Custom Modes and Nested Modes

[samholmes]

From #2948 (comment):

Instead of binding key sequences, we could achieve the same result with "custom modes". First we bind p to enter a custom mode called "ModeP", then in that mode we can define "k" and "h" to their respective commands.

Additionally, we can apply a "backspace" keybinding to all modes to issue the hypothetical command "ReturnToLastMode" which will exit to the previous mode. Such a command implies that modes are maintained in a stack... So instead of having an unbounded stack-based mode system, we could opt to have "nested modes" where a mode can be nested in another mode and then we can have a command "GoToOuterMode" which will exit the current nested mode for the outer mode. Such a nested mode system could be complimented with some syntax to enter a nested mode deeply instead of always requiring nested modes be drilled down by commands. This flexibility could lead to a full state-machine with some contextual awareness via nesting.

The idea with custom modes and nested modes is to allow the user to configure any number of modes by the config file. Each mode would be identified by its name. Any mode could be defined within another mode, which we call "nested modes". Commands to go to a top-level mode versus a nested mode could be done with some delimitated syntax "normal/foo/bar". There are also commands to enter a relative mode and exit the current nested mode to the outer mode.

This proposal would allow for emulation of multi-key bindings. In addition, this would allow for the user to create more complex state machines within their config such that they can fully emulate vim-like keybindings and more.

Aside from this feature, each mode could trigger some action on enter/exit allowing for commands to run that change some UI, which could signal modal state to the user leading to a better UX. Knowing whether the user is in a particular mode could be shown by color changes in the UI, text changes to status bars, etc. This sort of discoverability would keep the users aware of the context they currently in. Such a feature could be done via plugin, so it's not necessarily apart of this feature request.

[samholmes]

One example of why this feature would be useful is when I want to assign dd to close a tab while within tab mode and also dd to delete pane while in the pane mode:

pane {
  bind "d" { SwitchToMode "delete" }
  delete {
    bind "d" { CloseTab; ExitMode }
  }
}
tab {
  bind "d" { SwitchToMode "delete" }
  delete {
    bind "d" { CloseFocus; ExitMode }
  }
}

This is a bit more complex then perhaps having a way to bind two characters to the two commands, but consider that the status bar could show whether you've entered "delete" mode in the respective tab/pane modes and show the next key bindings available while in that mode.

Additionally, if there was a way to hook into enter/exit events in a mode then we can issue hypothetical commands to change the UI.

pane {
  enter { SetStatusLine "Pane" }
  bind "d" { SwitchToMode "delete" }
  delete {
    enter { SetStatusLine "Pane Delete"; SetStatusColor "red" }
    exit { ResetStatusColor }
    bind "d" { CloseTab; ExitMode }
  }
}

Though this example is a little contrived when it comes to the use-case here for enter/exit hooks. A better example might run some shell command instead which opens the door to many things.

Of course binding "Escape" to all modes to issue an ExitMode command may be useful as I mentioned in the OP.