Dropdowns, menus and lists should highlight items in reaction to key presses ("Initial Letter Navigation")

[cbjeukendrup]

Task description
When a dropdown, menu or list is focused, you should be able to type text on the keyboard, to quickly jump to the corresponding item.

For example, in the dropdown shown below,

• when you press S, it should highlight the first next item that comes lexicographically after "S", so in this case "Sans Serif Collection".

• when you press S E in quick succession, it should highlight the first next item that comes lexicographically after "Se", so in this case "Segoe Fluent Icons".

• when you press S G in quick succession, it should highlight the first next item that comes lexicographically after "Sg", so in this case "Showcard Gothic".

Edits by @shoogle:

• Changed 'first' to 'next'.

• The delay for "in quick succession" is one second (see WM_CHAR), but could be made configurable in Preferences > Advanced. Within this period, letters are added to a string buffer and a search is performed for the entire string. If the current item starts with the string then selection remains on this item, otherwise it goes to the next item that starts with the string. After the period expires, the string buffer is emptied. A subsequent key press starts a fresh search with a new string buffer, and will move to the next matching item even if the current item is a match.

• If the first letter is pressed repeatedly (e.g. S S S) then the selection cycles through items that begin with that letter (e.g. "Sans", "Sergeo", "Showcard"). However, the buffer is still saved as SSS, and if a different character is pressed before it empties (e.g. SSSQ) then the selection jumps to the next item starting with this entire string (e.g. "Sssquash").

• See PR #5829 for previous implementation in MuseScore 3. If in doubt, check the behaviour of Windows File Explorer.

Technical note
This will need to be implemented for all QML dropdowns/menus/lists/treeViews.
In Qt Widgets, this already works out of the box (and for those few cases where it doesn't, we shouldn't invest time to fix it since Qt Widgets is going to be replaced eventually anyway).

[Tantacrul]

I agree. I also think you've basically designed it, so no input required from me. Let's do it!

[jmwmusic]

Really hope this one can be addressed as it is standard accessibility practice and makes a significant difference to productivity. Take for example adding the swing item from the tempo palette, currently that takes 31 down arrow presses to get to!

[jmwmusic]

Can I check whether the scope of this issue currently includes palette lists, and if not can that be added?

[jmwmusic]

I think this might be related - Currently screen reader is not announcing the Mnemonic Shortcut options in menus. So for example in the File menu if you arrow down to Open it just says "Open Ctrl O" but would usually say "Open Ctrl O O". But maybe this is deliberate till it is working as currently if I press Alt F O I get stuck in nomans land with no focus, tab & arrow do not work and I have to press Esc to get out of it.

[cbjeukendrup]

I think the Mnemonics issue is unrelated, and it would be preferable to have a separate issue for that. MuseScore uses a custom menu implementation on Windows, so speech for mnemonic shortcuts was probably just not implemented. However, mnemonics themselves should be working correctly. If not, that would be a regression and would deserve an issue on its own.

[re1san]

Hey, I would like to work on it :)

[scorster]

Issue reported here on .org, along with issues with the Up arrow keystroke.

[Jojo-Schmitz]

Came up again in https://musescore.org/en/node/366117

[cbjeukendrup]

As noted in #22179 (comment), this is a bit worse than just "not working yet": pressing a letter key while a list is focused is interpreted as activating a global keyboard shortcut, resulting in notes unexpectedly being repitched for example.

[shubham-shinde-442]

Working on this.

[re1san]

@cbjeukendrup I worked on this issue earlier. It seems like the focus is disabled for dropdowns by this line: openPolicies: PopupView.NoActivateFocus in StyledDropdownView.qml. Because of this, the code in Keys.onShortcutOverride doesn't work and instead the global keyboard shortcut is activated as ShortcutsController::activate is called on pressing any key.

@Eism mentioned that this was done intentionally so that opening dropdowns wouldn't steal focus from the current window.

Hope this helps you, @shubham-shinde-442 :)

[scorster]

@Eism mentioned that this was done intentionally so that opening dropdowns wouldn't steal focus from the current window.

Why should the score respond to keystrokes made when the user has a menu open? If the user expects/wants their keystroke to interact with the score, but notices that it doesn't, they can simple escape the menu and press the keystroke again. Seems like standard fare with lists and menus.

What am I missing here?

MuseScore exhibited lots of focus issues over the years. Thankfully many of them have been fixed, resulting in improved and mainstream UX. I hope to see this resolved. Searching font menus can be a pain.

[shoogle]

What am I missing here?

I'm not sure, but possible reasons might be:

1. To allow changing the dropdown value with Up and Down arrow keys without opening it first (but I think this only works in MS3 and not MS4 anyway).

2. The screen reader might treat the dropdown as a new window and say the window name when you focus/unfocus it, which could be confusing/unhelpful.

3. Focusing might break speech in some other way, perhaps because we use our own accessibility system rather than Qt's system.

4. Focusing might break keyboard navigation somehow, perhaps because we use our own shortcut-based navigation rather than Qt navigation.

5. If you open the Playback or Appearance popup in the Properties panel, these popups are supposed to stay open while you navigate the score. Perhaps they share some code with dropdowns, and keeping them open means focus has to stay in the main window.

@shubham-shinde-442, you could try focusing the dropdown if you like. If you do, those are the things I would watch out for.