Accessibility: setting caret position via IAccessible2 doesn't work

[mltony]

Type: Bug

Hello, this is an accessibility regression. cc: @isidorn

Steps to reproduce:

1. On Windows with NVDA screenreader open any text file in VSCode. Please leave the cursor at the very beginning of the document.
2. Press NVDA+Control+z to open NVDA python console.
3. Paste the following text and press enter:

   import api, core, tones, time, ui
   
   def f():
       f = api.getFocusObject()
       t = f.makeTextInfo('caret')
       result = t.move('character', 10)
       if result == 0:
           raise RuntimeError
       newOffset = t._start._startOffset
       t.updateCaret()
       t.updateCaret()
       time.sleep(0.5)
       tt = f.makeTextInfo('caret')
       ttOffset = tt._start._startOffset
       if ttOffset != newOffset:
           raise RuntimeError(f"Invalid offset: {newOffset=} {ttOffset=}")
       ui.message("Success")
   
   
   
   

   This snippet creates a function f() that retrieves cursor state, then moves it by 10 characters right.
4. Paste the following line to NVDA Python console. Press enter to run it and immediately alt-tab to VSCode window (you need to do it within 2 seconds of pressing Enter).

   core.callLater(2000, f)
   

5. Wait a second or two until NVDA says "Success".

Expected result

Cursor should be moved to position 10 in the file.

Actual result

VSCode is left in a very weird state:

1. I am blind so I can't tell whether the cursor has moved visually.
2. If you press RightArrow LeftArrow in VSCode, then it reads 1st and 0th characters, so it behaves as if cursor didn't move at all.
3. If I retrieve cursor position via IAccessible2 API again (see tt = f.makeTextInfo('caret') statement in my snippet above) then it reports correct cursor location (otherwise exception would have been raised.
   So TLDR: VSCode cursor ends up in an inconsistent state, but from the point of view of keyboard commands LeftArrow and RightArrow cursor position doesn't move at all.

Technical information

NVDA uses IAccessible2 to talk to VSCode. Cursor position is updated via setCaretOffset method:
https://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/interface_i_accessible_text.html#ae0943c1f912e4f35704fe4a0ee98e862

Thanks!

VS Code version: Code 1.88.1 (e170252, 2024-04-10T17:41:02.734Z)
OS version: Windows_NT x64 10.0.22631
Modes:

System Info

Item	Value
CPUs	AMD Ryzen 7 4700U with Radeon Graphics (8 x 1996)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	15.37GB (0.82GB free)
Process Argv	--crash-reporter-id 9a96d9f3-a840-44d8-9d62-0bac44c9d4b9
Screen Reader	yes
VM	67%

Extensions (17)

Extension	Author (truncated)	Version
tsl-problem-matcher	amo	0.6.2
doxdocgen	csc	1.4.0
vscode-eslint	dba	2.4.4
better-cpp-syntax	jef	1.17.2
debugpy	ms-	2024.4.0
python	ms-	2024.4.1
vscode-pylance	ms-	2024.4.1
jupyter	ms-	2024.3.1
jupyter-keymap	ms-	1.1.2
jupyter-renderers	ms-	1.0.17
vscode-jupyter-cell-tags	ms-	0.1.9
vscode-jupyter-slideshow	ms-	0.1.6
cmake-tools	ms-	1.17.17
cpptools	ms-	1.19.9
cpptools-extension-pack	ms-	1.3.0
extension-test-runner	ms-	0.0.9
cmake	twx	0.0.17

(1 theme extensions excluded)

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscoreces:30445986
vscod805cf:30301675
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
vscaat:30438848
c4g48928:30535728
azure-dev_surveyone:30548225
2i9eh265:30646982
962ge761:30959799
pythongtdpath:30769146
welcomedialogc:30910334
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pyreplss1:30897532
pythonmypyd1:30879173
pythoncet0:30885854
2e7ec940:31000449
pythontbext0:30879054
accentitlementsc:30995553
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
d34g3935:30971562
fegfb526:30981948
bg6jg535:30979843
ccp2r3:30993541
dsvsc020:30976470
pythonait:31006305
gee8j676:31009558
chatpanelc:31018788
dsvsc021:30996838
01bff139:31013167
pythoncenvpt:31022790

[isidorn]

@mltony thanks for filling this issue. Does this also surface in some user interaction when using nvda with VS Code?
@meganrogge is on vacation today, but I expect her to investigate more once she gets back later in the week.

[mltony]

Does this also surface in some user interaction when using nvda with VS Code?

Currently no, but I am reworking word navigation in NVDA (via control+leftArrow/rightArrow), so if this can be fixed, we will make word navigation happen entirely on NVDA's side and NVDA will only update cursor location, thus entirely avoiding issues like #34319.

[isidorn]

Thanks.
Do you know for a fact that this used to work? If yes, with what VS Code version was it working?
I am asking to understand if this was caused by an Electron update.

[mltony]

My best recollection is it was working back in 2020 - that's when I was playing with word navigation last time. I don't remember what VS Code version was that.

[meganrogge]

I am not aware of any changes that could have caused this on our side. Since it cannot be reproduced in VS Code with NVDA, I am thinking it's an issue with IAccessible2, which we cannot investigate.

cc @rperez030 for any thoughts

[mltony]

@meganrogge ,

Since it cannot be reproduced in VS Code with NVDA

It can be reproduced - see steps to reproduce in my original description.
Or if you are referring to the fact that NVDA does not use this for word navigation - well this seems to be proverbial chicken and egg problem: we can't make use of it since it's broken, and if you use the argument that you won't fix it since NVDA doesn't use it - that's a logical deadlock.

I am thinking it's an issue with IAccessible2

Yes, I stated in the description that this is an issue of IAccessible implementation in VSCode. Similar behavior works fine in Chrome.

which we cannot investigate.

Why? Chromium doesn't have this problem. Who should investigate this then?

[cary-rowen]

Using word navigation in VS Code is very necessary and will greatly improve the efficiency of screen reader users in reading code.

[meganrogge]

@deepak1556 any idea what could have changed here for us since 2020?

[Neurrone]

Word navigation has never worked reliably as far as I'm aware due to underlying bugs in Chrome.

[mltony]

After further investigation it seems like this might be related to this Chromium issue. But on the other hand, that chromium issue can be easily mitigated by calling setCaretOffset twice in a row with the same argument, while this mitigation doesn't appear to work with VSCode.

[meganrogge]

Thanks for investigating. I am not sure how I can help here. I've added our chromium experts, @deepak1556 and @rzhao271, to this issue, in case they have thoughts.

[deepak1556]

Sorry for the delay here, the difference between Chromium and VS Code could be attributed to us disabling the occlusion tracker

vscode/src/main.js

Lines 291 to 295 in 26120e5

	// Following features are disabled from the runtime:
	// `CalculateNativeWinOcclusion` - Disable native window occlusion tracker (https://groups.google.com/a/chromium.org/g/embedder-dev/c/ZF3uHHyWLKw/m/VDN2hDXMAAAJ)
	const featuresToDisable =
	`CalculateNativeWinOcclusion,${app.commandLine.getSwitchValue('disable-features')}`;
	app.commandLine.appendSwitch('disable-features', featuresToDisable);

but I would have expected a positive difference since disabling the tracker means renderer does not get throttled when occluded.

I am yet to try the repro steps, but based on the linked upstream issue this does seem related to incorrect focus handling. I will try a bisect to see if we are able to narrow down the issue on our end.

[deepak1556]

Was able to repro with using https://vscode.dev and electron fiddle. Bisect revealed the following upstream CL https://chromium-review.googlesource.com/c/chromium/src/+/5356613 to have addressed the issue is newer chromium versions. We are promoting Electron 29 to insiders this month, I will backport the change to this version in upstream.

[isidorn]

Thank you very much for back-porting the fix @deepak1556