CSI 58 (underline color) integration with zellij

[tranzystorekk]

Environment:

• wezterm version: 20240520-135708-b8f94c47
• zellij version: 0.40.1
• neovim version: 0.10.0
• OS: Void Linux via WSL2

Description

I recently noticed that after updating neovim to 0.10.0 the error highlights look completely broken (the xxx text should be just red undercurls):

A short investigation revealed that this happens only when using wezterm together with zellij. I managed to isolate the sequence of ansi codes produced by zellij that I believe leads to this behavior:

ansi.txt:

\x1b[58;2;234;105;98m\x1b[4:3mxxx\x1b[59m\x1b[24m \x1b[38;2;140;248;247mcterm=\x1b[38;2;212;190;152mundercurl \x1b[38;2;140;248;247mgui=\x1b[38;2;212;190;152mundercurl \x1b[38;2;140;248;247mguisp=\x1b[38;2;212;190;152m#ea6962\x1b[39m

Note that the CSI 58 is encoded in the standard ANSI format (params separated by semicolons). Printing that in wezterm with zellij disabled produces:

Now if we adjust it to the 24bit color format (used e.g. by the wezterm undercurl FAQ):

ansi-corrected.txt

\x1b[58:2::234:105:98m\x1b[4:3mxxx\x1b[59m\x1b[24m \x1b[38;2;140;248;247mcterm=\x1b[38;2;212;190;152mundercurl \x1b[38;2;140;248;247mgui=\x1b[38;2;212;190;152mundercurl \x1b[38;2;140;248;247mguisp=\x1b[38;2;212;190;152m#ea6962\x1b[39m

This printed produces the correct render:

Discussion

My educated guess is that the sequence of events looks like this:

1. neovim with termguicolors enabled produces the 24bit non-standard ansi sequence (with colons)
2. zellij processes and normalizes that into the standard ansi sequence (with semicolons)
3. wezterm only understands the non-normalized variant and thus fails to correctly parse what it got from zellij

The first step to resolving this is to determine if this is in fact what wezterm expects for CSI 58, or if it can be some other underlying issue?

(I've started an initial discussion with zellij (@imsnif) on this topic and there is openness to adjust this behavior, provided that there isn't anything else that I have missed)

[bew]

The docs:
https://wezfurlong.org/wezterm/escape-sequences.html#graphic-rendition-sgr
With mention of the format (below the table):

The canonical representation of these sequences is to have the multiple parameters be separated by colons (:), but for compatibility reasons WezTerm also accepts an ambiguous semicolon (;) separated variation.
The colon form is unambiguous and should be preferred; the semicolon form should not be used by new applications and is not documented here in the interest of avoiding accidental new implementations.

[tranzystorekk]

for compatibility reasons WezTerm also accepts an ambiguous semicolon (;) separated variation

I'm not entirely sure if the broken render in my experiment is caused by the said ambiguity or if it's a bug in wezterm.

[imsnif]

The canonical representation of these sequences is to have the multiple parameters be separated by colons (:)

Citation needed ;P

But joking aside, we in Zellij prefer the semicolon in order to be compatible with old software as well (eg. the bare TTY console which I'm not sure will play nicely with colons, super old proprietary terminals, etc.)

In this specific case, as @tranzystorekk mentioned - I'd be open to changing it to colons seeing as anything that supports undercurls is very likely to support them as well.

But before we do, since the semicolon version does work for me on Wezterm on my arch machine and doesn't work for @tranzystorekk on Wezterm on windows - I'm guessing maybe this is an internal Wezterm issue? Is the intent to support both here?

[tranzystorekk]

I'm guessing maybe this is an internal Wezterm issue?

It clicked for me, that wezterm on windows relies on the conpty implementation of undercurls, so I did one more test: port the zellij sequence to powershell escapes and print it on wezterm in local powershell, and maybe unsurprisingly:

Makes me think this isn't wezterm's fault per se, but rather the windows implementation it delegates to.

[tranzystorekk]

Zellij accepted a workaround in zellij-org/zellij#3440 and main discussion is now in the microsoft terminal repo issue mentioned above, so this issue can probably be closed unless we want to keep it for monitoring further developments?