Bad performance when refresh highlight in large file

[kevinhwang91]

Describe the bug
didopen and didsave will trigger cxx to highlight the overall symbols in the buffer.
Without triggering these events, the highlight looks weird after editing.
If the file is large enough, it will make neovim hang for a few hundred milliseconds.

Maybe cxx should use asynchronous tech such as RPC or refresh the current view window first?

To Reproduce

1. git clone https://github.com/neovim/neovim.git && cd neovim
2. build compile_commands.json: bear make CMAKE_INSTALL_PREFIX=$HOME/.local
3. nvim -u ~/mini.vim src/nvim/window.c +1000
4. yy p :w

cat ~/mini.vim

call plug#begin('~/.config/nvim/plugged')
Plug 'neoclide/coc.nvim', {'branch': 'release'}
Plug 'jackguo380/vim-lsp-cxx-highlight'
call plug#end()

let g:lsp_cxx_hl_log_file = '/tmp/lsp-cxx-hl.log'
let g:lsp_cxx_hl_verbose_log = 1

Expected behavior
highlight as smooth as possible.

Configuration (Fill this out):

• NVIM v0.5.0-642-gb3f5083b8
• coc.vim 0.0.78-9f9d8a32c4 + ccls

Log File:
the content of log is redundant, so I post the file instead of paste its content directly.
lsp-cxx-hl.log.tar.gz

[jackguo380]

Hi, sorry for taking so long to reply.

There's a bug in vim-lsp-cxx-highlight with newer versions of neovim which causes highlighting to clear whenever it's updated. You're not supposed to notice when the highlight refreshes (apart from when adding new lines) as the old highlight shouldn't clear until the new highlighting is fully applied. I fixed it in 054d571 but I'm not sure if you are experiencing this exact bug or just feel that the plugin is sluggish.

If it's general slowness there's a lot of things that factor into this. I agree that a parsing time of usually 100-500ms isn't ideal but I personally haven't noticed vim or neovim visibly locked up/hanging as a result. Generally I would attribute most of the performance issues to vimscript and vim/neovim's single threaded execution. As much as I don't particularly like writing vimscript and it's super slow, it still offers the best user experience for most.

I also don't think breaking this plugin into a separate application is necessary given the small scope of what this plugin does. This would make the installation process much more complicated and some users would likely consider it a non-starter to have to do anything other than just clone the repo, even worse if they have to compile rust or c++ code. One possibility I had in mind was to write some of the compute portions in lua and enable it on neovim via the builtin lua system. Using lua could provide a moderate speedup but I haven't written lua in many years so I haven't got around to it.

Please let me know if the mentioned fix makes a difference for you.

[kevinhwang91]

I fixed it in 054d571 but I'm not sure if you are experiencing this exact bug or just feel that the plugin is sluggish.

It's sluggish, and I don't feel the patch exactly speeds up the cxx.

100-500ms of STW for the large file is improvable, I found that the sluggish code is nvim_buf_add_highlight in my profile.

FUNCTION  <SNR>130_buf_add_hl()
    Defined: ~/.config/nvim/plugged/vim-lsp-cxx-highlight/autoload/lsp_cxx_hl/textprop_nvim.vim:16
Called 21316 times
Total time:   0.336146
 Self time:   0.335502

count  total (s)   self (s)
                            
                                " single line symbol
21316              0.034396     if a:s_line == a:e_line
21232              0.028302         if a:e_char - a:s_char > 0
21232              0.220552             call nvim_buf_add_highlight(a:buf, a:ns_id, a:hl_group, a:s_line, a:s_char, a:e_char)
21232              0.020543             return
                                    else
                                        return
                                    endif
   84              0.000032     endif
                            
   84   0.001437   0.000793     call lsp_cxx_hl#log('Error (textprop_nvim): symbol (', a:hl_group, ') spans multiple lines: ', a:s_line, ':', a:s_char, ' to ', a:e_line, ':', a:e_char)

It maybe uses the cache to prevent the rendered line from render again.

BTW

vim-lsp-cxx-highlight/syntax/lsp_cxx_highlight.vim

Lines 26 to 34 in 054d571

	if g:lsp_cxx_hl_light_bg
	hi LspCxxHlGroupEnumConstant ctermfg=Magenta guifg=#573F54 cterm=none gui=none
	hi LspCxxHlGroupNamespace ctermfg=Yellow guifg=#3D3D00 cterm=none gui=none
	hi LspCxxHlGroupMemberVariable ctermfg=Black guifg=Black
	else
	hi LspCxxHlGroupEnumConstant ctermfg=Magenta guifg=#AD7FA8 cterm=none gui=none
	hi LspCxxHlGroupNamespace ctermfg=Yellow guifg=#BBBB00 cterm=none gui=none
	hi LspCxxHlGroupMemberVariable ctermfg=White guifg=White
	endif

should keep the default for highlight

[jackguo380]

Sorry to hear that wasn't the fix, but if you believe there's room for optimizing I would be happy to accept a PR.

Thanks for noticing the syntax file bug. I missed that since I copied the suggestion from one of the comments and forgot to adjust it.

[kevinhwang91]

Do you think using CursorMoved can solve the issue?

When cxx received the lsRanges data, only parse it and compare the value of line whether is in the [current_lnum - win_height + 1, current_lnum + win_height -1] range, if yes, render them right now, otherwise, add them into a list which will be used in CursorMoved event and then render then latter.

BTW, I use nvim-treesitter, if cxx could provide a whitelist to render is the best, most of highlighting could be done by nvim-treesitter.

[skysky97]

I have met the perfomance problem too. Each edit causes a long time slag.
Is this the case you mentioned before? Help.

Generally I would attribute most of the performance issues to vimscript and vim/neovim's single threaded execution.

hl_symbols (textprop) highlighted 9343 symbols in file src/vsomeip/implementation/configuration/src/configuration_impl.cpp
operation hl_symbols (textprop) src/vsomeip/implementation/configuration/src/configuration_impl.cpp took   1.546854s to complete