feat: add vim.filetype.get_option()

[lewis6991]

Problem

It is not possible to get the default option value for a specific filetype. This is required for a number of applications, including working with treesitter injections, e.g: automatically updating 'commentstring' based on cursor position.

Solution

• Add filetype option to nvim_get_option_value() to get the default option value for a specific filetype.
• Add vim.filetype.get_option(filetype, option) as a wrapper around nvim_get_option_value() with caching.

---

Use this to set a context aware commentstring

local function get_lang()
  local ok, parser = pcall(vim.treesitter.get_parser)
  if not ok then
    return
  end

  local cpos = api.nvim_win_get_cursor(0)
  local row, col = cpos[1] - 1, cpos[2]
  local range = { row, col, row, col + 1 }

  local lang ---@type string?
  parser:for_each_child(function(tree, lang_)
    if tree:contains(range) then
      lang = lang_
      return
    end
  end)

  return lang
end

local function enable_commenstrings()
  -- Any better events?
  api.nvim_create_autocmd({'CursorMoved', 'CursorMovedI'}, {
    buffer = 0,
    callback = function()
      local lang = get_lang() or vim.bo.filetype
      if lang == 'comment' then
        lang = vim.bo.filetype   -- Exclude this
      end

      local commentstring = vim.filetype.get_option(lang, 'commentstring')

      if commentstring ~= vim.bo.commentstring then
        vim.bo.commentstring = commentstring
      end
    end
  })
end

api.nvim_create_autocmd('FileType', {
  callback = function()
    if not pcall(vim.treesitter.start) then
      return
    end

    enable_commenstrings()
  end
})

[clason]

@echasnovski @numToStr

[echasnovski]

Nice! Implementation seems a bit hacky, but can't suggest any better. Will definitely use it.

[clason]

Are there other filetype-local options that could be interesting (and thus warrant a generic vim.treesitter.get_lang_option)?

[lewis6991]

It is what it is. Fundementally we need to load runtime/ftplugin/<filetype>.vim which needs a buffer to pollute information into.

[lewis6991]

Are there other filetype-local options that could be interesting (and thus warrant a generic vim.treesitter.get_lang_option)?

Good question. Maybe.

[lewis6991]

Quick look:

setlocal cinkeys-=0#
setlocal indentkeys-=0#
setlocal include=^\\s*\\(from\\\|import\\)
setlocal define=^\\s*\\(def\\\|class\\)
setlocal includeexpr=...
setlocal suffixesadd=.py
setlocal comments=b:#,fb:-
setlocal omnifunc=python3complete#Complete
setlocal keywordprg=python3\ -m\ pydoc
setlocal isk+=#

I don't think we heavily use many (any?) of these.

[clason]

Quick look:

setlocal cinkeys-=0#
setlocal indentkeys-=0#
setlocal include=^\\s*\\(from\\\|import\\)
setlocal define=^\\s*\\(def\\\|class\\)
setlocal includeexpr=...
setlocal suffixesadd=.py
setlocal comments=b:#,fb:-
setlocal omnifunc=python3complete#Complete
setlocal keywordprg=python3\ -m\ pydoc
setlocal isk+=#

I don't think we heavily use many (any?) of these.

Omnifunc sounds very useful but would need to be handled at a lower level; same with idk and keywordprg.

Would have to be something that plugins need to know about. (If we had comment in core...)

[clason]

Either that or we cache the commentstring values.

Or even more radical, do something similar to filetype.lua and just maintain a big table.

That should definitely be more generic then... (But hard to make backward compatible.)

If we go that route, we should integrate all such options into filetype.lua, which already supports setting filetype options (because filetype.vim does, because why not add another mechanism on top of existing ones).

[clason]

Either that or we cache the commentstring values.

I think that would be a reasonable and natural optimization -- not necessarily for this PR, and not until someone complains about performance...

[lewis6991]

If we go that route, we should integrate all such options into filetype.lua, which already supports setting filetype options (because filetype.vim does, because why not add another mechanism on top of existing ones).

Well...as per my comments on the other PR, I'm not sure define or include should even exist, so maybe not all.

[justinmk]

I think I'd rather do that than have a buffer lying around.

fwiw, it can be :bdelete 'd and revived whenever needed (set 'bufhidden=hide' for good measure). So it's not really costing anything, though the logic is a bit annoying if you want to handle accidental :bwipe or other corruption.

The interface matters more, and it sounds like we are heading in a good direction here. vim.treesitter.get_option() seems actually more intuitive than having to look around to see which options are and aren't "provided" by vim.treesitter. For that reason I would even suggest hiding vim.treesitter.foldexpr and encapsulating it in vim.treesitter.get_option() , to further emphasize the pattern--though foldexpr may be special-cased internally?

[lewis6991]

foldexpr is a window-local option so it doesn't quite work here. Only buffer-local options make sense.

[clason]

Thoughts on the generic function:

1. vim.treesitter.get_option() is not great since it can be interpreted to mean "get treesitter option" (whatever those are or will be). Maybe get_local_option or get_buf_option?
2. The idea is to have a generic interface that will not require hard-coding some "blessed" options, so whether, e.g., include is pointless or not should make no difference. This is assuming this can be implemented transparently (just returning vim.bo[buf][bufoption], where bufoption is an argument to get_local_option -- with or without validation), otherwise it becomes a harder sell.
3. If this is not the case (options need to be handled manually), I'd be fine with saying "currently only 'commentstring' is supported" and worry about other options when and if they are requested (for 0.10).

[echasnovski]

2. The idea is to have a generic interface that will not require hard-coding some "blessed" options, so whether, e.g., include is pointless or not should make no difference. This is assuming this can be implemented transparently (just returning vim.bo[buf][bufoption], where bufoption is an argument to get_local_option -- with or without validation), otherwise it becomes a harder sell.

I think my initial expectation for allowed options in a function vim.treesitter.get_buf_option() would be the ones for which vim.api.nvim_get_option_info(option_name).scope is 'buf'.

[clason]

Sure, that would be one method of validation.

[clason]

Thinking about it, to pull this off properly we also need a reverse mapping "parser name -> filetype" to look up the correct commentstring for languages where the parser name is different from the filetype (latex vs. tex; bibtex vs. bib; markdown_inline vs. markdown, ...)

(Or any other way of associating language to canonical filetype, no matter how we end up implementing the filetype -> parser name mapping.)

[lewis6991]

Been thinking about this, and there may be some performance concerns. This is because we need to fire the FileType autocmd which is dynamic and can source any number of files. For infrequent calls, like to fetch the commentstring when commenting a single line, this wouldn't be a problem. But for example, if you want to comment every line in a file correctly, a naive implementation (on both sides) would result in FileType being fired N times and source O(n) files.

A good implementation of a commenting function could know about this perf characteristic and decide to keep a short lived local cache, however I would expect other implementation/applications to not take this into account.

Due to how dynamic autocmds make this, I don't think it's possible to implement a private cache in get_option, or atleast not easily as we'd need to keep track of sourced file paths (difficult) and autocmd entries for FileType (easy).

The only think I can think of would be to just document that this new get_option function is expensive and shouldn't be called often.

[clason]

I agree with the analysis, but since we have to add a caveat either way, we could also say "value is cached and may not update in every situation"? (And wait for complaints to see if these situations are worth supporting.)

[seblj]

Maybe, I'm not sure. What's the worst that could happen? However if we can't run the FileType autocmd safely, then there's no way we can safely use ftplugin as a namespace for filetype options.

Sorry to comment here on a PR that is already merged instead of creating an issue, but I have some issues with LSP attaching to the dummy-buffer and notifiying me of different things, but I don't feel that it's worth a separate issue. I see that lspconfig don't try to attach to a buffer if the buftype is nofile: https://github.com/neovim/nvim-lspconfig/blob/0f94c5fded29c0024254259f3d8a0284bfb507ea/lua/lspconfig/configs.lua#L233

Is this something that we can set on the dummy buffer? Or is there some other option already set or something that we can set? I can then send a PR to lspconfig to avoid trying to attach to a buffer with that said option#lewis6991

@lewis6991

[lewis6991]

Issue with repro steps please.

[seblj]

I have tried to come up with something, but I haven't been able to consistently reproduce it with a super minimal config without lspconfig. I can try some more, but I do think it would be nice to have some way to detect a dummy buffer

[lewis6991]

The issue needs to be analyzed and understood. The dummy buffer is purely an implementation detail, not something you should be messing with.

[clason]

You can replicate everything nvim-lspconfig does with vim.lsp.start in a FileType autocommand.

[mhanberg]

I will also try to come up with a repro.

I was attempting to replicate the commentstring example from the description and when i have codelens auto refreshing, i get tons of errors about an invalid buffer id.

it's also extremely slow (each time you move the cursor takes like 2 seconds), but that is merely tangential.

Thanks for all the hard work on Neovim, y'all have been on fire recently!!

[clason]

yeah, the snippet is not the best way to realize that (you'd query the commentstring on demand, not run a CursorMoved autocommand just in case) -- it's just for the sake of illustration

[lewis6991]

The whole point of vim.filetype.get_option() is that it's cached, so something in your system is invalidating it, or it's not working.

[seblj]

This is the most minimal I could get it:

local lazypath = vim.fn.stdpath("data") .. "/lazy/lazy.nvim"
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({
    "git",
    "clone",
    "--filter=blob:none",
    "https://github.com/folke/lazy.nvim.git",
    "--branch=stable", -- latest stable release
    lazypath,
  })
end
vim.opt.rtp:prepend(lazypath)

require("lazy").setup({
    {
        "nvim-treesitter/nvim-treesitter",
        config = function()
            require("nvim-treesitter.configs").setup({
                highlight = {
                    enable = true,
                },
            })
        end,
        priority = 100,
        build = ":TSUpdate",
    },

    {
        "neovim/nvim-lspconfig",
        config = function()
            require("lspconfig")["lua_ls"].setup({})
        end,
    },
})

vim.keymap.set("n", "gcc", function()
    vim.filetype.get_option(vim.bo.ft, "commentstring")
end)

I used NVIM_APPNAME with a fresh config. Then open up and let lazy install everything

Then do: nvim test.lua and gcc. Notice the message: [LSP] Client with id 1 not attached to buffer 2

Notice that I use priority = 100 on nvim-treesitter (the default is 50). This means that it should get loaded before nvim-lspconfig. I don't get the error here if I set priority on nvim-lspconfig so that it gets loaded before treesitter. I have no idea why. I tried to make it even more minimal with adding to rtp myself, but couldn't make it work...

(Deleted the message first as I thought for a second I didn't use the minimal config, but I did. So this should reproduce it)

[lewis6991]

Repro both issues (LSP and bad cache), should be fixed in #22753

[lewis6991]

@seblj you should never need a package manager for a minimal repro. Best to use this template from gitsigns:

         vim.o.packpath = '/tmp/nvim/site'

         local plugins = {
           gitsigns = 'https://github.com/lewis6991/gitsigns.nvim',
           -- ADD OTHER PLUGINS _NECESSARY_ TO REPRODUCE THE ISSUE
         }

         for name, url in pairs(plugins) do
           local install_path = '/tmp/nvim/site/pack/test/start/'..name
           if vim.fn.isdirectory(install_path) == 0 then
             vim.fn.system { 'git', 'clone', '--depth=1', url, install_path }
           end
         end

         require('gitsigns').setup{
           debug_mode = true, -- You must add this to enable debug messages
           -- ADD GITSIGNS CONFIG THAT IS _NECESSARY_ FOR REPRODUCING THE ISSUE
         }

[seblj]

@seblj you should never need a package manager for a minimal repro. Best to use this template from gitsigns:

         vim.o.packpath = '/tmp/nvim/site'



         local plugins = {

           gitsigns = 'https://github.com/lewis6991/gitsigns.nvim',

           -- ADD OTHER PLUGINS _NECESSARY_ TO REPRODUCE THE ISSUE

         }



         for name, url in pairs(plugins) do

           local install_path = '/tmp/nvim/site/pack/test/start/'..name

           if vim.fn.isdirectory(install_path) == 0 then

             vim.fn.system { 'git', 'clone', '--depth=1', url, install_path }

           end

         end



         require('gitsigns').setup{

           debug_mode = true, -- You must add this to enable debug messages

           -- ADD GITSIGNS CONFIG THAT IS _NECESSARY_ FOR REPRODUCING THE ISSUE

         }

I know... I didn't want to use it but couldn't get it to work without it. Thanks for the snippet from gitsigns! Will use it in the future