Provide comparison/benchmarks vs guess-indent.nvim

[krishnakumarg1984]

Thank you for making this plugin. Are there advantages/USPs of using this over guess-indent, a newer lua plugin with a similar mandate?

Perhaps you could provide an explanation and/or benchmarks comparing these two?

[Darazaki]

Oh, I wasn't aware of guess-indent.nvim's existence thanks! I've skimmed through its source code and will try to provide an in-depth answer here

Perhaps you could provide an explanation

Both plugin do essentially the same thing: they auto-detect indentation quickly. The only difference is the algorithm they use:

• guess-indent.nvim uses the classic tried and tested statistical algorithm: taking a sample of lines and guessing what is the most likely indentation
• indent-o-matic uses a new dumber algorithm which I created since I've noticed doing a full on analysis of a sample of lines wasn't needed most of the time

In theory, here's the main advantage of each one:

• guess-indent.nvim should detect indentations more accurately, especially in files with weird indentations (as explained here by NMAC427)
• indent-o-matic should be able to analyze more lines (if configured to do so) without sacrificing much performance because of its simpler algorithm

In practice, both algorithm usually detect indentations correctly and are done in an instant so in most cases it's pretty much up to whichever name you think look fancier in your init file

(Though I have to admit that I love NMAC427's idea of ignoring multi-line comments since those usually don't follow standard indentation. This should definitely be added to indent-o-matic!)

benchmarks comparing these two?

In practice, both algorithms finish executing before anyone can notice so benchmarks shouldn't matter too much. However, it could be fun to see how fast they can run and how things like "using pattern matching vs raw lua code" affect performance

I may try to attach some benchmarks to this issue once I get a bit more free time, I have far too much work nowadays :/

[Darazaki]

(I'll keep this issue open for now so that I remember to try doing benchmarks later. Also, I'll mention this issue inside the one you opened for guess-indent.nvim so that NMAC427 can correct me if I got anything wrong)

[krishnakumarg1984]

Great. Thank you so much for clarifying various aspects raised here!

[NMAC427]

I think @Darazaki summed it up perfectly - both plugins have the same goal just with a different approach.
Thanks to Lau, both plugins are blazingly fast, no matter what you throw at them.

I ran some benchmarks (each time running the corresponding command 1000 times) and, except for some edge cases, indent-o-matic almost always runs in about 10-20µs (yes, you read that right; micro seconds). On the other hand, guess-indent can take anywhere between 60µs to 1ms to run (depending on the length of the file). In the real world though, this difference doesn't really matter, because the biggest bottle neck is not detecting the indentation style, but adjusting the buffer options accordingly. Running a single :set command takes about 350µs. This means, that when you open a buffer that is indented using spaces, simply updating the buffer options takes 1.4ms. Both plugins mitigate this overhead by only changing the buffer options if the value differs from the detected indentation style (this is also why we don't see this overhead in the benchmarks I ran).

In case you're interested, these are the raw numbers I got from running the :GuessIndent, :IndentOMatic, and :Sleuth (vim-sleuth, probably the smartest indentation plugin) commands in various buffers:

Approx 20 lines:
GI:   0.063 ms
IOM:  0.012 ms
VS:   4.622 ms

Approx 300 lines:
GI:   0.204 ms
IOM:  0.010 ms
VS:  17.040 ms

Approx 1000 lines, no indentation (this is the one case where 
    indent-o-matic becomes just as slow as every other plugin):
GI:   0.995 ms
IOM:  0.876 ms
VS:   1.406 ms

[GI]  = GuessIndent  
[IOM] = IndentOMatic 
[VS]  = VimSleuth

So what's the conclusion? I think the best thing is to just try out both plugins and see which one works best for your personal workflow and with the kind of files you're working with. There are cases in which indent-o-matic is too dumb and there are cases where guess-indent is too smart to determine the proper indentation style, but those cases are rare. It all comes down to your use cases. I think you can be happy with either option, because there is one thing that I can guarantee: No matter what plugin you are using, both will make your workflow easier, because you almost never have to worry about indentation again.

[Darazaki]

Wow thank you very much for the benchmarks! Also I completely agree with your conclusion

Closing this since every question has been answered now :)

[krishnakumarg1984]

This is absolutely comprehensive response from both the plugin authors. Thank you so much.

May I suggest pinning this issue and provide a link in the README (under a section titled alternatives/benchmarks or something equivalent) for each of the two plugins?

[Darazaki]

Added to README! I don't think pinning this issue is necessary since there's already a link to it in the README

Also, you're welcome :)

[rag-hav]

Hi, I am interested in knowing how these benchmarks were run for my own project. Please point me to any resources if you can. Thanks

[NMAC427]

Hi, I am interested in knowing how these benchmarks were run for my own project. Please point me to any resources if you can. Thanks

I filled a buffer with some text and then ran the following lua script to test how much time each command took:

local function profileCmd(cmd, times)
  local start = tonumber(vim.fn.reltimestr(vim.fn.reltime()))
  for _ = 1, times do
    vim.cmd(cmd)
  end
  local stop = tonumber(vim.fn.reltimestr(vim.fn.reltime()))
  return (stop - start) / times * 1000 -- ms
end

print("IOM: ", profileCmd(":silent IndentOMatic", 1000))
print("GI:  ", profileCmd(":silent GuessIndent", 1000))
print("VS:  ", profileCmd(":silent Sleuth", 1000))

print(profileCmd(":set sw=4", 1000))