Make search progress more user friendly

[bercly0b]

Hi guys.
I'm trying to switch from fzf to telescope and there is one thing that blocks me.

Is your feature request related to a problem? Please describe.
When I'm searching (find_files, live_grep), I don't understand the search is over and nothing is found or the search is in progress.
In gif, after a few seconds, the total number of files will change

telescope gif

Describe the solution you'd like
In fzf.vim search progress is more user friendly

fzf gif

Is it possible to make the same "live" progress in a telescope?

[alefpereira]

Dear @bercly0b, have you tried telescope-fzf-native? I'm away from my computer now but I'm almost sure it behaves the same as in the fzf gif.

[bercly0b]

Hi @alefpereira
Thanks you for your answer.
Yes, I tried telescope-fzf-native, but I have the same result. It shows the count of files and only updates this number after a few seconds..
here is my telescope config

[alefpereira]

@bercly0b I'm curious about that.

I see you have a lot of patterns in your file_ignore_patterns table, if I'm not wrong, those patterns are filtered by telescope, only after the file list is retrieved by the external command, this may be causing this strange behavior in your setup.

Try to remove those patterns from telescope and use a custom fd command for it to filter them before passing the file list to telescope.

I use this command fd --type file --type symlink --hidden --exclude .git, but you can custom the way you want:

require('telescope').setup({
  defaults = {
    pickers = {
      find_files = {
        find_command = {
          'fd',
          '--type',
          'file',
          '--type',
          'symlink',
          '--hidden',
          '--exclude', 
          '.git',
          -- put your other patterns here
        }
      }
    }
  }
}) 

[alefpereira]

@bercly0b please let me know if it worked.

I'm almost sure the custom fd command will do the trick for the find_files, but if it doesn't work, or if you just want a faster finder for git repositories, try to use git_files instead. The fd command is really fast, but for a git repository, git_files is faster.

There is even this recipe that setup a function to use git_files if you are working on a git repository and if you are not, it will fallback to find_files, you can then set a keymap to it.

[bercly0b]

With fd and removing the file_ignore_patterns table it works much better!
Thanks a lot, now I can switch to telescope from fzf.vim

But there is one interesting thing.
When I just open the picker, it provides "live" search progress. But when I start typing, it causes a delay in displaying the search progress.
I specifically did not ignore node_modules in this example to provide a large number of files to search for..

gif

[hbiel]

I think it would be helpful to have an visual indicator while a search process is running in the background. Like FZF does with this animated circle of dots for example.

This could also be displayed on right side of the prompt where the result count is shown.

[alefpereira]

You're welcome @bercly0b, glad to help.

I tested both telescope and the terminal fzf tool in the linux repository which have more than 75K files and I notice the behaviour you mentioned in telescope.

For me it seems that telescope do the sorting in two different moments/stages, one as you type, and the other, showing a different sorting after a second or so, runs on background, or maybe debounced idk, while in fzf its done only as you type. This doesn't mean that telescope-fzf-native implementation is slower than fzf but sure the UX feels slower and a bit odd.

I also tested both telescope-fzf-native and the fzf tool in my home dir with fd --hidden -I which brought over 900K files. I notice that the time it took for telescope to show the "second sorting" was almost if not the same as with 75K files. This brings me the feeling that this is caused by how telescope interface interacts with the fzf-native search result, and not the sorting itself freezing or being slow.

[alefpereira]

In fzf the animated dots are shown during the files load, the search shows a percentage like (42%) in the right side of the number of files, for me I'm already content with only the files counting growing, but both indications would be good to have.

@bercly0b, @hbiel I'm not a Telescope member but I like to contribute to projects I really like in my free time. But I'm not having much of that these days, If you guys could help, at least with some analysis or understanding how telescope handles this, we could work on that together.

[hbiel]

In fzf the animated dots are shown during the files load, the search shows a percentage like (42%) in the right side of the number of files, for me I'm already content with only the files counting growing, but both indications would be good to have.

@bercly0b, @hbiel I'm not a Telescope member but I like to contribute to projects I really like in my free time. But I'm not having much of that these days, If you guys could help, at least with some analysis or understanding how telescope handles this, we could work on that together.

I feel the same, not having as much time myself. But i can try. :-)

What i found so far regarding the progress indicator:

The picker has an an status_updater which can be called to update the status text in the prompt.
This is defined here:

telescope.nvim/lua/telescope/pickers.lua

Lines 1194 to 1223 in f262e7d

	--- Returns a function that sets virtual text for the count indicator
	--- e.g. "10/50" as "filtered"/"processed"
	---@param prompt_win number
	---@param prompt_bufnr number
	---@return function
	function Picker:get_status_updater(prompt_win, prompt_bufnr)
	return function(opts)
	if self.closed or not vim.api.nvim_buf_is_valid(prompt_bufnr) then
	return
	end
	local current_prompt = self:_get_prompt()
	if not current_prompt then
	return
	end
	if not vim.api.nvim_win_is_valid(prompt_win) then
	return
	end
	local text = self:get_status_text(opts)
	vim.api.nvim_buf_clear_namespace(prompt_bufnr, ns_telescope_prompt, 0, -1)
	vim.api.nvim_buf_set_extmark(prompt_bufnr, ns_telescope_prompt, 0, 0, {
	virt_text = { { text, "TelescopePromptCounter" } },
	virt_text_pos = "right_align",
	})
	self:_increment "status"
	end
	end

This updater uses the function get_status_text to determine which text to display:
This function can be customized via user configuration, the defaults are set here:

telescope.nvim/lua/telescope/config.lua

Lines 363 to 390 in f262e7d

	append(
	"get_status_text",
	function(self)
	local ww = #(self:get_multi_selection())
	local xx = (self.stats.processed or 0) - (self.stats.filtered or 0)
	local yy = self.stats.processed or 0
	if xx == 0 and yy == 0 then
	return ""
	end
	-- local status_icon
	-- if opts.completed then
	-- status_icon = "✔️"
	-- else
	-- status_icon = "*"
	-- end
	if ww == 0 then
	return string.format("%s / %s", xx, yy)
	else
	return string.format("%s / %s / %s", ww, xx, yy)
	end
	end,
	[[
	A function that determines what the virtual text looks like.
	Signature: function(picker) -> str
	Default: function that shows current count / all]]
	)

This even has some commented code which should display an indicator whether the search is completed or still running.
Simply uncommenting it and using the status_icon in the returned text doesn't work however as opts isn't set.
Changing opts to self helps but then the icon never gets updated to the checkmark.

Something seems to be missing here.

[bercly0b]

@hbiel thanks you for links to code.
I found what was missing, and I have made PR into my fork.

And I think there are a couple more things that need to be discussed:

• need to add the ability to pass icon from the user config
• need to choose where to display the icon (after files counter or before)
• need to update README file

I think it would be great if someone from the telescope team will advise us

[Conni2461]

We talked about adding search animations some time ago but the whole ui is currently bottle necking so its not on top of our priority list. The ui rewrite happens here #1491

[jelmansouri]

Hi! Was looking into the same issue as using the raw fzf command feels more snappy than telescope on large code bases such as the UnrealEngine one. I need to do more thorough timing but the overall timing of a similar search seems close, but the fact that results starts showing up progressively makes it feel like actually fzf is quite faster.

[jamestrew]

@jelmansouri telescope won't be as fast as raw fzf
see #2884 (comment)