Ignoring files based on their size

[mensfeld]

Hey, is there an option to use ripgrep and exclude files based on their size? I have a lot of +10MB text files that I don't want to grep and would love to be able to exclude them all. Thank you.

[BurntSushi]

No. Can you use some other attribute of the text files? You can specify paths to ignore at the command line with -g '!dir/to/ignore' or -g '!*.txt'.

[mensfeld]

Hello @BurntSushi. Thank you for such fast reaction. Unfortunately I can't. I use rg as a part in a "chain" where I get some locations and I need to do search based on that. If it is not possible, I will just wrap it around with a shell script with a list of dirs/files to ignore.

[BurntSushi]

It is indeed not possible right now. It would have to be explicitly added to ripgrep as a new flag. Which I'd be OK with, since it should be relatively easy to add. It will have a performance cost, but that seems fine since the end user would have to specifically ask for it anyway.

[mensfeld]

@BurntSushi - yes it would have a perf. impact but only when needed which shouldn't be that bad. For now I will just find those files on my own and exclude them. Thank you

[tiehuis]

I'll probably end up looking at implementing this over the next few days. Just to outline the intended approach from a quick glance before beginning.

• Add a new size matcher into Ignore and co in the ignore crate (in file dir.rs)
• Add the size check into matched (in file dir.rs)
• Add the top-level options into all the required locations
• Add some tests

There probably should be some nice intuitive values expected for the command line option, too (i.e. --limit-size=2K --limit-size=1G).

[BurntSushi]

Add a new size matcher into Ignore and co in the ignore crate (in file dir.rs)

I think the ignore crate is probably the right place, but it feels like the filter should be implemented in walk.rs, not dir.rs. The stuff in dir.rs is pretty tightly coupled to glob matching, where as the stuff in walk.dir is generalized to recursive directory walking. Note that you will have to implement the check on both the single threaded and parallel walkers.

There probably should be some nice intuitive values expected for the command line option, too (i.e. --limit-size=2K --limit-size=1G).

That seems like nice extra credit. Feel free to punt on that if you like. :-)

[BurntSushi]

Fixed in #385