-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
--files performance regression in 0.8.0 #820
Comments
@chrmarti Thanks for another great bug report! I can indeed reproduce this. Your tip about ripgrep stat'ing more files is right on the money. Basically, this is fallout from fixing #705. In particular, in order to fix #705, I had to work-around the corresponding bug in Rust's standard library, and I was apparently careless enough that I introduced an extra stat call for each file that ripgrep visits. This should be possible to fix, but I'll need to take a closer look tomorrow. Assuming I get this fixed, and after I get some of the PR backlog cleared, I can put out another release. |
Note that this regression is also in walkdir, so it impacts both single threaded and multi-threaded directory traversal. |
This commit fixes a performance regression introduced in commit 0f4441, which aimed to fix OneDrive traversals. In particular, we added an additional stat call to every directory entry, which can be quite disastrous for performance. We fix this by being more fastidious about reusing the Metadata that comes from fs::DirEntry, which is, conveniently, cheap to acquire specifically on Windows. The performance regression was reported against ripgrep: BurntSushi/ripgrep#820
This commit fixes a performance regression introduced in commit 0f4441, which aimed to fix OneDrive traversals. In particular, we added an additional stat call to every directory entry, which can be quite disastrous for performance. We fix this by being more fastidious about reusing the Metadata that comes from fs::DirEntry, which is, conveniently, cheap to acquire specifically on Windows. The performance regression was reported against ripgrep: BurntSushi/ripgrep#820
This commit fixes a performance regression in Windows that resulted from fallout from fixing #705. In particular, we introduced an additional stat call for every single directory entry, which can be quite disastrous for performance. There is a corresponding companion PR that fixes the same bug in walkdir: BurntSushi/walkdir#96 Fixes #820
This commit fixes a performance regression in Windows that resulted from fallout from fixing #705. In particular, we introduced an additional stat call for every single directory entry, which can be quite disastrous for performance. There is a corresponding companion PR that fixes the same bug in walkdir: BurntSushi/walkdir#96 Fixes #820
This should be fixed in master. I manually QAed this and confirmed that performance is back to If all goes well, I'm going to try and squeak a 0.8.1 release out tonight. |
What version of ripgrep are you using?
0.8.0
What operating system are you using ripgrep on?
Windows 10
If this is a bug, what are the steps to reproduce the behavior?
Using an old commit of the Chromium sources (
git clone https://chromium.googlesource.com/chromium/src
) as corpus (~230k files).With 0.8.0
--files
runs about 4x longer:Than with 0.7.1:
Using Process Monitor it looks like 0.8.0 accesses individual files a lot more:
![image](https://user-images.githubusercontent.com/9205389/36384556-fa76474a-158f-11e8-9c7a-208f270c1b0f.png)
Than 0.7.1, which seems to only access folders:
![image](https://user-images.githubusercontent.com/9205389/36384611-27245f16-1590-11e8-8a2e-b54f6abe0b1c.png)
/cc @roblourens
The text was updated successfully, but these errors were encountered: