-
Notifications
You must be signed in to change notification settings - Fork 496
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Slow seeking through large file #288
Comments
Is it any faster if you don't do it through SSHFS (if you run the program on your remote machine instead?) Just an idea to determine if the latency comes from the disk of your remote machine (because reading files are slow there) or if it's SSHFS or network speed that's causing the problem. Which version of SSHFS are you using? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
I wrote a program that extracts certain meta data from a large file (~100GB) mounted via sshfs. The extracted data is just about 1MB but this data is spread in many but small chunks across the large file at known byte offsets.
Now the problem is that seeking through this file takes longer than expected. So I believe too much (unwanted) bytes are read due to too large buffer/block sizes or because of some readahead optimization.
I already tried to set
-o no_readahead
and to reduce-o max_read=1000
. Both don't seem to have an effect in my case. Has anyone an idea what else I could try? Does anyone know whether there is also some buffering or large block sizes in the kernel or on the server side?Thank you very much in advance
The text was updated successfully, but these errors were encountered: