-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Slow or too much data? #4
Comments
Hello @ofionnad, INSAR4SM/insar4sm/inversion.py Line 24 in 9caa75d
Based on my understanding and some experimentation, the computational time increases a lot the more observations (number of images) you provide. This is because the inversion scheme uses combinations of the provided images. So the more images you provide the bigger the number of combinations it becomes. I suggest you to use to segment temporally our images from one dry season to the next dry one (e.g. a year span). Hopefully this will be helpful for speedup reasons without significant accuracy losses. I hope my suggestion will help you 😃 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi @kleok,
I am running your soil moisture code on a ISCE TopsStack. The soil inversion is taking a very long time however (after 1 day it is at 35% and 7-8 day completion).
The tops stack is about 750 GB in size, and my AOI is roughly 4-5 times larger than your example image of the ford dry lake, over a period of about 14 months of Sentinel-1 data.
(I previously ran it on a small 2km x 2km sample and it worked)
I am running it on a medium sized cluster (64 cores, 128 GB RAM).
It seems to be using all the cores, but they are only at 30-40% capacity.
It is only using about 1/3 of the available RAM.
Do you know what the bottleneck in the code would be? Read/write speeds?
If it is not # cores, RAM, or IO speeds, then it is possibly the code somewhere?
It would be great if I could achieve some sort of speedup. If you could point me in the right direction that would be great.
The text was updated successfully, but these errors were encountered: