Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trying to combinemultiple lazy loaded files kills julia #481

Open
felixcremer opened this issue Jul 25, 2023 · 1 comment
Open

Trying to combinemultiple lazy loaded files kills julia #481

felixcremer opened this issue Jul 25, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@felixcremer
Copy link
Contributor

When I try to combine multiple large files that are loaded lazily into a RasterSeries into a new Raster this kills my Julia process.
I suspect, that the views load all the data from disk into RAM and that gets Julia killed with OOM.
Could we have a lazy version of combine so that only the metadata is handled and the data will be loaded later when it is actually needed?

I will post a MWE that breaks my julia process tomorrow.

@rafaqz
Copy link
Owner

rafaqz commented Jul 25, 2023

I have a PR to DiskArrays that adds lazy cat, which should be a solution to this?

If you want to review it ;)

@rafaqz rafaqz added the bug Something isn't working label May 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants