Skip to content

Commit

Permalink
Fix #70 by bypassing lazy loading of datasets (#71)
Browse files Browse the repository at this point in the history
By default, rioxarray lazyily loads datasets. On Windows, this causes
a PermissionError when the temporary directory containing the downloaded
files is deleted, as it is still open in rioxarray. To avoid that issue,
we now explicitly load the dataset into memory, allowing tempfiles to be
removed. This will potentially increase memory usage in some use cases, but
given the size limitations of datasets downloaded from Earth Engine, that
should never be a practical problem.
  • Loading branch information
aazuspan committed Jun 28, 2023
1 parent ff335b4 commit 77ba981
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion wxee/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,12 @@ def _dataarray_from_file(file: str, masked: bool, nodata: int) -> xr.DataArray:
The file name must follow the format "{dimension}.{coordinate}.{variable}.{extension}".
"""
da = rioxarray.open_rasterio(file)
with rioxarray.open_rasterio(file) as da:
# Load fully into memory rather than reading lazily from disk. This is needed to allow reading from tempfiles
# that will be deleted after the function returns. See https://github.com/corteva/rioxarray/issues/485 and
# https://github.com/aazuspan/wxee/issues/70.
da.load()

dim, coord, var = _parse_filename(file)

da = da.expand_dims({dim: [coord]}).rename(var).squeeze("band").drop_vars("band")
Expand Down

0 comments on commit 77ba981

Please sign in to comment.