PermissionError when using to_xarray on Windows

[aazuspan]

It looks like there's a bug in the temp directory handling for 0.4.0 that only seems to affect Windows (and therefore not the CI workflow). It's also possible this was introduced with Windows 11, since I upgraded recently.

This would be handled automatically by resolving #19, so if this isn't a quick fix I may need to wait for that.

Reproducing

import wxee
import ee

wxee.Initialize()

img = ee.ImageCollection("IDAHO_EPSCOR/GRIDMET").first()
img.wx.to_xarray(scale=100_000)

Raises:

---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
File [c:\ProgramData\Miniconda3\envs\ee\lib\shutil.py:627](file:///C:/ProgramData/Miniconda3/envs/ee/lib/shutil.py:627), in _rmtree_unsafe(path, onerror)
    626 try:
--> 627     os.unlink(fullname)
    628 except OSError:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\az\\AppData\\Local\\Temp\\wxee_tmpstdzug4g\\IDAHO_EPSCOR_GRIDMET_19790101.time.19790101T060000.pr.tif'

During handling of the above exception, another exception occurred:

PermissionError                           Traceback (most recent call last)
File [c:\ProgramData\Miniconda3\envs\ee\lib\tempfile.py:805](file:///C:/ProgramData/Miniconda3/envs/ee/lib/tempfile.py:805), in TemporaryDirectory._rmtree..onerror(func, path, exc_info)
    804 try:
--> 805     _os.unlink(path)
    806 # PermissionError is raised on FreeBSD for directories

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\az\\AppData\\Local\\Temp\\wxee_tmpstdzug4g\\IDAHO_EPSCOR_GRIDMET_19790101.time.19790101T060000.pr.tif'

During handling of the above exception, another exception occurred:

NotADirectoryError                        Traceback (most recent call last)
[c:\Users\az\wxee\ee_computepixels.ipynb](file:///C:/Users/az/wxee/ee_computepixels.ipynb) Cell 9 in ()
      [4](vscode-notebook-cell:/c%3A/Users/az/wxee/ee_computepixels.ipynb#X11sZmlsZQ%3D%3D?line=3) wxee.Initialize()
      [6](vscode-notebook-cell:/c%3A/Users/az/wxee/ee_computepixels.ipynb#X11sZmlsZQ%3D%3D?line=5) img = ee.ImageCollection("IDAHO_EPSCOR/GRIDMET").first()
----> [7](vscode-notebook-cell:/c%3A/Users/az/wxee/ee_computepixels.ipynb#X11sZmlsZQ%3D%3D?line=6) img.wx.to_xarray(scale=100_000)

File [c:\Users\az\wxee\wxee\image.py:90](file:///C:/Users/az/wxee/wxee/image.py:90), in Image.to_xarray(self, path, region, scale, crs, masked, nodata, progress, max_attempts)
     77 with tempfile.TemporaryDirectory(prefix=constants.TMP_PREFIX) as tmp:
     78     files = self.to_tif(
     79         out_dir=tmp,
     80         region=region,
   (...)
     87         progress=progress,
     88     )
---> 90     ds = _dataset_from_files(files, masked, nodata)
     92 if path:
     93     msg = (
     94         "The path argument is deprecated and will be removed in a future "
     95         "release. Use the `xarray.Dataset.to_netcdf` method instead."
     96     )

File [c:\ProgramData\Miniconda3\envs\ee\lib\tempfile.py:830](file:///C:/ProgramData/Miniconda3/envs/ee/lib/tempfile.py:830), in TemporaryDirectory.__exit__(self, exc, value, tb)
    829 def __exit__(self, exc, value, tb):
--> 830     self.cleanup()

File [c:\ProgramData\Miniconda3\envs\ee\lib\tempfile.py:834](file:///C:/ProgramData/Miniconda3/envs/ee/lib/tempfile.py:834), in TemporaryDirectory.cleanup(self)
    832 def cleanup(self):
    833     if self._finalizer.detach():
--> 834         self._rmtree(self.name)

File [c:\ProgramData\Miniconda3\envs\ee\lib\tempfile.py:816](file:///C:/ProgramData/Miniconda3/envs/ee/lib/tempfile.py:816), in TemporaryDirectory._rmtree(cls, name)
    813     else:
    814         raise
--> 816 _shutil.rmtree(name, onerror=onerror)

File [c:\ProgramData\Miniconda3\envs\ee\lib\shutil.py:759](file:///C:/ProgramData/Miniconda3/envs/ee/lib/shutil.py:759), in rmtree(path, ignore_errors, onerror)
    757     # can't continue even if onerror hook returns
    758     return
--> 759 return _rmtree_unsafe(path, onerror)

File [c:\ProgramData\Miniconda3\envs\ee\lib\shutil.py:629](file:///C:/ProgramData/Miniconda3/envs/ee/lib/shutil.py:629), in _rmtree_unsafe(path, onerror)
    627             os.unlink(fullname)
    628         except OSError:
--> 629             onerror(os.unlink, fullname, sys.exc_info())
    630 try:
    631     os.rmdir(path)

File [c:\ProgramData\Miniconda3\envs\ee\lib\tempfile.py:808](file:///C:/ProgramData/Miniconda3/envs/ee/lib/tempfile.py:808), in TemporaryDirectory._rmtree..onerror(func, path, exc_info)
    806     # PermissionError is raised on FreeBSD for directories
    807     except (IsADirectoryError, PermissionError):
--> 808         cls._rmtree(path)
    809 except FileNotFoundError:
    810     pass

File [c:\ProgramData\Miniconda3\envs\ee\lib\tempfile.py:816](file:///C:/ProgramData/Miniconda3/envs/ee/lib/tempfile.py:816), in TemporaryDirectory._rmtree(cls, name)
    813     else:
    814         raise
--> 816 _shutil.rmtree(name, onerror=onerror)

File [c:\ProgramData\Miniconda3\envs\ee\lib\shutil.py:759](file:///C:/ProgramData/Miniconda3/envs/ee/lib/shutil.py:759), in rmtree(path, ignore_errors, onerror)
    757     # can't continue even if onerror hook returns
    758     return
--> 759 return _rmtree_unsafe(path, onerror)

File [c:\ProgramData\Miniconda3\envs\ee\lib\shutil.py:610](file:///C:/ProgramData/Miniconda3/envs/ee/lib/shutil.py:610), in _rmtree_unsafe(path, onerror)
    608         entries = list(scandir_it)
    609 except OSError:
--> 610     onerror(os.scandir, path, sys.exc_info())
    611     entries = []
    612 for entry in entries:

File [c:\ProgramData\Miniconda3\envs\ee\lib\shutil.py:607](file:///C:/ProgramData/Miniconda3/envs/ee/lib/shutil.py:607), in _rmtree_unsafe(path, onerror)
    605 def _rmtree_unsafe(path, onerror):
    606     try:
--> 607         with os.scandir(path) as scandir_it:
    608             entries = list(scandir_it)
    609     except OSError:

NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\az\\AppData\\Local\\Temp\\wxee_tmpstdzug4g\\IDAHO_EPSCOR_GRIDMET_19790101.time.19790101T060000.pr.tif'

[aazuspan]

This was apparently caused by the switch from xarray.open_rasterio to rioxarray.open_rasterio in #63, confirmed by reverting that change. I'm guessing this is a difference in how they lock opened files. As mentioned above, removing the need for temp files by directly downloading would resolve this. Otherwise, I need to dig a little deeper into rioxarray to see if I can close the file prior to temp directory cleanup to fix this.

Possibly relevant issue.