Windows test failures for test_accessor_sliced_datacube and test_grdview_drapegrid_dataarray

[weiji14]

Description of the problem

Two unit tests have been failing consistenly on the Windows CI tests (both the regular and Dev versions). See e.g. https://github.com/GenericMappingTools/pygmt/runs/7890620920?check_suite_focus=true#step:11:661

Full code that generated the error

First test failure is due to this code, in particular, the os.remove line

pygmt/pygmt/tests/test_accessor.py

Lines 77 to 100 in 58a9b30

	@pytest.mark.skipif(
	gmt_version < Version("6.4.0"),
	reason="Upstream bug fixed in https://github.com/GenericMappingTools/gmt/pull/6615",
	)
	def test_accessor_sliced_datacube():
	"""
	Check that a 2D grid which is sliced from an n-dimensional datacube works
	with accessor methods.
	This is a regression test for
	https://github.com/GenericMappingTools/pygmt/issues/1578.
	"""
	try:
	fname = which(
	"https://github.com/pydata/xarray-data/raw/master/eraint_uvz.nc",
	download="u",
	)
	with xr.open_dataset(fname) as dataset:
	grid = dataset.sel(level=500, month=1, drop=True).z
	assert grid.gmt.registration == 0 # gridline registration
	assert grid.gmt.gtype == 1 # geographic coordinate type
	finally:
	os.remove(fname)

Second test failure is due to some mismatch between baseline and actual output image

Full error message

================================== FAILURES ===================================
________________________ test_accessor_sliced_datacube ________________________

    @pytest.mark.skipif(
        gmt_version < Version("6.4.0"),
        reason="Upstream bug fixed in https://github.com/GenericMappingTools/gmt/pull/6615",
    )
    def test_accessor_sliced_datacube():
        """
        Check that a 2D grid which is sliced from an n-dimensional datacube works
        with accessor methods.
    
        This is a regression test for
        https://github.com/GenericMappingTools/pygmt/issues/1578.
        """
        try:
            fname = which(
                "https://github.com/pydata/xarray-data/raw/master/eraint_uvz.nc",
                download="u",
            )
            with xr.open_dataset(fname) as dataset:
                grid = dataset.sel(level=500, month=1, drop=True).z
    
            assert grid.gmt.registration == 0  # gridline registration
            assert grid.gmt.gtype == 1  # geographic coordinate type
        finally:
>           os.remove(fname)
E           PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'eraint_uvz.nc'

C:\Miniconda3\envs\pygmt\lib\site-packages\pygmt\tests\test_accessor.py:100: PermissionError
---------------------------- Captured stderr call -----------------------------
Warning: WARNING]: No 3-D array in file D:\a\pygmt\pygmt\tmp-test-dir-with-unique-name\eraint_uvz.nc.  Selecting first 3-D slice in the 4-D array z
______________________ test_grdview_drapegrid_dataarray _______________________
Error: Image files did not match.
  RMS Value: 51.91310207540863
  Expected:  
    D:\a\pygmt\pygmt\tmp-test-dir-with-unique-name\results\pygmt.tests.test_grdview.test_grdview_drapegrid_dataarray\baseline.png
  Actual:    
    D:\a\pygmt\pygmt\tmp-test-dir-with-unique-name\results\pygmt.tests.test_grdview.test_grdview_drapegrid_dataarray\result.png
  Difference:
    D:\a\pygmt\pygmt\tmp-test-dir-with-unique-name\results\pygmt.tests.test_grdview.test_grdview_drapegrid_dataarray\result-failed-diff.png
  Tolerance: 
    2

System information

Please paste the output of python -c "import pygmt; pygmt.show_versions()":

PyGMT information:
  version: v0.7.1.dev43+g58a9b302
System information:
  python: 3.10.5 | packaged by conda-forge | (main, Jun 14 2022, 06:57:19) [MSC v.1929 64 bit (AMD64)]
  executable: C:\Miniconda3\envs\pygmt\python.exe
  machine: Windows-10-10.0.20348-SP0
Dependency information:
  numpy: 1.23.2
  pandas: 1.4.3
  xarray: 2022.6.0
  netCDF4: 1.6.0
  packaging: 21.3
  geopandas: 0.11.1
  ghostscript: 9.54.0
  gmt: 6.4.0
GMT library information:
  binary dir: C:/Miniconda3/envs/pygmt
  cores: 2
  grid layout: rows
  library path: C:/Miniconda3/envs/pygmt/Library/bin/gmt.dll
  padding: 2
  plugin dir: C:/Miniconda3/envs/pygmt/Library/bin/gmt_plugins
  share dir: C:/Miniconda3/envs/pygmt/Library/share/gmt
  version: 6.4.0

[seisman]

For the 2nd test failure, the equivalent PyGMT script is:

import pygmt
from pygmt.helpers.testing import load_static_earth_relief
grid = load_static_earth_relief()
xrgrid = pygmt.grdcut(grid=grid, region=[-55, -50, -18, -12])
drapegrid = 1.1 * xrgrid
fig = pygmt.Figure()
fig.grdview(grid=xrgrid, drapegrid=drapegrid, cmap="oleron", surftype="c")
fig.savefig("pygmt-test.png")

And the equivalent GMT bash script is:

gmt grdcut @static_earth_relief.nc -R-55/-50/-18/-12 -Ggrid.nc
gmt grdmath grid.nc 1.1 MUL = drape.nc
gmt grdview grid.nc -Gdrape.nc -Coleron -Qs -png gmt-test

The outputs are different on macOS (ignoring the colorbar which is automatically added in one-liner mode):

bash GMT	PyGMT

Our baseline image is the same as the right-side image, and the resulting image on Windows looks the same as the left-side image. So, I think:

1. the baseline image for test_grdview_drapegrid_dataarray is wrong
2. after correcting the baseline image, the test is likely to pass on Windows but fail on Linux and macOS, which may point to an upstream bug

[seisman]

Both failures are not easy to debug and are likely due to upstream bugs.

They are tracked by PR #2068 and #2073.

I think we should mark the two tests as xfail temporarily (#2075) so that all checks are green.

[seisman]

@weiji14 I believe the test_accessor_sliced_datacube failure on Windows is because GMT forgets to close the datacube NetCDF file after reading it. The bug was fixed in GenericMappingTools/gmt#7573.

Could you please trigger a new dev build on conda-forge so that we can verify it?

[weiji14]

Ok, new GMT 6.5.0.dev6+95c3431 build at conda-forge/gmt-feedstock#254.

[weiji14]

Ok, test_accessor_sliced_datacube has been fixed in #2073 when using GMT 6.5.0, still need to fix test_grdview_drapegrid_dataarray in #2068.