testing.assert_allclose on string dtype gives error with NumPy 2.0.0

[pont-us]

What happened?

This error came up in an nc2zarr unit test in a nightly CI run, just after the release of NumPy 2.0.0. This is the test in question, and here is the call to xarray.testing.allclose which triggers the error:

AttributeError: Module 'numpy._core' has no attribute 'defchararray'

What did you expect to happen?

We expected that xarray.testing.allclose would perform as stated rather than producing an error.

Minimal Complete Verifiable Example

import numpy as np
import xarray as xr

xr.testing.assert_allclose(
    da := xr.DataArray(np.array("a", dtype="|S1")),
    da.copy()
)

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• New issue — a search of GitHub Issues suggests this is not a duplicate.
• Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

Traceback (most recent call last):
  File "/home/pont/allclose_error.py", line 6, in <module>
    xr.testing.assert_allclose(
  File "/home/pont/loc/repos/xarray/xarray/testing/assertions.py", line 32, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/pont/loc/repos/xarray/xarray/testing/assertions.py", line 282, in assert_allclose
    ) and compat_variable(a.variable, b.variable)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pont/loc/repos/xarray/xarray/testing/assertions.py", line 274, in compat_variable
    return a.dims == b.dims and (a._data is b._data or equiv(a.data, b.data))
                                                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/pont/loc/repos/xarray/xarray/testing/assertions.py", line 45, in _data_allclose_or_equiv
    arr1 = _decode_string_data(arr1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pont/loc/repos/xarray/xarray/testing/assertions.py", line 39, in _decode_string_data
    return np.core.defchararray.decode(data, "utf-8", "replace")
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/pont/mambaforge/envs/xrrepo/lib/python3.12/site-packages/numpy/core/__init__.py", line 30, in __getattr__
    attr = getattr(_core, attr_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pont/mambaforge/envs/xrrepo/lib/python3.12/site-packages/numpy/_core/__init__.py", line 167, in __getattr__
    raise AttributeError(f"Module {__name__!r} has no attribute {name!r}")
AttributeError: Module 'numpy._core' has no attribute 'defchararray'

Anything else we need to know?

Error appears to be due to a failed import of defchararray from a path that's no longer valid in NumPy 2. numpy._core.defchararray says "The preferred alias for defchararray is numpy.char.". Replacing np.core.defchararray with np.char.chararray in xarray.testing.assertions seems to fix it.

Environment

INSTALLED VERSIONS

commit: None
python: 3.12.4 | packaged by conda-forge | (main, Jun 17 2024, 10:23:07) [GCC 12.3.0]
python-bits: 64
OS: Linux
OS-release: 5.15.0-107-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: ('en_GB', 'UTF-8')
libhdf5: 1.14.2
libnetcdf: 4.9.3-development

xarray: 0.1.dev5453+g872c1c5
pandas: 2.2.2
numpy: 2.0.0
scipy: 1.13.1
netCDF4: 1.7.1
pydap: None
h5netcdf: 1.3.0
h5py: 3.11.0
zarr: 2.18.2
cftime: 1.6.4
nc_time_axis: 1.4.1
iris: None
bottleneck: 1.4.0
dask: 2024.6.2
distributed: 2024.6.2
matplotlib: 3.9.0
cartopy: None
seaborn: 0.13.2
numbagg: 0.8.1
fsspec: 2024.6.0
cupy: None
pint: None
sparse: None
flox: 0.9.8
numpy_groupies: 0.11.1
setuptools: 70.1.0
pip: 24.0
conda: None
pytest: 8.2.2
mypy: 1.10.0
IPython: 8.25.0
sphinx: None

[keewis]

Thanks for the report, @pont-us.

Only the most recent version of xarray is compatible with numpy>=2 (I remember fixing that kind of issue in the past). Your environment shows that you installed from a git repo, can you make sure you're at main?

[keewis]

actually, I can reproduce this locally... maybe our tests are not as extensive as I had hoped they are?

[keewis]

good news is, this can be easily fixed by replacing np.core.defchararray.decode with np.char.decode. We'll probably need to extend our tests, though.

[pont-us]

Your environment shows that you installed from a git repo, can you make sure you're at main?

Yes, I'm on main at commit 872c1c5.