Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError when formatting sparse-backed DataArray in f-string #6822

Closed
4 tasks done
khaeru opened this issue Jul 25, 2022 · 2 comments · Fixed by #6823
Closed
4 tasks done

RuntimeError when formatting sparse-backed DataArray in f-string #6822

khaeru opened this issue Jul 25, 2022 · 2 comments · Fixed by #6823

Comments

@khaeru
Copy link

khaeru commented Jul 25, 2022

What happened?

On upgrading from xarray 2022.3.0 to 2022.6.0, f-string formatting of sparse-backed DataArray raises an exception.

What did you expect to happen?

Minimal Complete Verifiable Example

import pandas as pd
import xarray as xr

s = pd.Series(
    range(4),
    index=pd.MultiIndex.from_product([list("ab"), list("cd")]),
)

da = xr.DataArray.from_series(s, sparse=True)

print(f"{da}")

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

# xarray 2022.3.0:

<xarray.DataArray (level_0: 2, level_1: 2)>
<COO: shape=(2, 2), dtype=float64, nnz=4, fill_value=nan>                                         
Coordinates:                                     
  * level_0  (level_0) object 'a' 'b'
  * level_1  (level_1) object 'c' 'd'

# xarray 2022.6.0:

Traceback (most recent call last):                                                                
  File "/home/khaeru/bug.py", line 11, in <module>
    print(f"{da}")
  File "/home/khaeru/.local/lib/python3.10/site-packages/xarray/core/common.py", line 168, in __format__                                           
    return self.values.__format__(format_spec)
  File "/home/khaeru/.local/lib/python3.10/site-packages/xarray/core/dataarray.py", line 685, in values                                            
    return self.variable.values
  File "/home/khaeru/.local/lib/python3.10/site-packages/xarray/core/variable.py", line 527, in values                                             
    return _as_array_or_item(self._data)
  File "/home/khaeru/.local/lib/python3.10/site-packages/xarray/core/variable.py", line 267, in _as_array_or_item                                                                                   
    data = np.asarray(data)
  File "/home/khaeru/.local/lib/python3.10/site-packages/sparse/_sparse_array.py", line 229, in __array__                                                                                           
    raise RuntimeError(
RuntimeError: Cannot convert a sparse array to dense automatically. To manually densify, use the todense method.

Anything else we need to know?

Along with the versions below, I have confirmed the error occurs with both sparse 0.12 and sparse 0.13.

Environment

INSTALLED VERSIONS ------------------ commit: None python: 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] python-bits: 64 OS: Linux OS-release: 5.15.0-41-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_CA.UTF-8 LOCALE: ('en_CA', 'UTF-8') libhdf5: 1.10.7 libnetcdf: 4.8.1

xarray: 2022.6.0
pandas: 1.4.2
numpy: 1.22.4
scipy: 1.8.0
netCDF4: 1.5.8
pydap: None
h5netcdf: 0.12.0
h5py: 3.6.0
Nio: None
zarr: None
cftime: 1.5.2
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.3.2
dask: 2022.01.0+dfsg
distributed: 2022.01.0+ds.1
matplotlib: 3.5.1
cartopy: 0.20.2
seaborn: 0.11.2
numbagg: None
fsspec: 2022.01.0
cupy: None
pint: 0.18
sparse: 0.13.0
flox: None
numpy_groupies: None
setuptools: 62.1.0
pip: 22.0.2
conda: None
pytest: 6.2.5
IPython: 7.31.1
sphinx: 4.5.0

@khaeru khaeru added bug needs triage Issue that has not been reviewed by xarray team member labels Jul 25, 2022
@Illviljan
Copy link
Contributor

This comes from #5981, I think it's better to just let the backend array (numpy, dask, sparse etc.) determine how to handle string formatting instead of forcing to numpy arrays.

@dcherian dcherian added release-blocker regression and removed release-blocker needs triage Issue that has not been reviewed by xarray team member labels Jul 25, 2022
@khaeru
Copy link
Author

khaeru commented Aug 9, 2022

Thanks @Illviljan for the fix! 🙏🏾

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants