-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fillna removes non-indexed coordinates in some cases #9124
Comments
Thanks for opening your first issue here at xarray! Be sure to follow the issue template! |
This does seem like confusing behavior. We'd def welcome a fix. |
So far I just have a workaround for our use case which merged the additional coordinates independently (treat them like variables). I looked into the xarray code, but without sufficient time to understand it, I think chances are high that I break more than I fix. I probably won't have time to dig deeper before November, but if it's still open then I'll take a look. |
What happened?
fillna
(and other functions which need aligning) remove non-indexed extra coordinates (along a dimensions which also have an indexed coordinates)Below I have an example with
fillna
, But I have experienced similar things withcombine_first
and assume it will happen whenever alignment is needed. More on what I think is happening below.What did you expect to happen?
I expected the extra coordinate to remain in place as it was defined consistently in both datasets. Alternatively an error message would also help to understand what's going on. Currently it's dropped silently.
Minimal Complete Verifiable Example
MVCE confirmation
Relevant log output
Anything else we need to know?
The problem only occurs when the
.loc
is done on the dimension with the additional coordinate. I think the reason for the problem is the following:align
fills the additional coordinate from the smaller dataset (same for dataarray) withnp.nan
to expand it to the larger dataset. Later in the process the aligned coordinates are passed tomerge_coordinates_without_align
which removes the additional coordinate as it containsnp.nan
in one of the datasets where the other dataset has values.So the non-index coordinates are neither combined like the indexed coordinates, nor filled like the data variables.
Below a short example involving only
align
andmerge_coordinates_without_align
.Environment
INSTALLED VERSIONS
commit: None
python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
python-bits: 64
OS: Linux
OS-release: 6.5.0-35-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.2
libnetcdf: None
xarray: 2024.6.0
pandas: 2.2.2
numpy: 1.26.4
scipy: 1.13.1
netCDF4: None
pydap: None
h5netcdf: 1.3.0
h5py: 3.11.0
zarr: None
cftime: None
nc_time_axis: None
iris: None
bottleneck: 1.3.8
dask: 2023.12.1
distributed: None
matplotlib: 3.9.0
cartopy: None
seaborn: None
numbagg: None
fsspec: 2024.6.0
cupy: None
pint: 0.24
sparse: None
flox: None
numpy_groupies: None
setuptools: 70.0.0
pip: 24.0
conda: None
pytest: 7.4.4
mypy: 1.10.0
IPython: 8.25.0
sphinx: 5.3.0
The text was updated successfully, but these errors were encountered: