Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy iris.analysis.cartography.area_weights #5611

Closed
schlunma opened this issue Dec 6, 2023 · 1 comment · Fixed by #5658
Closed

Lazy iris.analysis.cartography.area_weights #5611

schlunma opened this issue Dec 6, 2023 · 1 comment · Fixed by #5658

Comments

@schlunma
Copy link
Contributor

schlunma commented Dec 6, 2023

✨ Feature Request

Currently, iris.analysis.cartography.area_weights always returns a numpy array. Depending on the shape of the input cube, this can use up a lot of memory. It would be really helpful to have a lazy version of it that returns a dask array.

Motivation

Using a dask distributed scheduler in ESMValTool in combination with a preprocessor that requires the calculation of area weights is currently not possible. A lazy version of iris.analysis.cartography.area_weights would solve that.

Design

Solving this is probably easy since broadcast_to_shape that is used to broadcast the 2D weights to the cube shape now supports dask arrays. I see two options here regarding the API:

  1. Add an additional keyword argument for iris.analysis.cartography.area_weights (maybe compute which is True by default?).
  2. Let the behavior depend on the input data, i.e., if the cube has lazy data, the function returns lazy data and vice versa.

I guess option 1. is preferable since it is fully backwards-compatible.

If we can agree on an implementation, I can open a PR.

@HGWright
Copy link
Contributor

@SciTools/peloton

@schlunma thanks for raising this and offering to put up a PR. We think option 1 is much preferred to option 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

4 participants