Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce CFA file size with small CFA variable HDF chunks #739

Closed
davidhassell opened this issue Mar 14, 2024 · 0 comments · Fixed by #743
Closed

Reduce CFA file size with small CFA variable HDF chunks #739

davidhassell opened this issue Mar 14, 2024 · 0 comments · Fixed by #743
Labels
CFA Relating to CFA datasets enhancement New feature or request netCDF write Relating to writing netCDF datasets
Milestone
3.16.2

Comments

@davidhassell
Copy link
Collaborator

davidhassell commented Mar 14, 2024 •
edited
Loading

CFA netCDF variables can be very small, much smaller than a default HDF chunk size (= 1 MiB). when there are a lot of these, this can unnecessarily inflate the output CFA netCDF file size.

We could reset the HDF chunk size for these variables to be the variable shape. Even a very large aggregation has a relatively small CFA variables (by design!), so this shouldn't ever be a problem.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CFA Relating to CFA datasets enhancement New feature or request netCDF write Relating to writing netCDF datasets
Projects
None yet
Milestone
3.16.2
1 participant
@davidhassell