Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardise access to multiple single slice arrays and a single multidimensional arrays #70

Closed
rafaqz opened this issue Oct 4, 2020 · 3 comments · Fixed by #168
Closed

Comments

@rafaqz
Copy link
Owner

rafaqz commented Oct 4, 2020

This problem comes up with NetCDF files with a time dimension, and similar data in tif that are just single slices in time.

You can make a series that holds multiple pointers to the same netcdf with a different window for each, and just point to each tif file separately. But it's not an efficient way to load the netcdf. Possibly using CachedStack would get around this somehow.

There is also the complication of multiple netcdf along a time dimension that also have their own time dimension - like annual files that have daily data. How can this be standardised to be accessed just like separate daily tif files for the same sequences? Indexing with just one Ti(x) object for both cases is the goal.

It seems like the solution to this is to fully abstract the child objects. So indexing should be:

series[Ti(Date(2001, 2, 4), :tmax, Lat(Between(10, 25)), Lon(Between(130, 150))]

Instead of :

series[Ti(Date(2001, 2, 4)][:tmax][Lat(Between(10, 25)), Lon(Between(130, 150))]

Multiple calls to getindex break the abstraction and mean we don't have control over doing things in either a general or performant way.

The tricky part will be when we need to index with Ti twice - with Contains for the series and At for the array. These dimensions may need some marker trait to specify that they are Partial or something. This only really makes sense for Sampled so they can just inherit from AbstractSampled? (It's seeming like we need to move to traits instead of inheritance one day)

@rafaqz rafaqz changed the title Sandardise access to multiple single slice arrays and a single multidimensional arrays Standardise access to multiple single slice arrays and a single multidimensional arrays Oct 7, 2020
@rafaqz
Copy link
Owner Author

rafaqz commented Oct 12, 2020

At, Near or Contains would be converted to Contains for the parent - which would have to specifically be Intervals. Between is more difficult as the child objects may be distributed across multiple parent objects - say three year of daily data where the parent object holds one year of data.

Probably initially Between would not be implemented.

@rafaqz
Copy link
Owner Author

rafaqz commented Oct 15, 2020

The Sampling trait could be ParentIntervals <: AbstractIntervals, and most dispatch on Intervals would be swapped to AbstractIntervals.

getindex for GeoSeries would need to be special-cased, as for GeoStack, to pass unused (via otherdims) or ParentIntervals arguments to the child object.

@rafaqz
Copy link
Owner Author

rafaqz commented May 21, 2021

The simple way to do this is to slice views of the lower dimension moving it to the sequence. Then none of the above applies.

slice(ser; dims=Ti)

Would do it. The catenated time dimensions of the contained geoarrays/stacks would become a series dimension.

stacks/geoarrays would then be views of a single slice in the time dimension.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant