Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Layer and sequence interpolation for interop between hourly temperature and min/max datasets (and similar) #65

Closed
rafaqz opened this issue Sep 14, 2020 · 1 comment

Comments

@rafaqz
Copy link
Owner

rafaqz commented Sep 14, 2020

It would be very useful to be able to be able to standardise time-series data of environmental variables between different formats,
as suggested by @jamesmaino

Some datasets like worldclim use tmax/tmin layers on a daily basis,
others use hourly/3 hourly temperature records, like SMAP.

Models written to work with hourly/3 hourly temperature can't use worldclim-style min/max data. But this could be resolved with a cyclic interpolator (sinosoidal or more complex) that presented data as if it was a regular GeoSeries containing regular GeoStacks with regular GeoArray layers.

There are a few components to this:

  • Interpolator wrapper: holds 2 arrays and fraction f to interpolate with
  • a function for calculating f for any value in the time series
  • an internal mutable wrapper inside the interpolators to make sure data shared across multiple timesteps are only loaded once -
    and only copied to the GPU once.

There are also some performance gains to be had interpolating between files, especially when running models on the GPU. Loading file data and moving it to the GPU is often a lot more costly than doing anything to them - so loading files to GPU only once should be the goal.

We may also need a way of specifying the kind of array the data is loaded to initially, such as CuArray.

@rafaqz
Copy link
Owner Author

rafaqz commented Oct 1, 2020

This will be a separate package for now: https://github.com/cesaraustralia/TimeInterpolatedGeoData.jl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant