You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
All our blocked operations (BlockReduce, BlockedMean, block_split) rely on KD-trees from Scipy or pykdtree to do distance calculations. None of these support calculating great circle distances so we have to project geographic the data before doing these operations.
The scikit-learn implementation of KD-trees has support for a harversine distance metric which calculates great circle distances on a unit sphere. No scaling would be required since we don't need to consider absolute distances at all, only relative distances between points. We could use the scikit-learn KDTree class with harversine metric instead of Scipy or pykdtree in all the functions/classes above if a keyword spherical=True is given.
Alternatively, we can create spherical versions of these classes/functions. I'm not entirely sure that is best since it would create quite a bit of duplicated code. But for documenting it might be best. We can have a section of the API dedicated to spherical coordinates with these classes and #173 as well.
Another option is to have spherical=True in block_split only (which is mostly used internally) and then have spherical versions of the classes only. As I write this, I kind of like this option best.
With this option and #173 we can have fully spherical processing without need of projections.
How to do this (each step should be a separate PR):
Start by adding spherical support to block_split. If spherical=True, use scikit-learn's KDTree with harversine metric. That is mostly it. Would need careful testing (near the poles for example).
Add BlockReduceSpherical which uses block_split with spherical=True by default. Should be a subclass of BlockReduce or might require creating a separate base class. Most work will be in creating the base classes and writing docstrings and examples.
Add BlockMeanSpherical which builds on top of BlockReduceSpherical.
Are you willing to help implement and maintain this feature? Yes but would welcome anyone who wants to give it a shot.
The text was updated successfully, but these errors were encountered:
Description of the desired feature
All our blocked operations (
BlockReduce
,BlockedMean
,block_split
) rely on KD-trees from Scipy or pykdtree to do distance calculations. None of these support calculating great circle distances so we have to project geographic the data before doing these operations.The scikit-learn implementation of KD-trees has support for a
harversine
distance metric which calculates great circle distances on a unit sphere. No scaling would be required since we don't need to consider absolute distances at all, only relative distances between points. We could use the scikit-learnKDTree
class withharversine
metric instead of Scipy or pykdtree in all the functions/classes above if a keywordspherical=True
is given.Alternatively, we can create spherical versions of these classes/functions. I'm not entirely sure that is best since it would create quite a bit of duplicated code. But for documenting it might be best. We can have a section of the API dedicated to spherical coordinates with these classes and #173 as well.
Another option is to have
spherical=True
inblock_split
only (which is mostly used internally) and then have spherical versions of the classes only. As I write this, I kind of like this option best.With this option and #173 we can have fully spherical processing without need of projections.
How to do this (each step should be a separate PR):
block_split
. Ifspherical=True
, use scikit-learn'sKDTree
withharversine
metric. That is mostly it. Would need careful testing (near the poles for example).BlockReduceSpherical
which usesblock_split
withspherical=True
by default. Should be a subclass ofBlockReduce
or might require creating a separate base class. Most work will be in creating the base classes and writing docstrings and examples.BlockMeanSpherical
which builds on top ofBlockReduceSpherical
.Are you willing to help implement and maintain this feature? Yes but would welcome anyone who wants to give it a shot.
The text was updated successfully, but these errors were encountered: