Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partitions #208

Open
vchuravy opened this issue Aug 3, 2019 · 0 comments
Open

Partitions #208

vchuravy opened this issue Aug 3, 2019 · 0 comments

Comments

@vchuravy
Copy link
Member

vchuravy commented Aug 3, 2019

Right now DArray is both the notion of where and the data. I propose to factor out the where into it's own Partitons type, so that we can have a type-hierarchy of partitions, from generic n-dim hypercubes, to partitionings that are only distributed across one axes, to block cyclic or cartesian partitioning.

One important improvement would be that a process can hold more than one chunk and that the chunk array type can be heterogeneous across the DArray allowing us to mix something like a FillArray in and represent
block sparsity more efficiently.

Partitions should support reasoning about moving data from A to B, including this data is local (even though it is a different chunk).

@vchuravy vchuravy self-assigned this Aug 3, 2019
@vchuravy vchuravy added this to To do in DArray NG via automation Aug 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
DArray NG
  
To do
Development

No branches or pull requests

1 participant