-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Retroactively shard dataset into subdatasets #2978
Labels
Comments
Related: datalad/datalad#3554 |
do you have examples of already existing openneuro datasets and how would you like to split? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What would you like to see added?
If a dataset becomes too large, it would be good to turn subject directories into submodules. For tagging purposes, we would need to find some way of identifying the old and new commits (grafting?).
At least for datasets with new subjects, one possibility would be to shard new subjects, but new sessions within existing subjects may or may not need to be handled differently.
Derivatives and sourcedata would also be candidates for sharding.
Alternatives
No response
Do you have any interest in helping implement the feature?
No
Additional information / screenshots
No response
The text was updated successfully, but these errors were encountered: