Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Investigate if we can use Pytorch webdataset library for accessing data #213

Closed
ant0nsc opened this issue Sep 10, 2020 · 0 comments
Closed

Investigate if we can use Pytorch webdataset library for accessing data #213

ant0nsc opened this issue Sep 10, 2020 · 0 comments
Labels
architecture Anything around possible extensions or re-structuring of code large scale Features needed for scale-out to very large datasets

Comments

@ant0nsc
Copy link
Contributor

ant0nsc commented Sep 10, 2020 •
edited by azure-boards bot
Loading

https://pytorch.org/blog/efficient-pytorch-io-library-for-large-datasets-many-files-many-gpus/

AB#3938
@ant0nsc ant0nsc added architecture Anything around possible extensions or re-structuring of code large scale Features needed for scale-out to very large datasets labels Sep 10, 2020
@ant0nsc ant0nsc added no changelog needed CHANGELOG.md does not need to be updated in this PR and removed no changelog needed CHANGELOG.md does not need to be updated in this PR labels Apr 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
architecture Anything around possible extensions or re-structuring of code large scale Features needed for scale-out to very large datasets
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ant0nsc @peterhessey