-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Designing a Datastore
class (for handling different storage backends for datasets)
#58
Comments
Methods:
Attributes:
|
@sadamov I made a dataclasses based class structure which matches the config yaml file you've made: https://github.com/leifdenby/neural-lam/blob/mllam-dataloader/neural_lam/multizarr_datastore_config.py I will explain this in more detail tomorrow, but the idea is that by using dataclasses the config structure is stored in code, it is validated, and the nested structure (enabling the dot-syntax nesting that you achieve with |
DataStorage
class (for handling different storage backends for datasets)Datastore
class (for handling different storage backends for datasets)
I am in the process of reading through @sadamov's PR #54 on using zarr-based datasets in
neural-lam
and I am going to use this issue to write down some notes. Everyone is free to read-along, but this will only over time become a coherent piece of information, so it is probably best to wait until I comment directly on #54.Uses of current
neural_lam.config.Config
attributes and methods outside the class itself:The text was updated successfully, but these errors were encountered: