EMMA: Datasets

To use this package in your project, you can install it by running

poetry add git+https://github.com/emma-simbot/datasets.git

You can then just import from emma_datasets or run commands using the CLI with

python -m emma_datasets

When running commands for emma_datasets, you can append --help to get more information on the commands and any arguments available to you.

This is organised in very similarly to structure from the Lightning-Hydra-Template to facilitate reproducible research code.

• scripts — sh scripts to run experiments
• notebooks — Jupyter notebook for analysis and exploration
• storage — data for training/inference (and maybe use symlinks to point to other parts of the file system)
• tests — pytest scripts to verify the code
• src/emma_datasets — where the main code lives

For more detail on how to use this library, check out the following specific pages on:

• Installing and preparing your developer environment
• Downloading and organising the raw datasets
• Adding a new downstream dataset