Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add conda environment.yml (and fix requirements.txt) #8

Merged
merged 3 commits into from
Sep 23, 2022

Conversation

sradc
Copy link
Contributor

@sradc sradc commented Sep 21, 2022

Created a PR in case this is useful.

Adds a minimal conda environment.yml, which has two benefits: one, an easy installation, and two, it tracks all package versions to make environment reproducibility easier X years in the future. (Plus it includes ffmpeg in addition to ffmpeg-python.)

To install from the yml: conda env create -f environment.yml.

@jongwook
Copy link
Collaborator

Thanks for the suggestions, but this package is intentionally designed to be not too picky in terms of dependencies, so that it can be imported into existing ML project without wrecking their dependency tree. Similarly, I didn't want to require the users to depend on conda and conda-forge (which is a great tool I use daily), to make it amenable to install in other kinds of environments such as python from the official installer, homebrew, APT, or virtualenv/pyenv on top of them which provides similar environments as conda does.

Reproducibility X years later would be great for individual experiments reported in a paper, but I'm trying to make this repo source-compatible with many Python/CUDA/PyTorch versions for the foreseeable future. Your environment.yml specifies Python 3.10, PyTorch 1.12.1, CUDA 11.3, etc., which is too narrower than the currently intended range of compatible versions, and these won't be a popular choice in, say, 5 years later.

PyPI renaming is a good catch, and pip was flexible enough that more_itertools gets redirected to more-itertools. Would you be okay if I merge only the commit bfd4f69?

@HenryLoenwind
Copy link

HenryLoenwind commented Sep 22, 2022

but this package is intentionally designed to be not too picky in terms of dependencies

Adding an environment.yml doesn't make the package picky, it only allows one to 1-click install it with a specific configuration. There's nothing in there that forces anyone to use it.

Edit: On the other hand, an env file that is specific to one operating system in exactly one flavour and version is rather useless.

@sradc
Copy link
Contributor Author

sradc commented Sep 22, 2022

Hey, happy for you to only merge bfd4f69!

As to the usefulness of the env, it would serve as a record that these exact package versions (for this operating system, (which is another dependency in a way)) worked with the codebase at the time it was committed. But feel free to leave out.

@jongwook jongwook merged commit a4fe05a into openai:main Sep 23, 2022
@jongwook
Copy link
Collaborator

Thank you for understanding!

I think it's definitely useful as a record that Whisper works on the exact set of dependencies listed in your environment.yml, but as one of the files in the top-level directory, I would like to avoid pinning to a specific platform/versions or having to maintain environment.yaml on every major platform and update them if some of the pinned packages turn out to have a security vulnerability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants