-
Notifications
You must be signed in to change notification settings - Fork 421
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Duplicated lists of dependencies #3370
Comments
I'm a big fan of solution 1, keep things as simple as possible. We only publish one package anyway, right? If anybody wants to make a custom build it should still be just as simple to keep doing that. |
Something to consider, with |
Our
requirements.txt
is mainly used as a lock / constraints file. We use it everywhere withpip install -c requirements.txt
(except in the docker file, see PR).pip-compile
is not able to generate a constraints file from all mentioned dependencies inpyproject.toml
(main + extras).pip
does not support extras dependencies in constraints files (that's why we havestrip-extras = true
), and does not support hashes when installing the package locally in editable mode (that's why we havegenerate-hashes = false
). We need editable mode to be able to make changes without having to reinstal the package between two tests runs for example.In #3359, we took this approach:
pyproject.toml
requirements.in
requirements.txt
generated usingpip-compile
.Ideally we would to avoid this duplication, and have one source of truth for all kinds of dependencies. Then have a constraints/lock file for reproducible builds/tests/installations.
Solution #0: status quo
This duplication is fine. The list of package does not change so often.
Solution #1: Get rid of extra dependencies
Simplify everything, and install all dependencies instead of having
monitoring
,postgresql
,memcached
extras.We would still need a distinction for development dependencies, but that's straightforward and doesn't really require pinning.
Solution #2: split into several
*.in
filessetuptools
is able to read*.in
files frompyproject.toml
. That's what we do in kinto-attachmentWe could have:
requirements.in
requirements-postgresql.in
requirements-memcached.in
requirements-monitoring.in
requirements-test.in
requirements-dev.in
The source of truth for the list of dependencies would be these
.in
files.Solution #3: use poetry
poetry
has a simple.Piplock
feature, and has many features around fine-grained installation of extra dependencies and groups.But
poetry
does not support the[project]
section (yet).We would have to move away from setuptools and setuptools-scm, and use a
[tools.poetry]
section.The text was updated successfully, but these errors were encountered: