Make version in pyproject.toml optional, or at least validate it *after* the plugins are run

[tiangolo]

• I have searched the issues of this repo and believe that this is not a duplicate.
• I have searched the documentation and believe that my question is not covered.

Feature Request

Thanks for Poetry! I'm a big fan. 🎉

I wanted to be able to keep my package versions in __init__.py, and with the new alpha supporting plugins I was able to create a plugin to read the version from it dynamically: https://github.com/tiangolo/poetry-version-plugin

But currently, Poetry checks and verifies if the version is set in pyproject.toml, and if it is not set there, it terminates with an error.

That means that even though the plugin is taking care of setting the package version, a dummy "version" is required in pyproject.toml, even though it is not really used.

My feature request is to make the version field optional in pyproject.toml.

Or alternatively, check if the version is set in the Poetry object in memory after running the plugins. That way my plugin can set the version.

Motivation

If users try to use that plugin (or any other similar one), they would have to set a version just to trick Poetry into thinking the pyproject.toml has all the fields, while the version is not really used in the end. So, for the final result, it's an extra workaround step.

[finswimmer]

Hello @tiangolo,

thanks for FastAPI! 😃 🥇

In general I think it is a good idea to validate the pyproject.toml after running the plugins. Doing this we can make sure that this file is still valid.

On the other hand you are touching a topic that is discussed quite emotional in various places (e.g. in the discussion around PEP 621): How static should/must metadata of a package be?

My personal opinion is, that name, version and dependencies are clearly static metadata and shouldn't be declared dynamically. AFAIK many people trying to receive the version from a git tag,because a build and/or deployment is triggered by adding the version as a git tag, right? At my company we have a different approach: A merge into the master branch triggers the deployment. The pipeline reads the version from the pyproject.toml and adds the git tag. A check within the corresponding Merge Request assures that the version is not already used as a git tag.

Until now Poetry follows the principal that there should be only one source of truth for the metadata and it's the pyproject.toml. That's why I think we should not make version optional. But that is just a personal opinion and doesn't necessarily match the opinion of the other maintainers. @sdispater: Would be good if you could add your opinion here.

fin swimmer

[tiangolo]

Thanks for the response @finswimmer!

Yep, I know it's a complex topic. 😅

And I understand that the expected main use case would be to set the version in pyproject.toml, and doing it otherwise could be considered a "corner case".

Let me share my points of view and motivations here:

---

I'm actually not that interested in setting the version in a git tag as much as in the __init__.py file.

I tried with a couple of packages using the sort-of-official trick you describe here: #2366 (comment)

Nevertheless, I ended up having issues with conflicts with other packages also requiring the backport importlib-metadata, with different version ranges, etc. Apart from the caveat that setting the version in __init__.py that way is somewhat cumbersome, as it requires try blocks checking for imports, etc. And it could also cause issues if the package is not actually installed, for example when using Poetry to manage a project and its dependencies directly in Docker without installing it.

---

That's why I got super excited with the new plugin system and being able to do that in my projects. ✨

With the plugin, I can make it easier for me and for others, especially newcomers, to build packages with Python.

That's one of the things I love about Poetry, that it makes it super simple and straightforward to manage projects, dependencies, and also packages. And after learning to use it just for dependencies, it's very straightforward to build an actual package, without having to use setuptools (that I always feared 😂 ).

---

I guess that the general intention of the Python community in the future is to stop declaring a __version__ in general, and just use importlib.metadata, but I feel there's probably still a bunch of people (:raising_hand: ) used to getting the version of a package from a __version__ variable.

But then, let's say that the general consensus or the general accepted approach was to not declare a __version__ variable and just declare the version in pyproject.toml (which is also perfectly valid, similar to the JS/NPM ecosystems).

Then, in that case, with no __version__ variable and only importlib.metadata, I think it would be unfortunate for users of Python < 3.8, that for them to get the version of a package they installed they currently should install another backported package, just to get the version.

---

As a side note, my plan is to move FastAPI, Typer, and everything else to Poetry using my plugin. 🚀

For me, the workaround of setting version = "0" in pyproject.toml is okay for my use cases.

My request is mainly because I feel it could be confusing for newcomers building a new package.

[dbanty]

FWIW I, too have encountered issues with using importlib.metadata with Poetry. Specifically for FastAPI apps, I want to have the version of the package available to the server at runtime so it shows up in the docs & OpenAPI document. Unfortunately, poetry install seems to be capable of retaining multiple versions of the same package. This lead to an issue in CI where restoring a cached virtualenv to build & deploy would actually sometimes grab the wrong version number from importlib.metadata 😨 .

This is probably a niche edge case, as the version is for an app, not a published library... but it's a real world example of the importlib method not working. Our solution was to parse pyproject.toml manually, grab the version number from it, and inject that into a separate version.py in the build process which is... messy and clunky to say the least 😓. Having a source of truth which is always available to the Python code at runtime (e.g. __version__ in __init.py__) would have made that much cleaner.

[Conchylicultor]

My personal opinion is, that name, version and dependencies are clearly static metadata and shouldn't be declared dynamically.

@finswimmer , I think there are many use-cases of dynamic versioning. For example, our project require nightly releases of our library. Updating the version manually every day is just not an option. So we need some automated way.

[edgarrmondragon]

Updating the version manually every day is just not an option.

@Conchylicultor I wonder why that's the case. Wouldn't running an automated version bump, e.g. poetry version $(poetry version --short)-nightly-$(date '+%s'), help with your use case?

[pbarker]

@Conchylicultor I wonder why that's the case. Wouldn't running an automated version bump, e.g. poetry version $(poetry version --short)-nightly-$(date '+%s'), help with your use case?

I also have this problem but poetry version won't solve it because we primarily install Python packages from git. This saves us the overhead of needing a private PyPi.

In the software world I am engaged with, using git versioning as the source of truth is best practice. Deviating from the git revision is only advised if the git revision is somehow lacking (rare). Idea being git is the root-most version, and we don't want to complicate the system needlessly.

This is a perfectly rational approach to versioning python packages and it would be great if Poetry could support it. No breaking change required, version could just be made optional. Ideally, Poetry would default to the git revision if version not present without a plugin, but needing one would be fine too.

[pbarker]

also worth noting PDM has a nice way of doing this https://pdm.fming.dev/pyproject/pep621/#determine-the-package-version-dynamically

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

As another use-case, my company uses a monorepo where the server (written in Haskell) and client are versioned in lock-step, using the same source-of-truth VERSION file. Aside from whether this is a good idea in the first place, we need to set the version of the Python client library to the contents of the VERSION file. An alternative option is to hardcode the version in both VERSION and pyproject.toml and add a test ensuring they're the same, but our release process is complex already, and we'd like to avoid two manual file edits.

[Conchylicultor]

For this, (and many other reason), I switched to flit (https://github.com/pypa/flit).

Flit is only trying to one problem, so it end up working very nicely. I think poetry try to do to much, so end up having many bugs, edge cases.