-
-
Notifications
You must be signed in to change notification settings - Fork 511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
native pyproject.toml config #999
Comments
Learning from the old format, I would add:
|
... and most importantly: a clear, versioned specification that can then evolve as we for sure won't get it right at first try :) |
I think, It would be great if we could organize a sprint around this whole topic at some point this year. |
Well, most importantly we need someone try to tackle this and implement it. Personally, in my priority queue, this is quite low. I've tackled it inside https://github.com/gaborbernat/toxn/blob/master/pyproject.toml#L5 only to prove it's quite hard to get this right. I'll only have some bandwidth for this after the #1102, #998 and #338. I also want to first split up configuration module from a mega file to sub-modules.
|
That's exactly why I think a sprint with a few more people might be a good way to get this off the ground. Maybe around a conference where a few of as will go anyway - e.g. EuroPython. |
I don't consider this a good candidate for a sprint. That creates an artificial deadline (e.g. end of sprint). Instead, we should probably propose each and every one of us our own proposal and iterate on that until we find something that can do at least what the |
Why not a |
Only in your head :) A sprint would in my eyes be a good way to make progress and share state, but in no way a deadline. Especially if you explored that area already with your
That would be simply the
Why do you think that? I might be a bit naive because I am really not that deep in the whole packaging topic, but I am pretty hopeful that this time around we'll do things better. |
I don't think it's going to be a mess. But it definitely can be hard to maintain once many things will try to use it. E.g. a complex tox + towncrier + black config can easily make |
Well ... how to put it ... I've looked at https://github.com/gaborbernat/toxn/blob/master/pyproject.toml#L5 and I thought "well I don't want to write all those ridiculously long section names, I'd already be at my wits' end after all those quotes and brackets". The second thing is that there will inevitably be broken tools that try to modify pyproject.toml - eg: bumpversion rewrites setup.cfg (if you're naive enough to use that - I was) but with slightly different values for the other sections. From my perspective the only benefit of toml is that tox will get more features. To give a bit more perspective, pyproject.toml is the new hotness but also these were too the new hotness some years ago:
What I'm getting at here: ignore the PEP518/pyproject.toml bandwagon - just think about a new configuration system for tox.
|
I'm a bit conflicted here now though, I don't think we can come up with anything as succinct as our current ini format. So if writing brackets and quotes is going to become a problem (which very easily could be, I found myself to be a bit awkward when I've created toxn), I don't think there's much benefit from moving off the ini format. |
Hmmm, can you elaborate a bit more on this goal:
|
so... I'm not sure about the original intention behind this ticket, but to me the most obvious reason to allow pyproject.toml has nothing to do with the superiority of toml over ini (debatable, but not the point), but the centralization of project configuration in one file. As a user, I would be very happy to avoid another configuration file if it meant I could fully specify build and test environments in one place. I believe this is the spirit of allowing extra sections in pyproject.toml in the PEP Is there a reason in particular we can't just transliterate the current ini features into toml? |
Because the ini offers a lot of flexibility/functionality that's not easy to express within TOML, e.g. conditional factors and value substituion. |
tox usage of ini is the most complex use of the ini "format" I ever have seen. It is quite a complex DSL, even so for basic usage you do not need to read the docs. I am not saying that I love or hate it but porting it to TOML would likely end up as a disaster due to incompatible/conflicting syntax. My impression is that tox should fix other areas before changing the configuration file format. IMHO, if the tox config should have to be rewritten, it would as well be an opportunity for the user to switch to another tool. It may worth noting that the all tool configuration in a single file may be the wrong solution for the issue of cluttering the project root folder, especially if the tools involved are likely to have bigger configuration files with more changes to them like tox. For linters I do not see a real problem to stick them to pyproject but if you try to stick everything in, you end up with a maintenance issue, problematic backporting to stable branches and merge conflicts on bigger projects. As I was also annoyed by the cluttering of the project root folder I started to allow configuration loading from |
I can see the appeal in having a TOML config. And we'll likely have one in tox 4. tox 4 (see rewrite branch) does a clean separation via abstraction on where the source comes from and how it is translated to python configuration objects - see https://github.com/tox-dev/tox/blob/rewrite/src/tox/config/source/__init__.py#L10. With that, we can implement a toml configuration format for tox that maybe is less succinct than the With that being said tox 4 is likely not to happen before the end of the year but hopeful for an RC in January. So in the meantime, this is blocked. |
Is the work related to this issue still in schedule? |
The current focus is on getting tox 4 ready for a final release. Just last week the final alpha was released. This feature was scheduled for 4.1. Once we get tox 4 out, there will be quite a transition period and possibly quite some edge cases coming up which need attention. That said - let's see what @gaborbernat has to add. |
Mostly what @jugmac00 said. The design of the tox 4 is made to allow this without too much of heavy-lift but I'd not pull myself into this when trying to make tox 4 actually come out the door. But would like to follow up on this quickly after that happens. |
Thank you for the information! @jugmac00 @gaborbernat |
|
Thanks, but that will not help. tox configuration is complex and it's not possible to have the same config in ini and toml. E.g. substitutions and references don't work well in toml, so we need to design a config file/logic that's better suited for TOML. |
We would prefer to configure tox using pyproject.toml, but that's unfortunately not properly supported yet (there's only a hacky way with a giant string that we were using before). The problem this causes for us is that other tools (like Renovate bot) don't support the hacky method, so we're losing out on updates to our tox config. According to the issue below (and the tox docs), pyproject will be properly supported by tox v4, so we should check back then to see if we can get rid of tox.ini. see: tox-dev/tox#999 see: tox-dev/tox#814
* Add tox >= 4.0 as a dev dependency * Add tox configuration to setup.cfg - I chose setup.cfg because pyproject.toml support in tox is awkward (tox-dev/tox#999) * Modify Makefile to use tox instead of running the commands directly * Update .gitignore for relevant new files and directories * Update requirements files - Newer tarball in freeze-requirements.sh because the older one was sometimes breaking in tox. A future commit will configure tox in Github actions.
* Add tox >= 4.0 as a dev dependency * Add tox configuration to setup.cfg - I chose setup.cfg because pyproject.toml support in tox is awkward (tox-dev/tox#999) * Modify Makefile to use tox instead of running the commands directly * Update .gitignore for relevant new files and directories * Update requirements files - Newer tarball in freeze-requirements.sh because the older one was sometimes breaking in tox. A future commit will configure tox in Github actions.
* Add tox >= 4.0 as a dev dependency * Add tox configuration to setup.cfg - I chose setup.cfg because pyproject.toml support in tox is awkward (tox-dev/tox#999) - Specified flake8 < 6.0.0 until we can get around to replacing type comments with type annotations. PyCQA/pyflakes#747 * Modify Makefile to use tox instead of running the commands directly * Update .gitignore for relevant new files and directories * Update requirements files - Newer tarball in freeze-requirements.sh because the older one was sometimes breaking in tox. * Set pylint configuration to ignore tests/legacy rather than manually excluding it A future commit will configure tox in Github actions.
* Add tox >= 4.0 as a dev dependency * Add tox configuration to setup.cfg - I chose setup.cfg because pyproject.toml support in tox is awkward (tox-dev/tox#999) - Specified flake8 < 6.0.0 until we can get around to replacing type comments with type annotations. PyCQA/pyflakes#747 * Modify Makefile to use tox instead of running the commands directly * Update .gitignore for relevant new files and directories * Update requirements files - Newer tarball in freeze-requirements.sh because the older one was sometimes breaking in tox. * Set pylint configuration to ignore tests/legacy rather than manually excluding it A future commit will configure tox in Github actions.
* Add tox >= 4.0 as a dev dependency * Add tox configuration to setup.cfg - I chose setup.cfg because pyproject.toml support in tox is awkward (tox-dev/tox#999) - Specified flake8 < 6.0.0 until we can get around to replacing type comments with type annotations. PyCQA/pyflakes#747 * Modify Makefile to use tox instead of running the commands directly * Update .gitignore for relevant new files and directories * Update requirements files - Newer tarball in freeze-requirements.sh because the older one was sometimes breaking in tox. * Set pylint configuration to ignore tests/legacy rather than manually excluding it A future commit will configure tox in Github actions.
* Move tox configurion to tox.ini until tox-dev/tox#999 is addressed. * Run tox on 3.9 - 3.11 * Improve poetry compatibility * Add isort
@gaborbernat Since this issue has gotten as complicated as it is, it would be useful to have a writeup of the complexities you've mentioned. |
Mostly boils down to the ini logic cannot be used by the pyproject.toml file. That's because a TOML file is a type bound structure, while ini is just raw strings. |
I'd like this to be implemented and I should have some time this summer to work on this. So I patched a prototype see #3309 to start with something. It has quite a number of limitations, see below, I'd like to confirm that I'm in the right direction first. I was hoping that in tox 4 this will be relatively easy, but this will be a long run. There is a bunch of things that hindered my progress and number of decisions to be made. My initial idea: add Status of the prototype: [tox]
min_version = '4.0'
env_list = [
'py310',
'py39',
'type'
]
[testenv]
deps = 'pytest'
commands = ['pytest tests']
['testenv:type']
deps = 'mypy'
commands = ['mypy src'] I completely ditched the idea of Major limitation: Tox seems still to be heavily designed around INI structure. INI files are easy, since everything in there is just a text. Despite the fact that basic APIs in TOX 4 are designed INI-neutral (or at least at first glance), I found following problems:
Known bugs and limitations:
I plan to start with some small PRs first and move gradually towards the TOML config. A feedback would be very appreciated. |
Ad env definitions: I don't think, it's possible to use |
It really should be tool.tox.env instead. |
This is a ticket to track and implement a native
pyproject.toml
configuration. Will update with the proposed plan and format.Core goals:
We'll use this opportunity to rename some keys (e.g.
skipsdist
should beskip_packaging
, etc).The text was updated successfully, but these errors were encountered: