-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove test command and tests_require #931
Comments
Perhaps its relevant here. After discussing with @RonnyPfannschmidt we added the following to the Tox docs.
As a downstream we prefer to have one command that tests the package for the interpreter that is available and not any other versions. |
I've discovered that |
Yes, I agree, as long as no other envs are defined, then there's no problem. |
|
@techtonik tox works totally fine for dev, you can use |
@graingert how much time does it take on you system to execute a single test with tox. On my it takes more than 5 seconds, which is not acceptable. Also, |
@jaraco instead of test_requires, i'd like to see something that allows |
I don't like the idea of replacing simple test calling with tox with my openSUSE Python maintenance hat on. We strive hard to achieve reproducible builds in openSUSE and thus we strongly prefer packages with small dependency trees. |
To be clear,
Eliminate where or how? For your own projects or for projects you are packaging? |
Well, the problem with whole virtual environments tox promotes is that it is completely useless in the Linux packaging world: we have all builds working in their own chrooted completely isolated environments so whatever issues virtenv is trying to solve are completely irrelevant for us. Also, we have different mechanisms (full blown isolated builds in separate virtual machines or something of that calibre) for testing with different versions of Python or something. So, we usually end up with digging into tox.ini and ripping out only |
@mcepl Well, in that case I think then One nice thing about the very declarative nature of In any case, fewer and fewer projects are actually using |
That's exactly what I’ve meant, so general feeling that more and more projects rely on |
So, they are doing massively different things. To get comparable experience, a user needs to somehow figure out which tox environment specified ini |
@native-api None of these things are true. While It certainly supports Yes there are differences between |
+1 for the explanations. It's not in the reference documentation so I had no way to know this. (The whole purpose of a reference is to be a complete list of the project's official guarantees.) |
In Nixpkgs the default Python builder (that expects I can understand the motivation for using Tox, but I suggest that at this point it makes more sense to standardize by going through a PEP. PEP 517/518 is a big improvement, and getting testing included there as well is where we should be going. |
Looks like Quick <1s test runs for TTD in IDEs that support test-on-edit and using |
its not .. |
You are welcome to write a draft PEP for this. I suspect that such a thing would be welcomed. I think the reason there has been no clear spec for this is that it's not very common for end users to run a project's tests - usually the people who run the tests are the developers of the package themselves and certain downstream redistributors. The package developers usually have to manually configure CI to run their tests and they don't have to do this at scale, so a standard is less necessary. The redistributors tend to be testing their whole package ecosystem, which means that something defined in terms of the PyPI package ecosystem will still need manual intervention, reducing the need for any sort of standard. Still, if you and other downstream redistributors of Python packages would like to propose such a standard, I recommend doing so. I think the packaging category or the users category on the Python discourse would be a good place to start. I think you will get a better response if you come in with a concrete proposal rather than creating a general "brainstorming" topic, but that is just my experience with posting such things. |
Discussion on Python Packaging discourse about a section in |
Python packaging is [moving away from embedding commands in setup.py][1]. It is difficult to maintain external commands in Python this way. Managing the dependencies needed to run commands from within setup.py is gnarly, because dependencies cannot be resolved so early. Furthermore, it's difficult to test code that happens at the point at which tests themselves are triggered, so that code needs to be very simple. A shell script is simple and more appropriate for this use case. At another time we should look to extract the lint command as well. [1][pypa/setuptools#931]
These are broken by the the new version of `setuptools`... https://setuptools.pypa.io/en/stable/history.html#v72-0-0 * pypa/setuptools#931 * pypa/setuptools#4458
For compatibility with setuptools>=72 (which removes the test command altogether) See pypa/setuptools#931
For compatibility with setuptools>=72 that remove the test command. See pypa/setuptools#931
Where to begin? It's just another Python screw-up. Developers of setuptools have announced, back in 2017 (!) their intention to remove the `test` command (found in setuptools.command.test) [1]. Today, they thought that 7 years of advance deprecation warnings were enough, and they published setuptools v72.0.0 which no longer provides that command. [2] But the Python ecosystem being what it is (a joke that has lasted long enough), it turns out that 7 years was not actually enough, and there are still many Python packages out there that didn't get the memo and that are still dependent on setuptools.command.test (including some packages that are used in the ODK). There are several ways of fixing this: a) Ditch Python altogether. This would be by far the best solution, but one that is, alas, unlikely to happen any time soon. b) Fix _all_ the packages used by the ODK (and their dependencies) so that they do what they should have done in the past 7 years. This is the correct solution, but one that will take time (7 years were already not enough, so who knows how long it is going to take?). c) Torture the setuptools developers until they agree to revert the removal of the `test` command. I believe this may be illegal in many jurisdictions. d) Force the ODK build process to use a version of `setuptools` older than 72.0.0. This is what we are doing here. Yes, this is a local workaround and we should favour "upstream fixing" instead (solution b), but if we want to be be able to build the ODK before 2031 (optimistic estimation of the time it will take to get all the Python packages fixed), we don't have much choice. [1] pypa/setuptools#931 [2] https://setuptools.pypa.io/en/stable/history.html
setuptools.command.test is removed in setuptools>=72.0.0. Discussion: pypa/setuptools#931 This breaks dateutil as it was using `setuptools.command.test.test` as `TestCommand`. Changed it to `setuptools.Command`. Signed-off-by: Md Safiyat Reza <[email protected]>
For compatibility with setuptools>=72 that remove the test command. See pypa/setuptools#931
Compatibility with setuptools 72.0 See pypa/setuptools#931
* scripts/run-tests: remove python2 relics * setup.py: drop test For compatibility with setuptools>=72 that remove the test command. See pypa/setuptools#931 * setup.py: remove unused imports
* scripts/run-tests: remove python2 relics * setup.py: drop test For compatibility with setuptools>=72 that remove the test command. See pypa/setuptools#931 * setup.py: remove unused imports
* scripts/run-tests: remove python2 relics * setup.py: drop test For compatibility with setuptools>=72 that remove the test command. See pypa/setuptools#931 * setup.py: remove unused imports
In this comment, @graingert proposes that we may be able to completely remove support for tests_require instead of transitioning that tooling from
easy_install
topip install
.While he didn't directly propose removal, the effort would only benefit that ticket if the install functionality were completely removed, so let's explore what that will entail.
While I agree that
tox
is an excellent, powerful, robust solution, it's more heavy thantests_require
, requiring that the user have tox installed in advance. As a more thorough solution, it also is subject to bugs and constraints that a simpler test runner is not. There are several advantages tosetup.py test
andtests_require
over tox:setup.py test
) instead of multiple steps (e.g.pip install --user tox; python -m tox
).setup.py test
allows for invocation under a number of different Python versions naturally (i.e.python3.3 setup.py test
orpy -3.3 setup.py test
) whereas tox offers "run under the python in which tox is installed" or "run for explicitly-declared python versions".setup.py test
doesn't use virtualenv, it's not subject to virtualenv bugs or other constraints imposed by virtualenv (such as version 14 dropping support for Python 3.2.).setup.py test
doesn't rely on pip, it's not subject to the bugs of pip (such as issues with--editable
installs or namespace packages) or other constraints imposed by pip (such as dropping support for Python 3.2).I consider these advantages small and easy enough to overcome, especially now that many of these issues have been resolved in setuptools, pip, and virtualenv. If we can get to a place that tox can broadly supplant the uses cases of setup.py test and pytest-runner (and thus tests_require) in practice, then yes, deprecating and removing it would be in order. Given the amount of activity and bugs I see around these tools, I'd asses they're still in active use.
Before flagging this functionality as deprecated, I'd like to survey the community about the possibility to see if there are use cases that would prove difficult to support with tox.
@graingert, would you be interested in being the champion for this effort (removing tests_require), starting with the outreach on distutils-sig and then implementing the deprecation/removal changes?
The text was updated successfully, but these errors were encountered: