Run test suite with pytest

[jdufresne]

pytest has better support for parameterized tests. Each test iteration
appears as a separate dot by the test runner and can run in isolation.

[francoisfreitag]

Maybe pytest should be listed as a tests_require dependency in setup.py.

[jdufresne]

Maybe pytest should be listed as a tests_require dependency in setup.py.

We don't use python3 setup.py test to run the test suite, so I'm not sure it will be of any practical use. If you think we should be doing this, here are the docs to use run pytest in this way:

https://docs.pytest.org/en/latest/goodpractices.html#integrating-with-setuptools-python-setup-py-test-pytest-runner

But I find the use of setup_requires=["pytest-runner", ... adds unnecessary overhead to every setup.py call as setuptools will need to install pytest-runner before doing anything else. Even for non-test commands. It also seems like unnecessary configuration for a feature we aren't using.

So, I'd prefer to leave this out for now, if it is okay. WDYT?

[francoisfreitag]

The idea of listing pytest as a dependency was mostly for documentation purposes, so that contributors know pytest is required to run the tests. Since the documented command to run tests is tox, I suppose contributors can look at tox.ini and get the dependencies there.

[jdufresne]

Just a bit more information, the setuptools developers have discussed potentially deprecating and removing the test integration features in setup.py:

pypa/setuptools#1684
pypa/setuptools#931