Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pytest to run doctests #6110

Open
trexfeathers opened this issue Aug 9, 2024 · 0 comments
Open

Use pytest to run doctests #6110

trexfeathers opened this issue Aug 9, 2024 · 0 comments
Assignees

Comments

@trexfeathers
Copy link
Contributor

trexfeathers commented Aug 9, 2024

📰 Custom Issue

How to run doctests - pytest documentation

We currently rely on our docs rendering to run our doctests. This introduces many problems:

  1. Unnecessarily long run time - must first build the docs before running the doctests.
  2. Confusingly wide CI scope - the CI doctest session must first build the docs. We regularly get doctest 'failures' that are actually docs build failures, and this has been known to confuse developers.
  3. Difficult to run individual doctests - must use a specific series of arguments with a docs build command. This took me 4 years to figure out! sphinx-build -b doctest -d _build/doctrees . _build/doctest generated/api/iris.util.rst.
  4. @tkknight and @bjlittle are discovering that the number of doctests that actually get run is had to understand. It seems to depend on how the docs are built, and is difficult to debug specifically because it is hidden behind a docs builder. DOCS: Adopt sphinx-autoapi #6102

pytest is a mature, purpose-built tool. It can directly run doctests from .rst files (points 1 and 2), its syntax is familiar and aimed at being easy (point 3), and its terminal output is also the subject of much refinement (point 4). I believe the only reason we're not already using it for doctests is that it didn't occur to us when we adopted pytest as the runner for our 'normal' tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants