Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show "short test summary info" after tracebacks and warnings #3255

Merged
merged 2 commits into from
Feb 27, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Show "short test summary info" after tracebacks and warnings
  • Loading branch information
nicoddemus committed Feb 27, 2018
commit 4e405dd9f96e69cdd53901db930a2d79bc4f7e58
12 changes: 10 additions & 2 deletions _pytest/hookspec.py
Original file line number Diff line number Diff line change
Expand Up @@ -489,8 +489,16 @@ def pytest_report_teststatus(report):
Stops at first non-None result, see :ref:`firstresult` """


def pytest_terminal_summary(terminalreporter, exitstatus):
""" add additional section in terminal summary reporting. """
def pytest_terminal_summary(config, terminalreporter, exitstatus):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a breaking change, any caller of the hook would need to add the extra parameter

this needs to be addressed in pluggy first

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hook is called only by terminal.py, and we've added parameters to hooks in the past, it's not clear to me why this hook in particular cannot receive an extra argument at this point.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this particular hooks its probably less of an issue, however all hooks and the required arguments they take are currently part of the public api

for example pytest_deselected is in dire need of a added reason argument however its one of the hooks that are supposed to be called by 3rd parties, and they would simply break

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean regarding pytest_deselected, but AFAIK this is the only hook where we expect to be called by 3rd parties, all other hooks are called by pytest itself.

So IMHO your concern is valid but not for most hooks.

Having said all that, if you really want I can remove the parameter, it was not really required by the hook implementations I touched.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im leaning torwards removing it, but as you said, its unlikely to be invoked by 3rd parties as of now
we ought to create a mechanism trough which 3rd parties defer to hooks later on

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries, removed it. 👍

"""Add a section to terminal summary reporting.

:param _pytest.config.Config config: pytest config object
:param _pytest.terminal.TerminalReporter terminalreporter: the internal terminal reporter object
:param int exitstatus: the exit status that will be reported back to the OS

.. versionadded:: 3.5
The ``config`` parameter.
"""


@hookspec(historic=True)
Expand Down
18 changes: 10 additions & 8 deletions _pytest/skipping.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
""" support for skip/xfail functions and markers. """
from __future__ import absolute_import, division, print_function


from _pytest.config import hookimpl
from _pytest.mark import MarkInfo, MarkDecorator
from _pytest.mark.evaluate import MarkEvaluator
Expand All @@ -14,11 +13,11 @@ def pytest_addoption(parser):
action="store_true", dest="runxfail", default=False,
help="run tests even if they are marked xfail")

parser.addini("xfail_strict", "default for the strict parameter of xfail "
"markers when not given explicitly (default: "
"False)",
default=False,
type="bool")
parser.addini("xfail_strict",
"default for the strict parameter of xfail "
"markers when not given explicitly (default: False)",
default=False,
type="bool")


def pytest_configure(config):
Expand Down Expand Up @@ -130,7 +129,7 @@ def pytest_runtest_makereport(item, call):
rep.outcome = "passed"
rep.wasxfail = rep.longrepr
elif item.config.option.runxfail:
pass # don't interefere
pass # don't interefere
elif call.excinfo and call.excinfo.errisinstance(xfail.Exception):
rep.wasxfail = "reason: " + call.excinfo.value.msg
rep.outcome = "skipped"
Expand Down Expand Up @@ -160,6 +159,7 @@ def pytest_runtest_makereport(item, call):
filename, line = item.location[:2]
rep.longrepr = filename, line, reason


# called by terminalreporter progress reporting


Expand All @@ -170,6 +170,7 @@ def pytest_report_teststatus(report):
elif report.passed:
return "xpassed", "X", ("XPASS", {'yellow': True})


# called by the terminalreporter instance/plugin


Expand Down Expand Up @@ -233,7 +234,7 @@ def folded_skips(skipped):
# TODO: revisit after marks scope would be fixed
when = getattr(event, 'when', None)
if when == 'setup' and 'skip' in keywords and 'pytestmark' not in keywords:
key = (key[0], None, key[2], )
key = (key[0], None, key[2])
d.setdefault(key, []).append(event)
values = []
for key, events in d.items():
Expand Down Expand Up @@ -269,6 +270,7 @@ def show_skipped(terminalreporter, lines):
def shower(stat, format):
def show_(terminalreporter, lines):
return show_simple(terminalreporter, lines, stat, format)

return show_


Expand Down
13 changes: 9 additions & 4 deletions _pytest/terminal.py
Original file line number Diff line number Diff line change
Expand Up @@ -480,16 +480,21 @@ def pytest_sessionfinish(self, exitstatus):
EXIT_NOTESTSCOLLECTED)
if exitstatus in summary_exit_codes:
self.config.hook.pytest_terminal_summary(terminalreporter=self,
config=self.config,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dito

exitstatus=exitstatus)
self.summary_errors()
self.summary_failures()
self.summary_warnings()
self.summary_passes()
if exitstatus == EXIT_INTERRUPTED:
self._report_keyboardinterrupt()
del self._keyboardinterrupt_memo
self.summary_stats()

@pytest.hookimpl(hookwrapper=True)
def pytest_terminal_summary(self):
self.summary_errors()
self.summary_failures()
yield
self.summary_warnings()
self.summary_passes()

def pytest_keyboard_interrupt(self, excinfo):
self._keyboardinterrupt_memo = excinfo.getrepr(funcargs=True)

Expand Down
1 change: 1 addition & 0 deletions changelog/3255.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The *short test summary info* section now is displayed after tracebacks and warnings in the terminal.
15 changes: 15 additions & 0 deletions testing/test_skipping.py
Original file line number Diff line number Diff line change
Expand Up @@ -1065,3 +1065,18 @@ def pytest_collect_file(path, parent):
assert not failed
xfailed = [r for r in skipped if hasattr(r, 'wasxfail')]
assert xfailed


def test_summary_list_after_errors(testdir):
"""Ensure the list of errors/fails/xfails/skips appear after tracebacks in terminal reporting."""
testdir.makepyfile("""
import pytest
def test_fail():
assert 0
""")
result = testdir.runpytest('-ra')
result.stdout.fnmatch_lines([
'=* FAILURES *=',
'*= short test summary info =*',
'FAIL test_summary_list_after_errors.py::test_fail',
])