OutcomeException should not be a subclass of Exception

[pytestbot]

Originally reported by: BitBucket: thiefmaster, GitHub: thiefmaster

---

Imagine this code:

def blackbox(callback, arg):
    # this is assumed to be application code
    try:
        callback(arg)
    except Exception:
        pass


def test_callback():
    def cb(x):
        if not x:
            import pytest
            pytest.fail('foo')

    blackbox(cb, True)
    blackbox(cb, False)

The testcase will pass since except Exception will catch the Failed raised by pytest.fail. However, I don't think there is any valid use-case for application code to prevent a testcase failure from propagating to the test runner.

A more practical example where this is a major issue is monkeypatching a library function to fail when called. For example, one might want to monkeypatch smtplib.SMTP.connect to fail when code tries to send an email unexpectedly - but chances are good that the application is wrapping that code in a try...except Exception block to prevent an email sending failure from breaking things (e.g. because the email is not very important). So now the pytest.fail called in the monkeypatched method will never propagate since it's caught by application code.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/580

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

When following this route, how can one also protect agains try: callback(); except: pass, that is a bare exception in user code?

(I must admit I've never tried to call pytest.fail() in callbacks before)

[pytestbot]

Original comment by BitBucket: thiefmaster, GitHub: thiefmaster:

---

That's such an unlikely case though...

[RonnyPfannschmidt]

@flub
its sensible to use BaseException to help people

we don't need to protect about the diaper antipattern

[nicoddemus]

Shouldn't this be in 2.9.0? It's not a bug per-se, but an improvement that might even break current test suites, IMO.

[The-Compiler]

Changing the milestone for this to 4.0 for now.

[blueyed]

Just for reference/info: using BaseException is problematic, e.g. when people are trying to have a safe repr (and only handling Exception, and where pytest-django might throw a failure for when DB access is not allowed), or with asyncio which only handles Exception (pytest-dev/pytest-asyncio#123).