nose compatibility problem with yield-base tests

[pchampin]

I wrote a test class for node, using yield-base tests, and it does not behave as expected with py-test 2.9.2.

Consider the following class:

class TestYield(object):

    SETUP = 0

    def setup(self):
        print "setup %r" % self
        self.SETUP += 1
        assert self.SETUP == 1

    def teardown(self):
        print "teardown %r" % self
        assert 0 <= self.SETUP <= 1 # setup has not been used more than once
        self.SETUP = 0

    def test_a(self):
        print "test_a %r" % self
        assert self.SETUP == 1 #check that class is setup
        self.SETUP -= 1 # "use" the setup

    def test_b(self):
        for i in range(3):
            print "test_b %r yields %r" % (self, i)
            yield self._do_test_b, i

    def _do_test_b(self, val):
        print "_do_test_b %r %r" % (self, val)
        assert self.SETUP == 1 # check that class is setup
        self.SETUP -= 1 # "use" the setup

When I run it with nosetests -qs, all four tests (1test_a and 3 test_b's) pass. For each of them, setup and teardown are called, as can be seen from the output:

setup <test_yield.TestYield object at 0x7f08de452110>
test_a <test_yield.TestYield object at 0x7f08de452110>
teardown <test_yield.TestYield object at 0x7f08de452110>
test_b <test_yield.TestYield object at 0x7f08de4520d0> yields 0
setup <test_yield.TestYield object at 0x7f08de452a50>
_do_test_b <test_yield.TestYield object at 0x7f08de452a50> 0
teardown <test_yield.TestYield object at 0x7f08de452a50>
test_b <test_yield.TestYield object at 0x7f08de4520d0> yields 1
setup <test_yield.TestYield object at 0x7f08de452bd0>
_do_test_b <test_yield.TestYield object at 0x7f08de452bd0> 1
teardown <test_yield.TestYield object at 0x7f08de452bd0>
test_b <test_yield.TestYield object at 0x7f08de4520d0> yields 2
setup <test_yield.TestYield object at 0x7f08de452d10>
_do_test_b <test_yield.TestYield object at 0x7f08de452d10> 2
teardown <test_yield.TestYield object at 0x7f08de452d10>
----------------------------------------------------------------------
Ran 4 tests in 0.001s

When I run the tests with py.test -qs, only test_a passes, all 3 test_b's fail on the assert in _do_test_b. The output is:

test_b <test_yield.TestYield object at 0x7f6df825b410> yields 0
test_b <test_yield.TestYield object at 0x7f6df825b410> yields 1
test_b <test_yield.TestYield object at 0x7f6df825b410> yields 2
setup <test_yield.TestYield object at 0x7f6df825b8d0>
test_a <test_yield.TestYield object at 0x7f6df825b8d0>
.teardown <test_yield.TestYield object at 0x7f6df825b8d0>
setup <test_yield.TestYield object at 0x7f6df825b8d0>
_do_test_b <test_yield.TestYield object at 0x7f6df825b410> 0
F_do_test_b <test_yield.TestYield object at 0x7f6df825b410> 1
F_do_test_b <test_yield.TestYield object at 0x7f6df825b410> 2
F
(...)

We can see that the three test_b's are run on the same instance, which is not consistent with Nose. Worse, setup is not even called on that instance, instead it is called on the instance that was previously used to run test_a, which does not make much sense... Finally, teardown is never called for test_b.

[pchampin]

I know that I can rewrite my class in order to make it work with pytest (using mark.parametrize), but this is not my point here: my concern is that the documentation claims that pytest is compatible with Nose yield-based tests, which is not entirely true.

The quickest way to fix it would be to amend the documentation, to make it clear that compatibility is restricted to generator functions, but that generator methods are not supported.

[nicoddemus]

Hey @pchampin,

Unfortunately support for yield-tests is being deprecated in pytest 3.0 (#1714), so unless someone which is willing to work on this I think we should go with your suggestion and change the documentation.

[iamveritas]

is there anywhere documented how to migrate "nose2 type of tests that yield self.assert* and each yield is collected as a separate test" into pytest?
I;ve been searching with no good result an easy way to migrate a quite large of nose2 tests into py.test, so any help would be appreciated.
thank you