Fix error that occurs when using pytest=3.7

[drdavella]

../../miniconda3/envs/env/lib/python3.6/site-packages/pluggy/hooks.py:258: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
../../miniconda3/envs/env/lib/python3.6/site-packages/pluggy/manager.py:67: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../miniconda3/envs/env/lib/python3.6/site-packages/pluggy/manager.py:61: in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
../../miniconda3/envs/env/lib/python3.6/site-packages/_pytest/python.py:210: in pytest_collect_file
    return ihook.pytest_pycollect_makemodule(path=path, parent=parent)
../../miniconda3/envs/env/lib/python3.6/site-packages/pluggy/hooks.py:258: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
../../miniconda3/envs/env/lib/python3.6/site-packages/pluggy/manager.py:67: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../miniconda3/envs/env/lib/python3.6/site-packages/pluggy/manager.py:61: in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
../../miniconda3/envs/env/lib/python3.6/site-packages/_pytest/python.py:215: in pytest_pycollect_makemodule
    return Package(path, parent)
../../miniconda3/envs/env/lib/python3.6/site-packages/_pytest/python.py:552: in __init__
    self, fspath, parent=parent, config=config, session=session, nodeid=nodeid
../../miniconda3/envs/env/lib/python3.6/site-packages/_pytest/nodes.py:371: in __init__
    name, parent, config, session, nodeid=nodeid, fspath=fspath
../../miniconda3/envs/env/lib/python3.6/site-packages/_pytest/python.py:275: in __init__
    super(PyobjMixin, self).__init__(*k, **kw)
../../miniconda3/envs/env/lib/python3.6/site-packages/_pytest/nodes.py:93: in __init__
    self.keywords = NodeKeywords(self)
E   RecursionError: maximum recursion depth exceeded

We first encountered a different problem when pytest-3.7.0 was released, and that was reported upstream (see pytest-dev/pytest#3742). They released a patch in 3.7.1, but that is what is causing this issue (see pytest-dev/pytest#3751).

The recommended solution for the time being is to use pytest==3.6.

[drdavella]

A bit of investigation has led me to believe that this issue has been fixed in the development version of pytest but the fix is not yet available in a released version. So we should hold off on using the latest pytest until 3.7.2 is released.

[bsipocz]

pytest 3.7.2 is out, so now at least the tests are run, but we have tons of failures.

[pllim]

@bsipocz , the failures are due to doctests got outdated, right? OK nvm, I was very confused. Astropy doctests seem fine again with pytest 3.6 https://travis-ci.org/astropy/astropy/jobs/419745266

[cdeil]

With Python 3.7 and pytest 3.7.3 and astropy/astropy#7779 applied the Astropy tests pass for me if I don't have pytest-astropy installed. After installing it, there's a ton of errors.

These:

________________________________________________________________________ ERROR at setup of test_warnings_logging_disable_no_enable ________________________________________________________________________

self = <CallInfo when='setup' exception: Plugin specs must be a ','-separated string or a list/tuple of strings for plugin names. Given: <module 'astropy.tests.pytest_plugins' from '/private/tmp/astropy/astropy/tests/pytest_plugins.py'>>
func = <function call_runtest_hook.<locals>.<lambda> at 0x11b3539d8>, when = 'setup', treat_keyboard_interrupt_as_exception = False

    def __init__(self, func, when, treat_keyboard_interrupt_as_exception=False):
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        self.when = when
        self.start = time()
        try:
>           self.result = func()

/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/runner.py:201: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/runner.py:183: in <lambda>
    lambda: ihook(item=item, **kwds),
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/pluggy/hooks.py:258: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/pluggy/manager.py:67: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/pluggy/manager.py:61: in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/runner.py:104: in pytest_runtest_setup
    item.session._setupstate.prepare(item)
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/runner.py:370: in prepare
    col.setup()
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/python.py:527: in setup
    setup_module = _get_xunit_setup_teardown(self.obj, "setUpModule")
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/python.py:273: in fget
    self._obj = obj = self._getobj()
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/python.py:470: in _getobj
    return self._importtestmodule()
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/python.py:523: in _importtestmodule
    self.config.pluginmanager.consider_module(mod)
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/config/__init__.py:462: in consider_module
    self._import_plugin_specs(getattr(mod, "pytest_plugins", []))
/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/config/__init__.py:465: in _import_plugin_specs
    plugins = _get_plugin_specs_as_list(spec)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

specs = <module 'astropy.tests.pytest_plugins' from '/private/tmp/astropy/astropy/tests/pytest_plugins.py'>

    def _get_plugin_specs_as_list(specs):
        """
        Parses a list of "plugin specs" and returns a list of plugin names.
    
        Plugin specs can be given as a list of strings separated by "," or already as a list/tuple in
        which case it is returned as a list. Specs can also be `None` in which case an
        empty list is returned.
        """
        if specs is not None:
            if isinstance(specs, str):
                specs = specs.split(",") if specs else []
            if not isinstance(specs, (list, tuple)):
                raise UsageError(
                    "Plugin specs must be a ','-separated string or a "
>                   "list/tuple of strings for plugin names. Given: %r" % specs
                )
E               _pytest.config.exceptions.UsageError: Plugin specs must be a ','-separated string or a list/tuple of strings for plugin names. Given: <module 'astropy.tests.pytest_plugins' from '/private/tmp/astropy/astropy/tests/pytest_plugins.py'>

/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/config/__init__.py:518: UsageError
__________________________________________________________________________ ERROR at setup of test_warnings_logging_enable_twice ___________________________________________________________________________

tp = <class '_pytest.config.exceptions.UsageError'>, value = None, tb = None

    def reraise(tp, value, tb=None):
        try:
            if value is None:
                value = tp()
            if value.__traceback__ is not tb:
>               raise value.with_traceback(tb)
E               _pytest.config.exceptions.UsageError: Plugin specs must be a ','-separated string or a list/tuple of strings for plugin names. Given: <module 'astropy.tests.pytest_plugins' from '/private/tmp/astropy/astropy/tests/pytest_plugins.py'>

/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/six.py:692: UsageError
___________________________________________________________________________ ERROR at setup of test_warnings_logging_overridden ____________________________________________________________________________

tp = <class '_pytest.config.exceptions.UsageError'>, value = None, tb = None

    def reraise(tp, value, tb=None):
        try:
            if value is None:
                value = tp()
            if value.__traceback__ is not tb:
>               raise value.with_traceback(tb)
E               _pytest.config.exceptions.UsageError: Plugin specs must be a ','-separated string or a list/tuple of strings for plugin names. Given: <module 'astropy.tests.pytest_plugins' from '/private/tmp/astropy/astropy/tests/pytest_plugins.py'>

/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/six.py:692: UsageError

and then a gadzillion of those:

>           out, err = capman.suspend_global_capture(in_=True)
E           TypeError: cannot unpack non-iterable NoneType object

/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/doctest.py:206: TypeError

https://gist.github.com/cdeil/36ebd9882e73a3b3dd1b8d94f1798f5b

[bsipocz]

cc @drdavella

[nicoddemus]

This error in particular:

>           out, err = capman.suspend_global_capture(in_=True)
E           TypeError: cannot unpack non-iterable NoneType object

/Users/deil/software/anaconda3/envs/astropy37/lib/python3.7/site-packages/_pytest/doctest.py:206: TypeError

Is a regression on OS-X which will be fixed in 3.7.4 (pytest-dev/pytest#3893).

The other seems to be that some file is importing astropy.tests.pytest_plugins like this:

from astropy.tests import pytest_plugins

Problem is that pytest_plugins is a special name used to load plugins dynamically, hence the error.

Unfortunately at the moment it raises that quite cryptic error. A quick fix is to rename the import:

from astropy.tests import pytest_plugins as plugins

[cdeil]

There's one more error that looks like an issue that should be fixed in Astropy I think; for that I've opened astropy/astropy#7780 .

[drdavella]

Thanks @nicoddemus! So it seems like no action is required as far as pytest-doctestplus is required, correct?

I guess the choice of name for astropy.tests.pytest_plugins is a bit unfortunate in retrospect, but I'm not sure how much we can do at this point. It should only matter if someone is importing that into a conftest.py, though, right?

[nicoddemus]

It should only matter if someone is importing that into a conftest.py, though, right?

Actually pytest will look for pytest_plugins in test modules as well, so this is quite error prone.

I see that the astropy.tests.pytest_plugins has been deprecated for 10 months now in favor of astropy.tests.plugins.display. One option would be to end the deprecation and update your warnings filter to transform that warning into an error (that's how we plan to end deprecations in pytest itself).

[drdavella]

Ah yes... I believe I actually was the one responsible for that deprecation for just those reasons (how soon we forget!). In fact, I opened an issue in pytest (here: pytest-dev/pytest#2891) because this warning does not show up for users who import through conftest.py.

@bsipocz, @astrofrog maybe we should discuss on Astropy proper, but do you have any thoughts about accelerating the deprecation of this?

[cdeil]

I see these warnings from Gammapy. Is there an explanation or astropy-dev or astropy-affiliated-maintainers mailing list post that explains what this is and what I should do to update in my Astropy affiliated packages? After this change will it still work with oder Astropy and pytest versions, i.e. is it possible to do this upgrade in a sane way?

[drdavella]

@cdeil there is a bit of documentation here, but if it's not actually helpful then I can make some updates. I can also look at gammapy and review changes there if you want.

The required changes should be compatible with any version of pytest, but will not be compatible with the 2.0.x series of Astropy. If you need to maintain compatibility, you'll need to do a version check.

[bsipocz]

@drdavella - I'm usually one of the most liberal maintainers in terms of deprecations/removal times, so you better open an issue in astropy proper and ask there. One project idea for dotastro I had was related to digging up usage info for deprecated functionality and set up a notification system for package maintainers. But now I don't think this sprint will happen before the coordination meeting.

[drdavella]

Closing since this resolved itself on the pytest side.