subprocess tests hang

[jaraco]

While working out a solution for #2022, I discovered a different issue with a subprocess test hanging when -s is passed to pytest.

$ python -m tox -e py35 -- -v -s -k test_pdb_collection_failure_is_shown
GLOB sdist-make: /Users/jaraco/Dropbox/code/public/pytest/setup.py
py35 inst-nodeps: /Users/jaraco/Dropbox/code/public/pytest/.tox/dist/pytest-3.0.4.dev0.zip
py35 installed: hypothesis==3.5.3,mock==2.0.0,nose==1.3.7,pbr==1.10.0,pexpect==4.2.1,ptyprocess==0.5.1,py==1.4.31,pytest==3.0.4.dev0,requests==2.11.1,six==1.10.0,spark-parser==1.4.0,uncompyle6==2.9.2,xdis==3.1.0
py35 runtests: PYTHONHASHSEED='318108626'
py35 runtests: commands[0] | pytest --lsof -rfsxX -v -s -k test_pdb_collection_failure_is_shown
=========================================== test session starts ===========================================
platform darwin -- Python 3.5.2, pytest-3.0.4.dev, py-1.4.31, pluggy-0.4.0 -- /Users/jaraco/Dropbox/code/public/pytest/.tox/py35/bin/python3.5
cachedir: .cache
rootdir: /Users/jaraco/Dropbox/code/public/pytest, inifile: tox.ini
plugins: hypothesis-3.5.3
collected 1724 items 

testing/test_pdb.py::TestPDB::test_pdb_collection_failure_is_shown running: /Users/jaraco/Dropbox/code/public/pytest/.tox/py35/bin/python3.5 /Users/jaraco/Dropbox/code/public/pytest/.tox/py35/lib/python3.5/site-packages/pytest.py --basetemp=/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-52/testdir/test_pdb_collection_failure_is_shown0/runpytest-0 --pdb /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-52/testdir/test_pdb_collection_failure_is_shown0/test_pdb_collection_failure_is_shown.py
     in: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-52/testdir/test_pdb_collection_failure_is_shown0
^CERROR: KEYBOARDINTERRUPT


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
to show a full traceback on KeyboardInterrupt use --fulltrace
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py:1608: KeyboardInterrupt
========================================== 1723 tests deselected ==========================================
==================================== 1723 deselected in 33.22 seconds =====================================
ERROR: keyboardinterrupt

Omit the -s from the test run, and the test passes just fine.

[andras-tim]

I think this bug is not critical because this bug based on pytest tests itself. The problem is based on insufficient encapsulation for running subprocess. So I think this isn't a critical bug.

The problem based on

• The pytester.py.Testdir.popen() didn't say anything about stdin, so
  • if you don't use -s, the stdin will be mocked by FDCapture
  • however with -s the stdin will remain the pytest's stdin, so in this case the console
• This the test_pdb_collection_failure_is_shown tests the behavior for a failure, therefore the pdb wants to show the problem.

And we are there! The pdb presents an interactive debug environment because it got a standard terminal as stdin.

Demo time

1. Let's run the problematic test with -s like this:

   python pytest.py -s testing/test_pdb.py::TestPDB::test_pdb_collection_failure_is_shown

2. And when the process is frozen, let's exit from pdb with q and an enter

3. 😉

Fix

Should set the stdin in pytester.py.Testdir.popen() to devnull for avoid this bad cases.