pytest consistently raises ImportError

[rickmcgeer]

I'm running pytest with the command

$ pytest -vvv --rootdir . tests/test_sdtp_utils.py

$ tree gives the following output: (irrelevant files and directories omitted for clarity)

.
├── __init__.py
├── conftest.py
├── sdtp
│   ├── README.md
│   ├── __init__.py
│   ├── sdtp_filter.py
│   ├── sdtp_table.py
│   └── sdtp_utils.py
├── sdtp_server
│   ├── README.md
│   ├── __init__.py
│   ├── sdtp_server.py
│   └── table_server.py
├── setup.cfg
├── setup.py
└── tests
    ├── __init__.py
    ├── pytest.ini
    ├── table_data_good.py
    ├── test_data_plane.py
    ├── test_sdtp_filter.py
    ├── test_sdtp_server.py
    ├── test_sdtp_table.py
    ├── test_sdtp_table_and_filter.py
    ├── test_sdtp_utils.py
    └── test_table_server.py

Pytest (run from the root directory) gives the following error:

pytest -vvv --rootdir . tests/test_sdtp_utils.py
============================================================ test session starts =============================================================
platform linux -- Python 3.11.4, pytest-8.2.2, pluggy-1.5.0 -- /opt/conda/bin/python
cachedir: .pytest_cache
rootdir: /workspaces/sdtp
configfile: tests/pytest.ini
plugins: cov-5.0.0, pytest_httpserver-1.0.10
collected 0 items / 1 error                                                                                                                  

=================================================================== ERRORS ===================================================================
_________________________________________________ ERROR collecting tests/test_sdtp_utils.py __________________________________________________
ImportError while importing test module '/workspaces/sdtp/tests/test_sdtp_utils.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/conda/lib/python3.11/site-packages/_pytest/python.py:492: in importtestmodule
    mod = import_path(
/opt/conda/lib/python3.11/site-packages/_pytest/pathlib.py:591: in import_path
    importlib.import_module(module_name)
/opt/conda/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1126: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1126: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:690: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:940: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
__init__.py:9: in <module>
    from sdtp.sdtp_utils import InvalidDataException
E   ModuleNotFoundError: No module named 'sdtp.sdtp_utils'

However, from the same directory

$ python
Python 3.11.4 (main, Jul  5 2023, 13:45:01) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sdtp.sdtp_utils
>>> quit()

No import error. The rootdir reported by pytest is correct, and the import failures are in a subdirectory which has an __init__.py file.

Here is the output of pip list:

Package                 Version
----------------------- -----------
blinker                 1.8.2
boltons                 23.0.0
brotlipy                0.7.0
certifi                 2023.7.22
cffi                    1.15.1
charset-normalizer      2.0.4
click                   8.1.7
conda                   23.7.4
conda-content-trust     0.2.0
conda-libmamba-solver   23.5.0
conda-package-handling  2.1.0
conda_package_streaming 0.8.0
coverage                7.5.4
cryptography            41.0.3
Flask                   3.0.3
idna                    3.4
iniconfig               2.0.0
itsdangerous            2.2.0
Jinja2                  3.1.4
jsonpatch               1.32
jsonpointer             2.1
libmambapy              1.4.1
MarkupSafe              2.1.5
numpy                   2.0.0
packaging               23.0
pandas                  2.2.2
pip                     23.1.2
pluggy                  1.5.0
pycosat                 0.6.4
pycparser               2.21
pyOpenSSL               23.2.0
PySocks                 1.7.1
pytest                  8.2.2
pytest-cov              5.0.0
pytest_httpserver       1.0.10
python-dateutil         2.9.0.post0
pytz                    2024.1
requests                2.31.0
ruamel.yaml             0.17.21
setuptools              67.8.0
six                     1.16.0
toolz                   0.12.0
tqdm                    4.65.0
tzdata                  2024.1
urllib3                 1.26.16
Werkzeug                3.0.3
wheel                   0.38.4
zstandard               0.19.0

I'm running pytest 8.2.2, Python 3.11.4, under VSCode in a Docker container. /etc/debian_version is 11.7

The repo for the distribution is in https://github.com/engageLively/stdp. Running the given command in the root directory excites the error

• [X ] a detailed description of the bug or problem you are having
• [X ] output of pip list from the virtual environment you are using
• [ X] pytest and operating system versions
• [X ] minimal example if possible

[rickmcgeer]

It turns out that this was due to a confused path on my part, and pytest's default path. /workspaces was in the python path, so it looked for /workspaces/sdtp/sdtp_utils.py, when the module was /workspaces/sdtp/sdtp/sdtp_utils.py. Fixed by changing this to /workspaces/sdtp/sdtp_data/sdtp_utils.py and importing from sdtp_data.
But I spent hours chasing this, and the directory naming was suggested for pip...so better feedback to the user would have helped. If I can think of a way to do that I'll file an enhancement request. Thanks and sorry to take your time on this one.