Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set_env substitution doesn't pass PYTHONHASHSEED in tox 4.x #2872

Open
mgoral opened this issue Jan 16, 2023 · 3 comments
Open

set_env substitution doesn't pass PYTHONHASHSEED in tox 4.x #2872

mgoral opened this issue Jan 16, 2023 · 3 comments
Labels
bug:minor does not affect many people or has no big impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.

Comments

@mgoral
Copy link

mgoral commented Jan 16, 2023

Issue

Hi, I have encountered what I believe is tox bug.

With 2 test environments: base ([testenv]) and non-base ([testenv:hs]), some environment variables from the base one aren't set in the non-base when they are passed as a substitution, like:

[testenv]
setenv =
    PYTHONHASHSEED=0
    OTHER=foo

[testenv:hs]
setenv = {[testenv]setenv}

In above example PYTHONHASHSEED will be random for every run of tox -e has and OTHER will be correctly set to foo. I found out that tox displays the same behaviour for some other variables, like PATH, but I didn't dig any further.

Funny thing is that if we mix setenv and set_env a little bit, tox suddenly starts passing PYTHONHASHSEED, for example below tox.ini sets PYTHONHASHSEED to 0.

[testenv]
set_env =
    PYTHONHASHSEED=0
    OTHER=foo

[testenv:hs]
setenv = {[testenv]set_env}

I checked this on tox 4.3.1, 4.3, 4.2, 4.1 and 4.0. Tox 3.x doesn't have this behaviour.

Full tox.ini files attached below.

Environment

Output of running tox

Provide the output of tox -rvv:

using tox.ini: /home/mgoral/test/tox.ini (pid 279807)
  removing /home/mgoral/test/.tox/log
could not satisfy requires MissingDependency(<Requirement('tox>=4.0')>)
using tox-3.21.4 from /usr/lib/python3/dist-packages/tox/__init__.py (pid 279807)
/usr/bin/python3 (/usr/bin/python3) is {'executable': '/usr/bin/python3', 'implementation': 'CPython', 'version_info': [3, 9, 2, 'final', 0], 'version': '3.9.2 (default, Feb 28 2021, 17:03:44) \n[GCC 10.2.1 20210110]', 'is_64': True, 'sysplatform': 'linux', 'extra_version_info': None}
.tox uses /usr/bin/python3
.tox start: getenv /home/mgoral/test/.tox/.tox
.tox cannot reuse: -r flag
.tox recreate: /home/mgoral/test/.tox/.tox
  removing /home/mgoral/test/.tox/.tox
setting PATH=/home/mgoral/test/.tox/.tox/bin:/home/mgoral/.cargo/bin:/home/mgoral/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
[279827] /home/mgoral/test/.tox$ /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3 .tox
created virtual environment CPython3.9.2.final.0-64 in 81ms
  creator CPython3Posix(dest=/home/mgoral/test/.tox/.tox, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mgoral/.local/share/virtualenv)
    added seed packages: pip==20.3.4, pkg_resources==0.0.0, setuptools==44.1.1, wheel==0.34.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
.tox installdeps: tox>=4.0
setting PATH=/home/mgoral/test/.tox/.tox/bin:/home/mgoral/.cargo/bin:/home/mgoral/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
[279835] /home/mgoral/test$ /home/mgoral/test/.tox/.tox/bin/python -m pip install 'tox>=4.0'
Collecting tox>=4.0
  Using cached tox-4.3.1-py3-none-any.whl (147 kB)
Collecting platformdirs>=2.6.2
  Using cached platformdirs-2.6.2-py3-none-any.whl (14 kB)
Collecting virtualenv>=20.17.1
  Using cached virtualenv-20.17.1-py3-none-any.whl (8.8 MB)
Collecting pyproject-api>=1.4
  Using cached pyproject_api-1.4.0-py3-none-any.whl (12 kB)
Collecting pluggy>=1
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting chardet>=5.1
  Using cached chardet-5.1.0-py3-none-any.whl (199 kB)
Collecting tomli>=2.0.1
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting filelock>=3.9
  Using cached filelock-3.9.0-py3-none-any.whl (9.7 kB)
Collecting colorama>=0.4.6
  Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting packaging>=23
  Using cached packaging-23.0-py3-none-any.whl (42 kB)
Collecting cachetools>=5.2.1
  Using cached cachetools-5.2.1-py3-none-any.whl (9.3 kB)
Collecting distlib<1,>=0.3.6
  Using cached distlib-0.3.6-py2.py3-none-any.whl (468 kB)
Installing collected packages: tomli, platformdirs, packaging, filelock, distlib, virtualenv, pyproject-api, pluggy, colorama, chardet, cachetools, tox
Successfully installed cachetools-5.2.1 chardet-5.1.0 colorama-0.4.6 distlib-0.3.6 filelock-3.9.0 packaging-23.0 platformdirs-2.6.2 pluggy-1.0.0 pyproject-api-1.4.0 tomli-2.0.1 tox-4.3.1 virtualenv-20.17.1
.tox finish: getenv /home/mgoral/test/.tox/.tox after 3.43 seconds
.tox start: finishvenv 
write config to /home/mgoral/test/.tox/.tox/.tox-config1 as '409276cb52787e20907912730020ca0f84204a375c30a79fcb494ffde0e0f116 /usr/bin/python3\n3.21.4 0 0 0\n00000000000000000000000000000000 tox>=4.0'
.tox finish: finishvenv  after 0.02 seconds
.tox start: provision 
[279851] /home/mgoral/test$ /home/mgoral/test/.tox/.tox/bin/python -m tox -rvv -e hs
hs: 135 W remove tox env folder /home/mgoral/test/.tox/hs [tox/tox_env/api.py:321]
hs: 159 I find interpreter for spec PythonSpec(major=3) [virtualenv/discovery/builtin.py:56]
hs: 159 D discover exe for PythonInfo(spec=CPython3.9.2.final.0-64, exe=/home/mgoral/test/.tox/.tox/bin/python, platform=linux, version='3.9.2 (default, Feb 28 2021, 17:03:44) \n[GCC 10.2.1 20210110]', encoding_fs_io=utf-8-utf-8) in /usr [virtualenv/discovery/py_info.py:437]
hs: 159 D filesystem is case-sensitive [virtualenv/info.py:24]
hs: 160 D got python info of /usr/bin/python3.9 from /home/mgoral/.local/share/virtualenv/py_info/1/36cf16204b8548560b1c020c4e8fb5b57f0e4c58016f52f2d4be01e192833930.json [virtualenv/app_data/via_disk_folder.py:129]
hs: 161 I proposed PythonInfo(spec=CPython3.9.2.final.0-64, system=/usr/bin/python3.9, exe=/home/mgoral/test/.tox/.tox/bin/python, platform=linux, version='3.9.2 (default, Feb 28 2021, 17:03:44) \n[GCC 10.2.1 20210110]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:63]
hs: 161 D accepted PythonInfo(spec=CPython3.9.2.final.0-64, system=/usr/bin/python3.9, exe=/home/mgoral/test/.tox/.tox/bin/python, platform=linux, version='3.9.2 (default, Feb 28 2021, 17:03:44) \n[GCC 10.2.1 20210110]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:65]
hs: 186 I create virtual environment via CPython3Posix(dest=/home/mgoral/test/.tox/hs, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:48]
hs: 186 D create folder /home/mgoral/test/.tox/hs/bin [virtualenv/util/path/_sync.py:9]
hs: 186 D create folder /home/mgoral/test/.tox/hs/lib/python3.9/site-packages [virtualenv/util/path/_sync.py:9]
hs: 187 D write /home/mgoral/test/.tox/hs/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
hs: 187 D 	home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
hs: 187 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
hs: 187 D 	version_info = 3.9.2.final.0 [virtualenv/create/pyenv_cfg.py:34]
hs: 187 D 	virtualenv = 20.17.1 [virtualenv/create/pyenv_cfg.py:34]
hs: 187 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
hs: 187 D 	base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
hs: 187 D 	base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
hs: 187 D 	base-executable = /usr/bin/python3.9 [virtualenv/create/pyenv_cfg.py:34]
hs: 187 D symlink /usr/bin/python3.9 to /home/mgoral/test/.tox/hs/bin/python [virtualenv/util/path/_sync.py:28]
hs: 187 D create virtualenv import hook file /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:89]
hs: 187 D create /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:92]
hs: 188 D ============================== target debug ============================== [virtualenv/run/session.py:50]
hs: 188 D debug via /home/mgoral/test/.tox/hs/bin/python /home/mgoral/test/.tox/.tox/lib/python3.9/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:197]
hs: 188 D {
  "sys": {
    "executable": "/home/mgoral/test/.tox/hs/bin/python",
    "_base_executable": "/home/mgoral/test/.tox/hs/bin/python",
    "prefix": "/home/mgoral/test/.tox/hs",
    "base_prefix": "/usr",
    "real_prefix": null,
    "exec_prefix": "/home/mgoral/test/.tox/hs",
    "base_exec_prefix": "/usr",
    "path": [
      "/usr/lib/python39.zip",
      "/usr/lib/python3.9",
      "/usr/lib/python3.9/lib-dynload",
      "/home/mgoral/test/.tox/hs/lib/python3.9/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.9.2 (default, Feb 28 2021, 17:03:44) \n[GCC 10.2.1 20210110]",
  "makefile_filename": "/usr/lib/python3.9/config-3.9-x86_64-linux-gnu/Makefile",
  "os": "<module 'os' from '/usr/lib/python3.9/os.py'>",
  "site": "<module 'site' from '/usr/lib/python3.9/site.py'>",
  "datetime": "<module 'datetime' from '/usr/lib/python3.9/datetime.py'>",
  "math": "<module 'math' (built-in)>",
  "json": "<module 'json' from '/usr/lib/python3.9/json/__init__.py'>"
} [virtualenv/run/session.py:51]
hs: 213 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mgoral/.local/share/virtualenv) [virtualenv/run/session.py:55]
hs: 215 D got embed update of distribution wheel from /home/mgoral/.local/share/virtualenv/wheel/3.9/embed/3/wheel.json [virtualenv/app_data/via_disk_folder.py:129]
hs: 216 D got embed update of distribution setuptools from /home/mgoral/.local/share/virtualenv/wheel/3.9/embed/3/setuptools.json [virtualenv/app_data/via_disk_folder.py:129]
hs: 216 D got embed update of distribution pip from /home/mgoral/.local/share/virtualenv/wheel/3.9/embed/3/pip.json [virtualenv/app_data/via_disk_folder.py:129]
hs: 219 D install wheel from wheel /home/mgoral/test/.tox/.tox/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/wheel-0.38.4-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
hs: 219 D install setuptools from wheel /home/mgoral/test/.tox/.tox/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/setuptools-65.6.3-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
hs: 219 D install pip from wheel /home/mgoral/test/.tox/.tox/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.3.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
hs: 220 D copy directory /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip-22.3.1.dist-info to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/pip-22.3.1.dist-info [virtualenv/util/path/_sync.py:36]
hs: 220 D copy /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools-65.6.3.virtualenv to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/setuptools-65.6.3.virtualenv [virtualenv/util/path/_sync.py:36]
hs: 220 D copy directory /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/wheel [virtualenv/util/path/_sync.py:36]
hs: 221 D copy directory /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/setuptools [virtualenv/util/path/_sync.py:36]
hs: 224 D copy directory /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/pip [virtualenv/util/path/_sync.py:36]
hs: 227 D copy directory /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.dist-info to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/wheel-0.38.4.dist-info [virtualenv/util/path/_sync.py:36]
hs: 229 D copy /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.virtualenv to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/wheel-0.38.4.virtualenv [virtualenv/util/path/_sync.py:36]
hs: 231 D generated console scripts wheel-3.9 wheel3 wheel3.9 wheel [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
hs: 256 D copy /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/distutils-precedence.pth to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:36]
hs: 256 D copy directory /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/pkg_resources to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/pkg_resources [virtualenv/util/path/_sync.py:36]
hs: 264 D copy directory /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/_distutils_hack to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:36]
hs: 264 D copy directory /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools-65.6.3.dist-info to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/setuptools-65.6.3.dist-info [virtualenv/util/path/_sync.py:36]
hs: 265 D generated console scripts  [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
hs: 286 D copy /home/mgoral/.local/share/virtualenv/wheel/3.9/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip-22.3.1.virtualenv to /home/mgoral/test/.tox/hs/lib/python3.9/site-packages/pip-22.3.1.virtualenv [virtualenv/util/path/_sync.py:36]
hs: 287 D generated console scripts pip3.9 pip3 pip-3.9 pip [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
hs: 287 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:61]
hs: 288 D write /home/mgoral/test/.tox/hs/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
hs: 288 D 	home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
hs: 288 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
hs: 288 D 	version_info = 3.9.2.final.0 [virtualenv/create/pyenv_cfg.py:34]
hs: 288 D 	virtualenv = 20.17.1 [virtualenv/create/pyenv_cfg.py:34]
hs: 288 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
hs: 288 D 	base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
hs: 288 D 	base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
hs: 288 D 	base-executable = /usr/bin/python3.9 [virtualenv/create/pyenv_cfg.py:34]
hs: 291 W commands[0]> python -c 'import os; print(os.environ["PYTHONHASHSEED"])' [tox/tox_env/api.py:427]
288846098
hs: 311 I exit 0 (0.02 seconds) /home/mgoral/test> python -c 'import os; print(os.environ["PYTHONHASHSEED"])' pid=279864 [tox/execute/api.py:275]
hs: 312 W commands[1]> python -c 'import os; print(os.environ["OTHER"])' [tox/tox_env/api.py:427]
foo
hs: 327 I exit 0 (0.01 seconds) /home/mgoral/test> python -c 'import os; print(os.environ["OTHER"])' pid=279870 [tox/execute/api.py:275]
  hs: OK (0.19=setup[0.16]+cmd[0.02,0.01] seconds)
  congratulations :) (0.24 seconds)
.tox finish: provision  after 0.39 seconds

Minimal example

tox.ini which doesn't pass PYTHONHASHSEED to hs environment, but passes OTHER:

[tox]
requires = tox>=4.0
skipsdist = True

[testenv]
basepython = python3

setenv =
    PYTHONHASHSEED=0
    OTHER=foo

[testenv:hs]
commands =
    python -c 'import os; print(os.environ["PYTHONHASHSEED"])'
    python -c 'import os; print(os.environ["OTHER"])'

setenv = {[testenv]setenv}

tox.ini which passes PYTHONHASHSEED to hs environment (watch the underscores in set_env and setenv)

[tox]
requires = tox>=4.0
skipsdist = True

[testenv]
basepython = python3

set_env =
    PYTHONHASHSEED=0
    OTHER=foo

[testenv:hs]
commands =
    python -c 'import os; print(os.environ["PYTHONHASHSEED"])'
    python -c 'import os; print(os.environ["OTHER"])'

setenv = {[testenv]set_env}
@gaborbernat gaborbernat added bug:minor does not affect many people or has no big impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted. labels Jan 16, 2023
@gaborbernat gaborbernat added this to the P-1 milestone Jan 16, 2023
@Perfect5th
Copy link

Perfect5th commented Feb 4, 2023

Looks like this is happening because of the order in which ConfigDynamicDefinition resolves keys in sections in the ini. The second example above "works" because:

  1. it looks for set_env in [testenv:hs]
  2. Failing that, it looks for set_env in [testenv]
  3. Failing that, it looks for setenv in [testenv:hs]
  4. Failing that, it looks for setenv in [testenv]

This is because of the way product works:

for key, loader in product(self.keys, loaders):

So in the second example, where it is set to 0, it's finding the value on step 2, so the substitution is never actually used.

PYTHONHASHSEED is special because it is set on the env before ini substitutions occur, using SetEnv.update (replacements don't happen until iteration does):

def __iter__(self) -> Iterator[str]:
# start with the materialized ones, maybe we don't need to materialize the raw ones
yield from self._materialized.keys()
yield from list(self._raw.keys()) # iterating over this may trigger materialization and change the dict
while self._needs_replacement:
line = self._needs_replacement.pop(0)
expanded_line = self._replacer(line, ConfigLoadArgs([], self._name, self._env_name))
sub_raw = dict(self._extract_key_value(sub_line) for sub_line in expanded_line.splitlines() if sub_line)
self._raw.update(sub_raw)
yield from sub_raw.keys()
def update(self, param: Mapping[str, str], *, override: bool = True) -> None:
for key, value in param.items():
# do not override something already set explicitly
if override or (key not in self._raw and key not in self._materialized):
self._materialized[key] = value
self.changed = True

Reversing the product args to be product(loaders, self.keys) would make the precedence [testenv:hs]set_env > [testenv:hs]setenv > [testenv]set_env > [testenv]setenv, which I think is more intuitive, but won't fix the special-casing of PYTHONHASHSEED:

def _default_set_env(self) -> dict[str, str]:
env = super()._default_set_env()
hash_seed: int | None = getattr(self.options, "hash_seed", None)
if hash_seed is not None:
env["PYTHONHASHSEED"] = str(hash_seed)
return env

mr-mixas added a commit to mr-mixas/Nested-Diff.py that referenced this issue May 12, 2023
setenv defined explicitly until regression fixed in tox:
* tox-dev/tox#2972
* tox-dev/tox#2872
mr-mixas added a commit to mr-mixas/Nested-Diff.py that referenced this issue May 12, 2023
setenv defined explicitly until regression fixed in tox:
* tox-dev/tox#2972
* tox-dev/tox#2872
mr-mixas added a commit to mr-mixas/Nested-Diff.py that referenced this issue May 12, 2023
setenv defined explicitly until regression fixed in tox:
* tox-dev/tox#2972
* tox-dev/tox#2872
@gaborbernat gaborbernat removed this from the P-1 milestone Jun 17, 2023
@francis-clairicia
Copy link

Any updates on this issue?

@gaborbernat
Copy link
Member

Yes, pull requests to fix this are welcome. We don't plan to work on this otherwise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug:minor does not affect many people or has no big impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.
Projects
None yet
Development

No branches or pull requests

4 participants