Keyring errors during non-publishing operations

[wahuneke]

I gather that keyring integration was added as a convenience feature so that you can publish packages using keys stored in your keyring. This ticket describes that feature:

#210

But it appears that poetry tries to access the keyring even for install operations. In one of my first times using poetry - I, as a security paranoid person, decided to abort installation of a py package I wanted because the installation process seemed to be wanting to access my keyring.

I think people should be careful about granting programs access to things they should not need to access. This behavior is an example of a problem that makes that hard to achieve.

For me, I worked around the problem by pip uninstalling the 'keyring' package from that virt env.

In the future, I think it would be beneficial to poetry's adoption and to its users, if users are not prompted for keyring unless keys should actually be needed (for a 'publish' operation).

Thanks! I'm happy to be trying out poetry.

[sdispater]

Do you have private repositories present in your pyproject.toml file?

I think you are affected by the issue that is fixed by #1892. The fix will be available in the next bug fix release.

[colatkinson]

I was affected by something similar on the latest release (1.0.5), but I'm not sure if it's entirely the same.

The keyring will be hit for private repos if there is an auth.toml with only the username set, even if credentials are provided via environment variables.

I believe the relevant code is here. In this case, auth will be truthy, causing the keyring path to be taken.

In our case, the "fix" was to remove any leftover auth.toml files in the environment. Manually uninstalling keyring had the same effect.

I was wondering your thoughts on adding a config variable that would prevent any keyring hits altogether (by e.g. unconditionally marking the keyring as unavailable). This would have largely the same effect as @wahuneke's workaround, but might be a bit cleaner for CI pipelines and such.

[asandeep]

+1 to this. I am facing similar issue while integrating poetry with Jenkins. Even though private repo credentials were provided via environment variables, poetry still tries to look into keyring for credentials.

[dariobig]

+1 running into this issue while working over ssh

[bdowling]

fwiw, I came across this issue and agree with the concerns mentioned above.

But fwiw, I came here due to a issue with my distro that shows a dependency mismatch on keyring. keyring==18 is required (circa 2019), where as my distro has keyring-21.2 available, not sure without looking on the compatibility issues, but thought I'd mention if in case that can be relaxed....

$ poetry init
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (keyring 21.2.1 (/usr/lib/python3.8/site-packages), Requirement.parse('keyring==18.*,>=18.0.0'), {'poetry'})

[kakarukeys]

I'm seeing this issue in poetry 1.1.6

My setup:
MacOS Mojave 10.14.6
localhost private pypiserver with authentication required for upload only

[[tool.poetry.source]]
name = "foo"
url = "http:https://localhost/simple/"
secondary = true

$ poetry config --list
cache-dir = "..."
experimental.new-installer = true
installer.parallel = true
repositories.foo.url = "http:https://localhost"
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.path = "{cache-dir}/virtualenvs" 

$ cat auth.toml
[http-basic]
[http-basic.foo]
username = "kakarukeys"

The commands that poetry asked for keyring access:
poetry add lxml ??? why? it should get the package from the official PyPI
poetry run python --version this makes no sense either.

if you need the installation scripts for the local private pypi let me know

[alesdakshanin]

+1 this

[abn]

Poetry searches for a package in all package sources by default.

Additionally you might want to add the publishing repository with a different name, if the source uses the same name authenticator will load the credentials.

[erooke]

Poetry 1.2 now hits the keyring for most operations (add, update, install, self update, etc) rendering it unusable for me. I can "fix" the problem by exporting PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring but it would be much nicer if poetry just didn't hit the keyring at all if it wasn't needed.

[txtsd]

I tried poetry for the first time today.

This command will guide you through creating your pyproject.toml config.

Package name [herpderp]:  
Version [0.1.0]:  
Description []:  
Author [txtsd <[email protected]>, n to skip]:  
License []:  GPL-3.0-only
Compatible Python versions [^3.10]:  

Would you like to define your main dependencies interactively? (yes/no) [yes] yes
You can specify a package in the following forms:
  - A single name (requests): this will search for matches on PyPI
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Package to add or search for (leave blank to skip): requests
Found 20 packages matching requests
Showing the first 10 matches

Enter package # to add, or the complete package name if it is not listed []:
 [ 0] requests
 [ 1] requests5
 [ 2] requests3
 [ 3] requests2
 [ 4] pycopy-requests
 [ 5] requests-middleware
 [ 6] grasspy-requests
 [ 7] requests-kerberos
 [ 8] fed-requests
 [ 9] requests-httpsproxy
 [10] 
 > 1
Enter the version constraint to require (or leave blank to use the latest version): 

Failed to open keyring: org.freedesktop.DBus.Error.ServiceUnknown: The name :1.146 was not provided by any .service files.

What a horrible experience. If there isn't a keyring, assume it doesn't exist and continue. Why hard fail at this stage? It's not even like I don't have a keyring. I use KeePassXC's freedesktop.org Secrets integration. It works fine system-wide.

[neersighted]

@txtsd Keyring support is a new feature, and this is a complex interaction across the matrix of possible platforms, configurations, versions, and backends that a user may invoke Poetry in. Many of these sharp edges simply haven't been filed off because of how large that matrix is and how dependent on external factors keychain interactions are.

MRs improving the authentication experience or docs are welcome -- we're going to have to gradually file down these rough edges as keychain support matures.

[willforde]

Had the same issue. Had to change the keyring package config to point to the keyring.backends.SecretService.Keyring backend. For me the keyring package was trying to use kwallet, but I'm using Gnome not KDE.

OS = Arch Linux
DE = Gnome

[txtsd]

Okay, I created this file ~/.config/python_keyring/keyringrc.cfg and populated it with

[backend]
default-keyring=keyring.backends.SecretService.Keyring

Then I unset PYTHON_KEYRING_BACKEND and ran poetry init:

λ poetry init

This command will guide you through creating your pyproject.toml config.

Package name [test]:  
Version [0.1.0]:  
Description []:  
Author [txtsd <[email protected]>, n to skip]:  
License []:  
Compatible Python versions [^3.10]:  

Would you like to define your main dependencies interactively? (yes/no) [yes] 
You can specify a package in the following forms:
  - A single name (requests): this will search for matches on PyPI
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Package to add or search for (leave blank to skip): lxml
Found 20 packages matching lxml
Showing the first 10 matches

Enter package # to add, or the complete package name if it is not listed []:
 [ 0] lxml
 [ 1] lxml-stubs
 [ 2] zsi-lxml
 [ 3] lxml-wrapper
 [ 4] readability-lxml
 [ 5] lxml2dict
 [ 6] suds-lxml
 [ 7] json-lxml
 [ 8] lxml-asserts
 [ 9] gocept.lxml
 [10] 
 > 0
Enter the version constraint to require (or leave blank to use the latest version): 

Empty module name

Different error now.

[neersighted]

There's some busted keyring behavior with your particular backends that seem to

1. invoke with poor error handling during init -- odd (though it's possible that the search code in poetry init was not made robust against keyring errors as an oversight)
2. bypass most of our error handling and spit out unhelpful messages

Some more insight into your system (what backend should be available, OS version, package versions), etc would be helpful for those that attempt to address yours/similar issues @txtsd.

In the meantime, you can revert to 1.1-style writing to disk using the keyring.backends.null.Keyring backend, of course.

[txtsd]

@neersighted
I use Arch Linux

λ pip freeze | grep poetry
poetry==1.2.0
poetry-core==1.1.0
poetry-plugin-export==1.0.6

I set the config to use keyring.backends.null.Keyring for now. Thanks!

[croqaz]

Happened to me today for the first time. I used Poetry until last month without any issues, but today I saw same error.
export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring seems to fix the issue.

[lazka]

I'm seeing the same error on Windows with libsecret installed:

  • Installing urllib3 (1.26.12): Failed
  Error
  g-dbus-error-quark: The name org.freedesktop.secrets is unknown (2)

[viteski]

you can use this in your dev environment to work around the issue on Debian/Ubuntu machines

function unlock_keyring() {
    export "$(echo -n "$(pass)" | gnome-keyring-daemon --replace --unlock)"
    unset "$(pass)"
}

you'll have to apt install the gnome-keyring package
gnome-keyring-pkcs11/jammy-updates,now 40.0-3ubuntu3 amd64 [installed,automatic]
gnome-keyring/jammy-updates,now 40.0-3ubuntu3 amd64 [installed,automatic]
libpam-gnome-keyring/jammy-updates,now 40.0-3ubuntu3 amd64 [installed,automatic]

But, it is troublesome.

[bgyarfas]

Another workaround in the case you don't need the keyring at all

python3 -m keyring --disable

If you are also running pyenv you'll need to run this with the system python

pyenv shell system
python3 -m keyring --disable

[vipcxj]

The same issue for the latest version. A big bong bug not fixed for 3 years?

[ericriff]

I'm just dealing with this myself, on the latest version (1.6.1).
This workaround worked for me

python3 -m keyring --disable

but it is not ideal. I handle the environment of many devs on a company, I can't ask them all to run this command.

[13steinj]

The same issue for the latest version. A big bong bug not fixed for 3 years?

Because

• this is open source software-- unless someone picks it up it won't be priority. I think this was picked up at some point but then dropped.
• I don't think the maintainers care about this particular issue as a "big deal"

In fairness to them, it doesn't happen in most desktop environments. However companies in various fashion have moved many devs to "use an ssh terminal and be stuck with it." or something of the like. Which is sometimes a necessity, but isn't what open source devs "target" in terms of usability testing (if they do usability testing at all).

Similarly in fairness, after doing some evil hack to avoid this issue once, it's not a recurring problem.

[remram44]

"this is open source software" is a terrible excuse, the community has provided many patches that have all been ignored.

[13steinj]

is a terrible excuse, the community has provided many patches that have all been ignored.

I don't disagree, it's just the excuse always given to me by every project I've made a similar comment on 🙃

It may be possible that the maintainers don't have enough time to go through the issues / prs at this point (there are quite a few, after all). It may be possible that they've stopped caring. It may be possible that they don't accept the patches made in various fashion.

In fact, yours, explicitly was asked for tests and it doesn't appear as if you've written them; so, bit odd to complain about ignored patches. And in that there is a reference (without response from the author) about #6612 which has been superseded (and relatively recent) by #8078

I can understand the frustration, but it's not as if things aren't moving and it's not as if the fault is purely on maintainers for not merging any one person's PR.

[dimbleby]

for what it's worth there have been a couple of abandoned tries at this (I know of #6612, #7037 - neither of which were ignored) - but #8227 looks to me as though it's pretty close. So with any luck that'll go in soon

[moberst]

Just came across this multi-year-old issue for the first time only to find that it was just merged into the master branch. Hurray!

For anyone else in a similar position who comes across this issue and is looking for the fix, you can install the current master branch via the following (taken from the installation docs here). Looking forward to this being in the stable version.

curl -sSL https://install.python-poetry.org | python3 - --git https://github.com/python-poetry/poetry.git@master

[simaotwx]

For now I'm using the workaround mentioned before

python3 -m keyring --disable

Bad timing that I just got to use poetry for the first time and got hit by a flood of error messages.

Looks like it has been fixed so I hope this will not be an issue in the future.
IMO it should not hard-fail since the keyring is not necessary to install packages.

[arturtoshev]

Installing the latest version from the master branch as suggested above did the job yesterday, but I now ran into poetry install getting stuck halfway through.

What I found to really solve the problem for me was clearing the cache:

curl -sSL https://install.python-poetry.org | python3 - --uninstall  # uninstall poetry
rm -r $HOME/.cache/pypoetry  # clear cache
curl -sSL https://install.python-poetry.org | python3 -  # install latest release

Could someone add clearing the cache as an option during uninstall or at least mention it in the documentation?

[ilyagr]

I installed poetry 1.7.0, and I now have poetry install and poetry lock hang forever.

Removing the cache as suggested by the previous comment did not help.

The workaround of export PYTHON_KEYRING_BACKEND=keyring.backends.fail.Keyring still works and solves the problem.

See some more details in https://gist.github.com/ilyagr/1bebabebc93e6662d139009d19ed8c1b; I'll probably file a proper bug later. Update: Filed #8623.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.