pep621 manager incorrectly includes build-backend dependencies during lockfile generation

[sigma67]

What would you like help with?

I think I found a bug

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Gitlab

Please tell us more about your question or problem

Sample pyproject.toml:

[project]
dependencies = [
    "filelock>=3.14.0",
]

[build-system]
requires = ["pdm-backend", "pdm-build-locked==2.0.0"]

Since we are grouping all dep updates in a single MR, renovate will correctly update both deps to their latest versions.

However, since #26440, it will attempt to perform lockfile maintenance on build-system deps, which are not included in the lockfile by pdm: pdm-project/pdm#2465

**Artifact update problem**

Command failed: pdm update --no-sync  filelock pdm-build-locked
WARNING: Project requires a python version of >=3.10, The virtualenv is being created for you as it cannot be matched to the right version.
INFO: python.use_venv is on, creating a virtualenv for this project...
Virtualenv is created successfully at /builds/pes/infrastructure/general-libraries/python/renovate-bot-python/renovate/repos/gitlab/pes/infrastructure/product-development/vibe/pes-vibe/.venv
[ProjectError]: pdm-build-locked does not exist in default dependencies nor is a transitive dependency.

Logs (if relevant)

Logs

Replace this text with your logs, between the starting and ending triple backticks

[HonkingGoose]

Maybe this discussion is related?

• [Reoccurred] Python with Poetry failed to update poetry.lock due to wrong Python version #29607

Also what version number of Renovate are you running now?

[sigma67]

I fail to see why it should be related. Seems to be a different manager and issue. Please elaborate.

We are running on 37.342.1 currently.

[HonkingGoose]

I meant that I'm seeing more Python version related problems on this repository lately. And the discussion I linked above came to mind. 😄

Thank you for telling us what version you're using now! ❤️

I see a maintainer is asking for a minimal reproduction. We have a nice template repository you can use as a base. See the links in the bot comment.

[sigma67]

Probably coincidence

I will create a minimal reproduction: https://github.com/sigma67/renovate-pep621-buildsystem-requires-reproduction

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

Please follow these steps:

1. Read our guide on creating a minimal reproduction.
2. Go to our minimal reproduction template repository.
3. Select the Use this template button to create a new repository based on the template.
4. Work on the minimal reproduction in your own repository.
5. Fill out the information in your repository's README.md.
6. Add the link to your reproduction to the first post of your Discussion. If you are not the original author, you can post a new comment with the link.

Good luck,

The Renovate team

[Churro]

I think the issue here is that pdm-build-locked is an optional build plugin for pdm. Your pyproject.toml includes a reference to it but renovate doesn't install it as a build-time dependency and thus updating the lock file fails.

[sigma67]

Not really - it's not building anything. You can test locally by running the same pdm update command as above. It fails instantly, even if you manually install the package.

[sigma67]

pdm update --no-sync filelock pdm-build-locked

[secustor]

So basically every package under build-system.requires should be excluded from pdm update commands?

[sigma67]

@HonkingGoose @Churro @secustor I tried pretty hard but I was unable to reproduce this on GitLab with latest renovate in my repro repo.

Our renovate version is from beginning of May. So I dug a bit deeper and it turns out it has been fixed (incidentally) thanks to a recent PR: #29183

I will test a bit more in our intranet with the latest version, but I expect no further issues with latest

[sigma67]

Update: It has indeed solved the issue on the repos/branches where we had it.