Artifact Update Problem - NuGet PackageSourceMapping

[gtbuchanan]

What would you like help with?

I think I found a bug

How are you running Renovate?

Mend Renovate hosted app on github.com

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

No response

Please tell us more about your question or problem

We have many repositories that successfully use Renovate with NuGet Package Source Mapping without lock files. However, our one repository that uses lock files is having issues updating them. Every new pull request ends up with an "Artifact Update Problem" comment from Renovate with the error:

PackageSourceMapping is enabled and there are multiple package sources associated with the same key(s): nuget.org

We have the same NuGet.Config in every repository (including the one that fails):

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="Azure-Internal" value="https://pkgs.dev.azure.com/**redacted**/nuget/v3/index.json" />
  </packageSources>
  <packageSourceMapping>
    <packageSource key="nuget.org">
      <package pattern="*" />
    </packageSource>
    <packageSource key="Azure-Internal">
      <package pattern="Ewn.*" />
    </packageSource>
  </packageSourceMapping>
</configuration>

We don't have a nuget section defined in renovate.json.

I don't see a problem with our configuration, so I suspect this is an issue with Renovate when dealing with NuGet lock files.

Logs (if relevant)

Logs

DEBUG: syncBranchState() (branch="renovate/fluid.core-2.x")
DEBUG: getBranchPr(renovate/fluid.core-2.x) (branch="renovate/fluid.core-2.x")
DEBUG: findPr(renovate/fluid.core-2.x, undefined, open) (branch="renovate/fluid.core-2.x")
DEBUG: findPr(renovate/fluid.core-2.x, undefined, closed) (branch="renovate/fluid.core-2.x")
DEBUG: branchExists=false (branch="renovate/fluid.core-2.x")
DEBUG: dependencyDashboardCheck=approve (branch="renovate/fluid.core-2.x")
DEBUG: Check for closed PR because recreating closed PRs is disabled. (branch="renovate/fluid.core-2.x")
DEBUG: findPr(renovate/fluid.core-2.x, Update dependency Fluid.Core to v2.10.0, !open) (branch="renovate/fluid.core-2.x")
DEBUG: prAlreadyExisted=false (branch="renovate/fluid.core-2.x")
DEBUG: Checking schedule(schedule=at any time, tz=America/Chicago, now=2024-06-04T14:52:23.402Z) (branch="renovate/fluid.core-2.x")
DEBUG: No schedule defined (branch="renovate/fluid.core-2.x")
DEBUG: Branch needs creating (branch="renovate/fluid.core-2.x")
DEBUG: Using reuseExistingBranch: false (branch="renovate/fluid.core-2.x")
DEBUG: Setting current branch to dev (branch="renovate/fluid.core-2.x")
DEBUG: latest commit (branch="renovate/fluid.core-2.x")
{
  "branchName": "dev"
  "latestCommitDate": "2024-06-04T14:41:56Z"
}

DEBUG: manager.getUpdatedPackageFiles() reuseExistingBranch=false (branch="renovate/fluid.core-2.x")
DEBUG: Starting search at index 4308 (branch="renovate/fluid.core-2.x")
{
  "packageFile": "Directory.Packages.props"
  "depName": "Fluid.Core"
}

DEBUG: Found match at index 4308 (branch="renovate/fluid.core-2.x")
{
  "packageFile": "Directory.Packages.props"
  "depName": "Fluid.Core"
}

DEBUG: Found NuGet.config at NuGet.Config (branch="renovate/fluid.core-2.x")
DEBUG: Adding registry URL https://api.nuget.org/v3/index.json#protocolVersion=3 (branch="renovate/fluid.core-2.x")
{
  "registryUrl": "https://api.nuget.org/v3/index.json#protocolVersion=3"
  "sourceMappedPackagePatterns": [
    "*"
  ]
}

DEBUG: Adding registry URL https://pkgs.dev.azure.com/**redacted**/nuget/v3/index.json (branch="renovate/fluid.core-2.x")
{
  "registryUrl": "https://pkgs.dev.azure.com/**redacted**/nuget/v3/index.json"
  "sourceMappedPackagePatterns": [
    "Ewn.*"
  ]
}

DEBUG: Contents updated (branch="renovate/fluid.core-2.x")
{
  "packageFile": "Directory.Packages.props"
  "depName": "Fluid.Core"
}

DEBUG: updateArtifacts for updatedPackageFiles (branch="renovate/fluid.core-2.x")
DEBUG: nuget.updateArtifacts(Directory.Packages.props) (branch="renovate/fluid.core-2.x")
DEBUG: Found NuGet.config at NuGet.Config (branch="renovate/fluid.core-2.x")
DEBUG: Adding registry URL https://api.nuget.org/v3/index.json#protocolVersion=3 (branch="renovate/fluid.core-2.x")
{
  "registryUrl": "https://api.nuget.org/v3/index.json#protocolVersion=3"
  "sourceMappedPackagePatterns": [
    "*"
  ]
}

DEBUG: Adding registry URL https://pkgs.dev.azure.com/**redacted**/nuget/v3/index.json (branch="renovate/fluid.core-2.x")
{
  "registryUrl": "https://pkgs.dev.azure.com/**redacted**/nuget/v3/index.json"
  "sourceMappedPackagePatterns": [
    "Ewn.*"
  ]
}

DEBUG: Setting CONTAINERBASE_CACHE_DIR to /tmp/renovate/cache/containerbase (branch="renovate/fluid.core-2.x")
DEBUG: Using containerbase dynamic installs (branch="renovate/fluid.core-2.x")
DEBUG: Resolved stable matching version (branch="renovate/fluid.core-2.x")
{
  "toolName": "dotnet"
  "resolvedVersion": "8.0.301"
}

DEBUG: Executing command (branch="renovate/fluid.core-2.x")
{
  "command": "install-tool dotnet 8.0.301"
}

DEBUG: exec completed (branch="renovate/fluid.core-2.x")
{
  "durationMs": 24200
  "stdout": "[14:52:30.519] INFO (227): Installing tool [email protected]...\n.NET SDK:\n Version:           8.0.301\n Commit:            1a0e9c0300\n Workload version:  8.0.300-manifests.f6879a9a\n MSBuild version:   17.10.4+10fbfbf2e\n\nRuntime Environment:\n OS Name:     ubuntu\n OS Version:  20.04\n OS Platform: Linux\n RID:         linux-x64\n Base Path:   /opt/containerbase/tools/dotnet/sdk/8.0.301/\n\n.NET workloads installed:\nThere are no installed workloads to display.\n\nHost:\n  Version:      8.0.6\n  Architecture: x64\n  Commit:       3b8b000a0e\n\n.NET SDKs installed:\n  8.0.301 [/opt/containerbase/tools/dotnet/sdk]\n\n.NET runtimes installed:\n  Microsoft.AspNetCore.App 8.0.6 [/opt/containerbase/tools/dotnet/shared/Microsoft.AspNetCore.App]\n  Microsoft.NETCore.App 8.0.6 [/opt/containerbase/tools/dotnet/shared/Microsoft.NETCore.App]\n\nOther architectures found:\n  None\n\nEnvironment variables:\n  DOTNET_ROOT       [/opt/containerbase/tools/dotnet]\n\nglobal.json file:\n  Not found\n\nLearn more:\n  https://aka.ms/dotnet/info\n\nDownload .NET:\n  https://aka.ms/dotnet/download\n[14:52:53.368] INFO (227): Installed tool dotnet in 22.8s.\n"
  "stderr": ""
}

DEBUG: Executing command (branch="renovate/fluid.core-2.x")
{
  "command": "dotnet restore build/Build.csproj --force-evaluate --configfile /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config"
}

DEBUG: rawExec err (branch="renovate/fluid.core-2.x")
{
  "err": {
    "cmd": "/bin/sh -c dotnet restore build/Build.csproj --force-evaluate --configfile /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config",
    "stderr": "",
    "stdout": "  Determining projects to restore...\n/opt/containerbase/tools/dotnet/sdk/8.0.301/NuGet.targets(694,5): error : PackageSourceMapping is enabled and there are multiple package sources associated with the same key(s): nuget.org. Path: /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config [/tmp/renovate/repos/github/**redacted**/build/Build.csproj]\n",
    "options": {
      "cwd": "/tmp/renovate/repos/github/**redacted**",
      "encoding": "utf-8",
      "env": {
        "NUGET_PACKAGES": "/tmp/renovate/cache/__renovate-private-cache/nuget/packages",
        "MSBUILDDISABLENODEREUSE": "1",
        "HOME": "/home/ubuntu",
        "PATH": "/home/ubuntu/.cargo/bin:/home/ubuntu/.local/bin:/go/bin:/home/ubuntu/bin:/home/ubuntu/.npm-global/bin:/home/ubuntu/.cargo/bin:/home/ubuntu/.local/bin:/go/bin:/home/ubuntu/bin:/home/ubuntu/.npm-global/bin:/home/ubuntu/.cargo/bin:/home/ubuntu/.local/bin:/go/bin:/home/ubuntu/bin:/home/ubuntu/.npm-global/bin:/home/ubuntu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "LC_ALL": "C.UTF-8",
        "LANG": "C.UTF-8",
        "CONTAINERBASE_CACHE_DIR": "/tmp/renovate/cache/containerbase"
      },
      "maxBuffer": 10485760,
      "timeout": 900000
    },
    "exitCode": 1,
    "name": "ExecError",
    "message": "Command failed: dotnet restore build/Build.csproj --force-evaluate --configfile /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config\n",
    "stack": "ExecError: Command failed: dotnet restore build/Build.csproj --force-evaluate --configfile /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config\n\n    at ChildProcess.<anonymous> (/usr/local/renovate/lib/util/exec/common.ts:99:11)\n    at ChildProcess.emit (node:events:529:35)\n    at ChildProcess.emit (node:domain:489:12)\n    at Process.ChildProcess._handle.onexit (node:internal/child_process:292:12)"
  }
  "durationMs": 1974
}

DEBUG: Failed to generate lock file (branch="renovate/fluid.core-2.x")
{
  "err": {
    "cmd": "/bin/sh -c dotnet restore build/Build.csproj --force-evaluate --configfile /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config",
    "stderr": "",
    "stdout": "  Determining projects to restore...\n/opt/containerbase/tools/dotnet/sdk/8.0.301/NuGet.targets(694,5): error : PackageSourceMapping is enabled and there are multiple package sources associated with the same key(s): nuget.org. Path: /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config [/tmp/renovate/repos/github/**redacted**/build/Build.csproj]\n",
    "options": {
      "cwd": "/tmp/renovate/repos/github/**redacted**",
      "encoding": "utf-8",
      "env": {
        "NUGET_PACKAGES": "/tmp/renovate/cache/__renovate-private-cache/nuget/packages",
        "MSBUILDDISABLENODEREUSE": "1",
        "HOME": "/home/ubuntu",
        "PATH": "/home/ubuntu/.cargo/bin:/home/ubuntu/.local/bin:/go/bin:/home/ubuntu/bin:/home/ubuntu/.npm-global/bin:/home/ubuntu/.cargo/bin:/home/ubuntu/.local/bin:/go/bin:/home/ubuntu/bin:/home/ubuntu/.npm-global/bin:/home/ubuntu/.cargo/bin:/home/ubuntu/.local/bin:/go/bin:/home/ubuntu/bin:/home/ubuntu/.npm-global/bin:/home/ubuntu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
        "LC_ALL": "C.UTF-8",
        "LANG": "C.UTF-8",
        "CONTAINERBASE_CACHE_DIR": "/tmp/renovate/cache/containerbase"
      },
      "maxBuffer": 10485760,
      "timeout": 900000
    },
    "exitCode": 1,
    "name": "ExecError",
    "message": "Command failed: dotnet restore build/Build.csproj --force-evaluate --configfile /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config\n",
    "stack": "ExecError: Command failed: dotnet restore build/Build.csproj --force-evaluate --configfile /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config\n\n    at ChildProcess.<anonymous> (/usr/local/renovate/lib/util/exec/common.ts:99:11)\n    at ChildProcess.emit (node:events:529:35)\n    at ChildProcess.emit (node:domain:489:12)\n    at Process.ChildProcess._handle.onexit (node:internal/child_process:292:12)"
  }
}

DEBUG: Updated 1 package files (branch="renovate/fluid.core-2.x")
DEBUG: Getting updated lock files (branch="renovate/fluid.core-2.x")
DEBUG: Writing package.json files (branch="renovate/fluid.core-2.x")
{
  "packageFiles": [
    "src/Ewn.Ui/package.json",
    "src/Ewn.Web.App/ClientApp/package.json",
    "src/Ewn.Web.Auth/ClientApp/package.json",
    "src/Ewn.Web.TestCenter/ClientApp/package.json"
  ]
}

DEBUG: Writing any updated package files (branch="renovate/fluid.core-2.x")
DEBUG: No updated lock files in branch (branch="renovate/fluid.core-2.x")
DEBUG: Branch timestamp: 2024-05-17T18:36:21.413Z (branch="renovate/fluid.core-2.x")
DEBUG: PR is older than 2 hours, raise PR with lock file errors (branch="renovate/fluid.core-2.x")
DEBUG: 1 file(s) to commit (branch="renovate/fluid.core-2.x")
DEBUG: Preparing files for committing to branch renovate/fluid.core-2.x (branch="renovate/fluid.core-2.x")
DEBUG: Setting git author name: renovate[bot] (branch="renovate/fluid.core-2.x")
DEBUG: Setting git author email: 29139614+renovate[bot]@users.noreply.github.com (branch="renovate/fluid.core-2.x")
DEBUG: git commit (branch="renovate/fluid.core-2.x")
{
  "deletedFiles": []
  "ignoredFiles": []
  "result": {
    "author": null,
    "branch": "renovate/fluid.core-2.x",
    "commit": "de1e7b7661b30de8466210d3ce347d83306a3862",
    "root": false,
    "summary": {
      "changes": 1,
      "insertions": 1,
      "deletions": 1
    }
  }
}

DEBUG: HEAD https://api.github.com/repos/**redacted**/git/refs/heads/renovate/fluid.core-2.x/ = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=126) (branch="renovate/fluid.core-2.x")
DEBUG: HEAD https://api.github.com/repos/**redacted**/git/refs/heads/renovate/fluid.core-2.x = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=149) (branch="renovate/fluid.core-2.x")
DEBUG: resetToCommit(b59012386343da3a9994c37f7df84e2dba3f5902) (branch="renovate/fluid.core-2.x")
DEBUG: Fetching branch renovate/fluid.core-2.x (branch="renovate/fluid.core-2.x")
DEBUG: Setting current branch to dev (branch="renovate/fluid.core-2.x")
DEBUG: latest commit (branch="renovate/fluid.core-2.x")
{
  "branchName": "dev"
  "latestCommitDate": "2024-06-04T14:41:56Z"
}

INFO: Branch created (branch="renovate/fluid.core-2.x")
{
  "commitSha": "bb1c779ea7673d3368fe30fbafad6737030972ba"
}

DEBUG: Updating status check state to failed (branch="renovate/fluid.core-2.x")
DEBUG: Setting branch status (branch="renovate/fluid.core-2.x")
{
  "context": "renovate/artifacts"
  "state": "red"
}

DEBUG: http cache: saving https://api.github.com/repos/**redacted**/commits/renovate/fluid.core-2.x/status (etag=W/"bfba464e5dd3e0fa92f565652b840bd438e73393a9f1026adf3f105d752fde68", lastModified=undefined) (branch="renovate/fluid.core-2.x")
DEBUG: Updating renovate/stability-days status check state to green (branch="renovate/fluid.core-2.x")
DEBUG: Setting branch status (branch="renovate/fluid.core-2.x")
{
  "context": "renovate/stability-days"
  "state": "green"
}

DEBUG: http cache: saving https://api.github.com/repos/**redacted**/commits/renovate/fluid.core-2.x/status (etag=W/"dbf7db2b4a112ef3d0a3fee3f7aee0fec0ae9af61bb63253e1c701592ccc755a", lastModified=undefined) (branch="renovate/fluid.core-2.x")
DEBUG: Ensuring PR (branch="renovate/fluid.core-2.x")
DEBUG: There are 0 errors and 0 warnings (branch="renovate/fluid.core-2.x")
DEBUG: getBranchPr(renovate/fluid.core-2.x) (branch="renovate/fluid.core-2.x")
DEBUG: findPr(renovate/fluid.core-2.x, undefined, open) (branch="renovate/fluid.core-2.x")
DEBUG: findPr(renovate/fluid.core-2.x, undefined, closed) (branch="renovate/fluid.core-2.x")
DEBUG: getPrCache() (branch="renovate/fluid.core-2.x")
DEBUG: Forcing PR because of artifact errors (branch="renovate/fluid.core-2.x")
DEBUG: Fetching changelog: https://github.com/sebastienros/fluid (2.9.0 -> 2.10.0) (branch="renovate/fluid.core-2.x")
DEBUG: Creating PR (branch="renovate/fluid.core-2.x")
{
  "prTitle": "Update dependency Fluid.Core to v2.10.0"
}

DEBUG: Creating PR (branch="renovate/fluid.core-2.x")
{
  "title": "Update dependency Fluid.Core to v2.10.0"
  "head": "energyworldnet:renovate/fluid.core-2.x"
  "base": "dev"
  "draft": true
}

DEBUG: PR created (branch="renovate/fluid.core-2.x")
{
  "pr": 1706
  "draft": true
}

DEBUG: Adding labels 'dependencies' to #1706 (branch="renovate/fluid.core-2.x")
INFO: PR created (branch="renovate/fluid.core-2.x")
{
  "pr": 1706
  "prTitle": "Update dependency Fluid.Core to v2.10.0"
}

DEBUG: addParticipants(pr=1706) (branch="renovate/fluid.core-2.x")
DEBUG: setPrCache() (branch="renovate/fluid.core-2.x")
DEBUG: Created Pull Request #1706 (branch="renovate/fluid.core-2.x")
WARN: artifactErrors (branch="renovate/fluid.core-2.x")
{
  "artifactErrors": [
    {
      "lockFile": "build/packages.lock.json, src/Ewn.Api/packages.lock.json, src/Ewn.Ui/packages.lock.json, test/Ewn.Ui.Test/packages.lock.json, test/Ewn.Api.Test/packages.lock.json, src/Ewn.Data/packages.lock.json, src/Ewn.MessageProcessor/packages.lock.json, test/Ewn.MessageProcessor.Test/packages.lock.json, src/Ewn.DataAccess/packages.lock.json, src/Ewn.Services/packages.lock.json, src/Ewn.Services.Autofac/packages.lock.json, src/Ewn.Web.Auth/packages.lock.json, test/Ewn.Web.Auth.Test/packages.lock.json, test/Ewn.Services.Autofac.Test/packages.lock.json, test/Ewn.Fluid.Test/packages.lock.json, test/Ewn.Services.Test.Integration/packages.lock.json, test/Ewn.Services.Test/packages.lock.json, src/Ewn.Validation/packages.lock.json, test/Ewn.Validation.Test/packages.lock.json, src/Ewn.Web.TestCenter/packages.lock.json, test/Ewn.Web.TestCenter.Test/packages.lock.json, test/Ewn.DataAccess.Test/packages.lock.json, src/Ewn.DataTransfer/packages.lock.json, test/Ewn.DataTransfer.Test/packages.lock.json, src/Ewn.Extensions.Options.Autofac/packages.lock.json, test/Ewn.Extensions.Options.Autofac.Test/packages.lock.json, src/Ewn.Extensions.Options/packages.lock.json, test/Ewn.Extensions.Options.Test/packages.lock.json, src/Ewn.Fluid/packages.lock.json, src/Ewn.Model/packages.lock.json, src/Ewn.Security/packages.lock.json, src/Ewn.Security.Autofac/packages.lock.json, test/Ewn.Security.Autofac.Test/packages.lock.json, src/Ewn.Web/packages.lock.json, src/Ewn.Web.App/packages.lock.json, test/Ewn.Web.App.Test/packages.lock.json, test/Ewn.Web.Test/packages.lock.json, test/Ewn.Security.Test/packages.lock.json, test/Ewn.Model.Test/packages.lock.json, src/Ewn.Net.Autofac/packages.lock.json, test/Ewn.Net.Autofac.Test/packages.lock.json, src/Ewn.Net/packages.lock.json, test/Ewn.Net.Test/packages.lock.json, src/Ewn.Resources/packages.lock.json, test/Ewn.Resources.Test/packages.lock.json, src/Ewn.Scheduler/packages.lock.json, test/Ewn.Scheduler.Test/packages.lock.json",
      "stderr": "  Determining projects to restore...\n/opt/containerbase/tools/dotnet/sdk/8.0.301/NuGet.targets(694,5): error : PackageSourceMapping is enabled and there are multiple package sources associated with the same key(s): nuget.org. Path: /tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config [/tmp/renovate/repos/github/**redacted**/build/Build.csproj]\n"
    }
  ]
}

DEBUG: Getting comments for #1706 (branch="renovate/fluid.core-2.x")
DEBUG: http cache: saving https://api.github.com/repos/**redacted**/issues/1706/comments?per_page=100 (etag="b3e3a71f4b6963d18f0cd56854db9de2d9aba19847a54abe5e87319ad504459e", lastModified=undefined) (branch="renovate/fluid.core-2.x")
DEBUG: Found 0 comments (branch="renovate/fluid.core-2.x")
DEBUG: Ensuring comment "⚠️ Artifact update problem" in #1706 (branch="renovate/fluid.core-2.x")
INFO: Comment added (branch="renovate/fluid.core-2.x")
{
  "issueNo": 1706
  "topic": "⚠️ Artifact update problem"
}

DEBUG: setBranchCommit() (branch="renovate/fluid.core-2.x")

[rarkins]

The error is thrown from nuget itself and not directly by Renovate.

I suggest searching for the term "PackageSourceMapping is enabled and there are multiple package sources associated with the same key(s)" to work out why it's an error and then we might be able to work out if there's anything different which Renovate can do to avoid it

[gtbuchanan]

I'm aware of why the error would occur, but I don't know why it would be occurring in this instance since my NuGet.Config is well-formed and NuGet does not report the error when run directly on the project. I assume that the cache file generated by Renovate (/tmp/renovate/cache/__renovate-private-cache/nuget/nuget.config) ends up having multiple packageSourceMapping entries for nuget.org when only one should be defined.

[gtbuchanan]

I found the bit of code that generates the cached NuGet.Config and it does look like that is the case. getDefaultRegistries (which contains nuget.org) is used as the base array and then all of the sources from the repository-level NuGet.Config are added on top of that, so us having nuget.org already defined in our config causes createNuGetConfigXml to generate duplicate packageSource/add[key=nuget.org] nodes.

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

To get started, please read our guide on creating a minimal reproduction.

Good luck,

The Renovate team