Auth token probably not included in the request

[aru-trackunit]

What would you like help with?

I think I found a bug

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Local

Please tell us more about your question or problem

Hi all,

I am struggling to authenticate into Databricks using token - Status Code 401. They use bearer tokens so constructing http request is easy.

Following curl command works well and returns status code 200
curl -X GET https://workspace_id.databricks.com/api/2.0/clusters/spark-versions -H 'authorization: Bearer AUTH_TOKEN' -H 'accept: application/json'

Installation: npm install -g [email protected]
Execution: LOG_LEVEL=debug renovate --platform=local

Regex to be found

# renovate:
DBR_VERSION: "12.2.x-scala2.12"

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "customDatasources": {
    "databricksSparkVersions": {
      "defaultRegistryUrlTemplate": "https://workspace_id.cloud.databricks.com/api/2.0/clusters/spark-versions",
      "format": "json",
      "transformTemplates": ["{\"releases\": $filter(versions, function($v) { $contains($v.key, \"x-scala\") }).{\"version\": key}}"]
    }
  },
  "customManagers": [
    {
      "customType": "regex",
      "datasourceTemplate": "custom.databricksSparkVersions",
      "fileMatch": ["dags/custom_fields/test"],
      "matchStrings": [
        "#\\s*renovate:\\s*DBR_VERSION:\\s*\"(?<major>\\d+)\\.(?<minor>\\d+)\\.x-(?<suffix>[\\w\\d\\.]+)\""
      ],
      "currentValueTemplate": "{{major}}.{{minor}}.x-{{suffix}}",
      "depNameTemplate": "test",
      "versioningTemplate": "regex:^(?<major>\\d+)\\.(?<minor>\\d+)\\.x-(?<compatibility>.*)$"
    }
  ],
  "enabledManagers": [
    "custom.regex"
  ],
  "hostRules": [{
    "hostType": "custom",
    "matchHost": "https://workspace_id.cloud.databricks.com",
    "token": "AUTH_TOKEN",
    "abortOnError": true
  }],
  "labels": [
    "dependencies"
  ],
  "prHourlyLimit": 5,
  "prConcurrentLimit": 5,
  "separateMajorMinor": true,
  "separateMultipleMajor": true
}

Logs

DEBUG: Using RE2 regex engine
DEBUG: Parsing configs
DEBUG: Checking for config file in config.js
(node:36772) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
DEBUG: No config file found on disk - skipping
DEBUG: File config
       "config": {}
DEBUG: CLI config
       "config": {"platform": "local"}
DEBUG: Env config
       "config": {"hostRules": []}
DEBUG: Combined config
       "config": {"hostRules": [], "platform": "local"}
DEBUG: Enabling forkProcessing while in non-autodiscover mode
DEBUG: Found valid git version: 2.39.3
DEBUG: Setting global hostRules
DEBUG: Using baseDir: /var/folders/9d/7r5ttgmd29vg0m8frvvcy35m0000gn/T/renovate
DEBUG: Using cacheDir: /var/folders/9d/7r5ttgmd29vg0m8frvvcy35m0000gn/T/renovate/cache
DEBUG: Using containerbaseDir: /var/folders/9d/7r5ttgmd29vg0m8frvvcy35m0000gn/T/renovate/cache/containerbase
DEBUG: Initializing Renovate internal cache into /var/folders/9d/7r5ttgmd29vg0m8frvvcy35m0000gn/T/renovate/cache/renovate/renovate-cache-v1
DEBUG: Commits limit = null
DEBUG: Setting global hostRules
DEBUG: validatePresets()
DEBUG: Reinitializing hostRules for repo
DEBUG: Clearing hostRules
 INFO: Repository started (repository=local)
       "renovateVersion": "37.279.4"
DEBUG: Using localDir: /Users/user/renovate-test (repository=local)
DEBUG: PackageFiles.clear() - Package files deleted (repository=local)
DEBUG: Resetting npmrc (repository=local)
DEBUG: Resetting npmrc (repository=local)
DEBUG: checkOnboarding() (repository=local)
DEBUG: isOnboarded() (repository=local)
DEBUG: findFile(renovate.json) (repository=local)
fatal: not a git repository (or any of the parent directories): .git
DEBUG: Could not get file list using git, using glob instead (repository=local)
DEBUG: Config file exists, fileName: renovate.json (repository=local)
DEBUG: Repo is onboarded (repository=local)
fatal: not a git repository (or any of the parent directories): .git
DEBUG: Could not get file list using git, using glob instead (repository=local)
DEBUG: Found renovate.json config file (repository=local)
DEBUG: Repository config (repository=local)
       "fileName": "renovate.json",
       "config": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "customDatasources": {
           "databricksSparkVersions": {
             "defaultRegistryUrlTemplate": "https://workspace_id.cloud.databricks.com/api/2.0/clusters/spark-versions",
             "format": "json",
             "transformTemplates": [
               "{\"releases\": $filter(versions, function($v) { $contains($v.key, \"x-scala\") }).{\"version\": key}}"
             ]
           }
         },
         "customManagers": [
           {
             "customType": "regex",
             "datasourceTemplate": "custom.databricksSparkVersions",
             "fileMatch": ["dags/custom_fields/test"],
             "matchStrings": [
               "#\\s*renovate:\\s*DBR_VERSION:\\s*\"(?<major>\\d+)\\.(?<minor>\\d+)\\.x-(?<suffix>[\\w\\d\\.]+)\""
             ],
             "currentValueTemplate": "{{major}}.{{minor}}.x-{{suffix}}",
             "depNameTemplate": "test",
             "versioningTemplate": "regex:^(?<major>\\d+)\\.(?<minor>\\d+)\\.x-(?<compatibility>.*)$"
           }
         ],
         "enabledManagers": ["custom.regex"],
         "hostRules": [
           {
             "hostType": "custom",
             "matchHost": "https://workspace_id.cloud.databricks.com",
             "token": "***********",
             "abortOnError": true,
             "enableHttp2": true
           }
         ],
         "labels": ["dependencies"],
         "prHourlyLimit": 5,
         "prConcurrentLimit": 5,
         "separateMajorMinor": true,
         "separateMultipleMajor": true
       }
DEBUG: migrateAndValidate() (repository=local)
DEBUG: No config migration necessary (repository=local)
DEBUG: Setting hostRules from config (repository=local)
DEBUG: Adding token authentication for https://workspace_id.cloud.databricks.com (hostType=custom) to hostRules (repository=local)
DEBUG: Found repo ignorePaths (repository=local)
       "ignorePaths": ["**/node_modules/**", "**/bower_components/**"]
DEBUG: No vulnerability alerts found (repository=local)
DEBUG: No baseBranches (repository=local)
DEBUG: extract() (repository=local)
fatal: not a git repository (or any of the parent directories): .git
DEBUG: Could not get file list using git, using glob instead (repository=local)
DEBUG: Using file match: dags/custom_fields/test for manager regex (repository=local)
DEBUG: Matched 1 file(s) for manager regex: dags/custom_fields/test (repository=local)
DEBUG: manager extract durations (ms) (repository=local)
       "managers": {"regex": 1}
DEBUG: Found regex package files (repository=local)
DEBUG: Found 1 package file(s) (repository=local)
 INFO: Dependency extraction complete (repository=local)
       "stats": {
         "managers": {"regex": {"fileCount": 1, "depCount": 1}},
         "total": {"fileCount": 1, "depCount": 1}
       }
DEBUG: hostRules: applying Bearer authentication for workspace_id.cloud.databricks.com (repository=local)
DEBUG: GET https://workspace_id.cloud.databricks.com/api/2.0/clusters/spark-versions = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=401 retryCount=0, duration=420) (repository=local)
DEBUG: Failed to look up custom.databricksSparkVersions package test (repository=local, packageFile=dags/custom_fields/test, dependency=test)
DEBUG: PackageFiles.add() - Package file saved for base branch (repository=local)
DEBUG: Package releases lookups complete (repository=local)
DEBUG: branchifyUpgrades (repository=local)
DEBUG: detectSemanticCommits() (repository=local)
DEBUG: getCommitMessages (repository=local)
DEBUG: semanticCommits: detected "unknown" (repository=local)
DEBUG: semanticCommits: disabled (repository=local)
DEBUG: 0 flattened updates found:  (repository=local)
DEBUG: Returning 0 branch(es) (repository=local)
DEBUG: config.repoIsOnboarded=true (repository=local)
DEBUG: packageFiles with updates (repository=local)
       "config": {
         "regex": [
           {
             "deps": [
               {
                 "depName": "test",
                 "currentValue": "12.2.x-scala2.12",
                 "datasource": "custom.databricksSparkVersions",
                 "versioning": "regex:^(?<major>\\d+)\\.(?<minor>\\d+)\\.x-(?<compatibility>.*)$",
                 "replaceString": "# renovate:\nDBR_VERSION: \"12.2.x-scala2.12\"",
                 "updates": [],
                 "packageName": "test",
                 "warnings": [
                   {
                     "topic": "test",
                     "message": "Failed to look up custom.databricksSparkVersions package test"
                   }
                 ]
               }
             ],
             "matchStrings": [
               "#\\s*renovate:\\s*DBR_VERSION:\\s*\"(?<major>\\d+)\\.(?<minor>\\d+)\\.x-(?<suffix>[\\w\\d\\.]+)\""
             ],
             "depNameTemplate": "test",
             "currentValueTemplate": "{{major}}.{{minor}}.x-{{suffix}}",
             "datasourceTemplate": "custom.databricksSparkVersions",
             "versioningTemplate": "regex:^(?<major>\\d+)\\.(?<minor>\\d+)\\.x-(?<compatibility>.*)$",
             "packageFile": "dags/custom_fields/test"
           }
         ]
       }
DEBUG: detectSemanticCommits() (repository=local)
DEBUG: semanticCommits: returning "disabled" from cache (repository=local)
DEBUG: Repository timing splits (milliseconds) (repository=local)
       "splits": {"init": 89, "extract": 28, "lookup": 746},
       "total": 864
DEBUG: Package cache statistics (repository=local)
       "get": {"count": 0, "avgMs": 0, "medianMs": 0, "maxMs": 0, "totalMs": 0},
       "set": {"count": 0, "avgMs": 0, "medianMs": 0, "maxMs": 0, "totalMs": 0}
DEBUG: HTTP statistics (repository=local)
       "urls": {
         "https://workspace_id.cloud.databricks.com/api/2.0/clusters/spark-versions": {
           "GET": {"401": 1}
         }
       },
       "hosts": {
         "workspace_id.cloud.databricks.com": {
           "count": 1,
           "reqAvgMs": 420,
           "reqMedianMs": 420,
           "reqMaxMs": 420,
           "queueAvgMs": 1,
           "queueMedianMs": 1,
           "queueMaxMs": 1
         }
       },
       "requests": 1
DEBUG: HTTP cache statistics (repository=local)
DEBUG: Lookup statistics (repository=local)
       "custom.databricksSparkVersions": {"count": 1, "avgMs": 742, "medianMs": 742, "maxMs": 742, "totalMs": 742}
DEBUG: dns cache (repository=local)
       "hosts": []
 INFO: Repository finished (repository=local)
       "cloned": undefined,
       "durationMs": 864
DEBUG: Checking file package cache for expired items
DEBUG: Deleted 0 of 0 file cached entries in 1ms

As a workaround I used allowedHeaders & headers setting, how can I encrypt the header?
Like below?

"hostRules": [{
    "hostType": "custom",
    "matchHost": "https://workspace_id.cloud.databricks.com",
    "encrypted": {
        "headers": {
          "Authorization": "Bearer AUTH_TOKEN"
        },
    },
    "abortOnError": true,
    "enableHttp2": true
  }],

Remarks:

1. It would be really nice to see entire request that is sent - because when reading logs I was unsure if headers are actually set (even using TRACE log level).

Probably related to: #26983 and #26625

Logs (if relevant)

No response

[rarkins]

Try removing the hostType from
your host rule

[aru-trackunit]

Come on - short living scope limited tokens as a proof of concept, it is to narrow down the problem instead of adding encryption complexity that could potentially fail

[rarkins]

I asked a simple question and did not expect rudeness in response

[rarkins]

Please create a minimal reproduction (instructions below) with a fake token. We can then see if the token is passed or not using debugging

[aru-trackunit]

Sorry for my impatience @rarkins , please don't take my comment personally. Thanks for a quick loop on the issue.

Will follow the instructions that are attached below.

[rarkins]

I asked you the question because:

• Clearly you had redacted the actual content of your config for the token
• I have seen dozens of people in the past unaware that their renovate.json can't read from env variables

I don't/didn't care if you're doing something moderately insecure for testing

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

To get started, please read our guide on creating a minimal reproduction.

Good luck,

The Renovate team

[aru-trackunit]

@rarkins minimal reproduction example. Please let me know what I can do more in case you are missing something.

https://github.com/aru-trackunit/renovate-token-minimal-reproduction

[rarkins]

A big thanks to you too for your patience/persistence. You were essentially right all along with your auth suspicions, but I was skeptical that such a bug could exist and we'd not realized before.

[aru-trackunit]

There is one more thing. When I encrypt the token then I get exactly the same issue - token prefix before auth token instead of Bearer

"hostRules": [{
    "matchHost": "https://workspace_id.cloud.databricks.com",
    "encrypted": {
      "token": "encrypted_token"
    }
  }],

Tested on: 37.288.0
Headers logged from the requestsreported

EnvironHeaders([('User-Agent', 'RenovateBot/37.288.0 (https://github.com/renovatebot/renovate)'), ('Accept', 'application/json'), ('Authorization', 'token AUTH_TOKEN'), ('Accept-Encoding', 'gzip, deflate, br'), ('Host', 'localhost:5000'), ('Connection', 'keep-alive')])
127.0.0.1 - - [12/Apr/2024 11:57:16] "GET / HTTP/1.1" 200 -

There is also more into it:
I debugged a lot before I found this issue: #27375 (comment)
It would be good if that "gotcha" could be read on mend website or somewhere in the docs, because gpg --full-generate-key does not work - I had to usegpg --full-generate-key --openpgp instead

[aru-trackunit]

Should I create another repository then? If yes, then how can I go about the organization using mend? Do public repos have organization assigned?

[rarkins]

Can you start a new discussion? It seems like a new topic.

[aru-trackunit]

Started a new discussion here: #28367