Updating Renovate comments with new Alpine versions

[syphernl]

Renovate discovered an update of FROM alpine:3.13 to FROM alpine:3.14 in one of our Dockerfiles. Great!
However, for all specific packages we are installing manually we have a Renovate comment like this one:

# renovate: datasource=repology depName=alpine_3_13/bash versioning=loose
ENV BASH_VERSION="5.1.0-r0"

These comments are being handled by a regex manager:

    {
      description: 'Update VERSION variables in Dockerfiles',
      fileMatch: ['^Dockerfile$'],
      matchStrings: [
        '# renovate: datasource=(?<datasource>[a-z-]+?) depName=(?<depName>.+?)(?: versioning=(?<versioning>[a-z-]+?))?\\s(?:ENV|ARG) .+?_VERSION="(?<currentValue>.+?)"\\s',
        '# renovate: datasource=(?<datasource>[a-z-]+?) depName=(?<depName>.+?)(?: versioning=(?<versioning>[a-z-]+?))?\\s(?:ENV|ARG) VERSION="(?<currentValue>.+?)"\\s',
      ],
      versioningTemplate: '{{#if versioning}}{{versioning}}{{else}}semver{{/if}}',
    },

Separately they would properly upgrade these packages, but not when Alpine switches versions.

Is there a way for Renovate to update the alpine_3_13 depName part to alpine_3_14 when it also upgrades the container source?
Because now our CI is failing on building the Dockerfile because these particular versions do not exist within Alpine 3.14 and need to be updated by hand. Which is something we'd like to prevent.

[viceice]

Maybe you can do it with another regex manager, regex versioningTemplate and extractVersionTemplate

[icereed]

Hi @syphernl, did you find a workaround or process to deal with this?

[syphernl]

Hi @icereed Unfortunately I haven't found a workaround or process.
"Luckily" Alpine doesn't often bump releases so the problem is fairly limited, yet still on my wishlist to fix.

[elchenberg]

I tried using versioningTemplate and extractVersionTemplate (as @viceice suggested) but I could not make it work.

This is the line I want to update in a Dockerfile (from 3_16 to 3_17):

# renovate: datasource=repology depName=alpine_3_16/iptables versioning=loose

And my regexManager looks like this:

{
  "regexManagers": [
    {
      "matchStrings": [
        "# renovate: datasource=repology depName=alpine_(?<currentValue>\\d+_\\d+)\\/"
      ],
      "fileMatch": [
        "^Dockerfile$"
      ],
      "datasourceTemplate": "docker",
      "depNameTemplate": "alpine",
      "versioningTemplate": "regex:^(?<major>\\d+)_(?<minor>\\d+)$",
      "extractVersionTemplate": "^(?<major>\\d+).(?<minor>\\d+).(?<patch>\\d+)$"
    }
  ]
}

My assumption is that versioningTemplate should extract the major and the minor version from the currentValue (3_16) and extractVersionTemplate will be used on the tags of the alpine image. Am I mistaken? Is there an error in my config?

PS. I tried to use docker.io/library/alpine as depNameTemplate and tried extractVersionTemplate without the patch version but this did not work either.

[elchenberg]

I think I found a solution.

Dockerfile:

FROM docker.io/library/alpine:3.16.5
# renovate: datasource=repology depName=alpine_3_16/iptables versioning=loose
ENV IPTABLES_VERSION=1.8.8-r1
RUN apk add "iptables=${IPTABLES_VERSION:?}"

renovate.json:

{
    "regexManagers": [
        {
            "fileMatch": [
                "Dockerfile"
            ],
            "matchStrings": [
                "# renovate: datasource=repology depName=alpine_(?<currentValue>\\d+[._]\\d+)"
            ],
            "currentValueTemplate": "{{{ replace '_' '.' currentValue }}}",
            "datasourceTemplate": "docker",
            "depNameTemplate": "docker.io/library/alpine",
            "versioningTemplate": "regex:^(?<major>\\d+)[._](?<minor>\\d+)$",
            "autoReplaceStringTemplate": "# renovate: datasource=repology depName=alpine_{{{newMajor}}}_{{{newMinor}}}"
        }
    ]
}

Diff:

diff --git a/Dockerfile b/Dockerfile
index 58acba9..7053596 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,4 +1,4 @@
-FROM docker.io/library/alpine:3.16.5
-# renovate: datasource=repology depName=alpine_3_16/iptables versioning=loose
+FROM docker.io/library/alpine:3.17.3
+# renovate: datasource=repology depName=alpine_3_17/iptables versioning=loose
 ENV IPTABLES_VERSION=1.8.8-r1
 RUN apk add "iptables=${IPTABLES_VERSION:?}"

[viceice]

can you add that sample to the regex manager docs please?

[syphernl]

Awesome, this indeed updates the depName=alpine_* to the new version. 🎉

However... this leaves the version for the package itself on the old version resulting in a failed Docker build if a build is attempted (e.g. in the MR pipeline).
Is there a practical way to combine this change with the repology datasource changes so that they both change at the same time in the same MR?

[elchenberg]

can you add that sample to the regex manager docs please?

Sure. But I do not know when I find the time to do that so I wouldn't mind if someone would does it. 🙂

Is there a practical way to combine this change with the repology datasource changes so that they both change at the same time in the same MR?

@syphernl If you use a self-hosted bot, you could maybe write a postUpgradeTask to do it. But I do not know if you would consider this a "practical way".

[romainjrd]

Hi,

Using the renovate.json @elchenberg provided, i tried updating Renovate's comment and its ENV variable.

When grouping updates in the same Merge Request, I have a bug. The update for my ENV variable is written in the MR body but no changes are written in the Dockerfile.

MR body :

MR changes :

I think there is a conflict after updating the comment, Renovate can't find anymore the matchStrings concerning the ENV variable. It only happens when I try to group updates by Dockerfile.

renovate.json :

 {
    "enabledManagers": ["regex", "dockerfile"],
    "additionalBranchPrefix": "{{baseDir}}-",
    "packageRules": [
        {
            "matchManagers": ["dockerfile", "regex"],
            "enabled": true,
            "groupName": "{{baseDir}}-dockerfile",
            "groupSlug": "dockerfile"
        }
    ],
    "customManagers": [
      {
        "customType": "regex",
        "fileMatch": ["^Dockerfile$"],
        "matchStrings": [
          "# renovate: datasource=repology depName=alpine_(?<currentValue>\\d+[._]\\d+)"
        ],
        "currentValueTemplate": "{{{ replace '_' '.' currentValue }}}",
        "datasourceTemplate": "docker",
        "depNameTemplate": "alpine",
        "versioningTemplate": "regex:^(?<major>\\d+)[._](?<minor>\\d+)$",
        "autoReplaceStringTemplate": "# renovate: datasource=repology depName=alpine_{{{newMajor}}}_{{{newMinor}}}"
      },
      {
        "customType": "regex",
        "fileMatch": ["^Dockerfile$"],
        "matchStrings": [
          "datasource=(?<datasource>repology) depName=(?<depName>.*?)( versioning=(?<versioning>.*?))?\\sENV .*?_VERSION=(?<currentValue>.*)\\s"
        ]
      }
    ]
  }

Dockerfile :

# renovate: datasource=repology depName=alpine_3_18/firefox-esr versioning=loose
ENV FIREFOX_ESR_VERSION=115.6.0-r0

Does anyone had the same issue ? Did I missed something in the configuration ?

Any help would be appreciated 🙂

Thanks !

[rarkins]

When alpine switches versions, what does the latest and second-latest release look like? e.g. does it switch from alpine_3_13/bash to alpine_3_14/bash and version remains as 5.1.0-r0, or does the version increment too?

[rarkins]

It's a bit tough as you're essentially switching both the depName and version at the same time. If you update either separately then it's not a valid combination.

[syphernl]

Yeah, either way the builds will fail. I also figured out that some packages can be missing in newer versions and in some cases the packages could be renamed.. Although in most cases it might work.

But does the "a bit tough" mean it's impossible or is it something that can be achieved?

[rarkins]

I was meaning it's a bit tough to think of even how to do this in a new feature

[syphernl]

Ah, understood!

[reitzig]

Spitballing:

• With named custom managers (which are coming, right?) we could group updates based on those names.
• After creating a patch for a group of changes, iterate with the same managers until the patch converges.

In the example, first iteration would bump the base image tag and depNames in renovate comments (two managers, grouped by name/label/whatever); second iteration would bump the versions in renovate comments (manager #3, same group).

This would open the potential of infinite loops and might double runtime of lookup overall, but if it could be restricted to "good" cases ... 🤔

Not sure if that's more or less ugly compared to expanding the docker manager to effectively make the all but canonical comments into first-class citizens.