Feat: BuildArgs in custom docker builds

[lorenzomigliorero]

What problem will this feature address?

Currently, there's no way to specify build-args for custom dockerfile projects.
Example use-case: In case of a private npm registry, there's no way to declare an NPM_TOKEN build variable.

Describe the solution you'd like

I see three possible ways:

• Create another table under Environment Settings called Build Args
• Create another top-level menu item called Build Args close to Environment
• Specify which environments are also build-args

This is just pseudo-code, but the buildImage command should accept a buildargs parameter:

server/utils/builders/docker-file.ts

import type { WriteStream } from "node:fs";
import { docker } from "@/server/constants";
import * as tar from "tar-fs";
import type { ApplicationNested } from ".";
import { getBuildAppDirectory } from "../filesystem/directory";
import { createEnvFile } from "./utils";

export const buildCustomDocker = async (
	application: ApplicationNested,
	writeStream: WriteStream,
) => {
++	const { appName, buildargs, env } = application;
	const dockerFilePath = getBuildAppDirectory(application);
	try {
		const image = `${appName}`;
		const contextPath =
			dockerFilePath.substring(0, dockerFilePath.lastIndexOf("/") + 1) || ".";
		const tarStream = tar.pack(contextPath);

		createEnvFile(dockerFilePath, env);

		const stream = await docker.buildImage(tarStream, {
			t: image,
++			buildargs,
			dockerfile: dockerFilePath.substring(dockerFilePath.lastIndexOf("/") + 1),
		});

		await new Promise((resolve, reject) => {
			docker.modem.followProgress(
				stream,
				(err, res) => (err ? reject(err) : resolve(res)),
				(event) => {
					if (event.stream) {
						writeStream.write(event.stream);
					}
				},
			);
		});
	} catch (error) {
		throw error;
	}
};

Dockerode issue: apocas/dockerode#326
Engine API docs: https://docs.docker.com/engine/api/v1.24/#build-image-from-a-dockerfile

Describe alternatives you've considered

There are no other alternatives since the buildargs argument is the only param available.

Additional context

No response

[Siumauricio]

hi, @lorenzomigliorero I recently add support for enviroment variables in build time for Dockerfiles (Not released yet), is not args but it makes almost the same purpose to have variables in build time #227

[lorenzomigliorero]

Oh cool! I didn't see that issue. Is there a way to test the feature?

[Siumauricio]

Yes, you can install the canary version curl -sSL https://dokploy.com/canary.sh | sh I don't recommend since it can introduce some bugs or something but if you want to test it out would be awesome!

[lorenzomigliorero]

I ran the canary branch locally, but variables are unavailable at build time.

[lorenzomigliorero]

@Siumauricio , I did a quick test, and I can confirm that the only way is to use the buildArgs flag. Creating the env file during the DockerFile creation isn't enough.

const stream = await docker.buildImage(tarStream, {
	t: image,
	buildargs: {
		foo: "bar" // this works
	},
	dockerfile: dockerFilePath.substring(dockerFilePath.lastIndexOf("/") + 1),
});

The easiest may be converting the env variable into a JSON: Eg: buildargs: convertEnvToJSON(env), however the Docker Engine API says: This is not meant for passing secret values.

So, I suggest creating another field or flagging which variables are also build-time (as Coolify does).

[Siumauricio]

@lorenzomigliorero can you share your dockerfile?

[lorenzomigliorero]

Sure, the NPM_TOKEN is needed as ARG for the yarn --production cmd.

FROM node:12-alpine as build

ARG NPM_TOKEN
ARG NODE_ENV=production

RUN env

WORKDIR /opt/
COPY package.json yarn.lock .npmrc ./
# Strapi requirements
RUN apk add --no-cache python2 make g++ vips-dev
RUN yarn --production

WORKDIR /opt/app
COPY . .
ENV PATH /opt/node_modules/.bin:$PATH
RUN yarn run build

FROM node:12-alpine

RUN apk add --no-cache vips-dev

ARG NPM_TOKEN
ENV NPM_TOKEN=$NPM_TOKEN

ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}

WORKDIR /opt/
COPY --from=build /opt/node_modules ./node_modules

ENV PATH /opt/node_modules/.bin:$PATH

WORKDIR /opt/app
COPY --from=build /opt/app .

EXPOSE 1337
CMD ["yarn", "start"]

[Siumauricio]

You have 2 options

1. Remove ARG and adapt to the ENV structure
2. Remove the ARG and ENV, and just assign the ENV values in the Enviroment Tab and it automatically will create a .env file which the dockerfile will pick, something like this..

FROM node:12-alpine as build

RUN env

WORKDIR /opt/
COPY package.json yarn.lock .npmrc ./
# Strapi requirements
RUN apk add --no-cache python2 make g++ vips-dev
RUN yarn --production

WORKDIR /opt/app
COPY . .
ENV PATH /opt/node_modules/.bin:$PATH
RUN yarn run build

FROM node:12-alpine

RUN apk add --no-cache vips-dev

WORKDIR /opt/
COPY --from=build /opt/node_modules ./node_modules

ENV PATH /opt/node_modules/.bin:$PATH

WORKDIR /opt/app
COPY --from=build /opt/app .

EXPOSE 1337
CMD ["yarn", "start"]

[lorenzomigliorero]

Both options can't work (removing the ARG FOO command won't solve).
Here is an example:

Environments

FOO=100
BAR=200
LOREM=300

Dockerfile

FROM node:12-alpine as build
ARG FOO
RUN env

Without buildArgs

NODE_VERSION=12.22.12
HOSTNAME=dbab621bb9ec
YARN_VERSION=1.22.18
SHLVL=1
HOME=/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/

With buildargs

const stream = await docker.buildImage(tarStream, {
	t: image,
	buildargs: {
		FOO: "bar"
	},
	dockerfile: dockerFilePath.substring(dockerFilePath.lastIndexOf("/") + 1),
});

NODE_VERSION=12.22.12
HOSTNAME=017e4d07ee41
YARN_VERSION=1.22.18
SHLVL=1
HOME=/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
FOO=bar
PWD=/

As described in the issue, I can work on a PR, but I need to know which option to choose.
I prefer the first one because it avoids duplication.

Curious to hear your thoughts on that!

[Siumauricio]

@lorenzomigliorero I understand, wouldn't it be easier and better to pass the enviroment variables and convert them to an object and pass them directly to the ARGS? so we don't have to create a new UI or new fields?

[lorenzomigliorero]

@Siumauricio It's easier for sure, but not better. Sharing all environment variables as build arguments in a Docker build command can have implications for both security and performance. We should share only the needed ones. I'd love to prepare a PR of the first screenshot above. I'm comfortable with the project stack, and I appreciate the project, so I want to contribute. What do you think?

[Siumauricio]

@lorenzomigliorero Thanks, if you have time would be awesome, use the second implementation you shared, and show only when we select the Dockerfile as BuildType

[lorenzomigliorero]

Sure thing, I'll let you know soon!