Feature request: support for environment variables in configuration files

[jmhostalet]

Given a configuration files, for example:

/etc/mosquitto/conf.d/my-bridge.conf

it would be nice to use environment variables, for example:

bridge_identity my-identity
bridge_psk ${MOSQUITTO_PSK}

This would help a lot to use the docker images without having to generate a custom image by extending the official one.
It would also help a lot in the use of kubernetes, making use of the resources of type "secret" and mounting them in the container itself as environment variables.

[ralight]

This seems like a good idea, I'm now wondering about what vulnerabilities this could introduce and how to mitigate them.

[sglux]

Don't know if this introduces weaknesses but maybe the requested "feature" can be turned on and of with a switch parameter? If the hypothetical switch --use-env is not specified nothing is changed to the interpretation of configuration files.

[onedr0p]

I'm now wondering about what vulnerabilities this could introduce and how to mitigate them.

What did you have in mind? If a bad actor has access to where mosquitto is running, them reading environment variables are the least of the operators worries and plus they'll be able to read the mosquitto config file anyways. Environment variables are a pretty common design pattern and follows the 12factor app.

[datenheim]

I'm now wondering about what vulnerabilities this could introduce and how to mitigate them.

Remembering all the apps that support environment variables (and are easily deployable as container) I wonder if that would introduce new attack vectors at all.

[beckerben]

Doesn't seem to be a issue for other brokers, my favorite is vernemq, easy to run as docker container because all settings can be configured from environment variables...also hive-ce supports environment variable tokens embedded in the file which is nice too. There is a version of mosquitto broker actually that someone made to support the variables, would be nice if this was baked into official build: https://github.com/efrecon/docker-mosquitto

[abiliojr]

I wanted this for docker, so I was half way into a prototype, when it struck me: wouldn't it be ok to use something like envsubst for the docker image. Isn't it good enough, while keeping mosquitto as simple as possible?

[beckerben]

@abiliojr that is a good solution for building an image, however the issue is once you have an image built and now want to use it and change the configuration for each instance of a container where the common image is used, this method doesn't work.

[abiliojr]

@beckerben, that is true. But my idea was to have it as part of the docker container itself, running before mosquitto runs (so mosquitto consumes it's output).

[beckerben]

@abiliojr interesting, would the base image of the image be a linux distro like alpine you install mosquitto on top of and use a script in the image to build the configuration and then start mosquitto or is there another method?

[onedr0p]

I tried giving envsubst a try with https://github.com/drone/envsubst and came to notice that this will not work with Mosquitto. Mosquitto doesn't differentiate between a variable that's defined or empty. It assumes if the variable is defined and empty it wants to use it and fails to start.

Anyways here some of my work.

Bash script to generate env based config file

#!/usr/bin/env bash

# ./hack/conf2env.sh | sort -n | uniq -u > mosquitto.conf.tmpl

test -t 0 && exec < <(curl -sL https://raw.githubusercontent.com/eclipse/mosquitto/v2.0.11/mosquitto.conf)
while IFS="" read -r line || [ -n "$line" ]
do
    # Skip lines not variables
    if [[ $line =~ ^\#[[:space:]] || $line =~ ^\#$ ]]; then
        continue
    fi
    # Skip lines that are empty
    [ -z "$line" ] && continue
    # Remove comment character
    line="${line:1}"
    # Handle variables
    variable=$(echo "${line}" | awk -F" " '{print $1}')
    uppercase_variable=$(echo "${variable}" | awk '{print toupper($0)}')
    # Replace any non alphanumeric character with underscores
    uppercase_variable="${uppercase_variable//[^[:alnum:]]/_}"
    value=$(echo "${line}" | awk -F" " '{print $2}')
    # Only use informatiom log_type
    if [[ $variable == "log_type" ]]; then
        if [[ "$value" =~ ^(notice|warning|error)$ ]]; then
            continue
        fi
    fi
    # Variables that do not have a clear default and instead some helper
    if [[ $value == \<* ]]; then
        value=""
    fi
    if [[ -z $value ]]; then
        value=""
    fi
    result="${variable} \${MOSQUITTO_${uppercase_variable}:-${value}}"
    printf '%s\n' "${result}"
done

This generates a files like:

acl_file ${MOSQUITTO_ACL_FILE:-}
address ${MOSQUITTO_ADDRESS:-}
allow_anonymous ${MOSQUITTO_ALLOW_ANONYMOUS:-false}
allow_zero_length_clientid ${MOSQUITTO_ALLOW_ZERO_LENGTH_CLIENTID:-true}
auth_plugin ${MOSQUITTO_AUTH_PLUGIN:-}

Error and debugging

❯ docker run -it --rm mosquitto-test
Mosquitto starting with the following configuration...
acl_file 
address 
allow_anonymous false
allow_zero_length_clientid true
auth_plugin 
auto_id_prefix auto-
autosave_interval 1800
autosave_on_changes false
bind_interface 
bridge_alpn 
bridge_attempt_unsubscribe true
bridge_bind_address 
bridge_cafile 
bridge_capath 
bridge_certfile 
bridge_identity 
bridge_insecure false
bridge_keyfile 
bridge_max_packet_size 0
bridge_outgoing_retain true
bridge_protocol_version mqttv311
bridge_psk 
cafile 
capath 
certfile 
check_retain_source true
ciphers_tls1.3 
cleansession false
clientid_prefixes 
connection 
connection_messages true
crlfile 
dhparamfile 
http_dir 
idle_timeout 60
include_dir 
keepalive_interval 60
keyfile 
listener 
local_clientid 
log_dest stderr
log_facility 
log_timestamp true
log_timestamp_format 
log_type information
max_connections -1
max_inflight_bytes 0
max_inflight_messages 20
max_keepalive 65535
max_packet_size 0
max_qos 2
max_queued_bytes 0
max_queued_messages 1000
memory_limit 0
message_size_limit 0
mount_point 
notification_topic 
notifications true
password_file 
per_listener_settings false
persistence true
persistence_file mosquitto.db
persistence_location /config
persistent_client_expiration 
pid_file 
protocol mqtt
psk_file 
psk_hint 
queue_qos0_messages false
remote_clientid 
remote_password 
remote_username 
require_certificate false
restart_timeout 5
retain_available true
round_robin false
set_tcp_nodelay false
socket_domain 
start_type automatic
sys_interval 10
threshold 10
topic 
try_private true
upgrade_outgoing_qos false
use_username_as_clientid 
user mosquitto
websockets_headers_size 
websockets_log_level 0
1629931283: Error: Empty acl_file value in configuration.
1629931283: Error found at /config/mosquitto.conf:1.