Challenging to configure mosquitto password_file file permissions correctly on Kubernetes

[jfly]

I'm deploying mosquitto to my Kubernetes cluster. I've got my configuration in a ConfigMap (which works fine), which has a password_file /mosquitto/secret/passwords directive in it. /mosquitto/secret is a Volume mounted from a Secret, which means that by default, it is mounted read-only (0644) in my container owned by root:root.

When mosquitto starts up, I see a number of errors and warnings:

chown: /mosquitto/config/..2024_03_12_07_21_00.613021741/mosquitto.conf: Read-only file system
chown: /mosquitto/config/..2024_03_12_07_21_00.613021741: Read-only file system
chown: /mosquitto/config/..2024_03_12_07_21_00.613021741: Read-only file system
chown: /mosquitto/config/..data: Read-only file system
chown: /mosquitto/config/mosquitto.conf: Read-only file system
chown: /mosquitto/config: Read-only file system
chown: /mosquitto/config: Read-only file system
chown: /mosquitto/secret/passwords: Read-only file system
chown: /mosquitto/secret/..data: Read-only file system
chown: /mosquitto/secret/..2024_03_12_07_21_00.3359112445/passwords: Read-only file system
chown: /mosquitto/secret/..2024_03_12_07_21_00.3359112445: Read-only file system
chown: /mosquitto/secret/..2024_03_12_07_21_00.3359112445: Read-only file system
chown: /mosquitto/secret: Read-only file system
chown: /mosquitto/secret: Read-only file system
1710228061: mosquitto version 2.0.18 starting
1710228061: Config loaded from /mosquitto/config/mosquitto.conf.
1710228061: Warning: File /mosquitto/secret/passwords has world readable permissions. Future versions will refuse to load this file.
To fix this, use `chmod 0700 /mosquitto/secret/passwords`.
1710228061: Warning: File /mosquitto/secret/passwords owner is not mosquitto. Future versions will refuse to load this file.To fix this, use `chown mosquitto /mosquitto/secret/passwords`.
1710228061: Warning: File /mosquitto/secret/passwords group is not mosquitto. Future versions will refuse to load this file.
1710228061: Opening ipv4 listen socket on port 1883.
1710228061: Opening ipv6 listen socket on port 1883.
1710228061: mosquitto version 2.0.18 running

Analysis

Various chown: /mosquitto/...: Read-only file system errors

All the chown errors are happening due to this call to chown -R in docker-entrypoint.sh. I can workaround that by telling Kubernetes to start my pod with user and group 1883 (ids copied from the Dockerfile) using the pod security context's runAsUser and runAsGroup as documented here: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod.

Warning: File /mosquitto/secret/passwords has world readable permissions

I can fix this by specifying defaultMode for my pod's volume secret as documented here: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#secretvolumesource-v1-core.

Warning: File /mosquitto/secret/passwords group is not mosquitto

I can fix this using the pod security context's fsGroup as documented here: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod.

Warning: File /mosquitto/secret/passwords owner is not mosquitto

I believe the only way to fix this is with an init container that copies this file and changes the ownership as described here. This is tedious, and doesn't provide any security benefit.

Conclusion

There are a lot of ways we could fix/workaround this (having an option to specify the user/group to run as is the first one that comes to mind). I totally understand if you consider this to be a Kubernetes limitation, and not something you want to work around.

Either way, I do think it would be useful to document "how to run Mosquitto in Kubernetes" somewhere. This took me quite some time to read through and figure out.

[ralight]

There are some good points here, thank you.

The chown was introduced because of earlier problems people had had with file permissions, however I think it should be restricted to the data directory only.

2.1 will be able to specify the user/group with the PUID/PGID environment variables, which should help.

The remaining warnings about world accessible file permissions and files owned by users other than the user that the broker is running as come out of a security audit quite rightly pointing out that secrets should not be modifiable by other users. This may have limited benefit in a k8s environment, but there are lots of people running outside of containers.

[jfly]

Awesome! Having the the PUID and PGID pattern should solve my troubles so long I can just get them to root/root in a k8s environment =)

The remaining warnings about world accessible file permissions and files owned by users other than the user that the broker is running as [...]

Yeah, I agree those checks seem completely reasonable to me, and should be addressed by the PUID/PGID pattern.

I'll stay subscribed to this issue, please feel free to reach out to me if you'd like a beta tester on any upcoming changes.

[bbkz]

Thank you for opening this issue, i have the same problem and came here looking for a answer :-D

It would be nice if hardening owner is not mosquitto could wait until k8s has a solution.

Or PUID/PGID's implemented :-)