Unable to open config file (Docker Compose - RPi 4)

[IonicOwl]

Hi, I've been struggling with getting this to work. I'm using Docker Compose on a Pi 4 to run all of my home automation stuff, and I can't get Mosquitto to start properly. The container itself starts up, but looking at the container log, I see this:

1654374192: Error: Unable to open config file /mosquitto/config/mosquitto.conf.

Here is the mosquitto section of my compose.yml file:

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    volumes:
      - /opt/cgten/mosquitto:/mosquitto
    ports:
      - 1883:1883
      - 9001:9001

I created a folder structure for the config file before starting the container, and pre-populated the config file.

pi@cgten:/opt/cgten $ ls -la /opt/cgten/
total 24
drwxr-xr-x 5 root root 4096 Jun  5 14:51 .
drwxr-xr-x 4 root root 4096 Jun  4 13:22 ..
-rw-r--r-- 1 root root  770 Jun  5 14:31 compose.yaml
drwxr-xr-x 7 root root 4096 Jun  4 20:37 homeassistant
drwxr-xr-x 5 1883 1883 4096 Jun  4 21:28 mosquitto
drwxr-xr-x 8 root root 4096 Jun  5 14:32 portainer

pi@cgten:/opt/cgten $ ls -la /opt/cgten/mosquitto/
total 20
drwxr-xr-x 5 1883 1883 4096 Jun  4 21:28 .
drwxr-xr-x 5 root root 4096 Jun  5 14:51 ..
drwxr-xr-x 2 1883 1883 4096 Jun  5 14:39 config
drwxr-xr-x 2 1883 1883 4096 Jun  4 21:23 data
drwxr-xr-x 2 1883 1883 4096 Jun  4 21:23 log

pi@cgten:/opt/cgten $ ls -la /opt/cgten/mosquitto/config/
total 12
drwxr-xr-x 2 1883 1883 4096 Jun  5 14:39 .
drwxr-xr-x 5 1883 1883 4096 Jun  4 21:28 ..
-rw-r--r-- 1 1883 1883  202 Jun  5 14:39 mosquitto.conf

I should point out that the /data and /log directories were automatically created after starting the container for the first time.
The contents of the config file:

persistence true
persistence_location /mosquitto/data
log_dest file /mosquitto/log/mosquitto.log
listener 1883

## Authentication ##
#allow_anonymous false
#password_file /mosquitto/config/password.txt

(I've commented out the auth stuff for now as i want to set this up after. I don't think it matters though because it's not picking up this file anyway)

One of the first things I checked was whether the files were available from within the container. If I exec into the container, I can see that the config file is indeed there:

/ # ls -la mosquitto/config/
total 12
drwxr-xr-x    2 mosquitt mosquitt      4096 Jun  5 13:39 .
drwxr-xr-x    5 mosquitt mosquitt      4096 Jun  4 20:28 ..
-rw-r--r--    1 mosquitt mosquitt       202 Jun  5 13:39 mosquitto.conf
/ # 

Read permissions appear to be present and correct. If I cat the file, it's all in there:

/ # cat /mosquitto/config/mosquitto.conf 
persistence true
persistence_location /mosquitto/data
log_dest file /mosquitto/log/mosquitto.log
listener 1883

## Authentication ##
#allow_anonymous false
#password_file /mosquitto/config/password.txt
/ #

I'm at a bit of a loss here. I get the feeling that I've missed something, or perhaps there's a problem with permissions/ownership somewhere. If you look at the directory listings above, everything is owned by root (from me running everything as sudo), aside from the mosquitto directory, which is owned by 1883:1883 recursively.

I'm not sure if I needed to specify something in the compose file, but It's clearly already looking in the right place for the config file, going by the container log. Perhaps it's missing a volume declaration? I did specify /opt/cgten/mosquitto:/mosquitto in the compose file though, and it put everything else in there as expected, so I don't know.

Any help in the right direction would be massively appreciated.

**System Information**

Mosquitto Image Version:
2.0.14

Docker Version:

 Version:           20.10.16
 API version:       1.41
 Go version:        go1.17.10
 Git commit:        aa7e414
 Built:             Thu May 12 09:15:56 2022
 OS/Arch:           linux/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.16
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.10
  Git commit:       f756502
  Built:            Thu May 12 09:14:19 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.4
  GitCommit:        212e8b6fa2f44b9c21b2798135fc6fb7c53efc16
 runc:
  Version:          1.1.1
  GitCommit:        v1.1.1-0-g52de29d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker-Compose Version:

docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.9.2
OpenSSL version: OpenSSL 1.1.1n  15 Mar 2022

Kernel Version:
5.15.32-v8+

Distribution Version:

Distributor ID:	Debian
Description:	Debian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

[IonicOwl]

Ok I think I may have figured out the problem by RTFM properly.
Despite having the volume mapping in my compose file at the top level with ./mosquitto:/mosquitto, I still needed to explicitly map the config file as a separate volume, so my compose.yaml now looks like this:

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    volumes:
      - /opt/cgten/mosquitto:/mosquitto
      - /opt/cgten/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf
    ports:
      - 1883:1883
      - 9001:9001

Rebuilding the container with this new config results in no errors in the logs. I hope this is the last of my issues. Still a little perplexed as to why it didn't work as it was before... perhaps someone smarter than me can provide a reason.

I'll keep this open a little longer until I have successfully connected Zigbee2mqtt and Home Assistant to the broker, and feed back shortly.

[IonicOwl]

This appears to be similar to #2512
I've since tried connecting to the broker with Home Assistant and it refuses to connect. I've also tried connecting to the service on both 1883 and 9001 directly in my browser and it will not connect.

Looking further, I noticed that the volumes aren't mapping properly. If I exec into the container and check the files, everything is there as it should be

Inside mosquitto/log, I see:

total 32
drwxr-xr-x    2 mosquitt mosquitt      4096 Jun  4 20:29 .
drwxr-xr-x    5 mosquitt mosquitt      4096 Jun  4 20:28 ..
-rw-------    1 mosquitt mosquitt     17081 Jun  8 15:56 mosquitto.log

...but if I check inside the mosquitto/log directory on the host machine, it's empty. Same goes for the data directory. They both contain data in the container itself.

I've spent the last few hours playing around with volume mappings, permissions, and other things. There has to be a correct way of doing this.

[IonicOwl]

Still trying to get this to work.
Since my last post I've taken compose out of the equation and attempted to create the container both by pulling/running the container using Portainer using manually created volumes, and manually via command line, as per the instructions at https://hub.docker.com/_/eclipse-mosquitto

Error message returned in the command line is

docker: Error response from daemon: source /var/lib/docker/overlay2/e9c211431f6c732693274edd89d487232c957ec35eedbad2e8147eb4498d8938/merged/mosquitto/config/mosquitto.conf is not directory.

Am I doing something wrong or is this image broken? Can someone help? I've spent way too many hours on this, no other image I've run on this host has taken so much of my time and energy.

[lulol]

@IonicOwl I'm not using a RPi but found that had to use mosquitto.conf and a password file (and hash it) in the config folder to be able to connect, and got around this empty directory issue mapping each volume separately like:

 -v /home/lulol/docker/mosquitto/config:/mosquitto/config \
 -v /home/lulol/docker/mosquitto/data:/mosquitto/data \
 -v /home/lulol/docker/mosquitto/log:/mosquitto/log \

Amusingly the config file works with just the top level directory but not the the other two if the explicit mappings are removed despite the directories being recreated.

 -v /home/lulol/docker/mosquitto:/mosquitto \
 -v /home/lulol/docker/mosquitto/data:/mosquitto/data \
 -v /home/lulol/docker/mosquitto/log:/mosquitto/log \

Interestingly the permissions aren't the same, but doesn't seem that should be the cause of this issue.

$ ls -l
total 12
drwxrwxr-x 2 mosquitto mosquitto 4096 jun 13 17:27 config
drwxr-xr-x 2 mosquitto mosquitto 4096 jun 14 00:28 data
drwxr-xr-x 2 mosquitto mosquitto 4096 jun 13 12:47 log

And with config/mosquitto.conf

persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log
log_dest stderr
listener 1883
password_file /mosquitto/config/passwd
#acl_file /mosquitto/config/test.acl

[IonicOwl]

I've managed to get this working now, thanks to some help from someone on Reddit. Here is my working compose snippet:

  mosquitto:
    container_name: mosquitto
    hostname: mqtt.cgten
    image: eclipse-mosquitto
    user: 1000:1000
    volumes:
      - /opt/cgten/mosquitto/certs:/mosquitto/config/certs
      - /opt/cgten/mosquitto/config:/mosquitto/config
      - /opt/cgten/mosquitto/data:/mosquitto/data
      - /opt/cgten/mosquitto/log:/mosquitto/log
    ports:
      - "1883:1883"
      - "9001:9001"
    restart: unless-stopped

Not sure if it helped, but I set the file and directory ownership of the mosquitto volume to the pi user (1000:1000), and specified the user in the compose file.

I made an error in assuming that DNS would work between containers, so when trying to connect to the broker from my Home Assistant container via he hostname, it was failing to connect. Using the IP address of the container instead works for me. Strangely using the container name doesn't work like I'm lead to believe it should either. That's a problem for another day, not related to this.