Docker Error: Unable to open log file /mqtt/logs/mosquitto.log for writing.

[celeroll]

I have a problem with running eclipse image under docker. Particular with the access to the log folder.
Compose file:

version: '2.1'

services:

  mqtt:
    image: eclipse-mosquitto:latest
    container_name: "mqtt2"
    restart: always
    ports:
      - 1883:1883
      - 9001:9001
    volumes:
      - /volume1/apps/configs/mqtt/data:/mosquitto/data
      - /volume1/apps/configs/mqtt/config:/mosquitto/config:ro
      - /volume1/apps/configs/mqtt/logs:/mosquitto/log
    environment:
      - TZ=Europe/Berlin

And the docker logs:

1544689704: Error: Unable to open log file /mqtt/logs/mosquitto.log for writing.
1544689704: Error: Unable to open log file /mqtt/logs/mosquitto.log for writing.
1544689705: Error: Unable to open log file /mqtt/logs/mosquitto.log for writing.
1544689706: Error: Unable to open log file /mqtt/logs/mosquitto.log for writing.
1544689708: Error: Unable to open log file /mqtt/logs/mosquitto.log for writing.
1544689710: Error: Unable to open log file /mqtt/logs/mosquitto.log for writing.

This is folder permission:

drwx------+ 5 myUser users 4096 Dec 12 23:42 .
drwx------+ 9 myUser users 4096 Dec 12 21:12 ..
drwxrwxrwx  2 root users 4096 Dec 12 23:22 config
drwxrwxrwx  2 root users 4096 Dec 12 21:12 data
d---------+ 2 root root  4096 Dec 12 23:42 logs

I have also tried creating the mosquitto.log file ahead, but no success.
When I run the same command with

mqtt:
    image: toke/mosquitto:latest

everything is fine. But I would like to run eclipse image.

P.S. I dont have mosquitto user in my system added.

[sqldiablo]

I'm also seeing this issue after upgrading my docker container (using eclipse-mosquitto:latest). Running on Docker 18.09.0 CE and Rancher 1.6.21 on CentOS Linux 7 (3.10.0).

I have the following volumes mapped to my host as read-write:
/mosquitto/config
/mosquitto/data
/mosquitto/log

I've tried deleting the log file and starting the container, but it still throws the same error.

[Assar63]

Have a check on uid and gid mapping between host and container.
Read/Write access is not trivial if this do not match.

[ralight]

The uid that the image used for the "mosquitto" user had to be set to a fixed value. In previous versions it was set to the next available uid, which was almost certain to clash with an existing system uid, potentially causing permission problems. The uid:gid it now uses is fixed at 1883:1883, so you should be able to update your permissions to match. Apologies for the inconvenience.

[sqldiablo]

Thanks. I'm using Rancher and NFS volumes, so I had to chown 1883:1883 <volume-name> -R on the NFS server to make sure my permissions were correct. Once I did that, the upgraded container started up fine.

[nilathj]

I have the same issue. I have created a mosquitto user with uid and gid 1883.

id mosquitto
uid=1883(mosquitto) gid=1883(mosquitto) groups=1883(mosquitto)

cd /srv/mosquitto
chown -R 1883:1883 config/ data/ log/
ls -al
drwxrwx--- 2 mosquitt mosquitt 4096 Feb 16 14:49 config
drwxrwx--- 2 mosquitt mosquitt 4096 Feb 16 14:39 data
drwxrwx--- 2 mosquitt mosquitt 4096 Feb 17 20:35 log

as root user>
docker run -it -p 1883:1883 -p 9001:9001 -e PGID=1883 -e PUID=1883 -v /srv/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf -v /srv/mosquitto/data -v /srv/mosquitto/log eclipse-mosquitto

1550395157: Error: Unable to open log file /srv/mosquitto/log/mosquitto.log for writing.

I have tried with root 0:0 as well. But I get the same error. I have tried without the -e PGID and -e PUID flag, and I get the same error. I have tried giving the log file global read write permissions (777) and I get the same error. How do I get this running?

[jonahfang]

@nilathj Try:

cd log
sudo touch mosquitto.log
sudo chmod o+w ./mosquitto.log
sudo chown 1883:1883 /srv/mosquitto/log -R

[nilathj]

No same error:

/srv/mosquitto/log# ls -al
total 8
drwxrwx--- 2 mosquitt mosquitt 4096 Feb 20 19:57 .
drwxr-xr-x 5 1001 1001 4096 Feb 16 17:08 ..
-rw-r--rw- 1 mosquitt mosquitt 0 Feb 20 19:57 mosquitto.log

/srv/mosquitto/log# docker run -it -p 1883:1883 -p 9001:9001 -v /srv/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.
conf -v /srv/mosquitto/data -v /srv/mosquitto/log eclipse-mosquitto
1550653867: Error: Unable to open log file /srv/mosquitto/log/mosquitto.log for writing.

I'm running, Alpine Linux v3.9 as host OS. Docker version 18.09.1-ce. I am running docker as root.
2961 root 56:17 /usr/bin/dockerd -p /run/docker.pid

If I don't specify an external config, this container runs fine and I can connect to it using an external mqtt client.

[solick]

I have the same problem and the mentioned solution also did not work for me. Any further hints?

[proddy]

same problem here too.

1552421249: Saving in-memory database to /mosquitto/data/mosquitto.db.
1552421249: Error saving in-memory database, unable to open /mosquitto/data/mosquitto.db.new for writing.
1552421249: Error: Permission denied.

I'll try a few things as suggested in this thread

[harrzz]

for me adding the --user parameter fixed this issue.

[jtomasrl]

same problem

services:
  mqtt:
    image: eclipse-mosquitto
    container_name: mqtt
    user: 1883:1883
    environment:
      - PUID=1883
      - PGID=1883

[proddy]

it's a permissions thing. In my case I use

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    user: "1000:996"
    ports:
      - 1883:1883
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${USERDIR}/docker/mosquitto/config:/mosquitto/config:ro
      - ${USERDIR}/docker/mosquitto/data:/mosquitto/data
      - ${USERDIR}/docker/mosquitto/log:/mosquitto/log
    restart: always
    network_mode: host

and the docker group in /etc/group (id 996) has the user 'mosquitto'

[jtomasrl]

it's a permissions thing. In my case I use

  mosquitto:
    container_name: mosquitto
    image: eclipse-mosquitto
    user: "1000:996"
    ports:
      - 1883:1883
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${USERDIR}/docker/mosquitto/config:/mosquitto/config:ro
      - ${USERDIR}/docker/mosquitto/data:/mosquitto/data
      - ${USERDIR}/docker/mosquitto/log:/mosquitto/log
    restart: always
    network_mode: host

and the docker group in /etc/group (id 996) has the user 'mosquitto'

indeed, its a permissions issue. But even though I replicate your configuration and adding the user mosquitto to docker group, doesn't fix my problem. I've tried almost everything so far

[proddy]

@jtomasrl also make sure user (with id 1883 in your example) is also in the docker group and the permissions are set correctly on the file and directory (g+w)

[jtomasrl]

@proddy the user 1883 is "mosquitto"

This is my actual configuration and whatever I do, it doesn't work

id mosquitto
uid=1883(mosquitto) gid=1883(mosquitto) groups=1883(mosquitto),996(docker)

ls -l
-rwxrwxrwx 1 mosquitto mosquitto 2 May 5 11:47 mosquitto.log

cat docker-compose.yml

version: "3"

services:
  mqtt:
    image: eclipse-mosquitto
    container_name: mqtt
    user: "1883:996"

cat mosquitto.conf

persistence true
persistence_location /mqtt/data/
log_dest file /mqtt/log/mosquitto.log
user mosquitto

[proddy]

the log directory in your compose file is /mosquitto/log and in your .conf its /mqtt/log/. I think they should at least match.

Also I don't use the id of mosquitto (1883) but 1000 which is the owner of my docker

[jtomasrl]

I was just using a placeholder for the directory, but they match. I tried using mi id 1000 (owner of docker) without luck

[tlvlp]

@nilathj Try:

cd log
sudo touch mosquitto.log
sudo chmod o+w ./mosquitto.log
sudo chown 1883:1883 /srv/mosquitto/log -R

It finally worked for me using @nilathj 's approach.

The key was not to create a new mosquitto user on the host system, so the below search returns no results:

sudo cat /etc/passwd | grep 1883

Instead I just gave owner rights to 1883 UID/GID for the mqtt root folder on the host machine where all data, logs and configs are stored. And also gave rwx rights.

sudo chown -R 1883:1883 /opt/mqtt
sudo chmod -R 770 /opt/mqtt

So the result is:

drwxrwx---  5 1883 1883 4096 Jun 15 15:05 mqtt/

I assumed that a user / group has to exist on a system in order to be granted ownership but it seems that I have to research this further :)

[tomasznguyen]

I ran into the same problem myself today.

My setup is running mosquitto in docker on a Synology nas.

For mosquitto, I created the following mqqt-folders:

/volume1/docker/mosquitto/config /volume1/docker/mosquitto/data /volume1/docker/mosquitto/log

Starting the image fails with the error message that mosquitto is unable to open the log file for writing.

After running the following command:

sudo chown -R 1883:1883 /volume1/docker/mosquitto

and restarting mosquitto, everything works like a charm.

Note: I did not create a new user for mosquitto on the nas.

[mchuc]

use of directives:
-e GUID and -e GUID do not help much, there is an error that the package does not include it at all.
Each time files with group number and user ID 1883 are created

>/mosquitto/log# ls -l
total 4
-rwxrwxrwx 1 1883 1883 45 Oct 23 19:18 mosquitto.log

the only solution was:

mkdir -p '/volume2/docker-data/mosquitto/'
mkdir -p '/volume2/docker-data/mosquitto/config/'
mkdir -p '/volume2/docker-data/mosquitto/log'
mkdir -p '/volume2/docker-data/mosquitto/data'

next:
complete the file:
/volume2/docker-data/mosquitto/config/mosquitto.conf:

persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log

then run:

docker run -d \
--name = MOSQUITTO1 \
--restart = always \
-p 1883: 1883 \
-p 9001: 9001 \
///// -e PGID = 100 \ <- delete it, its my Moquitto local user GroupID: how to show in ssh?: > id mosquitto
//// - e PUID = 1031 \ <- delete its my Mosquitto local user UserID
////-e TZ = Europe/Warsaw \
-v /etc/localtime:/etc/localtime:ro \                       <- it's more universal than -e TZ...
-v '/ volume2/docker-data/mosquitto/config': '/mosquitto/config': ro \
-v '/ volume2/docker-data/mosquitto/log': '/mosquitto/log' \
-v '/ volume2/docker-data/mosquitto/data': '/mosquitto/data' \
eclipse-mosquitto: latest

next:

chmod a + rwx '/volume2/docker-data/mosquitto/data' -R
chmod a + rwx '/volume2/docker-data/mosquitto/log' -R

and a few minutes after the server starts again:

chmod a + rwx '/volume2/docker-data/mosquitto/data' -R
chmod a + rwx '/volume2/docker-data/mosquitto/log' -R

so that a normal user can read these files from Synology

so the mistake is that PID / GUID 1883 is always used maniacally, regardless of what I specified in the startup file

[LavI-0]

Tried all of the above. Still not working.

[crisycochea]

I've changed the ownership of the volume's directories and it works.
sudo chown -hR $MOSQUITO_USER $DIRECTORY
In my case:
sudo chown -hR mosquitto ./mosquitto

[bugs181]

I ran into the same problem myself today.

My setup is running mosquitto in docker on a Synology nas.

For mosquitto, I created the following mqqt-folders:

/volume1/docker/mosquitto/config /volume1/docker/mosquitto/data /volume1/docker/mosquitto/log

Starting the image fails with the error message that mosquitto is unable to open the log file for writing.

After running the following command:

sudo chown -R 1883:1883 /volume1/docker/mosquitto

and restarting mosquitto, everything works like a charm.

Note: I did not create a new user for mosquitto on the nas.

After trying several things from this thread, this is what worked for me.

[elchelun]

After a while I got it to work.
This are the steps that made the trick:
My yaml:

 mosquitto:
    image: eclipse-mosquitto
    hostname: mosquitto
    container_name: mosquitto
    ports:
      - 1883:1883
      - 9001:9001
    volumes:
      - /opt/appdata/mosquitto/config/mosquitto.conf:/mosquitto/config/mosquitto.conf:ro
      - /opt/appdata/mosquitto/data:/mosquitto/data
      - /opt/appdata/mosquitto/log:/mosquitto/log
    restart: always
    environment:
      - PUID=${PUID}
      - PGID=${PGID}

I use the same user/group for all the containers:

$ echo ${PUID}
1001

$ echo ${PGID}
1001

$ id 1001
uid=1001(dockeradmin) gid=1001(dockeradmin) groups=1001(dockeradmin)

My problem was permissions like most of the people in this tread, so I ran this command:

$ chmod -R 777 /opt/appdata/mosquitto/

After all that, restarted the container and no more errors.

[donnib]

I have exact same issue and i have literally tried EVERYTHING in this issue and more, have spent two days on this before i am writing this.

I have a NFS mount on the machine and i use docker-compose and use latest mosquitto. I have tried setting the owner to 1883:1883, i have tried to set permissions using chmod -R 777, i have tried to set the user: 1883:1883 and i also tried to set the

environment:
  - PGID=1883
  - PUID=1883

No matter what i do i get :

Error: Unable to open pwfile "/mosquitto/config/mosquitto.passwd".
Error opening password file "/mosquitto/config/mosquitto.passwd".
Error: Unable to open log file /mosquitto/log/mosquitto.log for writing.

Here is my docker-compose :

services:
  mosquitto:
      container_name: mosquitto
      restart: always
      image: eclipse-mosquitto:latest
      volumes:
        - "{{ app_data_dir }}/mosquitto/config:/mosquitto/config"
        - "{{ app_data_dir }}/mosquitto/data:/mosquitto/data"
        - "{{ app_data_dir }}/mosquitto/log:/mosquitto/log"
        - /etc/localtime:/etc/localtime:ro
      ports:
        - "1883:1883"
        - "9001:9001"
      networks:
        internal_net:
          ipv4_address: {{mosquitto_ip}}
      labels:
        - "traefik.enable=false"
      environment:
        - PUID=1883
        - PGID=1883
networks:
  proxy:
    external: true
  internal_net:
    external: true

Here is the output of the permissions and owner :

ansible@myvm-0-dev:~$ sudo ls -lR /var/docker_data/mosquitto
/var/docker_data/mosquitto:
total 0
drwxrwxrwx 1 1883 1883 102 Feb  3 16:57 config
drwxrwxrwx 1 1883 1883  24 Mar 11 15:32 data
drwxrwxrwx 1 1883 1883   0 Mar 13 23:20 log

/var/docker_data/mosquitto/config:
total 12
-rwxrwxrwx 1 1883 1883 186 Mar 13 22:50  mosquitto.conf
-rwxrwxrwx 1 1883 1883 351 Mar 11 17:00  mosquitto.passwd
-rwxrwxrwx 1 1883 1883 235 Mar 30  2020 'mosquitto.passwd copy'

/var/docker_data/mosquitto/data:
total 4
-rwxrwxrwx 1 1883 1883 143 Mar 11 15:32 mosquitto.db

/var/docker_data/mosquitto/log:
total 0

Here is my NFS mount in fstab mount :

10.10.0.15:/volume1/docker_test /var/docker_data nfs rw,sync 0 0

If anybody has some idea more that has been described in this issue please do share since i don't know what else i can try ? I can see the dockerfile sets the owner of the file to 1883 so even in the cases where i tries to run as another user the container sets the owner of the files to 1883.

[ralight]

@donnib I'm sorry to hear you're having a tricky time of it.

Error: Unable to open pwfile "/mosquitto/config/mosquitto.passwd".

Starting with the most simple of cases, this message says that the broker doesn't have any read access at all to config directory, so something must not be consistent somewhere because what you've show suggests that it should have the correct permissions.

Does the 1883 user exist on the nfs server?

You should also note that the chown in the docker entrypoint only runs if the docker user is root. If you are running as your own user it should not change the permissions.

[donnib]

Does the 1883 user exist on the nfs server?

@ralight thank you replying. No there is no user with that id on the NFS server, i have another NFS folder i am sharing out from same server to a mosquitto broken on another docker host and that works so there seem not to be a requirement that the user must exist on the server other wise t wouldn't have worked there either.

You should also note that the chown in the docker entrypoint only runs if the docker user is root. If you are running as your own user it should not change the permissions.

Point taken but in in all my desperate trials i also tried with root so i guess that's when the owner was changed.

Is there a way i can troubleshoot more for example have a print of what user tries to read on the docker volume (NFS share) since as you said there must be something that is inconsistent ?

[BrentonPoke]

I'm having the same issue with permissions, except I'm trying to get a conf file in. I get this no matter what I do.

chown: /mosquitto/config/mosquitto.conf: Permission denied
1620592029: Error: Unable to open config file /mosquitto/config/mosquitto.conf

I have no idea why this doesn't work, my permissions are just my user account and i'm not sure why this container can't just read the file. The permissions are this, and i'm not sure if it's SELinux or something else. I'm on Fedora, and I don't think this is a podman issue.
-rwxrwxrwx. 1 brentonpoke brentonpoke 35 May 8 19:47 mosquitto.conf

[chrisn-au]

I had this issue as I was using a file system mount rather than a volume mount so I could store config files in Github. After 3 -4 hours of very great frustration, I gave in and used volume mount and magically problems went away (I needed the config changes manually into the config file)

btw I needed to pin mosquito to 2.0.11 as anything more recent does not work with the Schneider PLC

  mosquitto:
    image: eclipse-mosquitto:2.0.11 
    volumes:
      - mqtt-sit:/mosquitto/:rw
    ports:
      - '1883:1883'
    networks:
      - sit-local

[Smyl3]

I have the same issue. With every other of my containers I using PUID, PGID docker run parameters to bind my local user to container but it doesnt work only in mosquitto. Mosquitto container files and folders permission always turns to 1883.
Really I have to use mosquitto without a container to work as it should? I never see that issue with any other container...

[matthewparkes]

I have the same issue. With every other of my containers I using PUID, PGID docker run parameters to bind my local user to container but it doesnt work only in mosquitto. Mosquitto container files and folders permission always turns to 1883. Really I have to use mosquitto without a container to work as it should? I never see that issue with any other container...

Added new issue: #2441

[arabezar]

I ran into the same problem myself today.

My setup is running mosquitto in docker on a Synology nas.

Same stuff. I've resolved it this way - after creating the main mosquitto folder mkdir mqtt just changed the permissions: chmod 777 mqtt then created subfolders (config, data, log) and started the container with the params above... and it worked for me with no changing owner and no user creation. I hope this helps someone.

[anschweitzer]

@ralight is there an example somewhere showing running the docker image and mounting its log directory onto the host machine, including, for example, any control of permissions, directories and mosquitto config file?

The example here appears to be insufficient. I found this paragraph but have not been able to use that information to allow the container to access logging directory on the host. I've tried various suggestions on this page (including chmod, chown, mkdir, --user, PGID/PUID) with no luck.

If there were a complete documented canonical example, that would be a great place to troubleshoot from.

Thanks

Andy

[lubrob]

I created a file with passwords in a place accessible to the "file station" application, gave everyone read/write permissions, gave a wipe to the file in the config and reinstalled Mosquito. This time it took off without any problems.