Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mosquitto Service Crash / Seg Fault No Restart #1425

Closed
peterchs opened this issue Sep 25, 2019 · 4 comments
Closed

Mosquitto Service Crash / Seg Fault No Restart #1425

peterchs opened this issue Sep 25, 2019 · 4 comments

Comments

@peterchs
Copy link
Contributor

peterchs commented Sep 25, 2019

I'm running the following packages on Ubuntu 18.04.3 server;

#dpkg -l | grep mosquitto
ii  libmosquitto1:amd64                                         1.6.6-0mosquitto1~bionic1                                   amd64        MQTT version 3.1/3.1.1 client library
ii  mosquitto                                                   1.6.6-0mosquitto1~bionic1                                   amd64        MQTT version 3.1/3.1.1 compatible message broker
ii  mosquitto-clients                                           1.6.6-0mosquitto1~bionic1                                   amd64        Mosquitto command line MQTT clients

Ever since I recently updated the packages the service runs for a few hours and then seq faults as can be seen in the kern.log below. I'd restarted the service manually between the two crashes. Its interesting that its at a similar time..

Sep 24 06:25:01 cougar kernel: [141409.020829] mosquitto[13165]: segfault at 88 ip 0000561f7a04914c sp 00007ffd5682ca90 error 4 in mosquitto[561f7a025000+3c000]
Sep 25 01:25:14 cougar kernel: [209824.740542] CE: hpet increased min_delta_ns to 67887 nsec
Sep 25 06:25:02 cougar kernel: [227813.703606] mosquitto[1915]: segfault at 88 ip 0000564cae88114c sp 00007ffedc03d8d0 error 4 in mosquitto[564cae85d000+3c000]

The last few lines from mosquitto.log file with debug all on before crash;


1569389100: Sending PUBLISH to mqtt_eb479221.eee4d (d0, q0, r0, m0, 'rtl_433/cougar/devices/Efergy_e2_CT/5656/current', ... (8 bytes))
1569389100: Received PUBLISH from rtl_433-65c8d7a3 (d0, q0, r0, m0, 'rtl_433/cougar/devices/Efergy_e2_CT/5656/interval', ... (1 bytes))
1569389100: Received PUBLISH from rtl_433-65c8d7a3 (d0, q0, r0, m0, 'rtl_433/cougar/devices/Efergy_e2_CT/5656/battery', ... (2 bytes))
1569389100: Received PUBLISH from rtl_433-65c8d7a3 (d0, q0, r0, m0, 'rtl_433/cougar/devices/Efergy_e2_CT/5656/learn', ... (2 bytes))
1569389100: Received PUBLISH from rtl_433-65c8d7a3 (d0, q0, r0, m0, 'rtl_433/cougar/devices/Efergy_e2_CT/5656/mic', ... (8 bytes))
1569389101: Sending PINGREQ to local.cougar.external-bridge
1569389102: Received PINGRESP from local.cougar.external-bridge

The mosquitto.conf file;


pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log
log_type all

include_dir /etc/mosquitto/conf.d

connection external-bridge
address 192.168.1.122:1883
bridge_protocol_version mqttv31
topic REDACTED both 0
remote_username REDACTED 
remote_password REDACTED 

Also the way the package sets up the service it appears it does not automatically restart the service on such a crash - is this something that could be fixed easily? As seen below the process has exited but wasn't restarted. ps shows no mosquitto process when this was run.


# service mosquitto status
● mosquitto.service - LSB: mosquitto MQTT v3.1 message broker
   Loaded: loaded (/etc/init.d/mosquitto; generated)
   Active: active (exited) since Tue 2019-09-24 08:49:39 BST; 24h ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1812 ExecStop=/etc/init.d/mosquitto stop (code=exited, status=0/SUCCESS)
  Process: 1898 ExecStart=/etc/init.d/mosquitto start (code=exited, status=0/SUCCESS)

Sep 24 08:49:39 systemd[1]: Starting LSB: mosquitto MQTT v3.1 message broker...
Sep 24 08:49:39 mosquitto[1898]:  * Starting network daemon: mosquitto
Sep 24 08:49:39 mosquitto[1898]:    ...done.
Sep 24 08:49:39 systemd[1]: Started LSB: mosquitto MQTT v3.1 message broker.

@peterchs
Copy link
Contributor Author

Have updated;
/run/systemd/generator.late/mosquitto.service
to have;

Restart=always

Hopefully that will tactically fix it so it automatically restarts on a crash?

Still dont know whats causing the crash, checked my crontabs around that time, nothing that I can see..

@ralight
Copy link
Contributor

ralight commented Sep 25, 2019

Are there any config files in /etc/mosquitto/conf.d ?

@peterchs
Copy link
Contributor Author

No nothing in /etc/mosquitto/conf.d other than the README.

ralight added a commit that referenced this issue Sep 25, 2019
Closes #1424, #1425. Thanks to JinPingChng and peteakalad.
@ralight
Copy link
Contributor

ralight commented Sep 25, 2019

Could you please update, both issues should now be fixed.

@ralight ralight closed this as completed Sep 25, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Dec 24, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants