-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mosquitto broker stops working, it stalls after working for a while #986
Comments
Can you use lsof to see if it has run out of file handles? Can you strace the running process to see where it's stopped? |
i have installed both, i am not sure how to use them
at the moment it worked for more than 3h, i did not change anything, it will stall again.
|
You want lsof and strace when it's stuck, not when it's working
normally.
alwashe <[email protected]> wrote:
… i have installed both, i am not sure how to use them
```
***@***.***:~ $ sudo strace -c -p 9215
strace: Process 9215 attached
^Cstrace: Process 9215 detached
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
56.38 0.300066 8 36885 read
19.14 0.101864 8 12294 12294 write
15.44 0.082198 7 12294 rt_sigprocmask
9.03 0.048073 8 6147 epoll_wait
------ ----------- ----------- --------- --------- ----------------
100.00 0.532201 67620 12294 total
```
```
***@***.***:~ $ sudo lsof -p 9215
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME mosquitto
9215 mosquitto cwd DIR 179,2 4096 2 / mosquitto 9215 mosquitto
rtd DIR 179,2 4096 2 / mosquitto 9215 mosquitto txt REG 179,2
162296 2049 /usr/sbin/mosquitto mosquitto 9215 mosquitto mem
REG 179,2 38560 1945
/lib/arm-linux-gnueabihf/libnss_files-2.24.so mosquitto 9215
mosquitto mem REG 179,2 38588 1953
/lib/arm-linux-gnueabihf/libnss_nis-2.24.so mosquitto 9215
mosquitto mem REG 179,2 26456 1943
/lib/arm-linux-gnueabihf/libnss_compat-2.24.so mosquitto 9215
mosquitto mem REG 179,2 129840 1068
/usr/lib/arm-linux-gnueabihf/libuv.so.1.0.0 mosquitto 9215
mosquitto mem REG 179,2 46832 973
/usr/lib/arm-linux-gnueabihf/libev.so.4.0.0 mosquitto 9215
mosquitto mem REG 179,2 91884 1990
/lib/arm-linux-gnueabihf/libz.so.1.2.8 mosquitto 9215 mosquitto
mem REG 179,2 71604 1942
/lib/arm-linux-gnueabihf/libnsl-2.24.so mosquitto 9215
mosquitto mem REG 179,2 127300 1967
/lib/arm-linux-gnueabihf/libpthread-2.24.so mosquitto 9215
mosquitto mem REG 179,2 1234700 1906
/lib/arm-linux-gnueabihf/libc-2.24.so mosquitto 9215 mosquitto
mem REG 179,2 133604 1095
/usr/lib/arm-linux-gnueabihf/libwebsockets.so.8 mosquitto 9215
mosquitto mem REG 179,2 9964 1899
/lib/arm-linux-gnueabihf/libanl-2.24.so mosquitto 9215
mosquitto mem REG 179,2 14048 1988
/lib/arm-linux-gnueabihf/libuuid.so.1.3.0 mosquitto 9215
mosquitto mem REG 179,2 1827956 10467
/usr/lib/arm-linux-gnueabihf/libcrypto.so.1.1 mosquitto 9215
mosquitto mem REG 179,2 327952 10582
/usr/lib/arm-linux-gnueabihf/libssl.so.1.1 mosquitto 9215
mosquitto mem REG 179,2 30548 1989
/lib/arm-linux-gnueabihf/libwrap.so.0.7.6 mosquitto 9215
mosquitto mem REG 179,2 26632 1971
/lib/arm-linux-gnueabihf/librt-2.24.so mosquitto 9215 mosquitto
mem REG 179,2 452152 1933 /lib/arm-linux-gnueabihf/libm-2.24.so
mosquitto 9215 mosquitto mem REG 179,2 9800 1915
/lib/arm-linux-gnueabihf/libdl-2.24.so mosquitto 9215 mosquitto
mem REG 179,2 21868 10441
/usr/lib/arm-linux-gnueabihf/libarmmem.so mosquitto 9215
mosquitto mem REG 179,2 138576 1859
/lib/arm-linux-gnueabihf/ld-2.24.so mosquitto 9215 mosquitto 0u
CHR 1,3 0t0 1028 /dev/null mosquitto 9215 mosquitto 1u CHR 1,3
0t0 1028 /dev/null mosquitto 9215 mosquitto 2u CHR 1,3 0t0 1028
/dev/null mosquitto 9215 mosquitto 3u sock 0,8 0t0 27983
protocol: TCP mosquitto 9215 mosquitto 4w REG 179,2 29062
131108 /var/log/mosquitto/mosquitto2.log mosquitto 9215
mosquitto 5u IPv4 27991 0t0 TCP *:1883 (LISTEN) mosquitto 9215
mosquitto 6u IPv6 27992 0t0 TCP *:1883 (LISTEN) mosquitto 9215
mosquitto 7u a_inode 0,11 0 6408 [eventpoll] mosquitto 9215
mosquitto 8u IPv4 34831 0t0 TCP
mqtt.fritz.box:1883->ESP-6E423C.fritz.box:60225 (ESTABLISHED)
mosquitto 9215 mosquitto 9u IPv4 29677 0t0 TCP
mqtt.fritz.box:1883->ESP-6E40C1.fritz.box:60200 (ESTABLISHED)
mosquitto 9215 mosquitto 11u IPv4 25200 0t0 TCP
mqtt.fritz.box:1883->JmDNS-IP-1.fritz.box:38106 (ESTABLISHED)
```
```
***@***.***:~ $ sudo strace -p 9215
rt_sigprocmask(SIG_SETMASK, [HUP INT USR1 USR2 TERM], [], 8) =
0 epoll_wait(7, [{EPOLLIN, {u32=8, u64=8}}, {EPOLLIN, {u32=9,
u64=9}}], 1000, 100) = 2 rt_sigprocmask(SIG_SETMASK, [], NULL,
8) = 0 read(8, "\202", 1) = 1 read(8, "\r", 1) = 1 read(8,
"\317z\0\10MQTT_ACT\0", 13) = 13 write(8, "\3\na\0", 4) = -1
EAGAIN (Resource temporarily unavailable) read(9, "\202", 1) =
1 read(9, "\r", 1) = 1 read(9, "\361\240\0\10MQTT_ACT\0", 13) =
13 write(9, "\3\nn\0", 4) = -1 EAGAIN (Resource temporarily
unavailable) rt_sigprocmask(SIG_SETMASK, [HUP INT USR1 USR2
TERM], [], 8) = 0 epoll_wait(7, [{EPOLLIN, {u32=8, u64=8}},
{EPOLLIN, {u32=9, u64=9}}], 1000, 100) = 2
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 read(8, "\202", 1)
= 1 read(8, "\r", 1) = 1 read(8, "\317{\0\10MQTT_ACT\0", 13) =
13 write(8, "\3\na\0", 4) = -1 EAGAIN (Resource temporarily
unavailable) read(9, "\202", 1) = 1 read(9, "\r", 1) = 1
read(9, "\361\241\0\10MQTT_ACT\0", 13) = 13 write(9, "\3\nn\0",
4^C) = -1 EAGAIN (Resource temporarily unavailable) strace:
Process 9215 detached
```
**at the moment it worked for more than 3h, i did not change
anything, it will stall again.**
```
***@***.***:~ $ sudo service mosquitto status
● mosquitto.service - LSB: mosquitto MQTT v3.1 message broker
Loaded: loaded (/etc/init.d/mosquitto; generated; vendor preset: enabled)
Active: active (running) since Wed 2018-10-10 08:27:39 UTC; 3h 35min ago
Docs: man:systemd-sysv-generator(8)
Process: 9201 ExecStop=/etc/init.d/mosquitto stop (code=exited, status=0/SUCCESS)
Process: 9209 ExecStart=/etc/init.d/mosquitto start (code=exited, status=0/SUCCESS)
CGroup: /system.slice/mosquitto.service
└─9215 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
Oct 10 08:27:39 mqtt systemd[1]: Starting LSB: mosquitto MQTT
v3.1 message broker... Oct 10 08:27:39 mqtt mosquitto[9209]:
Starting network daemon:: mosquitto. Oct 10 08:27:39 mqtt
systemd[1]: Started LSB: mosquitto MQTT v3.1 message broker.
```
|
Yes, this makes sense, are the commands right? |
I have exact the same problem after upgrading and fresh reinstall of 1.5.3 (on debian) It works for a while and then the service is exited (random time range) |
the Server ran for about 12 h, longest time. |
maybe
this is for 2 days |
Hope this helps
|
we're seeing this too, but the server was restarted before I could grab any debug. WIll be watching closer in future.... |
wow, still no official reaction, how is this possible.? |
really dont know what I changed, |
same here |
@alwashe How did you resolve this one? I'm also investigating an issue similar to you have. Thank you. |
Raspberry Pi 3b, where i have Raspbian 4.14, mosquito 1.5.3. and mosquito-clienets
After about 30 to 120 min i have to restart the service because no connection is anymore possible.
The text was updated successfully, but these errors were encountered: