900MB OOM Error #888
Comments
|
I presume you weren't expecting it? It's no problem to get a lot of memory consumption with a lot of queued messages and offline subscribers for instance. I highly recommend at least 1.4.15 (the last 1.4) or 1.5, which all have known memory and performance fixes. If you are still finding this to be unexpected, you'll need to elaborate on what your scenario is and what you are expecting. |
|
Hmm, I wasn't expecting it to continue to consume that much memory. If when the internet goes down, will it continue to buffer data in memory indefinitely? My setup is: I'm assuming the issue may have occurred if broker1 lost connectivity with the Internet possibly? So that it would start storing the messages in memory? Is it possible instead of consuming memory to flush the message straight to disk instead? Or flush to disk when the memory usage is above X MB? |
|
yes and no. persistence at present is simply having the in memory db written to disk perdioically, it won't reduce your ram at all. (And you wouldn't really want it, you'd just have to be churning the disk reading little pieces all the time so you weren't "using ram") No, there's no magic knobs for "I prefer disk io to ram usage" or thresholds for that. I still highly recommend running a more recent version for starters. |
|
I guess it is not a bug of mosquitto. You need a tips of OS memory management. OOM error comes from the memory overcommitted. You can get some hint by Googling "memory overcommit". |
|
Or could you resize your swap? Anyway, I guess it should be solved by optimizing your OS. You can get more useful comments at any OS forum. |
|
It looks like max_queued_messages and max_queued_bytes solve my issue. It would be nice to have the ability to set a global threshold too, as well as per client. |
|
@anfractuosity that would be #100 |
|
I guess @anfractuosity 's situation can be resolved by max_queued_... without #100 . Because his broker's uplink is only one bridge. |
|
I have an arbitrary number of clients behind the bridge though. Doesn't that mean there are multiple clients, so a global value could be useful? I'm assuming each of those clients has a separate queue at the bridge, before forwarding? |
|
Is your issue at pi (broker1) ? If so, there is only one queue for the uplink bridge context, although there are a lot of clients behind the bridge target. |
|
Sorry I'm a little confused. Yeah the issue was at broker 1. I was under the impression the broker has queues for each client? "Currently (1.1.x) mosquitto broker allows configuration of the maximum number of outstanding messages that are queued per client" seems to indicate there are multiple queues? |
|
Your issue is not at receiving messages from a lot of client to the broker 1 (multiple queues), but at sending messages from the broker 1 (one queue) to the one bridge target, right? I think to control one sending queue will be a solution for your issue. |
|
Thanks for the info. I'm not sure though, which queue 'max_queued_messages' controls. I'll have a look at the code when I have some time. I'll do some more analysis too soon, as I'm gonna setup a small testbed and can then play with different queue values. Cheers |
|
queues are per client, and there are only limits on those queues. #100 is a feature request to add limits that are broker wide as well. |
|
ah, gotcha, cheers |
|
@anfractuosity could you close this issue? Your issue is definitely not a bug for mosquitto. This thread became change into the consultation for your environment. If you want to continue to discuss, use mosquitto-dev ML. |
|
Sure yeah |
Hi,
Today I got an OOM error, when it was killed at about 900MB if I'm reading
the logs correctly:
Using version:
mosquitto version 1.4.10 (build date Fri, 22 Dec 2017 08:19:25 +0000)
Cheers
Chris
The text was updated successfully, but these errors were encountered: