Message delivery over the websockets is too slow #411
Comments
|
+1 |
|
I encountered this using Mosquitto on alpinelinux. It was fixed when I upgraded |
|
@gdraynz can you add a bit more details about what libwebsockets version fixed the issue for you? I still see it with v2.1.1 here |
|
I reported a similar issue to the mailing list last month, so far without any feedback: |
|
I just tried to reproduce the environment on which I encountered this issue but could not reproduce it again... |
|
Could you please give the latest fixes branch a try with regards to websockets performance? |
|
OK. I did some benchmarking. I started with the The tl;dr :
Overview of the test. I stood up 7 processes that each subscribed to '+', and published to a string comprised of their pid. Every time they received a message on the topic of their pid, they would publish again, they count all received messages, as well as their number of publishes. ) Test machineA 12 core mosquitto buildA docker container running debian stretch. Dependencies provided by apt. Versions: Node.js test codevar mqtt = require('mqtt')
var now = require('performance-now')
var pubcount = 0
var recvcount = 0
var recvstart
var recvend
var pubstart
var pubend
var topic = '' + process.pid
console.log("starting on topic ", topic)
var client = mqtt.connect('ws:https://localhost:1883')
client.on('connect', function () {
client.subscribe(topic)
client.subscribe('+')
pubstart = now();
recvstart = pubstart;
client.publish(topic, 'Hello mqtt')
})
client.on('message', function (intopic, message) {
// message is Buffer
// console.log(message.toString())
if (intopic === topic) {
if (++pubcount >= 10000) {
pubcount = 0;
//client.end()
pubend = now()
console.log('pubs per sec :', 1000 * 10000 / (pubend - pubstart));
pubstart = pubend
}
client.publish(topic, 'Hello mqtt')
}
if (++recvcount >= 50000) {
recvcount = 0
recvend = now()
console.log('receives per sec :', 1000 * 50000 / (recvend - recvstart) );
recvstart = recvend
}
})sample output (single process)top showing mem and cpu |
|
@rrichardson awesome, thanks for confirming. |
|
This is fixed in 1.4.13. @rrichardson as you've hopefully still got your test set up, I don't suppose you'd mind trying the test again with the version at #495 ? |
|
I can provide more information on this, had the same problems, large packets > 4k had a huge delay. 1.5Mb took 40 seconds to arrive on the client over websockets. I finally found the problems, and there are multiple problems.
Meaning a package of size 1.5Mb will be split into multiple packets by libwebsocket and those will be sent on average every 90ms .... each time the mosquitto loop runs. I solved these in two ways.
The performance improvement was huge. `------------------------------------------------------------------------------- -> Single call to libwebsocket_service per mosquitto loop
I'm on windows and the results are from I can do a pull request for this, however I am not sure what impact this has on anything else since I don't have tests setup. Also the point 2 for the fix should be loop until packet is sent, and I don't know how to do that for now. The new configuration options are ` #the size of the chunk to send to libwebsockets PS. For anyone not wanting to do the full fix and wants to do a quick test, in loop.c find the call to libwebsocket_service and wrap it in a for loop. |
|
@arminbaljic I don't know if you found the issue or not, or if you are still using mosquitto. If you can, please do a quick test with the finding above? |
|
Hi @MRazvan , unfortunately, due to performance issues and some other bugs we had to change MQTT broker. The client was becoming increasingly agitated by these issues and we decided to switch to RabbitMQ with web MQTT plugin. |
|
Has anyone found a fix to this issue? |
|
@aalibasic this is a separate issue to the one you are seeing. This one was due to the websockets listeners not being serviced properly and should be closed. I believe the issue with large messages is entirely different. |
|
Closing because the original issue is addressed. |
I am having an issue with MQTT message delivery over websockets. On the server side, I have been using mosquitto broker version 1.4.8. installed on an AWS t2.medium server instance
with the following configuration:
After the data processing server publishes about 900 messages to a single subscriber (with QOS1). This is where I have noticed large latency (about 30 seconds) in message delivery.
This only happens with clients that are using websockets. If I subscribe with the MQTT.fx client (1883 port) then the message delivery is almost instant. Please see timings below:
Additionally, please notice that I have tried the following: different client library (MQTT.js from https://github.com/mqttjs), MQHive web client, replaced all processing in onMessageReceived client method with the console output, upgraded mosquitto broker to 1.4.11 version. All with the same results. Any thoughts about this?
The text was updated successfully, but these errors were encountered: