High CPU usage with mosquitto 2.0 under FreeBSD 12.1

[ptjm]

I'm seeing about 25% CPU usage in top with mosquitto 2.0.0 and 2.0.2 and either no or very few clients connected. This is on FreeBSD 12.1 using the system compiler, clang 8.0.1. The problem appears to be in mux_poll__handle(). pollfd_max is set to 1881756, and it's taking a bit of time to deal with all those invalid file descriptors. I found capping that value to a reasonable number fixed the problem. I suggest keeping track of the high-water index into pollfds and using that as the nfds argument to poll().

I haven't signed a contribution agreement so I don't want to taint a simple change by putting too much detail here, but hypothetically, one could have a static variable pollfd_highwater, initialize it to the value of pollfd_index in mux_poll__init(), and increment it if the index into pollfds reaches it in mux_poll__add_*() and then use pollfd_highwater as the nfds argument to poll(). That might hypothetically appeared to have fix the problem.

[ralight]

Thanks @ptjm, I'll get that fixed.

Out of interest, is it typical on FreeBSD to have the open file limit set so high by default?

[ptjm]

This is tied to the amount of memory on the machine (64GB in this case). On an RPI2, it's 25074. I looked up the formula in the kernal, and the default formula is 9/10 of 1/8 the number of physical pages in the machine.

Thanks a lot, and best wishes.

[ralight]

I've pushed a fix for this, but not tested it on FreeBSD specifically. Are you able to give it a test and report back?

[ptjm]

Thanks. It appears to have fixed my problem. I found the listener tests failed, but I'm wondering if there's a missing allow_anon there somewhere. Seems unlikely it's related to this change.