persistence: autosave_on_changes does not trigger when number of queued messages decreases

[nduhme]

Hi,
i found a special case that seems to violate QoS2 in case of the broker crashing.

I am using mosquitto 1.6.9 to collect data that should be delivered exactly once (QoS2). In my system there are multiple publishers (50-100) and a single persistent subscriber for that i want to be sure that he gets all queued messages (no Clean Start).

To prevent potential loss of messages when the broker might crash i set
autosave_on_changes true
autosave_interval 1
persistence true
in the configuration file.

I found that when i have a subscriber with QoS2 that is temporarily not available every queued message is saved to the mosquitto.db file. So far so good.
If i now reconnect the subscriber all messages are sent to it, but the mosquitto.db file still contains all the queued messages. This seems to be the case because successfully sending a message (decreasing the queue) does not trigger the autosave_on_change. If the broker crashes and is restarted now the information that these messages were already recevied by the subscriber is lost. After restarting the broker and reconnecting the subscriber the messages will be sent a second time which violates QoS2 as the subscriber received them twice.

Steps to reproduce:
0 - Set configuration file: autosave_on_changes true, autosave_interval 1, persistence true
1 - Subscribe to topic T with QoS2
2 - Disconnect subscriber
3 - Publish messages with QoS2 to topic T (these are queued and trigger the autosave_on_change)
4 - Reconnect subcriber (no Clean Start)
5 - wait till all queued messages are revecied by subscriber
6 - Stop mosquitto broker by killing the task (simulating a crash)
7 - Restart mosquitto broker
8 - Reconnect subcriber (no Clean Start)
=> now the messages that were already received in step 5 will be received again

It would be great if there was an option to also autosave when the queue decreases otherwise i would have to downgrade my subscriber to QoS1.

[ralight]

Thanks for the nice details and sorry to not get to this sooner. I've pushed a change to the fixes branch that should sort this.