Update mqtt proxy to broker

[suchen-sci]

this pr is to make MQTTProxy support broker mode by set brokerMode true in spec. Original MQTTProxy only publish messages to backend Kafka. Now, MQTTProxy not only publish message to Kafka but also send messages to the subscribed clients directly.

Main changes:

1. cachedTopicManager: it contains topic and mqtt client mapping information from all easegress nodes.
2. SessionCacheManager: it contains all session information for all mqtt clients from all easegress nodes and update cachedTopicManager.
3. broker: watch etcd changes and update SessionCacheManager.
4. other updates include optim resend and store, solve conflict of client connect to new easegress instance without disconnect previous one (more details in comment).

[suchen-sci]

Previous comment is in #874 . Sorry for inconvince...

[codecov-commenter]

Codecov Report

Base: 76.10% // Head: 75.63% // Decreases project coverage by -0.46% ⚠️

Coverage data is based on head (36dd17a) compared to base (4c9b3a1).
Patch coverage: 76.57% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #876      +/-   ##
==========================================
- Coverage   76.10%   75.63%   -0.47%     
==========================================
  Files         110      114       +4     
  Lines       12698    13221     +523     
==========================================
+ Hits         9664    10000     +336     
- Misses       2490     2645     +155     
- Partials      544      576      +32     

Impacted Files	Coverage Δ
pkg/object/mqttproxy/spec.go	100.00% <ø> (ø)
pkg/object/mqttproxy/client.go	77.45% <33.33%> (-4.16%)	⬇️
pkg/object/mqttproxy/mqttproxy.go	46.83% <33.33%> (ø)
pkg/object/mqttproxy/topic.go	47.05% <53.84%> (-52.95%)	⬇️
pkg/object/mqttproxy/sessioncache.go	61.85% <61.85%> (ø)
pkg/object/mqttproxy/session.go	67.70% <68.08%> (-14.51%)	⬇️
pkg/object/mqttproxy/broker.go	74.37% <68.75%> (-0.43%)	⬇️
pkg/object/mqttproxy/cachetopicmgr.go	79.86% <79.86%> (ø)
pkg/object/mqttproxy/nocachetopicmgr.go	92.85% <92.85%> (ø)
pkg/object/mqttproxy/session_manager.go	86.76% <100.00%> (+0.40%)	⬆️
... and 9 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[zhao-kun]

I prefer to enable the broker model by default because:

1. The backward-compatible, users can upgrade to the broker model by default, they needn't update their specs.
2. Broker model has more extensive use cases.

[suchen-sci]

I am more prefer not to enable it because previous users can upgrade it and the behavior of MQTTProxy stays unchanged unless they update their spec... But I am open to update it.