Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MQTT performance optimization #304

Merged
merged 9 commits into from
Oct 19, 2021
Merged

MQTT performance optimization #304

merged 9 commits into from
Oct 19, 2021

Conversation

suchen-sci
Copy link
Contributor

@suchen-sci suchen-sci commented Oct 15, 2021

This pr contains following optimizations and updates for MQTT proxy.

  1. remove unnecessary defer
  2. remove client writePacket lock since golang write to Conn already contain locks.
  3. since client writePacket has no lock, then cancel create goroutine in session.publish (no risk of dead lock), use buffered channel instead.
  4. use github.com/hashicorp/golang-lru package in topic cache. Keep both speed and memory security.
  5. merge topic split and validation check into one function to speed up. (new splitTopic function in topic.go)
  6. drop qos0 message when sending channel is full.
  7. clean session from DB when cleanSession flag is set
  8. clean topicMgr when client is closed to improve performance.

Thanks a lot for advice from @faceair.

Copy link

@megaeasex megaeasex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[TASK:easegress-pr-test SUCCESS]megaease/easegress Pull Request 304 Deploy Test Success

Copy link

@megaeasex megaeasex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[TASK:easegress-pr-test SUCCESS]megaease/easegress Pull Request 304 Deploy Test Success

@codecov-commenter
Copy link

codecov-commenter commented Oct 15, 2021

Codecov Report

Merging #304 (fd29438) into main (c0f5200) will increase coverage by 0.25%.
The diff coverage is 85.71%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #304      +/-   ##
==========================================
+ Coverage   80.22%   80.47%   +0.25%     
==========================================
  Files          53       53              
  Lines        5897     5973      +76     
==========================================
+ Hits         4731     4807      +76     
- Misses        907      908       +1     
+ Partials      259      258       -1     
Impacted Files Coverage Δ
pkg/object/mqttproxy/spec.go 100.00% <ø> (ø)
pkg/object/mqttproxy/session_manager.go 86.36% <40.00%> (-3.81%) ⬇️
pkg/object/mqttproxy/session.go 82.31% <66.66%> (+0.83%) ⬆️
pkg/object/mqttproxy/storage.go 94.87% <77.77%> (-5.13%) ⬇️
pkg/object/mqttproxy/client.go 81.09% <80.76%> (+2.15%) ⬆️
pkg/object/mqttproxy/broker.go 79.68% <90.00%> (+2.35%) ⬆️
pkg/object/mqttproxy/topic.go 100.00% <100.00%> (+0.46%) ⬆️
pkg/object/meshcontroller/spec/spec.go 89.52% <0.00%> (+0.01%) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c0f5200...fd29438. Read the comment docs.

Copy link

@megaeasex megaeasex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[TASK:easegress-pr-test SUCCESS]megaease/easegress Pull Request 304 Deploy Test Success

Copy link

@megaeasex megaeasex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[TASK:easegress-pr-test SUCCESS]megaease/easegress Pull Request 304 Deploy Test Success

Copy link

@megaeasex megaeasex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[TASK:easegress-pr-test SUCCESS]megaease/easegress Pull Request 304 Deploy Test Success

Copy link

@megaeasex megaeasex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[TASK:easegress-pr-test SUCCESS]megaease/easegress Pull Request 304 Deploy Test Success

@xxx7xxxx xxx7xxxx merged commit 69ef753 into easegress-io:main Oct 19, 2021
@suchen-sci suchen-sci deleted the mqtt-optim branch November 30, 2021 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants