These commit is mostly resolve the memory leak problem and optimize memory usage to handle millions of requests #81
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These commits optimize memory allocation to reuse buffer instead of keep allocating and release the memory on every request which push a lot of pressure on GC.
This pull request also add a notice and an config option to prevent memory leak if client sending Authorization while Pulsar broker does not activate it. And an option to keeps both Pulsar token authorization and normal HTTP Authorization without confliction.
This pull request also allow to set a limit on number of concurrency (config
WorkerPoolSize
) to limit number of concurrent request to protect the brokers as same as leverage reusable memory buffer. Without this one, an accidentally peak time can cause crashing of service and uncontrolable of resource usage.This pull request is tested and using on production environment of https://doopage.com (we are using only Pulsar receiver) to handle few millions requests per day and it definitely improve the performance by using less CPU (mostly spend on GC) and of course much less Memory.