Reduce memory footprint by reusing buffers #229
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds some benchmarks for
Collection.Insert()
and changesmongoSocket.Query()
andaddBSON()
to reuse allocated[]byte
buffers by getting and putting then into async.Pool
.Also the function
MarshalBuffer
was added to the bson package. It behaves in the exact same way as Marshal but instead of allocating a []byte buffer it receives an already existing buffer.I came upon these changes after profiling a real world application that inserts a lot of documents simultaneously.
The added benchmarks show a really nice decrease in memory footprint specially when large messages are being sent to MongoDB, as is the case of inserting multiple documents at once:
I did't use gocheck for the benchmarks because it was returning some weird numbers. I'm not sure whether I did something wrong but these are the numbers for the exact same code only adapting it to use gocheck:
Compared to the use of native benchmarks:
Maybe I did something wrong and I'm open to work on it further if you have any tips.