Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rfc(decision): Batch multiple files together into single large file to improve network throughput #98

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open

rfc(decision): Batch multiple files together into single large file to improve network throughput #98

Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
912301d
rfc(decision): Store Multiple Replay Segments in a Single Blob
cmanallen May 24, 2023
7dfe76c
Initial commit
cmanallen May 24, 2023
f82f9e6
Add semicolons
cmanallen May 24, 2023
721c28c
Fix flowchart
cmanallen May 24, 2023
4ff4376
Use correct byte start
cmanallen May 24, 2023
4ec773a
Add extension
cmanallen May 24, 2023
86c1819
Remove ancillary goal motivation
cmanallen May 24, 2023
9f9d123
Add more detail
cmanallen May 25, 2023
8b25e3b
Use clearer language
cmanallen May 25, 2023
d34401e
Add technical details section
cmanallen May 26, 2023
8af3d50
Merge branch 'rfc/store-multiple-replay-segments-in-a-single-blob' of…
cmanallen May 26, 2023
812a110
Re-write to include notes about encryption of files
cmanallen Jun 8, 2023
f5fe8e8
Update questions section
cmanallen Jun 8, 2023
78a0d6e
Add created-at column
cmanallen Jun 8, 2023
82f222d
Add high-level overview
cmanallen Jun 8, 2023
a7ad1d6
Improve formatting
cmanallen Jun 9, 2023
d63a30a
Clean up managing effects section slightly
cmanallen Jun 9, 2023
354431d
Add more to supporting data section and add an FAQ
cmanallen Jul 18, 2023
15430b4
Re-write RFC to be product-neutral
cmanallen Aug 29, 2023
71f1715
Add conclusion
cmanallen Aug 29, 2023
3759346
To to with
cmanallen Aug 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add conclusion
  • Loading branch information
@cmanallen
cmanallen committed Aug 29, 2023
commit 71f1715b5e363dc6f1110b185a206f76eba6d262
2 changes: 1 addition & 1 deletion text/0098-store-multiple-replay-segments-in-a-single-blob.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,4 +183,4 @@ def zero_bytes_in_range(blob: io.BytesIO, start: int, length: int) -> None:

# Conclusion

Buffering files can provide significant throughput and cost advantages and poses minimal downside.
Buffering files can provide significant throughput and cost advantages and poses minimal downside. Kafka is the suggested implementation mechanism for batching files. It has strong client support for batched network-operations and is a well understood software-system within Sentry.