Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Encoding back pressure system so that offset payloads are prevented from being too large #47

Closed
astubbs opened this issue Dec 7, 2020 · 1 comment · Fixed by #51
Closed

Encoding back pressure system so that offset payloads are prevented from being too large #47

astubbs opened this issue Dec 7, 2020 · 1 comment · Fixed by #51
Labels
enhancement New feature or request ver:0.3.0.0

Comments

@astubbs
Copy link
Contributor

astubbs commented Dec 7, 2020 •
edited

At the moment the system does't protect against the space required to encode the offset overrun data from becoming too large to fit in the space available.

An approximation of the continuous approach is to keep some history of record per byte performance, and use that rate to restrict new messages.

For an exact approach, see #53
@astubbs astubbs mentioned this issue Dec 7, 2020
Closed
@astubbs astubbs changed the title Proper offset encoding back pressure system so that too large payloads cannot be needed Proper offset encoding back pressure system so that offset payloads can't ever be too large Dec 13, 2020
@astubbs astubbs linked a pull request Dec 14, 2020 that will close this issue
Final improvements before releasing 3.0 #51
Merged
@astubbs astubbs mentioned this issue Dec 14, 2020
Closed
@astubbs astubbs changed the title Proper offset encoding back pressure system so that offset payloads can't ever be too large Encoding back pressure system so that offset payloads are prevented from being too large Dec 14, 2020
@astubbs astubbs added enhancement New feature or request ver:0.3.0.0 labels Dec 14, 2020
@astubbs
Copy link
Contributor Author

astubbs commented Dec 14, 2020

Approximate approach merged for 0.3.0

@astubbs astubbs closed this as completed Dec 14, 2020
@what-the-diff what-the-diff bot mentioned this issue Jan 13, 2023
Closed
3 tasks
@botPotyo botPotyo mentioned this issue Sep 25, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request ver:0.3.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Final improvements before releasing 3.0 astubbs/parallel-consumer
1 participant
@astubbs