Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Periodic committer does not commit already committed offsets #22

Open
bsideup opened this issue Feb 17, 2018 · 0 comments
Open

Periodic committer does not commit already committed offsets #22

bsideup opened this issue Feb 17, 2018 · 0 comments
Assignees
Labels
type/bug A general bug

Comments

@bsideup
Copy link
Contributor

bsideup commented Feb 17, 2018

In Kafka, the offsets are stored for a limited period of time (24h by default), and if the consumer did not commit them during that period, they get wiped.
There is "periodic commit" logic in reactor-kafka, but it removes consumedOffsets during the commit:

If partition did not receive any events for the past 24h, no commit will be performed and Kafka will delete the offset for that partition.

Reactor Kafka should support a "refresh" of already committed offsets to avoid that.

See a similar issue in Akka's reactive-kafka:
akka/alpakka-kafka#375

@smaldini smaldini added this to the 1.0.1.RELEASE milestone Jun 15, 2018
@smaldini smaldini added type/bug A general bug and removed type/bug A general bug labels Jun 15, 2018
@OlegDokuka OlegDokuka self-assigned this Jul 9, 2018
@OlegDokuka OlegDokuka added the WIP label Jul 9, 2018
@bsideup bsideup removed the WIP label Jun 10, 2019
@simonbasle simonbasle modified the milestones: 1.1.1.RELEASE, 1.1.2.RELEASE, 1.1.3.RELEASE Nov 26, 2019
@bsideup bsideup removed this from the 1.1.3.RELEASE milestone Apr 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug A general bug
Projects
None yet
Development

No branches or pull requests

4 participants