Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky tests during release. One test has now been disabled #287

Open
simonbasle opened this issue Aug 9, 2022 · 0 comments
Open

Flaky tests during release. One test has now been disabled #287

simonbasle opened this issue Aug 9, 2022 · 0 comments
Labels
type/chore A task not related to code (build, formatting, process, ...)
Milestone

Comments

@simonbasle
Copy link
Member

After the 1.3.13 release, when going back to snapshots CI was unable to build new snapshots due to flaky tests.

We've made 6 attempts total, and have seen 3 different flaky tests. One is the culprit in several attempts, so it has been @Ignore in commit here. Fortunately in that last ditch attempt CI was finally able to pass, but this needs fixing.

Job with 6 failed attempts is https://github.com/reactor/reactor-kafka/runs/7745848334

Tests that failed were:

reactor.kafka.receiver.KafkaReceiverTest > autoCommitFailurePropagationAfterRetries FAILED
    java.lang.AssertionError: expected:<[7, 11]> but was:<[7, 3, 7, 11]>

(most times)
then:

reactor.kafka.samples.SampleScenariosTest > atmostOnce FAILED
    java.lang.AssertionError: Incorrect number of messages received, expected=20, received=11

and:

reactor.kafka.receiver.KafkaReceiverTest > transactionalOffsetCommit FAILED
    java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at reactor.kafka.receiver.KafkaReceiverTest.transactionalOffsetCommit(KafkaReceiverTest.java:1224)
@simonbasle simonbasle added the type/chore A task not related to code (build, formatting, process, ...) label Aug 9, 2022
@simonbasle simonbasle added this to the 1.3.x Backlog milestone Aug 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/chore A task not related to code (build, formatting, process, ...)
Projects
None yet
Development

No branches or pull requests

1 participant