Skip to content

Commit

Permalink
[FLINK-7300] Fix unstable Kafka end-to-end tests
Browse files Browse the repository at this point in the history
Instead of specifying a timeout for the console consumer we now speciy
an uppper message boundary that matches our expected number of messages.
The timout was to unstable because it could sometimes return even though
messages were in fact available.
  • Loading branch information
aljoscha committed Aug 8, 2017
1 parent 6f5fa7f commit 65402e0
Showing 1 changed file with 1 addition and 18 deletions.
19 changes: 1 addition & 18 deletions test-infra/end-to-end-test/test_streaming_kafka010.sh
Original file line number Diff line number Diff line change
Expand Up @@ -73,24 +73,7 @@ $FLINK_DIR/bin/flink run -d build-target/examples/streaming/Kafka010Example.jar
# send some data to Kafka
echo -e "hello\nwhats\nup" | $KAFKA_DIR/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test-input

# wait at most (roughly) 5 minutes until the results are there
for i in {1..300}; do
DATA_FROM_KAFKA=$($KAFKA_DIR/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-output --from-beginning --timeout-ms 0 2> /dev/null)

# make sure we have actual newlines in the string, not "\n"
EXPECTED=$(printf "PREFIX:hello\nPREFIX:whats\nPREFIX:up")

if [[ "$DATA_FROM_KAFKA" == "$EXPECTED" ]]; then
echo -e "Retrieved data from Kafka: --$DATA_FROM_KAFKA--"
break
fi

echo "Waiting for results from Kafka..."
sleep 1
done

# verify again to set the PASS variable
DATA_FROM_KAFKA=$($KAFKA_DIR/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-output --from-beginning --timeout-ms 0 2> /dev/null)
DATA_FROM_KAFKA=$($KAFKA_DIR/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-output --from-beginning --max-messages 3 2> /dev/null)

# make sure we have actual newlines in the string, not "\n"
EXPECTED=$(printf "PREFIX:hello\nPREFIX:whats\nPREFIX:up")
Expand Down

0 comments on commit 65402e0

Please sign in to comment.