Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FLINK-1560 - Add ITCases for streaming examples #519

Closed
wants to merge 3 commits into from
Closed

FLINK-1560 - Add ITCases for streaming examples #519

wants to merge 3 commits into from

Conversation

szape
Copy link
Contributor

@szape szape commented Mar 23, 2015

ITCases for streaming examples have been added, except for IterateExample and StockPrices. I replaced the IterateExample with one that generates Fibonacci-sequences. Something is bugous around iteration, so the ITCase have to wait. StockPrices is untestable because of the windowJoin operator, hence it will not have an ITCase right now.

rmetzger
rmetzger reviewed Mar 23, 2015
View reviewed changes
...st/java/org/apache/flink/streaming/examples/test/socket/SocketTextStreamWordCountITCase.java
try {
waitForAccept();
} catch (Exception e) {
e.printStackTrace();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would fail the test in case of an exception.

@mbalassi
Copy link
Contributor

Please update this for parallelism of 4. Besides that and the inline comments looks good.

@szape
Copy link
Contributor Author

szape commented Mar 23, 2015

Thank you guys! I corrected the errors.
Updating the ITCases for parallelism 4 will be easier than I thought, but windowing makes it hard to test WindowJoin and TopSpeedWindowingExample with higher parallelism. So, parallelism will be set to 4 by default, but in the two mentioned cases it is better set to 1. What dou you think? Is there an easy workaround to this problem?

@szape szape force-pushed the FLINK-1560 branch 3 times, most recently from 49f49e8 to 0b21e3b Compare March 25, 2015 10:29
@szape
Copy link
Contributor Author

szape commented Mar 26, 2015

I tried to fix issue FLINK-1662, that is related to IterateExample. Gyula, what do you think?
To see the actual bug, run IterateExample with parallelism 1 without the last bugfix commit.

@mbalassi
Copy link
Contributor

@szape: I've started merging this: The TwitterStreamITCase seems to fail spuriously. Could you look into it?

@szape
Copy link
Contributor Author

szape commented Mar 26, 2015

It runs just fine on my computer. Did you pull the latest state of the branch? The TwitterStreamData class was updated not so long ago.
Also, the IterateExample isn't running with parallelism higher than one. It throws something like "not enough slot". I couldn't resolve this bug by now.

@szape szape force-pushed the FLINK-1560 branch 4 times, most recently from 2e2a63f to b2d8e8b Compare April 1, 2015 12:16
@mbalassi
Copy link
Contributor

mbalassi commented Apr 6, 2015

Thanks, now the Twitter one runs fine.

mbalassi added a commit to mbalassi/flink that referenced this pull request Apr 6, 2015
@mbalassi
[FLINK-1560] [streaming] Streaming example ITCases cleanup
a7e333b 
This closes apache#519
mbalassi added a commit to mbalassi/flink that referenced this pull request Apr 7, 2015
@mbalassi
[FLINK-1560] [streaming] Streaming example ITCases cleanup
aed208f 
This closes apache#519
@asfgit asfgit closed this in 954beca Apr 7, 2015
bhatsachin pushed a commit to bhatsachin/flink that referenced this pull request May 5, 2015
@mbalassi @bhatsachin
[FLINK-1560] [streaming] Streaming example ITCases cleanup
9512142 
This closes apache#519
marthavk pushed a commit to marthavk/flink that referenced this pull request Jun 9, 2015
@mbalassi @marthavk
[FLINK-1560] [streaming] Streaming example ITCases cleanup
f42bdac 
This closes apache#519
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
component=API/DataStream
Projects
None yet
Milestone
No milestone
3 participants
@szape @mbalassi @rmetzger