Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-10401] Port ProcessFailureCancelingITCase to new code base #6749

Conversation

tillrohrmann
Copy link
Contributor

What is the purpose of the change

This PR is based on #6743 and #6742 and is part of the removal of Flink's legacy mode.

Port ProcessFailureCancelingITCase to new code base using the SessionClusterComponent.

Verifying this change

  • Checked manually that the test works and still tests the cancellation

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicable)

Copy link
Contributor

@StefanRRichter StefanRRichter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@@ -130,6 +131,13 @@ public T getDispatcher() {
}
}

@VisibleForTesting
public WebMonitorEndpoint<?> getWebMonitorEndpoint() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coming back to my longer comment on #6749, I think this is only required because the collaborator is lazily created in a lifecycle method. I wonder if we could not have a building object the will create all collaborators at the place where currently startComponent is called because clusterComponent = createClusterComponent(configuration) and clusterComponent.startComponent(...) are even called in a straight sequence. Tests could then simply pass WebMonitorEndpoint into the component.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes you're right. I've incorporated your feedback from #6743 and with that it should no longer be necessary to have this getter.

Introduce a ClusterComponent which encapsulates the logic for starting the cluster
components, Dispatcher, RestServerEndpoint and the ResourceManager. The individual
components are created by using a factory instance. The ClusterEntrypoint is now
only responsible for managing the required services needed by the ClusterComponent.
This design should make the testing of these components easier, improve reusability
and reduce code duplication.
Move the logic of when to exit the JVM process out of the ClusterEntrypoint
so that the caller is now responsible to make this call. This improves the
usage of the ClusterEntrypoint for testing purposes.
This commit introduces the DispatcherResourceManagerComponentFactory which is used
to create a DispatcherResourceManagerComponent. That way, it is possible to eagerly
initialize all fields of the DispatcherResourceManagerComponent making it possible
to make all fields final and remove the lock.
@tillrohrmann
Copy link
Contributor Author

Thanks for the review @StefanRRichter. Addressing your comment and then merging this PR.

@tillrohrmann tillrohrmann force-pushed the portProcessFailureCancelingITCase branch from a4f768e to 77ca1e2 Compare September 27, 2018 15:04
tillrohrmann added a commit to tillrohrmann/flink that referenced this pull request Sep 28, 2018
@asfgit asfgit closed this in ff97d1c Sep 28, 2018
@tillrohrmann tillrohrmann deleted the portProcessFailureCancelingITCase branch September 28, 2018 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants