-
Notifications
You must be signed in to change notification settings - Fork 13.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[FLINK-2291] [runtime] Add ZooKeeper support to elect a leader from a…
… set of JobManager. The leader will then be retrieved from ZooKeeper by the TaskManagers. Refactors FlinkMiniCluster to support multiple JobManager Adds proper remote address resolution for actors Clean up of LeaderElection and LeaderRetrievalService. Removes synchronization to avoid deadlock. Adds ZooKeeper start option to TestBaseUtils.startCluster Removes registration session IDs, using the leader session IDs instead. Sets the leader session ID directly in the grantLeadership method. Let the LeaderElectionService select the leader session I D. Return leader session ID to LeaderRetrievalListeners. Removes direct ActorRef interaction Introduces LeaderRetrievalService for the Client and the CliFrontend. Make ApplicationClient to use the LeaderRetrievalService for JobManager resolution Adds LeaderElection/Retrieval tests Added test for exception forwarding from the CuratorFramework to a Contender Adds test job submission with changing leaders Adds new test cases for job cleanup after leader election change Adds new LeaderChangeStateCleanup test case Adds LeaderElectionRetrievalTestingCluster Introduces ListeningBehaviour for job submissions Relocation of org.apache.curator in flink-shaded-hadoop jar Adds Apache ZooKeeper and Apache Curator to LICENSE and NOTICE files Increases zookeeper connection timeout to 20000 ms for the KafkaITCase to fix failing tests on Travis Increased timeouts of ZooKeeperLeaderElectionTest for Travis Makes the WebInfoServer and the WebRuntimeMonitor to use the LeaderRetrievalService to retrieve the current leading JobManager Adds proper synchronization to ZooKeeperLeaderElectionService. Fixes StateCheckpointedITCase and PartitionedStateCheckpointingITCase Adds configuration description for new ZooKeeper configuration values Fixed port selection of JobManager at startup Improves logging output Extends masters file to also specify the webui ports Adds proper network interface resolution by retrieving the current leader address Makes the ZooKeeperLeaderElectionService write the leader information in ephemeral nodes so that the information is deleted once the leader has terminated. Fixes a bug in the TaskManager due to call by name semantics of scheduler.scheduleOnce. Adds jobManagerURL to TriggerTaskManagerRegistration message Enables findConnectingAddress to use the ZooKeeperLeaderRetrievalService. This allows to test the connection to a possibly changing master node. Changes startup scripts to respect the recovery mode instead of the ZK_QUORUM Adjust travis log file to only log zookeeper errors Updates high availability setup guide Adds TestLogger to leader election tests This closes #1016.
- Loading branch information
1 parent
9e20147
commit 4b0ce08
Showing
187 changed files
with
8,004 additions
and
2,704 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.