org.apache.flink.runtime.highavailability.zookeeper

Class ZooKeeperHaServices

java.lang.Object

org.apache.flink.runtime.highavailability.AbstractHaServices

org.apache.flink.runtime.highavailability.zookeeper.AbstractZooKeeperHaServices

org.apache.flink.runtime.highavailability.zookeeper.ZooKeeperHaServices

All Implemented Interfaces:

AutoCloseable, GloballyCleanableResource, ClientHighAvailabilityServices, HighAvailabilityServices

Deprecated.

in favour of ZooKeeperMultipleComponentLeaderElectionHaServices

@Deprecated
public class ZooKeeperHaServices
extends AbstractZooKeeperHaServices

An implementation of the AbstractHaServices using Apache ZooKeeper. The services store data in ZooKeeper's nodes as illustrated by the following tree structure:

 /flink
      +/cluster_id_1/leader/resource_manager/latch
      |            |                        /connection_info
      |            |       /dispatcher/latch
      |            |                  /connection_info
      |            |       /rest_server/latch
      |            |                   /connection_info
      |            |
      |            |
      |            +jobgraphs/job-id-1
      |            |         /job-id-2
      |            +jobs/job-id-1/leader/latch
      |                 |               /connection_info
      |                 |        /checkpoints/latest
      |                 |                    /latest-1
      |                 |                    /latest-2
      |                 |       /checkpoint_id_counter
      |
      +/cluster_id_2/leader/resource_manager/latch
      |            |                        /connection_info
      |            |       /dispatcher/latch
      |            |                  /connection_info
      |            |       /rest_server/latch
      |            |                   /connection_info
      |            |
      |            +jobgraphs/job-id-2
      |            +jobs/job-id-2/leader/latch
      |                 |               /connection_info
      |                 |        /checkpoints/latest
      |                 |                    /latest-1
      |                 |                    /latest-2
      |                 |       /checkpoint_id_counter
 

The root path "/flink" is configurable via the option HighAvailabilityOptions.HA_ZOOKEEPER_ROOT. This makes sure Flink stores its data under specific subtrees in ZooKeeper, for example to accommodate specific permission.

The "cluster_id" part identifies the data stored for a specific Flink "cluster". This "cluster" can be either a standalone or containerized Flink cluster, or it can be job on a framework like YARN (in a "per-job-cluster" mode).

In case of a "per-job-cluster" on YARN, the cluster-id is generated and configured automatically by the client or dispatcher that submits the Job to YARN.

In the case of a standalone cluster, that cluster-id needs to be configured via HighAvailabilityOptions.HA_CLUSTER_ID. All nodes with the same cluster id will join the same cluster and participate in the execution of the same set of jobs.

Field Summary

Fields inherited from class org.apache.flink.runtime.highavailability.AbstractHaServices

configuration, ioExecutor, logger

Fields inherited from interface org.apache.flink.runtime.highavailability.HighAvailabilityServices

DEFAULT_JOB_ID, DEFAULT_LEADER_ID

Constructor Summary

Constructors

Constructor and Description
ZooKeeperHaServices(CuratorFrameworkWithUnhandledErrorListener curatorFrameworkWrapper, Executor executor, org.apache.flink.configuration.Configuration configuration, BlobStoreService blobStoreService) Deprecated.

Method Summary

Modifier and Type	Method and Description
protected LeaderElectionService	createLeaderElectionService(String leaderPath) Deprecated. Create leader election service with specified leaderName.
protected LeaderRetrievalService	createLeaderRetrievalService(String leaderPath) Deprecated. Create leader retrieval service with specified leaderName.
protected String	getLeaderPathForDispatcher() Deprecated. Get the leader path for Dispatcher.
String	getLeaderPathForJobManager(org.apache.flink.api.common.JobID jobID) Deprecated. Get the leader path for specific JobManager.
protected String	getLeaderPathForResourceManager() Deprecated. Get the leader path for ResourceManager.
protected String	getLeaderPathForRestServer() Deprecated. Get the leader path for RestServer.

Methods inherited from class org.apache.flink.runtime.highavailability.zookeeper.AbstractZooKeeperHaServices

createCheckpointRecoveryFactory, createJobGraphStore, deleteZNode, getCuratorFramework, internalCleanup, internalCleanupJobData, internalClose

Methods inherited from class org.apache.flink.runtime.highavailability.AbstractHaServices

close, closeAndCleanupAllData, createBlobStore, getCheckpointRecoveryFactory, getClusterRestEndpointLeaderElectionService, getClusterRestEndpointLeaderRetriever, getDispatcherLeaderElectionService, getDispatcherLeaderRetriever, getJobGraphStore, getJobManagerLeaderElectionService, getJobManagerLeaderRetriever, getJobManagerLeaderRetriever, getJobResultStore, getResourceManagerLeaderElectionService, getResourceManagerLeaderRetriever, globalCleanupAsync

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.highavailability.HighAvailabilityServices

getWebMonitorLeaderElectionService, getWebMonitorLeaderRetriever

Constructor Detail

ZooKeeperHaServices

public ZooKeeperHaServices(CuratorFrameworkWithUnhandledErrorListener curatorFrameworkWrapper,
                           Executor executor,
                           org.apache.flink.configuration.Configuration configuration,
                           BlobStoreService blobStoreService)
                    throws IOException

Deprecated.

Throws:

IOException

Method Detail

createLeaderElectionService

protected LeaderElectionService createLeaderElectionService(String leaderPath)

Deprecated.

Description copied from class: AbstractHaServices

Create leader election service with specified leaderName.

Specified by:

createLeaderElectionService in class AbstractHaServices

Parameters:

leaderPath - ConfigMap name in Kubernetes or child node path in Zookeeper.

Returns:

Return LeaderElectionService using Zookeeper or Kubernetes.

createLeaderRetrievalService

protected LeaderRetrievalService createLeaderRetrievalService(String leaderPath)

Deprecated.

Description copied from class: AbstractHaServices

Create leader retrieval service with specified leaderName.

Specified by:

createLeaderRetrievalService in class AbstractHaServices

Parameters:

leaderPath - ConfigMap name in Kubernetes or child node path in Zookeeper.

Returns:

Return LeaderRetrievalService using Zookeeper or Kubernetes.

getLeaderPathForResourceManager

protected String getLeaderPathForResourceManager()

Deprecated.

Description copied from class: AbstractHaServices

Get the leader path for ResourceManager.

Specified by:

getLeaderPathForResourceManager in class AbstractHaServices

Returns:

Return the ResourceManager leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

getLeaderPathForDispatcher

protected String getLeaderPathForDispatcher()

Deprecated.

Description copied from class: AbstractHaServices

Get the leader path for Dispatcher.

Specified by:

getLeaderPathForDispatcher in class AbstractHaServices

Returns:

Return the Dispatcher leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

getLeaderPathForJobManager

public String getLeaderPathForJobManager(org.apache.flink.api.common.JobID jobID)

Deprecated.

Description copied from class: AbstractHaServices

Get the leader path for specific JobManager.

Specified by:

getLeaderPathForJobManager in class AbstractHaServices

Parameters:

jobID - job id

Returns:

Return the JobManager leader name for specified job id. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

getLeaderPathForRestServer

protected String getLeaderPathForRestServer()

Deprecated.

Description copied from class: AbstractHaServices

Get the leader path for RestServer.

Specified by:

getLeaderPathForRestServer in class AbstractHaServices

Returns:

Return the RestServer leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.