org.apache.flink.runtime.scheduler

Class DefaultScheduler

java.lang.Object

org.apache.flink.runtime.scheduler.SchedulerBase

org.apache.flink.runtime.scheduler.DefaultScheduler

All Implemented Interfaces:

AutoCloseable, CheckpointScheduling, GlobalFailureHandler, SchedulerNG, SchedulerOperations, org.apache.flink.util.AutoCloseableAsync

Direct Known Subclasses:

AdaptiveBatchScheduler

public class DefaultScheduler
extends SchedulerBase
implements SchedulerOperations

The future default scheduler.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.slf4j.Logger	log
protected SchedulingStrategy	schedulingStrategy

Fields inherited from class org.apache.flink.runtime.scheduler.SchedulerBase

executionVertexVersioner, inputsLocationsRetriever, operatorCoordinatorHandler, stateLocationRetriever

Constructor Summary

Constructors

Modifier

Constructor and Description

protected

DefaultScheduler(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, java.util.function.Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction, org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor, ClassLoader userCodeLoader, CheckpointsCleaner checkpointsCleaner, CheckpointRecoveryFactory checkpointRecoveryFactory, JobManagerJobMetricGroup jobManagerJobMetricGroup, SchedulingStrategyFactory schedulingStrategyFactory, FailoverStrategy.Factory failoverStrategyFactory, RestartBackoffTimeStrategy restartBackoffTimeStrategy, ExecutionVertexOperations executionVertexOperations, ExecutionVertexVersioner executionVertexVersioner, ExecutionSlotAllocatorFactory executionSlotAllocatorFactory, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, JobStatusListener jobStatusListener, ExecutionGraphFactory executionGraphFactory, ShuffleMaster<?> shuffleMaster, org.apache.flink.api.common.time.Time rpcTimeout, VertexParallelismStore vertexParallelismStore)

Method Summary

Modifier and Type	Method and Description
void	allocateSlotsAndDeploy(List<ExecutionVertexDeploymentOption> executionVertexDeploymentOptions) Allocate slots and deploy the vertex when slots are returned.
protected void	cancelAllPendingSlotRequestsInternal()
protected long	getNumberOfRestarts()
void	handleGlobalFailure(Throwable error) Handles a global failure.
protected void	notifyPartitionDataAvailableInternal(IntermediateResultPartitionID partitionId)
protected void	startSchedulingInternal()
protected void	updateTaskExecutionStateInternal(ExecutionVertexID executionVertexId, TaskExecutionStateTransition taskExecutionState)

Methods inherited from class org.apache.flink.runtime.scheduler.SchedulerBase

acknowledgeCheckpoint, archiveFromFailureHandlingResult, archiveGlobalFailure, cancel, closeAsync, computeVertexParallelismStore, computeVertexParallelismStore, computeVertexParallelismStore, computeVertexParallelismStore, declineCheckpoint, deliverCoordinationRequestToCoordinator, deliverOperatorEventToCoordinator, failJob, getDefaultMaxParallelism, getExecutionGraph, getExecutionJobVertex, getExecutionVertex, getExecutionVertexId, getExecutionVertexIdOrThrow, getJobGraph, getJobTerminationFuture, getMainThreadExecutor, getResultPartitionAvailabilityChecker, getSchedulingTopology, notifyKvStateRegistered, notifyKvStateUnregistered, notifyPartitionDataAvailable, registerJobMetrics, reportCheckpointMetrics, requestJob, requestJobDetails, requestJobStatus, requestKvStateLocation, requestNextInputSplit, requestPartitionState, resetForNewExecutions, restoreState, setGlobalFailureCause, startCheckpointScheduler, startScheduling, stopCheckpointScheduler, stopWithSavepoint, transitionExecutionGraphState, transitionToRunning, transitionToScheduled, triggerCheckpoint, triggerSavepoint, updateAccumulators, updateTaskExecutionState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.scheduler.SchedulerNG

updateTaskExecutionState

Methods inherited from interface org.apache.flink.util.AutoCloseableAsync

close

Field Detail

log

protected final org.slf4j.Logger log

schedulingStrategy

protected final SchedulingStrategy schedulingStrategy

Constructor Detail

DefaultScheduler

protected DefaultScheduler(org.slf4j.Logger log,
                           JobGraph jobGraph,
                           Executor ioExecutor,
                           org.apache.flink.configuration.Configuration jobMasterConfiguration,
                           java.util.function.Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction,
                           org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor,
                           ClassLoader userCodeLoader,
                           CheckpointsCleaner checkpointsCleaner,
                           CheckpointRecoveryFactory checkpointRecoveryFactory,
                           JobManagerJobMetricGroup jobManagerJobMetricGroup,
                           SchedulingStrategyFactory schedulingStrategyFactory,
                           FailoverStrategy.Factory failoverStrategyFactory,
                           RestartBackoffTimeStrategy restartBackoffTimeStrategy,
                           ExecutionVertexOperations executionVertexOperations,
                           ExecutionVertexVersioner executionVertexVersioner,
                           ExecutionSlotAllocatorFactory executionSlotAllocatorFactory,
                           long initializationTimestamp,
                           org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor,
                           JobStatusListener jobStatusListener,
                           ExecutionGraphFactory executionGraphFactory,
                           ShuffleMaster<?> shuffleMaster,
                           org.apache.flink.api.common.time.Time rpcTimeout,
                           VertexParallelismStore vertexParallelismStore)
                    throws Exception

Throws:

Exception

Method Detail

getNumberOfRestarts

protected long getNumberOfRestarts()

Specified by:

getNumberOfRestarts in class SchedulerBase

cancelAllPendingSlotRequestsInternal

protected void cancelAllPendingSlotRequestsInternal()

Specified by:

cancelAllPendingSlotRequestsInternal in class SchedulerBase

startSchedulingInternal

protected void startSchedulingInternal()

Specified by:

startSchedulingInternal in class SchedulerBase

updateTaskExecutionStateInternal

protected void updateTaskExecutionStateInternal(ExecutionVertexID executionVertexId,
                                                TaskExecutionStateTransition taskExecutionState)

Overrides:

updateTaskExecutionStateInternal in class SchedulerBase

handleGlobalFailure

public void handleGlobalFailure(Throwable error)

Description copied from interface: GlobalFailureHandler

Handles a global failure.

Specified by:

handleGlobalFailure in interface GlobalFailureHandler

Parameters:

error - A cause that describes the global failure.

notifyPartitionDataAvailableInternal

protected void notifyPartitionDataAvailableInternal(IntermediateResultPartitionID partitionId)

Overrides:

notifyPartitionDataAvailableInternal in class SchedulerBase

allocateSlotsAndDeploy

public void allocateSlotsAndDeploy(List<ExecutionVertexDeploymentOption> executionVertexDeploymentOptions)

Description copied from interface: SchedulerOperations

Allocate slots and deploy the vertex when slots are returned. Vertices will be deployed only after all of them have been assigned slots. The given order will be respected, i.e. tasks with smaller indices will be deployed earlier. Only vertices in CREATED state will be accepted. Errors will happen if scheduling Non-CREATED vertices.

Specified by:

allocateSlotsAndDeploy in interface SchedulerOperations

Parameters:

executionVertexDeploymentOptions - The deployment options of tasks to be deployed