org.apache.flink.runtime.scheduler.adaptivebatch

Class AdaptiveBatchScheduler

java.lang.Object

org.apache.flink.runtime.scheduler.SchedulerBase

org.apache.flink.runtime.scheduler.DefaultScheduler

org.apache.flink.runtime.scheduler.adaptivebatch.AdaptiveBatchScheduler

All Implemented Interfaces:

AutoCloseable, CheckpointScheduling, GlobalFailureHandler, SchedulerNG, SchedulerOperations, org.apache.flink.util.AutoCloseableAsync

public class AdaptiveBatchScheduler
extends DefaultScheduler
implements SchedulerOperations

This scheduler decides the parallelism of JobVertex according to the data volume it consumes. A dynamically built up ExecutionGraph is used for this purpose.

Field Summary

Fields inherited from class org.apache.flink.runtime.scheduler.DefaultScheduler

log, schedulingStrategy

Fields inherited from class org.apache.flink.runtime.scheduler.SchedulerBase

executionVertexVersioner, inputsLocationsRetriever, operatorCoordinatorHandler, stateLocationRetriever

Method Summary

Modifier and Type	Method and Description
static VertexParallelismStore	computeVertexParallelismStoreForDynamicGraph(Iterable<JobVertex> vertices, int defaultMaxParallelism) Compute the VertexParallelismStore for all given vertices in a dynamic graph, which will set defaults and ensure that the returned store contains valid parallelisms, with the configured default max parallelism.
void	startSchedulingInternal()
protected void	updateTaskExecutionStateInternal(ExecutionVertexID executionVertexId, TaskExecutionStateTransition taskExecutionState)

Methods inherited from class org.apache.flink.runtime.scheduler.DefaultScheduler

allocateSlotsAndDeploy, cancelAllPendingSlotRequestsInternal, getNumberOfRestarts, handleGlobalFailure, notifyPartitionDataAvailableInternal

Methods inherited from class org.apache.flink.runtime.scheduler.SchedulerBase

acknowledgeCheckpoint, archiveFromFailureHandlingResult, archiveGlobalFailure, cancel, closeAsync, computeVertexParallelismStore, computeVertexParallelismStore, computeVertexParallelismStore, computeVertexParallelismStore, declineCheckpoint, deliverCoordinationRequestToCoordinator, deliverOperatorEventToCoordinator, failJob, getDefaultMaxParallelism, getExecutionGraph, getExecutionJobVertex, getExecutionVertex, getExecutionVertexId, getExecutionVertexIdOrThrow, getJobGraph, getJobTerminationFuture, getMainThreadExecutor, getResultPartitionAvailabilityChecker, getSchedulingTopology, notifyKvStateRegistered, notifyKvStateUnregistered, notifyPartitionDataAvailable, registerJobMetrics, reportCheckpointMetrics, requestJob, requestJobDetails, requestJobStatus, requestKvStateLocation, requestNextInputSplit, requestPartitionState, resetForNewExecutions, restoreState, setGlobalFailureCause, startCheckpointScheduler, startScheduling, stopCheckpointScheduler, stopWithSavepoint, transitionExecutionGraphState, transitionToRunning, transitionToScheduled, triggerCheckpoint, triggerSavepoint, updateAccumulators, updateTaskExecutionState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.scheduler.SchedulerOperations

allocateSlotsAndDeploy

Methods inherited from interface org.apache.flink.runtime.scheduler.SchedulerNG

updateTaskExecutionState

Methods inherited from interface org.apache.flink.util.AutoCloseableAsync

close

Method Detail

startSchedulingInternal

public void startSchedulingInternal()

Overrides:

startSchedulingInternal in class DefaultScheduler

updateTaskExecutionStateInternal

protected void updateTaskExecutionStateInternal(ExecutionVertexID executionVertexId,
                                                TaskExecutionStateTransition taskExecutionState)

Overrides:

updateTaskExecutionStateInternal in class DefaultScheduler

computeVertexParallelismStoreForDynamicGraph

@VisibleForTesting
public static VertexParallelismStore computeVertexParallelismStoreForDynamicGraph(Iterable<JobVertex> vertices,
                                                                                                     int defaultMaxParallelism)

Compute the VertexParallelismStore for all given vertices in a dynamic graph, which will set defaults and ensure that the returned store contains valid parallelisms, with the configured default max parallelism.

Parameters:

vertices - the vertices to compute parallelism for

defaultMaxParallelism - the global default max parallelism

Returns:

the computed parallelism store