Package org.apache.flink.runtime.scheduler.adaptivebatch

Class DefaultVertexParallelismAndInputInfosDecider

java.lang.Object

org.apache.flink.runtime.scheduler.adaptivebatch.DefaultVertexParallelismAndInputInfosDecider

All Implemented Interfaces:

VertexParallelismAndInputInfosDecider

public class DefaultVertexParallelismAndInputInfosDecider
extends Object
implements VertexParallelismAndInputInfosDecider

Default implementation of VertexParallelismAndInputInfosDecider. This implementation will decide parallelism and JobVertexInputInfos as follows:

1. We will first attempt to: evenly distribute data to downstream subtasks, make different downstream subtasks consume roughly the same amount of data.

2. If step 1 fails or is not applicable, we will proceed to: evenly distribute subpartitions to downstream subtasks, make different downstream subtasks consume roughly the same number of subpartitions.

Method Summary

Modifier and Type	Method	Description
int	computeSourceParallelismUpperBound(JobVertexID jobVertexId, int maxParallelism)	Compute source parallelism upper bound for the source vertex.
ParallelismAndInputInfos	decideParallelismAndInputInfosForVertex(JobVertexID jobVertexId, List<BlockingInputInfo> consumedResults, int vertexInitialParallelism, int vertexMinParallelism, int vertexMaxParallelism)	Decide the parallelism and JobVertexInputInfos for this job vertex.
long	getDataVolumePerTask()	Get the average size of data volume to expect each task instance to process.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

decideParallelismAndInputInfosForVertex

public ParallelismAndInputInfos decideParallelismAndInputInfosForVertex(JobVertexID jobVertexId,
 List<BlockingInputInfo> consumedResults,
 int vertexInitialParallelism,
 int vertexMinParallelism,
 int vertexMaxParallelism)

Description copied from interface: VertexParallelismAndInputInfosDecider

Decide the parallelism and JobVertexInputInfos for this job vertex.

Specified by:

decideParallelismAndInputInfosForVertex in interface VertexParallelismAndInputInfosDecider

Parameters:

jobVertexId - The job vertex id

consumedResults - The information of consumed blocking results

vertexInitialParallelism - The initial parallelism of the job vertex. If it's a positive number, it will be respected. If it's not set(equals to ExecutionConfig.PARALLELISM_DEFAULT), a parallelism will be automatically decided for the vertex.

vertexMinParallelism - The min parallelism of the job vertex.

vertexMaxParallelism - The max parallelism of the job vertex.

Returns:

the parallelism and vertex input infos.

computeSourceParallelismUpperBound

public int computeSourceParallelismUpperBound(JobVertexID jobVertexId,
 int maxParallelism)

Description copied from interface: VertexParallelismAndInputInfosDecider

Compute source parallelism upper bound for the source vertex.

Specified by:

computeSourceParallelismUpperBound in interface VertexParallelismAndInputInfosDecider

Parameters:

jobVertexId - The job vertex id

maxParallelism - The max parallelism of the job vertex.

Returns:

the upper bound parallelism for the source vertex.

getDataVolumePerTask

public long getDataVolumePerTask()

Description copied from interface: VertexParallelismAndInputInfosDecider

Get the average size of data volume to expect each task instance to process.

Specified by:

getDataVolumePerTask in interface VertexParallelismAndInputInfosDecider

Returns:

the data volume.