Package org.apache.flink.runtime.io.network.partition

Class ResourceManagerPartitionTrackerImpl

java.lang.Object

org.apache.flink.runtime.io.network.partition.ResourceManagerPartitionTrackerImpl

All Implemented Interfaces:

ResourceManagerPartitionTracker

public class ResourceManagerPartitionTrackerImpl
extends Object
implements ResourceManagerPartitionTracker

Default ResourceManagerPartitionTracker implementation.

Internal tracking info must only be updated upon reception of a ClusterPartitionReport, as the task executor state is the source of truth.

Constructor Summary

Constructors

Constructor	Description
ResourceManagerPartitionTrackerImpl(TaskExecutorClusterPartitionReleaser taskExecutorClusterPartitionReleaser)

Method Summary

Modifier and Type	Method	Description
List<ShuffleDescriptor>	getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID)	Returns all the shuffle descriptors of cluster partitions for the intermediate dataset.
Map<IntermediateDataSetID,DataSetMetaInfo>	listDataSets()	Returns all data sets for which partitions are being tracked.
void	processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId, ClusterPartitionReport clusterPartitionReport)	Processes ClusterPartitionReport of a task executor.
void	processTaskExecutorShutdown(ResourceID taskExecutorId)	Processes the shutdown of task executor.
CompletableFuture<Void>	releaseClusterPartitions(IntermediateDataSetID dataSetId)	Issues a release calls to all task executors that are hosting partitions of the given data set.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ResourceManagerPartitionTrackerImpl

public ResourceManagerPartitionTrackerImpl(TaskExecutorClusterPartitionReleaser taskExecutorClusterPartitionReleaser)

Method Details

processTaskExecutorClusterPartitionReport

public void processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId,
 ClusterPartitionReport clusterPartitionReport)

Description copied from interface: ResourceManagerPartitionTracker

Processes ClusterPartitionReport of a task executor. Updates the tracking information for the respective task executor. Any partition no longer being hosted on the task executor is considered lost, corrupting the corresponding data set. For any such data set this method issues partition release calls to all task executors that are hosting partitions of this data set.

Specified by:

processTaskExecutorClusterPartitionReport in interface ResourceManagerPartitionTracker

Parameters:

taskExecutorId - origin of the report

clusterPartitionReport - partition report

processTaskExecutorShutdown

public void processTaskExecutorShutdown(ResourceID taskExecutorId)

Description copied from interface: ResourceManagerPartitionTracker

Processes the shutdown of task executor. Removes all tracking information for the given executor, determines datasets that may be corrupted by the shutdown (and implied loss of partitions). For any such data set this method issues partition release calls to all task executors that are hosting partitions of this data set, and issues release calls.

Specified by:

processTaskExecutorShutdown in interface ResourceManagerPartitionTracker

Parameters:

taskExecutorId - task executor that shut down

releaseClusterPartitions

public CompletableFuture<Void> releaseClusterPartitions(IntermediateDataSetID dataSetId)

Description copied from interface: ResourceManagerPartitionTracker

Issues a release calls to all task executors that are hosting partitions of the given data set.

Specified by:

releaseClusterPartitions in interface ResourceManagerPartitionTracker

Parameters:

dataSetId - data set to release

getClusterPartitionShuffleDescriptors

public List<ShuffleDescriptor> getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID)

Description copied from interface: ResourceManagerPartitionTracker

Returns all the shuffle descriptors of cluster partitions for the intermediate dataset.

Specified by:

getClusterPartitionShuffleDescriptors in interface ResourceManagerPartitionTracker

Parameters:

dataSetID - The id of the intermediate dataset.

Returns:

the shuffle descriptors.

listDataSets

public Map<IntermediateDataSetID,DataSetMetaInfo> listDataSets()

Description copied from interface: ResourceManagerPartitionTracker

Returns all data sets for which partitions are being tracked.

Specified by:

listDataSets in interface ResourceManagerPartitionTracker

Returns:

tracked datasets