Package org.apache.flink.runtime.io.network.partition

Interface ResourceManagerPartitionTracker

All Known Implementing Classes:
ResourceManagerPartitionTrackerImpl
public interface ResourceManagerPartitionTracker
Utility for tracking and releasing partitions on the ResourceManager.

  • Method Details

    • processTaskExecutorClusterPartitionReport

      void processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId, ClusterPartitionReport clusterPartitionReport)
      Processes ClusterPartitionReport of a task executor. Updates the tracking information for the respective task executor. Any partition no longer being hosted on the task executor is considered lost, corrupting the corresponding data set. For any such data set this method issues partition release calls to all task executors that are hosting partitions of this data set.
      Parameters:
      taskExecutorId - origin of the report
      clusterPartitionReport - partition report

    • processTaskExecutorShutdown

      void processTaskExecutorShutdown(ResourceID taskExecutorId)
      Processes the shutdown of task executor. Removes all tracking information for the given executor, determines datasets that may be corrupted by the shutdown (and implied loss of partitions). For any such data set this method issues partition release calls to all task executors that are hosting partitions of this data set, and issues release calls.
      Parameters:
      taskExecutorId - task executor that shut down

    • releaseClusterPartitions

      CompletableFuture<Void> releaseClusterPartitions(IntermediateDataSetID dataSetId)
      Issues a release calls to all task executors that are hosting partitions of the given data set.
      Parameters:
      dataSetId - data set to release

    • listDataSets

      Map<IntermediateDataSetID,DataSetMetaInfo> listDataSets()
      Returns all data sets for which partitions are being tracked.
      Returns:
      tracked datasets

    • getClusterPartitionShuffleDescriptors

      List<ShuffleDescriptor> getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID)
      Returns all the shuffle descriptors of cluster partitions for the intermediate dataset.
      Parameters:
      dataSetID - The id of the intermediate dataset.
      Returns:
      the shuffle descriptors.