Package org.apache.flink.runtime.source.coordinator

Class SplitAssignmentTracker<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

java.lang.Object
org.apache.flink.runtime.source.coordinator.SplitAssignmentTracker<SplitT>
@Internal public class SplitAssignmentTracker<SplitT extends org.apache.flink.api.connector.source.SourceSplit> extends Object
A class that is responsible for tracking the past split assignments made by SplitEnumerator.

  • Constructor Details

    • SplitAssignmentTracker

      public SplitAssignmentTracker()

  • Method Details

    • onCheckpoint

      public void onCheckpoint(long checkpointId) throws Exception
      Behavior of SplitAssignmentTracker on checkpoint. Tracker will mark uncheckpointed assignment as checkpointed with current checkpoint ID.
      Parameters:
      checkpointId - the id of the ongoing checkpoint
      Throws:
      Exception

    • snapshotState

      public byte[] snapshotState(org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer) throws Exception
      Take a snapshot of the split assignments.
      Throws:
      Exception

    • restoreState

      public void restoreState(org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer, byte[] assignmentData) throws Exception
      Restore the state of the SplitAssignmentTracker.
      Parameters:
      splitSerializer - The serializer of the splits.
      assignmentData - The state of the SplitAssignmentTracker.
      Throws:
      Exception - when the state deserialization fails.

    • onCheckpointComplete

      public void onCheckpointComplete(long checkpointId)
      when a checkpoint has been successfully made, this method is invoked to clean up the assignment history before this successful checkpoint.
      Parameters:
      checkpointId - the id of the successful checkpoint.

    • recordSplitAssignment

      public void recordSplitAssignment(org.apache.flink.api.connector.source.SplitsAssignment<SplitT> splitsAssignment)
      Record a new split assignment.
      Parameters:
      splitsAssignment - the new split assignment.

    • getAndRemoveUncheckpointedAssignment

      public List<SplitT> getAndRemoveUncheckpointedAssignment(int subtaskId, long restoredCheckpointId)
      This method is invoked when a source reader fails over. In this case, the source reader will restore its split assignment to the last successful checkpoint. Any split assignment to that source reader after the last successful checkpoint will be lost on the source reader side as if those splits were never assigned. To handle this case, the coordinator needs to find those splits and return them back to the SplitEnumerator for re-assignment.
      Parameters:
      subtaskId - the subtask id of the reader that failed over.
      restoredCheckpointId - the ID of the checkpoint that the reader was restored to.
      Returns:
      A list of splits that needs to be added back to the SplitEnumerator.