Package org.apache.flink.streaming.runtime.operators.sink.committables

Interface CheckpointCommittableManager<CommT>

Type Parameters:
CommT - type of the committable
@Internal public interface CheckpointCommittableManager<CommT>
A CheckpointCommittableManager collects committables for one checkpoint across potentially multiple upstream subtasks.

While it collects committables from multiple upstream subtasks, it belongs to exactly one committer subtask.

Each upstream subtask of this particular checkpoint is represented by a SubtaskCommittableManager.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    commit(org.apache.flink.api.connector.sink2.Committer<CommT> committer, int maxRetries)
    Commits all due committables if all respective committables of the specific subtask and checkpoint have been received.
    long
    getCheckpointId()
    Returns the checkpoint id in which the committables were created.
    int
    getNumberOfSubtasks()
    Returns the number of upstream subtasks belonging to the checkpoint.
    int
    getNumFailed()
    Returns the number of committables that have failed with a known error.
    Collection<CommT>
    getSuccessfulCommittables()
    Returns the number of committables that have been successfully committed; that is, the corresponding Committer.CommitRequest was not used to signal an error of any kind (retryable or not).
    boolean
    hasGloballyReceivedAll()
    Returns true if all committables of all upstream subtasks arrived, which is only guaranteed to happen if the DOP of the caller is 1.
    boolean
    isFinished()
     

  • Method Details

    • getCheckpointId

      long getCheckpointId()
      Returns the checkpoint id in which the committables were created.
      Returns:
      checkpoint id

    • getNumberOfSubtasks

      int getNumberOfSubtasks()
      Returns the number of upstream subtasks belonging to the checkpoint.

    • isFinished

      boolean isFinished()

    • hasGloballyReceivedAll

      boolean hasGloballyReceivedAll()
      Returns true if all committables of all upstream subtasks arrived, which is only guaranteed to happen if the DOP of the caller is 1.

    • commit

      void commit(org.apache.flink.api.connector.sink2.Committer<CommT> committer, int maxRetries) throws IOException, InterruptedException
      Commits all due committables if all respective committables of the specific subtask and checkpoint have been received.
      Parameters:
      committer - used to commit to the external system
      maxRetries -
      Throws:
      IOException
      InterruptedException

    • getSuccessfulCommittables

      Collection<CommT> getSuccessfulCommittables()
      Returns the number of committables that have been successfully committed; that is, the corresponding Committer.CommitRequest was not used to signal an error of any kind (retryable or not).
      Returns:
      number of successful committables

    • getNumFailed

      int getNumFailed()
      Returns the number of committables that have failed with a known error. By the current semantics of Committer.CommitRequest.signalFailedWithKnownReason(Throwable) discards the committable but proceeds processing. The returned number should be emitted downstream in a CommittableSummary, such that downstream can assess if all committables have been processed.
      Returns:
      number of failed committables