Package org.apache.flink.runtime.io.network.partition.consumer

Class InputChannel

java.lang.Object
org.apache.flink.runtime.io.network.partition.consumer.InputChannel
Direct Known Subclasses:
LocalInputChannel, RecoveredInputChannel, RemoteInputChannel
public abstract class InputChannel extends Object
An input channel consumes a single ResultSubpartitionView.

For each channel, the consumption life cycle is as follows:

  1. requestSubpartitions()
  2. getNextBuffer()
  3. releaseAllResources()

  • Field Details

    • channelInfo

      protected final InputChannelInfo channelInfo
      The info of the input channel to identify it globally within a task.

    • partitionId

      protected final ResultPartitionID partitionId
      The parent partition of the subpartitions consumed by this channel.

    • consumedSubpartitionIndexSet

      protected final ResultSubpartitionIndexSet consumedSubpartitionIndexSet
      The indexes of the subpartitions consumed by this channel.

    • inputGate

      protected final SingleInputGate inputGate

    • initialBackoff

      protected final int initialBackoff
      The initial backoff (in ms).

    • maxBackoff

      protected final int maxBackoff
      The maximum backoff (in ms).

    • numBytesIn

      protected final org.apache.flink.metrics.Counter numBytesIn

    • numBuffersIn

      protected final org.apache.flink.metrics.Counter numBuffersIn

    • currentBackoff

      protected int currentBackoff
      The current backoff (in ms).

  • Constructor Details

    • InputChannel

      protected InputChannel(SingleInputGate inputGate, int channelIndex, ResultPartitionID partitionId, ResultSubpartitionIndexSet consumedSubpartitionIndexSet, int initialBackoff, int maxBackoff, org.apache.flink.metrics.Counter numBytesIn, org.apache.flink.metrics.Counter numBuffersIn)

  • Method Details

    • getChannelIndex

      public int getChannelIndex()
      Returns the index of this channel within its SingleInputGate.

    • getChannelInfo

      public InputChannelInfo getChannelInfo()
      Returns the info of this channel, which uniquely identifies the channel in respect to its operator instance.

    • getPartitionId

      public ResultPartitionID getPartitionId()

    • getConsumedSubpartitionIndexSet

      public ResultSubpartitionIndexSet getConsumedSubpartitionIndexSet()

    • resumeConsumption

      public abstract void resumeConsumption() throws IOException
      After sending a CheckpointBarrier of exactly-once mode, the upstream will be blocked and become unavailable. This method tries to unblock the corresponding upstream and resume data consumption.
      Throws:
      IOException

    • acknowledgeAllRecordsProcessed

      public abstract void acknowledgeAllRecordsProcessed() throws IOException
      When received EndOfData from one channel, it need to acknowledge after this event get processed.
      Throws:
      IOException

    • notifyChannelNonEmpty

      protected void notifyChannelNonEmpty()
      Notifies the owning SingleInputGate that this channel became non-empty.

      This is guaranteed to be called only when a Buffer was added to a previously empty input channel. The notion of empty is atomically consistent with the flag InputChannel.BufferAndAvailability.moreAvailable() when polling the next buffer from this channel.

      Note: When the input channel observes an exception, this method is called regardless of whether the channel was empty before. That ensures that the parent InputGate will always be notified about the exception.

    • notifyPriorityEvent

      public void notifyPriorityEvent(int priorityBufferNumber)

    • notifyBufferAvailable

      protected void notifyBufferAvailable(int numAvailableBuffers) throws IOException
      Throws:
      IOException

    • peekNextBufferSubpartitionId

      public int peekNextBufferSubpartitionId() throws IOException
      Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.
      Throws:
      IOException

    • peekNextBufferSubpartitionIdInternal

      protected abstract int peekNextBufferSubpartitionIdInternal() throws IOException
      Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.
      Throws:
      IOException

    • getNextBuffer

      public abstract Optional<InputChannel.BufferAndAvailability> getNextBuffer() throws IOException, InterruptedException
      Returns the next buffer from the consumed subpartitions or Optional.empty() if there is no data to return.
      Throws:
      IOException
      InterruptedException

    • checkpointStarted

      public void checkpointStarted(CheckpointBarrier barrier) throws CheckpointException
      Called by task thread when checkpointing is started (e.g., any input channel received barrier).
      Throws:
      CheckpointException

    • checkpointStopped

      public void checkpointStopped(long checkpointId)
      Called by task thread on cancel/complete to clean-up temporary data.

    • convertToPriorityEvent

      public void convertToPriorityEvent(int sequenceNumber) throws IOException
      Throws:
      IOException

    • checkError

      protected void checkError() throws IOException
      Checks for an error and rethrows it if one was reported.

      Note: Any PartitionException instances should not be transformed and make sure they are always visible in task failure cause.

      Throws:
      IOException

    • setError

      protected void setError(Throwable cause)
      Atomically sets an error for this channel and notifies the input gate about available data to trigger querying this channel by the task thread.

    • getCurrentBackoff

      protected int getCurrentBackoff()
      Returns the current backoff in ms.

    • increaseBackoff

      protected boolean increaseBackoff()
      Increases the current backoff and returns whether the operation was successful.
      Returns:
      true, iff the operation was successful. Otherwise, false.

    • unsynchronizedGetNumberOfQueuedBuffers

      public int unsynchronizedGetNumberOfQueuedBuffers()

    • unsynchronizedGetSizeOfQueuedBuffers

      public long unsynchronizedGetSizeOfQueuedBuffers()

    • notifyRequiredSegmentId

      public void notifyRequiredSegmentId(int subpartitionId, int segmentId) throws IOException
      Notify the upstream the id of required segment that should be sent to netty connection.
      Parameters:
      subpartitionId - The id of the corresponding subpartition.
      segmentId - The id of required segment.
      Throws:
      IOException