Package org.apache.flink.runtime.io.network.partition.consumer

Class RemoteInputChannel

java.lang.Object
org.apache.flink.runtime.io.network.partition.consumer.InputChannel
org.apache.flink.runtime.io.network.partition.consumer.RemoteInputChannel
public class RemoteInputChannel extends InputChannel
An input channel, which requests a remote partition queue.

  • Constructor Details

  • Method Details

    • requestSubpartitions

      @VisibleForTesting public void requestSubpartitions() throws IOException, InterruptedException
      Requests a remote subpartition.
      Throws:
      IOException
      InterruptedException

    • increaseBackoff

      protected boolean increaseBackoff()
      The remote task manager creates partition request listener and returns PartitionNotFoundException until the listener is timeout, so the backoff should add the timeout milliseconds if it exists.
      Overrides:
      increaseBackoff in class InputChannel
      Returns:
      true, iff the operation was successful. Otherwise, false.

    • peekNextBufferSubpartitionIdInternal

      protected int peekNextBufferSubpartitionIdInternal() throws IOException
      Description copied from class: InputChannel
      Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.
      Specified by:
      peekNextBufferSubpartitionIdInternal in class InputChannel
      Throws:
      IOException

    • getNextBuffer

      public Optional<InputChannel.BufferAndAvailability> getNextBuffer() throws IOException
      Description copied from class: InputChannel
      Returns the next buffer from the consumed subpartitions or Optional.empty() if there is no data to return.
      Specified by:
      getNextBuffer in class InputChannel
      Throws:
      IOException

    • isReleased

      public boolean isReleased()

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getNumberOfAvailableBuffers

      @VisibleForTesting public int getNumberOfAvailableBuffers()

    • getNumberOfRequiredBuffers

      @VisibleForTesting public int getNumberOfRequiredBuffers()

    • getSenderBacklog

      @VisibleForTesting public int getSenderBacklog()

    • getNextReceivedBuffer

      @VisibleForTesting public Buffer getNextReceivedBuffer()

    • notifyBufferAvailable

      public void notifyBufferAvailable(int numAvailableBuffers) throws IOException
      The unannounced credit is increased by the given amount and might notify increased credit to the producer.
      Overrides:
      notifyBufferAvailable in class InputChannel
      Throws:
      IOException

    • resumeConsumption

      public void resumeConsumption() throws IOException
      Description copied from class: InputChannel
      After sending a CheckpointBarrier of exactly-once mode, the upstream will be blocked and become unavailable. This method tries to unblock the corresponding upstream and resume data consumption.
      Specified by:
      resumeConsumption in class InputChannel
      Throws:
      IOException

    • acknowledgeAllRecordsProcessed

      public void acknowledgeAllRecordsProcessed() throws IOException
      Description copied from class: InputChannel
      When received EndOfData from one channel, it need to acknowledge after this event get processed.
      Specified by:
      acknowledgeAllRecordsProcessed in class InputChannel
      Throws:
      IOException

    • getUnannouncedCredit

      public int getUnannouncedCredit()
      Gets the currently unannounced credit.
      Returns:
      Credit which was not announced to the sender yet.

    • getAndResetUnannouncedCredit

      public int getAndResetUnannouncedCredit()
      Gets the unannounced credit and resets it to 0 atomically.
      Returns:
      Credit which was not announced to the sender yet.

    • getNumberOfQueuedBuffers

      public int getNumberOfQueuedBuffers()
      Gets the current number of received buffers which have not been processed yet.
      Returns:
      Buffers queued for processing.

    • unsynchronizedGetNumberOfQueuedBuffers

      public int unsynchronizedGetNumberOfQueuedBuffers()
      Overrides:
      unsynchronizedGetNumberOfQueuedBuffers in class InputChannel

    • unsynchronizedGetSizeOfQueuedBuffers

      public long unsynchronizedGetSizeOfQueuedBuffers()
      Overrides:
      unsynchronizedGetSizeOfQueuedBuffers in class InputChannel

    • unsynchronizedGetExclusiveBuffersUsed

      public int unsynchronizedGetExclusiveBuffersUsed()

    • unsynchronizedGetFloatingBuffersAvailable

      public int unsynchronizedGetFloatingBuffersAvailable()

    • getInputChannelId

      public InputChannelID getInputChannelId()

    • getInitialCredit

      public int getInitialCredit()

    • getBufferProvider

      public BufferProvider getBufferProvider() throws IOException
      Throws:
      IOException

    • requestBuffer

      @Nullable public Buffer requestBuffer()
      Requests buffer from input channel directly for receiving network data. It should always return an available buffer in credit-based mode unless the channel has been released.
      Returns:
      The available buffer.

    • onSenderBacklog

      public void onSenderBacklog(int backlog) throws IOException
      Receives the backlog from the producer's buffer response. If the number of available buffers is less than backlog + initialCredit, it will request floating buffers from the buffer manager, and then notify unannounced credits to the producer.
      Parameters:
      backlog - The number of unsent buffers in the producer's sub partition.
      Throws:
      IOException

    • onBuffer

      public void onBuffer(Buffer buffer, int sequenceNumber, int backlog, int subpartitionId) throws IOException
      Handles the input buffer. This method is taking over the ownership of the buffer and is fully responsible for cleaning it up both on the happy path and in case of an error.
      Throws:
      IOException

    • checkpointStarted

      public void checkpointStarted(CheckpointBarrier barrier) throws CheckpointException
      Spills all queued buffers on checkpoint start. If barrier has already been received (and reordered), spill only the overtaken buffers.
      Overrides:
      checkpointStarted in class InputChannel
      Throws:
      CheckpointException

    • checkpointStopped

      public void checkpointStopped(long checkpointId)
      Description copied from class: InputChannel
      Called by task thread on cancel/complete to clean-up temporary data.
      Overrides:
      checkpointStopped in class InputChannel

    • convertToPriorityEvent

      public void convertToPriorityEvent(int sequenceNumber) throws IOException
      Overrides:
      convertToPriorityEvent in class InputChannel
      Throws:
      IOException

    • onEmptyBuffer

      public void onEmptyBuffer(int sequenceNumber, int backlog) throws IOException
      Throws:
      IOException

    • onFailedPartitionRequest

      public void onFailedPartitionRequest()

    • onError

      public void onError(Throwable cause)

    • notifyRequiredSegmentId

      public void notifyRequiredSegmentId(int subpartitionId, int segmentId) throws IOException
      Description copied from class: InputChannel
      Notify the upstream the id of required segment that should be sent to netty connection.
      Overrides:
      notifyRequiredSegmentId in class InputChannel
      Parameters:
      subpartitionId - The id of the corresponding subpartition.
      segmentId - The id of required segment.
      Throws:
      IOException