Package org.apache.flink.runtime.io.network.api.writer

Class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>

java.lang.Object
org.apache.flink.runtime.io.network.api.writer.RecordWriter<T>
Type Parameters:
T - the type of the record that can be emitted with this record writer
All Implemented Interfaces:
AvailabilityProvider
Direct Known Subclasses:
BroadcastRecordWriter, ChannelSelectorRecordWriter
public abstract class RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable> extends Object implements AvailabilityProvider
An abstract record-oriented runtime result writer.

The RecordWriter wraps the runtime's ResultPartitionWriter and takes care of subpartition selection and serializing records into bytes.

  • Field Details

    • DEFAULT_OUTPUT_FLUSH_THREAD_NAME

      @VisibleForTesting public static final String DEFAULT_OUTPUT_FLUSH_THREAD_NAME
      Default name for the output flush thread, if no name with a task reference is given.
      See Also:

    • targetPartition

      protected final ResultPartitionWriter targetPartition

    • numberOfSubpartitions

      protected final int numberOfSubpartitions

    • serializer

      protected final org.apache.flink.core.memory.DataOutputSerializer serializer

    • rng

      protected final Random rng

    • flushAlways

      protected final boolean flushAlways

  • Method Details

    • emit

      public void emit(T record, int targetSubpartition) throws IOException
      Throws:
      IOException

    • emit

      protected void emit(ByteBuffer record, int targetSubpartition) throws IOException
      Throws:
      IOException

    • broadcastEvent

      public void broadcastEvent(AbstractEvent event) throws IOException
      Throws:
      IOException

    • broadcastEvent

      public void broadcastEvent(AbstractEvent event, boolean isPriorityEvent) throws IOException
      Throws:
      IOException

    • alignedBarrierTimeout

      public void alignedBarrierTimeout(long checkpointId) throws IOException
      Throws:
      IOException

    • abortCheckpoint

      public void abortCheckpoint(long checkpointId, CheckpointException cause)

    • serializeRecord

      @VisibleForTesting public static ByteBuffer serializeRecord(org.apache.flink.core.memory.DataOutputSerializer serializer, org.apache.flink.core.io.IOReadableWritable record) throws IOException
      Throws:
      IOException

    • flushAll

      public void flushAll()

    • setMetricGroup

      public void setMetricGroup(TaskIOMetricGroup metrics)
      Sets the metric group for this RecordWriter.

    • getNumberOfSubpartitions

      public int getNumberOfSubpartitions()

    • isSubpartitionDerivable

      public boolean isSubpartitionDerivable()
      Whether the subpartition where an element comes from can be derived from the existing information. If false, the caller of this writer should attach the subpartition information onto an element before writing it to a subpartition, if the element needs this information afterward.

    • getAvailableFuture

      public CompletableFuture<?> getAvailableFuture()
      Specified by:
      getAvailableFuture in interface AvailabilityProvider
      Returns:
      a future that is completed if the respective provider is available.

    • emit

      public abstract void emit(T record) throws IOException
      This is used to send regular records.
      Throws:
      IOException

    • randomEmit

      public void randomEmit(T record) throws IOException
      This is used to send LatencyMarks to a random target subpartition.
      Throws:
      IOException

    • broadcastEmit

      public abstract void broadcastEmit(T record) throws IOException
      This is used to broadcast streaming Watermarks in-band with records.
      Throws:
      IOException

    • close

      public void close()
      Closes the writer. This stops the flushing thread (if there is one).

    • checkErroneous

      protected void checkErroneous() throws IOException
      Throws:
      IOException

    • setMaxOverdraftBuffersPerGate

      public void setMaxOverdraftBuffersPerGate(int maxOverdraftBuffersPerGate)
      Sets the max overdraft buffer size of per gate.