Package org.apache.flink.table.runtime.operators.deduplicate

Class ProcTimeMiniBatchDeduplicateKeepFirstRowFunction

java.lang.Object
org.apache.flink.table.runtime.operators.bundle.MapBundleFunction<K,V,IN,OUT>
org.apache.flink.table.runtime.operators.deduplicate.ProcTimeMiniBatchDeduplicateKeepFirstRowFunction
All Implemented Interfaces:
Serializable, org.apache.flink.api.common.functions.Function
public class ProcTimeMiniBatchDeduplicateKeepFirstRowFunction extends MapBundleFunction<K,V,IN,OUT>
This function is used to get the first row for every key partition in miniBatch mode.

The state stores a boolean flag to indicate whether key appears before as an optimization.

See Also:

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected final long
    minRetentionTime
     
    protected org.apache.flink.api.common.state.ValueState<Boolean>
    state
     
    protected final org.apache.flink.api.common.typeinfo.TypeInformation<Boolean>
    stateType
     

    Fields inherited from class org.apache.flink.table.runtime.operators.bundle.MapBundleFunction

    ctx

  • Constructor Summary

    Constructors
    Constructor
    Description
    ProcTimeMiniBatchDeduplicateKeepFirstRowFunction(org.apache.flink.api.common.typeutils.TypeSerializer<org.apache.flink.table.data.RowData> serializer, long stateRetentionTime)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.flink.table.data.RowData
    addInput(org.apache.flink.table.data.RowData value, org.apache.flink.table.data.RowData input)
    Adds the given input to the given value, returning the new bundle value.
    void
    finishBundle(Map<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> buffer, org.apache.flink.util.Collector<org.apache.flink.table.data.RowData> out)
    Called when a bundle is finished.
    void
    open(ExecutionContext ctx)
     

    Methods inherited from class org.apache.flink.table.runtime.operators.bundle.MapBundleFunction

    close

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • stateType

      protected final org.apache.flink.api.common.typeinfo.TypeInformation<Boolean> stateType

    • minRetentionTime

      protected final long minRetentionTime

    • state

      protected org.apache.flink.api.common.state.ValueState<Boolean> state

  • Constructor Details

    • ProcTimeMiniBatchDeduplicateKeepFirstRowFunction

      public ProcTimeMiniBatchDeduplicateKeepFirstRowFunction(org.apache.flink.api.common.typeutils.TypeSerializer<org.apache.flink.table.data.RowData> serializer, long stateRetentionTime)

  • Method Details

    • addInput

      public org.apache.flink.table.data.RowData addInput(@Nullable org.apache.flink.table.data.RowData value, org.apache.flink.table.data.RowData input)
      Description copied from class: MapBundleFunction
      Adds the given input to the given value, returning the new bundle value.
      Specified by:
      addInput in class MapBundleFunction<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>
      Parameters:
      value - the existing bundle value, maybe null
      input - the given input, not null

    • finishBundle

      public void finishBundle(Map<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> buffer, org.apache.flink.util.Collector<org.apache.flink.table.data.RowData> out) throws Exception
      Description copied from class: MapBundleFunction
      Called when a bundle is finished. Transform a bundle to zero, one, or more output elements.
      Specified by:
      finishBundle in class MapBundleFunction<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • open

      public void open(ExecutionContext ctx) throws Exception
      Overrides:
      open in class MapBundleFunction<K,V,IN,OUT>
      Throws:
      Exception