Package org.apache.flink.table.runtime.operators.deduplicate

Class RowTimeMiniBatchDeduplicateFunction

java.lang.Object
org.apache.flink.table.runtime.operators.bundle.MapBundleFunction<K,V,IN,OUT>
org.apache.flink.table.runtime.operators.deduplicate.RowTimeMiniBatchDeduplicateFunction
All Implemented Interfaces:
Serializable, org.apache.flink.api.common.functions.Function
public class RowTimeMiniBatchDeduplicateFunction extends MapBundleFunction<K,V,IN,OUT>
This function is used to get the first or last row for every key partition in miniBatch mode.
See Also:

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected final long
    minRetentionTime
     
    protected org.apache.flink.api.common.state.ValueState<org.apache.flink.table.data.RowData>
    state
     
    protected final org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.table.data.RowData>
    stateType
     

    Fields inherited from class org.apache.flink.table.runtime.operators.bundle.MapBundleFunction

    ctx

  • Constructor Summary

    Constructors
    Constructor
    Description
    RowTimeMiniBatchDeduplicateFunction(InternalTypeInfo<org.apache.flink.table.data.RowData> typeInfo, org.apache.flink.api.common.typeutils.TypeSerializer<org.apache.flink.table.data.RowData> serializer, long minRetentionTime, int rowtimeIndex, boolean generateUpdateBefore, boolean generateInsert, boolean keepLastRow)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    List<org.apache.flink.table.data.RowData>
    addInput(List<org.apache.flink.table.data.RowData> value, org.apache.flink.table.data.RowData input)
    Adds the given input to the given value, returning the new bundle value.
    void
    finishBundle(Map<org.apache.flink.table.data.RowData,List<org.apache.flink.table.data.RowData>> buffer, org.apache.flink.util.Collector<org.apache.flink.table.data.RowData> out)
    Called when a bundle is finished.
    void
    open(ExecutionContext ctx)
     

    Methods inherited from class org.apache.flink.table.runtime.operators.bundle.MapBundleFunction

    close

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • stateType

      protected final org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.table.data.RowData> stateType

    • minRetentionTime

      protected final long minRetentionTime

    • state

      protected org.apache.flink.api.common.state.ValueState<org.apache.flink.table.data.RowData> state

  • Constructor Details

    • RowTimeMiniBatchDeduplicateFunction

      public RowTimeMiniBatchDeduplicateFunction(InternalTypeInfo<org.apache.flink.table.data.RowData> typeInfo, org.apache.flink.api.common.typeutils.TypeSerializer<org.apache.flink.table.data.RowData> serializer, long minRetentionTime, int rowtimeIndex, boolean generateUpdateBefore, boolean generateInsert, boolean keepLastRow)

  • Method Details

    • addInput

      public List<org.apache.flink.table.data.RowData> addInput(@Nullable List<org.apache.flink.table.data.RowData> value, org.apache.flink.table.data.RowData input) throws Exception
      Description copied from class: MapBundleFunction
      Adds the given input to the given value, returning the new bundle value.
      Specified by:
      addInput in class MapBundleFunction<org.apache.flink.table.data.RowData,List<org.apache.flink.table.data.RowData>,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>
      Parameters:
      value - the existing bundle value, maybe null
      input - the given input, not null
      Throws:
      Exception

    • finishBundle

      public void finishBundle(Map<org.apache.flink.table.data.RowData,List<org.apache.flink.table.data.RowData>> buffer, org.apache.flink.util.Collector<org.apache.flink.table.data.RowData> out) throws Exception
      Description copied from class: MapBundleFunction
      Called when a bundle is finished. Transform a bundle to zero, one, or more output elements.
      Specified by:
      finishBundle in class MapBundleFunction<org.apache.flink.table.data.RowData,List<org.apache.flink.table.data.RowData>,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • open

      public void open(ExecutionContext ctx) throws Exception
      Overrides:
      open in class MapBundleFunction<K,V,IN,OUT>
      Throws:
      Exception