Package org.apache.flink.table.runtime.operators.join.temporal

Class BaseTwoInputStreamOperatorWithStateRetention

java.lang.Object
org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
org.apache.flink.table.runtime.operators.join.temporal.BaseTwoInputStreamOperatorWithStateRetention
All Implemented Interfaces:
Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.Triggerable<Object,org.apache.flink.runtime.state.VoidNamespace>, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.YieldingOperator<org.apache.flink.table.data.RowData>
Direct Known Subclasses:
TemporalProcessTimeJoinOperator, TemporalRowTimeJoinOperator
@Internal public abstract class BaseTwoInputStreamOperatorWithStateRetention extends org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData> implements org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.Triggerable<Object,org.apache.flink.runtime.state.VoidNamespace>
An abstract TwoInputStreamOperator that allows its subclasses to clean up their state based on a TTL. This TTL should be specified in the provided minRetentionTime and maxRetentionTime.

For each known key, this operator registers a timer (in processing time) to fire after the TTL expires. When the timer fires, the subclass can decide which state to cleanup and what further action to take.

This class takes care of maintaining at most one timer per key.

IMPORTANT NOTE TO USERS: When extending this class, do not use processing time timers in your business logic. The reason is that:

1) if your timers collide with clean up timers and you delete them, then state clean-up will not be performed, and

2) (this one is the reason why this class does not allow to override the onProcessingTime()) the onProcessingTime with your logic would be also executed on each clean up timer.

See Also:

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected final boolean
    stateCleaningEnabled
     

    Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

    combinedWatermark, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    BaseTwoInputStreamOperatorWithStateRetention(long minRetentionTime, long maxRetentionTime)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected void
    cleanupLastTimer()
     
    abstract void
    cleanupState(long time)
    The method to be called when a cleanup timer fires.
    final void
    onProcessingTime(org.apache.flink.streaming.api.operators.InternalTimer<Object,org.apache.flink.runtime.state.VoidNamespace> timer)
    The users of this class are not allowed to use processing time timers.
    void
    open()
     
    protected void
    registerProcessingCleanupTimer()
    If the user has specified a minRetentionTime and maxRetentionTime, this method registers a cleanup timer for currentProcessingTime + minRetentionTime.
    boolean
    useSplittableTimers()
     

    Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

    beforeInitializeStateHandler, close, finish, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

    notifyCheckpointAborted, notifyCheckpointComplete

    Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

    getCurrentKey, setCurrentKey

    Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

    hasKeyContext

    Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

    close, finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

    Methods inherited from interface org.apache.flink.streaming.api.operators.Triggerable

    onEventTime

    Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

    processElement1, processElement2, processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus1, processWatermarkStatus2

  • Field Details

    • stateCleaningEnabled

      protected final boolean stateCleaningEnabled

  • Constructor Details

    • BaseTwoInputStreamOperatorWithStateRetention

      protected BaseTwoInputStreamOperatorWithStateRetention(long minRetentionTime, long maxRetentionTime)

  • Method Details

    • useSplittableTimers

      public boolean useSplittableTimers()
      Overrides:
      useSplittableTimers in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

    • open

      public void open() throws Exception
      Specified by:
      open in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>
      Overrides:
      open in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • registerProcessingCleanupTimer

      protected void registerProcessingCleanupTimer() throws IOException
      If the user has specified a minRetentionTime and maxRetentionTime, this method registers a cleanup timer for currentProcessingTime + minRetentionTime.

      When this timer fires, the cleanupState(long) method is called.

      Throws:
      IOException

    • cleanupLastTimer

      protected void cleanupLastTimer() throws IOException
      Throws:
      IOException

    • onProcessingTime

      public final void onProcessingTime(org.apache.flink.streaming.api.operators.InternalTimer<Object,org.apache.flink.runtime.state.VoidNamespace> timer) throws Exception
      The users of this class are not allowed to use processing time timers. See class javadoc.
      Specified by:
      onProcessingTime in interface org.apache.flink.streaming.api.operators.Triggerable<Object,org.apache.flink.runtime.state.VoidNamespace>
      Throws:
      Exception

    • cleanupState

      public abstract void cleanupState(long time)
      The method to be called when a cleanup timer fires.
      Parameters:
      time - The timestamp of the fired timer.