Package org.apache.flink.table.runtime.operators.window.tvf.common

Class WindowAggOperator<K,W>

java.lang.Object
org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>
org.apache.flink.table.runtime.operators.TableStreamOperator<org.apache.flink.table.data.RowData>
org.apache.flink.table.runtime.operators.window.tvf.common.WindowAggOperator<K,W>
All Implemented Interfaces:
Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.Triggerable<K,W>, org.apache.flink.streaming.api.operators.YieldingOperator<org.apache.flink.table.data.RowData>
@Internal public final class WindowAggOperator<K,W> extends TableStreamOperator<org.apache.flink.table.data.RowData> implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.Triggerable<K,W>, org.apache.flink.streaming.api.operators.KeyContext
We divide windows into 2 categories: Aligned Windows and Unaligned Windows.

Concept of Aligned Window and Unaligned Window

Aligned Windows are windows have predetermined window boundaries and windows can be divided into finite number of non-overlapping chunks. The boundary of an aligned window is determined independently from the time characteristic of the data stream, or messages it receives. For example, hopping (sliding) window is an aligned window as the window boundaries are predetermined based on the window size and slide. Aligned windows include tumbling, hopping, cumulative windows.

Unaligned Windows are windows determined dynamically based on elements. For example, session window is an unaligned window as the window boundaries are determined based on the messages timestamps and their correlations. Currently, unaligned windows include session window only.

Because aligned windows can be divided into finite number of non-overlapping chunks (a.k.a. slices), which can apply efficient processing to share intermediate results. 

 Window
 |
 +-- Aligned Window (Slicing Window)
 |    |
 |    +-- Tumble (Slice Unshared Window)
 |    |
 |    +-- Hop (Slice Shared Window)
 |    |
 |    +-- Cumulate (Slice Shared Window)
 |
 +-- Unaligned Window (Unslice Window)
      |
      +-- Session

Note: currently, WindowAggOperator doesn't support early-fire and late-arrival. Thus late elements (elements belong to emitted windows) will be simply dropped.

See more in SlicingSyncStateWindowProcessor and UnslicingSyncStateWindowProcessor.

See Also:

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.flink.table.runtime.operators.TableStreamOperator

    TableStreamOperator.ContextImpl

  • Field Summary

    Fields inherited from class org.apache.flink.table.runtime.operators.TableStreamOperator

    ctx, currentWatermark

    Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

    combinedWatermark, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

  • Constructor Summary

    Constructors
    Constructor
    Description
    WindowAggOperator(SyncStateWindowProcessor<W> windowProcessor, boolean isEventTime)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    org.apache.flink.metrics.Counter
    getNumLateRecordsDropped()
     
    org.apache.flink.metrics.Gauge<Long>
    getWatermarkLatency()
     
    void
    initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
     
    void
    onEventTime(org.apache.flink.streaming.api.operators.InternalTimer<K,W> timer)
     
    void
    onProcessingTime(org.apache.flink.streaming.api.operators.InternalTimer<K,W> timer)
     
    void
    open()
     
    void
    prepareSnapshotPreBarrier(long checkpointId)
     
    void
    processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
     
    void
    processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
     
    void
    snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)
     

    Methods inherited from class org.apache.flink.table.runtime.operators.TableStreamOperator

    computeMemorySize, useSplittableTimers

    Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

    beforeInitializeStateHandler, finish, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

    notifyCheckpointAborted, notifyCheckpointComplete

    Methods inherited from interface org.apache.flink.streaming.api.operators.Input

    processLatencyMarker, processRecordAttributes, processWatermark, processWatermarkStatus

    Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

    getCurrentKey, setCurrentKey

    Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

    hasKeyContext

    Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

    setKeyContextElement

    Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

    finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState

  • Constructor Details

  • Method Details

    • open

      public void open() throws Exception
      Specified by:
      open in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
      Overrides:
      open in class TableStreamOperator<org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • initializeState

      public void initializeState(org.apache.flink.runtime.state.StateInitializationContext context) throws Exception
      Specified by:
      initializeState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator
      Overrides:
      initializeState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • snapshotState

      public void snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context) throws Exception
      Specified by:
      snapshotState in interface org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator
      Overrides:
      snapshotState in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • close

      public void close() throws Exception
      Specified by:
      close in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
      Overrides:
      close in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • processElement

      public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element) throws Exception
      Specified by:
      processElement in interface org.apache.flink.streaming.api.operators.Input<K>
      Throws:
      Exception

    • processWatermark

      public void processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark) throws Exception
      Specified by:
      processWatermark in interface org.apache.flink.streaming.api.operators.Input<K>
      Overrides:
      processWatermark in class TableStreamOperator<org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • onEventTime

      public void onEventTime(org.apache.flink.streaming.api.operators.InternalTimer<K,W> timer) throws Exception
      Specified by:
      onEventTime in interface org.apache.flink.streaming.api.operators.Triggerable<K,W>
      Throws:
      Exception

    • onProcessingTime

      public void onProcessingTime(org.apache.flink.streaming.api.operators.InternalTimer<K,W> timer) throws Exception
      Specified by:
      onProcessingTime in interface org.apache.flink.streaming.api.operators.Triggerable<K,W>
      Throws:
      Exception

    • prepareSnapshotPreBarrier

      public void prepareSnapshotPreBarrier(long checkpointId) throws Exception
      Specified by:
      prepareSnapshotPreBarrier in interface org.apache.flink.streaming.api.operators.StreamOperator<K>
      Overrides:
      prepareSnapshotPreBarrier in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
      Throws:
      Exception

    • getNumLateRecordsDropped

      @VisibleForTesting public org.apache.flink.metrics.Counter getNumLateRecordsDropped()

    • getWatermarkLatency

      @VisibleForTesting public org.apache.flink.metrics.Gauge<Long> getWatermarkLatency()