Package org.apache.flink.table.runtime.operators.wmassigners

Class ProcTimeMiniBatchAssignerOperator

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

org.apache.flink.table.runtime.operators.wmassigners.ProcTimeMiniBatchAssignerOperator

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.YieldingOperator<org.apache.flink.table.data.RowData>

public class ProcTimeMiniBatchAssignerOperator
extends org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback

A stream operator that emits mini-batch marker in a given period. This mini-batch assigner works in processing time, which means the mini-batch marker is generated in the given period using the processing time. The downstream operators will trigger mini-batch once the received mini-batch id advanced.

NOTE: currently, we use Watermark to represents the mini-batch marker.

The difference between this operator and RowTimeMiniBatchAssginerOperator is that, this operator generates watermarks by itself using processing time, but the other forwards watermarks from upstream.

See Also:

• Serialized Form

Field Summary

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

combinedWatermark, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor	Description
ProcTimeMiniBatchAssignerOperator(long intervalMs)

Method Summary

Modifier and Type	Method	Description
void	onProcessingTime(long timestamp)
void	open()
void	processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
void	processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)	Override the base implementation to completely ignore watermarks propagated from upstream (we rely only on the WatermarkStrategyWithPeriodicWatermarks to emit watermarks from here).

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

beforeInitializeStateHandler, close, finish, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processRecordAttributes, processWatermark, processWatermarkStatus

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Constructor Details

ProcTimeMiniBatchAssignerOperator

public ProcTimeMiniBatchAssignerOperator(long intervalMs)

Method Details

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

open in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

processElement

public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                    throws Exception

Specified by:

processElement in interface org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>

Throws:

Exception

onProcessingTime

public void onProcessingTime(long timestamp)
                      throws Exception

Specified by:

onProcessingTime in interface org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback

Throws:

Exception

processWatermark

public void processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception

Override the base implementation to completely ignore watermarks propagated from upstream (we rely only on the WatermarkStrategyWithPeriodicWatermarks to emit watermarks from here).

Specified by:

processWatermark in interface org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>

Overrides:

processWatermark in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception