Package org.apache.flink.streaming.api.operators.python.embedded

Class EmbeddedPythonBatchCoBroadcastProcessOperator<IN1,IN2,OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.embedded.AbstractTwoInputEmbeddedPythonFunctionOperator<IN1,IN2,OUT>

org.apache.flink.streaming.api.operators.python.embedded.EmbeddedPythonCoProcessOperator<IN1,IN2,OUT>

org.apache.flink.streaming.api.operators.python.embedded.EmbeddedPythonBatchCoBroadcastProcessOperator<IN1,IN2,OUT>

Type Parameters:

IN1 - The input type of the regular stream

IN2 - The input type of the broadcast stream

OUT - The output type of the CoBroadcastProcess function

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.java.typeutils.ResultTypeQueryable<OUT>, org.apache.flink.streaming.api.operators.BoundedMultiInput, org.apache.flink.streaming.api.operators.InputSelectable, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, DataStreamPythonFunctionOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT>, org.apache.flink.streaming.api.operators.YieldingOperator<OUT>

@Internal
public class EmbeddedPythonBatchCoBroadcastProcessOperator<IN1,IN2,OUT>
extends EmbeddedPythonCoProcessOperator<IN1,IN2,OUT>
implements org.apache.flink.streaming.api.operators.BoundedMultiInput, org.apache.flink.streaming.api.operators.InputSelectable

The EmbeddedPythonBatchCoBroadcastProcessOperator is responsible for executing the Python CoBroadcastProcess Function under BATCH mode, EmbeddedPythonCoProcessOperator is used under STREAMING mode. This operator forces to run out data from broadcast side first, and then process data from regular side.

See Also:

• Serialized Form

Field Summary

Fields inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractTwoInputEmbeddedPythonFunctionOperator

timestamp

Fields inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator

collector, hasSideOutput, outputTypeInfo, sideOutputContext

Fields inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator

interpreter

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

combinedWatermark, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor	Description
EmbeddedPythonBatchCoBroadcastProcessOperator(org.apache.flink.configuration.Configuration config, DataStreamPythonFunctionInfo pythonFunctionInfo, org.apache.flink.api.common.typeinfo.TypeInformation<IN1> inputTypeInfo1, org.apache.flink.api.common.typeinfo.TypeInformation<IN2> inputTypeInfo2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

Method Summary

Modifier and Type	Method	Description
void	endInput(int inputId)
org.apache.flink.streaming.api.operators.InputSelection	nextSelection()
void	processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN1> element)

Methods inherited from class org.apache.flink.streaming.api.operators.python.embedded.EmbeddedPythonCoProcessOperator

copy, createUserDefinedFunctionsProto, getFunctionContext, getTimerContext, open, processWatermark

Methods inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractTwoInputEmbeddedPythonFunctionOperator

close, openPythonInterpreter, processElement2

Methods inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedDataStreamPythonFunctionOperator

addSideOutputTags, getJobParameters, getProducedType, getPythonFunctionInfo, getSideOutputTags, setNumPartitions

Methods inherited from class org.apache.flink.streaming.api.operators.python.embedded.AbstractEmbeddedPythonFunctionOperator

createPythonEnvironmentManager, invokeFinishBundle

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, drainUnregisteredTimers, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, setCurrentKey

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

beforeInitializeStateHandler, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus1, processWatermarkStatus2

Constructor Details

EmbeddedPythonBatchCoBroadcastProcessOperator

public EmbeddedPythonBatchCoBroadcastProcessOperator(org.apache.flink.configuration.Configuration config,
 DataStreamPythonFunctionInfo pythonFunctionInfo,
 org.apache.flink.api.common.typeinfo.TypeInformation<IN1> inputTypeInfo1,
 org.apache.flink.api.common.typeinfo.TypeInformation<IN2> inputTypeInfo2,
 org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

Method Details

endInput

public void endInput(int inputId)
              throws Exception

Specified by:

endInput in interface org.apache.flink.streaming.api.operators.BoundedMultiInput

Overrides:

endInput in class AbstractTwoInputEmbeddedPythonFunctionOperator<IN1,IN2,OUT>

Throws:

Exception

nextSelection

public org.apache.flink.streaming.api.operators.InputSelection nextSelection()

Specified by:

nextSelection in interface org.apache.flink.streaming.api.operators.InputSelectable

processElement1

public void processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN1> element)
                     throws Exception

Specified by:

processElement1 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT>

Overrides:

processElement1 in class AbstractTwoInputEmbeddedPythonFunctionOperator<IN1,IN2,OUT>

Throws:

Exception