Package org.apache.flink.streaming.api.operators.python.process

Class AbstractExternalDataStreamPythonFunctionOperator<OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalDataStreamPythonFunctionOperator<OUT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.java.typeutils.ResultTypeQueryable<OUT>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, DataStreamPythonFunctionOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.YieldingOperator<OUT>

Direct Known Subclasses:

AbstractExternalOneInputPythonFunctionOperator, AbstractExternalTwoInputPythonFunctionOperator

@Internal
public abstract class AbstractExternalDataStreamPythonFunctionOperator<OUT>
extends AbstractExternalPythonFunctionOperator<OUT>
implements DataStreamPythonFunctionOperator<OUT>

Base class for all Python DataStream operators executed in Python process.

See Also:

• Serialized Form

Field Summary

Fields inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator

pythonFunctionRunner

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

combinedWatermark, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor	Description
AbstractExternalDataStreamPythonFunctionOperator(org.apache.flink.configuration.Configuration config, DataStreamPythonFunctionInfo pythonFunctionInfo, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

Method Summary

Modifier and Type	Method	Description
void	addSideOutputTags(Collection<org.apache.flink.util.OutputTag<?>> outputTags)	Add a collection of OutputTags to the operator.
protected Map<String,FlinkFnApi.CoderInfoDescriptor>	createSideOutputCoderDescriptors()
Map<String,String>	getInternalParameters()
protected org.apache.flink.util.OutputTag<?>	getOutputTagById(String id)
org.apache.flink.api.common.typeinfo.TypeInformation<OUT>	getProducedType()
org.apache.flink.table.functions.python.PythonEnv	getPythonEnv()	Returns the PythonEnv used to create PythonEnvironmentManager..
DataStreamPythonFunctionInfo	getPythonFunctionInfo()	Returns the underlying DataStreamPythonFunctionInfo.
Collection<org.apache.flink.util.OutputTag<?>>	getSideOutputTags()	Gets the OutputTags belongs to the operator.
protected org.apache.flink.api.common.typeutils.TypeSerializer<org.apache.flink.types.Row>	getSideOutputTypeSerializerById(String id)
void	open()
void	setNumPartitions(int numPartitions)	Sets the number of partitions.

Methods inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator

close, createPythonEnvironmentManager, createPythonFunctionRunner, drainUnregisteredTimers, emitResult, emitResults, invokeFinishBundle

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

beforeInitializeStateHandler, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.python.DataStreamPythonFunctionOperator

copy

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

getOperatorAttributes

Constructor Details

AbstractExternalDataStreamPythonFunctionOperator

public AbstractExternalDataStreamPythonFunctionOperator(org.apache.flink.configuration.Configuration config,
 DataStreamPythonFunctionInfo pythonFunctionInfo,
 org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

Method Details

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<OUT>

Overrides:

open in class AbstractExternalPythonFunctionOperator<OUT>

Throws:

Exception

getPythonEnv

public org.apache.flink.table.functions.python.PythonEnv getPythonEnv()

Description copied from class: AbstractExternalPythonFunctionOperator

Returns the PythonEnv used to create PythonEnvironmentManager..

Specified by:

getPythonEnv in class AbstractExternalPythonFunctionOperator<OUT>

getProducedType

public org.apache.flink.api.common.typeinfo.TypeInformation<OUT> getProducedType()

Specified by:

getProducedType in interface org.apache.flink.api.java.typeutils.ResultTypeQueryable<OUT>

getPythonFunctionInfo

public DataStreamPythonFunctionInfo getPythonFunctionInfo()

Description copied from interface: DataStreamPythonFunctionOperator

Returns the underlying DataStreamPythonFunctionInfo.

Specified by:

getPythonFunctionInfo in interface DataStreamPythonFunctionOperator<OUT>

setNumPartitions

public void setNumPartitions(int numPartitions)

Description copied from interface: DataStreamPythonFunctionOperator

Sets the number of partitions. This is used for partitionCustom which takes the number of partitions to partition into as input.

Specified by:

setNumPartitions in interface DataStreamPythonFunctionOperator<OUT>

getInternalParameters

public Map<String,String> getInternalParameters()

getOutputTagById

protected org.apache.flink.util.OutputTag<?> getOutputTagById(String id)

addSideOutputTags

public void addSideOutputTags(Collection<org.apache.flink.util.OutputTag<?>> outputTags)

Description copied from interface: DataStreamPythonFunctionOperator

Add a collection of OutputTags to the operator.

Specified by:

addSideOutputTags in interface DataStreamPythonFunctionOperator<OUT>

getSideOutputTags

public Collection<org.apache.flink.util.OutputTag<?>> getSideOutputTags()

Description copied from interface: DataStreamPythonFunctionOperator

Gets the OutputTags belongs to the operator.

Specified by:

getSideOutputTags in interface DataStreamPythonFunctionOperator<OUT>

createSideOutputCoderDescriptors

protected Map<String,FlinkFnApi.CoderInfoDescriptor> createSideOutputCoderDescriptors()

getSideOutputTypeSerializerById

protected org.apache.flink.api.common.typeutils.TypeSerializer<org.apache.flink.types.Row> getSideOutputTypeSerializerById(String id)