Package org.apache.flink.table.runtime.operators.python

Class AbstractStatelessFunctionOperator<IN,OUT,UDFIN>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator<OUT>

org.apache.flink.table.runtime.operators.python.AbstractOneInputPythonFunctionOperator<IN,OUT>

org.apache.flink.table.runtime.operators.python.AbstractStatelessFunctionOperator<IN,OUT,UDFIN>

Type Parameters:

IN - Type of the input elements.

OUT - Type of the output elements.

UDFIN - Type of the UDF input type.

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<IN>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<IN,OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.YieldingOperator<OUT>

Direct Known Subclasses:

AbstractArrowPythonAggregateFunctionOperator, AbstractPythonScalarFunctionOperator, PythonTableFunctionOperator

@Internal
public abstract class AbstractStatelessFunctionOperator<IN,OUT,UDFIN>
extends AbstractOneInputPythonFunctionOperator<IN,OUT>

Base class for all stream operators to execute Python Stateless Functions.

See Also:

• Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
protected org.apache.flink.core.memory.ByteArrayInputStreamWithPos	bais	Reusable InputStream used to holding the execution results to be deserialized.
protected org.apache.flink.core.memory.DataInputViewStreamWrapper	baisWrapper	InputStream Wrapper.
protected org.apache.flink.core.memory.ByteArrayOutputStreamWithPos	baos	Reusable OutputStream used to holding the serialized input elements.
protected org.apache.flink.core.memory.DataOutputViewStreamWrapper	baosWrapper	OutputStream Wrapper.
protected LinkedList<IN>	forwardedInputQueue	The queue holding the input elements for which the execution results have not been received.
protected final org.apache.flink.table.types.logical.RowType	inputType	The input logical type.
protected final org.apache.flink.table.types.logical.RowType	udfInputType	The user-defined function input logical type.
protected final org.apache.flink.table.types.logical.RowType	udfOutputType	The user-defined function output logical type.

Fields inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator

pythonFunctionRunner

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

combinedWatermark, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor	Description
AbstractStatelessFunctionOperator(org.apache.flink.configuration.Configuration config, org.apache.flink.table.types.logical.RowType inputType, org.apache.flink.table.types.logical.RowType udfInputType, org.apache.flink.table.types.logical.RowType udfOutputType)

Method Summary

Modifier and Type	Method	Description
abstract void	bufferInput(IN input)	Buffers the specified input, it will be used to construct the operator result together with the user-defined function execution result.
abstract FlinkFnApi.CoderInfoDescriptor	createInputCoderInfoDescriptor(org.apache.flink.table.types.logical.RowType runnerInputType)
abstract FlinkFnApi.CoderInfoDescriptor	createOutputCoderInfoDescriptor(org.apache.flink.table.types.logical.RowType runnerOutType)
PythonFunctionRunner	createPythonFunctionRunner()	Creates the PythonFunctionRunner which is responsible for Python user-defined function execution.
abstract FlinkFnApi.UserDefinedFunctions	createUserDefinedFunctionsProto()	Gets the proto representation of the Python user-defined functions to be executed.
abstract UDFIN	getFunctionInput(IN element)
abstract String	getFunctionUrn()
void	open()
void	processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)
abstract void	processElementInternal(IN value)

Methods inherited from class org.apache.flink.table.runtime.operators.python.AbstractOneInputPythonFunctionOperator

endInput

Methods inherited from class org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator

close, createPythonEnvironmentManager, drainUnregisteredTimers, emitResult, emitResults, getPythonEnv, invokeFinishBundle

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

beforeInitializeStateHandler, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processRecordAttributes, processWatermark, processWatermark, processWatermarkStatus

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Field Details

inputType

protected final org.apache.flink.table.types.logical.RowType inputType

The input logical type.

udfInputType

protected final org.apache.flink.table.types.logical.RowType udfInputType

The user-defined function input logical type.

udfOutputType

protected final org.apache.flink.table.types.logical.RowType udfOutputType

The user-defined function output logical type.

forwardedInputQueue

protected transient LinkedList<IN> forwardedInputQueue

The queue holding the input elements for which the execution results have not been received.

bais

protected transient org.apache.flink.core.memory.ByteArrayInputStreamWithPos bais

Reusable InputStream used to holding the execution results to be deserialized.

baisWrapper

protected transient org.apache.flink.core.memory.DataInputViewStreamWrapper baisWrapper

InputStream Wrapper.

baos

protected transient org.apache.flink.core.memory.ByteArrayOutputStreamWithPos baos

Reusable OutputStream used to holding the serialized input elements.

baosWrapper

protected transient org.apache.flink.core.memory.DataOutputViewStreamWrapper baosWrapper

OutputStream Wrapper.

Constructor Details

AbstractStatelessFunctionOperator

public AbstractStatelessFunctionOperator(org.apache.flink.configuration.Configuration config,
 org.apache.flink.table.types.logical.RowType inputType,
 org.apache.flink.table.types.logical.RowType udfInputType,
 org.apache.flink.table.types.logical.RowType udfOutputType)

Method Details

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<IN>

Overrides:

open in class AbstractExternalPythonFunctionOperator<OUT>

Throws:

Exception

processElement

public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)
                    throws Exception

Throws:

Exception

createPythonFunctionRunner

public PythonFunctionRunner createPythonFunctionRunner()
                                                throws IOException

Description copied from class: AbstractExternalPythonFunctionOperator

Creates the PythonFunctionRunner which is responsible for Python user-defined function execution.

Specified by:

createPythonFunctionRunner in class AbstractExternalPythonFunctionOperator<OUT>

Throws:

IOException

bufferInput

public abstract void bufferInput(IN input)
                          throws Exception

Buffers the specified input, it will be used to construct the operator result together with the user-defined function execution result.

Throws:

Exception

getFunctionInput

public abstract UDFIN getFunctionInput(IN element)

createUserDefinedFunctionsProto

public abstract FlinkFnApi.UserDefinedFunctions createUserDefinedFunctionsProto()

Gets the proto representation of the Python user-defined functions to be executed.

getFunctionUrn

public abstract String getFunctionUrn()

createInputCoderInfoDescriptor

public abstract FlinkFnApi.CoderInfoDescriptor createInputCoderInfoDescriptor(org.apache.flink.table.types.logical.RowType runnerInputType)

createOutputCoderInfoDescriptor

public abstract FlinkFnApi.CoderInfoDescriptor createOutputCoderInfoDescriptor(org.apache.flink.table.types.logical.RowType runnerOutType)

processElementInternal

public abstract void processElementInternal(IN value)
                                     throws Exception

Throws:

Exception