Package org.apache.flink.streaming.api.runners.python.beam

Class BeamDataStreamPythonFunctionRunner

java.lang.Object

org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner

org.apache.flink.streaming.api.runners.python.beam.BeamDataStreamPythonFunctionRunner

All Implemented Interfaces:

AutoCloseable, PythonFunctionRunner

@Internal
public class BeamDataStreamPythonFunctionRunner
extends BeamPythonFunctionRunner

BeamDataStreamPythonFunctionRunner is responsible for starting a beam python harness to execute user defined python function.

Field Summary

Fields inherited from class org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner

inputCoderDescriptor, LOG, mainInputReceiver, outputCoderDescriptor, resultBuffer, sideOutputCoderDescriptors

Constructor Summary

Constructors

Constructor

Description

BeamDataStreamPythonFunctionRunner(org.apache.flink.runtime.execution.Environment environment, String taskName, ProcessPythonEnvironmentManager environmentManager, String headOperatorFunctionUrn, List<FlinkFnApi.UserDefinedDataStreamFunction> userDefinedDataStreamFunctions, FlinkMetricContainer flinkMetricContainer, org.apache.flink.runtime.state.KeyedStateBackend<?> keyedStateBackend, org.apache.flink.runtime.state.OperatorStateBackend operatorStateBackend, org.apache.flink.api.common.typeutils.TypeSerializer<?> keySerializer, org.apache.flink.api.common.typeutils.TypeSerializer<?> namespaceSerializer, TimerRegistration timerRegistration, org.apache.flink.runtime.memory.MemoryManager memoryManager, double managedMemoryFraction, FlinkFnApi.CoderInfoDescriptor inputCoderDescriptor, FlinkFnApi.CoderInfoDescriptor outputCoderDescriptor, FlinkFnApi.CoderInfoDescriptor timerCoderDescriptor, Map<String,FlinkFnApi.CoderInfoDescriptor> sideOutputCoderDescriptors)

Method Summary

Modifier and Type	Method	Description
protected void	buildTransforms(org.apache.beam.model.pipeline.v1.RunnerApi.Components.Builder componentsBuilder)
protected Optional<org.apache.beam.model.pipeline.v1.RunnerApi.Coder>	getOptionalTimerCoderProto()
protected List<org.apache.beam.runners.core.construction.graph.TimerReference>	getTimers(org.apache.beam.model.pipeline.v1.RunnerApi.Components components)

Methods inherited from class org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner

close, createJobBundleFactory, drainUnregisteredTimers, flush, notifyNoMoreResults, open, pollResult, process, processTimer, startBundle, takeResult

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

BeamDataStreamPythonFunctionRunner

public BeamDataStreamPythonFunctionRunner(org.apache.flink.runtime.execution.Environment environment,
 String taskName,
 ProcessPythonEnvironmentManager environmentManager,
 String headOperatorFunctionUrn,
 List<FlinkFnApi.UserDefinedDataStreamFunction> userDefinedDataStreamFunctions,
 @Nullable
 FlinkMetricContainer flinkMetricContainer,
 @Nullable
 org.apache.flink.runtime.state.KeyedStateBackend<?> keyedStateBackend,
 @Nullable
 org.apache.flink.runtime.state.OperatorStateBackend operatorStateBackend,
 @Nullable
 org.apache.flink.api.common.typeutils.TypeSerializer<?> keySerializer,
 @Nullable
 org.apache.flink.api.common.typeutils.TypeSerializer<?> namespaceSerializer,
 @Nullable
 TimerRegistration timerRegistration,
 org.apache.flink.runtime.memory.MemoryManager memoryManager,
 double managedMemoryFraction,
 FlinkFnApi.CoderInfoDescriptor inputCoderDescriptor,
 FlinkFnApi.CoderInfoDescriptor outputCoderDescriptor,
 @Nullable
 FlinkFnApi.CoderInfoDescriptor timerCoderDescriptor,
 Map<String,FlinkFnApi.CoderInfoDescriptor> sideOutputCoderDescriptors)

Method Details

buildTransforms

protected void buildTransforms(org.apache.beam.model.pipeline.v1.RunnerApi.Components.Builder componentsBuilder)

Specified by:

buildTransforms in class BeamPythonFunctionRunner

getTimers

protected List<org.apache.beam.runners.core.construction.graph.TimerReference> getTimers(org.apache.beam.model.pipeline.v1.RunnerApi.Components components)

Specified by:

getTimers in class BeamPythonFunctionRunner

getOptionalTimerCoderProto

protected Optional<org.apache.beam.model.pipeline.v1.RunnerApi.Coder> getOptionalTimerCoderProto()

Specified by:

getOptionalTimerCoderProto in class BeamPythonFunctionRunner