Package org.apache.flink.streaming.api.graph

Class StreamConfig

java.lang.Object
org.apache.flink.streaming.api.graph.StreamConfig
All Implemented Interfaces:
Serializable
@Internal public class StreamConfig extends Object implements Serializable
Internal configuration for a StreamOperator. This is created and populated by the StreamingJobGraphGenerator.
See Also:

  • Field Details

  • Constructor Details

    • StreamConfig

      public StreamConfig(org.apache.flink.configuration.Configuration config)

  • Method Details

    • getConfiguration

      public org.apache.flink.configuration.Configuration getConfiguration()

    • getSerializationFuture

      public CompletableFuture<StreamConfig> getSerializationFuture()

    • triggerSerializationAndReturnFuture

      public CompletableFuture<StreamConfig> triggerSerializationAndReturnFuture(Executor ioExecutor)
      Trigger the object config serialization and return the completable future.

    • serializeAllConfigs

      public void serializeAllConfigs()
      Serialize all object configs synchronously. Only used for operators which need to reconstruct the StreamConfig internally or test.

    • setAndSerializeTransitiveChainedTaskConfigs

      @VisibleForTesting public void setAndSerializeTransitiveChainedTaskConfigs(Map<Integer,StreamConfig> chainedTaskConfigs)

    • setVertexID

      public void setVertexID(Integer vertexID)

    • getVertexID

      public Integer getVertexID()

    • setManagedMemoryFractionOperatorOfUseCase

      public void setManagedMemoryFractionOperatorOfUseCase(org.apache.flink.core.memory.ManagedMemoryUseCase managedMemoryUseCase, double fraction)
      Fraction of managed memory reserved for the given use case that this operator should use.

    • getManagedMemoryFractionOperatorUseCaseOfSlot

      public double getManagedMemoryFractionOperatorUseCaseOfSlot(org.apache.flink.core.memory.ManagedMemoryUseCase managedMemoryUseCase, org.apache.flink.configuration.Configuration jobConfig, org.apache.flink.configuration.Configuration taskManagerConfig, ClassLoader cl)
      Fraction of total managed memory in the slot that this operator should use for the given use case.

    • setTypeSerializerOut

      public void setTypeSerializerOut(org.apache.flink.api.common.typeutils.TypeSerializer<?> serializer)

    • getTypeSerializerOut

      public <T> org.apache.flink.api.common.typeutils.TypeSerializer<T> getTypeSerializerOut(ClassLoader cl)

    • setTypeSerializerSideOut

      public void setTypeSerializerSideOut(org.apache.flink.util.OutputTag<?> outputTag, org.apache.flink.api.common.typeutils.TypeSerializer<?> serializer)

    • getTypeSerializerSideOut

      public <T> org.apache.flink.api.common.typeutils.TypeSerializer<T> getTypeSerializerSideOut(org.apache.flink.util.OutputTag<?> outputTag, ClassLoader cl)

    • setWatermarkDeclarations

      public void setWatermarkDeclarations(byte[] serializedWatermarkDeclarations)

    • getWatermarkDeclarations

      public Set<AbstractInternalWatermarkDeclaration<?>> getWatermarkDeclarations(ClassLoader cl)

    • setupNetworkInputs

      public void setupNetworkInputs(org.apache.flink.api.common.typeutils.TypeSerializer<?>... serializers)

    • setInputs

      public void setInputs(StreamConfig.InputConfig... inputs)

    • getInputs

      public StreamConfig.InputConfig[] getInputs(ClassLoader cl)

    • getTypeSerializerIn1

      @Deprecated public <T> org.apache.flink.api.common.typeutils.TypeSerializer<T> getTypeSerializerIn1(ClassLoader cl)
      Deprecated.

    • getTypeSerializerIn2

      @Deprecated public <T> org.apache.flink.api.common.typeutils.TypeSerializer<T> getTypeSerializerIn2(ClassLoader cl)
      Deprecated.

    • getTypeSerializerIn

      public <T> org.apache.flink.api.common.typeutils.TypeSerializer<T> getTypeSerializerIn(int index, ClassLoader cl)

    • setStreamOperator

      @VisibleForTesting public void setStreamOperator(StreamOperator<?> operator)

    • setStreamOperatorFactory

      public void setStreamOperatorFactory(StreamOperatorFactory<?> factory)

    • getStreamOperator

      @VisibleForTesting public <T extends StreamOperator<?>> T getStreamOperator(ClassLoader cl)

    • getStreamOperatorFactory

      public <T extends StreamOperatorFactory<?>> T getStreamOperatorFactory(ClassLoader cl)

    • getStreamOperatorFactoryClass

      public <T extends StreamOperatorFactory<?>> Class<T> getStreamOperatorFactoryClass(ClassLoader cl)

    • setIterationId

      public void setIterationId(String iterationId)

    • getIterationId

      public String getIterationId()

    • setIterationWaitTime

      public void setIterationWaitTime(long time)

    • getIterationWaitTime

      public long getIterationWaitTime()

    • setNumberOfNetworkInputs

      public void setNumberOfNetworkInputs(int numberOfInputs)

    • getNumberOfNetworkInputs

      public int getNumberOfNetworkInputs()

    • setNumberOfOutputs

      public void setNumberOfOutputs(int numberOfOutputs)

    • getNumberOfOutputs

      public int getNumberOfOutputs()

    • setOperatorNonChainedOutputs

      public void setOperatorNonChainedOutputs(List<NonChainedOutput> nonChainedOutputs)
      Sets the operator level non-chained outputs.

    • getOperatorNonChainedOutputs

      public List<NonChainedOutput> getOperatorNonChainedOutputs(ClassLoader cl)

    • setChainedOutputs

      public void setChainedOutputs(List<StreamEdge> chainedOutputs)

    • getChainedOutputs

      public List<StreamEdge> getChainedOutputs(ClassLoader cl)

    • setInPhysicalEdges

      public void setInPhysicalEdges(List<StreamEdge> inEdges)

    • getInPhysicalEdges

      public List<StreamEdge> getInPhysicalEdges(ClassLoader cl)

    • setCheckpointingEnabled

      public void setCheckpointingEnabled(boolean enabled)

    • isCheckpointingEnabled

      public boolean isCheckpointingEnabled()

    • setCheckpointMode

      public void setCheckpointMode(org.apache.flink.core.execution.CheckpointingMode mode)

    • getCheckpointMode

      public org.apache.flink.core.execution.CheckpointingMode getCheckpointMode()

    • setUnalignedCheckpointsEnabled

      public void setUnalignedCheckpointsEnabled(boolean enabled)

    • isUnalignedCheckpointsEnabled

      public boolean isUnalignedCheckpointsEnabled()

    • setUnalignedCheckpointsSplittableTimersEnabled

      public void setUnalignedCheckpointsSplittableTimersEnabled(boolean enabled)

    • isUnalignedCheckpointsSplittableTimersEnabled

      public boolean isUnalignedCheckpointsSplittableTimersEnabled()

    • isExactlyOnceCheckpointMode

      public boolean isExactlyOnceCheckpointMode()

    • getAlignedCheckpointTimeout

      public Duration getAlignedCheckpointTimeout()

    • setAlignedCheckpointTimeout

      public void setAlignedCheckpointTimeout(Duration alignedCheckpointTimeout)

    • setMaxConcurrentCheckpoints

      public void setMaxConcurrentCheckpoints(int maxConcurrentCheckpoints)

    • getMaxConcurrentCheckpoints

      public int getMaxConcurrentCheckpoints()

    • getMaxSubtasksPerChannelStateFile

      public int getMaxSubtasksPerChannelStateFile()

    • setMaxSubtasksPerChannelStateFile

      public void setMaxSubtasksPerChannelStateFile(int maxSubtasksPerChannelStateFile)

    • setVertexNonChainedOutputs

      public void setVertexNonChainedOutputs(List<NonChainedOutput> nonChainedOutputs)
      Sets the job vertex level non-chained outputs. The given output list must have the same order with JobVertex.getProducedDataSets().

    • getVertexNonChainedOutputs

      public List<NonChainedOutput> getVertexNonChainedOutputs(ClassLoader cl)

    • setTransitiveChainedTaskConfigs

      public void setTransitiveChainedTaskConfigs(Map<Integer,StreamConfig> chainedTaskConfigs)

    • getTransitiveChainedTaskConfigs

      public Map<Integer,StreamConfig> getTransitiveChainedTaskConfigs(ClassLoader cl)

    • getTransitiveChainedTaskConfigsWithSelf

      public Map<Integer,StreamConfig> getTransitiveChainedTaskConfigsWithSelf(ClassLoader cl)

    • setOperatorID

      public void setOperatorID(OperatorID operatorID)

    • getOperatorID

      public OperatorID getOperatorID()

    • setOperatorName

      public void setOperatorName(String name)

    • getOperatorName

      public String getOperatorName()

    • setChainIndex

      public void setChainIndex(int index)

    • getChainIndex

      public int getChainIndex()

    • setStateBackend

      public void setStateBackend(StateBackend backend)

    • setChangelogStateBackendEnabled

      public void setChangelogStateBackendEnabled(org.apache.flink.util.TernaryBoolean enabled)

    • setStateBackendUsesManagedMemory

      @VisibleForTesting public void setStateBackendUsesManagedMemory(boolean usesManagedMemory)

    • setSerializedStateBackend

      public void setSerializedStateBackend(org.apache.flink.util.SerializedValue<StateBackend> serializedStateBackend, boolean useManagedMemory)

    • setSerializedCheckpointStorage

      public void setSerializedCheckpointStorage(org.apache.flink.util.SerializedValue<CheckpointStorage> serializedCheckpointStorage)

    • getStateBackend

      public StateBackend getStateBackend(ClassLoader cl)

    • isChangelogStateBackendEnabled

      public org.apache.flink.util.TernaryBoolean isChangelogStateBackendEnabled(ClassLoader cl)

    • setCheckpointStorage

      @VisibleForTesting public void setCheckpointStorage(CheckpointStorage storage)

    • getCheckpointStorage

      public CheckpointStorage getCheckpointStorage(ClassLoader cl)

    • setTimerServiceProvider

      public void setTimerServiceProvider(InternalTimeServiceManager.Provider timerServiceProvider)

    • getTimerServiceProvider

      public InternalTimeServiceManager.Provider getTimerServiceProvider(ClassLoader cl)

    • setStatePartitioner

      public void setStatePartitioner(int input, org.apache.flink.api.java.functions.KeySelector<?,?> partitioner)

    • getStatePartitioner

      public <IN, K extends Serializable> org.apache.flink.api.java.functions.KeySelector<IN,K> getStatePartitioner(int input, ClassLoader cl)

    • setStateKeySerializer

      public void setStateKeySerializer(org.apache.flink.api.common.typeutils.TypeSerializer<?> serializer)

    • getStateKeySerializer

      public <K> org.apache.flink.api.common.typeutils.TypeSerializer<K> getStateKeySerializer(ClassLoader cl)

    • setChainStart

      public void setChainStart()

    • isChainStart

      public boolean isChainStart()

    • setChainEnd

      public void setChainEnd()

    • isChainEnd

      public boolean isChainEnd()

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • setGraphContainingLoops

      public void setGraphContainingLoops(boolean graphContainingLoops)

    • isGraphContainingLoops

      public boolean isGraphContainingLoops()

    • setAttribute

      public void setAttribute(org.apache.flink.api.common.attribute.Attribute attribute)

    • getAttribute

      public org.apache.flink.api.common.attribute.Attribute getAttribute(ClassLoader cl)

    • clearInitialConfigs

      public void clearInitialConfigs()
      In general, we don't clear any configuration. However, the SERIALIZED_UDF may be very large when operator includes some large objects, the SERIALIZED_UDF is used to create a StreamOperator and usually only needs to be called once. CHAINED_TASK_CONFIG may be large as well due to the StreamConfig of all non-head operators in OperatorChain will be serialized and stored in CHAINED_TASK_CONFIG. They can be cleared to reduce the memory after StreamTask is initialized. If so, TM will have more memory during running. See FLINK-33315 and FLINK-33317 for more information.

    • requiresSorting

      public static boolean requiresSorting(StreamConfig.InputConfig inputConfig)