Package org.apache.flink.datastream.impl.utils

Class StreamUtils

java.lang.Object
org.apache.flink.datastream.impl.utils.StreamUtils
public final class StreamUtils extends Object
This class encapsulates the common logic for all type of streams. It can be used to handle things like extract type information, create a new transformation and so on for AbstractDataStream.

  • Constructor Summary

    Constructors
    Constructor
    Description
    StreamUtils()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static <T> DataStreamV2SinkTransformation<T,T>
    addSinkOperator(AbstractDataStream<T> inputStream, org.apache.flink.api.connector.dsv2.Sink<T> sink, org.apache.flink.api.common.typeinfo.TypeInformation<T> typeInformation)
    Add sink operator to the input stream.
    static <T, R, K> org.apache.flink.streaming.api.transformations.OneInputTransformation<T,R>
    getOneInputKeyedTransformation(String operatorName, AbstractDataStream<T> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInformation, org.apache.flink.streaming.api.operators.OneInputStreamOperator<T,R> operator, org.apache.flink.api.java.functions.KeySelector<T,K> keySelector, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType)
    Construct and return a OneInputTransformation from keyed input streams.
    static <T, R> org.apache.flink.streaming.api.transformations.OneInputTransformation<T,R>
    getOneInputTransformation(String operatorName, AbstractDataStream<T> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInformation, org.apache.flink.streaming.api.operators.OneInputStreamOperator<T,R> operator)
    Construct and return a OneInputTransformation from non-keyed input streams.
    static <IN, OUT> org.apache.flink.api.common.typeinfo.TypeInformation<OUT>
    getOutputTypeForOneInputProcessFunction(org.apache.flink.datastream.api.function.OneInputStreamProcessFunction<IN,OUT> processFunction, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inTypeInformation)
    Get the output type information for OneInputStreamProcessFunction from input type information.
    static <IN1, IN2, OUT>
    org.apache.flink.api.common.typeinfo.TypeInformation<OUT>
    getOutputTypeForTwoInputBroadcastProcessFunction(org.apache.flink.datastream.api.function.TwoInputBroadcastStreamProcessFunction<IN1,IN2,OUT> processFunction, org.apache.flink.api.common.typeinfo.TypeInformation<IN1> in1TypeInformation, org.apache.flink.api.common.typeinfo.TypeInformation<IN2> in2TypeInformation)
    Get the output type information for TwoInputBroadcastStreamProcessFunction from two input type information .
    static <IN1, IN2, OUT>
    org.apache.flink.api.common.typeinfo.TypeInformation<OUT>
    getOutputTypeForTwoInputNonBroadcastProcessFunction(org.apache.flink.datastream.api.function.TwoInputNonBroadcastStreamProcessFunction<IN1,IN2,OUT> processFunction, org.apache.flink.api.common.typeinfo.TypeInformation<IN1> in1TypeInformation, org.apache.flink.api.common.typeinfo.TypeInformation<IN2> in2TypeInformation)
    Get the output type information for TwoInputNonBroadcastStreamProcessFunction from two input type information .
    static <IN, OUT1, OUT2>
    org.apache.flink.api.java.tuple.Tuple2<org.apache.flink.api.common.typeinfo.TypeInformation<OUT1>,org.apache.flink.api.common.typeinfo.TypeInformation<OUT2>>
    getOutputTypesForTwoOutputProcessFunction(org.apache.flink.datastream.api.function.TwoOutputStreamProcessFunction<IN,OUT1,OUT2> twoOutputStreamProcessFunction, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inTypeInformation)
    Get output types information for TwoOutputStreamProcessFunction from the input type information.
    static <K, IN1, IN2, OUT>
    org.apache.flink.streaming.api.transformations.TwoInputTransformation<IN1,IN2,OUT>
    getTwoInputTransformation(String operatorName, org.apache.flink.api.dag.Transformation<IN1> inputStream1, org.apache.flink.api.dag.Transformation<IN2> inputStream2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outTypeInformation, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT> operator, org.apache.flink.api.java.functions.KeySelector<IN1,K> keySelector1, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType1, org.apache.flink.api.java.functions.KeySelector<IN2,K> keySelector2, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType2)
    Construct and return a TwoInputTransformation from two input streams.
    static <IN1, IN2, OUT>
    org.apache.flink.streaming.api.transformations.TwoInputTransformation<IN1,IN2,OUT>
    getTwoInputTransformation(String operatorName, AbstractDataStream<IN1> inputStream1, AbstractDataStream<IN2> inputStream2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outTypeInformation, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT> operator)
    Construct and return a TwoInputTransformation from two input streams.
    static <T, R> AbstractDataStream<R>
    transformOneInputOperator(String operatorName, AbstractDataStream<T> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo, org.apache.flink.streaming.api.operators.StreamOperatorFactory<R> operatorFactory)
    Construct and return a new DataStream with one input operator.
    static <K, IN, OUT, W extends org.apache.flink.streaming.api.windowing.windows.Window>
    org.apache.flink.api.dag.Transformation<OUT>
    transformOneInputWindow(org.apache.flink.api.common.ExecutionConfig executionConfig, AbstractDataStream<IN> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inputType, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outType, InternalOneInputWindowStreamProcessFunction<IN,OUT,W> internalWindowFunction, org.apache.flink.api.java.functions.KeySelector<IN,K> keySelector, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType)
    Construct and return a OneInputTransformation of one input window operator.
    static <K, IN1, IN2, OUT, W extends org.apache.flink.streaming.api.windowing.windows.Window>
    org.apache.flink.api.dag.Transformation<OUT>
    transformTwoInputNonBroadcastWindow(org.apache.flink.api.common.ExecutionConfig executionConfig, org.apache.flink.api.dag.Transformation<IN1> inputStream1, org.apache.flink.api.common.typeinfo.TypeInformation<IN1> inputType1, org.apache.flink.api.dag.Transformation<IN2> inputStream2, org.apache.flink.api.common.typeinfo.TypeInformation<IN2> inputType2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outType, InternalTwoInputWindowStreamProcessFunction<IN1,IN2,OUT,W> internalWindowFunction, org.apache.flink.api.java.functions.KeySelector<IN1,K> keySelector1, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType1, org.apache.flink.api.java.functions.KeySelector<IN2,K> keySelector2, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType2)
    Construct and return a OneInputTransformation of one input window operator.
    static <K, IN, OUT1, OUT2, W extends org.apache.flink.streaming.api.windowing.windows.Window>
    org.apache.flink.api.dag.Transformation<OUT1>
    transformTwoOutputWindow(org.apache.flink.api.common.ExecutionConfig executionConfig, AbstractDataStream<IN> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inputType, org.apache.flink.api.common.typeinfo.TypeInformation<OUT1> outType1, org.apache.flink.api.common.typeinfo.TypeInformation<OUT2> outType2, org.apache.flink.util.OutputTag<OUT2> secondOutputTag, InternalTwoOutputWindowStreamProcessFunction<IN,OUT1,OUT2,W> internalWindowFunction, org.apache.flink.api.java.functions.KeySelector<IN,K> keySelector, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType)
    Construct and return a OneInputTransformation of two output window operator.
    static void
    validateStates(Set<org.apache.flink.api.common.state.StateDeclaration> inputStateDeclarations, Set<org.apache.flink.api.common.state.StateDeclaration.RedistributionMode> invalidStateDeclarations)
    Wrap a GlobalStreamImpl with configure handle.
    static <T> org.apache.flink.datastream.api.stream.GlobalStream.ProcessConfigurableAndGlobalStream<T>
    wrapWithConfigureHandle(GlobalStreamImpl<T> stream)
    Wrap a GlobalStreamImpl with configure handle.
    static <K, T> org.apache.flink.datastream.api.stream.KeyedPartitionStream.ProcessConfigurableAndKeyedPartitionStream<K,T>
    wrapWithConfigureHandle(KeyedPartitionStreamImpl<K,T> stream)
    Wrap a KeyedPartitionStreamImpl with configure handle.
    static <T> org.apache.flink.datastream.api.stream.NonKeyedPartitionStream.ProcessConfigurableAndNonKeyedPartitionStream<T>
    wrapWithConfigureHandle(NonKeyedPartitionStreamImpl<T> stream)
    Wrap a NonKeyedPartitionStreamImpl with configure handle.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • StreamUtils

      public StreamUtils()

  • Method Details

    • getOutputTypeForOneInputProcessFunction

      public static <IN, OUT> org.apache.flink.api.common.typeinfo.TypeInformation<OUT> getOutputTypeForOneInputProcessFunction(org.apache.flink.datastream.api.function.OneInputStreamProcessFunction<IN,OUT> processFunction, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inTypeInformation)
      Get the output type information for OneInputStreamProcessFunction from input type information.

    • getOutputTypeForTwoInputNonBroadcastProcessFunction

      public static <IN1, IN2, OUT> org.apache.flink.api.common.typeinfo.TypeInformation<OUT> getOutputTypeForTwoInputNonBroadcastProcessFunction(org.apache.flink.datastream.api.function.TwoInputNonBroadcastStreamProcessFunction<IN1,IN2,OUT> processFunction, org.apache.flink.api.common.typeinfo.TypeInformation<IN1> in1TypeInformation, org.apache.flink.api.common.typeinfo.TypeInformation<IN2> in2TypeInformation)
      Get the output type information for TwoInputNonBroadcastStreamProcessFunction from two input type information .

    • getOutputTypeForTwoInputBroadcastProcessFunction

      public static <IN1, IN2, OUT> org.apache.flink.api.common.typeinfo.TypeInformation<OUT> getOutputTypeForTwoInputBroadcastProcessFunction(org.apache.flink.datastream.api.function.TwoInputBroadcastStreamProcessFunction<IN1,IN2,OUT> processFunction, org.apache.flink.api.common.typeinfo.TypeInformation<IN1> in1TypeInformation, org.apache.flink.api.common.typeinfo.TypeInformation<IN2> in2TypeInformation)
      Get the output type information for TwoInputBroadcastStreamProcessFunction from two input type information .

    • getOutputTypesForTwoOutputProcessFunction

      public static <IN, OUT1, OUT2> org.apache.flink.api.java.tuple.Tuple2<org.apache.flink.api.common.typeinfo.TypeInformation<OUT1>,org.apache.flink.api.common.typeinfo.TypeInformation<OUT2>> getOutputTypesForTwoOutputProcessFunction(org.apache.flink.datastream.api.function.TwoOutputStreamProcessFunction<IN,OUT1,OUT2> twoOutputStreamProcessFunction, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inTypeInformation)
      Get output types information for TwoOutputStreamProcessFunction from the input type information.

    • getOneInputTransformation

      public static <T, R> org.apache.flink.streaming.api.transformations.OneInputTransformation<T,R> getOneInputTransformation(String operatorName, AbstractDataStream<T> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInformation, org.apache.flink.streaming.api.operators.OneInputStreamOperator<T,R> operator)
      Construct and return a OneInputTransformation from non-keyed input streams.

    • getOneInputKeyedTransformation

      public static <T, R, K> org.apache.flink.streaming.api.transformations.OneInputTransformation<T,R> getOneInputKeyedTransformation(String operatorName, AbstractDataStream<T> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInformation, org.apache.flink.streaming.api.operators.OneInputStreamOperator<T,R> operator, org.apache.flink.api.java.functions.KeySelector<T,K> keySelector, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType)
      Construct and return a OneInputTransformation from keyed input streams.

    • getTwoInputTransformation

      public static <IN1, IN2, OUT> org.apache.flink.streaming.api.transformations.TwoInputTransformation<IN1,IN2,OUT> getTwoInputTransformation(String operatorName, AbstractDataStream<IN1> inputStream1, AbstractDataStream<IN2> inputStream2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outTypeInformation, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT> operator)
      Construct and return a TwoInputTransformation from two input streams.

    • getTwoInputTransformation

      public static <K, IN1, IN2, OUT> org.apache.flink.streaming.api.transformations.TwoInputTransformation<IN1,IN2,OUT> getTwoInputTransformation(String operatorName, org.apache.flink.api.dag.Transformation<IN1> inputStream1, org.apache.flink.api.dag.Transformation<IN2> inputStream2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outTypeInformation, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<IN1,IN2,OUT> operator, org.apache.flink.api.java.functions.KeySelector<IN1,K> keySelector1, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType1, org.apache.flink.api.java.functions.KeySelector<IN2,K> keySelector2, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType2)
      Construct and return a TwoInputTransformation from two input streams.

    • transformOneInputOperator

      public static <T, R> AbstractDataStream<R> transformOneInputOperator(String operatorName, AbstractDataStream<T> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<R> outTypeInfo, org.apache.flink.streaming.api.operators.StreamOperatorFactory<R> operatorFactory)
      Construct and return a new DataStream with one input operator.

    • transformOneInputWindow

      public static <K, IN, OUT, W extends org.apache.flink.streaming.api.windowing.windows.Window> org.apache.flink.api.dag.Transformation<OUT> transformOneInputWindow(org.apache.flink.api.common.ExecutionConfig executionConfig, AbstractDataStream<IN> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inputType, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outType, InternalOneInputWindowStreamProcessFunction<IN,OUT,W> internalWindowFunction, org.apache.flink.api.java.functions.KeySelector<IN,K> keySelector, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType)
      Construct and return a OneInputTransformation of one input window operator.

    • transformTwoInputNonBroadcastWindow

      public static <K, IN1, IN2, OUT, W extends org.apache.flink.streaming.api.windowing.windows.Window> org.apache.flink.api.dag.Transformation<OUT> transformTwoInputNonBroadcastWindow(org.apache.flink.api.common.ExecutionConfig executionConfig, org.apache.flink.api.dag.Transformation<IN1> inputStream1, org.apache.flink.api.common.typeinfo.TypeInformation<IN1> inputType1, org.apache.flink.api.dag.Transformation<IN2> inputStream2, org.apache.flink.api.common.typeinfo.TypeInformation<IN2> inputType2, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outType, InternalTwoInputWindowStreamProcessFunction<IN1,IN2,OUT,W> internalWindowFunction, org.apache.flink.api.java.functions.KeySelector<IN1,K> keySelector1, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType1, org.apache.flink.api.java.functions.KeySelector<IN2,K> keySelector2, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType2)
      Construct and return a OneInputTransformation of one input window operator.

    • transformTwoOutputWindow

      public static <K, IN, OUT1, OUT2, W extends org.apache.flink.streaming.api.windowing.windows.Window> org.apache.flink.api.dag.Transformation<OUT1> transformTwoOutputWindow(org.apache.flink.api.common.ExecutionConfig executionConfig, AbstractDataStream<IN> inputStream, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inputType, org.apache.flink.api.common.typeinfo.TypeInformation<OUT1> outType1, org.apache.flink.api.common.typeinfo.TypeInformation<OUT2> outType2, org.apache.flink.util.OutputTag<OUT2> secondOutputTag, InternalTwoOutputWindowStreamProcessFunction<IN,OUT1,OUT2,W> internalWindowFunction, org.apache.flink.api.java.functions.KeySelector<IN,K> keySelector, org.apache.flink.api.common.typeinfo.TypeInformation<K> keyType)
      Construct and return a OneInputTransformation of two output window operator.

    • addSinkOperator

      public static <T> DataStreamV2SinkTransformation<T,T> addSinkOperator(AbstractDataStream<T> inputStream, org.apache.flink.api.connector.dsv2.Sink<T> sink, org.apache.flink.api.common.typeinfo.TypeInformation<T> typeInformation)
      Add sink operator to the input stream.

    • wrapWithConfigureHandle

      public static <T> org.apache.flink.datastream.api.stream.NonKeyedPartitionStream.ProcessConfigurableAndNonKeyedPartitionStream<T> wrapWithConfigureHandle(NonKeyedPartitionStreamImpl<T> stream)
      Wrap a NonKeyedPartitionStreamImpl with configure handle.

    • wrapWithConfigureHandle

      public static <K, T> org.apache.flink.datastream.api.stream.KeyedPartitionStream.ProcessConfigurableAndKeyedPartitionStream<K,T> wrapWithConfigureHandle(KeyedPartitionStreamImpl<K,T> stream)
      Wrap a KeyedPartitionStreamImpl with configure handle.

    • wrapWithConfigureHandle

      public static <T> org.apache.flink.datastream.api.stream.GlobalStream.ProcessConfigurableAndGlobalStream<T> wrapWithConfigureHandle(GlobalStreamImpl<T> stream)
      Wrap a GlobalStreamImpl with configure handle.

    • validateStates

      public static void validateStates(Set<org.apache.flink.api.common.state.StateDeclaration> inputStateDeclarations, Set<org.apache.flink.api.common.state.StateDeclaration.RedistributionMode> invalidStateDeclarations)
      Wrap a GlobalStreamImpl with configure handle.