Package org.apache.flink.streaming.api.datastream

Class AsyncDataStream

java.lang.Object
org.apache.flink.streaming.api.datastream.AsyncDataStream
@PublicEvolving public class AsyncDataStream extends Object
A helper class to apply AsyncFunction to a data stream. 

 DataStream<String> input = ...
 AsyncFunction<String, Tuple<String, String>> asyncFunc = ...

 AsyncDataStream.orderedWait(input, asyncFunc, timeout, TimeUnit.MILLISECONDS, 100);

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static enum 
    AsyncDataStream.OutputMode
    Output mode for asynchronous operations.

  • Constructor Summary

    Constructors
    Constructor
    Description
    AsyncDataStream()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>
    orderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit)
    Adds an AsyncWaitOperator.
    static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>
    orderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity)
    Adds an AsyncWaitOperator.
    static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>
    orderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity, AsyncRetryStrategy<OUT> asyncRetryStrategy)
    Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
    static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>
    orderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, AsyncRetryStrategy<OUT> asyncRetryStrategy)
    Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
    static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>
    unorderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit)
    Adds an AsyncWaitOperator.
    static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>
    unorderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity)
    Adds an AsyncWaitOperator.
    static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>
    unorderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity, AsyncRetryStrategy<OUT> asyncRetryStrategy)
    Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.
    static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT>
    unorderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, AsyncRetryStrategy<OUT> asyncRetryStrategy)
    Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • AsyncDataStream

      public AsyncDataStream()

  • Method Details

    • unorderedWait

      public static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> unorderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity)
      Adds an AsyncWaitOperator. The order of output stream records may be reordered.
      Type Parameters:
      IN - Type of input record
      OUT - Type of output record
      Parameters:
      in - Input DataStream
      func - AsyncFunction
      timeout - for the asynchronous operation to complete
      timeUnit - of the given timeout
      capacity - The max number of async i/o operation that can be triggered
      Returns:
      A new SingleOutputStreamOperator.

    • unorderedWait

      public static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> unorderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit)
      Adds an AsyncWaitOperator. The order of output stream records may be reordered.
      Type Parameters:
      IN - Type of input record
      OUT - Type of output record
      Parameters:
      in - Input DataStream
      func - AsyncFunction
      timeout - for the asynchronous operation to complete
      timeUnit - of the given timeout
      Returns:
      A new SingleOutputStreamOperator.

    • orderedWait

      public static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> orderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity)
      Adds an AsyncWaitOperator. The order to process input records is guaranteed to be the same as input ones.
      Type Parameters:
      IN - Type of input record
      OUT - Type of output record
      Parameters:
      in - Input DataStream
      func - AsyncFunction
      timeout - for the asynchronous operation to complete
      timeUnit - of the given timeout
      capacity - The max number of async i/o operation that can be triggered
      Returns:
      A new SingleOutputStreamOperator.

    • orderedWait

      public static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> orderedWait(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit)
      Adds an AsyncWaitOperator. The order to process input records is guaranteed to be the same as input ones.
      Type Parameters:
      IN - Type of input record
      OUT - Type of output record
      Parameters:
      in - Input DataStream
      func - AsyncFunction
      timeout - for the asynchronous operation to complete
      timeUnit - of the given timeout
      Returns:
      A new SingleOutputStreamOperator.

    • unorderedWaitWithRetry

      public static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> unorderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, AsyncRetryStrategy<OUT> asyncRetryStrategy)
      Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction. The order of output stream records may be reordered.
      Type Parameters:
      IN - Type of input record
      OUT - Type of output record
      Parameters:
      in - Input DataStream
      func - AsyncFunction
      timeout - from first invoke to final completion of asynchronous operation, may include multiple retries, and will be reset in case of restart
      timeUnit - of the given timeout
      asyncRetryStrategy - The strategy of reattempt async i/o operation that can be triggered
      Returns:
      A new SingleOutputStreamOperator.

    • unorderedWaitWithRetry

      public static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> unorderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity, AsyncRetryStrategy<OUT> asyncRetryStrategy)
      Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction. The order of output stream records may be reordered.
      Type Parameters:
      IN - Type of input record
      OUT - Type of output record
      Parameters:
      in - Input DataStream
      func - AsyncFunction
      timeout - from first invoke to final completion of asynchronous operation, may include multiple retries, and will be reset in case of restart
      timeUnit - of the given timeout
      capacity - The max number of async i/o operation that can be triggered
      asyncRetryStrategy - The strategy of reattempt async i/o operation that can be triggered
      Returns:
      A new SingleOutputStreamOperator.

    • orderedWaitWithRetry

      public static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> orderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, AsyncRetryStrategy<OUT> asyncRetryStrategy)
      Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction. The order to process input records is guaranteed to be the same as * input ones.
      Type Parameters:
      IN - Type of input record
      OUT - Type of output record
      Parameters:
      in - Input DataStream
      func - AsyncFunction
      timeout - from first invoke to final completion of asynchronous operation, may include multiple retries, and will be reset in case of restart
      timeUnit - of the given timeout
      asyncRetryStrategy - The strategy of reattempt async i/o operation that can be triggered
      Returns:
      A new SingleOutputStreamOperator.

    • orderedWaitWithRetry

      public static <IN, OUT> org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator<OUT> orderedWaitWithRetry(org.apache.flink.streaming.api.datastream.DataStream<IN> in, AsyncFunction<IN,OUT> func, long timeout, TimeUnit timeUnit, int capacity, AsyncRetryStrategy<OUT> asyncRetryStrategy)
      Adds an AsyncWaitOperator with an AsyncRetryStrategy to support retry of AsyncFunction. The order to process input records is guaranteed to be the same as * input ones.
      Type Parameters:
      IN - Type of input record
      OUT - Type of output record
      Parameters:
      in - Input DataStream
      func - AsyncFunction
      timeout - from first invoke to final completion of asynchronous operation, may include multiple retries, and will be reset in case of restart
      timeUnit - of the given timeout
      capacity - The max number of async i/o operation that can be triggered
      asyncRetryStrategy - The strategy of reattempt async i/o operation that can be triggered
      Returns:
      A new SingleOutputStreamOperator.