Package org.apache.flink.table.connector.sink

Interface DataStreamSinkProvider

All Superinterfaces:
org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider, org.apache.flink.table.connector.ParallelismProvider
@PublicEvolving public interface DataStreamSinkProvider extends org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider, org.apache.flink.table.connector.ParallelismProvider
Provider that consumes a Java DataStream as a runtime implementation for DynamicTableSink.

Note: This provider is only meant for advanced connector developers. Usually, a sink should consist of a single entity expressed via SinkFunctionProvider, or OutputFormatProvider. When using a DataStream an implementer needs to pay attention to how changes are shuffled to not mess up the changelog per parallel subtask.

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.flink.streaming.api.datastream.DataStreamSink<?>
    consumeDataStream(ProviderContext providerContext, org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> dataStream)
    Consumes the given Java DataStream and returns the sink transformation DataStreamSink.
    default Optional<Integer>
    getParallelism()

  • Method Details

    • consumeDataStream

      org.apache.flink.streaming.api.datastream.DataStreamSink<?> consumeDataStream(ProviderContext providerContext, org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> dataStream)
      Consumes the given Java DataStream and returns the sink transformation DataStreamSink.

      Note: If the CompiledPlan feature should be supported, this method MUST set a unique identifier for each transformation/operator in the data stream. This enables stateful Flink version upgrades for streaming jobs. The identifier is used to map state back from a savepoint to an actual operator in the topology. The framework can generate topology-wide unique identifiers with ProviderContext.generateUid(String).

      See Also:
      • SingleOutputStreamOperator.uid(String)

    • getParallelism

      default Optional<Integer> getParallelism()

      Note: If a custom parallelism is returned and consumeDataStream(ProviderContext, DataStream) applies multiple transformations, make sure to set the same custom parallelism to each operator to not mess up the changelog.

      Specified by:
      getParallelism in interface org.apache.flink.table.connector.ParallelismProvider