Package org.apache.flink.streaming.api.datastream

Class DataStreamSink<T>

java.lang.Object
org.apache.flink.streaming.api.datastream.DataStreamSink<T>
Type Parameters:
T - The type of the elements in the Stream
Direct Known Subclasses:
CollectStreamSink
@Public public class DataStreamSink<T> extends Object
A Stream Sink. This is used for emitting elements from a streaming topology.

  • Constructor Details

  • Method Details

    • forSink

      @Internal public static <T> DataStreamSink<T> forSink(DataStream<T> inputStream, org.apache.flink.api.connector.sink2.Sink<T> sink, CustomSinkOperatorUidHashes customSinkOperatorUidHashes)

    • getTransformation

      @Internal public org.apache.flink.api.dag.Transformation<T> getTransformation()
      Returns the transformation that contains the actual sink operator of this sink.

    • getLegacyTransformation

      @Internal public LegacySinkTransformation<T> getLegacyTransformation()

    • name

      public DataStreamSink<T> name(String name)
      Sets the name of this sink. This name is used by the visualization and logging during runtime.
      Returns:
      The named sink.

    • uid

      @PublicEvolving public DataStreamSink<T> uid(String uid)
      Sets an ID for this operator.

      The specified ID is used to assign the same operator ID across job submissions (for example when starting a job from a savepoint).

      Important: this ID needs to be unique per transformation and job. Otherwise, job submission will fail.

      Parameters:
      uid - The unique user-specified ID of this transformation.
      Returns:
      The operator with the specified ID.

    • setUidHash

      @PublicEvolving public DataStreamSink<T> setUidHash(String uidHash)
      Sets an user provided hash for this operator. This will be used AS IS the create the JobVertexID.

      The user provided hash is an alternative to the generated hashes, that is considered when identifying an operator through the default hash mechanics fails (e.g. because of changes between Flink versions).

      Important: this should be used as a workaround or for trouble shooting. The provided hash needs to be unique per transformation and job. Otherwise, job submission will fail. Furthermore, you cannot assign user-specified hash to intermediate nodes in an operator chain and trying so will let your job fail.

      A use case for this is in migration between Flink versions or changing the jobs in a way that changes the automatically generated hashes. In this case, providing the previous hashes directly through this method (e.g. obtained from old logs) can help to reestablish a lost mapping from states to their target operator.

      Parameters:
      uidHash - The user provided hash for this operator. This will become the JobVertexID, which is shown in the logs and web ui.
      Returns:
      The operator with the user provided hash.

    • setParallelism

      public DataStreamSink<T> setParallelism(int parallelism)
      Sets the parallelism for this sink. The degree must be higher than zero.
      Parameters:
      parallelism - The parallelism for this sink.
      Returns:
      The sink with set parallelism.

    • setMaxParallelism

      public DataStreamSink<T> setMaxParallelism(int maxParallelism)
      Sets the max parallelism for this sink.

      The maximum parallelism specifies the upper bound for dynamic scaling. The degree must be higher than zero and less than the upper bound.

      Parameters:
      maxParallelism - The max parallelism for this sink.
      Returns:
      The sink with set parallelism.

    • setDescription

      @PublicEvolving public DataStreamSink<T> setDescription(String description)
      Sets the description for this sink.

      Description is used in json plan and web ui, but not in logging and metrics where only name is available. Description is expected to provide detailed information about the sink, while name is expected to be more simple, providing summary information only, so that we can have more user-friendly logging messages and metric tags without losing useful messages for debugging.

      Parameters:
      description - The description for this sink.
      Returns:
      The sink with new description.

    • disableChaining

      @PublicEvolving public DataStreamSink<T> disableChaining()
      Turns off chaining for this operator so thread co-location will not be used as an optimization.

      Chaining can be turned off for the whole job by StreamExecutionEnvironment.disableOperatorChaining() however it is not advised for performance considerations.

      Returns:
      The sink with chaining disabled

    • slotSharingGroup

      @PublicEvolving public DataStreamSink<T> slotSharingGroup(String slotSharingGroup)
      Sets the slot sharing group of this operation. Parallel instances of operations that are in the same slot sharing group will be co-located in the same TaskManager slot, if possible.

      Operations inherit the slot sharing group of input operations if all input operations are in the same slot sharing group and no slot sharing group was explicitly specified.

      Initially an operation is in the default slot sharing group. An operation can be put into the default group explicitly by setting the slot sharing group to "default".

      Parameters:
      slotSharingGroup - The slot sharing group name.

    • slotSharingGroup

      @PublicEvolving public DataStreamSink<T> slotSharingGroup(org.apache.flink.api.common.operators.SlotSharingGroup slotSharingGroup)
      Sets the slot sharing group of this operation. Parallel instances of operations that are in the same slot sharing group will be co-located in the same TaskManager slot, if possible.

      Operations inherit the slot sharing group of input operations if all input operations are in the same slot sharing group and no slot sharing group was explicitly specified.

      Initially an operation is in the default slot sharing group. An operation can be put into the default group explicitly by setting the slot sharing group with name "default".

      Parameters:
      slotSharingGroup - which contains name and its resource spec.