Package org.apache.flink.connector.base.sink

Class AsyncSinkBaseBuilder<InputT,RequestEntryT extends Serializable,ConcreteBuilderT extends AsyncSinkBaseBuilder<?,?,?>>

java.lang.Object
org.apache.flink.connector.base.sink.AsyncSinkBaseBuilder<InputT,RequestEntryT,ConcreteBuilderT>
Type Parameters:
InputT - type of elements that should be persisted in the destination
RequestEntryT - type of payload that contains the element and additional metadata that is required to submit a single element to the destination
ConcreteBuilderT - type of concrete implementation of this builder class
@PublicEvolving public abstract class AsyncSinkBaseBuilder<InputT,RequestEntryT extends Serializable,ConcreteBuilderT extends AsyncSinkBaseBuilder<?,?,?>> extends Object
Abstract builder for constructing a concrete implementation of AsyncSinkBase.

  • Constructor Details

    • AsyncSinkBaseBuilder

      public AsyncSinkBaseBuilder()

  • Method Details

    • setMaxBatchSize

      public ConcreteBuilderT setMaxBatchSize(int maxBatchSize)
      Parameters:
      maxBatchSize - maximum number of elements that may be passed in a list to be written downstream.
      Returns:
      AsyncSinkBaseBuilder itself

    • setMaxInFlightRequests

      public ConcreteBuilderT setMaxInFlightRequests(int maxInFlightRequests)
      Parameters:
      maxInFlightRequests - maximum number of uncompleted calls to submitRequestEntries that the SinkWriter will allow at any given point. Once this point has reached, writes and callbacks to add elements to the buffer may block until one or more requests to submitRequestEntries completes.
      Returns:
      AsyncSinkBaseBuilder itself

    • setMaxBufferedRequests

      public ConcreteBuilderT setMaxBufferedRequests(int maxBufferedRequests)
      Parameters:
      maxBufferedRequests - the maximum buffer length. Callbacks to add elements to the buffer and calls to write will block if this length has been reached and will only unblock if elements from the buffer have been removed for flushing.
      Returns:
      AsyncSinkBaseBuilder itself

    • setMaxBatchSizeInBytes

      public ConcreteBuilderT setMaxBatchSizeInBytes(long maxBatchSizeInBytes)
      Parameters:
      maxBatchSizeInBytes - a flush will be attempted if the most recent call to write introduces an element to the buffer such that the total size of the buffer is greater than or equal to this threshold value. If this happens, the maximum number of elements from the head of the buffer will be selected, that is smaller than maxBatchSizeInBytes in size will be flushed.
      Returns:
      AsyncSinkBaseBuilder itself

    • setMaxTimeInBufferMS

      public ConcreteBuilderT setMaxTimeInBufferMS(long maxTimeInBufferMS)
      Parameters:
      maxTimeInBufferMS - the maximum amount of time an element may remain in the buffer. In most cases elements are flushed as a result of the batch size (in bytes or number) being reached or during a snapshot. However, there are scenarios where an element may remain in the buffer forever or a long period of time. To mitigate this, a timer is constantly active in the buffer such that: while the buffer is not empty, it will flush every maxTimeInBufferMS milliseconds.
      Returns:
      AsyncSinkBaseBuilder itself

    • setMaxRecordSizeInBytes

      public ConcreteBuilderT setMaxRecordSizeInBytes(long maxRecordSizeInBytes)
      Parameters:
      maxRecordSizeInBytes - the maximum size of each records in bytes. If a record larger than this is passed to the sink, it will throw an IllegalArgumentException.
      Returns:
      AsyncSinkBaseBuilder itself

    • build

      public abstract AsyncSinkBase<InputT,RequestEntryT> build()
      Builds the Sink with the settings applied to this builder.

    • getMaxBatchSize

      protected Integer getMaxBatchSize()

    • getMaxInFlightRequests

      protected Integer getMaxInFlightRequests()

    • getMaxBufferedRequests

      protected Integer getMaxBufferedRequests()

    • getMaxBatchSizeInBytes

      protected Long getMaxBatchSizeInBytes()

    • getMaxTimeInBufferMS

      protected Long getMaxTimeInBufferMS()

    • getMaxRecordSizeInBytes

      protected Long getMaxRecordSizeInBytes()