Package org.apache.flink.connector.base.sink

Class AsyncSinkBase<InputT,RequestEntryT extends Serializable>

java.lang.Object

org.apache.flink.connector.base.sink.AsyncSinkBase<InputT,RequestEntryT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.connector.sink2.Sink<InputT>, org.apache.flink.api.connector.sink2.SupportsWriterState<InputT,BufferedRequestState<RequestEntryT>>

@PublicEvolving
public abstract class AsyncSinkBase<InputT,RequestEntryT extends Serializable>
extends Object
implements org.apache.flink.api.connector.sink2.SupportsWriterState<InputT,BufferedRequestState<RequestEntryT>>, org.apache.flink.api.connector.sink2.Sink<InputT>

A generic sink for destinations that provide an async client to persist data.

The design of the sink focuses on extensibility and a broad support of destinations. The core of the sink is kept generic and free of any connector specific dependencies. The sink is designed to participate in checkpointing to provide at-least once semantics, but it is limited to destinations that provide a client that supports async requests.

Limitations:

• The sink is designed for destinations that provide an async client. Destinations that cannot ingest events in an async fashion cannot be supported by the sink.
• The sink usually persist InputTs in the order they are added to the sink, but reorderings may occur, eg, when RequestEntryTs need to be retried.
• We are not considering support for exactly-once semantics at this point.

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink2.SupportsWriterState

org.apache.flink.api.connector.sink2.SupportsWriterState.WithCompatibleState

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	AsyncSinkBase(ElementConverter<InputT,RequestEntryT> elementConverter, int maxBatchSize, int maxInFlightRequests, int maxBufferedRequests, long maxBatchSizeInBytes, long maxTimeInBufferMS, long maxRecordSizeInBytes)
protected	AsyncSinkBase(ElementConverter<InputT,RequestEntryT> elementConverter, int maxBatchSize, int maxInFlightRequests, int maxBufferedRequests, long maxBatchSizeInBytes, long maxTimeInBufferMS, long maxRecordSizeInBytes, long requestTimeoutMS, boolean failOnTimeout)

Method Summary

Modifier and Type	Method	Description
protected ElementConverter<InputT,RequestEntryT>	getElementConverter()
protected boolean	getFailOnTimeout()
protected int	getMaxBatchSize()
protected long	getMaxBatchSizeInBytes()
protected int	getMaxBufferedRequests()
protected int	getMaxInFlightRequests()
protected long	getMaxRecordSizeInBytes()
protected long	getMaxTimeInBufferMS()
protected long	getRequestTimeoutMS()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.connector.sink2.Sink

createWriter

Methods inherited from interface org.apache.flink.api.connector.sink2.SupportsWriterState

getWriterStateSerializer, restoreWriter

Constructor Details

AsyncSinkBase

protected AsyncSinkBase(ElementConverter<InputT,RequestEntryT> elementConverter,
 int maxBatchSize,
 int maxInFlightRequests,
 int maxBufferedRequests,
 long maxBatchSizeInBytes,
 long maxTimeInBufferMS,
 long maxRecordSizeInBytes)

AsyncSinkBase

protected AsyncSinkBase(ElementConverter<InputT,RequestEntryT> elementConverter,
 int maxBatchSize,
 int maxInFlightRequests,
 int maxBufferedRequests,
 long maxBatchSizeInBytes,
 long maxTimeInBufferMS,
 long maxRecordSizeInBytes,
 long requestTimeoutMS,
 boolean failOnTimeout)

Method Details

getElementConverter

protected ElementConverter<InputT,RequestEntryT> getElementConverter()

getMaxBatchSize

protected int getMaxBatchSize()

getMaxInFlightRequests

protected int getMaxInFlightRequests()

getMaxBufferedRequests

protected int getMaxBufferedRequests()

getMaxBatchSizeInBytes

protected long getMaxBatchSizeInBytes()

getMaxTimeInBufferMS

protected long getMaxTimeInBufferMS()

getMaxRecordSizeInBytes

protected long getMaxRecordSizeInBytes()

getRequestTimeoutMS

protected long getRequestTimeoutMS()

getFailOnTimeout

protected boolean getFailOnTimeout()