Package org.apache.flink.streaming.runtime.operators.sink.committables

Class CommittableCollector<CommT>

java.lang.Object

org.apache.flink.streaming.runtime.operators.sink.committables.CommittableCollector<CommT>

Type Parameters:

CommT - type of committable

@Internal
public class CommittableCollector<CommT>
extends Object

This class is responsible to book-keep the committing progress across checkpoints and upstream subtasks.

Each checkpoint in turn is handled by a CheckpointCommittableManager.

Constructor Summary

Constructors

Constructor	Description
CommittableCollector(org.apache.flink.metrics.groups.SinkCommitterMetricGroup metricGroup)

Method Summary

Modifier and Type	Method	Description
void	addMessage(CommittableMessage<CommT> message)	Adds a CommittableMessage to the collector to hold it until emission.
CommittableCollector<CommT>	copy()	Returns a new committable collector that deep copies all internals.
boolean	equals(Object o)
Collection<? extends CheckpointCommittableManager<CommT>>	getCheckpointCommittablesUpTo(long checkpointId)	Returns all CheckpointCommittableManager until the requested checkpoint id.
Optional<CheckpointCommittableManager<CommT>>	getEndOfInputCommittable()	Returns CheckpointCommittableManager belonging to the last input.
int	hashCode()
boolean	isFinished()	Returns whether all CheckpointCommittableManager currently hold by the collector are either committed or failed.
void	merge(CommittableCollector<CommT> cc)	Merges all information from an external collector into this collector.
static <CommT> CommittableCollector<CommT>	of(org.apache.flink.metrics.groups.SinkCommitterMetricGroup metricGroup)	Creates a CommittableCollector based on the current runtime information.
void	remove(CheckpointCommittableManager<CommT> manager)	Removes the manager for a specific checkpoint and all it's metadata.
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

CommittableCollector

public CommittableCollector(org.apache.flink.metrics.groups.SinkCommitterMetricGroup metricGroup)

Method Details

of

public static <CommT> CommittableCollector<CommT> of(org.apache.flink.metrics.groups.SinkCommitterMetricGroup metricGroup)

Creates a CommittableCollector based on the current runtime information. This method should be used for to instantiate a collector for all Sink V2.

Type Parameters:

CommT - type of the committable

Parameters:

metricGroup - storing the committable metrics

Returns:

CommittableCollector

addMessage

public void addMessage(CommittableMessage<CommT> message)

Adds a CommittableMessage to the collector to hold it until emission.

Parameters:

message - either CommittableSummary or CommittableWithLineage

getCheckpointCommittablesUpTo

public Collection<? extends CheckpointCommittableManager<CommT>> getCheckpointCommittablesUpTo(long checkpointId)

Returns all CheckpointCommittableManager until the requested checkpoint id.

Parameters:

checkpointId - counter

Returns:

collection of CheckpointCommittableManager

getEndOfInputCommittable

public Optional<CheckpointCommittableManager<CommT>> getEndOfInputCommittable()

Returns CheckpointCommittableManager belonging to the last input.

Returns:

CheckpointCommittableManager

isFinished

public boolean isFinished()

Returns whether all CheckpointCommittableManager currently hold by the collector are either committed or failed.

Returns:

state of the CheckpointCommittableManager

merge

public void merge(CommittableCollector<CommT> cc)

Merges all information from an external collector into this collector.

This method is important during recovery from existing state.

Parameters:

cc - other CommittableCollector

copy

public CommittableCollector<CommT> copy()

Returns a new committable collector that deep copies all internals.

Returns:

CommittableCollector

remove

public void remove(CheckpointCommittableManager<CommT> manager)

Removes the manager for a specific checkpoint and all it's metadata.

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object