Package org.apache.flink.state.rocksdb.snapshot

Class RocksDBSnapshotStrategyBase<K,R extends org.apache.flink.runtime.state.SnapshotResources>

java.lang.Object

org.apache.flink.state.rocksdb.snapshot.RocksDBSnapshotStrategyBase<K,R>

Type Parameters:

K - type of the backend keys.

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.runtime.state.SnapshotStrategy<org.apache.flink.runtime.state.KeyedStateHandle,RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources>

Direct Known Subclasses:

RocksIncrementalSnapshotStrategy, RocksNativeFullSnapshotStrategy

public abstract class RocksDBSnapshotStrategyBase<K,R extends org.apache.flink.runtime.state.SnapshotResources>
extends Object
implements org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.runtime.state.SnapshotStrategy<org.apache.flink.runtime.state.KeyedStateHandle,RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources>, AutoCloseable

Abstract base class for SnapshotStrategy implementations for RocksDB state backend.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static class	RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources	A SnapshotResources for native rocksdb snapshot.
protected static class	RocksDBSnapshotStrategyBase.PreviousSnapshot	Previous snapshot with uploaded sst files.
protected class	RocksDBSnapshotStrategyBase.RocksDBSnapshotOperation	Common operation in native rocksdb snapshot result supplier.

Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.SnapshotStrategy

org.apache.flink.runtime.state.SnapshotStrategy.SnapshotResultSupplier<S extends org.apache.flink.runtime.state.StateObject>

Field Summary

Fields

Modifier and Type	Field	Description
protected final UUID	backendUID	The state handle ids of all sst files materialized in snapshots for previous checkpoints.
protected org.rocksdb.RocksDB	db	RocksDB instance from the backend.
protected static final RocksDBSnapshotStrategyBase.PreviousSnapshot	EMPTY_PREVIOUS_SNAPSHOT
protected final File	instanceBasePath	Base path of the RocksDB instance.
protected final int	keyGroupPrefixBytes	Number of bytes in the key-group prefix.
protected final org.apache.flink.runtime.state.KeyGroupRange	keyGroupRange	The key-group range for the task.
protected final org.apache.flink.api.common.typeutils.TypeSerializer<K>	keySerializer	The key serializer of the backend.
protected final LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo>	kvStateInformation	Key/Value state meta info from the backend.
protected final String	localDirectoryName	The local directory name of the current snapshot strategy.
protected final org.apache.flink.runtime.state.LocalRecoveryConfig	localRecoveryConfig	The configuration for local recovery.
protected final org.apache.flink.util.ResourceGuard	rocksDBResourceGuard	Resource guard for the RocksDB instance.

Constructor Summary

Constructors

Constructor

Description

RocksDBSnapshotStrategyBase(String description, org.rocksdb.RocksDB db, org.apache.flink.util.ResourceGuard rocksDBResourceGuard, org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer, LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation, org.apache.flink.runtime.state.KeyGroupRange keyGroupRange, int keyGroupPrefixBytes, org.apache.flink.runtime.state.LocalRecoveryConfig localRecoveryConfig, File instanceBasePath, UUID backendUID)

Method Summary

Modifier and Type	Method	Description
protected void	cleanupIncompleteSnapshot(org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry, org.apache.flink.runtime.state.SnapshotDirectory localBackupDirectory)
abstract void	close()
String	getDescription()
protected org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.StreamStateHandle>	materializeMetaData(org.apache.flink.core.fs.CloseableRegistry snapshotCloseableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry, List<org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshot> stateMetaInfoSnapshots, long checkpointId, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory)
protected org.apache.flink.runtime.state.SnapshotDirectory	prepareLocalSnapshotDirectory(long checkpointId)
protected abstract RocksDBSnapshotStrategyBase.PreviousSnapshot	snapshotMetaData(long checkpointId, List<org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshot> stateMetaInfoSnapshots)
RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources	syncPrepareResources(long checkpointId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.runtime.state.SnapshotStrategy

asyncSnapshot

Field Details

db

@Nonnull
protected org.rocksdb.RocksDB db

RocksDB instance from the backend.

rocksDBResourceGuard

@Nonnull
protected final org.apache.flink.util.ResourceGuard rocksDBResourceGuard

Resource guard for the RocksDB instance.

keySerializer

@Nonnull
protected final org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer

The key serializer of the backend.

kvStateInformation

@Nonnull
protected final LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation

Key/Value state meta info from the backend.

keyGroupRange

@Nonnull
protected final org.apache.flink.runtime.state.KeyGroupRange keyGroupRange

The key-group range for the task.

keyGroupPrefixBytes

@Nonnegative
protected final int keyGroupPrefixBytes

Number of bytes in the key-group prefix.

localRecoveryConfig

@Nonnull
protected final org.apache.flink.runtime.state.LocalRecoveryConfig localRecoveryConfig

The configuration for local recovery.

instanceBasePath

@Nonnull
protected final File instanceBasePath

Base path of the RocksDB instance.

localDirectoryName

protected final String localDirectoryName

The local directory name of the current snapshot strategy.

backendUID

@Nonnull
protected final UUID backendUID

The state handle ids of all sst files materialized in snapshots for previous checkpoints.

EMPTY_PREVIOUS_SNAPSHOT

protected static final RocksDBSnapshotStrategyBase.PreviousSnapshot EMPTY_PREVIOUS_SNAPSHOT

Constructor Details

RocksDBSnapshotStrategyBase

public RocksDBSnapshotStrategyBase(@Nonnull
 String description,
 @Nonnull
 org.rocksdb.RocksDB db,
 @Nonnull
 org.apache.flink.util.ResourceGuard rocksDBResourceGuard,
 @Nonnull
 org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer,
 @Nonnull
 LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
 @Nonnull
 org.apache.flink.runtime.state.KeyGroupRange keyGroupRange,
 @Nonnegative
 int keyGroupPrefixBytes,
 @Nonnull
 org.apache.flink.runtime.state.LocalRecoveryConfig localRecoveryConfig,
 @Nonnull
 File instanceBasePath,
 @Nonnull
 UUID backendUID)

Method Details

getDescription

@Nonnull
public String getDescription()

syncPrepareResources

public RocksDBSnapshotStrategyBase.NativeRocksDBSnapshotResources syncPrepareResources(long checkpointId)
                                                                                throws Exception

Specified by:

syncPrepareResources in interface org.apache.flink.runtime.state.SnapshotStrategy<K,R extends org.apache.flink.runtime.state.SnapshotResources>

Throws:

Exception

snapshotMetaData

protected abstract RocksDBSnapshotStrategyBase.PreviousSnapshot snapshotMetaData(long checkpointId,
 @Nonnull
 List<org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshot> stateMetaInfoSnapshots)

prepareLocalSnapshotDirectory

@Nonnull
protected org.apache.flink.runtime.state.SnapshotDirectory prepareLocalSnapshotDirectory(long checkpointId)
                                                                                  throws IOException

Throws:

IOException

cleanupIncompleteSnapshot

protected void cleanupIncompleteSnapshot(@Nonnull
 org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry,
 @Nonnull
 org.apache.flink.runtime.state.SnapshotDirectory localBackupDirectory)

materializeMetaData

@Nonnull
protected org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.StreamStateHandle> materializeMetaData(@Nonnull
 org.apache.flink.core.fs.CloseableRegistry snapshotCloseableRegistry,
 @Nonnull
 org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry,
 @Nonnull
 List<org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshot> stateMetaInfoSnapshots,
 long checkpointId,
 @Nonnull
 org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory)
                                                                                                                       throws Exception

Throws:

Exception

close

public abstract void close()
                    throws IOException

Specified by:

close in interface AutoCloseable

Throws:

IOException