Package org.apache.kafka.streams.state.internals

Class RocksDBStore

java.lang.Object

org.apache.kafka.streams.state.internals.RocksDBStore

All Implemented Interfaces:

StateStore, BatchWritingStore, KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>, ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Direct Known Subclasses:

RocksDBTimestampedStore

public class RocksDBStore
extends java.lang.Object
implements KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>, BatchWritingStore

A persistent key-value store based on RocksDB.

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	open

Method Summary

Modifier and Type	Method	Description
void	addToBatch(KeyValue<byte[],byte[]> record, org.rocksdb.WriteBatch batch)
KeyValueIterator<org.apache.kafka.common.utils.Bytes,byte[]>	all()	Return an iterator over all keys in this store.
long	approximateNumEntries()	Return an approximate count of key-value mappings in this store.
void	close()	Close the storage engine.
byte[]	delete(org.apache.kafka.common.utils.Bytes key)	Delete the value from the store (if there is one).
void	flush()	Flush any cached data
byte[]	get(org.apache.kafka.common.utils.Bytes key)	Get the value corresponding to this key.
org.rocksdb.Options	getOptions()
void	init(ProcessorContext context, StateStore root)	Initializes this state store.
boolean	isOpen()	Is this store open for reading and writing
java.lang.String	name()	The name of this store.
boolean	persistent()	Return if the storage is persistent or not.
void	put(org.apache.kafka.common.utils.Bytes key, byte[] value)	Update the value associated with this key.
void	putAll(java.util.List<KeyValue<org.apache.kafka.common.utils.Bytes,byte[]>> entries)	Update all the given key/value pairs.
byte[]	putIfAbsent(org.apache.kafka.common.utils.Bytes key, byte[] value)	Update the value associated with this key, unless a value is already associated with the key.
KeyValueIterator<org.apache.kafka.common.utils.Bytes,byte[]>	range(org.apache.kafka.common.utils.Bytes from, org.apache.kafka.common.utils.Bytes to)	Get an iterator over a given range of keys.
void	write(org.rocksdb.WriteBatch batch)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

open

protected volatile boolean open

Method Detail

init

public void init(ProcessorContext context,
                 StateStore root)

Description copied from interface: StateStore

Initializes this state store.

The implementation of this function must register the root store in the context via the ProcessorContext.register(StateStore, StateRestoreCallback) function, where the first StateStore parameter should always be the passed-in root object, and the second parameter should be an object of user's implementation of the StateRestoreCallback interface used for restoring the state store from the changelog.

Note that if the state store engine itself supports bulk writes, users can implement another interface BatchingStateRestoreCallback which extends StateRestoreCallback to let users implement bulk-load restoration logic instead of restoring one record at a time.

Specified by:

init in interface StateStore

name

public java.lang.String name()

Description copied from interface: StateStore

The name of this store.

Specified by:

name in interface StateStore

Returns:

the storage name

persistent

public boolean persistent()

Description copied from interface: StateStore

Return if the storage is persistent or not.

Specified by:

persistent in interface StateStore

Returns:

true if the storage is persistent—false otherwise

isOpen

public boolean isOpen()

Description copied from interface: StateStore

Is this store open for reading and writing

Specified by:

isOpen in interface StateStore

Returns:

true if the store is open

put

public void put(org.apache.kafka.common.utils.Bytes key,
                byte[] value)

Description copied from interface: KeyValueStore

Update the value associated with this key.

Specified by:

put in interface KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Parameters:

key - The key to associate the value to

value - The value to update, it can be null; if the serialized bytes are also null it is interpreted as deletes

putIfAbsent

public byte[] putIfAbsent(org.apache.kafka.common.utils.Bytes key,
                          byte[] value)

Description copied from interface: KeyValueStore

Update the value associated with this key, unless a value is already associated with the key.

Specified by:

putIfAbsent in interface KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Parameters:

key - The key to associate the value to

value - The value to update, it can be null; if the serialized bytes are also null it is interpreted as deletes

Returns:

The old value or null if there is no such key.

putAll

public void putAll(java.util.List<KeyValue<org.apache.kafka.common.utils.Bytes,byte[]>> entries)

Description copied from interface: KeyValueStore

Update all the given key/value pairs.

Specified by:

putAll in interface KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Parameters:

entries - A list of entries to put into the store; if the serialized bytes are also null it is interpreted as deletes

get

public byte[] get(org.apache.kafka.common.utils.Bytes key)

Description copied from interface: ReadOnlyKeyValueStore

Get the value corresponding to this key.

Specified by:

get in interface ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Parameters:

key - The key to fetch

Returns:

The value or null if no value is found.

delete

public byte[] delete(org.apache.kafka.common.utils.Bytes key)

Description copied from interface: KeyValueStore

Delete the value from the store (if there is one).

Specified by:

delete in interface KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Parameters:

key - The key

Returns:

The old value or null if there is no such key.

range

public KeyValueIterator<org.apache.kafka.common.utils.Bytes,byte[]> range(org.apache.kafka.common.utils.Bytes from,
                                                                                org.apache.kafka.common.utils.Bytes to)

Description copied from interface: ReadOnlyKeyValueStore

Get an iterator over a given range of keys. This iterator must be closed after use. The returned iterator must be safe from ConcurrentModificationExceptions and must not return null values. No ordering guarantees are provided.

Specified by:

range in interface ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Parameters:

from - The first key that could be in the range

to - The last key that could be in the range

Returns:

The iterator for this range.

all

public KeyValueIterator<org.apache.kafka.common.utils.Bytes,byte[]> all()

Description copied from interface: ReadOnlyKeyValueStore

Return an iterator over all keys in this store. This iterator must be closed after use. The returned iterator must be safe from ConcurrentModificationExceptions and must not return null values. No ordering guarantees are provided.

Specified by:

all in interface ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Returns:

An iterator of all key/value pairs in the store.

approximateNumEntries

public long approximateNumEntries()

Return an approximate count of key-value mappings in this store. RocksDB cannot return an exact entry count without doing a full scan, so this method relies on the rocksdb.estimate-num-keys property to get an approximate count. The returned size also includes a count of dirty keys in the store's in-memory cache, which may lead to some double-counting of entries and inflate the estimate.

Specified by:

approximateNumEntries in interface ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>

Returns:

an approximate count of key-value mappings in the store.

flush

public void flush()

Description copied from interface: StateStore

Flush any cached data

Specified by:

flush in interface StateStore

addToBatch

public void addToBatch(KeyValue<byte[],byte[]> record,
                       org.rocksdb.WriteBatch batch)
                throws org.rocksdb.RocksDBException

Specified by:

addToBatch in interface BatchWritingStore

Throws:

org.rocksdb.RocksDBException

write

public void write(org.rocksdb.WriteBatch batch)
           throws org.rocksdb.RocksDBException

Specified by:

write in interface BatchWritingStore

Throws:

org.rocksdb.RocksDBException

close

public void close()

Description copied from interface: StateStore

Close the storage engine. Note that this function needs to be idempotent since it may be called several times on the same state store.

Users only need to implement this function but should NEVER need to call this api explicitly as it will be called by the library automatically when necessary

Specified by:

close in interface StateStore

getOptions

public org.rocksdb.Options getOptions()