Class RocksDBStore

  • All Implemented Interfaces:
    StateStore, KeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>, ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>

    public class RocksDBStore
    extends java.lang.Object
    implements KeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
    A persistent key-value store based on RocksDB. Note that the use of array-typed keys is discouraged because they result in incorrect caching behavior. If you intend to work on byte arrays as key, for example, you may want to wrap them with the Bytes class, i.e. use RocksDBStore<Bytes, ...> rather than RocksDBStore<byte[], ...>.
    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected boolean open  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      KeyValueIterator<org.apache.kafka.common.utils.Bytes,​byte[]> all()
      Return an iterator over all keys in this store.
      long approximateNumEntries()
      Return an approximate count of key-value mappings in this store.
      void close()
      Close the storage engine.
      byte[] delete​(org.apache.kafka.common.utils.Bytes key)
      Delete the value from the store (if there is one).
      void flush()
      Flush any cached data
      byte[] get​(org.apache.kafka.common.utils.Bytes key)
      Get the value corresponding to this key.
      org.rocksdb.Options getOptions()  
      void init​(ProcessorContext context, StateStore root)
      Initializes this state store.
      boolean isOpen()
      Is this store open for reading and writing
      java.lang.String name()
      The name of this store.
      void openDB​(ProcessorContext context)  
      boolean persistent()
      Return if the storage is persistent or not.
      void put​(org.apache.kafka.common.utils.Bytes key, byte[] value)
      Update the value associated with this key.
      void putAll​(java.util.List<KeyValue<org.apache.kafka.common.utils.Bytes,​byte[]>> entries)
      Update all the given key/value pairs.
      byte[] putIfAbsent​(org.apache.kafka.common.utils.Bytes key, byte[] value)
      Update the value associated with this key, unless a value is already associated with the key.
      KeyValueIterator<org.apache.kafka.common.utils.Bytes,​byte[]> range​(org.apache.kafka.common.utils.Bytes from, org.apache.kafka.common.utils.Bytes to)
      Get an iterator over a given range of keys.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • open

        protected volatile boolean open
    • Method Detail

      • name

        public java.lang.String name()
        Description copied from interface: StateStore
        The name of this store.
        Specified by:
        name in interface StateStore
        Returns:
        the storage name
      • persistent

        public boolean persistent()
        Description copied from interface: StateStore
        Return if the storage is persistent or not.
        Specified by:
        persistent in interface StateStore
        Returns:
        true if the storage is persistent—false otherwise
      • isOpen

        public boolean isOpen()
        Description copied from interface: StateStore
        Is this store open for reading and writing
        Specified by:
        isOpen in interface StateStore
        Returns:
        true if the store is open
      • get

        public byte[] get​(org.apache.kafka.common.utils.Bytes key)
        Description copied from interface: ReadOnlyKeyValueStore
        Get the value corresponding to this key.
        Specified by:
        get in interface ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
        Parameters:
        key - The key to fetch
        Returns:
        The value or null if no value is found.
      • put

        public void put​(org.apache.kafka.common.utils.Bytes key,
                        byte[] value)
        Description copied from interface: KeyValueStore
        Update the value associated with this key.
        Specified by:
        put in interface KeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
        Parameters:
        key - The key to associate the value to
        value - The value to update, it can be null; if the serialized bytes are also null it is interpreted as deletes
      • putIfAbsent

        public byte[] putIfAbsent​(org.apache.kafka.common.utils.Bytes key,
                                  byte[] value)
        Description copied from interface: KeyValueStore
        Update the value associated with this key, unless a value is already associated with the key.
        Specified by:
        putIfAbsent in interface KeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
        Parameters:
        key - The key to associate the value to
        value - The value to update, it can be null; if the serialized bytes are also null it is interpreted as deletes
        Returns:
        The old value or null if there is no such key.
      • putAll

        public void putAll​(java.util.List<KeyValue<org.apache.kafka.common.utils.Bytes,​byte[]>> entries)
        Description copied from interface: KeyValueStore
        Update all the given key/value pairs.
        Specified by:
        putAll in interface KeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
        Parameters:
        entries - A list of entries to put into the store; if the serialized bytes are also null it is interpreted as deletes
      • delete

        public byte[] delete​(org.apache.kafka.common.utils.Bytes key)
        Description copied from interface: KeyValueStore
        Delete the value from the store (if there is one).
        Specified by:
        delete in interface KeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
        Parameters:
        key - The key
        Returns:
        The old value or null if there is no such key.
      • range

        public KeyValueIterator<org.apache.kafka.common.utils.Bytes,​byte[]> range​(org.apache.kafka.common.utils.Bytes from,
                                                                                        org.apache.kafka.common.utils.Bytes to)
        Description copied from interface: ReadOnlyKeyValueStore
        Get an iterator over a given range of keys. This iterator must be closed after use. The returned iterator must be safe from ConcurrentModificationExceptions and must not return null values. No ordering guarantees are provided.
        Specified by:
        range in interface ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
        Parameters:
        from - The first key that could be in the range
        to - The last key that could be in the range
        Returns:
        The iterator for this range.
      • all

        public KeyValueIterator<org.apache.kafka.common.utils.Bytes,​byte[]> all()
        Description copied from interface: ReadOnlyKeyValueStore
        Return an iterator over all keys in this store. This iterator must be closed after use. The returned iterator must be safe from ConcurrentModificationExceptions and must not return null values. No ordering guarantees are provided.
        Specified by:
        all in interface ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
        Returns:
        An iterator of all key/value pairs in the store.
      • approximateNumEntries

        public long approximateNumEntries()
        Return an approximate count of key-value mappings in this store. RocksDB cannot return an exact entry count without doing a full scan, so this method relies on the rocksdb.estimate-num-keys property to get an approximate count. The returned size also includes a count of dirty keys in the store's in-memory cache, which may lead to some double-counting of entries and inflate the estimate.
        Specified by:
        approximateNumEntries in interface ReadOnlyKeyValueStore<org.apache.kafka.common.utils.Bytes,​byte[]>
        Returns:
        an approximate count of key-value mappings in the store.
      • flush

        public void flush()
        Description copied from interface: StateStore
        Flush any cached data
        Specified by:
        flush in interface StateStore
      • close

        public void close()
        Description copied from interface: StateStore
        Close the storage engine. Note that this function needs to be idempotent since it may be called several times on the same state store.

        Users only need to implement this function but should NEVER need to call this api explicitly as it will be called by the library automatically when necessary

        Specified by:
        close in interface StateStore
      • getOptions

        public org.rocksdb.Options getOptions()