Packages

trait StateStoreProvider extends AnyRef

Trait representing a provider that provide StateStore instances representing versions of state data.

The life cycle of a provider and its provide stores are as follows.

- A StateStoreProvider is created in a executor for each unique StateStoreId when the first batch of a streaming query is executed on the executor. All subsequent batches reuse this provider instance until the query is stopped.

- Every batch of streaming data request a specific version of the state data by invoking getStore(version) which returns an instance of StateStore through which the required version of the data can be accessed. It is the responsible of the provider to populate this store with context information like the schema of keys and values, etc.

- After the streaming query is stopped, the created provider instances are lazily disposed off.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StateStoreProvider
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def close(): Unit

    Called when the provider instance is unloaded from the executor

  2. abstract def getStore(version: Long): StateStore

    Return an instance of StateStore representing state data of the given version

  3. abstract def init(stateStoreId: StateStoreId, keySchema: StructType, valueSchema: StructType, numColsPrefixKey: Int, storeConfs: StateStoreConf, hadoopConf: Configuration): Unit

    Initialize the provide with more contextual information from the SQL operator.

    Initialize the provide with more contextual information from the SQL operator. This method will be called first after creating an instance of the StateStoreProvider by reflection.

    stateStoreId

    Id of the versioned StateStores that this provider will generate

    keySchema

    Schema of keys to be stored

    valueSchema

    Schema of value to be stored

    numColsPrefixKey

    The number of leftmost columns to be used as prefix key. A value not greater than 0 means the operator doesn't activate prefix key, and the operator should not call prefixScan method in StateStore.

    storeConfs

    Configurations used by the StateStores

    hadoopConf

    Hadoop configuration that could be used by StateStore to save state data

  4. abstract def stateStoreId: StateStoreId

    Return the id of the StateStores this provider will generate.

    Return the id of the StateStores this provider will generate. Should be the same as the one passed in init().

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  6. def doMaintenance(): Unit

    Optional method for providers to allow for background maintenance (e.g.

    Optional method for providers to allow for background maintenance (e.g. compactions)

  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  10. def getReadStore(version: Long): ReadStateStore

    Return an instance of ReadStateStore representing state data of the given version.

    Return an instance of ReadStateStore representing state data of the given version. By default it will return the same instance as getStore(version) but wrapped to prevent modification. Providers can override and return optimized version of ReadStateStore based on the fact the instance will be only used for reading.

  11. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  16. def supportedCustomMetrics: Seq[StateStoreCustomMetric]

    Optional custom metrics that the implementation may want to report.

    Optional custom metrics that the implementation may want to report.

    Note

    The StateStore objects created by this provider must report the same custom metrics (specifically, same names) through StateStore.metrics.

  17. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  18. def toString(): String
    Definition Classes
    AnyRef → Any
  19. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  21. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from AnyRef

Inherited from Any

Ungrouped