Package org.apache.flink.state.forst

Class ForStStateBackend

java.lang.Object
org.apache.flink.runtime.state.AbstractStateBackend
org.apache.flink.runtime.state.AbstractManagedMemoryStateBackend
org.apache.flink.state.forst.ForStStateBackend
All Implemented Interfaces:
Serializable, org.apache.flink.runtime.state.ConfigurableStateBackend, org.apache.flink.runtime.state.StateBackend
@Experimental public class ForStStateBackend extends org.apache.flink.runtime.state.AbstractManagedMemoryStateBackend implements org.apache.flink.runtime.state.ConfigurableStateBackend
A StateBackend that stores its state in a ForSt instance. This state backend can store very large state that exceeds memory even disk and spills to remote storage.

The behavior of the ForSt instances can be parametrized by setting ForSt Options using the methods setForStOptions(ForStOptionsFactory).

See Also:

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static enum 
    ForStStateBackend.PriorityQueueStateType
    The options to chose for the type of priority queue state.

    Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.StateBackend

    org.apache.flink.runtime.state.StateBackend.CustomInitializationMetrics, org.apache.flink.runtime.state.StateBackend.KeyedStateBackendParameters<K extends Object>, org.apache.flink.runtime.state.StateBackend.OperatorStateBackendParameters

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    CHECKPOINT_DIR_AS_PRIMARY_SHORTCUT
     
    static final String
    LOCAL_DIR_AS_PRIMARY_SHORTCUT
     

    Fields inherited from class org.apache.flink.runtime.state.AbstractStateBackend

    latencyTrackingConfigBuilder

  • Constructor Summary

    Constructors
    Constructor
    Description
    ForStStateBackend()
    Creates a new ForStStateBackend for storing state.

  • Method Summary

    Modifier and Type
    Method
    Description
    ForStStateBackend
    configure(org.apache.flink.configuration.ReadableConfig config, ClassLoader classLoader)
    Creates a copy of this state backend that uses the values defined in the configuration for fields where that were not yet specified in this state backend.
    <K> ForStKeyedStateBackend<K>
    createAsyncKeyedStateBackend(org.apache.flink.runtime.state.StateBackend.KeyedStateBackendParameters<K> parameters)
     
    <K> org.apache.flink.runtime.state.AbstractKeyedStateBackend<K>
    createKeyedStateBackend(org.apache.flink.runtime.state.StateBackend.KeyedStateBackendParameters<K> parameters)
     
    org.apache.flink.runtime.state.OperatorStateBackend
    createOperatorStateBackend(org.apache.flink.runtime.state.StateBackend.OperatorStateBackendParameters parameters)
     
    ForStOptionsFactory
    getForStOptions()
    Gets Options for the ForSt instances.
    String[]
    getLocalDbStoragePaths()
    Gets the configured local DB storage paths, or null, if none were configured.
    boolean
    isIncrementalCheckpointsEnabled()
    Gets whether incremental checkpoints are enabled for this state backend.
    void
    setForStOptions(ForStOptionsFactory optionsFactory)
    Sets Options for the ForSt instances.
    void
    setLocalDbStoragePath(String path)
    Sets the path where the ForSt local files should be stored on the local file system.
    void
    setLocalDbStoragePaths(String... paths)
    Sets the local directories in which the ForSt database puts some files (like metadata files).
    boolean
    supportsAsyncKeyedStateBackend()
     
    boolean
    supportsNoClaimRestoreMode()
     
    boolean
    supportsSavepointFormat(org.apache.flink.core.execution.SavepointFormatType formatType)
     
    String
    toString()
     

    Methods inherited from class org.apache.flink.runtime.state.AbstractManagedMemoryStateBackend

    useManagedMemory

    Methods inherited from class org.apache.flink.runtime.state.AbstractStateBackend

    getCompressionDecorator

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface org.apache.flink.runtime.state.StateBackend

    getName, useManagedMemory

  • Field Details

  • Constructor Details

    • ForStStateBackend

      public ForStStateBackend()
      Creates a new ForStStateBackend for storing state.

  • Method Details

    • configure

      public ForStStateBackend configure(org.apache.flink.configuration.ReadableConfig config, ClassLoader classLoader)
      Creates a copy of this state backend that uses the values defined in the configuration for fields where that were not yet specified in this state backend.
      Specified by:
      configure in interface org.apache.flink.runtime.state.ConfigurableStateBackend
      Parameters:
      config - The configuration.
      classLoader - The class loader.
      Returns:
      The re-configured variant of the state backend

    • supportsAsyncKeyedStateBackend

      public boolean supportsAsyncKeyedStateBackend()
      Specified by:
      supportsAsyncKeyedStateBackend in interface org.apache.flink.runtime.state.StateBackend

    • createAsyncKeyedStateBackend

      public <K> ForStKeyedStateBackend<K> createAsyncKeyedStateBackend(org.apache.flink.runtime.state.StateBackend.KeyedStateBackendParameters<K> parameters) throws IOException
      Specified by:
      createAsyncKeyedStateBackend in interface org.apache.flink.runtime.state.StateBackend
      Throws:
      IOException

    • createKeyedStateBackend

      public <K> org.apache.flink.runtime.state.AbstractKeyedStateBackend<K> createKeyedStateBackend(org.apache.flink.runtime.state.StateBackend.KeyedStateBackendParameters<K> parameters) throws IOException
      Specified by:
      createKeyedStateBackend in interface org.apache.flink.runtime.state.StateBackend
      Specified by:
      createKeyedStateBackend in class org.apache.flink.runtime.state.AbstractManagedMemoryStateBackend
      Throws:
      IOException

    • createOperatorStateBackend

      public org.apache.flink.runtime.state.OperatorStateBackend createOperatorStateBackend(org.apache.flink.runtime.state.StateBackend.OperatorStateBackendParameters parameters) throws Exception
      Specified by:
      createOperatorStateBackend in interface org.apache.flink.runtime.state.StateBackend
      Specified by:
      createOperatorStateBackend in class org.apache.flink.runtime.state.AbstractStateBackend
      Throws:
      Exception

    • isIncrementalCheckpointsEnabled

      public boolean isIncrementalCheckpointsEnabled()
      Gets whether incremental checkpoints are enabled for this state backend.

    • supportsNoClaimRestoreMode

      public boolean supportsNoClaimRestoreMode()
      Specified by:
      supportsNoClaimRestoreMode in interface org.apache.flink.runtime.state.StateBackend

    • supportsSavepointFormat

      public boolean supportsSavepointFormat(org.apache.flink.core.execution.SavepointFormatType formatType)
      Specified by:
      supportsSavepointFormat in interface org.apache.flink.runtime.state.StateBackend

    • setLocalDbStoragePath

      public void setLocalDbStoragePath(String path)
      Sets the path where the ForSt local files should be stored on the local file system. Setting this path overrides the default behavior, where the files are stored across the configured temp directories.

      Passing null to this function restores the default behavior, where the configured temp directories will be used.

      Parameters:
      path - The path where the local ForSt database files are stored.

    • setLocalDbStoragePaths

      public void setLocalDbStoragePaths(String... paths)
      Sets the local directories in which the ForSt database puts some files (like metadata files). These directories do not need to be persistent, they can be ephemeral, meaning that they are lost on a machine failure, because state in ForSt is persisted in checkpoints.

      If nothing is configured, these directories default to the TaskManager's local temporary file directories.

      Each distinct state will be stored in one path, but when the state backend creates multiple states, they will store their files on different paths.

      Passing null to this function restores the default behavior, where the configured temp directories will be used.

      Parameters:
      paths - The paths across which the local ForSt database files will be spread.

    • getLocalDbStoragePaths

      public String[] getLocalDbStoragePaths()
      Gets the configured local DB storage paths, or null, if none were configured.

      Under these directories on the TaskManager, ForSt stores some metadata files. These directories do not need to be persistent, they can be ephermeral, meaning that they are lost on a machine failure, because state in ForSt is persisted in checkpoints.

      If nothing is configured, these directories default to the TaskManager's local temporary file directories.

    • setForStOptions

      public void setForStOptions(ForStOptionsFactory optionsFactory)
      Sets Options for the ForSt instances. Because the options are not serializable and hold native code references, they must be specified through a factory.

      The options created by the factory here are applied on top of user-configured options from configuration set by configure(ReadableConfig, ClassLoader) with keys in ForStConfigurableOptions.

      Parameters:
      optionsFactory - The options factory that lazily creates the ForSt options.

    • getForStOptions

      @Nullable public ForStOptionsFactory getForStOptions()
      Gets Options for the ForSt instances.

    • toString

      public String toString()
      Overrides:
      toString in class Object