Package org.apache.flink.runtime.state.filesystem

Class AbstractFileStateBackend

java.lang.Object
org.apache.flink.runtime.state.AbstractStateBackend
org.apache.flink.runtime.state.filesystem.AbstractFileStateBackend
All Implemented Interfaces:
Serializable, CheckpointStorage, StateBackend
@Deprecated @PublicEvolving public abstract class AbstractFileStateBackend extends AbstractStateBackend implements CheckpointStorage
Deprecated.
State backends should no longer implement CheckpointStorage functionality. Please inherit AbstractStateBackend instead. Custom checkpoint storage can be additionally implemented as a separate class.
A base class for all state backends that store their metadata (and data) in files.

This class takes the base checkpoint- and savepoint directory paths, but also accepts null for both of then, in which case creating externalized checkpoint is not possible, and it is not possible to create a savepoint with a default path. Null is accepted to enable implementations that only optionally support default savepoints and externalized checkpoints.

Checkpoint Layout

The state backend is configured with a base directory and persists the checkpoint data of specific checkpoints in specific subdirectories. For example, if the base directory was set to hdfs://namenode:port/flink-checkpoints/, the state backend will create a subdirectory with the job's ID that will contain the actual checkpoints: ( hdfs://namenode:port/flink-checkpoints/1b080b6e710aabbef8993ab18c6de98b)

Each checkpoint individually will store all its files in a subdirectory that includes the checkpoint number, such as hdfs://namenode:port/flink-checkpoints/1b080b6e710aabbef8993ab18c6de98b/chk-17/.

Savepoint Layout

A savepoint that is set to be stored in path hdfs://namenode:port/flink-savepoints/, will create a subdirectory savepoint-jobId(0, 6)-randomDigits in which it stores all savepoint data. The random digits are added as "entropy" to avoid directory collisions.

Metadata File

A completed checkpoint writes its metadata into a file '"_metadata"'.

See Also:

  • Constructor Details

    • AbstractFileStateBackend

      protected AbstractFileStateBackend(@Nullable URI baseCheckpointPath, @Nullable URI baseSavepointPath)
      Deprecated.
      Creates a backend with the given optional checkpoint- and savepoint base directories.
      Parameters:
      baseCheckpointPath - The base directory for checkpoints, or null, if none is configured.
      baseSavepointPath - The default directory for savepoints, or null, if none is set.

    • AbstractFileStateBackend

      protected AbstractFileStateBackend(@Nullable org.apache.flink.core.fs.Path baseCheckpointPath, @Nullable org.apache.flink.core.fs.Path baseSavepointPath)
      Deprecated.
      Creates a backend with the given optional checkpoint- and savepoint base directories.
      Parameters:
      baseCheckpointPath - The base directory for checkpoints, or null, if none is configured.
      baseSavepointPath - The default directory for savepoints, or null, if none is set.

    • AbstractFileStateBackend

      protected AbstractFileStateBackend(@Nullable org.apache.flink.core.fs.Path baseCheckpointPath, @Nullable org.apache.flink.core.fs.Path baseSavepointPath, org.apache.flink.configuration.ReadableConfig configuration)
      Deprecated.
      Creates a new backend using the given checkpoint-/savepoint directories, or the values defined in the given configuration. If a checkpoint-/savepoint parameter is not null, that value takes precedence over the value in the configuration. If the configuration does not specify a value, it is possible that the checkpoint-/savepoint directories in the backend will be null.

      This constructor can be used to create a backend that is based partially on a given backend and partially on a configuration.

      Parameters:
      baseCheckpointPath - The checkpoint base directory to use (or null).
      baseSavepointPath - The default savepoint directory to use (or null).
      configuration - The configuration to read values from.

  • Method Details

    • getCheckpointPath

      @Nullable public org.apache.flink.core.fs.Path getCheckpointPath()
      Deprecated.
      Gets the checkpoint base directory. Jobs will create job-specific subdirectories for checkpoints within this directory. May be null, if not configured.
      Returns:
      The checkpoint base directory

    • getSavepointPath

      @Nullable public org.apache.flink.core.fs.Path getSavepointPath()
      Deprecated.
      Gets the directory where savepoints are stored by default (when no custom path is given to the savepoint trigger command).
      Returns:
      The default directory for savepoints, or null, if no default directory has been configured.

    • resolveCheckpoint

      public CompletedCheckpointStorageLocation resolveCheckpoint(String pointer) throws IOException
      Deprecated.
      Description copied from interface: CheckpointStorage
      Resolves the given pointer to a checkpoint/savepoint into a checkpoint location. The location supports reading the checkpoint metadata, or disposing the checkpoint storage location.
      Specified by:
      resolveCheckpoint in interface CheckpointStorage
      Parameters:
      pointer - The external checkpoint pointer to resolve.
      Returns:
      The checkpoint location handle.
      Throws:
      IOException - Thrown, if the state backend does not understand the pointer, or if the pointer could not be resolved due to an I/O error.