Package org.apache.flink.state.api

Class SavepointWriter

java.lang.Object
org.apache.flink.state.api.SavepointWriter
@PublicEvolving public class SavepointWriter extends Object
A SavepointWriter can create new savepoints from bounded data streams. This can allow for boostrapping state for new applications or modifying the savepoints of existing jobs.

  • Field Details

    • metadata

      protected final SavepointMetadataV2 metadata
      The savepoint metadata, which maintains the current set of existing / newly added operator states.

    • stateBackend

      @Nullable protected final org.apache.flink.runtime.state.StateBackend stateBackend
      The state backend to use when writing this savepoint.

  • Method Details

    • fromExistingSavepoint

      public static SavepointWriter fromExistingSavepoint(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment executionEnvironment, String path) throws IOException
      Loads an existing savepoint. Useful if you want to modify or extend the state of an existing application. The savepoint will be written using the state backend defined via the clusters configuration.
      Parameters:
      path - The path to an existing savepoint on disk.
      Returns:
      A SavepointWriter.
      Throws:
      IOException
      See Also:

    • fromExistingSavepoint

      public static SavepointWriter fromExistingSavepoint(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment executionEnvironment, String path, org.apache.flink.runtime.state.StateBackend stateBackend) throws IOException
      Loads an existing savepoint. Useful if you want to modify or extend the state of an existing application.
      Parameters:
      path - The path to an existing savepoint on disk.
      stateBackend - The state backend of the savepoint.
      Returns:
      A SavepointWriter.
      Throws:
      IOException

    • newSavepoint

      public static SavepointWriter newSavepoint(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment executionEnvironment, int maxParallelism)
      Creates a new savepoint. The savepoint will be written using the state backend defined via the clusters configuration.
      Parameters:
      maxParallelism - The max parallelism of the savepoint.
      Returns:
      A SavepointWriter.
      See Also:

    • newSavepoint

      public static SavepointWriter newSavepoint(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment executionEnvironment, org.apache.flink.runtime.state.StateBackend stateBackend, int maxParallelism)
      Creates a new savepoint.
      Parameters:
      stateBackend - The state backend of the savepoint used for keyed state.
      maxParallelism - The max parallelism of the savepoint.
      Returns:
      A SavepointWriter.
      See Also:

    • removeOperator

      public SavepointWriter removeOperator(OperatorIdentifier identifier)
      Drop an existing operator from the savepoint.
      Parameters:
      identifier - The identifier of the operator.
      Returns:
      A modified savepoint.

    • withOperator

      public <T> SavepointWriter withOperator(OperatorIdentifier identifier, StateBootstrapTransformation<T> transformation)
      Adds a new operator to the savepoint.
      Parameters:
      identifier - The identifier of the operator.
      transformation - The operator to be included.
      Returns:
      The modified savepoint.

    • withConfiguration

      public <T> SavepointWriter withConfiguration(org.apache.flink.configuration.ConfigOption<T> option, T value)
      Sets a configuration that will be applied to the stream operators used to bootstrap a new savepoint.
      Type Parameters:
      T - type of the value to be stored
      Parameters:
      option - metadata information
      value - value to be stored
      Returns:
      The modified savepoint.

    • changeOperatorIdentifier

      public SavepointWriter changeOperatorIdentifier(OperatorIdentifier from, OperatorIdentifier to)
      Changes the identifier of an operator.

      This method is comparatively cheap since it only modifies savepoint metadata without reading the entire savepoint data.

      Use-cases include, but are not limited to:

      • assigning a UID to an operator that did not have a UID assigned before
      • changing the UID of an operator
      • swapping the states of 2 operators

      Identifier changes are applied after all other operations; in the following example the savepoint will only contain UID_2. 

           SavepointWriter savepoint = ...
     savepoint.withOperator(UID_1, ...)
     savepoint.changeOperatorIdentifier(UID_1, UID_2)
     savepoint.write(...)

      You cannot define a chain of changes; in the following example the savepoint will only contain UID_2. 

           SavepointWriter savepoint = ...
     savepoint.withOperator(UID_1, ...)
     savepoint.changeOperatorIdentifier(UID_1, UID_2)
     savepoint.changeOperatorIdentifier(UID_2, UID_3)
     savepoint.write(...)
      Parameters:
      from - operator whose identifier should be changed
      to - desired identifier
      Returns:
      The modified savepoint.

    • write

      public final void write(String path)
      Write out a new or updated savepoint.
      Parameters:
      path - The path to where the savepoint should be written.