Package org.apache.flink.runtime.state.restore

Class FullSnapshotRestoreOperation<K>

java.lang.Object

org.apache.flink.runtime.state.restore.FullSnapshotRestoreOperation<K>

Type Parameters:

K - The data type of the key.

All Implemented Interfaces:

RestoreOperation<ThrowingIterator<SavepointRestoreResult>>

@Internal
public class FullSnapshotRestoreOperation<K>
extends Object
implements RestoreOperation<ThrowingIterator<SavepointRestoreResult>>

The procedure of restoring state from a savepoint written with the unified binary format. All state backends should support restoring from that format.

The format was adopted from the RocksDB state backend. It is as follows:

                    +-------------------------------------------------------------+
                    |                Keyed Backend Meta Information               |
  Meta Information  +--------------------+-----+----------------------------------+
                    |    State Meta 0    | ... |           State Meta M           |
                    +-------------------------------------------------------------+
                    |                       State ID (short)                      |
  State 0           +--------------------+-----+----------------------------------+
                    | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                       State ID (short)                      |
  State 1           +--------------------+-----+----------------------------------+
                    | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                       State ID (short)                      |
                    +--------------------+-----+----------------------------------+
  State M           | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                END_OF_KEY_GROUP_MARK (0xFFFF)               |
                    +-------------------------------------------------------------+
 

Additionally the format of the (K,V) pairs differs slightly depending on the type of the state object:

 +------------------+-------------------------------------+
 |    ValueState    |    [CompositeKey(KG, K, NS), SV]    |
 |------------------+-------------------------------------+
 |    ListState     |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |     MapState     | [CompositeKey(KG, K, NS) :: UK, UV] |
 +------------------+-------------------------------------+
 | AggregatingState |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |   ReducingState  |    [CompositeKey(KG, K, NS), SV]    |
 |                  +-------------------------------------+
 |                  |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |      Timers      |    [KG :: TS :: K :: NS, (empty)]   |
 +------------------+-------------------------------------+
 

For detailed information see FLIP-41: https://cwiki.apache.org/confluence/x/VJDiBg

Constructor Summary

Constructors

Constructor	Description
FullSnapshotRestoreOperation(KeyGroupRange keyGroupRange, ClassLoader userCodeClassLoader, Collection<KeyedStateHandle> restoreStateHandles, StateSerializerProvider<K> keySerializerProvider)

Method Summary

Modifier and Type	Method	Description
ThrowingIterator<SavepointRestoreResult>	restore()	Restores state that was previously snapshot-ed from the provided state handles.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FullSnapshotRestoreOperation

public FullSnapshotRestoreOperation(KeyGroupRange keyGroupRange,
 ClassLoader userCodeClassLoader,
 Collection<KeyedStateHandle> restoreStateHandles,
 StateSerializerProvider<K> keySerializerProvider)

Method Details

restore

public ThrowingIterator<SavepointRestoreResult> restore()
                                                 throws IOException,
org.apache.flink.util.StateMigrationException

Description copied from interface: RestoreOperation

Restores state that was previously snapshot-ed from the provided state handles.

Specified by:

restore in interface RestoreOperation<K>

Throws:

IOException

org.apache.flink.util.StateMigrationException