Package org.apache.flink.state.forst.datatransfer

Class ForStStateDataTransfer

java.lang.Object
org.apache.flink.state.forst.datatransfer.ForStStateDataTransfer
All Implemented Interfaces:
Closeable, AutoCloseable
public class ForStStateDataTransfer extends Object implements Closeable
Data transfer util class for ForStKeyedStateBackend.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    DEFAULT_THREAD_NUM
     
    protected final ExecutorService
    executorService
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    ForStStateDataTransfer(int threadNum)
     
    ForStStateDataTransfer(int threadNum, ForStFlinkFileSystem forStFs)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    void
    transferAllStateDataToDirectory(Collection<StateHandleTransferSpec> transferSpecs, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.execution.RecoveryClaimMode recoveryClaimMode)
    Transfer all state data to the target directory, as specified in the transfer requests.
    List<org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath>
    transferFilesToCheckpointFs(org.apache.flink.runtime.checkpoint.SnapshotType.SharingFilesStrategy sharingFilesStrategy, List<org.apache.flink.core.fs.Path> files, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry, boolean forceCopy)
    Transfer a batch of files to checkpoint filesystem.
    org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath
    transferFileToCheckpointFs(org.apache.flink.runtime.checkpoint.SnapshotType.SharingFilesStrategy sharingFilesStrategy, org.apache.flink.core.fs.Path file, long transferBytes, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry snapshotCloseableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry, boolean forceCopy)
    Transfer a single file to checkpoint filesystem.
    org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath
    writeFileToCheckpointFs(String filename, String fileContent, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry)
    Write a file to checkpoint filesystem.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

    • ForStStateDataTransfer

      public ForStStateDataTransfer(int threadNum)

    • ForStStateDataTransfer

      public ForStStateDataTransfer(int threadNum, ForStFlinkFileSystem forStFs)

  • Method Details

    • transferFileToCheckpointFs

      public org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath transferFileToCheckpointFs(org.apache.flink.runtime.checkpoint.SnapshotType.SharingFilesStrategy sharingFilesStrategy, org.apache.flink.core.fs.Path file, long transferBytes, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry snapshotCloseableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry, boolean forceCopy) throws Exception
      Transfer a single file to checkpoint filesystem.
      Parameters:
      transferBytes - Bytes will be transfer from the head of the file. If < 0, the whole file will be transferred.
      Throws:
      Exception

    • transferFilesToCheckpointFs

      public List<org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath> transferFilesToCheckpointFs(org.apache.flink.runtime.checkpoint.SnapshotType.SharingFilesStrategy sharingFilesStrategy, List<org.apache.flink.core.fs.Path> files, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry, boolean forceCopy) throws Exception
      Transfer a batch of files to checkpoint filesystem.
      Throws:
      Exception

    • writeFileToCheckpointFs

      public org.apache.flink.runtime.state.IncrementalKeyedStateHandle.HandleAndLocalPath writeFileToCheckpointFs(String filename, String fileContent, org.apache.flink.runtime.state.CheckpointStreamFactory checkpointStreamFactory, org.apache.flink.runtime.state.CheckpointedStateScope stateScope, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.fs.CloseableRegistry tmpResourcesRegistry) throws IOException
      Write a file to checkpoint filesystem.
      Throws:
      IOException

    • transferAllStateDataToDirectory

      public void transferAllStateDataToDirectory(Collection<StateHandleTransferSpec> transferSpecs, org.apache.flink.core.fs.CloseableRegistry closeableRegistry, org.apache.flink.core.execution.RecoveryClaimMode recoveryClaimMode) throws Exception
      Transfer all state data to the target directory, as specified in the transfer requests.
      Parameters:
      transferSpecs - the list of transfers.
      Throws:
      Exception - If anything about the transfer goes wrong.

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable