Package org.apache.flink.state.forst.fs

Class ForStFlinkFileSystem

java.lang.Object
org.apache.flink.core.fs.FileSystem
org.apache.flink.state.forst.fs.ForStFlinkFileSystem
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.flink.core.fs.IFileSystem
public class ForStFlinkFileSystem extends org.apache.flink.core.fs.FileSystem implements Closeable
A FileSystem delegates some requests to file system loaded by Flink FileSystem mechanism.

All methods in this class maybe used by ForSt, please start a discussion firstly if it has to be modified.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    ForStFlinkFileSystem.FileStatusWrapper
     

    Nested classes/interfaces inherited from class org.apache.flink.core.fs.FileSystem

    org.apache.flink.core.fs.FileSystem.FSKey, org.apache.flink.core.fs.FileSystem.WriteMode

  • Constructor Summary

    Constructors
    Constructor
    Description
    ForStFlinkFileSystem(org.apache.flink.core.fs.FileSystem delegateFS, String remoteBase, String localBase, FileBasedCache fileBasedCache)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    ByteBufferWritableFSDataOutputStream
    create(org.apache.flink.core.fs.Path path)
    Create ByteBufferWritableFSDataOutputStream from specific path which supports to write data to ByteBuffer with FileSystem.WriteMode.OVERWRITE mode.
    ByteBufferWritableFSDataOutputStream
    create(org.apache.flink.core.fs.Path dbFilePath, org.apache.flink.core.fs.FileSystem.WriteMode overwriteMode)
     
    boolean
    delete(org.apache.flink.core.fs.Path path, boolean recursive)
     
    boolean
    exists(org.apache.flink.core.fs.Path f)
     
    static ForStFlinkFileSystem
    get(URI uri)
    Returns a reference to the FileSystem instance for accessing the file system identified by the given URI.
    static ForStFlinkFileSystem
    get(URI uri, org.apache.flink.core.fs.Path localBase, FileBasedCache fileBasedCache)
     
    org.apache.flink.core.fs.FileSystem
    getDelegateFS()
     
    static FileBasedCache
    getFileBasedCache(org.apache.flink.configuration.ReadableConfig config, org.apache.flink.core.fs.Path cacheBase, org.apache.flink.core.fs.Path remoteForStPath, long cacheCapacity, long cacheReservedSize, org.apache.flink.metrics.MetricGroup metricGroup)
     
    org.apache.flink.core.fs.BlockLocation[]
    getFileBlockLocations(org.apache.flink.core.fs.FileStatus file, long start, long len)
     
    org.apache.flink.core.fs.FileStatus
    getFileStatus(org.apache.flink.core.fs.Path path)
     
    org.apache.flink.core.fs.Path
    getHomeDirectory()
     
    MappingEntry
    getMappingEntry(org.apache.flink.core.fs.Path path)
     
    String
    getRemoteBase()
     
    URI
    getUri()
     
    org.apache.flink.core.fs.Path
    getWorkingDirectory()
     
    void
    giveUpOwnership(org.apache.flink.core.fs.Path path, org.apache.flink.runtime.state.StreamStateHandle stateHandle)
     
    boolean
    isDistributedFS()
     
    int
    link(String src, org.apache.flink.core.fs.Path dst)
     
    int
    link(org.apache.flink.core.fs.Path src, org.apache.flink.core.fs.Path dst)
     
    org.apache.flink.core.fs.FileStatus[]
    listStatus(org.apache.flink.core.fs.Path path)
     
    boolean
    mkdirs(org.apache.flink.core.fs.Path path)
     
    ByteBufferReadableFSDataInputStream
    open(org.apache.flink.core.fs.Path dbFilePath)
     
    ByteBufferReadableFSDataInputStream
    open(org.apache.flink.core.fs.Path dbFilePath, int bufferSize)
     
    void
    registerReusedRestoredFile(String key, org.apache.flink.runtime.state.StreamStateHandle stateHandle, org.apache.flink.core.fs.Path dbFilePath)
     
    boolean
    rename(org.apache.flink.core.fs.Path src, org.apache.flink.core.fs.Path dst)
     

    Methods inherited from class org.apache.flink.core.fs.FileSystem

    create, create, createRecoverableWriter, createRecoverableWriter, getDefaultBlockSize, getDefaultFsUri, getLocalFileSystem, getUnguardedFileSystem, initialize, initialize, initOutPathDistFS, initOutPathLocalFS

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.flink.core.fs.IFileSystem

    canCopyPaths

  • Constructor Details

    • ForStFlinkFileSystem

      public ForStFlinkFileSystem(org.apache.flink.core.fs.FileSystem delegateFS, String remoteBase, String localBase, @Nullable FileBasedCache fileBasedCache)

  • Method Details

    • get

      public static ForStFlinkFileSystem get(URI uri) throws IOException
      Returns a reference to the FileSystem instance for accessing the file system identified by the given URI.
      Parameters:
      uri - the URI identifying the file system.
      Returns:
      a reference to the FileSystem instance for accessing the file system identified by the given URI.
      Throws:
      IOException - thrown if a reference to the file system instance could not be obtained.

    • get

      public static ForStFlinkFileSystem get(URI uri, org.apache.flink.core.fs.Path localBase, FileBasedCache fileBasedCache) throws IOException
      Throws:
      IOException

    • getFileBasedCache

      public static FileBasedCache getFileBasedCache(org.apache.flink.configuration.ReadableConfig config, org.apache.flink.core.fs.Path cacheBase, org.apache.flink.core.fs.Path remoteForStPath, long cacheCapacity, long cacheReservedSize, org.apache.flink.metrics.MetricGroup metricGroup) throws IOException
      Throws:
      IOException

    • getDelegateFS

      public org.apache.flink.core.fs.FileSystem getDelegateFS()

    • getRemoteBase

      public String getRemoteBase()

    • create

      public ByteBufferWritableFSDataOutputStream create(org.apache.flink.core.fs.Path path) throws IOException
      Create ByteBufferWritableFSDataOutputStream from specific path which supports to write data to ByteBuffer with FileSystem.WriteMode.OVERWRITE mode.
      Parameters:
      path - The file path to write to.
      Returns:
      The stream to the new file at the target path.
      Throws:
      IOException - Thrown, if the stream could not be opened because of an I/O, or because a file already exists at that path and the write mode indicates to not overwrite the file.

    • create

      public ByteBufferWritableFSDataOutputStream create(org.apache.flink.core.fs.Path dbFilePath, org.apache.flink.core.fs.FileSystem.WriteMode overwriteMode) throws IOException
      Specified by:
      create in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      create in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • open

      public ByteBufferReadableFSDataInputStream open(org.apache.flink.core.fs.Path dbFilePath, int bufferSize) throws IOException
      Specified by:
      open in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      open in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • open

      public ByteBufferReadableFSDataInputStream open(org.apache.flink.core.fs.Path dbFilePath) throws IOException
      Specified by:
      open in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      open in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • rename

      public boolean rename(org.apache.flink.core.fs.Path src, org.apache.flink.core.fs.Path dst) throws IOException
      Specified by:
      rename in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      rename in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • getWorkingDirectory

      public org.apache.flink.core.fs.Path getWorkingDirectory()
      Specified by:
      getWorkingDirectory in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      getWorkingDirectory in class org.apache.flink.core.fs.FileSystem

    • getHomeDirectory

      public org.apache.flink.core.fs.Path getHomeDirectory()
      Specified by:
      getHomeDirectory in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      getHomeDirectory in class org.apache.flink.core.fs.FileSystem

    • getUri

      public URI getUri()
      Specified by:
      getUri in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      getUri in class org.apache.flink.core.fs.FileSystem

    • exists

      public boolean exists(org.apache.flink.core.fs.Path f) throws IOException
      Specified by:
      exists in interface org.apache.flink.core.fs.IFileSystem
      Overrides:
      exists in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • getFileStatus

      public org.apache.flink.core.fs.FileStatus getFileStatus(org.apache.flink.core.fs.Path path) throws IOException
      Specified by:
      getFileStatus in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      getFileStatus in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • getFileBlockLocations

      public org.apache.flink.core.fs.BlockLocation[] getFileBlockLocations(org.apache.flink.core.fs.FileStatus file, long start, long len) throws IOException
      Specified by:
      getFileBlockLocations in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      getFileBlockLocations in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • listStatus

      public org.apache.flink.core.fs.FileStatus[] listStatus(org.apache.flink.core.fs.Path path) throws IOException
      Specified by:
      listStatus in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      listStatus in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • delete

      public boolean delete(org.apache.flink.core.fs.Path path, boolean recursive) throws IOException
      Specified by:
      delete in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      delete in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • mkdirs

      public boolean mkdirs(org.apache.flink.core.fs.Path path) throws IOException
      Specified by:
      mkdirs in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      mkdirs in class org.apache.flink.core.fs.FileSystem
      Throws:
      IOException

    • isDistributedFS

      public boolean isDistributedFS()
      Specified by:
      isDistributedFS in interface org.apache.flink.core.fs.IFileSystem
      Specified by:
      isDistributedFS in class org.apache.flink.core.fs.FileSystem

    • link

      public int link(org.apache.flink.core.fs.Path src, org.apache.flink.core.fs.Path dst) throws IOException
      Throws:
      IOException

    • link

      public int link(String src, org.apache.flink.core.fs.Path dst) throws IOException
      Throws:
      IOException

    • registerReusedRestoredFile

      public void registerReusedRestoredFile(String key, org.apache.flink.runtime.state.StreamStateHandle stateHandle, org.apache.flink.core.fs.Path dbFilePath)

    • getMappingEntry

      @Nullable public MappingEntry getMappingEntry(org.apache.flink.core.fs.Path path)

    • giveUpOwnership

      public void giveUpOwnership(org.apache.flink.core.fs.Path path, org.apache.flink.runtime.state.StreamStateHandle stateHandle)

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException