Package org.apache.flink.connector.file.table.utils

Class CompactFileUtils

java.lang.Object
org.apache.flink.connector.file.table.utils.CompactFileUtils
public class CompactFileUtils extends Object
Utils for compacting files.

  • Constructor Summary

    Constructors
    Constructor
    Description
    CompactFileUtils()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static <T> org.apache.flink.core.fs.Path
    doCompact(org.apache.flink.core.fs.FileSystem fileSystem, String partition, List<org.apache.flink.core.fs.Path> paths, org.apache.flink.core.fs.Path target, org.apache.flink.configuration.Configuration config, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory)
    Do Compaction: - Target file exists, do nothing. - Can do compaction: - Single file, do atomic renaming, there are optimizations for FileSystem. - Multiple file, do reading and writing.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • CompactFileUtils

      public CompactFileUtils()

  • Method Details

    • doCompact

      @Nullable public static <T> org.apache.flink.core.fs.Path doCompact(org.apache.flink.core.fs.FileSystem fileSystem, String partition, List<org.apache.flink.core.fs.Path> paths, org.apache.flink.core.fs.Path target, org.apache.flink.configuration.Configuration config, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory) throws IOException
      Do Compaction: - Target file exists, do nothing. - Can do compaction: - Single file, do atomic renaming, there are optimizations for FileSystem. - Multiple file, do reading and writing.
      Throws:
      IOException