Uses of Interface
org.apache.flink.connector.file.table.stream.compact.CompactReader.Factory
Packages that use CompactReader.Factory
Package
Description
-
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.batch
Methods in org.apache.flink.connector.file.table.batch with parameters of type CompactReader.FactoryModifier and TypeMethodDescriptionstatic <T> org.apache.flink.streaming.api.datastream.DataStreamSink<?>BatchSink.createBatchCompactSink(org.apache.flink.streaming.api.datastream.DataStream<CompactMessages.CoordinatorInput> dataStream, org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<T, String, ? extends org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<T, String, ?>> builder, CompactReader.Factory<T> readFactory, FileSystemFactory fsFactory, TableMetaStoreFactory metaStoreFactory, PartitionCommitPolicyFactory partitionCommitPolicyFactory, String[] partitionColumns, LinkedHashMap<String, String> staticPartitionSpec, org.apache.flink.core.fs.Path tmpPath, org.apache.flink.table.catalog.ObjectIdentifier identifier, long compactAverageSize, long compactTargetSize, boolean isToLocal, boolean overwrite, int compactParallelism, boolean compactParallelismConfigured) -
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.batch.compact
Constructors in org.apache.flink.connector.file.table.batch.compact with parameters of type CompactReader.FactoryModifierConstructorDescriptionBatchCompactOperator(org.apache.flink.util.function.SupplierWithException<org.apache.flink.core.fs.FileSystem, IOException> fsFactory, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory) -
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.stream
Methods in org.apache.flink.connector.file.table.stream with parameters of type CompactReader.FactoryModifier and TypeMethodDescriptionstatic <T> org.apache.flink.streaming.api.datastream.DataStream<PartitionCommitInfo>StreamingSink.compactionWriter(org.apache.flink.table.connector.ProviderContext providerContext, org.apache.flink.streaming.api.datastream.DataStream<T> inputStream, long bucketCheckInterval, org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<T, String, ? extends org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<T, String, ?>> bucketsBuilder, FileSystemFactory fsFactory, org.apache.flink.core.fs.Path path, CompactReader.Factory<T> readFactory, long targetFileSize, int parallelism, boolean parallelismConfigured) Create a file writer with compaction operators by input stream. -
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.stream.compact
Methods in org.apache.flink.connector.file.table.stream.compact that return CompactReader.FactoryModifier and TypeMethodDescriptionstatic <T> CompactReader.Factory<T>CompactBulkReader.factory(BulkFormat<T, FileSourceSplit> format) static <T> CompactReader.Factory<T>FileInputFormatCompactReader.factory(org.apache.flink.api.common.io.FileInputFormat<T> format) Constructors in org.apache.flink.connector.file.table.stream.compact with parameters of type CompactReader.FactoryModifierConstructorDescriptionCompactOperator(org.apache.flink.util.function.SupplierWithException<org.apache.flink.core.fs.FileSystem, IOException> fsFactory, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory) -
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.utils
Methods in org.apache.flink.connector.file.table.utils with parameters of type CompactReader.FactoryModifier and TypeMethodDescriptionstatic <T> org.apache.flink.core.fs.PathCompactFileUtils.doCompact(org.apache.flink.core.fs.FileSystem fileSystem, String partition, List<org.apache.flink.core.fs.Path> paths, org.apache.flink.core.fs.Path target, org.apache.flink.configuration.Configuration config, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory) Do Compaction: - Target file exists, do nothing. - Can do compaction: - Single file, do atomic renaming, there are optimizations for FileSystem. - Multiple file, do reading and writing.