Package org.apache.flink.connector.file.table.batch.compact

Class BatchFileWriter<T>

java.lang.Object
org.apache.flink.streaming.api.operators.AbstractStreamOperator<CompactMessages.CoordinatorInput>
org.apache.flink.connector.file.table.batch.compact.BatchFileWriter<T>
All Implemented Interfaces:
Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<T>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<T,CompactMessages.CoordinatorInput>, org.apache.flink.streaming.api.operators.StreamOperator<CompactMessages.CoordinatorInput>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.YieldingOperator<CompactMessages.CoordinatorInput>
public class BatchFileWriter<T> extends org.apache.flink.streaming.api.operators.AbstractStreamOperator<CompactMessages.CoordinatorInput> implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<T,CompactMessages.CoordinatorInput>, org.apache.flink.streaming.api.operators.BoundedOneInput
An operator for writing files in batch mode. Once creating a new file to write, the writing operator will emit the written file to downstream.
See Also:

  • Field Summary

    Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

    combinedWatermark, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

  • Constructor Summary

    Constructors
    Constructor
    Description
    BatchFileWriter(FileSystemFactory fsFactory, org.apache.flink.core.fs.Path tmpPath, String[] partitionColumns, boolean dynamicGrouped, LinkedHashMap<String,String> staticPartitions, OutputFormatFactory<T> formatFactory, PartitionComputer<T> computer, org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig outputFileConfig)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
     
    void
    endInput()
     
    void
    open()
     
    void
    processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<T> element)
     

    Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

    beforeInitializeStateHandler, finish, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark, processWatermark1, processWatermark1, processWatermark2, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

    notifyCheckpointAborted, notifyCheckpointComplete

    Methods inherited from interface org.apache.flink.streaming.api.operators.Input

    processLatencyMarker, processRecordAttributes, processWatermark, processWatermark, processWatermarkStatus

    Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

    getCurrentKey, setCurrentKey

    Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

    hasKeyContext

    Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

    setKeyContextElement

    Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

    finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

  • Constructor Details

  • Method Details

    • open

      public void open() throws Exception
      Specified by:
      open in interface org.apache.flink.streaming.api.operators.StreamOperator<T>
      Overrides:
      open in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<CompactMessages.CoordinatorInput>
      Throws:
      Exception

    • processElement

      public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<T> element) throws Exception
      Specified by:
      processElement in interface org.apache.flink.streaming.api.operators.Input<T>
      Throws:
      Exception

    • endInput

      public void endInput() throws Exception
      Specified by:
      endInput in interface org.apache.flink.streaming.api.operators.BoundedOneInput
      Throws:
      Exception

    • close

      public void close() throws Exception
      Specified by:
      close in interface org.apache.flink.streaming.api.operators.StreamOperator<T>
      Overrides:
      close in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<CompactMessages.CoordinatorInput>
      Throws:
      Exception