Package org.apache.flink.table.runtime.operators.sort

Class AbstractBinaryExternalMerger<Entry>

java.lang.Object
org.apache.flink.table.runtime.operators.sort.AbstractBinaryExternalMerger<Entry>
Type Parameters:
Entry - Type of Entry to Merge sort.
All Implemented Interfaces:
Closeable, AutoCloseable
Direct Known Subclasses:
BinaryExternalMerger, BinaryKVExternalMerger
public abstract class AbstractBinaryExternalMerger<Entry> extends Object implements Closeable
Spilled files Merger of BinaryExternalSorter. It merges maxFanIn spilled files at most once.

  • Field Details

    • pageSize

      protected final int pageSize

    • ioManager

      protected final org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager

  • Constructor Details

    • AbstractBinaryExternalMerger

      public AbstractBinaryExternalMerger(org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, int pageSize, int maxFanIn, SpillChannelManager channelManager, boolean compressionEnabled, org.apache.flink.runtime.io.compression.BlockCompressionFactory compressionCodecFactory, int compressionBlockSize)

  • Method Details

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable

    • getMergingIterator

      public BinaryMergeIterator<Entry> getMergingIterator(List<ChannelWithMeta> channelIDs, List<org.apache.flink.runtime.io.disk.iomanager.FileIOChannel> openChannels) throws IOException
      Returns an iterator that iterates over the merged result from all given channels.
      Parameters:
      channelIDs - The channels that are to be merged and returned.
      Returns:
      An iterator over the merged records of the input channels.
      Throws:
      IOException - Thrown, if the readers encounter an I/O problem.

    • mergeChannelList

      public List<ChannelWithMeta> mergeChannelList(List<ChannelWithMeta> channelIDs) throws IOException
      Merges the given sorted runs to a smaller number of sorted runs.
      Parameters:
      channelIDs - The IDs of the sorted runs that need to be merged.
      Returns:
      A list of the IDs of the merged channels.
      Throws:
      IOException - Thrown, if the readers or writers encountered an I/O problem.

    • channelReaderInputViewIterator

      protected abstract org.apache.flink.util.MutableObjectIterator<Entry> channelReaderInputViewIterator(org.apache.flink.runtime.io.disk.iomanager.AbstractChannelReaderInputView inView)
      Returns:
      entry iterator reading from inView.

    • mergeComparator

      protected abstract Comparator<Entry> mergeComparator()
      Returns:
      merging comparator used in merging.

    • mergeReusedEntries

      protected abstract List<Entry> mergeReusedEntries(int size)
      Returns:
      reused entry object used in merging.

    • writeMergingOutput

      protected abstract void writeMergingOutput(org.apache.flink.util.MutableObjectIterator<Entry> mergeIterator, org.apache.flink.runtime.memory.AbstractPagedOutputView output) throws IOException
      read the merged stream and write the data back.
      Throws:
      IOException