Package org.apache.flink.runtime.operators.sort

Class AbstractMergeIterator<T1,T2,O>

java.lang.Object
org.apache.flink.runtime.operators.sort.AbstractMergeIterator<T1,T2,O>
All Implemented Interfaces:
JoinTaskIterator<T1,T2,O>
Direct Known Subclasses:
AbstractMergeInnerJoinIterator, AbstractMergeOuterJoinIterator
public abstract class AbstractMergeIterator<T1,T2,O> extends Object implements JoinTaskIterator<T1,T2,O>

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected T2
    blockHeadCopy
     
    protected T1
    copy1
     
    protected T2
    copy2
     
    protected KeyGroupedIterator<T1>
    iterator1
     
    protected KeyGroupedIterator<T2>
    iterator2
     
    protected org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2>
    pairComparator
     
    protected final org.apache.flink.api.common.typeutils.TypeSerializer<T1>
    serializer1
     
    protected final org.apache.flink.api.common.typeutils.TypeSerializer<T2>
    serializer2
     
    protected T1
    spillHeadCopy
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    AbstractMergeIterator(org.apache.flink.util.MutableObjectIterator<T1> input1, org.apache.flink.util.MutableObjectIterator<T2> input2, org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1, org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1, org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2, org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2, org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator, MemoryManager memoryManager, IOManager ioManager, int numMemoryPages, AbstractInvokable parentTask)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    abort()
    Aborts the matching process.
    abstract boolean
    callWithNextKey(org.apache.flink.api.common.functions.FlatJoinFunction<T1,T2,O> joinFunction, org.apache.flink.util.Collector<O> collector)
    Calls the JoinFunction#join() method for all two key-value pairs that share the same key and come from different inputs.
    void
    close()
    General-purpose close method.
    protected abstract <T> T
    createCopy(org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer, T value, T reuse)
    Copies an instance of the given type, potentially reusing the object passed as the reuse parameter, which may be null.
    protected abstract <T> KeyGroupedIterator<T>
    createKeyGroupedIterator(org.apache.flink.util.MutableObjectIterator<T> input, org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer, org.apache.flink.api.common.typeutils.TypeComparator<T> comparator)
     
    protected void
    crossMatchingGroup(Iterator<T1> values1, Iterator<T2> values2, org.apache.flink.api.common.functions.FlatJoinFunction<T1,T2,O> joinFunction, org.apache.flink.util.Collector<O> collector)
     
    void
    open()
    General-purpose open method.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • pairComparator

      protected org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator

    • iterator1

      protected KeyGroupedIterator<T1> iterator1

    • iterator2

      protected KeyGroupedIterator<T2> iterator2

    • serializer1

      protected final org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1

    • serializer2

      protected final org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2

    • copy1

      protected T1 copy1

    • spillHeadCopy

      protected T1 spillHeadCopy

    • copy2

      protected T2 copy2

    • blockHeadCopy

      protected T2 blockHeadCopy

  • Constructor Details

    • AbstractMergeIterator

      public AbstractMergeIterator(org.apache.flink.util.MutableObjectIterator<T1> input1, org.apache.flink.util.MutableObjectIterator<T2> input2, org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1, org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1, org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2, org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2, org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator, MemoryManager memoryManager, IOManager ioManager, int numMemoryPages, AbstractInvokable parentTask) throws MemoryAllocationException
      Throws:
      MemoryAllocationException

  • Method Details

    • open

      public void open() throws IOException
      Description copied from interface: JoinTaskIterator
      General-purpose open method. Initializes the internal strategy (for example triggers the sorting of the inputs or starts building hash tables).
      Specified by:
      open in interface JoinTaskIterator<T1,T2,O>
      Throws:
      IOException - Thrown, if an I/O error occurred while preparing the data. An example is a failing external sort.

    • close

      public void close()
      Description copied from interface: JoinTaskIterator
      General-purpose close method. Works after the principle of best effort. The internal structures are released, but errors that occur on the way are not reported.
      Specified by:
      close in interface JoinTaskIterator<T1,T2,O>

    • abort

      public void abort()
      Description copied from interface: JoinTaskIterator
      Aborts the matching process. This extra abort method is supplied, because a significant time may pass while calling the match stub with the cross product of all values that share the same key. A call to this abort method signals an interrupt to that procedure.
      Specified by:
      abort in interface JoinTaskIterator<T1,T2,O>

    • callWithNextKey

      public abstract boolean callWithNextKey(org.apache.flink.api.common.functions.FlatJoinFunction<T1,T2,O> joinFunction, org.apache.flink.util.Collector<O> collector) throws Exception
      Calls the JoinFunction#join() method for all two key-value pairs that share the same key and come from different inputs. The output of the join() method is forwarded.

      This method first zig-zags between the two sorted inputs in order to find a common key, and then calls the join stub with the cross product of the values.

      Specified by:
      callWithNextKey in interface JoinTaskIterator<T1,T2,O>
      Parameters:
      joinFunction - The match stub containing the match function which is called with the keys.
      collector - The collector to pass the match function.
      Returns:
      True, if a next key exists, false if no more keys exist.
      Throws:
      Exception - Forwards all exceptions from the user code and the I/O system.
      See Also:

    • crossMatchingGroup

      protected void crossMatchingGroup(Iterator<T1> values1, Iterator<T2> values2, org.apache.flink.api.common.functions.FlatJoinFunction<T1,T2,O> joinFunction, org.apache.flink.util.Collector<O> collector) throws Exception
      Throws:
      Exception

    • createKeyGroupedIterator

      protected abstract <T> KeyGroupedIterator<T> createKeyGroupedIterator(org.apache.flink.util.MutableObjectIterator<T> input, org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer, org.apache.flink.api.common.typeutils.TypeComparator<T> comparator)

    • createCopy

      protected abstract <T> T createCopy(org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer, T value, T reuse)
      Copies an instance of the given type, potentially reusing the object passed as the reuse parameter, which may be null.