Package org.apache.flink.runtime.operators.sort

Class ExternalSorter<E>

java.lang.Object

org.apache.flink.runtime.operators.sort.ExternalSorter<E>

All Implemented Interfaces:

Closeable, AutoCloseable, Sorter<E>, CloseableInputProvider<E>

public class ExternalSorter<E>
extends Object
implements Sorter<E>

The ExternalSorter is a full fledged sorter. It implements a multi-way merge sort. Internally, the logic is factored into two or three threads (read, sort, spill) which communicate through a set of blocking queues, forming a closed loop. Memory is allocated using the MemoryManager interface. Thus the component will not exceed the provided memory limits.

Method Summary

Modifier and Type	Method	Description
void	close()	Shuts down all the threads initiated by this sort/merger.
org.apache.flink.util.MutableObjectIterator<E>	getIterator()	Gets the iterator over this input.
boolean	isClosed()
static <E> ExternalSorterBuilder<E>	newBuilder(MemoryManager memoryManager, AbstractInvokable parentTask, org.apache.flink.api.common.typeutils.TypeSerializer<E> serializer, org.apache.flink.api.common.typeutils.TypeComparator<E> comparator)	Creates a builder for the ExternalSorter.
static <E> ExternalSorterBuilder<E>	newBuilder(MemoryManager memoryManager, TaskInvokable parentTask, org.apache.flink.api.common.typeutils.TypeSerializer<E> serializer, org.apache.flink.api.common.typeutils.TypeComparator<E> comparator, org.apache.flink.api.common.ExecutionConfig executionConfig)	Creates a builder for the ExternalSorter.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

close

public void close()

Shuts down all the threads initiated by this sort/merger. Also releases all previously allocated memory, if it has not yet been released by the threads, and closes and deletes all channels (removing the temporary files).

The threads are set to exit directly, but depending on their operation, it may take a while to actually happen. The sorting thread will for example not finish before the current batch is sorted. This method attempts to wait for the working thread to exit. If it is however interrupted, the method exits immediately and is not guaranteed how long the threads continue to exist and occupy resources afterwards.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

See Also:

• Closeable.close()

isClosed

public boolean isClosed()

getIterator

public org.apache.flink.util.MutableObjectIterator<E> getIterator()
                                                           throws InterruptedException

Description copied from interface: CloseableInputProvider

Gets the iterator over this input.

Specified by:

getIterator in interface CloseableInputProvider<E>

Returns:

The iterator provided by this iterator provider.

Throws:

InterruptedException

newBuilder

public static <E> ExternalSorterBuilder<E> newBuilder(MemoryManager memoryManager,
 TaskInvokable parentTask,
 org.apache.flink.api.common.typeutils.TypeSerializer<E> serializer,
 org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
 org.apache.flink.api.common.ExecutionConfig executionConfig)

Creates a builder for the ExternalSorter.

newBuilder

public static <E> ExternalSorterBuilder<E> newBuilder(MemoryManager memoryManager,
 AbstractInvokable parentTask,
 org.apache.flink.api.common.typeutils.TypeSerializer<E> serializer,
 org.apache.flink.api.common.typeutils.TypeComparator<E> comparator)

Creates a builder for the ExternalSorter.