Package org.apache.flink.runtime.io.network.api.serialization

Class SpillingAdaptiveSpanningRecordDeserializer<T extends org.apache.flink.core.io.IOReadableWritable>

java.lang.Object

org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer<T>

Type Parameters:

T - The type of the record to be deserialized.

All Implemented Interfaces:

RecordDeserializer<T>

public class SpillingAdaptiveSpanningRecordDeserializer<T extends org.apache.flink.core.io.IOReadableWritable>
extends Object
implements RecordDeserializer<T>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.network.api.serialization.RecordDeserializer

RecordDeserializer.DeserializationResult

Field Summary

Fields

Modifier and Type	Field	Description
static final int	DEFAULT_FILE_BUFFER_SIZE
static final int	DEFAULT_THRESHOLD_FOR_SPILLING

Constructor Summary

Constructors

Constructor	Description
SpillingAdaptiveSpanningRecordDeserializer(String[] tmpDirectories)
SpillingAdaptiveSpanningRecordDeserializer(String[] tmpDirectories, int thresholdForSpilling, int fileBufferSize)

Method Summary

Modifier and Type	Method	Description
void	clear()
RecordDeserializer.DeserializationResult	getNextRecord(T target)
org.apache.flink.util.CloseableIterator<Buffer>	getUnconsumedBuffer()	Gets the unconsumed buffer which needs to be persisted in unaligned checkpoint scenario.
void	setNextBuffer(Buffer buffer)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DEFAULT_THRESHOLD_FOR_SPILLING

public static final int DEFAULT_THRESHOLD_FOR_SPILLING

See Also:

• Constant Field Values

DEFAULT_FILE_BUFFER_SIZE

public static final int DEFAULT_FILE_BUFFER_SIZE

See Also:

• Constant Field Values

Constructor Details

SpillingAdaptiveSpanningRecordDeserializer

public SpillingAdaptiveSpanningRecordDeserializer(String[] tmpDirectories)

SpillingAdaptiveSpanningRecordDeserializer

public SpillingAdaptiveSpanningRecordDeserializer(String[] tmpDirectories,
 int thresholdForSpilling,
 int fileBufferSize)

Method Details

setNextBuffer

public void setNextBuffer(Buffer buffer)
                   throws IOException

Specified by:

setNextBuffer in interface RecordDeserializer<T extends org.apache.flink.core.io.IOReadableWritable>

Throws:

IOException

getUnconsumedBuffer

public org.apache.flink.util.CloseableIterator<Buffer> getUnconsumedBuffer()
                                                                    throws IOException

Description copied from interface: RecordDeserializer

Gets the unconsumed buffer which needs to be persisted in unaligned checkpoint scenario.

Note that the unconsumed buffer might be null if the whole buffer was already consumed before and there are no partial length or data remained in the end of buffer.

Specified by:

getUnconsumedBuffer in interface RecordDeserializer<T extends org.apache.flink.core.io.IOReadableWritable>

Throws:

IOException

getNextRecord

public RecordDeserializer.DeserializationResult getNextRecord(T target)
                                                       throws IOException

Specified by:

getNextRecord in interface RecordDeserializer<T extends org.apache.flink.core.io.IOReadableWritable>

Throws:

IOException

clear

public void clear()

Specified by:

clear in interface RecordDeserializer<T extends org.apache.flink.core.io.IOReadableWritable>