Package org.apache.flink.runtime.io.disk.iomanager

Class AsynchronousBlockReader

java.lang.Object

org.apache.flink.runtime.io.disk.iomanager.AbstractFileIOChannel

org.apache.flink.runtime.io.disk.iomanager.AsynchronousFileIOChannel<org.apache.flink.core.memory.MemorySegment,org.apache.flink.runtime.io.disk.iomanager.ReadRequest>

org.apache.flink.runtime.io.disk.iomanager.AsynchronousBlockReader

All Implemented Interfaces:

BlockChannelReader<org.apache.flink.core.memory.MemorySegment>, FileIOChannel

public class AsynchronousBlockReader
extends AsynchronousFileIOChannel<org.apache.flink.core.memory.MemorySegment,org.apache.flink.runtime.io.disk.iomanager.ReadRequest>
implements BlockChannelReader<org.apache.flink.core.memory.MemorySegment>

A reader that reads data in blocks from a file channel. The reader reads the blocks into a MemorySegment in an asynchronous fashion. That is, a read request is not processed by the thread that issues it, but by an asynchronous reader thread. Once the read request is done, the asynchronous reader adds the full MemorySegment to a return queue where it can be popped by the worker thread, once it needs the data. The return queue is in this case a LinkedBlockingQueue, such that the working thread blocks until the request has been served, if the request is still pending when the it requires the data.

Typical pre-fetching reads are done by issuing the read requests early and popping the return queue once the data is actually needed.

The reader has no notion whether the size of the memory segments is actually the size of the blocks on disk, or even whether the file was written in blocks of the same size, or in blocks at all. Ensuring that the writing and reading is consistent with each other (same blocks sizes) is up to the programmer.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.disk.iomanager.FileIOChannel

FileIOChannel.Enumerator, FileIOChannel.ID

Field Summary

Fields inherited from class org.apache.flink.runtime.io.disk.iomanager.AsynchronousFileIOChannel

closed, closeLock, exception, requestQueue, requestsNotReturned, resultHandler

Fields inherited from class org.apache.flink.runtime.io.disk.iomanager.AbstractFileIOChannel

fileChannel, id, LOG

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	AsynchronousBlockReader(FileIOChannel.ID channelID, RequestQueue<org.apache.flink.runtime.io.disk.iomanager.ReadRequest> requestQueue, LinkedBlockingQueue<org.apache.flink.core.memory.MemorySegment> returnSegments)	Creates a new block channel reader for the given channel.

Method Summary

Modifier and Type	Method	Description
org.apache.flink.core.memory.MemorySegment	getNextReturnedBlock()	Gets the next memory segment that has been filled with data by the reader.
LinkedBlockingQueue<org.apache.flink.core.memory.MemorySegment>	getReturnQueue()	Gets the queue in which the full memory segments are queued after the asynchronous read is complete.
void	readBlock(org.apache.flink.core.memory.MemorySegment segment)	Issues a read request, which will asynchronously fill the given segment with the next block in the underlying file channel.
void	seekToPosition(long position)

Methods inherited from class org.apache.flink.runtime.io.disk.iomanager.AsynchronousFileIOChannel

addRequest, checkErroneous, close, closeAndDelete, handleProcessedBuffer, isClosed, registerAllRequestsProcessedListener

Methods inherited from class org.apache.flink.runtime.io.disk.iomanager.AbstractFileIOChannel

deleteChannel, getChannelID, getNioFileChannel, getSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.io.disk.iomanager.FileIOChannel

close, closeAndDelete, deleteChannel, getChannelID, getNioFileChannel, getSize, isClosed

Constructor Details

AsynchronousBlockReader

protected AsynchronousBlockReader(FileIOChannel.ID channelID,
 RequestQueue<org.apache.flink.runtime.io.disk.iomanager.ReadRequest> requestQueue,
 LinkedBlockingQueue<org.apache.flink.core.memory.MemorySegment> returnSegments)
                           throws IOException

Creates a new block channel reader for the given channel.

Parameters:

channelID - The ID of the channel to read.

requestQueue - The request queue of the asynchronous reader thread, to which the I/O requests are added.

returnSegments - The return queue, to which the full Memory Segments are added.

Throws:

IOException - Thrown, if the underlying file channel could not be opened.

Method Details

readBlock

public void readBlock(org.apache.flink.core.memory.MemorySegment segment)
               throws IOException

Issues a read request, which will asynchronously fill the given segment with the next block in the underlying file channel. Once the read request is fulfilled, the segment will be added to this reader's return queue.

Specified by:

readBlock in interface BlockChannelReader<org.apache.flink.core.memory.MemorySegment>

Parameters:

segment - The segment to read the block into.

Throws:

IOException - Thrown, when the reader encounters an I/O error. Due to the asynchronous nature of the reader, the exception thrown here may have been caused by an earlier read request.

seekToPosition

public void seekToPosition(long position)
                    throws IOException

Specified by:

seekToPosition in interface BlockChannelReader<org.apache.flink.core.memory.MemorySegment>

Throws:

IOException

getNextReturnedBlock

public org.apache.flink.core.memory.MemorySegment getNextReturnedBlock()
                                                                throws IOException

Gets the next memory segment that has been filled with data by the reader. This method blocks until such a segment is available, or until an error occurs in the reader, or the reader is closed.

WARNING: If this method is invoked without any segment ever returning (for example, because the readBlock(MemorySegment) method has not been invoked appropriately), the method may block forever.

Specified by:

getNextReturnedBlock in interface BlockChannelReader<org.apache.flink.core.memory.MemorySegment>

Returns:

The next memory segment from the reader's return queue.

Throws:

IOException - Thrown, if an I/O error occurs in the reader while waiting for the request to return.

getReturnQueue

public LinkedBlockingQueue<org.apache.flink.core.memory.MemorySegment> getReturnQueue()

Gets the queue in which the full memory segments are queued after the asynchronous read is complete.

Specified by:

getReturnQueue in interface BlockChannelReader<org.apache.flink.core.memory.MemorySegment>

Returns:

The queue with the full memory segments.