Package org.apache.flink.runtime.io.disk.iomanager

Class AsynchronousFileIOChannel<T,R extends org.apache.flink.runtime.io.disk.iomanager.IORequest>

java.lang.Object
org.apache.flink.runtime.io.disk.iomanager.AbstractFileIOChannel
org.apache.flink.runtime.io.disk.iomanager.AsynchronousFileIOChannel<T,R>
Type Parameters:
R - The type of request (e.g. ReadRequest or WriteRequest issued by this access to the I/O threads.
All Implemented Interfaces:
FileIOChannel
Direct Known Subclasses:
AsynchronousBlockReader, AsynchronousBlockWriterWithCallback, AsynchronousBufferFileReader, AsynchronousBufferFileSegmentReader, AsynchronousBufferFileWriter, AsynchronousBulkBlockReader
public abstract class AsynchronousFileIOChannel<T,R extends org.apache.flink.runtime.io.disk.iomanager.IORequest> extends AbstractFileIOChannel
A base class for readers and writers that accept read or write requests for whole blocks. The request is delegated to an asynchronous I/O thread. After completion of the I/O request, the memory segment of the block is added to a collection to be returned.

The asynchrony of the access makes it possible to implement read-ahead or write-behind types of I/O accesses.

  • Field Details

    • closeLock

      protected final Object closeLock
      The lock that is used during closing to synchronize the thread that waits for all requests to be handled with the asynchronous I/O thread.

    • requestQueue

      protected final RequestQueue<R extends org.apache.flink.runtime.io.disk.iomanager.IORequest> requestQueue
      A request queue for submitting asynchronous requests to the corresponding IO worker thread.

    • requestsNotReturned

      protected final AtomicInteger requestsNotReturned
      An atomic integer that counts the number of requests that we still wait for to return.

    • resultHandler

      protected final RequestDoneCallback<T> resultHandler
      Handler for completed requests

    • exception

      protected volatile IOException exception
      An exception that was encountered by the asynchronous request handling thread.

    • closed

      protected volatile boolean closed
      Flag marking this channel as closed

  • Constructor Details

    • AsynchronousFileIOChannel

      protected AsynchronousFileIOChannel(FileIOChannel.ID channelID, RequestQueue<R> requestQueue, RequestDoneCallback<T> callback, boolean writeEnabled) throws IOException
      Creates a new channel access to the path indicated by the given ID. The channel accepts buffers to be read/written and hands them to the asynchronous I/O thread. After being processed, the buffers are returned by adding the to the given queue.
      Parameters:
      channelID - The id describing the path of the file that the channel accessed.
      requestQueue - The queue that this channel hands its IO requests to.
      callback - The callback to be invoked when a request is done.
      writeEnabled - Flag describing whether the channel should be opened in read/write mode, rather than in read-only mode.
      Throws:
      IOException - Thrown, if the channel could no be opened.

  • Method Details

    • isClosed

      public boolean isClosed()
      Description copied from interface: FileIOChannel
      Checks whether the channel has been closed.
      Specified by:
      isClosed in interface FileIOChannel
      Specified by:
      isClosed in class AbstractFileIOChannel
      Returns:
      True if the channel has been closed, false otherwise.

    • close

      public void close() throws IOException
      Closes the channel and waits until all pending asynchronous requests are processed. The underlying FileChannel is closed even if an exception interrupts the closing.

      Important: the isClosed() method returns true immediately after this method has been called even when there are outstanding requests.

      Specified by:
      close in interface FileIOChannel
      Specified by:
      close in class AbstractFileIOChannel
      Throws:
      IOException - Thrown, if an I/O exception occurred while waiting for the buffers, or if the closing was interrupted.

    • closeAndDelete

      public void closeAndDelete() throws IOException
      This method waits for all pending asynchronous requests to return. When the last request has returned, the channel is closed and deleted.

      Even if an exception interrupts the closing, such that not all request are handled, the underlying FileChannel is closed and deleted.

      Specified by:
      closeAndDelete in interface FileIOChannel
      Overrides:
      closeAndDelete in class AbstractFileIOChannel
      Throws:
      IOException - Thrown, if an I/O exception occurred while waiting for the buffers, or if the closing was interrupted.

    • checkErroneous

      public final void checkErroneous() throws IOException
      Checks the exception state of this channel. The channel is erroneous, if one of its requests could not be processed correctly.
      Throws:
      IOException - Thrown, if the channel is erroneous. The thrown exception contains the original exception that defined the erroneous state as its cause.

    • handleProcessedBuffer

      protected final void handleProcessedBuffer(T buffer, IOException ex)
      Handles a processed Buffer. This method is invoked by the asynchronous IO worker threads upon completion of the IO request with the provided buffer and/or an exception that occurred while processing the request for that buffer.
      Parameters:
      buffer - The buffer to be processed.
      ex - The exception that occurred in the I/O threads when processing the buffer's request.

    • addRequest

      protected final void addRequest(R request) throws IOException
      Throws:
      IOException

    • registerAllRequestsProcessedListener

      protected boolean registerAllRequestsProcessedListener(NotificationListener listener) throws IOException
      Registers a listener to be notified when all outstanding requests have been processed.

      New requests can arrive right after the listener got notified. Therefore, it is not safe to assume that the number of outstanding requests is still zero after a notification unless there was a close right before the listener got called.

      Returns true, if the registration was successful. A registration can fail, if there are no outstanding requests when trying to register a listener.

      Throws:
      IOException