Package org.apache.hadoop.fs

Class BufferedFSInputStream

java.lang.Object

java.io.InputStream

java.io.FilterInputStream

java.io.BufferedInputStream

org.apache.hadoop.fs.BufferedFSInputStream

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, HasFileDescriptor, PositionedReadable, Seekable, IOStatisticsSource, StreamCapabilities

@Private
@Unstable
public class BufferedFSInputStream
extends java.io.BufferedInputStream
implements Seekable, PositionedReadable, HasFileDescriptor, IOStatisticsSource, StreamCapabilities

A class that optimizes reading from FSInputStream by buffering.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.fs.StreamCapabilities

StreamCapabilities.StreamCapability

Field Summary

Fields inherited from class java.io.BufferedInputStream

buf, count, marklimit, markpos, pos

Fields inherited from class java.io.FilterInputStream

in

Fields inherited from interface org.apache.hadoop.fs.StreamCapabilities

ABORTABLE_STREAM, DROPBEHIND, HFLUSH, HSYNC, IOSTATISTICS, IOSTATISTICS_CONTEXT, PREADBYTEBUFFER, READAHEAD, READBYTEBUFFER, UNBUFFER, VECTOREDIO

Constructor Summary

Constructors

Constructor	Description
BufferedFSInputStream(FSInputStream in, int size)	Creates a BufferedFSInputStream with the specified buffer size, and saves its argument, the input stream in, for later use.

Method Summary

Modifier and Type	Method	Description
java.io.FileDescriptor	getFileDescriptor()
IOStatistics	getIOStatistics()	Return a statistics instance.
long	getPos()	Return the current offset from the start of the file
boolean	hasCapability(java.lang.String capability)	If the inner stream supports StreamCapabilities, forward the probe to it.
int	maxReadSizeForVectorReads()	What is the largest size that we should group ranges together as?
int	minSeekForVectorReads()	What is the smallest reasonable seek?
int	read(long position, byte[] buffer, int offset, int length)	Read up to the specified number of bytes, from a given position within a file, and return the number of bytes read.
void	readFully(long position, byte[] buffer)	Read number of bytes equal to the length of the buffer, from a given position within a file.
void	readFully(long position, byte[] buffer, int offset, int length)	Read the specified number of bytes, from a given position within a file.
void	readVectored(java.util.List<? extends FileRange> ranges, java.util.function.IntFunction<java.nio.ByteBuffer> allocate)	Read fully a list of file ranges asynchronously from this file.
void	seek(long pos)	Seek to the given offset from the start of the file.
boolean	seekToNewSource(long targetPos)	Seeks a different copy of the data.
long	skip(long n)
java.lang.String	toString()

Methods inherited from class java.io.BufferedInputStream

available, close, mark, markSupported, read, read, reset

Methods inherited from class java.io.FilterInputStream

read

Methods inherited from class java.io.InputStream

nullInputStream, readAllBytes, readNBytes, readNBytes, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

BufferedFSInputStream

public BufferedFSInputStream(FSInputStream in,
                             int size)

Creates a BufferedFSInputStream with the specified buffer size, and saves its argument, the input stream in, for later use. An internal buffer array of length size is created and stored in buf.

Parameters:

in - the underlying input stream.

size - the buffer size.

Throws:

java.lang.IllegalArgumentException - if size <= 0.

Method Detail

getPos

public long getPos()
            throws java.io.IOException

Description copied from interface: Seekable

Return the current offset from the start of the file

Specified by:

getPos in interface Seekable

Returns:

offset from the start of the file.

Throws:

java.io.IOException - raised on errors performing I/O.

skip

public long skip(long n)
          throws java.io.IOException

Overrides:

skip in class java.io.BufferedInputStream

Throws:

java.io.IOException

seek

public void seek(long pos)
          throws java.io.IOException

Description copied from interface: Seekable

Seek to the given offset from the start of the file. The next read() will be from that location. Can't seek past the end of the file.

Specified by:

seek in interface Seekable

Parameters:

pos - offset from the start of the file.

Throws:

java.io.IOException - raised on errors performing I/O.

seekToNewSource

public boolean seekToNewSource(long targetPos)
                        throws java.io.IOException

Description copied from interface: Seekable

Seeks a different copy of the data. Returns true if found a new source, false otherwise.

Specified by:

seekToNewSource in interface Seekable

Parameters:

targetPos - target position.

Returns:

true if found a new source, false otherwise.

Throws:

java.io.IOException - raised on errors performing I/O.

read

public int read(long position,
                byte[] buffer,
                int offset,
                int length)
         throws java.io.IOException

Description copied from interface: PositionedReadable

Read up to the specified number of bytes, from a given position within a file, and return the number of bytes read. This does not change the current offset of a file, and is thread-safe. Warning: Not all filesystems satisfy the thread-safety requirement.

Specified by:

read in interface PositionedReadable

Parameters:

position - position within file

buffer - destination buffer

offset - offset in the buffer

length - number of bytes to read

Returns:

actual number of bytes read; -1 means "none"

Throws:

java.io.IOException - IO problems.

readFully

public void readFully(long position,
                      byte[] buffer,
                      int offset,
                      int length)
               throws java.io.IOException

Description copied from interface: PositionedReadable

Read the specified number of bytes, from a given position within a file. This does not change the current offset of a file, and is thread-safe. Warning: Not all filesystems satisfy the thread-safety requirement.

Specified by:

readFully in interface PositionedReadable

Parameters:

position - position within file

buffer - destination buffer

offset - offset in the buffer

length - number of bytes to read

Throws:

java.io.IOException - IO problems.

java.io.EOFException - the end of the data was reached before the read operation completed

readFully

public void readFully(long position,
                      byte[] buffer)
               throws java.io.IOException

Description copied from interface: PositionedReadable

Read number of bytes equal to the length of the buffer, from a given position within a file. This does not change the current offset of a file, and is thread-safe. Warning: Not all filesystems satisfy the thread-safety requirement.

Specified by:

readFully in interface PositionedReadable

Parameters:

position - position within file

buffer - destination buffer

Throws:

java.io.IOException - IO problems.

java.io.EOFException - the end of the data was reached before the read operation completed

getFileDescriptor

public java.io.FileDescriptor getFileDescriptor()
                                         throws java.io.IOException

Specified by:

getFileDescriptor in interface HasFileDescriptor

Returns:

the FileDescriptor

Throws:

java.io.IOException - raised on errors performing I/O.

hasCapability

public boolean hasCapability(java.lang.String capability)

If the inner stream supports StreamCapabilities, forward the probe to it. Otherwise: return false.

Specified by:

hasCapability in interface StreamCapabilities

Parameters:

capability - string to query the stream support for.

Returns:

true if a capability is known to be supported.

getIOStatistics

public IOStatistics getIOStatistics()

Description copied from interface: IOStatisticsSource

Return a statistics instance.

It is not a requirement that the same instance is returned every time. IOStatisticsSource.

If the object implementing this is Closeable, this method may return null if invoked on a closed object, even if it returns a valid instance when called earlier.

Specified by:

getIOStatistics in interface IOStatisticsSource

Returns:

an IOStatistics instance or null

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

minSeekForVectorReads

public int minSeekForVectorReads()

Description copied from interface: PositionedReadable

What is the smallest reasonable seek?

Specified by:

minSeekForVectorReads in interface PositionedReadable

Returns:

the minimum number of bytes

maxReadSizeForVectorReads

public int maxReadSizeForVectorReads()

Description copied from interface: PositionedReadable

What is the largest size that we should group ranges together as?

Specified by:

maxReadSizeForVectorReads in interface PositionedReadable

Returns:

the number of bytes to read at once

readVectored

public void readVectored(java.util.List<? extends FileRange> ranges,
                         java.util.function.IntFunction<java.nio.ByteBuffer> allocate)
                  throws java.io.IOException

Description copied from interface: PositionedReadable

Read fully a list of file ranges asynchronously from this file. The default iterates through the ranges to read each synchronously, but the intent is that FSDataInputStream subclasses can make more efficient readers. As a result of the call, each range will have FileRange.setData(CompletableFuture) called with a future that when complete will have a ByteBuffer with the data from the file's range.

The position returned by getPos() after readVectored() is undefined.

If a file is changed while the readVectored() operation is in progress, the output is undefined. Some ranges may have old data, some may have new and some may have both.

While a readVectored() operation is in progress, normal read api calls may block.

Specified by:

readVectored in interface PositionedReadable

Parameters:

ranges - the byte ranges to read

allocate - the function to allocate ByteBuffer

Throws:

java.io.IOException - any IOE.