Package org.apache.hadoop.hdfs.web

Class ByteRangeInputStream

java.lang.Object

java.io.InputStream

org.apache.hadoop.fs.FSInputStream

org.apache.hadoop.hdfs.web.ByteRangeInputStream

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.fs.PositionedReadable, org.apache.hadoop.fs.Seekable

public abstract class ByteRangeInputStream
extends org.apache.hadoop.fs.FSInputStream

To support HTTP byte streams, a new connection to an HTTP server needs to be created each time. This class hides the complexity of those multiple connections from the client. Whenever seek() is called, a new connection is made on the successive read(). The normal input stream functions are connected to the currently active input stream.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ByteRangeInputStream.URLOpener	This class wraps a URL and provides method to open connection.

Field Summary

Fields

Modifier and Type	Field	Description
protected long	currentPos
protected java.lang.Long	fileLength
protected java.io.InputStream	in
protected ByteRangeInputStream.URLOpener	originalURL
protected ByteRangeInputStream.URLOpener	resolvedURL
protected long	startPos

Constructor Summary

Constructors

Constructor	Description
ByteRangeInputStream(ByteRangeInputStream.URLOpener o, ByteRangeInputStream.URLOpener r)	Create with the specified URLOpeners.

Method Summary

Modifier and Type	Method	Description
int	available()
void	close()
protected java.io.InputStream	getInputStream()
long	getPos()	Return the current offset from the start of the file
protected abstract java.net.URL	getResolvedUrl(java.net.HttpURLConnection connection)
protected org.apache.hadoop.hdfs.web.ByteRangeInputStream.InputStreamAndFileLength	openInputStream(long startOffset)
int	read()
int	read(byte[] b, int off, int len)
int	read(long position, byte[] buffer, int offset, int length)
void	readFully(long position, byte[] buffer, int offset, int length)
void	seek(long pos)	Seek to the given offset from the start of the file.
boolean	seekToNewSource(long targetPos)	Seeks a different copy of the data.

Methods inherited from class org.apache.hadoop.fs.FSInputStream

readFully, toString, validatePositionedReadArgs

Methods inherited from class java.io.InputStream

mark, markSupported, nullInputStream, read, readAllBytes, readNBytes, readNBytes, reset, skip, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.fs.PositionedReadable

maxReadSizeForVectorReads, minSeekForVectorReads, readVectored

Field Detail

in

protected java.io.InputStream in

originalURL

protected final ByteRangeInputStream.URLOpener originalURL

resolvedURL

protected final ByteRangeInputStream.URLOpener resolvedURL

startPos

protected long startPos

currentPos

protected long currentPos

fileLength

protected java.lang.Long fileLength

Constructor Detail

ByteRangeInputStream

public ByteRangeInputStream(ByteRangeInputStream.URLOpener o,
                            ByteRangeInputStream.URLOpener r)
                     throws java.io.IOException

Create with the specified URLOpeners. Original url is used to open the stream for the first time. Resolved url is used in subsequent requests.

Parameters:

o - Original url

r - Resolved url

Throws:

java.io.IOException

Method Detail

getResolvedUrl

protected abstract java.net.URL getResolvedUrl(java.net.HttpURLConnection connection)
                                        throws java.io.IOException

Throws:

java.io.IOException

getInputStream

@VisibleForTesting
protected java.io.InputStream getInputStream()
                                      throws java.io.IOException

Throws:

java.io.IOException

openInputStream

@VisibleForTesting
protected org.apache.hadoop.hdfs.web.ByteRangeInputStream.InputStreamAndFileLength openInputStream(long startOffset)
                                                                                            throws java.io.IOException

Throws:

java.io.IOException

read

public int read()
         throws java.io.IOException

Specified by:

read in class java.io.InputStream

Throws:

java.io.IOException

read

public int read(@Nonnull
                byte[] b,
                int off,
                int len)
         throws java.io.IOException

Overrides:

read in class java.io.InputStream

Throws:

java.io.IOException

seek

public void seek(long pos)
          throws java.io.IOException

Seek to the given offset from the start of the file. The next read() will be from that location. Can't seek past the end of the file.

Specified by:

seek in interface org.apache.hadoop.fs.Seekable

Specified by:

seek in class org.apache.hadoop.fs.FSInputStream

Throws:

java.io.IOException

read

public int read(long position,
                byte[] buffer,
                int offset,
                int length)
         throws java.io.IOException

Specified by:

read in interface org.apache.hadoop.fs.PositionedReadable

Overrides:

read in class org.apache.hadoop.fs.FSInputStream

Throws:

java.io.IOException

readFully

public void readFully(long position,
                      byte[] buffer,
                      int offset,
                      int length)
               throws java.io.IOException

Specified by:

readFully in interface org.apache.hadoop.fs.PositionedReadable

Overrides:

readFully in class org.apache.hadoop.fs.FSInputStream

Throws:

java.io.IOException

getPos

public long getPos()
            throws java.io.IOException

Return the current offset from the start of the file

Specified by:

getPos in interface org.apache.hadoop.fs.Seekable

Specified by:

getPos in class org.apache.hadoop.fs.FSInputStream

Throws:

java.io.IOException

seekToNewSource

public boolean seekToNewSource(long targetPos)
                        throws java.io.IOException

Seeks a different copy of the data. Returns true if found a new source, false otherwise.

Specified by:

seekToNewSource in interface org.apache.hadoop.fs.Seekable

Specified by:

seekToNewSource in class org.apache.hadoop.fs.FSInputStream

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Overrides:

close in class java.io.InputStream

Throws:

java.io.IOException

available

public int available()
              throws java.io.IOException

Overrides:

available in class java.io.InputStream

Throws:

java.io.IOException