Package org.apache.hadoop.hdfs.web

Class WebHdfsFileSystem.ReadRunner

java.lang.Object

org.apache.hadoop.hdfs.web.WebHdfsFileSystem.ReadRunner

Enclosing class:

WebHdfsFileSystem

@VisibleForTesting
protected class WebHdfsFileSystem.ReadRunner
extends java.lang.Object

This class will allow retries to occur for both open and read operations. The first WebHdfsFileSystem#open creates a new WebHdfsInputStream object, which creates a new ReadRunner object that will be used to open a connection and read or seek into the input stream. ReadRunner is a subclass of the AbstractRunner class, which will run the ReadRunner#getUrl(), ReadRunner#connect(URL), and ReadRunner#getResponse methods within a retry loop, based on the configured retry policy. ReadRunner#connect will create a connection if one has not already been created. Otherwise, it will return the previously created connection object. This is necessary because a new connection should not be created for every read. Likewise, ReadRunner#getUrl will construct a new URL object only if the connection has not previously been established. Otherwise, it will return the previously created URL object. ReadRunner#getResponse will initialize the input stream if it has not already been initialized and read the requested data from the specified input stream.

Field Summary

Fields

Modifier and Type	Field	Description
protected ExcludeDatanodesParam	excludeDatanodes
protected HttpOpParam.Op	op

Method Summary

Modifier and Type	Method	Description
void	close()
protected java.net.HttpURLConnection	connect(java.net.URL url)	Two-step requests redirected to a DN Create/Append: Step 1) Submit a Http request with neither auto-redirect nor data.
protected org.apache.hadoop.fs.FileEncryptionInfo	getFileEncryptionInfo()
protected java.io.InputStream	getInputStream()
protected java.net.URL	getUrl()
protected void	setInputStream(java.io.InputStream inStream)
protected void	updateURLParameters(Param<?,?>... p)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

op

protected final HttpOpParam.Op op

excludeDatanodes

protected ExcludeDatanodesParam excludeDatanodes

Method Detail

close

public void close()
           throws java.io.IOException

Throws:

java.io.IOException

getUrl

protected java.net.URL getUrl()
                       throws java.io.IOException

Throws:

java.io.IOException

connect

protected java.net.HttpURLConnection connect(java.net.URL url)
                                      throws java.io.IOException

Two-step requests redirected to a DN Create/Append: Step 1) Submit a Http request with neither auto-redirect nor data. Step 2) Submit another Http request with the URL from the Location header with data. The reason of having two-step create/append is for preventing clients to send out the data before the redirect. This issue is addressed by the "Expect: 100-continue" header in HTTP/1.1; see RFC 2616, Section 8.2.3. Unfortunately, there are software library bugs (e.g. Jetty 6 http server and Java 6 http client), which do not correctly implement "Expect: 100-continue". The two-step create/append is a temporary workaround for the software library bugs. Open/Checksum Also implements two-step connects for other operations redirected to a DN such as open and checksum

Throws:

java.io.IOException

getInputStream

@VisibleForTesting
protected java.io.InputStream getInputStream()

setInputStream

@VisibleForTesting
protected void setInputStream(java.io.InputStream inStream)

getFileEncryptionInfo

protected org.apache.hadoop.fs.FileEncryptionInfo getFileEncryptionInfo()

updateURLParameters

protected void updateURLParameters(Param<?,?>... p)