Package org.apache.hadoop.hdfs

Class StripedDataStreamer

java.lang.Object

java.lang.Thread

org.apache.hadoop.util.Daemon

org.apache.hadoop.hdfs.StripedDataStreamer

All Implemented Interfaces:

java.lang.Runnable

@Private
public class StripedDataStreamer
extends org.apache.hadoop.util.Daemon

This class extends DataStreamer to support writing striped blocks to datanodes. A DFSStripedOutputStream has multiple StripedDataStreamers. Whenever the streamers need to talk the namenode, only the fastest streamer sends an rpc call to the namenode and then populates the result for the other streamers.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.util.Daemon

org.apache.hadoop.util.Daemon.DaemonFactory

Nested classes/interfaces inherited from class java.lang.Thread

java.lang.Thread.State, java.lang.Thread.UncaughtExceptionHandler

Field Summary

Fields

Modifier and Type	Field	Description
protected org.apache.hadoop.security.token.Token<BlockTokenIdentifier>	accessToken
protected org.apache.hadoop.hdfs.DataStreamer.BlockToWrite	block
protected long	bytesSent
protected java.util.LinkedList<DFSPacket>	dataQueue
protected DFSClient	dfsClient
protected org.apache.hadoop.thirdparty.com.google.common.cache.LoadingCache<DatanodeInfo,DatanodeInfo>	excludedNodes
protected java.lang.String	src
protected HdfsFileStatus	stat

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Method Summary

Modifier and Type	Method	Description
protected void	endBlock()
void	run()
protected void	setPipeline(DatanodeInfo[] newNodes, org.apache.hadoop.fs.StorageType[] newStorageTypes, java.lang.String[] newStorageIDs)
protected void	setPipeline(LocatedBlock lb)
protected void	setupPipelineForCreate()	Open a DataStreamer to a DataNode so that it can be written to.
protected boolean	setupPipelineInternal(DatanodeInfo[] nodes, org.apache.hadoop.fs.StorageType[] nodeStorageTypes, java.lang.String[] nodeStorageIDs)
java.lang.String	toString()
void	updatePipeline(long newGS)	update pipeline at the namenode

Methods inherited from class org.apache.hadoop.util.Daemon

getRunnable

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, onSpinWait, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, suspend, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

block

protected final org.apache.hadoop.hdfs.DataStreamer.BlockToWrite block

accessToken

protected org.apache.hadoop.security.token.Token<BlockTokenIdentifier> accessToken

bytesSent

protected long bytesSent

dfsClient

protected final DFSClient dfsClient

src

protected final java.lang.String src

stat

protected final HdfsFileStatus stat

dataQueue

protected final java.util.LinkedList<DFSPacket> dataQueue

excludedNodes

protected final org.apache.hadoop.thirdparty.com.google.common.cache.LoadingCache<DatanodeInfo,DatanodeInfo> excludedNodes

Method Detail

endBlock

protected void endBlock()

setupPipelineForCreate

protected void setupPipelineForCreate()
                               throws java.io.IOException

Open a DataStreamer to a DataNode so that it can be written to. This happens when a file is created and each time a new block is allocated. Must get block ID and the IDs of the destinations from the namenode. Returns the list of target datanodes.

Throws:

java.io.IOException

setupPipelineInternal

protected boolean setupPipelineInternal(DatanodeInfo[] nodes,
                                        org.apache.hadoop.fs.StorageType[] nodeStorageTypes,
                                        java.lang.String[] nodeStorageIDs)
                                 throws java.io.IOException

Throws:

java.io.IOException

toString

public java.lang.String toString()

setPipeline

protected void setPipeline(LocatedBlock lb)

setPipeline

protected void setPipeline(DatanodeInfo[] newNodes,
                           org.apache.hadoop.fs.StorageType[] newStorageTypes,
                           java.lang.String[] newStorageIDs)

run

public void run()

Specified by:

run in interface java.lang.Runnable

Overrides:

run in class java.lang.Thread

updatePipeline

@VisibleForTesting
public void updatePipeline(long newGS)
                    throws java.io.IOException

update pipeline at the namenode

Throws:

java.io.IOException