Package org.apache.hadoop.hdfs.protocol.datatransfer

Class Sender

java.lang.Object

org.apache.hadoop.hdfs.protocol.datatransfer.Sender

All Implemented Interfaces:

DataTransferProtocol

@Private
@Evolving
public class Sender
extends java.lang.Object
implements DataTransferProtocol

Sender

Field Summary

Fields inherited from interface org.apache.hadoop.hdfs.protocol.datatransfer.DataTransferProtocol

DATA_TRANSFER_VERSION, LOG

Constructor Summary

Constructors

Constructor	Description
Sender(java.io.DataOutputStream out)	Create a sender for DataTransferProtocol with a output stream.

Method Summary

Modifier and Type	Method	Description
void	blockChecksum(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, BlockChecksumOptions blockChecksumOptions)	Get block checksum (MD5 of CRC32).
void	blockGroupChecksum(StripedBlockInfo stripedBlockInfo, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, long requestedNumBytes, BlockChecksumOptions blockChecksumOptions)	Get striped block group checksum (MD5 of CRC32).
void	copyBlock(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken)	Copy a block.
void	readBlock(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, java.lang.String clientName, long blockOffset, long length, boolean sendChecksum, CachingStrategy cachingStrategy)	Read a block.
void	releaseShortCircuitFds(ShortCircuitShm.SlotId slotId)	Release a pair of short-circuit FDs requested earlier.
void	replaceBlock(ExtendedBlock blk, org.apache.hadoop.fs.StorageType storageType, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, java.lang.String delHint, DatanodeInfo source, java.lang.String storageId)	Receive a block from a source datanode and then notifies the namenode to remove the copy from the original datanode.
void	requestShortCircuitFds(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, ShortCircuitShm.SlotId slotId, int maxVersion, boolean supportsReceiptVerification)	Request short circuit access file descriptors from a DataNode.
void	requestShortCircuitShm(java.lang.String clientName)	Request a short circuit shared memory area from a DataNode.
void	transferBlock(ExtendedBlock blk, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, java.lang.String clientName, DatanodeInfo[] targets, org.apache.hadoop.fs.StorageType[] targetStorageTypes, java.lang.String[] targetStorageIds)	Transfer a block to another datanode.
void	writeBlock(ExtendedBlock blk, org.apache.hadoop.fs.StorageType storageType, org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken, java.lang.String clientName, DatanodeInfo[] targets, org.apache.hadoop.fs.StorageType[] targetStorageTypes, DatanodeInfo source, BlockConstructionStage stage, int pipelineSize, long minBytesRcvd, long maxBytesRcvd, long latestGenerationStamp, org.apache.hadoop.util.DataChecksum requestedChecksum, CachingStrategy cachingStrategy, boolean allowLazyPersist, boolean pinning, boolean[] targetPinnings, java.lang.String storageId, java.lang.String[] targetStorageIds)	Write a block to a datanode pipeline.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Sender

public Sender(java.io.DataOutputStream out)

Create a sender for DataTransferProtocol with a output stream.

Method Detail

readBlock

public void readBlock(ExtendedBlock blk,
                      org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken,
                      java.lang.String clientName,
                      long blockOffset,
                      long length,
                      boolean sendChecksum,
                      CachingStrategy cachingStrategy)
               throws java.io.IOException

Description copied from interface: DataTransferProtocol

Read a block.

Specified by:

readBlock in interface DataTransferProtocol

Parameters:

blk - the block being read.

blockToken - security token for accessing the block.

clientName - client's name.

blockOffset - offset of the block.

length - maximum number of bytes for this read.

sendChecksum - if false, the DN should skip reading and sending checksums

cachingStrategy - The caching strategy to use.

Throws:

java.io.IOException

writeBlock

public void writeBlock(ExtendedBlock blk,
                       org.apache.hadoop.fs.StorageType storageType,
                       org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken,
                       java.lang.String clientName,
                       DatanodeInfo[] targets,
                       org.apache.hadoop.fs.StorageType[] targetStorageTypes,
                       DatanodeInfo source,
                       BlockConstructionStage stage,
                       int pipelineSize,
                       long minBytesRcvd,
                       long maxBytesRcvd,
                       long latestGenerationStamp,
                       org.apache.hadoop.util.DataChecksum requestedChecksum,
                       CachingStrategy cachingStrategy,
                       boolean allowLazyPersist,
                       boolean pinning,
                       boolean[] targetPinnings,
                       java.lang.String storageId,
                       java.lang.String[] targetStorageIds)
                throws java.io.IOException

Description copied from interface: DataTransferProtocol

Write a block to a datanode pipeline. The receiver datanode of this call is the next datanode in the pipeline. The other downstream datanodes are specified by the targets parameter. Note that the receiver DatanodeInfo is not required in the parameter list since the receiver datanode knows its info. However, the StorageType for storing the replica in the receiver datanode is a parameter since the receiver datanode may support multiple storage types.

Specified by:

writeBlock in interface DataTransferProtocol

Parameters:

blk - the block being written.

storageType - for storing the replica in the receiver datanode.

blockToken - security token for accessing the block.

clientName - client's name.

targets - other downstream datanodes in the pipeline.

targetStorageTypes - target StorageTypes corresponding to the target datanodes.

source - source datanode.

stage - pipeline stage.

pipelineSize - the size of the pipeline.

minBytesRcvd - minimum number of bytes received.

maxBytesRcvd - maximum number of bytes received.

latestGenerationStamp - the latest generation stamp of the block.

requestedChecksum - the requested checksum mechanism

cachingStrategy - the caching strategy

allowLazyPersist - hint to the DataNode that the block can be allocated on transient storage i.e. memory and written to disk lazily

pinning - whether to pin the block, so Balancer won't move it.

targetPinnings - whether to pin the block on target datanode

storageId - optional StorageIDs designating where to write the block. An empty String or null indicates that this has not been provided.

targetStorageIds - target StorageIDs corresponding to the target datanodes.

Throws:

java.io.IOException

transferBlock

public void transferBlock(ExtendedBlock blk,
                          org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken,
                          java.lang.String clientName,
                          DatanodeInfo[] targets,
                          org.apache.hadoop.fs.StorageType[] targetStorageTypes,
                          java.lang.String[] targetStorageIds)
                   throws java.io.IOException

Description copied from interface: DataTransferProtocol

Transfer a block to another datanode. The block stage must be either BlockConstructionStage.TRANSFER_RBW or BlockConstructionStage.TRANSFER_FINALIZED.

Specified by:

transferBlock in interface DataTransferProtocol

Parameters:

blk - the block being transferred.

blockToken - security token for accessing the block.

clientName - client's name.

targets - target datanodes.

targetStorageIds - StorageID designating where to write the block.

Throws:

java.io.IOException

requestShortCircuitFds

public void requestShortCircuitFds(ExtendedBlock blk,
                                   org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken,
                                   ShortCircuitShm.SlotId slotId,
                                   int maxVersion,
                                   boolean supportsReceiptVerification)
                            throws java.io.IOException

Description copied from interface: DataTransferProtocol

Request short circuit access file descriptors from a DataNode.

Specified by:

requestShortCircuitFds in interface DataTransferProtocol

Parameters:

blk - The block to get file descriptors for.

blockToken - Security token for accessing the block.

slotId - The shared memory slot id to use, or null to use no slot id.

maxVersion - Maximum version of the block data the client can understand.

supportsReceiptVerification - True if the client supports receipt verification.

Throws:

java.io.IOException

releaseShortCircuitFds

public void releaseShortCircuitFds(ShortCircuitShm.SlotId slotId)
                            throws java.io.IOException

Description copied from interface: DataTransferProtocol

Release a pair of short-circuit FDs requested earlier.

Specified by:

releaseShortCircuitFds in interface DataTransferProtocol

Parameters:

slotId - SlotID used by the earlier file descriptors.

Throws:

java.io.IOException

requestShortCircuitShm

public void requestShortCircuitShm(java.lang.String clientName)
                            throws java.io.IOException

Description copied from interface: DataTransferProtocol

Request a short circuit shared memory area from a DataNode.

Specified by:

requestShortCircuitShm in interface DataTransferProtocol

Parameters:

clientName - The name of the client.

Throws:

java.io.IOException

replaceBlock

public void replaceBlock(ExtendedBlock blk,
                         org.apache.hadoop.fs.StorageType storageType,
                         org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken,
                         java.lang.String delHint,
                         DatanodeInfo source,
                         java.lang.String storageId)
                  throws java.io.IOException

Description copied from interface: DataTransferProtocol

Receive a block from a source datanode and then notifies the namenode to remove the copy from the original datanode. Note that the source datanode and the original datanode can be different. It is used for balancing purpose.

Specified by:

replaceBlock in interface DataTransferProtocol

Parameters:

blk - the block being replaced.

storageType - the StorageType for storing the block.

blockToken - security token for accessing the block.

delHint - the hint for deleting the block in the original datanode.

source - the source datanode for receiving the block.

storageId - an optional storage ID to designate where the block is replaced to.

Throws:

java.io.IOException

copyBlock

public void copyBlock(ExtendedBlock blk,
                      org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken)
               throws java.io.IOException

Description copied from interface: DataTransferProtocol

Copy a block. It is used for balancing purpose.

Specified by:

copyBlock in interface DataTransferProtocol

Parameters:

blk - the block being copied.

blockToken - security token for accessing the block.

Throws:

java.io.IOException

blockChecksum

public void blockChecksum(ExtendedBlock blk,
                          org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken,
                          BlockChecksumOptions blockChecksumOptions)
                   throws java.io.IOException

Description copied from interface: DataTransferProtocol

Get block checksum (MD5 of CRC32).

Specified by:

blockChecksum in interface DataTransferProtocol

Parameters:

blk - a block.

blockToken - security token for accessing the block.

blockChecksumOptions - determines how the block-level checksum is computed from underlying block metadata.

Throws:

java.io.IOException

blockGroupChecksum

public void blockGroupChecksum(StripedBlockInfo stripedBlockInfo,
                               org.apache.hadoop.security.token.Token<BlockTokenIdentifier> blockToken,
                               long requestedNumBytes,
                               BlockChecksumOptions blockChecksumOptions)
                        throws java.io.IOException

Description copied from interface: DataTransferProtocol

Get striped block group checksum (MD5 of CRC32).

Specified by:

blockGroupChecksum in interface DataTransferProtocol

Parameters:

stripedBlockInfo - a striped block info.

blockToken - security token for accessing the block.

requestedNumBytes - requested number of bytes in the block group to compute the checksum.

blockChecksumOptions - determines how the block-level checksum is computed from underlying block metadata.

Throws:

java.io.IOException