Package org.apache.hadoop.hdfs.server.protocol

Interface DatanodeProtocol

All Known Subinterfaces:

NamenodeProtocols

All Known Implementing Classes:

DatanodeProtocolClientSideTranslatorPB, NameNodeRpcServer

@Private
public interface DatanodeProtocol

Protocol that a DFS datanode uses to communicate with the NameNode. It's used to upload current load information and block reports. The only way a NameNode can communicate with a DataNode is by returning values from these functions.

Field Summary

Fields

Modifier and Type	Field	Description
static int	DISK_ERROR
static int	DNA_ACCESSKEYUPDATE
static int	DNA_BALANCERBANDWIDTHUPDATE
static int	DNA_BLOCK_STORAGE_MOVEMENT
static int	DNA_CACHE
static int	DNA_DROP_SPS_WORK_COMMAND
static int	DNA_ERASURE_CODING_RECONSTRUCTION
static int	DNA_FINALIZE
static int	DNA_INVALIDATE
static int	DNA_RECOVERBLOCK
static int	DNA_REGISTER
static int	DNA_SHUTDOWN
static int	DNA_TRANSFER
static int	DNA_UNCACHE
static int	DNA_UNKNOWN	Determines actions that data node should perform when receiving a datanode command.
static int	FATAL_DISK_ERROR
static int	INVALID_BLOCK
static int	NOTIFY
static long	versionID	This class is used by both the Namenode (client) and BackupNode (server) to insulate from the protocol serialization.

Method Summary

Modifier and Type	Method	Description
void	blockReceivedAndDeleted(DatanodeRegistration registration, java.lang.String poolId, StorageReceivedDeletedBlocks[] rcvdAndDeletedBlocks)	blockReceivedAndDeleted() allows the DataNode to tell the NameNode about recently-received and -deleted block data.
DatanodeCommand	blockReport(DatanodeRegistration registration, java.lang.String poolId, StorageBlockReport[] reports, BlockReportContext context)	blockReport() tells the NameNode about all the locally-stored blocks.
DatanodeCommand	cacheReport(DatanodeRegistration registration, java.lang.String poolId, java.util.List<java.lang.Long> blockIds)	Communicates the complete list of locally cached blocks to the NameNode.
void	commitBlockSynchronization(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, long newgenerationstamp, long newlength, boolean closeFile, boolean deleteblock, org.apache.hadoop.hdfs.protocol.DatanodeID[] newtargets, java.lang.String[] newtargetstorages)	Commit block synchronization in lease recovery
void	errorReport(DatanodeRegistration registration, int errorCode, java.lang.String msg)	errorReport() tells the NameNode about something that has gone awry.
DatanodeRegistration	registerDatanode(DatanodeRegistration registration)	Register Datanode.
void	reportBadBlocks(org.apache.hadoop.hdfs.protocol.LocatedBlock[] blocks)	same as ClientProtocol.reportBadBlocks(LocatedBlock[]) }
HeartbeatResponse	sendHeartbeat(DatanodeRegistration registration, org.apache.hadoop.hdfs.server.protocol.StorageReport[] reports, long dnCacheCapacity, long dnCacheUsed, int xmitsInProgress, int xceiverCount, int failedVolumes, VolumeFailureSummary volumeFailureSummary, boolean requestFullBlockReportLease, org.apache.hadoop.hdfs.server.protocol.SlowPeerReports slowPeers, org.apache.hadoop.hdfs.server.protocol.SlowDiskReports slowDisks)	sendHeartbeat() tells the NameNode that the DataNode is still alive and well.
NamespaceInfo	versionRequest()

Field Detail

versionID

static final long versionID

This class is used by both the Namenode (client) and BackupNode (server) to insulate from the protocol serialization. If you are adding/changing DN's interface then you need to change both this class and ALSO related protocol buffer wire protocol definition in DatanodeProtocol.proto. For more details on protocol buffer wire protocol, please see .../org/apache/hadoop/hdfs/protocolPB/overview.html

See Also:

Constant Field Values

NOTIFY

static final int NOTIFY

See Also:

Constant Field Values

DISK_ERROR

static final int DISK_ERROR

See Also:

Constant Field Values

INVALID_BLOCK

static final int INVALID_BLOCK

See Also:

Constant Field Values

FATAL_DISK_ERROR

static final int FATAL_DISK_ERROR

See Also:

Constant Field Values

DNA_UNKNOWN

static final int DNA_UNKNOWN

Determines actions that data node should perform when receiving a datanode command.

See Also:

Constant Field Values

DNA_TRANSFER

static final int DNA_TRANSFER

See Also:

Constant Field Values

DNA_INVALIDATE

static final int DNA_INVALIDATE

See Also:

Constant Field Values

DNA_SHUTDOWN

static final int DNA_SHUTDOWN

See Also:

Constant Field Values

DNA_REGISTER

static final int DNA_REGISTER

See Also:

Constant Field Values

DNA_FINALIZE

static final int DNA_FINALIZE

See Also:

Constant Field Values

DNA_RECOVERBLOCK

static final int DNA_RECOVERBLOCK

See Also:

Constant Field Values

DNA_ACCESSKEYUPDATE

static final int DNA_ACCESSKEYUPDATE

See Also:

Constant Field Values

DNA_BALANCERBANDWIDTHUPDATE

static final int DNA_BALANCERBANDWIDTHUPDATE

See Also:

Constant Field Values

DNA_CACHE

static final int DNA_CACHE

See Also:

Constant Field Values

DNA_UNCACHE

static final int DNA_UNCACHE

See Also:

Constant Field Values

DNA_ERASURE_CODING_RECONSTRUCTION

static final int DNA_ERASURE_CODING_RECONSTRUCTION

See Also:

Constant Field Values

DNA_BLOCK_STORAGE_MOVEMENT

static final int DNA_BLOCK_STORAGE_MOVEMENT

See Also:

Constant Field Values

DNA_DROP_SPS_WORK_COMMAND

static final int DNA_DROP_SPS_WORK_COMMAND

See Also:

Constant Field Values

Method Detail

registerDatanode

DatanodeRegistration registerDatanode(DatanodeRegistration registration)
                               throws java.io.IOException

Register Datanode.

Parameters:

registration - datanode registration information

Returns:

the given DatanodeRegistration with updated registration information

Throws:

java.io.IOException

See Also:

FSNamesystem.registerDatanode(DatanodeRegistration)

sendHeartbeat

HeartbeatResponse sendHeartbeat(DatanodeRegistration registration,
                                org.apache.hadoop.hdfs.server.protocol.StorageReport[] reports,
                                long dnCacheCapacity,
                                long dnCacheUsed,
                                int xmitsInProgress,
                                int xceiverCount,
                                int failedVolumes,
                                VolumeFailureSummary volumeFailureSummary,
                                boolean requestFullBlockReportLease,
                                @Nonnull
                                org.apache.hadoop.hdfs.server.protocol.SlowPeerReports slowPeers,
                                @Nonnull
                                org.apache.hadoop.hdfs.server.protocol.SlowDiskReports slowDisks)
                         throws java.io.IOException

sendHeartbeat() tells the NameNode that the DataNode is still alive and well. Includes some status info, too. It also gives the NameNode a chance to return an array of "DatanodeCommand" objects in HeartbeatResponse. A DatanodeCommand tells the DataNode to invalidate local block(s), or to copy them to other DataNodes, etc.

Parameters:

registration - datanode registration information.

reports - utilization report per storage.

dnCacheCapacity - the total cache capacity of the datanode (in bytes).

dnCacheUsed - the amount of cache used by the datanode (in bytes).

xmitsInProgress - number of transfers from this datanode to others.

xceiverCount - number of active transceiver threads.

failedVolumes - number of failed volumes.

volumeFailureSummary - info about volume failures.

requestFullBlockReportLease - whether to request a full block report lease.

slowPeers - Details of peer DataNodes that were detected as being slow to respond to packet writes. Empty report if no slow peers were detected by the DataNode.

slowDisks - Details of disks on DataNodes that were detected as being slow. Empty report if no slow disks were detected.

Throws:

java.io.IOException - on error.

blockReport

DatanodeCommand blockReport(DatanodeRegistration registration,
                            java.lang.String poolId,
                            StorageBlockReport[] reports,
                            BlockReportContext context)
                     throws java.io.IOException

blockReport() tells the NameNode about all the locally-stored blocks. The NameNode returns an array of Blocks that have become obsolete and should be deleted. This function is meant to upload *all* the locally-stored blocks. It's invoked upon startup and then infrequently afterwards.

Parameters:

registration - datanode registration

poolId - the block pool ID for the blocks

reports - report of blocks per storage Each finalized block is represented as 3 longs. Each under- construction replica is represented as 4 longs. This is done instead of Block[] to reduce memory used by block reports.

reports - report of blocks per storage

context - Context information for this block report.

Returns:

- the next command for DN to process.

Throws:

java.io.IOException

cacheReport

DatanodeCommand cacheReport(DatanodeRegistration registration,
                            java.lang.String poolId,
                            java.util.List<java.lang.Long> blockIds)
                     throws java.io.IOException

Communicates the complete list of locally cached blocks to the NameNode. This method is similar to blockReport(DatanodeRegistration, String, StorageBlockReport[], BlockReportContext), which is used to communicated blocks stored on disk.

Parameters:

registration - The datanode registration.

poolId - The block pool ID for the blocks.

blockIds - A list of block IDs.

Returns:

The DatanodeCommand.

Throws:

java.io.IOException

blockReceivedAndDeleted

void blockReceivedAndDeleted(DatanodeRegistration registration,
                             java.lang.String poolId,
                             StorageReceivedDeletedBlocks[] rcvdAndDeletedBlocks)
                      throws java.io.IOException

blockReceivedAndDeleted() allows the DataNode to tell the NameNode about recently-received and -deleted block data. For the case of received blocks, a hint for preferred replica to be deleted when there is any excessive blocks is provided. For example, whenever client code writes a new Block here, or another DataNode copies a Block to this DataNode, it will call blockReceived().

Throws:

java.io.IOException

errorReport

void errorReport(DatanodeRegistration registration,
                 int errorCode,
                 java.lang.String msg)
          throws java.io.IOException

errorReport() tells the NameNode about something that has gone awry. Useful for debugging.

Throws:

java.io.IOException

versionRequest

NamespaceInfo versionRequest()
                      throws java.io.IOException

Throws:

java.io.IOException

reportBadBlocks

void reportBadBlocks(org.apache.hadoop.hdfs.protocol.LocatedBlock[] blocks)
              throws java.io.IOException

same as ClientProtocol.reportBadBlocks(LocatedBlock[]) }

Throws:

java.io.IOException

commitBlockSynchronization

void commitBlockSynchronization(org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                                long newgenerationstamp,
                                long newlength,
                                boolean closeFile,
                                boolean deleteblock,
                                org.apache.hadoop.hdfs.protocol.DatanodeID[] newtargets,
                                java.lang.String[] newtargetstorages)
                         throws java.io.IOException

Commit block synchronization in lease recovery

Throws:

java.io.IOException