BlockManager (Apache Hadoop HDFS 3.4.1.0-eep-940 API)

java.lang.Object
- org.apache.hadoop.hdfs.server.blockmanagement.BlockManager

All Implemented Interfaces:

BlockStatsMXBean
```
@Private
public class BlockManager
extends java.lang.Object
implements BlockStatsMXBean
```
Keeps information related to the blocks stored in the Hadoop cluster. For block state management, it tries to maintain the safety property of "# of live replicas == # of expected redundancy" under any events such as decommission, namenode failover, datanode failure. The motivation of maintenance mode is to allow admins quickly repair nodes without paying the cost of decommission. Thus with maintenance mode, # of live replicas doesn't have to be equal to # of expected redundancy. If any of the replica is in maintenance mode, the safety property is extended as follows. These property still apply for the case of zero maintenance replicas, thus we can use these safe property for all scenarios. a. # of live replicas >= # of min replication for maintenance. b. # of live replicas <= # of expected redundancy. c. # of live replicas and maintenance replicas >= # of expected redundancy. For regular replication, # of min live replicas for maintenance is determined by DFSConfigKeys.DFS_NAMENODE_MAINTENANCE_REPLICATION_MIN_KEY. This number has to <= DFSConfigKeys.DFS_NAMENODE_REPLICATION_MIN_KEY. For erasure encoding, # of min live replicas for maintenance is BlockInfoStriped.getRealDataBlockNum(). Another safety property is to satisfy the block placement policy. While the policy is configurable, the replicas the policy is applied to are the live replicas + maintenance replicas.

Field Summary

Fields
Modifier and Type	Field	Description
`static org.slf4j.Logger`	`blockLog`
`int`	`defaultReplication`	Default number of replicas
`static org.slf4j.Logger`	`LOG`
`short`	`maxReplication`	The maximum number of replicas allowed for a block
`short`	`minReplication`	Minimum copies needed or else write is disallowed
`org.apache.hadoop.hdfs.server.blockmanagement.LowRedundancyBlocks`	`neededReconstruction`	Store set of Blocks that need to be replicated 1 or more times.

Constructor Summary

Constructors
Constructor Description

BlockManager(Namesystem namesystem, boolean haEnabled, org.apache.hadoop.conf.Configuration conf)

Constructors
Constructor	Description
`BlockManager(Namesystem namesystem, boolean haEnabled, org.apache.hadoop.conf.Configuration conf)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`activate(org.apache.hadoop.conf.Configuration conf, long blockTotal)`
`void`	`addBlock(DatanodeStorageInfo storageInfo, org.apache.hadoop.hdfs.protocol.Block block, java.lang.String delHint)`	The given node is reporting that it received a certain block.
`BlockInfo`	`addBlockCollection(BlockInfo block, BlockCollection bc)`
`BlockInfo`	`addBlockCollectionWithCheck(BlockInfo block, BlockCollection bc)`	Do some check when adding a block to blocksmap.
`boolean`	`addBlockRecoveryAttempt(BlockInfo b)`	Checks whether a recovery attempt has been made for the given block.
`void`	`addBLocksToMarkedDeleteQueue(java.util.List<BlockInfo> blockInfos)`
`void`	`addExpectedReplicasToPending(BlockInfo blk)`	If IBR is not sent from expected locations yet, add the datanodes to pendingReconstruction in order to keep RedundancyMonitor from scheduling the block.
`short`	`adjustReplication(short replication)`	Clamp the specified replication between the minimum and the maximum replication levels.
`boolean`	`checkBlockReportLease(BlockReportContext context, org.apache.hadoop.hdfs.protocol.DatanodeID nodeID)`	Check block report lease.
`void`	`checkRedundancy(BlockCollection bc)`	Check sufficient redundancy of the blocks in the collection.
`void`	`checkSafeMode()`
`DatanodeStorageInfo[]`	`chooseTarget4AdditionalDatanode(java.lang.String src, int numAdditionalNodes, org.apache.hadoop.net.Node clientnode, java.util.List<DatanodeStorageInfo> chosen, java.util.Set<org.apache.hadoop.net.Node> excludes, long blocksize, byte storagePolicyID, org.apache.hadoop.hdfs.protocol.BlockType blockType)`	Choose target for getting additional datanodes for an existing pipeline.
`DatanodeStorageInfo[]`	`chooseTarget4NewBlock(java.lang.String src, int numOfReplicas, org.apache.hadoop.net.Node client, java.util.Set<org.apache.hadoop.net.Node> excludedNodes, long blocksize, java.util.List<java.lang.String> favoredNodes, byte storagePolicyID, org.apache.hadoop.hdfs.protocol.BlockType blockType, org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy ecPolicy, java.util.EnumSet<org.apache.hadoop.hdfs.AddBlockFlag> flags)`	Choose target datanodes for creating a new block.
`DatanodeStorageInfo[]`	`chooseTarget4WebHDFS(java.lang.String src, DatanodeDescriptor clientnode, java.util.Set<org.apache.hadoop.net.Node> excludes, long blocksize)`	Choose target for WebHDFS redirection.
`void`	`clear()`
`void`	`clearQueues()`	Clear all queues that hold decisions previously made by this NameNode.
`void`	`close()`
`boolean`	`commitOrCompleteLastBlock(BlockCollection bc, org.apache.hadoop.hdfs.protocol.Block commitBlock, INodesInPath iip)`	Commit the last block of the file and mark it as complete if it has meets the minimum redundancy requirement
`boolean`	`containsInvalidateBlock(org.apache.hadoop.hdfs.protocol.DatanodeInfo dn, org.apache.hadoop.hdfs.protocol.Block block)`
`org.apache.hadoop.hdfs.protocol.LocatedBlock`	`convertLastBlockToUnderConstruction(BlockCollection bc, long bytesToRemove)`	Convert the last block of the file to an under construction block.
`NumberReplicas`	`countNodes(BlockInfo b)`	Return the number of nodes hosting a given block, grouped by the state of those replicas.
`org.apache.hadoop.hdfs.protocol.LocatedBlocks`	`createLocatedBlocks(BlockInfo[] blocks, long fileSizeExcludeBlocksUnderConstruction, boolean isFileUnderConstruction, long offset, long length, boolean needBlockToken, boolean inSnapshot, org.apache.hadoop.fs.FileEncryptionInfo feInfo, org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy ecPolicy)`	Create a LocatedBlocks.
`boolean`	`createSPSManager(org.apache.hadoop.conf.Configuration conf, java.lang.String spsMode)`	Create SPS manager instance.
`void`	`disableSPS()`	Nullify SPS manager as this feature is disabled fully.
`void`	`enqueueBlockOp(java.lang.Runnable action)`
`void`	`findAndMarkBlockAsCorrupt(org.apache.hadoop.hdfs.protocol.ExtendedBlock blk, org.apache.hadoop.hdfs.protocol.DatanodeInfo dn, java.lang.String storageID, java.lang.String reason)`	Mark the block belonging to datanode as corrupt
`void`	`flushBlockOps()`
`void`	`forceCompleteBlock(BlockInfo block)`	Force the given block in the given file to be marked as complete, regardless of whether enough replicas are present.
`org.apache.hadoop.hdfs.security.token.block.DataEncryptionKey`	`generateDataEncryptionKey()`
`int`	`getActiveBlockCount()`
`BlockIdManager`	`getBlockIdManager()`
`ExportedBlockKeys`	`getBlockKeys()`
`int`	`getBlockOpQueueLength()`
`BlockPlacementPolicy`	`getBlockPlacementPolicy()`
`java.lang.String`	`getBlockPoolId()`
`org.apache.hadoop.hdfs.server.blockmanagement.BlockReportLeaseManager`	`getBlockReportLeaseManager()`
`int`	`getBlocksReplWorkMultiplier()`	Returns the current setting for blocksReplWorkMultiplier, set by `DFSConfigKeys. DFS_NAMENODE_REPLICATION_WORK_MULTIPLIER_PER_ITERATION`.
`BlocksWithLocations`	`getBlocksWithLocations(org.apache.hadoop.hdfs.protocol.DatanodeID datanode, long size, long minBlockSize, long timeInterval, org.apache.hadoop.fs.StorageType storageType)`	Get all blocks with location information from a datanode.
`BlockTokenSecretManager`	`getBlockTokenSecretManager()`	get the BlockTokenSecretManager
`long`	`getBytesInFuture()`
`long`	`getBytesInFutureECBlockGroups()`
`long`	`getBytesInFutureReplicatedBlocks()`
`int`	`getCapacity()`
`long`	`getCorruptBlocks()`	Used by metrics.
`long`	`getCorruptECBlockGroups()`	Used by metrics.
`java.lang.String`	`getCorruptReason(org.apache.hadoop.hdfs.protocol.Block block, DatanodeDescriptor node)`	Get reason for certain corrupted replicas for a given block and a given dn.
`java.util.Iterator<BlockInfo>`	`getCorruptReplicaBlockIterator()`	Return an iterator over the set of blocks for which there are no replicas.
`long`	`getCorruptReplicaBlocksCount()`	Used by metrics
`java.util.Collection<DatanodeDescriptor>`	`getCorruptReplicas(org.apache.hadoop.hdfs.protocol.Block block)`	Get the replicas which are corrupt for a given block.
`DatanodeManager`	`getDatanodeManager()`
`int`	`getDefaultStorageNum(BlockInfo block)`
`long`	`getExcessBlocksCount()`	Used by metrics
`boolean`	`getExcludeSlowNodesEnabled(org.apache.hadoop.hdfs.protocol.BlockType blockType)`
`short`	`getExpectedLiveRedundancyNum(BlockInfo block, NumberReplicas numberReplicas)`
`short`	`getExpectedRedundancyNum(BlockInfo block)`
`long`	`getHighestPriorityECBlockCount()`
`long`	`getHighestPriorityReplicatedBlockCount()`
`long`	`getLastRedundancyMonitorTS()`	Used as ad hoc to check the time stamp of the last full cycle of `redundancyThread`.
`long`	`getLowRedundancyBlocks()`	Used by metrics.
`long`	`getLowRedundancyBlocksCount()`	Used by metrics
`long`	`getLowRedundancyECBlockGroups()`	Used by metrics.
`java.util.concurrent.ConcurrentLinkedQueue<java.util.List<BlockInfo>>`	`getMarkedDeleteQueue()`
`int`	`getMaxReplicationStreams()`	Returns the current setting for maxReplicationStreams, which is set by `DFSConfigKeys.DFS_NAMENODE_REPLICATION_MAX_STREAMS_KEY`.
`int`	`getMinBlocksForWrite(org.apache.hadoop.hdfs.protocol.BlockType blockType)`
`short`	`getMinReplication()`
`short`	`getMinReplicationToBeInMaintenance()`
`short`	`getMinStorageNum(BlockInfo block)`
`long`	`getMissingBlocks()`	Used by metrics.
`long`	`getMissingBlocksCount()`
`long`	`getMissingECBlockGroups()`	Used by metrics.
`long`	`getMissingReplicationOneBlocks()`	Used by metrics.
`long`	`getMissingReplOneBlocksCount()`
`long`	`getNumTimedOutPendingReconstructions()`	Used by metrics.
`int`	`getPendingDataNodeMessageCount()`	Used by metrics
`long`	`getPendingDeletionBlocksCount()`	Used by metrics
`long`	`getPendingDeletionECBlocks()`	Used by metrics.
`long`	`getPendingDeletionReplicatedBlocks()`	Used by metrics.
`long`	`getPendingReconstructionBlocksCount()`	Used by metrics
`int`	`getPendingSPSPaths()`	Used by metrics.
`long`	`getPostponedMisreplicatedBlocksCount()`	Used by metrics
`long`	`getProvidedCapacity()`
`ProvidedStorageMap`	`getProvidedStorageMap()`
`int`	`getReconstructionPendingTimeout()`	Returns the current setting for pendingReconstruction timeout, set by `DFSConfigKeys.DFS_NAMENODE_RECONSTRUCTION_PENDING_TIMEOUT_SEC_KEY`.
`float`	`getReconstructionQueuesInitProgress()`	Get the progress of the reconstruction queues initialisation
`int`	`getReplicationStreamsHardLimit()`	Returns the current setting for maxReplicationStreamsHardLimit, set by `DFSConfigKeys.DFS_NAMENODE_REPLICATION_STREAMS_HARD_LIMIT_KEY`.
`java.lang.String`	`getSafeModeTip()`
`long`	`getScheduledReplicationBlocksCount()`	Used by metrics
`StoragePolicySatisfyManager`	`getSPSManager()`
`long`	`getStartupDelayBlockDeletionInMs()`	Used by metrics
`org.apache.hadoop.hdfs.protocol.BlockStoragePolicy[]`	`getStoragePolicies()`
`org.apache.hadoop.hdfs.protocol.BlockStoragePolicy`	`getStoragePolicy(byte policyId)`
`org.apache.hadoop.hdfs.protocol.BlockStoragePolicy`	`getStoragePolicy(java.lang.String policyName)`
`BlockStoragePolicySuite`	`getStoragePolicySuite()`
`java.lang.Iterable<DatanodeStorageInfo>`	`getStorages(org.apache.hadoop.hdfs.protocol.Block block)`
`DatanodeStorageInfo[]`	`getStorages(BlockInfo block)`
`java.util.Map<org.apache.hadoop.fs.StorageType,StorageTypeStats>`	`getStorageTypeStats()`	The statistics of storage types.
`BlockInfo`	`getStoredBlock(org.apache.hadoop.hdfs.protocol.Block block)`
`BlockPlacementPolicy`	`getStriptedBlockPlacementPolicy()`
`int`	`getTotalBlocks()`
`long`	`getTotalECBlockGroups()`	Used by metrics.
`long`	`getTotalReplicatedBlocks()`	Used by metrics.
`int`	`getUnderReplicatedNotMissingBlocks()`	Return number of low redundancy blocks but not missing blocks.
`boolean`	`hasMinStorage(BlockInfo block)`
`boolean`	`hasMinStorage(BlockInfo block, int liveNum)`
`boolean`	`hasNonEcBlockUsingStripedID()`	Get the value of whether there are any non-EC blocks using StripedID.
`void`	`initializeReplQueues()`	Initialize replication queues.
`boolean`	`isExcess(DatanodeDescriptor dn, BlockInfo blk)`
`boolean`	`isInSafeMode()`
`boolean`	`isLegacyBlock(org.apache.hadoop.hdfs.protocol.Block block)`
`boolean`	`isPopulatingReplQueues()`	Check if replication queues are to be populated
`boolean`	`isSufficientlyReplicated(BlockInfo b)`	Check if a block is replicated to at least the minimum replication.
`boolean`	`leaveSafeMode(boolean force)`
`void`	`markBlockReplicasAsCorrupt(org.apache.hadoop.hdfs.protocol.Block oldBlock, BlockInfo block, long oldGenerationStamp, long oldNumBytes, DatanodeStorageInfo[] newStorages)`	Mark block replicas as corrupt except those on the storages in newStorages list.
`void`	`metaSave(java.io.PrintWriter out)`	Dump meta data to out.
`static org.apache.hadoop.hdfs.protocol.LocatedBlock`	`newLocatedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock eb, BlockInfo info, DatanodeStorageInfo[] locs, long offset)`
`static org.apache.hadoop.hdfs.protocol.LocatedBlock`	`newLocatedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock b, DatanodeStorageInfo[] storages, long startOffset, boolean corrupt)`
`static org.apache.hadoop.hdfs.protocol.LocatedStripedBlock`	`newLocatedStripedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock b, DatanodeStorageInfo[] storages, byte[] indices, long startOffset, boolean corrupt)`
`long`	`nextBlockId(org.apache.hadoop.hdfs.protocol.BlockType blockType)`
`long`	`nextGenerationStamp(boolean legacyBlock)`
`int`	`numCorruptReplicas(org.apache.hadoop.hdfs.protocol.Block block)`
`int`	`numOfUnderReplicatedBlocks()`
`void`	`processAllPendingDNMessages()`	Process any remaining queued datanode messages after entering active state.
`void`	`processIncrementalBlockReport(org.apache.hadoop.hdfs.protocol.DatanodeID nodeID, StorageReceivedDeletedBlocks srdb)`	The given node is reporting incremental information about some blocks.
`void`	`processMisReplicatedBlocks()`	For each block in the name-node verify whether it belongs to any file, extra or low redundancy.
`int`	`processMisReplicatedBlocks(java.util.List<BlockInfo> blocks)`	Schedule replication work for a specified list of mis-replicated blocks and return total number of blocks scheduled for replication.
`void`	`processQueuedMessagesForBlock(org.apache.hadoop.hdfs.protocol.Block b)`	Try to process any messages that were previously queued for the given block.
`boolean`	`processReport(org.apache.hadoop.hdfs.protocol.DatanodeID nodeID, org.apache.hadoop.hdfs.server.protocol.DatanodeStorage storage, BlockListAsLongs newReport, BlockReportContext context)`	The given storage is reporting all its blocks.
`void`	`refreshBlockPlacementPolicy(org.apache.hadoop.conf.Configuration conf)`
`void`	`registerDatanode(DatanodeRegistration nodeReg)`
`void`	`removeBlock(BlockInfo block)`
`void`	`removeBlockFromMap(BlockInfo block)`
`void`	`removeBlocksAndUpdateSafemodeTotal(INode.BlocksMapUpdateInfo blocks)`	Removes the blocks from blocksmap and updates the safemode blocks total.
`void`	`removeBRLeaseIfNeeded(org.apache.hadoop.hdfs.protocol.DatanodeID nodeID, BlockReportContext context)`
`void`	`removeStoredBlock(BlockInfo storedBlock, DatanodeDescriptor node)`	Modify (block-->datanode) map.
`long`	`requestBlockReportLeaseId(DatanodeRegistration nodeReg)`
`<T> T`	`runBlockOp(java.util.concurrent.Callable<T> action)`
`void`	`setBlockPoolId(java.lang.String blockPoolId)`
`void`	`setBlockRecoveryTimeout(long blockRecoveryTimeout)`
`void`	`setBlocksReplWorkMultiplier(int newVal)`	Updates the value used for blocksReplWorkMultiplier, set by `DFSConfigKeys. DFS_NAMENODE_REPLICATION_WORK_MULTIPLIER_PER_ITERATION` initially.
`void`	`setBlockToken(org.apache.hadoop.hdfs.protocol.LocatedBlock b, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode)`	Generate a block token for the located block.
`void`	`setBlockTotal(long total)`	Set the total number of blocks in the system.
`void`	`setExcessRedundancyTimeout(long timeout)`	Sets the timeout (in seconds) for excess redundancy blocks, if the provided timeout is less than or equal to 0, the default value is used (converted to milliseconds).
`void`	`setExcessRedundancyTimeoutCheckLimit(long limit)`	Sets the limit number of blocks for checking excess redundancy timeout.
`void`	`setExcludeSlowNodesEnabled(boolean enable)`
`void`	`setInitializedReplQueues(boolean v)`
`void`	`setMaxReplicationStreams(int newVal)`
`void`	`setMaxReplicationStreams(int newVal, boolean ensurePositiveInt)`	Updates the value used for maxReplicationStreams, which is set by `DFSConfigKeys.DFS_NAMENODE_REPLICATION_MAX_STREAMS_KEY` initially.
`void`	`setMinBlocksForWrite(int minBlocksForWrite)`
`void`	`setPostponeBlocksFromFuture(boolean postpone)`
`void`	`setReconstructionPendingTimeout(int newVal)`	Updates the value used for pendingReconstruction timeout, which is set by `DFSConfigKeys. DFS_NAMENODE_RECONSTRUCTION_PENDING_TIMEOUT_SEC_KEY` initially.
`void`	`setReplication(short oldRepl, short newRepl, BlockInfo b)`	Set replication for the blocks.
`void`	`setReplicationStreamsHardLimit(int newVal)`	Updates the value used for replicationStreamsHardLimit, which is set by `DFSConfigKeys.DFS_NAMENODE_REPLICATION_STREAMS_HARD_LIMIT_KEY` initially.
`boolean`	`shouldPopulateReplQueues()`
`void`	`shutdown()`
`void`	`stopReconstructionInitializer()`
`void`	`successfulBlockRecovery(BlockInfo block)`	Notification of a successful block recovery.
`void`	`updateLastBlock(BlockInfo lastBlock, org.apache.hadoop.hdfs.protocol.ExtendedBlock newBlock)`
`void`	`verifyReplication(java.lang.String src, short replication, java.lang.String clientName)`	Check whether the replication parameter is within the range determined by system configuration and throw an exception if it's not.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail
- LOG
```
public static final org.slf4j.Logger LOG
```
- blockLog
```
public static final org.slf4j.Logger blockLog
```
- neededReconstruction
```
public final org.apache.hadoop.hdfs.server.blockmanagement.LowRedundancyBlocks neededReconstruction
```
  Store set of Blocks that need to be replicated 1 or more times. We also store pending reconstruction-orders.
- maxReplication
```
public final short maxReplication
```
  The maximum number of replicas allowed for a block
- minReplication
```
public final short minReplication
```
  Minimum copies needed or else write is disallowed
- defaultReplication
```
public final int defaultReplication
```
  Default number of replicas

Constructor Detail

BlockManager

public BlockManager(Namesystem namesystem,
                    boolean haEnabled,
                    org.apache.hadoop.conf.Configuration conf)
             throws java.io.IOException

Throws:: java.io.IOException

Method Detail

getPendingReconstructionBlocksCount

public long getPendingReconstructionBlocksCount()

Used by metrics

getLowRedundancyBlocksCount

public long getLowRedundancyBlocksCount()

Used by metrics

getCorruptReplicaBlocksCount

public long getCorruptReplicaBlocksCount()

Used by metrics

getScheduledReplicationBlocksCount

public long getScheduledReplicationBlocksCount()

Used by metrics

getPendingDeletionBlocksCount

public long getPendingDeletionBlocksCount()

Used by metrics

getStartupDelayBlockDeletionInMs

public long getStartupDelayBlockDeletionInMs()

Used by metrics

getExcessBlocksCount
```
public long getExcessBlocksCount()
```
Used by metrics

getPostponedMisreplicatedBlocksCount

public long getPostponedMisreplicatedBlocksCount()

Used by metrics

getPendingDataNodeMessageCount

public int getPendingDataNodeMessageCount()

Used by metrics

getNumTimedOutPendingReconstructions

public long getNumTimedOutPendingReconstructions()

Used by metrics.

getLowRedundancyBlocks
```
public long getLowRedundancyBlocks()
```
Used by metrics.

getCorruptBlocks
```
public long getCorruptBlocks()
```
Used by metrics.

getMissingBlocks
```
public long getMissingBlocks()
```
Used by metrics.

getMissingReplicationOneBlocks

public long getMissingReplicationOneBlocks()

Used by metrics.

getPendingDeletionReplicatedBlocks

public long getPendingDeletionReplicatedBlocks()

Used by metrics.

getTotalReplicatedBlocks

public long getTotalReplicatedBlocks()

Used by metrics.

getLowRedundancyECBlockGroups

public long getLowRedundancyECBlockGroups()

Used by metrics.

getCorruptECBlockGroups
```
public long getCorruptECBlockGroups()
```
Used by metrics.

getMissingECBlockGroups
```
public long getMissingECBlockGroups()
```
Used by metrics.

getPendingDeletionECBlocks

public long getPendingDeletionECBlocks()

Used by metrics.

getTotalECBlockGroups
```
public long getTotalECBlockGroups()
```
Used by metrics.

getPendingSPSPaths
```
public int getPendingSPSPaths()
```
Used by metrics.

getStoragePolicy

public org.apache.hadoop.hdfs.protocol.BlockStoragePolicy getStoragePolicy(java.lang.String policyName)

getStoragePolicy

public org.apache.hadoop.hdfs.protocol.BlockStoragePolicy getStoragePolicy(byte policyId)

getStoragePolicies

public org.apache.hadoop.hdfs.protocol.BlockStoragePolicy[] getStoragePolicies()

setBlockPoolId

public void setBlockPoolId(java.lang.String blockPoolId)

getBlockPoolId

public java.lang.String getBlockPoolId()

getStoragePolicySuite

public BlockStoragePolicySuite getStoragePolicySuite()

getBlockTokenSecretManager

@VisibleForTesting
public BlockTokenSecretManager getBlockTokenSecretManager()

get the BlockTokenSecretManager

activate

public void activate(org.apache.hadoop.conf.Configuration conf,
                     long blockTotal)

close
```
public void close()
```

getDatanodeManager

public DatanodeManager getDatanodeManager()

Returns:: the datanodeManager

getBlockPlacementPolicy

@VisibleForTesting
public BlockPlacementPolicy getBlockPlacementPolicy()

getStriptedBlockPlacementPolicy

@VisibleForTesting
public BlockPlacementPolicy getStriptedBlockPlacementPolicy()

refreshBlockPlacementPolicy

public void refreshBlockPlacementPolicy(org.apache.hadoop.conf.Configuration conf)

metaSave

public void metaSave(java.io.PrintWriter out)

Dump meta data to out.

getMaxReplicationStreams
```
public int getMaxReplicationStreams()
```
Returns the current setting for maxReplicationStreams, which is set by DFSConfigKeys.DFS_NAMENODE_REPLICATION_MAX_STREAMS_KEY.

Returns:

maxReplicationStreams

setMaxReplicationStreams
```
@VisibleForTesting
public void setMaxReplicationStreams(int newVal,
                                     boolean ensurePositiveInt)
```
Updates the value used for maxReplicationStreams, which is set by DFSConfigKeys.DFS_NAMENODE_REPLICATION_MAX_STREAMS_KEY initially.

Parameters:

newVal - - Must be a positive non-zero integer.

setMaxReplicationStreams

public void setMaxReplicationStreams(int newVal)

getReplicationStreamsHardLimit
```
public int getReplicationStreamsHardLimit()
```
Returns the current setting for maxReplicationStreamsHardLimit, set by DFSConfigKeys.DFS_NAMENODE_REPLICATION_STREAMS_HARD_LIMIT_KEY.

Returns:

maxReplicationStreamsHardLimit

setReplicationStreamsHardLimit
```
public void setReplicationStreamsHardLimit(int newVal)
```
Updates the value used for replicationStreamsHardLimit, which is set by DFSConfigKeys.DFS_NAMENODE_REPLICATION_STREAMS_HARD_LIMIT_KEY initially.

Parameters:

newVal - - Must be a positive non-zero integer.

getBlocksReplWorkMultiplier
```
public int getBlocksReplWorkMultiplier()
```
Returns the current setting for blocksReplWorkMultiplier, set by DFSConfigKeys. DFS_NAMENODE_REPLICATION_WORK_MULTIPLIER_PER_ITERATION.

Returns:

maxReplicationStreamsHardLimit

setBlocksReplWorkMultiplier
```
public void setBlocksReplWorkMultiplier(int newVal)
```
Updates the value used for blocksReplWorkMultiplier, set by DFSConfigKeys. DFS_NAMENODE_REPLICATION_WORK_MULTIPLIER_PER_ITERATION initially.

Parameters:

newVal - - Must be a positive non-zero integer.

setReconstructionPendingTimeout
```
public void setReconstructionPendingTimeout(int newVal)
```
Updates the value used for pendingReconstruction timeout, which is set by DFSConfigKeys. DFS_NAMENODE_RECONSTRUCTION_PENDING_TIMEOUT_SEC_KEY initially.

Parameters:

newVal - - Must be a positive non-zero integer.

getReconstructionPendingTimeout
```
public int getReconstructionPendingTimeout()
```
Returns the current setting for pendingReconstruction timeout, set by DFSConfigKeys.DFS_NAMENODE_RECONSTRUCTION_PENDING_TIMEOUT_SEC_KEY.

getDefaultStorageNum

public int getDefaultStorageNum(BlockInfo block)

getMinReplication
```
public short getMinReplication()
```

getMinStorageNum

public short getMinStorageNum(BlockInfo block)

getMinReplicationToBeInMaintenance

public short getMinReplicationToBeInMaintenance()

hasMinStorage

public boolean hasMinStorage(BlockInfo block)

hasMinStorage

public boolean hasMinStorage(BlockInfo block,
                             int liveNum)

commitOrCompleteLastBlock
```
public boolean commitOrCompleteLastBlock(BlockCollection bc,
                                         org.apache.hadoop.hdfs.protocol.Block commitBlock,
                                         INodesInPath iip)
                                  throws java.io.IOException
```
Commit the last block of the file and mark it as complete if it has meets the minimum redundancy requirement

Parameters:

bc - block collection

commitBlock - - contains client reported block length and generation

iip - - INodes in path to bc

Returns:

true if the last block is changed to committed state.

Throws:

java.io.IOException - if the block does not have at least a minimal number of replicas reported from data-nodes.

addExpectedReplicasToPending
```
public void addExpectedReplicasToPending(BlockInfo blk)
```
If IBR is not sent from expected locations yet, add the datanodes to pendingReconstruction in order to keep RedundancyMonitor from scheduling the block. In case of erasure coding blocks, adds only in case there isn't any missing node.

forceCompleteBlock
```
public void forceCompleteBlock(BlockInfo block)
                        throws java.io.IOException
```
Force the given block in the given file to be marked as complete, regardless of whether enough replicas are present. This is necessary when tailing edit logs as a Standby.

Throws:

java.io.IOException

convertLastBlockToUnderConstruction
```
public org.apache.hadoop.hdfs.protocol.LocatedBlock convertLastBlockToUnderConstruction(BlockCollection bc,
                                                                                        long bytesToRemove)
                                                                                 throws java.io.IOException
```
Convert the last block of the file to an under construction block.
The block is converted only if the file has blocks and the last one is a partial block (its size is less than the preferred block size). The converted block is returned to the client. The client uses the returned block locations to form the data pipeline for this block.
The methods returns null if there is no partial block at the end. The client is supposed to allocate a new block with the next call.

Parameters:

bc - file

bytesToRemove - num of bytes to remove from block

Returns:

the last block locations if the block is partial or null otherwise

Throws:

java.io.IOException

createLocatedBlocks

public org.apache.hadoop.hdfs.protocol.LocatedBlocks createLocatedBlocks(BlockInfo[] blocks,
                                                                         long fileSizeExcludeBlocksUnderConstruction,
                                                                         boolean isFileUnderConstruction,
                                                                         long offset,
                                                                         long length,
                                                                         boolean needBlockToken,
                                                                         boolean inSnapshot,
                                                                         org.apache.hadoop.fs.FileEncryptionInfo feInfo,
                                                                         org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy ecPolicy)
                                                                  throws java.io.IOException

Create a LocatedBlocks.

Throws:: java.io.IOException

getBlockKeys

public ExportedBlockKeys getBlockKeys()

Returns:: current access keys.

setBlockToken

public void setBlockToken(org.apache.hadoop.hdfs.protocol.LocatedBlock b,
                          org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode)
                   throws java.io.IOException

Generate a block token for the located block.

Throws:: java.io.IOException

generateDataEncryptionKey

public org.apache.hadoop.hdfs.security.token.block.DataEncryptionKey generateDataEncryptionKey()

adjustReplication
```
public short adjustReplication(short replication)
```
Clamp the specified replication between the minimum and the maximum replication levels.

verifyReplication
```
public void verifyReplication(java.lang.String src,
                              short replication,
                              java.lang.String clientName)
                       throws java.io.IOException
```
Check whether the replication parameter is within the range determined by system configuration and throw an exception if it's not.

Parameters:

src - the path to the target file

replication - the requested replication factor

clientName - the name of the client node making the request

Throws:

java.io.IOException - thrown if the requested replication factor is out of bounds

isSufficientlyReplicated
```
public boolean isSufficientlyReplicated(BlockInfo b)
```
Check if a block is replicated to at least the minimum replication.

getBlocksWithLocations

public BlocksWithLocations getBlocksWithLocations(org.apache.hadoop.hdfs.protocol.DatanodeID datanode,
                                                  long size,
                                                  long minBlockSize,
                                                  long timeInterval,
                                                  org.apache.hadoop.fs.StorageType storageType)
                                           throws UnregisteredNodeException

Get all blocks with location information from a datanode.

Throws:: UnregisteredNodeException

findAndMarkBlockAsCorrupt

public void findAndMarkBlockAsCorrupt(org.apache.hadoop.hdfs.protocol.ExtendedBlock blk,
                                      org.apache.hadoop.hdfs.protocol.DatanodeInfo dn,
                                      java.lang.String storageID,
                                      java.lang.String reason)
                               throws java.io.IOException

Mark the block belonging to datanode as corrupt

Parameters:: blk - Block to be marked as corrupt; dn - Datanode which holds the corrupt replica; storageID - if known, null otherwise.; reason - a textual reason why the block should be marked corrupt, for logging purposes
Throws:: java.io.IOException

setPostponeBlocksFromFuture

public void setPostponeBlocksFromFuture(boolean postpone)

getUnderReplicatedNotMissingBlocks
```
public int getUnderReplicatedNotMissingBlocks()
```
Return number of low redundancy blocks but not missing blocks.

chooseTarget4WebHDFS

public DatanodeStorageInfo[] chooseTarget4WebHDFS(java.lang.String src,
                                                  DatanodeDescriptor clientnode,
                                                  java.util.Set<org.apache.hadoop.net.Node> excludes,
                                                  long blocksize)

Choose target for WebHDFS redirection.

chooseTarget4AdditionalDatanode

public DatanodeStorageInfo[] chooseTarget4AdditionalDatanode(java.lang.String src,
                                                             int numAdditionalNodes,
                                                             org.apache.hadoop.net.Node clientnode,
                                                             java.util.List<DatanodeStorageInfo> chosen,
                                                             java.util.Set<org.apache.hadoop.net.Node> excludes,
                                                             long blocksize,
                                                             byte storagePolicyID,
                                                             org.apache.hadoop.hdfs.protocol.BlockType blockType)

Choose target for getting additional datanodes for an existing pipeline.

chooseTarget4NewBlock

public DatanodeStorageInfo[] chooseTarget4NewBlock(java.lang.String src,
                                                   int numOfReplicas,
                                                   org.apache.hadoop.net.Node client,
                                                   java.util.Set<org.apache.hadoop.net.Node> excludedNodes,
                                                   long blocksize,
                                                   java.util.List<java.lang.String> favoredNodes,
                                                   byte storagePolicyID,
                                                   org.apache.hadoop.hdfs.protocol.BlockType blockType,
                                                   org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy ecPolicy,
                                                   java.util.EnumSet<org.apache.hadoop.hdfs.AddBlockFlag> flags)
                                            throws java.io.IOException

Choose target datanodes for creating a new block.

Throws:: java.io.IOException - if the number of targets < minimum replication.
See Also:: BlockPlacementPolicy.chooseTarget(String, int, Node, Set, long, List, BlockStoragePolicy, EnumSet)

requestBlockReportLeaseId

public long requestBlockReportLeaseId(DatanodeRegistration nodeReg)

registerDatanode

public void registerDatanode(DatanodeRegistration nodeReg)
                      throws java.io.IOException

Throws:: java.io.IOException

setBlockTotal
```
public void setBlockTotal(long total)
```
Set the total number of blocks in the system. If safe mode is not currently on, this is a no-op.

isInSafeMode
```
public boolean isInSafeMode()
```

getSafeModeTip

public java.lang.String getSafeModeTip()

leaveSafeMode

public boolean leaveSafeMode(boolean force)

checkSafeMode
```
public void checkSafeMode()
```

getBytesInFuture
```
public long getBytesInFuture()
```

getBytesInFutureReplicatedBlocks

public long getBytesInFutureReplicatedBlocks()

getBytesInFutureECBlockGroups

public long getBytesInFutureECBlockGroups()

removeBlocksAndUpdateSafemodeTotal
```
public void removeBlocksAndUpdateSafemodeTotal(INode.BlocksMapUpdateInfo blocks)
```
Removes the blocks from blocksmap and updates the safemode blocks total.

Parameters:

blocks - An instance of INode.BlocksMapUpdateInfo which contains a list of blocks that need to be removed from blocksMap

getProvidedCapacity
```
public long getProvidedCapacity()
```

checkBlockReportLease

public boolean checkBlockReportLease(BlockReportContext context,
                                     org.apache.hadoop.hdfs.protocol.DatanodeID nodeID)
                              throws UnregisteredNodeException

Check block report lease.

Returns:: true if lease exist and not expire
Throws:: UnregisteredNodeException

processReport

public boolean processReport(org.apache.hadoop.hdfs.protocol.DatanodeID nodeID,
                             org.apache.hadoop.hdfs.server.protocol.DatanodeStorage storage,
                             BlockListAsLongs newReport,
                             BlockReportContext context)
                      throws java.io.IOException

The given storage is reporting all its blocks. Update the (storage-->block list) and (block-->storage list) maps.

Returns:: true if all known storages of the given DN have finished reporting.
Throws:: java.io.IOException

removeBRLeaseIfNeeded

public void removeBRLeaseIfNeeded(org.apache.hadoop.hdfs.protocol.DatanodeID nodeID,
                                  BlockReportContext context)
                           throws java.io.IOException

Throws:: java.io.IOException

setExcessRedundancyTimeout
```
public void setExcessRedundancyTimeout(long timeout)
```
Sets the timeout (in seconds) for excess redundancy blocks, if the provided timeout is less than or equal to 0, the default value is used (converted to milliseconds).

Parameters:

timeout - The time (in seconds) to set as the excess redundancy block timeout.

setExcessRedundancyTimeoutCheckLimit
```
public void setExcessRedundancyTimeoutCheckLimit(long limit)
```
Sets the limit number of blocks for checking excess redundancy timeout. If the provided limit is less than or equal to 0, the default limit is used.

Parameters:

limit - The limit number of blocks used to check for excess redundancy timeout.

markBlockReplicasAsCorrupt

public void markBlockReplicasAsCorrupt(org.apache.hadoop.hdfs.protocol.Block oldBlock,
                                       BlockInfo block,
                                       long oldGenerationStamp,
                                       long oldNumBytes,
                                       DatanodeStorageInfo[] newStorages)
                                throws java.io.IOException

Mark block replicas as corrupt except those on the storages in newStorages list.

Throws:: java.io.IOException

processQueuedMessagesForBlock
```
public void processQueuedMessagesForBlock(org.apache.hadoop.hdfs.protocol.Block b)
                                   throws java.io.IOException
```
Try to process any messages that were previously queued for the given block. This is called from FSEditLogLoader whenever a block's state in the namespace has changed or a new block has been created.

Throws:

java.io.IOException

processAllPendingDNMessages
```
public void processAllPendingDNMessages()
                                 throws java.io.IOException
```
Process any remaining queued datanode messages after entering active state. At this point they will not be re-queued since we are the definitive master node and thus should be up-to-date with the namespace information.

Throws:

java.io.IOException

processMisReplicatedBlocks
```
public void processMisReplicatedBlocks()
```
For each block in the name-node verify whether it belongs to any file, extra or low redundancy. Place it into the respective queue.

stopReconstructionInitializer

public void stopReconstructionInitializer()

getReconstructionQueuesInitProgress
```
public float getReconstructionQueuesInitProgress()
```
Get the progress of the reconstruction queues initialisation

Returns:

Returns values between 0 and 1 for the progress.

hasNonEcBlockUsingStripedID
```
public boolean hasNonEcBlockUsingStripedID()
```
Get the value of whether there are any non-EC blocks using StripedID.

Returns:

Returns the value of whether there are any non-EC blocks using StripedID.

processMisReplicatedBlocks
```
public int processMisReplicatedBlocks(java.util.List<BlockInfo> blocks)
```
Schedule replication work for a specified list of mis-replicated blocks and return total number of blocks scheduled for replication.

Parameters:

blocks - A list of blocks for which replication work needs to be scheduled.

Returns:

Total number of blocks for which replication work is scheduled.

setReplication

public void setReplication(short oldRepl,
                           short newRepl,
                           BlockInfo b)

Set replication for the blocks.

removeStoredBlock

public void removeStoredBlock(BlockInfo storedBlock,
                              DatanodeDescriptor node)

Modify (block-->datanode) map. Possibly generate replication tasks, if the removed block is still valid.

addBlock

@VisibleForTesting
public void addBlock(DatanodeStorageInfo storageInfo,
                     org.apache.hadoop.hdfs.protocol.Block block,
                     java.lang.String delHint)
              throws java.io.IOException

The given node is reporting that it received a certain block.

Throws:: java.io.IOException

processIncrementalBlockReport

public void processIncrementalBlockReport(org.apache.hadoop.hdfs.protocol.DatanodeID nodeID,
                                          StorageReceivedDeletedBlocks srdb)
                                   throws java.io.IOException

The given node is reporting incremental information about some blocks. This includes blocks that are starting to be received, completed being received, or deleted. This method must be called with FSNamesystem lock held.

Throws:: java.io.IOException

countNodes
```
public NumberReplicas countNodes(BlockInfo b)
```
Return the number of nodes hosting a given block, grouped by the state of those replicas. For a striped block, this includes nodes storing blocks belonging to the striped block group. But note we exclude duplicated internal block replicas for calculating NumberReplicas.liveReplicas(). If the replica on a decommissioning node is the same as the replica on a live node, the internal block for this replica is live, not decommissioning.

isExcess

public boolean isExcess(DatanodeDescriptor dn,
                        BlockInfo blk)

getActiveBlockCount
```
public int getActiveBlockCount()
```

getStorages

public DatanodeStorageInfo[] getStorages(BlockInfo block)

getStorages

public java.lang.Iterable<DatanodeStorageInfo> getStorages(org.apache.hadoop.hdfs.protocol.Block block)

Returns:: an iterator of the datanodes.

getTotalBlocks
```
public int getTotalBlocks()
```

removeBlock

public void removeBlock(BlockInfo block)

getStoredBlock

public BlockInfo getStoredBlock(org.apache.hadoop.hdfs.protocol.Block block)

updateLastBlock

public void updateLastBlock(BlockInfo lastBlock,
                            org.apache.hadoop.hdfs.protocol.ExtendedBlock newBlock)

checkRedundancy
```
public void checkRedundancy(BlockCollection bc)
```
Check sufficient redundancy of the blocks in the collection. If any block is needed reconstruction, insert it into the reconstruction queue. Otherwise, if the block is more than the expected replication factor, process it as an extra redundancy block.

containsInvalidateBlock

@VisibleForTesting
public boolean containsInvalidateBlock(org.apache.hadoop.hdfs.protocol.DatanodeInfo dn,
                                       org.apache.hadoop.hdfs.protocol.Block block)

getExpectedLiveRedundancyNum

public short getExpectedLiveRedundancyNum(BlockInfo block,
                                          NumberReplicas numberReplicas)

getExpectedRedundancyNum

public short getExpectedRedundancyNum(BlockInfo block)

getMissingBlocksCount
```
public long getMissingBlocksCount()
```

getMissingReplOneBlocksCount

public long getMissingReplOneBlocksCount()

getHighestPriorityReplicatedBlockCount

public long getHighestPriorityReplicatedBlockCount()

getHighestPriorityECBlockCount

public long getHighestPriorityECBlockCount()

addBlockCollection

public BlockInfo addBlockCollection(BlockInfo block,
                                    BlockCollection bc)

addBlockCollectionWithCheck

public BlockInfo addBlockCollectionWithCheck(BlockInfo block,
                                             BlockCollection bc)

Do some check when adding a block to blocksmap. For HDFS-7994 to check whether then block is a NonEcBlockUsingStripedID.

numCorruptReplicas

public int numCorruptReplicas(org.apache.hadoop.hdfs.protocol.Block block)

removeBlockFromMap

public void removeBlockFromMap(BlockInfo block)

getCapacity
```
public int getCapacity()
```

getCorruptReplicaBlockIterator
```
public java.util.Iterator<BlockInfo> getCorruptReplicaBlockIterator()
```
Return an iterator over the set of blocks for which there are no replicas.

getCorruptReplicas

public java.util.Collection<DatanodeDescriptor> getCorruptReplicas(org.apache.hadoop.hdfs.protocol.Block block)

Get the replicas which are corrupt for a given block.

getCorruptReason

public java.lang.String getCorruptReason(org.apache.hadoop.hdfs.protocol.Block block,
                                         DatanodeDescriptor node)

Get reason for certain corrupted replicas for a given block and a given dn.

numOfUnderReplicatedBlocks
```
public int numOfUnderReplicatedBlocks()
```
Returns:

the size of UnderReplicatedBlocks

getLastRedundancyMonitorTS
```
@VisibleForTesting
public long getLastRedundancyMonitorTS()
```
Used as ad hoc to check the time stamp of the last full cycle of redundancyThread. This is used by the Junit tests to block until lastRedundancyCycleTS is updated.

Returns:

the current lastRedundancyCycleTS.

clearQueues
```
public void clearQueues()
```
Clear all queues that hold decisions previously made by this NameNode.

newLocatedBlock

public static org.apache.hadoop.hdfs.protocol.LocatedBlock newLocatedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock b,
                                                                           DatanodeStorageInfo[] storages,
                                                                           long startOffset,
                                                                           boolean corrupt)

newLocatedStripedBlock

public static org.apache.hadoop.hdfs.protocol.LocatedStripedBlock newLocatedStripedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock b,
                                                                                         DatanodeStorageInfo[] storages,
                                                                                         byte[] indices,
                                                                                         long startOffset,
                                                                                         boolean corrupt)

newLocatedBlock

public static org.apache.hadoop.hdfs.protocol.LocatedBlock newLocatedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock eb,
                                                                           BlockInfo info,
                                                                           DatanodeStorageInfo[] locs,
                                                                           long offset)
                                                                    throws java.io.IOException

Throws:: java.io.IOException

shutdown
```
public void shutdown()
```

clear
```
public void clear()
```

getBlockReportLeaseManager

public org.apache.hadoop.hdfs.server.blockmanagement.BlockReportLeaseManager getBlockReportLeaseManager()

getStorageTypeStats
```
public java.util.Map<org.apache.hadoop.fs.StorageType,StorageTypeStats> getStorageTypeStats()
```
Description copied from interface: BlockStatsMXBean

The statistics of storage types.

Specified by:

getStorageTypeStats in interface BlockStatsMXBean

Returns:

get storage statistics per storage type

initializeReplQueues
```
public void initializeReplQueues()
```
Initialize replication queues.

isPopulatingReplQueues
```
public boolean isPopulatingReplQueues()
```
Check if replication queues are to be populated

Returns:

true when node is HAState.Active and not in the very first safemode

setInitializedReplQueues

public void setInitializedReplQueues(boolean v)

shouldPopulateReplQueues

public boolean shouldPopulateReplQueues()

enqueueBlockOp

public void enqueueBlockOp(java.lang.Runnable action)
                    throws java.io.IOException

Throws:: java.io.IOException

runBlockOp

public <T> T runBlockOp(java.util.concurrent.Callable<T> action)
                 throws java.io.IOException

Throws:: java.io.IOException

successfulBlockRecovery
```
public void successfulBlockRecovery(BlockInfo block)
```
Notification of a successful block recovery.

Parameters:

block - for which the recovery succeeded

addBlockRecoveryAttempt
```
public boolean addBlockRecoveryAttempt(BlockInfo b)
```
Checks whether a recovery attempt has been made for the given block. If so, checks whether that attempt has timed out.

Parameters:

b - block for which recovery is being attempted

Returns:

true if no recovery attempt has been made or the previous attempt timed out

flushBlockOps

@VisibleForTesting
public void flushBlockOps()
                   throws java.io.IOException

Throws:: java.io.IOException

getBlockOpQueueLength
```
public int getBlockOpQueueLength()
```

getBlockIdManager

public BlockIdManager getBlockIdManager()

getMarkedDeleteQueue

@VisibleForTesting
public java.util.concurrent.ConcurrentLinkedQueue<java.util.List<BlockInfo>> getMarkedDeleteQueue()

addBLocksToMarkedDeleteQueue

public void addBLocksToMarkedDeleteQueue(java.util.List<BlockInfo> blockInfos)

nextGenerationStamp

public long nextGenerationStamp(boolean legacyBlock)
                         throws java.io.IOException

Throws:: java.io.IOException

isLegacyBlock

public boolean isLegacyBlock(org.apache.hadoop.hdfs.protocol.Block block)

nextBlockId

public long nextBlockId(org.apache.hadoop.hdfs.protocol.BlockType blockType)

setBlockRecoveryTimeout

@VisibleForTesting
public void setBlockRecoveryTimeout(long blockRecoveryTimeout)

getProvidedStorageMap

@VisibleForTesting
public ProvidedStorageMap getProvidedStorageMap()

createSPSManager
```
public boolean createSPSManager(org.apache.hadoop.conf.Configuration conf,
                                java.lang.String spsMode)
```
Create SPS manager instance. It manages the user invoked sps paths and does the movement.

Parameters:

conf - configuration

spsMode - satisfier mode

Returns:

true if the instance is successfully created, false otherwise.

disableSPS
```
public void disableSPS()
```
Nullify SPS manager as this feature is disabled fully.

getSPSManager

public StoragePolicySatisfyManager getSPSManager()

Returns:: sps manager.

setExcludeSlowNodesEnabled

public void setExcludeSlowNodesEnabled(boolean enable)

getExcludeSlowNodesEnabled

@VisibleForTesting
public boolean getExcludeSlowNodesEnabled(org.apache.hadoop.hdfs.protocol.BlockType blockType)

setMinBlocksForWrite

public void setMinBlocksForWrite(int minBlocksForWrite)

getMinBlocksForWrite

@VisibleForTesting
public int getMinBlocksForWrite(org.apache.hadoop.hdfs.protocol.BlockType blockType)

Class BlockManager

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

blockLog

neededReconstruction

maxReplication

minReplication

defaultReplication

Constructor Detail

BlockManager

Method Detail

getPendingReconstructionBlocksCount

getLowRedundancyBlocksCount

getCorruptReplicaBlocksCount

getScheduledReplicationBlocksCount

getPendingDeletionBlocksCount

getStartupDelayBlockDeletionInMs

getExcessBlocksCount

getPostponedMisreplicatedBlocksCount

getPendingDataNodeMessageCount

getNumTimedOutPendingReconstructions

getLowRedundancyBlocks

getCorruptBlocks

getMissingBlocks

getMissingReplicationOneBlocks

getPendingDeletionReplicatedBlocks

getTotalReplicatedBlocks

getLowRedundancyECBlockGroups

getCorruptECBlockGroups

getMissingECBlockGroups

getPendingDeletionECBlocks

getTotalECBlockGroups

getPendingSPSPaths

getStoragePolicy

getStoragePolicy

getStoragePolicies

setBlockPoolId

getBlockPoolId

getStoragePolicySuite

getBlockTokenSecretManager

activate

close

getDatanodeManager

getBlockPlacementPolicy

getStriptedBlockPlacementPolicy

refreshBlockPlacementPolicy

metaSave

getMaxReplicationStreams

setMaxReplicationStreams

setMaxReplicationStreams

getReplicationStreamsHardLimit

setReplicationStreamsHardLimit

getBlocksReplWorkMultiplier

setBlocksReplWorkMultiplier

setReconstructionPendingTimeout

getReconstructionPendingTimeout

getDefaultStorageNum

getMinReplication

getMinStorageNum

getMinReplicationToBeInMaintenance

hasMinStorage

hasMinStorage

commitOrCompleteLastBlock

addExpectedReplicasToPending

forceCompleteBlock

convertLastBlockToUnderConstruction

createLocatedBlocks

getBlockKeys

setBlockToken

generateDataEncryptionKey

adjustReplication

verifyReplication

isSufficientlyReplicated

getBlocksWithLocations

findAndMarkBlockAsCorrupt

setPostponeBlocksFromFuture