Package org.apache.hadoop.hdfs.server.blockmanagement

Class BlockPlacementPolicyWithNodeGroup

java.lang.Object

org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyWithNodeGroup

public class BlockPlacementPolicyWithNodeGroup
extends BlockPlacementPolicyDefault

The class is responsible for choosing the desired number of targets for placing block replicas on environment with node-group layer. The replica placement strategy is adjusted to: If the writer is on a datanode, the 1st replica is placed on the local node(or local node-group or on local rack), otherwise a random datanode. The 2nd replica is placed on a datanode that is on a different rack with 1st replica node. The 3rd replica is placed on a datanode which is on a different node-group but the same rack as the second replica node.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

BlockPlacementPolicy.NotEnoughReplicasException

Field Summary

Fields inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

clusterMap, considerLoad, considerLoadFactor, heartbeatInterval, host2datanodeMap, tolerateHeartbeatMultiplier

Fields inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

LOG

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BlockPlacementPolicyWithNodeGroup()

Method Summary

Modifier and Type	Method	Description
protected int	addToExcludedNodes(DatanodeDescriptor chosenNode, java.util.Set<org.apache.hadoop.net.Node> excludedNodes)	Find other nodes in the same nodegroup of localMachine and add them into excludeNodes as replica should not be duplicated for nodes within the same nodegroup
protected void	chooseFavouredNodes(java.lang.String src, int numOfReplicas, java.util.List<DatanodeDescriptor> favoredNodes, java.util.Set<org.apache.hadoop.net.Node> favoriteAndExcludedNodes, long blocksize, int maxNodesPerRack, java.util.List<DatanodeStorageInfo> results, boolean avoidStaleNodes, java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes)	choose all good favored nodes as target.
protected DatanodeStorageInfo	chooseLocalRack(org.apache.hadoop.net.Node localMachine, java.util.Set<org.apache.hadoop.net.Node> excludedNodes, long blocksize, int maxNodesPerRack, java.util.List<DatanodeStorageInfo> results, boolean avoidStaleNodes, java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes)	Choose one node from the rack that localMachine is on.
protected DatanodeStorageInfo	chooseLocalStorage(org.apache.hadoop.net.Node localMachine, java.util.Set<org.apache.hadoop.net.Node> excludedNodes, long blocksize, int maxNodesPerRack, java.util.List<DatanodeStorageInfo> results, boolean avoidStaleNodes, java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes, boolean fallbackToNodeGroupAndLocalRack)	choose local node of localMachine as the target.
protected void	chooseRemoteRack(int numOfReplicas, DatanodeDescriptor localMachine, java.util.Set<org.apache.hadoop.net.Node> excludedNodes, long blocksize, int maxReplicasPerRack, java.util.List<DatanodeStorageInfo> results, boolean avoidStaleNodes, java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes)	Choose numOfReplicas nodes from the racks that localMachine is NOT on.
protected java.lang.String	getRack(org.apache.hadoop.hdfs.protocol.DatanodeInfo cur)	Get rack string from a data node
void	initialize(org.apache.hadoop.conf.Configuration conf, FSClusterStats stats, org.apache.hadoop.net.NetworkTopology clusterMap, org.apache.hadoop.hdfs.server.blockmanagement.Host2NodesMap host2datanodeMap)	Used to setup a BlockPlacementPolicy object.
boolean	isMovable(java.util.Collection<org.apache.hadoop.hdfs.protocol.DatanodeInfo> locs, org.apache.hadoop.hdfs.protocol.DatanodeInfo source, org.apache.hadoop.hdfs.protocol.DatanodeInfo target)	Check if there are any replica (other than source) on the same node group with target.
java.util.Collection<DatanodeStorageInfo>	pickupReplicaSet(java.util.Collection<DatanodeStorageInfo> first, java.util.Collection<DatanodeStorageInfo> second, java.util.Map<java.lang.String,java.util.List<DatanodeStorageInfo>> rackMap)	Pick up replica node set for deleting replica as over-replicated.
BlockPlacementStatus	verifyBlockPlacement(org.apache.hadoop.hdfs.protocol.DatanodeInfo[] locs, int numberOfReplicas)	Verify if the block's placement meets requirement of placement policy, i.e.

Methods inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

chooseDataNode, chooseDataNode, chooseLocalOrFavoredStorage, chooseLocalStorage, chooseRandom, chooseRandom, chooseReplicasToDelete, chooseReplicaToDelete, chooseTarget, chooseTarget, chooseTargetInOrder, getExcludeSlowNodesEnabled, getMaxNodesPerRack, getMinBlocksForWrite, setExcludeSlowNodesEnabled, setMinBlocksForWrite

Methods inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

adjustSetsWithChosenReplica, getDatanodeInfo, splitNodesWithRack

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlockPlacementPolicyWithNodeGroup

protected BlockPlacementPolicyWithNodeGroup()

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration conf,
                       FSClusterStats stats,
                       org.apache.hadoop.net.NetworkTopology clusterMap,
                       org.apache.hadoop.hdfs.server.blockmanagement.Host2NodesMap host2datanodeMap)

Description copied from class: BlockPlacementPolicy

Used to setup a BlockPlacementPolicy object. This should be defined by all implementations of a BlockPlacementPolicy.

Overrides:

initialize in class BlockPlacementPolicyDefault

Parameters:

conf - the configuration object

stats - retrieve cluster status from here

clusterMap - cluster topology

chooseFavouredNodes

protected void chooseFavouredNodes(java.lang.String src,
                                   int numOfReplicas,
                                   java.util.List<DatanodeDescriptor> favoredNodes,
                                   java.util.Set<org.apache.hadoop.net.Node> favoriteAndExcludedNodes,
                                   long blocksize,
                                   int maxNodesPerRack,
                                   java.util.List<DatanodeStorageInfo> results,
                                   boolean avoidStaleNodes,
                                   java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes)
                            throws BlockPlacementPolicy.NotEnoughReplicasException

choose all good favored nodes as target. If no enough targets, then choose one replica from each bad favored node's node group.

Overrides:

chooseFavouredNodes in class BlockPlacementPolicyDefault

Throws:

BlockPlacementPolicy.NotEnoughReplicasException

chooseLocalStorage

protected DatanodeStorageInfo chooseLocalStorage(org.apache.hadoop.net.Node localMachine,
                                                 java.util.Set<org.apache.hadoop.net.Node> excludedNodes,
                                                 long blocksize,
                                                 int maxNodesPerRack,
                                                 java.util.List<DatanodeStorageInfo> results,
                                                 boolean avoidStaleNodes,
                                                 java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes,
                                                 boolean fallbackToNodeGroupAndLocalRack)
                                          throws BlockPlacementPolicy.NotEnoughReplicasException

choose local node of localMachine as the target. If localMachine is not available, will fallback to nodegroup/rack when flag fallbackToNodeGroupAndLocalRack is set.

Overrides:

chooseLocalStorage in class BlockPlacementPolicyDefault

Returns:

the chosen node

Throws:

BlockPlacementPolicy.NotEnoughReplicasException

chooseLocalRack

protected DatanodeStorageInfo chooseLocalRack(org.apache.hadoop.net.Node localMachine,
                                              java.util.Set<org.apache.hadoop.net.Node> excludedNodes,
                                              long blocksize,
                                              int maxNodesPerRack,
                                              java.util.List<DatanodeStorageInfo> results,
                                              boolean avoidStaleNodes,
                                              java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes)
                                       throws BlockPlacementPolicy.NotEnoughReplicasException

Description copied from class: BlockPlacementPolicyDefault

Choose one node from the rack that localMachine is on. if no such node is available, choose one node from the rack where a second replica is on. if still no such node is available, choose a random node in the cluster.

Overrides:

chooseLocalRack in class BlockPlacementPolicyDefault

Returns:

the chosen node

Throws:

BlockPlacementPolicy.NotEnoughReplicasException

chooseRemoteRack

protected void chooseRemoteRack(int numOfReplicas,
                                DatanodeDescriptor localMachine,
                                java.util.Set<org.apache.hadoop.net.Node> excludedNodes,
                                long blocksize,
                                int maxReplicasPerRack,
                                java.util.List<DatanodeStorageInfo> results,
                                boolean avoidStaleNodes,
                                java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes)
                         throws BlockPlacementPolicy.NotEnoughReplicasException

Description copied from class: BlockPlacementPolicyDefault

Choose numOfReplicas nodes from the racks that localMachine is NOT on. If not enough nodes are available, choose the remaining ones from the local rack

Overrides:

chooseRemoteRack in class BlockPlacementPolicyDefault

Throws:

BlockPlacementPolicy.NotEnoughReplicasException

getRack

protected java.lang.String getRack(org.apache.hadoop.hdfs.protocol.DatanodeInfo cur)

Description copied from class: BlockPlacementPolicy

Get rack string from a data node

Overrides:

getRack in class BlockPlacementPolicy

Returns:

rack of data node

addToExcludedNodes

protected int addToExcludedNodes(DatanodeDescriptor chosenNode,
                                 java.util.Set<org.apache.hadoop.net.Node> excludedNodes)

Find other nodes in the same nodegroup of localMachine and add them into excludeNodes as replica should not be duplicated for nodes within the same nodegroup

Overrides:

addToExcludedNodes in class BlockPlacementPolicyDefault

Returns:

number of new excluded nodes

pickupReplicaSet

public java.util.Collection<DatanodeStorageInfo> pickupReplicaSet(java.util.Collection<DatanodeStorageInfo> first,
                                                                  java.util.Collection<DatanodeStorageInfo> second,
                                                                  java.util.Map<java.lang.String,java.util.List<DatanodeStorageInfo>> rackMap)

Pick up replica node set for deleting replica as over-replicated. First set contains replica nodes on rack with more than one replica while second set contains remaining replica nodes. If first is not empty, divide first set into two subsets: moreThanOne contains nodes on nodegroup with more than one replica exactlyOne contains the remaining nodes in first set then pickup priSet if not empty. If first is empty, then pick second.

Overrides:

pickupReplicaSet in class BlockPlacementPolicyDefault

isMovable

public boolean isMovable(java.util.Collection<org.apache.hadoop.hdfs.protocol.DatanodeInfo> locs,
                         org.apache.hadoop.hdfs.protocol.DatanodeInfo source,
                         org.apache.hadoop.hdfs.protocol.DatanodeInfo target)

Check if there are any replica (other than source) on the same node group with target. If true, then target is not a good candidate for placing specific replica as we don't want 2 replicas under the same nodegroup.

Overrides:

isMovable in class BlockPlacementPolicyDefault

Parameters:

locs - all replicas including source and target

source - source replica of the move

target - target replica of the move

Returns:

true if there are any replica (other than source) on the same node group with target

verifyBlockPlacement

public BlockPlacementStatus verifyBlockPlacement(org.apache.hadoop.hdfs.protocol.DatanodeInfo[] locs,
                                                 int numberOfReplicas)

Description copied from class: BlockPlacementPolicy

Verify if the block's placement meets requirement of placement policy, i.e. replicas are placed on no less than minRacks racks in the system.

Overrides:

verifyBlockPlacement in class BlockPlacementPolicyDefault

Parameters:

locs - block with locations

numberOfReplicas - replica number of file to be verified

Returns:

the result of verification