Package org.apache.hadoop.hdfs.server.blockmanagement

Class AvailableSpaceBlockPlacementPolicy

java.lang.Object

org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

org.apache.hadoop.hdfs.server.blockmanagement.AvailableSpaceBlockPlacementPolicy

public class AvailableSpaceBlockPlacementPolicy
extends BlockPlacementPolicyDefault

Space balanced block placement policy.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

BlockPlacementPolicy.NotEnoughReplicasException

Field Summary

Fields inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

clusterMap, considerLoad, considerLoadFactor, heartbeatInterval, host2datanodeMap, tolerateHeartbeatMultiplier

Constructor Summary

Constructors

Constructor	Description
AvailableSpaceBlockPlacementPolicy()

Method Summary

Modifier and Type	Method	Description
protected DatanodeDescriptor	chooseDataNode(java.lang.String scope, java.util.Collection<org.apache.hadoop.net.Node> excludedNode)	Choose a datanode from the given scope.
protected DatanodeDescriptor	chooseDataNode(java.lang.String scope, java.util.Collection<org.apache.hadoop.net.Node> excludedNode, org.apache.hadoop.fs.StorageType type)	Choose a datanode from the given scope with specified storage type.
protected DatanodeStorageInfo	chooseLocalStorage(org.apache.hadoop.net.Node localMachine, java.util.Set<org.apache.hadoop.net.Node> excludedNodes, long blocksize, int maxNodesPerRack, java.util.List<DatanodeStorageInfo> results, boolean avoidStaleNodes, java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes, boolean fallbackToLocalRack)	Choose localMachine as the target.
protected int	compareDataNode(DatanodeDescriptor a, DatanodeDescriptor b, boolean isBalanceLocal)	Compare the two data nodes.
void	initialize(org.apache.hadoop.conf.Configuration conf, FSClusterStats stats, org.apache.hadoop.net.NetworkTopology clusterMap, org.apache.hadoop.hdfs.server.blockmanagement.Host2NodesMap host2datanodeMap)	Used to setup a BlockPlacementPolicy object.

Methods inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

addToExcludedNodes, chooseFavouredNodes, chooseLocalOrFavoredStorage, chooseLocalRack, chooseLocalStorage, chooseRandom, chooseRandom, chooseRemoteRack, chooseReplicasToDelete, chooseReplicaToDelete, chooseTarget, chooseTarget, chooseTargetInOrder, getExcludeSlowNodesEnabled, getMaxNodesPerRack, getMinBlocksForWrite, isMovable, pickupReplicaSet, setExcludeSlowNodesEnabled, setMinBlocksForWrite, verifyBlockPlacement

Methods inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

adjustSetsWithChosenReplica, getDatanodeInfo, getRack, splitNodesWithRack

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvailableSpaceBlockPlacementPolicy

public AvailableSpaceBlockPlacementPolicy()

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration conf,
                       FSClusterStats stats,
                       org.apache.hadoop.net.NetworkTopology clusterMap,
                       org.apache.hadoop.hdfs.server.blockmanagement.Host2NodesMap host2datanodeMap)

Description copied from class: BlockPlacementPolicy

Used to setup a BlockPlacementPolicy object. This should be defined by all implementations of a BlockPlacementPolicy.

Overrides:

initialize in class BlockPlacementPolicyDefault

Parameters:

conf - the configuration object

stats - retrieve cluster status from here

clusterMap - cluster topology

chooseDataNode

protected DatanodeDescriptor chooseDataNode(java.lang.String scope,
                                            java.util.Collection<org.apache.hadoop.net.Node> excludedNode,
                                            org.apache.hadoop.fs.StorageType type)

Description copied from class: BlockPlacementPolicyDefault

Choose a datanode from the given scope with specified storage type.

Overrides:

chooseDataNode in class BlockPlacementPolicyDefault

Returns:

the chosen node, if there is any.

chooseLocalStorage

protected DatanodeStorageInfo chooseLocalStorage(org.apache.hadoop.net.Node localMachine,
                                                 java.util.Set<org.apache.hadoop.net.Node> excludedNodes,
                                                 long blocksize,
                                                 int maxNodesPerRack,
                                                 java.util.List<DatanodeStorageInfo> results,
                                                 boolean avoidStaleNodes,
                                                 java.util.EnumMap<org.apache.hadoop.fs.StorageType,java.lang.Integer> storageTypes,
                                                 boolean fallbackToLocalRack)
                                          throws BlockPlacementPolicy.NotEnoughReplicasException

Description copied from class: BlockPlacementPolicyDefault

Choose localMachine as the target. if localMachine is not available, choose a node on the same rack

Overrides:

chooseLocalStorage in class BlockPlacementPolicyDefault

Returns:

the chosen storage

Throws:

BlockPlacementPolicy.NotEnoughReplicasException

chooseDataNode

protected DatanodeDescriptor chooseDataNode(java.lang.String scope,
                                            java.util.Collection<org.apache.hadoop.net.Node> excludedNode)

Description copied from class: BlockPlacementPolicyDefault

Choose a datanode from the given scope.

Overrides:

chooseDataNode in class BlockPlacementPolicyDefault

Returns:

the chosen node, if there is any.

compareDataNode

protected int compareDataNode(DatanodeDescriptor a,
                              DatanodeDescriptor b,
                              boolean isBalanceLocal)

Compare the two data nodes.