Package org.apache.hadoop.hdfs.server.datanode

Class BlockPoolSliceStorage

java.lang.Object

org.apache.hadoop.hdfs.server.common.StorageInfo

org.apache.hadoop.hdfs.server.common.Storage

org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceStorage

@Private
public class BlockPoolSliceStorage
extends Storage

Manages storage for the set of BlockPoolSlices which share a particular block pool id, on this DataNode. This class supports the following functionality:

• Formatting a new block pool storage
• Recovering a storage state to a consistent state (if possible)
• Taking a snapshot of the block pool during upgrade
• Rolling back a block pool to a previous snapshot
• Finalizing block storage by deletion of a snapshot

See Also:

Storage

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.server.common.Storage

Storage.FormatConfirmable, Storage.StorageDirectory, Storage.StorageDirType, Storage.StorageState

Field Summary

Fields inherited from class org.apache.hadoop.hdfs.server.common.Storage

LAST_PRE_UPGRADE_LAYOUT_VERSION, LAST_UPGRADABLE_HADOOP_VERSION, LAST_UPGRADABLE_LAYOUT_VERSION, LAYOUT_VERSIONS_203, LOG, STORAGE_1_BBW, STORAGE_DIR_CURRENT, STORAGE_DIR_PREVIOUS, STORAGE_FILE_LOCK, STORAGE_PREVIOUS_CKPT, STORAGE_TMP_FINALIZED, STORAGE_TMP_LAST_CKPT, STORAGE_TMP_PREVIOUS, STORAGE_TMP_REMOVED

Fields inherited from class org.apache.hadoop.hdfs.server.common.StorageInfo

clusterID, cTime, layoutVersion, namespaceID, STORAGE_FILE_VERSION, storageType

Constructor Summary

Constructors

Constructor	Description
BlockPoolSliceStorage(StorageInfo storageInfo, java.lang.String bpid)

Method Summary

Modifier and Type	Method	Description
void	addStorageDir(Storage.StorageDirectory sd)
void	clearRollingUpgradeMarkers(java.util.List<Storage.StorageDirectory> dnStorageDirs)	Check whether the rolling upgrade marker file exists for each BP storage root.
void	clearTrash()	Delete all files and directories in the trash directories.
static java.io.File	getBpRoot(java.lang.String bpID, java.io.File dnCurDir)	Get a block pool storage root based on data node storage root
java.lang.String	getTrashDirectory(ReplicaInfo info)	Get a target subdirectory under trash/ for a given block file that is being deleted.
boolean	isPreUpgradableLayout(Storage.StorageDirectory sd)	Return true if the layout of the given storage directory is from a version of Hadoop prior to the introduction of the "current" and "previous" directories which allow upgrade and rollback.
boolean	isTrashAllowed(java.io.File blockFile)	Determine whether we can use trash for the given blockFile.
protected void	setFieldsFromProperties(java.util.Properties props, Storage.StorageDirectory sd)	Get common storage fields.
protected void	setPropertiesFromFields(java.util.Properties props, Storage.StorageDirectory sd)	Set layoutVersion, namespaceID and blockpoolID into block pool storage VERSION file
void	setRollingUpgradeMarkers(java.util.List<Storage.StorageDirectory> dnStorageDirs)	Create a rolling upgrade marker file for each BP storage root, if it does not exist already.
void	stopTrashCleaner()
java.lang.String	toString()
boolean	trashEnabled()	trash is enabled if at least one storage directory contains trash root

Methods inherited from class org.apache.hadoop.hdfs.server.common.Storage

checkVersionUpgradable, confirmFormat, containsStorageDir, containsStorageDir, containsStorageDir, deleteDir, dirIterable, dirIterator, dirIterator, dirIterator, dirIterator, getBuildVersion, getFiles, getNamespaceInfo, getNumStorageDirs, getRegistrationID, getSingularStorageDir, getStorageDir, getStorageDirs, is203LayoutVersion, listStorageDirectories, nativeCopyFileUnbuffered, rename, unlockAll, writeAll, writeProperties, writeProperties, writeProperties

Methods inherited from class org.apache.hadoop.hdfs.server.common.StorageInfo

checkStorageType, getClusterID, getClusterIdFromColonSeparatedString, getCTime, getLayoutVersion, getNamespaceID, getNsIdFromColonSeparatedString, getProperty, getServiceLayoutFeatureMap, getServiceLayoutVersion, readPreviousVersionProperties, readProperties, readPropertiesFile, setClusterId, setcTime, setLayoutVersion, setNamespaceID, setServiceLayoutVersion, setStorageInfo, toColonSeparatedString, toMapString, versionSupportsFederation

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

BlockPoolSliceStorage

public BlockPoolSliceStorage(StorageInfo storageInfo,
                             java.lang.String bpid)

Method Detail

addStorageDir

public void addStorageDir(Storage.StorageDirectory sd)

Overrides:

addStorageDir in class Storage

setPropertiesFromFields

protected void setPropertiesFromFields(java.util.Properties props,
                                       Storage.StorageDirectory sd)
                                throws java.io.IOException

Set layoutVersion, namespaceID and blockpoolID into block pool storage VERSION file

Overrides:

setPropertiesFromFields in class Storage

Parameters:

props - the Properties object to write into

Throws:

java.io.IOException

setFieldsFromProperties

protected void setFieldsFromProperties(java.util.Properties props,
                                       Storage.StorageDirectory sd)
                                throws java.io.IOException

Description copied from class: StorageInfo

Get common storage fields. Should be overloaded if additional fields need to be get.

Overrides:

setFieldsFromProperties in class StorageInfo

Parameters:

props - properties

Throws:

java.io.IOException - on error

toString

public java.lang.String toString()

Overrides:

toString in class StorageInfo

getBpRoot

public static java.io.File getBpRoot(java.lang.String bpID,
                                     java.io.File dnCurDir)

Get a block pool storage root based on data node storage root

Parameters:

bpID - block pool ID

dnCurDir - data node storage root directory

Returns:

root directory for block pool storage

isPreUpgradableLayout

public boolean isPreUpgradableLayout(Storage.StorageDirectory sd)
                              throws java.io.IOException

Description copied from class: Storage

Return true if the layout of the given storage directory is from a version of Hadoop prior to the introduction of the "current" and "previous" directories which allow upgrade and rollback.

Specified by:

isPreUpgradableLayout in class Storage

Throws:

java.io.IOException

isTrashAllowed

@VisibleForTesting
public boolean isTrashAllowed(java.io.File blockFile)

Determine whether we can use trash for the given blockFile. Trash is disallowed if a 'previous' directory exists for the storage directory containing the block.

getTrashDirectory

public java.lang.String getTrashDirectory(ReplicaInfo info)

Get a target subdirectory under trash/ for a given block file that is being deleted. The subdirectory structure under trash/ mirrors that under current/ to keep implicit memory of where the files are to be restored (if necessary).

Returns:

the trash directory for a given block file that is being deleted.

clearTrash

public void clearTrash()

Delete all files and directories in the trash directories.

stopTrashCleaner

public void stopTrashCleaner()

trashEnabled

@VisibleForTesting
public boolean trashEnabled()

trash is enabled if at least one storage directory contains trash root

setRollingUpgradeMarkers

public void setRollingUpgradeMarkers(java.util.List<Storage.StorageDirectory> dnStorageDirs)
                              throws java.io.IOException

Create a rolling upgrade marker file for each BP storage root, if it does not exist already.

Parameters:

dnStorageDirs -

Throws:

java.io.IOException

clearRollingUpgradeMarkers

public void clearRollingUpgradeMarkers(java.util.List<Storage.StorageDirectory> dnStorageDirs)
                                throws java.io.IOException

Check whether the rolling upgrade marker file exists for each BP storage root. If it does exist, then the marker file is cleared and more importantly the layout upgrade is finalized.

Parameters:

dnStorageDirs -

Throws:

java.io.IOException