Package org.apache.hadoop.hdfs.server.common

Class Storage.StorageDirectory

java.lang.Object

org.apache.hadoop.hdfs.server.common.Storage.StorageDirectory

All Implemented Interfaces:

Storage.FormatConfirmable

Enclosing class:

Storage

@Private
public static class Storage.StorageDirectory
extends java.lang.Object
implements Storage.FormatConfirmable

One of the storage directories.

Constructor Summary

Constructors

Constructor	Description
StorageDirectory(java.io.File dir)
StorageDirectory(java.io.File dir, Storage.StorageDirType dirType)
StorageDirectory(java.io.File dir, Storage.StorageDirType dirType, boolean isShared)	Constructor
StorageDirectory(java.io.File dir, Storage.StorageDirType dirType, boolean isShared, org.apache.hadoop.fs.permission.FsPermission permission)
StorageDirectory(java.lang.String bpid, Storage.StorageDirType dirType, boolean isShared, StorageLocation location)	Constructor
StorageDirectory(Storage.StorageDirType dirType, boolean isShared, StorageLocation location)	Constructor
StorageDirectory(StorageLocation location)

Method Summary

Modifier and Type	Method	Description
Storage.StorageState	analyzeStorage(HdfsServerConstants.StartupOption startOpt, Storage storage)	Check consistency of the storage directory.
Storage.StorageState	analyzeStorage(HdfsServerConstants.StartupOption startOpt, Storage storage, boolean checkCurrentIsEmpty)	Check consistency of the storage directory.
void	clearDirectory()	Clear and re-create storage directory.
void	doRecover(Storage.StorageState curState)	Complete or recover storage state from previously failed transition.
java.io.File	getCurrentDir()	Directory current contains latest files defining the file system meta-data.
long	getDirecorySize()	Get storage directory size.
java.io.File	getFinalizedTmp()	finalized.tmp is a transient directory, which holds the previous file system state while it is being removed in response to the finalize request.
java.io.File	getLastCheckpointTmp()	lastcheckpoint.tmp is a transient directory, which holds current file system state while the new state is saved into the new current during regular namespace updates.
java.io.File	getPreviousCheckpoint()	previous.checkpoint is a directory, which holds the previous (before the last save) state of the storage directory.
java.io.File	getPreviousDir()	Directory previous contains the previous file system state, which the system can be rolled back to.
java.io.File	getPreviousTmp()	previous.tmp is a transient directory, which holds current file system state while the new state is saved into the new current during upgrade.
java.io.File	getPreviousVersionFile()	File VERSION from the previous directory.
java.io.File	getRemovedTmp()	removed.tmp is a transient directory, which holds current file system state while the previous state is moved into current during rollback.
java.io.File	getRoot()	Get root directory of this storage
Storage.StorageDirType	getStorageDirType()	Get storage directory type
StorageLocation	getStorageLocation()
java.lang.String	getStorageUuid()
java.io.File	getVersionFile()	File VERSION contains the following fields: node type layout version namespaceID fs state creation time other fields specific for this node type The version file is always written last during storage directory updates.
boolean	hasSomeData()
boolean	isLockSupported()	Check whether underlying file system supports file locking.
boolean	isShared()
void	lock()	Lock storage to provide exclusive access.
void	read(java.io.File from, Storage storage)
void	setStorageUuid(java.lang.String storageUuid)
java.lang.String	toString()
void	unlock()	Unlock storage.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

StorageDirectory

public StorageDirectory(java.io.File dir)

StorageDirectory

public StorageDirectory(StorageLocation location)

StorageDirectory

public StorageDirectory(java.io.File dir,
                        Storage.StorageDirType dirType)

StorageDirectory

public StorageDirectory(java.io.File dir,
                        Storage.StorageDirType dirType,
                        boolean isShared)

Constructor

Parameters:

dir - directory corresponding to the storage

dirType - storage directory type

isShared - whether or not this dir is shared between two NNs. true disables locking on the storage directory, false enables locking

StorageDirectory

public StorageDirectory(java.io.File dir,
                        Storage.StorageDirType dirType,
                        boolean isShared,
                        org.apache.hadoop.fs.permission.FsPermission permission)

StorageDirectory

public StorageDirectory(Storage.StorageDirType dirType,
                        boolean isShared,
                        StorageLocation location)

Constructor

Parameters:

dirType - storage directory type

isShared - whether or not this dir is shared between two NNs. true disables locking on the storage directory, false enables locking

location - the StorageLocation for this directory

StorageDirectory

public StorageDirectory(java.lang.String bpid,
                        Storage.StorageDirType dirType,
                        boolean isShared,
                        StorageLocation location)

Constructor

Parameters:

bpid - the block pool id

dirType - storage directory type

isShared - whether or not this dir is shared between two NNs. true disables locking on the storage directory, false enables locking

location - the StorageLocation for this directory

Method Detail

setStorageUuid

public void setStorageUuid(java.lang.String storageUuid)

getStorageUuid

public java.lang.String getStorageUuid()

getRoot

public java.io.File getRoot()

Get root directory of this storage

getStorageDirType

public Storage.StorageDirType getStorageDirType()

Get storage directory type

getDirecorySize

public long getDirecorySize()

Get storage directory size.

read

public void read(java.io.File from,
                 Storage storage)
          throws java.io.IOException

Throws:

java.io.IOException

clearDirectory

public void clearDirectory()
                    throws java.io.IOException

Clear and re-create storage directory.

Removes contents of the current directory and creates an empty directory. This does not fully format storage directory. It cannot write the version file since it should be written last after all other storage type dependent files are written. Derived storage is responsible for setting specific storage values and writing the version file to disk.

Throws:

java.io.IOException

getCurrentDir

public java.io.File getCurrentDir()

Directory current contains latest files defining the file system meta-data.

Returns:

the directory path

getVersionFile

public java.io.File getVersionFile()

File VERSION contains the following fields:

1. node type
2. layout version
3. namespaceID
4. fs state creation time
5. other fields specific for this node type

The version file is always written last during storage directory updates. The existence of the version file indicates that all other files have been successfully written in the storage directory, the storage is valid and does not need to be recovered.

Returns:

the version file path

getPreviousVersionFile

public java.io.File getPreviousVersionFile()

File VERSION from the previous directory.

Returns:

the previous version file path

getPreviousDir

public java.io.File getPreviousDir()

Directory previous contains the previous file system state, which the system can be rolled back to.

Returns:

the directory path

getPreviousTmp

public java.io.File getPreviousTmp()

previous.tmp is a transient directory, which holds current file system state while the new state is saved into the new current during upgrade. If the saving succeeds previous.tmp will be moved to previous, otherwise it will be renamed back to current by the recovery procedure during startup.

Returns:

the directory path

getRemovedTmp

public java.io.File getRemovedTmp()

removed.tmp is a transient directory, which holds current file system state while the previous state is moved into current during rollback. If the moving succeeds removed.tmp will be removed, otherwise it will be renamed back to current by the recovery procedure during startup.

Returns:

the directory path

getFinalizedTmp

public java.io.File getFinalizedTmp()

finalized.tmp is a transient directory, which holds the previous file system state while it is being removed in response to the finalize request. Finalize operation will remove finalized.tmp when completed, otherwise the removal will resume upon the system startup.

Returns:

the directory path

getLastCheckpointTmp

public java.io.File getLastCheckpointTmp()

lastcheckpoint.tmp is a transient directory, which holds current file system state while the new state is saved into the new current during regular namespace updates. If the saving succeeds lastcheckpoint.tmp will be moved to previous.checkpoint, otherwise it will be renamed back to current by the recovery procedure during startup.

Returns:

the directory path

getPreviousCheckpoint

public java.io.File getPreviousCheckpoint()

previous.checkpoint is a directory, which holds the previous (before the last save) state of the storage directory. The directory is created as a reference only, it does not play role in state recovery procedures, and is recycled automatically, but it may be useful for manual recovery of a stale state of the system.

Returns:

the directory path

analyzeStorage

public Storage.StorageState analyzeStorage(HdfsServerConstants.StartupOption startOpt,
                                           Storage storage)
                                    throws java.io.IOException

Check consistency of the storage directory.

Parameters:

startOpt - a startup option.

storage - The Storage object that manages this StorageDirectory.

Returns:

state Storage.StorageState of the storage directory

Throws:

InconsistentFSStateException - if directory state is not consistent and cannot be recovered.

java.io.IOException

analyzeStorage

public Storage.StorageState analyzeStorage(HdfsServerConstants.StartupOption startOpt,
                                           Storage storage,
                                           boolean checkCurrentIsEmpty)
                                    throws java.io.IOException

Check consistency of the storage directory.

Parameters:

startOpt - a startup option.

storage - The Storage object that manages this StorageDirectory.

checkCurrentIsEmpty - if true, make sure current/ directory is empty before determining to format it.

Returns:

state Storage.StorageState of the storage directory

Throws:

InconsistentFSStateException - if directory state is not consistent and cannot be recovered.

java.io.IOException

doRecover

public void doRecover(Storage.StorageState curState)
               throws java.io.IOException

Complete or recover storage state from previously failed transition.

Parameters:

curState - specifies what/how the state should be recovered

Throws:

java.io.IOException

hasSomeData

public boolean hasSomeData()
                    throws java.io.IOException

Specified by:

hasSomeData in interface Storage.FormatConfirmable

Returns:

true if the storage directory should prompt the user prior to formatting (i.e if the directory appears to contain some data)

Throws:

java.io.IOException - if the SD cannot be accessed due to an IO error

isShared

public boolean isShared()

lock

public void lock()
          throws java.io.IOException

Lock storage to provide exclusive access.

Locking is not supported by all file systems. E.g., NFS does not consistently support exclusive locks.

If locking is supported we guarantee exclusive access to the storage directory. Otherwise, no guarantee is given.

Throws:

java.io.IOException - if locking fails

unlock

public void unlock()
            throws java.io.IOException

Unlock storage.

Throws:

java.io.IOException

toString

public java.lang.String toString()

Specified by:

toString in interface Storage.FormatConfirmable

Overrides:

toString in class java.lang.Object

Returns:

a string representation of the formattable item, suitable for display to the user inside a prompt

isLockSupported

public boolean isLockSupported()
                        throws java.io.IOException

Check whether underlying file system supports file locking.

Returns:

true if exclusive locks are supported or false otherwise.

Throws:

java.io.IOException

See Also:

lock()

getStorageLocation

public StorageLocation getStorageLocation()