Package org.apache.hadoop.hdfs.server.namenode

Class FSDirectory

java.lang.Object

org.apache.hadoop.hdfs.server.namenode.FSDirectory

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

@Private
public class FSDirectory
extends java.lang.Object
implements java.io.Closeable

Both FSDirectory and FSNamesystem manage the state of the namespace. FSDirectory is a pure in-memory data structure, all of whose operations happen entirely in memory. In contrast, FSNamesystem persists the operations to the disk.

See Also:

FSNamesystem

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	FSDirectory.DirOp

Field Summary

Fields

Modifier and Type	Field	Description
static byte[]	DOT_INODES
static java.lang.String	DOT_INODES_STRING
static byte[]	DOT_RESERVED
static java.lang.String	DOT_RESERVED_PATH_PREFIX
static org.apache.hadoop.hdfs.protocol.HdfsFileStatus	DOT_RESERVED_STATUS
static java.lang.String	DOT_RESERVED_STRING
static org.apache.hadoop.hdfs.protocol.HdfsFileStatus	DOT_SNAPSHOT_DIR_STATUS
EncryptionZoneManager	ezManager

Method Summary

Modifier and Type	Method	Description
INodesInPath	addLastINode(INodesInPath existing, INode inode, org.apache.hadoop.fs.permission.FsPermission modes, boolean checkQuota)	Add a child to the end of the path specified by INodesInPath.
void	addRootDirToEncryptionZone(XAttrFeature xaf)	This is to handle encryption zone for rootDir when loading from fsimage, and should only be called during NN restart.
void	addToInodeMap(INode inode)	This method is always called with writeLock of FSDirectory held.
void	close()	Shutdown the filestore
BlockStoragePolicySuite	getBlockStoragePolicySuite()
INode	getInode(long id)	Get the inode from inodeMap based on its inode id.
INode	getINode(java.lang.String src)	Get INode associated with the file / directory.
INode	getINode(java.lang.String src, FSDirectory.DirOp dirOp)	Get INode associated with the file / directory.
INode	getINode4Write(java.lang.String src)	Get INode associated with the file / directory.
INodeMap	getINodeMap()
INodesInPath	getINodesInPath(byte[][] components, FSDirectory.DirOp dirOp)
INodesInPath	getINodesInPath(java.lang.String src, FSDirectory.DirOp dirOp)	Resolves the given path into inodes.
long	getLastInodeId()
int	getListLimit()
static byte[][]	getPathComponents(INode inode)	Given an INode get all the path complents leading to it from the root.
java.util.SortedSet<java.lang.String>	getProtectedDirectories()
int	getReadHoldCount()	Deprecated.
INodeDirectory	getRoot()
EnumCounters<org.apache.hadoop.fs.StorageType>	getStorageTypeDeltas(byte storagePolicyID, long dsDelta, short oldRep, short newRep)
int	getWriteHoldCount()	Deprecated.
long	getYieldCount()
static boolean	isExactReservedName(byte[][] components)
static boolean	isExactReservedName(java.lang.String src)
boolean	isImageLoaded()	Indicates whether the image loading is complete or not.
boolean	isNonEmptyDirectory(INodesInPath inodesInPath)
boolean	isPosixAclInheritanceEnabled()
boolean	isProtectedSubDirectoriesEnable()
static boolean	isReservedName(java.lang.String src)	Check if a given path is reserved
static boolean	isReservedName(INode inode)	Check if a given inode name is reserved
void	removeFromInodeMap(java.util.List<? extends INode> inodes)	This method is always called with writeLock of FSDirectory held.
long	removeLastINode(INodesInPath iip)	Remove the last inode in the path from the namespace.
INodesInPath	resolvePath(FSPermissionChecker pc, java.lang.String src, FSDirectory.DirOp dirOp)	Resolves a given path into an INodesInPath.
void	setINodeAttributeProvider(INodeAttributeProvider provider)
void	setPosixAclInheritanceEnabled(boolean posixAclInheritanceEnabled)
INodesInPath	unprotectedResolvePath(java.lang.String src)	This method should only be used from internal paths and not those provided directly by a user.
void	updateCount(INodesInPath iip, INode.QuotaDelta quotaDelta, boolean check)
void	updateSpaceForCompleteBlock(BlockInfo completeBlk, INodesInPath inodes)	Update the cached quota space for a block that is being completed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DOT_RESERVED_STRING

public static final java.lang.String DOT_RESERVED_STRING

See Also:

Constant Field Values

DOT_RESERVED_PATH_PREFIX

public static final java.lang.String DOT_RESERVED_PATH_PREFIX

See Also:

Constant Field Values

DOT_RESERVED

public static final byte[] DOT_RESERVED

DOT_INODES_STRING

public static final java.lang.String DOT_INODES_STRING

See Also:

Constant Field Values

DOT_INODES

public static final byte[] DOT_INODES

DOT_RESERVED_STATUS

public static final org.apache.hadoop.hdfs.protocol.HdfsFileStatus DOT_RESERVED_STATUS

DOT_SNAPSHOT_DIR_STATUS

public static final org.apache.hadoop.hdfs.protocol.HdfsFileStatus DOT_SNAPSHOT_DIR_STATUS

ezManager

@VisibleForTesting
public final EncryptionZoneManager ezManager

Method Detail

setINodeAttributeProvider

public void setINodeAttributeProvider(@Nullable
                                      INodeAttributeProvider provider)

getReadHoldCount

@Deprecated
public int getReadHoldCount()

Deprecated.

getWriteHoldCount

@Deprecated
public int getWriteHoldCount()

Deprecated.

getListLimit

public int getListLimit()

isImageLoaded

public boolean isImageLoaded()

Indicates whether the image loading is complete or not.

Returns:

true if image loading is complete, false otherwise

getProtectedDirectories

public java.util.SortedSet<java.lang.String> getProtectedDirectories()

isProtectedSubDirectoriesEnable

public boolean isProtectedSubDirectoriesEnable()

getRoot

public INodeDirectory getRoot()

Returns:

the root directory inode.

getBlockStoragePolicySuite

public BlockStoragePolicySuite getBlockStoragePolicySuite()

isPosixAclInheritanceEnabled

@VisibleForTesting
public boolean isPosixAclInheritanceEnabled()

setPosixAclInheritanceEnabled

@VisibleForTesting
public void setPosixAclInheritanceEnabled(boolean posixAclInheritanceEnabled)

close

public void close()
           throws java.io.IOException

Shutdown the filestore

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Throws:

java.io.IOException

resolvePath

@VisibleForTesting
public INodesInPath resolvePath(FSPermissionChecker pc,
                                java.lang.String src,
                                FSDirectory.DirOp dirOp)
                         throws org.apache.hadoop.fs.UnresolvedLinkException,
                                java.io.FileNotFoundException,
                                org.apache.hadoop.security.AccessControlException,
                                org.apache.hadoop.fs.ParentNotDirectoryException

Resolves a given path into an INodesInPath. All ancestor inodes that exist are validated as traversable directories. Symlinks in the ancestry will generate an UnresolvedLinkException. The returned IIP will be an accessible path that also passed additional sanity checks based on how the path will be used as specified by the DirOp. READ: Expands reserved paths and performs permission checks during traversal. Raw paths are only accessible by a superuser. WRITE: In addition to READ checks, ensures the path is not a snapshot path. CREATE: In addition to WRITE checks, ensures path does not contain illegal character sequences.

Parameters:

pc - A permission checker for traversal checks. Pass null for no permission checks.

src - The path to resolve.

dirOp - The FSDirectory.DirOp that controls additional checks.

Returns:

if the path indicates an inode, return path after replacing up to <inodeid> with the corresponding path of the inode, else the path in src as is. If the path refers to a path in the "raw" directory, return the non-raw pathname.

Throws:

java.io.FileNotFoundException

org.apache.hadoop.security.AccessControlException

org.apache.hadoop.fs.ParentNotDirectoryException

org.apache.hadoop.fs.UnresolvedLinkException

unprotectedResolvePath

public INodesInPath unprotectedResolvePath(java.lang.String src)
                                    throws java.io.FileNotFoundException

This method should only be used from internal paths and not those provided directly by a user. It resolves a given path into an INodesInPath in a similar way to resolvePath(...), only traversal and permissions are not checked.

Parameters:

src - The path to resolve.

Returns:

if the path indicates an inode, return path after replacing up to <inodeid> with the corresponding path of the inode, else the path in src as is. If the path refers to a path in the "raw" directory, return the non-raw pathname.

Throws:

java.io.FileNotFoundException

isNonEmptyDirectory

public boolean isNonEmptyDirectory(INodesInPath inodesInPath)

Returns:

true if the path is a non-empty directory; otherwise, return false.

updateCount

public void updateCount(INodesInPath iip,
                        INode.QuotaDelta quotaDelta,
                        boolean check)
                 throws org.apache.hadoop.hdfs.protocol.QuotaExceededException

Throws:

org.apache.hadoop.hdfs.protocol.QuotaExceededException

updateSpaceForCompleteBlock

public void updateSpaceForCompleteBlock(BlockInfo completeBlk,
                                        INodesInPath inodes)
                                 throws java.io.IOException

Update the cached quota space for a block that is being completed. Must only be called once, as the block is being completed.

Parameters:

completeBlk - - Completed block for which to update space

inodes - - INodes in path to file containing completeBlk; if null this will be resolved internally

Throws:

java.io.IOException

getStorageTypeDeltas

public EnumCounters<org.apache.hadoop.fs.StorageType> getStorageTypeDeltas(byte storagePolicyID,
                                                                           long dsDelta,
                                                                           short oldRep,
                                                                           short newRep)

addLastINode

@VisibleForTesting
public INodesInPath addLastINode(INodesInPath existing,
                                 INode inode,
                                 org.apache.hadoop.fs.permission.FsPermission modes,
                                 boolean checkQuota)
                          throws org.apache.hadoop.hdfs.protocol.QuotaExceededException

Add a child to the end of the path specified by INodesInPath.

Parameters:

existing - the INodesInPath containing all the ancestral INodes

inode - the new INode to add

modes - create modes

checkQuota - whether to check quota

Returns:

an INodesInPath instance containing the new INode

Throws:

org.apache.hadoop.hdfs.protocol.QuotaExceededException

removeLastINode

@VisibleForTesting
public long removeLastINode(INodesInPath iip)

Remove the last inode in the path from the namespace. Note: the caller needs to update the ancestors' quota count.

Returns:

-1 for failing to remove; 0 for removing a reference whose referred inode has other reference nodes; 1 otherwise.

getYieldCount

@VisibleForTesting
public long getYieldCount()

getINodeMap

public INodeMap getINodeMap()

addToInodeMap

public final void addToInodeMap(INode inode)

This method is always called with writeLock of FSDirectory held.

addRootDirToEncryptionZone

public final void addRootDirToEncryptionZone(XAttrFeature xaf)

This is to handle encryption zone for rootDir when loading from fsimage, and should only be called during NN restart.

removeFromInodeMap

public final void removeFromInodeMap(java.util.List<? extends INode> inodes)

This method is always called with writeLock of FSDirectory held.

getInode

public INode getInode(long id)

Get the inode from inodeMap based on its inode id.

Parameters:

id - The given id

Returns:

The inode associated with the given id

getPathComponents

public static byte[][] getPathComponents(INode inode)

Given an INode get all the path complents leading to it from the root. If an Inode corresponding to C is given in /A/B/C, the returned patch components will be {root, A, B, C}. Note that this method cannot handle scenarios where the inode is in a snapshot.

isReservedName

public static boolean isReservedName(INode inode)

Check if a given inode name is reserved

isReservedName

public static boolean isReservedName(java.lang.String src)

Check if a given path is reserved

isExactReservedName

public static boolean isExactReservedName(java.lang.String src)

isExactReservedName

public static boolean isExactReservedName(byte[][] components)

getINodesInPath

public INodesInPath getINodesInPath(java.lang.String src,
                                    FSDirectory.DirOp dirOp)
                             throws org.apache.hadoop.fs.UnresolvedLinkException,
                                    org.apache.hadoop.security.AccessControlException,
                                    org.apache.hadoop.fs.ParentNotDirectoryException

Resolves the given path into inodes. Reserved paths are not handled and permissions are not verified. Client supplied paths should be resolved via resolvePath(FSPermissionChecker, String, DirOp). This method should only be used by internal methods.

Returns:

the INodesInPath containing all inodes in the path.

Throws:

org.apache.hadoop.fs.UnresolvedLinkException

org.apache.hadoop.fs.ParentNotDirectoryException

org.apache.hadoop.security.AccessControlException

getINodesInPath

public INodesInPath getINodesInPath(byte[][] components,
                                    FSDirectory.DirOp dirOp)
                             throws org.apache.hadoop.fs.UnresolvedLinkException,
                                    org.apache.hadoop.security.AccessControlException,
                                    org.apache.hadoop.fs.ParentNotDirectoryException

Throws:

org.apache.hadoop.fs.UnresolvedLinkException

org.apache.hadoop.security.AccessControlException

org.apache.hadoop.fs.ParentNotDirectoryException

getINode

@VisibleForTesting
public INode getINode(java.lang.String src)
               throws org.apache.hadoop.fs.UnresolvedLinkException,
                      org.apache.hadoop.security.AccessControlException,
                      org.apache.hadoop.fs.ParentNotDirectoryException

Get INode associated with the file / directory. See getINode(String, DirOp)

Throws:

org.apache.hadoop.fs.UnresolvedLinkException

org.apache.hadoop.security.AccessControlException

org.apache.hadoop.fs.ParentNotDirectoryException

getINode4Write

@VisibleForTesting
public INode getINode4Write(java.lang.String src)
                     throws org.apache.hadoop.fs.UnresolvedLinkException,
                            org.apache.hadoop.security.AccessControlException,
                            java.io.FileNotFoundException,
                            org.apache.hadoop.fs.ParentNotDirectoryException

Get INode associated with the file / directory. See getINode(String, DirOp)

Throws:

org.apache.hadoop.fs.UnresolvedLinkException

org.apache.hadoop.security.AccessControlException

java.io.FileNotFoundException

org.apache.hadoop.fs.ParentNotDirectoryException

getINode

public INode getINode(java.lang.String src,
                      FSDirectory.DirOp dirOp)
               throws org.apache.hadoop.fs.UnresolvedLinkException,
                      org.apache.hadoop.security.AccessControlException,
                      org.apache.hadoop.fs.ParentNotDirectoryException

Get INode associated with the file / directory.

Throws:

org.apache.hadoop.fs.UnresolvedLinkException

org.apache.hadoop.security.AccessControlException

org.apache.hadoop.fs.ParentNotDirectoryException

getLastInodeId

public long getLastInodeId()

Returns:

the last inode ID.