Package org.apache.hadoop.hdfs.protocol

Interface ClientProtocol

All Known Implementing Classes:

ClientNamenodeProtocolTranslatorPB

@Private
@Evolving
public interface ClientProtocol

ClientProtocol is used by user code via the DistributedFileSystem class to communicate with the NameNode. User code can manipulate the directory namespace, as well as open/close file streams, etc.

Field Summary

Fields

Modifier and Type	Field	Description
static int	GET_STATS_BYTES_IN_FUTURE_BLOCKS_IDX
static int	GET_STATS_CAPACITY_IDX	Constants to index the array of aggregated stats returned by getStats().
static int	GET_STATS_CORRUPT_BLOCKS_IDX
static int	GET_STATS_LOW_REDUNDANCY_IDX
static int	GET_STATS_MISSING_BLOCKS_IDX
static int	GET_STATS_MISSING_REPL_ONE_BLOCKS_IDX
static int	GET_STATS_PENDING_DELETION_BLOCKS_IDX
static int	GET_STATS_REMAINING_IDX
static int	GET_STATS_UNDER_REPLICATED_IDX	Deprecated.
static int	GET_STATS_USED_IDX
static int	STATS_ARRAY_LENGTH
static long	versionID	Until version 69, this class ClientProtocol served as both the client interface to the NN AND the RPC protocol used to communicate with the NN.

Method Summary

Modifier and Type	Method	Description
void	abandonBlock(ExtendedBlock b, long fileId, java.lang.String src, java.lang.String holder)	The client can give up on a block by calling abandonBlock().
LocatedBlock	addBlock(java.lang.String src, java.lang.String clientName, ExtendedBlock previous, DatanodeInfo[] excludeNodes, long fileId, java.lang.String[] favoredNodes, java.util.EnumSet<AddBlockFlag> addBlockFlags)	A client that wants to write an additional block to the indicated filename (which must currently be open for writing) should call addBlock().
long	addCacheDirective(CacheDirectiveInfo directive, java.util.EnumSet<CacheFlag> flags)	Add a CacheDirective to the CacheManager.
void	addCachePool(CachePoolInfo info)	Add a new cache pool.
AddErasureCodingPolicyResponse[]	addErasureCodingPolicies(ErasureCodingPolicy[] policies)	Add Erasure coding policies to HDFS.
void	allowSnapshot(java.lang.String snapshotRoot)	Allow snapshot on a directory.
LastBlockWithStatus	append(java.lang.String src, java.lang.String clientName, org.apache.hadoop.io.EnumSetWritable<org.apache.hadoop.fs.CreateFlag> flag)	Append to the end of the file.
void	cancelDelegationToken(org.apache.hadoop.security.token.Token<DelegationTokenIdentifier> token)	Cancel an existing delegation token.
void	checkAccess(java.lang.String path, org.apache.hadoop.fs.permission.FsAction mode)	Checks if the user can access a path.
boolean	complete(java.lang.String src, java.lang.String clientName, ExtendedBlock last, long fileId)	The client is done writing data to the given filename, and would like to complete it.
void	concat(java.lang.String trg, java.lang.String[] srcs)	Moves blocks from srcs to trg and delete srcs.
HdfsFileStatus	create(java.lang.String src, org.apache.hadoop.fs.permission.FsPermission masked, java.lang.String clientName, org.apache.hadoop.io.EnumSetWritable<org.apache.hadoop.fs.CreateFlag> flag, boolean createParent, short replication, long blockSize, org.apache.hadoop.crypto.CryptoProtocolVersion[] supportedVersions, java.lang.String ecPolicyName, java.lang.String storagePolicy)	Create a new file entry in the namespace.
void	createEncryptionZone(java.lang.String src, java.lang.String keyName)	Create an encryption zone.
java.lang.String	createSnapshot(java.lang.String snapshotRoot, java.lang.String snapshotName)	Create a snapshot.
void	createSymlink(java.lang.String target, java.lang.String link, org.apache.hadoop.fs.permission.FsPermission dirPerm, boolean createParent)	Create symlink to a file or directory.
boolean	delete(java.lang.String src, boolean recursive)	Delete the given file or directory from the file system.
void	deleteSnapshot(java.lang.String snapshotRoot, java.lang.String snapshotName)	Delete a specific snapshot of a snapshottable directory.
void	disableErasureCodingPolicy(java.lang.String ecPolicyName)	Disable erasure coding policy.
void	disallowSnapshot(java.lang.String snapshotRoot)	Disallow snapshot on a directory.
void	enableErasureCodingPolicy(java.lang.String ecPolicyName)	Enable erasure coding policy.
void	finalizeUpgrade()	Finalize previous upgrade.
void	fsync(java.lang.String src, long inodeId, java.lang.String client, long lastBlockLength)	Write all metadata for this file into persistent storage.
org.apache.hadoop.fs.permission.AclStatus	getAclStatus(java.lang.String src)	Gets the ACLs of files and directories.
LocatedBlock	getAdditionalDatanode(java.lang.String src, long fileId, ExtendedBlock blk, DatanodeInfo[] existings, java.lang.String[] existingStorageIDs, DatanodeInfo[] excludes, int numAdditionalNodes, java.lang.String clientName)	Get a datanode for an existing pipeline.
BatchedDirectoryListing	getBatchedListing(java.lang.String[] srcs, byte[] startAfter, boolean needLocation)	Get a partial listing of the input directories
LocatedBlocks	getBlockLocations(java.lang.String src, long offset, long length)	Get locations of the blocks of the specified file within the specified range.
org.apache.hadoop.fs.ContentSummary	getContentSummary(java.lang.String path)	Get ContentSummary rooted at the specified directory.
long	getCurrentEditLogTxid()	Get the highest txid the NameNode knows has been written to the edit log, or -1 if the NameNode's edit log is not yet open for write.
DataEncryptionKey	getDataEncryptionKey()
DatanodeInfo[]	getDatanodeReport(HdfsConstants.DatanodeReportType type)	Get a report on the system's current datanodes.
DatanodeStorageReport[]	getDatanodeStorageReport(HdfsConstants.DatanodeReportType type)	Get a report on the current datanode storages.
org.apache.hadoop.security.token.Token<DelegationTokenIdentifier>	getDelegationToken(org.apache.hadoop.io.Text renewer)	Get a valid Delegation Token.
ECBlockGroupStats	getECBlockGroupStats()	Get statistics pertaining to blocks of type BlockType.STRIPED in the filesystem.
ECTopologyVerifierResult	getECTopologyResultForPolicies(java.lang.String... policyNames)	Verifies if the given policies are supported in the given cluster setup.
EventBatchList	getEditsFromTxid(long txid)	Get an ordered list of batches of events corresponding to the edit log transactions for txids equal to or greater than txid.
org.apache.hadoop.fs.Path	getEnclosingRoot(java.lang.String src)	Get the enclosing root for a path.
java.util.Map<java.lang.String,java.lang.String>	getErasureCodingCodecs()	Get the erasure coding codecs loaded in Namenode.
ErasureCodingPolicyInfo[]	getErasureCodingPolicies()	Get the erasure coding policies loaded in Namenode, excluding REPLICATION policy.
ErasureCodingPolicy	getErasureCodingPolicy(java.lang.String src)	Get the information about the EC policy for the path.
EncryptionZone	getEZForPath(java.lang.String src)	Get the encryption zone for a path.
HdfsFileStatus	getFileInfo(java.lang.String src)	Get the file info for a specific file or directory.
HdfsFileStatus	getFileLinkInfo(java.lang.String src)	Get the file info for a specific file or directory.
org.apache.hadoop.ha.HAServiceProtocol.HAServiceState	getHAServiceState()	Get HA service state of the server.
java.lang.String	getLinkTarget(java.lang.String path)	Return the target of the given symlink.
DirectoryListing	getListing(java.lang.String src, byte[] startAfter, boolean needLocation)	Get a partial listing of the indicated directory.
HdfsLocatedFileStatus	getLocatedFileInfo(java.lang.String src, boolean needBlockToken)	Get the file info for a specific file or directory with LocatedBlocks.
long	getPreferredBlockSize(java.lang.String filename)	Get the block size for the given file.
org.apache.hadoop.fs.QuotaUsage	getQuotaUsage(java.lang.String path)	Get QuotaUsage rooted at the specified directory.
ReplicatedBlockStats	getReplicatedBlockStats()	Get statistics pertaining to blocks of type BlockType.CONTIGUOUS in the filesystem.
org.apache.hadoop.fs.FsServerDefaults	getServerDefaults()	Get server default values for a number of configuration params.
DatanodeInfo[]	getSlowDatanodeReport()	Get report on all of the slow Datanodes.
SnapshotDiffReport	getSnapshotDiffReport(java.lang.String snapshotRoot, java.lang.String fromSnapshot, java.lang.String toSnapshot)	Get the difference between two snapshots, or between a snapshot and the current tree of a directory.
SnapshotDiffReportListing	getSnapshotDiffReportListing(java.lang.String snapshotRoot, java.lang.String fromSnapshot, java.lang.String toSnapshot, byte[] startPath, int index)	Get the difference between two snapshots of a directory iteratively.
SnapshotStatus[]	getSnapshotListing(java.lang.String snapshotRoot)	Get listing of all the snapshots for a snapshottable directory.
SnapshottableDirectoryStatus[]	getSnapshottableDirListing()	Get the list of snapshottable directories that are owned by the current user.
long[]	getStats()	Get an array of aggregated statistics combining blocks of both type BlockType.CONTIGUOUS and BlockType.STRIPED in the filesystem.
BlockStoragePolicy[]	getStoragePolicies()	Get all the available block storage policies.
BlockStoragePolicy	getStoragePolicy(java.lang.String path)	Get the storage policy for a file/directory.
java.util.List<XAttr>	getXAttrs(java.lang.String src, java.util.List<XAttr> xAttrs)	Get xattrs of a file or directory.
boolean	isFileClosed(java.lang.String src)	Get the close status of a file.
org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<CacheDirectiveEntry>	listCacheDirectives(long prevId, CacheDirectiveInfo filter)	List the set of cached paths of a cache pool.
org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<CachePoolEntry>	listCachePools(java.lang.String prevPool)	List the set of cache pools.
CorruptFileBlocks	listCorruptFileBlocks(java.lang.String path, java.lang.String cookie)
org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<EncryptionZone>	listEncryptionZones(long prevId)	Used to implement cursor-based batched listing of EncryptionZones.
org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<OpenFileEntry>	listOpenFiles(long prevId)	Deprecated.
org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<OpenFileEntry>	listOpenFiles(long prevId, java.util.EnumSet<OpenFilesIterator.OpenFilesType> openFilesTypes, java.lang.String path)	List open files in the system in batches.
org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<ZoneReencryptionStatus>	listReencryptionStatus(long prevId)	Used to implement cursor-based batched listing of ZoneReencryptionStatuss.
java.util.List<XAttr>	listXAttrs(java.lang.String src)	List the xattrs names for a file or directory.
void	metaSave(java.lang.String filename)	Dumps namenode data structures into specified file.
boolean	mkdirs(java.lang.String src, org.apache.hadoop.fs.permission.FsPermission masked, boolean createParent)	Create a directory (or hierarchy of directories) with the given name and permission.
void	modifyAclEntries(java.lang.String src, java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)	Modifies ACL entries of files and directories.
void	modifyCacheDirective(CacheDirectiveInfo directive, java.util.EnumSet<CacheFlag> flags)	Modify a CacheDirective in the CacheManager.
void	modifyCachePool(CachePoolInfo req)	Modify an existing cache pool.
void	msync()	Called by client to wait until the server has reached the state id of the client.
boolean	recoverLease(java.lang.String src, java.lang.String clientName)	Start lease recovery.
void	reencryptEncryptionZone(java.lang.String zone, HdfsConstants.ReencryptAction action)	Used to implement re-encryption of encryption zones.
void	refreshNodes()	Tells the namenode to reread the hosts and exclude files.
void	removeAcl(java.lang.String src)	Removes all but the base ACL entries of files and directories.
void	removeAclEntries(java.lang.String src, java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)	Removes ACL entries from files and directories.
void	removeCacheDirective(long id)	Remove a CacheDirectiveInfo from the CacheManager.
void	removeCachePool(java.lang.String pool)	Remove a cache pool.
void	removeDefaultAcl(java.lang.String src)	Removes all default ACL entries from files and directories.
void	removeErasureCodingPolicy(java.lang.String ecPolicyName)	Remove erasure coding policy.
void	removeXAttr(java.lang.String src, XAttr xAttr)	Remove xattr of a file or directory.Value in xAttr parameter is ignored.
boolean	rename(java.lang.String src, java.lang.String dst)	Rename an item in the file system namespace.
void	rename2(java.lang.String src, java.lang.String dst, org.apache.hadoop.fs.Options.Rename... options)	Rename src to dst.
void	renameSnapshot(java.lang.String snapshotRoot, java.lang.String snapshotOldName, java.lang.String snapshotNewName)	Rename a snapshot.
long	renewDelegationToken(org.apache.hadoop.security.token.Token<DelegationTokenIdentifier> token)	Renew an existing delegation token.
void	renewLease(java.lang.String clientName, java.util.List<java.lang.String> namespaces)	Client programs can cause stateful changes in the NameNode that affect other clients.
void	reportBadBlocks(LocatedBlock[] blocks)	The client wants to report corrupted blocks (blocks with specified locations on datanodes).
boolean	restoreFailedStorage(java.lang.String arg)	Enable/Disable restore failed storage.
long	rollEdits()	Roll the edit log.
RollingUpgradeInfo	rollingUpgrade(HdfsConstants.RollingUpgradeAction action)	Rolling upgrade operations.
void	satisfyStoragePolicy(java.lang.String path)	Satisfy the storage policy for a file/directory.
boolean	saveNamespace(long timeWindow, long txGap)	Save namespace image.
void	setAcl(java.lang.String src, java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)	Fully replaces ACL of files and directories, discarding all existing entries.
void	setBalancerBandwidth(long bandwidth)	Tell all datanodes to use a new, non-persistent bandwidth value for dfs.datanode.balance.bandwidthPerSec.
void	setErasureCodingPolicy(java.lang.String src, java.lang.String ecPolicyName)	Set an erasure coding policy on a specified path.
void	setOwner(java.lang.String src, java.lang.String username, java.lang.String groupname)	Set Owner of a path (i.e.
void	setPermission(java.lang.String src, org.apache.hadoop.fs.permission.FsPermission permission)	Set permissions for an existing file/directory.
void	setQuota(java.lang.String path, long namespaceQuota, long storagespaceQuota, org.apache.hadoop.fs.StorageType type)	Set the quota for a directory.
boolean	setReplication(java.lang.String src, short replication)	Set replication for an existing file.
boolean	setSafeMode(HdfsConstants.SafeModeAction action, boolean isChecked)	Enter, leave or get safe mode.
void	setStoragePolicy(java.lang.String src, java.lang.String policyName)	Set the storage policy for a file/directory.
void	setTimes(java.lang.String src, long mtime, long atime)	Sets the modification and access time of the file to the specified time.
void	setXAttr(java.lang.String src, XAttr xAttr, java.util.EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag)	Set xattr of a file or directory.
boolean	truncate(java.lang.String src, long newLength, java.lang.String clientName)	Truncate file src to new size.
void	unsetErasureCodingPolicy(java.lang.String src)	Unset erasure coding policy from a specified path.
void	unsetStoragePolicy(java.lang.String src)	Unset the storage policy set for a given file or directory.
LocatedBlock	updateBlockForPipeline(ExtendedBlock block, java.lang.String clientName)	Get a new generation stamp together with an access token for a block under construction This method is called only when a client needs to recover a failed pipeline or set up a pipeline for appending to a block.
void	updatePipeline(java.lang.String clientName, ExtendedBlock oldBlock, ExtendedBlock newBlock, DatanodeID[] newNodes, java.lang.String[] newStorageIDs)	Update a pipeline for a block under construction.
boolean	upgradeStatus()	Get status of upgrade - finalized or not.

Field Detail

versionID

static final long versionID

Until version 69, this class ClientProtocol served as both the client interface to the NN AND the RPC protocol used to communicate with the NN. This class is used by both the DFSClient and the NN server side to insulate from the protocol serialization. If you are adding/changing this interface then you need to change both this class and ALSO related protocol buffer wire protocol definition in ClientNamenodeProtocol.proto. For more details on protocol buffer wire protocol, please see .../org/apache/hadoop/hdfs/protocolPB/overview.html The log of historical changes can be retrieved from the svn). 69: Eliminate overloaded method names. 69L is the last version id when this class was used for protocols serialization. DO not update this version any further.

See Also:

Constant Field Values

GET_STATS_CAPACITY_IDX

static final int GET_STATS_CAPACITY_IDX

Constants to index the array of aggregated stats returned by getStats().

See Also:

Constant Field Values

GET_STATS_USED_IDX

static final int GET_STATS_USED_IDX

See Also:

Constant Field Values

GET_STATS_REMAINING_IDX

static final int GET_STATS_REMAINING_IDX

See Also:

Constant Field Values

GET_STATS_UNDER_REPLICATED_IDX

@Deprecated
static final int GET_STATS_UNDER_REPLICATED_IDX

Deprecated.

Use GET_STATS_LOW_REDUNDANCY_IDX instead.

See Also:

Constant Field Values

GET_STATS_LOW_REDUNDANCY_IDX

static final int GET_STATS_LOW_REDUNDANCY_IDX

See Also:

Constant Field Values

GET_STATS_CORRUPT_BLOCKS_IDX

static final int GET_STATS_CORRUPT_BLOCKS_IDX

See Also:

Constant Field Values

GET_STATS_MISSING_BLOCKS_IDX

static final int GET_STATS_MISSING_BLOCKS_IDX

See Also:

Constant Field Values

GET_STATS_MISSING_REPL_ONE_BLOCKS_IDX

static final int GET_STATS_MISSING_REPL_ONE_BLOCKS_IDX

See Also:

Constant Field Values

GET_STATS_BYTES_IN_FUTURE_BLOCKS_IDX

static final int GET_STATS_BYTES_IN_FUTURE_BLOCKS_IDX

See Also:

Constant Field Values

GET_STATS_PENDING_DELETION_BLOCKS_IDX

static final int GET_STATS_PENDING_DELETION_BLOCKS_IDX

See Also:

Constant Field Values

STATS_ARRAY_LENGTH

static final int STATS_ARRAY_LENGTH

See Also:

Constant Field Values

Method Detail

getBlockLocations

LocatedBlocks getBlockLocations(java.lang.String src,
                                long offset,
                                long length)
                         throws java.io.IOException

Get locations of the blocks of the specified file within the specified range. DataNode locations for each block are sorted by the proximity to the client.

Return LocatedBlocks which contains file length, blocks and their locations. DataNode locations for each block are sorted by the distance to the client's address.

The client will then have to contact one of the indicated DataNodes to obtain the actual data.

Parameters:

src - file name

offset - range start offset

length - range length

Returns:

file length and array of blocks with their locations

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - If file src does not exist

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

java.io.IOException - If an I/O error occurred

getServerDefaults

org.apache.hadoop.fs.FsServerDefaults getServerDefaults()
                                                 throws java.io.IOException

Get server default values for a number of configuration params.

Returns:

a set of server default configuration values

Throws:

java.io.IOException

create

HdfsFileStatus create(java.lang.String src,
                      org.apache.hadoop.fs.permission.FsPermission masked,
                      java.lang.String clientName,
                      org.apache.hadoop.io.EnumSetWritable<org.apache.hadoop.fs.CreateFlag> flag,
                      boolean createParent,
                      short replication,
                      long blockSize,
                      org.apache.hadoop.crypto.CryptoProtocolVersion[] supportedVersions,
                      java.lang.String ecPolicyName,
                      java.lang.String storagePolicy)
               throws java.io.IOException

Create a new file entry in the namespace.

This will create an empty file specified by the source path. The path should reflect a full path originated at the root. The name-node does not have a notion of "current" directory for a client.

Once created, the file is visible and available for read to other clients. Although, other clients cannot delete(String, boolean), re-create or rename(String, String) it until the file is completed or explicitly as a result of lease expiration.

Blocks have a maximum size. Clients that intend to create multi-block files must also use addBlock(java.lang.String, java.lang.String, org.apache.hadoop.hdfs.protocol.ExtendedBlock, org.apache.hadoop.hdfs.protocol.DatanodeInfo[], long, java.lang.String[], java.util.EnumSet<org.apache.hadoop.hdfs.AddBlockFlag>)

Parameters:

src - path of the file being created.

masked - masked permission.

clientName - name of the current client.

flag - indicates whether the file should be overwritten if it already exists or create if it does not exist or append, or whether the file should be a replicate file, no matter what its ancestor's replication or erasure coding policy is.

createParent - create missing parent directory if true

replication - block replication factor.

blockSize - maximum block size.

supportedVersions - CryptoProtocolVersions supported by the client

ecPolicyName - the name of erasure coding policy. A null value means this file will inherit its parent directory's policy, either traditional replication or erasure coding policy. ecPolicyName and SHOULD_REPLICATE CreateFlag are mutually exclusive. It's invalid to set both SHOULD_REPLICATE flag and a non-null ecPolicyName.

storagePolicy - the name of the storage policy.

Returns:

the status of the created file, it could be null if the server doesn't support returning the file status

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

AlreadyBeingCreatedException - if the path does not exist.

DSQuotaExceededException - If file creation violates disk space quota restriction

org.apache.hadoop.fs.FileAlreadyExistsException - If file src already exists

java.io.FileNotFoundException - If parent of src does not exist and createParent is false

org.apache.hadoop.fs.ParentNotDirectoryException - If parent of src is not a directory.

NSQuotaExceededException - If file creation violates name space quota restriction

SafeModeException - create not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred RuntimeExceptions:

org.apache.hadoop.fs.InvalidPathException - Path src is invalid

Note that create with CreateFlag.OVERWRITE is idempotent.

append

LastBlockWithStatus append(java.lang.String src,
                           java.lang.String clientName,
                           org.apache.hadoop.io.EnumSetWritable<org.apache.hadoop.fs.CreateFlag> flag)
                    throws java.io.IOException

Append to the end of the file.

Parameters:

src - path of the file being created.

clientName - name of the current client.

flag - indicates whether the data is appended to a new block.

Returns:

wrapper with information about the last partial block and file status if any

Throws:

org.apache.hadoop.security.AccessControlException - if permission to append file is denied by the system. As usually on the client side the exception will be wrapped into RemoteException. Allows appending to an existing file if the server is configured with the parameter dfs.support.append set to true, otherwise throws an IOException.

org.apache.hadoop.security.AccessControlException - If permission to append to file is denied

java.io.FileNotFoundException - If file src is not found

DSQuotaExceededException - If append violates disk space quota restriction

SafeModeException - append not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred. RuntimeExceptions:

java.lang.UnsupportedOperationException - if append is not supported

setReplication

boolean setReplication(java.lang.String src,
                       short replication)
                throws java.io.IOException

Set replication for an existing file.

The NameNode sets replication to the new value and returns. The actual block replication is not expected to be performed during this method call. The blocks will be populated or removed in the background as the result of the routine block maintenance procedures.

Parameters:

src - file name

replication - new replication

Returns:

true if successful; false if file does not exist or is a directory

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

DSQuotaExceededException - If replication violates disk space quota restriction

java.io.FileNotFoundException - If file src is not found

SafeModeException - not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - if src contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred

getStoragePolicies

BlockStoragePolicy[] getStoragePolicies()
                                 throws java.io.IOException

Get all the available block storage policies.

Returns:

All the in-use block storage policies currently.

Throws:

java.io.IOException

setStoragePolicy

void setStoragePolicy(java.lang.String src,
                      java.lang.String policyName)
               throws java.io.IOException

Set the storage policy for a file/directory.

Parameters:

src - Path of an existing file/directory.

policyName - The name of the storage policy

Throws:

SnapshotAccessControlException - If access is denied

org.apache.hadoop.fs.UnresolvedLinkException - if src contains a symlink

java.io.FileNotFoundException - If file/dir src is not found

QuotaExceededException - If changes violate the quota restriction

java.io.IOException

unsetStoragePolicy

void unsetStoragePolicy(java.lang.String src)
                 throws java.io.IOException

Unset the storage policy set for a given file or directory.

Parameters:

src - Path of an existing file/directory.

Throws:

SnapshotAccessControlException - If access is denied

org.apache.hadoop.fs.UnresolvedLinkException - if src contains a symlink

java.io.FileNotFoundException - If file/dir src is not found

QuotaExceededException - If changes violate the quota restriction

java.io.IOException

getStoragePolicy

BlockStoragePolicy getStoragePolicy(java.lang.String path)
                             throws java.io.IOException

Get the storage policy for a file/directory.

Parameters:

path - Path of an existing file/directory.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

org.apache.hadoop.fs.UnresolvedLinkException - if src contains a symlink

java.io.FileNotFoundException - If file/dir src is not found

java.io.IOException

setPermission

void setPermission(java.lang.String src,
                   org.apache.hadoop.fs.permission.FsPermission permission)
            throws java.io.IOException

Set permissions for an existing file/directory.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - If file src is not found

SafeModeException - not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred

setOwner

void setOwner(java.lang.String src,
              java.lang.String username,
              java.lang.String groupname)
       throws java.io.IOException

Set Owner of a path (i.e. a file or a directory). The parameters username and groupname cannot both be null.

Parameters:

src - file path

username - If it is null, the original username remains unchanged.

groupname - If it is null, the original groupname remains unchanged.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - If file src is not found

SafeModeException - not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred

abandonBlock

void abandonBlock(ExtendedBlock b,
                  long fileId,
                  java.lang.String src,
                  java.lang.String holder)
           throws java.io.IOException

The client can give up on a block by calling abandonBlock(). The client can then either obtain a new block, or complete or abandon the file. Any partial writes to the block will be discarded.

Parameters:

b - Block to abandon

fileId - The id of the file where the block resides. Older clients will pass GRANDFATHER_INODE_ID here.

src - The path of the file where the block resides.

holder - Lease holder.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - file src is not found

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

java.io.IOException - If an I/O error occurred

addBlock

LocatedBlock addBlock(java.lang.String src,
                      java.lang.String clientName,
                      ExtendedBlock previous,
                      DatanodeInfo[] excludeNodes,
                      long fileId,
                      java.lang.String[] favoredNodes,
                      java.util.EnumSet<AddBlockFlag> addBlockFlags)
               throws java.io.IOException

A client that wants to write an additional block to the indicated filename (which must currently be open for writing) should call addBlock(). addBlock() allocates a new block and datanodes the block data should be replicated to. addBlock() also commits the previous block by reporting to the name-node the actual generation stamp and the length of the block that the client has transmitted to data-nodes.

Parameters:

src - the file being created

clientName - the name of the client that adds the block

previous - previous block

excludeNodes - a list of nodes that should not be allocated for the current block

fileId - the id uniquely identifying a file

favoredNodes - the list of nodes where the client wants the blocks. Nodes are identified by either host name or address.

addBlockFlags - flags to advise the behavior of allocating and placing a new block.

Returns:

LocatedBlock allocated block information.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - If file src is not found

NotReplicatedYetException - previous blocks of the file are not replicated yet. Blocks cannot be added until replication completes.

SafeModeException - create not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

java.io.IOException - If an I/O error occurred

getAdditionalDatanode

LocatedBlock getAdditionalDatanode(java.lang.String src,
                                   long fileId,
                                   ExtendedBlock blk,
                                   DatanodeInfo[] existings,
                                   java.lang.String[] existingStorageIDs,
                                   DatanodeInfo[] excludes,
                                   int numAdditionalNodes,
                                   java.lang.String clientName)
                            throws java.io.IOException

Get a datanode for an existing pipeline.

Parameters:

src - the file being written

fileId - the ID of the file being written

blk - the block being written

existings - the existing nodes in the pipeline

excludes - the excluded nodes

numAdditionalNodes - number of additional datanodes

clientName - the name of the client

Returns:

the located block.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - If file src is not found

SafeModeException - create not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

java.io.IOException - If an I/O error occurred

complete

boolean complete(java.lang.String src,
                 java.lang.String clientName,
                 ExtendedBlock last,
                 long fileId)
          throws java.io.IOException

The client is done writing data to the given filename, and would like to complete it. The function returns whether the file has been closed successfully. If the function returns false, the caller should try again. close() also commits the last block of file by reporting to the name-node the actual generation stamp and the length of the block that the client has transmitted to data-nodes. A call to complete() will not return true until all the file's blocks have been replicated the minimum number of times. Thus, DataNode failures may cause a client to call complete() several times before succeeding.

Parameters:

src - the file being created

clientName - the name of the client that adds the block

last - the last block info

fileId - the id uniquely identifying a file

Returns:

true if all file blocks are minimally replicated or false otherwise

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - If file src is not found

SafeModeException - create not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

java.io.IOException - If an I/O error occurred

reportBadBlocks

void reportBadBlocks(LocatedBlock[] blocks)
              throws java.io.IOException

The client wants to report corrupted blocks (blocks with specified locations on datanodes).

Parameters:

blocks - Array of located blocks to report

Throws:

java.io.IOException

rename

boolean rename(java.lang.String src,
               java.lang.String dst)
        throws java.io.IOException

Rename an item in the file system namespace.

Parameters:

src - existing file or directory name.

dst - new name.

Returns:

true if successful, or false if the old name does not exist or if the new name already belongs to the namespace.

Throws:

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - an I/O error occurred

concat

void concat(java.lang.String trg,
            java.lang.String[] srcs)
     throws java.io.IOException

Moves blocks from srcs to trg and delete srcs.

Parameters:

trg - existing file

srcs - - list of existing files (same block size, same replication)

Throws:

java.io.IOException - if some arguments are invalid

org.apache.hadoop.fs.UnresolvedLinkException - if trg or srcs contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

rename2

void rename2(java.lang.String src,
             java.lang.String dst,
             org.apache.hadoop.fs.Options.Rename... options)
      throws java.io.IOException

Rename src to dst.

• Fails if src is a file and dst is a directory.
• Fails if src is a directory and dst is a file.
• Fails if the parent of dst does not exist or is a file.

Without OVERWRITE option, rename fails if the dst already exists. With OVERWRITE option, rename overwrites the dst, if it is a file or an empty directory. Rename fails if dst is a non-empty directory.

This implementation of rename is atomic.

Parameters:

src - existing file or directory name.

dst - new name.

options - Rename options

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

DSQuotaExceededException - If rename violates disk space quota restriction

org.apache.hadoop.fs.FileAlreadyExistsException - If dst already exists and options has Options.Rename.OVERWRITE option false.

java.io.FileNotFoundException - If src does not exist

NSQuotaExceededException - If rename violates namespace quota restriction

org.apache.hadoop.fs.ParentNotDirectoryException - If parent of dst is not a directory

SafeModeException - rename not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src or dst contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred

truncate

boolean truncate(java.lang.String src,
                 long newLength,
                 java.lang.String clientName)
          throws java.io.IOException

Truncate file src to new size.

• Fails if src is a directory.
• Fails if src does not exist.
• Fails if src is not closed.
• Fails if new size is greater than current size.

This implementation of truncate is purely a namespace operation if truncate occurs at a block boundary. Requires DataNode block recovery otherwise.

Parameters:

src - existing file

newLength - the target size

Returns:

true if client does not need to wait for block recovery, false if client needs to wait for block recovery.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - If file src is not found

SafeModeException - truncate not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred

delete

boolean delete(java.lang.String src,
               boolean recursive)
        throws java.io.IOException

Delete the given file or directory from the file system.

same as delete but provides a way to avoid accidentally deleting non empty directories programmatically.

Parameters:

src - existing name

recursive - if true deletes a non empty directory recursively, else throws an exception.

Returns:

true only if the existing file or directory was actually removed from the file system.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

java.io.FileNotFoundException - If file src is not found

SafeModeException - create not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

org.apache.hadoop.fs.PathIsNotEmptyDirectoryException - if path is a non-empty directory and recursive is set to false

java.io.IOException - If an I/O error occurred

mkdirs

boolean mkdirs(java.lang.String src,
               org.apache.hadoop.fs.permission.FsPermission masked,
               boolean createParent)
        throws java.io.IOException

Create a directory (or hierarchy of directories) with the given name and permission.

Parameters:

src - The path of the directory being created

masked - The masked permission of the directory being created

createParent - create missing parent directory if true

Returns:

True if the operation success.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied

org.apache.hadoop.fs.FileAlreadyExistsException - If src already exists

java.io.FileNotFoundException - If parent of src does not exist and createParent is false

NSQuotaExceededException - If file creation violates quota restriction

org.apache.hadoop.fs.ParentNotDirectoryException - If parent of src is not a directory

SafeModeException - create not allowed in safemode

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred. RunTimeExceptions:

org.apache.hadoop.fs.InvalidPathException - If src is invalid

getListing

DirectoryListing getListing(java.lang.String src,
                            byte[] startAfter,
                            boolean needLocation)
                     throws java.io.IOException

Get a partial listing of the indicated directory.

Parameters:

src - the directory name

startAfter - the name to start listing after encoded in java UTF8

needLocation - if the FileStatus should contain block locations

Returns:

a partial listing starting after startAfter

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file src is not found

org.apache.hadoop.fs.UnresolvedLinkException - If src contains a symlink

java.io.IOException - If an I/O error occurred

getBatchedListing

BatchedDirectoryListing getBatchedListing(java.lang.String[] srcs,
                                          byte[] startAfter,
                                          boolean needLocation)
                                   throws java.io.IOException

Get a partial listing of the input directories

Parameters:

srcs - the input directories

startAfter - the name to start listing after encoded in Java UTF8

needLocation - if the FileStatus should contain block locations

Returns:

a partial listing starting after startAfter. null if the input is empty

Throws:

java.io.IOException - if an I/O error occurred

getSnapshottableDirListing

SnapshottableDirectoryStatus[] getSnapshottableDirListing()
                                                   throws java.io.IOException

Get the list of snapshottable directories that are owned by the current user. Return all the snapshottable directories if the current user is a super user.

Returns:

The list of all the current snapshottable directories.

Throws:

java.io.IOException - If an I/O error occurred.

getSnapshotListing

SnapshotStatus[] getSnapshotListing(java.lang.String snapshotRoot)
                             throws java.io.IOException

Get listing of all the snapshots for a snapshottable directory.

Returns:

Information about all the snapshots for a snapshottable directory

Throws:

java.io.IOException - If an I/O error occurred

renewLease

void renewLease(java.lang.String clientName,
                java.util.List<java.lang.String> namespaces)
         throws java.io.IOException

Client programs can cause stateful changes in the NameNode that affect other clients. A client may obtain a file and neither abandon nor complete it. A client might hold a series of locks that prevent other clients from proceeding. Clearly, it would be bad if a client held a bunch of locks that it never gave up. This can happen easily if the client dies unexpectedly.

So, the NameNode will revoke the locks and live file-creates for clients that it thinks have died. A client tells the NameNode that it is still alive by periodically calling renewLease(). If a certain amount of time passes since the last call to renewLease(), the NameNode assumes the client has died.

Parameters:

namespaces - The full Namespace list that the renewLease rpc should be forwarded by RBF. Tips: NN side, this value should be null. RBF side, if this value is null, this rpc will be forwarded to all available namespaces, else this rpc will be forwarded to the special namespaces.

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.IOException - If an I/O error occurred

recoverLease

boolean recoverLease(java.lang.String src,
                     java.lang.String clientName)
              throws java.io.IOException

Start lease recovery. Lightweight NameNode operation to trigger lease recovery

Parameters:

src - path of the file to start lease recovery

clientName - name of the current client

Returns:

true if the file is already closed

Throws:

java.io.IOException

getStats

long[] getStats()
         throws java.io.IOException

Get an array of aggregated statistics combining blocks of both type BlockType.CONTIGUOUS and BlockType.STRIPED in the filesystem. Use public constants like GET_STATS_CAPACITY_IDX in place of actual numbers to index into the array.

• [0] contains the total storage capacity of the system, in bytes.
• [1] contains the total used space of the system, in bytes.
• [2] contains the available storage of the system, in bytes.
• [3] contains number of low redundancy blocks in the system.
• [4] contains number of corrupt blocks.
• [5] contains number of blocks without any good replicas left.
• [6] contains number of blocks which have replication factor 1 and have lost the only replica.
• [7] contains number of bytes that are at risk for deletion.
• [8] contains number of pending deletion blocks.

Throws:

java.io.IOException

getReplicatedBlockStats

ReplicatedBlockStats getReplicatedBlockStats()
                                      throws java.io.IOException

Get statistics pertaining to blocks of type BlockType.CONTIGUOUS in the filesystem.

Throws:

java.io.IOException

getECBlockGroupStats

ECBlockGroupStats getECBlockGroupStats()
                                throws java.io.IOException

Get statistics pertaining to blocks of type BlockType.STRIPED in the filesystem.

Throws:

java.io.IOException

getDatanodeReport

DatanodeInfo[] getDatanodeReport(HdfsConstants.DatanodeReportType type)
                          throws java.io.IOException

Get a report on the system's current datanodes. One DatanodeInfo object is returned for each DataNode. Return live datanodes if type is LIVE; dead datanodes if type is DEAD; otherwise all datanodes if type is ALL.

Throws:

java.io.IOException

getDatanodeStorageReport

DatanodeStorageReport[] getDatanodeStorageReport(HdfsConstants.DatanodeReportType type)
                                          throws java.io.IOException

Get a report on the current datanode storages.

Throws:

java.io.IOException

getPreferredBlockSize

long getPreferredBlockSize(java.lang.String filename)
                    throws java.io.IOException

Get the block size for the given file.

Parameters:

filename - The name of the file

Returns:

The number of bytes in each block

Throws:

java.io.IOException

org.apache.hadoop.fs.UnresolvedLinkException - if the path contains a symlink.

setSafeMode

boolean setSafeMode(HdfsConstants.SafeModeAction action,
                    boolean isChecked)
             throws java.io.IOException

Enter, leave or get safe mode.

Safe mode is a name node state when it

1. does not accept changes to name space (read-only), and
2. does not replicate or delete blocks.

Safe mode is entered automatically at name node startup. Safe mode can also be entered manually using setSafeMode(SafeModeAction.SAFEMODE_ENTER,false).

At startup the name node accepts data node reports collecting information about block locations. In order to leave safe mode it needs to collect a configurable percentage called threshold of blocks, which satisfy the minimal replication condition. The minimal replication condition is that each block must have at least dfs.namenode.replication.min replicas. When the threshold is reached the name node extends safe mode for a configurable amount of time to let the remaining data nodes to check in before it will start replicating missing blocks. Then the name node leaves safe mode.

If safe mode is turned on manually using setSafeMode(SafeModeAction.SAFEMODE_ENTER,false) then the name node stays in safe mode until it is manually turned off using setSafeMode(SafeModeAction.SAFEMODE_LEAVE,false). Current state of the name node can be verified using setSafeMode(SafeModeAction.SAFEMODE_GET,false)

Configuration parameters:

dfs.safemode.threshold.pct is the threshold parameter.
dfs.safemode.extension is the safe mode extension parameter.
dfs.namenode.replication.min is the minimal replication parameter.

Special cases:

The name node does not enter safe mode at startup if the threshold is set to 0 or if the name space is empty.
If the threshold is set to 1 then all blocks need to have at least minimal replication.
If the threshold value is greater than 1 then the name node will not be able to turn off safe mode automatically.
Safe mode can always be turned off manually.

Parameters:

action -

• 0 leave safe mode;
• 1 enter safe mode;
• 2 get safe mode state.

isChecked - If true then action will be done only in ActiveNN.

Returns:

• 0 if the safe mode is OFF or
• 1 if the safe mode is ON.

Throws:

java.io.IOException

saveNamespace

boolean saveNamespace(long timeWindow,
                      long txGap)
               throws java.io.IOException

Save namespace image.

Saves current namespace into storage directories and reset edits log. Requires superuser privilege and safe mode.

Parameters:

timeWindow - NameNode does a checkpoint if the latest checkpoint was done beyond the given time period (in seconds).

txGap - NameNode does a checkpoint if the gap between the latest checkpoint and the latest transaction id is greater this gap.

Returns:

whether an extra checkpoint has been done

Throws:

java.io.IOException - if image creation failed.

rollEdits

long rollEdits()
        throws java.io.IOException

Roll the edit log. Requires superuser privileges.

Returns:

the txid of the new segment

Throws:

org.apache.hadoop.security.AccessControlException - if the superuser privilege is violated

java.io.IOException - if log roll fails

restoreFailedStorage

boolean restoreFailedStorage(java.lang.String arg)
                      throws java.io.IOException

Enable/Disable restore failed storage.

sets flag to enable restore of failed storage replicas

Throws:

org.apache.hadoop.security.AccessControlException - if the superuser privilege is violated.

java.io.IOException

refreshNodes

void refreshNodes()
           throws java.io.IOException

Tells the namenode to reread the hosts and exclude files.

Throws:

java.io.IOException

finalizeUpgrade

void finalizeUpgrade()
              throws java.io.IOException

Finalize previous upgrade. Remove file system state saved during the upgrade. The upgrade will become irreversible.

Throws:

java.io.IOException

upgradeStatus

boolean upgradeStatus()
               throws java.io.IOException

Get status of upgrade - finalized or not.

Returns:

true if upgrade is finalized or if no upgrade is in progress and false otherwise.

Throws:

java.io.IOException

rollingUpgrade

RollingUpgradeInfo rollingUpgrade(HdfsConstants.RollingUpgradeAction action)
                           throws java.io.IOException

Rolling upgrade operations.

Parameters:

action - either query, prepare or finalize.

Returns:

rolling upgrade information. On query, if no upgrade is in progress, returns null.

Throws:

java.io.IOException

listCorruptFileBlocks

CorruptFileBlocks listCorruptFileBlocks(java.lang.String path,
                                        java.lang.String cookie)
                                 throws java.io.IOException

Returns:

CorruptFileBlocks, containing a list of corrupt files (with duplicates if there is more than one corrupt block in a file) and a cookie

Throws:

java.io.IOException - Each call returns a subset of the corrupt files in the system. To obtain all corrupt files, call this method repeatedly and each time pass in the cookie returned from the previous call.

metaSave

void metaSave(java.lang.String filename)
       throws java.io.IOException

Dumps namenode data structures into specified file. If the file already exists, then append.

Throws:

java.io.IOException

setBalancerBandwidth

void setBalancerBandwidth(long bandwidth)
                   throws java.io.IOException

Tell all datanodes to use a new, non-persistent bandwidth value for dfs.datanode.balance.bandwidthPerSec.

Parameters:

bandwidth - Blanacer bandwidth in bytes per second for this datanode.

Throws:

java.io.IOException

getFileInfo

HdfsFileStatus getFileInfo(java.lang.String src)
                    throws java.io.IOException

Get the file info for a specific file or directory.

Parameters:

src - The string representation of the path to the file

Returns:

object containing information regarding the file or null if file not found

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file src is not found

org.apache.hadoop.fs.UnresolvedLinkException - if the path contains a symlink.

java.io.IOException - If an I/O error occurred

isFileClosed

boolean isFileClosed(java.lang.String src)
              throws java.io.IOException

Get the close status of a file.

Parameters:

src - The string representation of the path to the file

Returns:

return true if file is closed

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file src is not found

org.apache.hadoop.fs.UnresolvedLinkException - if the path contains a symlink.

java.io.IOException - If an I/O error occurred

getFileLinkInfo

HdfsFileStatus getFileLinkInfo(java.lang.String src)
                        throws java.io.IOException

Get the file info for a specific file or directory. If the path refers to a symlink then the FileStatus of the symlink is returned.

Parameters:

src - The string representation of the path to the file

Returns:

object containing information regarding the file or null if file not found

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

org.apache.hadoop.fs.UnresolvedLinkException - if src contains a symlink

java.io.IOException - If an I/O error occurred

getLocatedFileInfo

HdfsLocatedFileStatus getLocatedFileInfo(java.lang.String src,
                                         boolean needBlockToken)
                                  throws java.io.IOException

Get the file info for a specific file or directory with LocatedBlocks.

Parameters:

src - The string representation of the path to the file

needBlockToken - Generate block tokens for LocatedBlocks

Returns:

object containing information regarding the file or null if file not found

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file src is not found

java.io.IOException - If an I/O error occurred

getContentSummary

org.apache.hadoop.fs.ContentSummary getContentSummary(java.lang.String path)
                                               throws java.io.IOException

Get ContentSummary rooted at the specified directory.

Parameters:

path - The string representation of the path

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file path is not found

org.apache.hadoop.fs.UnresolvedLinkException - if path contains a symlink.

java.io.IOException - If an I/O error occurred

setQuota

void setQuota(java.lang.String path,
              long namespaceQuota,
              long storagespaceQuota,
              org.apache.hadoop.fs.StorageType type)
       throws java.io.IOException

Set the quota for a directory.

Parameters:

path - The string representation of the path to the directory

namespaceQuota - Limit on the number of names in the tree rooted at the directory

storagespaceQuota - Limit on storage space occupied all the files under this directory.

type - StorageType that the space quota is intended to be set on. It may be null when called by traditional space/namespace quota. When type is is not null, the storagespaceQuota parameter is for type specified and namespaceQuota must be HdfsConstants.QUOTA_DONT_SET.

The quota can have three types of values : (1) 0 or more will set the quota to that value, (2) HdfsConstants.QUOTA_DONT_SET implies the quota will not be changed, and (3) HdfsConstants.QUOTA_RESET implies the quota will be reset. Any other value is a runtime error.

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file path is not found

QuotaExceededException - if the directory size is greater than the given quota

org.apache.hadoop.fs.UnresolvedLinkException - if the path contains a symlink.

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred

fsync

void fsync(java.lang.String src,
           long inodeId,
           java.lang.String client,
           long lastBlockLength)
    throws java.io.IOException

Write all metadata for this file into persistent storage. The file must be currently open for writing.

Parameters:

src - The string representation of the path

inodeId - The inode ID, or GRANDFATHER_INODE_ID if the client is too old to support fsync with inode IDs.

client - The string representation of the client

lastBlockLength - The length of the last block (under construction) to be reported to NameNode

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file src is not found

org.apache.hadoop.fs.UnresolvedLinkException - if src contains a symlink.

java.io.IOException - If an I/O error occurred

setTimes

void setTimes(java.lang.String src,
              long mtime,
              long atime)
       throws java.io.IOException

Sets the modification and access time of the file to the specified time.

Parameters:

src - The string representation of the path

mtime - The number of milliseconds since Jan 1, 1970. Setting negative mtime means that modification time should not be set by this call.

atime - The number of milliseconds since Jan 1, 1970. Setting negative atime means that access time should not be set by this call.

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file src is not found

org.apache.hadoop.fs.UnresolvedLinkException - if src contains a symlink.

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred

createSymlink

void createSymlink(java.lang.String target,
                   java.lang.String link,
                   org.apache.hadoop.fs.permission.FsPermission dirPerm,
                   boolean createParent)
            throws java.io.IOException

Create symlink to a file or directory.

Parameters:

target - The path of the destination that the link points to.

link - The path of the link being created.

dirPerm - permissions to use when creating parent directories

createParent - - if true then missing parent dirs are created if false then parent must exist

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

org.apache.hadoop.fs.FileAlreadyExistsException - If file link already exists

java.io.FileNotFoundException - If parent of link does not exist and createParent is false

org.apache.hadoop.fs.ParentNotDirectoryException - If parent of link is not a directory.

org.apache.hadoop.fs.UnresolvedLinkException - if link contains a symlink.

SnapshotAccessControlException - if path is in RO snapshot

java.io.IOException - If an I/O error occurred

getLinkTarget

java.lang.String getLinkTarget(java.lang.String path)
                        throws java.io.IOException

Return the target of the given symlink. If there is an intermediate symlink in the path (ie a symlink leading up to the final path component) then the given path is returned with this symlink resolved.

Parameters:

path - The path with a link that needs resolution.

Returns:

The path after resolving the first symbolic link in the path.

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - If path does not exist

java.io.IOException - If the given path does not refer to a symlink or an I/O error occurred

updateBlockForPipeline

LocatedBlock updateBlockForPipeline(ExtendedBlock block,
                                    java.lang.String clientName)
                             throws java.io.IOException

Get a new generation stamp together with an access token for a block under construction This method is called only when a client needs to recover a failed pipeline or set up a pipeline for appending to a block.

Parameters:

block - a block

clientName - the name of the client

Returns:

a located block with a new generation stamp and an access token

Throws:

java.io.IOException - if any error occurs

updatePipeline

void updatePipeline(java.lang.String clientName,
                    ExtendedBlock oldBlock,
                    ExtendedBlock newBlock,
                    DatanodeID[] newNodes,
                    java.lang.String[] newStorageIDs)
             throws java.io.IOException

Update a pipeline for a block under construction.

Parameters:

clientName - the name of the client

oldBlock - the old block

newBlock - the new block containing new generation stamp and length

newNodes - datanodes in the pipeline

Throws:

java.io.IOException - if any error occurs

getDelegationToken

org.apache.hadoop.security.token.Token<DelegationTokenIdentifier> getDelegationToken(org.apache.hadoop.io.Text renewer)
                                                                              throws java.io.IOException

Get a valid Delegation Token.

Parameters:

renewer - the designated renewer for the token

Throws:

java.io.IOException

renewDelegationToken

long renewDelegationToken(org.apache.hadoop.security.token.Token<DelegationTokenIdentifier> token)
                   throws java.io.IOException

Renew an existing delegation token.

Parameters:

token - delegation token obtained earlier

Returns:

the new expiration time

Throws:

java.io.IOException

cancelDelegationToken

void cancelDelegationToken(org.apache.hadoop.security.token.Token<DelegationTokenIdentifier> token)
                    throws java.io.IOException

Cancel an existing delegation token.

Parameters:

token - delegation token

Throws:

java.io.IOException

getDataEncryptionKey

DataEncryptionKey getDataEncryptionKey()
                                throws java.io.IOException

Returns:

encryption key so a client can encrypt data sent via the DataTransferProtocol to/from DataNodes.

Throws:

java.io.IOException

createSnapshot

java.lang.String createSnapshot(java.lang.String snapshotRoot,
                                java.lang.String snapshotName)
                         throws java.io.IOException

Create a snapshot.

Parameters:

snapshotRoot - the path that is being snapshotted

snapshotName - name of the snapshot created

Returns:

the snapshot path.

Throws:

java.io.IOException

deleteSnapshot

void deleteSnapshot(java.lang.String snapshotRoot,
                    java.lang.String snapshotName)
             throws java.io.IOException

Delete a specific snapshot of a snapshottable directory.

Parameters:

snapshotRoot - The snapshottable directory

snapshotName - Name of the snapshot for the snapshottable directory

Throws:

java.io.IOException

renameSnapshot

void renameSnapshot(java.lang.String snapshotRoot,
                    java.lang.String snapshotOldName,
                    java.lang.String snapshotNewName)
             throws java.io.IOException

Rename a snapshot.

Parameters:

snapshotRoot - the directory path where the snapshot was taken

snapshotOldName - old name of the snapshot

snapshotNewName - new name of the snapshot

Throws:

java.io.IOException

allowSnapshot

void allowSnapshot(java.lang.String snapshotRoot)
            throws java.io.IOException

Allow snapshot on a directory.

Parameters:

snapshotRoot - the directory to be snapped

Throws:

java.io.IOException - on error

disallowSnapshot

void disallowSnapshot(java.lang.String snapshotRoot)
               throws java.io.IOException

Disallow snapshot on a directory.

Parameters:

snapshotRoot - the directory to disallow snapshot

Throws:

java.io.IOException - on error

getSnapshotDiffReport

SnapshotDiffReport getSnapshotDiffReport(java.lang.String snapshotRoot,
                                         java.lang.String fromSnapshot,
                                         java.lang.String toSnapshot)
                                  throws java.io.IOException

Get the difference between two snapshots, or between a snapshot and the current tree of a directory.

Parameters:

snapshotRoot - full path of the directory where snapshots are taken

fromSnapshot - snapshot name of the from point. Null indicates the current tree

toSnapshot - snapshot name of the to point. Null indicates the current tree.

Returns:

The difference report represented as a SnapshotDiffReport.

Throws:

java.io.IOException - on error

getSnapshotDiffReportListing

SnapshotDiffReportListing getSnapshotDiffReportListing(java.lang.String snapshotRoot,
                                                       java.lang.String fromSnapshot,
                                                       java.lang.String toSnapshot,
                                                       byte[] startPath,
                                                       int index)
                                                throws java.io.IOException

Get the difference between two snapshots of a directory iteratively.

Parameters:

snapshotRoot - full path of the directory where snapshots are taken

fromSnapshot - snapshot name of the from point. Null indicates the current tree

toSnapshot - snapshot name of the to point. Null indicates the current tree.

startPath - path relative to the snapshottable root directory from where the snapshotdiff computation needs to start across multiple rpc calls

index - index in the created or deleted list of the directory at which the snapshotdiff computation stopped during the last rpc call as the no of entries exceeded the snapshotdiffentry limit. -1 indicates, the snapshotdiff compuatation needs to start right from the startPath provided.

Returns:

The difference report represented as a SnapshotDiffReport.

Throws:

java.io.IOException - on error

addCacheDirective

long addCacheDirective(CacheDirectiveInfo directive,
                       java.util.EnumSet<CacheFlag> flags)
                throws java.io.IOException

Add a CacheDirective to the CacheManager.

Parameters:

directive - A CacheDirectiveInfo to be added

flags - CacheFlags to use for this operation.

Returns:

A CacheDirectiveInfo associated with the added directive

Throws:

java.io.IOException - if the directive could not be added

modifyCacheDirective

void modifyCacheDirective(CacheDirectiveInfo directive,
                          java.util.EnumSet<CacheFlag> flags)
                   throws java.io.IOException

Modify a CacheDirective in the CacheManager.

Parameters:

flags - CacheFlags to use for this operation.

Throws:

java.io.IOException - if the directive could not be modified

removeCacheDirective

void removeCacheDirective(long id)
                   throws java.io.IOException

Remove a CacheDirectiveInfo from the CacheManager.

Parameters:

id - of a CacheDirectiveInfo

Throws:

java.io.IOException - if the cache directive could not be removed

listCacheDirectives

org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<CacheDirectiveEntry> listCacheDirectives(long prevId,
                                                                                                   CacheDirectiveInfo filter)
                                                                                            throws java.io.IOException

List the set of cached paths of a cache pool. Incrementally fetches results from the server.

Parameters:

prevId - The last listed entry ID, or -1 if this is the first call to listCacheDirectives.

filter - Parameters to use to filter the list results, or null to display all directives visible to us.

Returns:

A batch of CacheDirectiveEntry objects.

Throws:

java.io.IOException

addCachePool

void addCachePool(CachePoolInfo info)
           throws java.io.IOException

Add a new cache pool.

Parameters:

info - Description of the new cache pool

Throws:

java.io.IOException - If the request could not be completed.

modifyCachePool

void modifyCachePool(CachePoolInfo req)
              throws java.io.IOException

Modify an existing cache pool.

Parameters:

req - The request to modify a cache pool.

Throws:

java.io.IOException - If the request could not be completed.

removeCachePool

void removeCachePool(java.lang.String pool)
              throws java.io.IOException

Remove a cache pool.

Parameters:

pool - name of the cache pool to remove.

Throws:

java.io.IOException - if the cache pool did not exist, or could not be removed.

listCachePools

org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<CachePoolEntry> listCachePools(java.lang.String prevPool)
                                                                                  throws java.io.IOException

List the set of cache pools. Incrementally fetches results from the server.

Parameters:

prevPool - name of the last pool listed, or the empty string if this is the first invocation of listCachePools

Returns:

A batch of CachePoolEntry objects.

Throws:

java.io.IOException

modifyAclEntries

void modifyAclEntries(java.lang.String src,
                      java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
               throws java.io.IOException

Modifies ACL entries of files and directories. This method can add new ACL entries or modify the permissions on existing ACL entries. All existing ACL entries that are not specified in this call are retained without changes. (Modifications are merged into the current ACL.)

Throws:

java.io.IOException

removeAclEntries

void removeAclEntries(java.lang.String src,
                      java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
               throws java.io.IOException

Removes ACL entries from files and directories. Other ACL entries are retained.

Throws:

java.io.IOException

removeDefaultAcl

void removeDefaultAcl(java.lang.String src)
               throws java.io.IOException

Removes all default ACL entries from files and directories.

Throws:

java.io.IOException

removeAcl

void removeAcl(java.lang.String src)
        throws java.io.IOException

Removes all but the base ACL entries of files and directories. The entries for user, group, and others are retained for compatibility with permission bits.

Throws:

java.io.IOException

setAcl

void setAcl(java.lang.String src,
            java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
     throws java.io.IOException

Fully replaces ACL of files and directories, discarding all existing entries.

Throws:

java.io.IOException

getAclStatus

org.apache.hadoop.fs.permission.AclStatus getAclStatus(java.lang.String src)
                                                throws java.io.IOException

Gets the ACLs of files and directories.

Throws:

java.io.IOException

createEncryptionZone

void createEncryptionZone(java.lang.String src,
                          java.lang.String keyName)
                   throws java.io.IOException

Create an encryption zone.

Throws:

java.io.IOException

getEZForPath

EncryptionZone getEZForPath(java.lang.String src)
                     throws java.io.IOException

Get the encryption zone for a path.

Throws:

java.io.IOException

listEncryptionZones

org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<EncryptionZone> listEncryptionZones(long prevId)
                                                                                       throws java.io.IOException

Used to implement cursor-based batched listing of EncryptionZones.

Parameters:

prevId - ID of the last item in the previous batch. If there is no previous batch, a negative value can be used.

Returns:

Batch of encryption zones.

Throws:

java.io.IOException

reencryptEncryptionZone

void reencryptEncryptionZone(java.lang.String zone,
                             HdfsConstants.ReencryptAction action)
                      throws java.io.IOException

Used to implement re-encryption of encryption zones.

Parameters:

zone - the encryption zone to re-encrypt.

action - the action for the re-encryption.

Throws:

java.io.IOException

listReencryptionStatus

org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<ZoneReencryptionStatus> listReencryptionStatus(long prevId)
                                                                                                  throws java.io.IOException

Used to implement cursor-based batched listing of ZoneReencryptionStatuss.

Parameters:

prevId - ID of the last item in the previous batch. If there is no previous batch, a negative value can be used.

Returns:

Batch of encryption zones.

Throws:

java.io.IOException

setXAttr

void setXAttr(java.lang.String src,
              XAttr xAttr,
              java.util.EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag)
       throws java.io.IOException

Set xattr of a file or directory. The name must be prefixed with the namespace followed by ".". For example, "user.attr".

Refer to the HDFS extended attributes user documentation for details.

Parameters:

src - file or directory

xAttr - XAttr to set

flag - set flag

Throws:

java.io.IOException

getXAttrs

java.util.List<XAttr> getXAttrs(java.lang.String src,
                                java.util.List<XAttr> xAttrs)
                         throws java.io.IOException

Get xattrs of a file or directory. Values in xAttrs parameter are ignored. If xAttrs is null or empty, this is the same as getting all xattrs of the file or directory. Only those xattrs for which the logged-in user has permissions to view are returned.

Refer to the HDFS extended attributes user documentation for details.

Parameters:

src - file or directory

xAttrs - xAttrs to get

Returns:

XAttr list

Throws:

java.io.IOException

listXAttrs

java.util.List<XAttr> listXAttrs(java.lang.String src)
                          throws java.io.IOException

List the xattrs names for a file or directory. Only the xattr names for which the logged in user has the permissions to access will be returned.

Refer to the HDFS extended attributes user documentation for details.

Parameters:

src - file or directory

Returns:

XAttr list

Throws:

java.io.IOException

removeXAttr

void removeXAttr(java.lang.String src,
                 XAttr xAttr)
          throws java.io.IOException

Remove xattr of a file or directory.Value in xAttr parameter is ignored. The name must be prefixed with the namespace followed by ".". For example, "user.attr".

Refer to the HDFS extended attributes user documentation for details.

Parameters:

src - file or directory

xAttr - XAttr to remove

Throws:

java.io.IOException

checkAccess

void checkAccess(java.lang.String path,
                 org.apache.hadoop.fs.permission.FsAction mode)
          throws java.io.IOException

Checks if the user can access a path. The mode specifies which access checks to perform. If the requested permissions are granted, then the method returns normally. If access is denied, then the method throws an AccessControlException. In general, applications should avoid using this method, due to the risk of time-of-check/time-of-use race conditions. The permissions on a file may change immediately after the access call returns.

Parameters:

path - Path to check

mode - type of access to check

Throws:

org.apache.hadoop.security.AccessControlException - if access is denied

java.io.FileNotFoundException - if the path does not exist

java.io.IOException - see specific implementation

getCurrentEditLogTxid

long getCurrentEditLogTxid()
                    throws java.io.IOException

Get the highest txid the NameNode knows has been written to the edit log, or -1 if the NameNode's edit log is not yet open for write. Used as the starting point for the inotify event stream.

Throws:

java.io.IOException

getEditsFromTxid

EventBatchList getEditsFromTxid(long txid)
                         throws java.io.IOException

Get an ordered list of batches of events corresponding to the edit log transactions for txids equal to or greater than txid.

Throws:

java.io.IOException

setErasureCodingPolicy

void setErasureCodingPolicy(java.lang.String src,
                            java.lang.String ecPolicyName)
                     throws java.io.IOException

Set an erasure coding policy on a specified path.

Parameters:

src - The path to set policy on.

ecPolicyName - The erasure coding policy name.

Throws:

java.io.IOException

addErasureCodingPolicies

AddErasureCodingPolicyResponse[] addErasureCodingPolicies(ErasureCodingPolicy[] policies)
                                                   throws java.io.IOException

Add Erasure coding policies to HDFS. For each policy input, schema and cellSize are musts, name and id are ignored. They will be automatically created and assigned by Namenode once the policy is successfully added, and will be returned in the response.

Parameters:

policies - The user defined ec policy list to add.

Returns:

Return the response list of adding operations.

Throws:

java.io.IOException

removeErasureCodingPolicy

void removeErasureCodingPolicy(java.lang.String ecPolicyName)
                        throws java.io.IOException

Remove erasure coding policy.

Parameters:

ecPolicyName - The name of the policy to be removed.

Throws:

java.io.IOException

enableErasureCodingPolicy

void enableErasureCodingPolicy(java.lang.String ecPolicyName)
                        throws java.io.IOException

Enable erasure coding policy.

Parameters:

ecPolicyName - The name of the policy to be enabled.

Throws:

java.io.IOException

disableErasureCodingPolicy

void disableErasureCodingPolicy(java.lang.String ecPolicyName)
                         throws java.io.IOException

Disable erasure coding policy.

Parameters:

ecPolicyName - The name of the policy to be disabled.

Throws:

java.io.IOException

getErasureCodingPolicies

ErasureCodingPolicyInfo[] getErasureCodingPolicies()
                                            throws java.io.IOException

Get the erasure coding policies loaded in Namenode, excluding REPLICATION policy.

Throws:

java.io.IOException

getErasureCodingCodecs

java.util.Map<java.lang.String,java.lang.String> getErasureCodingCodecs()
                                                                       throws java.io.IOException

Get the erasure coding codecs loaded in Namenode.

Throws:

java.io.IOException

getErasureCodingPolicy

ErasureCodingPolicy getErasureCodingPolicy(java.lang.String src)
                                    throws java.io.IOException

Get the information about the EC policy for the path. Null will be returned if directory or file has REPLICATION policy.

Parameters:

src - path to get the info for

Throws:

java.io.IOException

unsetErasureCodingPolicy

void unsetErasureCodingPolicy(java.lang.String src)
                       throws java.io.IOException

Unset erasure coding policy from a specified path.

Parameters:

src - The path to unset policy.

Throws:

java.io.IOException

getECTopologyResultForPolicies

ECTopologyVerifierResult getECTopologyResultForPolicies(java.lang.String... policyNames)
                                                 throws java.io.IOException

Verifies if the given policies are supported in the given cluster setup. If not policy is specified checks for all enabled policies.

Parameters:

policyNames - name of policies.

Returns:

the result if the given policies are supported in the cluster setup

Throws:

java.io.IOException

getQuotaUsage

org.apache.hadoop.fs.QuotaUsage getQuotaUsage(java.lang.String path)
                                       throws java.io.IOException

Get QuotaUsage rooted at the specified directory. Note: due to HDFS-6763, standby/observer doesn't keep up-to-date info about quota usage, and thus even though this is ReadOnly, it can only be directed to the active namenode.

Parameters:

path - The string representation of the path

Throws:

org.apache.hadoop.security.AccessControlException - permission denied

java.io.FileNotFoundException - file path is not found

org.apache.hadoop.fs.UnresolvedLinkException - if path contains a symlink.

java.io.IOException - If an I/O error occurred

listOpenFiles

@Deprecated
org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<OpenFileEntry> listOpenFiles(long prevId)
                                                                                throws java.io.IOException

Deprecated.

List open files in the system in batches. INode id is the cursor and the open files returned in a batch will have their INode ids greater than the cursor INode id. Open files can only be requested by super user and the the list across batches are not atomic.

Parameters:

prevId - the cursor INode id.

Throws:

java.io.IOException

listOpenFiles

org.apache.hadoop.fs.BatchedRemoteIterator.BatchedEntries<OpenFileEntry> listOpenFiles(long prevId,
                                                                                       java.util.EnumSet<OpenFilesIterator.OpenFilesType> openFilesTypes,
                                                                                       java.lang.String path)
                                                                                throws java.io.IOException

List open files in the system in batches. INode id is the cursor and the open files returned in a batch will have their INode ids greater than the cursor INode id. Open files can only be requested by super user and the the list across batches are not atomic.

Parameters:

prevId - the cursor INode id.

openFilesTypes - types to filter the open files.

path - path to filter the open files.

Throws:

java.io.IOException

getHAServiceState

org.apache.hadoop.ha.HAServiceProtocol.HAServiceState getHAServiceState()
                                                                 throws java.io.IOException

Get HA service state of the server.

Returns:

server HA state

Throws:

java.io.IOException

msync

void msync()
    throws java.io.IOException

Called by client to wait until the server has reached the state id of the client. The client and server state id are given by client side and server side alignment context respectively. This can be a blocking call.

Throws:

java.io.IOException

satisfyStoragePolicy

void satisfyStoragePolicy(java.lang.String path)
                   throws java.io.IOException

Satisfy the storage policy for a file/directory.

Parameters:

path - Path of an existing file/directory.

Throws:

org.apache.hadoop.security.AccessControlException - If access is denied.

org.apache.hadoop.fs.UnresolvedLinkException - if src contains a symlink.

java.io.FileNotFoundException - If file/dir src is not found.

SafeModeException - append not allowed in safemode.

java.io.IOException

getSlowDatanodeReport

DatanodeInfo[] getSlowDatanodeReport()
                              throws java.io.IOException

Get report on all of the slow Datanodes. Slow running datanodes are identified based on the Outlier detection algorithm, if slow peer tracking is enabled for the DFS cluster.

Returns:

Datanode report for slow running datanodes.

Throws:

java.io.IOException - If an I/O error occurs.

getEnclosingRoot

org.apache.hadoop.fs.Path getEnclosingRoot(java.lang.String src)
                                    throws java.io.IOException

Get the enclosing root for a path.

Throws:

java.io.IOException