Package org.apache.hadoop.fs.azurebfs

Class AzureBlobFileSystem

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.fs.BulkDeleteSource, org.apache.hadoop.fs.PathCapabilities, org.apache.hadoop.fs.statistics.IOStatisticsSource, org.apache.hadoop.security.token.DelegationTokenIssuer

Direct Known Subclasses:

SecureAzureBlobFileSystem

@Evolving
public class AzureBlobFileSystem
extends org.apache.hadoop.fs.FileSystem
implements org.apache.hadoop.fs.statistics.IOStatisticsSource

A FileSystem for reading and writing files stored on Windows Azure

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	AzureBlobFileSystem.ResilientCommitByRenameImpl	Resilient commit support.

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.FileSystem.DirectoryEntries, org.apache.hadoop.fs.FileSystem.DirListingIterator<T extends org.apache.hadoop.fs.FileStatus>, org.apache.hadoop.fs.FileSystem.Statistics

Field Summary

Fields

Modifier and Type	Field	Description
static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.fs.FileSystem

DEFAULT_FS, FS_DEFAULT_NAME_KEY, SHUTDOWN_HOOK_PRIORITY, statistics, TRASH_PREFIX, USER_HOME_PREFIX

Fields inherited from interface org.apache.hadoop.security.token.DelegationTokenIssuer

TOKEN_LOG

Constructor Summary

Constructors

Constructor	Description
AzureBlobFileSystem()

Method Summary

Modifier and Type	Method	Description
void	access(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsAction mode)	Checks if the user can access a path.
org.apache.hadoop.fs.FSDataOutputStream	append(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress)
void	breakLease(org.apache.hadoop.fs.Path f)	Break the current lease on an ABFS file if it exists.
static void	checkException(org.apache.hadoop.fs.Path path, AzureBlobFileSystemException exception, AzureServiceErrorCode... allowedErrorCodesList)	Given a path and exception, choose which IOException subclass to create.
void	close()
org.apache.hadoop.fs.FSDataOutputStream	create(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
org.apache.hadoop.fs.FSDataOutputStream	createNonRecursive(org.apache.hadoop.fs.Path f, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
org.apache.hadoop.fs.FSDataOutputStream	createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
org.apache.hadoop.fs.FSDataOutputStream	createNonRecursive(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission, java.util.EnumSet<org.apache.hadoop.fs.CreateFlag> flags, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
ResilientCommitByRename	createResilientCommitSupport(org.apache.hadoop.fs.Path path)	Private method to create resilient commit support.
boolean	delete(org.apache.hadoop.fs.Path f, boolean recursive)
boolean	exists(org.apache.hadoop.fs.Path f)	Incrementing exists() calls from superclass for statistic collection.
protected void	finalize()
AzureBlobFileSystemStore	getAbfsStore()
org.apache.hadoop.fs.permission.AclStatus	getAclStatus(org.apache.hadoop.fs.Path path)	Gets the ACL of a file or directory.
java.lang.String	getCanonicalServiceName()	If Delegation tokens are enabled, the canonical service name of this filesystem is the filesystem URI.
org.apache.hadoop.security.token.Token<?>	getDelegationToken(java.lang.String renewer)	Get a delegation token from remote service endpoint if 'fs.azure.enable.kerberos.support' is set to 'true', and 'fs.azure.enable.delegation.token' is set to 'true'.
org.apache.hadoop.fs.BlockLocation[]	getFileBlockLocations(org.apache.hadoop.fs.FileStatus file, long start, long len)	Return an array containing hostnames, offset and size of portions of the given file.
org.apache.hadoop.fs.FileStatus	getFileStatus(org.apache.hadoop.fs.Path f)
org.apache.hadoop.fs.Path	getHomeDirectory()
org.apache.hadoop.fs.statistics.IOStatistics	getIOStatistics()	Getter for IOStatistic instance in AzureBlobFilesystem.
java.lang.String	getOwnerUser()	Get the username of the FS.
java.lang.String	getOwnerUserPrimaryGroup()	Get the group name of the owner of the FS.
java.lang.String	getScheme()
java.net.URI	getUri()
org.apache.hadoop.fs.Path	getWorkingDirectory()
byte[]	getXAttr(org.apache.hadoop.fs.Path path, java.lang.String name)	Get the value of an attribute for a non-root path.
boolean	hasPathCapability(org.apache.hadoop.fs.Path path, java.lang.String capability)
void	initialize(java.net.URI uri, org.apache.hadoop.conf.Configuration configuration)
boolean	isSecureScheme()
protected org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus>	listLocatedStatus(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter filter)	Incremental listing of located status entries, preserving etags.
org.apache.hadoop.fs.FileStatus[]	listStatus(org.apache.hadoop.fs.Path f)
org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus>	listStatusIterator(org.apache.hadoop.fs.Path path)
org.apache.hadoop.fs.Path	makeQualified(org.apache.hadoop.fs.Path path)	Qualify a path to one which uses this FileSystem and, if relative, made absolute.
boolean	mkdirs(org.apache.hadoop.fs.Path f, org.apache.hadoop.fs.permission.FsPermission permission)
void	modifyAclEntries(org.apache.hadoop.fs.Path path, java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)	Modifies ACL entries of files and directories.
org.apache.hadoop.fs.FSDataInputStream	open(org.apache.hadoop.fs.Path path, int bufferSize)
protected java.util.concurrent.CompletableFuture<org.apache.hadoop.fs.FSDataInputStream>	openFileWithOptions(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.impl.OpenFileParameters parameters)	Takes config and other options through OpenFileParameters.
void	registerListener(Listener listener1)
void	removeAcl(org.apache.hadoop.fs.Path path)	Removes all but the base ACL entries of files and directories.
void	removeAclEntries(org.apache.hadoop.fs.Path path, java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)	Removes ACL entries from files and directories.
void	removeDefaultAcl(org.apache.hadoop.fs.Path path)	Removes all default ACL entries from files and directories.
boolean	rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)
void	setAcl(org.apache.hadoop.fs.Path path, java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)	Fully replaces ACL of files and directories, discarding all existing entries.
void	setOwner(org.apache.hadoop.fs.Path path, java.lang.String owner, java.lang.String group)	Set owner of a path (i.e.
void	setPermission(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsPermission permission)	Set permission of a path.
void	setWorkingDirectory(org.apache.hadoop.fs.Path newDir)
void	setXAttr(org.apache.hadoop.fs.Path path, java.lang.String name, byte[] value, java.util.EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag)	Set the value of an attribute for a non-root path.
java.lang.String	toString()

Methods inherited from class org.apache.hadoop.fs.FileSystem

append, append, append, append, appendFile, areSymlinksEnabled, cancelDeleteOnExit, canonicalizeUri, checkPath, clearStatistics, closeAll, closeAllForUGI, completeLocalOutput, concat, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createBulkDelete, createDataInputStreamBuilder, createDataInputStreamBuilder, createDataOutputStreamBuilder, createFid, createFile, createMultipartUploader, createNewFile, createPathHandle, createPathId, createSnapshot, createSnapshot, createSymlink, delete, deleteFid, deleteOnExit, deleteSnapshot, enableSymlinks, fixRelativePart, get, get, get, getAdditionalTokenIssuers, getAllStatistics, getAllStoragePolicies, getBlockSize, getCanonicalUri, getChildFileSystems, getContentSummary, getDefaultBlockSize, getDefaultBlockSize, getDefaultPort, getDefaultReplication, getDefaultReplication, getDefaultUri, getEnclosingRoot, getFileBlockLocations, getFileChecksum, getFileChecksum, getFileLinkStatus, getFileSystemClass, getFSofPath, getGlobalStorageStatistics, getInitialWorkingDirectory, getJobTrackerAddrs, getLength, getLinkTarget, getLocal, getName, getNamed, getPathHandle, getQuotaUsage, getReplication, getServerDefaults, getServerDefaults, getStatistics, getStatistics, getStatus, getStatus, getStoragePolicy, getStorageStatistics, getTrashRoot, getTrashRoots, getUsed, getUsed, getXAttrs, getXAttrs, getZkConnectString, globStatus, globStatus, isDirectory, isFile, listCorruptFileBlocks, listFiles, listLocatedStatus, listStatus, listStatus, listStatus, listStatusBatch, listXAttrs, mkdirs, mkdirs, mkdirsFid, mkdirsFid, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, msync, newInstance, newInstance, newInstance, newInstanceLocal, open, open, open, openFid, openFid, openFid2, openFile, openFile, openFileWithOptions, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, removeXAttr, rename, renameSnapshot, resolveLink, resolvePath, satisfyStoragePolicy, setDefaultUri, setDefaultUri, setOwnerFid, setQuota, setQuotaByStorageType, setReplication, setStoragePolicy, setTimes, setVerifyChecksum, setWriteChecksum, setXAttr, startLocalOutput, supportsSymlinks, truncate, unsetStoragePolicy

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.security.token.DelegationTokenIssuer

addDelegationTokens

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

AzureBlobFileSystem

public AzureBlobFileSystem()

Method Detail

initialize

public void initialize(java.net.URI uri,
                       org.apache.hadoop.conf.Configuration configuration)
                throws java.io.IOException

Overrides:

initialize in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

isSecureScheme

public boolean isSecureScheme()

getUri

public java.net.URI getUri()

Specified by:

getUri in class org.apache.hadoop.fs.FileSystem

registerListener

public void registerListener(Listener listener1)

open

public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.Path path,
                                                   int bufferSize)
                                            throws java.io.IOException

Specified by:

open in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

openFileWithOptions

protected java.util.concurrent.CompletableFuture<org.apache.hadoop.fs.FSDataInputStream> openFileWithOptions(org.apache.hadoop.fs.Path path,
                                                                                                             org.apache.hadoop.fs.impl.OpenFileParameters parameters)
                                                                                                      throws java.io.IOException

Takes config and other options through OpenFileParameters. Ensure that FileStatus entered is up-to-date, as it will be used to create the InputStream (with info such as contentLength, eTag)

Overrides:

openFileWithOptions in class org.apache.hadoop.fs.FileSystem

Parameters:

path - The location of file to be opened

parameters - OpenFileParameters instance; can hold FileStatus, Configuration, bufferSize and mandatoryKeys

Throws:

java.io.IOException

create

public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path f,
                                                      org.apache.hadoop.fs.permission.FsPermission permission,
                                                      boolean overwrite,
                                                      int bufferSize,
                                                      short replication,
                                                      long blockSize,
                                                      org.apache.hadoop.util.Progressable progress)
                                               throws java.io.IOException

Specified by:

create in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

createNonRecursive

public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f,
                                                                  org.apache.hadoop.fs.permission.FsPermission permission,
                                                                  boolean overwrite,
                                                                  int bufferSize,
                                                                  short replication,
                                                                  long blockSize,
                                                                  org.apache.hadoop.util.Progressable progress)
                                                           throws java.io.IOException

Overrides:

createNonRecursive in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

createNonRecursive

public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f,
                                                                  org.apache.hadoop.fs.permission.FsPermission permission,
                                                                  java.util.EnumSet<org.apache.hadoop.fs.CreateFlag> flags,
                                                                  int bufferSize,
                                                                  short replication,
                                                                  long blockSize,
                                                                  org.apache.hadoop.util.Progressable progress)
                                                           throws java.io.IOException

Overrides:

createNonRecursive in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

createNonRecursive

public org.apache.hadoop.fs.FSDataOutputStream createNonRecursive(org.apache.hadoop.fs.Path f,
                                                                  boolean overwrite,
                                                                  int bufferSize,
                                                                  short replication,
                                                                  long blockSize,
                                                                  org.apache.hadoop.util.Progressable progress)
                                                           throws java.io.IOException

Overrides:

createNonRecursive in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

append

public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f,
                                                      int bufferSize,
                                                      org.apache.hadoop.util.Progressable progress)
                                               throws java.io.IOException

Specified by:

append in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

rename

public boolean rename(org.apache.hadoop.fs.Path src,
                      org.apache.hadoop.fs.Path dst)
               throws java.io.IOException

Specified by:

rename in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

createResilientCommitSupport

@Private
public ResilientCommitByRename createResilientCommitSupport(org.apache.hadoop.fs.Path path)
                                                     throws java.io.IOException

Private method to create resilient commit support.

Parameters:

path - destination path

Returns:

a new instance

Throws:

java.io.IOException - problem probing store capabilities

java.lang.UnsupportedOperationException - if the store lacks this support

delete

public boolean delete(org.apache.hadoop.fs.Path f,
                      boolean recursive)
               throws java.io.IOException

Specified by:

delete in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

listStatus

public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path f)
                                             throws java.io.IOException

Specified by:

listStatus in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

mkdirs

public boolean mkdirs(org.apache.hadoop.fs.Path f,
                      org.apache.hadoop.fs.permission.FsPermission permission)
               throws java.io.IOException

Specified by:

mkdirs in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Overrides:

close in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

getFileStatus

public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.Path f)
                                              throws java.io.IOException

Specified by:

getFileStatus in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

breakLease

public void breakLease(org.apache.hadoop.fs.Path f)
                throws java.io.IOException

Break the current lease on an ABFS file if it exists. A lease that is broken cannot be renewed. A new lease may be obtained on the file immediately.

Parameters:

f - file name

Throws:

java.io.IOException - on any exception while breaking the lease

makeQualified

public org.apache.hadoop.fs.Path makeQualified(org.apache.hadoop.fs.Path path)

Qualify a path to one which uses this FileSystem and, if relative, made absolute.

Overrides:

makeQualified in class org.apache.hadoop.fs.FileSystem

Parameters:

path - to qualify.

Returns:

this path if it contains a scheme and authority and is absolute, or a new path that includes a path and authority and is fully qualified

Throws:

java.lang.IllegalArgumentException - if the path has a schema/URI different from this FileSystem.

See Also:

Path.makeQualified(URI, Path)

getWorkingDirectory

public org.apache.hadoop.fs.Path getWorkingDirectory()

Specified by:

getWorkingDirectory in class org.apache.hadoop.fs.FileSystem

setWorkingDirectory

public void setWorkingDirectory(org.apache.hadoop.fs.Path newDir)

Specified by:

setWorkingDirectory in class org.apache.hadoop.fs.FileSystem

getScheme

public java.lang.String getScheme()

Overrides:

getScheme in class org.apache.hadoop.fs.FileSystem

getHomeDirectory

public org.apache.hadoop.fs.Path getHomeDirectory()

Overrides:

getHomeDirectory in class org.apache.hadoop.fs.FileSystem

getFileBlockLocations

public org.apache.hadoop.fs.BlockLocation[] getFileBlockLocations(org.apache.hadoop.fs.FileStatus file,
                                                                  long start,
                                                                  long len)

Return an array containing hostnames, offset and size of portions of the given file. For ABFS we'll just lie and give fake hosts to make sure we get many splits in MR jobs.

Overrides:

getFileBlockLocations in class org.apache.hadoop.fs.FileSystem

finalize

protected void finalize()
                 throws java.lang.Throwable

Overrides:

finalize in class java.lang.Object

Throws:

java.lang.Throwable

getOwnerUser

public java.lang.String getOwnerUser()

Get the username of the FS.

Returns:

the short name of the user who instantiated the FS

getOwnerUserPrimaryGroup

public java.lang.String getOwnerUserPrimaryGroup()

Get the group name of the owner of the FS.

Returns:

primary group name

setOwner

public void setOwner(org.apache.hadoop.fs.Path path,
                     java.lang.String owner,
                     java.lang.String group)
              throws java.io.IOException

Set owner of a path (i.e. a file or a directory). The parameters owner and group cannot both be null.

Overrides:

setOwner in class org.apache.hadoop.fs.FileSystem

Parameters:

path - The path

owner - If it is null, the original username remains unchanged.

group - If it is null, the original groupname remains unchanged.

Throws:

java.io.IOException

setXAttr

public void setXAttr(org.apache.hadoop.fs.Path path,
                     java.lang.String name,
                     byte[] value,
                     java.util.EnumSet<org.apache.hadoop.fs.XAttrSetFlag> flag)
              throws java.io.IOException

Set the value of an attribute for a non-root path.

Overrides:

setXAttr in class org.apache.hadoop.fs.FileSystem

Parameters:

path - The path on which to set the attribute

name - The attribute to set

value - The byte value of the attribute to set (encoded in latin-1)

flag - The mode in which to set the attribute

Throws:

java.io.IOException - If there was an issue setting the attribute on Azure

java.lang.IllegalArgumentException - If name is null or empty or if value is null

getXAttr

public byte[] getXAttr(org.apache.hadoop.fs.Path path,
                       java.lang.String name)
                throws java.io.IOException

Get the value of an attribute for a non-root path.

Overrides:

getXAttr in class org.apache.hadoop.fs.FileSystem

Parameters:

path - The path on which to get the attribute

name - The attribute to get

Returns:

The bytes of the attribute's value (encoded in latin-1) or null if the attribute does not exist

Throws:

java.io.IOException - If there was an issue getting the attribute from Azure

java.lang.IllegalArgumentException - If name is null or empty

setPermission

public void setPermission(org.apache.hadoop.fs.Path path,
                          org.apache.hadoop.fs.permission.FsPermission permission)
                   throws java.io.IOException

Set permission of a path.

Overrides:

setPermission in class org.apache.hadoop.fs.FileSystem

Parameters:

path - The path

permission - Access permission

Throws:

java.io.IOException

modifyAclEntries

public void modifyAclEntries(org.apache.hadoop.fs.Path path,
                             java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
                      throws java.io.IOException

Modifies ACL entries of files and directories. This method can add new ACL entries or modify the permissions on existing ACL entries. All existing ACL entries that are not specified in this call are retained without changes. (Modifications are merged into the current ACL.)

Overrides:

modifyAclEntries in class org.apache.hadoop.fs.FileSystem

Parameters:

path - Path to modify

aclSpec - List of AbfsAclEntry describing modifications

Throws:

java.io.IOException - if an ACL could not be modified

removeAclEntries

public void removeAclEntries(org.apache.hadoop.fs.Path path,
                             java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
                      throws java.io.IOException

Removes ACL entries from files and directories. Other ACL entries are retained.

Overrides:

removeAclEntries in class org.apache.hadoop.fs.FileSystem

Parameters:

path - Path to modify

aclSpec - List of AclEntry describing entries to remove

Throws:

java.io.IOException - if an ACL could not be modified

removeDefaultAcl

public void removeDefaultAcl(org.apache.hadoop.fs.Path path)
                      throws java.io.IOException

Removes all default ACL entries from files and directories.

Overrides:

removeDefaultAcl in class org.apache.hadoop.fs.FileSystem

Parameters:

path - Path to modify

Throws:

java.io.IOException - if an ACL could not be modified

removeAcl

public void removeAcl(org.apache.hadoop.fs.Path path)
               throws java.io.IOException

Removes all but the base ACL entries of files and directories. The entries for user, group, and others are retained for compatibility with permission bits.

Overrides:

removeAcl in class org.apache.hadoop.fs.FileSystem

Parameters:

path - Path to modify

Throws:

java.io.IOException - if an ACL could not be removed

setAcl

public void setAcl(org.apache.hadoop.fs.Path path,
                   java.util.List<org.apache.hadoop.fs.permission.AclEntry> aclSpec)
            throws java.io.IOException

Fully replaces ACL of files and directories, discarding all existing entries.

Overrides:

setAcl in class org.apache.hadoop.fs.FileSystem

Parameters:

path - Path to modify

aclSpec - List of AclEntry describing modifications, must include entries for user, group, and others for compatibility with permission bits.

Throws:

java.io.IOException - if an ACL could not be modified

getAclStatus

public org.apache.hadoop.fs.permission.AclStatus getAclStatus(org.apache.hadoop.fs.Path path)
                                                       throws java.io.IOException

Gets the ACL of a file or directory.

Overrides:

getAclStatus in class org.apache.hadoop.fs.FileSystem

Parameters:

path - Path to get

Returns:

AbfsAclStatus describing the ACL of the file or directory

Throws:

java.io.IOException - if an ACL could not be read

access

public void access(org.apache.hadoop.fs.Path path,
                   org.apache.hadoop.fs.permission.FsAction mode)
            throws java.io.IOException

Checks if the user can access a path. The mode specifies which access checks to perform. If the requested permissions are granted, then the method returns normally. If access is denied, then the method throws an AccessControlException.

Overrides:

access in class org.apache.hadoop.fs.FileSystem

Parameters:

path - Path to check

mode - type of access to check

Throws:

org.apache.hadoop.security.AccessControlException - if access is denied

java.io.FileNotFoundException - if the path does not exist

java.io.IOException - see specific implementation

exists

public boolean exists(org.apache.hadoop.fs.Path f)
               throws java.io.IOException

Incrementing exists() calls from superclass for statistic collection.

Overrides:

exists in class org.apache.hadoop.fs.FileSystem

Parameters:

f - source path.

Returns:

true if the path exists.

Throws:

java.io.IOException - if some issue in checking path.

listStatusIterator

public org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.FileStatus> listStatusIterator(org.apache.hadoop.fs.Path path)
                                                                                        throws java.io.IOException

Overrides:

listStatusIterator in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

listLocatedStatus

protected org.apache.hadoop.fs.RemoteIterator<org.apache.hadoop.fs.LocatedFileStatus> listLocatedStatus(org.apache.hadoop.fs.Path path,
                                                                                                        org.apache.hadoop.fs.PathFilter filter)
                                                                                                 throws java.io.FileNotFoundException,
                                                                                                        java.io.IOException

Incremental listing of located status entries, preserving etags.

Overrides:

listLocatedStatus in class org.apache.hadoop.fs.FileSystem

Parameters:

path - path to list

filter - a path filter

Returns:

iterator of results.

Throws:

java.io.FileNotFoundException - source path not found.

java.io.IOException - other values.

checkException

@VisibleForTesting
public static void checkException(org.apache.hadoop.fs.Path path,
                                  AzureBlobFileSystemException exception,
                                  AzureServiceErrorCode... allowedErrorCodesList)
                           throws java.io.IOException

Given a path and exception, choose which IOException subclass to create. Will return if and only iff the error code is in the list of allowed error codes.

Parameters:

path - path of operation triggering exception; may be null

exception - the exception caught

allowedErrorCodesList - varargs list of error codes.

Throws:

java.io.IOException - if the exception error code is not on the allowed list.

getDelegationToken

public org.apache.hadoop.security.token.Token<?> getDelegationToken(java.lang.String renewer)
                                                             throws java.io.IOException

Get a delegation token from remote service endpoint if 'fs.azure.enable.kerberos.support' is set to 'true', and 'fs.azure.enable.delegation.token' is set to 'true'.

Specified by:

getDelegationToken in interface org.apache.hadoop.security.token.DelegationTokenIssuer

Overrides:

getDelegationToken in class org.apache.hadoop.fs.FileSystem

Parameters:

renewer - the account name that is allowed to renew the token.

Returns:

delegation token

Throws:

java.io.IOException - thrown when getting the current user.

getCanonicalServiceName

public java.lang.String getCanonicalServiceName()

If Delegation tokens are enabled, the canonical service name of this filesystem is the filesystem URI.

Specified by:

getCanonicalServiceName in interface org.apache.hadoop.security.token.DelegationTokenIssuer

Overrides:

getCanonicalServiceName in class org.apache.hadoop.fs.FileSystem

Returns:

either the filesystem URI as a string, or null.

getAbfsStore

@VisibleForTesting
public AzureBlobFileSystemStore getAbfsStore()

hasPathCapability

public boolean hasPathCapability(org.apache.hadoop.fs.Path path,
                                 java.lang.String capability)
                          throws java.io.IOException

Specified by:

hasPathCapability in interface org.apache.hadoop.fs.PathCapabilities

Overrides:

hasPathCapability in class org.apache.hadoop.fs.FileSystem

Throws:

java.io.IOException

getIOStatistics

public org.apache.hadoop.fs.statistics.IOStatistics getIOStatistics()

Getter for IOStatistic instance in AzureBlobFilesystem.

Specified by:

getIOStatistics in interface org.apache.hadoop.fs.statistics.IOStatisticsSource

Returns:

the IOStatistic instance from abfsCounters.