org.apache.hadoop.fs.s3

Class S3FileSystem

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.s3.S3FileSystem

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable

@InterfaceAudience.Public
@InterfaceStability.Stable
public class S3FileSystem
extends org.apache.hadoop.fs.FileSystem

A block-based FileSystem backed by Amazon S3.

See Also:

NativeS3FileSystem

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

org.apache.hadoop.fs.FileSystem.Statistics

Field Summary

Fields inherited from class org.apache.hadoop.fs.FileSystem

DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics

Constructor Summary

Constructors

Constructor and Description
S3FileSystem()
S3FileSystem(FileSystemStore store)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.fs.FSDataOutputStream	append(org.apache.hadoop.fs.Path f, int bufferSize, org.apache.hadoop.util.Progressable progress) This optional operation is not yet supported.
org.apache.hadoop.fs.FSDataOutputStream	create(org.apache.hadoop.fs.Path file, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)
boolean	delete(org.apache.hadoop.fs.Path path, boolean recursive)
String	getCanonicalServiceName()
long	getDefaultBlockSize()
org.apache.hadoop.fs.FileStatus	getFileStatus(org.apache.hadoop.fs.Path f) FileStatus for S3 file systems.
String	getScheme() Return the protocol scheme for the FileSystem.
URI	getUri()
org.apache.hadoop.fs.Path	getWorkingDirectory()
void	initialize(URI uri, org.apache.hadoop.conf.Configuration conf)
boolean	isFile(org.apache.hadoop.fs.Path path)
org.apache.hadoop.fs.FileStatus[]	listStatus(org.apache.hadoop.fs.Path f)
boolean	mkdirs(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.permission.FsPermission permission)
org.apache.hadoop.fs.FSDataInputStream	open(org.apache.hadoop.fs.Path path, int bufferSize)
boolean	rename(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)
void	setWorkingDirectory(org.apache.hadoop.fs.Path dir)

Methods inherited from class org.apache.hadoop.fs.FileSystem

access, addDelegationTokens, append, append, areSymlinksEnabled, cancelDeleteOnExit, canonicalizeUri, checkPath, clearStatistics, close, closeAll, closeAllForUGI, completeLocalOutput, concat, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createFid, createNewFile, createNonRecursive, createNonRecursive, createNonRecursive, createPathId, createSnapshot, createSnapshot, createSymlink, delete, deleteFid, deleteOnExit, deleteSnapshot, enableSymlinks, exists, fixRelativePart, get, get, get, getAclStatus, getAllStatistics, getBlockSize, getCanonicalUri, getChildFileSystems, getContentSummary, getDefaultBlockSize, getDefaultPort, getDefaultReplication, getDefaultReplication, getDefaultUri, getDelegationToken, getFileBlockLocations, getFileBlockLocations, getFileChecksum, getFileChecksum, getFileLinkStatus, getFileSystemClass, getFSofPath, getHomeDirectory, getInitialWorkingDirectory, getJobTrackerAddrs, getLength, getLinkTarget, getLocal, getName, getNamed, getReplication, getServerDefaults, getServerDefaults, getStatistics, getStatistics, getStatus, getStatus, getUsed, getXAttr, getXAttrs, getXAttrs, getZkConnectString, globStatus, globStatus, isDirectory, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, listStatusIterator, listXAttrs, makeQualified, mkdirs, mkdirs, mkdirsFid, mkdirsFid, modifyAclEntries, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, openFid, openFid, openFid2, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, rename, renameSnapshot, resolveLink, resolvePath, setAcl, setDefaultUri, setDefaultUri, setOwner, setOwnerFid, setPermission, setReplication, setTimes, setVerifyChecksum, setWriteChecksum, setXAttr, setXAttr, startLocalOutput, supportsSymlinks, truncate

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

S3FileSystem

public S3FileSystem()

S3FileSystem

public S3FileSystem(FileSystemStore store)

Method Detail

getScheme

public String getScheme()

Return the protocol scheme for the FileSystem.

Overrides:

getScheme in class org.apache.hadoop.fs.FileSystem

Returns:

s3

getUri

public URI getUri()

Specified by:

getUri in class org.apache.hadoop.fs.FileSystem

initialize

public void initialize(URI uri,
              org.apache.hadoop.conf.Configuration conf)
                throws IOException

Overrides:

initialize in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getWorkingDirectory

public org.apache.hadoop.fs.Path getWorkingDirectory()

Specified by:

getWorkingDirectory in class org.apache.hadoop.fs.FileSystem

setWorkingDirectory

public void setWorkingDirectory(org.apache.hadoop.fs.Path dir)

Specified by:

setWorkingDirectory in class org.apache.hadoop.fs.FileSystem

mkdirs

public boolean mkdirs(org.apache.hadoop.fs.Path path,
             org.apache.hadoop.fs.permission.FsPermission permission)
               throws IOException

Specified by:

mkdirs in class org.apache.hadoop.fs.FileSystem

Parameters:

permission - Currently ignored.

Throws:

IOException

isFile

public boolean isFile(org.apache.hadoop.fs.Path path)
               throws IOException

Overrides:

isFile in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

listStatus

public org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.fs.Path f)
                                             throws IOException

Specified by:

listStatus in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

append

public org.apache.hadoop.fs.FSDataOutputStream append(org.apache.hadoop.fs.Path f,
                                             int bufferSize,
                                             org.apache.hadoop.util.Progressable progress)
                                               throws IOException

This optional operation is not yet supported.

Specified by:

append in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

create

public org.apache.hadoop.fs.FSDataOutputStream create(org.apache.hadoop.fs.Path file,
                                             org.apache.hadoop.fs.permission.FsPermission permission,
                                             boolean overwrite,
                                             int bufferSize,
                                             short replication,
                                             long blockSize,
                                             org.apache.hadoop.util.Progressable progress)
                                               throws IOException

Specified by:

create in class org.apache.hadoop.fs.FileSystem

Parameters:

permission - Currently ignored.

Throws:

IOException

open

public org.apache.hadoop.fs.FSDataInputStream open(org.apache.hadoop.fs.Path path,
                                          int bufferSize)
                                            throws IOException

Specified by:

open in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

rename

public boolean rename(org.apache.hadoop.fs.Path src,
             org.apache.hadoop.fs.Path dst)
               throws IOException

Specified by:

rename in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

delete

public boolean delete(org.apache.hadoop.fs.Path path,
             boolean recursive)
               throws IOException

Specified by:

delete in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getFileStatus

public org.apache.hadoop.fs.FileStatus getFileStatus(org.apache.hadoop.fs.Path f)
                                              throws IOException

FileStatus for S3 file systems.

Specified by:

getFileStatus in class org.apache.hadoop.fs.FileSystem

Throws:

IOException

getDefaultBlockSize

public long getDefaultBlockSize()

Overrides:

getDefaultBlockSize in class org.apache.hadoop.fs.FileSystem

getCanonicalServiceName

public String getCanonicalServiceName()

Overrides:

getCanonicalServiceName in class org.apache.hadoop.fs.FileSystem