Package org.apache.hadoop.fs

Class FileUtil

java.lang.Object

org.apache.hadoop.fs.FileUtil

@Public
@Evolving
public class FileUtil
extends java.lang.Object

A collection of file-processing util methods

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	FileUtil.HardLink	Deprecated. Use HardLink

Field Summary

Fields

Modifier and Type	Field	Description
static int	SYMLINK_NO_PRIVILEGE

Constructor Summary

Constructors

Constructor	Description
FileUtil()

Method Summary

Modifier and Type	Method	Description
static boolean	canExecute(java.io.File f)	Platform independent implementation for File.canExecute()
static boolean	canRead(java.io.File f)	Platform independent implementation for File.canRead()
static boolean	canWrite(java.io.File f)	Platform independent implementation for File.canWrite()
static PathData	checkExistItem(PathData item)
static PathData	checkItemForSymlink(PathData item)
static PathData	checkPathForSymlink(Path path, Configuration conf)
static int	chmod(java.lang.String filename, java.lang.String perm)	Change the permissions on a filename.
static int	chmod(java.lang.String filename, java.lang.String perm, boolean recursive)	Change the permissions on a file / directory, recursively, if needed.
static boolean	compareFs(FileSystem srcFs, FileSystem destFs)
static boolean	copy(java.io.File src, FileSystem dstFS, Path dst, boolean deleteSource, Configuration conf)	Copy local files to a FileSystem.
static boolean	copy(FileSystem srcFS, FileStatus srcStatus, FileSystem dstFS, Path dst, boolean deleteSource, boolean overwrite, Configuration conf)	Copy a file/directory tree within/between filesystems.
static boolean	copy(FileSystem srcFS, Path[] srcs, FileSystem dstFS, Path dst, boolean deleteSource, boolean overwrite, Configuration conf)
static boolean	copy(FileSystem srcFS, Path src, java.io.File dst, boolean deleteSource, Configuration conf)	Copy FileSystem files to local files.
static boolean	copy(FileSystem srcFS, Path src, FileSystem dstFS, Path dst, boolean deleteSource, boolean overwrite, Configuration conf)	Copy files between FileSystems.
static boolean	copy(FileSystem srcFS, Path src, FileSystem dstFS, Path dst, boolean deleteSource, Configuration conf)	Copy files between FileSystems.
static boolean	copyAsMove(FileSystem srcFS, FileStatus srcStatus, FileSystem dstFS, Path dst, boolean deleteSource, boolean overwrite, Configuration conf)	Copy files between FileSystems with preserved permissions, ownership, links and ACEs.
static boolean	copyMerge(FileSystem srcFS, Path srcDir, FileSystem dstFS, Path dstFile, boolean deleteSource, Configuration conf, java.lang.String addString)	Copy all files in a directory to one output file (merge).
static java.lang.String[]	createJarWithClassPath(java.lang.String inputClassPath, Path pwd, java.util.Map<java.lang.String,java.lang.String> callerEnv)
static java.lang.String[]	createJarWithClassPath(java.lang.String inputClassPath, Path pwd, Path targetDir, java.util.Map<java.lang.String,java.lang.String> callerEnv)	Create a jar file at the given path, containing a manifest with a classpath that references all specified entries.
static java.io.File	createLocalTempFile(java.io.File basefile, java.lang.String prefix, boolean isDeleteOnExit)	Create a tmp file for a base file.
static Path	fixSymlinkFileStatus(FileStatus stat)
static Path	fixSymlinkPath(PathData path)
static boolean	fullyDelete(java.io.File dir)	Delete a directory and all its contents.
static boolean	fullyDelete(java.io.File dir, boolean tryGrantPermissions)	Delete a directory and all its contents.
static void	fullyDelete(FileSystem fs, Path dir)	Deprecated. Use FileSystem.delete(Path, boolean)
static boolean	fullyDeleteContents(java.io.File dir)	Delete the contents of a directory, not the directory itself.
static boolean	fullyDeleteContents(java.io.File dir, boolean tryGrantPermissions)	Delete the contents of a directory, not the directory itself.
static void	fullyDeleteOnExit(java.io.File file)	Register all files recursively to be deleted on exit.
static long	getDU(java.io.File dir)	Takes an input dir and returns the du on that local directory.
static java.util.List<Path>	getJarsInDirectory(java.lang.String path)	Returns all jars that are in the directory.
static java.util.List<Path>	getJarsInDirectory(java.lang.String path, boolean useLocal)	Returns all jars that are in the directory.
static boolean	isRegularFile(java.io.File file)
static boolean	isRegularFile(java.io.File file, boolean allowLinks)	Check if the file is regular.
static java.lang.String[]	list(java.io.File dir)	A wrapper for File.list().
static java.io.File[]	listFiles(java.io.File dir)	A wrapper for File.listFiles().
static java.lang.String	makeSecureShellPath(java.io.File file)	Convert a os-native filename to a path that works for the shell and avoids script injection attacks.
static java.lang.String	makeShellPath(java.io.File file)	Convert a os-native filename to a path that works for the shell.
static java.lang.String	makeShellPath(java.io.File file, boolean makeCanonicalPath)	Convert a os-native filename to a path that works for the shell.
static java.lang.String	makeShellPath(java.lang.String filename)	Convert a os-native filename to a path that works for the shell.
static void	maybeIgnoreMissingDirectory(FileSystem fs, Path path, java.io.FileNotFoundException e)	Method to call after a FNFE has been raised on a treewalk, so as to decide whether to throw the exception (default), or, if the FS supports inconsistent directory listings, to log and ignore it.
static java.lang.String	readLink(java.io.File f)	Returns the target of the given symlink.
static void	rename(FileSystem srcFs, Path src, Path dst, Options.Rename... options)
static void	replaceFile(java.io.File src, java.io.File target)	Move the src file to the name specified by target.
static boolean	setExecutable(java.io.File f, boolean executable)	Platform independent implementation for File.setExecutable(boolean) File#setExecutable does not work as expected on Windows.
static void	setOwner(java.io.File file, java.lang.String username, java.lang.String groupname)	Set the ownership on a file / directory.
static void	setPermission(java.io.File f, FsPermission permission)	Set permissions to the required value.
static boolean	setReadable(java.io.File f, boolean readable)	Platform independent implementation for File.setReadable(boolean) File#setReadable does not work as expected on Windows.
static boolean	setWritable(java.io.File f, boolean writable)	Platform independent implementation for File.setWritable(boolean) File#setWritable does not work as expected on Windows.
static Path[]	stat2Paths(FileStatus[] stats)	convert an array of FileStatus to an array of Path
static Path[]	stat2Paths(FileStatus[] stats, Path path)	convert an array of FileStatus to an array of Path.
static int	symLink(java.lang.String target, java.lang.String linkname)	Create a soft link between a src and destination only on a local disk.
static void	unTar(java.io.File inFile, java.io.File untarDir)	Given a Tar File as input it will untar the file in a the untar directory passed as the second parameter This utility will untar ".tar" files and ".tar.gz","tgz" files.
static void	unTar(java.io.InputStream inputStream, java.io.File untarDir, boolean gzipped)	Given a Tar File as input it will untar the file in a the untar directory passed as the second parameter This utility will untar ".tar" files and ".tar.gz","tgz" files.
static void	unZip(java.io.File inFile, java.io.File unzipDir)	Given a File input it will unzip it in the unzip directory.
static void	unZip(java.io.InputStream inputStream, java.io.File toDir)	Given a stream input it will unzip the it in the unzip directory.
static FileContext	write(FileContext fileContext, Path path, byte[] bytes)	Writes bytes to a file.
static FileContext	write(FileContext fileContext, Path path, java.lang.CharSequence charseq)	Write a line of text to a file.
static FileContext	write(FileContext fs, Path path, java.lang.CharSequence charseq, java.nio.charset.Charset cs)	Write a line of text to a file.
static FileContext	write(FileContext fileContext, Path path, java.lang.Iterable<? extends java.lang.CharSequence> lines, java.nio.charset.Charset cs)	Write lines of text to a file.
static FileSystem	write(FileSystem fs, Path path, byte[] bytes)	Writes bytes to a file.
static FileSystem	write(FileSystem fs, Path path, java.lang.CharSequence charseq)	Write a line of text to a file.
static FileSystem	write(FileSystem fs, Path path, java.lang.CharSequence charseq, java.nio.charset.Charset cs)	Write a line of text to a file.
static FileSystem	write(FileSystem fs, Path path, java.lang.Iterable<? extends java.lang.CharSequence> lines, java.nio.charset.Charset cs)	Write lines of text to a file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SYMLINK_NO_PRIVILEGE

public static final int SYMLINK_NO_PRIVILEGE

See Also:

Constant Field Values

Constructor Detail

FileUtil

public FileUtil()

Method Detail

stat2Paths

public static Path[] stat2Paths(FileStatus[] stats)

convert an array of FileStatus to an array of Path

Parameters:

stats - an array of FileStatus objects

Returns:

an array of paths corresponding to the input

stat2Paths

public static Path[] stat2Paths(FileStatus[] stats,
                                Path path)

convert an array of FileStatus to an array of Path. If stats if null, return path

Parameters:

stats - an array of FileStatus objects

path - default path to return in stats is null

Returns:

an array of paths corresponding to the input

fullyDeleteOnExit

public static void fullyDeleteOnExit(java.io.File file)

Register all files recursively to be deleted on exit.

Parameters:

file - File/directory to be deleted

fullyDelete

public static boolean fullyDelete(java.io.File dir)

Delete a directory and all its contents. If we return false, the directory may be partially-deleted. (1) If dir is symlink to a file, the symlink is deleted. The file pointed to by the symlink is not deleted. (2) If dir is symlink to a directory, symlink is deleted. The directory pointed to by symlink is not deleted. (3) If dir is a normal file, it is deleted. (4) If dir is a normal directory, then dir and all its contents recursively are deleted.

Parameters:

dir - dir.

Returns:

fully delete status.

fullyDelete

public static boolean fullyDelete(java.io.File dir,
                                  boolean tryGrantPermissions)

Delete a directory and all its contents. If we return false, the directory may be partially-deleted. (1) If dir is symlink to a file, the symlink is deleted. The file pointed to by the symlink is not deleted. (2) If dir is symlink to a directory, symlink is deleted. The directory pointed to by symlink is not deleted. (3) If dir is a normal file, it is deleted. (4) If dir is a normal directory, then dir and all its contents recursively are deleted.

Parameters:

dir - the file or directory to be deleted

tryGrantPermissions - true if permissions should be modified to delete a file.

Returns:

true on success false on failure.

readLink

public static java.lang.String readLink(java.io.File f)

Returns the target of the given symlink. Returns the empty string if the given path does not refer to a symlink or there is an error accessing the symlink.

Parameters:

f - File representing the symbolic link.

Returns:

The target of the symbolic link, empty string on error or if not a symlink.

fixSymlinkPath

public static Path fixSymlinkPath(PathData path)
                           throws java.io.IOException

Throws:

java.io.IOException

fixSymlinkFileStatus

public static Path fixSymlinkFileStatus(FileStatus stat)
                                 throws java.io.IOException

Throws:

java.io.IOException

checkItemForSymlink

public static PathData checkItemForSymlink(PathData item)
                                    throws java.io.IOException

Throws:

java.io.IOException

checkExistItem

public static PathData checkExistItem(PathData item)
                               throws java.io.IOException

Throws:

java.io.IOException

checkPathForSymlink

public static PathData checkPathForSymlink(Path path,
                                           Configuration conf)
                                    throws java.io.IOException

Throws:

java.io.IOException

fullyDeleteContents

public static boolean fullyDeleteContents(java.io.File dir)

Delete the contents of a directory, not the directory itself. If we return false, the directory may be partially-deleted. If dir is a symlink to a directory, all the contents of the actual directory pointed to by dir will be deleted.

Parameters:

dir - dir.

Returns:

fullyDeleteContents Status.

fullyDeleteContents

public static boolean fullyDeleteContents(java.io.File dir,
                                          boolean tryGrantPermissions)

Delete the contents of a directory, not the directory itself. If we return false, the directory may be partially-deleted. If dir is a symlink to a directory, all the contents of the actual directory pointed to by dir will be deleted.

Parameters:

dir - dir.

tryGrantPermissions - if 'true', try grant +rwx permissions to this and all the underlying directories before trying to delete their contents.

Returns:

fully delete contents status.

fullyDelete

@Deprecated
public static void fullyDelete(FileSystem fs,
                               Path dir)
                        throws java.io.IOException

Deprecated.

Use FileSystem.delete(Path, boolean)

Recursively delete a directory.

Parameters:

fs - FileSystem on which the path is present

dir - directory to recursively delete

Throws:

java.io.IOException - raised on errors performing I/O.

copy

public static boolean copy(FileSystem srcFS,
                           Path src,
                           FileSystem dstFS,
                           Path dst,
                           boolean deleteSource,
                           Configuration conf)
                    throws java.io.IOException

Copy files between FileSystems.

Parameters:

srcFS - src fs.

src - src.

dstFS - dst fs.

dst - dst.

deleteSource - delete source.

conf - configuration.

Returns:

if copy success true, not false.

Throws:

java.io.IOException - raised on errors performing I/O.

copy

public static boolean copy(FileSystem srcFS,
                           Path[] srcs,
                           FileSystem dstFS,
                           Path dst,
                           boolean deleteSource,
                           boolean overwrite,
                           Configuration conf)
                    throws java.io.IOException

Throws:

java.io.IOException

copy

public static boolean copy(FileSystem srcFS,
                           Path src,
                           FileSystem dstFS,
                           Path dst,
                           boolean deleteSource,
                           boolean overwrite,
                           Configuration conf)
                    throws java.io.IOException

Copy files between FileSystems.

Parameters:

srcFS - srcFs.

src - src.

dstFS - dstFs.

dst - dst.

deleteSource - delete source.

overwrite - overwrite.

conf - configuration.

Returns:

true if the operation succeeded.

Throws:

java.io.IOException - raised on errors performing I/O.

copy

public static boolean copy(FileSystem srcFS,
                           FileStatus srcStatus,
                           FileSystem dstFS,
                           Path dst,
                           boolean deleteSource,
                           boolean overwrite,
                           Configuration conf)
                    throws java.io.IOException

Copy a file/directory tree within/between filesystems.

returns true if the operation succeeded. When deleteSource is true, this means "after the copy, delete(source) returned true" If the destination is a directory, and mkdirs (dest) fails, the operation will return false rather than raise any exception.

The overwrite flag is about overwriting files; it has no effect about handing an attempt to copy a file atop a directory (expect an IOException), or a directory over a path which contains a file (mkdir will fail, so "false").

The operation is recursive, and the deleteSource operation takes place as each subdirectory is copied. Therefore, if an operation fails partway through, the source tree may be partially deleted.

Parameters:

srcFS - source filesystem

srcStatus - status of source

dstFS - destination filesystem

dst - path of source

deleteSource - delete the source?

overwrite - overwrite files at destination?

conf - configuration to use when opening files

Returns:

true if the operation succeeded.

Throws:

java.io.IOException - failure

copyAsMove

public static boolean copyAsMove(FileSystem srcFS,
                                 FileStatus srcStatus,
                                 FileSystem dstFS,
                                 Path dst,
                                 boolean deleteSource,
                                 boolean overwrite,
                                 Configuration conf)
                          throws java.io.IOException

Copy files between FileSystems with preserved permissions, ownership, links and ACEs.

Throws:

java.io.IOException

copyMerge

public static boolean copyMerge(FileSystem srcFS,
                                Path srcDir,
                                FileSystem dstFS,
                                Path dstFile,
                                boolean deleteSource,
                                Configuration conf,
                                java.lang.String addString)
                         throws java.io.IOException

Copy all files in a directory to one output file (merge).

Throws:

java.io.IOException

copy

public static boolean copy(java.io.File src,
                           FileSystem dstFS,
                           Path dst,
                           boolean deleteSource,
                           Configuration conf)
                    throws java.io.IOException

Copy local files to a FileSystem.

Parameters:

src - src.

dstFS - dstFs.

dst - dst.

deleteSource - delete source.

conf - configuration.

Returns:

true if the operation succeeded.

Throws:

java.io.IOException - raised on errors performing I/O.

copy

public static boolean copy(FileSystem srcFS,
                           Path src,
                           java.io.File dst,
                           boolean deleteSource,
                           Configuration conf)
                    throws java.io.IOException

Copy FileSystem files to local files.

Parameters:

srcFS - srcFs.

src - src.

dst - dst.

deleteSource - delete source.

conf - configuration.

Returns:

true if the operation succeeded.

Throws:

java.io.IOException - raised on errors performing I/O.

isRegularFile

public static boolean isRegularFile(java.io.File file)

isRegularFile

public static boolean isRegularFile(java.io.File file,
                                    boolean allowLinks)

Check if the file is regular.

Parameters:

file - The file being checked.

allowLinks - Whether to allow matching links.

Returns:

Returns the result of checking whether the file is a regular file.

makeShellPath

public static java.lang.String makeShellPath(java.lang.String filename)
                                      throws java.io.IOException

Convert a os-native filename to a path that works for the shell.

Parameters:

filename - The filename to convert

Returns:

The unix pathname

Throws:

java.io.IOException - on windows, there can be problems with the subprocess

makeShellPath

public static java.lang.String makeShellPath(java.io.File file)
                                      throws java.io.IOException

Convert a os-native filename to a path that works for the shell.

Parameters:

file - The filename to convert

Returns:

The unix pathname

Throws:

java.io.IOException - on windows, there can be problems with the subprocess

makeSecureShellPath

public static java.lang.String makeSecureShellPath(java.io.File file)
                                            throws java.io.IOException

Convert a os-native filename to a path that works for the shell and avoids script injection attacks.

Parameters:

file - The filename to convert

Returns:

The unix pathname

Throws:

java.io.IOException - on windows, there can be problems with the subprocess

makeShellPath

public static java.lang.String makeShellPath(java.io.File file,
                                             boolean makeCanonicalPath)
                                      throws java.io.IOException

Convert a os-native filename to a path that works for the shell.

Parameters:

file - The filename to convert

makeCanonicalPath - Whether to make canonical path for the file passed

Returns:

The unix pathname

Throws:

java.io.IOException - on windows, there can be problems with the subprocess

getDU

public static long getDU(java.io.File dir)

Takes an input dir and returns the du on that local directory. Very basic implementation.

Parameters:

dir - The input dir to get the disk space of this local dir

Returns:

The total disk space of the input local directory

unZip

public static void unZip(java.io.InputStream inputStream,
                         java.io.File toDir)
                  throws java.io.IOException

Given a stream input it will unzip the it in the unzip directory. passed as the second parameter

Parameters:

inputStream - The zip file as input

toDir - The unzip directory where to unzip the zip file.

Throws:

java.io.IOException - an exception occurred

unZip

public static void unZip(java.io.File inFile,
                         java.io.File unzipDir)
                  throws java.io.IOException

Given a File input it will unzip it in the unzip directory. passed as the second parameter

Parameters:

inFile - The zip file as input

unzipDir - The unzip directory where to unzip the zip file.

Throws:

java.io.IOException - An I/O exception has occurred

unTar

public static void unTar(java.io.InputStream inputStream,
                         java.io.File untarDir,
                         boolean gzipped)
                  throws java.io.IOException,
                         java.lang.InterruptedException,
                         java.util.concurrent.ExecutionException

Given a Tar File as input it will untar the file in a the untar directory passed as the second parameter This utility will untar ".tar" files and ".tar.gz","tgz" files.

Parameters:

inputStream - The tar file as input.

untarDir - The untar directory where to untar the tar file.

gzipped - The input stream is gzipped TODO Use magic number and PusbackInputStream to identify

Throws:

java.io.IOException - an exception occurred

java.lang.InterruptedException - command interrupted

java.util.concurrent.ExecutionException - task submit failed

unTar

public static void unTar(java.io.File inFile,
                         java.io.File untarDir)
                  throws java.io.IOException

Given a Tar File as input it will untar the file in a the untar directory passed as the second parameter This utility will untar ".tar" files and ".tar.gz","tgz" files.

Parameters:

inFile - The tar file as input.

untarDir - The untar directory where to untar the tar file.

Throws:

java.io.IOException - an exception occurred.

symLink

public static int symLink(java.lang.String target,
                          java.lang.String linkname)
                   throws java.io.IOException

Create a soft link between a src and destination only on a local disk. HDFS does not support this. On Windows, when symlink creation fails due to security setting, we will log a warning. The return code in this case is 2.

Parameters:

target - the target for symlink

linkname - the symlink

Returns:

0 on success

Throws:

java.io.IOException - raised on errors performing I/O.

chmod

public static int chmod(java.lang.String filename,
                        java.lang.String perm)
                 throws java.io.IOException,
                        java.lang.InterruptedException

Change the permissions on a filename.

Parameters:

filename - the name of the file to change

perm - the permission string

Returns:

the exit code from the command

Throws:

java.io.IOException - raised on errors performing I/O.

java.lang.InterruptedException - command interrupted.

chmod

public static int chmod(java.lang.String filename,
                        java.lang.String perm,
                        boolean recursive)
                 throws java.io.IOException

Change the permissions on a file / directory, recursively, if needed.

Parameters:

filename - name of the file whose permissions are to change

perm - permission string

recursive - true, if permissions should be changed recursively

Returns:

the exit code from the command.

Throws:

java.io.IOException - raised on errors performing I/O.

setOwner

public static void setOwner(java.io.File file,
                            java.lang.String username,
                            java.lang.String groupname)
                     throws java.io.IOException

Set the ownership on a file / directory. User name and group name cannot both be null.

Parameters:

file - the file to change

username - the new user owner name

groupname - the new group owner name

Throws:

java.io.IOException - raised on errors performing I/O.

setReadable

public static boolean setReadable(java.io.File f,
                                  boolean readable)

Platform independent implementation for File.setReadable(boolean) File#setReadable does not work as expected on Windows.

Parameters:

f - input file

readable - readable.

Returns:

true on success, false otherwise

setWritable

public static boolean setWritable(java.io.File f,
                                  boolean writable)

Platform independent implementation for File.setWritable(boolean) File#setWritable does not work as expected on Windows.

Parameters:

f - input file

writable - writable.

Returns:

true on success, false otherwise

setExecutable

public static boolean setExecutable(java.io.File f,
                                    boolean executable)

Platform independent implementation for File.setExecutable(boolean) File#setExecutable does not work as expected on Windows. Note: revoking execute permission on folders does not have the same behavior on Windows as on Unix platforms. Creating, deleting or renaming a file within that folder will still succeed on Windows.

Parameters:

f - input file

executable - executable.

Returns:

true on success, false otherwise

canRead

public static boolean canRead(java.io.File f)

Platform independent implementation for File.canRead()

Parameters:

f - input file

Returns:

On Unix, same as File.canRead() On Windows, true if process has read access on the path

canWrite

public static boolean canWrite(java.io.File f)

Platform independent implementation for File.canWrite()

Parameters:

f - input file

Returns:

On Unix, same as File.canWrite() On Windows, true if process has write access on the path

canExecute

public static boolean canExecute(java.io.File f)

Platform independent implementation for File.canExecute()

Parameters:

f - input file

Returns:

On Unix, same as File.canExecute() On Windows, true if process has execute access on the path

setPermission

public static void setPermission(java.io.File f,
                                 FsPermission permission)
                          throws java.io.IOException

Set permissions to the required value. Uses the java primitives instead of forking if group == other.

Parameters:

f - the file to change

permission - the new permissions

Throws:

java.io.IOException - raised on errors performing I/O.

createLocalTempFile

public static final java.io.File createLocalTempFile(java.io.File basefile,
                                                     java.lang.String prefix,
                                                     boolean isDeleteOnExit)
                                              throws java.io.IOException

Create a tmp file for a base file.

Parameters:

basefile - the base file of the tmp

prefix - file name prefix of tmp

isDeleteOnExit - if true, the tmp will be deleted when the VM exits

Returns:

a newly created tmp file

Throws:

java.io.IOException - If a tmp file cannot created

See Also:

File.createTempFile(String, String, File), File.deleteOnExit()

replaceFile

public static void replaceFile(java.io.File src,
                               java.io.File target)
                        throws java.io.IOException

Move the src file to the name specified by target.

Parameters:

src - the source file

target - the target file

Throws:

java.io.IOException - If this operation fails

listFiles

public static java.io.File[] listFiles(java.io.File dir)
                                throws java.io.IOException

A wrapper for File.listFiles(). This java.io API returns null when a dir is not a directory or for any I/O error. Instead of having null check everywhere File#listFiles() is used, we will add utility API to get around this problem. For the majority of cases where we prefer an IOException to be thrown.

Parameters:

dir - directory for which listing should be performed

Returns:

list of files or empty list

Throws:

java.io.IOException - for invalid directory or for a bad disk.

list

public static java.lang.String[] list(java.io.File dir)
                               throws java.io.IOException

A wrapper for File.list(). This java.io API returns null when a dir is not a directory or for any I/O error. Instead of having null check everywhere File#list() is used, we will add utility API to get around this problem. For the majority of cases where we prefer an IOException to be thrown.

Parameters:

dir - directory for which listing should be performed

Returns:

list of file names or empty string list

Throws:

java.nio.file.AccessDeniedException - for unreadable directory

java.io.IOException - for invalid directory or for bad disk

createJarWithClassPath

public static java.lang.String[] createJarWithClassPath(java.lang.String inputClassPath,
                                                        Path pwd,
                                                        java.util.Map<java.lang.String,java.lang.String> callerEnv)
                                                 throws java.io.IOException

Throws:

java.io.IOException

createJarWithClassPath

public static java.lang.String[] createJarWithClassPath(java.lang.String inputClassPath,
                                                        Path pwd,
                                                        Path targetDir,
                                                        java.util.Map<java.lang.String,java.lang.String> callerEnv)
                                                 throws java.io.IOException

Create a jar file at the given path, containing a manifest with a classpath that references all specified entries. Some platforms may have an upper limit on command line length. For example, the maximum command line length on Windows is 8191 characters, but the length of the classpath may exceed this. To work around this limitation, use this method to create a small intermediate jar with a manifest that contains the full classpath. It returns the absolute path to the new jar, which the caller may set as the classpath for a new process. Environment variable evaluation is not supported within a jar manifest, so this method expands environment variables before inserting classpath entries to the manifest. The method parses environment variables according to platform-specific syntax (%VAR% on Windows, or $VAR otherwise). On Windows, environment variables are case-insensitive. For example, %VAR% and %var% evaluate to the same value. Specifying the classpath in a jar manifest does not support wildcards, so this method expands wildcards internally. Any classpath entry that ends with * is translated to all files at that path with extension .jar or .JAR.

Parameters:

inputClassPath - String input classpath to bundle into the jar manifest

pwd - Path to working directory to save jar

targetDir - path to where the jar execution will have its working dir

callerEnv - Map <String, String> caller's environment variables to use for expansion

Returns:

String[] with absolute path to new jar in position 0 and unexpanded wild card entry path in position 1

Throws:

java.io.IOException - if there is an I/O error while writing the jar file

getJarsInDirectory

public static java.util.List<Path> getJarsInDirectory(java.lang.String path)

Returns all jars that are in the directory. It is useful in expanding a wildcard path to return all jars from the directory to use in a classpath. It operates only on local paths.

Parameters:

path - the path to the directory. The path may include the wildcard.

Returns:

the list of jars as URLs, or an empty list if there are no jars, or the directory does not exist locally

getJarsInDirectory

public static java.util.List<Path> getJarsInDirectory(java.lang.String path,
                                                      boolean useLocal)

Returns all jars that are in the directory. It is useful in expanding a wildcard path to return all jars from the directory to use in a classpath.

Parameters:

path - the path to the directory. The path may include the wildcard.

useLocal - use local.

Returns:

the list of jars as URLs, or an empty list if there are no jars, or the directory does not exist

compareFs

public static boolean compareFs(FileSystem srcFs,
                                FileSystem destFs)

write

public static FileSystem write(FileSystem fs,
                               Path path,
                               byte[] bytes)
                        throws java.io.IOException

Writes bytes to a file. This utility method opens the file for writing, creating the file if it does not exist, or overwrites an existing file. All bytes in the byte array are written to the file.

Parameters:

fs - the file system with which to create the file

path - the path to the file

bytes - the byte array with the bytes to write

Returns:

the file system

Throws:

java.lang.NullPointerException - if any of the arguments are null

java.io.IOException - if an I/O error occurs creating or writing to the file

write

public static FileContext write(FileContext fileContext,
                                Path path,
                                byte[] bytes)
                         throws java.io.IOException

Writes bytes to a file. This utility method opens the file for writing, creating the file if it does not exist, or overwrites an existing file. All bytes in the byte array are written to the file.

Parameters:

fileContext - the file context with which to create the file

path - the path to the file

bytes - the byte array with the bytes to write

Returns:

the file context

Throws:

java.lang.NullPointerException - if any of the arguments are null

java.io.IOException - if an I/O error occurs creating or writing to the file

write

public static FileSystem write(FileSystem fs,
                               Path path,
                               java.lang.Iterable<? extends java.lang.CharSequence> lines,
                               java.nio.charset.Charset cs)
                        throws java.io.IOException

Write lines of text to a file. Each line is a char sequence and is written to the file in sequence with each line terminated by the platform's line separator, as defined by the system property line.separator. Characters are encoded into bytes using the specified charset. This utility method opens the file for writing, creating the file if it does not exist, or overwrites an existing file.

Parameters:

fs - the file system with which to create the file

path - the path to the file

lines - a Collection to iterate over the char sequences

cs - the charset to use for encoding

Returns:

the file system

Throws:

java.lang.NullPointerException - if any of the arguments are null

java.io.IOException - if an I/O error occurs creating or writing to the file

write

public static FileContext write(FileContext fileContext,
                                Path path,
                                java.lang.Iterable<? extends java.lang.CharSequence> lines,
                                java.nio.charset.Charset cs)
                         throws java.io.IOException

Write lines of text to a file. Each line is a char sequence and is written to the file in sequence with each line terminated by the platform's line separator, as defined by the system property line.separator. Characters are encoded into bytes using the specified charset. This utility method opens the file for writing, creating the file if it does not exist, or overwrites an existing file.

Parameters:

fileContext - the file context with which to create the file

path - the path to the file

lines - a Collection to iterate over the char sequences

cs - the charset to use for encoding

Returns:

the file context

Throws:

java.lang.NullPointerException - if any of the arguments are null

java.io.IOException - if an I/O error occurs creating or writing to the file

write

public static FileSystem write(FileSystem fs,
                               Path path,
                               java.lang.CharSequence charseq,
                               java.nio.charset.Charset cs)
                        throws java.io.IOException

Write a line of text to a file. Characters are encoded into bytes using the specified charset. This utility method opens the file for writing, creating the file if it does not exist, or overwrites an existing file.

Parameters:

fs - the file system with which to create the file

path - the path to the file

charseq - the char sequence to write to the file

cs - the charset to use for encoding

Returns:

the file system

Throws:

java.lang.NullPointerException - if any of the arguments are null

java.io.IOException - if an I/O error occurs creating or writing to the file

write

public static FileContext write(FileContext fs,
                                Path path,
                                java.lang.CharSequence charseq,
                                java.nio.charset.Charset cs)
                         throws java.io.IOException

Write a line of text to a file. Characters are encoded into bytes using the specified charset. This utility method opens the file for writing, creating the file if it does not exist, or overwrites an existing file.

Parameters:

fs - the file context with which to create the file

path - the path to the file

charseq - the char sequence to write to the file

cs - the charset to use for encoding

Returns:

the file context

Throws:

java.lang.NullPointerException - if any of the arguments are null

java.io.IOException - if an I/O error occurs creating or writing to the file

write

public static FileSystem write(FileSystem fs,
                               Path path,
                               java.lang.CharSequence charseq)
                        throws java.io.IOException

Write a line of text to a file. Characters are encoded into bytes using UTF-8. This utility method opens the file for writing, creating the file if it does not exist, or overwrites an existing file.

Parameters:

fs - the files system with which to create the file

path - the path to the file

charseq - the char sequence to write to the file

Returns:

the file system

Throws:

java.lang.NullPointerException - if any of the arguments are null

java.io.IOException - if an I/O error occurs creating or writing to the file

write

public static FileContext write(FileContext fileContext,
                                Path path,
                                java.lang.CharSequence charseq)
                         throws java.io.IOException

Write a line of text to a file. Characters are encoded into bytes using UTF-8. This utility method opens the file for writing, creating the file if it does not exist, or overwrites an existing file.

Parameters:

fileContext - the files system with which to create the file

path - the path to the file

charseq - the char sequence to write to the file

Returns:

the file context

Throws:

java.lang.NullPointerException - if any of the arguments are null

java.io.IOException - if an I/O error occurs creating or writing to the file

rename

@LimitedPrivate("ViewDistributedFileSystem")
@Unstable
public static void rename(FileSystem srcFs,
                          Path src,
                          Path dst,
                          Options.Rename... options)
                   throws java.io.IOException

Throws:

java.io.IOException

maybeIgnoreMissingDirectory

public static void maybeIgnoreMissingDirectory(FileSystem fs,
                                               Path path,
                                               java.io.FileNotFoundException e)
                                        throws java.io.FileNotFoundException

Method to call after a FNFE has been raised on a treewalk, so as to decide whether to throw the exception (default), or, if the FS supports inconsistent directory listings, to log and ignore it. If this returns then the caller should ignore the failure and continue.

Parameters:

fs - filesystem

path - path

e - exception caught

Throws:

java.io.FileNotFoundException - the exception passed in, if rethrown.