DF, Shell.ShellCommandExecutor, Stat@Public
@Evolving
public abstract class Shell
extends java.lang.Object
Shell can be used to run shell commands like du or
df. It also offers facilities to gate commands by
time-intervals.| Modifier and Type | Class | Description |
|---|---|---|
static interface |
Shell.CommandExecutor |
|
static class |
Shell.ExitCodeException |
This is an IOException with exit code added.
|
static class |
Shell.OSType |
|
static class |
Shell.ShellCommandExecutor |
A simple shell command executor.
|
| Modifier and Type | Field | Description |
|---|---|---|
static java.lang.String |
ENV_DEBUG_OPTS |
|
static java.lang.String |
ENV_HADOOP_HOME |
Environment variable for Hadoop's home dir: "HADOOP_HOME".
|
static java.lang.String |
ENV_HADOOP_OPTS |
|
static java.lang.String |
ENV_MAPR_JMX_OPTS |
|
static java.lang.String |
ENV_NAME_REGEX |
Regular expression for environment variables: "[A-Za-z_][A-Za-z0-9_]*".
|
static boolean |
FREEBSD |
|
protected boolean |
inheritParentEnv |
Indicates if the parent env vars should be inherited or not
|
static boolean |
isSetsidAvailable |
Flag which is true if setsid exists.
|
static java.lang.String |
LINK_COMMAND |
a Unix command to create a link: "ln".
|
static boolean |
LINUX |
|
static org.slf4j.Logger |
LOG |
|
static boolean |
MAC |
|
static Shell.OSType |
osType |
Get the type of the operating system, as determined from parsing
the
os.name property. |
static boolean |
OTHER |
|
static boolean |
PPC_64 |
|
static java.lang.String |
READ_LINK_COMMAND |
a Unix command to get a link target: "readlink".
|
static java.lang.String |
SET_GROUP_COMMAND |
a Unix command to set the change user's groups list: "chgrp".
|
static java.lang.String |
SET_OWNER_COMMAND |
a Unix command to set owner: "chown".
|
static java.lang.String |
SET_PERMISSION_COMMAND |
a Unix command to set permission: "chmod".
|
static boolean |
SOLARIS |
|
static java.lang.String |
SYSPROP_HADOOP_HOME_DIR |
System property for the Hadoop home directory: "hadoop.home.dir".
|
protected long |
timeOutInterval |
Time after which the executing script would be timedout.
|
static java.lang.String |
TOKEN_SEPARATOR_REGEX |
Token separator regex used to parse Shell tool outputs.
|
static java.lang.String |
USER_NAME_COMMAND |
a Unix command to get the current user's name: "whoami".
|
static boolean |
WINDOWS |
|
static int |
WINDOWS_MAX_SHELL_LENGHT |
Deprecated.
use the correctly spelled constant.
|
static int |
WINDOWS_MAX_SHELL_LENGTH |
Maximum command line length in Windows
KB830473 documents this as 8191
|
static java.lang.Object |
WindowsProcessLaunchLock |
Windows
CreateProcess synchronization object. |
static java.lang.String |
WINUTILS |
Deprecated.
use one of the exception-raising getter methods,
specifically
getWinUtilsPath() or getWinUtilsFile() |
| Modifier | Constructor | Description |
|---|---|---|
protected |
Shell() |
Create an instance with no minimum interval between runs; stderr is
not merged with stdout.
|
protected |
Shell(long interval) |
Create an instance with a minimum interval between executions; stderr is
not merged with stdout.
|
protected |
Shell(long interval,
boolean redirectErrorStream) |
Create a shell instance which can be re-executed when the
run()
method is invoked with a given elapsed time between calls. |
| Modifier and Type | Method | Description |
|---|---|---|
static java.io.File |
appendScriptExtension(java.io.File parent,
java.lang.String basename) |
Returns a File referencing a script with the given basename, inside the
given parent directory.
|
static java.lang.String |
appendScriptExtension(java.lang.String basename) |
Returns a script file name with the given basename.
|
static java.lang.String |
bashQuote(java.lang.String arg) |
Quote the given arg so that bash will interpret it as a single value.
|
static boolean |
checkIsBashSupported() |
|
static boolean |
checkUserExists(java.lang.String user) |
Check that user exists on the node
|
static void |
checkWindowsCommandLineLength(java.lang.String... commands) |
Checks if a given command (String[]) fits in the Windows maximum command
line length Note that the input is expected to already include space
delimiters, no extra count will be added for delimiters.
|
static void |
destroyAllShellProcesses() |
Static method to destroy all running
Shell processes. |
static java.lang.String |
execCommand(java.lang.String... cmd) |
Static method to execute a shell command.
|
static java.lang.String |
execCommand(java.util.Map<java.lang.String,java.lang.String> env,
java.lang.String... cmd) |
Static method to execute a shell command.
|
static java.lang.String |
execCommand(java.util.Map<java.lang.String,java.lang.String> env,
java.lang.String[] cmd,
long timeout) |
Static method to execute a shell command.
|
static java.util.Set<Shell> |
getAllShells() |
Static method to return a Set of all
Shell objects. |
static java.lang.String[] |
getCheckProcessIsAliveCommand(java.lang.String pid) |
Return a command for determining if process with specified pid is alive.
|
java.lang.String |
getEnvironment(java.lang.String env) |
Get an environment variable.
|
static java.lang.String |
getEnvironmentVariableRegex() |
Return a regular expression string that match environment variables.
|
protected abstract java.lang.String[] |
getExecString() |
return an array containing the command name and its parameters.
|
int |
getExitCode() |
get the exit code.
|
static java.lang.String[] |
getGetPermissionCommand() |
Return a command to get permission information.
|
static java.lang.String[] |
getGroupsCommand() |
a Unix command to get the current user's groups list.
|
static java.lang.String[] |
getGroupsForUserCommand(java.lang.String user) |
A command to get a given user's groups list.
|
static java.lang.String[] |
getGroupsIDForUserCommand(java.lang.String user) |
A command to get a given user's group id list.
|
static java.lang.String |
getHadoopHome() |
Get the Hadoop home directory.
|
static java.lang.Long |
getMemlockLimit(java.lang.Long ulimit) |
Static method to return the memory lock limit for datanode.
|
java.lang.Process |
getProcess() |
get the current sub-process executing the given command.
|
static java.io.File |
getQualifiedBin(java.lang.String executable) |
Fully qualify the path to a binary that should be in a known hadoop
bin location.
|
static java.lang.String |
getQualifiedBinPath(java.lang.String executable) |
Fully qualify the path to a binary that should be in a known hadoop
bin location.
|
static java.lang.String[] |
getReadlinkCommand(java.lang.String link) |
Return a command to read the target of the a symbolic link.
|
static java.lang.String[] |
getRunScriptCommand(java.io.File script) |
Returns a command to run the given script.
|
static java.lang.String[] |
getSetOwnerCommand(java.lang.String owner) |
Return a command to set owner.
|
static java.lang.String[] |
getSetPermissionCommand(java.lang.String perm,
boolean recursive) |
Return a command to set permission.
|
static java.lang.String[] |
getSetPermissionCommand(java.lang.String perm,
boolean recursive,
java.lang.String file) |
Return a command to set permission for specific file.
|
static java.lang.String[] |
getSignalKillCommand(int code,
java.lang.String pid) |
Return a command to send a signal to a given pid.
|
static java.lang.String[] |
getSymlinkCommand(java.lang.String target,
java.lang.String link) |
Return a command to create symbolic links.
|
static java.lang.String[] |
getUsersForNetgroupCommand(java.lang.String netgroup) |
A command to get a given netgroup's user list.
|
java.lang.Thread |
getWaitingThread() |
get the thread that is waiting on this instance of
Shell. |
static java.io.File |
getWinUtilsFile() |
Get a file reference to winutils.
|
static java.lang.String |
getWinUtilsPath() |
Locate the winutils binary, or fail with a meaningful
exception and stack trace as an RTE.
|
static boolean |
hasWinutilsPath() |
Predicate to indicate whether or not the path to winutils is known.
|
static boolean |
isJava7OrAbove() |
Deprecated.
This call isn't needed any more: please remove uses of it.
|
static boolean |
isJavaVersionAtLeast(int version) |
Query to see if major version of Java specification of the system
is equal or greater than the parameter.
|
boolean |
isTimedOut() |
To check if the passed script to shell command executor timed out or
not.
|
protected abstract void |
parseExecResult(java.io.BufferedReader lines) |
Parse the execution result.
|
protected void |
run() |
Check to see if a command needs to be executed and execute if needed.
|
protected void |
setEnvironment(java.util.Map<java.lang.String,java.lang.String> env) |
Set the environment for the command.
|
protected void |
setWorkingDirectory(java.io.File dir) |
Set the working directory.
|
public static final org.slf4j.Logger LOG
public static final java.lang.String SYSPROP_HADOOP_HOME_DIR
public static final java.lang.String ENV_HADOOP_HOME
public static final java.lang.String ENV_HADOOP_OPTS
public static final java.lang.String ENV_MAPR_JMX_OPTS
public static final java.lang.String ENV_DEBUG_OPTS
public static final int WINDOWS_MAX_SHELL_LENGTH
@Deprecated public static final int WINDOWS_MAX_SHELL_LENGHT
WINDOWS_MAX_SHELL_LENGTH.public static final java.lang.String USER_NAME_COMMAND
public static final java.lang.Object WindowsProcessLaunchLock
CreateProcess synchronization object.public static final Shell.OSType osType
os.name property.public static final boolean WINDOWS
public static final boolean SOLARIS
public static final boolean MAC
public static final boolean FREEBSD
public static final boolean LINUX
public static final boolean OTHER
public static final boolean PPC_64
public static final java.lang.String ENV_NAME_REGEX
public static final java.lang.String SET_PERMISSION_COMMAND
public static final java.lang.String SET_OWNER_COMMAND
public static final java.lang.String SET_GROUP_COMMAND
public static final java.lang.String LINK_COMMAND
public static final java.lang.String READ_LINK_COMMAND
protected long timeOutInterval
protected boolean inheritParentEnv
@Deprecated public static final java.lang.String WINUTILS
getWinUtilsPath() or getWinUtilsFile()Important: caller must check for this value being null. The lack of such checks has led to many support issues being raised.
public static final boolean isSetsidAvailable
public static final java.lang.String TOKEN_SEPARATOR_REGEX
protected Shell()
protected Shell(long interval)
interval - interval in milliseconds between command executions.protected Shell(long interval,
boolean redirectErrorStream)
run()
method is invoked with a given elapsed time between calls.interval - the minimum duration in milliseconds to wait before
re-executing the command. If set to 0, there is no minimum.redirectErrorStream - should the error stream be merged with
the normal output stream?@Deprecated public static boolean isJava7OrAbove()
public static boolean isJavaVersionAtLeast(int version)
version - 8, 9, 10 etc.public static void checkWindowsCommandLineLength(java.lang.String... commands)
throws java.io.IOException
commands - command parts, including any space delimitersjava.io.IOException - raised on errors performing I/O.@Private public static java.lang.String bashQuote(java.lang.String arg)
arg - the argument to quotepublic static java.lang.String[] getGroupsCommand()
public static java.lang.String[] getGroupsForUserCommand(java.lang.String user)
user - user.public static java.lang.String[] getGroupsIDForUserCommand(java.lang.String user)
user - user.public static boolean checkUserExists(java.lang.String user)
user - - usernamepublic static java.lang.String[] getUsersForNetgroupCommand(java.lang.String netgroup)
netgroup - net group.public static java.lang.String[] getGetPermissionCommand()
public static java.lang.String[] getSetPermissionCommand(java.lang.String perm,
boolean recursive)
perm - permission.recursive - recursive.public static java.lang.String[] getSetPermissionCommand(java.lang.String perm,
boolean recursive,
java.lang.String file)
perm - String permission to setrecursive - boolean true to apply to all sub-directories recursivelyfile - String file to setpublic static java.lang.String[] getSetOwnerCommand(java.lang.String owner)
owner - owner.public static java.lang.String[] getSymlinkCommand(java.lang.String target,
java.lang.String link)
target - target.link - link.public static java.lang.String[] getReadlinkCommand(java.lang.String link)
link - link.public static java.lang.String[] getCheckProcessIsAliveCommand(java.lang.String pid)
pid - process IDkill -0 command or equivalentpublic static java.lang.String[] getSignalKillCommand(int code,
java.lang.String pid)
code - code.pid - pid.public static java.lang.String getEnvironmentVariableRegex()
public static java.io.File appendScriptExtension(java.io.File parent,
java.lang.String basename)
".cmd" on Windows, or ".sh" otherwise.parent - File parent directorybasename - String script file basenamepublic static java.lang.String appendScriptExtension(java.lang.String basename)
".cmd" on Windows, or ".sh" otherwise.basename - String script file basenamepublic static java.lang.String[] getRunScriptCommand(java.io.File script)
script - File script to runpublic static java.lang.String getHadoopHome()
throws java.io.IOException
java.io.IOException - if the home directory cannot be located.public static java.io.File getQualifiedBin(java.lang.String executable)
throws java.io.FileNotFoundException
executable - executablejava.io.FileNotFoundException - if the path does not existpublic static java.lang.String getQualifiedBinPath(java.lang.String executable)
throws java.io.IOException
executable - executablejava.io.FileNotFoundException - if the path does not existjava.io.IOException - on path canonicalization failurespublic static boolean hasWinutilsPath()
WINUTILS is non-null, and both
getWinUtilsPath() and getWinUtilsFile()
will successfully return this value. Always false on non-windows systems.public static java.lang.String getWinUtilsPath()
IOException.WINUTILS_EXEjava.lang.RuntimeException - if the path is not resolvablepublic static java.io.File getWinUtilsFile()
throws java.io.FileNotFoundException
java.io.FileNotFoundException - on any failure to locate that file.public static boolean checkIsBashSupported()
throws java.io.InterruptedIOException
java.io.InterruptedIOExceptionprotected void setEnvironment(java.util.Map<java.lang.String,java.lang.String> env)
env - Mapping of environment variablesprotected void setWorkingDirectory(java.io.File dir)
dir - The directory where the command will be executedprotected void run()
throws java.io.IOException
java.io.IOException - raised on errors performing I/O.protected abstract java.lang.String[] getExecString()
protected abstract void parseExecResult(java.io.BufferedReader lines)
throws java.io.IOException
lines - lines.java.io.IOException - raised on errors performing I/O.public java.lang.String getEnvironment(java.lang.String env)
env - the environment varpublic java.lang.Process getProcess()
public int getExitCode()
public java.lang.Thread getWaitingThread()
Shell.public boolean isTimedOut()
public static java.lang.String execCommand(java.lang.String... cmd)
throws java.io.IOException
Shell interface.cmd - shell command to execute.java.io.IOException - raised on errors performing I/O.public static java.lang.String execCommand(java.util.Map<java.lang.String,java.lang.String> env,
java.lang.String[] cmd,
long timeout)
throws java.io.IOException
Shell interface.env - the map of environment key=valuecmd - shell command to execute.timeout - time in milliseconds after which script should be marked timeoutjava.io.IOException - on any problem.public static java.lang.String execCommand(java.util.Map<java.lang.String,java.lang.String> env,
java.lang.String... cmd)
throws java.io.IOException
Shell interface.env - the map of environment key=valuecmd - shell command to execute.java.io.IOException - on any problem.public static void destroyAllShellProcesses()
Shell processes.
Iterates through a map of all currently running Shell
processes and destroys them one by one. This method is thread safepublic static java.util.Set<Shell> getAllShells()
Shell objects.public static java.lang.Long getMemlockLimit(java.lang.Long ulimit)
ulimit - max value at which memory locked should be capped.Copyright © 2008–2025 Apache Software Foundation. All rights reserved.