Package org.apache.hadoop.util

Common utilities.

Interface Summary

Interface	Description
CleanerUtil.BufferCleaner	Pass in an implementation of this interface to cleanup ByteBuffers.
DiskValidator	A interface for disk validators.
ExitCodeProvider	Get the exit code of an exception.
GcTimeMonitor.GcTimeAlertHandler	The user can provide an instance of a class implementing this interface when initializing a GcTimeMonitor to receive alerts when GC time percentage exceeds the specified threshold.
GSet<K,E extends K>	A GSet is set, which supports the GSet.get(Object) operation.
IdentityHashStore.Visitor<K,V>
IdGenerator	Generic ID generator used for generating various types of number sequences.
IndexedSortable	Interface for collections capable of being sorted by IndexedSorter algorithms.
IndexedSorter	Interface for sort algorithms accepting IndexedSortable items.
IntrusiveCollection.Element	An element contained in this list.
IPList
LightWeightCache.Entry	Entries of LightWeightCache.
LightWeightGSet.LinkedElement	Elements of LightWeightGSet.
Progressable	A facility for reporting progress.
RateLimiting	Minimal subset of google rate limiter class.
ServicePlugin	Service plug-in interface.
Shell.CommandExecutor
Tool	A tool interface that supports handling of generic command-line options.

Class Summary

Class	Description
ApplicationClassLoader	A URLClassLoader for application isolation.
AsyncDiskService
AutoCloseableLock	This is a wrap class of a ReentrantLock.
BaseMapRUtil
BasicDiskValidator	BasicDiskValidator is a wrapper around DiskChecker.
BlockingThreadPoolExecutorService	This ExecutorService blocks the submission of new tasks when its queue is already full by using a semaphore.
CacheableIPList	CacheableIPList loads a list of subnets from a file.
ChunkedArrayList<T>	Simplified List implementation which stores elements as a list of chunks, each chunk having a maximum size.
Classpath	Command-line utility for getting the full classpath needed to launch a Hadoop client application.
ClassUtil
CleanerUtil	sun.misc.Cleaner has moved in OpenJDK 9 and sun.misc.Unsafe#invokeCleaner(ByteBuffer) is the replacement.
CloseableReferenceCount	A closeable object that maintains a reference count.
CombinedIPList	Util class to stores ips/hosts/subnets.
CombinedIPWhiteList
ComparableVersion	Generic implementation of version comparison.
ConfigurationHelper	Configuration Helper class to provide advanced configuration parsing.
ConfTest	This class validates configuration XML files in ${HADOOP_CONF_DIR} or specified ones.
CpuTimeTracker	Utility for sampling and computing CPU usage.
CrcComposer	Encapsulates logic for composing multiple CRCs into one or more combined CRCs corresponding to concatenated underlying data ranges.
CrcUtil	This class provides utilities for working with CRCs.
Daemon	A thread that has called Thread.setDaemon(boolean) with true.
Daemon.DaemonFactory	Provide a factory for named daemon threads, for use in ExecutorServices constructors
DataChecksum	This class provides interface and utilities for processing checksums for DFS data transfers.
DirectBufferPool	A simple class for pooling direct ByteBuffers.
DiskChecker	Class that provides utility functions for checking disk problem
DiskValidatorFactory	The factory class to create instance of DiskValidator.
DurationInfo	A duration with logging of final state at info or debug in the close() call.
ExitUtil	Facilitates hooking process termination for tests, debugging and embedding.
FastNumberFormat	Fast thread-safe version of NumberFormat
FileBasedIPList	FileBasedIPList loads a list of subnets in CIDR format and ip addresses from a file.
FindClass	This entry point exists for diagnosing classloader problems: is a class or resource present -and if so, where?
GcTimeMonitor	This class monitors the percentage of time the JVM is paused in GC within the specified observation window, say 1 minute.
GcTimeMonitor.Builder
GcTimeMonitor.GcData	Encapsulates data about GC pauses measured at the specific timestamp.
GenericOptionsParser	GenericOptionsParser is a utility to parse command line arguments generic to the Hadoop framework.
GenericsUtil	Contains utility methods for dealing with Java Generics.
GSetByHashMap<K,E extends K>	A GSet implementation by HashMap.
HeapSort	An implementation of the core algorithm of HeapSort.
HostsFileReader
HostsFileReader.HostDetails	An atomic view of the included and excluded hosts
HttpExceptionUtils	HTTP utility class to help propagate server side exception to the client over HTTP as a JSON payload.
IdentityHashStore<K,V>	The IdentityHashStore stores (key, value) mappings in an array.
InstrumentedLock	This is a debugging class that can be used by callers to track whether a specific lock is being held for too long and periodically log a warning and stack trace, if so.
InstrumentedLock.SuppressedSnapshot	Immutable class to capture a snapshot of suppressed log message stats.
InstrumentedReadLock	This is a wrap class of a ReadLock.
InstrumentedReadWriteLock	This is a wrap class of a ReentrantReadWriteLock.
InstrumentedWriteLock	This is a wrap class of a WriteLock.
IntrusiveCollection<E extends IntrusiveCollection.Element>	Implements an intrusive doubly-linked list.
JsonSerialization<T>	Support for marshalling objects to and from JSON.
JvmPauseMonitor	Class which sets up a simple thread which runs in a loop sleeping for a short interval of time.
JWTConfiguration
KMSUtil	Utils for KMS.
LambdaUtils	Lambda-expression utilities be they generic or specific to Hadoop datatypes.
LightWeightCache<K,E extends K>	A low memory footprint Cache which extends LightWeightGSet.
LightWeightGSet<K,E extends K>	A low memory footprint GSet implementation, which uses an array for storing the elements and linked lists for collision resolution.
LightWeightResizableGSet<K,E extends K>	A low memory footprint GSet implementation, which uses an array for storing the elements and linked lists for collision resolution.
LimitInputStream	Copied from guava source code v15 (LimitedInputStream) Guava deprecated LimitInputStream in v14 and removed it in v15.
LineReader	A class that provides a line reader from an input stream.
Lists	Static utility methods pertaining to List instances.
MachineList	Container class which holds a list of ip/host addresses and answers membership queries.
MachineList.InetAddressFactory	InetAddressFactory is used to obtain InetAddress from host.
MapRCommonSecurityUtil
MaprShellCommandExecutor	A base class for running a maprcli unix command.
MergeSort	An implementation of the core algorithm of MergeSort.
NativeCodeLoader	A helper to load the native hadoop code i.e.
NativeLibraryChecker
OperationDuration	Little duration counter.
Options	This class allows generic access to variable length type-safe parameter lists.
Options.BooleanOption
Options.ClassOption
Options.FSDataInputStreamOption
Options.FSDataOutputStreamOption
Options.IntegerOption
Options.LongOption
Options.PathOption
Options.ProgressableOption
Options.StringOption
PerformanceAdvisory
Preconditions	This class replaces guava.Preconditions which provides helpers to validate the following conditions: An invalid null obj causes a NullPointerException. An invalid argument causes an IllegalArgumentException. An invalid state causes an IllegalStateException. An invalid index causes an IndexOutOfBoundsException.
PrintJarMainClass	A micro-application that prints the main class name out of a jar file.
PriorityQueue<T>	A PriorityQueue maintains a partial ordering of its elements such that the least element can always be found in constant time.
ProcessUtils	Process related utilities.
ProgramDriver	A driver that is used to run programs added to it
Progress	Utility to assist with generation of progress reports.
ProtoUtil
PureJavaCrc32	A pure-java implementation of the CRC32 checksum that uses the same polynomial as the built-in native CRC32.
PureJavaCrc32C	A pure-java implementation of the CRC32 checksum that uses the CRC32-C polynomial, the same polynomial used by iSCSI and implemented on many Intel chipsets supporting SSE4.2.
QuickSort	An implementation of the core algorithm of QuickSort.
RateLimitingFactory	Factory for Rate Limiting.
ReadWriteDiskValidator	ReadWriteDiskValidator is the class to check a directory by to create a file, write some bytes into it, read back, and verify if they are identical.
ReadWriteDiskValidatorMetrics	The metrics for a directory generated by ReadWriteDiskValidator.
ReflectionUtils	General reflection utils
RMVolumeShardingUtil
RunJar	Run a Hadoop job jar.
SemaphoredDelegatingExecutor	This ExecutorService blocks the submission of new tasks when its queue is already full by using a semaphore.
SequentialNumber	Sequential number generator.
ServletUtil
Sets	Static utility methods pertaining to Set instances.
Shell	A base class for running a Shell command.
Shell.ShellCommandExecutor	A simple shell command executor.
ShutdownHookManager	The ShutdownHookManager enables running shutdownHook in a deterministic order, higher priority first.
ShutdownThreadsHelper	Helper class to shutdown Threads and ExecutorServices.
StopWatch	A simplified StopWatch implementation which can measure times in nanoseconds.
StringInterner	Provides string interning utility methods.
StringUtils	General string utils
SysInfo	Plugin to calculate resource information on the system.
SysInfoLinux	Plugin to calculate resource information on Linux systems.
SysInfoWindows	Plugin to calculate resource information on Windows systems.
ThreadUtil
Time	Utility methods for getting the time and computing intervals.
Timer	Utility methods for getting the time and computing intervals.
ToolRunner	A utility to help run Tools.
UTF8ByteArrayUtils
VersionInfo	This class returns build information about Hadoop components.
VersionUtil	A wrapper class to maven's ComparableVersion class, to comply with maven's version name string convention
Waitable<T>	Represents an object that you can wait for.
WeakReferenceMap<K,V>	A map of keys type K to objects of type V which uses weak references, so does lot leak memory through long-lived references at the expense of losing references when GC takes place..
XMLUtils	General xml utilities.
ZKUtil	Utilities for working with ZooKeeper.
ZKUtil.ZKAuthInfo	An authentication token passed to ZooKeeper.addAuthInfo

Enum Summary

Enum	Description
DataChecksum.Type	The checksum types
Shell.OSType
SignalLogger	This class logs a message whenever we're about to exit on a UNIX signal.
StringUtils.TraditionalBinaryPrefix	The traditional binary prefixes, kilo, mega, ..., exa, which can be represented by a 64-bit integer.

Exception Summary

Exception	Description
DiskChecker.DiskErrorException
DiskChecker.DiskOutOfSpaceException
ExitUtil.ExitException	An exception raised when a call to ExitUtil.terminate(int) was called and system exits were blocked.
ExitUtil.HaltException	An exception raised when a call to ExitUtil.terminate(int) was called and system halts were blocked.
InvalidChecksumSizeException	Thrown when bytesPerChecksun field in the meta file is less than or equal to 0 or type is invalid.
MapRCommonSecurityException
Shell.ExitCodeException	This is an IOException with exit code added.
ZKUtil.BadAclFormatException
ZKUtil.BadAuthFormatException