Package org.apache.hadoop.fs.statistics.impl

Class IOStatisticsBinding

java.lang.Object

org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding

public final class IOStatisticsBinding
extends java.lang.Object

Support for implementing IOStatistics interfaces.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	ENTRY_PATTERN	Pattern used for each entry.
static java.lang.String	NULL_SOURCE	String to return when a source is null.

Method Summary

Modifier and Type	Method	Description
static java.lang.Long	aggregateCounters(java.lang.Long l, java.lang.Long r)	Aggregate two counters.
static java.lang.Long	aggregateGauges(java.lang.Long l, java.lang.Long r)	Add two gauges.
static <E> void	aggregateMaps(java.util.Map<java.lang.String,E> dest, java.util.Map<java.lang.String,E> other, java.util.function.BiFunction<E,E,E> aggregateFn, java.util.function.Function<E,E> copyFn)	Aggregate two maps so that the destination.
static java.lang.Long	aggregateMaximums(java.lang.Long l, java.lang.Long r)	Aggregate two maximum values.
static MeanStatistic	aggregateMeanStatistics(MeanStatistic l, MeanStatistic r)	Aggregate the mean statistics.
static java.lang.Long	aggregateMinimums(java.lang.Long l, java.lang.Long r)	Aggregate two minimum values.
static DurationTracker	createTracker(DurationTrackerFactory factory, java.lang.String statistic)	Create the tracker.
static DynamicIOStatisticsBuilder	dynamicIOStatistics()	Create a builder for dynamic IO Statistics.
static IOStatistics	emptyStatistics()	Get the shared instance of the immutable empty statistics object.
static IOStatisticsStore	emptyStatisticsStore()	Get the shared instance of the immutable empty statistics store.
static <E> java.lang.String	entryToString(java.lang.String name, E value)	Convert entry values to the string format used in logging.
static <E> java.lang.String	entryToString(java.util.Map.Entry<java.lang.String,E> entry)	Convert an entry to the string format used in logging.
static IOStatistics	fromStorageStatistics(StorageStatistics storageStatistics)	Create IOStatistics from a storage statistics instance.
static <B> B	invokeTrackingDuration(DurationTracker tracker, CallableRaisingIOE<B> input)	Given an IOException raising callable/lambda expression, execute it, updating the tracker on success/failure.
static IOStatisticsStoreBuilder	iostatisticsStore()	Create a builder for an IOStatisticsStore.
static void	maybeUpdateMaximum(java.util.concurrent.atomic.AtomicLong dest, long sample)	Update a maximum value tracked in an atomic long.
static void	maybeUpdateMinimum(java.util.concurrent.atomic.AtomicLong dest, long sample)	Update a maximum value tracked in an atomic long.
static java.time.Duration	measureDurationOfInvocation(DurationTrackerFactory factory, java.lang.String statistic, InvocationRaisingIOE input)	Given an IOException raising callable/lambda expression, execute it and update the relevant statistic, returning the measured duration.
static DurationTrackerFactory	pairedTrackerFactory(DurationTrackerFactory first, DurationTrackerFactory second)	Create a DurationTrackerFactory which aggregates the tracking of two other factories.
static <E extends java.io.Serializable> E	passthroughFn(E src)	A passthrough copy operation suitable for immutable types, including numbers.
static StorageStatistics	publishAsStorageStatistics(java.lang.String name, java.lang.String scheme, IOStatistics source)	Publish the IOStatistics as a set of storage statistics.
static <E extends java.io.Serializable> java.util.Map<java.lang.String,E>	snapshotMap(java.util.Map<java.lang.String,E> source)	Take a snapshot of a supplied map, where the copy option simply uses the existing value.
static <E extends java.io.Serializable> java.util.concurrent.ConcurrentHashMap<java.lang.String,E>	snapshotMap(java.util.Map<java.lang.String,E> source, java.util.function.Function<E,E> copyFn)	Take a snapshot of a supplied map, using the copy function to replicate the source values.
static <B> B	trackDuration(DurationTrackerFactory factory, java.lang.String statistic, CallableRaisingIOE<B> input)	Given an IOException raising callable/lambda expression, execute it and update the relevant statistic.
static <B> ConsumerRaisingIOE<B>	trackDurationConsumer(DurationTrackerFactory factory, java.lang.String statistic, ConsumerRaisingIOE<B> input)	Given an IOException raising Consumer, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
static <B> java.util.concurrent.Callable<B>	trackDurationOfCallable(DurationTrackerFactory factory, java.lang.String statistic, java.util.concurrent.Callable<B> input)	Given a callable/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
static void	trackDurationOfInvocation(DurationTrackerFactory factory, java.lang.String statistic, InvocationRaisingIOE input)	Given an IOException raising callable/lambda expression, execute it and update the relevant statistic.
static <B> CallableRaisingIOE<B>	trackDurationOfOperation(DurationTrackerFactory factory, java.lang.String statistic, CallableRaisingIOE<B> input)	Given an IOException raising callable/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
static <B> B	trackDurationOfSupplier(DurationTrackerFactory factory, java.lang.String statistic, java.util.function.Supplier<B> input)	Given a Java supplier, evaluate it while tracking the duration of the operation and success/failure.
static <A,B> FunctionRaisingIOE<A,B>	trackFunctionDuration(DurationTrackerFactory factory, java.lang.String statistic, FunctionRaisingIOE<A,B> inputFn)	Given an IOException raising function/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
static <A,B> java.util.function.Function<A,B>	trackJavaFunctionDuration(DurationTrackerFactory factory, java.lang.String statistic, java.util.function.Function<A,B> inputFn)	Given a java function/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.
static IOStatisticsSource	wrap(IOStatistics statistics)	Take an IOStatistics instance and wrap it in a source.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ENTRY_PATTERN

public static final java.lang.String ENTRY_PATTERN

Pattern used for each entry.

See Also:

Constant Field Values

NULL_SOURCE

@VisibleForTesting
public static final java.lang.String NULL_SOURCE

String to return when a source is null.

See Also:

Constant Field Values

Method Detail

fromStorageStatistics

public static IOStatistics fromStorageStatistics(StorageStatistics storageStatistics)

Create IOStatistics from a storage statistics instance. This will be updated as the storage statistics change.

Parameters:

storageStatistics - source data.

Returns:

an IO statistics source.

dynamicIOStatistics

public static DynamicIOStatisticsBuilder dynamicIOStatistics()

Create a builder for dynamic IO Statistics.

Returns:

a builder to be completed.

emptyStatistics

public static IOStatistics emptyStatistics()

Get the shared instance of the immutable empty statistics object.

Returns:

an empty statistics object.

emptyStatisticsStore

public static IOStatisticsStore emptyStatisticsStore()

Get the shared instance of the immutable empty statistics store.

Returns:

an empty statistics object.

wrap

public static IOStatisticsSource wrap(IOStatistics statistics)

Take an IOStatistics instance and wrap it in a source.

Parameters:

statistics - statistics.

Returns:

a source which will return the values

iostatisticsStore

public static IOStatisticsStoreBuilder iostatisticsStore()

Create a builder for an IOStatisticsStore.

Returns:

a builder instance.

entryToString

public static <E> java.lang.String entryToString(java.util.Map.Entry<java.lang.String,E> entry)

Convert an entry to the string format used in logging.

Type Parameters:

E - entry type

Parameters:

entry - entry to evaluate

Returns:

formatted string

entryToString

public static <E> java.lang.String entryToString(java.lang.String name,
                                                 E value)

Convert entry values to the string format used in logging.

Type Parameters:

E - type of values.

Parameters:

name - statistic name

value - stat value

Returns:

formatted string

passthroughFn

public static <E extends java.io.Serializable> E passthroughFn(E src)

A passthrough copy operation suitable for immutable types, including numbers.

Type Parameters:

E - type of values.

Parameters:

src - source object

Returns:

the source object

snapshotMap

public static <E extends java.io.Serializable> java.util.Map<java.lang.String,E> snapshotMap(java.util.Map<java.lang.String,E> source)

Take a snapshot of a supplied map, where the copy option simply uses the existing value. For this to be safe, the map must refer to immutable objects.

Type Parameters:

E - type of values.

Parameters:

source - source map

Returns:

a new map referencing the same values.

snapshotMap

public static <E extends java.io.Serializable> java.util.concurrent.ConcurrentHashMap<java.lang.String,E> snapshotMap(java.util.Map<java.lang.String,E> source,
                                                                                                                            java.util.function.Function<E,E> copyFn)

Take a snapshot of a supplied map, using the copy function to replicate the source values.

Type Parameters:

E - type of values.

Parameters:

source - source map

copyFn - function to copy the value

Returns:

a concurrent hash map referencing the same values.

aggregateMaps

public static <E> void aggregateMaps(java.util.Map<java.lang.String,E> dest,
                                     java.util.Map<java.lang.String,E> other,
                                     java.util.function.BiFunction<E,E,E> aggregateFn,
                                     java.util.function.Function<E,E> copyFn)

Aggregate two maps so that the destination.

Type Parameters:

E - type of values

Parameters:

dest - destination map.

other - other map

aggregateFn - function to aggregate the values.

copyFn - function to copy the value

aggregateCounters

public static java.lang.Long aggregateCounters(java.lang.Long l,
                                               java.lang.Long r)

Aggregate two counters.

Parameters:

l - left value

r - right value

Returns:

the aggregate value

aggregateGauges

public static java.lang.Long aggregateGauges(java.lang.Long l,
                                             java.lang.Long r)

Add two gauges.

Parameters:

l - left value

r - right value

Returns:

aggregate value

aggregateMinimums

public static java.lang.Long aggregateMinimums(java.lang.Long l,
                                               java.lang.Long r)

Aggregate two minimum values.

Parameters:

l - left

r - right

Returns:

the new minimum.

aggregateMaximums

public static java.lang.Long aggregateMaximums(java.lang.Long l,
                                               java.lang.Long r)

Aggregate two maximum values.

Parameters:

l - left

r - right

Returns:

the new minimum.

aggregateMeanStatistics

public static MeanStatistic aggregateMeanStatistics(MeanStatistic l,
                                                    MeanStatistic r)

Aggregate the mean statistics. This returns a new instance.

Parameters:

l - left value

r - right value

Returns:

aggregate value

maybeUpdateMaximum

public static void maybeUpdateMaximum(java.util.concurrent.atomic.AtomicLong dest,
                                      long sample)

Update a maximum value tracked in an atomic long. This is thread safe -it uses compareAndSet to ensure that Thread T1 whose sample is greater than the current value never overwrites an update from thread T2 whose sample was also higher -and which completed first.

Parameters:

dest - destination for all changes.

sample - sample to update.

maybeUpdateMinimum

public static void maybeUpdateMinimum(java.util.concurrent.atomic.AtomicLong dest,
                                      long sample)

Update a maximum value tracked in an atomic long. This is thread safe -it uses compareAndSet to ensure that Thread T1 whose sample is greater than the current value never overwrites an update from thread T2 whose sample was also higher -and which completed first.

Parameters:

dest - destination for all changes.

sample - sample to update.

trackFunctionDuration

public static <A,B> FunctionRaisingIOE<A,B> trackFunctionDuration(@Nullable
                                                                              DurationTrackerFactory factory,
                                                                              java.lang.String statistic,
                                                                              FunctionRaisingIOE<A,B> inputFn)

Given an IOException raising function/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.

Type Parameters:

A - type of argument to the input function.

B - return type.

Parameters:

factory - factory of duration trackers

statistic - statistic key

inputFn - input function

Returns:

a new function which tracks duration and failure.

trackJavaFunctionDuration

public static <A,B> java.util.function.Function<A,B> trackJavaFunctionDuration(@Nullable
                                                                                           DurationTrackerFactory factory,
                                                                                           java.lang.String statistic,
                                                                                           java.util.function.Function<A,B> inputFn)

Given a java function/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.

Type Parameters:

A - type of argument to the input function.

B - return type.

Parameters:

factory - factory of duration trackers

statistic - statistic key

inputFn - input function

Returns:

a new function which tracks duration and failure.

trackDuration

public static <B> B trackDuration(DurationTrackerFactory factory,
                                  java.lang.String statistic,
                                  CallableRaisingIOE<B> input)
                           throws java.io.IOException

Given an IOException raising callable/lambda expression, execute it and update the relevant statistic.

Type Parameters:

B - return type.

Parameters:

factory - factory of duration trackers

statistic - statistic key

input - input callable.

Returns:

the result of the operation.

Throws:

java.io.IOException - raised on errors performing I/O.

trackDurationOfInvocation

public static void trackDurationOfInvocation(DurationTrackerFactory factory,
                                             java.lang.String statistic,
                                             InvocationRaisingIOE input)
                                      throws java.io.IOException

Given an IOException raising callable/lambda expression, execute it and update the relevant statistic.

Parameters:

factory - factory of duration trackers

statistic - statistic key

input - input callable.

Throws:

java.io.IOException - IO failure.

measureDurationOfInvocation

public static java.time.Duration measureDurationOfInvocation(DurationTrackerFactory factory,
                                                             java.lang.String statistic,
                                                             InvocationRaisingIOE input)
                                                      throws java.io.IOException

Given an IOException raising callable/lambda expression, execute it and update the relevant statistic, returning the measured duration. trackDurationOfInvocation(DurationTrackerFactory, String, InvocationRaisingIOE) with the duration returned for logging etc.; added as a new method to avoid linking problems with any code calling the existing method.

Parameters:

factory - factory of duration trackers

statistic - statistic key

input - input callable.

Returns:

the duration of the operation, as measured by the duration tracker.

Throws:

java.io.IOException - IO failure.

trackDurationOfOperation

public static <B> CallableRaisingIOE<B> trackDurationOfOperation(@Nullable
                                                                 DurationTrackerFactory factory,
                                                                 java.lang.String statistic,
                                                                 CallableRaisingIOE<B> input)

Given an IOException raising callable/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.

Type Parameters:

B - return type.

Parameters:

factory - factory of duration trackers

statistic - statistic key

input - input callable.

Returns:

a new callable which tracks duration and failure.

invokeTrackingDuration

public static <B> B invokeTrackingDuration(DurationTracker tracker,
                                           CallableRaisingIOE<B> input)
                                    throws java.io.IOException

Given an IOException raising callable/lambda expression, execute it, updating the tracker on success/failure.

Type Parameters:

B - return type.

Parameters:

tracker - duration tracker.

input - input callable.

Returns:

the result of the invocation

Throws:

java.io.IOException - on failure.

trackDurationConsumer

public static <B> ConsumerRaisingIOE<B> trackDurationConsumer(@Nullable
                                                              DurationTrackerFactory factory,
                                                              java.lang.String statistic,
                                                              ConsumerRaisingIOE<B> input)

Given an IOException raising Consumer, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.

Type Parameters:

B - return type.

Parameters:

factory - factory of duration trackers

statistic - statistic key

input - input callable.

Returns:

a new consumer which tracks duration and failure.

trackDurationOfCallable

public static <B> java.util.concurrent.Callable<B> trackDurationOfCallable(@Nullable
                                                                           DurationTrackerFactory factory,
                                                                           java.lang.String statistic,
                                                                           java.util.concurrent.Callable<B> input)

Given a callable/lambda expression, return a new one which wraps the inner and tracks the duration of the operation, including whether it passes/fails.

Type Parameters:

B - return type.

Parameters:

factory - factory of duration trackers

statistic - statistic key

input - input callable.

Returns:

a new callable which tracks duration and failure.

trackDurationOfSupplier

public static <B> B trackDurationOfSupplier(@Nullable
                                            DurationTrackerFactory factory,
                                            java.lang.String statistic,
                                            java.util.function.Supplier<B> input)

Given a Java supplier, evaluate it while tracking the duration of the operation and success/failure.

Type Parameters:

B - return type.

Parameters:

factory - factory of duration trackers

statistic - statistic key

input - input callable.

Returns:

the output of the supplier.

createTracker

public static DurationTracker createTracker(@Nullable
                                            DurationTrackerFactory factory,
                                            java.lang.String statistic)

Create the tracker. If the factory is null, a stub tracker is returned.

Parameters:

factory - tracker factory

statistic - statistic to track

Returns:

a duration tracker.

pairedTrackerFactory

public static DurationTrackerFactory pairedTrackerFactory(DurationTrackerFactory first,
                                                          DurationTrackerFactory second)

Create a DurationTrackerFactory which aggregates the tracking of two other factories.

Parameters:

first - first tracker factory

second - second tracker factory

Returns:

a factory

publishAsStorageStatistics

public static StorageStatistics publishAsStorageStatistics(java.lang.String name,
                                                           java.lang.String scheme,
                                                           IOStatistics source)

Publish the IOStatistics as a set of storage statistics. This is dynamic.

Parameters:

name - storage statistics name.

scheme - FS scheme; may be null.

source - IOStatistics source.

Returns:

a dynamic storage statistics object.