Package org.apache.hadoop.hdfs.server.datanode.metrics

Class OutlierDetector

java.lang.Object

org.apache.hadoop.hdfs.server.datanode.metrics.OutlierDetector

@Private
@Unstable
public class OutlierDetector
extends java.lang.Object

A utility class to help detect resources (nodes/ disks) whose aggregate latency is an outlier within a given set. We use the median absolute deviation for outlier detection as described in the following publication: Leys, C., et al., Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. http://dx.doi.org/10.1016/j.jesp.2013.03.013 We augment the above scheme with the following heuristics to be even more conservative: 1. Skip outlier detection if the sample size is too small. 2. Never flag resources whose aggregate latency is below a low threshold. 3. Never flag resources whose aggregate latency is less than a small multiple of the median.

Field Summary

Fields

Modifier and Type	Field	Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
OutlierDetector(long minNumResources, long lowThresholdMs)

Method Summary

Modifier and Type	Method	Description
static java.lang.Double	computeMad(java.util.List<java.lang.Double> sortedValues)	Compute the Median Absolute Deviation of a sorted list.
static java.lang.Double	computeMedian(java.util.List<java.lang.Double> sortedValues)	Compute the median of a sorted list.
long	getLowThresholdMs()
long	getMinOutlierDetectionNodes()
java.util.Map<java.lang.String,org.apache.hadoop.hdfs.server.protocol.OutlierMetrics>	getOutlierMetrics(java.util.Map<java.lang.String,java.lang.Double> stats)	Return a set of nodes whose latency is much higher than their counterparts.
java.util.Map<java.lang.String,java.lang.Double>	getOutliers(java.util.Map<java.lang.String,java.lang.Double> stats)	Return a set of nodes/ disks whose latency is much higher than their counterparts.
void	setLowThresholdMs(long thresholdMs)
void	setMinNumResources(long minNodes)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

OutlierDetector

public OutlierDetector(long minNumResources,
                       long lowThresholdMs)

Method Detail

getOutliers

public java.util.Map<java.lang.String,java.lang.Double> getOutliers(java.util.Map<java.lang.String,java.lang.Double> stats)

Return a set of nodes/ disks whose latency is much higher than their counterparts. The input is a map of (resource -> aggregate latency) entries. The aggregate may be an arithmetic mean or a percentile e.g. 90th percentile. Percentiles are a better choice than median since latency is usually not a normal distribution. This method allocates temporary memory O(n) and has run time O(n.log(n)), where n = stats.size().

Returns:

getOutlierMetrics

public java.util.Map<java.lang.String,org.apache.hadoop.hdfs.server.protocol.OutlierMetrics> getOutlierMetrics(java.util.Map<java.lang.String,java.lang.Double> stats)

Return a set of nodes whose latency is much higher than their counterparts. The input is a map of (resource -> aggregate latency) entries. The aggregate may be an arithmetic mean or a percentile e.g. 90th percentile. Percentiles are a better choice than median since latency is usually not a normal distribution.

Parameters:

stats - map of aggregate latency entries.

Returns:

map of outlier nodes to outlier metrics.

computeMad

public static java.lang.Double computeMad(java.util.List<java.lang.Double> sortedValues)

Compute the Median Absolute Deviation of a sorted list.

computeMedian

public static java.lang.Double computeMedian(java.util.List<java.lang.Double> sortedValues)

Compute the median of a sorted list.

setMinNumResources

public void setMinNumResources(long minNodes)

getMinOutlierDetectionNodes

public long getMinOutlierDetectionNodes()

setLowThresholdMs

public void setLowThresholdMs(long thresholdMs)

getLowThresholdMs

public long getLowThresholdMs()