org.apache.hadoop.yarn.util.resource

Class DiskBasedDominantResourceCalculator

java.lang.Object

org.apache.hadoop.yarn.util.resource.ResourceCalculator

org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

org.apache.hadoop.yarn.util.resource.DiskBasedDominantResourceCalculator

public class DiskBasedDominantResourceCalculator
extends DominantResourceCalculator

Constructor Summary

Constructors

Constructor and Description
DiskBasedDominantResourceCalculator()

Method Summary

Methods

Modifier and Type	Method and Description
int	computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available, org.apache.hadoop.yarn.api.records.Resource required) Compute the number of containers which can be allocated given available and required resources.
org.apache.hadoop.yarn.api.records.Resource	divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, int denominator) Divide-and-ceil numerator by denominator.
static double	divideAndCeilDouble(double a, double b)
static double	divideAndFloorDouble(double a, double b)
protected float	getResourceAsValue(org.apache.hadoop.yarn.api.records.Resource clusterResource, org.apache.hadoop.yarn.api.records.Resource resource, boolean dominant) Use 'dominant' for now since we only have 2 resources - gives us a slight performance boost.
org.apache.hadoop.yarn.api.records.Resource	multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor) Multiply resource r by factor by and normalize down using step-factor stepFactor.
org.apache.hadoop.yarn.api.records.Resource	multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor) Multiply resource r by factor by and normalize up using step-factor stepFactor.
org.apache.hadoop.yarn.api.records.Resource	normalize(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.Resource stepFactor) Normalize resource r given the base minimumResource and verify against max allowed maximumResource using a step factor for hte normalization.
float	ratio(org.apache.hadoop.yarn.api.records.Resource a, org.apache.hadoop.yarn.api.records.Resource b) Ratio of resource a to resource b.
org.apache.hadoop.yarn.api.records.Resource	roundDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor) Round-down resource r given factor stepFactor.
static double	roundDownDisk(double a, double b)
org.apache.hadoop.yarn.api.records.Resource	roundUp(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor) Round-up resource r given factor stepFactor.
static double	roundUpDisk(double a, double b)

Methods inherited from class org.apache.hadoop.yarn.util.resource.DominantResourceCalculator

compare, divide

Methods inherited from class org.apache.hadoop.yarn.util.resource.ResourceCalculator

divideAndCeil, normalize, roundDown, roundUp

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DiskBasedDominantResourceCalculator

public DiskBasedDominantResourceCalculator()

Method Detail

divideAndCeilDouble

public static double divideAndCeilDouble(double a,
                         double b)

roundUpDisk

public static double roundUpDisk(double a,
                 double b)

divideAndFloorDouble

public static double divideAndFloorDouble(double a,
                          double b)

roundDownDisk

public static double roundDownDisk(double a,
                   double b)

getResourceAsValue

protected float getResourceAsValue(org.apache.hadoop.yarn.api.records.Resource clusterResource,
                       org.apache.hadoop.yarn.api.records.Resource resource,
                       boolean dominant)

Use 'dominant' for now since we only have 2 resources - gives us a slight performance boost. Once we add more resources, we'll need a more complicated (and slightly less performant algorithm).

Overrides:

getResourceAsValue in class DominantResourceCalculator

computeAvailableContainers

public int computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available,
                             org.apache.hadoop.yarn.api.records.Resource required)

Description copied from class: ResourceCalculator

Compute the number of containers which can be allocated given available and required resources.

Overrides:

computeAvailableContainers in class DominantResourceCalculator

Parameters:

available - available resources

required - required resources

Returns:

number of containers which can be allocated

ratio

public float ratio(org.apache.hadoop.yarn.api.records.Resource a,
          org.apache.hadoop.yarn.api.records.Resource b)

Description copied from class: ResourceCalculator

Ratio of resource a to resource b.

Overrides:

ratio in class DominantResourceCalculator

Parameters:

a - resource

b - resource

Returns:

ratio of resource a to resource b

divideAndCeil

public org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
                                                        int denominator)

Description copied from class: ResourceCalculator

Divide-and-ceil numerator by denominator.

Overrides:

divideAndCeil in class DominantResourceCalculator

Parameters:

numerator - numerator resource

denominator - denominator

Returns:

resultant resource

normalize

public org.apache.hadoop.yarn.api.records.Resource normalize(org.apache.hadoop.yarn.api.records.Resource r,
                                                    org.apache.hadoop.yarn.api.records.Resource minimumResource,
                                                    org.apache.hadoop.yarn.api.records.Resource maximumResource,
                                                    org.apache.hadoop.yarn.api.records.Resource stepFactor)

Description copied from class: ResourceCalculator

Normalize resource r given the base minimumResource and verify against max allowed maximumResource using a step factor for hte normalization.

Overrides:

normalize in class DominantResourceCalculator

Parameters:

r - resource

minimumResource - minimum value

maximumResource - the upper bound of the resource to be allocated

stepFactor - the increment for resources to be allocated

Returns:

normalized resource

roundUp

public org.apache.hadoop.yarn.api.records.Resource roundUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                  org.apache.hadoop.yarn.api.records.Resource stepFactor)

Description copied from class: ResourceCalculator

Round-up resource r given factor stepFactor.

Overrides:

roundUp in class DominantResourceCalculator

Parameters:

r - resource

stepFactor - step-factor

Returns:

rounded resource

roundDown

public org.apache.hadoop.yarn.api.records.Resource roundDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                    org.apache.hadoop.yarn.api.records.Resource stepFactor)

Description copied from class: ResourceCalculator

Round-down resource r given factor stepFactor.

Overrides:

roundDown in class DominantResourceCalculator

Parameters:

r - resource

stepFactor - step-factor

Returns:

rounded resource

multiplyAndNormalizeUp

public org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
                                                                 double by,
                                                                 org.apache.hadoop.yarn.api.records.Resource stepFactor)

Description copied from class: ResourceCalculator

Multiply resource r by factor by and normalize up using step-factor stepFactor.

Overrides:

multiplyAndNormalizeUp in class DominantResourceCalculator

Parameters:

r - resource to be multiplied

by - multiplier

stepFactor - factor by which to normalize up

Returns:

resulting normalized resource

multiplyAndNormalizeDown

public org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
                                                                   double by,
                                                                   org.apache.hadoop.yarn.api.records.Resource stepFactor)

Description copied from class: ResourceCalculator

Multiply resource r by factor by and normalize down using step-factor stepFactor.

Overrides:

multiplyAndNormalizeDown in class DominantResourceCalculator

Parameters:

r - resource to be multiplied

by - multiplier

stepFactor - factor by which to normalize down

Returns:

resulting normalized resource