Package org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources

Class CGroupsResourceCalculator

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsResourceCalculator

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class CGroupsResourceCalculator
extends org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

A cgroups file-system based Resource calculator without the process tree features. CGroups has its limitations. It can only be enabled, if both CPU and memory cgroups are enabled with yarn.nodemanager.resource.cpu.enabled and yarn.nodemanager.resource.memory.enabled respectively. This means that memory limits are enforced by default. You can turn this off and keep memory reporting only with yarn.nodemanager.resource.memory.enforced. Another limitation is virtual memory measurement. CGroups does not have the ability to measure virtual memory usage. This includes memory reserved but not used. CGroups measures used memory as sa sum of physical memory and swap usage. This will be returned in the virtual memory counters. If the real virtual memory is required please use the legacy procfs based resource calculator or CombinedResourceCalculator.

Field Summary

Fields

Modifier and Type	Field	Description
protected static org.slf4j.Logger	LOG

Fields inherited from class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

UNAVAILABLE

Constructor Summary

Constructors

Constructor	Description
CGroupsResourceCalculator()	Create resource calculator for all Yarn containers.
CGroupsResourceCalculator(java.lang.String pid)	Create resource calculator for the container that has the specified pid.

Method Summary

Modifier and Type	Method	Description
boolean	checkPidPgrpidForMatch()
float	getCpuUsagePercent()
long	getCumulativeCpuTime()
java.lang.String	getProcessTreeDump()
long	getRssMemorySize(int olderThanAge)
long	getVirtualMemorySize(int olderThanAge)
void	initialize()
static boolean	isAvailable()	Checks if the CGroupsResourceCalculator is available on this system.
void	updateProcessTree()

Methods inherited from class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

getResourceCalculatorProcessTree, getRssMemorySize, getVirtualMemorySize

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

CGroupsResourceCalculator

public CGroupsResourceCalculator()
                          throws org.apache.hadoop.yarn.exceptions.YarnException

Create resource calculator for all Yarn containers.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

CGroupsResourceCalculator

public CGroupsResourceCalculator(java.lang.String pid)

Create resource calculator for the container that has the specified pid.

Parameters:

pid - A pid from the cgroup or null for all containers

Method Detail

initialize

public void initialize()
                throws org.apache.hadoop.yarn.exceptions.YarnException

Overrides:

initialize in class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

getCpuUsagePercent

public float getCpuUsagePercent()

Overrides:

getCpuUsagePercent in class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

getCumulativeCpuTime

public long getCumulativeCpuTime()

Overrides:

getCumulativeCpuTime in class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

getRssMemorySize

public long getRssMemorySize(int olderThanAge)

Overrides:

getRssMemorySize in class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

getVirtualMemorySize

public long getVirtualMemorySize(int olderThanAge)

Overrides:

getVirtualMemorySize in class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

updateProcessTree

public void updateProcessTree()

Specified by:

updateProcessTree in class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

getProcessTreeDump

public java.lang.String getProcessTreeDump()

Specified by:

getProcessTreeDump in class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

checkPidPgrpidForMatch

public boolean checkPidPgrpidForMatch()

Specified by:

checkPidPgrpidForMatch in class org.apache.hadoop.yarn.util.ResourceCalculatorProcessTree

isAvailable

public static boolean isAvailable()

Checks if the CGroupsResourceCalculator is available on this system. This assumes that Linux container executor is already initialized.

Returns:

true if CGroupsResourceCalculator is available. False otherwise.