Package org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources

Class CGroupsCpuResourceHandlerImpl

java.lang.Object

org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsCpuResourceHandlerImpl

All Implemented Interfaces:

CpuResourceHandler, ResourceHandler

@Unstable
@Private
public class CGroupsCpuResourceHandlerImpl
extends java.lang.Object
implements CpuResourceHandler

An implementation for using CGroups to restrict CPU usage on Linux. The implementation supports 3 different controls - restrict usage of all YARN containers, restrict relative usage of individual YARN containers and restrict usage of individual YARN containers. Admins can set the overall CPU to be used by all YARN containers - this is implemented by setting cpu.cfs_period_us and cpu.cfs_quota_us to the ratio desired. If strict resource usage mode is not enabled, cpu.shares is set for individual containers - this prevents containers from exceeding the overall limit for YARN containers but individual containers can use as much of the CPU as available(under the YARN limit). If strict resource usage is enabled, then container can only use the percentage of CPU allocated to them and this is again implemented using cpu.cfs_period_us and cpu.cfs_quota_us.

Method Summary

Modifier and Type	Method	Description
java.util.List<PrivilegedOperation>	bootstrap(org.apache.hadoop.conf.Configuration conf)	Bootstrap resource susbsystem.
static boolean	cpuLimitsExist(java.lang.String path)
static int[]	getOverallLimits(float yarnProcessors)
java.util.List<PrivilegedOperation>	postComplete(org.apache.hadoop.yarn.api.records.ContainerId containerId)	Perform any tasks necessary after container completion.
java.util.List<PrivilegedOperation>	preStart(Container container)	Prepare a resource environment for container launch
java.util.List<PrivilegedOperation>	reacquireContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)	Require state for container that was already launched
java.util.List<PrivilegedOperation>	teardown()	Teardown environment for resource subsystem if requested.
java.lang.String	toString()
java.util.List<PrivilegedOperation>	updateContainer(Container container)	Update state for container that was already launched

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

bootstrap

public java.util.List<PrivilegedOperation> bootstrap(org.apache.hadoop.conf.Configuration conf)
                                              throws ResourceHandlerException

Description copied from interface: ResourceHandler

Bootstrap resource susbsystem.

Specified by:

bootstrap in interface ResourceHandler

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

cpuLimitsExist

@Private
public static boolean cpuLimitsExist(java.lang.String path)
                              throws java.io.IOException

Throws:

java.io.IOException

getOverallLimits

@VisibleForTesting
@Private
public static int[] getOverallLimits(float yarnProcessors)

preStart

public java.util.List<PrivilegedOperation> preStart(Container container)
                                             throws ResourceHandlerException

Description copied from interface: ResourceHandler

Prepare a resource environment for container launch

Specified by:

preStart in interface ResourceHandler

Parameters:

container - Container being launched

Returns:

(possibly empty) list of operations that require elevated privileges e.g a) create a custom cgroup b) add pid for container to tasks file for a cgroup.

Throws:

ResourceHandlerException

reacquireContainer

public java.util.List<PrivilegedOperation> reacquireContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                                                       throws ResourceHandlerException

Description copied from interface: ResourceHandler

Require state for container that was already launched

Specified by:

reacquireContainer in interface ResourceHandler

Parameters:

containerId - id of the container being reacquired.

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

updateContainer

public java.util.List<PrivilegedOperation> updateContainer(Container container)
                                                    throws ResourceHandlerException

Description copied from interface: ResourceHandler

Update state for container that was already launched

Specified by:

updateContainer in interface ResourceHandler

Parameters:

container - the container being updated.

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

postComplete

public java.util.List<PrivilegedOperation> postComplete(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                                                 throws ResourceHandlerException

Description copied from interface: ResourceHandler

Perform any tasks necessary after container completion.

Specified by:

postComplete in interface ResourceHandler

Parameters:

containerId - of the container that was completed.

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

teardown

public java.util.List<PrivilegedOperation> teardown()
                                             throws ResourceHandlerException

Description copied from interface: ResourceHandler

Teardown environment for resource subsystem if requested. This method needs to be used with care since it could impact running containers.

Specified by:

teardown in interface ResourceHandler

Returns:

(possibly empty) list of operations that require elevated privileges

Throws:

ResourceHandlerException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object