Package org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime

Class RuncContainerRuntime

java.lang.Object

org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.OCIContainerRuntime

org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.RuncContainerRuntime

All Implemented Interfaces:

LinuxContainerRuntime, ContainerRuntime

@Private
@Unstable
public class RuncContainerRuntime
extends OCIContainerRuntime

This class is an extension of OCIContainerRuntime that uses the native container-executor binary via a PrivilegedOperationExecutor instance to launch processes inside Runc containers.

The following environment variables are used to configure the Runc engine:

• YARN_CONTAINER_RUNTIME_TYPE ultimately determines whether a runC container will be used. If the value is runc, a runC container will be used. Otherwise a regular process tree container will be used. This environment variable is checked by the isRuncContainerRequested(org.apache.hadoop.conf.Configuration, java.util.Map<java.lang.String, java.lang.String>) method, which is called by the DelegatingLinuxContainerRuntime.
• YARN_CONTAINER_RUNTIME_RUNC_IMAGE names which image will be used to launch the Runc container.
• YARN_CONTAINER_RUNTIME_RUNC_MOUNTS allows users to specify additional volume mounts for the runC container. The value of the environment variable should be a comma-separated list of mounts. All such mounts must be given as source:dest[:mode] and the mode must be "ro" (read-only) or "rw" (read-write) to specify the type of access being requested. If neither is specified, read-write will be assumed. The requested mounts will be validated by container-executor based on the values set in container-executor.cfg for runc.allowed.ro-mounts and runc.allowed.rw-mounts.
• YARN_CONTAINER_RUNTIME_RUNC_CONTAINER_HOSTNAME sets the hostname to be used by the Runc container. If not specified, a hostname will be derived from the container ID and set as default hostname for networks other than 'host'.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	ENV_RUNC_CONTAINER_HOSTNAME
static java.lang.String	ENV_RUNC_CONTAINER_IMAGE
static java.lang.String	ENV_RUNC_CONTAINER_MOUNTS
static java.lang.String	ENV_RUNC_CONTAINER_PID_NAMESPACE
static java.lang.String	ENV_RUNC_CONTAINER_RUN_PRIVILEGED_CONTAINER

Fields inherited from class org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.OCIContainerRuntime

CONTAINER_PID_NAMESPACE_SUFFIX, RUN_PRIVILEGED_CONTAINER_SUFFIX, RUNTIME_PREFIX

Constructor Summary

Constructors

Constructor	Description
RuncContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor)
RuncContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor, CGroupsHandler cGroupsHandler)

Method Summary

Modifier and Type	Method	Description
protected RuncImageTagToManifestPlugin	chooseImageTagToManifestPlugin()
protected RuncManifestToResourcesPlugin	chooseManifestToResourcesPlugin()
org.apache.hadoop.hdfs.protocol.datatransfer.IOStreamPair	execContainer(ContainerExecContext ctx)	Run a program in container.
protected java.util.List<java.lang.String>	extractImageEntrypoint(java.io.File config)
protected java.util.List<java.lang.String>	extractImageEnv(java.io.File config)
java.lang.String	getExposedPorts(Container container)	Return the exposed ports of the container.
java.lang.String[]	getIpAndHost(Container container)	Return the host and ip of the container.
java.util.Map<java.lang.String,org.apache.hadoop.yarn.api.records.LocalResource>	getLocalResources(Container container)
void	initialize(org.apache.hadoop.conf.Configuration configuration, Context nmCtx)	Initialize the runtime.
static boolean	isRuncContainerRequested(org.apache.hadoop.conf.Configuration daemonConf, java.util.Map<java.lang.String,java.lang.String> env)	Return whether the given environment variables indicate that the operation is requesting a Runc container.
boolean	isRuntimeRequested(java.util.Map<java.lang.String,java.lang.String> env)	Return whether the given environment variables indicate that the operation is requesting this runtime.
void	launchContainer(ContainerRuntimeContext ctx)	Launch a container.
void	reapContainer(ContainerRuntimeContext ctx)	Perform any container cleanup that may be required.
void	relaunchContainer(ContainerRuntimeContext ctx)	Relaunch a container.
void	signalContainer(ContainerRuntimeContext ctx)	Signal a container.
void	start()
void	stop()
java.lang.String	writeCommandToFile(RuncContainerExecutorConfig runcContainerExecutorConfig, Container container)

Methods inherited from class org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.OCIContainerRuntime

allowHostPidNamespace, allowPrivilegedContainerExecution, formatOciEnvKey, getCsiClients, getGroupIdInfo, getUserIdInfo, initiateCsiClients, isContainerRequestedAsPrivileged, isOCICompliantContainerRequested, mountReadOnlyPath, prepareContainer, validateContainerNetworkType, validateContainerRuntimeType, validateHostname

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ENV_RUNC_CONTAINER_IMAGE

@Private
public static final java.lang.String ENV_RUNC_CONTAINER_IMAGE

See Also:

Constant Field Values

ENV_RUNC_CONTAINER_MOUNTS

@Private
public static final java.lang.String ENV_RUNC_CONTAINER_MOUNTS

See Also:

Constant Field Values

ENV_RUNC_CONTAINER_HOSTNAME

@Private
public static final java.lang.String ENV_RUNC_CONTAINER_HOSTNAME

See Also:

Constant Field Values

ENV_RUNC_CONTAINER_PID_NAMESPACE

@Private
public static final java.lang.String ENV_RUNC_CONTAINER_PID_NAMESPACE

ENV_RUNC_CONTAINER_RUN_PRIVILEGED_CONTAINER

@Private
public static final java.lang.String ENV_RUNC_CONTAINER_RUN_PRIVILEGED_CONTAINER

Constructor Detail

RuncContainerRuntime

public RuncContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor)

RuncContainerRuntime

@VisibleForTesting
public RuncContainerRuntime(PrivilegedOperationExecutor privilegedOperationExecutor,
                            CGroupsHandler cGroupsHandler)

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration configuration,
                       Context nmCtx)
                throws ContainerExecutionException

Description copied from interface: LinuxContainerRuntime

Initialize the runtime.

Specified by:

initialize in interface LinuxContainerRuntime

Overrides:

initialize in class OCIContainerRuntime

Parameters:

configuration - the Configuration to use

nmCtx - NMContext

Throws:

ContainerExecutionException - if an error occurs while initializing the runtime

start

public void start()

stop

public void stop()

launchContainer

public void launchContainer(ContainerRuntimeContext ctx)
                     throws ContainerExecutionException

Description copied from interface: ContainerRuntime

Launch a container.

Parameters:

ctx - the ContainerRuntimeContext

Throws:

ContainerExecutionException - if an error occurs while launching the container

getLocalResources

public java.util.Map<java.lang.String,org.apache.hadoop.yarn.api.records.LocalResource> getLocalResources(Container container)
                                                                                                         throws java.io.IOException

Throws:

java.io.IOException

chooseImageTagToManifestPlugin

protected RuncImageTagToManifestPlugin chooseImageTagToManifestPlugin()
                                                               throws ContainerExecutionException

Throws:

ContainerExecutionException

chooseManifestToResourcesPlugin

protected RuncManifestToResourcesPlugin chooseManifestToResourcesPlugin()
                                                                 throws ContainerExecutionException

Throws:

ContainerExecutionException

extractImageEnv

protected java.util.List<java.lang.String> extractImageEnv(java.io.File config)
                                                    throws java.io.IOException

Throws:

java.io.IOException

extractImageEntrypoint

protected java.util.List<java.lang.String> extractImageEntrypoint(java.io.File config)
                                                           throws java.io.IOException

Throws:

java.io.IOException

writeCommandToFile

public java.lang.String writeCommandToFile(RuncContainerExecutorConfig runcContainerExecutorConfig,
                                           Container container)
                                    throws ContainerExecutionException

Throws:

ContainerExecutionException

getExposedPorts

public java.lang.String getExposedPorts(Container container)

Description copied from interface: ContainerRuntime

Return the exposed ports of the container.

Parameters:

container - the Container

Returns:

List of exposed ports

getIpAndHost

public java.lang.String[] getIpAndHost(Container container)

Description copied from interface: ContainerRuntime

Return the host and ip of the container.

Parameters:

container - the Container

execContainer

public org.apache.hadoop.hdfs.protocol.datatransfer.IOStreamPair execContainer(ContainerExecContext ctx)
                                                                        throws ContainerExecutionException

Description copied from interface: ContainerRuntime

Run a program in container.

Parameters:

ctx - the ContainerExecContext

Returns:

stdin and stdout of container exec

Throws:

ContainerExecutionException

reapContainer

public void reapContainer(ContainerRuntimeContext ctx)
                   throws ContainerExecutionException

Description copied from interface: ContainerRuntime

Perform any container cleanup that may be required.

Parameters:

ctx - the ContainerRuntimeContext

Throws:

ContainerExecutionException - if an error occurs while reaping the container

relaunchContainer

public void relaunchContainer(ContainerRuntimeContext ctx)
                       throws ContainerExecutionException

Description copied from interface: ContainerRuntime

Relaunch a container.

Parameters:

ctx - the ContainerRuntimeContext

Throws:

ContainerExecutionException - if an error occurs while relaunching the container

isRuncContainerRequested

public static boolean isRuncContainerRequested(org.apache.hadoop.conf.Configuration daemonConf,
                                               java.util.Map<java.lang.String,java.lang.String> env)

Return whether the given environment variables indicate that the operation is requesting a Runc container. If the environment contains a key called YARN_CONTAINER_RUNTIME_TYPE whose value is runc, this method will return true. Otherwise it will return false.

Parameters:

daemonConf - the NodeManager daemon configuration

env - the environment variable settings for the operation

Returns:

whether a Runc container is requested

isRuntimeRequested

public boolean isRuntimeRequested(java.util.Map<java.lang.String,java.lang.String> env)

Description copied from interface: LinuxContainerRuntime

Return whether the given environment variables indicate that the operation is requesting this runtime.

Parameters:

env - the environment variable settings for the operation

Returns:

whether this runtime is requested

signalContainer

public void signalContainer(ContainerRuntimeContext ctx)
                     throws ContainerExecutionException

Description copied from interface: ContainerRuntime

Signal a container. Signals may be a request to terminate, a status check, etc.

Parameters:

ctx - the ContainerRuntimeContext

Throws:

ContainerExecutionException - if an error occurs while signaling the container