org.apache.hadoop.yarn.client.api.impl
Class NMClientImpl

java.lang.Object
  extended by org.apache.hadoop.service.AbstractService
      extended by org.apache.hadoop.yarn.client.api.NMClient
          extended by org.apache.hadoop.yarn.client.api.impl.NMClientImpl
All Implemented Interfaces:
Closeable, org.apache.hadoop.service.Service

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class NMClientImpl
extends NMClient

This class implements NMClient. All the APIs are blocking.

By default, this client stops all the running containers that are started by it when it stops. It can be disabled via cleanupRunningContainersOnStop(boolean), in which case containers will continue to run even after this client is stopped and till the application runs at which point ResourceManager will forcefully kill them.

Note that the blocking APIs ensure the RPC calls to NodeManager are executed immediately, and the responses are received before these APIs return. However, when startContainer(org.apache.hadoop.yarn.api.records.Container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext) or stopContainer(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId) returns, NodeManager may still need some time to either start or stop the container because of its asynchronous implementation. Therefore, getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId) is likely to return a transit container status if it is executed immediately after startContainer(org.apache.hadoop.yarn.api.records.Container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext) or stopContainer(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId).


Nested Class Summary
protected static class NMClientImpl.StartedContainer
           
 
Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service
org.apache.hadoop.service.Service.STATE
 
Field Summary
protected  ConcurrentMap<org.apache.hadoop.yarn.api.records.ContainerId,NMClientImpl.StartedContainer> startedContainers
           
 
Constructor Summary
NMClientImpl()
           
NMClientImpl(String name)
           
 
Method Summary
protected  void cleanupRunningContainers()
           
 void cleanupRunningContainersOnStop(boolean enabled)
          Set whether the containers that are started by this client, and are still running should be stopped when the client stops.
protected  NMClientImpl.StartedContainer createStartedContainer(org.apache.hadoop.yarn.api.records.Container container)
           
 AtomicBoolean getCleanupRunningContainers()
           
 org.apache.hadoop.yarn.api.records.ContainerStatus getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId)
          Query the status of a container.
protected  NMClientImpl.StartedContainer getStartedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
           
protected  void removeStartedContainer(NMClientImpl.StartedContainer container)
           
protected  void serviceInit(org.apache.hadoop.conf.Configuration conf)
           
protected  void serviceStop()
           
 Map<String,ByteBuffer> startContainer(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext)
          Start an allocated container.
 void stopContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.yarn.api.records.NodeId nodeId)
          Stop an started container.
 
Methods inherited from class org.apache.hadoop.yarn.client.api.NMClient
createNMClient, createNMClient, getNMTokenCache, setNMTokenCache
 
Methods inherited from class org.apache.hadoop.service.AbstractService
close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceStart, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

startedContainers

protected ConcurrentMap<org.apache.hadoop.yarn.api.records.ContainerId,NMClientImpl.StartedContainer> startedContainers
Constructor Detail

NMClientImpl

public NMClientImpl()

NMClientImpl

public NMClientImpl(String name)
Method Detail

serviceStop

protected void serviceStop()
                    throws Exception
Overrides:
serviceStop in class org.apache.hadoop.service.AbstractService
Throws:
Exception

cleanupRunningContainers

protected void cleanupRunningContainers()

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws Exception
Overrides:
serviceInit in class org.apache.hadoop.service.AbstractService
Throws:
Exception

cleanupRunningContainersOnStop

public void cleanupRunningContainersOnStop(boolean enabled)
Description copied from class: NMClient

Set whether the containers that are started by this client, and are still running should be stopped when the client stops. By default, the feature should be enabled.

However, containers will be stopped only when service is stopped. i.e. after AbstractService.stop().

Specified by:
cleanupRunningContainersOnStop in class NMClient
Parameters:
enabled - whether the feature is enabled or not

startContainer

public Map<String,ByteBuffer> startContainer(org.apache.hadoop.yarn.api.records.Container container,
                                             org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext)
                                      throws org.apache.hadoop.yarn.exceptions.YarnException,
                                             IOException
Description copied from class: NMClient

Start an allocated container.

The ApplicationMaster or other applications that use the client must provide the details of the allocated container, including the Id, the assigned node's Id and the token via Container. In addition, the AM needs to provide the ContainerLaunchContext as well.

Specified by:
startContainer in class NMClient
Parameters:
container - the allocated container
containerLaunchContext - the context information needed by the NodeManager to launch the container
Returns:
a map between the auxiliary service names and their outputs
Throws:
org.apache.hadoop.yarn.exceptions.YarnException
IOException

stopContainer

public void stopContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                          org.apache.hadoop.yarn.api.records.NodeId nodeId)
                   throws org.apache.hadoop.yarn.exceptions.YarnException,
                          IOException
Description copied from class: NMClient

Stop an started container.

Specified by:
stopContainer in class NMClient
Parameters:
containerId - the Id of the started container
nodeId - the Id of the NodeManager
Throws:
org.apache.hadoop.yarn.exceptions.YarnException
IOException

getContainerStatus

public org.apache.hadoop.yarn.api.records.ContainerStatus getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                                                                             org.apache.hadoop.yarn.api.records.NodeId nodeId)
                                                                      throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                             IOException
Description copied from class: NMClient

Query the status of a container.

Specified by:
getContainerStatus in class NMClient
Parameters:
containerId - the Id of the started container
nodeId - the Id of the NodeManager
Returns:
the status of a container
Throws:
org.apache.hadoop.yarn.exceptions.YarnException
IOException

createStartedContainer

protected NMClientImpl.StartedContainer createStartedContainer(org.apache.hadoop.yarn.api.records.Container container)
                                                        throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                               IOException
Throws:
org.apache.hadoop.yarn.exceptions.YarnException
IOException

removeStartedContainer

protected void removeStartedContainer(NMClientImpl.StartedContainer container)

getStartedContainer

protected NMClientImpl.StartedContainer getStartedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

getCleanupRunningContainers

public AtomicBoolean getCleanupRunningContainers()


Copyright © 2014 Apache Software Foundation. All Rights Reserved.