org.apache.hadoop.yarn.client.api.impl
Class AMRMClientImpl<T extends AMRMClient.ContainerRequest>

java.lang.Object
  extended by org.apache.hadoop.service.AbstractService
      extended by org.apache.hadoop.yarn.client.api.AMRMClient<T>
          extended by org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl<T>
All Implemented Interfaces:
Closeable, org.apache.hadoop.service.Service

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class AMRMClientImpl<T extends AMRMClient.ContainerRequest>
extends AMRMClient<T>


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.hadoop.yarn.client.api.AMRMClient
AMRMClient.ContainerRequest
 
Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service
org.apache.hadoop.service.Service.STATE
 
Field Summary
protected  Set<org.apache.hadoop.yarn.api.records.ResourceRequest> ask
           
protected  Set<String> blacklistAdditions
           
protected  Set<String> blacklistRemovals
           
protected  org.apache.hadoop.yarn.api.records.Resource clusterAvailableResources
           
protected  int clusterNodeCount
           
protected  Set<org.apache.hadoop.yarn.api.records.ContainerId> release
           
protected  Map<org.apache.hadoop.yarn.api.records.Priority,Map<String,TreeMap<org.apache.hadoop.yarn.api.records.Resource,org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.ResourceRequestInfo>>> remoteRequestsTable
           
protected  org.apache.hadoop.yarn.api.ApplicationMasterProtocol rmClient
           
 
Constructor Summary
AMRMClientImpl()
           
 
Method Summary
 void addContainerRequest(T req)
          Request containers for resources before calling allocate
 org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse allocate(float progressIndicator)
          Request additional containers and receive new container allocations.
 org.apache.hadoop.yarn.api.records.Resource getAvailableResources()
          Get the currently available resources in the cluster.
 int getClusterNodeCount()
          Get the current number of nodes in the cluster.
 List<? extends Collection<T>> getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority, String resourceName, org.apache.hadoop.yarn.api.records.Resource capability)
          Get outstanding ContainerRequests matching the given parameters.
protected  void populateNMTokens(org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse allocateResponse)
           
 org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(String appHostName, int appHostPort, String appTrackingUrl)
          Register the application master.
 void releaseAssignedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
          Release containers assigned by the Resource Manager.
 void removeContainerRequest(T req)
          Remove previous container request.
protected  void serviceInit(org.apache.hadoop.conf.Configuration conf)
           
protected  void serviceStart()
           
protected  void serviceStop()
           
 void unregisterApplicationMaster(org.apache.hadoop.yarn.api.records.FinalApplicationStatus appStatus, String appMessage, String appTrackingUrl)
          Unregister the application master.
 void updateBlacklist(List<String> blacklistAdditions, List<String> blacklistRemovals)
          Update application's blacklist with addition or removal resources.
 
Methods inherited from class org.apache.hadoop.yarn.client.api.AMRMClient
createAMRMClient, getNMTokenCache, setNMTokenCache
 
Methods inherited from class org.apache.hadoop.service.AbstractService
close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

rmClient

protected org.apache.hadoop.yarn.api.ApplicationMasterProtocol rmClient

clusterAvailableResources

protected org.apache.hadoop.yarn.api.records.Resource clusterAvailableResources

clusterNodeCount

protected int clusterNodeCount

blacklistAdditions

protected final Set<String> blacklistAdditions

blacklistRemovals

protected final Set<String> blacklistRemovals

remoteRequestsTable

protected final Map<org.apache.hadoop.yarn.api.records.Priority,Map<String,TreeMap<org.apache.hadoop.yarn.api.records.Resource,org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.ResourceRequestInfo>>> remoteRequestsTable

ask

protected final Set<org.apache.hadoop.yarn.api.records.ResourceRequest> ask

release

protected final Set<org.apache.hadoop.yarn.api.records.ContainerId> release
Constructor Detail

AMRMClientImpl

public AMRMClientImpl()
Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws Exception
Overrides:
serviceInit in class org.apache.hadoop.service.AbstractService
Throws:
Exception

serviceStart

protected void serviceStart()
                     throws Exception
Overrides:
serviceStart in class org.apache.hadoop.service.AbstractService
Throws:
Exception

serviceStop

protected void serviceStop()
                    throws Exception
Overrides:
serviceStop in class org.apache.hadoop.service.AbstractService
Throws:
Exception

registerApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(String appHostName,
                                                                                                              int appHostPort,
                                                                                                              String appTrackingUrl)
                                                                                                       throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                              IOException
Description copied from class: AMRMClient
Register the application master. This must be called before any other interaction

Specified by:
registerApplicationMaster in class AMRMClient<T extends AMRMClient.ContainerRequest>
Parameters:
appHostName - Name of the host on which master is running
appHostPort - Port master is listening on
appTrackingUrl - URL at which the master info can be seen
Returns:
RegisterApplicationMasterResponse
Throws:
org.apache.hadoop.yarn.exceptions.YarnException
IOException

allocate

public org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse allocate(float progressIndicator)
                                                                     throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                            IOException
Description copied from class: AMRMClient
Request additional containers and receive new container allocations. Requests made via addContainerRequest are sent to the ResourceManager. New containers assigned to the master are retrieved. Status of completed containers and node health updates are also retrieved. This also doubles up as a heartbeat to the ResourceManager and must be made periodically. The call may not always return any new allocations of containers. App should not make concurrent allocate requests. May cause request loss.

Specified by:
allocate in class AMRMClient<T extends AMRMClient.ContainerRequest>
Parameters:
progressIndicator - Indicates progress made by the master
Returns:
the response of the allocate request
Throws:
org.apache.hadoop.yarn.exceptions.YarnException
IOException

populateNMTokens

@InterfaceAudience.Private
protected void populateNMTokens(org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse allocateResponse)

unregisterApplicationMaster

public void unregisterApplicationMaster(org.apache.hadoop.yarn.api.records.FinalApplicationStatus appStatus,
                                        String appMessage,
                                        String appTrackingUrl)
                                 throws org.apache.hadoop.yarn.exceptions.YarnException,
                                        IOException
Description copied from class: AMRMClient
Unregister the application master. This must be called in the end.

Specified by:
unregisterApplicationMaster in class AMRMClient<T extends AMRMClient.ContainerRequest>
Parameters:
appStatus - Success/Failure status of the master
appMessage - Diagnostics message on failure
appTrackingUrl - New URL to get master info
Throws:
org.apache.hadoop.yarn.exceptions.YarnException
IOException

addContainerRequest

public void addContainerRequest(T req)
Description copied from class: AMRMClient
Request containers for resources before calling allocate

Specified by:
addContainerRequest in class AMRMClient<T extends AMRMClient.ContainerRequest>
Parameters:
req - Resource request

removeContainerRequest

public void removeContainerRequest(T req)
Description copied from class: AMRMClient
Remove previous container request. The previous container request may have already been sent to the ResourceManager. So even after the remove request the app must be prepared to receive an allocation for the previous request even after the remove request

Specified by:
removeContainerRequest in class AMRMClient<T extends AMRMClient.ContainerRequest>
Parameters:
req - Resource request

releaseAssignedContainer

public void releaseAssignedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
Description copied from class: AMRMClient
Release containers assigned by the Resource Manager. If the app cannot use the container or wants to give up the container then it can release them. The app needs to make new requests for the released resource capability if it still needs it. eg. it released non-local resources

Specified by:
releaseAssignedContainer in class AMRMClient<T extends AMRMClient.ContainerRequest>

getAvailableResources

public org.apache.hadoop.yarn.api.records.Resource getAvailableResources()
Description copied from class: AMRMClient
Get the currently available resources in the cluster. A valid value is available after a call to allocate has been made

Specified by:
getAvailableResources in class AMRMClient<T extends AMRMClient.ContainerRequest>
Returns:
Currently available resources

getClusterNodeCount

public int getClusterNodeCount()
Description copied from class: AMRMClient
Get the current number of nodes in the cluster. A valid values is available after a call to allocate has been made

Specified by:
getClusterNodeCount in class AMRMClient<T extends AMRMClient.ContainerRequest>
Returns:
Current number of nodes in the cluster

getMatchingRequests

public List<? extends Collection<T>> getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority,
                                                         String resourceName,
                                                         org.apache.hadoop.yarn.api.records.Resource capability)
Description copied from class: AMRMClient
Get outstanding ContainerRequests matching the given parameters. These ContainerRequests should have been added via addContainerRequest earlier in the lifecycle. For performance, the AMRMClient may return its internal collection directly without creating a copy. Users should not perform mutable operations on the return value. Each collection in the list contains requests with identical Resource size that fit in the given capability. In a collection, requests will be returned in the same order as they were added.

Specified by:
getMatchingRequests in class AMRMClient<T extends AMRMClient.ContainerRequest>
Returns:
Collection of request matching the parameters

updateBlacklist

public void updateBlacklist(List<String> blacklistAdditions,
                            List<String> blacklistRemovals)
Description copied from class: AMRMClient
Update application's blacklist with addition or removal resources.

Specified by:
updateBlacklist in class AMRMClient<T extends AMRMClient.ContainerRequest>
Parameters:
blacklistAdditions - list of resources which should be added to the application blacklist
blacklistRemovals - list of resources which should be removed from the application blacklist


Copyright © 2014 Apache Software Foundation. All Rights Reserved.