Package org.apache.hadoop.yarn.client.api.async.impl

Class AMRMClientAsyncImpl<T extends AMRMClient.ContainerRequest>

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.client.api.async.AMRMClientAsync<T>

org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl<T>

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.service.Service

@Private
@Unstable
public class AMRMClientAsyncImpl<T extends AMRMClient.ContainerRequest>
extends AMRMClientAsync<T>

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.yarn.client.api.async.AMRMClientAsync

AMRMClientAsync.AbstractCallbackHandler, AMRMClientAsync.CallbackHandler

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields inherited from class org.apache.hadoop.yarn.client.api.async.AMRMClientAsync

client, handler, heartbeatIntervalMs

Constructor Summary

Constructors

Constructor	Description
AMRMClientAsyncImpl(int intervalMs, AMRMClientAsync.AbstractCallbackHandler callbackHandler)
AMRMClientAsyncImpl(int intervalMs, AMRMClientAsync.CallbackHandler callbackHandler)	Deprecated. Use AMRMClientAsyncImpl(int, AMRMClientAsync.AbstractCallbackHandler) instead.
AMRMClientAsyncImpl(AMRMClient<T> client, int intervalMs, AMRMClientAsync.AbstractCallbackHandler callbackHandler)
AMRMClientAsyncImpl(AMRMClient<T> client, int intervalMs, AMRMClientAsync.CallbackHandler callbackHandler)	Deprecated.

Method Summary

Modifier and Type	Method	Description
void	addContainerRequest(T req)	Request containers for resources before calling allocate
void	addSchedulingRequests(java.util.Collection<org.apache.hadoop.yarn.api.records.SchedulingRequest> schedulingRequests)	Add a Collection of SchedulingRequests.
org.apache.hadoop.yarn.api.records.Resource	getAvailableResources()	Get the currently available resources in the cluster.
int	getClusterNodeCount()	Get the current number of nodes in the cluster.
java.util.List<? extends java.util.Collection<T>>	getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority, java.lang.String resourceName, org.apache.hadoop.yarn.api.records.Resource capability)
org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse	registerApplicationMaster(java.lang.String appHostName, int appHostPort, java.lang.String appTrackingUrl)	Registers this application master with the resource manager.
org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse	registerApplicationMaster(java.lang.String appHostName, int appHostPort, java.lang.String appTrackingUrl, java.util.Map<java.util.Set<java.lang.String>,org.apache.hadoop.yarn.api.resource.PlacementConstraint> placementConstraintsMap)	Registers this application master with the resource manager.
void	releaseAssignedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)	Release containers assigned by the Resource Manager.
void	removeContainerRequest(T req)	Remove previous container request.
void	requestContainerUpdate(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.UpdateContainerRequest updateContainerRequest)	Request a container update before calling allocate.
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf)
protected void	serviceStart()
protected void	serviceStop()	Tells the heartbeat and handler threads to stop and waits for them to terminate.
void	unregisterApplicationMaster(org.apache.hadoop.yarn.api.records.FinalApplicationStatus appStatus, java.lang.String appMessage, java.lang.String appTrackingUrl)	Unregister the application master.
void	updateBlacklist(java.util.List<java.lang.String> blacklistAdditions, java.util.List<java.lang.String> blacklistRemovals)	Update application's blacklist with addition or removal resources.
void	updateTrackingUrl(java.lang.String trackingUrl)	Update application's tracking url on next heartbeat.

Methods inherited from class org.apache.hadoop.yarn.client.api.async.AMRMClientAsync

createAMRMClientAsync, createAMRMClientAsync, createAMRMClientAsync, createAMRMClientAsync, getMatchingRequests, getMatchingRequests, getRegisteredTimelineV2Client, registerTimelineV2Client, requestContainerResourceChange, setHeartbeatInterval, waitFor, waitFor, waitFor

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

AMRMClientAsyncImpl

public AMRMClientAsyncImpl(int intervalMs,
                           AMRMClientAsync.AbstractCallbackHandler callbackHandler)

Parameters:

intervalMs - heartbeat interval in milliseconds between AM and RM

callbackHandler - callback handler that processes responses from the ResourceManager

AMRMClientAsyncImpl

public AMRMClientAsyncImpl(AMRMClient<T> client,
                           int intervalMs,
                           AMRMClientAsync.AbstractCallbackHandler callbackHandler)

AMRMClientAsyncImpl

@Deprecated
public AMRMClientAsyncImpl(int intervalMs,
                           AMRMClientAsync.CallbackHandler callbackHandler)

Deprecated.

Use AMRMClientAsyncImpl(int, AMRMClientAsync.AbstractCallbackHandler) instead.

AMRMClientAsyncImpl

@Private
@VisibleForTesting
@Deprecated
public AMRMClientAsyncImpl(AMRMClient<T> client,
                           int intervalMs,
                           AMRMClientAsync.CallbackHandler callbackHandler)

Deprecated.

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws java.lang.Exception

Overrides:

serviceInit in class org.apache.hadoop.service.AbstractService

Throws:

java.lang.Exception

serviceStart

protected void serviceStart()
                     throws java.lang.Exception

Overrides:

serviceStart in class org.apache.hadoop.service.AbstractService

Throws:

java.lang.Exception

serviceStop

protected void serviceStop()
                    throws java.lang.Exception

Tells the heartbeat and handler threads to stop and waits for them to terminate.

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

java.lang.Exception

getMatchingRequests

public java.util.List<? extends java.util.Collection<T>> getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority,
                                                                             java.lang.String resourceName,
                                                                             org.apache.hadoop.yarn.api.records.Resource capability)

Specified by:

getMatchingRequests in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

addSchedulingRequests

public void addSchedulingRequests(java.util.Collection<org.apache.hadoop.yarn.api.records.SchedulingRequest> schedulingRequests)

Description copied from class: AMRMClientAsync

Add a Collection of SchedulingRequests. The AMRMClient will ensure that all requests in the same batch are sent in the same allocate call.

Overrides:

addSchedulingRequests in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

schedulingRequests - Collection of Scheduling Requests.

registerApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(java.lang.String appHostName,
                                                                                                              int appHostPort,
                                                                                                              java.lang.String appTrackingUrl)
                                                                                                       throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                              java.io.IOException

Registers this application master with the resource manager. On successful registration, starts the heartbeating thread.

Specified by:

registerApplicationMaster in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

appHostName - Name of the host on which master is running

appHostPort - Port master is listening on

appTrackingUrl - URL at which the master info can be seen

Returns:

Register AM Response.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

java.io.IOException

registerApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(java.lang.String appHostName,
                                                                                                              int appHostPort,
                                                                                                              java.lang.String appTrackingUrl,
                                                                                                              java.util.Map<java.util.Set<java.lang.String>,org.apache.hadoop.yarn.api.resource.PlacementConstraint> placementConstraintsMap)
                                                                                                       throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                              java.io.IOException

Registers this application master with the resource manager. On successful registration, starts the heartbeating thread.

Overrides:

registerApplicationMaster in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

appHostName - Name of the host on which master is running

appHostPort - Port master is listening on

appTrackingUrl - URL at which the master info can be seen

placementConstraintsMap - Placement Constraints Mapping.

Returns:

Register AM Response.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

java.io.IOException

unregisterApplicationMaster

public void unregisterApplicationMaster(org.apache.hadoop.yarn.api.records.FinalApplicationStatus appStatus,
                                        java.lang.String appMessage,
                                        java.lang.String appTrackingUrl)
                                 throws org.apache.hadoop.yarn.exceptions.YarnException,
                                        java.io.IOException

Unregister the application master. This must be called in the end.

Specified by:

unregisterApplicationMaster in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

appStatus - Success/Failure status of the master

appMessage - Diagnostics message on failure

appTrackingUrl - New URL to get master info

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

java.io.IOException

addContainerRequest

public void addContainerRequest(T req)

Request containers for resources before calling allocate

Specified by:

addContainerRequest in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

req - Resource request

removeContainerRequest

public void removeContainerRequest(T req)

Remove previous container request. The previous container request may have already been sent to the ResourceManager. So even after the remove request the app must be prepared to receive an allocation for the previous request even after the remove request

Specified by:

removeContainerRequest in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

req - Resource request

requestContainerUpdate

public void requestContainerUpdate(org.apache.hadoop.yarn.api.records.Container container,
                                   org.apache.hadoop.yarn.api.records.UpdateContainerRequest updateContainerRequest)

Description copied from class: AMRMClientAsync

Request a container update before calling allocate. Any previous pending update request of the same container will be removed.

Specified by:

requestContainerUpdate in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

container - The container returned from the last successful resource allocation or update

updateContainerRequest - The UpdateContainerRequest.

releaseAssignedContainer

public void releaseAssignedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Release containers assigned by the Resource Manager. If the app cannot use the container or wants to give up the container then it can release them. The app needs to make new requests for the released resource capability if it still needs it. eg. it released non-local resources

Specified by:

releaseAssignedContainer in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

containerId -

getAvailableResources

public org.apache.hadoop.yarn.api.records.Resource getAvailableResources()

Get the currently available resources in the cluster. A valid value is available after a call to allocate has been made

Specified by:

getAvailableResources in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Returns:

Currently available resources

getClusterNodeCount

public int getClusterNodeCount()

Get the current number of nodes in the cluster. A valid values is available after a call to allocate has been made

Specified by:

getClusterNodeCount in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Returns:

Current number of nodes in the cluster

updateBlacklist

public void updateBlacklist(java.util.List<java.lang.String> blacklistAdditions,
                            java.util.List<java.lang.String> blacklistRemovals)

Update application's blacklist with addition or removal resources.

Specified by:

updateBlacklist in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

blacklistAdditions - list of resources which should be added to the application blacklist

blacklistRemovals - list of resources which should be removed from the application blacklist

updateTrackingUrl

public void updateTrackingUrl(java.lang.String trackingUrl)

Description copied from class: AMRMClientAsync

Update application's tracking url on next heartbeat.

Overrides:

updateTrackingUrl in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

trackingUrl - new tracking url for this application