Package org.apache.hadoop.yarn.server.uam

Class UnmanagedApplicationManager

java.lang.Object

org.apache.hadoop.yarn.server.uam.UnmanagedApplicationManager

@Public
@Unstable
public class UnmanagedApplicationManager
extends java.lang.Object

UnmanagedApplicationManager is used to register unmanaged application and negotiate for resources from resource managers. An unmanagedAM is an AM that is not launched and managed by the RM. Allocate calls are handled asynchronously using AsyncCallback.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	APP_NAME

Constructor Summary

Constructors

Constructor	Description
UnmanagedApplicationManager(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.yarn.api.records.ApplicationId appId, java.lang.String queueName, java.lang.String submitter, java.lang.String appNameSuffix, boolean keepContainersAcrossApplicationAttempts, java.lang.String rmName, org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext originalApplicationSubmissionContext)	Constructor.

Method Summary

Modifier and Type	Method	Description
void	allocateAsync(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request, org.apache.hadoop.yarn.util.AsyncCallback<org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse> callback)	Sends the specified heart beat request to the resource manager and invokes the callback asynchronously with the response.
protected AMHeartbeatRequestHandler	createAMHeartbeatRequestHandler(org.apache.hadoop.conf.Configuration config, org.apache.hadoop.yarn.api.records.ApplicationId appId, AMRMClientRelayer relayer)
protected <T> T	createRMProxy(java.lang.Class<T> protocol, org.apache.hadoop.conf.Configuration config, org.apache.hadoop.security.UserGroupInformation user, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> token)	Returns RM proxy for the specified protocol type.
protected void	createUAMProxy(org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken)
protected void	drainHeartbeatThread()
org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse	finishApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request)	Unregisters from the resource manager and stops the request handler thread.
org.apache.hadoop.yarn.api.protocolrecords.KillApplicationResponse	forceKillApplication()	Force kill the UAM.
AMRMClientRelayer	getAMRMClientRelayer()	Returns the rmProxy relayer of this UAM.
org.apache.hadoop.yarn.api.records.ApplicationId	getAppId()	Returns the application id of the UAM.
org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext	getApplicationSubmissionContext()
int	getRequestQueueSize()
protected org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier>	getUAMToken()	Gets the amrmToken of the unmanaged AM.
protected org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier>	initializeUnmanagedAM(org.apache.hadoop.yarn.api.records.ApplicationId appId)	Launch and initialize an unmanaged AM.
protected boolean	isHeartbeatThreadAlive()
org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier>	launchUAM()	Launch a new UAM in the resource manager.
void	reAttachUAM(org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken)	Re-attach to an existing UAM in the resource manager.
org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse	registerApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest request)	Registers this UnmanagedApplicationManager with the resource manager.
void	shutDownConnections()	Shutdown this UAM client, without killing the UAM in the YarnRM side.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

APP_NAME

public static final java.lang.String APP_NAME

See Also:

Constant Field Values

Constructor Detail

UnmanagedApplicationManager

public UnmanagedApplicationManager(org.apache.hadoop.conf.Configuration conf,
                                   org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                   java.lang.String queueName,
                                   java.lang.String submitter,
                                   java.lang.String appNameSuffix,
                                   boolean keepContainersAcrossApplicationAttempts,
                                   java.lang.String rmName,
                                   org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext originalApplicationSubmissionContext)

Constructor.

Parameters:

conf - configuration

appId - application Id to use for this UAM

queueName - the queue of the UAM

submitter - user name of the app

appNameSuffix - the app name suffix to use

rmName - name of the YarnRM

originalApplicationSubmissionContext - ApplicationSubmissionContext

keepContainersAcrossApplicationAttempts - keep container flag for UAM recovery. See #setKeepContainersAcrossApplicationAttempts(boolean)

Method Detail

createAMHeartbeatRequestHandler

@VisibleForTesting
protected AMHeartbeatRequestHandler createAMHeartbeatRequestHandler(org.apache.hadoop.conf.Configuration config,
                                                                    org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                    AMRMClientRelayer relayer)

launchUAM

public org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> launchUAM()
                                                                                                      throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                             java.io.IOException

Launch a new UAM in the resource manager.

Returns:

identifier uam identifier

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if fails

java.io.IOException - if fails

reAttachUAM

public void reAttachUAM(org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken)
                 throws java.io.IOException,
                        org.apache.hadoop.yarn.exceptions.YarnException

Re-attach to an existing UAM in the resource manager.

Parameters:

amrmToken - the UAM token

Throws:

java.io.IOException - if re-attach fails

org.apache.hadoop.yarn.exceptions.YarnException - if re-attach fails

createUAMProxy

protected void createUAMProxy(org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken)
                       throws java.io.IOException

Throws:

java.io.IOException

registerApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest request)
                                                                                                       throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                              java.io.IOException

Registers this UnmanagedApplicationManager with the resource manager.

Parameters:

request - RegisterApplicationMasterRequest

Returns:

register response

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if register fails

java.io.IOException - if register fails

finishApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse finishApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request)
                                                                                                   throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                          java.io.IOException

Unregisters from the resource manager and stops the request handler thread.

Parameters:

request - the finishApplicationMaster request

Returns:

the response

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if finishAM call fails

java.io.IOException - if finishAM call fails

forceKillApplication

public org.apache.hadoop.yarn.api.protocolrecords.KillApplicationResponse forceKillApplication()
                                                                                        throws java.io.IOException,
                                                                                               org.apache.hadoop.yarn.exceptions.YarnException

Force kill the UAM.

Returns:

kill response

Throws:

java.io.IOException - if fails to create rmProxy

org.apache.hadoop.yarn.exceptions.YarnException - if force kill fails

allocateAsync

public void allocateAsync(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request,
                          org.apache.hadoop.yarn.util.AsyncCallback<org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse> callback)
                   throws org.apache.hadoop.yarn.exceptions.YarnException

Sends the specified heart beat request to the resource manager and invokes the callback asynchronously with the response.

Parameters:

request - the allocate request

callback - the callback method for the request

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if registerAM is not called yet

shutDownConnections

public void shutDownConnections()

Shutdown this UAM client, without killing the UAM in the YarnRM side.

getAppId

public org.apache.hadoop.yarn.api.records.ApplicationId getAppId()

Returns the application id of the UAM.

Returns:

application id of the UAM

getAMRMClientRelayer

public AMRMClientRelayer getAMRMClientRelayer()

Returns the rmProxy relayer of this UAM.

Returns:

rmProxy relayer of the UAM

createRMProxy

protected <T> T createRMProxy(java.lang.Class<T> protocol,
                              org.apache.hadoop.conf.Configuration config,
                              org.apache.hadoop.security.UserGroupInformation user,
                              org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> token)
                       throws java.io.IOException

Returns RM proxy for the specified protocol type. Unit test cases can override this method and return mock proxy instances.

Type Parameters:

T - type of the proxy

Parameters:

protocol - protocol of the proxy

config - configuration

user - ugi for the proxy connection

token - token for the connection

Returns:

the proxy instance

Throws:

java.io.IOException - if fails to create the proxy

initializeUnmanagedAM

protected org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> initializeUnmanagedAM(org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                                                                                                     throws java.io.IOException,
                                                                                                                            org.apache.hadoop.yarn.exceptions.YarnException

Launch and initialize an unmanaged AM. First, it creates a new application on the RM and negotiates a new attempt id. Then it waits for the RM application attempt state to reach YarnApplicationAttemptState.LAUNCHED after which it returns the AM-RM token.

Parameters:

appId - application id

Returns:

the UAM token

Throws:

java.io.IOException - if initialize fails

org.apache.hadoop.yarn.exceptions.YarnException - if initialize fails

getUAMToken

protected org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> getUAMToken()
                                                                                                           throws java.io.IOException,
                                                                                                                  org.apache.hadoop.yarn.exceptions.YarnException

Gets the amrmToken of the unmanaged AM.

Returns:

the amrmToken of the unmanaged AM.

Throws:

java.io.IOException - if getApplicationReport fails

org.apache.hadoop.yarn.exceptions.YarnException - if getApplicationReport fails

getRequestQueueSize

@VisibleForTesting
public int getRequestQueueSize()

drainHeartbeatThread

@VisibleForTesting
protected void drainHeartbeatThread()

isHeartbeatThreadAlive

@VisibleForTesting
protected boolean isHeartbeatThreadAlive()

getApplicationSubmissionContext

@VisibleForTesting
public org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext getApplicationSubmissionContext()