Package org.apache.hadoop.yarn.server.nodemanager.amrmproxy

Class AMRMProxyService

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.service.CompositeService

org.apache.hadoop.yarn.server.nodemanager.amrmproxy.AMRMProxyService

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.service.Service, org.apache.hadoop.yarn.api.ApplicationMasterProtocol

public class AMRMProxyService
extends org.apache.hadoop.service.CompositeService
implements org.apache.hadoop.yarn.api.ApplicationMasterProtocol

AMRMProxyService is a service that runs on each node manager that can be used to intercept and inspect messages from application master to the cluster resource manager. It listens to messages from the application master and creates a request intercepting pipeline instance for each application. The pipeline is a chain of interceptor instances that can inspect and modify the request/response as needed.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AMRMProxyService.RequestInterceptorChainWrapper	Private structure for encapsulating RequestInterceptor and ApplicationAttemptId instances.

Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

org.apache.hadoop.service.CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields inherited from class org.apache.hadoop.service.CompositeService

STOP_ONLY_STARTED_SERVICES

Constructor Summary

Constructors

Constructor	Description
AMRMProxyService(Context nmContext, org.apache.hadoop.yarn.event.AsyncDispatcher dispatcher)	Creates an instance of the service.

Method Summary

Modifier and Type	Method	Description
org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse	allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request)	This is called by the AMs started on this node to send heart beat to RM.
protected RequestInterceptor	createRequestInterceptorChain()	This method creates and returns reference of the first interceptor in the chain of request interceptor instances.
org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse	finishApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request)	This is called by the AMs started on this node to unregister from the RM.
java.net.InetSocketAddress	getBindAddress()
protected java.util.Map<org.apache.hadoop.yarn.api.records.ApplicationId,AMRMProxyService.RequestInterceptorChainWrapper>	getPipelines()	Gets the Request interceptor chains for all the applications.
AMRMProxyTokenSecretManager	getSecretManager()
protected void	initializePipeline(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, java.lang.String user, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken, org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> localToken, java.util.Map<java.lang.String,byte[]> recoveredDataMap, boolean isRecovery, org.apache.hadoop.security.Credentials credentials)	Initializes the request interceptor pipeline for the specified application.
void	processApplicationStartRequest(org.apache.hadoop.yarn.api.protocolrecords.StartContainerRequest request)	Callback from the ContainerManager implementation for initializing the application request processing pipeline.
void	recover()	Recover from NM state store.
org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse	registerApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest request)	This is called by the AMs started on this node to register with the RM.
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf)
protected void	serviceStart()
protected void	serviceStop()
protected void	stopApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)	Shuts down the request processing pipeline for the specified application attempt id.

Methods inherited from class org.apache.hadoop.service.CompositeService

addIfService, addService, getServices, removeService

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

AMRMProxyService

public AMRMProxyService(Context nmContext,
                        org.apache.hadoop.yarn.event.AsyncDispatcher dispatcher)

Creates an instance of the service.

Parameters:

nmContext - NM context

dispatcher - NM dispatcher

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws java.lang.Exception

Overrides:

serviceInit in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

serviceStart

protected void serviceStart()
                     throws java.lang.Exception

Overrides:

serviceStart in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

serviceStop

protected void serviceStop()
                    throws java.lang.Exception

Overrides:

serviceStop in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

recover

public void recover()
             throws java.io.IOException

Recover from NM state store. Called after serviceInit before serviceStart.

Throws:

java.io.IOException - if recover fails

registerApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest request)
                                                                                                       throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                              java.io.IOException

This is called by the AMs started on this node to register with the RM. This method does the initial authorization and then forwards the request to the application instance specific interceptor chain.

Specified by:

registerApplicationMaster in interface org.apache.hadoop.yarn.api.ApplicationMasterProtocol

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

java.io.IOException

finishApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterResponse finishApplicationMaster(org.apache.hadoop.yarn.api.protocolrecords.FinishApplicationMasterRequest request)
                                                                                                   throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                          java.io.IOException

This is called by the AMs started on this node to unregister from the RM. This method does the initial authorization and then forwards the request to the application instance specific interceptor chain.

Specified by:

finishApplicationMaster in interface org.apache.hadoop.yarn.api.ApplicationMasterProtocol

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

java.io.IOException

allocate

public org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest request)
                                                                     throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                            java.io.IOException

This is called by the AMs started on this node to send heart beat to RM. This method does the initial authorization and then forwards the request to the application instance specific pipeline, which is a chain of request interceptor objects. One application request processing pipeline is created per AM instance.

Specified by:

allocate in interface org.apache.hadoop.yarn.api.ApplicationMasterProtocol

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

java.io.IOException

processApplicationStartRequest

public void processApplicationStartRequest(org.apache.hadoop.yarn.api.protocolrecords.StartContainerRequest request)
                                    throws java.io.IOException,
                                           org.apache.hadoop.yarn.exceptions.YarnException

Callback from the ContainerManager implementation for initializing the application request processing pipeline.

Parameters:

request - - encapsulates information for starting an AM

Throws:

java.io.IOException - if fails

org.apache.hadoop.yarn.exceptions.YarnException - if fails

initializePipeline

protected void initializePipeline(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId,
                                  java.lang.String user,
                                  org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> amrmToken,
                                  org.apache.hadoop.security.token.Token<org.apache.hadoop.yarn.security.AMRMTokenIdentifier> localToken,
                                  java.util.Map<java.lang.String,byte[]> recoveredDataMap,
                                  boolean isRecovery,
                                  org.apache.hadoop.security.Credentials credentials)

Initializes the request interceptor pipeline for the specified application.

Parameters:

applicationAttemptId - attempt id

user - user name

amrmToken - amrmToken issued by RM

localToken - amrmToken issued by AMRMProxy

recoveredDataMap - the recovered states for AMRMProxy from NMSS

isRecovery - whether this is to recover a previously existing pipeline

stopApplication

protected void stopApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)

Shuts down the request processing pipeline for the specified application attempt id.

Parameters:

applicationId - application id

getPipelines

protected java.util.Map<org.apache.hadoop.yarn.api.records.ApplicationId,AMRMProxyService.RequestInterceptorChainWrapper> getPipelines()

Gets the Request interceptor chains for all the applications.

Returns:

the request interceptor chains.

createRequestInterceptorChain

protected RequestInterceptor createRequestInterceptorChain()

This method creates and returns reference of the first interceptor in the chain of request interceptor instances.

Returns:

the reference of the first interceptor in the chain

getBindAddress

@Private
public java.net.InetSocketAddress getBindAddress()

getSecretManager

@Private
public AMRMProxyTokenSecretManager getSecretManager()