Package org.apache.hadoop.mapred

Class TaskAttemptListenerImpl

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.service.CompositeService

org.apache.hadoop.mapred.TaskAttemptListenerImpl

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.ipc.VersionedProtocol, org.apache.hadoop.mapred.TaskUmbilicalProtocol, TaskAttemptListener, org.apache.hadoop.service.Service

public class TaskAttemptListenerImpl
extends org.apache.hadoop.service.CompositeService
implements org.apache.hadoop.mapred.TaskUmbilicalProtocol, TaskAttemptListener

This class is responsible for talking to the task umblical. It also converts all the old data structures to yarn data structures. This class HAS to be in this package to access package private methods/classes.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

org.apache.hadoop.service.CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field	Description
protected TaskHeartbeatHandler	taskHeartbeatHandler

Fields inherited from class org.apache.hadoop.service.CompositeService

STOP_ONLY_STARTED_SERVICES

Fields inherited from interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

versionID

Constructor Summary

Constructors

Constructor	Description
TaskAttemptListenerImpl(AppContext context, org.apache.hadoop.mapreduce.security.token.JobTokenSecretManager jobTokenSecretManager, RMHeartbeatHandler rmHeartbeatHandler, AMPreemptionPolicy preemptionPolicy)
TaskAttemptListenerImpl(AppContext context, org.apache.hadoop.mapreduce.security.token.JobTokenSecretManager jobTokenSecretManager, RMHeartbeatHandler rmHeartbeatHandler, AMPreemptionPolicy preemptionPolicy, byte[] secretShuffleKey)

Method Summary

Modifier and Type	Method	Description
boolean	canCommit(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID)	Child checking whether it can commit.
void	commitPending(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID, org.apache.hadoop.mapred.TaskStatus taskStatsu)	TaskAttempt is reporting that it is in commit_pending and it is waiting for the commit Response Commit it a two-phased protocol.
void	done(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID)
void	fatalError(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID, java.lang.String msg, boolean fastFail)
void	fsError(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID, java.lang.String message)
java.net.InetSocketAddress	getAddress()
org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID	getCheckpointID(org.apache.hadoop.mapred.TaskID taskId)
org.apache.hadoop.mapred.MapTaskCompletionEventsUpdate	getMapCompletionEvents(org.apache.hadoop.mapred.JobID jobIdentifier, int startIndex, int maxEvents, org.apache.hadoop.mapred.TaskAttemptID taskAttemptID)
org.apache.hadoop.ipc.ProtocolSignature	getProtocolSignature(java.lang.String protocol, long clientVersion, int clientMethodsHash)
long	getProtocolVersion(java.lang.String arg0, long arg1)
org.apache.hadoop.mapred.JvmTask	getTask(org.apache.hadoop.mapred.JvmContext context)
void	preempted(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID, org.apache.hadoop.mapred.TaskStatus taskStatus)
protected void	registerHeartbeatHandler(org.apache.hadoop.conf.Configuration conf)
void	registerLaunchedTask(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID, WrappedJvmID jvmId)	Register task attempt.
void	registerPendingTask(org.apache.hadoop.mapred.Task task, WrappedJvmID jvmID)	Register a JVM with the listener.
void	reportDiagnosticInfo(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID, java.lang.String diagnosticInfo)
void	reportNextRecordRange(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID, org.apache.hadoop.mapred.SortedRanges.Range range)
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf)
protected void	serviceStart()
protected void	serviceStop()
void	setCheckpointID(org.apache.hadoop.mapred.TaskID taskId, org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID cid)
void	shuffleError(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID, java.lang.String message)
protected void	startRpcServer()
org.apache.hadoop.mapred.AMFeedback	statusUpdate(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID, org.apache.hadoop.mapred.TaskStatus taskStatus)
protected void	stopRpcServer()
void	unregister(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID, WrappedJvmID jvmID)	Unregister the JVM and the attempt associated with it.

Methods inherited from class org.apache.hadoop.service.CompositeService

addIfService, addService, getServices, removeService

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

taskHeartbeatHandler

protected TaskHeartbeatHandler taskHeartbeatHandler

Constructor Detail

TaskAttemptListenerImpl

public TaskAttemptListenerImpl(AppContext context,
                               org.apache.hadoop.mapreduce.security.token.JobTokenSecretManager jobTokenSecretManager,
                               RMHeartbeatHandler rmHeartbeatHandler,
                               AMPreemptionPolicy preemptionPolicy)

TaskAttemptListenerImpl

public TaskAttemptListenerImpl(AppContext context,
                               org.apache.hadoop.mapreduce.security.token.JobTokenSecretManager jobTokenSecretManager,
                               RMHeartbeatHandler rmHeartbeatHandler,
                               AMPreemptionPolicy preemptionPolicy,
                               byte[] secretShuffleKey)

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws java.lang.Exception

Overrides:

serviceInit in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

serviceStart

protected void serviceStart()
                     throws java.lang.Exception

Overrides:

serviceStart in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

registerHeartbeatHandler

protected void registerHeartbeatHandler(org.apache.hadoop.conf.Configuration conf)

startRpcServer

protected void startRpcServer()

serviceStop

protected void serviceStop()
                    throws java.lang.Exception

Overrides:

serviceStop in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

stopRpcServer

protected void stopRpcServer()

getAddress

public java.net.InetSocketAddress getAddress()

Specified by:

getAddress in interface TaskAttemptListener

canCommit

public boolean canCommit(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID)
                  throws java.io.IOException

Child checking whether it can commit.
Commit is a two-phased protocol. First the attempt informs the ApplicationMaster that it is commitPending(TaskAttemptID, TaskStatus). Then it repeatedly polls the ApplicationMaster whether it canCommit(TaskAttemptID) This is a legacy from the centralized commit protocol handling by the JobTracker.

Specified by:

canCommit in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

commitPending

public void commitPending(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID,
                          org.apache.hadoop.mapred.TaskStatus taskStatsu)
                   throws java.io.IOException,
                          java.lang.InterruptedException

TaskAttempt is reporting that it is in commit_pending and it is waiting for the commit Response
Commit it a two-phased protocol. First the attempt informs the ApplicationMaster that it is commitPending(TaskAttemptID, TaskStatus). Then it repeatedly polls the ApplicationMaster whether it canCommit(TaskAttemptID) This is a legacy from the centralized commit protocol handling by the JobTracker.

Specified by:

commitPending in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

java.lang.InterruptedException

preempted

public void preempted(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID,
                      org.apache.hadoop.mapred.TaskStatus taskStatus)
               throws java.io.IOException,
                      java.lang.InterruptedException

Specified by:

preempted in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

java.lang.InterruptedException

done

public void done(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID)
          throws java.io.IOException

Specified by:

done in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

fatalError

public void fatalError(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID,
                       java.lang.String msg,
                       boolean fastFail)
                throws java.io.IOException

Specified by:

fatalError in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

fsError

public void fsError(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID,
                    java.lang.String message)
             throws java.io.IOException

Specified by:

fsError in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

shuffleError

public void shuffleError(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID,
                         java.lang.String message)
                  throws java.io.IOException

Specified by:

shuffleError in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

getMapCompletionEvents

public org.apache.hadoop.mapred.MapTaskCompletionEventsUpdate getMapCompletionEvents(org.apache.hadoop.mapred.JobID jobIdentifier,
                                                                                     int startIndex,
                                                                                     int maxEvents,
                                                                                     org.apache.hadoop.mapred.TaskAttemptID taskAttemptID)
                                                                              throws java.io.IOException

Specified by:

getMapCompletionEvents in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

reportDiagnosticInfo

public void reportDiagnosticInfo(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID,
                                 java.lang.String diagnosticInfo)
                          throws java.io.IOException

Specified by:

reportDiagnosticInfo in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

statusUpdate

public org.apache.hadoop.mapred.AMFeedback statusUpdate(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID,
                                                        org.apache.hadoop.mapred.TaskStatus taskStatus)
                                                 throws java.io.IOException,
                                                        java.lang.InterruptedException

Specified by:

statusUpdate in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

java.lang.InterruptedException

getProtocolVersion

public long getProtocolVersion(java.lang.String arg0,
                               long arg1)
                        throws java.io.IOException

Specified by:

getProtocolVersion in interface org.apache.hadoop.ipc.VersionedProtocol

Throws:

java.io.IOException

reportNextRecordRange

public void reportNextRecordRange(org.apache.hadoop.mapred.TaskAttemptID taskAttemptID,
                                  org.apache.hadoop.mapred.SortedRanges.Range range)
                           throws java.io.IOException

Specified by:

reportNextRecordRange in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

getTask

public org.apache.hadoop.mapred.JvmTask getTask(org.apache.hadoop.mapred.JvmContext context)
                                         throws java.io.IOException

Specified by:

getTask in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

Throws:

java.io.IOException

registerPendingTask

public void registerPendingTask(org.apache.hadoop.mapred.Task task,
                                WrappedJvmID jvmID)

Description copied from interface: TaskAttemptListener

Register a JVM with the listener. This should be called as soon as a JVM ID is assigned to a task attempt, before it has been launched.

Specified by:

registerPendingTask in interface TaskAttemptListener

Parameters:

task - the task itself for this JVM.

jvmID - The ID of the JVM .

registerLaunchedTask

public void registerLaunchedTask(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID,
                                 WrappedJvmID jvmId)

Description copied from interface: TaskAttemptListener

Register task attempt. This should be called when the JVM has been launched.

Specified by:

registerLaunchedTask in interface TaskAttemptListener

Parameters:

attemptID - the id of the attempt for this JVM.

jvmId - the ID of the JVM.

unregister

public void unregister(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID,
                       WrappedJvmID jvmID)

Description copied from interface: TaskAttemptListener

Unregister the JVM and the attempt associated with it. This should be called when the attempt/JVM has finished executing and is being cleaned up.

Specified by:

unregister in interface TaskAttemptListener

Parameters:

attemptID - the ID of the attempt.

jvmID - the ID of the JVM for that attempt.

getProtocolSignature

public org.apache.hadoop.ipc.ProtocolSignature getProtocolSignature(java.lang.String protocol,
                                                                    long clientVersion,
                                                                    int clientMethodsHash)
                                                             throws java.io.IOException

Specified by:

getProtocolSignature in interface org.apache.hadoop.ipc.VersionedProtocol

Throws:

java.io.IOException

getCheckpointID

public org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID getCheckpointID(org.apache.hadoop.mapred.TaskID taskId)

Specified by:

getCheckpointID in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol

setCheckpointID

public void setCheckpointID(org.apache.hadoop.mapred.TaskID taskId,
                            org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID cid)

Specified by:

setCheckpointID in interface org.apache.hadoop.mapred.TaskUmbilicalProtocol