Package org.apache.hadoop.mapreduce.v2.app.speculate

Class DefaultSpeculator

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, Speculator, org.apache.hadoop.service.Service, org.apache.hadoop.yarn.event.EventHandler<SpeculatorEvent>

public class DefaultSpeculator
extends org.apache.hadoop.service.AbstractService
implements Speculator

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Nested classes/interfaces inherited from interface org.apache.hadoop.mapreduce.v2.app.speculate.Speculator

Speculator.EventType

Constructor Summary

Constructors

Constructor	Description
DefaultSpeculator(org.apache.hadoop.conf.Configuration conf, AppContext context)
DefaultSpeculator(org.apache.hadoop.conf.Configuration conf, AppContext context, TaskRuntimeEstimator estimator, org.apache.hadoop.yarn.util.Clock clock)
DefaultSpeculator(org.apache.hadoop.conf.Configuration conf, AppContext context, org.apache.hadoop.yarn.util.Clock clock)

Method Summary

Modifier and Type	Method	Description
protected void	addSpeculativeAttempt(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
boolean	eventQueueEmpty()
int	getMinimumAllowedSpeculativeTasks()
double	getProportionRunningTasksSpeculatable()
double	getProportionTotalTasksSpeculatable()
long	getSoonestRetryAfterNoSpeculate()
long	getSoonestRetryAfterSpeculate()
void	handle(SpeculatorEvent event)
void	handleAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status)
void	scanForSpeculations()
protected void	serviceStart()
protected void	serviceStop()
protected void	statusUpdate(TaskAttemptStatusUpdateEvent.TaskAttemptStatus reportedStatus, long timestamp)	Absorbs one TaskAttemptStatus

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

DefaultSpeculator

public DefaultSpeculator(org.apache.hadoop.conf.Configuration conf,
                         AppContext context)

DefaultSpeculator

public DefaultSpeculator(org.apache.hadoop.conf.Configuration conf,
                         AppContext context,
                         org.apache.hadoop.yarn.util.Clock clock)

DefaultSpeculator

public DefaultSpeculator(org.apache.hadoop.conf.Configuration conf,
                         AppContext context,
                         TaskRuntimeEstimator estimator,
                         org.apache.hadoop.yarn.util.Clock clock)

Method Detail

serviceStart

protected void serviceStart()
                     throws java.lang.Exception

Overrides:

serviceStart in class org.apache.hadoop.service.AbstractService

Throws:

java.lang.Exception

serviceStop

protected void serviceStop()
                    throws java.lang.Exception

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

java.lang.Exception

handleAttempt

public void handleAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status)

Specified by:

handleAttempt in interface Speculator

eventQueueEmpty

public boolean eventQueueEmpty()

scanForSpeculations

public void scanForSpeculations()

statusUpdate

protected void statusUpdate(TaskAttemptStatusUpdateEvent.TaskAttemptStatus reportedStatus,
                            long timestamp)

Absorbs one TaskAttemptStatus

Parameters:

reportedStatus - the status report that we got from a task attempt that we want to fold into the speculation data for this job

timestamp - the time this status corresponds to. This matters because statuses contain progress.

addSpeculativeAttempt

protected void addSpeculativeAttempt(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)

handle

public void handle(SpeculatorEvent event)

Specified by:

handle in interface org.apache.hadoop.yarn.event.EventHandler<SpeculatorEvent>

getSoonestRetryAfterNoSpeculate

@VisibleForTesting
public long getSoonestRetryAfterNoSpeculate()

getSoonestRetryAfterSpeculate

@VisibleForTesting
public long getSoonestRetryAfterSpeculate()

getProportionRunningTasksSpeculatable

@VisibleForTesting
public double getProportionRunningTasksSpeculatable()

getProportionTotalTasksSpeculatable

@VisibleForTesting
public double getProportionTotalTasksSpeculatable()

getMinimumAllowedSpeculativeTasks

@VisibleForTesting
public int getMinimumAllowedSpeculativeTasks()