org.apache.hadoop.mapreduce.v2.app.speculate
Class LegacyTaskRuntimeEstimator

java.lang.Object
  org.apache.hadoop.mapreduce.v2.app.speculate.LegacyTaskRuntimeEstimator

All Implemented Interfaces:

TaskRuntimeEstimator

public class LegacyTaskRuntimeEstimator
extends Object

Field Summary

protected AppContext	context
protected Set<Task>	doneTasks
protected Map<Job,DataStatistics>	mapperStatistics
protected Map<Job,DataStatistics>	reducerStatistics
protected Map<org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId,Long>	startTimes

Constructor Summary

LegacyTaskRuntimeEstimator()

Method Summary

long	attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
void	contextualize(org.apache.hadoop.conf.Configuration conf, AppContext context)
protected DataStatistics	dataStatisticsForTask(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
void	enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status, long timestamp)
long	estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id) Estimates how long a new attempt on this task will take if we start one now
long	estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID) Estimate a task attempt's total runtime.
long	runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID) Computes the width of the error band of our estimate of the task runtime as returned by TaskRuntimeEstimator.estimatedRuntime(TaskAttemptId)
long	thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID) Find a maximum reasonable execution wallclock time.
void	updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status, long timestamp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

context

protected AppContext context

startTimes

protected final Map<org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId,Long> startTimes

mapperStatistics

protected final Map<Job,DataStatistics> mapperStatistics

reducerStatistics

protected final Map<Job,DataStatistics> reducerStatistics

doneTasks

protected final Set<Task> doneTasks

Constructor Detail

LegacyTaskRuntimeEstimator

public LegacyTaskRuntimeEstimator()

Method Detail

updateAttempt

public void updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status,
                          long timestamp)

Specified by:

updateAttempt in interface TaskRuntimeEstimator

estimatedRuntime

public long estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)

Description copied from interface: TaskRuntimeEstimator

Estimate a task attempt's total runtime. Includes the time already elapsed.

Parameters:

attemptID - the TaskAttemptId of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

runtimeEstimateVariance

public long runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)

Description copied from interface: TaskRuntimeEstimator

Computes the width of the error band of our estimate of the task runtime as returned by TaskRuntimeEstimator.estimatedRuntime(TaskAttemptId)

Parameters:

attemptID - the TaskAttemptId of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

enrollAttempt

public void enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status,
                          long timestamp)

Specified by:

enrollAttempt in interface TaskRuntimeEstimator

attemptEnrolledTime

public long attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)

Specified by:

attemptEnrolledTime in interface TaskRuntimeEstimator

contextualize

public void contextualize(org.apache.hadoop.conf.Configuration conf,
                          AppContext context)

Specified by:

contextualize in interface TaskRuntimeEstimator

dataStatisticsForTask

protected DataStatistics dataStatisticsForTask(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)

thresholdRuntime

public long thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)

Description copied from interface: TaskRuntimeEstimator

Find a maximum reasonable execution wallclock time. Includes the time already elapsed. Find a maximum reasonable execution time. Includes the time already elapsed. If the projected total execution time for this task ever exceeds its reasonable execution time, we may speculate it.

Specified by:

thresholdRuntime in interface TaskRuntimeEstimator

Parameters:

taskID - the TaskId of the task we are asking about

Returns:

the task's maximum reasonable runtime, or MAX_VALUE if we don't have enough information to rule out any runtime, however long.

estimatedNewAttemptRuntime

public long estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id)

Description copied from interface: TaskRuntimeEstimator

Estimates how long a new attempt on this task will take if we start one now

Specified by:

estimatedNewAttemptRuntime in interface TaskRuntimeEstimator

Parameters:

id - the TaskId of the task we are asking about

Returns:

our best estimate of a new attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.