org.apache.hadoop.mapreduce.task.reduce

Class DirectShuffleFetcher<K,V>

java.lang.Object

java.lang.Thread

org.apache.hadoop.mapreduce.task.reduce.DirectShuffleFetcher<K,V>

All Implemented Interfaces:

Runnable

public class DirectShuffleFetcher<K,V>
extends Thread

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

Thread.State, Thread.UncaughtExceptionHandler

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.fs.FileSystem	rfs

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors

Constructor and Description

DirectShuffleFetcher(int id, org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.mapreduce.TaskAttemptID reduceId, DirectShuffleSchedulerImpl<K,V> scheduler, org.apache.hadoop.mapreduce.task.reduce.MergeManager<K,V> merger, org.apache.hadoop.mapred.Reporter reporter, org.apache.hadoop.mapreduce.task.reduce.ShuffleClientMetrics metrics, org.apache.hadoop.mapreduce.task.reduce.ExceptionReporter exceptionReporter, org.apache.hadoop.mapred.MapOutputFile mapOutputFile) Represents the result of an attempt to copy a map output

Method Summary

Methods

Modifier and Type	Method and Description
protected long	copyOutput(MapOutputLocation loc) Copies a a map output from a remote host.
boolean	fail() Fail the current file that we are fetching
MapOutputLocation	getLocation() Get the current map output location.
void	run() Loop forever and fetch map outputs as they become available.
void	shutDown()

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

rfs

protected org.apache.hadoop.fs.FileSystem rfs

Constructor Detail

DirectShuffleFetcher

public DirectShuffleFetcher(int id,
                    org.apache.hadoop.mapred.JobConf jobConf,
                    org.apache.hadoop.mapreduce.TaskAttemptID reduceId,
                    DirectShuffleSchedulerImpl<K,V> scheduler,
                    org.apache.hadoop.mapreduce.task.reduce.MergeManager<K,V> merger,
                    org.apache.hadoop.mapred.Reporter reporter,
                    org.apache.hadoop.mapreduce.task.reduce.ShuffleClientMetrics metrics,
                    org.apache.hadoop.mapreduce.task.reduce.ExceptionReporter exceptionReporter,
                    org.apache.hadoop.mapred.MapOutputFile mapOutputFile)

Represents the result of an attempt to copy a map output

Method Detail

fail

public boolean fail()

Fail the current file that we are fetching

Returns:

were we currently fetching?

getLocation

public MapOutputLocation getLocation()

Get the current map output location.

shutDown

public void shutDown()

run

public void run()

Loop forever and fetch map outputs as they become available.

Specified by:

run in interface Runnable

Overrides:

run in class Thread

copyOutput

protected long copyOutput(MapOutputLocation loc)
                   throws IOException,
                          InterruptedException

Copies a a map output from a remote host.

Throws:

IOException - if there is an error copying the file

InterruptedException - if the copier should give up