org.apache.hadoop.mapreduce.task.reduce
Class DirectShuffleFetcher<K,V>

java.lang.Object
  java.lang.Thread
      org.apache.hadoop.mapreduce.task.reduce.DirectShuffleFetcher<K,V>

All Implemented Interfaces:

Runnable

public class DirectShuffleFetcher<K,V>
extends Thread

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

Thread.State, Thread.UncaughtExceptionHandler

Field Summary

protected org.apache.hadoop.fs.FileSystem

rfs

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

DirectShuffleFetcher(int id, org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.mapreduce.TaskAttemptID reduceId, DirectShuffleSchedulerImpl<K,V> scheduler, org.apache.hadoop.mapreduce.task.reduce.MergeManager<K,V> merger, org.apache.hadoop.mapred.Reporter reporter, org.apache.hadoop.mapreduce.task.reduce.ShuffleClientMetrics metrics, org.apache.hadoop.mapreduce.task.reduce.ExceptionReporter exceptionReporter, org.apache.hadoop.mapred.MapOutputFile mapOutputFile)
Represents the result of an attempt to copy a map output

Method Summary

protected long	copyOutput(MapOutputLocation loc) Copies a a map output from a remote host.
boolean	fail() Fail the current file that we are fetching
MapOutputLocation	getLocation() Get the current map output location.
void	run() Loop forever and fetch map outputs as they become available.
void	shutDown()

Methods inherited from class java.lang.Thread

activeCount, checkAccess, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

rfs

protected org.apache.hadoop.fs.FileSystem rfs

Constructor Detail

DirectShuffleFetcher

public DirectShuffleFetcher(int id,
                            org.apache.hadoop.mapred.JobConf jobConf,
                            org.apache.hadoop.mapreduce.TaskAttemptID reduceId,
                            DirectShuffleSchedulerImpl<K,V> scheduler,
                            org.apache.hadoop.mapreduce.task.reduce.MergeManager<K,V> merger,
                            org.apache.hadoop.mapred.Reporter reporter,
                            org.apache.hadoop.mapreduce.task.reduce.ShuffleClientMetrics metrics,
                            org.apache.hadoop.mapreduce.task.reduce.ExceptionReporter exceptionReporter,
                            org.apache.hadoop.mapred.MapOutputFile mapOutputFile)

Represents the result of an attempt to copy a map output

Method Detail

fail

public boolean fail()

Fail the current file that we are fetching

Returns:

were we currently fetching?

getLocation

public MapOutputLocation getLocation()

Get the current map output location.

shutDown

public void shutDown()

run

public void run()

Loop forever and fetch map outputs as they become available.

Specified by:

run in interface Runnable

Overrides:

run in class Thread

copyOutput

protected long copyOutput(MapOutputLocation loc)
                   throws IOException,
                          InterruptedException

Copies a a map output from a remote host.

Throws:

IOException - if there is an error copying the file

InterruptedException - if the copier should give up