org.apache.hadoop.examples

Class SleepJob

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.examples.SleepJob

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable>, org.apache.hadoop.mapred.Partitioner<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable>, org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>, org.apache.hadoop.util.Tool

public class SleepJob
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool, org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable>, org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>, org.apache.hadoop.mapred.Partitioner<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable>

Dummy class for testing MR framefork. Sleeps for a defined period of time in mapper and reducer. Generates fake input for map / reduce jobs. Note that generated number of input pairs is in the order of numMappers * mapSleepTime / 100, so the job uses some disk space.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SleepJob.EmptySplit
static class	SleepJob.SleepInputFormat

Constructor Summary

Constructors

Constructor and Description
SleepJob()

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	configure(org.apache.hadoop.mapred.JobConf job)
int	getPartition(org.apache.hadoop.io.IntWritable k, org.apache.hadoop.io.NullWritable v, int numPartitions)
static void	main(String[] args)
void	map(org.apache.hadoop.io.IntWritable key, org.apache.hadoop.io.IntWritable value, org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable> output, org.apache.hadoop.mapred.Reporter reporter)
void	reduce(org.apache.hadoop.io.IntWritable key, Iterator<org.apache.hadoop.io.NullWritable> values, org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable> output, org.apache.hadoop.mapred.Reporter reporter)
int	run(int numMapper, int numReducer, long mapSleepTime, int mapSleepCount, long reduceSleepTime, int reduceSleepCount)
int	run(String[] args)
org.apache.hadoop.mapred.JobConf	setupJobConf(int numMapper, int numReducer, long mapSleepTime, int mapSleepCount, long reduceSleepTime, int reduceSleepCount)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Constructor Detail

SleepJob

public SleepJob()

Method Detail

getPartition

public int getPartition(org.apache.hadoop.io.IntWritable k,
               org.apache.hadoop.io.NullWritable v,
               int numPartitions)

Specified by:

getPartition in interface org.apache.hadoop.mapred.Partitioner<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable>

map

public void map(org.apache.hadoop.io.IntWritable key,
       org.apache.hadoop.io.IntWritable value,
       org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable> output,
       org.apache.hadoop.mapred.Reporter reporter)
         throws IOException

Specified by:

map in interface org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable>

Throws:

IOException

reduce

public void reduce(org.apache.hadoop.io.IntWritable key,
          Iterator<org.apache.hadoop.io.NullWritable> values,
          org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable> output,
          org.apache.hadoop.mapred.Reporter reporter)
            throws IOException

Specified by:

reduce in interface org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.IntWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.NullWritable>

Throws:

IOException

configure

public void configure(org.apache.hadoop.mapred.JobConf job)

Specified by:

configure in interface org.apache.hadoop.mapred.JobConfigurable

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception

run

public int run(int numMapper,
      int numReducer,
      long mapSleepTime,
      int mapSleepCount,
      long reduceSleepTime,
      int reduceSleepCount)
        throws IOException

Throws:

IOException

setupJobConf

public org.apache.hadoop.mapred.JobConf setupJobConf(int numMapper,
                                            int numReducer,
                                            long mapSleepTime,
                                            int mapSleepCount,
                                            long reduceSleepTime,
                                            int reduceSleepCount)

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception