Package org.apache.hadoop.tools.mapred.lib

Class DynamicRecordReader<K,V>

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<K,V>

org.apache.hadoop.tools.mapred.lib.DynamicRecordReader<K,V>

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public class DynamicRecordReader<K,V>
extends org.apache.hadoop.mapreduce.RecordReader<K,V>

The DynamicRecordReader is used in conjunction with the DynamicInputFormat to implement the "Worker pattern" for DistCp. The DynamicRecordReader is responsible for: 1. Presenting the contents of each chunk to DistCp's mapper. 2. Acquiring a new chunk when the current chunk has been completely consumed, transparently.

Method Summary

Modifier and Type	Method	Description
void	close()	Implementation of RecordReader::close().
K	getCurrentKey()	Implementation of RecordReader::getCurrentKey().
V	getCurrentValue()	Implementation of RecordReader::getCurrentValue().
float	getProgress()	Implementation of RecordReader::getProgress().
void	initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)	Implementation for RecordReader::initialize().
boolean	nextKeyValue()	Implementation of RecordReader::nextValue().

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                       org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
                throws java.io.IOException,
                       java.lang.InterruptedException

Implementation for RecordReader::initialize(). Initializes the internal RecordReader to read from chunks.

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Parameters:

inputSplit - The InputSplit for the map. Ignored entirely.

taskAttemptContext - The AttemptContext.

Throws:

java.io.IOException

java.lang.InterruptedException

nextKeyValue

public boolean nextKeyValue()
                     throws java.io.IOException,
                            java.lang.InterruptedException

Implementation of RecordReader::nextValue(). Reads the contents of the current chunk and returns them. When a chunk has been completely exhausted, an new chunk is acquired and read, transparently.

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Returns:

True, if the nextValue() could be traversed to. False, otherwise.

Throws:

java.io.IOException

java.lang.InterruptedException

getCurrentKey

public K getCurrentKey()
                throws java.io.IOException,
                       java.lang.InterruptedException

Implementation of RecordReader::getCurrentKey().

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Returns:

The key of the current record. (i.e. the source-path.)

Throws:

java.io.IOException

java.lang.InterruptedException

getCurrentValue

public V getCurrentValue()
                  throws java.io.IOException,
                         java.lang.InterruptedException

Implementation of RecordReader::getCurrentValue().

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Returns:

The value of the current record. (i.e. the target-path.)

Throws:

java.io.IOException

java.lang.InterruptedException

getProgress

public float getProgress()
                  throws java.io.IOException,
                         java.lang.InterruptedException

Implementation of RecordReader::getProgress().

Specified by:

getProgress in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Returns:

A fraction [0.0,1.0] indicating the progress of a DistCp mapper.

Throws:

java.io.IOException

java.lang.InterruptedException

close

public void close()
           throws java.io.IOException

Implementation of RecordReader::close(). Closes the RecordReader.

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Specified by:

close in class org.apache.hadoop.mapreduce.RecordReader<K,V>

Throws:

java.io.IOException