org.apache.hadoop.mapred

Class MapRFsOutputBuffer<K,V>

java.lang.Object

org.apache.hadoop.mapred.MapRFsOutputBuffer<K,V>

All Implemented Interfaces:

org.apache.hadoop.mapred.MapOutputCollector<K,V>, org.apache.hadoop.util.IndexedSortable

public class MapRFsOutputBuffer<K,V>
extends Object
implements org.apache.hadoop.mapred.MapOutputCollector<K,V>, org.apache.hadoop.util.IndexedSortable

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	MapRFsOutputBuffer.BlockingBuffer Inner class managing the spill of serialized records to disk.
class	MapRFsOutputBuffer.Buffer
protected class	MapRFsOutputBuffer.InMemValBytes Inner class wrapping valuebytes, used for appendRaw.

Nested classes/interfaces inherited from interface org.apache.hadoop.mapred.MapOutputCollector

org.apache.hadoop.mapred.MapOutputCollector.Context

Constructor Summary

Constructors

Constructor and Description
MapRFsOutputBuffer()

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	collect(K key, V value, int partition) Serialize the key, value to intermediate storage.
int	compare(int mi, int mj) Compare logical range, st i, j MOD offset capacity.
void	flush()
void	init(org.apache.hadoop.mapred.MapOutputCollector.Context context)
protected void	mergeParts()
protected void	sortAndSpill()
protected boolean	sortAndSpillWrapper()
void	swap(int mi, int mj) Swap metadata for items i, j

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapRFsOutputBuffer

public MapRFsOutputBuffer()

Method Detail

init

public void init(org.apache.hadoop.mapred.MapOutputCollector.Context context)
          throws IOException,
                 ClassNotFoundException

Specified by:

init in interface org.apache.hadoop.mapred.MapOutputCollector<K,V>

Throws:

IOException

ClassNotFoundException

compare

public int compare(int mi,
          int mj)

Compare logical range, st i, j MOD offset capacity. Compare by partition, then by key.

Specified by:

compare in interface org.apache.hadoop.util.IndexedSortable

See Also:

IndexedSortable.compare(int, int)

swap

public void swap(int mi,
        int mj)

Swap metadata for items i, j

Specified by:

swap in interface org.apache.hadoop.util.IndexedSortable

See Also:

IndexedSortable.swap(int, int)

close

public void close()

Specified by:

close in interface org.apache.hadoop.mapred.MapOutputCollector<K,V>

sortAndSpillWrapper

protected boolean sortAndSpillWrapper()
                               throws IOException,
                                      ClassNotFoundException,
                                      InterruptedException

Throws:

IOException

ClassNotFoundException

InterruptedException

sortAndSpill

protected void sortAndSpill()
                     throws IOException,
                            ClassNotFoundException,
                            InterruptedException

Throws:

IOException

ClassNotFoundException

InterruptedException

mergeParts

protected void mergeParts()
                   throws IOException,
                          InterruptedException,
                          ClassNotFoundException

Throws:

IOException

InterruptedException

ClassNotFoundException

collect

public void collect(K key,
           V value,
           int partition)
             throws IOException

Serialize the key, value to intermediate storage. When this method returns, kvindex must refer to sufficient unused storage to store one METADATA.

Specified by:

collect in interface org.apache.hadoop.mapred.MapOutputCollector<K,V>

Throws:

IOException

flush

public void flush()
           throws IOException,
                  ClassNotFoundException,
                  InterruptedException

Specified by:

flush in interface org.apache.hadoop.mapred.MapOutputCollector<K,V>

Throws:

IOException

ClassNotFoundException

InterruptedException