Package org.apache.hadoop.mapred

Class MapRFsOutputBuffer<K,V>

java.lang.Object

org.apache.hadoop.mapred.MapRFsOutputBuffer<K,V>

All Implemented Interfaces:

org.apache.hadoop.mapred.MapOutputCollector<K,V>, org.apache.hadoop.util.IndexedSortable

public class MapRFsOutputBuffer<K,V>
extends java.lang.Object
implements org.apache.hadoop.mapred.MapOutputCollector<K,V>, org.apache.hadoop.util.IndexedSortable

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	MapRFsOutputBuffer.BlockingBuffer	Inner class managing the spill of serialized records to disk.
class	MapRFsOutputBuffer.Buffer
protected class	MapRFsOutputBuffer.InMemValBytes	Inner class wrapping valuebytes, used for appendRaw.

Nested classes/interfaces inherited from interface org.apache.hadoop.mapred.MapOutputCollector

org.apache.hadoop.mapred.MapOutputCollector.Context

Constructor Summary

Constructors

Constructor	Description
MapRFsOutputBuffer()

Method Summary

Modifier and Type	Method	Description
void	close()
void	collect(K key, V value, int partition)	Serialize the key, value to intermediate storage.
int	compare(int mi, int mj)	Compare logical range, st i, j MOD offset capacity.
void	flush()
void	init(org.apache.hadoop.mapred.MapOutputCollector.Context context)
protected void	mergeParts()
protected void	sortAndSpill()
protected boolean	sortAndSpillWrapper()
void	swap(int mi, int mj)	Swap metadata for items i, j

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapRFsOutputBuffer

public MapRFsOutputBuffer()

Method Detail

init

public void init(org.apache.hadoop.mapred.MapOutputCollector.Context context)
          throws java.io.IOException,
                 java.lang.ClassNotFoundException

Specified by:

init in interface org.apache.hadoop.mapred.MapOutputCollector<K,V>

Throws:

java.io.IOException

java.lang.ClassNotFoundException

compare

public int compare(int mi,
                   int mj)

Compare logical range, st i, j MOD offset capacity. Compare by partition, then by key.

Specified by:

compare in interface org.apache.hadoop.util.IndexedSortable

See Also:

IndexedSortable.compare(int, int)

swap

public void swap(int mi,
                 int mj)

Swap metadata for items i, j

Specified by:

swap in interface org.apache.hadoop.util.IndexedSortable

See Also:

IndexedSortable.swap(int, int)

close

public void close()

Specified by:

close in interface org.apache.hadoop.mapred.MapOutputCollector<K,V>

sortAndSpillWrapper

protected boolean sortAndSpillWrapper()
                               throws java.io.IOException,
                                      java.lang.ClassNotFoundException,
                                      java.lang.InterruptedException

Throws:

java.io.IOException

java.lang.ClassNotFoundException

java.lang.InterruptedException

sortAndSpill

protected void sortAndSpill()
                     throws java.io.IOException,
                            java.lang.ClassNotFoundException,
                            java.lang.InterruptedException

Throws:

java.io.IOException

java.lang.ClassNotFoundException

java.lang.InterruptedException

mergeParts

protected void mergeParts()
                   throws java.io.IOException,
                          java.lang.InterruptedException,
                          java.lang.ClassNotFoundException

Throws:

java.io.IOException

java.lang.InterruptedException

java.lang.ClassNotFoundException

collect

public void collect(K key,
                    V value,
                    int partition)
             throws java.io.IOException

Serialize the key, value to intermediate storage. When this method returns, kvindex must refer to sufficient unused storage to store one METADATA.

Specified by:

collect in interface org.apache.hadoop.mapred.MapOutputCollector<K,V>

Throws:

java.io.IOException

flush

public void flush()
           throws java.io.IOException,
                  java.lang.ClassNotFoundException,
                  java.lang.InterruptedException

Specified by:

flush in interface org.apache.hadoop.mapred.MapOutputCollector<K,V>

Throws:

java.io.IOException

java.lang.ClassNotFoundException

java.lang.InterruptedException