Package org.apache.hadoop.io.erasurecode.rawcoder

Class RawErasureEncoder

java.lang.Object

org.apache.hadoop.io.erasurecode.rawcoder.RawErasureEncoder

Direct Known Subclasses:

DummyRawEncoder, NativeRSRawEncoder, NativeXORRawEncoder, RSLegacyRawEncoder, RSRawEncoder, XORRawEncoder

@Private
public abstract class RawErasureEncoder
extends java.lang.Object

An abstract raw erasure encoder that's to be inherited by new encoders. Raw erasure coder is part of erasure codec framework, where erasure coder is used to encode/decode a group of blocks (BlockGroup) according to the codec specific BlockGroup layout and logic. An erasure coder extracts chunks of data from the blocks and can employ various low level raw erasure coders to perform encoding/decoding against the chunks. To distinguish from erasure coder, here raw erasure coder is used to mean the low level constructs, since it only takes care of the math calculation with a group of byte buffers. Note it mainly provides encode() calls, which should be stateless and may be made thread-safe in future.

Constructor Summary

Constructors

Constructor	Description
RawErasureEncoder(ErasureCoderOptions coderOptions)

Method Summary

Modifier and Type	Method	Description
boolean	allowChangeInputs()	Allow change into input buffers or not while perform encoding/decoding.
boolean	allowVerboseDump()	Allow to dump verbose info during encoding/decoding.
protected abstract void	doEncode(org.apache.hadoop.io.erasurecode.rawcoder.ByteArrayEncodingState encodingState)	Perform the real encoding work using bytes array, supporting offsets and lengths.
protected abstract void	doEncode(org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferEncodingState encodingState)	Perform the real encoding work using direct ByteBuffer.
void	encode(byte[][] inputs, byte[][] outputs)	Encode with inputs and generates outputs.
void	encode(java.nio.ByteBuffer[] inputs, java.nio.ByteBuffer[] outputs)	Encode with inputs and generates outputs.
void	encode(ECChunk[] inputs, ECChunk[] outputs)	Encode with inputs and generates outputs.
int	getNumAllUnits()
int	getNumDataUnits()
int	getNumParityUnits()
boolean	preferDirectBuffer()	Tell if direct buffer is preferred or not.
void	release()	Should be called when release this coder.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RawErasureEncoder

public RawErasureEncoder(ErasureCoderOptions coderOptions)

Method Detail

encode

public void encode(java.nio.ByteBuffer[] inputs,
                   java.nio.ByteBuffer[] outputs)
            throws java.io.IOException

Encode with inputs and generates outputs. Note, for both inputs and outputs, no mixing of on-heap buffers and direct buffers are allowed. If the coder option ALLOW_CHANGE_INPUTS is set true (false by default), the content of input buffers may change after the call, subject to concrete implementation. Anyway the positions of input buffers will move forward.

Parameters:

inputs - input buffers to read data from. The buffers' remaining will be 0 after encoding

outputs - output buffers to put the encoded data into, ready to read after the call

Throws:

java.io.IOException - if the encoder is closed.

doEncode

protected abstract void doEncode(org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferEncodingState encodingState)
                          throws java.io.IOException

Perform the real encoding work using direct ByteBuffer.

Parameters:

encodingState - the encoding state.

Throws:

java.io.IOException - raised on errors performing I/O.

encode

public void encode(byte[][] inputs,
                   byte[][] outputs)
            throws java.io.IOException

Encode with inputs and generates outputs. More see above.

Parameters:

inputs - input buffers to read data from

outputs - output buffers to put the encoded data into, read to read after the call

Throws:

java.io.IOException - raised on errors performing I/O.

doEncode

protected abstract void doEncode(org.apache.hadoop.io.erasurecode.rawcoder.ByteArrayEncodingState encodingState)
                          throws java.io.IOException

Perform the real encoding work using bytes array, supporting offsets and lengths.

Parameters:

encodingState - the encoding state

Throws:

java.io.IOException - raised on errors performing I/O.

encode

public void encode(ECChunk[] inputs,
                   ECChunk[] outputs)
            throws java.io.IOException

Encode with inputs and generates outputs. More see above.

Parameters:

inputs - input buffers to read data from

outputs - output buffers to put the encoded data into, read to read after the call

Throws:

java.io.IOException - if the encoder is closed.

getNumDataUnits

public int getNumDataUnits()

getNumParityUnits

public int getNumParityUnits()

getNumAllUnits

public int getNumAllUnits()

preferDirectBuffer

public boolean preferDirectBuffer()

Tell if direct buffer is preferred or not. It's for callers to decide how to allocate coding chunk buffers, using DirectByteBuffer or bytes array. It will return false by default.

Returns:

true if native buffer is preferred for performance consideration, otherwise false.

allowChangeInputs

public boolean allowChangeInputs()

Allow change into input buffers or not while perform encoding/decoding.

Returns:

true if it's allowed to change inputs, false otherwise

allowVerboseDump

public boolean allowVerboseDump()

Allow to dump verbose info during encoding/decoding.

Returns:

true if it's allowed to do verbose dump, false otherwise.

release

public void release()

Should be called when release this coder. Good chance to release encoding or decoding buffers