DataChecksum (Apache Hadoop Common 3.4.1.200-dep-1000 API)

java.lang.Object
- org.apache.hadoop.util.DataChecksum

All Implemented Interfaces:

java.util.zip.Checksum
```
@LimitedPrivate({"HDFS","MapReduce"})
@Evolving
public class DataChecksum
extends java.lang.Object
implements java.util.zip.Checksum
```
This class provides interface and utilities for processing checksums for DFS data transfers.

Nested Class Summary

Nested Classes
Modifier and Type Class Description

static class DataChecksum.Type
The checksum types

Nested Classes
Modifier and Type	Class	Description
`static class`	`DataChecksum.Type`	The checksum types

Field Summary

Fields
Modifier and Type	Field	Description
`static int`	`CHECKSUM_CRC32`
`static int`	`CHECKSUM_CRC32C`
`static int`	`CHECKSUM_DEFAULT`
`static int`	`CHECKSUM_MIXED`
`static int`	`CHECKSUM_NULL`
`static int`	`SIZE_OF_INTEGER`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`calculateChunkedSums(byte[] data, int dataOffset, int dataLength, byte[] sums, int sumsOffset)`	Implementation of chunked calculation specifically on byte arrays.
`void`	`calculateChunkedSums(java.nio.ByteBuffer data, java.nio.ByteBuffer checksums)`	Calculate checksums for the given data.
`boolean`	`compare(byte[] buf, int offset)`	Compares the checksum located at buf[offset] with the current checksum.
`boolean`	`equals(java.lang.Object other)`
`int`	`getBytesPerChecksum()`
`static int`	`getChecksumHeaderSize()`
`int`	`getChecksumSize()`	the size for a checksum.
`int`	`getChecksumSize(int dataSize)`	the required checksum size given the data length.
`DataChecksum.Type`	`getChecksumType()`
`static int`	`getCrcPolynomialForType(DataChecksum.Type type)`	getCrcPolynomialForType.
`byte[]`	`getHeader()`
`int`	`getNumBytesInSum()`
`long`	`getValue()`
`int`	`hashCode()`
`static java.util.zip.Checksum`	`newCrc32()`	Create a Crc32 Checksum object.
`static DataChecksum`	`newDataChecksum(byte[] bytes, int offset)`	Creates a DataChecksum from HEADER_LEN bytes from arr[offset].
`static DataChecksum`	`newDataChecksum(java.io.DataInputStream in)`	This constructs a DataChecksum by reading HEADER_LEN bytes from input stream in.
`static DataChecksum`	`newDataChecksum(DataChecksum.Type type, int bytesPerChecksum)`
`void`	`reset()`
`java.lang.String`	`toString()`
`void`	`update(byte[] b, int off, int len)`
`void`	`update(int b)`
`void`	`verifyChunkedSums(java.nio.ByteBuffer data, java.nio.ByteBuffer checksums, java.lang.String fileName, long basePos)`	Verify that the given checksums match the given data.
`void`	`writeHeader(java.io.DataOutputStream out)`	Writes the checksum header to the output stream out.
`int`	`writeValue(byte[] buf, int offset, boolean reset)`	Writes the current checksum to a buffer.
`int`	`writeValue(java.io.DataOutputStream out, boolean reset)`	Writes the current checksum to the stream.

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.zip.Checksum
update, update

- Field Detail
  - CHECKSUM_NULL
```
public static final int CHECKSUM_NULL
```
    See Also:
    
    Constant Field Values
  - CHECKSUM_CRC32
```
public static final int CHECKSUM_CRC32
```
    See Also:
    
    Constant Field Values
  - CHECKSUM_CRC32C
```
public static final int CHECKSUM_CRC32C
```
    See Also:
    
    Constant Field Values
  - CHECKSUM_DEFAULT
```
public static final int CHECKSUM_DEFAULT
```
    See Also:
    
    Constant Field Values
  - CHECKSUM_MIXED
```
public static final int CHECKSUM_MIXED
```
    See Also:
    
    Constant Field Values
  - SIZE_OF_INTEGER
```
public static final int SIZE_OF_INTEGER
```
    See Also:
    
    Constant Field Values
- Method Detail
  - newCrc32
```
public static java.util.zip.Checksum newCrc32()
```
    Create a Crc32 Checksum object. The implementation of the Crc32 algorithm is chosen depending on the platform.
    
    Returns:
    
    Checksum.
  - getCrcPolynomialForType
```
public static int getCrcPolynomialForType(DataChecksum.Type type)
                                   throws java.io.IOException
```
    getCrcPolynomialForType.
    
    Parameters:
    
    type - type.
    
    Returns:
    
    the int representation of the polynomial associated with the CRC type, suitable for use with further CRC arithmetic.
    
    Throws:
    
    java.io.IOException - if there is no CRC polynomial applicable to the given type.
  - newDataChecksum
```
public static DataChecksum newDataChecksum(DataChecksum.Type type,
                                           int bytesPerChecksum)
```
  - newDataChecksum
```
public static DataChecksum newDataChecksum(byte[] bytes,
                                           int offset)
                                    throws java.io.IOException
```
    Creates a DataChecksum from HEADER_LEN bytes from arr[offset].
    
    Parameters:
    
    bytes - bytes.
    
    offset - offset.
    
    Returns:
    
    DataChecksum of the type in the array or null in case of an error.
    
    Throws:
    
    java.io.IOException - raised on errors performing I/O.
  - newDataChecksum
```
public static DataChecksum newDataChecksum(java.io.DataInputStream in)
                                    throws java.io.IOException
```
    This constructs a DataChecksum by reading HEADER_LEN bytes from input stream in.
    
    Parameters:
    
    in - data input stream.
    
    Returns:
    
    DataChecksum by reading HEADER_LEN bytes from input stream.
    
    Throws:
    
    java.io.IOException - raised on errors performing I/O.
  - writeHeader
```
public void writeHeader(java.io.DataOutputStream out)
                 throws java.io.IOException
```
    Writes the checksum header to the output stream out.
    
    Parameters:
    
    out - output stream.
    
    Throws:
    
    java.io.IOException - raised on errors performing I/O.
  - getHeader
```
public byte[] getHeader()
```
  - writeValue
```
public int writeValue(java.io.DataOutputStream out,
                      boolean reset)
               throws java.io.IOException
```
    Writes the current checksum to the stream. If reset is true, then resets the checksum.
    
    Parameters:
    
    out - out.
    
    reset - reset.
    
    Returns:
    
    number of bytes written. Will be equal to getChecksumSize();
    
    Throws:
    
    java.io.IOException - raised on errors performing I/O.
  - writeValue
```
public int writeValue(byte[] buf,
                      int offset,
                      boolean reset)
               throws java.io.IOException
```
    Writes the current checksum to a buffer. If reset is true, then resets the checksum.
    
    Parameters:
    
    buf - buf.
    
    offset - offset.
    
    reset - reset.
    
    Returns:
    
    number of bytes written. Will be equal to getChecksumSize();
    
    Throws:
    
    java.io.IOException - raised on errors performing I/O.
  - compare
```
public boolean compare(byte[] buf,
                       int offset)
```
    Compares the checksum located at buf[offset] with the current checksum.
    
    Parameters:
    
    buf - buf.
    
    offset - offset.
    
    Returns:
    
    true if the checksum matches and false otherwise.
  - getChecksumType
```
public DataChecksum.Type getChecksumType()
```
    Returns:
    
    the checksum algorithm type.
  - getChecksumSize
```
public int getChecksumSize()
```
    the size for a checksum.
    
    Returns:
    
    the size for a checksum.
  - getChecksumSize
```
public int getChecksumSize(int dataSize)
```
    the required checksum size given the data length.
    
    Parameters:
    
    dataSize - data size.
    
    Returns:
    
    the required checksum size given the data length.
  - getBytesPerChecksum
```
public int getBytesPerChecksum()
```
  - getNumBytesInSum
```
public int getNumBytesInSum()
```
  - getChecksumHeaderSize
```
public static int getChecksumHeaderSize()
```
  - getValue
```
public long getValue()
```
    Specified by:
    
    getValue in interface java.util.zip.Checksum
  - reset
```
public void reset()
```
    Specified by:
    
    reset in interface java.util.zip.Checksum
  - update
```
public void update(byte[] b,
                   int off,
                   int len)
```
    Specified by:
    
    update in interface java.util.zip.Checksum
  - update
```
public void update(int b)
```
    Specified by:
    
    update in interface java.util.zip.Checksum
  - verifyChunkedSums
```
public void verifyChunkedSums(java.nio.ByteBuffer data,
                              java.nio.ByteBuffer checksums,
                              java.lang.String fileName,
                              long basePos)
                       throws ChecksumException
```
    Verify that the given checksums match the given data. The 'mark' of the ByteBuffer parameters may be modified by this function,. but the position is maintained.
    
    Parameters:
    
    data - the DirectByteBuffer pointing to the data to verify.
    
    checksums - the DirectByteBuffer pointing to a series of stored checksums
    
    fileName - the name of the file being read, for error-reporting
    
    basePos - the file position to which the start of 'data' corresponds
    
    Throws:
    
    ChecksumException - if the checksums do not match
  - calculateChunkedSums
```
public void calculateChunkedSums(java.nio.ByteBuffer data,
                                 java.nio.ByteBuffer checksums)
```
    Calculate checksums for the given data. The 'mark' of the ByteBuffer parameters may be modified by this function, but the position is maintained.
    
    Parameters:
    
    data - the DirectByteBuffer pointing to the data to checksum.
    
    checksums - the DirectByteBuffer into which checksums will be stored. Enough space must be available in this buffer to put the checksums.
  - calculateChunkedSums
```
public void calculateChunkedSums(byte[] data,
                                 int dataOffset,
                                 int dataLength,
                                 byte[] sums,
                                 int sumsOffset)
```
    Implementation of chunked calculation specifically on byte arrays. This is to avoid the copy when dealing with ByteBuffers that have array backing.
    
    Parameters:
    
    data - data.
    
    dataOffset - dataOffset.
    
    dataLength - dataLength.
    
    sums - sums.
    
    sumsOffset - sumsOffset.
  - equals
```
public boolean equals(java.lang.Object other)
```
    Overrides:
    
    equals in class java.lang.Object
  - hashCode
```
public int hashCode()
```
    Overrides:
    
    hashCode in class java.lang.Object
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object

Class DataChecksum

Nested Class Summary

Field Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface java.util.zip.Checksum

Field Detail

CHECKSUM_NULL

CHECKSUM_CRC32

CHECKSUM_CRC32C

CHECKSUM_DEFAULT

CHECKSUM_MIXED

SIZE_OF_INTEGER

Method Detail

newCrc32

getCrcPolynomialForType

newDataChecksum

newDataChecksum

newDataChecksum

writeHeader

getHeader

writeValue

writeValue

compare

getChecksumType

getChecksumSize

getChecksumSize

getBytesPerChecksum

getNumBytesInSum

getChecksumHeaderSize

getValue

reset

update

update

verifyChunkedSums

calculateChunkedSums

calculateChunkedSums

equals

hashCode

toString