Package org.apache.hadoop.fs.impl.prefetch

Class CachingBlockManager

java.lang.Object

org.apache.hadoop.fs.impl.prefetch.BlockManager

org.apache.hadoop.fs.impl.prefetch.CachingBlockManager

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public abstract class CachingBlockManager
extends BlockManager

Provides read access to the underlying file one block at a time. Improve read performance by prefetching and locall caching blocks.

Constructor Summary

Constructors

Constructor	Description
CachingBlockManager(BlockManagerParameters blockManagerParameters)	Constructs an instance of a CachingBlockManager.

Method Summary

Modifier and Type	Method	Description
protected void	cachePut(int blockNumber, java.nio.ByteBuffer buffer)
void	cancelPrefetches()	Requests cancellation of any previously issued prefetch requests.
void	close()
protected BlockCache	createCache(int maxBlocksCount, DurationTrackerFactory trackerFactory)
BufferData	get(int blockNumber)	Gets the block having the given blockNumber.
int	numAvailable()	Number of ByteBuffers available to be acquired.
int	numCached()	Number of caching operations completed.
int	numCachingErrors()	Number of errors encountered when caching.
int	numReadErrors()	Number of errors encountered when reading.
void	release(BufferData data)	Releases resources allocated to the given block.
void	requestCaching(BufferData data)	Requests that the given block should be copied to the local cache.
void	requestPrefetch(int blockNumber)	Requests optional prefetching of the given block.
java.lang.String	toString()

Methods inherited from class org.apache.hadoop.fs.impl.prefetch.BlockManager

getBlockData, read

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CachingBlockManager

public CachingBlockManager(@Nonnull
                           BlockManagerParameters blockManagerParameters)

Constructs an instance of a CachingBlockManager.

Parameters:

blockManagerParameters - params for block manager.

Throws:

java.lang.IllegalArgumentException - if bufferPoolSize is zero or negative.

Method Detail

get

public BufferData get(int blockNumber)
               throws java.io.IOException

Gets the block having the given blockNumber.

Overrides:

get in class BlockManager

Parameters:

blockNumber - the number of the block to be read and returned.

Returns:

BufferData having data from the given block.

Throws:

java.lang.IllegalArgumentException - if blockNumber is negative.

java.io.IOException - if there an error reading the given block.

release

public void release(BufferData data)

Releases resources allocated to the given block.

Overrides:

release in class BlockManager

Parameters:

data - the BufferData to release.

Throws:

java.lang.IllegalArgumentException - if data is null.

close

public void close()

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Overrides:

close in class BlockManager

requestPrefetch

public void requestPrefetch(int blockNumber)

Requests optional prefetching of the given block. The block is prefetched only if we can acquire a free buffer.

Overrides:

requestPrefetch in class BlockManager

Parameters:

blockNumber - the id of the block to prefetch.

Throws:

java.lang.IllegalArgumentException - if blockNumber is negative.

cancelPrefetches

public void cancelPrefetches()

Requests cancellation of any previously issued prefetch requests.

Overrides:

cancelPrefetches in class BlockManager

requestCaching

public void requestCaching(BufferData data)

Requests that the given block should be copied to the local cache. The block must not be accessed by the caller after calling this method because it will released asynchronously relative to the caller.

Overrides:

requestCaching in class BlockManager

Parameters:

data - the BufferData instance to optionally cache.

Throws:

java.lang.IllegalArgumentException - if data is null.

createCache

protected BlockCache createCache(int maxBlocksCount,
                                 DurationTrackerFactory trackerFactory)

cachePut

protected void cachePut(int blockNumber,
                        java.nio.ByteBuffer buffer)
                 throws java.io.IOException

Throws:

java.io.IOException

numAvailable

public int numAvailable()

Number of ByteBuffers available to be acquired.

Returns:

the number of available buffers.

numCached

public int numCached()

Number of caching operations completed.

Returns:

the number of cached buffers.

numCachingErrors

public int numCachingErrors()

Number of errors encountered when caching.

Returns:

the number of errors encountered when caching.

numReadErrors

public int numReadErrors()

Number of errors encountered when reading.

Returns:

the number of errors encountered when reading.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object