Package org.apache.hadoop.fs.impl.prefetch

Class SingleFilePerBlockCache

java.lang.Object

org.apache.hadoop.fs.impl.prefetch.SingleFilePerBlockCache

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, BlockCache

public class SingleFilePerBlockCache
extends java.lang.Object
implements BlockCache

Provides functionality necessary for caching blocks of data read from FileSystem. Each cache block is stored on the local disk as a separate file.

Constructor Summary

Constructors

Constructor	Description
SingleFilePerBlockCache(PrefetchingStatistics prefetchingStatistics, int maxBlocksCount, DurationTrackerFactory trackerFactory)	Constructs an instance of a SingleFilePerBlockCache.

Method Summary

Modifier and Type	Method	Description
java.lang.Iterable<java.lang.Integer>	blocks()	Gets the blocks in this cache.
void	close()
boolean	containsBlock(int blockNumber)	Indicates whether the given block is in this cache.
void	get(int blockNumber, java.nio.ByteBuffer buffer)	Gets the block having the given blockNumber.
protected java.nio.file.Path	getCacheFilePath(Configuration conf, LocalDirAllocator localDirAllocator)	Return temporary file created based on the file path retrieved from local dir allocator.
static boolean	isCacheSpaceAvailable(long fileSize, Configuration conf, LocalDirAllocator localDirAllocator)	Determine if the cache space is available on the local FS.
void	put(int blockNumber, java.nio.ByteBuffer buffer, Configuration conf, LocalDirAllocator localDirAllocator)	Puts the given block in this cache.
protected int	readFile(java.nio.file.Path path, java.nio.ByteBuffer buffer)
int	size()	Gets the number of blocks in this cache.
java.lang.String	toString()
protected void	writeFile(java.nio.file.Path path, java.nio.ByteBuffer buffer)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

SingleFilePerBlockCache

public SingleFilePerBlockCache(PrefetchingStatistics prefetchingStatistics,
                               int maxBlocksCount,
                               DurationTrackerFactory trackerFactory)

Constructs an instance of a SingleFilePerBlockCache.

Parameters:

prefetchingStatistics - statistics for this stream.

maxBlocksCount - max blocks count to be kept in cache at any time.

trackerFactory - tracker with statistics to update

Method Detail

containsBlock

public boolean containsBlock(int blockNumber)

Indicates whether the given block is in this cache.

Specified by:

containsBlock in interface BlockCache

Parameters:

blockNumber - the id of the given block.

Returns:

true if the given block is in this cache, false otherwise.

blocks

public java.lang.Iterable<java.lang.Integer> blocks()

Gets the blocks in this cache.

Specified by:

blocks in interface BlockCache

Returns:

the blocks in this cache.

size

public int size()

Gets the number of blocks in this cache.

Specified by:

size in interface BlockCache

Returns:

the number of blocks in this cache.

get

public void get(int blockNumber,
                java.nio.ByteBuffer buffer)
         throws java.io.IOException

Gets the block having the given blockNumber.

Specified by:

get in interface BlockCache

Parameters:

blockNumber - the id of the desired block.

buffer - contents of the desired block are copied to this buffer.

Throws:

java.lang.IllegalArgumentException - if buffer is null.

java.io.IOException - if there is an error reading the given block.

readFile

protected int readFile(java.nio.file.Path path,
                       java.nio.ByteBuffer buffer)
                throws java.io.IOException

Throws:

java.io.IOException

put

public void put(int blockNumber,
                java.nio.ByteBuffer buffer,
                Configuration conf,
                LocalDirAllocator localDirAllocator)
         throws java.io.IOException

Puts the given block in this cache.

Specified by:

put in interface BlockCache

Parameters:

blockNumber - the block number, used as a key for blocks map.

buffer - buffer contents of the given block to be added to this cache.

conf - the configuration.

localDirAllocator - the local dir allocator instance.

Throws:

java.io.IOException - if either local dir allocator fails to allocate file or if IO error occurs while writing the buffer content to the file.

java.lang.IllegalArgumentException - if buffer is null, or if buffer.limit() is zero or negative.

writeFile

protected void writeFile(java.nio.file.Path path,
                         java.nio.ByteBuffer buffer)
                  throws java.io.IOException

Throws:

java.io.IOException

getCacheFilePath

protected java.nio.file.Path getCacheFilePath(Configuration conf,
                                              LocalDirAllocator localDirAllocator)
                                       throws java.io.IOException

Return temporary file created based on the file path retrieved from local dir allocator.

Parameters:

conf - The configuration object.

localDirAllocator - Local dir allocator instance.

Returns:

Path of the temporary file created.

Throws:

java.io.IOException - if IO error occurs while local dir allocator tries to retrieve path from local FS or file creation fails or permission set fails.

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Throws:

java.io.IOException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

isCacheSpaceAvailable

public static boolean isCacheSpaceAvailable(long fileSize,
                                            Configuration conf,
                                            LocalDirAllocator localDirAllocator)

Determine if the cache space is available on the local FS.

Parameters:

fileSize - The size of the file.

conf - The configuration.

localDirAllocator - Local dir allocator instance.

Returns:

True if the given file size is less than the available free space on local FS, False otherwise.