Package org.apache.hadoop.hdfs.server.datanode.fsdataset.impl

Class PmemVolumeManager

java.lang.Object

org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.PmemVolumeManager

@Private
@Unstable
public final class PmemVolumeManager
extends java.lang.Object

Manage the persistent memory volumes.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	CACHE_DIR

Method Summary

Modifier and Type	Method	Description
void	createBlockPoolDir(java.lang.String bpid)	Create cache subdirectory specified with blockPoolId.
long	getCacheCapacity()
java.lang.String	getCachePath(org.apache.hadoop.hdfs.ExtendedBlockId key)	The cache file path is pmemVolume/BlockPoolId/subdir#/subdir#/BlockId.
long	getCacheUsed()
static PmemVolumeManager	getInstance()
static java.lang.String	getRealPmemDir(java.lang.String rawPmemDir)
java.lang.String	idToCacheFileName(org.apache.hadoop.hdfs.ExtendedBlockId key)	A cache file is named after the corresponding BlockId.
java.lang.String	idToCacheFilePath(java.lang.Byte volumeIndex, org.apache.hadoop.hdfs.ExtendedBlockId key)	Create and get the directory where a cache file with this key and volumeIndex should be stored.
static void	init(java.lang.String[] pmemVolumesConfig, boolean cacheRecoveryEnabled)
void	recoverBlockKeyToVolume(org.apache.hadoop.hdfs.ExtendedBlockId key, byte volumeIndex)
java.util.Map<org.apache.hadoop.hdfs.ExtendedBlockId,MappableBlock>	recoverCache(java.lang.String bpid, MappableBlockLoader cacheLoader)	Recover cache from the cached files in the configured pmem volumes.
static void	reset()
static void	setMaxBytes(long maxBytes)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CACHE_DIR

public static final java.lang.String CACHE_DIR

See Also:

Constant Field Values

Method Detail

init

public static void init(java.lang.String[] pmemVolumesConfig,
                        boolean cacheRecoveryEnabled)
                 throws java.io.IOException

Throws:

java.io.IOException

getInstance

public static PmemVolumeManager getInstance()

reset

@VisibleForTesting
public static void reset()

setMaxBytes

@VisibleForTesting
public static void setMaxBytes(long maxBytes)

getCacheUsed

public long getCacheUsed()

getCacheCapacity

public long getCacheCapacity()

recoverCache

public java.util.Map<org.apache.hadoop.hdfs.ExtendedBlockId,MappableBlock> recoverCache(java.lang.String bpid,
                                                                                              MappableBlockLoader cacheLoader)
                                                                                       throws java.io.IOException

Recover cache from the cached files in the configured pmem volumes.

Throws:

java.io.IOException

recoverBlockKeyToVolume

public void recoverBlockKeyToVolume(org.apache.hadoop.hdfs.ExtendedBlockId key,
                                    byte volumeIndex)

createBlockPoolDir

public void createBlockPoolDir(java.lang.String bpid)
                        throws java.io.IOException

Create cache subdirectory specified with blockPoolId.

Throws:

java.io.IOException

getRealPmemDir

public static java.lang.String getRealPmemDir(java.lang.String rawPmemDir)

idToCacheFileName

public java.lang.String idToCacheFileName(org.apache.hadoop.hdfs.ExtendedBlockId key)

A cache file is named after the corresponding BlockId. Thus, cache file name can be inferred according to BlockId.

idToCacheFilePath

public java.lang.String idToCacheFilePath(java.lang.Byte volumeIndex,
                                          org.apache.hadoop.hdfs.ExtendedBlockId key)
                                   throws java.io.IOException

Create and get the directory where a cache file with this key and volumeIndex should be stored. Use hierarchical strategy of storing blocks to avoid keeping cache files under one directory.

Parameters:

volumeIndex - The index of pmem volume where a replica will be cached to or has been cached to.

key - The replica's ExtendedBlockId.

Returns:

A path to which the block replica is mapped.

Throws:

java.io.IOException

getCachePath

public java.lang.String getCachePath(org.apache.hadoop.hdfs.ExtendedBlockId key)
                              throws java.io.IOException

The cache file path is pmemVolume/BlockPoolId/subdir#/subdir#/BlockId.

Throws:

java.io.IOException