Package org.apache.hadoop.hdfs.server.namenode

Class FSImageFormat

java.lang.Object

org.apache.hadoop.hdfs.server.namenode.FSImageFormat

@Private
@Evolving
public class FSImageFormat
extends java.lang.Object

Contains inner classes for reading or writing the on-disk format for FSImages. In particular, the format of the FSImage looks like:

 FSImage {
   layoutVersion: int, namespaceID: int, numberItemsInFSDirectoryTree: long,
   namesystemGenerationStampV1: long, namesystemGenerationStampV2: long,
   generationStampAtBlockIdSwitch:long, lastAllocatedBlockId:
   long transactionID: long, snapshotCounter: int, numberOfSnapshots: int,
   numOfSnapshottableDirs: int,
   {FSDirectoryTree, FilesUnderConstruction, SecretManagerState} (can be compressed)
 }

 FSDirectoryTree (if LayoutVersion.Feature.FSIMAGE_NAME_OPTIMIZATION is supported) {
   INodeInfo of root, numberOfChildren of root: int
   [list of INodeInfo of root's children],
   [list of INodeDirectoryInfo of root's directory children]
 }

 FSDirectoryTree (if LayoutVersion.Feature.FSIMAGE_NAME_OPTIMIZATION not supported){
   [list of INodeInfo of INodes in topological order]
 }

 INodeInfo {
   {
     localName: short + byte[]
   } when LayoutVersion.Feature.FSIMAGE_NAME_OPTIMIZATION is supported
   or
   {
     fullPath: byte[]
   } when LayoutVersion.Feature.FSIMAGE_NAME_OPTIMIZATION is not supported
   replicationFactor: short, modificationTime: long,
   accessTime: long, preferredBlockSize: long,
   numberOfBlocks: int (-1 for INodeDirectory, -2 for INodeSymLink),
   {
     nsQuota: long, dsQuota: long,
     {
       isINodeSnapshottable: byte,
       isINodeWithSnapshot: byte (if isINodeSnapshottable is false)
     } (when LayoutVersion.Feature.SNAPSHOT is supported),
     fsPermission: short, PermissionStatus
   } for INodeDirectory
   or
   {
     symlinkString, fsPermission: short, PermissionStatus
   } for INodeSymlink
   or
   {
     [list of BlockInfo]
     [list of FileDiff]
     {
       isINodeFileUnderConstructionSnapshot: byte,
       {clientName: short + byte[], clientMachine: short + byte[]} (when
       isINodeFileUnderConstructionSnapshot is true),
     } (when LayoutVersion.Feature.SNAPSHOT is supported and writing snapshotINode),
     fsPermission: short, PermissionStatus
   } for INodeFile
 }

 INodeDirectoryInfo {
   fullPath of the directory: short + byte[],
   numberOfChildren: int, [list of INodeInfo of children INode],
   {
     numberOfSnapshots: int,
     [list of Snapshot] (when NumberOfSnapshots is positive),
     numberOfDirectoryDiffs: int,
     [list of DirectoryDiff] (NumberOfDirectoryDiffs is positive),
     number of children that are directories,
     [list of INodeDirectoryInfo of the directory children] (includes
     snapshot copies of deleted sub-directories)
   } (when LayoutVersion.Feature.SNAPSHOT is supported),
 }

 Snapshot {
   snapshotID: int, root of Snapshot: INodeDirectoryInfo (its local name is
   the name of the snapshot)
 }

 DirectoryDiff {
   full path of the root of the associated Snapshot: short + byte[],
   childrenSize: int,
   isSnapshotRoot: byte,
   snapshotINodeIsNotNull: byte (when isSnapshotRoot is false),
   snapshotINode: INodeDirectory (when SnapshotINodeIsNotNull is true), Diff
 }

 Diff {
   createdListSize: int, [Local name of INode in created list],
   deletedListSize: int, [INode in deleted list: INodeInfo]
 }

 FileDiff {
   full path of the root of the associated Snapshot: short + byte[],
   fileSize: long,
   snapshotINodeIsNotNull: byte,
   snapshotINode: INodeFile (when SnapshotINodeIsNotNull is true), Diff
 }
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	FSImageFormat.Loader	A one-shot class responsible for loading an image.

Field Summary

Fields

Modifier and Type	Field	Description
static java.util.TreeMap<java.lang.String,java.lang.String>	renameReservedMap

Method Summary

Modifier and Type	Method	Description
static org.apache.hadoop.hdfs.server.namenode.FSImageFormat.LoaderDelegator	newLoader(org.apache.hadoop.conf.Configuration conf, FSNamesystem fsn)	Construct a loader class to load the image.
static void	setRenameReservedPairs(java.lang.String renameReserved)	Set the key-value pairs that will be used to determine how to rename reserved paths on upgrade.
static void	useDefaultRenameReservedPairs()	Use the default key-value pairs that will be used to determine how to rename reserved paths on upgrade.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

renameReservedMap

@VisibleForTesting
public static final java.util.TreeMap<java.lang.String,java.lang.String> renameReservedMap

Method Detail

newLoader

public static org.apache.hadoop.hdfs.server.namenode.FSImageFormat.LoaderDelegator newLoader(org.apache.hadoop.conf.Configuration conf,
                                                                                             FSNamesystem fsn)

Construct a loader class to load the image. It chooses the loader based on the layout version.

useDefaultRenameReservedPairs

@VisibleForTesting
public static void useDefaultRenameReservedPairs()

Use the default key-value pairs that will be used to determine how to rename reserved paths on upgrade.

setRenameReservedPairs

@VisibleForTesting
public static void setRenameReservedPairs(java.lang.String renameReserved)

Set the key-value pairs that will be used to determine how to rename reserved paths on upgrade.