org.apache.hadoop.conf.Configurable, org.apache.hadoop.conf.Reconfigurable, TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>, NameNodeStatusMXBeanBackupNode@Private @Metrics(context="dfs") public class NameNode extends org.apache.hadoop.conf.ReconfigurableBase implements NameNodeStatusMXBean, TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>
ClientProtocol interface, which
allows clients to ask for DFS services.
ClientProtocol is not designed for
direct use by authors of DFS client code. End-users should instead use the
FileSystem class.
NameNode also implements the
DatanodeProtocol interface,
used by DataNodes that actually store DFS data blocks. These
methods are invoked repeatedly and automatically by all the
DataNodes in a DFS deployment.
NameNode also implements the
NamenodeProtocol interface,
used by secondary namenodes or rebalancing processes to get partial
NameNode state, for example partial blocksMap etc.| Modifier and Type | Class | Description |
|---|---|---|
protected class |
NameNode.NameNodeHAContext |
|
static class |
NameNode.OperationCategory |
Categories of operations supported by the namenode.
|
| Modifier and Type | Field | Description |
|---|---|---|
static HAState |
ACTIVE_STATE |
|
protected boolean |
allowStaleStandbyReads |
|
static org.slf4j.Logger |
blockStateChangeLog |
|
static int |
DEFAULT_PORT |
Deprecated.
Use
HdfsClientConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT
instead. |
protected NameNodeHttpServer |
httpServer |
httpServer
|
static org.slf4j.Logger |
LOG |
|
static java.lang.String |
METRICS_LOG_NAME |
|
static java.lang.String[] |
NAMENODE_SPECIFIC_KEYS |
HDFS configuration can have three types of parameters:
Parameters that are common for all the name services in the cluster.
Parameters that are specific to a name service.
|
static java.lang.String[] |
NAMESERVICE_SPECIFIC_KEYS |
|
protected FSNamesystem |
namesystem |
|
protected NamenodeRegistration |
nodeRegistration |
Registration information of this name-node
|
static HAState |
OBSERVER_STATE |
|
protected HdfsServerConstants.NamenodeRole |
role |
|
static HAState |
STANDBY_STATE |
|
static org.slf4j.Logger |
stateChangeLog |
|
protected boolean |
stopRequested |
only used for testing purposes
|
protected org.apache.hadoop.tracing.Tracer |
tracer |
| Modifier | Constructor | Description |
|---|---|---|
|
NameNode(org.apache.hadoop.conf.Configuration conf) |
Start NameNode.
|
protected |
NameNode(org.apache.hadoop.conf.Configuration conf,
HdfsServerConstants.NamenodeRole role) |
| Modifier and Type | Method | Description |
|---|---|---|
static void |
checkAllowFormat(org.apache.hadoop.conf.Configuration conf) |
|
static java.lang.String |
composeNotStartedMessage(HdfsServerConstants.NamenodeRole role) |
|
protected HAContext |
createHAContext() |
|
protected HAState |
createHAState(org.apache.hadoop.conf.Configuration conf) |
|
static NameNode |
createNameNode(java.lang.String[] argv,
org.apache.hadoop.conf.Configuration conf) |
|
protected NameNodeRpcServer |
createRpcServer(org.apache.hadoop.conf.Configuration conf) |
Create the RPC server implementation.
|
protected void |
doImmediateShutdown(java.lang.Throwable t) |
Shutdown the NN immediately in an ungraceful way.
|
static boolean |
doRollback(org.apache.hadoop.conf.Configuration conf,
boolean isConfirmationNeeded) |
|
static void |
format(org.apache.hadoop.conf.Configuration conf) |
Format a new filesystem.
|
InMemoryLevelDBAliasMapServer |
getAliasMapServer() |
|
java.util.Set<java.net.InetSocketAddress> |
getAuxiliaryNameNodeAddresses() |
|
long |
getBytesWithFutureGenerationStamps() |
Gets number of bytes in blocks with future generation stamps.
|
static org.apache.commons.lang3.tuple.Pair<byte[],java.lang.Integer> |
getClientIdAndCallId(java.lang.String[] ipProxyUsers) |
Try to obtain the actual client's id and call id
according to the current user.
|
static java.lang.String |
getClientMachine(java.lang.String[] ipProxyUsers) |
Try to obtain the actual client's machine according to the current user.
|
java.lang.String |
getClientNamenodeAddress() |
Get the namenode address to be used by clients.
|
FSImage |
getFSImage() |
get FSImage
|
java.lang.String |
getHostAndPort() |
Gets the host and port colon separated.
|
java.net.InetSocketAddress |
getHttpAddress() |
|
static java.net.InetSocketAddress |
getHttpAddress(org.apache.hadoop.conf.Configuration conf) |
|
java.net.InetSocketAddress |
getHttpsAddress() |
|
org.apache.hadoop.http.HttpServer2 |
getHttpServer() |
|
protected java.net.InetSocketAddress |
getHttpServerAddress(org.apache.hadoop.conf.Configuration conf) |
|
protected java.net.InetSocketAddress |
getHttpServerBindAddress(org.apache.hadoop.conf.Configuration conf) |
HTTP server address for binding the endpoint.
|
long |
getLastHATransitionTime() |
Gets the most recent HA transition time in milliseconds from the epoch.
|
java.net.InetSocketAddress |
getNameNodeAddress() |
|
java.lang.String |
getNameNodeAddressHostPortString() |
|
static NameNodeMetrics |
getNameNodeMetrics() |
|
int |
getNameNodeState() |
Emit Namenode HA service state as an integer so that one can monitor NN HA
state based on this metric.
|
protected java.lang.String |
getNameServiceId(org.apache.hadoop.conf.Configuration conf) |
Get the name service Id for the node
|
FSNamesystem |
getNamesystem() |
Return the
FSNamesystem object. |
protected org.apache.hadoop.conf.Configuration |
getNewConf() |
|
java.lang.String |
getNNAuxiliaryRpcAddress() |
Return a host:port format string corresponds to an auxiliary
port configured on NameNode.
|
java.lang.String |
getNNRole() |
Gets the NameNode role.
|
long |
getProtocolVersion(java.lang.String protocol,
long clientVersion) |
|
java.util.Collection<java.lang.String> |
getReconfigurableProperties() |
|
static org.apache.hadoop.security.UserGroupInformation |
getRemoteUser() |
|
HdfsServerConstants.NamenodeRole |
getRole() |
|
NamenodeProtocols |
getRpcServer() |
|
protected java.net.InetSocketAddress |
getRpcServerAddress(org.apache.hadoop.conf.Configuration conf) |
|
protected java.lang.String |
getRpcServerBindHost(org.apache.hadoop.conf.Configuration conf) |
Given a configuration get the bind host of the client rpc server
If the bind host is not configured returns null.
|
static java.net.InetSocketAddress |
getServiceAddress(org.apache.hadoop.conf.Configuration conf,
boolean fallback) |
Fetches the address for services to use when connecting to namenode
based on the value of fallback returns null if the special
address is not specified or returns the default namenode address
to be used by both clients and services.
|
java.net.InetSocketAddress |
getServiceRpcAddress() |
|
protected java.net.InetSocketAddress |
getServiceRpcServerAddress(org.apache.hadoop.conf.Configuration conf) |
Given a configuration get the address of the service rpc server
If the service rpc is not configured returns null
|
protected java.lang.String |
getServiceRpcServerBindHost(org.apache.hadoop.conf.Configuration conf) |
Given a configuration get the bind host of the service rpc server
If the bind host is not configured returns null.
|
java.lang.String |
getSlowDisksReport() |
Gets the topN slow disks in the cluster, if the feature is enabled.
|
java.lang.String |
getSlowPeersReport() |
Retrieves information about slow DataNodes, if the feature is
enabled.
|
static HdfsServerConstants.StartupOption |
getStartupOption(org.apache.hadoop.conf.Configuration conf) |
|
static StartupProgress |
getStartupProgress() |
Returns object used for reporting namenode startup progress.
|
java.lang.String |
getState() |
Gets the NameNode state.
|
java.lang.String |
getTokenServiceName() |
Return the service name of the issued delegation token.
|
protected void |
initialize(org.apache.hadoop.conf.Configuration conf) |
Initialize name-node.
|
static void |
initializeGenericKeys(org.apache.hadoop.conf.Configuration conf,
java.lang.String nameserviceId,
java.lang.String namenodeId) |
In federation configuration is set for a set of
namenode and secondary namenode/backup/checkpointer, which are
grouped under a logical nameservice ID.
|
static boolean |
initializeSharedEdits(org.apache.hadoop.conf.Configuration conf) |
|
static boolean |
initializeSharedEdits(org.apache.hadoop.conf.Configuration conf,
boolean force) |
|
static void |
initMetrics(org.apache.hadoop.conf.Configuration conf,
HdfsServerConstants.NamenodeRole role) |
|
boolean |
isActiveState() |
|
boolean |
isInSafeMode() |
Is the cluster currently in safe mode?
|
boolean |
isObserverState() |
|
boolean |
isSecurityEnabled() |
Gets if security is enabled.
|
boolean |
isStandbyState() |
|
void |
join() |
Wait for service to finish.
|
void |
joinHttpServer() |
NameNodeHttpServer, used by unit tests to ensure a full shutdown,
so that no bind exception is thrown during restart.
|
protected void |
loadNamesystem(org.apache.hadoop.conf.Configuration conf) |
|
static void |
main(java.lang.String[] argv) |
|
static java.lang.String |
parseSpecialValue(java.lang.String content,
java.lang.String key) |
Try to obtain the value corresponding to the key by parsing the content.
|
void |
queueExternalCall(org.apache.hadoop.ipc.ExternalCall<?> extCall) |
|
protected java.lang.String |
reconfigurePropertyImpl(java.lang.String property,
java.lang.String newVal) |
|
protected void |
setRpcServerAddress(org.apache.hadoop.conf.Configuration conf,
java.net.InetSocketAddress rpcAddress) |
|
protected void |
setRpcServiceServerAddress(org.apache.hadoop.conf.Configuration conf,
java.net.InetSocketAddress serviceRPCAddress) |
Modifies the configuration passed to contain the service rpc address setting
|
static void |
setServiceAddress(org.apache.hadoop.conf.Configuration conf,
java.lang.String address) |
Set the configuration property for the service rpc address
to address
|
protected void |
startMetricsLogger(org.apache.hadoop.conf.Configuration conf) |
Start a timer to periodically write NameNode metrics to the log
file.
|
void |
stop() |
Stop all NameNode threads and wait for all to finish.
|
protected void |
stopMetricsLogger() |
|
void |
verifyToken(org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier id,
byte[] password) |
getChangedProperties, getReconfigurationTaskStatus, isPropertyReconfigurable, reconfigureProperty, setReconfigurationUtil, shutdownReconfigurationTask, startReconfigurationTaskpublic static final java.lang.String[] NAMENODE_SPECIFIC_KEYS
public static final java.lang.String[] NAMESERVICE_SPECIFIC_KEYS
@Deprecated public static final int DEFAULT_PORT
HdfsClientConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT
instead.public static final org.slf4j.Logger LOG
public static final org.slf4j.Logger stateChangeLog
public static final org.slf4j.Logger blockStateChangeLog
public static final HAState ACTIVE_STATE
public static final HAState STANDBY_STATE
public static final HAState OBSERVER_STATE
public static final java.lang.String METRICS_LOG_NAME
protected FSNamesystem namesystem
protected final HdfsServerConstants.NamenodeRole role
protected final boolean allowStaleStandbyReads
protected NameNodeHttpServer httpServer
protected boolean stopRequested
protected NamenodeRegistration nodeRegistration
protected final org.apache.hadoop.tracing.Tracer tracer
public NameNode(org.apache.hadoop.conf.Configuration conf)
throws java.io.IOException
The name-node can be started with one of the following startup options:
REGULAR - normal name node startupFORMAT - format name nodeBACKUP - start backup nodeCHECKPOINT - start checkpoint nodeUPGRADE - start the cluster
UPGRADEONLY - upgrade the cluster
upgrade and create a snapshot of the current file system stateRECOVERY - recover name node
metadataROLLBACK - roll the
cluster back to the previous stateIMPORT - import checkpointzero in the conf.conf - confirgurationjava.io.IOExceptionprotected NameNode(org.apache.hadoop.conf.Configuration conf,
HdfsServerConstants.NamenodeRole role)
throws java.io.IOException
java.io.IOExceptionpublic long getProtocolVersion(java.lang.String protocol,
long clientVersion)
throws java.io.IOException
java.io.IOExceptionpublic static void format(org.apache.hadoop.conf.Configuration conf)
throws java.io.IOException
java.io.IOExceptionpublic FSNamesystem getNamesystem()
FSNamesystem object.FSNamesystem object.public NamenodeProtocols getRpcServer()
@VisibleForTesting public org.apache.hadoop.http.HttpServer2 getHttpServer()
public void queueExternalCall(org.apache.hadoop.ipc.ExternalCall<?> extCall)
throws java.io.IOException,
java.lang.InterruptedException
java.io.IOExceptionjava.lang.InterruptedExceptionpublic static void initMetrics(org.apache.hadoop.conf.Configuration conf,
HdfsServerConstants.NamenodeRole role)
public static NameNodeMetrics getNameNodeMetrics()
@VisibleForTesting
public static java.lang.String parseSpecialValue(java.lang.String content,
java.lang.String key)
content - the full content to be parsed.key - trying to obtain the value of the key.public static java.lang.String getClientMachine(java.lang.String[] ipProxyUsers)
ipProxyUsers - Users who can override client infos.public static org.apache.commons.lang3.tuple.Pair<byte[],java.lang.Integer> getClientIdAndCallId(java.lang.String[] ipProxyUsers)
ipProxyUsers - Users who can override client infospublic static StartupProgress getStartupProgress()
public java.lang.String getTokenServiceName()
public java.lang.String getClientNamenodeAddress()
public static void setServiceAddress(org.apache.hadoop.conf.Configuration conf,
java.lang.String address)
public static java.net.InetSocketAddress getServiceAddress(org.apache.hadoop.conf.Configuration conf,
boolean fallback)
public HdfsServerConstants.NamenodeRole getRole()
public static java.lang.String composeNotStartedMessage(HdfsServerConstants.NamenodeRole role)
protected java.net.InetSocketAddress getServiceRpcServerAddress(org.apache.hadoop.conf.Configuration conf)
protected java.net.InetSocketAddress getRpcServerAddress(org.apache.hadoop.conf.Configuration conf)
protected java.lang.String getServiceRpcServerBindHost(org.apache.hadoop.conf.Configuration conf)
protected java.lang.String getRpcServerBindHost(org.apache.hadoop.conf.Configuration conf)
protected void setRpcServiceServerAddress(org.apache.hadoop.conf.Configuration conf,
java.net.InetSocketAddress serviceRPCAddress)
protected void setRpcServerAddress(org.apache.hadoop.conf.Configuration conf,
java.net.InetSocketAddress rpcAddress)
protected java.net.InetSocketAddress getHttpServerAddress(org.apache.hadoop.conf.Configuration conf)
protected java.net.InetSocketAddress getHttpServerBindAddress(org.apache.hadoop.conf.Configuration conf)
DFSConfigKeys.DFS_NAMENODE_HTTP_BIND_HOST_KEYconf - public static java.net.InetSocketAddress getHttpAddress(org.apache.hadoop.conf.Configuration conf)
protected void loadNamesystem(org.apache.hadoop.conf.Configuration conf)
throws java.io.IOException
java.io.IOExceptionpublic static org.apache.hadoop.security.UserGroupInformation getRemoteUser()
throws java.io.IOException
java.io.IOExceptionpublic void verifyToken(org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier id,
byte[] password)
throws java.io.IOException
verifyToken in interface TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>java.io.IOExceptionprotected void initialize(org.apache.hadoop.conf.Configuration conf)
throws java.io.IOException
conf - the configurationjava.io.IOException@VisibleForTesting public InMemoryLevelDBAliasMapServer getAliasMapServer()
protected void startMetricsLogger(org.apache.hadoop.conf.Configuration conf)
conf - protected void stopMetricsLogger()
protected NameNodeRpcServer createRpcServer(org.apache.hadoop.conf.Configuration conf) throws java.io.IOException
java.io.IOExceptionprotected HAState createHAState(org.apache.hadoop.conf.Configuration conf)
protected HAContext createHAContext()
public void join()
public void stop()
public boolean isInSafeMode()
@VisibleForTesting public FSImage getFSImage()
public java.net.InetSocketAddress getNameNodeAddress()
public java.util.Set<java.net.InetSocketAddress> getAuxiliaryNameNodeAddresses()
public java.lang.String getNameNodeAddressHostPortString()
@VisibleForTesting public java.lang.String getNNAuxiliaryRpcAddress()
public java.net.InetSocketAddress getServiceRpcAddress()
public java.net.InetSocketAddress getHttpAddress()
public java.net.InetSocketAddress getHttpsAddress()
@VisibleForTesting public void joinHttpServer()
public static void checkAllowFormat(org.apache.hadoop.conf.Configuration conf)
throws java.io.IOException
java.io.IOException@VisibleForTesting
public static boolean initializeSharedEdits(org.apache.hadoop.conf.Configuration conf)
throws java.io.IOException
java.io.IOException@VisibleForTesting
public static boolean initializeSharedEdits(org.apache.hadoop.conf.Configuration conf,
boolean force)
throws java.io.IOException
java.io.IOException@VisibleForTesting
public static boolean doRollback(org.apache.hadoop.conf.Configuration conf,
boolean isConfirmationNeeded)
throws java.io.IOException
java.io.IOExceptionpublic static HdfsServerConstants.StartupOption getStartupOption(org.apache.hadoop.conf.Configuration conf)
public static NameNode createNameNode(java.lang.String[] argv, org.apache.hadoop.conf.Configuration conf) throws java.io.IOException
java.io.IOExceptionpublic static void initializeGenericKeys(org.apache.hadoop.conf.Configuration conf,
java.lang.String nameserviceId,
java.lang.String namenodeId)
conf - Configuration object to lookup specific key and to set the value
to the key passed. Note the conf object is modifiednameserviceId - name service Id (to distinguish federated NNs)namenodeId - the namenode ID (to distinguish HA NNs)DFSUtil.setGenericConf(Configuration, String, String, String...)protected java.lang.String getNameServiceId(org.apache.hadoop.conf.Configuration conf)
public static void main(java.lang.String[] argv)
throws java.lang.Exception
java.lang.Exception@Metric({"NameNodeState","Namenode HA service state"})
public int getNameNodeState()
public java.lang.String getNNRole()
NameNodeStatusMXBeangetNNRole in interface NameNodeStatusMXBeanpublic java.lang.String getState()
NameNodeStatusMXBeangetState in interface NameNodeStatusMXBeanpublic java.lang.String getHostAndPort()
NameNodeStatusMXBeangetHostAndPort in interface NameNodeStatusMXBeanpublic boolean isSecurityEnabled()
NameNodeStatusMXBeanisSecurityEnabled in interface NameNodeStatusMXBeanpublic long getLastHATransitionTime()
NameNodeStatusMXBeangetLastHATransitionTime in interface NameNodeStatusMXBeanpublic long getBytesWithFutureGenerationStamps()
NameNodeStatusMXBeangetBytesWithFutureGenerationStamps in interface NameNodeStatusMXBeanpublic java.lang.String getSlowPeersReport()
NameNodeStatusMXBeangetSlowPeersReport in interface NameNodeStatusMXBeanpublic java.lang.String getSlowDisksReport()
NameNodeStatusMXBeangetSlowDisksReport in interface NameNodeStatusMXBeanprotected void doImmediateShutdown(java.lang.Throwable t)
throws org.apache.hadoop.util.ExitUtil.ExitException
t - exception which warrants the shutdown. Printed to the NN log
before exit.org.apache.hadoop.util.ExitUtil.ExitException - thrown only for testing.public boolean isStandbyState()
public boolean isActiveState()
public boolean isObserverState()
public java.util.Collection<java.lang.String> getReconfigurableProperties()
getReconfigurableProperties in interface org.apache.hadoop.conf.ReconfigurablegetReconfigurableProperties in class org.apache.hadoop.conf.ReconfigurableBaseprotected java.lang.String reconfigurePropertyImpl(java.lang.String property,
java.lang.String newVal)
throws org.apache.hadoop.conf.ReconfigurationException
reconfigurePropertyImpl in class org.apache.hadoop.conf.ReconfigurableBaseorg.apache.hadoop.conf.ReconfigurationExceptionprotected org.apache.hadoop.conf.Configuration getNewConf()
getNewConf in class org.apache.hadoop.conf.ReconfigurableBaseCopyright © 2008–2025 Apache Software Foundation. All rights reserved.