Package org.apache.hadoop.hdfs.server.federation.router

Class Router

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.service.CompositeService

org.apache.hadoop.hdfs.server.federation.router.Router

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.hdfs.server.common.TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>, org.apache.hadoop.service.Service

@Private
@Evolving
public class Router
extends org.apache.hadoop.service.CompositeService
implements org.apache.hadoop.hdfs.server.common.TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>

Router that provides a unified view of multiple federated HDFS clusters. It has two main roles: (1) federated interface and (2) NameNode heartbeat.

For the federated interface, the Router receives a client request, checks the State Store for the correct subcluster, and forwards the request to the active Namenode of that subcluster. The reply from the Namenode then flows in the opposite direction. The Routers are stateless and can be behind a load balancer. HDFS clients connect to the router using the same interfaces as are used to communicate with a namenode, namely the ClientProtocol RPC interface and the WebHdfs HTTP interface exposed by the router. RouterRpcServer RouterHttpServer

For NameNode heartbeat, the Router periodically checks the state of a NameNode (usually on the same server) and reports their high availability (HA) state and load/space status to the State Store. Note that this is an optional role as a Router can be independent of any subcluster. StateStoreService NamenodeHeartbeatService

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

org.apache.hadoop.service.CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields inherited from class org.apache.hadoop.service.CompositeService

STOP_ONLY_STARTED_SERVICES

Constructor Summary

Constructors

Constructor	Description
Router()

Method Summary

Modifier and Type	Method	Description
protected RouterAdminServer	createAdminServer()	Create a new router admin server to handle the router admin interface.
protected RouterHttpServer	createHttpServer()	Create an HTTP server for this Router.
NamenodeHeartbeatService	createLocalNamenodeHeartbeatService()	Create a new status updater for the local Namenode.
protected NamenodeHeartbeatService	createNamenodeHeartbeatService(java.lang.String nsId, java.lang.String nnId)	Create a heartbeat monitor for a particular Namenode.
protected NamenodeHeartbeatService	createNamenodeHeartbeatService(java.lang.String nsId, java.lang.String nnId, java.lang.String resolvedHost)
protected java.util.Collection<NamenodeHeartbeatService>	createNamenodeHeartbeatServices()	Create each of the services that will monitor a Namenode.
protected RouterRpcServer	createRpcServer()	Create a new Router RPC server to proxy ClientProtocol requests.
RouterAdminServer	getAdminServer()	Get router admin server.
java.net.InetSocketAddress	getAdminServerAddress()	Get the current Admin socket address for the router.
java.net.InetSocketAddress	getHttpServerAddress()	Get the current HTTP socket address for the router.
RBFMetrics	getMetrics()	Get the federation metrics.
NamenodeBeanMetrics	getNamenodeMetrics()	Get the namenode metrics.
ActiveNamenodeResolver	getNamenodeResolver()	Get the namenode resolver for a subcluster.
RouterQuotaManager	getQuotaManager()	Get route quota manager.
RouterClientMetrics	getRouterClientMetrics()	Get the metrics system for the Router Client.
java.lang.String	getRouterId()	Unique ID for the router, typically the hostname:port string for the router's RPC server.
RouterMetrics	getRouterMetrics()	Get the metrics system for the Router.
RouterServiceState	getRouterState()	Get the status of the router.
RouterStore	getRouterStateManager()	Get the state store interface for the router heartbeats.
RouterRpcServer	getRpcServer()	Get the Router RPC server.
java.net.InetSocketAddress	getRpcServerAddress()	Get the current RPC socket address for the router.
long	getStartTime()	Get the start date of the Router.
StateStoreService	getStateStore()	Get the State Store service.
FileSubclusterResolver	getSubclusterResolver()	Get the subcluster resolver for files.
boolean	isQuotaEnabled()	Check if the quota system is enabled in Router.
boolean	isRouterState(RouterServiceState routerState)	Compare router state.
protected void	serviceInit(org.apache.hadoop.conf.Configuration configuration)
protected void	serviceStart()
protected void	serviceStop()
protected void	setAdminServerAddress(java.net.InetSocketAddress address)	Set the current Admin socket for the router.
void	setConf(org.apache.hadoop.conf.Configuration conf)	Set router configuration.
void	setRouterId(java.lang.String id)	Sets a unique ID for this router.
protected void	setRpcServerAddress(java.net.InetSocketAddress address)	Set the current RPC socket for the router.
void	shutDown()	Shutdown the router.
void	updateRouterState(RouterServiceState newState)	Update the router state and heartbeat to the state store.
void	verifyToken(org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier tokenId, byte[] password)

Methods inherited from class org.apache.hadoop.service.CompositeService

addIfService, addService, getServices, removeService

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Router

public Router()

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration configuration)
                    throws java.lang.Exception

Overrides:

serviceInit in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

serviceStart

protected void serviceStart()
                     throws java.lang.Exception

Overrides:

serviceStart in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

serviceStop

protected void serviceStop()
                    throws java.lang.Exception

Overrides:

serviceStop in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

shutDown

public void shutDown()

Shutdown the router.

createRpcServer

protected RouterRpcServer createRpcServer()
                                   throws java.io.IOException

Create a new Router RPC server to proxy ClientProtocol requests.

Returns:

New Router RPC Server.

Throws:

java.io.IOException - If the router RPC server was not started.

getRpcServer

public RouterRpcServer getRpcServer()

Get the Router RPC server.

Returns:

Router RPC server.

setRpcServerAddress

protected void setRpcServerAddress(java.net.InetSocketAddress address)

Set the current RPC socket for the router.

Parameters:

address - RPC address.

getRpcServerAddress

public java.net.InetSocketAddress getRpcServerAddress()

Get the current RPC socket address for the router.

Returns:

InetSocketAddress

createAdminServer

protected RouterAdminServer createAdminServer()
                                       throws java.io.IOException

Create a new router admin server to handle the router admin interface.

Returns:

RouterAdminServer

Throws:

java.io.IOException - If the admin server was not successfully started.

setAdminServerAddress

protected void setAdminServerAddress(java.net.InetSocketAddress address)

Set the current Admin socket for the router.

Parameters:

address - Admin RPC address.

getAdminServerAddress

public java.net.InetSocketAddress getAdminServerAddress()

Get the current Admin socket address for the router.

Returns:

InetSocketAddress Admin address.

createHttpServer

protected RouterHttpServer createHttpServer()

Create an HTTP server for this Router.

Returns:

HTTP server for this Router.

getHttpServerAddress

public java.net.InetSocketAddress getHttpServerAddress()

Get the current HTTP socket address for the router.

Returns:

InetSocketAddress HTTP address.

verifyToken

public void verifyToken(org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier tokenId,
                        byte[] password)
                 throws java.io.IOException

Specified by:

verifyToken in interface org.apache.hadoop.hdfs.server.common.TokenVerifier<org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier>

Throws:

java.io.IOException

createNamenodeHeartbeatServices

protected java.util.Collection<NamenodeHeartbeatService> createNamenodeHeartbeatServices()

Create each of the services that will monitor a Namenode.

Returns:

List of heartbeat services.

createLocalNamenodeHeartbeatService

@VisibleForTesting
public NamenodeHeartbeatService createLocalNamenodeHeartbeatService()

Create a new status updater for the local Namenode.

Returns:

Updater of the status for the local Namenode.

createNamenodeHeartbeatService

protected NamenodeHeartbeatService createNamenodeHeartbeatService(java.lang.String nsId,
                                                                  java.lang.String nnId)

Create a heartbeat monitor for a particular Namenode.

Parameters:

nsId - Identifier of the nameservice to monitor.

nnId - Identifier of the namenode (HA) to monitor.

Returns:

Updater of the status for the specified Namenode.

createNamenodeHeartbeatService

protected NamenodeHeartbeatService createNamenodeHeartbeatService(java.lang.String nsId,
                                                                  java.lang.String nnId,
                                                                  java.lang.String resolvedHost)

updateRouterState

public void updateRouterState(RouterServiceState newState)

Update the router state and heartbeat to the state store.

Parameters:

newState - The new router state.

getRouterState

public RouterServiceState getRouterState()

Get the status of the router.

Returns:

Status of the router.

isRouterState

public boolean isRouterState(RouterServiceState routerState)

Compare router state.

Parameters:

routerState - the router service state.

Returns:

true if the given router state is same as the state maintained by the router object.

getStateStore

public StateStoreService getStateStore()

Get the State Store service.

Returns:

State Store service.

getRouterMetrics

public RouterMetrics getRouterMetrics()

Get the metrics system for the Router.

Returns:

Router metrics.

getRouterClientMetrics

public RouterClientMetrics getRouterClientMetrics()

Get the metrics system for the Router Client.

Returns:

Router Client metrics.

getMetrics

public RBFMetrics getMetrics()

Get the federation metrics.

Returns:

Federation metrics.

getNamenodeMetrics

public NamenodeBeanMetrics getNamenodeMetrics()
                                       throws java.io.IOException

Get the namenode metrics.

Returns:

the namenode metrics.

Throws:

java.io.IOException - if the namenode metrics are not initialized.

getSubclusterResolver

public FileSubclusterResolver getSubclusterResolver()

Get the subcluster resolver for files.

Returns:

Subcluster resolver for files.

getNamenodeResolver

public ActiveNamenodeResolver getNamenodeResolver()

Get the namenode resolver for a subcluster.

Returns:

The namenode resolver for a subcluster.

getRouterStateManager

public RouterStore getRouterStateManager()

Get the state store interface for the router heartbeats.

Returns:

RouterStore state store API handle.

getStartTime

public long getStartTime()

Get the start date of the Router.

Specified by:

getStartTime in interface org.apache.hadoop.service.Service

Overrides:

getStartTime in class org.apache.hadoop.service.AbstractService

Returns:

Start date of the router.

getRouterId

public java.lang.String getRouterId()

Unique ID for the router, typically the hostname:port string for the router's RPC server. This ID may be null on router startup before the RPC server has bound to a port.

Returns:

Router identifier.

setRouterId

public void setRouterId(java.lang.String id)

Sets a unique ID for this router.

Parameters:

id - Identifier of the Router.

isQuotaEnabled

public boolean isQuotaEnabled()

Check if the quota system is enabled in Router.

Returns:

True if the quota system is enabled in Router.

getQuotaManager

public RouterQuotaManager getQuotaManager()

Get route quota manager.

Returns:

RouterQuotaManager Quota manager.

getAdminServer

public RouterAdminServer getAdminServer()

Get router admin server.

Returns:

Null if admin is not enabled.

setConf

@VisibleForTesting
public void setConf(org.apache.hadoop.conf.Configuration conf)

Set router configuration.

Parameters:

conf - the configuration.