Package org.apache.hadoop.hdfs.server.federation.router

Class RouterRpcClient

java.lang.Object

org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient

public class RouterRpcClient
extends java.lang.Object

A client proxy for Router to NN communication using the NN ClientProtocol.

Provides routers to invoke remote ClientProtocol methods and handle retries/failover.

• invokeSingle Make a single request to a single namespace
• invokeSequential Make a sequential series of requests to multiple ordered namespaces until a condition is met.
• invokeConcurrent Make concurrent requests to multiple namespaces and return all of the results.

Also maintains a cached pool of connections to NNs. Connections are managed by the ConnectionManager and are unique to each user + NN. The size of the connection pool can be configured. Larger pools allow for more simultaneous requests to a single NN from a single user.

Constructor Summary

Constructors

Constructor	Description
RouterRpcClient(org.apache.hadoop.conf.Configuration conf, Router router, ActiveNamenodeResolver resolver, RouterRpcMonitor monitor, org.apache.hadoop.hdfs.server.federation.router.RouterStateIdContext routerStateIdContext)	Create a router RPC client to manage remote procedure calls to NNs.

Method Summary

Modifier and Type	Method	Description
java.lang.Long	getAcceptedPermitForNs(java.lang.String ns)
java.lang.String	getAcceptedPermitsPerNsJSON()	JSON representation of the accepted permits for each nameservice.
java.lang.String	getAsyncCallerPoolJson()	JSON representation of the async caller thread pool.
java.lang.String	getJSON()	JSON representation of the connection pool.
ActiveNamenodeResolver	getNamenodeResolver()	Get the active namenode resolver used by this client.
int	getNumActiveConnections()	Total number of available sockets between the router and NNs.
int	getNumActiveConnectionsRecently()	Total number of active sockets between the router and NNs.
int	getNumConnectionPools()	Total number of open connection pools to a NN.
int	getNumConnections()	Total number of available sockets between the router and NNs.
int	getNumCreatingConnections()	Number of connections between the router and NNs being created sockets.
int	getNumIdleConnections()	Total number of idle sockets between the router and NNs.
java.lang.Long	getRejectedPermitForNs(java.lang.String ns)
java.lang.String	getRejectedPermitsPerNsJSON()	JSON representation of the rejected permits for each nameservice.
RouterRpcFairnessPolicyController	getRouterRpcFairnessPolicyController()
<T extends RemoteLocationContext> boolean	invokeAll(java.util.Collection<T> locations, RemoteMethod method)	Invoke method in all locations and return success if any succeeds.
<T extends RemoteLocationContext,R> void	invokeConcurrent(java.util.Collection<T> locations, RemoteMethod method)	Invoke multiple concurrent proxy calls to different clients.
<T extends RemoteLocationContext,R> void	invokeConcurrent(java.util.Collection<T> locations, RemoteMethod method, boolean requireResponse, boolean standby)	Invoke multiple concurrent proxy calls to different clients.
<T extends RemoteLocationContext,R> java.util.Map<T,R>	invokeConcurrent(java.util.Collection<T> locations, RemoteMethod method, boolean requireResponse, boolean standby, long timeOutMs, java.lang.Class<R> clazz)	Invokes multiple concurrent proxy calls to different clients.
<T extends RemoteLocationContext,R> java.util.Map<T,R>	invokeConcurrent(java.util.Collection<T> locations, RemoteMethod method, boolean requireResponse, boolean standby, java.lang.Class<R> clazz)	Invokes multiple concurrent proxy calls to different clients.
<T extends RemoteLocationContext,R> java.util.List<RemoteResult<T,R>>	invokeConcurrent(java.util.Collection<T> locations, RemoteMethod method, boolean standby, long timeOutMs, java.lang.Class<R> clazz)	Invokes multiple concurrent proxy calls to different clients.
<T extends RemoteLocationContext,R> java.util.Map<T,R>	invokeConcurrent(java.util.Collection<T> locations, RemoteMethod method, java.lang.Class<R> clazz)	Invoke multiple concurrent proxy calls to different clients.
java.lang.Object	invokeMethod(org.apache.hadoop.security.UserGroupInformation ugi, java.util.List<? extends FederationNamenodeContext> namenodes, boolean useObserver, java.lang.Class<?> protocol, java.lang.reflect.Method method, java.lang.Object... params)	Invokes a method against the ClientProtocol proxy server.
<T> T	invokeSequential(java.util.List<? extends RemoteLocationContext> locations, RemoteMethod remoteMethod)	Invokes sequential proxy calls to different locations.
<T> T	invokeSequential(java.util.List<? extends RemoteLocationContext> locations, RemoteMethod remoteMethod, java.lang.Class<T> expectedResultClass, java.lang.Object expectedResultValue)	Invokes sequential proxy calls to different locations.
<R extends RemoteLocationContext,T> RemoteResult	invokeSequential(RemoteMethod remoteMethod, java.util.List<R> locations, java.lang.Class<T> expectedResultClass, java.lang.Object expectedResultValue)	Invokes sequential proxy calls to different locations.
java.lang.Object	invokeSingle(java.lang.String nsId, RemoteMethod method)	Invokes a ClientProtocol method against the specified namespace.
<T> T	invokeSingle(java.lang.String nsId, RemoteMethod method, java.lang.Class<T> clazz)	Invokes a remote method against the specified namespace.
<T> T	invokeSingle(org.apache.hadoop.hdfs.protocol.ExtendedBlock extendedBlock, RemoteMethod method, java.lang.Class<T> clazz)	Invokes a remote method against the specified extendedBlock.
<T> T	invokeSingle(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, RemoteMethod method, java.util.List<RemoteLocation> locations, java.lang.Class<T> clazz)	Invokes a ClientProtocol method.
<T> T	invokeSingle(RemoteLocationContext location, RemoteMethod remoteMethod, java.lang.Class<T> clazz)	Invokes a single proxy call for a single location.
java.lang.Object	invokeSingleBlockPool(java.lang.String bpId, RemoteMethod method)	Invokes a ClientProtocol method.
static boolean	isUnavailableException(java.io.IOException ioe)	Check if the exception comes from an unavailable subcluster.
java.lang.String	refreshFairnessPolicyController(org.apache.hadoop.conf.Configuration conf)	Refreshes/changes the fairness policy controller implementation if possible and returns the controller class name.
void	shutdown()	Shutdown the client.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RouterRpcClient

public RouterRpcClient(org.apache.hadoop.conf.Configuration conf,
                       Router router,
                       ActiveNamenodeResolver resolver,
                       RouterRpcMonitor monitor,
                       org.apache.hadoop.hdfs.server.federation.router.RouterStateIdContext routerStateIdContext)

Create a router RPC client to manage remote procedure calls to NNs.

Parameters:

conf - Hdfs Configuration.

router - A router using this RPC client.

resolver - A NN resolver to determine the currently active NN in HA.

monitor - Optional performance monitor.

routerStateIdContext - the router state context object to hold the state ids for all namespaces.

Method Detail

getNamenodeResolver

public ActiveNamenodeResolver getNamenodeResolver()

Get the active namenode resolver used by this client.

Returns:

Active namenode resolver.

shutdown

public void shutdown()

Shutdown the client.

getNumConnections

public int getNumConnections()

Total number of available sockets between the router and NNs.

Returns:

Number of namenode clients.

getNumActiveConnections

public int getNumActiveConnections()

Total number of available sockets between the router and NNs.

Returns:

Number of namenode clients.

getNumIdleConnections

public int getNumIdleConnections()

Total number of idle sockets between the router and NNs.

Returns:

Number of namenode clients.

getNumActiveConnectionsRecently

public int getNumActiveConnectionsRecently()

Total number of active sockets between the router and NNs.

Returns:

Number of recently active namenode clients.

getNumConnectionPools

public int getNumConnectionPools()

Total number of open connection pools to a NN. Each connection pool. represents one user + one NN.

Returns:

Number of connection pools.

getNumCreatingConnections

public int getNumCreatingConnections()

Number of connections between the router and NNs being created sockets.

Returns:

Number of connections waiting to be created.

getJSON

public java.lang.String getJSON()

JSON representation of the connection pool.

Returns:

String representation of the JSON.

getAsyncCallerPoolJson

public java.lang.String getAsyncCallerPoolJson()

JSON representation of the async caller thread pool.

Returns:

String representation of the JSON.

getRejectedPermitsPerNsJSON

public java.lang.String getRejectedPermitsPerNsJSON()

JSON representation of the rejected permits for each nameservice.

Returns:

String representation of the rejected permits for each nameservice.

getAcceptedPermitsPerNsJSON

public java.lang.String getAcceptedPermitsPerNsJSON()

JSON representation of the accepted permits for each nameservice.

Returns:

String representation of the accepted permits for each nameservice.

invokeMethod

@VisibleForTesting
public java.lang.Object invokeMethod(org.apache.hadoop.security.UserGroupInformation ugi,
                                     java.util.List<? extends FederationNamenodeContext> namenodes,
                                     boolean useObserver,
                                     java.lang.Class<?> protocol,
                                     java.lang.reflect.Method method,
                                     java.lang.Object... params)
                              throws java.net.ConnectException,
                                     org.apache.hadoop.ipc.StandbyException,
                                     java.io.IOException

Invokes a method against the ClientProtocol proxy server. If a standby exception is generated by the call to the client, retries using the alternate server.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Parameters:

ugi - User group information.

namenodes - A prioritized list of namenodes within the same nameservice.

useObserver - Whether to use observer namenodes.

protocol - the protocol of the connection.

method - Remote ClientProtocol method to invoke.

params - Variable list of parameters matching the method.

Returns:

The result of invoking the method.

Throws:

java.net.ConnectException - If it cannot connect to any Namenode.

org.apache.hadoop.ipc.StandbyException - If all Namenodes are in Standby.

java.io.IOException - If it cannot invoke the method.

isUnavailableException

public static boolean isUnavailableException(java.io.IOException ioe)

Check if the exception comes from an unavailable subcluster.

Parameters:

ioe - IOException to check.

Returns:

If the exception comes from an unavailable subcluster.

invokeSingle

public <T> T invokeSingle(org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                          RemoteMethod method,
                          java.util.List<RemoteLocation> locations,
                          java.lang.Class<T> clazz)
                   throws java.io.IOException

Invokes a ClientProtocol method. Determines the target nameservice via a provided block.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote method return.

Parameters:

block - Block used to determine appropriate nameservice.

method - The remote method and parameters to invoke.

locations - The remote locations will be used.

clazz - Class for the return type.

Returns:

The result of invoking the method.

Throws:

java.io.IOException - If the invoke generated an error.

invokeSingleBlockPool

public java.lang.Object invokeSingleBlockPool(java.lang.String bpId,
                                              RemoteMethod method)
                                       throws java.io.IOException

Invokes a ClientProtocol method. Determines the target nameservice using the block pool id.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Parameters:

bpId - Block pool identifier.

method - The remote method and parameters to invoke.

Returns:

The result of invoking the method.

Throws:

java.io.IOException - If the invoke generated an error.

invokeSingle

public java.lang.Object invokeSingle(java.lang.String nsId,
                                     RemoteMethod method)
                              throws java.io.IOException

Invokes a ClientProtocol method against the specified namespace.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Parameters:

nsId - Target namespace for the method.

method - The remote method and parameters to invoke.

Returns:

The result of invoking the method.

Throws:

java.io.IOException - If the invoke generated an error.

invokeSingle

public <T> T invokeSingle(java.lang.String nsId,
                          RemoteMethod method,
                          java.lang.Class<T> clazz)
                   throws java.io.IOException

Invokes a remote method against the specified namespace.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote method return.

Parameters:

nsId - Target namespace for the method.

method - The remote method and parameters to invoke.

clazz - Class for the return type.

Returns:

The result of invoking the method.

Throws:

java.io.IOException - If the invoke generated an error.

invokeSingle

public <T> T invokeSingle(org.apache.hadoop.hdfs.protocol.ExtendedBlock extendedBlock,
                          RemoteMethod method,
                          java.lang.Class<T> clazz)
                   throws java.io.IOException

Invokes a remote method against the specified extendedBlock.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote method return.

Parameters:

extendedBlock - Target extendedBlock for the method.

method - The remote method and parameters to invoke.

clazz - Class for the return type.

Returns:

The result of invoking the method.

Throws:

java.io.IOException - If the invoke generated an error.

invokeSingle

public <T> T invokeSingle(RemoteLocationContext location,
                          RemoteMethod remoteMethod,
                          java.lang.Class<T> clazz)
                   throws java.io.IOException

Invokes a single proxy call for a single location.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote method return.

Parameters:

location - RemoteLocation to invoke.

remoteMethod - The remote method and parameters to invoke.

clazz - Class for the return type.

Returns:

The result of invoking the method if successful.

Throws:

java.io.IOException - If the invoke generated an error.

invokeSequential

public <T> T invokeSequential(java.util.List<? extends RemoteLocationContext> locations,
                              RemoteMethod remoteMethod)
                       throws java.io.IOException

Invokes sequential proxy calls to different locations. Continues to invoke calls until a call returns without throwing a remote exception.

Type Parameters:

T - The type of the remote method return.

Parameters:

locations - List of locations/nameservices to call concurrently.

remoteMethod - The remote method and parameters to invoke.

Returns:

The result of the first successful call, or if no calls are successful, the result of the last RPC call executed.

Throws:

java.io.IOException - if the success condition is not met and one of the RPC calls generated a remote exception.

invokeSequential

public <T> T invokeSequential(java.util.List<? extends RemoteLocationContext> locations,
                              RemoteMethod remoteMethod,
                              java.lang.Class<T> expectedResultClass,
                              java.lang.Object expectedResultValue)
                       throws java.io.IOException

Invokes sequential proxy calls to different locations. Continues to invoke calls until the success condition is met, or until all locations have been attempted. The success condition may be specified by:

• An expected result class
• An expected result value

If no expected result class/values are specified, the success condition is a call that does not throw a remote exception.

Type Parameters:

T - The type of the remote method return.

Parameters:

locations - List of locations/nameservices to call concurrently.

remoteMethod - The remote method and parameters to invoke.

expectedResultClass - In order to be considered a positive result, the return type must be of this class.

expectedResultValue - In order to be considered a positive result, the return value must equal the value of this object.

Returns:

The result of the first successful call, or if no calls are successful, the result of the first RPC call executed.

Throws:

java.io.IOException - if the success condition is not met, return the first remote exception generated.

invokeSequential

public <R extends RemoteLocationContext,T> RemoteResult invokeSequential(RemoteMethod remoteMethod,
                                                                               java.util.List<R> locations,
                                                                               java.lang.Class<T> expectedResultClass,
                                                                               java.lang.Object expectedResultValue)
                                                                        throws java.io.IOException

Invokes sequential proxy calls to different locations. Continues to invoke calls until the success condition is met, or until all locations have been attempted. The success condition may be specified by:

• An expected result class
• An expected result value

If no expected result class/values are specified, the success condition is a call that does not throw a remote exception. This returns RemoteResult, which contains the invoked location as well as the result.

Type Parameters:

R - The type of the remote location.

T - The type of the remote method return.

Parameters:

remoteMethod - The remote method and parameters to invoke.

locations - List of locations/nameservices to call concurrently.

expectedResultClass - In order to be considered a positive result, the return type must be of this class.

expectedResultValue - In order to be considered a positive result, the return value must equal the value of this object.

Returns:

The result of the first successful call, or if no calls are successful, the result of the first RPC call executed, along with the invoked location in form of RemoteResult.

Throws:

java.io.IOException - if the success condition is not met, return the first remote exception generated.

invokeAll

public <T extends RemoteLocationContext> boolean invokeAll(java.util.Collection<T> locations,
                                                           RemoteMethod method)
                                                    throws java.io.IOException

Invoke method in all locations and return success if any succeeds.

Type Parameters:

T - The type of the remote location.

Parameters:

locations - List of remote locations to call concurrently.

method - The remote method and parameters to invoke.

Returns:

If the call succeeds in any location.

Throws:

java.io.IOException - If any of the calls return an exception.

invokeConcurrent

public <T extends RemoteLocationContext,R> void invokeConcurrent(java.util.Collection<T> locations,
                                                                       RemoteMethod method)
                                                                throws java.io.IOException

Invoke multiple concurrent proxy calls to different clients. Returns an array of results.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote location.

R - The type of the remote method return.

Parameters:

locations - List of remote locations to call concurrently.

method - The remote method and parameters to invoke.

Throws:

java.io.IOException - If all the calls throw an exception.

invokeConcurrent

public <T extends RemoteLocationContext,R> java.util.Map<T,R> invokeConcurrent(java.util.Collection<T> locations,
                                                                                           RemoteMethod method,
                                                                                           java.lang.Class<R> clazz)
                                                                                    throws java.io.IOException

Invoke multiple concurrent proxy calls to different clients. Returns an array of results.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote location.

R - The type of the remote method return.

Parameters:

locations - List of remote locations to call concurrently.

method - The remote method and parameters to invoke.

clazz - Type of the remote return type.

Returns:

Result of invoking the method per subcluster: nsId to result.

Throws:

java.io.IOException - If all the calls throw an exception.

invokeConcurrent

public <T extends RemoteLocationContext,R> void invokeConcurrent(java.util.Collection<T> locations,
                                                                       RemoteMethod method,
                                                                       boolean requireResponse,
                                                                       boolean standby)
                                                                throws java.io.IOException

Invoke multiple concurrent proxy calls to different clients. Returns an array of results.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote location.

R - The type of the remote method return.

Parameters:

locations - List of remote locations to call concurrently.

method - The remote method and parameters to invoke.

requireResponse - If true an exception will be thrown if all calls do not complete. If false exceptions are ignored and all data results successfully received are returned.

standby - If the requests should go to the standby namenodes too.

Throws:

java.io.IOException - If all the calls throw an exception.

invokeConcurrent

public <T extends RemoteLocationContext,R> java.util.Map<T,R> invokeConcurrent(java.util.Collection<T> locations,
                                                                                           RemoteMethod method,
                                                                                           boolean requireResponse,
                                                                                           boolean standby,
                                                                                           java.lang.Class<R> clazz)
                                                                                    throws java.io.IOException

Invokes multiple concurrent proxy calls to different clients. Returns an array of results.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote location.

R - The type of the remote method return.

Parameters:

locations - List of remote locations to call concurrently.

method - The remote method and parameters to invoke.

requireResponse - If true an exception will be thrown if all calls do not complete. If false exceptions are ignored and all data results successfully received are returned.

standby - If the requests should go to the standby namenodes too.

clazz - Type of the remote return type.

Returns:

Result of invoking the method per subcluster: nsId to result.

Throws:

java.io.IOException - If requiredResponse=true and any of the calls throw an exception.

invokeConcurrent

public <T extends RemoteLocationContext,R> java.util.Map<T,R> invokeConcurrent(java.util.Collection<T> locations,
                                                                                           RemoteMethod method,
                                                                                           boolean requireResponse,
                                                                                           boolean standby,
                                                                                           long timeOutMs,
                                                                                           java.lang.Class<R> clazz)
                                                                                    throws java.io.IOException

Invokes multiple concurrent proxy calls to different clients. Returns an array of results.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote location.

R - The type of the remote method return.

Parameters:

locations - List of remote locations to call concurrently.

method - The remote method and parameters to invoke.

requireResponse - If true an exception will be thrown if all calls do not complete. If false exceptions are ignored and all data results successfully received are returned.

standby - If the requests should go to the standby namenodes too.

timeOutMs - Timeout for each individual call.

clazz - Type of the remote return type.

Returns:

Result of invoking the method per subcluster: nsId to result.

Throws:

java.io.IOException - If requiredResponse=true and any of the calls throw an exception.

invokeConcurrent

public <T extends RemoteLocationContext,R> java.util.List<RemoteResult<T,R>> invokeConcurrent(java.util.Collection<T> locations,
                                                                                                          RemoteMethod method,
                                                                                                          boolean standby,
                                                                                                          long timeOutMs,
                                                                                                          java.lang.Class<R> clazz)
                                                                                                   throws java.io.IOException

Invokes multiple concurrent proxy calls to different clients. Returns an array of results.

Re-throws exceptions generated by the remote RPC call as either RemoteException or IOException.

Type Parameters:

T - The type of the remote location.

R - The type of the remote method return

Parameters:

locations - List of remote locations to call concurrently.

method - The remote method and parameters to invoke.

standby - If the requests should go to the standby namenodes too.

timeOutMs - Timeout for each individual call.

clazz - Type of the remote return type.

Returns:

Result of invoking the method per subcluster (list of results). This includes the exception for each remote location.

Throws:

java.io.IOException - If there are errors invoking the method.

getRouterRpcFairnessPolicyController

public RouterRpcFairnessPolicyController getRouterRpcFairnessPolicyController()

getRejectedPermitForNs

public java.lang.Long getRejectedPermitForNs(java.lang.String ns)

getAcceptedPermitForNs

public java.lang.Long getAcceptedPermitForNs(java.lang.String ns)

refreshFairnessPolicyController

public java.lang.String refreshFairnessPolicyController(org.apache.hadoop.conf.Configuration conf)

Refreshes/changes the fairness policy controller implementation if possible and returns the controller class name.

Parameters:

conf - Configuration

Returns:

New controller class name if successfully refreshed, else old controller class name