Package org.apache.hadoop.hdfs.server.federation.router

Class ConnectionPool

java.lang.Object

org.apache.hadoop.hdfs.server.federation.router.ConnectionPool

@Private
@Evolving
public class ConnectionPool
extends java.lang.Object

Maintains a pool of connections for each User (including tokens) + NN. The RPC client maintains a single socket, to achieve throughput similar to a NN, each request is multiplexed across multiple sockets/connections from a pool.

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	ConnectionPool(org.apache.hadoop.conf.Configuration config, java.lang.String address, org.apache.hadoop.security.UserGroupInformation user, int minPoolSize, int maxPoolSize, float minActiveRatio, java.lang.Class<?> proto, PoolAlignmentContext alignmentContext)

Method Summary

Modifier and Type	Method	Description
void	addConnection(ConnectionContext conn)	Add a connection to the current pool.
protected void	close()	Close the connection pool.
java.util.concurrent.atomic.AtomicInteger	getClientIndex()	Get the clientIndex used to calculate index for lookup.
protected ConnectionContext	getConnection()	Return the next connection round-robin.
protected ConnectionPoolId	getConnectionPoolId()	Get the connection pool identifier.
java.lang.String	getJSON()	JSON representation of the connection pool.
protected long	getLastActiveTime()	Get the last time the connection pool was used.
protected int	getMaxSize()	Get the maximum number of connections allowed in this pool.
protected float	getMinActiveRatio()	Get the minimum ratio of active connections in this pool.
protected int	getMinSize()	Get the minimum number of connections in this pool.
protected int	getNumActiveConnections()	Number of active connections in the pool.
protected int	getNumActiveConnectionsRecently()	Number of active connections recently in the pool.
protected int	getNumConnections()	Number of connections in the pool.
protected int	getNumIdleConnections()	Number of usable i.e.
PoolAlignmentContext	getPoolAlignmentContext()	Get the alignment context for this pool.
ConnectionContext	newConnection()	Create a new proxy wrapper for a client NN connection.
protected static <T> ConnectionContext	newConnection(org.apache.hadoop.conf.Configuration conf, java.lang.String nnAddress, org.apache.hadoop.security.UserGroupInformation ugi, java.lang.Class<T> proto, boolean enableMultiSocket, int socketIndex, org.apache.hadoop.ipc.AlignmentContext alignmentContext)	Creates a proxy wrapper for a client NN connection.
java.util.List<ConnectionContext>	removeConnections(int num)	Remove connections from the current pool.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ConnectionPool

protected ConnectionPool(org.apache.hadoop.conf.Configuration config,
                         java.lang.String address,
                         org.apache.hadoop.security.UserGroupInformation user,
                         int minPoolSize,
                         int maxPoolSize,
                         float minActiveRatio,
                         java.lang.Class<?> proto,
                         PoolAlignmentContext alignmentContext)
                  throws java.io.IOException

Throws:

java.io.IOException

Method Detail

getMaxSize

protected int getMaxSize()

Get the maximum number of connections allowed in this pool.

Returns:

Maximum number of connections.

getMinSize

protected int getMinSize()

Get the minimum number of connections in this pool.

Returns:

Minimum number of connections.

getMinActiveRatio

protected float getMinActiveRatio()

Get the minimum ratio of active connections in this pool.

Returns:

Minimum ratio of active connections.

getConnectionPoolId

protected ConnectionPoolId getConnectionPoolId()

Get the connection pool identifier.

Returns:

Connection pool identifier.

getClientIndex

@VisibleForTesting
public java.util.concurrent.atomic.AtomicInteger getClientIndex()

Get the clientIndex used to calculate index for lookup.

Returns:

Client index.

getPoolAlignmentContext

public PoolAlignmentContext getPoolAlignmentContext()

Get the alignment context for this pool.

Returns:

Alignment context

getConnection

protected ConnectionContext getConnection()

Return the next connection round-robin.

Returns:

Connection context.

addConnection

public void addConnection(ConnectionContext conn)

Add a connection to the current pool. It uses a Copy-On-Write approach.

Parameters:

conn - New connection to add to the pool.

removeConnections

public java.util.List<ConnectionContext> removeConnections(int num)

Remove connections from the current pool.

Parameters:

num - Number of connections to remove.

Returns:

Removed connections.

close

protected void close()

Close the connection pool.

getNumConnections

protected int getNumConnections()

Number of connections in the pool.

Returns:

Number of connections.

getNumActiveConnections

protected int getNumActiveConnections()

Number of active connections in the pool.

Returns:

Number of active connections.

getNumIdleConnections

protected int getNumIdleConnections()

Number of usable i.e. no active thread connections.

Returns:

Number of idle connections

getNumActiveConnectionsRecently

protected int getNumActiveConnectionsRecently()

Number of active connections recently in the pool.

Returns:

Number of active connections recently.

getLastActiveTime

protected long getLastActiveTime()

Get the last time the connection pool was used.

Returns:

Last time the connection pool was used.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

getJSON

public java.lang.String getJSON()

JSON representation of the connection pool.

Returns:

String representation of the JSON.

newConnection

public ConnectionContext newConnection()
                                throws java.io.IOException

Create a new proxy wrapper for a client NN connection.

Returns:

Proxy for the target ClientProtocol that contains the user's security context.

Throws:

java.io.IOException - If it cannot get a new connection.

newConnection

protected static <T> ConnectionContext newConnection(org.apache.hadoop.conf.Configuration conf,
                                                     java.lang.String nnAddress,
                                                     org.apache.hadoop.security.UserGroupInformation ugi,
                                                     java.lang.Class<T> proto,
                                                     boolean enableMultiSocket,
                                                     int socketIndex,
                                                     org.apache.hadoop.ipc.AlignmentContext alignmentContext)
                                              throws java.io.IOException

Creates a proxy wrapper for a client NN connection. Each proxy contains context for a single user/security context. To maximize throughput it is recommended to use multiple connection per user+server, allowing multiple writes and reads to be dispatched in parallel.

Type Parameters:

T - Input type T.

Parameters:

conf - Configuration for the connection.

nnAddress - Address of server supporting the ClientProtocol.

ugi - User context.

proto - Interface of the protocol.

enableMultiSocket - Enable multiple socket or not.

socketIndex - Index for FederationConnectionId.

alignmentContext - Client alignment context.

Returns:

proto for the target ClientProtocol that contains the user's security context.

Throws:

java.io.IOException - If it cannot be created.