Package org.apache.hadoop.registry.client.impl.zk

Class CuratorService

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.service.CompositeService

org.apache.hadoop.registry.client.impl.zk.CuratorService

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, RegistryConstants, RegistryBindingSource, org.apache.hadoop.service.Service

Direct Known Subclasses:

RegistryOperationsService

@Private
@Evolving
public class CuratorService
extends org.apache.hadoop.service.CompositeService
implements RegistryConstants, RegistryBindingSource

This service binds to Zookeeper via Apache Curator. It is more generic than just the YARN service registry; it does not implement any of the Registry Operations API.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

org.apache.hadoop.service.CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields inherited from class org.apache.hadoop.service.CompositeService

STOP_ONLY_STARTED_SERVICES

Fields inherited from interface org.apache.hadoop.registry.client.api.RegistryConstants

DEFAULT_DNS_ENABLED, DEFAULT_DNS_PORT, DEFAULT_DNS_SPLIT_REVERSE_ZONE, DEFAULT_DNSSEC_PRIVATE_KEY_FILE, DEFAULT_REGISTRY_CLIENT_JAAS_CONTEXT, DEFAULT_REGISTRY_SECURE, DEFAULT_REGISTRY_SYSTEM_ACCOUNTS, DEFAULT_REGISTRY_USER_ACCOUNTS, DEFAULT_REGISTRY_ZK_QUORUM, DEFAULT_ZK_CONNECTION_TIMEOUT, DEFAULT_ZK_REGISTRY_ROOT, DEFAULT_ZK_RETRY_CEILING, DEFAULT_ZK_RETRY_INTERVAL, DEFAULT_ZK_RETRY_TIMES, DEFAULT_ZK_SESSION_TIMEOUT, DNS_PREFIX, KEY_DNS_BIND_ADDRESS, KEY_DNS_DOMAIN, KEY_DNS_ENABLED, KEY_DNS_PORT, KEY_DNS_SPLIT_REVERSE_ZONE, KEY_DNS_SPLIT_REVERSE_ZONE_RANGE, KEY_DNS_TTL, KEY_DNS_ZONE_IP_MAX, KEY_DNS_ZONE_IP_MIN, KEY_DNS_ZONE_MASK, KEY_DNS_ZONE_SUBNET, KEY_DNS_ZONES_DIR, KEY_DNSSEC_ENABLED, KEY_DNSSEC_PRIVATE_KEY_FILE, KEY_DNSSEC_PUBLIC_KEY, KEY_REGISTRY_CLIENT_AUTH, KEY_REGISTRY_CLIENT_AUTHENTICATION_ID, KEY_REGISTRY_CLIENT_AUTHENTICATION_PASSWORD, KEY_REGISTRY_CLIENT_JAAS_CONTEXT, KEY_REGISTRY_KERBEROS_REALM, KEY_REGISTRY_SECURE, KEY_REGISTRY_SYSTEM_ACCOUNTS, KEY_REGISTRY_USER_ACCOUNTS, KEY_REGISTRY_ZK_CONNECTION_TIMEOUT, KEY_REGISTRY_ZK_QUORUM, KEY_REGISTRY_ZK_RETRY_CEILING, KEY_REGISTRY_ZK_RETRY_INTERVAL, KEY_REGISTRY_ZK_RETRY_TIMES, KEY_REGISTRY_ZK_ROOT, KEY_REGISTRY_ZK_SESSION_TIMEOUT, MAX_FQDN_LABEL_LENGTH, PATH_SYSTEM_SERVICES, PATH_USER_SERVICES, PATH_USERS, REGISTRY_CLIENT_AUTH_ANONYMOUS, REGISTRY_CLIENT_AUTH_DIGEST, REGISTRY_CLIENT_AUTH_KERBEROS, REGISTRY_CLIENT_AUTH_SIMPLE, REGISTRY_PREFIX, SUBPATH_COMPONENTS, ZK_PREFIX

Constructor Summary

Constructors

Constructor	Description
CuratorService(java.lang.String name)	Create an instance using this service as the binding source (i.e.
CuratorService(java.lang.String name, RegistryBindingSource bindingSource)	Construct the service.

Method Summary

Modifier and Type	Method	Description
boolean	addWriteAccessor(java.lang.String id, java.lang.String pass)	Add a new write access entry for all future write operations.
java.lang.String	bindingDiagnosticDetails()	Get the binding diagnostics.
protected java.lang.String	buildConnectionString()	Override point: get the connection string used to connect to the ZK service.
protected java.lang.String	buildSecurityDiagnostics()	Build the security diagnostics string.
void	clearWriteAccessors()	Clear all write accessors.
protected void	createEnsembleProvider()	Create the ensemble provider for this registry, by invoking RegistryBindingSource.supplyBindingInformation() on the provider stored in bindingSource.
protected java.lang.String	createFullPath(java.lang.String path)	Create a full path from the registry root and the supplied subdir.
ZKPathDumper	dumpPath(boolean verbose)	Return a path dumper instance which can do a full dump of the registry tree in its toString() operation.
protected java.lang.String	dumpRegistryRobustly(boolean verbose)	Diagnostics method to dump a registry robustly.
RegistryBindingSource	getBindingSource()	Get the registry binding source ...
protected RegistrySecurity	getRegistrySecurity()	Get the registry security helper.
void	instantiateCacheForRegistry()	Instantiate the Curator cache that monitors the registry for node addition, update and deletion.
boolean	isSecure()	Flag to indicate whether or not the registry is secure.
boolean	maybeCreate(java.lang.String path, org.apache.zookeeper.CreateMode mode, java.util.List<org.apache.zookeeper.data.ACL> acl, boolean createParents)	Create a path if it does not exist.
protected java.io.IOException	operationFailure(java.lang.String path, java.lang.String operation, java.lang.Exception exception)	Create an IOE when an operation fails.
protected java.io.IOException	operationFailure(java.lang.String path, java.lang.String operation, java.lang.Exception exception, java.util.List<org.apache.zookeeper.data.ACL> acls)	Create an IOE when an operation fails.
void	registerPathListener(PathListener listener)	Registers a listener to path related events.
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf)	Init the service.
protected void	serviceStart()	Start the service.
protected void	serviceStop()	Close the ZK connection if it is open.
void	setKerberosPrincipalAndKeytab(java.lang.String principal, java.lang.String keytab)
void	startCache()
BindingInformation	supplyBindingInformation()	Supply the binding information.
java.lang.String	toString()
void	zkCreate(java.lang.String path, org.apache.zookeeper.CreateMode mode, byte[] data, java.util.List<org.apache.zookeeper.data.ACL> acls)	Create a path with given data.
void	zkDelete(java.lang.String path, boolean recursive, org.apache.curator.framework.api.BackgroundCallback backgroundCallback)	Delete a directory/directory tree.
java.util.List<org.apache.zookeeper.data.ACL>	zkGetACLS(java.lang.String path)	Get the ACLs of a path.
java.util.List<java.lang.String>	zkList(java.lang.String path)	List all children of a path.
void	zkMkParentPath(java.lang.String path, java.util.List<org.apache.zookeeper.data.ACL> acl)	Recursively make a path.
boolean	zkMkPath(java.lang.String path, org.apache.zookeeper.CreateMode mode, boolean createParents, java.util.List<org.apache.zookeeper.data.ACL> acls)	Create a directory.
boolean	zkPathExists(java.lang.String path)	Probe for a path existing.
java.lang.String	zkPathMustExist(java.lang.String path)	Verify a path exists.
byte[]	zkRead(java.lang.String path)	Read data on a path.
boolean	zkSet(java.lang.String path, org.apache.zookeeper.CreateMode mode, byte[] data, java.util.List<org.apache.zookeeper.data.ACL> acl, boolean overwrite)	Create or update an entry.
org.apache.zookeeper.data.Stat	zkStat(java.lang.String path)	Stat the file.
void	zkUpdate(java.lang.String path, byte[] data)	Update the data for a path.

Methods inherited from class org.apache.hadoop.service.CompositeService

addIfService, addService, getServices, removeService

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CuratorService

public CuratorService(java.lang.String name,
                      RegistryBindingSource bindingSource)

Construct the service.

Parameters:

name - service name

bindingSource - source of binding information. If null: use this instance

CuratorService

public CuratorService(java.lang.String name)

Create an instance using this service as the binding source (i.e. read configuration options from the registry).

Parameters:

name - service name

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws java.lang.Exception

Init the service. This is where the security bindings are set up.

Overrides:

serviceInit in class org.apache.hadoop.service.CompositeService

Parameters:

conf - configuration of the service

Throws:

java.lang.Exception

setKerberosPrincipalAndKeytab

public void setKerberosPrincipalAndKeytab(java.lang.String principal,
                                          java.lang.String keytab)

serviceStart

protected void serviceStart()
                     throws java.lang.Exception

Start the service. This is where the curator instance is started.

Overrides:

serviceStart in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

serviceStop

protected void serviceStop()
                    throws java.lang.Exception

Close the ZK connection if it is open.

Overrides:

serviceStop in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

isSecure

public boolean isSecure()

Flag to indicate whether or not the registry is secure. Valid once the service is inited.

Returns:

service security policy

getRegistrySecurity

protected RegistrySecurity getRegistrySecurity()

Get the registry security helper.

Returns:

the registry security helper

buildSecurityDiagnostics

protected java.lang.String buildSecurityDiagnostics()

Build the security diagnostics string.

Returns:

a string for diagnostics

toString

public java.lang.String toString()

Overrides:

toString in class org.apache.hadoop.service.AbstractService

bindingDiagnosticDetails

public java.lang.String bindingDiagnosticDetails()

Get the binding diagnostics.

Returns:

a diagnostics string valid after the service is started.

createFullPath

protected java.lang.String createFullPath(java.lang.String path)
                                   throws java.io.IOException

Create a full path from the registry root and the supplied subdir.

Parameters:

path - path of operation

Returns:

an absolute path

Throws:

java.lang.IllegalArgumentException - if the path is invalide

java.io.IOException

getBindingSource

public RegistryBindingSource getBindingSource()

Get the registry binding source ... this can be used to create new ensemble providers

Returns:

the registry binding source in use

createEnsembleProvider

protected void createEnsembleProvider()

Create the ensemble provider for this registry, by invoking RegistryBindingSource.supplyBindingInformation() on the provider stored in bindingSource. Sets ensembleProvider to that value; sets connectionDescription to the binding info for use in toString and logging;

supplyBindingInformation

public BindingInformation supplyBindingInformation()

Supply the binding information. This implementation returns a fixed ensemble bonded to the quorum supplied by buildConnectionString().

Specified by:

supplyBindingInformation in interface RegistryBindingSource

Returns:

the binding information

buildConnectionString

protected java.lang.String buildConnectionString()

Override point: get the connection string used to connect to the ZK service.

Returns:

a registry quorum

operationFailure

protected java.io.IOException operationFailure(java.lang.String path,
                                               java.lang.String operation,
                                               java.lang.Exception exception)

Create an IOE when an operation fails.

Parameters:

path - path of operation

operation - operation attempted

exception - caught the exception caught

Returns:

an IOE to throw that contains the path and operation details.

operationFailure

protected java.io.IOException operationFailure(java.lang.String path,
                                               java.lang.String operation,
                                               java.lang.Exception exception,
                                               java.util.List<org.apache.zookeeper.data.ACL> acls)

Create an IOE when an operation fails.

Parameters:

path - path of operation

operation - operation attempted

exception - caught the exception caught

Returns:

an IOE to throw that contains the path and operation details.

maybeCreate

@VisibleForTesting
public boolean maybeCreate(java.lang.String path,
                           org.apache.zookeeper.CreateMode mode,
                           java.util.List<org.apache.zookeeper.data.ACL> acl,
                           boolean createParents)
                    throws java.io.IOException

Create a path if it does not exist. The check is poll + create; there's a risk that another process may create the same path before the create() operation is executed/ propagated to the ZK node polled.

Parameters:

path - path to create

acl - ACL for path -used when creating a new entry

createParents - flag to trigger parent creation

Returns:

true iff the path was created

Throws:

java.io.IOException

zkStat

public org.apache.zookeeper.data.Stat zkStat(java.lang.String path)
                                      throws java.io.IOException

Stat the file.

Parameters:

path - path of operation

Returns:

a curator stat entry

Throws:

java.io.IOException - on a failure

org.apache.hadoop.fs.PathNotFoundException - if the path was not found

zkGetACLS

public java.util.List<org.apache.zookeeper.data.ACL> zkGetACLS(java.lang.String path)
                                                        throws java.io.IOException

Get the ACLs of a path.

Parameters:

path - path of operation

Returns:

a possibly empty list of ACLs

Throws:

java.io.IOException

zkPathExists

public boolean zkPathExists(java.lang.String path)
                     throws java.io.IOException

Probe for a path existing.

Parameters:

path - path of operation

Returns:

true if the path was visible from the ZK server queried.

Throws:

java.io.IOException - on any exception other than PathNotFoundException

zkPathMustExist

public java.lang.String zkPathMustExist(java.lang.String path)
                                 throws java.io.IOException

Verify a path exists.

Parameters:

path - path of operation

Throws:

org.apache.hadoop.fs.PathNotFoundException - if the path is absent

java.io.IOException

zkMkPath

public boolean zkMkPath(java.lang.String path,
                        org.apache.zookeeper.CreateMode mode,
                        boolean createParents,
                        java.util.List<org.apache.zookeeper.data.ACL> acls)
                 throws java.io.IOException

Create a directory. It is not an error if it already exists.

Parameters:

path - path to create

mode - mode for path

createParents - flag to trigger parent creation

acls - ACL for path

Throws:

java.io.IOException - any problem

zkMkParentPath

public void zkMkParentPath(java.lang.String path,
                           java.util.List<org.apache.zookeeper.data.ACL> acl)
                    throws java.io.IOException

Recursively make a path.

Parameters:

path - path to create

acl - ACL for path

Throws:

java.io.IOException - any problem

zkCreate

public void zkCreate(java.lang.String path,
                     org.apache.zookeeper.CreateMode mode,
                     byte[] data,
                     java.util.List<org.apache.zookeeper.data.ACL> acls)
              throws java.io.IOException

Create a path with given data. byte[0] is used for a path without data.

Parameters:

path - path of operation

data - initial data

acls -

Throws:

java.io.IOException

zkUpdate

public void zkUpdate(java.lang.String path,
                     byte[] data)
              throws java.io.IOException

Update the data for a path.

Parameters:

path - path of operation

data - new data

Throws:

java.io.IOException

zkSet

public boolean zkSet(java.lang.String path,
                     org.apache.zookeeper.CreateMode mode,
                     byte[] data,
                     java.util.List<org.apache.zookeeper.data.ACL> acl,
                     boolean overwrite)
              throws java.io.IOException

Create or update an entry.

Parameters:

path - path

data - data

acl - ACL for path -used when creating a new entry

overwrite - enable overwrite

Returns:

true if the entry was created, false if it was simply updated.

Throws:

java.io.IOException

zkDelete

public void zkDelete(java.lang.String path,
                     boolean recursive,
                     org.apache.curator.framework.api.BackgroundCallback backgroundCallback)
              throws java.io.IOException

Delete a directory/directory tree. It is not an error to delete a path that does not exist.

Parameters:

path - path of operation

recursive - flag to trigger recursive deletion

backgroundCallback - callback; this being set converts the operation into an async/background operation. task

Throws:

java.io.IOException - on problems other than no-such-path

zkList

public java.util.List<java.lang.String> zkList(java.lang.String path)
                                        throws java.io.IOException

List all children of a path.

Parameters:

path - path of operation

Returns:

a possibly empty list of children

Throws:

java.io.IOException

zkRead

public byte[] zkRead(java.lang.String path)
              throws java.io.IOException

Read data on a path.

Parameters:

path - path of operation

Returns:

the data

Throws:

java.io.IOException - read failure

dumpPath

public ZKPathDumper dumpPath(boolean verbose)

Return a path dumper instance which can do a full dump of the registry tree in its toString() operation.

Parameters:

verbose - verbose flag - includes more details (such as ACLs)

Returns:

a class to dump the registry

addWriteAccessor

public boolean addWriteAccessor(java.lang.String id,
                                java.lang.String pass)
                         throws java.io.IOException

Add a new write access entry for all future write operations.

Parameters:

id - ID to use

pass - password

Throws:

java.io.IOException - on any failure to build the digest

clearWriteAccessors

public void clearWriteAccessors()

Clear all write accessors.

dumpRegistryRobustly

protected java.lang.String dumpRegistryRobustly(boolean verbose)

Diagnostics method to dump a registry robustly. Any exception raised is swallowed.

Parameters:

verbose - verbose path dump

Returns:

the registry tree

registerPathListener

public void registerPathListener(PathListener listener)

Registers a listener to path related events.

Parameters:

listener - the listener.

instantiateCacheForRegistry

public void instantiateCacheForRegistry()

Instantiate the Curator cache that monitors the registry for node addition, update and deletion.

startCache

public void startCache()