Package org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf

Class ZKConfigurationStore

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.YarnConfigurationStore

org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.ZKConfigurationStore

All Implemented Interfaces:

java.lang.AutoCloseable

public class ZKConfigurationStore
extends YarnConfigurationStore

A Zookeeper-based implementation of YarnConfigurationStore.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.YarnConfigurationStore

YarnConfigurationStore.LogMutation

Field Summary

Fields

Modifier and Type	Field	Description
protected static org.apache.hadoop.yarn.server.records.Version	CURRENT_VERSION_INFO
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
ZKConfigurationStore()

Method Summary

Modifier and Type	Method	Description
void	close()	Closes the configuration store, releasing any required resources.
void	confirmMutation(YarnConfigurationStore.LogMutation pendingMutation, boolean isValid)	Should be called after logMutation.
void	format()	Format the persisted configuration.
long	getConfigVersion()	Get the last updated config version.
java.util.List<YarnConfigurationStore.LogMutation>	getConfirmedConfHistory(long fromId)	Get a list of confirmed configuration mutations starting from a given id.
org.apache.hadoop.yarn.server.records.Version	getConfStoreVersion()	Get schema version of persisted conf store, for detecting compatibility issues when changing conf store schema.
org.apache.hadoop.yarn.server.records.Version	getCurrentVersion()	Get the hard-coded schema version, for comparison against the schema version currently persisted.
protected java.util.LinkedList<YarnConfigurationStore.LogMutation>	getLogs()	Get a list of configuration mutations.
protected byte[]	getZkData(java.lang.String path)
void	initialize(org.apache.hadoop.conf.Configuration config, org.apache.hadoop.conf.Configuration schedConf, RMContext rmContext)	Initialize the configuration store, with schedConf as the initial scheduler configuration.
void	logMutation(YarnConfigurationStore.LogMutation logMutation)	Logs the configuration change to backing store.
org.apache.hadoop.conf.Configuration	retrieve()	Retrieve the persisted configuration.
protected void	safeCreateZkData(java.lang.String path, byte[] data)
protected void	setZkData(java.lang.String path, byte[] data)
void	storeVersion()	Persist the hard-coded schema version to the conf store.

Methods inherited from class org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.YarnConfigurationStore

checkVersion

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

CURRENT_VERSION_INFO

@VisibleForTesting
protected static final org.apache.hadoop.yarn.server.records.Version CURRENT_VERSION_INFO

Constructor Detail

ZKConfigurationStore

public ZKConfigurationStore()

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration config,
                       org.apache.hadoop.conf.Configuration schedConf,
                       RMContext rmContext)
                throws java.lang.Exception

Description copied from class: YarnConfigurationStore

Initialize the configuration store, with schedConf as the initial scheduler configuration. If a persisted store already exists, use the scheduler configuration stored there, and ignore schedConf.

Specified by:

initialize in class YarnConfigurationStore

Parameters:

config - configuration to initialize store with

schedConf - Initial key-value scheduler configuration to persist.

rmContext - RMContext for this configuration store

Throws:

java.io.IOException - if initialization fails

java.lang.Exception

getLogs

@VisibleForTesting
protected java.util.LinkedList<YarnConfigurationStore.LogMutation> getLogs()
                                                                    throws java.lang.Exception

Description copied from class: YarnConfigurationStore

Get a list of configuration mutations.

Specified by:

getLogs in class YarnConfigurationStore

Returns:

list of configuration mutations.

Throws:

java.lang.Exception - On mutation fetch failure

getCurrentVersion

public org.apache.hadoop.yarn.server.records.Version getCurrentVersion()

Description copied from class: YarnConfigurationStore

Get the hard-coded schema version, for comparison against the schema version currently persisted.

Specified by:

getCurrentVersion in class YarnConfigurationStore

Returns:

Current hard-coded schema version

getConfStoreVersion

public org.apache.hadoop.yarn.server.records.Version getConfStoreVersion()
                                                                  throws java.lang.Exception

Description copied from class: YarnConfigurationStore

Get schema version of persisted conf store, for detecting compatibility issues when changing conf store schema.

Specified by:

getConfStoreVersion in class YarnConfigurationStore

Returns:

Schema version currently used by the persisted configuration store.

Throws:

java.lang.Exception - On version fetch failure

format

public void format()
            throws java.lang.Exception

Description copied from class: YarnConfigurationStore

Format the persisted configuration.

Specified by:

format in class YarnConfigurationStore

Throws:

java.io.IOException - on failure to format

java.lang.Exception

storeVersion

public void storeVersion()
                  throws java.lang.Exception

Description copied from class: YarnConfigurationStore

Persist the hard-coded schema version to the conf store.

Specified by:

storeVersion in class YarnConfigurationStore

Throws:

java.lang.Exception - On storage failure

logMutation

public void logMutation(YarnConfigurationStore.LogMutation logMutation)
                 throws java.lang.Exception

Description copied from class: YarnConfigurationStore

Logs the configuration change to backing store.

Specified by:

logMutation in class YarnConfigurationStore

Parameters:

logMutation - configuration change to be persisted in write ahead log

Throws:

java.io.IOException - if logging fails

java.lang.Exception

confirmMutation

public void confirmMutation(YarnConfigurationStore.LogMutation pendingMutation,
                            boolean isValid)
                     throws java.lang.Exception

Description copied from class: YarnConfigurationStore

Should be called after logMutation. Gets the pending mutation last logged by logMutation and marks the mutation as persisted (no longer pending). If isValid is true, merge the mutation with the persisted configuration.

Specified by:

confirmMutation in class YarnConfigurationStore

Parameters:

pendingMutation - the log mutation to apply

isValid - if true, update persisted configuration with pending mutation.

Throws:

java.lang.Exception - if mutation confirmation fails

retrieve

public org.apache.hadoop.conf.Configuration retrieve()

Description copied from class: YarnConfigurationStore

Retrieve the persisted configuration.

Specified by:

retrieve in class YarnConfigurationStore

Returns:

configuration as key-value

getConfigVersion

public long getConfigVersion()
                      throws java.lang.Exception

Description copied from class: YarnConfigurationStore

Get the last updated config version.

Specified by:

getConfigVersion in class YarnConfigurationStore

Returns:

Last updated config version.

Throws:

java.lang.Exception - On version fetch failure.

getConfirmedConfHistory

public java.util.List<YarnConfigurationStore.LogMutation> getConfirmedConfHistory(long fromId)

Description copied from class: YarnConfigurationStore

Get a list of confirmed configuration mutations starting from a given id.

Specified by:

getConfirmedConfHistory in class YarnConfigurationStore

Parameters:

fromId - id from which to start getting mutations, inclusive

Returns:

list of configuration mutations

getZkData

@VisibleForTesting
protected byte[] getZkData(java.lang.String path)
                    throws java.lang.Exception

Throws:

java.lang.Exception

setZkData

@VisibleForTesting
protected void setZkData(java.lang.String path,
                         byte[] data)
                  throws java.lang.Exception

Throws:

java.lang.Exception

safeCreateZkData

@VisibleForTesting
protected void safeCreateZkData(java.lang.String path,
                                byte[] data)
                         throws java.lang.Exception

Throws:

java.lang.Exception

close

public void close()
           throws java.io.IOException

Description copied from class: YarnConfigurationStore

Closes the configuration store, releasing any required resources.

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in class YarnConfigurationStore

Throws:

java.io.IOException - on failure to close