Package org.apache.hadoop.yarn.server.sharedcachemanager.store

Class SCMStore

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.service.CompositeService

org.apache.hadoop.yarn.server.sharedcachemanager.store.SCMStore

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.service.Service

Direct Known Subclasses:

InMemorySCMStore

@Private
@Evolving
public abstract class SCMStore
extends org.apache.hadoop.service.CompositeService

An abstract class for the data store used by the shared cache manager service. All implementations of methods in this interface need to be thread safe and atomic.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.service.CompositeService

org.apache.hadoop.service.CompositeService.CompositeServiceShutdownHook

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field	Description
protected AppChecker	appChecker

Fields inherited from class org.apache.hadoop.service.CompositeService

STOP_ONLY_STARTED_SERVICES

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	SCMStore(java.lang.String name)

Method Summary

Modifier and Type	Method	Description
abstract java.lang.String	addResource(java.lang.String key, java.lang.String fileName)	Add a resource to the shared cache and it's associated filename.
abstract java.lang.String	addResourceReference(java.lang.String key, SharedCacheResourceReference ref)	Add a SharedCacheResourceReference to a resource and update the resource access time.
void	cleanResourceReferences(java.lang.String key)	Clean all resource references to a cache resource that contain application ids pointing to finished applications.
static AppChecker	createAppCheckerService(org.apache.hadoop.conf.Configuration conf)	Create an instance of the AppChecker service via reflection based on the YarnConfiguration.SCM_APP_CHECKER_CLASS parameter.
abstract java.util.Collection<SharedCacheResourceReference>	getResourceReferences(java.lang.String key)	Get the SharedCacheResourceReference(s) associated with the resource.
abstract boolean	isResourceEvictable(java.lang.String key, org.apache.hadoop.fs.FileStatus file)	Check if a specific resource is evictable according to the store's enabled cache eviction policies.
abstract boolean	removeResource(java.lang.String key)	Remove a resource from the shared cache.
abstract boolean	removeResourceReference(java.lang.String key, SharedCacheResourceReference ref, boolean updateAccessTime)	Remove a SharedCacheResourceReference from a resource.
abstract void	removeResourceReferences(java.lang.String key, java.util.Collection<SharedCacheResourceReference> refs, boolean updateAccessTime)	Remove a collection of SharedCacheResourceReferences from a resource.
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class org.apache.hadoop.service.CompositeService

addIfService, addService, getServices, removeService, serviceStart, serviceStop

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

appChecker

protected AppChecker appChecker

Constructor Detail

SCMStore

protected SCMStore(java.lang.String name)

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws java.lang.Exception

Overrides:

serviceInit in class org.apache.hadoop.service.CompositeService

Throws:

java.lang.Exception

addResource

@Private
public abstract java.lang.String addResource(java.lang.String key,
                                             java.lang.String fileName)

Add a resource to the shared cache and it's associated filename. The resource is identified by a unique key. If the key already exists no action is taken and the filename of the existing resource is returned. If the key does not exist, the resource is added, it's access time is set, and the filename of the resource is returned.

Parameters:

key - a unique identifier for a resource

fileName - the filename of the resource

Returns:

the filename of the resource as represented by the cache

removeResource

@Private
public abstract boolean removeResource(java.lang.String key)

Remove a resource from the shared cache.

Parameters:

key - a unique identifier for a resource

Returns:

true if the resource was removed or did not exist, false if the resource existed, contained at least one SharedCacheResourceReference and was not removed.

addResourceReference

@Private
public abstract java.lang.String addResourceReference(java.lang.String key,
                                                      SharedCacheResourceReference ref)

Add a SharedCacheResourceReference to a resource and update the resource access time.

Parameters:

key - a unique identifier for a resource

ref - the SharedCacheResourceReference to add

Returns:

String the filename of the resource if the SharedCacheResourceReference was added or already existed. null if the resource did not exist

getResourceReferences

@Private
public abstract java.util.Collection<SharedCacheResourceReference> getResourceReferences(java.lang.String key)

Get the SharedCacheResourceReference(s) associated with the resource.

Parameters:

key - a unique identifier for a resource

Returns:

an unmodifiable collection of SharedCacheResourceReferences. If the resource does not exist, an empty set is returned.

removeResourceReference

@Private
public abstract boolean removeResourceReference(java.lang.String key,
                                                SharedCacheResourceReference ref,
                                                boolean updateAccessTime)

Remove a SharedCacheResourceReference from a resource.

Parameters:

key - a unique identifier for a resource

ref - the SharedCacheResourceReference to remove

updateAccessTime - true if the call should update the access time for the resource

Returns:

true if the reference was removed, false otherwise

removeResourceReferences

@Private
public abstract void removeResourceReferences(java.lang.String key,
                                              java.util.Collection<SharedCacheResourceReference> refs,
                                              boolean updateAccessTime)

Remove a collection of SharedCacheResourceReferences from a resource.

Parameters:

key - a unique identifier for a resource

refs - the collection of SharedCacheResourceReferences to remove

updateAccessTime - true if the call should update the access time for the resource

cleanResourceReferences

@Private
public void cleanResourceReferences(java.lang.String key)
                             throws org.apache.hadoop.yarn.exceptions.YarnException

Clean all resource references to a cache resource that contain application ids pointing to finished applications. If the resource key does not exist, do nothing.

Parameters:

key - a unique identifier for a resource

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

isResourceEvictable

@Private
public abstract boolean isResourceEvictable(java.lang.String key,
                                            org.apache.hadoop.fs.FileStatus file)

Check if a specific resource is evictable according to the store's enabled cache eviction policies.

Parameters:

key - a unique identifier for a resource

file - the FileStatus object for the resource file in the file system.

Returns:

true if the resource is evicatble, false otherwise

createAppCheckerService

@Private
public static AppChecker createAppCheckerService(org.apache.hadoop.conf.Configuration conf)

Create an instance of the AppChecker service via reflection based on the YarnConfiguration.SCM_APP_CHECKER_CLASS parameter.

Parameters:

conf -

Returns:

an instance of the AppChecker class