org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity
Class CapacityScheduler

java.lang.Object
  extended by org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable, org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent>, Recoverable, CapacitySchedulerContext, PreemptableResourceScheduler, ResourceScheduler, YarnScheduler

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Evolving
public class CapacityScheduler
extends Object
implements PreemptableResourceScheduler, CapacitySchedulerContext, org.apache.hadoop.conf.Configurable


Field Summary
protected  Map<org.apache.hadoop.yarn.api.records.ApplicationAttemptId,FiCaSchedulerApp> applications
           
static String ROOT_QUEUE
           
 
Constructor Summary
CapacityScheduler()
           
 
Method Summary
 Allocation allocate(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask, List<org.apache.hadoop.yarn.api.records.ContainerId> release, List<String> blacklistAdditions, List<String> blacklistRemovals)
          The main api between the ApplicationMaster and the Scheduler.
 boolean checkAccess(org.apache.hadoop.security.UserGroupInformation callerUGI, org.apache.hadoop.yarn.api.records.QueueACL acl, String queueName)
          Check if the user has permission to perform the operation.
 void dropContainerReservation(RMContainer container)
          If the scheduler support container reservations, this method is used to ask the scheduler to drop the reservation for the given container.
 Comparator<FiCaSchedulerApp> getApplicationComparator()
           
 org.apache.hadoop.yarn.api.records.Resource getClusterResources()
           
 org.apache.hadoop.conf.Configuration getConf()
          Get the yarn configuration.
 CapacitySchedulerConfiguration getConfiguration()
           
 RMContainerTokenSecretManager getContainerTokenSecretManager()
           
 org.apache.hadoop.yarn.api.records.Resource getMaximumResourceCapability()
          Get maximum allocatable Resource.
 org.apache.hadoop.yarn.api.records.Resource getMinimumResourceCapability()
          Get minimum allocatable Resource.
 SchedulerNodeReport getNodeReport(org.apache.hadoop.yarn.api.records.NodeId nodeId)
          Get node resource usage report.
 int getNumClusterNodes()
          Get the number of nodes available in the cluster.
 Comparator<CSQueue> getQueueComparator()
           
 org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo(String queueName, boolean includeChildQueues, boolean recursive)
          Get queue information
 List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo> getQueueUserAclInfo()
          Get acls for queues for current user.
 org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
           
 RMContext getRMContext()
           
 CSQueue getRootQueue()
           
 QueueMetrics getRootQueueMetrics()
          Get the root queue for the scheduler.
 SchedulerAppReport getSchedulerAppInfo(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId)
          Get the Scheduler app for a given app attempt Id.
 void handle(SchedulerEvent event)
           
 void killContainer(RMContainer cont)
          Ask the scheduler to forcibly interrupt the container given as input
 void preemptContainer(org.apache.hadoop.yarn.api.records.ApplicationAttemptId aid, RMContainer cont)
          Ask the scheduler to obtain back the container from a specific application by issuing a preemption request
 void recover(RMStateStore.RMState state)
           
 void reinitialize(org.apache.hadoop.conf.Configuration conf, RMContext rmContext)
          Re-initialize the ResourceScheduler.
 void setConf(org.apache.hadoop.conf.Configuration conf)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

applications

protected Map<org.apache.hadoop.yarn.api.records.ApplicationAttemptId,FiCaSchedulerApp> applications

ROOT_QUEUE

@InterfaceAudience.Private
public static final String ROOT_QUEUE
See Also:
Constant Field Values
Constructor Detail

CapacityScheduler

public CapacityScheduler()
Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)
Specified by:
setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()
Description copied from interface: CapacitySchedulerContext
Get the yarn configuration.

Specified by:
getConf in interface org.apache.hadoop.conf.Configurable
Specified by:
getConf in interface CapacitySchedulerContext

getRootQueueMetrics

public QueueMetrics getRootQueueMetrics()
Description copied from interface: YarnScheduler
Get the root queue for the scheduler.

Specified by:
getRootQueueMetrics in interface YarnScheduler
Returns:
the root queue for the scheduler.

getRootQueue

public CSQueue getRootQueue()

getConfiguration

public CapacitySchedulerConfiguration getConfiguration()
Specified by:
getConfiguration in interface CapacitySchedulerContext

getContainerTokenSecretManager

public RMContainerTokenSecretManager getContainerTokenSecretManager()
Specified by:
getContainerTokenSecretManager in interface CapacitySchedulerContext

getMinimumResourceCapability

public org.apache.hadoop.yarn.api.records.Resource getMinimumResourceCapability()
Description copied from interface: YarnScheduler
Get minimum allocatable Resource.

Specified by:
getMinimumResourceCapability in interface CapacitySchedulerContext
Specified by:
getMinimumResourceCapability in interface YarnScheduler
Returns:
minimum allocatable resource

getMaximumResourceCapability

public org.apache.hadoop.yarn.api.records.Resource getMaximumResourceCapability()
Description copied from interface: YarnScheduler
Get maximum allocatable Resource.

Specified by:
getMaximumResourceCapability in interface CapacitySchedulerContext
Specified by:
getMaximumResourceCapability in interface YarnScheduler
Returns:
maximum allocatable resource

getApplicationComparator

public Comparator<FiCaSchedulerApp> getApplicationComparator()
Specified by:
getApplicationComparator in interface CapacitySchedulerContext

getResourceCalculator

public org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
Specified by:
getResourceCalculator in interface CapacitySchedulerContext

getQueueComparator

public Comparator<CSQueue> getQueueComparator()
Specified by:
getQueueComparator in interface CapacitySchedulerContext

getNumClusterNodes

public int getNumClusterNodes()
Description copied from interface: YarnScheduler
Get the number of nodes available in the cluster.

Specified by:
getNumClusterNodes in interface CapacitySchedulerContext
Specified by:
getNumClusterNodes in interface YarnScheduler
Returns:
the number of available nodes.

getRMContext

public RMContext getRMContext()
Specified by:
getRMContext in interface CapacitySchedulerContext

getClusterResources

public org.apache.hadoop.yarn.api.records.Resource getClusterResources()
Specified by:
getClusterResources in interface CapacitySchedulerContext

reinitialize

public void reinitialize(org.apache.hadoop.conf.Configuration conf,
                         RMContext rmContext)
                  throws IOException
Description copied from interface: ResourceScheduler
Re-initialize the ResourceScheduler.

Specified by:
reinitialize in interface ResourceScheduler
Parameters:
conf - configuration
Throws:
IOException

allocate

@Lock(value=org.apache.hadoop.yarn.server.utils.Lock.NoLock.class)
public Allocation allocate(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId,
                                                                                     List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask,
                                                                                     List<org.apache.hadoop.yarn.api.records.ContainerId> release,
                                                                                     List<String> blacklistAdditions,
                                                                                     List<String> blacklistRemovals)
Description copied from interface: YarnScheduler
The main api between the ApplicationMaster and the Scheduler. The ApplicationMaster is updating his future resource requirements and may release containers he doens't need.

Specified by:
allocate in interface YarnScheduler
Returns:
the Allocation for the application

getQueueInfo

@Lock(value=org.apache.hadoop.yarn.server.utils.Lock.NoLock.class)
public org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo(String queueName,
                                                                                                                           boolean includeChildQueues,
                                                                                                                           boolean recursive)
                                                          throws IOException
Description copied from interface: YarnScheduler
Get queue information

Specified by:
getQueueInfo in interface YarnScheduler
Parameters:
queueName - queue name
includeChildQueues - include child queues?
recursive - get children queues?
Returns:
queue information
Throws:
IOException

getQueueUserAclInfo

@Lock(value=org.apache.hadoop.yarn.server.utils.Lock.NoLock.class)
public List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo> getQueueUserAclInfo()
Description copied from interface: YarnScheduler
Get acls for queues for current user.

Specified by:
getQueueUserAclInfo in interface YarnScheduler
Returns:
acls for queues for current user

handle

public void handle(SchedulerEvent event)
Specified by:
handle in interface org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent>

getSchedulerAppInfo

public SchedulerAppReport getSchedulerAppInfo(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId)
Description copied from interface: YarnScheduler
Get the Scheduler app for a given app attempt Id.

Specified by:
getSchedulerAppInfo in interface YarnScheduler
Parameters:
applicationAttemptId - the id of the application attempt
Returns:
SchedulerApp for this given attempt.

recover

@Lock(value=org.apache.hadoop.yarn.server.utils.Lock.NoLock.class)
public void recover(RMStateStore.RMState state)
             throws Exception
Specified by:
recover in interface Recoverable
Throws:
Exception

getNodeReport

public SchedulerNodeReport getNodeReport(org.apache.hadoop.yarn.api.records.NodeId nodeId)
Description copied from interface: YarnScheduler
Get node resource usage report.

Specified by:
getNodeReport in interface YarnScheduler
Returns:
the SchedulerNodeReport for the node or null if nodeId does not point to a defined node.

dropContainerReservation

public void dropContainerReservation(RMContainer container)
Description copied from interface: PreemptableResourceScheduler
If the scheduler support container reservations, this method is used to ask the scheduler to drop the reservation for the given container.

Specified by:
dropContainerReservation in interface PreemptableResourceScheduler
Parameters:
container - Reference to reserved container allocation.

preemptContainer

public void preemptContainer(org.apache.hadoop.yarn.api.records.ApplicationAttemptId aid,
                             RMContainer cont)
Description copied from interface: PreemptableResourceScheduler
Ask the scheduler to obtain back the container from a specific application by issuing a preemption request

Specified by:
preemptContainer in interface PreemptableResourceScheduler
Parameters:
aid - the application from which we want to get a container back
cont - the container we want back

killContainer

public void killContainer(RMContainer cont)
Description copied from interface: PreemptableResourceScheduler
Ask the scheduler to forcibly interrupt the container given as input

Specified by:
killContainer in interface PreemptableResourceScheduler

checkAccess

public boolean checkAccess(org.apache.hadoop.security.UserGroupInformation callerUGI,
                           org.apache.hadoop.yarn.api.records.QueueACL acl,
                           String queueName)
Description copied from interface: YarnScheduler
Check if the user has permission to perform the operation. If the user has QueueACL.ADMINISTER_QUEUE permission, this user can view/modify the applications in this queue

Specified by:
checkAccess in interface YarnScheduler
Returns:
true if the user has the permission, false otherwise


Copyright © 2014 Apache Software Foundation. All Rights Reserved.