org.apache.hadoop.yarn.server.resourcemanager.scheduler

Class SchedulerUtils

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class SchedulerUtils
extends Object

Utilities shared by schedulers.

Field Summary

Fields

Modifier and Type	Field and Description
static String	COMPLETED_APPLICATION
static String	EXPIRED_CONTAINER
static String	LOST_CONTAINER
static String	PREEMPTED_CONTAINER
static String	RELEASED_CONTAINER
static String	UNRESERVED_CONTAINER

Constructor Summary

Constructors

Constructor and Description
SchedulerUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static void	checkIfLabelInClusterNodeLabels(RMNodeLabelsManager mgr, Set<String> labels)
static boolean	checkNodeLabelExpression(Set<String> nodeLabels, String labelExpression)
static boolean	checkQueueAccessToNode(Set<String> queueLabels, Set<String> nodeLabels)
static boolean	checkQueueLabelExpression(Set<String> queueLabels, String labelExpression)
static org.apache.hadoop.yarn.api.records.ContainerStatus	createAbnormalContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId, String diagnostics) Utility to create a ContainerStatus during exceptional circumstances.
static org.apache.hadoop.yarn.api.records.ContainerStatus	createPreemptedContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId, String diagnostics) Utility to create a ContainerStatus during exceptional circumstances.
static void	normalizeAndvalidateRequest(org.apache.hadoop.yarn.api.records.ResourceRequest resReq, org.apache.hadoop.yarn.api.records.Resource maximumResource, String queueName, YarnScheduler scheduler)
static void	normalizeAndValidateRequest(org.apache.hadoop.yarn.api.records.ResourceRequest resReq, org.apache.hadoop.yarn.api.records.Resource maximumResource, String queueName, YarnScheduler scheduler, boolean isRecovery)
static void	normalizeRequest(org.apache.hadoop.yarn.api.records.ResourceRequest ask, org.apache.hadoop.yarn.util.resource.ResourceCalculator resourceCalculator, org.apache.hadoop.yarn.api.records.Resource clusterResource, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource) Utility method to normalize a resource request, by insuring that the requested memory is a multiple of minMemory and is not zero.
static void	normalizeRequest(org.apache.hadoop.yarn.api.records.ResourceRequest ask, org.apache.hadoop.yarn.util.resource.ResourceCalculator resourceCalculator, org.apache.hadoop.yarn.api.records.Resource clusterResource, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.Resource incrementResource) Utility method to normalize a resource request, by insuring that the requested memory is a multiple of minMemory and is not zero.
static void	normalizeRequests(List<org.apache.hadoop.yarn.api.records.ResourceRequest> asks, org.apache.hadoop.yarn.util.resource.ResourceCalculator resourceCalculator, org.apache.hadoop.yarn.api.records.Resource clusterResource, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource) Utility method to normalize a list of resource requests, by insuring that the memory for each request is a multiple of minMemory and is not zero.
static void	normalizeRequests(List<org.apache.hadoop.yarn.api.records.ResourceRequest> asks, org.apache.hadoop.yarn.util.resource.ResourceCalculator resourceCalculator, org.apache.hadoop.yarn.api.records.Resource clusterResource, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.Resource incrementResource) Utility method to normalize a list of resource requests, by insuring that the memory for each request is a multiple of minMemory and is not zero.
static org.apache.hadoop.yarn.security.AccessType	toAccessType(org.apache.hadoop.yarn.api.records.QueueACL acl)
static void	validateResourceRequest(org.apache.hadoop.yarn.api.records.ResourceRequest resReq, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.QueueInfo queueInfo) Utility method to validate a resource request, by insuring that the requested memory/vcore/disk is non-negative and not greater than max

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

RELEASED_CONTAINER

public static final String RELEASED_CONTAINER

See Also:

Constant Field Values

LOST_CONTAINER

public static final String LOST_CONTAINER

See Also:

Constant Field Values

PREEMPTED_CONTAINER

public static final String PREEMPTED_CONTAINER

See Also:

Constant Field Values

COMPLETED_APPLICATION

public static final String COMPLETED_APPLICATION

See Also:

Constant Field Values

EXPIRED_CONTAINER

public static final String EXPIRED_CONTAINER

See Also:

Constant Field Values

UNRESERVED_CONTAINER

public static final String UNRESERVED_CONTAINER

See Also:

Constant Field Values

Constructor Detail

SchedulerUtils

public SchedulerUtils()

Method Detail

createAbnormalContainerStatus

public static org.apache.hadoop.yarn.api.records.ContainerStatus createAbnormalContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                                                                               String diagnostics)

Utility to create a ContainerStatus during exceptional circumstances.

Parameters:

containerId - ContainerId of returned/released/lost container.

diagnostics - diagnostic message

Returns:

ContainerStatus for an returned/released/lost container

createPreemptedContainerStatus

public static org.apache.hadoop.yarn.api.records.ContainerStatus createPreemptedContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                                                                                String diagnostics)

Utility to create a ContainerStatus during exceptional circumstances.

Parameters:

containerId - ContainerId of returned/released/lost container.

diagnostics - diagnostic message

Returns:

ContainerStatus for an returned/released/lost container

normalizeRequests

public static void normalizeRequests(List<org.apache.hadoop.yarn.api.records.ResourceRequest> asks,
                     org.apache.hadoop.yarn.util.resource.ResourceCalculator resourceCalculator,
                     org.apache.hadoop.yarn.api.records.Resource clusterResource,
                     org.apache.hadoop.yarn.api.records.Resource minimumResource,
                     org.apache.hadoop.yarn.api.records.Resource maximumResource)

Utility method to normalize a list of resource requests, by insuring that the memory for each request is a multiple of minMemory and is not zero.

normalizeRequest

public static void normalizeRequest(org.apache.hadoop.yarn.api.records.ResourceRequest ask,
                    org.apache.hadoop.yarn.util.resource.ResourceCalculator resourceCalculator,
                    org.apache.hadoop.yarn.api.records.Resource clusterResource,
                    org.apache.hadoop.yarn.api.records.Resource minimumResource,
                    org.apache.hadoop.yarn.api.records.Resource maximumResource)

Utility method to normalize a resource request, by insuring that the requested memory is a multiple of minMemory and is not zero.

normalizeRequests

public static void normalizeRequests(List<org.apache.hadoop.yarn.api.records.ResourceRequest> asks,
                     org.apache.hadoop.yarn.util.resource.ResourceCalculator resourceCalculator,
                     org.apache.hadoop.yarn.api.records.Resource clusterResource,
                     org.apache.hadoop.yarn.api.records.Resource minimumResource,
                     org.apache.hadoop.yarn.api.records.Resource maximumResource,
                     org.apache.hadoop.yarn.api.records.Resource incrementResource)

Utility method to normalize a list of resource requests, by insuring that the memory for each request is a multiple of minMemory and is not zero.

normalizeRequest

public static void normalizeRequest(org.apache.hadoop.yarn.api.records.ResourceRequest ask,
                    org.apache.hadoop.yarn.util.resource.ResourceCalculator resourceCalculator,
                    org.apache.hadoop.yarn.api.records.Resource clusterResource,
                    org.apache.hadoop.yarn.api.records.Resource minimumResource,
                    org.apache.hadoop.yarn.api.records.Resource maximumResource,
                    org.apache.hadoop.yarn.api.records.Resource incrementResource)

Utility method to normalize a resource request, by insuring that the requested memory is a multiple of minMemory and is not zero.

normalizeAndValidateRequest

public static void normalizeAndValidateRequest(org.apache.hadoop.yarn.api.records.ResourceRequest resReq,
                               org.apache.hadoop.yarn.api.records.Resource maximumResource,
                               String queueName,
                               YarnScheduler scheduler,
                               boolean isRecovery)
                                        throws org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

Throws:

org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

normalizeAndvalidateRequest

public static void normalizeAndvalidateRequest(org.apache.hadoop.yarn.api.records.ResourceRequest resReq,
                               org.apache.hadoop.yarn.api.records.Resource maximumResource,
                               String queueName,
                               YarnScheduler scheduler)
                                        throws org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

Throws:

org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

validateResourceRequest

public static void validateResourceRequest(org.apache.hadoop.yarn.api.records.ResourceRequest resReq,
                           org.apache.hadoop.yarn.api.records.Resource maximumResource,
                           org.apache.hadoop.yarn.api.records.QueueInfo queueInfo)
                                    throws org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException

Utility method to validate a resource request, by insuring that the requested memory/vcore/disk is non-negative and not greater than max

Throws:

org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException - when there is invalid request

checkQueueAccessToNode

public static boolean checkQueueAccessToNode(Set<String> queueLabels,
                             Set<String> nodeLabels)

checkIfLabelInClusterNodeLabels

public static void checkIfLabelInClusterNodeLabels(RMNodeLabelsManager mgr,
                                   Set<String> labels)
                                            throws IOException

Throws:

IOException

checkNodeLabelExpression

public static boolean checkNodeLabelExpression(Set<String> nodeLabels,
                               String labelExpression)

checkQueueLabelExpression

public static boolean checkQueueLabelExpression(Set<String> queueLabels,
                                String labelExpression)

toAccessType

public static org.apache.hadoop.yarn.security.AccessType toAccessType(org.apache.hadoop.yarn.api.records.QueueACL acl)