Package org.apache.hadoop.yarn.server.resourcemanager.reservation

Class CapacityOverTimePolicy

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.reservation.NoOverCommitPolicy

org.apache.hadoop.yarn.server.resourcemanager.reservation.CapacityOverTimePolicy

All Implemented Interfaces:

SharingPolicy

@LimitedPrivate("yarn")
@Unstable
public class CapacityOverTimePolicy
extends NoOverCommitPolicy

This policy enforces a time-extended notion of Capacity. In particular it guarantees that the allocation received in input when combined with all previous allocation for the user does not violate an instantaneous max limit on the resources received, and that for every window of time of length validWindow, the integral of the allocations for a user (sum of the currently submitted allocation and all prior allocations for the user) does not exceed validWindow * maxAvg. This allows flexibility, in the sense that an allocation can instantaneously use large portions of the available capacity, but prevents abuses by bounding the average use over time. By controlling maxInst, maxAvg, validWindow the administrator configuring this policy can obtain a behavior ranging from instantaneously enforced capacity (akin to existing queues), or fully flexible allocations (likely reserved to super-users, or trusted systems).

Constructor Summary

Constructors

Constructor	Description
CapacityOverTimePolicy()

Method Summary

Modifier and Type	Method	Description
RLESparseResourceAllocation	availableResources(RLESparseResourceAllocation available, Plan plan, java.lang.String user, org.apache.hadoop.yarn.api.records.ReservationId oldId, long start, long end)	This method provide a (partial) instantaneous validation by applying business rules (such as max number of parallel containers allowed for a user).
long	getValidWindow()	Returns the time range before and after the current reservation considered by this policy.
void	init(java.lang.String reservationQueuePath, ReservationSchedulerConfiguration conf)	Initialize this policy.
void	validate(Plan plan, ReservationAllocation reservation)	The validation algorithm walks over the RLE encoded allocation and checks that for all transition points (when the start or end of the checking window encounters a value in the RLE).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CapacityOverTimePolicy

public CapacityOverTimePolicy()

Method Detail

init

public void init(java.lang.String reservationQueuePath,
                 ReservationSchedulerConfiguration conf)

Description copied from interface: SharingPolicy

Initialize this policy.

Specified by:

init in interface SharingPolicy

Overrides:

init in class NoOverCommitPolicy

Parameters:

reservationQueuePath - the name of the queue for this plan

conf - the system configuration

validate

public void validate(Plan plan,
                     ReservationAllocation reservation)
              throws PlanningException

The validation algorithm walks over the RLE encoded allocation and checks that for all transition points (when the start or end of the checking window encounters a value in the RLE). At this point it checkes whether the integral computed exceeds the quota limit. Note that this might not find the exact time of a violation, but if a violation exists it will find it. The advantage is a much lower number of checks as compared to time-slot by time-slot checks.

Specified by:

validate in interface SharingPolicy

Overrides:

validate in class NoOverCommitPolicy

Parameters:

plan - the plan to validate against

reservation - the reservation allocation to test.

Throws:

PlanningException - if the validation fails.

availableResources

public RLESparseResourceAllocation availableResources(RLESparseResourceAllocation available,
                                                      Plan plan,
                                                      java.lang.String user,
                                                      org.apache.hadoop.yarn.api.records.ReservationId oldId,
                                                      long start,
                                                      long end)
                                               throws PlanningException

Description copied from interface: SharingPolicy

This method provide a (partial) instantaneous validation by applying business rules (such as max number of parallel containers allowed for a user). To provide the agent with more feedback the returned parameter is expressed in number of containers that can be fit in this time according to the business rules.

Specified by:

availableResources in interface SharingPolicy

Overrides:

availableResources in class NoOverCommitPolicy

Parameters:

available - the amount of resources that would be offered if not constrained by the policy

plan - reference the the current Plan

user - the username

oldId - (optional) the id of a reservation being updated

start - the start time for the range we are querying

end - the end time for the range we are querying

Returns:

the available resources expressed as a RLESparseResourceAllocation

Throws:

PlanningException - throws if the request is not valid

getValidWindow

public long getValidWindow()

Description copied from interface: SharingPolicy

Returns the time range before and after the current reservation considered by this policy. In particular, this informs the archival process for the Plan, i.e., reservations regarding times before (now - validWindow) can be deleted.

Specified by:

getValidWindow in interface SharingPolicy

Overrides:

getValidWindow in class NoOverCommitPolicy

Returns:

validWindow the window of validity considered by the policy.