Package org.apache.hadoop.yarn.server.resourcemanager.reservation

Class InMemoryPlan

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.reservation.InMemoryPlan

All Implemented Interfaces:

Plan, PlanContext, PlanEdit

public class InMemoryPlan
extends java.lang.Object
implements Plan

This class represents an in memory representation of the state of our reservation system, and provides accelerated access to both individual reservations and aggregate utilization of resources over time.

Constructor Summary

Constructors

Constructor	Description
InMemoryPlan(QueueMetrics queueMetrics, SharingPolicy policy, ReservationAgent agent, org.apache.hadoop.yarn.api.records.Resource totalCapacity, long step, org.apache.hadoop.yarn.util.resource.ResourceCalculator resCalc, org.apache.hadoop.yarn.api.records.Resource minAlloc, org.apache.hadoop.yarn.api.records.Resource maxAlloc, java.lang.String queueName, Planner replanner, boolean getMoveOnExpiry, long maxPeriodicity, RMContext rmContext)
InMemoryPlan(QueueMetrics queueMetrics, SharingPolicy policy, ReservationAgent agent, org.apache.hadoop.yarn.api.records.Resource totalCapacity, long step, org.apache.hadoop.yarn.util.resource.ResourceCalculator resCalc, org.apache.hadoop.yarn.api.records.Resource minAlloc, org.apache.hadoop.yarn.api.records.Resource maxAlloc, java.lang.String queueName, Planner replanner, boolean getMoveOnExpiry, long maxPeriodicty, RMContext rmContext, org.apache.hadoop.yarn.util.Clock clock)
InMemoryPlan(QueueMetrics queueMetrics, SharingPolicy policy, ReservationAgent agent, org.apache.hadoop.yarn.api.records.Resource totalCapacity, long step, org.apache.hadoop.yarn.util.resource.ResourceCalculator resCalc, org.apache.hadoop.yarn.api.records.Resource minAlloc, org.apache.hadoop.yarn.api.records.Resource maxAlloc, java.lang.String queueName, Planner replanner, boolean getMoveOnExpiry, RMContext rmContext)

Method Summary

Modifier and Type	Method	Description
boolean	addReservation(ReservationAllocation reservation, boolean isRecovering)	Add a new ReservationAllocation to the plan.
void	archiveCompletedReservations(long tick)	Method invoked to garbage collect old reservations.
boolean	deleteReservation(org.apache.hadoop.yarn.api.records.ReservationId reservationID)	Delete an existing ReservationAllocation from the plan identified uniquely by its ReservationId.
java.util.Set<ReservationAllocation>	getAllReservations()
RLESparseResourceAllocation	getAvailableResourceOverTime(java.lang.String user, org.apache.hadoop.yarn.api.records.ReservationId oldId, long start, long end, long period)
RLESparseResourceAllocation	getConsumptionForUserOverTime(java.lang.String user, long start, long end)
RLESparseResourceAllocation	getCumulativeLoadOverTime(long start, long end)
long	getEarliestStartTime()
long	getLastEndTime()
org.apache.hadoop.yarn.api.records.Resource	getMaximumAllocation()	Returns the single largest Resource allocation that can be reserved in this plan
long	getMaximumPeriodicity()	Returns the maximum periodicity allowed in a recurrence expression for reservations of a particular plan.
org.apache.hadoop.yarn.api.records.Resource	getMinimumAllocation()	Returns the single smallest Resource allocation that can be reserved in this plan
boolean	getMoveOnExpiry()	Instructs the PlanFollower on what to do for applications which are still running when the reservation is expiring (move-to-default vs kill)
QueueMetrics	getQueueMetrics()	Return the QueueMetrics for the queue in the ResourceScheduler corresponding to this plan
java.lang.String	getQueueName()	Return the name of the queue in the ResourceScheduler corresponding to this plan
Planner	getReplanner()	Return an instance of a Planner, which will be invoked in response to unexpected reduction in the resources of this plan
ReservationAgent	getReservationAgent()	Return the ReservationAgent configured for this plan that is responsible for optimally placing various reservation requests
ReservationAllocation	getReservationById(org.apache.hadoop.yarn.api.records.ReservationId reservationID)
java.util.Set<ReservationAllocation>	getReservationByUserAtTime(java.lang.String user, long t)
RLESparseResourceAllocation	getReservationCountForUserOverTime(java.lang.String user, long start, long end)
java.util.Set<ReservationAllocation>	getReservations(org.apache.hadoop.yarn.api.records.ReservationId reservationID, ReservationInterval interval)
java.util.Set<ReservationAllocation>	getReservations(org.apache.hadoop.yarn.api.records.ReservationId reservationID, ReservationInterval interval, java.lang.String user)
java.util.Set<ReservationAllocation>	getReservationsAtTime(long tick)
org.apache.hadoop.yarn.util.resource.ResourceCalculator	getResourceCalculator()	Returns the system ResourceCalculator
SharingPolicy	getSharingPolicy()	Return the configured SharingPolicy that governs the sharing of the resources of the plan between its various users
long	getStep()	Returns the configured "step" or granularity of time of the plan in millis.
org.apache.hadoop.yarn.api.records.Resource	getTotalCapacity()
org.apache.hadoop.yarn.api.records.Resource	getTotalCommittedResources(long t)
void	setTotalCapacity(org.apache.hadoop.yarn.api.records.Resource cap)	Sets the overall capacity in terms of Resource assigned to this plan.
java.lang.String	toCumulativeString()
java.lang.String	toString()
boolean	updateReservation(ReservationAllocation reservation)	Updates an existing ReservationAllocation in the plan.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

InMemoryPlan

public InMemoryPlan(QueueMetrics queueMetrics,
                    SharingPolicy policy,
                    ReservationAgent agent,
                    org.apache.hadoop.yarn.api.records.Resource totalCapacity,
                    long step,
                    org.apache.hadoop.yarn.util.resource.ResourceCalculator resCalc,
                    org.apache.hadoop.yarn.api.records.Resource minAlloc,
                    org.apache.hadoop.yarn.api.records.Resource maxAlloc,
                    java.lang.String queueName,
                    Planner replanner,
                    boolean getMoveOnExpiry,
                    RMContext rmContext)

InMemoryPlan

public InMemoryPlan(QueueMetrics queueMetrics,
                    SharingPolicy policy,
                    ReservationAgent agent,
                    org.apache.hadoop.yarn.api.records.Resource totalCapacity,
                    long step,
                    org.apache.hadoop.yarn.util.resource.ResourceCalculator resCalc,
                    org.apache.hadoop.yarn.api.records.Resource minAlloc,
                    org.apache.hadoop.yarn.api.records.Resource maxAlloc,
                    java.lang.String queueName,
                    Planner replanner,
                    boolean getMoveOnExpiry,
                    long maxPeriodicity,
                    RMContext rmContext)

InMemoryPlan

public InMemoryPlan(QueueMetrics queueMetrics,
                    SharingPolicy policy,
                    ReservationAgent agent,
                    org.apache.hadoop.yarn.api.records.Resource totalCapacity,
                    long step,
                    org.apache.hadoop.yarn.util.resource.ResourceCalculator resCalc,
                    org.apache.hadoop.yarn.api.records.Resource minAlloc,
                    org.apache.hadoop.yarn.api.records.Resource maxAlloc,
                    java.lang.String queueName,
                    Planner replanner,
                    boolean getMoveOnExpiry,
                    long maxPeriodicty,
                    RMContext rmContext,
                    org.apache.hadoop.yarn.util.Clock clock)

Method Detail

getQueueMetrics

public QueueMetrics getQueueMetrics()

Description copied from interface: PlanContext

Return the QueueMetrics for the queue in the ResourceScheduler corresponding to this plan

Specified by:

getQueueMetrics in interface PlanContext

Returns:

the QueueMetrics for the queue in the ResourceScheduler corresponding to this plan

getAllReservations

public java.util.Set<ReservationAllocation> getAllReservations()

addReservation

public boolean addReservation(ReservationAllocation reservation,
                              boolean isRecovering)
                       throws PlanningException

Description copied from interface: PlanEdit

Add a new ReservationAllocation to the plan.

Specified by:

addReservation in interface PlanEdit

Parameters:

reservation - the ReservationAllocation to be added to the plan

isRecovering - flag to indicate if reservation is being added as part of failover or not

Returns:

true if addition is successful, false otherwise

Throws:

PlanningException - if addition is unsuccessful

updateReservation

public boolean updateReservation(ReservationAllocation reservation)
                          throws PlanningException

Description copied from interface: PlanEdit

Updates an existing ReservationAllocation in the plan. This is required for re-negotiation.

Specified by:

updateReservation in interface PlanEdit

Parameters:

reservation - the ReservationAllocation to be updated the plan

Returns:

true if update is successful, false otherwise

Throws:

PlanningException - if update is unsuccessful

deleteReservation

public boolean deleteReservation(org.apache.hadoop.yarn.api.records.ReservationId reservationID)

Description copied from interface: PlanEdit

Delete an existing ReservationAllocation from the plan identified uniquely by its ReservationId. This will generally be used for garbage collection.

Specified by:

deleteReservation in interface PlanEdit

Parameters:

reservationID - the ReservationAllocation to be deleted from the plan identified uniquely by its ReservationId

Returns:

true if delete is successful, false otherwise

archiveCompletedReservations

public void archiveCompletedReservations(long tick)

Description copied from interface: PlanEdit

Method invoked to garbage collect old reservations. It cleans up expired reservations that have fallen out of the sliding archival window.

Specified by:

archiveCompletedReservations in interface PlanEdit

Parameters:

tick - the current time from which the archival window is computed

getReservationsAtTime

public java.util.Set<ReservationAllocation> getReservationsAtTime(long tick)

getStep

public long getStep()

Description copied from interface: PlanContext

Returns the configured "step" or granularity of time of the plan in millis.

Specified by:

getStep in interface PlanContext

Returns:

plan step in millis

getSharingPolicy

public SharingPolicy getSharingPolicy()

Description copied from interface: PlanContext

Return the configured SharingPolicy that governs the sharing of the resources of the plan between its various users

Specified by:

getSharingPolicy in interface PlanContext

Returns:

the configured SharingPolicy that governs the sharing of the resources of the plan between its various users

getReservationAgent

public ReservationAgent getReservationAgent()

Description copied from interface: PlanContext

Return the ReservationAgent configured for this plan that is responsible for optimally placing various reservation requests

Specified by:

getReservationAgent in interface PlanContext

Returns:

the ReservationAgent configured for this plan

getReservationCountForUserOverTime

public RLESparseResourceAllocation getReservationCountForUserOverTime(java.lang.String user,
                                                                      long start,
                                                                      long end)

getConsumptionForUserOverTime

public RLESparseResourceAllocation getConsumptionForUserOverTime(java.lang.String user,
                                                                 long start,
                                                                 long end)

getTotalCommittedResources

public org.apache.hadoop.yarn.api.records.Resource getTotalCommittedResources(long t)

getReservations

public java.util.Set<ReservationAllocation> getReservations(org.apache.hadoop.yarn.api.records.ReservationId reservationID,
                                                            ReservationInterval interval)

getReservations

public java.util.Set<ReservationAllocation> getReservations(org.apache.hadoop.yarn.api.records.ReservationId reservationID,
                                                            ReservationInterval interval,
                                                            java.lang.String user)

getReservationById

public ReservationAllocation getReservationById(org.apache.hadoop.yarn.api.records.ReservationId reservationID)

getTotalCapacity

public org.apache.hadoop.yarn.api.records.Resource getTotalCapacity()

getAvailableResourceOverTime

public RLESparseResourceAllocation getAvailableResourceOverTime(java.lang.String user,
                                                                org.apache.hadoop.yarn.api.records.ReservationId oldId,
                                                                long start,
                                                                long end,
                                                                long period)
                                                         throws PlanningException

Throws:

PlanningException

getMinimumAllocation

public org.apache.hadoop.yarn.api.records.Resource getMinimumAllocation()

Description copied from interface: PlanContext

Returns the single smallest Resource allocation that can be reserved in this plan

Specified by:

getMinimumAllocation in interface PlanContext

Returns:

the single smallest Resource allocation that can be reserved in this plan

setTotalCapacity

public void setTotalCapacity(org.apache.hadoop.yarn.api.records.Resource cap)

Description copied from interface: PlanEdit

Sets the overall capacity in terms of Resource assigned to this plan.

Specified by:

setTotalCapacity in interface PlanEdit

Parameters:

cap - the overall capacity in terms of Resource assigned to this plan

getEarliestStartTime

public long getEarliestStartTime()

getLastEndTime

public long getLastEndTime()

getResourceCalculator

public org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()

Description copied from interface: PlanContext

Returns the system ResourceCalculator

Specified by:

getResourceCalculator in interface PlanContext

Returns:

the system ResourceCalculator

getQueueName

public java.lang.String getQueueName()

Description copied from interface: PlanContext

Return the name of the queue in the ResourceScheduler corresponding to this plan

Specified by:

getQueueName in interface PlanContext

Returns:

the name of the queue in the ResourceScheduler corresponding to this plan

getMaximumAllocation

public org.apache.hadoop.yarn.api.records.Resource getMaximumAllocation()

Description copied from interface: PlanContext

Returns the single largest Resource allocation that can be reserved in this plan

Specified by:

getMaximumAllocation in interface PlanContext

Returns:

the single largest Resource allocation that can be reserved in this plan

getMaximumPeriodicity

public long getMaximumPeriodicity()

Description copied from interface: PlanContext

Returns the maximum periodicity allowed in a recurrence expression for reservations of a particular plan. This value must be divisible by the recurrence expression of a newly submitted reservation. Otherwise, the reservation submission will fail.

Specified by:

getMaximumPeriodicity in interface PlanContext

Returns:

the maximum periodicity allowed in a recurrence expression for reservations of a particular plan.

toCumulativeString

public java.lang.String toCumulativeString()

getReplanner

public Planner getReplanner()

Description copied from interface: PlanContext

Return an instance of a Planner, which will be invoked in response to unexpected reduction in the resources of this plan

Specified by:

getReplanner in interface PlanContext

Returns:

an instance of a Planner, which will be invoked in response to unexpected reduction in the resources of this plan

getMoveOnExpiry

public boolean getMoveOnExpiry()

Description copied from interface: PlanContext

Instructs the PlanFollower on what to do for applications which are still running when the reservation is expiring (move-to-default vs kill)

Specified by:

getMoveOnExpiry in interface PlanContext

Returns:

true if remaining applications have to be killed, false if they have to migrated

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

getReservationByUserAtTime

public java.util.Set<ReservationAllocation> getReservationByUserAtTime(java.lang.String user,
                                                                       long t)

getCumulativeLoadOverTime

public RLESparseResourceAllocation getCumulativeLoadOverTime(long start,
                                                             long end)
                                                      throws PlanningException

Throws:

PlanningException