Package org.apache.hadoop.yarn.server.scheduler

Class OpportunisticContainerAllocator

java.lang.Object

org.apache.hadoop.yarn.server.scheduler.OpportunisticContainerAllocator

Direct Known Subclasses:

DistributedOpportunisticContainerAllocator

public abstract class OpportunisticContainerAllocator
extends java.lang.Object

Base abstract class for Opportunistic container allocations, that provides common functions required for Opportunistic container allocation.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	OpportunisticContainerAllocator.Allocation	This class encapsulates container and resourceName for an allocation.
static class	OpportunisticContainerAllocator.AllocationParams	This class encapsulates application specific parameters used to build a Container.
static class	OpportunisticContainerAllocator.ContainerIdGenerator	A Container Id Generator.
static class	OpportunisticContainerAllocator.EnrichedResourceRequest	This class encapsulates Resource Request and provides requests per node and rack.
static class	OpportunisticContainerAllocator.PartitionedResourceRequests	Class that includes two lists of ResourceRequests: one for GUARANTEED and one for OPPORTUNISTIC ResourceRequests.

Constructor Summary

Constructors

Constructor	Description
OpportunisticContainerAllocator(BaseContainerTokenSecretManager tokenSecretManager)	Create a new Opportunistic Container Allocator.
OpportunisticContainerAllocator(BaseContainerTokenSecretManager tokenSecretManager, int maxAllocationsPerAMHeartbeat)	Create a new Opportunistic Container Allocator.

Method Summary

Modifier and Type	Method	Description
abstract java.util.List<org.apache.hadoop.yarn.api.records.Container>	allocateContainers(org.apache.hadoop.yarn.api.records.ResourceBlacklistRequest blackList, java.util.List<org.apache.hadoop.yarn.api.records.ResourceRequest> oppResourceReqs, org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, OpportunisticContainerContext opportContext, long rmIdentifier, java.lang.String appSubmitter)	Allocate OPPORTUNISTIC containers.
protected org.apache.hadoop.yarn.api.records.Container	createContainer(long rmIdentifier, OpportunisticContainerAllocator.AllocationParams appParams, OpportunisticContainerAllocator.ContainerIdGenerator idCounter, org.apache.hadoop.yarn.api.records.ApplicationAttemptId id, java.lang.String userName, java.util.Map<org.apache.hadoop.yarn.api.records.Resource,java.util.List<OpportunisticContainerAllocator.Allocation>> allocations, java.lang.String location, org.apache.hadoop.yarn.api.records.ResourceRequest anyAsk, RemoteNode rNode)
int	getMaxAllocationsPerAMHeartbeat()	Get the Max Allocations per AM heartbeat.
protected java.lang.String	getRemoteNodePartition(RemoteNode node)
protected java.lang.String	getRequestPartition(OpportunisticContainerAllocator.EnrichedResourceRequest enrichedRR)
protected int	getTotalAllocations(java.util.List<java.util.Map<org.apache.hadoop.yarn.api.records.Resource,java.util.List<OpportunisticContainerAllocator.Allocation>>> allocations)
protected void	matchAllocation(java.util.List<java.util.Map<org.apache.hadoop.yarn.api.records.Resource,java.util.List<OpportunisticContainerAllocator.Allocation>>> allocations, java.util.List<org.apache.hadoop.yarn.api.records.Container> allocatedContainers, OpportunisticContainerContext oppContext)
OpportunisticContainerAllocator.PartitionedResourceRequests	partitionAskList(java.util.List<org.apache.hadoop.yarn.api.records.ResourceRequest> askList)	Partitions a list of ResourceRequest to two separate lists, one for GUARANTEED and one for OPPORTUNISTIC ResourceRequests.
void	setMaxAllocationsPerAMHeartbeat(int maxAllocationsPerAMHeartbeat)
protected void	updateBlacklist(org.apache.hadoop.yarn.api.records.ResourceBlacklistRequest blackList, OpportunisticContainerContext oppContext)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OpportunisticContainerAllocator

public OpportunisticContainerAllocator(BaseContainerTokenSecretManager tokenSecretManager)

Create a new Opportunistic Container Allocator.

Parameters:

tokenSecretManager - TokenSecretManager

OpportunisticContainerAllocator

public OpportunisticContainerAllocator(BaseContainerTokenSecretManager tokenSecretManager,
                                       int maxAllocationsPerAMHeartbeat)

Create a new Opportunistic Container Allocator.

Parameters:

tokenSecretManager - TokenSecretManager

maxAllocationsPerAMHeartbeat - max number of containers to be allocated in one AM heartbeat

Method Detail

setMaxAllocationsPerAMHeartbeat

public void setMaxAllocationsPerAMHeartbeat(int maxAllocationsPerAMHeartbeat)

getMaxAllocationsPerAMHeartbeat

public int getMaxAllocationsPerAMHeartbeat()

Get the Max Allocations per AM heartbeat.

Returns:

maxAllocationsPerAMHeartbeat.

allocateContainers

public abstract java.util.List<org.apache.hadoop.yarn.api.records.Container> allocateContainers(org.apache.hadoop.yarn.api.records.ResourceBlacklistRequest blackList,
                                                                                                java.util.List<org.apache.hadoop.yarn.api.records.ResourceRequest> oppResourceReqs,
                                                                                                org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId,
                                                                                                OpportunisticContainerContext opportContext,
                                                                                                long rmIdentifier,
                                                                                                java.lang.String appSubmitter)
                                                                                         throws org.apache.hadoop.yarn.exceptions.YarnException

Allocate OPPORTUNISTIC containers.

Parameters:

blackList - Resource BlackList Request

oppResourceReqs - Opportunistic Resource Requests

applicationAttemptId - ApplicationAttemptId

opportContext - App specific OpportunisticContainerContext

rmIdentifier - RM Identifier

appSubmitter - App Submitter

Returns:

List of Containers.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - YarnException

updateBlacklist

protected void updateBlacklist(org.apache.hadoop.yarn.api.records.ResourceBlacklistRequest blackList,
                               OpportunisticContainerContext oppContext)

matchAllocation

protected void matchAllocation(java.util.List<java.util.Map<org.apache.hadoop.yarn.api.records.Resource,java.util.List<OpportunisticContainerAllocator.Allocation>>> allocations,
                               java.util.List<org.apache.hadoop.yarn.api.records.Container> allocatedContainers,
                               OpportunisticContainerContext oppContext)

getTotalAllocations

protected int getTotalAllocations(java.util.List<java.util.Map<org.apache.hadoop.yarn.api.records.Resource,java.util.List<OpportunisticContainerAllocator.Allocation>>> allocations)

createContainer

protected org.apache.hadoop.yarn.api.records.Container createContainer(long rmIdentifier,
                                                                       OpportunisticContainerAllocator.AllocationParams appParams,
                                                                       OpportunisticContainerAllocator.ContainerIdGenerator idCounter,
                                                                       org.apache.hadoop.yarn.api.records.ApplicationAttemptId id,
                                                                       java.lang.String userName,
                                                                       java.util.Map<org.apache.hadoop.yarn.api.records.Resource,java.util.List<OpportunisticContainerAllocator.Allocation>> allocations,
                                                                       java.lang.String location,
                                                                       org.apache.hadoop.yarn.api.records.ResourceRequest anyAsk,
                                                                       RemoteNode rNode)
                                                                throws org.apache.hadoop.yarn.exceptions.YarnException

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

partitionAskList

public OpportunisticContainerAllocator.PartitionedResourceRequests partitionAskList(java.util.List<org.apache.hadoop.yarn.api.records.ResourceRequest> askList)

Partitions a list of ResourceRequest to two separate lists, one for GUARANTEED and one for OPPORTUNISTIC ResourceRequests.

Parameters:

askList - the list of ResourceRequests to be partitioned

Returns:

the partitioned ResourceRequests

getRequestPartition

protected java.lang.String getRequestPartition(OpportunisticContainerAllocator.EnrichedResourceRequest enrichedRR)

getRemoteNodePartition

protected java.lang.String getRemoteNodePartition(RemoteNode node)