Package org.apache.hadoop.yarn.api.records

Class Resource

java.lang.Object

org.apache.hadoop.yarn.api.records.Resource

All Implemented Interfaces:

java.lang.Comparable<Resource>

Direct Known Subclasses:

LightWeightResource

@Public
@Stable
public abstract class Resource
extends java.lang.Object
implements java.lang.Comparable<Resource>

Resource models a set of computer resources in the cluster.

Currently it models both memory and CPU.

The unit for memory is megabytes. CPU is modeled with virtual cores (vcores), a unit for expressing parallelism. A node's capacity should be configured with virtual cores equal to its number of physical cores. A container should be requested with the number of cores it can saturate, i.e. the average number of threads it expects to have runnable at a time.

Virtual cores take integer values and thus currently CPU-scheduling is very coarse. A complementary axis for CPU requests that represents processing power will likely be added in the future to enable finer-grained resource configuration.

Typically, applications request Resource of suitable capability to run their component tasks.

See Also:

ResourceRequest, ApplicationMasterProtocol.allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest)

Field Summary

Fields

Modifier and Type	Field	Description
static int	MEMORY_INDEX
protected ResourceInformation[]	resources
static int	VCORES_INDEX

Constructor Summary

Constructors

Constructor	Description
Resource()

Method Summary

Modifier and Type	Method	Description
protected static int	castToIntSafely(long value)	Convert long to int for a resource value safely.
int	compareTo(Resource other)
static void	copy(Resource source, Resource dest)
boolean	equals(java.lang.Object obj)
java.util.List<ResourceInformation>	getAllResourcesListCopy()	Get list of resource information, this will be used by JAXB.
double	getDisks()	Get number of disks of the resource.
java.lang.String	getFormattedString()	This method is to get memory in terms of KB|MB|GB.
java.lang.String	getFormattedString(long memory)
abstract int	getMemory()	Deprecated.
long	getMemorySize()	Get memory of the resource.
ResourceInformation	getResourceInformation(int index)	Get ResourceInformation for a specified resource from a given index.
ResourceInformation	getResourceInformation(java.lang.String resource)	Get ResourceInformation for a specified resource.
ResourceInformation[]	getResources()	Get ResourceInformation for all resources.
long	getResourceValue(java.lang.String resource)	Get the value for a specified resource.
abstract int	getVirtualCores()	Get number of virtual cpu cores of the resource.
int	hashCode()
protected static ResourceInformation	newDefaultInformation(java.lang.String name, java.lang.String unit, long value)	Create ResourceInformation with basic fields.
static Resource	newInstance(int memory, int vCores)
static Resource	newInstance(long memory, int vCores)
static Resource	newInstance(long memory, int vCores, java.util.Map<java.lang.String,java.lang.Long> others)	Create a new Resource instance with the given CPU and memory values and additional resource values as set in the others parameter.
static Resource	newInstance(Resource resource)
void	setDisks(double disks)	Set number of disks of the resource.
abstract void	setMemory(int memory)	Deprecated.
void	setMemorySize(long memory)	Set memory of the resource.
void	setResourceInformation(int index, ResourceInformation resourceInformation)	Set the ResourceInformation object for a particular resource.
void	setResourceInformation(java.lang.String resource, ResourceInformation resourceInformation)	Set the ResourceInformation object for a particular resource.
protected void	setResources(ResourceInformation[] resources)
void	setResourceValue(int index, long value)	Set the value of a resource in the ResourceInformation object.
void	setResourceValue(java.lang.String resource, long value)	Set the value of a resource in the ResourceInformation object.
abstract void	setVirtualCores(int vCores)	Set number of virtual cpu cores of the resource.
protected void	throwExceptionWhenArrayOutOfBound(int index)
java.lang.String	toFormattedString()
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

resources

protected ResourceInformation[] resources

MEMORY_INDEX

@Private
public static final int MEMORY_INDEX

See Also:

Constant Field Values

VCORES_INDEX

@Private
public static final int VCORES_INDEX

See Also:

Constant Field Values

Constructor Detail

Resource

public Resource()

Method Detail

newInstance

@Public
@Stable
public static Resource newInstance(int memory,
                                   int vCores)

newInstance

@Public
@Stable
public static Resource newInstance(long memory,
                                   int vCores)

newInstance

@Public
@Stable
public static Resource newInstance(long memory,
                                   int vCores,
                                   java.util.Map<java.lang.String,java.lang.Long> others)

Create a new Resource instance with the given CPU and memory values and additional resource values as set in the others parameter. Note that the CPU and memory settings in the others parameter will be ignored.

Parameters:

memory - the memory value

vCores - the CPU value

others - a map of other resource values indexed by resource name

Returns:

a Resource instance with the given resource values

newInstance

@Private
@Unstable
public static Resource newInstance(Resource resource)

copy

@Private
@Unstable
public static void copy(Resource source,
                        Resource dest)

getMemory

@Public
@Deprecated
public abstract int getMemory()

Deprecated.

This method is DEPRECATED: Use getMemorySize() instead Get memory of the resource. Note - while memory has never had a unit specified, all YARN configurations have specified memory in MB. The assumption has been that the daemons and applications are always using the same units. With the introduction of the ResourceInformation class we have support for units - so this function will continue to return memory but in the units of MB

Returns:

memory(in MB) of the resource

getMemorySize

@Public
@Stable
public long getMemorySize()

Get memory of the resource. Note - while memory has never had a unit specified, all YARN configurations have specified memory in MB. The assumption has been that the daemons and applications are always using the same units. With the introduction of the ResourceInformation class we have support for units - so this function will continue to return memory but in the units of MB

Returns:

memory of the resource

setMemory

@Public
@Deprecated
public abstract void setMemory(int memory)

Deprecated.

Set memory of the resource. Note - while memory has never had a unit specified, all YARN configurations have specified memory in MB. The assumption has been that the daemons and applications are always using the same units. With the introduction of the ResourceInformation class we have support for units - so this function will continue to set memory but the assumption is that the value passed is in units of MB.

Parameters:

memory - memory(in MB) of the resource

setMemorySize

@Public
@Stable
public void setMemorySize(long memory)

Set memory of the resource.

Parameters:

memory - memory of the resource

getVirtualCores

@Public
@Evolving
public abstract int getVirtualCores()

Get number of virtual cpu cores of the resource. Virtual cores are a unit for expressing CPU parallelism. A node's capacity should be configured with virtual cores equal to its number of physical cores. A container should be requested with the number of cores it can saturate, i.e. the average number of threads it expects to have runnable at a time.

Returns:

num of virtual cpu cores of the resource

setVirtualCores

@Public
@Evolving
public abstract void setVirtualCores(int vCores)

Set number of virtual cpu cores of the resource. Virtual cores are a unit for expressing CPU parallelism. A node's capacity should be configured with virtual cores equal to its number of physical cores. A container should be requested with the number of cores it can saturate, i.e. the average number of threads it expects to have runnable at a time.

Parameters:

vCores - number of virtual cpu cores of the resource

getResources

@Private
@Unstable
public ResourceInformation[] getResources()

Get ResourceInformation for all resources.

Returns:

Map of resource name to ResourceInformation

getAllResourcesListCopy

@Private
@Unstable
public java.util.List<ResourceInformation> getAllResourcesListCopy()

Get list of resource information, this will be used by JAXB.

Returns:

list of resources copy.

getResourceInformation

@Public
@Unstable
public ResourceInformation getResourceInformation(java.lang.String resource)

Get ResourceInformation for a specified resource.

Parameters:

resource - name of the resource

Returns:

the ResourceInformation object for the resource

getResourceInformation

@Private
@Unstable
public ResourceInformation getResourceInformation(int index)
                                           throws ResourceNotFoundException

Get ResourceInformation for a specified resource from a given index.

Parameters:

index - of the resource

Returns:

the ResourceInformation object for the resource

Throws:

ResourceNotFoundException - if the resource can't be found

getResourceValue

@Public
@Unstable
public long getResourceValue(java.lang.String resource)

Get the value for a specified resource. No information about the units is returned.

Parameters:

resource - name of the resource

Returns:

the value for the resource

setResourceInformation

@Public
@Unstable
public void setResourceInformation(java.lang.String resource,
                                   ResourceInformation resourceInformation)

Set the ResourceInformation object for a particular resource.

Parameters:

resource - the resource for which the ResourceInformation is provided

resourceInformation - ResourceInformation object

setResourceInformation

@Private
@Unstable
public void setResourceInformation(int index,
                                   ResourceInformation resourceInformation)
                            throws ResourceNotFoundException

Set the ResourceInformation object for a particular resource.

Parameters:

index - the resource index for which the ResourceInformation is provided

resourceInformation - ResourceInformation object

Throws:

ResourceNotFoundException - if the resource is not found

setResourceValue

@Public
@Unstable
public void setResourceValue(java.lang.String resource,
                             long value)

Set the value of a resource in the ResourceInformation object. The unit of the value is assumed to be the one in the ResourceInformation object.

Parameters:

resource - the resource for which the value is provided.

value - the value to set

setResourceValue

@Private
@Unstable
public void setResourceValue(int index,
                             long value)
                      throws ResourceNotFoundException

Set the value of a resource in the ResourceInformation object. The unit of the value is assumed to be the one in the ResourceInformation object.

Parameters:

index - the resource index for which the value is provided.

value - the value to set

Throws:

ResourceNotFoundException - if the resource is not found

getDisks

@Public
@Stable
public double getDisks()

Get number of disks of the resource.

Returns:

number of disks of the resource

setDisks

@Public
@Stable
public void setDisks(double disks)

Set number of disks of the resource. Multiples to 1000 because disks handle as milli value

Parameters:

disks - number of disks of the resource

throwExceptionWhenArrayOutOfBound

protected void throwExceptionWhenArrayOutOfBound(int index)

equals

public boolean equals(java.lang.Object obj)

Overrides:

equals in class java.lang.Object

compareTo

public int compareTo(Resource other)

Specified by:

compareTo in interface java.lang.Comparable<Resource>

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

toFormattedString

public java.lang.String toFormattedString()

getFormattedString

public java.lang.String getFormattedString()

This method is to get memory in terms of KB|MB|GB.

Returns:

string containing all resources

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

castToIntSafely

protected static int castToIntSafely(long value)

Convert long to int for a resource value safely. This method assumes resource value is positive.

Parameters:

value - long resource value

Returns:

int resource value

newDefaultInformation

protected static ResourceInformation newDefaultInformation(java.lang.String name,
                                                           java.lang.String unit,
                                                           long value)

Create ResourceInformation with basic fields.

Parameters:

name - Resource Type Name

unit - Default unit of provided resource type

value - Value associated with giveb resource

Returns:

ResourceInformation object

setResources

@VisibleForTesting
protected void setResources(ResourceInformation[] resources)

getFormattedString

public java.lang.String getFormattedString(long memory)