java.lang.Iterable<java.util.Map.Entry<java.lang.String,java.lang.String>>, org.apache.hadoop.io.Writable@Private
@Evolving
public class FairSchedulerConfiguration
extends org.apache.hadoop.conf.Configuration
| Modifier and Type | Field | Description |
|---|---|---|
static java.lang.String |
ALLOCATION_FILE |
|
static java.lang.String |
ALLOW_UNDECLARED_POOLS |
Whether pools can be created that were not specified in the FS configuration file
|
protected static java.lang.String |
AM_PREEMPTION |
|
protected static java.lang.String |
AM_PREEMPTION_PREFIX |
|
static java.lang.String |
ASSIGN_MULTIPLE |
Whether to assign multiple containers in one check-in.
|
static java.lang.String |
CONTINUOUS_SCHEDULING_ENABLED |
Deprecated.
Continuous scheduling is known to cause locking issue inside
the scheduler in larger cluster, more than 100 nodes, use
ASSIGN_MULTIPLE to improve container allocation ramp up. |
static java.lang.String |
CONTINUOUS_SCHEDULING_SLEEP_MS |
Deprecated.
Continuous scheduling is known to cause locking issue inside
Only used when
CONTINUOUS_SCHEDULING_ENABLED is enabled |
protected static java.lang.String |
DEFAULT_ALLOCATION_FILE |
|
static boolean |
DEFAULT_ALLOW_UNDECLARED_POOLS |
|
protected static boolean |
DEFAULT_AM_PREEMPTION |
|
static boolean |
DEFAULT_ASSIGN_MULTIPLE |
|
static boolean |
DEFAULT_CONTINUOUS_SCHEDULING_ENABLED |
Deprecated.
|
static int |
DEFAULT_CONTINUOUS_SCHEDULING_SLEEP_MS |
Deprecated.
|
protected static long |
DEFAULT_LOCALITY_DELAY_NODE_MS |
Deprecated.
|
protected static long |
DEFAULT_LOCALITY_DELAY_RACK_MS |
Deprecated.
|
protected static float |
DEFAULT_LOCALITY_THRESHOLD |
|
static float |
DEFAULT_LOCALITY_THRESHOLD_NODE |
|
static float |
DEFAULT_LOCALITY_THRESHOLD_RACK |
|
static int |
DEFAULT_MAX_ASSIGN |
|
static boolean |
DEFAULT_PREEMPTION |
|
protected static float |
DEFAULT_PREEMPTION_THRESHOLD |
|
protected static boolean |
DEFAULT_PREEMPTION_THRESHOLD_BASED_ON_LABELS_ENABLED |
|
static boolean |
DEFAULT_RESOURCES_BASED_ON_LABELS_ENABLED |
|
static int |
DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_MB |
Deprecated.
|
static int |
DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES |
Deprecated.
|
static float |
DEFAULT_RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE |
|
static boolean |
DEFAULT_SIZE_BASED_WEIGHT |
|
static int |
DEFAULT_UPDATE_INTERVAL_MS |
|
static boolean |
DEFAULT_USER_AS_DEFAULT_QUEUE |
|
static int |
DEFAULT_WAIT_TIME_BEFORE_KILL |
|
static long |
DEFAULT_WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK_MS |
|
static java.lang.String |
DYNAMIC_MAX_ASSIGN |
Maximum number of containers to assign on each check-in.
|
protected static java.lang.String |
LOCALITY_DELAY_NODE_MS |
Deprecated.
Continuous scheduling is known to cause locking issue inside
Only used when
CONTINUOUS_SCHEDULING_ENABLED is enabled |
protected static java.lang.String |
LOCALITY_DELAY_RACK_MS |
Deprecated.
Continuous scheduling is known to cause locking issue inside
Only used when
CONTINUOUS_SCHEDULING_ENABLED is enabled |
static java.lang.String |
LOCALITY_THRESHOLD_NODE |
Cluster threshold for node locality.
|
static java.lang.String |
LOCALITY_THRESHOLD_RACK |
Cluster threshold for rack locality.
|
static org.slf4j.Logger |
LOG |
|
static java.lang.String |
MAX_ASSIGN |
Specify exact number of containers to assign on each heartbeat, if dynamic
max assign is turned off.
|
static java.lang.String |
MIGRATION_MODE |
Used during FS->CS conversion.
|
static java.lang.String |
NO_TERMINAL_RULE_CHECK |
Disables checking whether a placement rule is terminal or not.
|
static java.lang.String |
PREEMPTION |
Whether preemption is enabled.
|
protected static java.lang.String |
PREEMPTION_THRESHOLD |
|
protected static java.lang.String |
PREEMPTION_THRESHOLD_BASED_ON_LABELS_ENABLED |
Whether preemption utilization threshold based on node labels is enabled.
|
static java.lang.String |
RESERVABLE_NODES |
Ratio of nodes available for an app to make an reservation on.
|
static float |
RESERVABLE_NODES_DEFAULT |
|
static java.lang.String |
RESOURCES_BASED_ON_LABELS_ENABLED |
Whether computing of resources based on node labels is enabled.
|
static java.lang.String |
RM_SCHEDULER_INCREMENT_ALLOCATION_MB |
Deprecated.
The preferred way to configure the increment is by using the
yarn.resource-types.{RESOURCE_NAME}.increment-allocation property,
for memory: yarn.resource-types.memory-mb.increment-allocation
|
static java.lang.String |
RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES |
Deprecated.
The preferred way to configure the increment is by using the
yarn.resource-types.{RESOURCE_NAME}.increment-allocation property,
for CPU: yarn.resource-types.vcores.increment-allocation
|
static java.lang.String |
RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE |
Threshold for container size for making a container reservation as a
multiple of increment allocation.
|
static java.lang.String |
SIZE_BASED_WEIGHT |
Whether to give more weight to apps requiring many resources.
|
static java.lang.String |
UPDATE_INTERVAL_MS |
The update interval for calculating resources in FairScheduler .
|
static java.lang.String |
USER_AS_DEFAULT_QUEUE |
Whether to use the user name as the queue name (instead of "default") if
the request does not specify a queue.
|
static java.lang.String |
WAIT_TIME_BEFORE_KILL |
|
static java.lang.String |
WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK_MS |
Configurable delay (ms) before an app's starvation is considered after
it is identified.
|
| Constructor | Description |
|---|---|
FairSchedulerConfiguration() |
|
FairSchedulerConfiguration(org.apache.hadoop.conf.Configuration conf) |
| Modifier and Type | Method | Description |
|---|---|---|
boolean |
getAMPreemptionEnabled(java.lang.String queueName) |
|
boolean |
getAssignMultiple() |
|
int |
getContinuousSchedulingSleepMs() |
Deprecated.
linked to
CONTINUOUS_SCHEDULING_ENABLED deprecation |
org.apache.hadoop.yarn.api.records.Resource |
getIncrementAllocation() |
|
long |
getLocalityDelayNodeMs() |
Deprecated.
linked to
CONTINUOUS_SCHEDULING_ENABLED deprecation |
long |
getLocalityDelayRackMs() |
Deprecated.
linked to
CONTINUOUS_SCHEDULING_ENABLED deprecation |
float |
getLocalityThresholdNode() |
|
float |
getLocalityThresholdRack() |
|
int |
getMaxAssign() |
|
org.apache.hadoop.yarn.api.records.Resource |
getMaximumAllocation() |
|
org.apache.hadoop.yarn.api.records.Resource |
getMinimumAllocation() |
|
boolean |
getPreemptionEnabled() |
|
float |
getPreemptionUtilizationThreshold() |
|
float |
getReservableNodes() |
|
float |
getReservationThresholdIncrementMultiple() |
|
boolean |
getSizeBasedWeight() |
|
long |
getUpdateInterval() |
|
boolean |
getUsePortForNodeName() |
|
int |
getWaitTimeBeforeKill() |
|
long |
getWaitTimeBeforeNextStarvationCheck() |
|
boolean |
isContinuousSchedulingEnabled() |
Deprecated.
use
ASSIGN_MULTIPLE to improve container allocation
ramp up. |
boolean |
isMaxAssignDynamic() |
|
boolean |
isPreemptionThresholdBasedOnLabelsEnabled() |
|
boolean |
isResourcesBasedOnLabelsEnabled() |
|
static ConfigurableResource |
parseResourceConfigValue(java.lang.String value) |
Parses a resource config value in one of three forms:
Percentage: "50%" or "40% memory, 60% cpu"
New style resources: "vcores=10, memory-mb=1024"
or "vcores=60%, memory-mb=40%"
Old style resources: "1024 mb, 10 vcores"
In new style resources, any resource that is not specified will be
set to
Long.MAX_VALUE or 100%, as appropriate. |
static ConfigurableResource |
parseResourceConfigValue(java.lang.String value,
long missing) |
Parses a resource config value in one of three forms:
Percentage: "50%" or "40% memory, 60% cpu"
New style resources: "vcores=10, memory-mb=1024"
or "vcores=60%, memory-mb=40%"
Old style resources: "1024 mb, 10 vcores"
In new style resources, any resource that is not specified will be
set to
missing or 0%, as appropriate. |
addDefaultResource, addDefaultResource, addDeprecation, addDeprecation, addDeprecation, addDeprecation, addDeprecations, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addResource, addTags, clear, dumpConfiguration, dumpConfiguration, dumpDeprecatedKeys, get, get, getAllPropertiesByTag, getAllPropertiesByTags, getBoolean, getClass, getClass, getClassByName, getClassByNameOrNull, getClasses, getClassLoader, getConfResourceAsInputStream, getConfResourceAsReader, getDouble, getEnum, getEnumSet, getFile, getFinalParameters, getFloat, getInstances, getInt, getInts, getLocalPath, getLong, getLongBytes, getPassword, getPasswordFromConfig, getPasswordFromCredentialProviders, getPattern, getPropertySources, getProps, getPropsWithPrefix, getRange, getRaw, getResource, getSocketAddr, getSocketAddr, getStorageSize, getStorageSize, getStringCollection, getStrings, getStrings, getTimeDuration, getTimeDuration, getTimeDuration, getTimeDuration, getTimeDurationHelper, getTimeDurations, getTrimmed, getTrimmed, getTrimmedStringCollection, getTrimmedStrings, getTrimmedStrings, getValByRegex, hasWarnedDeprecation, isDeprecated, isPropertyTag, iterator, main, onlyKeyExists, readFields, reloadConfiguration, reloadExistingConfigurations, set, set, setAllowNullValueProperties, setBoolean, setBooleanIfUnset, setClass, setClassLoader, setDeprecatedProperties, setDouble, setEnum, setFloat, setIfUnset, setInt, setLong, setPattern, setQuietMode, setRestrictSystemProperties, setRestrictSystemPropertiesDefault, setRestrictSystemProps, setSocketAddr, setStorageSize, setStrings, setTimeDuration, size, substituteCommonVariables, toString, unset, updateConnectAddr, updateConnectAddr, write, writeXml, writeXml, writeXml, writeXmlpublic static final org.slf4j.Logger LOG
@Deprecated public static final java.lang.String RM_SCHEDULER_INCREMENT_ALLOCATION_MB
@Deprecated public static final int DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_MB
@Deprecated public static final java.lang.String RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES
@Deprecated public static final int DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES
public static final java.lang.String RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE
public static final float DEFAULT_RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE
public static final java.lang.String MIGRATION_MODE
public static final java.lang.String NO_TERMINAL_RULE_CHECK
public static final java.lang.String ALLOCATION_FILE
protected static final java.lang.String DEFAULT_ALLOCATION_FILE
public static final java.lang.String ALLOW_UNDECLARED_POOLS
public static final boolean DEFAULT_ALLOW_UNDECLARED_POOLS
public static final java.lang.String USER_AS_DEFAULT_QUEUE
public static final boolean DEFAULT_USER_AS_DEFAULT_QUEUE
protected static final float DEFAULT_LOCALITY_THRESHOLD
public static final java.lang.String LOCALITY_THRESHOLD_NODE
public static final float DEFAULT_LOCALITY_THRESHOLD_NODE
public static final java.lang.String LOCALITY_THRESHOLD_RACK
public static final float DEFAULT_LOCALITY_THRESHOLD_RACK
@Deprecated protected static final java.lang.String LOCALITY_DELAY_NODE_MS
CONTINUOUS_SCHEDULING_ENABLED is enabled@Deprecated protected static final long DEFAULT_LOCALITY_DELAY_NODE_MS
@Deprecated protected static final java.lang.String LOCALITY_DELAY_RACK_MS
CONTINUOUS_SCHEDULING_ENABLED is enabled@Deprecated protected static final long DEFAULT_LOCALITY_DELAY_RACK_MS
@Deprecated public static final java.lang.String CONTINUOUS_SCHEDULING_ENABLED
ASSIGN_MULTIPLE to improve container allocation ramp up.@Deprecated public static final boolean DEFAULT_CONTINUOUS_SCHEDULING_ENABLED
@Deprecated public static final java.lang.String CONTINUOUS_SCHEDULING_SLEEP_MS
CONTINUOUS_SCHEDULING_ENABLED is enabled@Deprecated public static final int DEFAULT_CONTINUOUS_SCHEDULING_SLEEP_MS
public static final java.lang.String PREEMPTION
public static final boolean DEFAULT_PREEMPTION
protected static final java.lang.String AM_PREEMPTION
protected static final java.lang.String AM_PREEMPTION_PREFIX
protected static final boolean DEFAULT_AM_PREEMPTION
protected static final java.lang.String PREEMPTION_THRESHOLD
protected static final float DEFAULT_PREEMPTION_THRESHOLD
protected static final java.lang.String PREEMPTION_THRESHOLD_BASED_ON_LABELS_ENABLED
protected static final boolean DEFAULT_PREEMPTION_THRESHOLD_BASED_ON_LABELS_ENABLED
public static final java.lang.String RESOURCES_BASED_ON_LABELS_ENABLED
public static final boolean DEFAULT_RESOURCES_BASED_ON_LABELS_ENABLED
public static final java.lang.String WAIT_TIME_BEFORE_KILL
public static final int DEFAULT_WAIT_TIME_BEFORE_KILL
public static final java.lang.String WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK_MS
WAIT_TIME_BEFORE_KILL and enough heartbeats.
This is intended to be a backdoor on production clusters, and hence
intentionally not documented.public static final long DEFAULT_WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK_MS
public static final java.lang.String ASSIGN_MULTIPLE
public static final boolean DEFAULT_ASSIGN_MULTIPLE
public static final java.lang.String SIZE_BASED_WEIGHT
public static final boolean DEFAULT_SIZE_BASED_WEIGHT
public static final java.lang.String DYNAMIC_MAX_ASSIGN
public static final java.lang.String MAX_ASSIGN
public static final int DEFAULT_MAX_ASSIGN
public static final java.lang.String UPDATE_INTERVAL_MS
public static final int DEFAULT_UPDATE_INTERVAL_MS
public static final java.lang.String RESERVABLE_NODES
public static final float RESERVABLE_NODES_DEFAULT
public FairSchedulerConfiguration()
public FairSchedulerConfiguration(org.apache.hadoop.conf.Configuration conf)
public org.apache.hadoop.yarn.api.records.Resource getMinimumAllocation()
public org.apache.hadoop.yarn.api.records.Resource getMaximumAllocation()
public org.apache.hadoop.yarn.api.records.Resource getIncrementAllocation()
public float getReservationThresholdIncrementMultiple()
public float getLocalityThresholdNode()
public float getLocalityThresholdRack()
@Deprecated public boolean isContinuousSchedulingEnabled()
ASSIGN_MULTIPLE to improve container allocation
ramp up.@Deprecated public int getContinuousSchedulingSleepMs()
CONTINUOUS_SCHEDULING_ENABLED deprecation@Deprecated public long getLocalityDelayNodeMs()
CONTINUOUS_SCHEDULING_ENABLED deprecation@Deprecated public long getLocalityDelayRackMs()
CONTINUOUS_SCHEDULING_ENABLED deprecationpublic boolean getPreemptionEnabled()
public boolean getAMPreemptionEnabled(java.lang.String queueName)
public float getPreemptionUtilizationThreshold()
public boolean isPreemptionThresholdBasedOnLabelsEnabled()
public boolean isResourcesBasedOnLabelsEnabled()
public boolean getAssignMultiple()
public boolean isMaxAssignDynamic()
public int getMaxAssign()
public boolean getSizeBasedWeight()
public long getWaitTimeBeforeNextStarvationCheck()
public int getWaitTimeBeforeKill()
public boolean getUsePortForNodeName()
public float getReservableNodes()
public static ConfigurableResource parseResourceConfigValue(java.lang.String value) throws AllocationConfigurationException
Long.MAX_VALUE or 100%, as appropriate. Also, in the new
style resources, units are not allowed. Units are assumed from the resource
manager's settings for the resources when the value isn't a percentage.value - the resource definition to parseConfigurableResource that represents the parsed valueAllocationConfigurationException - if the raw value is not a valid
resource definitionpublic static ConfigurableResource parseResourceConfigValue(java.lang.String value, long missing) throws AllocationConfigurationException
missing or 0%, as appropriate. Also, in the new style
resources, units are not allowed. Units are assumed from the resource
manager's settings for the resources when the value isn't a percentage.
The missing parameter is only used in the case of new style
resources without percentages. With new style resources with percentages,
any missing resources will be assumed to be 100% because percentages are
only used with maximum resource limits.value - the resource definition to parsemissing - the value to use for any unspecified resourcesConfigurableResource that represents the parsed valueAllocationConfigurationException - if the raw value is not a valid
resource definitionpublic long getUpdateInterval()
Copyright © 2008–2025 Apache Software Foundation. All rights reserved.