Package org.apache.flink.yarn.configuration

Class YarnConfigOptions

java.lang.Object
org.apache.flink.yarn.configuration.YarnConfigOptions
@PublicEvolving public class YarnConfigOptions extends Object
This class holds configuration constants used by Flink's YARN runners.

These options are not expected to be ever configured by users explicitly.

  • Field Details

    • APP_MASTER_VCORES

      public static final org.apache.flink.configuration.ConfigOption<Integer> APP_MASTER_VCORES
      The vcores used by YARN application master.

    • CLASSPATH_INCLUDE_USER_JAR

      public static final org.apache.flink.configuration.ConfigOption<YarnConfigOptions.UserJarInclusion> CLASSPATH_INCLUDE_USER_JAR
      Defines whether user-jars are included in the system class path as well as their positioning in the path. They can be positioned at the beginning (FIRST), at the end (LAST), or be positioned based on their name (ORDER). DISABLED means the user-jars are excluded from the system class path and as a result these jars will be loaded by user classloader.

    • VCORES

      public static final org.apache.flink.configuration.ConfigOption<Integer> VCORES
      The vcores exposed by YARN.

    • APPLICATION_ATTEMPTS

      public static final org.apache.flink.configuration.ConfigOption<Integer> APPLICATION_ATTEMPTS
      Set the number of retries for failed YARN ApplicationMasters/JobManagers in high availability mode. This value is usually limited by YARN. By default, it's 1 in the standalone case and 2 in the high availability case.

      >Note: This option returns a String since Integer options must have a static default value.

    • APPLICATION_ATTEMPT_FAILURE_VALIDITY_INTERVAL

      public static final org.apache.flink.configuration.ConfigOption<Long> APPLICATION_ATTEMPT_FAILURE_VALIDITY_INTERVAL
      The config parameter defining the attemptFailuresValidityInterval of Yarn application.

    • HEARTBEAT_DELAY_SECONDS

      public static final org.apache.flink.configuration.ConfigOption<Integer> HEARTBEAT_DELAY_SECONDS
      The heartbeat interval between the Application Master and the YARN Resource Manager.

    • CONTAINER_REQUEST_HEARTBEAT_INTERVAL_MILLISECONDS

      public static final org.apache.flink.configuration.ConfigOption<Duration> CONTAINER_REQUEST_HEARTBEAT_INTERVAL_MILLISECONDS
      The heartbeat interval between the Application Master and the YARN Resource Manager if Flink is requesting containers.

    • PROPERTIES_FILE_LOCATION

      public static final org.apache.flink.configuration.ConfigOption<String> PROPERTIES_FILE_LOCATION
      When a Flink job is submitted to YARN, the JobManager's host and the number of available processing slots is written into a properties file, so that the Flink client is able to pick those details up. This configuration parameter allows changing the default location of that file (for example for environments sharing a Flink installation between users)

    • APPLICATION_MASTER_PORT

      public static final org.apache.flink.configuration.ConfigOption<String> APPLICATION_MASTER_PORT
      The config parameter defining the Pekko actor system port for the ApplicationMaster and JobManager. The port can either be a port, such as "9123", a range of ports: "50100-50200" or a list of ranges and or points: "50100-50200,50300-50400,51234". Setting the port to 0 will let the OS choose an available port.

    • APPLICATION_PRIORITY

      public static final org.apache.flink.configuration.ConfigOption<Integer> APPLICATION_PRIORITY
      A non-negative integer indicating the priority for submitting a Flink YARN application. It will only take effect if YARN priority scheduling setting is enabled. Larger integer corresponds with higher priority. If priority is negative or set to '-1'(default), Flink will unset yarn priority setting and use cluster default priority.
      See Also:

    • FILE_REPLICATION

      public static final org.apache.flink.configuration.ConfigOption<Integer> FILE_REPLICATION
      Yarn session client uploads flink jar and user libs to file system (hdfs/s3) as local resource for yarn application context. The replication number changes the how many replica of each of these files in hdfs/s3. It is useful to accelerate this container bootstrap, when a Flink application needs more than one hundred of containers. If it is not configured, Flink will use the default replication value in hadoop configuration.

    • APPLICATION_TAGS

      public static final org.apache.flink.configuration.ConfigOption<String> APPLICATION_TAGS
      A comma-separated list of strings to use as YARN application tags.

    • APPLICATION_VIEW_ACLS

      public static final org.apache.flink.configuration.ConfigOption<String> APPLICATION_VIEW_ACLS
      Users and groups to give VIEW access. https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_yarn_acl.html

    • APPLICATION_MODIFY_ACLS

      public static final org.apache.flink.configuration.ConfigOption<String> APPLICATION_MODIFY_ACLS
      Users and groups to give MODIFY access.

    • STAGING_DIRECTORY

      public static final org.apache.flink.configuration.ConfigOption<String> STAGING_DIRECTORY

    • SHIP_FILES

      public static final org.apache.flink.configuration.ConfigOption<List<String>> SHIP_FILES

    • SHIP_ARCHIVES

      public static final org.apache.flink.configuration.ConfigOption<List<String>> SHIP_ARCHIVES

    • APPLICATION_ID

      public static final org.apache.flink.configuration.ConfigOption<String> APPLICATION_ID

    • APPLICATION_QUEUE

      public static final org.apache.flink.configuration.ConfigOption<String> APPLICATION_QUEUE

    • APPLICATION_NAME

      public static final org.apache.flink.configuration.ConfigOption<String> APPLICATION_NAME

    • APPLICATION_TYPE

      public static final org.apache.flink.configuration.ConfigOption<String> APPLICATION_TYPE

    • NODE_LABEL

      public static final org.apache.flink.configuration.ConfigOption<String> NODE_LABEL

    • TASK_MANAGER_NODE_LABEL

      public static final org.apache.flink.configuration.ConfigOption<String> TASK_MANAGER_NODE_LABEL

    • SHIP_LOCAL_KEYTAB

      public static final org.apache.flink.configuration.ConfigOption<Boolean> SHIP_LOCAL_KEYTAB

    • LOCALIZED_KEYTAB_PATH

      public static final org.apache.flink.configuration.ConfigOption<String> LOCALIZED_KEYTAB_PATH

    • APP_MASTER_TOKEN_SERVICES

      public static final org.apache.flink.configuration.ConfigOption<List<String>> APP_MASTER_TOKEN_SERVICES

    • PROVIDED_LIB_DIRS

      public static final org.apache.flink.configuration.ConfigOption<List<String>> PROVIDED_LIB_DIRS

    • PROVIDED_USRLIB_DIR

      public static final org.apache.flink.configuration.ConfigOption<String> PROVIDED_USRLIB_DIR
      Allows users to directly utilize usrlib directory in HDFS for YARN application mode. The classloader for loading jars under the usrlib will be controlled by CLASSPATH_INCLUDE_USER_JAR.

    • HADOOP_CONFIG_KEY

      public static final org.apache.flink.configuration.ConfigOption<String> HADOOP_CONFIG_KEY

    • YARN_CONFIG_KEY

      public static final org.apache.flink.configuration.ConfigOption<String> YARN_CONFIG_KEY

    • YARN_CONTAINER_START_COMMAND_TEMPLATE

      public static final org.apache.flink.configuration.ConfigOption<String> YARN_CONTAINER_START_COMMAND_TEMPLATE

    • EXTERNAL_RESOURCE_YARN_CONFIG_KEY_SUFFIX

      public static final String EXTERNAL_RESOURCE_YARN_CONFIG_KEY_SUFFIX
      Defines the configuration key of that external resource in Yarn. This is used as a suffix in an actual config.
      See Also:

    • EXTERNAL_RESOURCE_YARN_CONFIG_KEY

      public static final org.apache.flink.configuration.ConfigOption<String> EXTERNAL_RESOURCE_YARN_CONFIG_KEY
      If configured, Flink will add this key to the resource profile of container request to Yarn. The value will be set to ExternalResourceOptions.EXTERNAL_RESOURCE_AMOUNT.

      It is intentionally included into user docs while unused.