Client (Apache Hadoop Dynamometer Cluster Simulator 3.4.1.300-dep-1001 API)

java.lang.Object
- org.apache.hadoop.conf.Configured
- - org.apache.hadoop.tools.dynamometer.Client

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
```
@Public
@Unstable
public class Client
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool
```
Client for submitting a Dynamometer YARN application, and optionally, a workload MapReduce job. This client uploads resources to HDFS as necessary for them to be accessed by the YARN app, then launches an ApplicationMaster, which is responsible for managing the lifetime of the application.
The Dynamometer YARN application starts up the DataNodes of an HDFS cluster. If the namenode_servicerpc_addr option is specified, it should point to the service RPC address of an existing namenode, which the datanodes will talk to. Else, a namenode will be launched internal to this YARN application. The ApplicationMaster's logs contain links to the NN / DN containers to be able to access their logs. Some of this information is also printed by the client.
The application will store files in the submitting user's home directory under a `.dynamometer/applicationID/` folder. This is mostly for uses internal to the application, but if the NameNode is launched through YARN, the NameNode's metrics will also be uploaded to a file `namenode_metrics` within this folder. This file is also accessible as part of the NameNode's logs, but this centralized location is easier to access for subsequent parsing.
If the NameNode is launched internally, this Client will monitor the status of the NameNode, printing information about its availability as the DataNodes register (e.g., outstanding under replicated blocks as block reports arrive). If this is configured to launch the workload job, once the NameNode has gathered information from all of its DataNodes, the client will launch a workload job which is configured to act against the newly launched NameNode. Once the workload job completes, the infrastructure application will be shut down. At this time only the audit log replay (AuditReplayMapper) workload is supported.
If there is no workload job configured, this application will, by default, persist indefinitely until killed by YARN. You can specify the timeout option to have it exit automatically after some time. This timeout will enforced if there is a workload job configured as well.

Field Summary

Fields
Modifier and Type	Field	Description
`static java.lang.String`	`APPNAME_ARG`
`static java.lang.String`	`APPNAME_DEFAULT`
`static java.lang.String`	`BLOCK_LIST_PATH_ARG`
`static java.lang.String`	`CONF_PATH_ARG`
`static java.lang.String`	`FS_IMAGE_DIR_ARG`
`static java.lang.String`	`HADOOP_BINARY_PATH_ARG`
`static java.lang.String`	`HADOOP_VERSION_ARG`
`static java.lang.String`	`MASTER_MEMORY_MB_ARG`
`static java.lang.String`	`MASTER_MEMORY_MB_DEFAULT`
`static java.lang.String`	`MASTER_VCORES_ARG`
`static java.lang.String`	`MASTER_VCORES_DEFAULT`
`static java.lang.String`	`NAMENODE_SERVICERPC_ADDR_ARG`
`static java.lang.String`	`QUEUE_ARG`
`static java.lang.String`	`QUEUE_DEFAULT`
`static java.lang.String`	`TIMEOUT_ARG`
`static java.lang.String`	`TIMEOUT_DEFAULT`
`static java.lang.String`	`TOKEN_FILE_LOCATION_ARG`
`static java.lang.String`	`WORKLOAD_CONFIG_ARG`
`static java.lang.String`	`WORKLOAD_INPUT_PATH_ARG`
`static java.lang.String`	`WORKLOAD_OUTPUT_PATH_ARG`
`static java.lang.String`	`WORKLOAD_RATE_FACTOR_ARG`
`static java.lang.String`	`WORKLOAD_RATE_FACTOR_DEFAULT`
`static java.lang.String`	`WORKLOAD_REPLAY_ENABLE_ARG`
`static java.lang.String`	`WORKLOAD_START_DELAY_ARG`
`static java.lang.String`	`WORKLOAD_THREADS_PER_MAPPER_ARG`

Constructor Summary

Constructors
Constructor Description

Client(java.lang.String... dependencyJars)

Constructors
Constructor	Description
`Client(java.lang.String... dependencyJars)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`attemptCleanup()`	Best-effort attempt to clean up any remaining applications (infrastructure or workload).
`boolean`	`init(java.lang.String[] args)`	Parse command line options.
`static void`	`main(java.lang.String[] args)`
`boolean`	`run()`	Main run function for the client.
`int`	`run(java.lang.String[] args)`

Methods inherited from class org.apache.hadoop.conf.Configured
getConf, setConf

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable
getConf, setConf

- Field Detail
  - APPNAME_ARG
```
public static final java.lang.String APPNAME_ARG
```
    See Also:
    
    Constant Field Values
  - APPNAME_DEFAULT
```
public static final java.lang.String APPNAME_DEFAULT
```
    See Also:
    
    Constant Field Values
  - QUEUE_ARG
```
public static final java.lang.String QUEUE_ARG
```
    See Also:
    
    Constant Field Values
  - QUEUE_DEFAULT
```
public static final java.lang.String QUEUE_DEFAULT
```
    See Also:
    
    Constant Field Values
  - TIMEOUT_ARG
```
public static final java.lang.String TIMEOUT_ARG
```
    See Also:
    
    Constant Field Values
  - TIMEOUT_DEFAULT
```
public static final java.lang.String TIMEOUT_DEFAULT
```
    See Also:
    
    Constant Field Values
  - HADOOP_VERSION_ARG
```
public static final java.lang.String HADOOP_VERSION_ARG
```
    See Also:
    
    Constant Field Values
  - HADOOP_BINARY_PATH_ARG
```
public static final java.lang.String HADOOP_BINARY_PATH_ARG
```
    See Also:
    
    Constant Field Values
  - NAMENODE_SERVICERPC_ADDR_ARG
```
public static final java.lang.String NAMENODE_SERVICERPC_ADDR_ARG
```
    See Also:
    
    Constant Field Values
  - FS_IMAGE_DIR_ARG
```
public static final java.lang.String FS_IMAGE_DIR_ARG
```
    See Also:
    
    Constant Field Values
  - BLOCK_LIST_PATH_ARG
```
public static final java.lang.String BLOCK_LIST_PATH_ARG
```
    See Also:
    
    Constant Field Values
  - CONF_PATH_ARG
```
public static final java.lang.String CONF_PATH_ARG
```
    See Also:
    
    Constant Field Values
  - MASTER_VCORES_ARG
```
public static final java.lang.String MASTER_VCORES_ARG
```
    See Also:
    
    Constant Field Values
  - MASTER_VCORES_DEFAULT
```
public static final java.lang.String MASTER_VCORES_DEFAULT
```
    See Also:
    
    Constant Field Values
  - MASTER_MEMORY_MB_ARG
```
public static final java.lang.String MASTER_MEMORY_MB_ARG
```
    See Also:
    
    Constant Field Values
  - MASTER_MEMORY_MB_DEFAULT
```
public static final java.lang.String MASTER_MEMORY_MB_DEFAULT
```
    See Also:
    
    Constant Field Values
  - TOKEN_FILE_LOCATION_ARG
```
public static final java.lang.String TOKEN_FILE_LOCATION_ARG
```
    See Also:
    
    Constant Field Values
  - WORKLOAD_REPLAY_ENABLE_ARG
```
public static final java.lang.String WORKLOAD_REPLAY_ENABLE_ARG
```
    See Also:
    
    Constant Field Values
  - WORKLOAD_INPUT_PATH_ARG
```
public static final java.lang.String WORKLOAD_INPUT_PATH_ARG
```
    See Also:
    
    Constant Field Values
  - WORKLOAD_OUTPUT_PATH_ARG
```
public static final java.lang.String WORKLOAD_OUTPUT_PATH_ARG
```
    See Also:
    
    Constant Field Values
  - WORKLOAD_THREADS_PER_MAPPER_ARG
```
public static final java.lang.String WORKLOAD_THREADS_PER_MAPPER_ARG
```
    See Also:
    
    Constant Field Values
  - WORKLOAD_START_DELAY_ARG
```
public static final java.lang.String WORKLOAD_START_DELAY_ARG
```
    See Also:
    
    Constant Field Values
  - WORKLOAD_RATE_FACTOR_ARG
```
public static final java.lang.String WORKLOAD_RATE_FACTOR_ARG
```
    See Also:
    
    Constant Field Values
  - WORKLOAD_RATE_FACTOR_DEFAULT
```
public static final java.lang.String WORKLOAD_RATE_FACTOR_DEFAULT
```
    See Also:
    
    Constant Field Values
  - WORKLOAD_CONFIG_ARG
```
public static final java.lang.String WORKLOAD_CONFIG_ARG
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - Client
```
public Client(java.lang.String... dependencyJars)
```
- Method Detail
  - main
```
public static void main(java.lang.String[] args)
                 throws java.lang.Exception
```
    Parameters:
    
    args - Command line arguments
    
    Throws:
    
    java.lang.Exception - on error
  - run
```
public int run(java.lang.String[] args)
```
    Specified by:
    
    run in interface org.apache.hadoop.util.Tool
  - init
```
public boolean init(java.lang.String[] args)
             throws org.apache.commons.cli.ParseException,
                    java.io.IOException
```
    Parse command line options.
    
    Parameters:
    
    args - Parsed command line options
    
    Returns:
    
    Whether the init was successful to run the client
    
    Throws:
    
    org.apache.commons.cli.ParseException - on error while parsing
    
    java.io.IOException - for other errors
  - run
```
public boolean run()
            throws java.io.IOException,
                   org.apache.hadoop.yarn.exceptions.YarnException
```
    Main run function for the client.
    
    Returns:
    
    true if application completed successfully
    
    Throws:
    
    java.io.IOException - for general issues
    
    org.apache.hadoop.yarn.exceptions.YarnException - for issues while contacting YARN daemons
  - attemptCleanup
```
public void attemptCleanup()
```
    Best-effort attempt to clean up any remaining applications (infrastructure or workload).

Class Client

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.conf.Configured

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.hadoop.conf.Configurable

Field Detail

APPNAME_ARG

APPNAME_DEFAULT

QUEUE_ARG

QUEUE_DEFAULT

TIMEOUT_ARG

TIMEOUT_DEFAULT

HADOOP_VERSION_ARG

HADOOP_BINARY_PATH_ARG

NAMENODE_SERVICERPC_ADDR_ARG

FS_IMAGE_DIR_ARG

BLOCK_LIST_PATH_ARG

CONF_PATH_ARG

MASTER_VCORES_ARG

MASTER_VCORES_DEFAULT

MASTER_MEMORY_MB_ARG

MASTER_MEMORY_MB_DEFAULT

TOKEN_FILE_LOCATION_ARG

WORKLOAD_REPLAY_ENABLE_ARG

WORKLOAD_INPUT_PATH_ARG

WORKLOAD_OUTPUT_PATH_ARG

WORKLOAD_THREADS_PER_MAPPER_ARG

WORKLOAD_START_DELAY_ARG

WORKLOAD_RATE_FACTOR_ARG

WORKLOAD_RATE_FACTOR_DEFAULT

WORKLOAD_CONFIG_ARG

Constructor Detail

Client

Method Detail

main

run

init

run

attemptCleanup