AuditReplayMapper (Apache Hadoop Dynamometer Workload Simulator 3.4.1.200-dep-1000 API)

java.lang.Object
- org.apache.hadoop.mapreduce.Mapper<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
- - org.apache.hadoop.tools.dynamometer.workloadgenerator.WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
  - - org.apache.hadoop.tools.dynamometer.workloadgenerator.audit.AuditReplayMapper

```
public class AuditReplayMapper
extends WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
```
AuditReplayMapper replays the given audit trace against the NameNode under test. Each mapper spawns a number of threads equal to the "auditreplay.num-threads" configuration (by default 1) to use for replaying. Each mapper reads a single input file which will be consumed by all of the available threads. A FileInputFormat with splitting disabled is used so any files present in the input path directory (given by the "auditreplay.input-path" configuration) will be used as input; one file per mapper. The expected format of these files is determined by the value of the "auditreplay.command-parser.class" configuration, which defaults to AuditLogDirectParser.
This generates a number of Counter values which can be used to get information into the replay, including the number of commands replayed, how many of them were "invalid" (threw an exception), how many were "late" (replayed later than they should have been), and the latency (from client perspective) of each command. If there are a large number of "late" commands, you likely need to increase the number of threads used and/or the number of mappers.
By default, commands will be replayed at the same rate as they were originally performed. However a rate factor can be specified via the "auditreplay.rate-factor" configuration; all of the (relative) timestamps will be divided by this rate factor, effectively changing the rate at which they are replayed. For example, a rate factor of 2 would make the replay occur twice as fast, and a rate factor of 0.5 would make it occur half as fast.

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`static class`	`AuditReplayMapper.CommandType`	Define the type of command, either read or write.
`static class`	`AuditReplayMapper.ReplayCommand`	Definitions of the various commands that can be replayed.
`static class`	`AuditReplayMapper.REPLAYCOUNTERS`	`Counter` definitions for replay.

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper
org.apache.hadoop.mapreduce.Mapper.Context

Field Summary

Fields
Modifier and Type	Field	Description
`static java.lang.Class<AuditLogDirectParser>`	`COMMAND_PARSER_DEFAULT`
`static java.lang.String`	`COMMAND_PARSER_KEY`
`static boolean`	`CREATE_BLOCKS_DEFAULT`
`static java.lang.String`	`CREATE_BLOCKS_KEY`
`static java.lang.String`	`INDIVIDUAL_COMMANDS_COUNT_SUFFIX`
`static java.lang.String`	`INDIVIDUAL_COMMANDS_COUNTER_GROUP`
`static java.lang.String`	`INDIVIDUAL_COMMANDS_INVALID_SUFFIX`
`static java.lang.String`	`INDIVIDUAL_COMMANDS_LATENCY_SUFFIX`
`static java.lang.String`	`INPUT_PATH_KEY`
`static int`	`NUM_THREADS_DEFAULT`
`static java.lang.String`	`NUM_THREADS_KEY`
`static java.lang.String`	`OUTPUT_PATH_KEY`
`static double`	`RATE_FACTOR_DEFAULT`
`static java.lang.String`	`RATE_FACTOR_KEY`

Constructor Summary

Constructors
Constructor Description

AuditReplayMapper()

Constructors
Constructor	Description
`AuditReplayMapper()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`cleanup(org.apache.hadoop.mapreduce.Mapper.Context context)`
`void`	`configureJob(org.apache.hadoop.mapreduce.Job job)`	Setup input and output formats and optional reducer.
`java.util.List<java.lang.String>`	`getConfigDescriptions()`	Get a list of the description of each configuration that this mapper accepts.
`java.lang.String`	`getDescription()`	Get the description of the behavior of this mapper.
`void`	`map(org.apache.hadoop.io.LongWritable lineNum, org.apache.hadoop.io.Text inputLine, org.apache.hadoop.mapreduce.Mapper.Context context)`
`void`	`setup(org.apache.hadoop.mapreduce.Mapper.Context context)`
`boolean`	`verifyConfigurations(org.apache.hadoop.conf.Configuration conf)`	Verify that the provided configuration contains all configurations required by this mapper.

Methods inherited from class org.apache.hadoop.mapreduce.Mapper
run

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - INPUT_PATH_KEY
```
public static final java.lang.String INPUT_PATH_KEY
```
    See Also:
    
    Constant Field Values
  - OUTPUT_PATH_KEY
```
public static final java.lang.String OUTPUT_PATH_KEY
```
    See Also:
    
    Constant Field Values
  - NUM_THREADS_KEY
```
public static final java.lang.String NUM_THREADS_KEY
```
    See Also:
    
    Constant Field Values
  - NUM_THREADS_DEFAULT
```
public static final int NUM_THREADS_DEFAULT
```
    See Also:
    
    Constant Field Values
  - CREATE_BLOCKS_KEY
```
public static final java.lang.String CREATE_BLOCKS_KEY
```
    See Also:
    
    Constant Field Values
  - CREATE_BLOCKS_DEFAULT
```
public static final boolean CREATE_BLOCKS_DEFAULT
```
    See Also:
    
    Constant Field Values
  - RATE_FACTOR_KEY
```
public static final java.lang.String RATE_FACTOR_KEY
```
    See Also:
    
    Constant Field Values
  - RATE_FACTOR_DEFAULT
```
public static final double RATE_FACTOR_DEFAULT
```
    See Also:
    
    Constant Field Values
  - COMMAND_PARSER_KEY
```
public static final java.lang.String COMMAND_PARSER_KEY
```
    See Also:
    
    Constant Field Values
  - COMMAND_PARSER_DEFAULT
```
public static final java.lang.Class<AuditLogDirectParser> COMMAND_PARSER_DEFAULT
```
  - INDIVIDUAL_COMMANDS_COUNTER_GROUP
```
public static final java.lang.String INDIVIDUAL_COMMANDS_COUNTER_GROUP
```
    See Also:
    
    Constant Field Values
  - INDIVIDUAL_COMMANDS_LATENCY_SUFFIX
```
public static final java.lang.String INDIVIDUAL_COMMANDS_LATENCY_SUFFIX
```
    See Also:
    
    Constant Field Values
  - INDIVIDUAL_COMMANDS_INVALID_SUFFIX
```
public static final java.lang.String INDIVIDUAL_COMMANDS_INVALID_SUFFIX
```
    See Also:
    
    Constant Field Values
  - INDIVIDUAL_COMMANDS_COUNT_SUFFIX
```
public static final java.lang.String INDIVIDUAL_COMMANDS_COUNT_SUFFIX
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - AuditReplayMapper
```
public AuditReplayMapper()
```
- Method Detail
  - getDescription
```
public java.lang.String getDescription()
```
    Description copied from class: WorkloadMapper
    
    Get the description of the behavior of this mapper.
    
    Specified by:
    
    getDescription in class WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Returns:
    
    description string.
  - getConfigDescriptions
```
public java.util.List<java.lang.String> getConfigDescriptions()
```
    Description copied from class: WorkloadMapper
    
    Get a list of the description of each configuration that this mapper accepts.
    
    Specified by:
    
    getConfigDescriptions in class WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Returns:
    
    list of the description of each configuration.
  - verifyConfigurations
```
public boolean verifyConfigurations(org.apache.hadoop.conf.Configuration conf)
```
    Description copied from class: WorkloadMapper
    
    Verify that the provided configuration contains all configurations required by this mapper.
    
    Specified by:
    
    verifyConfigurations in class WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Parameters:
    
    conf - configuration.
    
    Returns:
    
    whether or not all configurations required are provided.
  - setup
```
public void setup(org.apache.hadoop.mapreduce.Mapper.Context context)
           throws java.io.IOException
```
    Overrides:
    
    setup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Throws:
    
    java.io.IOException
  - map
```
public void map(org.apache.hadoop.io.LongWritable lineNum,
                org.apache.hadoop.io.Text inputLine,
                org.apache.hadoop.mapreduce.Mapper.Context context)
         throws java.io.IOException,
                java.lang.InterruptedException
```
    Overrides:
    
    map in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Throws:
    
    java.io.IOException
    
    java.lang.InterruptedException
  - cleanup
```
public void cleanup(org.apache.hadoop.mapreduce.Mapper.Context context)
             throws java.lang.InterruptedException,
                    java.io.IOException
```
    Overrides:
    
    cleanup in class org.apache.hadoop.mapreduce.Mapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>
    
    Throws:
    
    java.lang.InterruptedException
    
    java.io.IOException
  - configureJob
```
public void configureJob(org.apache.hadoop.mapreduce.Job job)
```
    Description copied from class: WorkloadMapper
    
    Setup input and output formats and optional reducer.
    
    Overrides:
    
    configureJob in class WorkloadMapper<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,UserCommandKey,CountTimeWritable>

Class AuditReplayMapper

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

Methods inherited from class java.lang.Object

Field Detail

INPUT_PATH_KEY

OUTPUT_PATH_KEY

NUM_THREADS_KEY

NUM_THREADS_DEFAULT

CREATE_BLOCKS_KEY

CREATE_BLOCKS_DEFAULT

RATE_FACTOR_KEY

RATE_FACTOR_DEFAULT

COMMAND_PARSER_KEY

COMMAND_PARSER_DEFAULT

INDIVIDUAL_COMMANDS_COUNTER_GROUP

INDIVIDUAL_COMMANDS_LATENCY_SUFFIX

INDIVIDUAL_COMMANDS_INVALID_SUFFIX

INDIVIDUAL_COMMANDS_COUNT_SUFFIX

Constructor Detail

AuditReplayMapper

Method Detail

getDescription

getConfigDescriptions

verifyConfigurations

setup

map

cleanup

configureJob