Package org.apache.hadoop.tools.dynamometer.workloadgenerator.audit

Class AuditLogHiveTableParser

java.lang.Object

org.apache.hadoop.tools.dynamometer.workloadgenerator.audit.AuditLogHiveTableParser

All Implemented Interfaces:

AuditCommandParser

public class AuditLogHiveTableParser
extends java.lang.Object
implements AuditCommandParser

This AuditCommandParser is used to read commands assuming that the input was generated by a Hive query storing uncompressed output files, in which fields should be separated by the start-of-heading (U+0001) character. The fields available should be, in order:

   relativeTimestampMs,ugi,command,src,dest,sourceIP
 

Where relativeTimestampMs represents the time elapsed between the start of the audit log and the occurrence of the audit event. Assuming your audit logs are available in Hive, this can be generated with a query looking like:

   INSERT OVERWRITE DIRECTORY '${outputPath}'
   SELECT (timestamp - ${startTime} AS relTime, ugi, cmd, src, dst, ip
   FROM '${auditLogTableLocation}'
   WHERE
     timestamp >= ${startTime}
     AND timestamp < ${endTime}
   DISTRIBUTE BY src
   SORT BY relTime ASC;
 

Note that the sorting step is important; events in each distinct file must be in time-ascending order.

Constructor Summary

Constructors

Constructor	Description
AuditLogHiveTableParser()

Method Summary

Modifier and Type	Method	Description
void	initialize(org.apache.hadoop.conf.Configuration conf)	Initialize this parser with the given configuration.
org.apache.hadoop.tools.dynamometer.workloadgenerator.audit.AuditReplayCommand	parse(org.apache.hadoop.io.Text inputLine, java.util.function.Function<java.lang.Long,java.lang.Long> relativeToAbsolute)	Convert a line of input into an AuditReplayCommand.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AuditLogHiveTableParser

public AuditLogHiveTableParser()

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration conf)
                throws java.io.IOException

Description copied from interface: AuditCommandParser

Initialize this parser with the given configuration. Guaranteed to be called prior to any calls to AuditCommandParser.parse(Text, Function).

Specified by:

initialize in interface AuditCommandParser

Parameters:

conf - The Configuration to be used to set up this parser.

Throws:

java.io.IOException - if error on initializing a parser.

parse

public org.apache.hadoop.tools.dynamometer.workloadgenerator.audit.AuditReplayCommand parse(org.apache.hadoop.io.Text inputLine,
                                                                                            java.util.function.Function<java.lang.Long,java.lang.Long> relativeToAbsolute)
                                                                                     throws java.io.IOException

Description copied from interface: AuditCommandParser

Convert a line of input into an AuditReplayCommand. Since AuditReplayCommands store absolute timestamps, relativeToAbsolute can be used to convert relative timestamps (i.e., milliseconds elapsed between the start of the audit log and this command) into absolute timestamps.

Specified by:

parse in interface AuditCommandParser

Parameters:

inputLine - Single input line to convert.

relativeToAbsolute - Function converting relative timestamps (in milliseconds) to absolute timestamps (in milliseconds).

Returns:

A command representing the input line.

Throws:

java.io.IOException - if error on parsing.