Package org.apache.hadoop.yarn.util

Class MapRFSLoggingHandler

java.lang.Object

org.apache.hadoop.yarn.util.DFSLoggingHandler

org.apache.hadoop.yarn.util.MapRFSLoggingHandler

public class MapRFSLoggingHandler
extends DFSLoggingHandler

Supports redirecting stdout and stderr generated by YARN applications to MapRFS using maprcp. It also overrides the APIs to get log directories since MapR uses local volumes to store the logs generated on each node. So there isn't a single base directory under which all the application logs are stored. For e.g., if the tasks for app-1 ran on 2 nodes, then the logs would be stored under: /var/mapr/local/host-1/... and /var/mapr/local/host-2/... So to retrieve its logs, we need to query 2 different directory hierarchies.

Constructor Summary

Constructors

Constructor	Description
MapRFSLoggingHandler()

Method Summary

Modifier and Type	Method	Description
org.apache.hadoop.fs.Path[]	getLogDir(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)	Returns the absolute log directories that contains logs for the given application.
org.apache.hadoop.fs.Path	getLogDir(org.apache.hadoop.yarn.api.records.ContainerId containerId)	Returns the absolute log directory that contains logs for the given container.
org.apache.hadoop.fs.Path	getLogDirForWrite(java.lang.String relativeContainerLogDir)	Returns the absolute log directory to write to.
org.apache.hadoop.fs.Path	getLogFileForWrite(org.apache.hadoop.fs.Path stdOutDir, java.lang.String stdOutFile)
static java.lang.String	getPathToMaprHome()
java.lang.String	getStdErrCommand(java.lang.String stderrFile)	Returns the stderr redirection command to be used.
java.lang.String	getStdOutCommand(java.lang.String filePath)	Returns the stdout redirection command to be used.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapRFSLoggingHandler

public MapRFSLoggingHandler()

Method Detail

getPathToMaprHome

public static java.lang.String getPathToMaprHome()

getStdOutCommand

public java.lang.String getStdOutCommand(java.lang.String filePath)

Description copied from class: DFSLoggingHandler

Returns the stdout redirection command to be used. This input to this command is the stdout generated by the task.

Specified by:

getStdOutCommand in class DFSLoggingHandler

Parameters:

filePath - name of the file to which stdout will be redirected

Returns:

stdout redirection command

getStdErrCommand

public java.lang.String getStdErrCommand(java.lang.String stderrFile)

Description copied from class: DFSLoggingHandler

Returns the stderr redirection command to be used. This input to this command is the stderr generated by the task.

Specified by:

getStdErrCommand in class DFSLoggingHandler

Parameters:

stderrFile - name of the file to which stdout will be redirected

Returns:

stderr redirection command

getLogDirForWrite

public org.apache.hadoop.fs.Path getLogDirForWrite(java.lang.String relativeContainerLogDir)
                                            throws java.io.IOException

Description copied from class: DFSLoggingHandler

Returns the absolute log directory to write to.

Overrides:

getLogDirForWrite in class DFSLoggingHandler

Parameters:

relativeContainerLogDir - path of the specific container directory to write to

Returns:

absolute log directory

Throws:

java.io.IOException

getLogDir

public org.apache.hadoop.fs.Path[] getLogDir(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)
                                      throws java.io.IOException

Description copied from class: DFSLoggingHandler

Returns the absolute log directories that contains logs for the given application. An application can contain one or more containers and each container logs can potentially be stored in different directories. Hence an array of paths is returned.

Overrides:

getLogDir in class DFSLoggingHandler

Parameters:

applicationId - application ID

Returns:

absolute log directories that contain the given application logs

Throws:

java.io.IOException

getLogDir

public org.apache.hadoop.fs.Path getLogDir(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                                    throws java.io.IOException

Description copied from class: DFSLoggingHandler

Returns the absolute log directory that contains logs for the given container.

Overrides:

getLogDir in class DFSLoggingHandler

Parameters:

containerId - container ID

Returns:

absolute log directory that contain then given container logs

Throws:

java.io.IOException

getLogFileForWrite

public org.apache.hadoop.fs.Path getLogFileForWrite(org.apache.hadoop.fs.Path stdOutDir,
                                                    java.lang.String stdOutFile)

Overrides:

getLogFileForWrite in class DFSLoggingHandler