com.mapr.hadoop.yarn.util

Class MapRFSLoggingHandler

java.lang.Object

org.apache.hadoop.yarn.util.DFSLoggingHandler

com.mapr.hadoop.yarn.util.MapRFSLoggingHandler

public class MapRFSLoggingHandler
extends org.apache.hadoop.yarn.util.DFSLoggingHandler

Supports redirecting stdout and stderr generated by YARN applications to MapRFS using maprcp. It also overrides the APIs to get log directories since MapR uses local volumes to store the logs generated on each node. So there isn't a single base directory under which all the application logs are stored. For e.g., if the tasks for app-1 ran on 2 nodes, then the logs would be stored under: /var/mapr/local/host-1/... and /var/mapr/local/host-2/... So to retrieve its logs, we need to query 2 different directory hierarchies.

Constructor Summary

Constructors

Constructor and Description
MapRFSLoggingHandler()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.fs.Path[]	getLogDir(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)
org.apache.hadoop.fs.Path	getLogDir(org.apache.hadoop.yarn.api.records.ContainerId containerId)
org.apache.hadoop.fs.Path	getLogDirForWrite(java.lang.String relativeContainerLogDir)
java.lang.String	getStdErrCommand(java.lang.String stderrFile)
java.lang.String	getStdOutCommand(java.lang.String filePath)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapRFSLoggingHandler

public MapRFSLoggingHandler()

Method Detail

getStdOutCommand

public java.lang.String getStdOutCommand(java.lang.String filePath)

Specified by:

getStdOutCommand in class org.apache.hadoop.yarn.util.DFSLoggingHandler

getStdErrCommand

public java.lang.String getStdErrCommand(java.lang.String stderrFile)

Specified by:

getStdErrCommand in class org.apache.hadoop.yarn.util.DFSLoggingHandler

getLogDirForWrite

public org.apache.hadoop.fs.Path getLogDirForWrite(java.lang.String relativeContainerLogDir)
                                            throws java.io.IOException

Overrides:

getLogDirForWrite in class org.apache.hadoop.yarn.util.DFSLoggingHandler

Throws:

java.io.IOException

getLogDir

public org.apache.hadoop.fs.Path[] getLogDir(org.apache.hadoop.yarn.api.records.ApplicationId applicationId)
                                      throws java.io.IOException

Overrides:

getLogDir in class org.apache.hadoop.yarn.util.DFSLoggingHandler

Throws:

java.io.IOException

getLogDir

public org.apache.hadoop.fs.Path getLogDir(org.apache.hadoop.yarn.api.records.ContainerId containerId)
                                    throws java.io.IOException

Overrides:

getLogDir in class org.apache.hadoop.yarn.util.DFSLoggingHandler

Throws:

java.io.IOException