Package org.apache.hadoop.yarn.server.timelineservice.documentstore

Class DocumentStoreUtils

java.lang.Object

org.apache.hadoop.yarn.server.timelineservice.documentstore.DocumentStoreUtils

public final class DocumentStoreUtils
extends java.lang.Object

This class consists of all the utils required for reading or writing documents for a DocumentStoreVendor.

Method Summary

Modifier and Type	Method	Description
static java.util.Map<java.lang.String,java.lang.String>	applyConfigFilter(org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList configsToRetrieve, java.util.Map<java.lang.String,java.lang.String> configs)
static java.lang.String	constructFlowActivityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext, long eventTimestamp)	Creates a composite key for storing FlowActivityDocument.
static java.lang.String	constructFlowRunDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext)	Creates a composite key for storing FlowRunDocument.
static java.lang.String	constructTimelineEntityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext, java.lang.String type)	Creates a composite key for storing TimelineEntityDocument.
static java.lang.String	constructTimelineEntityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext, java.lang.String type, java.lang.String id)	Creates a composite key for storing TimelineEntityDocument.
static com.microsoft.azure.cosmosdb.rx.AsyncDocumentClient	createCosmosDBAsyncClient(org.apache.hadoop.conf.Configuration conf)	Creates CosmosDB Async Document Client.
static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity	createEntityToBeReturned(TimelineEntityDocument timelineEntityDocument, org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList confsToRetrieve, org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList metricsToRetrieve)	Creates the final entity to be returned as the result.
static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity	createEntityToBeReturned(TimelineEntityDocument timelineEntityDocument, org.apache.hadoop.yarn.server.timelineservice.reader.TimelineDataToRetrieve dataToRetrieve)	Creates the final entity to be returned as the result.
static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEvent	fetchEvent(org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity timelineEntity, java.lang.String eventType)	Retrieves a TimelineEvent from TimelineEntity.events.
static java.lang.String	getCosmosDBDatabaseName(org.apache.hadoop.conf.Configuration conf)
static DocumentStoreVendor	getStoreVendor(org.apache.hadoop.conf.Configuration conf)	Retrieves DocumentStoreVendor configured.
static long	getTopOfTheDayTimestamp(long timeStamp)	Returns the timestamp of the day's start (which is midnight 00:00:00 AM) for a given input timestamp.
static boolean	isNullOrEmpty(java.lang.String... values)	Checks if the string is null or empty.
static java.util.Set<org.apache.hadoop.yarn.api.records.timelineservice.TimelineMetric>	transformMetrics(org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList metricsToRetrieve, java.util.Map<java.lang.String,java.util.Set<TimelineMetricSubDoc>> metrics)
static void	validateCosmosDBConf(org.apache.hadoop.conf.Configuration conf)	Checks whether the cosmosdb conf are set properly in yarn-site.xml conf.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

validateCosmosDBConf

public static void validateCosmosDBConf(org.apache.hadoop.conf.Configuration conf)
                                 throws org.apache.hadoop.yarn.exceptions.YarnException

Checks whether the cosmosdb conf are set properly in yarn-site.xml conf.

Parameters:

conf - related to yarn

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if required config properties are missing

getStoreVendor

public static DocumentStoreVendor getStoreVendor(org.apache.hadoop.conf.Configuration conf)

Retrieves DocumentStoreVendor configured.

Parameters:

conf - related to yarn

Returns:

Returns the DocumentStoreVendor that is configured, else uses DocumentStoreVendor.COSMOS_DB as default

fetchEvent

public static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEvent fetchEvent(org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity timelineEntity,
                                                                                          java.lang.String eventType)

Retrieves a TimelineEvent from TimelineEntity.events.

Parameters:

timelineEntity - from which the set of events are examined.

eventType - that has to be checked.

Returns:

TimelineEvent if found else null

isNullOrEmpty

public static boolean isNullOrEmpty(java.lang.String... values)

Checks if the string is null or empty.

Parameters:

values - array of string to be checked

Returns:

false if any of the string is null or empty else true

createCosmosDBAsyncClient

public static com.microsoft.azure.cosmosdb.rx.AsyncDocumentClient createCosmosDBAsyncClient(org.apache.hadoop.conf.Configuration conf)

Creates CosmosDB Async Document Client.

Parameters:

conf - to retrieve cosmos db endpoint and key

Returns:

async document client for CosmosDB

getTopOfTheDayTimestamp

public static long getTopOfTheDayTimestamp(long timeStamp)

Returns the timestamp of the day's start (which is midnight 00:00:00 AM) for a given input timestamp.

Parameters:

timeStamp - Timestamp.

Returns:

timestamp of that day's beginning (midnight)

constructTimelineEntityDocId

public static java.lang.String constructTimelineEntityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext,
                                                            java.lang.String type)

Creates a composite key for storing TimelineEntityDocument.

Parameters:

collectorContext - of the timeline writer

type - of the entity

Returns:

composite key delimited with !

constructTimelineEntityDocId

public static java.lang.String constructTimelineEntityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext,
                                                            java.lang.String type,
                                                            java.lang.String id)

Creates a composite key for storing TimelineEntityDocument.

Parameters:

collectorContext - of the timeline writer

type - of the entity

id - of the entity

Returns:

composite key delimited with !

constructFlowRunDocId

public static java.lang.String constructFlowRunDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext)

Creates a composite key for storing FlowRunDocument.

Parameters:

collectorContext - of the timeline writer

Returns:

composite key delimited with !

constructFlowActivityDocId

public static java.lang.String constructFlowActivityDocId(org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorContext collectorContext,
                                                          long eventTimestamp)

Creates a composite key for storing FlowActivityDocument.

Parameters:

collectorContext - of the timeline writer

eventTimestamp - of the timeline entity

Returns:

composite key delimited with !

getCosmosDBDatabaseName

public static java.lang.String getCosmosDBDatabaseName(org.apache.hadoop.conf.Configuration conf)

createEntityToBeReturned

public static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity createEntityToBeReturned(TimelineEntityDocument timelineEntityDocument,
                                                                                                         org.apache.hadoop.yarn.server.timelineservice.reader.TimelineDataToRetrieve dataToRetrieve)

Creates the final entity to be returned as the result.

Parameters:

timelineEntityDocument - which has all the information for the entity

dataToRetrieve - specifies filters and fields to retrieve

Returns:

TimelineEntity as the result

createEntityToBeReturned

public static org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity createEntityToBeReturned(TimelineEntityDocument timelineEntityDocument,
                                                                                                         org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList confsToRetrieve,
                                                                                                         org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList metricsToRetrieve)

Creates the final entity to be returned as the result.

Parameters:

timelineEntityDocument - which has all the information for the entity

confsToRetrieve - specifies config filters to be applied

metricsToRetrieve - specifies metric filters to be applied

Returns:

TimelineEntity as the result

transformMetrics

public static java.util.Set<org.apache.hadoop.yarn.api.records.timelineservice.TimelineMetric> transformMetrics(org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList metricsToRetrieve,
                                                                                                                java.util.Map<java.lang.String,java.util.Set<TimelineMetricSubDoc>> metrics)

applyConfigFilter

public static java.util.Map<java.lang.String,java.lang.String> applyConfigFilter(org.apache.hadoop.yarn.server.timelineservice.reader.filter.TimelineFilterList configsToRetrieve,
                                                                                       java.util.Map<java.lang.String,java.lang.String> configs)