abstract class GpuBaseWindowExecMeta[WindowExecType <: SparkPlan] extends SparkPlanMeta[WindowExecType] with Logging
Base class for GPU Execs that implement window functions. This abstracts the method by which the window function's input expressions, partition specs, order-by specs, etc. are extracted from the specific WindowExecType.
- WindowExecType
The Exec class that implements window functions (E.g. o.a.s.sql.execution.window.WindowExec.)
- Alphabetic
- By Inheritance
- GpuBaseWindowExecMeta
- Logging
- SparkPlanMeta
- RapidsMeta
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
- new GpuBaseWindowExecMeta(windowExec: WindowExecType, conf: RapidsConf, parent: Option[RapidsMeta[_, _, _]], rule: DataFromReplacementRule)
Abstract Value Members
-
abstract
def
getInputWindowExpressions: Seq[NamedExpression]
Extracts window-expression from WindowExecType.
Extracts window-expression from WindowExecType. The implementation varies, depending on the WindowExecType class.
-
abstract
def
getOrderSpecs: Seq[SortOrder]
Extracts order-by spec from WindowExecType.
Extracts order-by spec from WindowExecType. The implementation varies, depending on the WindowExecType class.
-
abstract
def
getPartitionSpecs: Seq[Expression]
Extracts partition-spec from WindowExecType.
Extracts partition-spec from WindowExecType. The implementation varies, depending on the WindowExecType class.
-
abstract
def
getResultColumnsOnly: Boolean
Indicates the output column semantics for the WindowExecType, i.e.
Indicates the output column semantics for the WindowExecType, i.e. whether to only return the window-expression result columns (as in some Spark distributions) or also include the input columns (as in Apache Spark).
Concrete Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
addConvertedDataType(expression: Expression, typeMeta: DataTypeMeta): Unit
Call this method to record information about type conversions via DataTypeMeta.
Call this method to record information about type conversions via DataTypeMeta.
- Definition Classes
- RapidsMeta
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
val
availableRuntimeDataTransition: Boolean
Whether there exists runtime data transition for the wrapped plan, if true, the overriding of output attributes will always work even when the wrapped plan can't be replaced by GPU overrides.
Whether there exists runtime data transition for the wrapped plan, if true, the overriding of output attributes will always work even when the wrapped plan can't be replaced by GPU overrides.
- Definition Classes
- SparkPlanMeta
-
def
canDataWriteCmdsBeReplaced: Boolean
Returns true iff all of the data writing commands can be replaced.
Returns true iff all of the data writing commands can be replaced.
- Definition Classes
- RapidsMeta
-
def
canExprTreeBeReplaced: Boolean
Returns true iff all of the expressions and their children could be replaced.
Returns true iff all of the expressions and their children could be replaced.
- Definition Classes
- RapidsMeta
-
def
canPartsBeReplaced: Boolean
Returns true iff all of the partitioning can be replaced.
Returns true iff all of the partitioning can be replaced.
- Definition Classes
- RapidsMeta
-
def
canScansBeReplaced: Boolean
Returns true iff all of the scans can be replaced.
Returns true iff all of the scans can be replaced.
- Definition Classes
- RapidsMeta
-
final
def
canThisBeReplaced: Boolean
Returns true iff this could be replaced.
Returns true iff this could be replaced.
- Definition Classes
- RapidsMeta
-
var
cannotBeReplacedReasons: Option[Set[String]]
- Attributes
- protected
- Definition Classes
- RapidsMeta
-
var
cannotRunOnGpuBecauseOfCost: Boolean
- Attributes
- protected
- Definition Classes
- RapidsMeta
-
var
cannotRunOnGpuBecauseOfSparkPlan: Boolean
- Attributes
- protected
- Definition Classes
- RapidsMeta
-
def
checkExistingTags(): Unit
When AQE is enabled and we are planning a new query stage, we need to look at meta-data previously stored on the spark plan to determine whether this operator can run on GPU
When AQE is enabled and we are planning a new query stage, we need to look at meta-data previously stored on the spark plan to determine whether this operator can run on GPU
- Definition Classes
- SparkPlanMeta
-
def
checkTimeZoneId(sessionZoneId: ZoneId): Unit
- Definition Classes
- RapidsMeta
-
val
childDataWriteCmds: Seq[DataWritingCommandMeta[_]]
The wrapped data writing commands that should be examined
The wrapped data writing commands that should be examined
- Definition Classes
- SparkPlanMeta → RapidsMeta
-
val
childExprs: Seq[BaseExprMeta[_]]
Define all dependency expressions as
childExprs.Define all dependency expressions as
childExprs. This ensures that they are tagged for GPU execution.- Definition Classes
- GpuBaseWindowExecMeta → SparkPlanMeta → RapidsMeta
-
val
childParts: Seq[PartMeta[_]]
The wrapped partitioning that should be examined
The wrapped partitioning that should be examined
- Definition Classes
- SparkPlanMeta → RapidsMeta
-
val
childPlans: Seq[SparkPlanMeta[SparkPlan]]
The wrapped plans that should be examined
The wrapped plans that should be examined
- Definition Classes
- SparkPlanMeta → RapidsMeta
-
val
childScans: Seq[ScanMeta[_]]
The wrapped scans that should be examined
The wrapped scans that should be examined
- Definition Classes
- SparkPlanMeta → RapidsMeta
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @HotSpotIntrinsicCandidate()
-
val
conf: RapidsConf
- Definition Classes
- RapidsMeta
-
def
confKey: String
- Definition Classes
- RapidsMeta
-
final
def
convertIfNeeded(): SparkPlan
If this is enabled to be converted to a GPU version convert it and return the result, else do what is needed to possibly convert the rest of the plan.
If this is enabled to be converted to a GPU version convert it and return the result, else do what is needed to possibly convert the rest of the plan.
- Definition Classes
- SparkPlanMeta
-
def
convertToCpu(): SparkPlan
Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled.
Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled. By default this just returns what is wrapped by this. For some types of operators/stages, like SparkPlan, each part of the query can be converted independent of other parts. As such in a subclass this should be overridden to do the correct thing.
- Definition Classes
- SparkPlanMeta → RapidsMeta
-
def
convertToGpu(): GpuExec
Convert what this wraps to a GPU enabled version.
Convert what this wraps to a GPU enabled version.
- Definition Classes
- GpuBaseWindowExecMeta → RapidsMeta
-
final
def
costPreventsRunningOnGpu(): Unit
- Definition Classes
- RapidsMeta
-
var
cpuCost: Double
- Definition Classes
- SparkPlanMeta
-
final
val
disabledMsg: Option[String]
- Definition Classes
- RapidsMeta
-
final
def
entirePlanExcludedReasons: Set[String]
Returns the list of reasons the entire plan can't be replaced.
Returns the list of reasons the entire plan can't be replaced. An empty set means the entire plan is ok to be replaced, do the normal checking per exec and children.
- Definition Classes
- RapidsMeta
-
final
def
entirePlanWillNotWork(because: String): Unit
Call this if there is a condition found that the entire plan is not allowed to run on the GPU.
Call this if there is a condition found that the entire plan is not allowed to run on the GPU.
- Definition Classes
- RapidsMeta
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
var
estimatedOutputRows: Option[BigInt]
- Definition Classes
- SparkPlanMeta
-
def
explain(all: Boolean): String
- Definition Classes
- RapidsMeta
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
def
getReasonsNotToReplaceEntirePlan: Set[String]
- Definition Classes
- SparkPlanMeta
-
var
gpuCost: Double
- Definition Classes
- SparkPlanMeta
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
val
incompatDoc: Option[String]
- Definition Classes
- RapidsMeta
-
def
indent(append: StringBuilder, depth: Int): Unit
- Attributes
- protected
- Definition Classes
- RapidsMeta
-
def
initReasons(): Unit
- Definition Classes
- RapidsMeta
-
def
initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
- Attributes
- protected
- Definition Classes
- Logging
-
def
initializeLogIfNecessary(isInterpreter: Boolean): Unit
- Attributes
- protected
- Definition Classes
- Logging
- lazy val inputFields: Seq[BaseExprMeta[Attribute]]
-
final
def
inputFilePreventsRunningOnGpu(): Unit
- Definition Classes
- RapidsMeta
-
def
isDisabledByDefault: Boolean
- Definition Classes
- RapidsMeta
-
def
isIncompat: Boolean
- Definition Classes
- RapidsMeta
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
isTraceEnabled(): Boolean
- Attributes
- protected
- Definition Classes
- Logging
-
def
log: Logger
- Attributes
- protected
- Definition Classes
- Logging
-
def
logDebug(msg: ⇒ String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logDebug(msg: ⇒ String): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logError(msg: ⇒ String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logError(msg: ⇒ String): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logInfo(msg: ⇒ String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logInfo(msg: ⇒ String): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logName: String
- Attributes
- protected
- Definition Classes
- Logging
-
def
logTrace(msg: ⇒ String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logTrace(msg: ⇒ String): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logWarning(msg: ⇒ String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
def
logWarning(msg: ⇒ String): Unit
- Attributes
- protected
- Definition Classes
- Logging
-
final
def
mustBeReplaced(because: String): Unit
- Definition Classes
- RapidsMeta
-
final
def
mustThisBeReplaced: Boolean
Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.
Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.
- Definition Classes
- RapidsMeta
-
def
namedChildExprs: Map[String, Seq[BaseExprMeta[_]]]
- Definition Classes
- GpuBaseWindowExecMeta → SparkPlanMeta
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
noReplacementPossibleMessage(reasons: String): String
- Definition Classes
- RapidsMeta
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
val
operationName: String
- Definition Classes
- RapidsMeta
- val orderSpec: Seq[BaseExprMeta[SortOrder]]
-
def
outputAttributes: Seq[Attribute]
Gets output attributes of current SparkPlanMeta, which is supposed to be called during type checking for the current plan.
Gets output attributes of current SparkPlanMeta, which is supposed to be called during type checking for the current plan.
By default, it simply returns the output of wrapped plan. For specific plans, they can override outputTypeMetas to apply custom conversions on the output of wrapped plan. For plans which just pass through the schema of childPlan, they can set useOutputAttributesOfChild to true, in order to propagate the custom conversions of childPlan if they exist.
- Definition Classes
- SparkPlanMeta
-
lazy val
outputTypeMetas: Option[Seq[DataTypeMeta]]
Overrides this method to implement custom conversions for specific plans.
Overrides this method to implement custom conversions for specific plans.
- Attributes
- protected
- Definition Classes
- SparkPlanMeta
-
val
parent: Option[RapidsMeta[_, _, _]]
- Definition Classes
- RapidsMeta
- val partitionSpec: Seq[BaseExprMeta[Expression]]
-
def
print(strBuilder: StringBuilder, depth: Int, all: Boolean): Unit
Create a string representation of this in append.
Create a string representation of this in append.
- strBuilder
where to place the string representation.
- depth
how far down the tree this is.
- all
should all the data be printed or just what does not work on the GPU?
- Attributes
- protected
- Definition Classes
- RapidsMeta
-
val
printWrapped: Boolean
When converting this to a string should we include the string representation of what this wraps too? This is off by default.
When converting this to a string should we include the string representation of what this wraps too? This is off by default.
- Attributes
- protected
- Definition Classes
- RapidsMeta
-
final
def
recursiveCostPreventsRunningOnGpu(): Unit
Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.
Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.
- Definition Classes
- RapidsMeta
-
final
def
recursiveSparkPlanPreventsRunningOnGpu(): Unit
- Definition Classes
- RapidsMeta
-
final
def
recursiveSparkPlanRemoved(): Unit
- Definition Classes
- RapidsMeta
-
def
replaceMessage: String
- Definition Classes
- RapidsMeta
-
def
requireAstForGpuOn(exprMeta: BaseExprMeta[_]): Unit
- Definition Classes
- SparkPlanMeta
-
def
runAfterTagRules(): Unit
Run rules that happen for the entire tree after it has been tagged initially.
Run rules that happen for the entire tree after it has been tagged initially.
- Definition Classes
- SparkPlanMeta
-
final
def
shouldBeRemoved(because: String): Unit
- Definition Classes
- RapidsMeta
-
final
def
shouldThisBeRemoved: Boolean
Returns true if this node should be removed.
Returns true if this node should be removed.
- Definition Classes
- RapidsMeta
-
def
suppressWillWorkOnGpuInfo: Boolean
- Definition Classes
- RapidsMeta
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
tagForExplain(): Unit
- Definition Classes
- SparkPlanMeta
-
final
def
tagForGpu(): Unit
Tag all of the children to see if they are GPU compatible first.
Tag all of the children to see if they are GPU compatible first. Do basic common verification for the operators, and then call tagSelfForGpu
- Definition Classes
- RapidsMeta
-
def
tagPlanForGpu(): Unit
Called to verify that this plan will work on the GPU.
Called to verify that this plan will work on the GPU. Generic checks will have already been done. In general this method should only tag this operator as bad. If it needs to tag one of its children please take special care to update the comment inside
tagSelfForGpuso we don't end up with something that could be cyclical.- Definition Classes
- GpuBaseWindowExecMeta → SparkPlanMeta
-
final
def
tagSelfForGpu(): Unit
Do any extra checks and tag yourself if you are compatible or not.
Do any extra checks and tag yourself if you are compatible or not. Be aware that this may already have been marked as incompatible for a number of reasons.
All of your children should have already been tagged so if there are situations where you may need to disqualify your children for various reasons you may do it here too.
- Definition Classes
- SparkPlanMeta → RapidsMeta
-
def
toString(): String
- Definition Classes
- RapidsMeta → AnyRef → Any
-
val
useOutputAttributesOfChild: Boolean
Whether to pass through the outputAttributes of childPlan's meta, only for UnaryPlan
Whether to pass through the outputAttributes of childPlan's meta, only for UnaryPlan
- Attributes
- protected
- Definition Classes
- SparkPlanMeta
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
willNotWorkOnGpu(because: String): Unit
Call this to indicate that this should not be replaced with a GPU enabled version
Call this to indicate that this should not be replaced with a GPU enabled version
- because
why it should not be replaced.
- Definition Classes
- RapidsMeta
- val windowExpressions: Seq[BaseExprMeta[NamedExpression]]
-
val
wrapped: WindowExecType
- Definition Classes
- RapidsMeta