Type Members

1.  case class AnalysisContext(catalogAndNamespace: Seq[String] = Nil, nestedViewDepth: Int = 0, maxNestedViewDepth: Int = -1, relationCache: Map[(Seq[String], Option[TimeTravelSpec]), LogicalPlan] = mutable.Map.empty, referredTempViewNames: Seq[Seq[String]] = Seq.empty, referredTempFunctionNames: Set[String] = mutable.Set.empty, outerPlan: Option[LogicalPlan] = None) extends Product with Serializable

   Provides a way to keep state during the analysis, mostly for resolving views and subqueries.

   Provides a way to keep state during the analysis, mostly for resolving views and subqueries. This enables us to decouple the concerns of analysis environment from the catalog and resolve star expressions in subqueries that reference the outer query plans. The state that is kept here is per-query.

   Note this is thread local.

   catalogAndNamespace

   The catalog and namespace used in the view resolution. This overrides the current catalog and namespace when resolving relations inside views.

   nestedViewDepth

   The nested depth in the view resolution, this enables us to limit the depth of nested views.

   maxNestedViewDepth

   The maximum allowed depth of nested view resolution.

   relationCache

   A mapping from qualified table names and time travel spec to resolved relations. This can ensure that the table is resolved only once if a table is used multiple times in a query.

   referredTempViewNames

   All the temp view names referred by the current view we are resolving. It's used to make sure the relation resolution is consistent between view creation and view resolution. For example, if t was a permanent table when the current view was created, it should still be a permanent table when resolving the current view, even if a temp view t has been created.

   outerPlan

   The query plan from the outer query that can be used to resolve star expressions in a subquery.

2.  implicit class AnalysisErrorAt extends AnyRef
3.  class Analyzer extends RuleExecutor[LogicalPlan] with CheckAnalysis with SQLConfHelper

   Provides a logical query plan analyzer, which translates UnresolvedAttributes and UnresolvedRelations into fully typed objects using information in a SessionCatalog.

4.  case class AsOfTimestamp(timestamp: Long) extends TimeTravelSpec with Product with Serializable
5.  case class AsOfVersion(version: String) extends TimeTravelSpec with Product with Serializable
6.  class CannotReplaceMissingTableException extends AnalysisException
7.  trait CastSupport extends AnyRef

   Mix-in trait for constructing valid Cast expressions.

8.  trait CheckAnalysis extends PredicateHelper with LookupCatalog

   Throws user facing errors when passed invalid queries that fail to analyze.

9.  class DatabaseAlreadyExistsException extends NamespaceAlreadyExistsException

   Thrown by a catalog when an item already exists.

   Thrown by a catalog when an item already exists. The analyzer will rethrow the exception as an org.apache.spark.sql.AnalysisException with the correct position information.

10.  trait EmptyFunctionRegistryBase[T] extends FunctionRegistryBase[T]

    A trivial catalog that returns an error when a function is requested.

    A trivial catalog that returns an error when a function is requested. Used for testing when all functions are already filled in and the analyzer needs only to resolve attribute references.

11.  trait ExpressionBuilder extends AnyRef
12.  sealed trait FieldName extends LeafExpression with Unevaluable
13.  sealed trait FieldPosition extends LeafExpression with Unevaluable
14.  class FunctionAlreadyExistsException extends AnalysisException
15.  trait FunctionRegistry extends FunctionRegistryBase[Expression]
16.  trait FunctionRegistryBase[T] extends AnyRef

    A catalog for looking up user defined functions, used by an Analyzer.

    A catalog for looking up user defined functions, used by an Analyzer.

    Note: 1) The implementation should be thread-safe to allow concurrent access. 2) the database name is always case-sensitive here, callers are responsible to format the database name w.r.t. case-sensitive config.

17.  case class GetColumnByOrdinal(ordinal: Int, dataType: DataType) extends LeafExpression with Unevaluable with NonSQLExpression with Product with Serializable
18.  case class GetViewColumnByNameAndOrdinal(viewName: String, colName: String, ordinal: Int, expectedNumCandidates: Int, viewDDL: Option[String]) extends LeafExpression with Unevaluable with NonSQLExpression with Product with Serializable
19.  class IndexAlreadyExistsException extends AnalysisException
20.  trait LeafNodeWithoutStats extends LogicalPlan with LeafNode

    A resolved leaf node whose statistics has no meaning.

21.  case class MultiAlias(child: Expression, names: Seq[String]) extends UnaryExpression with NamedExpression with Unevaluable with Product with Serializable

    Used to assign new names to Generator's output, such as hive udtf.

    Used to assign new names to Generator's output, such as hive udtf. For example the SQL expression "stack(2, key, value, key, value) as (a, b)" could be represented as follows: MultiAlias(stack_function, Seq(a, b))

    child

    the computation being performed

    names

    the names to be associated with each output of computing child.

22.  trait MultiInstanceRelation extends AnyRef

    A trait that should be mixed into query operators where a single instance might appear multiple times in a logical query plan.

    A trait that should be mixed into query operators where a single instance might appear multiple times in a logical query plan. It is invalid to have multiple copies of the same attribute produced by distinct operators in a query tree as this breaks the guarantee that expression ids, which are used to differentiate attributes, are unique.

    During analysis, operators that include this trait may be asked to produce a new version of itself with globally unique expression ids.

23.  trait NamedRelation extends LogicalPlan
24.  class NamespaceAlreadyExistsException extends AnalysisException
25.  case class NoSuchDatabaseException(db: String) extends AnalysisException with Product with Serializable

    Thrown by a catalog when an item cannot be found.

    Thrown by a catalog when an item cannot be found. The analyzer will rethrow the exception as an org.apache.spark.sql.AnalysisException with the correct position information.

26.  case class NoSuchFunctionException(message: String, cause: Option[Throwable]) extends AnalysisException with Product with Serializable
27.  class NoSuchIndexException extends AnalysisException
28.  case class NoSuchNamespaceException(message: String, cause: Option[Throwable] = None) extends AnalysisException with Product with Serializable
29.  case class NoSuchPartitionException(message: String) extends AnalysisException with Product with Serializable
30.  case class NoSuchPartitionsException(message: String) extends AnalysisException with Product with Serializable
31.  case class NoSuchPermanentFunctionException(db: String, func: String) extends AnalysisException with Product with Serializable
32.  case class NoSuchTableException(message: String, cause: Option[Throwable] = None) extends AnalysisException with Product with Serializable
33.  case class NoSuchTempFunctionException(func: String) extends AnalysisException with Product with Serializable
34.  case class NonEmptyNamespaceException(message: String, cause: Option[Throwable] = None) extends AnalysisException with Product with Serializable

    Thrown by a catalog when an item already exists.

    Thrown by a catalog when an item already exists. The analyzer will rethrow the exception as an org.apache.spark.sql.AnalysisException with the correct position information.

35.  class PartitionAlreadyExistsException extends AnalysisException
36.  sealed trait PartitionSpec extends LeafExpression with Unevaluable
37.  class PartitionsAlreadyExistException extends AnalysisException
38.  case class ReferenceEqualPlanWrapper(plan: LogicalPlan) extends Product with Serializable

    A helper class used to detect duplicate relations fast in DeduplicateRelations

39.  case class RelationTimeTravel(relation: LogicalPlan, timestamp: Option[Expression], version: Option[String]) extends LogicalPlan with LeafNode with Product with Serializable

    A logical node used to time travel the child relation to the given timestamp or version.

    A logical node used to time travel the child relation to the given timestamp or version. The child must support time travel, e.g. a v2 source, and cannot be a view, subquery or stream. The timestamp expression cannot refer to any columns and cannot contain subqueries.

40.  class ResolveCatalogs extends Rule[LogicalPlan] with LookupCatalog

    Resolves catalogs from the multi-part identifiers in SQL statements, and convert the statements to the corresponding v2 commands if the resolved catalog is not the session catalog.

41.  case class ResolveTableValuedFunctions(catalog: SessionCatalog) extends Rule[LogicalPlan] with Product with Serializable

    Rule that resolves table-valued function references.

42.  case class ResolvedDBObjectName(catalog: CatalogPlugin, nameParts: Seq[String]) extends LogicalPlan with LeafNodeWithoutStats with Product with Serializable

    A plan containing resolved database object name with catalog determined.

43.  case class ResolvedFieldName(path: Seq[String], field: StructField) extends LeafExpression with FieldName with Product with Serializable
44.  case class ResolvedFieldPosition(position: ColumnPosition) extends LeafExpression with FieldPosition with Product with Serializable
45.  case class ResolvedNamespace(catalog: CatalogPlugin, namespace: Seq[String]) extends LogicalPlan with LeafNodeWithoutStats with Product with Serializable

    A plan containing resolved namespace.

46.  case class ResolvedNonPersistentFunc(name: String, func: UnboundFunction) extends LogicalPlan with LeafNodeWithoutStats with Product with Serializable

    A plan containing resolved non-persistent (temp or built-in) function.

47.  case class ResolvedPartitionSpec(names: Seq[String], ident: InternalRow, location: Option[String] = None) extends LeafExpression with PartitionSpec with Product with Serializable
48.  case class ResolvedPersistentFunc(catalog: FunctionCatalog, identifier: Identifier, func: UnboundFunction) extends LogicalPlan with LeafNodeWithoutStats with Product with Serializable

    A plan containing resolved persistent function.

49.  case class ResolvedStar(expressions: Seq[NamedExpression]) extends Star with Unevaluable with Product with Serializable

    Represents all the resolved input attributes to a given relational operator.

    Represents all the resolved input attributes to a given relational operator. This is used in the data frame DSL.

    expressions

    Expressions to expand.

50.  case class ResolvedTable(catalog: TableCatalog, identifier: Identifier, table: Table, outputAttributes: Seq[Attribute]) extends LogicalPlan with LeafNodeWithoutStats with Product with Serializable

    A plan containing resolved table.

51.  case class ResolvedView(identifier: Identifier, isTemp: Boolean) extends LogicalPlan with LeafNodeWithoutStats with Product with Serializable

    A plan containing resolved (temp) views.

52.  type Resolver = (String, String) ⇒ Boolean

    Resolver should return true if the first string refers to the same entity as the second string.

    Resolver should return true if the first string refers to the same entity as the second string. For example, by using case insensitive equality.

53.  trait RewriteRowLevelCommand extends Rule[LogicalPlan]
54.  class SimpleFunctionRegistry extends SimpleFunctionRegistryBase[Expression] with FunctionRegistry
55.  trait SimpleFunctionRegistryBase[T] extends FunctionRegistryBase[T] with Logging
56.  class SimpleTableFunctionRegistry extends SimpleFunctionRegistryBase[LogicalPlan] with TableFunctionRegistry
57.  abstract class Star extends LeafExpression with NamedExpression

    Represents all of the input attributes to a given relational operator, for example in "SELECT * FROM ...".

    Represents all of the input attributes to a given relational operator, for example in "SELECT * FROM ...". A Star gets automatically expanded during analysis.

58.  class TableAlreadyExistsException extends AnalysisException
59.  trait TableFunctionRegistry extends FunctionRegistryBase[LogicalPlan]

    A catalog for looking up table functions.

60.  case class TempResolvedColumn(child: Expression, nameParts: Seq[String]) extends UnaryExpression with Unevaluable with Product with Serializable

    An intermediate expression to hold a resolved (nested) column.

    An intermediate expression to hold a resolved (nested) column. Some rules may need to undo the column resolution and use this expression to keep the original column name.

61.  class TempTableAlreadyExistsException extends TableAlreadyExistsException
62.  sealed trait TimeTravelSpec extends AnyRef
63.  trait TypeCheckResult extends AnyRef

    Represents the result of Expression.checkInputDataTypes.

    Represents the result of Expression.checkInputDataTypes. We will throw AnalysisException in CheckAnalysis if isFailure is true.

64.  abstract class TypeCoercionBase extends AnyRef
65.  trait TypeCoercionRule extends Rule[LogicalPlan] with Logging
66.  case class UnresolvedAlias(child: Expression, aliasFunc: Option[(Expression) ⇒ String] = None) extends UnaryExpression with NamedExpression with Unevaluable with Product with Serializable

    Holds the expression that has yet to be aliased.

    Holds the expression that has yet to be aliased.

    child

    The computation that is needs to be resolved during analysis.

    aliasFunc

    The function if specified to be called to generate an alias to associate with the result of computing child

67.  case class UnresolvedAttribute(nameParts: Seq[String]) extends Attribute with Unevaluable with Product with Serializable

    Holds the name of an attribute that has yet to be resolved.

68.  case class UnresolvedDBObjectName(nameParts: Seq[String], isNamespace: Boolean) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a database object (table, view, namespace, function, etc.) that is to be created and we need to determine the catalog to store it.

    Holds the name of a database object (table, view, namespace, function, etc.) that is to be created and we need to determine the catalog to store it. It will be resolved to ResolvedDBObjectName during analysis.

69.  case class UnresolvedDeserializer(deserializer: Expression, inputAttributes: Seq[Attribute] = Nil) extends UnaryExpression with Unevaluable with NonSQLExpression with Product with Serializable

    Holds the deserializer expression and the attributes that are available during the resolution for it.

    Holds the deserializer expression and the attributes that are available during the resolution for it. Deserializer expression is a special kind of expression that is not always resolved by children output, but by given attributes, e.g. the keyDeserializer in MapGroups should be resolved by groupingAttributes instead of children output.

    deserializer

    The unresolved deserializer expression

    inputAttributes

    The input attributes used to resolve deserializer expression, can be empty if we want to resolve deserializer by children output.

70.  class UnresolvedException extends AnalysisException

    Thrown when an invalid attempt is made to access a property of a tree that has yet to be fully resolved.

71.  case class UnresolvedExtractValue(child: Expression, extraction: Expression) extends BinaryExpression with Unevaluable with Product with Serializable

    Extracts a value or values from an Expression

    Extracts a value or values from an Expression

    child

    The expression to extract value from, can be Map, Array, Struct or array of Structs.

    extraction

    The expression to describe the extraction, can be key of Map, index of Array, field name of Struct.

72.  case class UnresolvedFieldName(name: Seq[String]) extends LeafExpression with FieldName with Product with Serializable
73.  case class UnresolvedFieldPosition(position: ColumnPosition) extends LeafExpression with FieldPosition with Product with Serializable
74.  case class UnresolvedFunc(multipartIdentifier: Seq[String], commandName: String, requirePersistent: Boolean, funcTypeMismatchHint: Option[String], possibleQualifiedName: Option[Seq[String]] = None) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a function that has yet to be looked up.

    Holds the name of a function that has yet to be looked up. It will be resolved to ResolvedPersistentFunc or ResolvedNonPersistentFunc during analysis.

75.  case class UnresolvedFunction(nameParts: Seq[String], arguments: Seq[Expression], isDistinct: Boolean, filter: Option[Expression] = None, ignoreNulls: Boolean = false) extends Expression with Unevaluable with Product with Serializable
76.  case class UnresolvedGenerator(name: FunctionIdentifier, children: Seq[Expression]) extends Expression with Generator with Product with Serializable

    Represents an unresolved generator, which will be created by the parser for the org.apache.spark.sql.catalyst.plans.logical.Generate operator.

    Represents an unresolved generator, which will be created by the parser for the org.apache.spark.sql.catalyst.plans.logical.Generate operator. The analyzer will resolve this generator.

77.  case class UnresolvedHaving(havingCondition: Expression, child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Represents unresolved having clause, the child for it can be Aggregate, GroupingSets, Rollup and Cube.

    Represents unresolved having clause, the child for it can be Aggregate, GroupingSets, Rollup and Cube. It is turned by the analyzer into a Filter.

78.  case class UnresolvedInlineTable(names: Seq[String], rows: Seq[Seq[Expression]]) extends LogicalPlan with LeafNode with Product with Serializable

    An inline table that has not been resolved yet.

    An inline table that has not been resolved yet. Once resolved, it is turned by the analyzer into a org.apache.spark.sql.catalyst.plans.logical.LocalRelation.

    names

    list of column names

    rows

    expressions for the data

79.  case class UnresolvedNamespace(multipartIdentifier: Seq[String]) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a namespace that has yet to be looked up in a catalog.

    Holds the name of a namespace that has yet to be looked up in a catalog. It will be resolved to ResolvedNamespace during analysis.

80.  case class UnresolvedOrdinal(ordinal: Int) extends LeafExpression with Unevaluable with NonSQLExpression with Product with Serializable

    Represents unresolved ordinal used in order by or group by.

    Represents unresolved ordinal used in order by or group by.

    For example:

    select a from table order by 1
    select a   from table group by 1

    ordinal

    ordinal starts from 1, instead of 0

81.  case class UnresolvedPartitionSpec(spec: TablePartitionSpec, location: Option[String] = None) extends LeafExpression with PartitionSpec with Product with Serializable
82.  case class UnresolvedRegex(regexPattern: String, table: Option[String], caseSensitive: Boolean) extends Star with Unevaluable with Product with Serializable

    Represents all of the input attributes to a given relational operator, for example in "SELECT (id)?+.+ FROM ...".

    Represents all of the input attributes to a given relational operator, for example in "SELECT (id)?+.+ FROM ...".

    table

    an optional table that should be the target of the expansion. If omitted all tables' columns are produced.

83.  case class UnresolvedRelation(multipartIdentifier: Seq[String], options: CaseInsensitiveStringMap = CaseInsensitiveStringMap.empty(), isStreaming: Boolean = false) extends LogicalPlan with LeafNode with NamedRelation with Product with Serializable

    Holds the name of a relation that has yet to be looked up in a catalog.

    Holds the name of a relation that has yet to be looked up in a catalog.

    multipartIdentifier

    table name

    options

    options to scan this relation.

84.  case class UnresolvedStar(target: Option[Seq[String]]) extends Star with Unevaluable with Product with Serializable

    Represents all of the input attributes to a given relational operator, for example in "SELECT * FROM ...".

    Represents all of the input attributes to a given relational operator, for example in "SELECT * FROM ...".

    This is also used to expand structs. For example: "SELECT record.* from (SELECT struct(a,b,c) as record ...)

    target

    an optional name that should be the target of the expansion. If omitted all targets' columns are produced. This can either be a table name or struct name. This is a list of identifiers that is the path of the expansion.

85.  case class UnresolvedSubqueryColumnAliases(outputColumnNames: Seq[String], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Aliased column names resolved by positions for subquery.

    Aliased column names resolved by positions for subquery. We could add alias names for output columns in the subquery:

    // Assign alias names for output columns
    SELECT col1, col2 FROM testData AS t(col1, col2);

    outputColumnNames

    the LogicalPlan on which this subquery column aliases apply.

    child

    the logical plan of this subquery.

86.  case class UnresolvedTable(multipartIdentifier: Seq[String], commandName: String, relationTypeMismatchHint: Option[String]) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a table that has yet to be looked up in a catalog.

    Holds the name of a table that has yet to be looked up in a catalog. It will be resolved to ResolvedTable during analysis.

87.  case class UnresolvedTableOrView(multipartIdentifier: Seq[String], commandName: String, allowTempView: Boolean) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a table or view that has yet to be looked up in a catalog.

    Holds the name of a table or view that has yet to be looked up in a catalog. It will be resolved to ResolvedTable or ResolvedView during analysis.

88.  case class UnresolvedTableValuedFunction(name: FunctionIdentifier, functionArgs: Seq[Expression], outputNames: Seq[String]) extends LogicalPlan with LeafNode with Product with Serializable

    A table-valued function, e.g.

    A table-valued function, e.g.

    select id from range(10);
    
    // Assign alias names
    select t.a from range(10) t(a);

    name

    qualified name of this table-value function

    functionArgs

    list of function arguments

    outputNames

    alias names of function output columns. If these names given, an analyzer adds Project to rename the output columns.

89.  case class UnresolvedView(multipartIdentifier: Seq[String], commandName: String, allowTemp: Boolean, relationTypeMismatchHint: Option[String]) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a view that has yet to be looked up.

    Holds the name of a view that has yet to be looked up. It will be resolved to ResolvedView during analysis.

90.  sealed trait ViewType extends AnyRef

    ViewType is used to specify the expected view type when we want to create or replace a view in CreateViewStatement.