Type Members

1.  case class AnalysisContext(catalogAndNamespace: Seq[String] = Nil, nestedViewDepth: Int = 0, maxNestedViewDepth: Int = -1, relationCache: Map[Seq[String], LogicalPlan] = mutable.Map.empty, referredTempViewNames: Seq[Seq[String]] = Seq.empty, referredTempFunctionNames: Seq[String] = Seq.empty, outerPlan: Option[LogicalPlan] = None) extends Product with Serializable

   Provides a way to keep state during the analysis, mostly for resolving views and subqueries.

   Provides a way to keep state during the analysis, mostly for resolving views and subqueries. This enables us to decouple the concerns of analysis environment from the catalog and resolve star expressions in subqueries that reference the outer query plans. The state that is kept here is per-query.

   Note this is thread local.

   catalogAndNamespace

   The catalog and namespace used in the view resolution. This overrides the current catalog and namespace when resolving relations inside views.

   nestedViewDepth

   The nested depth in the view resolution, this enables us to limit the depth of nested views.

   maxNestedViewDepth

   The maximum allowed depth of nested view resolution.

   relationCache

   A mapping from qualified table names to resolved relations. This can ensure that the table is resolved only once if a table is used multiple times in a query.

   referredTempViewNames

   All the temp view names referred by the current view we are resolving. It's used to make sure the relation resolution is consistent between view creation and view resolution. For example, if t was a permanent table when the current view was created, it should still be a permanent table when resolving the current view, even if a temp view t has been created.

   outerPlan

   The query plan from the outer query that can be used to resolve star expressions in a subquery.

2.  implicit class AnalysisErrorAt extends AnyRef
3.  class Analyzer extends RuleExecutor[LogicalPlan] with CheckAnalysis with SQLConfHelper

   Provides a logical query plan analyzer, which translates UnresolvedAttributes and UnresolvedRelations into fully typed objects using information in a SessionCatalog.

4.  class CannotReplaceMissingTableException extends AnalysisException
5.  trait CastSupport extends AnyRef

   Mix-in trait for constructing valid Cast expressions.

6.  trait CheckAnalysis extends PredicateHelper with LookupCatalog

   Throws user facing errors when passed invalid queries that fail to analyze.

7.  class DatabaseAlreadyExistsException extends NamespaceAlreadyExistsException

   Thrown by a catalog when an item already exists.

   Thrown by a catalog when an item already exists. The analyzer will rethrow the exception as an org.apache.spark.sql.AnalysisException with the correct position information.

8.  trait EmptyFunctionRegistryBase[T] extends FunctionRegistryBase[T]

   A trivial catalog that returns an error when a function is requested.

   A trivial catalog that returns an error when a function is requested. Used for testing when all functions are already filled in and the analyzer needs only to resolve attribute references.

9.  sealed trait FieldName extends LeafExpression with Unevaluable
10.  sealed trait FieldPosition extends LeafExpression with Unevaluable
11.  class FunctionAlreadyExistsException extends AnalysisException
12.  trait FunctionRegistry extends FunctionRegistryBase[Expression]
13.  trait FunctionRegistryBase[T] extends AnyRef

    A catalog for looking up user defined functions, used by an Analyzer.

    A catalog for looking up user defined functions, used by an Analyzer.

    Note: 1) The implementation should be thread-safe to allow concurrent access. 2) the database name is always case-sensitive here, callers are responsible to format the database name w.r.t. case-sensitive config.

14.  case class GetColumnByOrdinal(ordinal: Int, dataType: DataType) extends LeafExpression with Unevaluable with NonSQLExpression with Product with Serializable
15.  case class GetViewColumnByNameAndOrdinal(viewName: String, colName: String, ordinal: Int, expectedNumCandidates: Int, viewDDL: Option[String]) extends LeafExpression with Unevaluable with NonSQLExpression with Product with Serializable
16.  case class MultiAlias(child: Expression, names: Seq[String]) extends UnaryExpression with NamedExpression with Unevaluable with Product with Serializable

    Used to assign new names to Generator's output, such as hive udtf.

    Used to assign new names to Generator's output, such as hive udtf. For example the SQL expression "stack(2, key, value, key, value) as (a, b)" could be represented as follows: MultiAlias(stack_function, Seq(a, b))

    child

    the computation being performed

    names

    the names to be associated with each output of computing child.

17.  trait MultiInstanceRelation extends AnyRef

    A trait that should be mixed into query operators where a single instance might appear multiple times in a logical query plan.

    A trait that should be mixed into query operators where a single instance might appear multiple times in a logical query plan. It is invalid to have multiple copies of the same attribute produced by distinct operators in a query tree as this breaks the guarantee that expression ids, which are used to differentiate attributes, are unique.

    During analysis, operators that include this trait may be asked to produce a new version of itself with globally unique expression ids.

18.  trait NamedRelation extends LogicalPlan
19.  class NamespaceAlreadyExistsException extends AnalysisException
20.  class NoSuchDatabaseException extends NoSuchNamespaceException

    Thrown by a catalog when an item cannot be found.

    Thrown by a catalog when an item cannot be found. The analyzer will rethrow the exception as an org.apache.spark.sql.AnalysisException with the correct position information.

21.  class NoSuchFunctionException extends AnalysisException
22.  class NoSuchNamespaceException extends AnalysisException
23.  class NoSuchPartitionException extends AnalysisException
24.  class NoSuchPartitionsException extends AnalysisException
25.  class NoSuchPermanentFunctionException extends AnalysisException
26.  class NoSuchTableException extends AnalysisException
27.  class NoSuchTempFunctionException extends AnalysisException
28.  class PartitionAlreadyExistsException extends AnalysisException
29.  sealed trait PartitionSpec extends LeafExpression with Unevaluable
30.  class PartitionsAlreadyExistException extends AnalysisException
31.  case class ReferenceEqualPlanWrapper(plan: LogicalPlan) extends Product with Serializable

    A helper class used to detect duplicate relations fast in DeduplicateRelations

32.  class ResolveCatalogs extends Rule[LogicalPlan] with LookupCatalog

    Resolves catalogs from the multi-part identifiers in SQL statements, and convert the statements to the corresponding v2 commands if the resolved catalog is not the session catalog.

33.  case class ResolveHigherOrderFunctions(catalogManager: CatalogManager) extends Rule[LogicalPlan] with LookupCatalog with Product with Serializable

    Resolve a higher order functions from the catalog.

    Resolve a higher order functions from the catalog. This is different from regular function resolution because lambda functions can only be resolved after the function has been resolved; so we need to resolve higher order function when all children are either resolved or a lambda function.

34.  case class ResolveTableValuedFunctions(catalog: SessionCatalog) extends Rule[LogicalPlan] with Product with Serializable

    Rule that resolves table-valued function references.

35.  case class ResolvedFieldName(path: Seq[String], field: StructField) extends LeafExpression with FieldName with Product with Serializable
36.  case class ResolvedFieldPosition(position: ColumnPosition) extends LeafExpression with FieldPosition with Product with Serializable
37.  case class ResolvedFunc(identifier: Identifier) extends LogicalPlan with LeafNode with Product with Serializable

    A plan containing resolved function.

38.  case class ResolvedNamespace(catalog: CatalogPlugin, namespace: Seq[String]) extends LogicalPlan with LeafNode with Product with Serializable

    A plan containing resolved namespace.

39.  case class ResolvedPartitionSpec(names: Seq[String], ident: InternalRow, location: Option[String] = None) extends LeafExpression with PartitionSpec with Product with Serializable
40.  case class ResolvedStar(expressions: Seq[NamedExpression]) extends Star with Unevaluable with Product with Serializable

    Represents all the resolved input attributes to a given relational operator.

    Represents all the resolved input attributes to a given relational operator. This is used in the data frame DSL.

    expressions

    Expressions to expand.

41.  case class ResolvedTable(catalog: TableCatalog, identifier: Identifier, table: Table, outputAttributes: Seq[Attribute]) extends LogicalPlan with LeafNode with Product with Serializable

    A plan containing resolved table.

42.  case class ResolvedView(identifier: Identifier, isTemp: Boolean) extends LogicalPlan with LeafNode with Product with Serializable

    A plan containing resolved (temp) views.

43.  type Resolver = (String, String) ⇒ Boolean

    Resolver should return true if the first string refers to the same entity as the second string.

    Resolver should return true if the first string refers to the same entity as the second string. For example, by using case insensitive equality.

44.  class SimpleFunctionRegistry extends SimpleFunctionRegistryBase[Expression] with FunctionRegistry
45.  trait SimpleFunctionRegistryBase[T] extends FunctionRegistryBase[T] with Logging
46.  class SimpleTableFunctionRegistry extends SimpleFunctionRegistryBase[LogicalPlan] with TableFunctionRegistry
47.  abstract class Star extends LeafExpression with NamedExpression

    Represents all of the input attributes to a given relational operator, for example in "SELECT * FROM ...".

    Represents all of the input attributes to a given relational operator, for example in "SELECT * FROM ...". A Star gets automatically expanded during analysis.

48.  class TableAlreadyExistsException extends AnalysisException
49.  trait TableFunctionRegistry extends FunctionRegistryBase[LogicalPlan]

    A catalog for looking up table functions.

50.  case class TempResolvedColumn(child: Expression, nameParts: Seq[String]) extends UnaryExpression with Unevaluable with Product with Serializable

    An intermediate expression to hold a resolved (nested) column.

    An intermediate expression to hold a resolved (nested) column. Some rules may need to undo the column resolution and use this expression to keep the original column name.

51.  class TempTableAlreadyExistsException extends TableAlreadyExistsException
52.  trait TypeCheckResult extends AnyRef

    Represents the result of Expression.checkInputDataTypes.

    Represents the result of Expression.checkInputDataTypes. We will throw AnalysisException in CheckAnalysis if isFailure is true.

53.  abstract class TypeCoercionBase extends AnyRef
54.  trait TypeCoercionRule extends Rule[LogicalPlan] with Logging
55.  case class UnresolvedAlias(child: Expression, aliasFunc: Option[(Expression) ⇒ String] = None) extends UnaryExpression with NamedExpression with Unevaluable with Product with Serializable

    Holds the expression that has yet to be aliased.

    Holds the expression that has yet to be aliased.

    child

    The computation that is needs to be resolved during analysis.

    aliasFunc

    The function if specified to be called to generate an alias to associate with the result of computing child

56.  case class UnresolvedAttribute(nameParts: Seq[String]) extends Attribute with Unevaluable with Product with Serializable

    Holds the name of an attribute that has yet to be resolved.

57.  case class UnresolvedDeserializer(deserializer: Expression, inputAttributes: Seq[Attribute] = Nil) extends UnaryExpression with Unevaluable with NonSQLExpression with Product with Serializable

    Holds the deserializer expression and the attributes that are available during the resolution for it.

    Holds the deserializer expression and the attributes that are available during the resolution for it. Deserializer expression is a special kind of expression that is not always resolved by children output, but by given attributes, e.g. the keyDeserializer in MapGroups should be resolved by groupingAttributes instead of children output.

    deserializer

    The unresolved deserializer expression

    inputAttributes

    The input attributes used to resolve deserializer expression, can be empty if we want to resolve deserializer by children output.

58.  class UnresolvedException extends AnalysisException

    Thrown when an invalid attempt is made to access a property of a tree that has yet to be fully resolved.

59.  case class UnresolvedExtractValue(child: Expression, extraction: Expression) extends BinaryExpression with Unevaluable with Product with Serializable

    Extracts a value or values from an Expression

    Extracts a value or values from an Expression

    child

    The expression to extract value from, can be Map, Array, Struct or array of Structs.

    extraction

    The expression to describe the extraction, can be key of Map, index of Array, field name of Struct.

60.  case class UnresolvedFieldName(name: Seq[String]) extends LeafExpression with FieldName with Product with Serializable
61.  case class UnresolvedFieldPosition(position: ColumnPosition) extends LeafExpression with FieldPosition with Product with Serializable
62.  case class UnresolvedFunc(multipartIdentifier: Seq[String]) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a function that has yet to be looked up in a catalog.

    Holds the name of a function that has yet to be looked up in a catalog. It will be resolved to ResolvedFunc during analysis.

63.  case class UnresolvedFunction(nameParts: Seq[String], arguments: Seq[Expression], isDistinct: Boolean, filter: Option[Expression] = None, ignoreNulls: Boolean = false) extends Expression with Unevaluable with Product with Serializable
64.  case class UnresolvedGenerator(name: FunctionIdentifier, children: Seq[Expression]) extends Expression with Generator with Product with Serializable

    Represents an unresolved generator, which will be created by the parser for the org.apache.spark.sql.catalyst.plans.logical.Generate operator.

    Represents an unresolved generator, which will be created by the parser for the org.apache.spark.sql.catalyst.plans.logical.Generate operator. The analyzer will resolve this generator.

65.  case class UnresolvedHaving(havingCondition: Expression, child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Represents unresolved having clause, the child for it can be Aggregate, GroupingSets, Rollup and Cube.

    Represents unresolved having clause, the child for it can be Aggregate, GroupingSets, Rollup and Cube. It is turned by the analyzer into a Filter.

66.  case class UnresolvedInlineTable(names: Seq[String], rows: Seq[Seq[Expression]]) extends LogicalPlan with LeafNode with Product with Serializable

    An inline table that has not been resolved yet.

    An inline table that has not been resolved yet. Once resolved, it is turned by the analyzer into a org.apache.spark.sql.catalyst.plans.logical.LocalRelation.

    names

    list of column names

    rows

    expressions for the data

67.  case class UnresolvedNamespace(multipartIdentifier: Seq[String]) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a namespace that has yet to be looked up in a catalog.

    Holds the name of a namespace that has yet to be looked up in a catalog. It will be resolved to ResolvedNamespace during analysis.

68.  case class UnresolvedOrdinal(ordinal: Int) extends LeafExpression with Unevaluable with NonSQLExpression with Product with Serializable

    Represents unresolved ordinal used in order by or group by.

    Represents unresolved ordinal used in order by or group by.

    For example:

    select a from table order by 1
    select a   from table group by 1

    ordinal

    ordinal starts from 1, instead of 0

69.  case class UnresolvedPartitionSpec(spec: TablePartitionSpec, location: Option[String] = None) extends LeafExpression with PartitionSpec with Product with Serializable
70.  case class UnresolvedRegex(regexPattern: String, table: Option[String], caseSensitive: Boolean) extends Star with Unevaluable with Product with Serializable

    Represents all of the input attributes to a given relational operator, for example in "SELECT (id)?+.+ FROM ...".

    Represents all of the input attributes to a given relational operator, for example in "SELECT (id)?+.+ FROM ...".

    table

    an optional table that should be the target of the expansion. If omitted all tables' columns are produced.

71.  case class UnresolvedRelation(multipartIdentifier: Seq[String], options: CaseInsensitiveStringMap = CaseInsensitiveStringMap.empty(), isStreaming: Boolean = false) extends LogicalPlan with LeafNode with NamedRelation with Product with Serializable

    Holds the name of a relation that has yet to be looked up in a catalog.

    Holds the name of a relation that has yet to be looked up in a catalog.

    multipartIdentifier

    table name

    options

    options to scan this relation.

72.  case class UnresolvedStar(target: Option[Seq[String]]) extends Star with Unevaluable with Product with Serializable

    Represents all of the input attributes to a given relational operator, for example in "SELECT * FROM ...".

    Represents all of the input attributes to a given relational operator, for example in "SELECT * FROM ...".

    This is also used to expand structs. For example: "SELECT record.* from (SELECT struct(a,b,c) as record ...)

    target

    an optional name that should be the target of the expansion. If omitted all targets' columns are produced. This can either be a table name or struct name. This is a list of identifiers that is the path of the expansion.

73.  case class UnresolvedSubqueryColumnAliases(outputColumnNames: Seq[String], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Aliased column names resolved by positions for subquery.

    Aliased column names resolved by positions for subquery. We could add alias names for output columns in the subquery:

    // Assign alias names for output columns
    SELECT col1, col2 FROM testData AS t(col1, col2);

    outputColumnNames

    the LogicalPlan on which this subquery column aliases apply.

    child

    the logical plan of this subquery.

74.  case class UnresolvedTable(multipartIdentifier: Seq[String], commandName: String, relationTypeMismatchHint: Option[String]) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a table that has yet to be looked up in a catalog.

    Holds the name of a table that has yet to be looked up in a catalog. It will be resolved to ResolvedTable during analysis.

75.  case class UnresolvedTableOrView(multipartIdentifier: Seq[String], commandName: String, allowTempView: Boolean) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a table or view that has yet to be looked up in a catalog.

    Holds the name of a table or view that has yet to be looked up in a catalog. It will be resolved to ResolvedTable or ResolvedView during analysis.

76.  case class UnresolvedTableValuedFunction(name: FunctionIdentifier, functionArgs: Seq[Expression], outputNames: Seq[String]) extends LogicalPlan with LeafNode with Product with Serializable

    A table-valued function, e.g.

    A table-valued function, e.g.

    select id from range(10);
    
    // Assign alias names
    select t.a from range(10) t(a);

    name

    qualified name of this table-value function

    functionArgs

    list of function arguments

    outputNames

    alias names of function output columns. If these names given, an analyzer adds Project to rename the output columns.

77.  case class UnresolvedV2Relation(originalNameParts: Seq[String], catalog: TableCatalog, tableName: Identifier) extends LogicalPlan with LeafNode with NamedRelation with Product with Serializable

    A variant of UnresolvedRelation which can only be resolved to a v2 relation (DataSourceV2Relation), not v1 relation or temp view.

    A variant of UnresolvedRelation which can only be resolved to a v2 relation (DataSourceV2Relation), not v1 relation or temp view.

    originalNameParts

    the original table identifier name parts before catalog is resolved.

    catalog

    The catalog which the table should be looked up from.

    tableName

    The name of the table to look up.

78.  case class UnresolvedView(multipartIdentifier: Seq[String], commandName: String, allowTemp: Boolean, relationTypeMismatchHint: Option[String]) extends LogicalPlan with LeafNode with Product with Serializable

    Holds the name of a view that has yet to be looked up in a catalog.

    Holds the name of a view that has yet to be looked up in a catalog. It will be resolved to ResolvedView during analysis.

79.  sealed trait ViewType extends AnyRef

    ViewType is used to specify the expected view type when we want to create or replace a view in CreateViewStatement.