SupportsPartitionManagement

trait SupportsPartitionManagement extends Table

A partition interface of Table. A partition is composed of identifier and properties, and properties contains metadata information of the partition.

These APIs are used to modify table partition identifier or partition metadata. In some cases, they will change the table data as well.

#createPartition: add a partition and any data it contains to the table
#dropPartition: remove a partition and any data it contains from the table
#purgePartition: remove a partition and any data it contains from the table by skipping a trash even if it is supported.
#replacePartitionMetadata: point a partition to a new location, which will swap one location's data for the other
#truncatePartition: remove partition data from the table

Annotations: @Experimental()
Since: 3.1.0

Linear Supertypes

Table, AnyRef, Any

Known Subclasses

SupportsAtomicPartitionManagement

Ordering

Alphabetic
By Inheritance

Inherited

SupportsPartitionManagement
Table
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Abstract Value Members

abstract def capabilities(): Set[TableCapability]
Returns the set of capabilities for this table.
Returns the set of capabilities for this table.

Definition Classes
Table
abstract def createPartition(ident: InternalRow, properties: Map[String, String]): Unit
Create a partition in table.
Create a partition in table.
ident
a new partition identifier
properties
the metadata of a partition

Exceptions thrown
PartitionAlreadyExistsException If a partition already exists for the identifier
UnsupportedOperationException If partition property is not supported
abstract def dropPartition(ident: InternalRow): Boolean
Drop a partition from table.
Drop a partition from table.
ident
a partition identifier
returns
true if a partition was deleted, false if no partition exists for the identifier
abstract def listPartitionIdentifiers(names: Array[String], ident: InternalRow): Array[InternalRow]
List the identifiers of all partitions that match to the ident by names.
List the identifiers of all partitions that match to the ident by names.
names
the names of partition values in the identifier.
ident
a partition identifier values.
returns
an array of Identifiers for the partitions
abstract def loadPartitionMetadata(ident: InternalRow): Map[String, String]
Retrieve the partition metadata of the existing partition.
Retrieve the partition metadata of the existing partition.
ident
a partition identifier
returns
the metadata of the partition

Exceptions thrown
UnsupportedOperationException If partition property is not supported
abstract def name(): String
A name to identify this table.
A name to identify this table. Implementations should provide a meaningful name, like the database and table name from catalog, or the location of files for this table.

Definition Classes
Table
abstract def partitionSchema(): StructType
Get the partition schema of table, this must be consistent with $Table#partitioning().
Get the partition schema of table, this must be consistent with $Table#partitioning().
returns
the partition schema of table
abstract def replacePartitionMetadata(ident: InternalRow, properties: Map[String, String]): Unit
Replace the partition metadata of the existing partition.
Replace the partition metadata of the existing partition.
ident
the partition identifier of the existing partition
properties
the new metadata of the partition

Exceptions thrown
NoSuchPartitionException If the partition identifier to alter doesn't exist
UnsupportedOperationException If partition property is not supported
abstract def schema(): StructType
Returns the schema of this table.
Returns the schema of this table. If the table is not readable and doesn't have a schema, an empty schema can be returned here.

Definition Classes
Table

Concrete Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native() @HotSpotIntrinsicCandidate()
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native() @HotSpotIntrinsicCandidate()
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native() @HotSpotIntrinsicCandidate()
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native() @HotSpotIntrinsicCandidate()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native() @HotSpotIntrinsicCandidate()
def partitionExists(ident: InternalRow): Boolean
Test whether a partition exists using an ident from the table.
Test whether a partition exists using an ident from the table.
ident
a partition identifier which must contain all partition fields in order
returns
true if the partition exists, false otherwise
def partitioning(): Array[Transform]
Returns the physical partitioning of this table.
Returns the physical partitioning of this table.

Definition Classes
Table
def properties(): Map[String, String]
Returns the string map of table properties.
Returns the string map of table properties.

Definition Classes
Table
def purgePartition(ident: InternalRow): Boolean
Drop a partition from the table and completely remove partition data by skipping a trash even if it is supported.
Drop a partition from the table and completely remove partition data by skipping a trash even if it is supported.
ident
a partition identifier
returns
true if a partition was deleted, false if no partition exists for the identifier

Since
3.2.0
Exceptions thrown
NoSuchPartitionException If the partition identifier to alter doesn't exist
UnsupportedOperationException If partition purging is not supported
def renamePartition(from: InternalRow, to: InternalRow): Boolean
Rename an existing partition of the table.
Rename an existing partition of the table.
from
an existing partition identifier to rename
to
new partition identifier
returns
true if renaming completes successfully otherwise false

Since
3.2.0
Exceptions thrown
NoSuchPartitionException If the from partition does not exist
PartitionAlreadyExistsException If the to partition exists already
UnsupportedOperationException If partition renaming is not supported
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toString(): String

Definition Classes
AnyRef → Any
def truncatePartition(ident: InternalRow): Boolean
Truncate a partition in the table by completely removing partition data.
Truncate a partition in the table by completely removing partition data.
ident
a partition identifier
returns
true if the partition was truncated successfully otherwise false

Since
3.2.0
Exceptions thrown
NoSuchPartitionException If the partition identifier to alter doesn't exist
UnsupportedOperationException If partition truncation is not supported
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Deprecated Value Members

def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] ) @Deprecated
Deprecated

Packages

SupportsPartitionManagement

trait SupportsPartitionManagement extends Table

Abstract Value Members

Concrete Value Members

Deprecated Value Members

Inherited from Table

Inherited from AnyRef

Inherited from Any

Ungrouped

Packages

SupportsPartitionManagement 

trait SupportsPartitionManagement extends Table

Abstract Value Members

Concrete Value Members

Deprecated Value Members

Inherited from Table

Inherited from AnyRef

Inherited from Any

Ungrouped

SupportsPartitionManagement