Package org.apache.polaris.service.it.ext

Class SparkSessionBuilder

java.lang.Object

org.apache.polaris.service.it.ext.SparkSessionBuilder

public class SparkSessionBuilder
extends Object

A fluent builder for configuring SparkSession instances with Polaris catalogs.

This builder creates a SparkSession with sensible test defaults and allows easy configuration of multiple Polaris catalogs. The resulting SparkSession will be configured for local execution with S3Mock support for testing.

Example usage:

 SparkSession session = SparkSessionBuilder
     .buildWithTestDefaults()
     .withExtensions("org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions")
     .withWarehouse(warehouseUri)
     .addCatalog("catalog1", "org.apache.iceberg.spark.SparkCatalog", endpoints, token)
     .addCatalog("catalog2", "org.apache.polaris.spark.SparkCatalog", endpoints, token)
     .createSession();
 

The final SparkSession will be configured with:

• Local master execution (local[1])
• Disabled Spark UI for clean test output
• S3A filesystem with mock credentials (foo/bar)
• Multiple Polaris catalogs with REST endpoints
• Iceberg extensions for table format support
• Custom warehouse directory location

Each catalog will be configured as:

 spark.sql.catalog.{catalogName} = {catalogType}
 spark.sql.catalog.{catalogName}.type = rest
 spark.sql.catalog.{catalogName}.uri = {polarisEndpoint}
 spark.sql.catalog.{catalogName}.token = {authToken}
 spark.sql.catalog.{catalogName}.warehouse = {catalogName}
 spark.sql.catalog.{catalogName}.scope = PRINCIPAL_ROLE:ALL
 

Method Summary

Modifier and Type	Method	Description
SparkSessionBuilder	addCatalog(String catalogName, String catalogImplClass, PolarisApiEndpoints endpoints, String token)
static SparkSessionBuilder	buildWithTestDefaults()	Create a SparkSessionBuilder with common test defaults
org.apache.spark.sql.SparkSession	getOrCreate()
SparkSessionBuilder	withConfig(String key, String value)
SparkSessionBuilder	withExtensions(String extensions)
SparkSessionBuilder	withWarehouse(URI warehouseDir)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

buildWithTestDefaults

public static SparkSessionBuilder buildWithTestDefaults()

Create a SparkSessionBuilder with common test defaults

Returns:

new builder instance with test defaults

withWarehouse

public SparkSessionBuilder withWarehouse(URI warehouseDir)

withExtensions

public SparkSessionBuilder withExtensions(String extensions)

addCatalog

public SparkSessionBuilder addCatalog(String catalogName,
 String catalogImplClass,
 PolarisApiEndpoints endpoints,
 String token)

withConfig

public SparkSessionBuilder withConfig(String key,
 String value)

getOrCreate

public org.apache.spark.sql.SparkSession getOrCreate()