Package org.apache.polaris.service.catalog.iceberg

Class IcebergCatalog

java.lang.Object
org.apache.iceberg.BaseMetastoreCatalog
org.apache.iceberg.view.BaseMetastoreViewCatalog
org.apache.polaris.service.catalog.iceberg.IcebergCatalog
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.iceberg.catalog.Catalog, org.apache.iceberg.catalog.SupportsNamespaces, org.apache.iceberg.catalog.ViewCatalog, SupportsCredentialDelegation, SupportsNotifications
public class IcebergCatalog extends org.apache.iceberg.view.BaseMetastoreViewCatalog implements org.apache.iceberg.catalog.SupportsNamespaces, SupportsNotifications, Closeable, SupportsCredentialDelegation
Defines the relationship between PolarisEntities and Iceberg's business logic.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    class 
    IcebergCatalog.BasePolarisTableOperations
    An implementation of TableOperations that integrates with IcebergCatalog.

    Nested classes/interfaces inherited from class org.apache.iceberg.view.BaseMetastoreViewCatalog

    org.apache.iceberg.view.BaseMetastoreViewCatalog.BaseMetastoreViewCatalogTableBuilder, org.apache.iceberg.view.BaseMetastoreViewCatalog.BaseViewBuilder

    Nested classes/interfaces inherited from class org.apache.iceberg.BaseMetastoreCatalog

    org.apache.iceberg.BaseMetastoreCatalog.BaseMetastoreCatalogTableBuilder

    Nested classes/interfaces inherited from interface org.apache.iceberg.catalog.Catalog

    org.apache.iceberg.catalog.Catalog.TableBuilder

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final Predicate<Exception>
    SHOULD_RETRY_REFRESH_PREDICATE
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    IcebergCatalog(org.apache.polaris.core.persistence.PolarisEntityManager entityManager, org.apache.polaris.core.persistence.PolarisMetaStoreManager metaStoreManager, org.apache.polaris.core.context.CallContext callContext, org.apache.polaris.core.persistence.resolver.PolarisResolutionManifestCatalogView resolvedEntityView, jakarta.ws.rs.core.SecurityContext securityContext, TaskExecutor taskExecutor, FileIOFactory fileIOFactory, PolarisEventListener polarisEventListener)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.iceberg.catalog.Catalog.TableBuilder
    buildTable(org.apache.iceberg.catalog.TableIdentifier identifier, org.apache.iceberg.Schema schema)
     
    org.apache.iceberg.view.ViewBuilder
    buildView(org.apache.iceberg.catalog.TableIdentifier identifier)
     
    void
    close()
     
    void
    createNamespace(org.apache.iceberg.catalog.Namespace namespace)
     
    void
    createNamespace(org.apache.iceberg.catalog.Namespace namespace, Map<String,String> metadata)
     
    protected String
    defaultWarehouseLocation(org.apache.iceberg.catalog.TableIdentifier tableIdentifier)
     
    boolean
    dropNamespace(org.apache.iceberg.catalog.Namespace namespace)
     
    boolean
    dropTable(org.apache.iceberg.catalog.TableIdentifier tableIdentifier, boolean purge)
     
    boolean
    dropView(org.apache.iceberg.catalog.TableIdentifier identifier)
     
    org.apache.polaris.core.storage.AccessConfig
    getAccessConfig(org.apache.iceberg.catalog.TableIdentifier tableIdentifier, org.apache.iceberg.TableMetadata tableMetadata, Set<org.apache.polaris.core.storage.PolarisStorageActions> storageActions)
     
    void
    initialize(String name, Map<String,String> properties)
     
    List<org.apache.iceberg.catalog.Namespace>
    listNamespaces()
     
    List<org.apache.iceberg.catalog.Namespace>
    listNamespaces(org.apache.iceberg.catalog.Namespace namespace)
     
    org.apache.polaris.core.persistence.pagination.Page<org.apache.iceberg.catalog.Namespace>
    listNamespaces(org.apache.iceberg.catalog.Namespace namespace, String pageToken, Integer pageSize)
     
    List<org.apache.iceberg.catalog.TableIdentifier>
    listTables(org.apache.iceberg.catalog.Namespace namespace)
     
    org.apache.polaris.core.persistence.pagination.Page<org.apache.iceberg.catalog.TableIdentifier>
    listTables(org.apache.iceberg.catalog.Namespace namespace, String pageToken, Integer pageSize)
     
    List<org.apache.iceberg.catalog.TableIdentifier>
    listViews(org.apache.iceberg.catalog.Namespace namespace)
     
    org.apache.polaris.core.persistence.pagination.Page<org.apache.iceberg.catalog.TableIdentifier>
    listViews(org.apache.iceberg.catalog.Namespace namespace, String pageToken, Integer pageSize)
     
    protected org.apache.iceberg.io.FileIO
    loadFileIO(String ioImpl, Map<String,String> properties)
    Load FileIO with provided impl and properties
    Map<String,String>
    loadNamespaceMetadata(org.apache.iceberg.catalog.Namespace namespace)
     
    String
    name()
     
    boolean
    namespaceExists(org.apache.iceberg.catalog.Namespace namespace)
     
    protected org.apache.iceberg.TableOperations
    newTableOps(org.apache.iceberg.catalog.TableIdentifier tableIdentifier)
     
    org.apache.iceberg.TableOperations
    newTableOps(org.apache.iceberg.catalog.TableIdentifier tableIdentifier, boolean makeMetadataCurrentOnCommit)
     
    protected org.apache.iceberg.view.ViewOperations
    newViewOps(org.apache.iceberg.catalog.TableIdentifier identifier)
     
    protected Map<String,String>
    properties()
     
    org.apache.iceberg.Table
    registerTable(org.apache.iceberg.catalog.TableIdentifier identifier, String metadataFileLocation)
     
    boolean
    removeProperties(org.apache.iceberg.catalog.Namespace namespace, Set<String> properties)
     
    void
    renameTable(org.apache.iceberg.catalog.TableIdentifier from, org.apache.iceberg.catalog.TableIdentifier to)
     
    void
    renameView(org.apache.iceberg.catalog.TableIdentifier from, org.apache.iceberg.catalog.TableIdentifier to)
     
    boolean
    sendNotification(org.apache.iceberg.catalog.TableIdentifier identifier, org.apache.polaris.service.types.NotificationRequest notificationRequest)
     
    void
    setCatalogFileIo(org.apache.iceberg.io.FileIO fileIO)
     
    void
    setFileIOFactory(FileIOFactory newFactory)
     
    void
    setMetaStoreManager(org.apache.polaris.core.persistence.PolarisMetaStoreManager newMetaStoreManager)
     
    boolean
    setProperties(org.apache.iceberg.catalog.Namespace namespace, Map<String,String> properties)
     
    String
    transformTableLikeLocation(String specifiedTableLikeLocation)
    Based on configuration settings, for callsites that need to handle potentially setting a new base location for a TableLike entity, produces the transformed location if applicable, or else the unaltered specified location.

    Methods inherited from class org.apache.iceberg.view.BaseMetastoreViewCatalog

    loadView

    Methods inherited from class org.apache.iceberg.BaseMetastoreCatalog

    fullTableName, isValidIdentifier, isValidMetadataIdentifier, loadTable, metricsReporter, toString

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface org.apache.iceberg.catalog.Catalog

    createTable, createTable, createTable, createTable, dropTable, invalidateTable, newCreateTableTransaction, newCreateTableTransaction, newCreateTableTransaction, newCreateTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, newReplaceTableTransaction, tableExists

    Methods inherited from interface org.apache.iceberg.catalog.ViewCatalog

    invalidateView, viewExists

  • Field Details

    • SHOULD_RETRY_REFRESH_PREDICATE

      public static final Predicate<Exception> SHOULD_RETRY_REFRESH_PREDICATE

  • Constructor Details

    • IcebergCatalog

      public IcebergCatalog(org.apache.polaris.core.persistence.PolarisEntityManager entityManager, org.apache.polaris.core.persistence.PolarisMetaStoreManager metaStoreManager, org.apache.polaris.core.context.CallContext callContext, org.apache.polaris.core.persistence.resolver.PolarisResolutionManifestCatalogView resolvedEntityView, jakarta.ws.rs.core.SecurityContext securityContext, TaskExecutor taskExecutor, FileIOFactory fileIOFactory, PolarisEventListener polarisEventListener)
      Parameters:
      entityManager - provides handle to underlying PolarisMetaStoreManager with which to perform mutations on entities.
      callContext - the current CallContext
      resolvedEntityView - accessor to resolved entity paths that have been pre-vetted to ensure this catalog instance only interacts with authorized resolved paths.
      taskExecutor - Executor we use to register cleanup task handlers

  • Method Details

    • name

      public String name()
      Specified by:
      name in interface org.apache.iceberg.catalog.Catalog
      Specified by:
      name in interface org.apache.iceberg.catalog.ViewCatalog
      Overrides:
      name in class org.apache.iceberg.view.BaseMetastoreViewCatalog

    • setCatalogFileIo

      public void setCatalogFileIo(org.apache.iceberg.io.FileIO fileIO)

    • initialize

      public void initialize(String name, Map<String,String> properties)
      Specified by:
      initialize in interface org.apache.iceberg.catalog.Catalog
      Specified by:
      initialize in interface org.apache.iceberg.catalog.ViewCatalog
      Overrides:
      initialize in class org.apache.iceberg.view.BaseMetastoreViewCatalog

    • setMetaStoreManager

      public void setMetaStoreManager(org.apache.polaris.core.persistence.PolarisMetaStoreManager newMetaStoreManager)

    • properties

      protected Map<String,String> properties()
      Overrides:
      properties in class org.apache.iceberg.BaseMetastoreCatalog

    • registerTable

      public org.apache.iceberg.Table registerTable(org.apache.iceberg.catalog.TableIdentifier identifier, String metadataFileLocation)
      Specified by:
      registerTable in interface org.apache.iceberg.catalog.Catalog
      Overrides:
      registerTable in class org.apache.iceberg.BaseMetastoreCatalog

    • buildTable

      public org.apache.iceberg.catalog.Catalog.TableBuilder buildTable(org.apache.iceberg.catalog.TableIdentifier identifier, org.apache.iceberg.Schema schema)
      Specified by:
      buildTable in interface org.apache.iceberg.catalog.Catalog
      Overrides:
      buildTable in class org.apache.iceberg.view.BaseMetastoreViewCatalog

    • buildView

      public org.apache.iceberg.view.ViewBuilder buildView(org.apache.iceberg.catalog.TableIdentifier identifier)
      Specified by:
      buildView in interface org.apache.iceberg.catalog.ViewCatalog
      Overrides:
      buildView in class org.apache.iceberg.view.BaseMetastoreViewCatalog

    • newTableOps

      public org.apache.iceberg.TableOperations newTableOps(org.apache.iceberg.catalog.TableIdentifier tableIdentifier, boolean makeMetadataCurrentOnCommit)

    • newTableOps

      protected org.apache.iceberg.TableOperations newTableOps(org.apache.iceberg.catalog.TableIdentifier tableIdentifier)
      Specified by:
      newTableOps in class org.apache.iceberg.BaseMetastoreCatalog

    • defaultWarehouseLocation

      protected String defaultWarehouseLocation(org.apache.iceberg.catalog.TableIdentifier tableIdentifier)
      Specified by:
      defaultWarehouseLocation in class org.apache.iceberg.BaseMetastoreCatalog

    • dropTable

      public boolean dropTable(org.apache.iceberg.catalog.TableIdentifier tableIdentifier, boolean purge)
      Specified by:
      dropTable in interface org.apache.iceberg.catalog.Catalog

    • listTables

      public List<org.apache.iceberg.catalog.TableIdentifier> listTables(org.apache.iceberg.catalog.Namespace namespace)
      Specified by:
      listTables in interface org.apache.iceberg.catalog.Catalog

    • listTables

      public org.apache.polaris.core.persistence.pagination.Page<org.apache.iceberg.catalog.TableIdentifier> listTables(org.apache.iceberg.catalog.Namespace namespace, String pageToken, Integer pageSize)

    • renameTable

      public void renameTable(org.apache.iceberg.catalog.TableIdentifier from, org.apache.iceberg.catalog.TableIdentifier to)
      Specified by:
      renameTable in interface org.apache.iceberg.catalog.Catalog

    • createNamespace

      public void createNamespace(org.apache.iceberg.catalog.Namespace namespace)
      Specified by:
      createNamespace in interface org.apache.iceberg.catalog.SupportsNamespaces

    • createNamespace

      public void createNamespace(org.apache.iceberg.catalog.Namespace namespace, Map<String,String> metadata)
      Specified by:
      createNamespace in interface org.apache.iceberg.catalog.SupportsNamespaces

    • namespaceExists

      public boolean namespaceExists(org.apache.iceberg.catalog.Namespace namespace)
      Specified by:
      namespaceExists in interface org.apache.iceberg.catalog.SupportsNamespaces

    • dropNamespace

      public boolean dropNamespace(org.apache.iceberg.catalog.Namespace namespace) throws org.apache.iceberg.exceptions.NamespaceNotEmptyException
      Specified by:
      dropNamespace in interface org.apache.iceberg.catalog.SupportsNamespaces
      Throws:
      org.apache.iceberg.exceptions.NamespaceNotEmptyException

    • setProperties

      public boolean setProperties(org.apache.iceberg.catalog.Namespace namespace, Map<String,String> properties) throws org.apache.iceberg.exceptions.NoSuchNamespaceException
      Specified by:
      setProperties in interface org.apache.iceberg.catalog.SupportsNamespaces
      Throws:
      org.apache.iceberg.exceptions.NoSuchNamespaceException

    • removeProperties

      public boolean removeProperties(org.apache.iceberg.catalog.Namespace namespace, Set<String> properties) throws org.apache.iceberg.exceptions.NoSuchNamespaceException
      Specified by:
      removeProperties in interface org.apache.iceberg.catalog.SupportsNamespaces
      Throws:
      org.apache.iceberg.exceptions.NoSuchNamespaceException

    • loadNamespaceMetadata

      public Map<String,String> loadNamespaceMetadata(org.apache.iceberg.catalog.Namespace namespace) throws org.apache.iceberg.exceptions.NoSuchNamespaceException
      Specified by:
      loadNamespaceMetadata in interface org.apache.iceberg.catalog.SupportsNamespaces
      Throws:
      org.apache.iceberg.exceptions.NoSuchNamespaceException

    • listNamespaces

      public List<org.apache.iceberg.catalog.Namespace> listNamespaces()
      Specified by:
      listNamespaces in interface org.apache.iceberg.catalog.SupportsNamespaces

    • listNamespaces

      public List<org.apache.iceberg.catalog.Namespace> listNamespaces(org.apache.iceberg.catalog.Namespace namespace) throws org.apache.iceberg.exceptions.NoSuchNamespaceException
      Specified by:
      listNamespaces in interface org.apache.iceberg.catalog.SupportsNamespaces
      Throws:
      org.apache.iceberg.exceptions.NoSuchNamespaceException

    • listNamespaces

      public org.apache.polaris.core.persistence.pagination.Page<org.apache.iceberg.catalog.Namespace> listNamespaces(org.apache.iceberg.catalog.Namespace namespace, String pageToken, Integer pageSize)

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class org.apache.iceberg.BaseMetastoreCatalog
      Throws:
      IOException

    • listViews

      public List<org.apache.iceberg.catalog.TableIdentifier> listViews(org.apache.iceberg.catalog.Namespace namespace)
      Specified by:
      listViews in interface org.apache.iceberg.catalog.ViewCatalog

    • listViews

      public org.apache.polaris.core.persistence.pagination.Page<org.apache.iceberg.catalog.TableIdentifier> listViews(org.apache.iceberg.catalog.Namespace namespace, String pageToken, Integer pageSize)

    • newViewOps

      protected org.apache.iceberg.view.ViewOperations newViewOps(org.apache.iceberg.catalog.TableIdentifier identifier)
      Specified by:
      newViewOps in class org.apache.iceberg.view.BaseMetastoreViewCatalog

    • dropView

      public boolean dropView(org.apache.iceberg.catalog.TableIdentifier identifier)
      Specified by:
      dropView in interface org.apache.iceberg.catalog.ViewCatalog

    • renameView

      public void renameView(org.apache.iceberg.catalog.TableIdentifier from, org.apache.iceberg.catalog.TableIdentifier to)
      Specified by:
      renameView in interface org.apache.iceberg.catalog.ViewCatalog

    • sendNotification

      public boolean sendNotification(org.apache.iceberg.catalog.TableIdentifier identifier, org.apache.polaris.service.types.NotificationRequest notificationRequest)
      Specified by:
      sendNotification in interface SupportsNotifications

    • getAccessConfig

      public org.apache.polaris.core.storage.AccessConfig getAccessConfig(org.apache.iceberg.catalog.TableIdentifier tableIdentifier, org.apache.iceberg.TableMetadata tableMetadata, Set<org.apache.polaris.core.storage.PolarisStorageActions> storageActions)
      Specified by:
      getAccessConfig in interface SupportsCredentialDelegation

    • transformTableLikeLocation

      public String transformTableLikeLocation(String specifiedTableLikeLocation)
      Based on configuration settings, for callsites that need to handle potentially setting a new base location for a TableLike entity, produces the transformed location if applicable, or else the unaltered specified location.

    • setFileIOFactory

      public void setFileIOFactory(FileIOFactory newFactory)

    • loadFileIO

      protected org.apache.iceberg.io.FileIO loadFileIO(String ioImpl, Map<String,String> properties)
      Load FileIO with provided impl and properties
      Parameters:
      ioImpl - full class name of a custom FileIO implementation
      properties - used to initialize the FileIO implementation
      Returns:
      FileIO object