Package org.apache.polaris.persistence.nosql.api

Interface Persistence

public interface Persistence
Polaris NoSQL persistence interface providing fundamental primitive operations to manage named-references including atomic updates and to read and write Objs. Batch operations are provided where applicable.

Objs are usually only written but never updated. This enables efficient caching of persisted data. In certain, exceptional use cases, which should always almost be avoided, CAS primitives allow conditional creates/updates/deletes. ObjType implementations can provide custom positive and negative caching rules.

Databases often have hard limits or at least more-or-less strong recommendations on the size of serialized Objs. The "main" implementation of this interface in :polaris-persistence-nosql-impl takes care of transparently splitting and re-assembling Objs across multiple database rows. The latter is not supported for conditionally updated Objs.

This interface is a Polaris-internal low-level API interface for NoSQL. Instances of this interface are scoped to a specific realm.

The behavior when fetching a non-existing reference is to throw, which is different from fetching non-existing Objs, because references are supposed to exist and a non-existence is usually a sign of a missing initialization step, whereas a missing Obj is often expected.

Database-specific implementations do implement the Backend interface, not this one.

  • Method Details

    • createReference

      @Nonnull Reference createReference(@Nonnull String name, @Nonnull Optional<ObjRef> pointer) throws ReferenceAlreadyExistsException
      Creates the reference with the given name and pointer value.

      Reference creation is always a strongly consistent operation.

      Throws:
      ReferenceAlreadyExistsException - if a reference with the same name already exists

    • createReferenceSilent

      default void createReferenceSilent(@Nonnull String name)
      Convenience function to create a reference with an empty pointer, if it does not already exist.
      See Also:

    • createReferencesSilent

      void createReferencesSilent(Set<String> referenceNames)
      Ensures that multiple references exist, leveraging bulk operations, if possible. References are created with empty pointers.

      This whole operation is not guaranteed to be atomic, the creation of each reference is atomic.

      See Also:

    • fetchOrCreateReference

      @Nonnull default Reference fetchOrCreateReference(@Nonnull String name, @Nonnull Supplier<Optional<ObjRef>> pointerForCreate)
      Convenience function to return an existing reference or to create the reference with a supplied pointer, if it does not already exist.

    • updateReferencePointer

      @Nonnull Optional<Reference> updateReferencePointer(@Nonnull Reference reference, @Nonnull ObjRef newPointer) throws ReferenceNotFoundException
      Updates the pointer to newPointer, if the reference exists and the current persisted pointer is the same as in reference.

      Reference update is always a strongly consistent operation.

      Parameters:
      reference - the existing reference including the expected pointer
      newPointer - the pointer to update the reference to. If the reference has a current pointer value, both the current and the new pointer must use the same ObjType.
      Returns:
      If the reference was successfully updated, an updated Reference instances will be returned.
      Throws:
      ReferenceNotFoundException - if the reference does not exist

    • fetchReference

      @Nonnull Reference fetchReference(@Nonnull String name) throws ReferenceNotFoundException
      Fetch the reference with the given name, leveraging the reference cache.
      Throws:
      ReferenceNotFoundException - if the reference does not exist
      See Also:

    • fetchReferenceForUpdate

      @Nonnull default Reference fetchReferenceForUpdate(@Nonnull String name) throws ReferenceNotFoundException
      Fetches the reference with the given name, but will always fetch the most recent state from the backend database.
      Throws:
      ReferenceNotFoundException
      See Also:

    • fetchReferenceHead

      default <T extends Obj> Optional<T> fetchReferenceHead(@Nonnull String name, @Nonnull Class<T> clazz) throws ReferenceNotFoundException
      Convenience function to return the Obj as pointed to from the reference with the given name.
      Throws:
      ReferenceNotFoundException
      See Also:

    • fetch

      @Nullable <T extends Obj> T fetch(@Nonnull ObjRef id, @Nonnull Class<T> clazz)
      Fetch the objects for the given object Ids.

      Supports assembling object splits across multiple rows by write(Obj, Class) or writeMany(Class, Obj[]).

      Type Parameters:
      T - returned type can also be just Obj
      Parameters:
      id - ID of the object to load
      clazz - expected Obj subtype, passing Obj.class is fine
      Returns:
      loaded object or null if it does not exist
      See Also:

    • fetchMany

      @Nonnull <T extends Obj> T[] fetchMany(@Nonnull Class<T> clazz, @Nonnull ObjRef... ids)
      Fetch multiple objects for the given object Ids.

      Supports assembling object splits across multiple rows by write(Obj, Class) or writeMany(Class, Obj[]).

      Type Parameters:
      T - returned type can also be just Obj
      Parameters:
      clazz - expected Obj subtype, passing Obj.class is fine
      ids - ID of the object to load, callers must ensure that the IDs are not duplicated within the array
      Returns:
      array of the same length as ids containing the loaded objects, with null elements for objects that do not exist
      See Also:

    • write

      @Nonnull <T extends Obj> T write(@Nonnull T obj, @Nonnull Class<T> clazz)
      Persist obj with eventually consistent guarantees.

      Supports splitting the serialized representation across multiple rows in the backend database, if the serialized representation does not fit entirely in a single row, limited by maxSerializedValueSize().

      This function (and writeMany(Class, Obj[])) are not meant to actually update existing objects with different information, especially not when the size of the serialized object changes the number of splits in the backend database. Note that there is no protection against this scenario.

      Returns:
      obj with the Obj.createdAtMicros() and Obj.numParts() fields updated
      See Also:

    • writeMany

      @Nonnull <T extends Obj> T[] writeMany(@Nonnull Class<T> clazz, @Nonnull T... objs)
      Persist multiple objs with eventually consistent guarantees.

      See write(Obj, Class) for more information.

      Supports splitting the serialized representation across multiple rows in the backend database, if the serialized representation does not fit entirely in a single row, limited by maxSerializedValueSize().

      This function and write(Obj, Class) are not meant to actually update existing objects with different information, especially not when the size of the serialized object changes the number of splits in the backend database. Note that there is no protection against this scenario.

      Returns:
      objs with the Obj.createdAtMicros() and Obj.numParts() fields updated, callers must ensure that the IDs are not duplicated within the array. null elements in the returned array will appear for null elements in the objs array.
      See Also:

    • delete

      void delete(@Nonnull ObjRef id)
      Unconditionally delete the object with the given id.

      Note that it is generally not advised to actively (or prematurely) delete objects. In general, it is better to just leave the object and let the maintenance service take care of purging it.

      If the object has been split across multiple database rows, only the number of parts mentioned in ObjRef.numParts() will be deleted. However, the maintenance service will take care of purging possibly left-over parts.

      See Also:

    • deleteMany

      void deleteMany(@Nonnull ObjRef... ids)
      Unconditionally delete the objects with the given ids.

      Note that it is generally not advised to actively (or prematurely) delete objects. In general, it is better to just leave the object and let the maintenance service take care of purging it.

      If the object has been split across multiple database rows, only the number of parts mentioned in ObjRef.numParts() will be deleted. However, the maintenance service will take care of purging possibly left-over parts.

      Parameters:
      ids - IDs of objects to delete, callers must ensure that the IDs are not duplicated within the array
      See Also:

    • conditionalInsert

      @Nullable <T extends Obj> T conditionalInsert(@Nonnull T obj, @Nonnull Class<T> clazz)
      Persist obj with strong consistent guarantees.

      Unlike eventually consistent writes, conditional write operations do not support splitting the serialized representation across multiple rows in the backend database.

      The serialized representation must fit entirely in a single row, limited by maxSerializedValueSize().

      Returns:
      obj with the Obj.createdAtMicros() field updated if and only if no other object with the same object id existed before, otherwise null

    • conditionalUpdate

      @Nullable <T extends Obj> T conditionalUpdate(@Nonnull T expected, @Nonnull T update, @Nonnull Class<T> clazz)
      Update an object with strong consistent guarantees.

      Unlike eventually consistent writes, conditional write operations do not support splitting the serialized representation across multiple rows in the backend database.

      The serialized representation must fit entirely in a single row, limited by maxSerializedValueSize().

      Parameters:
      expected - the object expected to have the same Obj.versionToken() as this one
      update - the object to be updated to, must have the same id, type but a different version token
      Returns:
      updated state in the database, if successful, otherwise null

    • conditionalDelete

      <T extends Obj> boolean conditionalDelete(@Nonnull T expected, Class<T> clazz)
      Delete an object with strong consistent guarantees.
      Parameters:
      expected - the object expected to have the same Obj.versionToken() as this one
      Returns:
      true if the object existed with the expected version token and was deleted in the database, if successful, otherwise false

    • params

      PersistenceParams params()

    • maxSerializedValueSize

      int maxSerializedValueSize()
      Defines the maximum allowed serialized object size. Serialized representation larger than this value will be split into multiple database rows.

    • generateId

      long generateId()

    • generateObjId

      ObjRef generateObjId(ObjType type)

    • getImmediate

      @Nullable <T extends Obj> T getImmediate(@Nonnull ObjRef id, @Nonnull Class<T> clazz)
      If the persistence implementation is caching, this function returns the object with the ID from the cache, but does not consult the backend.

      Non-caching implementations default to fetch(ObjRef, Class).

    • commits

      Commits commits()

    • createCommitter

      <REF_OBJ extends BaseCommitObj, RESULT> Committer<REF_OBJ,RESULT> createCommitter(@Nonnull String refName, @Nonnull Class<REF_OBJ> referencedObjType, @Nonnull Class<RESULT> resultType)

    • buildReadIndex

      <V> Index<V> buildReadIndex(@Nullable IndexContainer<V> indexContainer, @Nonnull IndexValueSerializer<V> indexValueSerializer)

    • buildWriteIndex

      <V> UpdatableIndex<V> buildWriteIndex(@Nullable IndexContainer<V> indexContainer, @Nonnull IndexValueSerializer<V> indexValueSerializer)

    • objAge

      @Nonnull default Duration objAge(@Nonnull Obj obj)

    • realmId

      String realmId()

    • monotonicClock

      org.apache.polaris.ids.api.MonotonicClock monotonicClock()

    • idGenerator

      org.apache.polaris.ids.api.IdGenerator idGenerator()

    • currentTimeMicros

      default long currentTimeMicros()
      Convenience for monotonicClock().MonotonicClock.currentTimeMicros().

    • currentTimeMillis

      default long currentTimeMillis()
      Convenience for monotonicClock().MonotonicClock.currentTimeMillis().

    • currentInstant

      default Instant currentInstant()
      Convenience for monotonicClock().MonotonicClock.currentInstant().

    • bucketizedBulkFetches

      default <O extends Obj> Stream<O> bucketizedBulkFetches(Stream<ObjRef> objRefs, Class<O> clazz)
      Convenience function to perform fetchMany(Class, ObjRef...) on an arbitrary number of objects to fetch.
      Parameters:
      objRefs - all ObjRefs to fetch
      clazz - type of Obj to fetch
      Returns:
      stream of fetched Objs, not found Objs are filtered out
      See Also: