Package org.apache.polaris.persistence.nosql.maintenance.api

Interface MaintenanceConfig

@ConfigMapping(prefix="polaris.persistence.maintenance") @PolarisImmutable public interface MaintenanceConfig
Maintenance service configuration.

  • Field Details

    • DEFAULT_EXPECTED_REFERENCE_COUNT

      static final long DEFAULT_EXPECTED_REFERENCE_COUNT
      See Also:

    • DEFAULT_EXPECTED_OBJ_COUNT

      static final long DEFAULT_EXPECTED_OBJ_COUNT
      See Also:

    • DEFAULT_COUNT_FROM_LAST_RUN_MULTIPLIER

      static final double DEFAULT_COUNT_FROM_LAST_RUN_MULTIPLIER
      See Also:

    • DEFAULT_INITIALIZED_FPP

      static final double DEFAULT_INITIALIZED_FPP
      See Also:

    • DEFAULT_MAX_ACCEPTABLE_FPP

      static final double DEFAULT_MAX_ACCEPTABLE_FPP
      See Also:

    • DEFAULT_RETAINED_RUNS

      static final int DEFAULT_RETAINED_RUNS
      See Also:

    • DEFAULT_CREATED_AT_GRACE_TIME_STRING

      static final String DEFAULT_CREATED_AT_GRACE_TIME_STRING
      See Also:

    • DEFAULT_CREATED_AT_GRACE_TIME

      static final Duration DEFAULT_CREATED_AT_GRACE_TIME

    • DEFAULT_DELETE_BATCH_SIZE

      static final int DEFAULT_DELETE_BATCH_SIZE
      See Also:

  • Method Details

    • expectedReferenceCount

      @WithDefault("100") OptionalLong expectedReferenceCount()
      Provides the expected number of references in all realms to retain, defaults to 100L, must be at least 100. This value is used as the default if no information of a previous maintenance run is present, it is also the minimum number of expected references.

    • expectedObjCount

      @WithDefault("100000") OptionalLong expectedObjCount()
      Provides the expected number of objects in all realms to retain, defaults to 100000L, must be at least 100000. This value is used as the default if no information of a previous maintenance run is present, it is also the minimum number of expected objects.

    • countFromLastRunMultiplier

      @WithDefault("1.1") OptionalDouble countFromLastRunMultiplier()
      Maintenance service sizes the bloom-filters used to hold the identified references and objects according to the expression lastRun.numberOfIdentified * countFromLastRunMultiplier. The default is to add 10% to the number of identified items.

    • filterInitializedFpp

      @WithDefault("1.0E-5") OptionalDouble filterInitializedFpp()
      False-positive-probability (FPP) used to initialize the bloom-filters for identified references and objects.

    • maxAcceptableFilterFpp

      @WithDefault("5.0E-5") OptionalDouble maxAcceptableFilterFpp()
      Expected maximum false-positive-probability (FPP) used to check the bloom-filters for identified references and objects.

      If the FPP of a bloom filter exceeds this value, no individual references or objects will be purged.

    • retainedRuns

      @WithDefault("50") OptionalInt retainedRuns()
      Number of retained maintenance run objects, must be at least 2.

    • createdAtGraceTime

      @WithDefault("PT3H") Optional<Duration> createdAtGraceTime()
      Objects and references that have been created after a maintenance run has started are never purged. This option defines an additional grace time to when the maintenance run has started.

      This value is a safety net for two reasons:

      • Respect the wall-clock drift between Polaris nodes.
      • Respect the order of writes in Polaris persistence. Objects are written before those become reachable via a commit. Commits may take a little time (milliseconds, up to a few seconds, depending on the system load) to complete. Therefore, implementations enforce a minimum of 5 minutes.

    • objectScanRateLimitPerSecond

      OptionalInt objectScanRateLimitPerSecond()
      Optionally limit the number of objects scanned per second. Default is to not throttle object scanning.

    • referenceScanRateLimitPerSecond

      OptionalInt referenceScanRateLimitPerSecond()
      Optionally limit the number of references scanned per second.

      Default is to not throttle reference scanning.

    • deleteBatchSize

      @WithDefault("10") OptionalInt deleteBatchSize()
      Size of the delete-batches when purging objects.

    • builder

      static org.apache.polaris.persistence.nosql.maintenance.api.ImmutableMaintenanceConfig.Builder builder()

    • check

      @Check default void check()