Class StoreChangelogReader

  • All Implemented Interfaces:
    ChangelogReader, ChangelogRegister

    public class StoreChangelogReader
    extends java.lang.Object
    implements ChangelogReader
    ChangelogReader is created and maintained by the stream thread and used for both updating standby tasks and restoring active tasks. It manages the restore consumer, including its assigned partitions, when to pause / resume these partitions, etc.

    The reader also maintains the source of truth for restoration state: only active tasks restoring changelog could be completed, while standby tasks updating changelog would always be in restoring state after being initialized.

    • Constructor Summary

      Constructors 
      Constructor Description
      StoreChangelogReader​(org.apache.kafka.common.utils.Time time, StreamsConfig config, org.apache.kafka.common.utils.LogContext logContext, org.apache.kafka.clients.admin.Admin adminClient, org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> restoreConsumer, StateRestoreListener stateRestoreListener)  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void clear()
      Clear all partitions
      java.util.Set<org.apache.kafka.common.TopicPartition> completedChangelogs()  
      void enforceRestoreActive()
      Transit to restore active changelogs mode
      boolean isEmpty()  
      void register​(org.apache.kafka.common.TopicPartition partition, ProcessorStateManager stateManager)
      Since it is shared for multiple tasks and hence multiple state managers, the registration would take its corresponding state manager as well for restoring.
      void restore()
      Restore all registered state stores by reading from their changelogs
      java.lang.String toString()  
      void transitToUpdateStandby()
      Transit to update standby changelogs mode
      void unregister​(java.util.Collection<org.apache.kafka.common.TopicPartition> revokedChangelogs)
      Unregisters and removes the passed in partitions from the set of changelogs
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Constructor Detail

      • StoreChangelogReader

        public StoreChangelogReader​(org.apache.kafka.common.utils.Time time,
                                    StreamsConfig config,
                                    org.apache.kafka.common.utils.LogContext logContext,
                                    org.apache.kafka.clients.admin.Admin adminClient,
                                    org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> restoreConsumer,
                                    StateRestoreListener stateRestoreListener)
    • Method Detail

      • register

        public void register​(org.apache.kafka.common.TopicPartition partition,
                             ProcessorStateManager stateManager)
        Since it is shared for multiple tasks and hence multiple state managers, the registration would take its corresponding state manager as well for restoring.
        Specified by:
        register in interface ChangelogRegister
        Parameters:
        partition - the state store's changelog partition for restoring
        stateManager - the state manager used for restoring (one per task)
      • completedChangelogs

        public java.util.Set<org.apache.kafka.common.TopicPartition> completedChangelogs()
        Specified by:
        completedChangelogs in interface ChangelogReader
        Returns:
        the changelog partitions that have been completed restoring
      • restore

        public void restore()
        Description copied from interface: ChangelogReader
        Restore all registered state stores by reading from their changelogs
        Specified by:
        restore in interface ChangelogReader
      • unregister

        public void unregister​(java.util.Collection<org.apache.kafka.common.TopicPartition> revokedChangelogs)
        Description copied from interface: ChangelogRegister
        Unregisters and removes the passed in partitions from the set of changelogs
        Specified by:
        unregister in interface ChangelogRegister
        Parameters:
        revokedChangelogs - the set of partitions to remove
      • isEmpty

        public boolean isEmpty()
        Specified by:
        isEmpty in interface ChangelogReader
        Returns:
        whether the changelog reader has just been cleared or is uninitialized
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object