Package kafka.tools

Class StreamsResetter

  • @Unstable
public class StreamsResetter
extends java.lang.Object
    StreamsResetter resets the processing state of a Kafka Streams application so that, for example, you can reprocess its input from scratch.

    This class is not part of public API. For backward compatibility, use the provided script in "bin/" instead of calling this class directly from your code.

    Resetting the processing state of an application includes the following actions:

    1. setting the application's consumer offsets for input and internal topics to zero
    2. skip over all intermediate user topics (i.e., "seekToEnd" for consumers of intermediate topics)
    3. deleting any topics created internally by Kafka Streams for this application

    Do only use this tool if no application instance is running. Otherwise, the application will get into an invalid state and crash or produce wrong results.

    If you run multiple application instances, running this tool once is sufficient. However, you need to call KafkaStreams#cleanUp() before re-starting any instance (to clean local state store directory). Otherwise, your application is in an invalid state.

    User output topics will not be deleted or modified by this tool. If downstream applications consume intermediate or output topics, it is the user's responsibility to adjust those applications manually if required.

    • Constructor Summary

      Constructors 
      Constructor Description
      StreamsResetter()  

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void doDeleteForStreamsAndAppDir​(java.lang.String internalStream, java.lang.String internalStreamCompacted, java.lang.String appDir)  
      long getDateTime​(java.lang.String timestamp)  
      static void main​(java.lang.String[] args)  
      boolean matchesInternalTopicFormat​(java.lang.String topicName)  
      void maybeSeekToEnd​(java.lang.String groupId, org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client, java.util.Set<org.apache.kafka.common.TopicPartition> intermediateTopicPartitions)  
      void resetOffsetsFromResetPlan​(org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client, java.util.Set<org.apache.kafka.common.TopicPartition> inputTopicPartitions, java.util.Map<org.apache.kafka.common.TopicPartition,​java.lang.Long> topicPartitionsAndOffset)  
      void resetOffsetsTo​(org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client, java.util.Set<org.apache.kafka.common.TopicPartition> inputTopicPartitions, java.lang.Long offset)  
      void resetToDatetime​(org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client, java.util.Set<org.apache.kafka.common.TopicPartition> inputTopicPartitions, java.lang.Long timestamp)  
      int run​(java.lang.String[] args)  
      int run​(java.lang.String[] args, java.util.Properties config)  
      void shiftOffsetsBy​(org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client, java.util.Set<org.apache.kafka.common.TopicPartition> inputTopicPartitions, long shiftBy)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • StreamsResetter

        public StreamsResetter()

    • Method Detail

      • run

        public int run​(java.lang.String[] args)

      • run

        public int run​(java.lang.String[] args,
               java.util.Properties config)

      • maybeSeekToEnd

        public void maybeSeekToEnd​(java.lang.String groupId,
                           org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client,
                           java.util.Set<org.apache.kafka.common.TopicPartition> intermediateTopicPartitions)

      • resetOffsetsFromResetPlan

        public void resetOffsetsFromResetPlan​(org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client,
                                      java.util.Set<org.apache.kafka.common.TopicPartition> inputTopicPartitions,
                                      java.util.Map<org.apache.kafka.common.TopicPartition,​java.lang.Long> topicPartitionsAndOffset)

      • resetToDatetime

        public void resetToDatetime​(org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client,
                            java.util.Set<org.apache.kafka.common.TopicPartition> inputTopicPartitions,
                            java.lang.Long timestamp)

      • shiftOffsetsBy

        public void shiftOffsetsBy​(org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client,
                           java.util.Set<org.apache.kafka.common.TopicPartition> inputTopicPartitions,
                           long shiftBy)

      • resetOffsetsTo

        public void resetOffsetsTo​(org.apache.kafka.clients.consumer.Consumer<byte[],​byte[]> client,
                           java.util.Set<org.apache.kafka.common.TopicPartition> inputTopicPartitions,
                           java.lang.Long offset)

      • getDateTime

        public long getDateTime​(java.lang.String timestamp)
                 throws java.text.ParseException
        Throws:
        java.text.ParseException

      • doDeleteForStreamsAndAppDir

        public void doDeleteForStreamsAndAppDir​(java.lang.String internalStream,
                                        java.lang.String internalStreamCompacted,
                                        java.lang.String appDir)

      • matchesInternalTopicFormat

        public boolean matchesInternalTopicFormat​(java.lang.String topicName)

      • main

        public static void main​(java.lang.String[] args)