Interface KGroupedStream<K,V>
-
- Type Parameters:
K- Type of keysV- Type of values
@Evolving public interface KGroupedStream<K,V>KGroupedStreamis an abstraction of a grouped record stream ofKeyValuepairs. It is an intermediate representation of aKStreamin order to apply an aggregation operation on the originalKStreamrecords.It is an intermediate representation after a grouping of a
KStreambefore an aggregation is applied to the new partitions resulting in aKTable.A
KGroupedStreammust be obtained from aKStreamviagroupByKey()orgroupBy(...).- See Also:
KStream
-
-
Method Summary
All Methods Instance Methods Abstract Methods Deprecated Methods Modifier and Type Method Description <T> KTable<Windowed<K>,T>aggregate(Initializer<T> initializer, Aggregator<? super K,? super V,T> aggregator, Merger<? super K,T> sessionMerger, SessionWindows sessionWindows, org.apache.kafka.common.serialization.Serde<T> aggValueSerde)Deprecated.<T> KTable<Windowed<K>,T>aggregate(Initializer<T> initializer, Aggregator<? super K,? super V,T> aggregator, Merger<? super K,T> sessionMerger, SessionWindows sessionWindows, org.apache.kafka.common.serialization.Serde<T> aggValueSerde, java.lang.String queryableStoreName)<T> KTable<Windowed<K>,T>aggregate(Initializer<T> initializer, Aggregator<? super K,? super V,T> aggregator, Merger<? super K,T> sessionMerger, SessionWindows sessionWindows, org.apache.kafka.common.serialization.Serde<T> aggValueSerde, StateStoreSupplier<SessionStore> storeSupplier)<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator)Aggregate the values of records in this stream by the grouped key.<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde)<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde, java.lang.String queryableStoreName)<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, Materialized<K,VR,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)Aggregate the values of records in this stream by the grouped key.<W extends Window,VR>
KTable<Windowed<K>,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, Windows<W> windows, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde)Deprecated.usewindowedBy(windows)followed byTimeWindowedKStream.aggregate(Initializer, Aggregator, Materialized)aggregate(initializer, aggregator, Materialized.with(null, aggValueSerde))}<W extends Window,VR>
KTable<Windowed<K>,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, Windows<W> windows, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde, java.lang.String queryableStoreName)<W extends Window,VR>
KTable<Windowed<K>,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, Windows<W> windows, StateStoreSupplier<WindowStore> storeSupplier)Deprecated.<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, StateStoreSupplier<KeyValueStore> storeSupplier)KTable<K,java.lang.Long>count()Count the number of records in this stream by the grouped key.KTable<K,java.lang.Long>count(java.lang.String queryableStoreName)Deprecated.KTable<K,java.lang.Long>count(Materialized<K,java.lang.Long,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)Count the number of records in this stream by the grouped key.KTable<Windowed<K>,java.lang.Long>count(SessionWindows sessionWindows)Deprecated.usewindowedBy(sessionWindows)followed bycount()KTable<Windowed<K>,java.lang.Long>count(SessionWindows sessionWindows, java.lang.String queryableStoreName)Deprecated.usewindowedBy(sessionWindows)followed bycount(Materialized.as(queryableStoreName))KTable<Windowed<K>,java.lang.Long>count(SessionWindows sessionWindows, StateStoreSupplier<SessionStore> storeSupplier)Deprecated.<W extends Window>
KTable<Windowed<K>,java.lang.Long>count(Windows<W> windows)Deprecated.usewindowedBy(windows)followed bycount()<W extends Window>
KTable<Windowed<K>,java.lang.Long>count(Windows<W> windows, java.lang.String queryableStoreName)Deprecated.usewindowedBy(windows)followed bycount(Materialized.as(queryableStoreName))<W extends Window>
KTable<Windowed<K>,java.lang.Long>count(Windows<W> windows, StateStoreSupplier<WindowStore> storeSupplier)Deprecated.usewindowedBy(windows)followed bycount(Materialized.as(KeyValueByteStoreSupplier))KTable<K,java.lang.Long>count(StateStoreSupplier<KeyValueStore> storeSupplier)Deprecated.KTable<K,V>reduce(Reducer<V> reducer)Combine the values of records in this stream by the grouped key.KTable<K,V>reduce(Reducer<V> reducer, java.lang.String queryableStoreName)Deprecated.KTable<K,V>reduce(Reducer<V> reducer, Materialized<K,V,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)Combine the value of records in this stream by the grouped key.KTable<Windowed<K>,V>reduce(Reducer<V> reducer, SessionWindows sessionWindows)Deprecated.usewindowedBy(sessionWindows)followed byreduce(reducer)KTable<Windowed<K>,V>reduce(Reducer<V> reducer, SessionWindows sessionWindows, java.lang.String queryableStoreName)Deprecated.KTable<Windowed<K>,V>reduce(Reducer<V> reducer, SessionWindows sessionWindows, StateStoreSupplier<SessionStore> storeSupplier)Deprecated.<W extends Window>
KTable<Windowed<K>,V>reduce(Reducer<V> reducer, Windows<W> windows)Deprecated.usewindowedBy(windows)followed byreduce(reducer)<W extends Window>
KTable<Windowed<K>,V>reduce(Reducer<V> reducer, Windows<W> windows, java.lang.String queryableStoreName)Deprecated.<W extends Window>
KTable<Windowed<K>,V>reduce(Reducer<V> reducer, Windows<W> windows, StateStoreSupplier<WindowStore> storeSupplier)Deprecated.KTable<K,V>reduce(Reducer<V> reducer, StateStoreSupplier<KeyValueStore> storeSupplier)Deprecated.SessionWindowedKStream<K,V>windowedBy(SessionWindows windows)Create a newSessionWindowedKStreaminstance that can be used to perform session windowed aggregations.<W extends Window>
TimeWindowedKStream<K,V>windowedBy(Windows<W> windows)Create a newTimeWindowedKStreaminstance that can be used to perform windowed aggregations.
-
-
-
Method Detail
-
count
@Deprecated KTable<K,java.lang.Long> count(java.lang.String queryableStoreName)
Deprecated.Count the number of records in this stream by the grouped key. Records withnullkey or value are ignored. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
queryableStoreName- the name of the underlyingKTablestate store; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthen this will be equivalent tocount().- Returns:
- a
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key
-
count
KTable<K,java.lang.Long> count()
Count the number of records in this stream by the grouped key. Records withnullkey or value are ignored. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view). Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Returns:
- a
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key
-
count
@Deprecated KTable<K,java.lang.Long> count(StateStoreSupplier<KeyValueStore> storeSupplier)
Deprecated.Count the number of records in this stream by the grouped key. Records withnullkey or value are ignored. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...). UseStateStore.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words String queryableStoreName = storeSupplier.name(); ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Parameters:
storeSupplier- user defined state store supplier. Cannot benull.- Returns:
- a
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key
-
count
KTable<K,java.lang.Long> count(Materialized<K,java.lang.Long,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)
Count the number of records in this stream by the grouped key. Records withnullkey or value are ignored. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenmaterialized. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...).
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words String queryableStoreName = "count-store"; // the queryableStoreName should be the name of the store as defined by the Materialized instance ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Parameters:
materialized- an instance ofMaterializedused to materialize a state store. Cannot benull. Note: the valueSerde will be automatically set toSerdes.Long()if there is no valueSerde provided- Returns:
- a
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key
-
count
@Deprecated <W extends Window> KTable<Windowed<K>,java.lang.Long> count(Windows<W> windows, java.lang.String queryableStoreName)
Deprecated.usewindowedBy(windows)followed bycount(Materialized.as(queryableStoreName))Count the number of records in this stream by the grouped key and the defined windows. Records withnullkey or value are ignored. The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyWindowStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>windowStore()); String key = "some-word"; long fromTime = ...; long toTime = ...; WindowStoreIterator<Long> countForWordsForWindows = localWindowStore.fetch(key, timeFrom, timeTo); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
windows- the specification of the aggregationWindowsqueryableStoreName- the name of the underlyingKTablestate store; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthen this will be equivalent tocount(Windows).- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key within a window.
-
count
@Deprecated <W extends Window> KTable<Windowed<K>,java.lang.Long> count(Windows<W> windows)
Deprecated.usewindowedBy(windows)followed bycount()Count the number of records in this stream by the grouped key and the defined windows. Records withnullkey or value are ignored. The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableName. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().
-
count
@Deprecated <W extends Window> KTable<Windowed<K>,java.lang.Long> count(Windows<W> windows, StateStoreSupplier<WindowStore> storeSupplier)
Deprecated.usewindowedBy(windows)followed bycount(Materialized.as(KeyValueByteStoreSupplier))Count the number of records in this stream by the grouped key and the defined windows. Records withnullkey or value are ignored. The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...). UseStateStoreSupplier.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words String queryableStoreName = storeSupplier.name(); ReadOnlyWindowStore<String,Long> localWindowStore = streams.store(queryableName, QueryableStoreTypes.<String, Long>windowStore()); String key = "some-word"; long fromTime = ...; long toTime = ...; WindowStoreIterator<Long> countForWordsForWindows = localWindowStore.fetch(key, timeFrom, timeTo); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Parameters:
windows- the specification of the aggregationWindowsstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key within a window
-
count
@Deprecated KTable<Windowed<K>,java.lang.Long> count(SessionWindows sessionWindows, java.lang.String queryableStoreName)
Deprecated.usewindowedBy(sessionWindows)followed bycount(Materialized.as(queryableStoreName))Count the number of records in this stream by the grouped key intoSessionWindows. Records withnullkey or value are ignored. The result is written into a localSessionStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...).
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum ReadOnlySessionStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>ReadOnlySessionStore<String, Long>); String key = "some-key"; KeyValueIterator<Windowed<String>, Long> sumForKeyForWindows = localWindowStore.fetch(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Parameters:
sessionWindows- the specification of the aggregationSessionWindowsqueryableStoreName- the name of the state store created from this operation; valid characters are ASCII alphanumerics, '.', '_' and '-. Ifnullthen this will be equivalent tocount(SessionWindows).- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key within a window
-
count
@Deprecated KTable<Windowed<K>,java.lang.Long> count(SessionWindows sessionWindows)
Deprecated.usewindowedBy(sessionWindows)followed bycount()Count the number of records in this stream by the grouped key intoSessionWindows. Records withnullkey or value are ignored. The result is written into a localSessionStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.- Parameters:
sessionWindows- the specification of the aggregationSessionWindows- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key within a window
-
count
@Deprecated KTable<Windowed<K>,java.lang.Long> count(SessionWindows sessionWindows, StateStoreSupplier<SessionStore> storeSupplier)
Deprecated.Count the number of records in this stream by the grouped key intoSessionWindows. Records withnullkey or value are ignored. The result is written into a localSessionStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...). UseStateStoreSupplier.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum Sting queryableStoreName = storeSupplier.name(); ReadOnlySessionStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>ReadOnlySessionStore<String, Long>); String key = "some-key"; KeyValueIterator<Windowed<String>, Long> sumForKeyForWindows = localWindowStore.fetch(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Parameters:
sessionWindows- the specification of the aggregationSessionWindowsstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key within a window
-
reduce
KTable<K,V> reduce(Reducer<V> reducer)
Combine the values of records in this stream by the grouped key. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator)). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate and the record's value. If there is no current aggregate theReduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, String)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().
-
reduce
@Deprecated KTable<K,V> reduce(Reducer<V> reducer, java.lang.String queryableStoreName)
Deprecated.Combine the values of records in this stream by the grouped key. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Serde, String)). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument):// At the example of a Reducer<Long> new Reducer<Long>() { public Long apply(Long aggValue, Long currValue) { return aggValue + currValue; } }If there is no current aggregate the
Reduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, String)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-key"; Long sumForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
reducer- aReducerthat computes a new aggregate result. Cannot benull.queryableStoreName- the name of the underlyingKTablestate store; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthen this will be equivalent toreduce(Reducer)()}.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
reduce
@Deprecated KTable<K,V> reduce(Reducer<V> reducer, StateStoreSupplier<KeyValueStore> storeSupplier)
Deprecated.Combine the value of records in this stream by the grouped key. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, org.apache.kafka.streams.processor.StateStoreSupplier)). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument):// At the example of a Reducer<Long> new Reducer<Long>() { public Long apply(Long aggValue, Long currValue) { return aggValue + currValue; } }If there is no current aggregate the
Reduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, org.apache.kafka.streams.processor.StateStoreSupplier)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...). UseStateStoreSupplier.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum String queryableStoreName = storeSupplier.name(); ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-key"; Long sumForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.
-
reduce
KTable<K,V> reduce(Reducer<V> reducer, Materialized<K,V,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)
Combine the value of records in this stream by the grouped key. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Materialized)). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenmaterialized. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument):// At the example of a Reducer<Long> new Reducer<Long>() { public Long apply(Long aggValue, Long currValue) { return aggValue + currValue; } }If there is no current aggregate the
Reduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, Materialized)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...).
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum String queryableStoreName = "storeName" // the queryableStoreName should be the name of the store as defined by the Materialized instance ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-key"; Long sumForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Parameters:
reducer- aReducerthat computes a new aggregate result. Cannot benull.materialized- an instance ofMaterializedused to materialize a state store. Cannot benull.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
reduce
@Deprecated <W extends Window> KTable<Windowed<K>,V> reduce(Reducer<V> reducer, Windows<W> windows, java.lang.String queryableStoreName)
Deprecated.Combine the number of records in this stream by the grouped key and the defined windows. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Windows, Serde, String)). The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument):// At the example of a Reducer<Long> new Reducer<Long>() { public Long apply(Long aggValue, Long currValue) { return aggValue + currValue; } }If there is no current aggregate the
Reduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, Windows, String)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum ReadOnlyWindowStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>windowStore()); String key = "some-key"; long fromTime = ...; long toTime = ...; WindowStoreIterator<Long> sumForKeyForWindows = localWindowStore.fetch(key, timeFrom, timeTo); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
reducer- aReducerthat computes a new aggregate resultwindows- the specification of the aggregationWindowsqueryableStoreName- the name of the state store created from this operation; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthen this will be equivalent toreduce(Reducer, Windows).- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
reduce
@Deprecated <W extends Window> KTable<Windowed<K>,V> reduce(Reducer<V> reducer, Windows<W> windows)
Deprecated.usewindowedBy(windows)followed byreduce(reducer)Combine the number of records in this stream by the grouped key and the defined windows. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Windows, Serde, String)). The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate and the record's value. If there is no current aggregate theReduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, Windows, String)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().
-
reduce
@Deprecated <W extends Window> KTable<Windowed<K>,V> reduce(Reducer<V> reducer, Windows<W> windows, StateStoreSupplier<WindowStore> storeSupplier)
Deprecated.Combine the values of records in this stream by the grouped key and the defined windows. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Windows, Serde, String)). The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument):// At the example of a Reducer<Long> new Reducer<Long>() { public Long apply(Long aggValue, Long currValue) { return aggValue + currValue; } }If there is no current aggregate the
Reduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, Windows, org.apache.kafka.streams.processor.StateStoreSupplier)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...). UseStateStoreSupplier.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum Sting queryableStoreName = storeSupplier.name(); ReadOnlyWindowStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>windowStore()); String key = "some-key"; long fromTime = ...; long toTime = ...; WindowStoreIterator<Long> sumForKeyForWindows = localWindowStore.fetch(key, timeFrom, timeTo); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Parameters:
reducer- aReducerthat computes a new aggregate result. Cannot benull.windows- the specification of the aggregationWindowsstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
reduce
@Deprecated KTable<Windowed<K>,V> reduce(Reducer<V> reducer, SessionWindows sessionWindows, java.lang.String queryableStoreName)
Deprecated.Combine values of this stream by the grouped key intoSessionWindows. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Merger, SessionWindows, Serde, String)). The result is written into a localSessionStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument):// At the example of a Reducer<Long> new Reducer<Long>() { public Long apply(Long aggValue, Long currValue) { return aggValue + currValue; } }If there is no current aggregate the
Reduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, SessionWindows, String)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...).
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum ReadOnlySessionStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>ReadOnlySessionStore<String, Long>); String key = "some-key"; KeyValueIterator<Windowed<String>, Long> sumForKeyForWindows = localWindowStore.fetch(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
reducer- aReducerthat computes a new aggregate result. Cannot benull.sessionWindows- the specification of the aggregationSessionWindowsqueryableStoreName- the name of the state store created from this operation; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthen this will be equivalent toreduce(Reducer, SessionWindows).- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
reduce
@Deprecated KTable<Windowed<K>,V> reduce(Reducer<V> reducer, SessionWindows sessionWindows)
Deprecated.usewindowedBy(sessionWindows)followed byreduce(reducer)Combine values of this stream by the grouped key intoSessionWindows. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Merger, SessionWindows, Serde, String)). The result is written into a localSessionStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate and the record's value. If there is no current aggregate theReduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, SessionWindows, String)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.- Parameters:
reducer- aReducerthat computes a new aggregate result. Cannot benull.sessionWindows- the specification of the aggregationSessionWindows- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
reduce
@Deprecated KTable<Windowed<K>,V> reduce(Reducer<V> reducer, SessionWindows sessionWindows, StateStoreSupplier<SessionStore> storeSupplier)
Deprecated.Combine values of this stream by the grouped key intoSessionWindows. Records withnullkey or value are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Merger, SessionWindows, Serde, String)). The result is written into a localSessionStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Reduceris applied for each input record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument):// At the example of a Reducer<Long> new Reducer<Long>() { public Long apply(Long aggValue, Long currValue) { return aggValue + currValue; } }If there is no current aggregate the
Reduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, SessionWindows, org.apache.kafka.streams.processor.StateStoreSupplier)can be used to compute aggregate functions like sum, min, or max.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...). UseStateStoreSupplier.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // compute sum Sting queryableStoreName = storeSupplier.name(); ReadOnlySessionStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>ReadOnlySessionStore<String, Long>); String key = "some-key"; KeyValueIterator<Windowed<String>, Long> sumForKeyForWindows = localWindowStore.fetch(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
reducer- aReducerthat computes a new aggregate result. Cannot benull.sessionWindows- the specification of the aggregationSessionWindowsstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
aggregate
@Deprecated <VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde, java.lang.String queryableStoreName)
Deprecated.Aggregate the values of records in this stream by the grouped key. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, Serde, String)can be used to compute aggregate functions like count (c.f.count(String)).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // some aggregation on value type double ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-key"; Long aggForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the resultingKTable- Parameters:
initializer- anInitializerthat computes an initial intermediate aggregation resultaggregator- anAggregatorthat computes a new aggregate resultaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be usedqueryableStoreName- the name of the state store created from this operation; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthen this will be equivalent toaggregate(Initializer, Aggregator, Serde).- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
<VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, Materialized<K,VR,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)
Aggregate the values of records in this stream by the grouped key. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, Serde, String)can be used to compute aggregate functions like count (c.f.count()).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // some aggregation on value type double ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-key"; Long aggForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the resultingKTable- Parameters:
initializer- anInitializerthat computes an initial intermediate aggregation resultaggregator- anAggregatorthat computes a new aggregate resultmaterialized- an instance ofMaterializedused to materialize a state store. Cannot benull.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
<VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator)
Aggregate the values of records in this stream by the grouped key. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator)can be used to compute aggregate functions like count (c.f.count()).The default value serde from config will be used for serializing the result. If a different serde is required then you should use
aggregate(Initializer, Aggregator, Materialized).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the resultingKTable- Parameters:
initializer- anInitializerthat computes an initial intermediate aggregation resultaggregator- anAggregatorthat computes a new aggregate result- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
@Deprecated <VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde)
Aggregate the values of records in this stream by the grouped key. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, Serde, String)can be used to compute aggregate functions like count (c.f.count()).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the resultingKTable- Parameters:
initializer- anInitializerthat computes an initial intermediate aggregation resultaggregator- anAggregatorthat computes a new aggregate resultaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be used- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
@Deprecated <VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, StateStoreSupplier<KeyValueStore> storeSupplier)
Aggregate the values of records in this stream by the grouped key. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, org.apache.kafka.streams.processor.StateStoreSupplier)can be used to compute aggregate functions like count (c.f.count()).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...). UseStateStoreSupplier.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // some aggregation on value type double Sting queryableStoreName = storeSupplier.name(); ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-key"; Long aggForKey = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Type Parameters:
VR- the value type of the resultingKTable- Parameters:
initializer- anInitializerthat computes an initial intermediate aggregation resultaggregator- anAggregatorthat computes a new aggregate resultstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
@Deprecated <W extends Window,VR> KTable<Windowed<K>,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, Windows<W> windows, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde, java.lang.String queryableStoreName)
Deprecated.Aggregate the values of records in this stream by the grouped key and defined windows. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Initializeris applied once per window directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, Windows, Serde, String)can be used to compute aggregate functions like count (c.f.count(Windows)).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // some windowed aggregation on value type double ReadOnlyWindowStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>windowStore()); String key = "some-key"; long fromTime = ...; long toTime = ...; WindowStoreIterator<Long> aggForKeyForWindows = localWindowStore.fetch(key, timeFrom, timeTo); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. Therefore, the store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the resultingKTable- Parameters:
initializer- anInitializerthat computes an initial intermediate aggregation resultaggregator- anAggregatorthat computes a new aggregate resultwindows- the specification of the aggregationWindowsaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be usedqueryableStoreName- the name of the state store created from this operation; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthen this will be equivalent toaggregate(Initializer, Aggregator, Windows, Serde).- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
aggregate
@Deprecated <W extends Window,VR> KTable<Windowed<K>,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, Windows<W> windows, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde)
Deprecated.usewindowedBy(windows)followed byTimeWindowedKStream.aggregate(Initializer, Aggregator, Materialized)aggregate(initializer, aggregator, Materialized.with(null, aggValueSerde))}Aggregate the values of records in this stream by the grouped key and defined windows. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Initializeris applied once per window directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, Windows, Serde, String)can be used to compute aggregate functions like count (c.f.count(Windows)).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the resultingKTable- Parameters:
initializer- anInitializerthat computes an initial intermediate aggregation resultaggregator- anAggregatorthat computes a new aggregate resultwindows- the specification of the aggregationWindowsaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be used- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
aggregate
@Deprecated <W extends Window,VR> KTable<Windowed<K>,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> aggregator, Windows<W> windows, StateStoreSupplier<WindowStore> storeSupplier)
Deprecated.Aggregate the values of records in this stream by the grouped key and defined windows. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The specifiedwindowsdefine either hopping time windows that can be overlapping or tumbling (c.f.TimeWindows) or they define landmark windows (c.f.UnlimitedWindows). The result is written into a local windowedKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Windows are retained until their retention time expires (c.f.Windows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Initializeris applied once per window directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, Windows, org.apache.kafka.streams.processor.StateStoreSupplier)can be used to compute aggregate functions like count (c.f.count(Windows)).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local windowed
KeyValueStoreit must be obtained viaKafkaStreams#store(...). UseStateStoreSupplier.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // some windowed aggregation on value type Long Sting queryableStoreName = storeSupplier.name(); ReadOnlyWindowStore<String,Long> localWindowStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>windowStore()); String key = "some-key"; long fromTime = ...; long toTime = ...; WindowStoreIterator<Long> aggForKeyForWindows = localWindowStore.fetch(key, timeFrom, timeTo); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Type Parameters:
VR- the value type of the resultingKTable- Parameters:
initializer- anInitializerthat computes an initial intermediate aggregation resultaggregator- anAggregatorthat computes a new aggregate resultwindows- the specification of the aggregationWindowsstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
aggregate
@Deprecated <T> KTable<Windowed<K>,T> aggregate(Initializer<T> initializer, Aggregator<? super K,? super V,T> aggregator, Merger<? super K,T> sessionMerger, SessionWindows sessionWindows, org.apache.kafka.common.serialization.Serde<T> aggValueSerde, java.lang.String queryableStoreName)
Deprecated.Aggregate the values of records in this stream by the grouped key and definedSessionWindows. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localSessionStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Initializeris applied once per session directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, Merger, SessionWindows, Serde, String)can be used to compute aggregate functions like count (c.f.count(SessionWindows))Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
SessionStoreit must be obtained viaKafkaStreams#store(...).
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // some windowed aggregation on value type double ReadOnlySessionStore<String, Long> sessionStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>sessionStore()); String key = "some-key"; KeyValueIterator<Windowed<String>, Long> aggForKeyForSession = localWindowStore.fetch(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Type Parameters:
T- the value type of the resultingKTable- Parameters:
initializer- the instance ofInitializeraggregator- the instance ofAggregatorsessionMerger- the instance ofMergersessionWindows- the specification of the aggregationSessionWindowsaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be usedqueryableStoreName- the name of the state store created from this operation; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthen this will be equivalent toaggregate(Initializer, Aggregator, Merger, SessionWindows, Serde).- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
aggregate
@Deprecated <T> KTable<Windowed<K>,T> aggregate(Initializer<T> initializer, Aggregator<? super K,? super V,T> aggregator, Merger<? super K,T> sessionMerger, SessionWindows sessionWindows, org.apache.kafka.common.serialization.Serde<T> aggValueSerde)
Deprecated.Aggregate the values of records in this stream by the grouped key and definedSessionWindows. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localSessionStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Initializeris applied once per session directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,aggregate(Initializer, Aggregator, Merger, SessionWindows, Serde, String)can be used to compute aggregate functions like count (c.f.count(SessionWindows))Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.- Type Parameters:
T- the value type of the resultingKTable- Parameters:
initializer- the instance ofInitializeraggregator- the instance ofAggregatorsessionMerger- the instance ofMergersessionWindows- the specification of the aggregationSessionWindowsaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be used- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
aggregate
@Deprecated <T> KTable<Windowed<K>,T> aggregate(Initializer<T> initializer, Aggregator<? super K,? super V,T> aggregator, Merger<? super K,T> sessionMerger, SessionWindows sessionWindows, org.apache.kafka.common.serialization.Serde<T> aggValueSerde, StateStoreSupplier<SessionStore> storeSupplier)
Deprecated.Aggregate the values of records in this stream by the grouped key and definedSessionWindows. Records withnullkey or value are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localSessionStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. SessionWindows are retained until their retention time expires (c.f.SessionWindows.until(long)). Furthermore, updates to the store are sent downstream into a windowedKTablechangelog stream, where "windowed" implies that theKTablekey is a combined key of the original record key and a window ID.The specified
Initializeris applied once per session directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. The specifiedAggregatoris applied for each input record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value. Thus,#aggregate(Initializer, Aggregator, Merger, SessionWindows, Serde, org.apache.kafka.streams.processor.StateStoreSupplier)can be used to compute aggregate functions like count (c.f.count(SessionWindows)).Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same window and key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
SessionStoreit must be obtained viaKafkaStreams#store(...). UseStateStoreSupplier.name()to get the store name:
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // some windowed aggregation on value type double Sting queryableStoreName = storeSupplier.name(); ReadOnlySessionStore<String, Long> sessionStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>sessionStore()); String key = "some-key"; KeyValueIterator<Windowed<String>, Long> aggForKeyForSession = localWindowStore.fetch(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.- Type Parameters:
T- the value type of the resultingKTable- Parameters:
initializer- the instance ofInitializeraggregator- the instance ofAggregatorsessionMerger- the instance ofMergersessionWindows- the specification of the aggregationSessionWindowsaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be usedstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a windowed
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key within a window
-
windowedBy
<W extends Window> TimeWindowedKStream<K,V> windowedBy(Windows<W> windows)
Create a newTimeWindowedKStreaminstance that can be used to perform windowed aggregations.- Type Parameters:
W- the window type- Parameters:
windows- the specification of the aggregationWindows- Returns:
- an instance of
TimeWindowedKStream
-
windowedBy
SessionWindowedKStream<K,V> windowedBy(SessionWindows windows)
Create a newSessionWindowedKStreaminstance that can be used to perform session windowed aggregations.- Parameters:
windows- the specification of the aggregationSessionWindows- Returns:
- an instance of
TimeWindowedKStream
-
-