Interface KGroupedTable<K,V>
-
- Type Parameters:
K- Type of keysV- Type of values
@Evolving public interface KGroupedTable<K,V>KGroupedTableis an abstraction of a re-grouped changelog stream from a primary-keyed table, usually on a different grouping key than the original primary key.It is an intermediate representation after a re-grouping of a
KTablebefore an aggregation is applied to the new partitions resulting in a newKTable.A
KGroupedTablemust be obtained from aKTableviagroupBy(...).- See Also:
KTable
-
-
Method Summary
All Methods Instance Methods Abstract Methods Deprecated Methods Modifier and Type Method Description <VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor)<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, java.lang.String queryableStoreName)<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde)<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde, java.lang.String queryableStoreName)<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, Materialized<K,VR,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)<VR> KTable<K,VR>aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, StateStoreSupplier<KeyValueStore> storeSupplier)KTable<K,java.lang.Long>count()KTable<K,java.lang.Long>count(java.lang.String queryableStoreName)Deprecated.KTable<K,java.lang.Long>count(Materialized<K,java.lang.Long,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)KTable<K,java.lang.Long>count(StateStoreSupplier<KeyValueStore> storeSupplier)Deprecated.KTable<K,V>reduce(Reducer<V> adder, Reducer<V> subtractor)KTable<K,V>reduce(Reducer<V> adder, Reducer<V> subtractor, java.lang.String queryableStoreName)Deprecated.KTable<K,V>reduce(Reducer<V> adder, Reducer<V> subtractor, Materialized<K,V,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)KTable<K,V>reduce(Reducer<V> adder, Reducer<V> subtractor, StateStoreSupplier<KeyValueStore> storeSupplier)
-
-
-
Method Detail
-
count
@Deprecated KTable<K,java.lang.Long> count(java.lang.String queryableStoreName)
Deprecated.Count number of records of the originalKTablethat gotmappedto the same key into a new instance ofKTable. Records withnullkey are ignored. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. The store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
queryableStoreName- the name of the underlyingKTablestate store; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthis is the equivalent ofcount().- Returns:
- a
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key
-
count
KTable<K,java.lang.Long> count(Materialized<K,java.lang.Long,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)
Count number of records of the originalKTablethat gotmappedto the same key into a new instance ofKTable. Records withnullkey are ignored. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. The store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
materialized- the instance ofMaterializedused to materialize the state store. Cannot benull- Returns:
- a
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key
-
count
KTable<K,java.lang.Long> count()
Count number of records of the originalKTablethat gotmappedto the same key into a new instance ofKTable. Records withnullkey are ignored. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Returns:
- a
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key
-
count
@Deprecated KTable<K,java.lang.Long> count(StateStoreSupplier<KeyValueStore> storeSupplier)
Deprecated.Count number of records of the originalKTablethat gotmappedto the same key into a new instance ofKTable. Records withnullkey are ignored. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the
configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words String queryableStoreName = storeSupplier.name(); ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
storeSupplier- user defined state store supplier. Cannot benull.- Returns:
- a
KTablethat contains "update" records with unmodified keys andLongvalues that represent the latest (rolling) count (i.e., number of records) for each key
-
reduce
@Deprecated KTable<K,V> reduce(Reducer<V> adder, Reducer<V> subtractor, java.lang.String queryableStoreName)
Deprecated.Combine the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTable. Records withnullkey are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Aggregator, Serde, String)). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Each update to the original
KTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument) by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate (first argument) and the record's value (second argument) by "removing" the "replaced" record from the aggregate. If there is no current aggregate theReduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, Reducer, String)can be used to compute aggregate functions like sum. For sum, the adder and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and thepublic class SumAdder implements Reducer<Integer> { public Integer apply(Integer currentAgg, Integer newValue) { return currentAgg + newValue; } } public class SumSubtractor implements Reducer<Integer> { public Integer apply(Integer currentAgg, Integer oldValue) { return currentAgg - oldValue; } }configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. The store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
adder- aReducerthat adds a new value to the aggregate resultsubtractor- aReducerthat removed an old value from the aggregate resultqueryableStoreName- the name of the underlyingKTablestate store; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthis is the equivalent ofreduce(Reducer, Reducer)()}.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
reduce
KTable<K,V> reduce(Reducer<V> adder, Reducer<V> subtractor, Materialized<K,V,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)
Combine the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTable. Records withnullkey are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Aggregator, Materialized)). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Each update to the original
KTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument) by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate (first argument) and the record's value (second argument) by "removing" the "replaced" record from the aggregate. If there is no current aggregate theReduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, Reducer, String)can be used to compute aggregate functions like sum. For sum, the adder and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and thepublic class SumAdder implements Reducer<Integer> { public Integer apply(Integer currentAgg, Integer newValue) { return currentAgg + newValue; } } public class SumSubtractor implements Reducer<Integer> { public Integer apply(Integer currentAgg, Integer oldValue) { return currentAgg - oldValue; } }configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. The store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
adder- aReducerthat adds a new value to the aggregate resultsubtractor- aReducerthat removed an old value from the aggregate resultmaterialized- the instance ofMaterializedused to materialize the state store. Cannot benull- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
reduce
KTable<K,V> reduce(Reducer<V> adder, Reducer<V> subtractor)
Combine the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTable. Records withnullkey are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Aggregator)). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Each update to the original
KTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate and the record's value by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate and the record's value by "removing" the "replaced" record from the aggregate. If there is no current aggregate theReduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, Reducer, String)can be used to compute aggregate functions like sum. For sum, the adder and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and thepublic class SumAdder implements Reducer<Integer> { public Integer apply(Integer currentAgg, Integer newValue) { return currentAgg + newValue; } } public class SumSubtractor implements Reducer<Integer> { public Integer apply(Integer currentAgg, Integer oldValue) { return currentAgg - oldValue; } }configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().
-
reduce
@Deprecated KTable<K,V> reduce(Reducer<V> adder, Reducer<V> subtractor, StateStoreSupplier<KeyValueStore> storeSupplier)
Combine the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTable. Records withnullkey are ignored. Combining implies that the type of the aggregate result is the same as the type of the input value (c.f.aggregate(Initializer, Aggregator, Aggregator, Serde, String)). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.Each update to the original
KTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (first argument) and the record's value (second argument) by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate (first argument) and the record's value (second argument) by "removing" the "replaced" record from the aggregate. If there is no current aggregate theReduceris not applied and the new aggregate will be the record's value as-is. Thus,reduce(Reducer, Reducer, String)can be used to compute aggregate functions like sum. For sum, the adder and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and thepublic class SumAdder implements Reducer<Integer> { public Integer apply(Integer currentAgg, Integer newValue) { return currentAgg + newValue; } } public class SumSubtractor implements Reducer<Integer> { public Integer apply(Integer currentAgg, Integer oldValue) { return currentAgg - oldValue; } }configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words String queryableStoreName = storeSupplier.name(); ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Parameters:
adder- aReducerthat adds a new value to the aggregate resultsubtractor- aReducerthat removed an old value from the aggregate resultstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
@Deprecated <VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, java.lang.String queryableStoreName)
Aggregate the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTableusing default serializers and deserializers. Records withnullkey are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. If the result value type does not match thedefault value serdeyou should useaggregate(Initializer, Aggregator, Aggregator, Serde, String). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. Each update to the originalKTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate and the record's value by "removing" the "replaced" record from the aggregate. Thus,aggregate(Initializer, Aggregator, Aggregator, String)can be used to compute aggregate functions like sum. For sum, the initializer, adder, and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the// in this example, LongSerde.class must be set as default value serde in StreamsConfig public class SumInitializer implements Initializer<Long> { public Long apply() { return 0L; } } public class SumAdder implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer newValue, Long aggregate) { return aggregate + newValue; } } public class SumSubstractor implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer oldValue, Long aggregate) { return aggregate - oldValue; } }configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the aggregatedKTable- Parameters:
initializer- aInitializerthat provides an initial aggregate result valueadder- aAggregatorthat adds a new record to the aggregate resultsubtractor- aAggregatorthat removed an old record from the aggregate resultqueryableStoreName- the name of the underlyingKTablestate store. Ifnullthis is the equivalent ofaggregate(Initializer, Aggregator, Aggregator)()}.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
<VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, Materialized<K,VR,KeyValueStore<org.apache.kafka.common.utils.Bytes,byte[]>> materialized)
Aggregate the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTableusing default serializers and deserializers. Records withnullkey are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. Each update to the originalKTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate and the record's value by "removing" the "replaced" record from the aggregate. Thus,aggregate(Initializer, Aggregator, Aggregator, String)can be used to compute aggregate functions like sum. For sum, the initializer, adder, and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the// in this example, LongSerde.class must be set as default value serde in StreamsConfig public class SumInitializer implements Initializer<Long> { public Long apply() { return 0L; } } public class SumAdder implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer newValue, Long aggregate) { return aggregate + newValue; } } public class SumSubstractor implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer oldValue, Long aggregate) { return aggregate - oldValue; } }configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the aggregatedKTable- Parameters:
initializer- anInitializerthat provides an initial aggregate result valueadder- anAggregatorthat adds a new record to the aggregate resultsubtractor- anAggregatorthat removed an old record from the aggregate resultmaterialized- the instance ofMaterializedused to materialize the state store. Cannot benull- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
<VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor)
Aggregate the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTableusing default serializers and deserializers. Records withnullkey are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. If the result value type does not match thedefault value serdeyou should useaggregate(Initializer, Aggregator, Aggregator, Serde). The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. Each update to the originalKTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate and the record's value by "removing" the "replaced" record from the aggregate. Thus,aggregate(Initializer, Aggregator, Aggregator, String)can be used to compute aggregate functions like sum. For sum, the initializer, adder, and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and the// in this example, LongSerde.class must be set as default value serde in StreamsConfig public class SumInitializer implements Initializer<Long> { public Long apply() { return 0L; } } public class SumAdder implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer newValue, Long aggregate) { return aggregate + newValue; } } public class SumSubstractor implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer oldValue, Long aggregate) { return aggregate - oldValue; } }configurationparameters forcache size, andcommit intervall. For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified inStreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the aggregatedKTable- Parameters:
initializer- aInitializerthat provides an initial aggregate result valueadder- aAggregatorthat adds a new record to the aggregate resultsubtractor- aAggregatorthat removed an old record from the aggregate result- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
@Deprecated <VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde, java.lang.String queryableStoreName)
Deprecated.Aggregate the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTableusing default serializers and deserializers. Records withnullkey are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. Each update to the originalKTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate and the record's value by "removing" the "replaced" record from the aggregate. Thus,aggregate(Initializer, Aggregator, Aggregator, String)can be used to compute aggregate functions like sum. For sum, the initializer, adder, and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and thepublic class SumInitializer implements Initializer<Long> { public Long apply() { return 0L; } } public class SumAdder implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer newValue, Long aggregate) { return aggregate + newValue; } } public class SumSubstractor implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer oldValue, Long aggregate) { return aggregate - oldValue; } }configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. The store name must be a valid Kafka topic name and cannot contain characters other than ASCII alphanumerics, '.', '_' and '-'. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the aggregatedKTable- Parameters:
initializer- aInitializerthat provides an initial aggregate result valueadder- aAggregatorthat adds a new record to the aggregate resultsubtractor- aAggregatorthat removed an old record from the aggregate resultaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be usedqueryableStoreName- the name of the underlyingKTablestate store; valid characters are ASCII alphanumerics, '.', '_' and '-'. Ifnullthis is the equivalent ofaggregate(Initializer, Aggregator, Aggregator, Serde)()}.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
@Deprecated <VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, org.apache.kafka.common.serialization.Serde<VR> aggValueSerde)
Aggregate the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTableusing default serializers and deserializers. Records withnullkey are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) that can be queried using the providedqueryableStoreName. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. Each update to the originalKTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate and the record's value by "removing" the "replaced" record from the aggregate. Thus,aggregate(Initializer, Aggregator, Aggregator, String)can be used to compute aggregate functions like sum. For sum, the initializer, adder, and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and thepublic class SumInitializer implements Initializer<Long> { public Long apply() { return 0L; } } public class SumAdder implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer newValue, Long aggregate) { return aggregate + newValue; } } public class SumSubstractor implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer oldValue, Long aggregate) { return aggregate - oldValue; } }configurationparameters forcache size, andcommit intervall.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${internalStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "internalStoreName" is an internal name and "-changelog" is a fixed suffix. Note that the internal store name may not be queriable through Interactive Queries. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the aggregatedKTable- Parameters:
initializer- aInitializerthat provides an initial aggregate result valueadder- aAggregatorthat adds a new record to the aggregate resultsubtractor- aAggregatorthat removed an old record from the aggregate resultaggValueSerde- aggregate value serdes for materializing the aggregated table, if not specified the default serdes defined in the configs will be used- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
aggregate
@Deprecated <VR> KTable<K,VR> aggregate(Initializer<VR> initializer, Aggregator<? super K,? super V,VR> adder, Aggregator<? super K,? super V,VR> subtractor, StateStoreSupplier<KeyValueStore> storeSupplier)
Deprecated.Aggregate the value of records of the originalKTablethat gotmappedto the same key into a new instance ofKTableusing default serializers and deserializers. Records withnullkey are ignored. Aggregating is a generalization ofcombining via reduce(...)as it, for example, allows the result to have a different type than the input values. The result is written into a localKeyValueStore(which is basically an ever-updating materialized view) provided by the givenstoreSupplier. Furthermore, updates to the store are sent downstream into aKTablechangelog stream.The specified
Initializeris applied once directly before the first input record is processed to provide an initial intermediate aggregation result that is used to process the first record. Each update to the originalKTableresults in a two step update of the resultKTable. The specifiedadderis applied for each update record and computes a new aggregate using the current aggregate (or for the very first record using the intermediate aggregation result provided via theInitializer) and the record's value by adding the new record to the aggregate. The specifiedsubstractoris applied for each "replaced" record of the originalKTableand computes a new aggregate using the current aggregate and the record's value by "removing" the "replaced" record from the aggregate. Thus,aggregate(Initializer, Aggregator, Aggregator, String)can be used to compute aggregate functions like sum. For sum, the initializer, adder, and substractor would work as follows:
Not all updates might get sent downstream, as an internal cache is used to deduplicate consecutive updates to the same key. The rate of propagated updates depends on your input data rate, the number of distinct keys, the number of parallel running Kafka Streams instances, and thepublic class SumInitializer implements Initializer<Long> { public Long apply() { return 0L; } } public class SumAdder implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer newValue, Long aggregate) { return aggregate + newValue; } } public class SumSubstractor implements Aggregator<String, Integer, Long> { public Long apply(String key, Integer oldValue, Long aggregate) { return aggregate - oldValue; } }configurationparameters forcache size, andcommit intervall.To query the local
KeyValueStoreit must be obtained viaKafkaStreams#store(...):
For non-local keys, a custom RPC mechanism must be implemented usingKafkaStreams streams = ... // counting words String queryableStoreName = storeSupplier.name(); ReadOnlyKeyValueStore<String,Long> localStore = streams.store(queryableStoreName, QueryableStoreTypes.<String, Long>keyValueStore()); String key = "some-word"; Long countForWord = localStore.get(key); // key must be local (application state is shared over all running Kafka Streams instances)KafkaStreams.allMetadata()to query the value of the key on a parallel running instance of your Kafka Streams application.For failure and recovery the store will be backed by an internal changelog topic that will be created in Kafka. The changelog topic will be named "${applicationId}-${queryableStoreName}-changelog", where "applicationId" is user-specified in
StreamsConfigvia parameterAPPLICATION_ID_CONFIG, "queryableStoreName" is the providequeryableStoreName, and "-changelog" is a fixed suffix. You can retrieve all generated internal topic names viaKafkaStreams.toString().- Type Parameters:
VR- the value type of the aggregatedKTable- Parameters:
initializer- aInitializerthat provides an initial aggregate result valueadder- aAggregatorthat adds a new record to the aggregate resultsubtractor- aAggregatorthat removed an old record from the aggregate resultstoreSupplier- user defined state store supplier. Cannot benull.- Returns:
- a
KTablethat contains "update" records with unmodified keys, and values that represent the latest (rolling) aggregate for each key
-
-