Class EventTimeExtension
java.lang.Object
org.apache.flink.datastream.api.extension.eventtime.EventTimeExtension
The entry point for the event-time extension, which provides the following functionality:
- defines the event-time watermark and idle status watermark. If you use the
EventTimeWatermarkGeneratorBuilderbelow, then you don't need to declare these watermarks manually in your application; otherwise you need to declare them in your ownProcessFunction. - provides the
EventTimeWatermarkGeneratorBuilderto facilitate the generation of event time watermarks. An example of usingEventTimeWatermarkGeneratorBuilderis as follows:OneInputStreamProcessFunction<POJO, POJO> watermarkGeneratorProcessFunction = EventTimeExtension .newWatermarkGeneratorBuilder(POJO::getEventTime) .periodicWatermark() .buildAsProcessFunction(); source.process(watermarkGeneratorProcessFunction) .process(...) - provides a tool to encapsulate a user-defined
EventTimeProcessFunctionto provide the relevant components of the event-time extension.stream.process( EventTimeExtension.wrapProcessFunction( new CustomEventTimeProcessFunction() ) ) .process(...)
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final org.apache.flink.api.common.watermark.LongWatermarkDeclarationDefinition of EventTimeWatermark.static final org.apache.flink.api.common.watermark.BoolWatermarkDeclarationDefinition of IdleStatusWatermark. -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic booleanisEventTimeWatermark(String watermarkIdentifier) Determine if the received watermark is an EventTimeWatermark by watermark identifier.static booleanisEventTimeWatermark(org.apache.flink.api.common.watermark.Watermark watermark) Determine if the received watermark is an EventTimeWatermark.static booleanisIdleStatusWatermark(String watermarkIdentifier) Determine if the received watermark is an IdleStatusWatermark by watermark identifier.static booleanisIdleStatusWatermark(org.apache.flink.api.common.watermark.Watermark watermark) Determine if the received watermark is an IdleStatusWatermark.static <T> EventTimeWatermarkGeneratorBuilder<T>newWatermarkGeneratorBuilder(EventTimeExtractor<T> eventTimeExtractor) Create an instance ofEventTimeWatermarkGeneratorBuilder, which contains aEventTimeExtractor.static <IN,OUT> OneInputStreamProcessFunction<IN, OUT> wrapProcessFunction(OneInputEventTimeStreamProcessFunction<IN, OUT> processFunction) Wrap the user-definedOneInputEventTimeStreamProcessFunction, which will provide related components such asEventTimeManagerand declare the necessary built-in state required for the Timer, etc.static <IN1,IN2, OUT>
TwoInputBroadcastStreamProcessFunction<IN1,IN2, OUT> wrapProcessFunction(TwoInputBroadcastEventTimeStreamProcessFunction<IN1, IN2, OUT> processFunction) Wrap the user-definedTwoInputBroadcastEventTimeStreamProcessFunction, which will provide related components such asEventTimeManagerand declare the necessary built-in state required for the Timer, etc.static <IN1,IN2, OUT>
TwoInputNonBroadcastStreamProcessFunction<IN1,IN2, OUT> wrapProcessFunction(TwoInputNonBroadcastEventTimeStreamProcessFunction<IN1, IN2, OUT> processFunction) Wrap the user-definedTwoInputNonBroadcastEventTimeStreamProcessFunction, which will provide related components such asEventTimeManagerand declare the necessary built-in state required for the Timer, etc.static <IN,OUT1, OUT2>
TwoOutputStreamProcessFunction<IN,OUT1, OUT2> wrapProcessFunction(TwoOutputEventTimeStreamProcessFunction<IN, OUT1, OUT2> processFunction) Wrap the user-definedTwoOutputStreamProcessFunction, which will provide related components such asEventTimeManagerand declare the necessary built-in state required for the Timer, etc.
-
Field Details
-
EVENT_TIME_WATERMARK_DECLARATION
public static final org.apache.flink.api.common.watermark.LongWatermarkDeclaration EVENT_TIME_WATERMARK_DECLARATIONDefinition of EventTimeWatermark. The EventTimeWatermark represents a specific timestamp, signifying the passage of time. Once a process function receives an EventTimeWatermark, it will no longer receive events with a timestamp earlier than that watermark. -
IDLE_STATUS_WATERMARK_DECLARATION
public static final org.apache.flink.api.common.watermark.BoolWatermarkDeclaration IDLE_STATUS_WATERMARK_DECLARATIONDefinition of IdleStatusWatermark. The IdleStatusWatermark indicates that a particular input is in an idle state. When a ProcessFunction receives an IdleStatusWatermark from an input, it should ignore that input when combining EventTimeWatermarks.
-
-
Constructor Details
-
EventTimeExtension
public EventTimeExtension()
-
-
Method Details
-
isEventTimeWatermark
public static boolean isEventTimeWatermark(org.apache.flink.api.common.watermark.Watermark watermark) Determine if the received watermark is an EventTimeWatermark.- Parameters:
watermark- The watermark to be checked.- Returns:
- true if the watermark is an EventTimeWatermark; false otherwise.
-
isEventTimeWatermark
Determine if the received watermark is an EventTimeWatermark by watermark identifier.- Parameters:
watermarkIdentifier- The identifier of the watermark to be checked.- Returns:
- true if the watermark is an EventTimeWatermark; false otherwise.
-
isIdleStatusWatermark
public static boolean isIdleStatusWatermark(org.apache.flink.api.common.watermark.Watermark watermark) Determine if the received watermark is an IdleStatusWatermark.- Parameters:
watermark- The watermark to be checked.- Returns:
- true if the watermark is an IdleStatusWatermark; false otherwise.
-
isIdleStatusWatermark
Determine if the received watermark is an IdleStatusWatermark by watermark identifier.- Parameters:
watermarkIdentifier- The identifier of the watermark to be checked.- Returns:
- true if the watermark is an IdleStatusWatermark; false otherwise.
-
newWatermarkGeneratorBuilder
public static <T> EventTimeWatermarkGeneratorBuilder<T> newWatermarkGeneratorBuilder(EventTimeExtractor<T> eventTimeExtractor) Create an instance ofEventTimeWatermarkGeneratorBuilder, which contains aEventTimeExtractor.- Type Parameters:
T- The type of data records.- Parameters:
eventTimeExtractor- An instance ofEventTimeExtractorused to extract event time information from data records.- Returns:
- An instance of
EventTimeWatermarkGeneratorBuildercontaining the specified event time extractor.
-
wrapProcessFunction
public static <IN,OUT> OneInputStreamProcessFunction<IN,OUT> wrapProcessFunction(OneInputEventTimeStreamProcessFunction<IN, OUT> processFunction) Wrap the user-definedOneInputEventTimeStreamProcessFunction, which will provide related components such asEventTimeManagerand declare the necessary built-in state required for the Timer, etc. Note that registering event timers ofEventTimeProcessFunctioncan only be used withKeyedPartitionStream.- Parameters:
processFunction- The user-definedOneInputEventTimeStreamProcessFunctionthat needs to be wrapped.- Returns:
- The wrapped
OneInputStreamProcessFunction.
-
wrapProcessFunction
public static <IN,OUT1, TwoOutputStreamProcessFunction<IN,OUT2> OUT1, wrapProcessFunctionOUT2> (TwoOutputEventTimeStreamProcessFunction<IN, OUT1, OUT2> processFunction) Wrap the user-definedTwoOutputStreamProcessFunction, which will provide related components such asEventTimeManagerand declare the necessary built-in state required for the Timer, etc. Note that registering event timers ofEventTimeProcessFunctioncan only be used withKeyedPartitionStream.- Parameters:
processFunction- The user-definedTwoOutputEventTimeStreamProcessFunctionthat needs to be wrapped.- Returns:
- The wrapped
TwoOutputStreamProcessFunction.
-
wrapProcessFunction
public static <IN1,IN2, TwoInputNonBroadcastStreamProcessFunction<IN1,OUT> IN2, wrapProcessFunctionOUT> (TwoInputNonBroadcastEventTimeStreamProcessFunction<IN1, IN2, OUT> processFunction) Wrap the user-definedTwoInputNonBroadcastEventTimeStreamProcessFunction, which will provide related components such asEventTimeManagerand declare the necessary built-in state required for the Timer, etc. Note that registering event timers ofEventTimeProcessFunctioncan only be used withKeyedPartitionStream.- Parameters:
processFunction- The user-definedTwoInputNonBroadcastEventTimeStreamProcessFunctionthat needs to be wrapped.- Returns:
- The wrapped
TwoInputNonBroadcastStreamProcessFunction.
-
wrapProcessFunction
public static <IN1,IN2, TwoInputBroadcastStreamProcessFunction<IN1,OUT> IN2, wrapProcessFunctionOUT> (TwoInputBroadcastEventTimeStreamProcessFunction<IN1, IN2, OUT> processFunction) Wrap the user-definedTwoInputBroadcastEventTimeStreamProcessFunction, which will provide related components such asEventTimeManagerand declare the necessary built-in state required for the Timer, etc. Note that registering event timers ofEventTimeProcessFunctioncan only be used withKeyedPartitionStream.- Parameters:
processFunction- The user-definedTwoInputBroadcastEventTimeStreamProcessFunctionthat needs to be wrapped.- Returns:
- The wrapped
TwoInputBroadcastStreamProcessFunction.
-