Package org.apache.flink.table.planner.plan.rules.physical.stream

Class TwoStageOptimizedWindowAggregateRule

java.lang.Object
org.apache.calcite.plan.RelOptRule
org.apache.flink.table.planner.plan.rules.physical.stream.TwoStageOptimizedWindowAggregateRule
public class TwoStageOptimizedWindowAggregateRule extends org.apache.calcite.plan.RelOptRule
Rule that matches StreamPhysicalWindowAggregate on StreamPhysicalExchange with following condition:
  • the applied windowing is not on processing-time, because processing-time should be materialized in a single node.
  • two-phase aggregation is enabled in given TableConfig.
  • all aggregate functions support merge() method.
  • the input of exchange does not satisfy the shuffle distribution

This optimization is as known as local-global optimization for reducing data-shuffling. It splits window aggregation into two-stage window aggregation, i.e. local-aggregation and global-aggregation. The local-aggregation produces a partial result for each group and window slice before shuffle in stage 1, and then the partially aggregated results are shuffled by group key to global-aggregation which produces the final result in stage 2.

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.calcite.plan.RelOptRule

    org.apache.calcite.plan.RelOptRule.ConverterRelOptRuleOperand

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final TwoStageOptimizedWindowAggregateRule
    INSTANCE
     

    Fields inherited from class org.apache.calcite.plan.RelOptRule

    description, operands, relBuilderFactory

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    matches(org.apache.calcite.plan.RelOptRuleCall call)
     
    void
    onMatch(org.apache.calcite.plan.RelOptRuleCall call)
     

    Methods inherited from class org.apache.calcite.plan.RelOptRule

    any, convert, convert, convertList, convertOperand, convertOperand, equals, equals, getOperand, getOperands, getOutConvention, getOutTrait, hashCode, none, operand, operand, operand, operand, operand, operandJ, operandJ, some, toString, unordered

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

  • Field Details

  • Method Details

    • matches

      public boolean matches(org.apache.calcite.plan.RelOptRuleCall call)
      Overrides:
      matches in class org.apache.calcite.plan.RelOptRule

    • onMatch

      public void onMatch(org.apache.calcite.plan.RelOptRuleCall call)
      Specified by:
      onMatch in class org.apache.calcite.plan.RelOptRule