Package org.apache.flink.table.planner.plan.optimize.program

Class FlinkDynamicPartitionPruningProgram

java.lang.Object
org.apache.flink.table.planner.plan.optimize.program.FlinkDynamicPartitionPruningProgram
public class FlinkDynamicPartitionPruningProgram extends Object
Planner program that tries to do partition prune in the execution phase, which can translate a BatchPhysicalTableSourceScan to a BatchPhysicalDynamicFilteringTableSourceScan whose source is a partition source. The OptimizerConfigOptions.TABLE_OPTIMIZER_DYNAMIC_FILTERING_ENABLED need to be true.

Suppose we have the original physical plan: 


 LogicalProject(...)
 HashJoin(joinType=[InnerJoin], where=[=(fact_partition_key, dim_key)], select=[xxx])
  * :- TableSourceScan(table=[[fact]], fields=[xxx, fact_partition_key],) # Is a partition table.
  * +- Exchange(distribution=[broadcast])
  *    +- Calc(select=[xxx], where=[<(xxx, xxx)]) # Need have an arbitrary filter condition.
  *       +- TableSourceScan(table=[[dim, filter=[]]], fields=[xxx, dim_key])

This physical plan will be rewritten to: 


 HashJoin(joinType=[InnerJoin], where=[=(fact_partition_key, dim_key)], select=[xxx])
 :- DynamicFilteringTableSourceScan(table=[[fact]], fields=[xxx, fact_partition_key]) # Is a partition table.
 :  +- DynamicFilteringDataCollector(fields=[dim_key])
 :     +- Calc(select=[xxx], where=[<(xxx, xxx)])
 :        +- TableSourceScan(table=[[dim, filter=[]]], fields=[xxx, dim_key])
 +- Exchange(distribution=[broadcast])
    +- Calc(select=[xxx], where=[<(xxx, xxx)]) # Need have an arbitrary filter condition.
       +- TableSourceScan(table=[[dim, filter=[]]], fields=[xxx, dim_key])

Note: We use a FlinkOptimizeProgram instead of a RelRule here because the HepPlanner doesn't support matching a partially determined pattern or dynamically replacing the inputs of matched nodes. Once we improve the HepPlanner, then class can be converted to RelRule.

  • Constructor Details

    • FlinkDynamicPartitionPruningProgram

      public FlinkDynamicPartitionPruningProgram()

  • Method Details

    • optimize

      public org.apache.calcite.rel.RelNode optimize(org.apache.calcite.rel.RelNode root, BatchOptimizeContext context)