Package org.apache.flink.table.planner.plan.utils

Class LookupJoinUtil

java.lang.Object

org.apache.flink.table.planner.plan.utils.LookupJoinUtil

@Internal
public final class LookupJoinUtil
extends Object

Utilities for lookup joins using LookupTableSource.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	LookupJoinUtil.AsyncLookupOptions	AsyncLookupOptions includes async related options.
static class	LookupJoinUtil.ConstantLookupKey	A LookupJoinUtil.LookupKey whose value is constant.
static class	LookupJoinUtil.FieldRefLookupKey	A LookupJoinUtil.LookupKey whose value comes from the left table field.
static class	LookupJoinUtil.LookupKey	A field used as an equal condition when querying content from a dimension table.
static class	LookupJoinUtil.RetryLookupOptions	RetryOptions includes retry lookup related options.
static class	LookupJoinUtil.ShuffleLookupOptions	ShuffleLookupOptions includes shuffle related options.

Method Summary

Modifier and Type	Method	Description
static boolean	enableLookupShuffle(org.apache.calcite.rel.hint.RelHint lookupJoinHint)	Determine whether to enable shuffle before lookup join.
static org.apache.flink.table.functions.UserDefinedFunction	getLookupFunction(org.apache.calcite.plan.RelOptTable temporalTable, Collection<Integer> lookupKeys, ClassLoader classLoader, boolean async, org.apache.flink.table.runtime.operators.join.lookup.ResultRetryStrategy retryStrategy, boolean preferCustomShuffle)	Gets required lookup function (async or sync) from temporal table , will raise an error if specified lookup function instance not found.
static LookupJoinUtil.AsyncLookupOptions	getMergedAsyncOptions(org.apache.calcite.rel.hint.RelHint lookupHint, org.apache.flink.table.api.TableConfig config, org.apache.flink.table.connector.ChangelogMode inputChangelogMode)
static int[]	getOrderedLookupKeys(Collection<Integer> allLookupKeys)	Gets lookup keys sorted by index in ascending order.
static boolean	isAsyncLookup(org.apache.calcite.plan.RelOptTable temporalTable, Collection<Integer> lookupKeys, org.apache.calcite.rel.hint.RelHint lookupHint, boolean upsertMaterialize, boolean preferCustomShuffle)	This method determines whether async lookup is enabled according to the given lookup keys with considering lookup RelHint and required upsertMaterialize.
static boolean	tableProvidesCustomPartitioner(org.apache.calcite.plan.RelOptTable table)	Determine whether table provides custom partitioner.
static org.apache.flink.api.dag.Transformation<org.apache.flink.table.data.RowData>	tryApplyCustomShufflePartitioner(PlannerBase planner, org.apache.calcite.plan.RelOptTable table, org.apache.flink.table.types.logical.RowType inputRowType, Map<Integer,LookupJoinUtil.LookupKey> allLookupKeys, org.apache.flink.api.dag.Transformation<org.apache.flink.table.data.RowData> inputTransformation, org.apache.flink.table.connector.ChangelogMode inputChangelogMode, TransformationMetadata metadata)	Try to apply custom partitioner to the input transformation of lookup join.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

getOrderedLookupKeys

public static int[] getOrderedLookupKeys(Collection<Integer> allLookupKeys)

Gets lookup keys sorted by index in ascending order.

getMergedAsyncOptions

public static LookupJoinUtil.AsyncLookupOptions getMergedAsyncOptions(org.apache.calcite.rel.hint.RelHint lookupHint,
 org.apache.flink.table.api.TableConfig config,
 org.apache.flink.table.connector.ChangelogMode inputChangelogMode)

isAsyncLookup

public static boolean isAsyncLookup(org.apache.calcite.plan.RelOptTable temporalTable,
 Collection<Integer> lookupKeys,
 org.apache.calcite.rel.hint.RelHint lookupHint,
 boolean upsertMaterialize,
 boolean preferCustomShuffle)

This method determines whether async lookup is enabled according to the given lookup keys with considering lookup RelHint and required upsertMaterialize. Note: it will not create the function instance to avoid potential heavy cost during optimization phase. if required upsertMaterialize is true, will return synchronous lookup function only, otherwise prefers asynchronous lookup function except there's a hint option 'async' = 'false', will raise an error if both candidates not found.

 1. if upsertMaterialize == true : return false

 2. preferAsync = except there is a hint option 'async' = 'false'
  if (preferAsync) {
    return asyncFound ? true : false
  } else {
    return syncFound ? false : true
  }
 

getLookupFunction

public static org.apache.flink.table.functions.UserDefinedFunction getLookupFunction(org.apache.calcite.plan.RelOptTable temporalTable,
 Collection<Integer> lookupKeys,
 ClassLoader classLoader,
 boolean async,
 org.apache.flink.table.runtime.operators.join.lookup.ResultRetryStrategy retryStrategy,
 boolean preferCustomShuffle)

Gets required lookup function (async or sync) from temporal table , will raise an error if specified lookup function instance not found.

enableLookupShuffle

public static boolean enableLookupShuffle(org.apache.calcite.rel.hint.RelHint lookupJoinHint)

Determine whether to enable shuffle before lookup join.

tableProvidesCustomPartitioner

public static boolean tableProvidesCustomPartitioner(org.apache.calcite.plan.RelOptTable table)

Determine whether table provides custom partitioner.

tryApplyCustomShufflePartitioner

public static org.apache.flink.api.dag.Transformation<org.apache.flink.table.data.RowData> tryApplyCustomShufflePartitioner(PlannerBase planner,
 org.apache.calcite.plan.RelOptTable table,
 org.apache.flink.table.types.logical.RowType inputRowType,
 Map<Integer,LookupJoinUtil.LookupKey> allLookupKeys,
 org.apache.flink.api.dag.Transformation<org.apache.flink.table.data.RowData> inputTransformation,
 org.apache.flink.table.connector.ChangelogMode inputChangelogMode,
 TransformationMetadata metadata)

Try to apply custom partitioner to the input transformation of lookup join.