Package org.apache.flink.table.planner.connectors

Class DynamicSourceUtils

java.lang.Object
org.apache.flink.table.planner.connectors.DynamicSourceUtils
@Internal public final class DynamicSourceUtils extends Object
Utilities for dealing with DynamicTableSource.

  • Method Summary

    Modifier and Type
    Method
    Description
    static boolean
    changelogNormalizeEnabled(boolean eventTimeSnapshotRequired, org.apache.flink.table.catalog.ResolvedSchema resolvedSchema, org.apache.flink.table.connector.source.DynamicTableSource tableSource, org.apache.flink.table.api.TableConfig tableConfig)
    Returns true if the changelogNormalize should be enabled.
    static org.apache.calcite.rel.RelNode
    convertDataStreamToRel(boolean isBatchMode, org.apache.flink.configuration.ReadableConfig config, FlinkRelBuilder relBuilder, org.apache.flink.table.catalog.ContextResolvedTable contextResolvedTable, org.apache.flink.streaming.api.datastream.DataStream<?> dataStream, org.apache.flink.table.types.DataType physicalDataType, boolean isTopLevelRecord, org.apache.flink.table.connector.ChangelogMode changelogMode)
    Converts a given DataStream to a RelNode.
    static org.apache.calcite.rel.RelNode
    convertSourceToRel(boolean isBatchMode, org.apache.flink.configuration.ReadableConfig config, FlinkRelBuilder relBuilder, org.apache.flink.table.catalog.ContextResolvedTable contextResolvedTable, FlinkStatistic statistic, List<org.apache.calcite.rel.hint.RelHint> hints, org.apache.flink.table.connector.source.DynamicTableSource tableSource)
    Converts a given DynamicTableSource to a RelNode.
    static Map<String,org.apache.flink.table.catalog.Column.MetadataColumn>
    createMetadataKeysToMetadataColumnsMap(org.apache.flink.table.catalog.ResolvedSchema schema)
    Returns a map record the mapping relation between metadataKeys to metadataColumns in input schema.
    static org.apache.flink.table.types.logical.RowType
    createProducedType(org.apache.flink.table.catalog.ResolvedSchema schema, org.apache.flink.table.connector.source.DynamicTableSource source)
    Returns the DataType that a source should produce as the input into the runtime.
    static List<org.apache.flink.table.catalog.Column.MetadataColumn>
    createRequiredMetadataColumns(org.apache.flink.table.catalog.ResolvedSchema schema, org.apache.flink.table.connector.source.DynamicTableSource source)
    Returns a list of required metadata columns.
    static List<org.apache.flink.table.catalog.Column.MetadataColumn>
    extractMetadataColumns(org.apache.flink.table.catalog.ResolvedSchema schema)
     
    static Map<String,org.apache.flink.table.types.DataType>
    extractMetadataMap(org.apache.flink.table.connector.source.DynamicTableSource source)
     
    static boolean
    isSourceChangeEventsDuplicate(org.apache.flink.table.catalog.ResolvedSchema resolvedSchema, org.apache.flink.table.connector.source.DynamicTableSource tableSource, org.apache.flink.table.api.TableConfig tableConfig)
    Returns true if the table source produces duplicate change events.
    static boolean
    isUpsertSource(org.apache.flink.table.catalog.ResolvedSchema resolvedSchema, org.apache.flink.table.connector.source.DynamicTableSource tableSource)
    Returns true if the table is an upsert source.
    static void
    prepareDynamicSource(String tableDebugName, org.apache.flink.table.catalog.ResolvedCatalogTable table, org.apache.flink.table.connector.source.DynamicTableSource source, boolean isBatchMode, org.apache.flink.configuration.ReadableConfig config, List<SourceAbilitySpec> sourceAbilities)
    Prepares the given DynamicTableSource.
    static void
    validateAndApplyMetadata(String tableDebugName, org.apache.flink.table.catalog.ResolvedSchema schema, org.apache.flink.table.connector.source.DynamicTableSource source, List<SourceAbilitySpec> sourceAbilities)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • convertDataStreamToRel

      public static org.apache.calcite.rel.RelNode convertDataStreamToRel(boolean isBatchMode, org.apache.flink.configuration.ReadableConfig config, FlinkRelBuilder relBuilder, org.apache.flink.table.catalog.ContextResolvedTable contextResolvedTable, org.apache.flink.streaming.api.datastream.DataStream<?> dataStream, org.apache.flink.table.types.DataType physicalDataType, boolean isTopLevelRecord, org.apache.flink.table.connector.ChangelogMode changelogMode)
      Converts a given DataStream to a RelNode. It adds helper projections if necessary.

    • convertSourceToRel

      public static org.apache.calcite.rel.RelNode convertSourceToRel(boolean isBatchMode, org.apache.flink.configuration.ReadableConfig config, FlinkRelBuilder relBuilder, org.apache.flink.table.catalog.ContextResolvedTable contextResolvedTable, FlinkStatistic statistic, List<org.apache.calcite.rel.hint.RelHint> hints, org.apache.flink.table.connector.source.DynamicTableSource tableSource)
      Converts a given DynamicTableSource to a RelNode. It adds helper projections if necessary.

    • prepareDynamicSource

      public static void prepareDynamicSource(String tableDebugName, org.apache.flink.table.catalog.ResolvedCatalogTable table, org.apache.flink.table.connector.source.DynamicTableSource source, boolean isBatchMode, org.apache.flink.configuration.ReadableConfig config, List<SourceAbilitySpec> sourceAbilities)
      Prepares the given DynamicTableSource. It check whether the source is compatible with the given schema and applies initial parameters.

    • createRequiredMetadataColumns

      public static List<org.apache.flink.table.catalog.Column.MetadataColumn> createRequiredMetadataColumns(org.apache.flink.table.catalog.ResolvedSchema schema, org.apache.flink.table.connector.source.DynamicTableSource source)
      Returns a list of required metadata columns. Ordered by the iteration order of SupportsReadingMetadata.listReadableMetadata().

      This method assumes that source and schema have been validated via prepareDynamicSource(String, ResolvedCatalogTable, DynamicTableSource, boolean, ReadableConfig, List).

    • createMetadataKeysToMetadataColumnsMap

      public static Map<String,org.apache.flink.table.catalog.Column.MetadataColumn> createMetadataKeysToMetadataColumnsMap(org.apache.flink.table.catalog.ResolvedSchema schema)
      Returns a map record the mapping relation between metadataKeys to metadataColumns in input schema.

    • createProducedType

      public static org.apache.flink.table.types.logical.RowType createProducedType(org.apache.flink.table.catalog.ResolvedSchema schema, org.apache.flink.table.connector.source.DynamicTableSource source)
      Returns the DataType that a source should produce as the input into the runtime.

      The format looks as follows: PHYSICAL COLUMNS + METADATA COLUMNS

      Physical columns use the table schema's name. Metadata column use the metadata key as name.

    • isUpsertSource

      public static boolean isUpsertSource(org.apache.flink.table.catalog.ResolvedSchema resolvedSchema, org.apache.flink.table.connector.source.DynamicTableSource tableSource)
      Returns true if the table is an upsert source.

    • isSourceChangeEventsDuplicate

      public static boolean isSourceChangeEventsDuplicate(org.apache.flink.table.catalog.ResolvedSchema resolvedSchema, org.apache.flink.table.connector.source.DynamicTableSource tableSource, org.apache.flink.table.api.TableConfig tableConfig)
      Returns true if the table source produces duplicate change events.

    • changelogNormalizeEnabled

      public static boolean changelogNormalizeEnabled(boolean eventTimeSnapshotRequired, org.apache.flink.table.catalog.ResolvedSchema resolvedSchema, org.apache.flink.table.connector.source.DynamicTableSource tableSource, org.apache.flink.table.api.TableConfig tableConfig)
      Returns true if the changelogNormalize should be enabled.

    • extractMetadataMap

      public static Map<String,org.apache.flink.table.types.DataType> extractMetadataMap(org.apache.flink.table.connector.source.DynamicTableSource source)

    • extractMetadataColumns

      public static List<org.apache.flink.table.catalog.Column.MetadataColumn> extractMetadataColumns(org.apache.flink.table.catalog.ResolvedSchema schema)

    • validateAndApplyMetadata

      public static void validateAndApplyMetadata(String tableDebugName, org.apache.flink.table.catalog.ResolvedSchema schema, org.apache.flink.table.connector.source.DynamicTableSource source, List<SourceAbilitySpec> sourceAbilities)