Package org.apache.flink.table.planner.plan.nodes.exec

Annotation Type ExecNodeMetadata

@Documented @Target(TYPE) @Retention(RUNTIME) @Repeatable(MultipleExecNodeMetadata.class) @Internal public @interface ExecNodeMetadata
Annotation to be used for ExecNodes to keep necessary metadata when serializing/deserializing them in a plan. It's used for internal bookkeeping across Flink versions, and to provide necessary information to the testing infrastructure.

Each ExecNode needs to be annotated and provide the necessary metadata info so that it can be correctly serialized and later on instantiated from a string (JSON) plan.

It's possible for one ExecNode class to use multiple annotations to denote ability to upgrade to more versions. an ExecNode class can be annotated directly with multiple ExecNodeMetadata annotations, or with a single MultipleExecNodeMetadata annotation where the MultipleExecNodeMetadata.value() is an array of ExecNodeMetadata annotations.

  • Required Element Summary

    Required Elements
    Modifier and Type
    Required Element
    Description
    org.apache.flink.FlinkVersion
    minPlanVersion
    Used for plan validation and potentially plan migration.
    org.apache.flink.FlinkVersion
    minStateVersion
    Used for operator and potentially savepoint migration.
    String
    name
    Unique name of the ExecNode for serialization/deserialization and uid() generation.
    int
    version
    A positive integer denoting the evolving version of an ExecNode, used for serialization/deserialization and uid() generation.

  • Optional Element Summary

    Optional Elements
    Modifier and Type
    Optional Element
    Description
    String[]
    consumedOptions
    Hard coded list of ExecutionConfigOptions keys of in the Flink version when the ExecNode was added.
    String[]
    producedTransformations
    Set of transformation names that can be part of the resulting Transformations.

  • Element Details

    • name

      String name
      Unique name of the ExecNode for serialization/deserialization and uid() generation. Together with version, uniquely identifies the ExecNode class.

    • version

      int version
      A positive integer denoting the evolving version of an ExecNode, used for serialization/deserialization and uid() generation. Together with name(), uniquely identifies the ExecNode class.

    • minPlanVersion

      org.apache.flink.FlinkVersion minPlanVersion
      Used for plan validation and potentially plan migration.

      Needs to be updated when the JSON for the ExecNode changes: e.g. after adding an attribute to the JSON spec of the ExecNode.

      The annotation does not need to be updated for every Flink version. As the name suggests it is about the "minimum" version for a restore. If the minimum version is higher than the current Flink version, plan migration is necessary.

      Changing this version will always result in a new version() for the ExecNode.

      Plan migration tests can use this information.

      Completeness tests can verify that restore tests exist for all JSON plan variations.

    • minStateVersion

      org.apache.flink.FlinkVersion minStateVersion
      Used for operator and potentially savepoint migration.

      Needs to be updated whenever the state layout of an ExecNode changes. In some cases, the operator can implement and perform state migration. If the minimum version is higher than the current Flink version, savepoint migration is necessary.

      Changing this version will always result in a new ExecNode version().

      Restore tests can verify that operator migration works for all Flink state versions.

      Completeness tests can verify that restore tests exist for all state variations.

    • consumedOptions

      String[] consumedOptions
      Hard coded list of ExecutionConfigOptions keys of in the Flink version when the ExecNode was added. Does not reference instances in the ExecutionConfigOptions class in case those get refactored.

      Completeness tests can verify that every option is set once in restore and change detection tests.

      Completeness tests can verify that the ExecutionConfigOptions class still contains an option (via key or fallback key) for the given key.

      Restore can verify whether the restored ExecNode config map contains only options of the given keys.

      Common options used for all StreamExecNodes:

      • ExecutionConfigOptions.TABLE_EXEC_SIMPLIFY_OPERATOR_NAME_ENABLED
      • ExecutionConfigOptions.TABLE_EXEC_RESOURCE_DEFAULT_PARALLELISM
      Default:
      {}

    • producedTransformations

      String[] producedTransformations
      Set of transformation names that can be part of the resulting Transformations.

      Restore and completeness tests can verify there exists at least one test that adds each operator and that the created Transformations contain only operators with Transformation.getUid() containing the given operator names.

      The concrete combinations or existence of these operators in the final pipeline depends on various parameters (both configuration and ExecNode-specific arguments such as interval size etc.).

      Default:
      {}