Package org.apache.flink.table.legacy.sinks

Interface PartitionableTableSink

@Deprecated @Internal public interface PartitionableTableSink
Deprecated.
This interface will not be supported in the new sink design around DynamicTableSink. Use SupportsPartitioning instead. See FLIP-95 for more information.
An interface for partitionable TableSink. A partitionable sink can writes query results to partitions.

Partition columns are defined via catalog table.

For example, a partitioned table named my_table with a table schema [a INT, b VARCHAR, c DOUBLE, dt VARCHAR, country VARCHAR] is partitioned on columns dt, country. Then dt is the first partition column, and country is the secondary partition column.

We can insert data into table partitions using INSERT INTO PARTITION syntax, for example: 

 
     INSERT INTO my_table PARTITION (dt='2019-06-20', country='bar') select a, b, c from my_view

When all the partition columns are set a value in PARTITION clause, it is inserting into a static partition. It will writes the query result into a static partition, i.e. dt='2019-06-20', country='bar'. The user specified static partitions will be told to the sink via setStaticPartition(Map).

The INSERT INTO PARTITION syntax also supports dynamic partition inserts. 

 
     INSERT INTO my_table PARTITION (dt='2019-06-20') select a, b, c, country from another_view

When partial partition columns (prefix part of all partition columns) are set a value in PARTITION clause, it is writing the query result into a dynamic partition. In the above example, the static partition part is dt='2019-06-20' which will be told to the sink via setStaticPartition(Map). And the country is the dynamic partition which will be get from each record.

  • Method Summary

    Modifier and Type
    Method
    Description
    default boolean
    configurePartitionGrouping(boolean supportsGrouping)
    Deprecated.
    If returns true, sink can trust all records will definitely be grouped by partition fields before consumed by the TableSink, i.e. the sink will receive all elements of one partition and then all elements of another partition, elements of different partitions will not be mixed.
    void
    setStaticPartition(Map<String,String> partitions)
    Deprecated.
    Sets the static partition into the TableSink.

  • Method Details

    • setStaticPartition

      void setStaticPartition(Map<String,String> partitions)
      Deprecated.
      Sets the static partition into the TableSink. The static partition may be partial of all partition columns. See the class Javadoc for more details.

      The static partition is represented as a Map<String, String> which maps from partition field name to partition value. The partition values are all encoded as strings, i.e. encoded using String.valueOf(...). For example, if we have a static partition f0=1024, f1="foo", f2="bar". f0 is an integer type, f1 and f2 are string types. They will all be encoded as strings: "1024", "foo", "bar". And can be decoded to original literals based on the field types.

      Parameters:
      partitions - user specified static partition

    • configurePartitionGrouping

      default boolean configurePartitionGrouping(boolean supportsGrouping)
      Deprecated.
      If returns true, sink can trust all records will definitely be grouped by partition fields before consumed by the TableSink, i.e. the sink will receive all elements of one partition and then all elements of another partition, elements of different partitions will not be mixed. For some sinks, this can be used to reduce number of the partition writers to improve writing performance.

      This method is used to configure the behavior of input whether to be grouped by partition, if true, at the same time the sink should also configure itself, i.e. set an internal field that changes the writing behavior (writing one partition at a time).

      Parameters:
      supportsGrouping - whether the execution mode supports grouping, e.g. grouping (usually use sort to implement) is only supported in batch mode, not supported in streaming mode.
      Returns:
      whether data need to be grouped by partition before consumed by the sink. Default is false. If supportsGrouping is false, it should never return true (requires grouping), otherwise it will fail.