Package org.apache.flink.table.utils

Class PartitionPathUtils

java.lang.Object
org.apache.flink.table.utils.PartitionPathUtils
@Internal public class PartitionPathUtils extends Object
Utils for file system.

  • Constructor Details

    • PartitionPathUtils

      public PartitionPathUtils()

  • Method Details

    • generatePartitionPath

      public static String generatePartitionPath(LinkedHashMap<String,String> partitionSpec)
      Make partition path from partition spec.
      Parameters:
      partitionSpec - The partition spec.
      Returns:
      An escaped, valid partition name.

    • extractPartitionValues

      public static List<String> extractPartitionValues(org.apache.flink.core.fs.Path currPath)
      Make partition values from path.
      Parameters:
      currPath - partition file path.
      Returns:
      Sequential partition specs.

    • extractPartitionSpecFromPath

      public static LinkedHashMap<String,String> extractPartitionSpecFromPath(org.apache.flink.core.fs.Path currPath)
      Make partition spec from path.
      Parameters:
      currPath - partition file path.
      Returns:
      Sequential partition specs.

    • unescapePathName

      public static String unescapePathName(String path)

    • listStatusWithoutHidden

      public static org.apache.flink.core.fs.FileStatus[] listStatusWithoutHidden(org.apache.flink.core.fs.FileSystem fs, org.apache.flink.core.fs.Path dir) throws IOException
      List file status without hidden files.
      Throws:
      IOException

    • searchPartSpecAndPaths

      public static List<org.apache.flink.api.java.tuple.Tuple2<LinkedHashMap<String,String>,org.apache.flink.core.fs.Path>> searchPartSpecAndPaths(org.apache.flink.core.fs.FileSystem fs, org.apache.flink.core.fs.Path path, int partitionNumber)
      Search all partitions in this path.
      Parameters:
      path - search path.
      partitionNumber - partition number, it will affect path structure.
      Returns:
      all partition specs to its path.

    • fillPartitionValueForRecord

      public static GenericRowData fillPartitionValueForRecord(String[] fieldNames, DataType[] fieldTypes, int[] selectFields, List<String> partitionKeys, org.apache.flink.core.fs.Path path, String defaultPartValue)
      Extract partition value from path and fill to record.
      Parameters:
      fieldNames - record field names.
      fieldTypes - record field types.
      selectFields - the selected fields.
      partitionKeys - the partition field names.
      path - the file path that the partition located.
      defaultPartValue - default value of partition field.
      Returns:
      the filled record.

    • convertStringToInternalValue

      public static Object convertStringToInternalValue(String valStr, DataType type)
      Restore partition value from string and type.
      Parameters:
      valStr - string partition value.
      type - type of partition field.
      Returns:
      partition value.