Package org.apache.flink.table.resource

Class ResourceManager

java.lang.Object
org.apache.flink.table.resource.ResourceManager
All Implemented Interfaces:
Closeable, AutoCloseable
@Internal public class ResourceManager extends Object implements Closeable
A manager for dealing with all user defined resource.

  • Field Details

    • localResourceDir

      protected final org.apache.flink.core.fs.Path localResourceDir

    • resourceInfos

      protected final Map<org.apache.flink.table.resource.ResourceUri,URL> resourceInfos

    • userClassLoader

      protected final org.apache.flink.util.MutableURLClassLoader userClassLoader

  • Constructor Details

    • ResourceManager

      public ResourceManager(org.apache.flink.configuration.ReadableConfig config, org.apache.flink.util.MutableURLClassLoader userClassLoader)

  • Method Details

    • createResourceManager

      public static ResourceManager createResourceManager(URL[] urls, ClassLoader parent, org.apache.flink.configuration.ReadableConfig config)

    • registerJarResources

      public void registerJarResources(List<org.apache.flink.table.resource.ResourceUri> resourceUris) throws IOException
      Due to anyone of the resource in list maybe fail during register, so we should stage it before actual register to guarantee transaction process. If all the resources are available, register them into the ResourceManager.
      Throws:
      IOException

    • registerFileResource

      public String registerFileResource(org.apache.flink.table.resource.ResourceUri resourceUri) throws IOException
      Register a file resource into ResourceManager and return the absolute local file path without the scheme.

      If the file is remote, it will be copied to a local file, with file name suffixed with a UUID.

      Parameters:
      resourceUri - resource with type as ResourceType.FILE, the resource uri might or might not contain the uri scheme, or it could be a relative path.
      Returns:
      the absolute local file path.
      Throws:
      IOException

    • declareFunctionResources

      public void declareFunctionResources(Set<org.apache.flink.table.resource.ResourceUri> resourceUris) throws IOException
      Declare a resource for function and add it to the function resource infos. If the file is remote, it will be copied to a local file. The declared resource will not be added to resources and classloader if it is not used in the job.
      Parameters:
      resourceUris - the resource uri for function.
      Throws:
      IOException

    • unregisterFunctionResources

      public void unregisterFunctionResources(List<org.apache.flink.table.resource.ResourceUri> resourceUris)
      Unregister the resource uri in function resources, when the reference count of the resource is 0, the resource will be removed from the function resources.
      Parameters:
      resourceUris - the uris to unregister in function resources.

    • registerPythonResources

      public void registerPythonResources()

    • getUserClassLoader

      public URLClassLoader getUserClassLoader()

    • createUserClassLoader

      public URLClassLoader createUserClassLoader(List<org.apache.flink.table.resource.ResourceUri> resourceUris)

    • getResources

      public Map<org.apache.flink.table.resource.ResourceUri,URL> getResources()

    • getLocalJarResources

      public Set<URL> getLocalJarResources()
      Get the local jars' URL. Return the URL corresponding to downloaded jars in the local file system for the remote jar. For the local jar, return the registered URL.

    • addJarConfiguration

      public void addJarConfiguration(TableConfig tableConfig)
      Adds the local jar resources to the given TableConfig. It implicitly considers the TableConfig.getRootConfiguration() and stores the merged result into TableConfig.getConfiguration().

    • copy

      public ResourceManager copy()

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • exists

      public boolean exists(org.apache.flink.core.fs.Path filePath) throws IOException
      Check whether the Path exists.
      Throws:
      IOException

    • syncFileResource

      public void syncFileResource(org.apache.flink.table.resource.ResourceUri resourceUri, Consumer<String> resourceGenerator) throws IOException
      Generate a local file resource by the given resource generator and then synchronize to the path identified by the given ResourceUri. The path passed to resource generator should be a local path retrieved from the given ResourceUri.

      NOTE: if the given ResourceUri represents a remote file path like "hdfs://path/to/file.json", then the retrieved local path will be "/localResourceDir/file-${uuid}.json"

      Parameters:
      resourceUri - the file resource uri to synchronize to
      resourceGenerator - a consumer that generates a local copy of the file resource
      Throws:
      IOException

    • checkPath

      protected void checkPath(org.apache.flink.core.fs.Path path, org.apache.flink.table.resource.ResourceType expectedType) throws IOException
      Throws:
      IOException

    • getURLFromPath

      @VisibleForTesting protected URL getURLFromPath(org.apache.flink.core.fs.Path path) throws IOException
      Throws:
      IOException