Package org.apache.flink.python.env

Class AbstractPythonEnvironmentManager

java.lang.Object

org.apache.flink.python.env.AbstractPythonEnvironmentManager

All Implemented Interfaces:

AutoCloseable, PythonEnvironmentManager

Direct Known Subclasses:

EmbeddedPythonEnvironmentManager, ProcessPythonEnvironmentManager

@Internal
public abstract class AbstractPythonEnvironmentManager
extends Object
implements PythonEnvironmentManager

The base class of python environment manager which is used to create the PythonEnvironment object used to execute Python functions.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	AbstractPythonEnvironmentManager.PythonLeasedResource	Python lease resource which includes environment variables and working directory of execution python environment.

Field Summary

Fields

Modifier and Type	Field	Description
protected final PythonDependencyInfo	dependencyInfo
static final String	PYFLINK_GATEWAY_DISABLED
static final String	PYTHON_ARCHIVES_DIR
static final String	PYTHON_FILES_DIR
static final String	PYTHON_REQUIREMENTS_CACHE
static final String	PYTHON_REQUIREMENTS_DIR
static final String	PYTHON_REQUIREMENTS_FILE
static final String	PYTHON_REQUIREMENTS_INSTALL_DIR
static final String	PYTHON_WORKING_DIR
protected AbstractPythonEnvironmentManager.PythonLeasedResource	resource

Constructor Summary

Constructors

Constructor	Description
AbstractPythonEnvironmentManager(PythonDependencyInfo dependencyInfo, String[] tmpDirectories, Map<String,String> systemEnv, org.apache.flink.api.common.JobID jobID)

Method Summary

Modifier and Type	Method	Description
void	close()
Map<String,String>	constructEnvironmentVariables(String baseDirectory)	Constructs the environment variables which is used to launch the python UDF worker.
String	getBaseDirectory()
Map<String,String>	getPythonEnv()
void	open()	Initialize the environment manager.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.python.env.PythonEnvironmentManager

createEnvironment

Field Details

resource

protected transient AbstractPythonEnvironmentManager.PythonLeasedResource resource

dependencyInfo

protected final PythonDependencyInfo dependencyInfo

PYTHON_REQUIREMENTS_DIR

@VisibleForTesting
public static final String PYTHON_REQUIREMENTS_DIR

See Also:

• Constant Field Values

PYTHON_REQUIREMENTS_FILE

@VisibleForTesting
public static final String PYTHON_REQUIREMENTS_FILE

See Also:

• Constant Field Values

PYTHON_REQUIREMENTS_CACHE

@VisibleForTesting
public static final String PYTHON_REQUIREMENTS_CACHE

See Also:

• Constant Field Values

PYTHON_REQUIREMENTS_INSTALL_DIR

@VisibleForTesting
public static final String PYTHON_REQUIREMENTS_INSTALL_DIR

See Also:

• Constant Field Values

PYTHON_WORKING_DIR

@VisibleForTesting
public static final String PYTHON_WORKING_DIR

See Also:

• Constant Field Values

PYTHON_FILES_DIR

@VisibleForTesting
public static final String PYTHON_FILES_DIR

See Also:

• Constant Field Values

PYTHON_ARCHIVES_DIR

@VisibleForTesting
public static final String PYTHON_ARCHIVES_DIR

See Also:

• Constant Field Values

PYFLINK_GATEWAY_DISABLED

@VisibleForTesting
public static final String PYFLINK_GATEWAY_DISABLED

See Also:

• Constant Field Values

Constructor Details

AbstractPythonEnvironmentManager

public AbstractPythonEnvironmentManager(PythonDependencyInfo dependencyInfo,
 String[] tmpDirectories,
 Map<String,String> systemEnv,
 org.apache.flink.api.common.JobID jobID)

Method Details

open

public void open()
          throws Exception

Description copied from interface: PythonEnvironmentManager

Initialize the environment manager.

Specified by:

open in interface PythonEnvironmentManager

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception

getBaseDirectory

@VisibleForTesting
public String getBaseDirectory()

getPythonEnv

@VisibleForTesting
public Map<String,String> getPythonEnv()

constructEnvironmentVariables

@VisibleForTesting
public Map<String,String> constructEnvironmentVariables(String baseDirectory)
                                                 throws IOException

Constructs the environment variables which is used to launch the python UDF worker.

Returns:

The environment variables which contain the paths of the python dependencies.

Throws:

IOException