NvidiaGPUPluginForRuntimeV2 (Apache Hadoop YARN NodeManager 3.4.1.100-eep-941 API)

java.lang.Object
- org.apache.hadoop.yarn.server.nodemanager.containermanager.resourceplugin.com.nvidia.NvidiaGPUPluginForRuntimeV2

All Implemented Interfaces:

DevicePlugin, DevicePluginScheduler
```
public class NvidiaGPUPluginForRuntimeV2
extends java.lang.Object
implements DevicePlugin, DevicePluginScheduler
```
Nvidia GPU plugin supporting both Nvidia container runtime v2 for Docker and non-Docker container. It has topology aware as well as simple scheduling ability.

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`static class`	`NvidiaGPUPluginForRuntimeV2.DeviceLinkType`	Different type of link.
`class`	`NvidiaGPUPluginForRuntimeV2.NvidiaCommandExecutor`	A shell wrapper class easy for test.

Field Summary

Fields
Modifier and Type	Field	Description
`static org.slf4j.Logger`	`LOG`
`static java.lang.String`	`NV_RESOURCE_NAME`
`static java.lang.String`	`TOPOLOGY_POLICY_ENV_KEY`	The container can set this environment variable.
`static java.lang.String`	`TOPOLOGY_POLICY_PACK`	Schedule policy that prefer the faster GPU-GPU communication.
`static java.lang.String`	`TOPOLOGY_POLICY_SPREAD`	Schedule policy that prefer the faster CPU-GPU communication.

Constructor Summary

Constructors
Constructor Description

NvidiaGPUPluginForRuntimeV2()

Constructors
Constructor	Description
`NvidiaGPUPluginForRuntimeV2()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`java.util.Set<Device>`	`allocateDevices(java.util.Set<Device> availableDevices, int count, java.util.Map<java.lang.String,java.lang.String> envs)`	Called when allocating devices.
`void`	`basicSchedule(java.util.Set<Device> allocation, int count, java.util.Set<Device> availableDevices)`
`int`	`computeCostOfDevices(Device[] devices)`	The cost function used to calculate costs of a sub set of devices.
`java.util.Map<java.lang.Integer,java.util.List<java.util.Map.Entry<java.util.Set<Device>,java.lang.Integer>>>`	`getCostTable()`
`java.util.Map<java.lang.String,java.lang.Integer>`	`getDevicePairToWeight()`
`java.util.Set<Device>`	`getDevices()`	Called when update node resource.
`DeviceRegisterRequest`	`getRegisterRequestInfo()`	Called first when device plugin framework wants to register.
`void`	`initCostTable()`
`boolean`	`isTopoInitialized()`
`DeviceRuntimeSpec`	`onDevicesAllocated(java.util.Set<Device> allocatedDevices, YarnRuntimeType yarnRuntime)`	Asking how these devices should be prepared/used before/when container launch.
`void`	`onDevicesReleased(java.util.Set<Device> releasedDevices)`	Called after device released.
`void`	`parseTopo(java.lang.String topo, java.util.Map<java.lang.String,java.lang.Integer> deviceLinkToWeight)`	A typical sample topo output: GPU0 GPU1 GPU2 GPU3 CPU Affinity GPU0 X PHB SOC SOC 0-31 GPU1 PHB X SOC SOC 0-31 GPU2 SOC SOC X PHB 0-31 GPU3 SOC SOC PHB X 0-31 Legend: X = Self SOC = Connection traversing PCIe as well as the SMP link between CPU sockets(e.g.
`void`	`setPathOfGpuBinary(java.lang.String pOfGpuBinary)`
`void`	`setShellExecutor(NvidiaGPUPluginForRuntimeV2.NvidiaCommandExecutor shellExecutor)`
`void`	`topologyAwareSchedule(java.util.Set<Device> allocation, int count, java.util.Map<java.lang.String,java.lang.String> envs, java.util.Set<Device> availableDevices, java.util.Map<java.lang.Integer,java.util.List<java.util.Map.Entry<java.util.Set<Device>,java.lang.Integer>>> cTable)`	Topology Aware schedule algorithm.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - LOG
```
public static final org.slf4j.Logger LOG
```
  - NV_RESOURCE_NAME
```
public static final java.lang.String NV_RESOURCE_NAME
```
    See Also:
    
    Constant Field Values
  - TOPOLOGY_POLICY_ENV_KEY
```
public static final java.lang.String TOPOLOGY_POLICY_ENV_KEY
```
    The container can set this environment variable. To tell the scheduler what's the policy to use when do scheduling
    
    See Also:
    
    Constant Field Values
  - TOPOLOGY_POLICY_PACK
```
public static final java.lang.String TOPOLOGY_POLICY_PACK
```
    Schedule policy that prefer the faster GPU-GPU communication. Suitable for heavy GPU computation workload generally.
    
    See Also:
    
    Constant Field Values
  - TOPOLOGY_POLICY_SPREAD
```
public static final java.lang.String TOPOLOGY_POLICY_SPREAD
```
    Schedule policy that prefer the faster CPU-GPU communication. Suitable for heavy CPU-GPU IO operations generally.
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - NvidiaGPUPluginForRuntimeV2
```
public NvidiaGPUPluginForRuntimeV2()
```
- Method Detail
  - getRegisterRequestInfo
```
public DeviceRegisterRequest getRegisterRequestInfo()
                                             throws java.lang.Exception
```
    Description copied from interface: DevicePlugin
    
    Called first when device plugin framework wants to register.
    
    Specified by:
    
    getRegisterRequestInfo in interface DevicePlugin
    
    Returns:
    
    DeviceRegisterRequest DeviceRegisterRequest
    
    Throws:
    
    java.lang.Exception
  - getDevices
```
public java.util.Set<Device> getDevices()
                                 throws java.lang.Exception
```
    Description copied from interface: DevicePlugin
    
    Called when update node resource.
    
    Specified by:
    
    getDevices in interface DevicePlugin
    
    Returns:
    
    a set of Device, TreeSet recommended
    
    Throws:
    
    java.lang.Exception
  - onDevicesAllocated
```
public DeviceRuntimeSpec onDevicesAllocated(java.util.Set<Device> allocatedDevices,
                                            YarnRuntimeType yarnRuntime)
                                     throws java.lang.Exception
```
    Description copied from interface: DevicePlugin
    
    Asking how these devices should be prepared/used before/when container launch. A plugin can do some tasks in its own or define it in DeviceRuntimeSpec to let the framework do it. For instance, define VolumeSpec to let the framework to create volume before running container.
    
    Specified by:
    
    onDevicesAllocated in interface DevicePlugin
    
    Parameters:
    
    allocatedDevices - A set of allocated Device.
    
    yarnRuntime - Indicate which runtime YARN will use Could be RUNTIME_DEFAULT or RUNTIME_DOCKER in DeviceRuntimeSpec constants. The default means YARN's non-docker container runtime is used. The docker means YARN's docker container runtime is used.
    
    Returns:
    
    a DeviceRuntimeSpec description about environment, VolumeSpec, MountVolumeSpec. etc
    
    Throws:
    
    java.lang.Exception
  - onDevicesReleased
```
public void onDevicesReleased(java.util.Set<Device> releasedDevices)
                       throws java.lang.Exception
```
    Description copied from interface: DevicePlugin
    
    Called after device released.
    
    Specified by:
    
    onDevicesReleased in interface DevicePlugin
    
    Parameters:
    
    releasedDevices - A set of released devices
    
    Throws:
    
    java.lang.Exception
  - allocateDevices
```
public java.util.Set<Device> allocateDevices(java.util.Set<Device> availableDevices,
                                             int count,
                                             java.util.Map<java.lang.String,java.lang.String> envs)
```
    Description copied from interface: DevicePluginScheduler
    
    Called when allocating devices. The framework will do all device book keeping and fail recovery. So this hook could be stateless and only do scheduling based on available devices passed in. It could be invoked multiple times by the framework. The hint in environment variables passed in could be potentially used in making better scheduling decision. For instance, GPU scheduling might support different kind of policy. The container can set it through environment variables.
    
    Specified by:
    
    allocateDevices in interface DevicePluginScheduler
    
    Parameters:
    
    availableDevices - Devices allowed to be chosen from.
    
    count - Number of device to be allocated.
    
    envs - Environment variables of the container.
    
    Returns:
    
    A set of Device allocated
  - initCostTable
```
@VisibleForTesting
public void initCostTable()
                   throws java.io.IOException
```
    Throws:
    
    java.io.IOException
  - computeCostOfDevices
```
@VisibleForTesting
public int computeCostOfDevices(Device[] devices)
```
    The cost function used to calculate costs of a sub set of devices. It calculate link weight of each pair in non-duplicated combination of devices.
  - topologyAwareSchedule
```
@VisibleForTesting
public void topologyAwareSchedule(java.util.Set<Device> allocation,
                                  int count,
                                  java.util.Map<java.lang.String,java.lang.String> envs,
                                  java.util.Set<Device> availableDevices,
                                  java.util.Map<java.lang.Integer,java.util.List<java.util.Map.Entry<java.util.Set<Device>,java.lang.Integer>>> cTable)
```
    Topology Aware schedule algorithm. It doesn't consider CPU affinity or NUMA or bus bandwidths. It support two plicy: "spread" and "pack" which can be set by container's environment variable. Use pack by default which means prefer the faster GPU-GPU. "Spread" means prefer the faster CPU-GPU. It can potentially be extend to take GPU attribute like GPU chip memory into consideration.
  - basicSchedule
```
@VisibleForTesting
public void basicSchedule(java.util.Set<Device> allocation,
                          int count,
                          java.util.Set<Device> availableDevices)
```
  - parseTopo
```
public void parseTopo(java.lang.String topo,
                      java.util.Map<java.lang.String,java.lang.Integer> deviceLinkToWeight)
```
    A typical sample topo output: GPU0 GPU1 GPU2 GPU3 CPU Affinity GPU0 X PHB SOC SOC 0-31 GPU1 PHB X SOC SOC 0-31 GPU2 SOC SOC X PHB 0-31 GPU3 SOC SOC PHB X 0-31 Legend: X = Self SOC = Connection traversing PCIe as well as the SMP link between CPU sockets(e.g. QPI) PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU) PXB = Connection traversing multiple PCIe switches (without traversing the PCIe Host Bridge) PIX = Connection traversing a single PCIe switch NV# = Connection traversing a bonded set of # NVLinks」
  - setPathOfGpuBinary
```
@VisibleForTesting
public void setPathOfGpuBinary(java.lang.String pOfGpuBinary)
```
  - setShellExecutor
```
@VisibleForTesting
public void setShellExecutor(NvidiaGPUPluginForRuntimeV2.NvidiaCommandExecutor shellExecutor)
```
  - isTopoInitialized
```
@VisibleForTesting
public boolean isTopoInitialized()
```
  - getCostTable
```
@VisibleForTesting
public java.util.Map<java.lang.Integer,java.util.List<java.util.Map.Entry<java.util.Set<Device>,java.lang.Integer>>> getCostTable()
```
  - getDevicePairToWeight
```
@VisibleForTesting
public java.util.Map<java.lang.String,java.lang.Integer> getDevicePairToWeight()
```

Class NvidiaGPUPluginForRuntimeV2

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

LOG

NV_RESOURCE_NAME

TOPOLOGY_POLICY_ENV_KEY

TOPOLOGY_POLICY_PACK

TOPOLOGY_POLICY_SPREAD

Constructor Detail

NvidiaGPUPluginForRuntimeV2

Method Detail

getRegisterRequestInfo

getDevices

onDevicesAllocated

onDevicesReleased

allocateDevices

initCostTable

computeCostOfDevices

topologyAwareSchedule

basicSchedule

parseTopo

setPathOfGpuBinary

setShellExecutor

isTopoInitialized

getCostTable

getDevicePairToWeight