This section provides guidelines for deploying a GPU cluster with vSphere 7.0 with A100 or A30 GPUs on PowerEdge servers. We focus on enabling MIG and configuring the VMs with access to a virtual GPU (vGPU).

Installing and configuring the NVIDIA AI Enterprise Host software 

See the NVIDIA AI Enterprise platform deployment guide for detailed instructions about installing and configuring the NVIDIA Virtual GPU Manager for vSphere, which includes these high-level steps:

1. Use the vSphere Installation bundle (VIB) to install the NVIDIA Virtual GPU Manager for vSphere. 

2. Configure vSphere vMotion with vGPU for vSphere by enabling an advanced vCenter Server setting.
3. Change the Default Graphics Type in vSphere. 
4. Configure a GPU for MIG-backed vGPUs. By default, MIG mode is not enabled on the A100 or A30 GPU.

Configuring a VM to use MIG

Follow these steps to configure a VM to use MIG:

1. Create a VM, provide virtual processor and memory configurations, and then assign a MIG partition, as shown in the following figure:

Figure 6.        Creating a VM and assigning a MIG profile

2. Boot your guest operating system in the VM in EFI mode or UEFI mode for correct GPU use:
   1. In the vSphere Client, select your VM.
   2. Go to Edit Settings àVM Options à Boot Options. In the Firmware field, ensure that UEFI or EFI is enabled, as shown in the following figure: 

Figure 7.        VM boot options

3. Click Advanced àConfiguration Parameters àEdit Configuration and edit the following parameters:   

pciPassthru.use64bitMMIO: TRUE   

pciPassthru.allowP2P: TRUE   

pciPassthru.64bitMMIOSizeGB: 64  

4. Install the Linux operating system on the VM. 
5. Install the NVIDIA vGPU Software Graphics Driver on the VM. 
6. License GRID vGPU on the VM.

For detailed instructions for steps 4 through 6, see the Virtual GPU Software User Guide.

Configuring a VM with multiple GPUs

The PowerEdge R750xa server supports configuring a maximum of two GPUs per VM. These GPUs are required to be connected through NVLink for peer-to-peer communication. The following steps determine which GPUs are connected through NVLink in the PowerEdge R750xa server:

1. Ensure that MIG mode is disabled on the PowerEdge server. MultiGPU configuration is supported only in non-MIG mode.

2. Detect the topology between the GPUs by typing the following command:

[root@esxi01:~] nvidia-smi topo -m

        GPU0    GPU1    GPU2    GPU3    CPU Affinity    NUMA Affinity

GPU0     X      SYS     SYS     SYS

GPU1    SYS      X      SYS     SYS

GPU2    SYS     SYS      X      SYS

GPU3    SYS     SYS     SYS      X

Legend:

  X    = Self

  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)

  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node

  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)

  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)

  PIX  = Connection traversing at most a single PCIe bridge

  NV#  = Connection traversing a bonded set of # NVLinks

The preceding example output is from a PowerEdge R750xa system with four A100 GPUs. GPU0 and GPU1 are connected by NVLink, as are GPU2 and GPU3. Assign GPU0 and GPU1 to a single VM.

3. Obtain the hexid from nvidia-smi by typing the following command:

4. In the VM advanced configuration, add keys for two GPUs:

pciPassthru0.cfg.gpu-pci-id = "00000000:17:00.0"

pciPassthru0.cfg.gpu-pci-id = "00000000:65:00.0"

Mount datasets from PowerScale to the VM

Use the following steps to mount AI datasets from PowerScale to the VMs:

1. Use SSH to connect to the Linux VM.

2. Create a directory to serve as the mount point for the remote NFS share: 

sudo mkdir /mnt/dataset

3. Mount the NFS share by running the following command as root or user with sudo privileges: 

sudo mount -t nfs <NFS IP addr>:<path> /mnt/dataset 

4. Verify that the remote NFS volume is successfully mounted: 

df -h

Installing NVIDIA AI Enterprise Suite

The AI and data science applications and frameworks are distributed as NGC container images through the NVIDIA NGC Enterprise Catalog. Each container image contains the entire user-space software stack that is required to run the application or framework such as the CUDA libraries, cuDNN, any required Magnum IO components, TensorRT, and the framework. See the Installing AI and data science applications and Frameworks chapter in the NVIDIA AI Enterprise documentation.

Multinode training using GPUDirect RDMA

You can configure multiple VMs to run distributed training of a neural network. Key steps in the configuration include:

• Disabling MIG on the GPU and assigning a non-MIG profile to the VMs
• Configuring a ConnectX network adapter and assigning it to a VM in passthrough mode 
• Ensuring the GPU and NIC pairs are on the same root complex and same NUMA node
• Configuring RDMA over Converged Ethernet (RoCE) on the network adapter and network switches

For detailed instructions about how to set up multinode training using GPUDirect RDMA, see NVIDIA’s AI Practitioners Deployment Guides.