Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Implementation Guide—Virtualizing GPUs for AI with VMware and NVIDIA Based on Dell Infrastructure
Introduction
Solution introduction
Document purpose
Audience
Document introduction
Revision history
Solution Components Deploying the Solution Verification Summary References

Document introduction

Document introduction

  • With VMware vSphere support for virtualized GPUs, IT administrators can run AI workloads such as neural network training, inference, or model development alongside their standard data center applications. 

    The following figure shows the high-level architecture for this validated design with PowerEdge R760 and R7625 servers, each with two NVIDIA A100 GPUs and a ConnectX network adapter, as part of a VMware vSphere cluster:

    Figure 1.   High-level architecture showing two scenarios for running AI workloads: virtual machines (VMs) and pods

    Customers can run AI workloads on VMware vSphere and access virtualized NVIDIA GPUs using two scenarios:

    • Scenario 1: Running AI workloads as VMs—Partitioned, full, or multiple GPUs are allocated to VMs. Required software and drives are installed in the VMs for workloads to take advantage of accelerated hardware.

    The following figure shows the high-level steps for deploying the solution:

    Figure 2.   Steps in deploying AI Workloads as VMs

    • Scenario 2: Running AI workloads as Kubernetes pods in a Tanzu Kubernetes cluster (TKC)—Worker node templates incorporate GPU resources. Worker nodes can be provisioned dynamically. NVIDIA operators automatically configure the worker nodes for workloads to take advantage of the accelerated hardware. Also, Tanzu offers a rich ecosystem of software products to manage, monitor, and operate the Kubernetes environment.

    The following figure shows the high-level steps for deploying the solution:

    Figure 3.   Steps in deploying AI workloads as Kubernetes pods

    Also, vSphere with Tanzu allows running VMs in Tanzu namespaces. Implementing this scenario requires combining steps from the preceding two methods and is outside the scope of this implementation guide.