Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Design Guide—Virtualizing GPUs for AI with VMware and NVIDIA Based on Dell Infrastructure
Introduction Architectural Concepts Solution Architecture
Architecture overview
Recommended configurations
Design considerations
Software architecture
Network architecture
Validation and Machine Learning Performance Conclusion References

Recommended configurations

Recommended configurations

  • The following table provides the recommended configurations for customer usage scenarios:

    Table 4.      Recommended configurations

    Configuration

    Mainstream performance

    High performance

    Compute server

    PowerEdge R760 or PowerEdge R7625

    PowerEdge R760 or PowerEdge R7625

    GPUs

    1 x NVIDIA A30

    2 x NVIDIA A100

    Number of nodes in a cluster

    Minimum 3 hosts. 4 ESXi hosts are recommended when using vSAN for resiliency during patching and updating.

    Network adapter

    ConnectX-6 Lx 25 GbE

    ConnectX-6 Lx 25 GbE

    Network switch

    2 x Dell S5248F-ON

    2 x Dell S5248F-ON

    Out-of-band (OOB) switch

    1 x Dell PowerSwitch N3248TE-ON

    VMware vSphere
    • VMware vSphere 8.0
    • VMware vSphere with Tanzu (Required for container orchestration)

    Internal storage for ESXi

    BOSS controller card with 2 M.2 Sticks 480 GB (RAID 1)

    Drive configuration for vSAN Original Storage Architecture (OSA)
    • Storage controllers: HBA355i
    • Cache tier: 2 x 800 GB SSD SAS Mixed-Use
    • Capacity tier: 6 x 1.92 TB SSD SAS Read Intensive

    Drive configuration for vSAN 8 ESA
    • 6 x 1.92 TB NVMe AGN Mixed-Use

    Storage
    • vSAN for VM storage
    • PowerScale all flash (F600, F800) or hybrid nodes (H400, H500, H600, H5600) for data lake

    NVIDIA software

    NVIDIA AI Enterprise 3.1

    Mainstream performance includes model development and inference on PowerEdge R760 and R7625 servers with NVIDIA A30 GPUs. It includes use cases like conversational AI, recommendation systems, and natural language processing. A high-performance configuration includes the PowerEdge R760/R7625 server with NVIDIA A100 GPUs. High-performance use cases include neural network training of large and complex models. High-performance configurations also include network architecture for multi-node training if customers want to harness GPU resources across multiple servers.