Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Design Guide—Generative AI in the Enterprise – Model Customization
Introduction Customizing Large Language Models Solution Components
Overivew
Model customization using NVIDIA AI Enterprise
Dell PowerEdge servers and NVIDIA GPUs
Dell PowerScale Storage
Dell and NVIDIA networking
Solution Architecture Validation Results Performance Characterization Dell Professional Services for Generative AI Conclusion References Sample Scripts

Dell PowerEdge servers and NVIDIA GPUs

Dell PowerEdge servers and NVIDIA GPUs

  • Dell Technologies provides a diverse selection of acceleration-optimized servers with an extensive portfolio of accelerators featuring NVIDIA GPUs.

    Dell Servers

    In this design, we showcase the PowerEdge XE9680 server, a flexible and high-performance AI application server purpose-built and optimized to develop, train, customize, and deploy LLMs and large datasets for advanced AI workloads.

    The PowerEdge XE9680 server enables fast time-to-value and no-compromise AI acceleration with two 4th generation Intel Xeon processors Intel CPUs with up to 56 cores, and up to eight NVIDIA H100 SXM5 Tensor Core GPUs for extreme performance. It also supports highly accelerated throughput with DDR5, NVLink, PCIe Gen 5.0, and NVMe SSDs to push the boundaries of data flow and computing possibilities.

    The server includes integrated NVIDIA NVLink technology to enable ultrafast communication between multiple NVIDIA GPUs, and supports NVSwitch to connect multiple NVLinks together for high scalability.

    GPU connectivity

    NVIDIA GPUs support various options to connect two or more GPUs, offering various bandwidths. GPU connectivity is often required for model customization, especially when higher performance and lower latency are crucial. Model customization is compute-resource intensive, requiring multiple GPUs. These GPUs require high-speed connectivity between them.

    NVIDIA NVLink is a high-speed interconnect technology developed by NVIDIA for connecting multiple NVIDIA GPUs to work in parallel. It allows for direct communication between the GPUs with high bandwidth and low latency, enabling them to share data and work collaboratively on compute-intensive tasks.

    NVIDIA NVSwitch is a high-performance and scalable switch technology that connects multiple NVLinks to provide all-to-all GPU communication at full NVLink speed within a single node and between nodes. Both are designed to facilitate high-bandwidth and low-latency data transfers, ideal for large-scale AI applications. The NVSwitch technology provides a bandwidth of 900 GB/s between any two GPUs.

    The following figure shows the NVIDIA GPU connectivity options for the PowerEdge XE9680 server used in this design:

    Figure 3. NVIDIA GPU connectivity in PowerEdge XE9680 servers