Dell Technologies provides a diverse selection of acceleration-optimized servers with an extensive portfolio of accelerators featuring NVIDIA GPUs.
In this design, we showcase the PowerEdge XE9680 server, a flexible and high-performance AI application server purpose-built and optimized to develop, train, customize, and deploy LLMs and large datasets for advanced AI workloads.
The PowerEdge XE9680 server enables fast time-to-value and no-compromise AI acceleration with two 4th generation Intel Xeon processors Intel CPUs with up to 56 cores, and up to eight NVIDIA H100 SXM5 Tensor Core GPUs for extreme performance. It also supports highly accelerated throughput with DDR5, NVLink, PCIe Gen 5.0, and NVMe SSDs to push the boundaries of data flow and computing possibilities.
The server includes integrated NVIDIA NVLink technology to enable ultrafast communication between multiple NVIDIA GPUs, and supports NVSwitch to connect multiple NVLinks together for high scalability.
NVIDIA GPUs support various options to connect two or more GPUs, offering various bandwidths. GPU connectivity is often required for model customization, especially when higher performance and lower latency are crucial. Model customization is compute-resource intensive, requiring multiple GPUs. These GPUs require high-speed connectivity between them.
NVIDIA NVLink is a high-speed interconnect technology developed by NVIDIA for connecting multiple NVIDIA GPUs to work in parallel. It allows for direct communication between the GPUs with high bandwidth and low latency, enabling them to share data and work collaboratively on compute-intensive tasks.
NVIDIA NVSwitch is a high-performance and scalable switch technology that connects multiple NVLinks to provide all-to-all GPU communication at full NVLink speed within a single node and between nodes. Both are designed to facilitate high-bandwidth and low-latency data transfers, ideal for large-scale AI applications. The NVSwitch technology provides a bandwidth of 900 GB/s between any two GPUs.
The following figure shows the NVIDIA GPU connectivity options for the PowerEdge XE9680 server used in this design: