Up to 29% Higher Inference Performance: PowerEdge R750xa and NVIDIA H100 PCIe GPU
Download PDFTue, 11 Apr 2023 22:40:39 -0000
|Read Time: 0 minutes
Executive Summary - PowerEdge R750xa
The Dell PowerEdge R750xa, powered by the 3rd Generation Intel® Xeon® Scalable processors, is a dual-socket/2U rack server that delivers outstanding performance for the most demanding emerging and intensive GPU workloads. It supports eight channels/CPU, and up to 32 DDR4 DIMMs @ 3200 MT/s DIMM speed. In addition, the PowerEdge R750xa supports PCIe Gen 4, and up to eight SAS/SATA SSD or NVMe drives.
Up to 29% higher inference performance PowerEdge R750xa and NVIDIA H100 PCIe GPU(1)
One platform that supports all of the PCIe GPUs in the PowerEdge portfolio makes the PowerEdge R750xa the ideal server for workloads including AI-ML/DL Training and Inferencing, High-Performance Computing, and virtualization environments. The PowerEdge R750xa includes all of the benefits of core PowerEdge: serviceability, consistent systems management with IDRAC, and the latest in extreme acceleration.
NVIDIA H100 PCIe GPU
The new NVIDIA® H100 PCIe GPU is optimal for delivering the fastest business outcomes with the latest accelerated servers in the Dell PowerEdge portfolio, starting with the R750xa. The PowerEdge R750xa boosts workloads to new performance heights with GPU and accelerator support for demanding workloads, including enterprise AI. With its enhanced, air-cooled design and support for up to four NVIDIA double-width GPUs, the PowerEdge R750xa server is purpose-built for optimal performance for the entire spectrum of HPC, AI-ML/DL training, and inferencing workloads. Learn more here.
Next-Generation GPU Performance Analysis
The Dell HPC & AI Innovation Lab compared the performance of the new NVIDIA® H100 PCIe 310W GPU to the last Gen A00 PCIe GPU in the Dell PowerEdge R750xa. They ran the popular TensorRT Inference benchmark across various batch sizes to evaluate inferencing performance.
The results are in Figure 1.
Figure 1. TensorRT
According to the industry standard TensorRT Inference Resnet50-v1.5 benchmark, the PowerEdge R750xa with NVIDIA's H100 PCIe 310W GPU processes approximately 29% more images per second than the NVIDIA A100 PCIe 300W GPU on the same server across various batch sizes. This significant improvement in image processing speed translates to higher overall throughput for inferencing workloads, making the PowerEdge R750xa with the H100 GPU an excellent choice for demanding applications.
Test Configuration
| R750xa with 4 NVIDIA H100 | R750xa with 4 NVIDIA A100 |
Server | PowerEdge R750xa | |
CPU | 2x Intel(R) Xeon(R) Gold 6338 CPU | |
Memory | 512G system memory | |
Storage | 1x 3.5T SSD | |
BIOS/iDRAC | 1.9.0/6.0.0.0 | |
Benchmark version | TensorRT Inference Resnet50-v1.5 | |
Operating System | Ubuntu 20.04 LTS | |
GPU | NVIDIA H100-PCIe-80GB (310W) | NVIDIA A100-PCIe-80GB (300W) |
Driver | CUDA 11.8 | CUDA 11.8 |
Conclusion
The PowerEdge R750xa supports up to four NVIDIA H100 PCIe adaptor GPUs and is available with new orders or as a customer upgrade kit for existing deployments.
Legal Disclosure
- Based on October 2022 Dell labs testing subjecting the PowerEdge R750xa 4x NVIDIA H100 PCIe Adaptor GPU configuration and the PowerEdge R750xa 4x NVIDIA A100 PCIe adaptor GPU configuration to TensorRT Inference Resnet50-v1.5 testing. Actual results will vary.