Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
MLPerf™ Inference v2.1 with NVIDIA GPU-Based Benchmarks on Dell PowerEdge Servers
Introduction What is new in MLPerf Inference v2.1? Growth of MLPerf Inference as a benchmark Dell results
Summary
Notable results Highlighting performance scores System performance details PowerEdge server, GPU, and Software specifications Performance improvements Conclusion and Key Takeaways Acknowledgment References

Summary

Home > AI Solutions > Artificial Intelligence > White Papers > MLPerf™ Inference v2.1 with NVIDIA GPU-Based Benchmarks on Dell PowerEdge Servers > Summary

Summary

  • The following table lists the Dell systems used for the inference benchmark:

    Table 5.      PowerEdge XE8545 system details

    Platform

    PowerEdge XE8545 4xA100 TensorRT

    PowerEdge XE8545 4xA100 TensorRT, MaxQ

    MLPerf system ID

    XE8545_A100_SXM_80GBx4_TRT

    XE8545_A100_SXM_80GBx4_TRT_MaxQ

    Operating system

    Ubuntu 20.04.3

    CPU

    AMD EPYC 7763

    Memory

    1 TB

    GPU

    NVIDIA A100-SXM-80GB CTS

    GPU form factor

    SXM

    GPU count

    4

    Software Stack

    TensorRT 8.4.2
    CUDA 11.6
    cuDNN 8.4.1
    Driver 510.73.08
    DALI 0.31.0

    System suite type

    Data center

    Table 6.      PowerEdge R750xa and PowerEdge XR12 system details

    Platform

    PowerEdge R750xa 4xA100 TensorRT

    PowerEdge XR12 1xA2 TensorRT

    MLPerf system ID

    R750xa_A100_PCIE_80GBx4_TRT

    XR12_A2x1_TRT_MaxQ

    Operating system

    CentOS 8.2

    CPU

    Intel Xeon Gold 6338 CPU @ 2.00 GHz

    Intel Xeon Gold 6312U CPU @ 2.40 GHz

    Memory

    512 GB

    256 GB

    GPU

    NVIDIA A100-PCIe-80GB

    NVIDIA A2

    GPU form factor

    PCIE

    GPU count

    4

    1

    Software Stack

    TensorRT 8.4.2
    CUDA 11.6
    cuDNN 8.4.1
    Driver 510.73.08
    DALI 0.31.0

    System suite type

    Data center

    Data center, Edge

    This round of submission marks our sixth submission to MLPerf Inference v2.1. Our submission had four different configurations that include NVIDIA-based accelerators. They were inclusive of data center and edge suites. For the benchmarks, we used Dell PowerEdge XR12, R750xa, and XE8545 servers with NVIDIA A2, A100 with PCIE, and A100 with SXM accelerators. We ran the configurations on the TensorRT backend and some configurations set to MaxQ. MaxQ refers to measuring both the performance and power consumed by the system.

    This round demonstrates higher performance on our data center systems and performance/watt on our MaxQ, data center, and edge systems. Overall, Dell servers delivered outstanding performance across different workloads and deployment scenarios, making them suitable for inference workloads.

    MLPerf system ID configurations

    NVIDIA accelerator-based system IDs in our submissions follow this syntax:

    <Dell server name>_<NVIDIA Accelerator name and config>x<Accelerator count>_<inference backend>_<MaxQ if power submission>

    Examples:

    • XR12_A2x1_TRT_MaxQ
    • R750xa_A100_PCIE_80GBx4_TRT
    • XE8545_A100_SXM_80GBx4_TRT
    • XE8545_A100_SXM_80GBx4_TRT_MaxQ