AI-related Performance Testing of PowerEdge MX760c vs. PowerEdge MX750c
Executive Summary Read the ReportThu, 14 Mar 2024 16:52:36 -0000
|Read Time: 0 minutes
Trust the numbers: Dell PowerEdge MX760c using 4th generation Intel® Xeon® Scalable Processors performs better machine learning.
Grid Dynamics set out to review whether taking a CPU-only (non-GPU) approach is effective in training and inference tasks for small and medium-sized machine learning models (up to ten million parameters), and whether it is a good solution for running larger models (with hundreds of millions of parameters) effectively in inference mode. They also wanted to review improvement in server performance, generation to generation.
To understand how the MX760c model series handles significant artificial intelligence/machine learning workloads as compared to the MX750c model series, Grid Dynamics developed four use cases that simulate the computational needs of retail, industrial, and IT infrastructure:
1. A recommendation system for analyzing user preferences and creating personalized recommendations.
2. A sales forecasting and inventory decision support tool for store managers for keeping the inventory optimized against actual and forecasted demand, and for planning for stock replenishment.
3. Anomaly detection for industrial timeseries for analyzing anomalies in telemetry data and detecting failure probability in industrial hardware.
4. Popular machine learning for evaluating server performance through a series of standardized tests.
(2) Based on testing performed by Grid Dynamics, February 2023.
The tests team concluded that Dell PowerEdge MX760c – leveraging Intel® Xeon® Gold 6430 (2.10 / 32 core) CPUs – performs better in machine learning than MX750c leveraging Intel® Xeon® Platinum 8368 (2.40 GHz /38 core) CPUs. The MX760C used Intel® Xeon® Gold 6430 that supports the Advanced Matrix Extensions accelerator that improves the performance of deep-learning training and inference. The Gold 6430 also supports faster DDR5 memory.
Figure 1. Inference speed (neural ML models – Inferences per Second (higher is better)
Advanced Matrix Extensions (AMX) technology implemented in “Sapphire Rapids” 4th generation of Intel® Xeon® Scalable Processors CPUs allows these servers to perform matrix operations for quantized (Int8 and Bf16) ML models much faster.
(8) Based on testing performed by Grid Dynamics, February 2023.
To summarize, Dell PowerEdge MX760c has proven itself to be a faster and a more capable solution than the previous generation MX750c server for the studied use cases. For more details, see the benchmark and testing results in these documents:
Executive Summary - Performance Comparison of Dell PowerEdge MX760c and MX750c Server Models
Technical Paper - Performance Comparison of Dell PowerEdge MX760c and MX750c Server Models