PowerEdge R760 ResNet50 Testing Overview and Results
Download PDFFri, 03 Mar 2023 17:20:51 -0000
|Read Time: 0 minutes
Summary
The testing outlined in this paper was conducted in conjunction with Intel and Solidigm. Server hardware was provided by Dell, processors and network devices were provided by Intel, and storage technology was provided by Solidigm. All tests were conducted in Dell Labs with contributions from Intel Performance Engineers and Dell System Performance Analysis Engineers.
With the introduction of the 4th Gen Intel® Xeon® Scalable processors, the new Dell PowerEdge R760 can benefit from important new features such as Advanced Matrix Extensions (AMX) to improve deep learning performance. To evaluate this, we recently tested the R760 using the TensorFlow framework with the ResNet50 (residual network) CNN model to determine the performance of these new features compared to previous generations of servers. This testing demonstrated more than 3x improvement in performance in the BF16 compared to FP32 precision and more than 2x improvement in performance compared to the previous generation R750 in INT8 precision.
Configurations tested
- BASELINE: Intel® Xeon Platinum 8380 (ICX Config): 4 Nodes, Each Node with 2x Intel® Xeon® Platinum 8380 Processor, 1x PowerEdge R750, Total Memory 1536 GB (16x 32GB + 16X64GB , DDR4 3200MHz), HyperThreading: Enabled, Turbo: Enabled, NUMA noSNC,, BIOS:Dell1.6.5 (ucode:0xd000375),Storage (boot): 1x 480 GB Micron SSD, Storage (cache): 2x 800 GB Intel® Optane™ DC SSD P5800X Series, Storage (capacity): 6x 3.2 TB SolidigmDC P5600 Series PCIe NVMe, Network devices: 1x Intel® Ethernet E810CQDA2 E810-CQDA2,at 100GbERoCEv2,Network speed: 100GbE, OS/Software: VMware 8.0, 20513097, Test by Dell & Intel as of 21/12/2022using Ubuntu Server 22.04 VM (vHW=20, vmxnet3), vSAN default policy (RAID-1, 2DG), Kernel 5.19.17, intel-optimized-tensorflow:2.11.0, ResNet50v1.5, Batch size=128, VM=80vCPU+96GBRAM
- SPRPlus: Intel® Xeon® Platinum 44 core Pre-Production Processors. 4 Nodes, Each Node with 2x Intel® Xeon® Platinum Pre-Production Processors, 1x PowerEdge R760, Total Memory 2048 GB (16x 128GB DDR5 4800MHz), HyperThreading: Enable, Turbo: Enabled, NUMA noSNC, BIOS: Dell 0.2.3.1(ucode:0x2b000081), Storage (boot):1x600GB Seagate Enterprise drive, Storage (cache): 2x 800 GB Intel® Optane™ DC SSD P5800X Series, Storage (capacity): 6x 3.2 TB Solidigm SSD DC P5600 Series PCIe NVMe, Network devices: 1x Intel® Ethernet E810CQDA2 E810-CQDA2,at 100GbERoCEv2,Network speed: 100GbE, OS/Software: VMware 8.0, 20513097, Test by Dell & Intel as of 11/21/2022using Ubuntu Server 22.04 (vHW=20, vmxnet3), vSANdefault policy (RAID-1, 2DG), Kernel 5.19.17, intel-optimized-tensorflow:2.11.0, ResNet50v1.5, Batch size=128, VM=88vCPU+96GBRAM
Security mitigations
The following security mitigations were evaluated and passed:
CVE-2017-5753, CVE-2017-5715, CVE-2017-5754, CVE-2018-3640, CVE-2018-3639, CVE-2018-3615, CVE-2018-3620, CVE-2018-3646, CVE-2018-12126, CVE-2018-12130, CVE-2018-12127, CVE-2018-11091, CVE-2018-11135, CVE-2018-12207, CVE-2020-0543, CVE-2022-0001, CVE-2022-0002
Systems architecture
Deep learning environments both process and generate large amounts of data. To facilitate this in our testing, we used a VMware vSAN 8 cluster to store all data.
Hypervisor, VM, and guest OS configuration
Benchmark configuration
Dell PowerEdge R750 Dell PowerEdge R760
Test results
ICX – 3rd Gen Intel® Xeon® processors used in the R750
SPR – 4th Gen Intel® Xeon® processors used in the R760
Conclusion
The new Dell PowerEdge R760 with 4th Gen Intel® Xeon® processors delivers outstanding machine learning (ML) performance. Using the Intel® AMX features and AVX-512 instruction set delivers performance levels up to 2.37x better than previous generations. As customers look to expand their deployments of ML workloads, the combination of 4th Gen Intel® Xeon® processors and the innovative Dell PowerEdge R760 provide a cost-effective solution that does not require the addition of expensive GPU technologies.
Related Documents
Microsoft SQL Server Solution Overview
Fri, 03 Mar 2023 17:23:10 -0000
|Read Time: 0 minutes
Summary
Microsoft SQL Server solution is a high-performance data platform that is optimized for Online Transaction Processing (OLTP) and Decision Support System or Analytics workloads. This solution helps to provide customers with system architectures that are optimized for a range of business operation and analysis needs. It also enables customers to achieve an efficient resource balance between the SQL Server data processing capability and the hardware throughput.
Market positioning
SQL Server enables organizations to gain intelligence from all types of data. By using SQL Server with Windows on the latest generation Dell PowerEdge servers with the latest Intel® Xeon® Scalable processors, organizations get faster insights from transaction processing and analytical processing.
Expanded product features
Intel Xeon Scalable Processors
The 4th Generation Intel® Xeon® Scalable processor family has the most built-in accelerators of any CPU on the market to speed up AI, databases, analytics, networking, storage, and HPC workloads.
Along with software optimizations, the following features help improve workload performance and power efficiency:
- Intel® Advanced Matrix Extensions (Intel® AMX)
- Intel® QuickAssist Technology (Intel® QAT)
- Intel® Data Streaming Accelerator (Intel® DSA)
- Intel® Dynamic Load Balancer (Intel® DLB)
- Intel® In-Memory Analytics Accelerator (Intel® IAA)
With Microsoft SQL Server 2022 and Intel® QuickAssist Technology, customers can efficiently speed-up compressed database backups without significanly increasing CPU utilization, leaving more resources for handling user queries and other database operations.
Memory
The latest Dell PowerEdge servers with Intel 4th Gen Xeon® Scalable processors supports eight channels of DDR5 memory modules per socket running at up to 4800MT/s with 1 DIMM per channel or up to 4400MT/s with 2 DIMMs per channel, offering up to 1.5x bandwidth improvement over previous generation platofrms with DDR4 memory, increased memory capacity, and power efficiency.
Storage/RAID
Intel® Optane™ SSDs deliver performance, Quality of Service (QoS), and capacity improvements to optimize storage efficiency, enabling data centers to do more per server, minimize service disruptions, and efficiently manage at scale. Intel® Optane™ SSD P5800X with next generation Intel® Optane™ storage media and advanced controller does not comprise I/O performance read or write (R/W) and high endurance, and provides unprecedented value over legacy storage. In the accelerating world of intelligent data, Intel® Optane™ SSD P5800X offers three times greater random 4k mixed R/W I/O operations per second (IOPS) over Intel® Optane™ SSD P4800X1 (PCIe* 3.x).
Key Considerations
- Higher performance with lower licensing costs - All configurations are based on the latest Dell PowerEdge servers with high-frequency 4th generation Intel® Xeon® Scalable Processors to achieve best Microsoft SQL Server performance and optimize software licensing costs.
- Data redundancy and high availability - The Dell PERC H755N NVMe RAID controller provides high performance local data redundancy, resilience, and reliability for critical workloads and applications such as Microsoft SQL Server. For high availability, performance, and capacity scaling, use multiple servers with SQL Server replication, log shipping, mirroring, clustering, or AlwaysOn Availability Groups (AG).
- Data throughput - Microsoft SQL Server Solution on Dell PowerEdge servers is built on high-performing Intel® architecture. The solution uses the high-performance Intel® Xeon® processors and better network, storage, and integrated platform acceleration products optimized for high workload density and performance.
- Optimized backup operations with QAT accelerator - The Intel® Xeon® Platinum 8462Y+ processor has a QAT accelerator that supports a high performance compression engine that can significantly shorten backup/restore operations on highly utilized servers running Microsoft SQL Server 2022.
Recommended Configurations
Base for SQL Server Standard Edition
Table 1. PowerEdge R660-based, up to 8 or 10 NVMe drives and optional HW RAID, 1RU
Feature | Description |
Platform[1] | Dell R660 chassis with NVMe backplane (10x 2.5” – direct connection without RAID, or 8x 2.5” with HW RAID support) |
CPU | 2x Xeon® Gold 6426Y with SST-PP (12c @ 2.5GHz base / 3.3GHz turbo), or 2x Xeon® Gold 5418Y with SST-PP (12c @ 2.7GHz base / 3.2GHz turbo) |
DRAM | 256GB (16x 16GB DDR5-4800) |
Boot device | Dell BOSS-S2 with 2x 480GB M.2 SATA SSD (RAID1) |
Storage adapter[1] | Optional Dell Front PERC H755N NVMe RAID |
Log drives | 2x 1.6TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 (RAID1) |
Data drives[2] | 4x (up to 6x/8x) 3.84TB (or larger) Enterprise NVMe Read Intensive AG Drive U.2 Gen4 |
NIC | Intel® E810-XXV for OCP3 (dual-port 25Gb) |
Base for SQL Server Enterprise Edition
Table 2. PowerEdge R660-based, up to 8 NVMe drives and HW RAID, 1RU
Feature | Description |
Platform | Dell R660 chassis with NVMe backplane (8x 2.5” with HW RAID support) |
CPU | 2x Xeon® Gold 6442Y (24c @ 2.6GHz base / 3.3GHz turbo) |
DRAM | 512GB (16x 32GB DDR5-4800) |
Boot device | Dell BOSS-S2 with 2x 480GB M.2 SATA SSD (RAID1) |
Storage adapter | Dell Front PERC H755N NVMe RAID |
Log drives | 2x 400GB or 800GB Intel Optane P5800X U.2 Gen4 (RAID1) |
Data drives | 6x 3.84TB (or larger) Enterprise NVMe Read Intensive AG Drive U.2 Gen4 |
NIC | Intel® E810-XXV for OCP3 (dual-port 25Gb) |
Plus for SQL Server Enterprise Edition
Table 3. PowerEdge R760-based, up to 16 or 24 NVMe drives and dual HW RAID, 2RU
Feature | Description |
Platform | Dell R750 chassis with NVMe backplane (16x 2.5” / 24x 2.5” with dual HW RAID support) |
CPU[3] | 2x Xeon® Platinum 8462Y+ (32c @ 2.8GHz base / 3.6GHz turbo) |
DRAM | 512GB (16x 32GB DDR5-4800) or more |
Boot device | Dell BOSS-S2 with 2x 480GB M.2 SATA SSD (RAID1) |
Storage adapter | Dual Dell Front PERC H755N NVMe RAID |
Log drives | 2x 400GB or 800GB Intel Optane P5800X U.2 Gen4 (RAID1) |
Data drives[4] | 6x (up to 14x/22x) 3.84TB (or larger) Enterprise NVMe Read Intensive AG Drive U.2 Gen4 |
NIC[5] | Intel® E810-XXV for OCP3 (dual-port 25Gb), or |
[1] optional Dell PERC H755N NVMe RAID controller supported only with 8-drive chassis
[2] max number of drives depends on the chassis version and HW RAID support
[3] The Xeon 8462Y+ SKU includes QAT engine for crypto and compression acceleration
[4] max number of drives depends on the chassis version and HW RAID support
[5] 100Gb NIC recommended for high throughput Data Warehouse loads and ETL processing
Learn more
Contact your Dell or Intel account team for a customized quote, at 1-877-289-3355.
Are Rugged Compact Platforms Ready for Edge AI?
Thu, 14 Mar 2024 16:47:06 -0000
|Read Time: 0 minutes
Scalers AI™ tested the Impellers Defect Inspection at the Edge on the Dell PowerEdge XR5610 server. Impellers are rotating components used in various industrial processes, including fluid handling in pumps and fans. Quality inspection of impellers is crucial to ensure their reliable performance and durability.
Dell™ PowerEdge™XR5610 Server supports 50 simultaneous streams running AI defect detection in a single CPU config with Dell™PowerEdge™ XR Portfolio offering scalability to 4CPUs at a near-linear scale. 1.4x Gen on Gen Performance Improvement Using Intel® Deep Learning Boost and Intel® OpenVINO™ Smart Factory Solution | Defect Detection Solution.
Dell™ PowerEdge™ XR 5610 servers, equipped with fourth Gen Intel® Xeon® scalable processors, are well suited to handle Edge AI applications with both AI inference and training at the Edge. The rugged form factor, extended temp, and scalability to four sockets enable compute to be deployed in the physical world closer to the point of data creation, allowing for near-real-time insights.
Fast-track development with access to the solution code:
Contact your Dell™ representative or Scalers AI™ at contact@scalers.ai for access. Save hundreds of hours of development with the solution code. As part of this effort, ScalersAI™ is making the solution code available.