PowerEdge “xs” Optimizations for Virtualization
Download PDFTue, 17 Jan 2023 05:36:08 -0000
|Read Time: 0 minutes
Summary
With the recent announcement of 3rd Gen Intel® Xeon® Scalable processors Dell has announced 2 new PowerEdge models designed for virtualization. The new R650xs is a 1U design with support for up to 10 hard drives. Customers can choose between the following options:
- (10) 2.5” SAS/SATA
- (10) 2.5” NVMe
- (4) 3.5” SAS/SATA
The new R750xs is a 2U design with support for a maximum of 24 hard drives. Customers can choose between the following options:
- (16) 2.5” SAS/SATA
- (16) 2.5” SAS/SATA + (8) NVMe
- (12) 3.5” SAS/SATA
- (12) 3.5” SAS/SATA + (2) rear mounted 2.5” drives
The R650xs and R750xs systems support CPU’s with TDP’s up to 220W and 32 cores as well as new RDMA based network interface cards designed specifically to improve performance in a Software Defined Storage environment like vSAN. Both models support a maximum of 1TB of memory using 64GB DIMM’s.
Introduction
Virtualization environments place significant demands on server Hardware. The CPU subsystem is the most obvious since key specifications like core count, core frequency and the availability of technologies like “hyperthreading” play a key role in determining the number of virtual machines that can be hosted. Memory capacity and performance is another key area of consideration since the ability of the system to deliver optimal virtualization performance is contingent on its ability to deliver data to the CPU subsystem as quickly as possible. The communications subsystem is equally important not only to deliver the required Input/Output necessary for applications but also to deliver optimal performance for technologies like vSAN or other software defined storage solutions. Storage capacity and performance also plays a role even in environments where “boot from SAN” are utilized.
The new PowerEdge R650xs and R750x have been specifically designed to meet these needs by combining high performance options for each subsystem with optimal capacity and flexibility for virtualized environments.
Design Optimizations – CPU Subsystem
The current VMWare licensing structure is based on the number of processors installed however, it is important to note that the standard processor license is limited to 32 cores. Customers can go beyond this to support higher core counts however, incremental licensing cost is incurred when doing so. In addition, virtualization solutions are typically deployed in large numbers so power and cooling efficiency is a key requirement.
The design of the R650xs and the R750xs addresses these elements in multiple ways. First, the highest core count CPU supported on these models is the Intel® Xeon® Gold 6338 Processor. This processor provides 32 cores (64 threads) and operates at a Thermal Design Point (TDP) of 205 Watts with each core operating at a base frequency of 2.00 GHz and a Turbo frequency of up to 3.20 GHz.
As noted above, power and cooling are key considerations as well. The R650xs and R750xs are designed to support CPU’s with a maximum TDP of 220Watts. By limiting the TDP rating for these systems, Dell Engineers were able to focus on designing these systems to reduce operating cost by reducing fan speeds and reducing overall system power budget.
The R650xs and R750xs support a wide range of processor options with core counts ranging from 8 cores per CPU (Intel® Xeon® Gold 6334 – 3.70 GHz) to 32 cores per CPU (Intel® Xeon® Gold 6338 – 2.00GHz) and with options ranging from “Silver” class CPU’s to “Gold” class CPU’s.
Design Optimizations – Memory Subsystem
The R650xs and R750xs are designed to deliver 1 memory DIMM per CPU memory channel and optimal performance can only be achieved with a fully balanced configuration. A “fully balanced” configuration means that all channels are populated with the same number of DIMM’s. 3rd Generation Intel® Xeon® processors have 8 memory channels so the R650xs and R750xs have been designed to support up to 16 DIMM’s per system. While these processors can support up to 2 DIMM’s per channel, research conducted by Dell indicates that 99% of customers configure their virtualized systems with less than 1TB of memory. The R650xs and R750xs offer options for 16GB DIMM’s (x16 = 256GB), 32GB DIMM’s (x16 = 512GB) and 64GB DIMM’s (x16 = 1TB).
Memory capacity requirements are often determined by the GB/VM ratio. The challenge many customers face with this approach is cost. Higher capacity DIMM’s cost more than lower capacity DIMM’s, however, the $/GB ratio of a 64GB DIMM is becoming similar to the ratio of a 32GB DIMM. This means that customers can achieve the same balance that was achieved for previous server generations with fewer DIMM sockets. As the chart below shows, an “xs” system with only 16 DIMM sockets populated with 64GB DIMM’s (1TB total) gives compelling GB/VM even with 32 core CPU’s.
| Threads/2P (with |
| VM's per |
|
Cores/CPU | Hyperthreading) | Threads/VM | Server | GB/VM |
32 | 128 | 2 | 64 | 16GB |
32 | 128 | 4 | 32 | 32GB |
32 | 128 | 8 | 16 | 64GB |
32 | 128 | 16 | 8 | 128GB |
There are several additional advantages to systems like the R650xs and R750xs that offer 16 DIMM sockets rather than 32. The first is reduced power and cooling requirements. For example, assuming a power requirement for memory of 5W per socket, by cutting the number of DIMM sockets in half, an “xs” power budget can be reduced by up to 80W. This in turn reduces the amount of cooling required which allows the use of more cost effective fans and potentially reduced cost by limiting baffles and other hardware used to direct air flow. This also helps explains why an “xs” system can be configured with a power supply as small as 600W while a “standard” system requires a minimum of 800W power supplies to operate. Note that the size of the power supply required is dependent upon the final configuration but in many cases, an “xs” system will operate with a smaller power supply than a system with 32 DIMM sockets.
Another advantage is cost. While the cost of a DIMM socket might be quite low, DDR4 DIMM’s have 288 pins. Each socket needs to connect to the CPU and a design with 16 DIMM sockets requires 4,608 (288 x 16 = 4,608) fewer connections. Fewer connections translates to less complexity of the motherboard design and a drop of this scale can reduce the number of layers the board requires which has a significant impact on the cost of the system.
Design Optimizations – Communications Subsystem
Networking subsystems are vital for virtualized environments. The new R650xs and R750xs address this need by offering a wealth of networking options. Integrated within each design is an OCP3.0 connector. This connector provides and industry standard mechanism for embedding network controllers such as 10Gb/s NIC’s, 25GB/s NIC’s, 40Gb/s NIC’s and 100Gb/s NIC’s. Further, customers can expand the networking capabilities of these system through the addition of PCIe based network interface cards.
An additional benefit is the availability of new networking options that utilize RDMA (Remote Direct Memory Access) such as the Dell E810-XXV, which is a 25Gb/s dual port controller that offers specialized firmware options specific for VMWare vSAN. By offloading networking processing for vSAN, this board is able to offer significant performance improvements over previous generation technologies. Recent testing by a 3rd party showed up to 1.9x better performance of systems utilizing RDMA based NIC’s for vSAN compared to the previous generation, as seen here. While these tests were run on a different system, much of the performance gains can be attributed to this NIC.
Design Optimizations – Storage Subsystem
The R650xs and R750xs offer a number of different storage options including:
- Discrete drive support for vSAN:
- The R650xs supports up to (10) 2.5” SAS/SATA devices or (10) 2.5” NVMe devices
- Alternatively, the R650xs can also be purchased with support for (4) 3.5” devices
- The R750xs supports up to (16) 2.5” SAS/SATA devices or (16) SAS/SATA plus (8) NVMe devices
- Alternatively, the R750xs can also be purchased with support for (12) 3.5” devices
- The R750xs also supports an additional (2) 2.5” devices option in the rear of the system for the 3.5” configuration only
- Both systems support the BOSS system with redundant storage devices. The R750xs also supports the “Hot Plug” BOSS controller
- The R650xs supports up to (10) 2.5” SAS/SATA devices or (10) 2.5” NVMe devices
- PERC 11 RAID Controllers
- For customers who wish to utilize local storage for maximum performance, both the R650xs and the R750xs support the new H755 PERC controller (RAID 0,1,10,5,6) and the new H755n NVMe based PERC controller (RAID 0,1,10,5,6)
- Recent 3rd party testing of the H755n NVMe RAID controller demonstrated up to 15x better performance than previous generation technologies, as seen here.
Design Optimizations – Other Subsystems
It is important to note that all models support key PowerEdge features, such as:
- iDRAC 9 and OpenManage
- Support for OpenManage Integration for VMWare vCenter (OMIVV)
- PCIe 4.0 slots – up to 3 in the R650xs series and up to 6 in the R750xs series both with SNAP I/O options
- 3200MT/s Memory
- Installation in standard depth racks
Conclusion
The new R650xs and R750xs deliver an optimal virtualization experience with support for the latest industry standard technologies and configuration options ideal for virtualized environments.