Understanding Thermal Design and Capabilities for the PowerEdge XR8000 Server
Download PDFThu, 27 Jul 2023 20:40:00 -0000
|Read Time: 0 minutes
Summary
This study is intended to help customers understand the behavior of the XR8000 PowerEdge server in harsh environmental conditions at the edge, and its resulting performance.
The need to improve power efficiency and provide sustainable solutions has been imminent for some time. According to a Bloomberg report, in some countries, data centers will account for an estimated 5-10% of energy consumption by 2030. This will include the demand for edge and cloud computing requirements[1]. Dell Technologies continues to innovate in this aspect and has launched its latest portfolio of XR servers for the edge and telecom this year.
The latest offering from the Dell XR portfolio is a series of rugged servers purpose-built for the edge and telecom, especially targeting workloads for retail, manufacturing, and defense. This document highlights the testing results for power consumption and fan speed across the varying temperature range of -5 to 55°C (23F to 122F) by running iPerf3 on the XR8000 server.
About PowerEdge XR8000 – a Flexible, innovative, sled-based, RAN-optimized server
The short-depth XR8000 server, which comes in a sledded server architecture (with 1U and 2U single-socket form factors), is optimized for total cost of ownership (TCO) and performance in O-RAN applications. It is RAN optimized with integrated networking and 1/0 PTP/SyncE support. Its front-accessible design radically simplifies sled serviceability in the field.
The PowerEdge XR8000 server is built rugged to operate in temperatures from -5°C to 55°C for select configurations. (For additional details, see the PowerEdge XR8000 Specification Sheet.)
Figure 1. Dell PowerEdge XR8000
Thermal chamber testing
For the purpose of conducting this test, we placed a 2U XR8000 inside the thermal chamber in our test lab. While in the thermal chamber, we ran the iPerf3 workload on the system for more than eight hours, stressing the system from 5-20%. We measured power consumption and fan speed using iDRAC at 10-degree intervals of Celsius temperature from 0C to 55C.
The iPerf3 throughput measured for 1GB, 10GB, and 25GB seemed consistent across the entire temperature range, with no impact on performance as temperature increased. The fan speed and power consumption increased with temperature, which is the expected behavior.
Figure 2. Thermal chamber in the Dell performance testing lab
System configuration
Table 1. System configuration
Node hardware configuration | Chassis configuration | SW configuration | |
1 x 6421N (4th Generation Intel® Xeon® Scalable Processors) | 2 x 8610t | BIOS | 1.1.0 |
8 x 64GB PC5 4800MT | 2 x 1400w PSU | CPLD | 1.1.1 |
1 x Dell NVMe 7400 M.2 960GB |
| iDRAC | 6.10.89.00 Build X15 |
1 x DP 25GB BCM 57414 |
| CM | 1.10 |
|
| PCIe SSD | 1.0.0 |
|
| BCM 57414 | 21.80.16.92 |
iPerf3
iPerf3 is an open-source tool for actively measuring the maximum achievable bandwidth on IP networks. It supports the tuning of various parameters related to timing, buffers, and protocols (TCP, UDP, SCTP with IPv4, and SCTP with IPv6). For each test it reports bandwidth, loss, and other parameters. An added advantage of using iPerf3 for testing network performance is that it is very reliable if you have two servers, in geographically different locations, and you want to measure network performance between them. (For additional details about iPerf3, see iPerf - The ultimate speed test tool for TCP, UDP and SCTP.)
Results
Figure 3. Constant networking performance with varying temperature and fan speed
Figure 3 shows that as the temperature and fan speed increases, the iPerf3 throughput stays the same. Fan speed is only 14% for temperatures near 20°C.
Figure 4. Power consumption and fan speed
Figure 4 shows that as temperature increases, Chassis power consumption for the system increases. It is 254W at 20°C.
A deep dive into the results
The consistent performance with increasing temperature and power can be attributed to several design considerations when designing and building these edge/telecom servers:
- RAF: The Reverse Airflow option offered in these servers is carried from Dell’s innovation in Multi-Vector Cooling technology. While most of the innovations for MVC center around optimizing thermal controls and management, the physical cooling hardware and its architecture layout help. XR servers are shallower, which can mean less airflow impedance, resulting in more airflow.
- Fans: XR servers are designed with high-performance fans, which have significantly increased airflow performance over previous fan generations.
- Layout: The T-shape system motherboard layout, along with PSUs that are located at each corner of the chassis, allows improved airflow balancing and system cooling, and consequently, improved system cooling efficiency. This layout also improves PSU cooling due to reduced risk from high pre-heat coming from CPU heat sinks. The streamlined airflow helps with PCIe cooling and enables support for PCIe Gen5 adapters.
- Smaller PSU Form Factor: In the 1U systems, a new, narrower, 60mm form factor PSU is implemented to increase the exhaust path space.
- XR servers usually support CPUs with higher TCase requirements. TCase stands for Case Temperature and is the maximum temperature allowed at the processor Integrated Heat Spreader (IHS)[2].
For more details about the design considerations used for edge servers, see the blog Computing on the Edge–Other Design Considerations for the Edge.
iDRAC
To best supplement the improved cooling hardware, the PowerEdge engineering team carried on the key features from the previous generation of PowerEdge servers to deliver autonomous thermal solutions capable of cooling next-generation PowerEdge servers.
An iDRAC feature in XR8000 detects Dell PCIe cards and automatically delivers the correct airflow to the slot to cool that card. When non-Dell PCIe cards are detected, the customer is given the option to enter the airflow requirement (LFM – Linear Feet per Minute) as specified by the card manufacturer. iDRAC and the fan algorithm ‘learn’ this information and the card is automatically cooled with the proper airflow. This feature saves power by not having to run the fans to cool the worst-case card in the system. Noise is also reduced.
More information about thermal management, see “Thermal Manage” Features and Benefits.
Figure 5. iDRAC settings to view fan status during our XR8000 testing in the thermal chamber
Conclusion
Dell Technologies is continuing its efforts to test other XR devices and to determine power consumption for various workloads and its variation with changes in temperature. This study is intended to help customers understand the behavior of XR servers in harsh environmental conditions at the edge and their resulting performance.
References
- PowerEdge XR8000 Specification Sheet
- iPerf - The ultimate speed test tool for TCP, UDP and SCTP
- Computing on the Edge–Other Design Considerations for the Edge
- “Thermal Manage” Features and Benefits
[1] https://stlpartners.com/articles/sustainability/edge-computing-sustainability
[2] https://www.intel.com/content/www/us/en/support/articles/000038309/processors/intel-xeon-processors.html