Foundation for Enterprise Security and Cyber Resiliency: Dell PowerEdge and Broadcom
Read the ReportWed, 13 Mar 2024 20:49:09 -0000
|Read Time: 0 minutes
Executive Summary
There are numerous challenges companies face to establish and maintain the cybersecurity capabilities required to operate in today’s complex and increasingly hostile digital world. Companies are reminded daily of the potential cost to their reputation, revenue and more for failing to maintain a competent cyber resiliency profile. Often, even a single breach can result in regulatory fines, loss of revenue and customers, and in some cases may even cause existential implications for a firm.
A Futurum Group study conducted in 2023, with more than 150 IT Security professionals from companies with 1,000 or more employees resulted in several insights. More than 80% of companies surveyed reported an increase in their cybersecurity budgets while nearly 50% are increasing their cybersecurity headcount in response to these growing threats. In terms of challenges faced, the top two items were limited budgets and high solution costs, indicating a need to improve cybersecurity without significant cost implications.
As part of The Futurum Group’s ongoing research and analysis, The Futurum Group Labs has developed a cyber-resiliency framework that is designed as a holistic tool for evaluating how well a product or service offering helps a company meet their cybersecurity requirements. This framework incorporates multiple industry standards and best practices to provide a comprehensive perspective for evaluating their security requirements. The Futurum Group utilizes this framework when evaluating security aspects of products and services.
In this paper we review the important security features of Dell and Broadcom components, and how they can help companies improve their security posture and overall cyber resiliency. The Futurum Group Labs tested Dell 16th Generation PowerEdge servers with Broadcom 57508 Ethernet Network Interface Card (NIC) and PERC RAID controller cards both within our lab facilities, and remotely within Dell’s lab. This analysis provides a real-world use case of managing security across a multi-site, distributed environment.
Summary of Findings
The Futurum Group Labs performed hands on testing and review of many of the security features and capabilities of Dell PowerEdge 16th Generation servers, together with Broadcom 57508 dual port 100 Gb Ethernet NIC and Broadcom SAS4116W ROC based Dell PERC 12, H965i RAID cards. Our evaluation included analyzing security best practices, along with overall system security features and capabilities.
Additionally, The Futurum Group has developed a security framework comprising areas for review across multiple areas of enterprise security. The Futurum Group’s Security Framework builds upon industry best practices and multiple NIST standards and provided the basis for our evaluation process.
Against this framework, we found the Dell PowerEdge servers, together with Dell PowerEdge RAID Controller 12 (PERC) and Broadcom NICs provided an integrated security solution that met or exceeded requirements in all areas evaluated. Strengths included the following areas:
- Adherence to standards, including the NIST concepts of Identify, Protect, Detect, Respond & Recover
- Zero Trust security principles utilized by both Dell and Broadcom for all aspects of the systems
- Integrated security management via iDRAC, OME and CloudIQ provide enterprise-wide capabilities
- Dell’s Secure Component Verification Process helps ensure security from order to delivery
Futurum Group’s Security Framework
The Futurum Group has developed a security framework for evaluating the overall cyber resiliency rating of products operating within IT environments. This framework encompasses multiple inter-related aspects of security including ensuring that secure design methodologies are utilized during product creation.
The first area assessed is the design methodology utilized for developing products and services. Next the security features of a product are evaluated, along with the security of the manufacturing and supply chain processes. Finally, the IT security implementation of a company using the product is important. Taken together, these components provide a comprehensive tool for evaluating and analyzing the security profile of a product operating within a particular company. An overview is shown in Figure 1.
Figure 1: Security Framework Overview (Source: The Futurum Group)
It should be noted that “IT Security Implementation” was outside of the scope of work for this project. Since security responsibilities fall upon each company to utilize and implement security features appropriately.
Dell Solution Overview
As a foundation for securing PowerEdge server products, intrinsic security practices are incorporated into hardware product design and software or firmware code development. These practices include the following processes to ensure security features are implemented at the time of product inception and continue throughout the development cycle. To perform this practice effectively, Dell engineers are required to take mandatory security training before handling the code. Security “champions” are assigned to each development team to drive a security culture within the organization. Dell combines all components to create an integrated solution with hardware, software and services that together address security threats companies are facing.
Dell and Broadcom asked The Futurum Group Labs to evaluate Dell’s servers together with Broadcom add-in cards with respect to how these products can help IT clients address security vulnerabilities and achieve the levels of cyber resiliency necessary for operations in today’s challenging environment.
Specifically, The Futurum Group Labs validated the latest, 16th Generation Dell PowerEdge servers, together with Dell - Broadcom PERC 12 RAID cards and Broadcom 57508 NIC cards, along with relevant firmware and software that address security and cyber resiliency within IT environments. The Broadcom components helped provide overall end-to-end security capabilities through integration into Dell’s management tools.
Figure 2: Dell PowerEdge and Broadcom NIC Products (Source: Dell and Broadcom)
Dell’s Security Principles for Cyber Resiliency
As companies have become increasingly sophisticated in their approach to security, it is clear that point solutions are insufficient to address today’s complex cybersecurity requirements. While there is naturally a focus on the hardware components themselves, an important part of the overall Dell and partner security ecosystem is the use of critical software elements that help enable security features and provide easy-to-use interfaces for IT users.
Futurum Group Comment: Too often, security features require IT administrators to choose between usability or security. As a result, IT administrators may ignore or circumvent security best practices to improve ease of use. The management tools of the PowerEdge systems enable IT users to easily follow best practices while utilizing the comprehensive security features of Dell servers and Broadcom. This functionality helps reduce attack surfaces for malicious actors, while maintaining usability and performance. |
As described previously, The Futurum Group’s Security Framework incorporates multiple best practices and industry standards, including NIST Cybersecurity Framework (CSF) which outlines five tenants:
- Identify: Ensure that an organization understands and can manage cybersecurity risk, assets, and data
- Protect: Develop and implement appropriate safeguards to ensure critical infrastructure is protected
- Detect: Quickly detect cyber events via anomaly detection and continuous monitoring
- Respond: Prepare and implement the ability to take appropriate action in response to a cyber event
- Recover: Establish the ability to maintain business resiliency and restore impacted systems
The Futurum Group Security Framework extends the NIST framework, including the following areas for assessing Dell’s Cyber Resilient Architecture for security:
- User authentication and authorization using multi-factor authentication applied to users and devices (User AA and Device AA) with role-based and least privileged access controls.
- Dell PowerEdge hardware RoT creates a chain of trust from firmware to devices and supports complete customization of Secure Boot, including removal of all industry-standard certificates provided by Microsoft, VMware, or the UEFI CA.
- Using PowerEdge cyber resilient platform RoT extends to Secure Enterprise Key Management for data-at-rest protection.
- Applications and workload security to protect runtime environments, leveraging data and network security technologies including end-to-end encryption and cryptographic signatures.
- Utilize Security Protocol and Data Model (SPDM) capabilities for device attestation as part of the Zero-Trust/cyber resiliency architecture. Future capabilities include internal encrypted communication capabilities.
- Auditing and analytics for visibility and alerting of security issues using persistent event logging and real-time code and firmware scanning.
- Automation and orchestration technologies to enable automated API and CLI driven actions, along with enterprise web UI tools to provide easy-to-use global management of security alerting, reporting and automation.
Test and Validation Findings
The Futurum Group Labs Security Framework shown previously in Figure 1, was used to analyze the overall security capabilities of products or features. Our analysis examined each aspect of the Security Framework with details provided in the following four sections.
1 - Secure Design Methodology
Requirements
Confirm that security considerations begin at product inception with cybersecurity and resiliency capabilities integrated throughout product design and development.
Verify that hardware security features based on Zero Trust, such as tamper proof TPM often form the basis of software security, to deliver an integrated secure system.
Asses if the software architectures leverage hardware security and utilize the available security capabilities. Additionally, validate that threat modeling and penetration testing along with external audits and ongoing mitigation of vulnerabilities are performed.
Dell PowerEdge and Broadcom Design
Dell utilizes a secure Software Development Lifecycle (SDL) as the basis for instilling security throughout the design process. A comprehensive SDL process should ensure that any identifiable attack methods are mitigated utilizing standards-based security mechanisms to thwart these attack vectors.
As outlined previously, security starts with using a secure design methodology, with multiple considerations required to help products provide cyber resiliency:
- Security is considered for each feature - Following industry best practices, Dell software is designed to obstruct, oppose and counter malicious attacks - External audits of security features - On-going evaluation of new vulnerabilities - Rapid response to common vulnerability exposures (CVEs) with remediation |
|
Table 1: Dell SDL Principles | Figure 2: Dell SDL Model |
Additional aspects of secure design include end-to-end secure communication based upon cryptographically signing certificates used to authenticate and authorize components such as with Device-ID, 802.1x, and SPDM.
Finally, even with good design and secure implementations, vulnerabilities and breaches may still occur. Thus, it is important for products to be designed with resiliency and Zero Trust implementation, so that when penetrations do occur, their impact is limited. Dell’s rapid response to critical CVEs and patching of identified components is an important aspect of mitigating threats from CVE’s as they arise.
Futurum Group Comment: Dell’s rapid response to critical CVEs and patching of identified components is an important aspect of mitigating threats from CVE’s as they arise. As one of the largest system vendors, Dell leverages their significant real-time monitoring via CloudIQ and other tools, to quickly respond to emerging threats and provide their customers with tools and processes to thwart attacks. |
2 – Product Security Features
Product security was an area of particular focus by The Futurum Group Labs when measuring how Dell PowerEdge servers, together with Broadcom based PERC 12 and 57508 Ethernet NICs met Futurum’s Security Framework. System security requires a layered approach, with hardware security features and capabilities utilized by the firmware, that provides a secure foundation for the operating system (OS) and applications running within the OS. The hardware root of trust provides several foundational security capabilities including a secure, tamper-proof key storage location, a unique hardware encryption key, and other facilities required as part of a cryptographic framework.
Specifically, The Futurum Group Labs tested the following system components:
- Dell PowerEdge 16th Generation systems with a Trusted Platform Module (TPM)
- Dell PERC 12 (PowerEdge RAID Controller, 12th generation)
- Broadcom 57508 Dual Port, 100 Gb Ethernet NIC with QSFP adapter connectivity
- Secure BIOS management system settings, via iDRAC
- Device identification via cryptographic framework (devices attestation uses a devices hardware RoT)
Each aspect of a system must be secured, ideally with each element utilizing industry best practices including Zero Trust principles, enabled via a silicon Root of Trust (RoT) as the foundation of physical security. Utilizing a certified RoT device that provides tamper resistance, the hardware protects the device firmware, which in turn is utilized to ensure critical software security features are verifiably operating securely. Security breach examples could include the exchange of entire PCIe cards, uploading compromised firmware or other attacks that target PCIe cards.
Delivering end-to-end security requires the entire solution integrates multiple hardware RoT devices, together with software features that leverage the underlying hardware capabilities. Dell’s PowerEdge servers together with Broadcom NIC and RAID controllers provide this capability by validating each layer, from hardware to firmware and the OS using cryptographic attestation. Dell’s iDRAC and other management tools leverage this integrated security environment to deliver a secure, easy to manage solution.
Futurum Group Comment: A key aspect of the Zero Trust approach is to utilize hardware RoT devices to provide cryptographic verification to build a secure ecosystem. Using Zero Trust, each component first verifies, then trusts their counterpart based upon certified key exchanges and mutual authentication, attestation and authorization. These features are critical for the additional security mechanisms used throughout the Dell servers along with supported devices. |
As part of a standards-based approach to secure system management, a new Security Protocol and Data Model (SPDM) standard has emerged. This standard leverages Zero Trust principles and enables devices to securely verify each other (device attestation) and then establish secure communications over a variety of internal, server transport connections. Dell’s 16th Generation servers, along with 12th generation PERC and Broadcom 57508 NICs now utilize SPDM, to enhance security. The SPDM standard leverages existing technologies including public key encryption, and cryptographic signing of certificates to provide device attestation between Dell’s servers and Broadcom devices. In future implementations, the SPDM standard can also enable secure communication channels between devices.
Broadcom Networking
Broadcom 57508 Ethernet NIC, with speeds from 25 Gb to 200 Gb operate with Dell’s security features and leverage some of the same Zero Trust principles. Broadcom’s NICs also have a hardware based, silicon RoT which enable multiple additional benefits, including:
- Silicon RoT with secure key storage on servers and PCIe devices
- Secure firmware loading, updates and recovery on servers and PCIe devices
- Signed UEFI drivers, with Secure PXE boot for onboard NICs and PCIe NICs
- Device attestation (SPDM 1.2) between servers and devices, and audit logging of all devices
Figure 3: Broadcom’s Silicon RoT w/ Secure Firmware Loading (Source: Broadcom)
Hardware TPM
The Futurum Group Labs verified that Dell PowerEdge’s hardware security operated correctly by testing the supplied TPM version 2.0. Testing verified that all supplied features operated correctly including key generation using a true random number generator, and a secure key store. Additionally, after verifying TPM functionality, we then enabled security features including secure boot, secure lifecycle management, out of band management, and firmware updating.
Dell PowerEdge RAID Controller (PERC)
Dell’s PowerEdge RAID Controllers, aka PERC 12th generation use Broadcom controller interfaces and support a variety of RAID levels and device connectivity. The PERC 12 H965i controllers tested are designed for internal and external device support and include multiple RAID levels along with SAS and NVMe connectivity.
The Dell 12th gen PERC security features include:
- Silicon based hardware RoT: Foundation for Zero Trust
- Onboard RoT builds a chain of trust by authenticating all PERC firmware prior to execution, permits only authenticated firmware upgrades
- SPDM support: Provides device attestation and secure communications
- UEFI secure boot: Helps reduce attack vectors, and root kit installations
- Secure Firmware Updates: Simplifies administrative security updates
As shown above previously in Figure 3, the process for secure booting from a Dell PERC 12 controller card is similar to securely booting via a NIC.
Dell PowerEdge 16th Generation Servers
Multiple areas of evaluation and testing were performed with the PowerEdge servers together with PERC RAID cards and Broadcom NICs. One of NIST’s specifications (NIST SP 800-193) specifically addresses platform firmware resiliency. It stipulates methods for securing the BIOS, boot ROMS, along with firmware and driver signatures to verify authenticity. This guideline also outlines a method for providing a “secure boot” mechanism, whereby each component verifies subsequent components in the stack from hardware all the way to the operating system.
Secure Boot
Dell PowerEdge secure boot via Broadcom NIC, Dell PERC or local disk occurs via similar mechanisms. The secure boot process is important to ensure that a validated OS image is ultimately loaded and is used to operate the system. If an OS is altered, system security cannot be guaranteed. At a high-level, secure boot validates the UEFI drivers and boot loaders which then guarantees the authenticity and integrity of each subsequent component, firmware or software element that is loaded. This chain of trust is one of the basic elements of the Zero Trust model and basic system security. Secure boot functionality was verified with multiple devices, including secure boot of a verified OS image from Dell’s PERC 12 controller, and via PXE boot using a Broadcom 100 GbE NIC card.
Authentication Access, Authorization and Auditing (IAM)
These areas include identity and access management of systems (IAM), components and users, multi-factor authentication, audit logging and alerting, and access based upon authenticated authorization. The IAM capabilities are implemented throughout Dell’s management tools, including iDRAC, OME and CloudIQ.
Secure Lifecycle Management
Secure lifecycle management features were verified by first establishing a baseline, and then updating to the latest firmware and drivers via iDRAC. The secure process was verified both via the successful updates and notices, along with one image that did not update due to an invalid security certificate. The test properly identified an invalid certificate, and would not allow the firmware to be updated, which was the correct behavior. Upon obtaining a firmware image with the proper security certificate, the firmware update was successfully applied, which again was the desired security behavior.
Management, Alerting and Reporting
A critical part of Dell’s cybersecurity capabilities are the management tools available. Dell’s iDRAC tool continues to evolve and add capabilities for BIOS and low-level system management of PowerEdge servers and attached devices. Incorporating standards such as Redfish for secure remote manageability, enables secure programmatic API, CLI and scripting access, while also enabling secure web UI access for users preferring a graphical interface.
The Futurum Group Labs utilized and validated numerous functions and capabilities of each of Dell’s enterprise management solution. iDRAC, Open Manage Enterprise and CloudIQ are designed to work together to enable a spectrum of management, alerting and reporting capabilities.
- Dell’s iDRAC: system level management software pre-installed on all Dell PowerEdge servers, providing secure out-of-the box management capabilities for individual systems.
- OpenManage Enterprise (OME): Designed for IT staff to monitor and manage Dell servers within a datacenter. OME provides roll-up features to aggregate information while still providing the ability to manage individual systems, either from within OME or via linking to iDRAC.
- CloudIQ: Enables multi-site and enterprise-wide monitoring and management, with high-level management and reporting while also providing drill-down system management of some features and function.
A key part of iDRAC with OME is the integration of Broadcom components throughout, including the ability to monitor Broadcom-based PERC and NICs for firmware levels, vulnerabilities and provide updates through the secure lifecycle management capabilities.
The Dell OME management server was utilized throughout testing. Installation was straightforward and accomplished without needing to consult manuals. Downloading a preconfigured VMware image (OVA) and deploying to a VMware cluster, the OME server was operational in under 30 minutes. OME provided a way to both perform health monitoring and alerting, along with active management of desired systems. The OME interface was intuitive and provided significant fleet management features, by helping to highlight potential issues and providing the ability to resolve problems.
Futurum Group Comment: Looking specifically at the security features of iDRAC, OME and CloudIQ, we found the health and security scoring mechanism and recommendations to be extremely effective. A significant challenge for most IT environments is prioritizing which updates are the most critical and then quickly remediating these issues. Dell’s management tools enable finding and applying critical updates quickly, and is one of the most effective solutions for security we have evaluated to date. |
With CloudIQ’s health scores and recommendations, it required only a few clicks to sort by the most critical issues, and then utilize the appropriate management tools to apply remediation for the vulnerabilities. Some issues may be resolved entirely from CloudIQ, with others enabled via links to OME for remediation.
Dell’s CloudIQ service has continued to evolve since its storage system origins into a Dell product-wide SaaS application that effectively provides health monitoring and alerting, along with recommendations. Dell’s overall management architecture provides robust security manageability, alerting and reporting capabilities that provide the critical link IT administrators require to effectively manage a secure IT environment.
3 - Manufacturing & Supply Chain Security
The concept of evaluating a product’s manufacturing and supply chain for security is relatively new to many companies. Although these practices have been standard for regulated industries for years, only recently have companies began to apply supply chain security practices to products designed for commercial use. One reason is that due to increased focus on other threats, the supply chain is now seen as an area of increasing vulnerability.
One area of supply chain security that sets Dell apart from their competitors is Dell’s Secured Component Verification (SCV), which is designed to help verify that the security of all components is maintained from order until it is delivered and installed at the customers location. Dell’s approach to supply chain threats includes supply chain integrity with ISO certification of sites, software bill of materials and the Dell SCV process, which all help to prevent counterfeit components or malware being inserted into systems or components.
Futurum Group Comment: Dell’s approach to securing product manufacturing and supply chain is perhaps the best processes the Futurum Group Labs has evaluated. In particular, Dell’s Secured Component Verification process provides unique value, enabling IT clients to quickly and easily ensure their products have not been altered or modified, including verification of both hardware and firmware components. Dell SCV is an important tool for addressing overall cybersecurity. |
An overview of the Dell SCV process is shown below in Figure 4.
Figure 4: Dell’s Secured Component Verification Process (Source: Dell)
The Futurum Group Labs verified that Dell’s SCV can accurately report on each major system element, showing whether it is a verifiable component that was part of the original bill of materials or not. The SCV leverages the earlier security principles and technologies outlined, including Secure RoT and hardware TPM modules, cryptographic signatures for device attestation and Zero Trust principles to create a process to verify that a system has not been altered in any way.
During testing The Futurum Group Labs verified that the SCV correctly verified components that were part of the original bill of material and order, and items that were added afterwards. This capability provides a unique and easy-to-use method for IT users to attest to the authenticity and integrity of their systems.
4 - IT Security Implementation
The last major component of the Futurum Group’s Security Framework is how well a company and the IT organization operates with respect to security. IT security has multiple standards, industry best practices and recommendations to help inform and guide companies towards implementations that provide the necessary security and cyber resiliency.
One guideline covering IT security is the NIST CSF [1] which includes the key phrases, Identify, Protect, Detect, Respond and Recover. The most recent update to this framework, the version 2.0 draft, adds a sixth area “Govern” to the previous list, placing more emphasis on corporate oversight and implementations.
Although important, this area falls largely outside the scope of vendors’ product offerings but may be supplemented via professional services and training. As a result, The Futurum Group did not formally assess the IT Security aspects of the Futurum Group’s Security Framework.
Futurum Group Comment: The Futurum Group’s Security Framework evaluates IT security through the lens of a products ease-of-use and ability to implement security best practices. In this regard, Dell enables IT security by making security best practices the default option when possible, and ensuring security features are easy-to-use. As a result of security integration between PowerEdge servers, Broadcom 57508 NICs and Dell management tools helps companies implement security best practices. |
For companies that desire security consulting to help supplement their capabilities, Dell and their partners offer a variety of security services, including cybersecurity advisory services that are designed to assess a company’s security posture, and help improve internal procedures and help elevate security capabilities to attain proficiency in this area. Based upon the principles outlined previously, Dell security services include Cybersecurity Advisory Services, Recovery Services for Cyber-attacks, along with managed services for cyber-attack detection and response.
Moreover, the product design, product features, together with the secure supply chain features and security consulting services all provide sufficient tools to enable IT organizations to manage and maintain a highly secure environment that meets NIST and industry best practices for maintaining a cyber resilient posture.
Final Thoughts
Security and cyber resiliency have become areas of critical focus for enterprises according to The Futurum Group’s research. Additionally, conversations with IT clients often include a discussion of ransomware and cyber resiliency. Organizations that follow best practices follow the NIST or other frameworks to integrate security throughout their organization and operational procedures.
Attempting to purchase point products, or simply “add security” to existing infrastructure is challenging at best. Starting with products that are designed from the ground up with security is a key priority. According to The Futurum Group’s research, a significant number of security breaches utilize compromised servers as part of their attack vector. Thus, securing servers, their networking interfaces and storage devices is an ideal area of focus for companies looking to improve their cyber resiliency.
In assessing Dell PowerEdge servers together with Broadcom add-in cards against the Futurum Group’s Security Framework, we found that the systems meet or exceed requirements in all categories. There were several areas of strength noted, where PowerEdge significantly exceeded expectations. These include several security integrations between Dell’s silicon RoT and Zero Trust approach, together with Broadcom NICs and PERC RAID cards own embedded RoT and Zero Trust capabilities and importantly with Dell’s management tools.
Another area where PowerEdge exceeded expectations is the seamless management capabilities of iDRAC for baseboard system management, with OME for datacenter management, coupled with CloudIQ for global monitoring, alerting and reporting of multiple datacenters. The particular focus on security alerting and recommended actions of CloudIQ help IT administrators move from inaction due to an overwhelming number of issues, to action focused on the most important configuration or security issues.
Finally, Dell’s SCV process significantly exceeded expectations, offering a best-in-class end-to-end process for ensuring system integrity from suppliers, manufacturing and configuration, shipping, delivery and installation. A significant area of concern for organizations is ensuring their systems do not contain malicious hardware elements that could circumvent even robust security practices. Dell’s SCV provides a simple and easy way for organizations to ensure their infrastructure has not been compromised.
Taken together, Dell PowerEdge servers, with Broadcom NIC cards provide comprehensive security capabilities that provide a foundation for IT users to quickly and efficiently establish a highly secure computing environment that leverages industry best practices.
Resources
[1] NIST Cybersecurity Framework, draft version 2.0, August 2023:
About The Futurum Group
The Futurum Group is dedicated to helping IT professionals and vendors create and implement strategies that make the most value of their storage and digital information. The Futurum Group services deliver in-depth, unbiased analysis on storage architectures, infrastructures, and management for IT professionals. Since 1997 The Futurum Group has provided services for thousands of end-users and vendor professionals through product and market evaluations, competitive analysis, and education.
Copyright 2023 The Futurum Group. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written consent of The Futurum Group. The information contained in this document is subject to change without notice. The Futurum Group assumes no responsibility for errors or omissions and makes no expressed or implied warranties in this document relating to the use or operation of the products described herein. In no event shall The Futurum Group be liable for any indirect, special, inconsequential, or incidental damages arising out of or associated with any aspect of this publication, even if advised of the possibility of such damages. All trademarks are the property of their respective companies.
This document was developed with funding from Dell Inc. and Broadcom. Although the document may utilize publicly available material from various vendors, including Dell, Broadcom and others, it does not necessarily reflect such vendors' positions on the issues addressed in this document.
Related Documents
How Dell and Broadcom can help you make the transition to IPv6
Thu, 14 Mar 2024 16:56:04 -0000
|Read Time: 0 minutes
IPv4 vs. IPv6: How we got here
As the internet grew and commercialized late last century, it became increasingly clear that Internet Protocol version 4 (IPv4) limitations would eventually present issues. Enter Internet Protocol version 6 (IPv6) in the 1990s. Despite the technology’s age, its adoption has been slow in the US. Until very recently, many companies and other entities still primarily used IPv4, as shown by IPv6 adoption trackers such as one from Google.[1] Recently, however, the transition to IPv6 has been ramping, as some of those IPv4 limitations—such as the dwindling pool of available IPv4 addresses—are quickly becoming reality. One effort to encourage this transition includes a 2020 mandate from the U.S. Office of Management and Budget (OMB) requiring federal government agency devices be at least 80 percent IPv6-only by 2025.[2]
Regardless of mandates or address pools, there are a host of other reasons to choose IPv6 over IPv4. IPv6 includes features such as support for larger packets and multicasting, simpler header formats, smaller routing tables, and the elimination of the network address translation (NAT) process—all of which can increase performance over IPv4 in certain use cases. IPv6 also has built-in end-to-end encryption and name resolution protocol enhancements that contribute to better base security than IPv4.
Despite the advantages of using IPv6, some companies have resisted transitioning because it’s not a small undertaking. In this paper, we explain why making the transition can be worth the investment and introduce a solution to help make the transition easier: Dell and Broadcom™ combine to have one of the first IPv6-only compliant end- to-end solutions. We also present the results of our testing, including performance advantages for IPv6 over IPv4 on read workloads and larger performance increases for IPv4 and IPv6 available by enabling the Offload feature in the Broadcom network interface cards (NIC).
Why organizations are shifting to IPv6
The transition to IPv6 has been a long, slow process that is complicated by the fact that IPv4 and IPv6 are not compatible, requiring companies to either choose just one or manage two networks via dual stack. For many companies, however, fully abandoning IPv4 is not an option. Doubling the number of networks you deploy means doubling the security concerns and hardware expenses. Additionally, applications built on IPv4 may need rebuilding or updating to work with IPv6. Despite these factors, we believe these complications to be worth the benefits you gain from taking advantage of the IPv6 landscape. Not only could organizations use the features we mention above, but as more companies and users move to IPv6, it will also be easier for others to follow. Thus, IPv6 will grow more valuable over time. In time, we hope this can lead to companies being able to shed their IPv4 network, leaving the single, more efficient IPv6 network in place.
Below, we detail some of the reasons to transition to IPv6 including current issues with IPv4, specific industries or government agencies with particular IPv6 requirements, and the benefits of IPv6.
The shortage of IPv4 addresses
One of the most important reasons for the push to transition to IPv6 is the limited number of possible IPv4 addresses. The IPv4 address space is a 32-bit field, meaning there are a total of 232, or roughly 4.3 billion, possible IPv4 addresses. As of November 2019, this address space was officially depleted, meaning there are no new IPv4 addresses to obtain.[3] This has created a premium on IPv4 addresses, allowing companies to sell or lease their addresses, increasing the costs of buying an IPv4 address. Amazon Web Services (AWS), for example, is adding a charge to every IPv4 address on its platform, citing a 300 percent cost increase of IPv4 addresses over the past 5 years.[4] Several workarounds for this problem exist, such as NAT, which allows organizations to map several private addresses within a local network to a single public address before transferring information to the internet. However, NAT comes with its own share of problems that can affect the performance and reliability of network applications. By adding an extra layer of translation and processing, NAT can introduce latency, errors, or packet loss.[5]
IPv6, by utilizing a 128-bit address field, increases available IP addresses to roughly 2128, or ~3.4*1028, essentially solving the address limitation for the foreseeable future. Companies that work with the Internet of Things (IoT), virtual reality (VR), self-driving vehicles, telecom, and other technologies requiring many IP addresses could avoid the IPv4 address market and limited address availability by moving to IPv6.
The federal government gets involved
US federal government agencies also find themselves impacted by the Office of Management and Budget (OMB) mandate, which claims that “full transition to IPv6 is the only viable option to ensure future growth and innovation in Internet technology and services.”[6] The latest version of the mandate states that running dual stack IPv4 and IPv6 networks, as previous versions of the mandate dictated, is too complex and no longer necessary. Instead, this new mandate requires IPv6-only networking, outlining four actions agencies must take:
- Create an IPv6 project team.
- Create and publish an agency-wide policy that states their intentions to phase out all IPv4 use and make all federal IT systems IPv6 enabled by the end of 2023.
- Identify and test at least one IPv6 pilot by the end of 2021.
- Develop a plan by 2021 for implementing IPv6-only networking, with milestones including at least 50 percent of IP-enabled assets, transitioned to IPv6-only by the end of 2023, and 80 percent on IPv6-only networks by the end of 2025.[7]
This means that by the end of the first quarter of 2024, federal agencies should already have half of their systems converted to IPv6-only, and the rest fully transitioned in just two more years.
Telecom and ISP industries are leading the way
With the development of the 5G cellular network and its need for high speeds and low latencies, much of the telecom industry has already converted to IPv6. As more and more devices connect over cellular networks with 5G, the increased address pool of IPv6 provides another benefit for internet service providers (ISPs) and cellular network companies. Additionally, the built-in quality of service (QoS) field in the IPv6 header allows ISPs to prioritize voice traffic over other traffic less vulnerable to latency such as http, SSH, and more. According to Akamai, a content delivery network and cloud computing company, ISPs and telecoms such as Comcast Cable, Verizon Business, AT&T, and T-Mobile have all reached IPv6 adoption above 70 percent—T-Mobile has as much as 92.7 percent IPv6 adoption.[8]
As these network providers continue to expand the IPv6 backbone, and mobile app developers continue to embrace IPv6 advantages, the rest of the world’s industries will lag behind if they continue to rely on IPv4.
How IPv6 helps your business
Even if you don’t need a wide range of IP addresses or aren’t part of the government mandate or the telecom industry, there are still benefits you can see from transitioning from IPv4 to IPv6. First, IPv6 could increase performance in several ways, mostly by increasing network efficiency. Second, IPv6 can offer some additional security benefits over IPv4. While IPv4 has had more security upgrades and patches to existing networks—simply by virtue of existing longer—the features that come with IPv6 offer stronger baseline security, which we examine in the following pages. Increased use and investment in IPv6 security enhancements should quickly close any existing gap between it and current IPv4 network security.
IPv6 can also improve network performance over IPv4 by using a more simplified header that takes less time and fewer resources to process. All IP packets include headers that contain the necessary information for proper route allocation and delivery. Much like the parts of a physical address tell postal workers the house, street, city, and country the letter originated from and is destined for, the IP header includes such information as the IP addresses of the source and destination devices, the version indicator, the total length of the packet, and other important information. Instead of requiring routers to perform a header checksum to ensure data integrity of every packet, IPv6 relies on Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and other existing protocol checksums. Additionally, this header allocates optional and non-essential header fields to header extensions, leaving only the most vital information in the header for processing. This practice increases the efficiency of processing data packets across the network by getting the metaphorical “letter” to the right “building” quickly and letting the extension headers then direct it more specifically, to the equivalent of the correct suite, floor, or person.
IPv6 also eliminates the need for NATs as each device on an IPv6 network can have its own unique IP address. If devices aren’t having to share IP addresses, then the routers do not need to translate the network addresses to send the packet to the correct device, thus eliminating a step in the data transmission process. Finally, while the Maximum Transmission Unit (MTU) technology limits packet sizes in many networks to 1,500 bytes, IPv6 networks are capable of much larger packet sizes, up to 4 GB. IPv4 packets, on the other hand, are limited to much smaller theoretical maximums of 64K bytes.[9]
Another way that IPv6 increases network efficiency is with its ability to multicast instead of broadcast. On IPv4, data transmission is broadcast: When a packet leaves a source device, the information is sent to every host connected to the network. Every single host, in turn, checks the packet to see if the data is meant for it. IPv4 is like a kid yelling out, “Mom!” at a crowded playground, causing every mother in the vicinity to stop and check if their child is the one in need. IPv6, on the other hand, uses multicast, which is the ability to transmit a packet only to the device or devices for which it is intended. Now, instead of a child yelling, “Mom!” into a crowd, IPv6 taps their parent on the shoulder and talks directly to them, allowing the other moms to focus on their own children.
IPv4 routing tables, the list of networks and other links a router consults to determine where a packet should go, are quite large—and they continue to grow. The larger the table, the longer it takes to search and find the relevant data. While IPv4 addresses and networks can be difficult to aggregate or simplify, the structure of the IPv6 address allows for just that. The IPv6 address contains three parts: the network or site prefix, the subnet ID, and the host or interface ID.[10] Routers use a site prefix to route the packet through the internet. Network creators and ISPs can also use site prefixes to create packet groups that aggregate packets going in the same general direction. Aggregating the IPs like this allows the internet and routers to act similarly to public transportation. A person boards the train at a specific stop, rides it to another stop, then leaves the train and follows the maps to the exit nearest the actual destination they have in mind. With IPv6, packets behave similarly, where several packets with completely different final destinations can all “exit” the internet at the same stop and travel more granularly from there. This behavior allows for smaller, more efficient routing tables, speeding up the routing process and lowering the overhead on router hardware.
As we mentioned above, comparing IPv4 and IPv6 security isn’t completely straightforward, as professionals have invested more time and effort into the older IPv4 network. However, IPv6 has the potential to be more secure than IPv4 due to at least two built-in advantages. First, IPv6 has end-to-end encryption and authentication built in via default Internet Protocol Security (IPsec) inclusion. While enabling IPsec on IPv6 networks may not be mandatory in some places, implementing it provides better security.[11] The second way IPv6 enhances security is simply by being much larger. With its more numerous IP addresses, IPv6 networks are nearly impossible to brute-force scan.[12]
One more benefit of IPv6 is the ability for users to implement stateful or stateless configurations. While IPv4 network devices rely heavily on dynamic host configuration protocol (DHCP) devices to assign IP addresses, IPv6 networks can use the stateless address auto configuration (SLAAC) technology to let devices generate their own IPs without manual application or the use of a third-party device such as DHCP. If users prefer to use a stateful network, IPv6’s version of DHCP is available.
The NIST and transitioning to IPv6
According to the National Institute of Standards and Technology (NIST), “the IPv6 protocol
suite offers a vastly greater address space than IPv4 and supports significant new capabilities necessary to enable modern network environments.”[13] To facilitate this transition, the NIST National Cyber-security Center of Excellence (NCCoE) “is planning a project to provide guidance and reference architecture that address operational, security, and privacy issues associated with the evolution to IPv6-only network infrastructures.”[14]
The project aims to provide enterprise organizations attempting to transition to IPv6 with guidance and tools that will “ensure that evolving enterprise IT environments to be IPv6-only can be accomplished in a secure and robust manner.”[15]
Regardless of the size of your network, the NIST Cybersecurity Practice Guide will provide best-practices and documentation to ensure that your transition to IPv6 is secure. For more information about the transition resources available from the NIST, visit https://www.nccoe.nist.gov/projects/IPv6-transition.
How Dell servers and Broadcom NICs can help on your journey to IPv6
The National Institute of Standards and Technology (NIST) is part of the U.S. Department of Commerce and serves to support and promote technology and science innovation and investment.[16] In addition to mandating that government agencies switch to IPv6-only networks, the OMB has mandated the NIST to create a set of standards and tools to support the transition. The resulting U.S. Government IPv6 (USGv6) Program develops, tests, and maintains IPv6 standards to help companies and government agencies ensure successful IPv6 transitions and deployments. Vendors can use the IPv6 tests and certifications that NIST developed to certify that their products meet the requirements and standards defined by NIST’s program.[17] In response to the 2020 OMB mandate for all government agencies, NIST revised their USGv6 program to include several objectives including updating specifications to add new and remove old technologies and streamlining their testing program based on previous experience.[18] With a set of agreed upon standards, definitions, and requirements, NIST and the USGv6-r1 provide OEMs a way to ensure their customers that their products are ready for IPv6 implementation.
Dell Technologies™ is the first company to offer a full USGv6-r1 certified server and storage stack for IPv6-only networking.[19] Dell’s certifications include:
- Dell™ PowerEdge™ servers – first in the industry to be fully IPv6 Ready Logo 5.1.2 compliant with RedHat 8.4 and Windows 2019 and 2022.
- Dell PowerEdge iDRAC9 with FW version 5.10.0.00 – first baseboard management controller (BMC) validated by USGv6-r1 as IPv6-only compliant.
- Unity-XT storage array – first storage product validated by USGv6-r1 as IPV6-only compliant meeting the requirements in the IPv6-Only Functional v1.1 (36277) profile.[20]
- Additionally, PowerStore, PowerEdge with VMware 8.0.1, and PowerEdge with SUSE SLES15 SP4 are on the USGv6-r1 registry.
Additionally, Dell servers and storage leverage Broadcom NICs and Adapters to ensure network performance and security for IPv6 customers. Broadcom NICs such as the Broadcom BCM957508-P2200G dual-port 100GbE NIC include several IPv6 offloads that can boost network performance.[21] These offloads allow the NIC to directly handle some of the computational needs of the network rather than use the OS stack, which can provide lower latencies and lower CPU utilization dedicated to network traffic.[22] Broadcom BCM957508-P2200G dual-port 100GbE NICs also offer features such as NVME over Fabrics (NVMe-oF) capabilities that allow NVMe storage traffic to travel through network instead of directly through PCIe channels. NVMe-oF allows users to connect storage via Ethernet (TCP), Fibre, and RDMA.[23] This rerouting of storage network allows for extremely low latency to get the most out of NVMe-based storage.
To show how Dell and Broadcom can provide one great hardware stack option for your IPv6 needs, we conducted some testing to highlight the performance you can expect with IPv6-connected Dell PowerEdge R660 servers to a PowerStore 1200T storage array using 100GbE Broadcom 57508 NICs.
Broadcom NICs
Broadcom NICs can serve most networking needs because they offer speeds ranging from 1G to 200G. According to Broadcom, their network cards feature:
- “Low power adapters and controllers with outstanding thermal performance
- Low latency and high throughput RoCEv2 [for] ground-breaking performance
for machine learning, HPC and
storage applications - Broadsafe™ embedded security [for] Silicon Root of Trust and attestation delivering industry’s most secure Ethernet controller
- Modern architecture [that] delivers industry’s lowest latency and lowest CPU utilization for real-world network conditions
- TruFlow™ engine [to accelerate] virtual switch processing, reduces server CPU usage
- TruManage™ [for] end-user manageability needs to allow fine-tuning of networks for maximum performance
- On-chip tunneling protocol processing for Geneve, VXLAN, and NVGRE [that] provides up to a 5x throughput increase
- Acceleration engines for SDN and NFV [to] enable leading-edge service provider solutions”[24]
Measuring performance
The goal of our performance testing was to show the benefits of the USGv6-r1 IPv6-only certified Dell PowerEdge server and Broadcom NIC solution. This included investigating the performance differences between IPv4 and IPv6 in a real-world environment. Most North American users continue to rely on IPv4, which typically requires NAT or packet fragmentation support from a network router.[25] In a typical scenario, a routing device need only read an IPv4 packet to determine its destination and send it on its way. In the case of IPv4 using NAT or requiring packet fragmentation, the routing device must modify the packet before it can send it along, which requires overhead. We wanted to quantify the impact of this overhead on network performance.
Other than the Layer 3 protocol, every aspect of the test scenario was the same. We used Linux standard tools and NVMe/TCP and/or NFS transport protocols for this test. We did not attempt to enable the best speed of each of these protocols; rather, we used those protocols to drive the tests to compare any differences in speed based on the journey of that data provided by Layer 3 (IP).
We configured two Dell PowerEdge servers as SUSE Linux Enterprise Server 15 SP4 hosts, sending data of diverse sizes to a Dell PowerStore storage array using the transport protocols we identified earlier. The data traversed multiple switches we configured to provide Border Gateway Protocol (BGP) routing and packet fragmentation within a heterogeneous multi-hop network.
We configured the host networks using an MTU of 9,000, with a 1,500 MTU on the switches emulating the core network (which forced packet fragmentation). The edge switches used BGP routing to communicate with the core network.
Comparing IPv6 and IPv4 performance without the Broadcom Offload feature
First, we tested the relative performance of IPv6 and IPv4 on a write workload with the Broadcom Offload feature off. Table 1 presents the results. In terms of both performance (IOPS and throughput in MB per second) and CPU utilization, we observed approximate parity between the two IP versions at both block sizes we tested.
Table 1. IPv6 vs. IPv4 performance on a write workload with Offload off. Higher IOPS and MB/sec and lower CPU utilization are better. Source: Principled Technologies.
Write workload, Offload off |
|
|
| |
IP version | Block size | IOPS | MB/sec | Percentage CPU utilization |
IPv4 | 256K | 8,696.1 | 2,174.01 | 4.9 |
IPv6 | 256K | 8,752.1 | 2,188.02 | 4.9 |
IPv6 % improvement | 0.64% | 0.64% | 0.00% | |
IPv4 | 64K | 34,862.7 | 2,178.92 | 6.6 |
IPv6 | 64K | 34,972.1 | 2,185.76 | 6.5 |
IPv6 % improvement | 0.31% | 0.31% | 1.51% |
Next, we ran the same test using a read workload. As Table 2 shows, in contrast to the comparable performance we observed on the write workload, IPv6 had a performance advantage over IPv4 on the read workload. At the larger block size of 256K, IPv6 delivered 13.83 percent greater performance. At the smaller block size of 64 K, IPv6 delivered 9.83 percent greater performance. These results indicate that users in a real-world setting would enjoy better performance by using IPv6. We also observed a CPU utilization improvement for IPv6.
Table 2. IPv6 vs. IPv4 performance on a read workload with Offload off. Higher IOPS and MB/sec and lower CPU utilization are better. Source: Principled Technologies.
Read workload, Offload off |
|
|
| |
IP version | Block size | IOPS | MB/sec | Percentage CPU utilization |
IPv4 | 256K | 19,987.8 | 4,996.95 | 14.1 |
IPv6 | 256K | 22,752.4 | 5,688.09 | 13.6 |
IPv6 % improvement | 13.83% | 13.83% | 3.54% | |
IPv4 | 64K | 73,194.1 | 4,574.63 | 13.9 |
IPv6 | 64K | 80,392.4 | 5,024.53 | 12.5 |
IPv6 % improvement | 9.83% | 9.83% | 10.07% |
Measuring the impact of the Broadcom Offload feature on IPv6 performance
A secondary component of our testing was investigating the capabilities of the Broadcom IP Offload feature. IP Offloading is a feature Broadcom has implemented in its NIC (Layer 2) to process IP (Layer 3) data to offload the processing of this data from the OS/CPU, leaving those clock cycles to process user data rather than managing the flow control of the protocol. We refer to this feature as Offload.
Table 3 presents IPv6 performance on a write workload with Offload off and with Offload on. While performance was comparable under both conditions, CPU utilization was lower with Offload on.
Table 3. IPv6 performance on a write workload with Offload off and Offload on. Higher IOPS and MB/sec and lower CPU utilization are better. Source: Principled Technologies.
Write workload, IPv6 |
|
|
| |
Block size | IOPS | MB/sec | Percentage CPU utilization | |
Offload off | 256K | 8,752.1 | 2,188.02 | 4.9 |
Offload on | 256K | 8,615.5 | 2,153.88 | 2 |
Offload on % improvement | -1.56% | -1.56% | 59.18% | |
Offload off | 64K | 34,972.1 | 2,185.76 | 6.5 |
Offload on | 64K | 34,895.6 | 2,180.97 | 3.4 |
Offload on % improvement | -0.21% | -0.21% | 47.69% |
Table 4 presents IPv6 performance on a read workload with Offload off and with Offload on. In contrast to the approximate parity we saw on the write workload, performance improved greatly with the use of Offload, particularly at the larger block size, where IPv6 delivered 58.15 percent greater performance than with Offload off. At the 64K block size, enabling Offload improved performance by 25.43 percent. Figure 1 illustrates these advantages. As we saw with the write workload, CPU utilization was lower with Offload on.
Table 4. IPv6 performance on a read workload with Offload off and Offload on. Higher IOPS and MB/sec and lower CPU utilization are better. Source: Principled Technologies.
Read workload, IPv6 |
|
|
| |
Block size | IOPS | MB/sec | Percentage CPU utilization | |
Offload off | 256K | 22,752.4 | 5,688.09 | 13.6% |
Offload on | 256K | 35,983.4 | 8,995.86 | 8.3% |
Offload on % improvement | 58.15% | 58.15% | 38.97% | |
Offload off | 64K | 80,392.4 | 5,024.53 | 12.5% |
Offload on | 64K | 100,840.9 | 6,302.55 | 7.3% |
Offload on % improvement | 25.43% | 25.43% | 41.60% |
Figure 1. Performance improvement of IPv6 using Offload feature on a read workload. Higher is better. Source: Principled Technologies.
Measuring the impact of the Broadcom Offload feature on IPv4 performance
Table 5 presents IPv4 performance on a write workload with Offload off and with Offload on. As we saw with IPv6, performance was comparable under both conditions and CPU utilization improved with Offload on.
Table 5. IPv4 performance on a write workload with Offload off and Offload on. Higher IOPS and MB/sec and lower CPU utilization are better. Source: Principled Technologies.
Write workload, IPv4 |
|
|
| |
Block size | IOPS | MB/sec | Percentage CPU utilization | |
Offload off | 256K | 8,696.1 | 2,174.01 | 4.9 |
Offload on | 256K | 8,596.0 | 2,148.99 | 2.0 |
Offload on % improvement | -1.15% | -1.15% | 59.18% | |
Offload off | 64K | 34,862.7 | 2,178.92 | 6.6 |
Offload on | 64K | 34,727.8 | 2,170.49 | 3.6 |
Offload on % improvement | -0.38% | -0.38% | 45.45% |
As Table 6 shows, the impact of enabling Offload on IPv4 read performance followed the same pattern we saw with IPv6. Using Offload dramatically improved read performance, by 77.90 percent at the larger block size and by 38.73 percent at the 64K block size. Figure 2 highlights these performance improvements. Once again, using Offload improved CPU utilization.
Table 6. IPv4 performance on a read workload with Offload off and Offload on. Higher IOPS and MB/sec and lower CPU utilization are better. Source: Principled Technologies.
Read workload, IPv4 |
|
|
| |
Block size | IOPS | MB/sec | Percentage CPU utilization | |
Offload off | 256K | 19,987.8 | 4,996.95 | 14.1 |
Offload on | 256K | 35,559.3 | 8,889.81 | 8.1 |
Offload on % improvement | 77.90% | 77.90% | 42.55% | |
Offload off | 64K | 73,194.1 | 4,574.63 | 13.9 |
Offload on | 64K | 101,545.6 | 6,346.60 | 7.4 |
Offload on % improvement | 38.73% | 38.73% | 46.76% |
Figure 2. Performance improvement of IPv4 using the Broadcom Offload feature on a read workload. Higher is better. Source: Principled Technologies.
While we have discussed many advantages to making the shift to IPv6, our test results demonstrate that companies who opt not to do so immediately could reap performance benefits on read workloads—and CPU utilization benefits on both read and write workloads—by using the Dell-Broadcom solution we tested and enabling the Broadcom NIC Offload feature.
Conclusion
With IPv4 address pools rapidly disappearing and a federal mandate for government agency devices to begin shifting to IPv6-only and telecom 5G with IoT and edge devices, it’s clear that IPv6 is the future. Transitioning from IPv4 to IPv6 can be a challenge, so organizations may be interested to learn that switching to IPv6 has the potential to improve performance. In our testing without the Broadcom Offload feature, IPv6 delivered comparable performance to IPv4 on write workloads and better performance on read workloads while also reducing CPU utilization. When we enabled the Broadcom Offload feature on both IPv6 and IPv4, read workload performance increased dramatically and CPU utilization on both read and write workloads improved. Whether your organization is transitioning to IPv6 right away or choosing to delay the shift, this feature can boost performance on read workloads, which can improve the experience for users, reduce backup windows, and allow databases to load more quickly.
This project was commissioned by Dell Technologies.
January 2024
Principled Technologies is a registered trademark of Principled Technologies, Inc.
All other product names are the trademarks of their respective owners.
[1] Google, “IPv6,” accessed December 21, 2023, https://www.google.com/intl/en/ipv6/statistics.html.
[2] Russell T. Vought, “MEMORANDUM FOR HEADS OF EXECUTIVE DEPARTMENTS AND AGENCIES,” accessed December 21, 2023, https://www.whitehouse.gov/wp-content/uploads/2020/11/M-21-07.pdf.
[3] RipeNCC, “What is IPv4 Run Out?” accessed December 21, 2023,
[4] Jeff Barr, “New – AWS Public IPv4 Address Charge + Public IP Insights,” accessed December 21, 2023,
https://aws.amazon.com/blogs/aws/new-aws-public-ipv4-address-charge-public-ip-insights/.
[5] Linkedin, “How can NAT affect the performance and reliability of network applications?” accessed December 21,
2023, https://www.linkedin.com/advice/0/how-can-nat-affect-performance-reliability-network.
[6] Russell T. Vought, “MEMORANDUM FOR HEADS OF EXECUTIVE DEPARTMENTS AND AGENCIES,” accessed December 21, 2023, https://www.whitehouse.gov/wp-content/uploads/2020/11/M-21-07.pdf.
[7] Russell T. Vought, “MEMORANDUM FOR HEADS OF EXECUTIVE DEPARTMENTS AND AGENCIES.”
[8] Akamai, “IPv6 Adoption Visualization,” accessed December 21, 2023,
[9] Network Academy, “IPv4 vs IPv6 - Understanding the differences,” accessed December 21, 2023,
[10] Diego Asturias, “Migration to IPv6: Benefits and Techniques,” accessed December 21, 2023,
[11] Diego Asturias, “Migration to IPv6: Benefits and Techniques.”
[12] Diego Asturias, “Migration to IPv6: Benefits and Techniques.”
[13] “IPv6 Transition,” accessed December 21, 2023, https://www.nccoe.nist.gov/projects/IPv6-transition.
[14] “IPv6 Transition.”
[15] “IPv6 Transition.”
[16] NIST, “About NIST,” accessed December 21, 2023, https://www.nist.gov/about-nist.
[17] NIST, “USGv6,” accessed December 21, 2023, https://www.nist.gov/programs-projects/usgv6-program/usgv6.
[18] NIST, “USGv6 Revision 1,” accessed December 21, 2023,
https://www.nist.gov/programs-projects/usgv6-program/usgv6-revision-1.
[19] George Dilger, “Industry First IPv6-only Support on Servers and Storage,” accessed December 21, 2023,
https://www.dell.com/en-us/blog/industry-first-ipv6-only-support-on-servers-and-storage/.
[20] George Dilger, “Industry First IPv6-only Support on Servers and Storage.”
[21] Broadcom, “Broadcom Ethernet NICs PCIe NIC Ethernet Adapters Specification Sheet,” accessed December 21,
2023, https://docs.broadcom.com/doc/PCIe-NIC-Ethernet-Adapters-Specification-Sheet.
[22] Eden Kim and Fred Zhang, “Optimizing NVMe® over Fabrics (NVMe-oFTM),” accessed December 21, 2023,
https://www.snia.org/sites/default/files/education/snia-optimizing-nvme-over-fabrics-nvme-of.pdf.
[23] Juan Mulford, “What is NVMe-oF?,” accessed December 21, 2023, https://www.storagereview.com/review/nvme-nvme-of-background-overview.
[24] “Ethernet Network Adapters,” accessed December 21, 2023,
https://www.broadcom.com/products/ethernet-connectivity/network-adapters.
[25] Google, “IPv6,” accessed December 21, 2023, https://www.google.com/intl/en/ipv6/statistics.html.
16th Generation Dell PowerEdge R760 server with Broadcom NICs: Save money, do more work, and use less energy
Mon, 15 Apr 2024 20:10:10 -0000
|Read Time: 0 minutes
Principled Technologies testing showed that a 16th Generation Dell PowerEdge R760 server featuring a Broadcom BCM57508-P2100G NIC delivered all three vs. previous-generation PowerEdge servers
The last 20 years have seen a dramatic increase in the amount of data businesses generate, analyze, and store. During this period of data proliferation, businesses gained access to a wealth of new sources of data from the rise of the internet, the widespread adoption of smart phones, and the growth of the internet of things (IoT). More powerful processors have made it possible for businesses to analyze this data in new ways using tools such as machine learning (ML) and artificial intelligence (AI). These developments mean many organizations need to add servers to their data centers to keep up with the growth in demand for compute capacity and storage, but it can be difficult to balance the need for higher performance and the challenge of reducing costs. Additionally, energy conservation has become increasingly important as energy costs have risen and businesses have made goals to reduce their carbon footprint to fight climate change.
We tested the performance and power efficiency of three generations of Dell PowerEdge servers to help businesses understand which of these servers to choose to meet the demands of data proliferation, reducing costs, and energy conservation. This report outlines our test methodologies and findings in the context of an increasingly cost-sensitive and conservation-conscious business landscape. We measured the Ops/s and the performance per watt ratio using the Memtier benchmark utility. We ran write and read commands against the popular open-source, in-memory data structure store Redis. Common use cases of Redis include message broker software[1] and real-time streaming analytics [2], among others.
Our testing showed that by choosing the 16th Generation Dell PowerEdge R760 featuring a Broadcom BCM57508-P2100G NIC, organizations can save more money, do more work, and use less energy.
Spend less to do the same amount of work with Dell PowerEdge R760 servers with Broadcom NICs
Figure 1: The Ops/s that the servers processed, the price of the servers in US dollars, and the Ops/s per US dollar of the servers. Higher Ops/s per US dollar is better. Source: Principled Technologies.
Figure 2: The Ops/s that the servers processed, the price of the servers in US dollars, and the Ops/s per US dollar of the servers. Higher Ops/s per US dollar is better. Source: Principled Technologies.
When looking at adding capacity, using the latest servers can boost performance, but as with any business decision, organizations should weigh the costs as well as the benefits associated with purchasing the latest equipment. Perhaps an organization has reasons they may consider equipment that is one or two generations back, say, to match existing equipment. However, this decision can be critical to both purchase and ongoing costs. To understand how much you can reduce costs by using the latest more powerful and more energy efficient servers, we obtained list pricing for the configurations we tested.[3] We configured the servers in these price comparisons as closely as possible to those we tested, but some minor differences were unavoidable because of product availability.
The PowerEdge R760 with a Broadcom NIC processed 2.29 times as many Ops/s as the PowerEdge R740 at a cost that is 13.7 percent lower. As Figure 1 shows, the PowerEdge R760 processed 166.1 percent more Ops/s per Dollar as the PowerEdge R740. In addition to the lower purchase price, the PowerEdge R760 was 24.2 percent more energy efficient in our tests, which could further reduce costs by requiring less energy for operation and cooling.
The PowerEdge R760 with a Broadcom NIC processed 30.6 percent more Ops/s than the PowerEdge R750 at a cost that is only 3.9 percent higher. The cost difference between the PowerEdge R750 and the PowerEdge R760 is relatively small compared to the difference in performance, and as Figure 2 shows, the PowerEdge R760 server offered 25.4 percent more Ops/s per US dollar than the PowerEdge R750.
The impact of these cost, energy efficiency, and performance numbers will vary depending on the needs of a given data center. Let’s examine what the performance and value of these servers would look like in several scenarios.
If an organization needed the additional database processing power of 10 PowerEdge R760 servers, they could meet those needs with any of the servers we tested, but they would spend the least ($318,236.20) if they bought 10 PowerEdge R760 servers. To meet those needs with the PowerEdge R750 servers, they’d spend $399,181.54, and with the PowerEdge R740 servers, they’d pay $847,046.62.
Figure 3: A comparison of how many servers it would take to do the same work as 10 Dell PowerEdge R760 servers and the cost in USD. Source: Principled Technologies.
The savings scale with the size of an organization’s database processing needs: 500 PowerEdge R760 servers would cost $15,911,810.00 and process 32,141,262,500 Ops/s. To do the same amount of work, the organization would need to purchase approximately 653 PowerEdge R750 servers, which would cost $19,956,907.00: over $4 million US dollars more than choosing the PowerEdge R760. Using this same hypothetical, an organization would need to purchase approximately 1,147 PowerEdge R740 servers to do the work of 500 PowerEdge R760 servers, at a cost of $42,319,539.95: over $26 million US dollars more than choosing the PowerEdge R760.
When you consider the higher performance, the savings discussed above, and the lower energy costs of the more efficient PowerEdge R760 with a Broadcom NIC, it becomes clear that organizations will save the most money by choosing the latest-generation Dell PowerEdge R760 server.
Better performance with the Dell PowerEdge R760 server
Figure 4: The Ops/s the servers processed. Higher is better. Source: Principled Technologies.
Figure 5: The throughput of the servers in MB/s. Higher is better. Source: Principled Technologies.
Figure 5: The performance per watt of the servers in Ops/s per watt. Higher is better. Source: Principled Technologies.
Bytes per second and operations per second (Ops/s) are both measures of throughput, or rates. Bytes per second are a system level statistic that typically come from either disk or network. We often refer to this metric in terms of a multiplier, such as megabytes per second (MB/s) or gigabytes per second (GB/s). In the case of our testing, we measured MB/s that passed through the ethernet controller. Ops/s typically refers to an application measure, in this case the operations per second that Redis processed during our memtier tests. Both measures of throughput are important to consider depending on what you are trying to measure. An increase in either number shows improvement in the amount of data (MB/s) or operations (Ops/s) that the server or application processed in a fixed amount of time.
When we compared the throughput of the three Dell PowerEdge servers, we found that the PowerEdge R760 server offered up to 126.9 percent more MB/s throughput and processed up to 129.5 percent more Ops/s than the PowerEdge R740 server. The PowerEdge R760 offered up to 30.8 percent more MB/s throughput and processed up to 30.6 percent more Ops/s than the previous-generation PowerEdge R750 server. Organizations that want the highest performing server should consider the PowerEdge R760, based on these findings.
Save money by getting more performance per watt with the PowerEdge R760 server featuring a Broadcom NIC
To understand the power efficiency of the servers, we measured the work a server did (Ops/s), and divided it by the power the server used (watts). We found that the PowerEdge R760 with a Broadcom NIC offered up to 24.2 percent better operations per second per watt than the 14th Generation Dell PowerEdge R740 server, and up to 9.2 percent better operations per second per watt than the 15th Generation Dell PowerEdge R750 server. Organizations that select these new servers with Broadcom ethernet controllers can reduce the amount of power necessary to process a given unit of work, which will help reduce their energy costs and carbon footprint.
Our hands-on testing of the Dell PowerEdge R760
To help organizations understand which PowerEdge server best meets their needs, we ran a series of tests comparing the performance and power efficiency of three servers that Dell is currently selling:
- 16th Generation Dell PowerEdge R760
- 15th Generation Dell PowerEdge R750
- 14th Generation Dell PowerEdge R740
These results might also be helpful for organizations considering upgrading their older servers to the latest-generation PowerEdge R760.
For testing, we used the database Redis, a widely-used, in-memory, open-source key-value store—a type of NoSQL database that uses pairs of keys and associated values to quickly access information. Companies including Twitter, GitHub, Snapchat, Craigslist, and StackOverflow rely on Redis for their operations.[4]
In the case of traditional relational databases, the architecture is such that the designers of the database must model, or structure, the data in a certain way. When applications request data and the data is not in memory, the database must retrieve the data from disk, causing a small delay. This architecture works well for some applications, but not all. In many types of applications, a developer may need to store data with less structure—perhaps strings, hashes, lists, or arrays. Redis works well for this, as its architecture is based on a key and a value. A key can have multiple values, resulting in databases that are comprised of a long list of keys and their associated values, with the potential for fewer complexities and slowdowns of relational databases. This data design makes Redis useful for web application user session data (such as cookies) and other small elements of data that web server applications need frequently. Caching that data in Redis allows web applications to return data very quickly.
Memtier, the benchmark utility we used, is a utility that runs set (write) and get (read) commands against a Redis database. Those commands simulate different request patterns. According to Redis, the memtier_benchmark is “a high-throughput benchmarking tool” that can “generate various traffic patterns against both Memcached and Redis instances.”[5] Various parameters that the operator can use, such as data size and key-pattern, can alter the traffic pattern that memtier returns. By operating the memtier utility on a client server and targeting a Redis server, we utilized the Redis server’s processor and ethernet controller subsystems heavily.
We configured all three of the servers we tested with Red Hat Enterprise Linux 8.7 and the most recent firmware as of March 14th, 2023 for the PowerEdge R740 and PowerEdge R750, and April 6th, 2023 for the PowerEdge R760. We recorded the operations per second (Ops/s) and throughput (MB/s) during the test and used those readings for our results. We used an external power meter to measure the power draw by the power supply units (PSUs) while the servers were under load and recorded the average power draw for the entire duration of the test. Then, we calculated the power efficiency (Ops/s/watt) of the devices using the results of our testing. We wanted to utilize close to 100 percent of the database server’s CPU capacity to see how the systems performed under stress. We used the memtier_benchmark to create a roughly 100GB database and ran a constant size, 32 byte (default/small), 100 percent read workload to create enough load to reach nearly 100 percent CPU usage. Read-only operations constitute the majority of the workload in applications such as content streaming and data analysis. Additionally, testing with read-only operations provides comparable performance metrics without the additional complexity of write operations.
Dell PowerEdge R760 servers with Broadcom BCM57508-P2100G NICs
The Dell PowerEdge R760 server is a 2U, two-socket server that features a Broadcom BCM57508-P2100G Dual-Port 100GbE PCl Ethernet Controller. Dell designed the PowerEdge R760 to handle “demanding workloads including artificial intelligence, database analytics, and high-density virtualization.”[6] A wide range of businesses can use the PowerEdge R760 for workloads that demand high performance, such as:
- Artificial intelligence
- Virtualized workloads
- Machine learning
- Database analysis
According to Dell, users of the PowerEdge R760 server can:
- “Add up to two 4th generation Intel® Xeon® scalable processors with up to 56 cores for fast and accurate performance
- Accelerate in-memory workloads with up to 32 DDR5 RDIMMS up to 4800 MT/s(1DPC)
- Maximize performance with up to 2x double-wide or 6x single-wide GPUs for finishing workloads faster
- [Use the] new Smart Flow chassis to air-cool some of the highest core count CPUs within current IT infrastructure”[7]
The PowerEdge R760 server includes built-in security features such as secured component verification, signed firmware and drift detection, dynamic system lockdown, and BIOS recovery.[8] The PowerEdge R760 also features intelligent automation features such as OpenManage Enterprise, CloudIQ solutions, and PowerEdge iDRAC. The OpenManage systems management portfolio enables administrators to deploy, configure, monitor, and manage PowerEdge servers, including easily obtaining power data and capping power as necessary. For more information about the Dell PowerEdge R760 server, visit: https://www.dell.com/en-us/shop/productdetailstxn/poweredge-r760.
About the Broadcom BCM57508-P2100G Dual-Port 100GbE PCle 4.0 ethernet controller
A network interface controller (NIC) is a hardware component in a server that connects the server to a network. The NIC translates outgoing data from the server into a format that the network can read and translates incoming data from the network into a format that the server can read. A higher performing NIC can reduce latency, increase throughput, and allow the server to transmit and receive a great volume of data.
The Dell PowerEdge R760 features the Broadcom BCM57508-P2100G Dual-Port 100GbE PCle 4.0 ethernet controller, which supports speeds of up to 200 Gigabits per second. Broadcom designed the BCM57508-P2100G “to build highly-scalable, feature-rich networking solutions in servers for enterprise and cloud-scale networking and storage applications, including high-performance computing, telco, machine learning, storage disaggregation, and data analytics.”[9]
The BCM57508-P2100G features TruFlow™, which Broadcom asserts can “increase VM density up to 50 percent.”[10] The BCM57508-P2100G also features BroadSAFE® technology, “to provide unparalleled platform security” and a “unique set of highly-optimized hardware acceleration engines to enhance network performance and improve server efficiency.”[11]
Efficiency and expansion: Unraveling the challenges of the growing data center market
We’ve shown that the PowerEdge R760 is more energy-efficient and offers better performance than previous-generation PowerEdge servers, but why does that matter? Organizations realize the value of their data and the insights they provide. Despite economic uncertainty, spending on servers has remained consistent with previous years, and many experts expect it to increase.[12] According to one industry analyst, “despite the general tightening of spending by governments and corporations..., organizations are still spending on infrastructure even if they are, in some cases, cutting back on personnel.”[13] Increasingly, to deal with the ever growing importance of their data, businesses are aware that adopting new technologies such as ML and AI is “both a necessity and a race.”[14]
While the new 16th Generation server we tested uses more power, it does so more efficiently—in this example resulting in an approximately 20 percent lower annual cost, based on the following calculations. In our example if we factor in power usage, our tested 16th Generation server used approximately 831W (.831kW), while the 14th Generation server used approximately 450W (.450kW). In an earlier section we established that an organization would need 1,147 PowerEdge R740 servers to do the work of 500 PowerEdge R760 servers. Thus we can calculate the data center power cost per year as follows:
Figure 7: Calculations showing the annual power cost of the servers in US dollars. Lower is better. Source: Principled Technologies.[15]
Sustainability challenges
As new technologies and data proliferation have increased the demands on data center performance and capacity, sustainability concerns have begun to put pressure on companies to increase the efficiency and reduce the carbon footprint of their data centers. By some estimates, the carbon footprint of data centers worldwide equals that of the airline industry.[16] According to S&P Global Market Intelligence, “the data center industry faces pressure from multiple sources— including government, financial markets, and corporate clients—to improve sustainability and reduce carbon emissions.”[17] Over the last few years, businesses have contended with volatile energy prices that have trended higher.[18] For many companies, the energy consumption of their servers is a significant expense.[19]
The sustainability challenges related to data centers are not limited to carbon emissions. Many data centers use large amounts of water to cool their servers. According to the U.S. Department of Energy, an average data center that uses evaporative cooling can consume 3 to 5 million gallons of water per day— similar to the daily capacity of a city of 30,000-50,000 people.[20] Roughly 20 percent of data centers in the U.S. rely on watersheds that are under stress from drought.[21] Choosing more efficient servers can help organizations meet internal and external climate goals and reduce their water usage and carbon footprints.
Managing costs over time
So far in this report we have explained why choosing the PowerEdge R760 to meet new capacity needs can save money and provide better performance, but what about organizations considering upgrading to newer, more efficient servers? In addition to the savings associated with reducing cooling costs, businesses that upgrade older servers may also save money by avoiding some of the other costs associated with delaying their server replacements. Older servers become more expensive to operate as they age. According to analysis by IDC, businesses reported an average annual operating cost of $7,541 per server during the first three years of use; during years 4 to 6, those servers reached a high of an average of $22,511 per server, a 199 percent increase.[22] Upgrading aging servers can reduce the amount of time IT spends on compliance, maintenance, and security, as newer servers often include the most recent manageability and security features. Aging servers may also contribute to slower response times and a diminished customer experience. Research by Deloitte found that “with a 0.1s improvement in site speed…consumers spent almost 10 percent more” on retail sites.[23] Upgrading to the latest-generation server technology can provide a better user experience.
Historically, server manufacturers have improved the performance of their systems in parallel with the growing demand for data processing, but the ever increasing reliance on data processing plus the rapid growth of resource-hungry technologies such as ML and AI have combined to make data center processing and power consumption challenges even more urgent. ML and AI workloads require a large amount of processing power, and therefore can lead to data centers running at higher power densities, which creates more demand on cooling systems. Some analysts estimate the use of ML will grow at a compound annual growth rate of 35.6 percent until 2030.[24] Data centers will need more powerful, more energy-efficient systems to keep up with this growing demand.
According to the International Monetary Fund (IMF), “energy prices surged to near historic highs in recent years amid the pandemic and the war in Ukraine.”[25] In their latest World Economic Outlook forecast, the IMF predicted “that growth will slow from 3.4 percent last year to 2.8 percent this year.”[26] Faced with volatile energy prices and growing economic uncertainty, many organizations are looking for ways to lower computing costs without sacrificing performance. Organizations can potentially gain performance and reduce cooling costs by upgrading their servers to more powerful models that consume less energy under load. Upgrading to more efficient servers can also help businesses meet their climate change goals.
Server power consumption
In the last two decades, server capabilities have increased dramatically, becoming more powerful and more efficient. While businesses have benefited from the advanced analytics and data processing capabilities of these new servers, they have come at a cost due to the energy required to run them. The design of a server and its components determine how much power it will consume during operation.
Thermal design power
Thermal design power (TDP) refers to the maximum theoretical amount of power, measured in watts, a computer chip (typically a processor or peripheral component) uses.
TDP can be a helpful starting point in considering what the total power draw of a server could be or what power supplies the server should be equipped with. Using these wattage approximations and other server energy usage information, data center designers can calculate the approximate British Thermal Units (BTUs) a data center will need for cooling. Because of advancements in technology and performance such as increased core count, the TDP of many processors has doubled in the last few years, making power considerations even more significant.[27]
Key factors in server design that affect power consumption
Multiple factors in server design affect power consumption. In many servers, the central processor uses more power than other components. Based on some models, the processor can consume as much as 32 percent of the server’s power.[28] A study by the Swedish Energy Agency examined data center and server energy consumption and found, “the central processing unit (CPU) is the largest contributor to the total server power consumption, followed by peripheral slots (including network card slot and input and output devices (I/O) devices), conduction losses, memory, motherboard, disk/storage, and cooling fan.”[29]
CPUs are, of course, not the only components that require power. Every component in a server consumes power, from disks to graphics processing units (GPUs) to power supply units (PSUs), and can therefore impact the server’s power efficiency. Adding more disks or GPUs might increase the server’s power consumption, while using fewer disks or no GPUs might decrease it. Similarly, using a higher efficiency power supply can reduce a server’s power consumption—according to one source, by 3 to 4 percent.[30]
The NIC also has an impact on the power consumption of a server. Based on the specifications Broadcom lists for the NICs in the servers we tested, the 100Gb Broadcom BCM957508-P2100G NIC in the PowerEdge R760 typically draws 16.0W and the 25Gb BCM957414-P225p NIC in the PowerEdge R740 typically draws 10.9W with Passive DAC at 55°C ambient temperature.[31] This means that choosing the PowerEdge R760 instead of the PowerEdge R750 would provide a NIC with four times the raw capacity that draws less than 50 percent more power. Based on the specifications Broadcom lists for a 10Gb BCM57712 NIC with the same configuration as the one in the PowerEdge R740 we tested, we estimate that selecting the PowerEdge R760 over the PowerEdge R740 would provide a NIC with ten times the raw capacity that draws less than 80 percent more power.[32]
Data center cooling
Cooling is critical to a data center’s operation because poorly managed heat can lead to “downtime, damage to critical components, and a shorter lifespan for equipment.”[33] A data center’s cooling generally falls into two categories: direct cooling of the servers and cooling the data center floor. Inside the servers, cooling technologies focus on moving the heat that the server generates away from the devices. On the data center floor, cooling technologies focus on maintaining a comfortable—if warm—ambient room temperature.[34]
The most common cooling system data centers use is a computer room air conditioner (CRAC). A CRAC works the same way as a residential air conditioner, but is inside of a server room. The CRAC method is energy inefficient but one of the most affordable methods of data center cooling.[35] A more energy-efficient method of cooling a data center is evaporative cooling. However, this requires a large amount of water, so it is not always the most environmentally friendly option depending on the watershed in the location of the data center.
Dell incorporated several cooling technologies into 16th Generation PowerEdge servers to help organizations meet the demands of reducing energy costs and environmental impact. These include high-performing fans and redesigned heatsinks to improve airflow and thermal management.[36] According to Dell, high airflow “is achieved using new impedance airflow paths, new storage backplane ingredients, and optimized mechanical structures all tuned to provide up to a 15% higher airflow compared to traditional designs.”[37] Software solutions such as OME Power Manager will automatically adjust power usage based on policies that admins can customize. To learn more about the power management benefits of OME, visit https://www.principledtechnologies.com/Dell/OpenManage-Power-Manager-1122.pdf.
Conclusion
Organizations don’t have to choose between saving money, doing more work, and using less energy; the new 16th Generation Dell PowerEdge R760 servers with Broadcom NICs do all three. In our hands-on testing, the new 16th Generation Dell PowerEdge R760 server featuring a Broadcom NIC offered 166.1 percent more Ops/s per US dollar, 129.5 percent more Ops/s, and 24.2 percent better performance per watt than the 14th Generation Dell PowerEdge R740 server. This means the PowerEdge R760 with a Broadcom NIC did over twice as much work at a lower cost with nearly 25 percent better power efficiency than the PowerEdge R740. When compared to the 15th Generation Dell PowerEdge R750, the PowerEdge R760 processed 29.8 percent more Ops/s at a lower cost in terms of Ops/s per US dollar and offered 3.8 percent better performance per watt.
In addition to lowering cooling costs and carbon emissions, choosing the latest-generation servers can help businesses lower other operating costs, such as maintenance and security compliance. Because the latest-generation Dell PowerEdge R760 servers offer more compute capacity, fewer servers can do more work, potentially reducing licensing costs and the physical footprint of a data center. Newer servers often feature the most recent manageability and security features, which can make it easier for businesses to manage their servers and protect their data. By choosing servers with better performance, businesses can reduce the response time of their systems and provide a better user experience.
Increasingly, businesses are under pressure to reduce the energy consumption and cost of their data centers while maintaining or increasing performance. Key web technologies such as Redis can offer businesses a competitive edge, but they also require a large amount of data processing. Businesses can meet these challenges with Dell PowerEdge R760 servers with Broadcom NICs that offer better cost-efficiency, performance, and power-efficiency.
This project was commissioned by Dell Technologies.
September 2023
Principled Technologies is a registered trademark of Principled Technologies, Inc.
All other product names are the trademarks of their respective owners.
[1] Redis, “Low-latency message queue & broker software,” accessed June 20, 2023, https://redis.com/solutions/use-cases/messaging/.
[2] Redis, “What is Redis?,” accessed June 20, 2023, https://aws.amazon.com/redis/.
[3] Prices obtained from Dell Marketing, US only. We used list pricing because it was the only pricing readily available for all three servers and because discounts vary.
[4] Redis, “Low-latency message queue & broker software.”
[5] Redis, “memtier_benchmark: A High-Throughput Benchmarking Tool for Redis and Memcached,” accessed June 15, 2023, https://redis.com/blog/memtier_benchmark-a-high-throughput-benchmarking-tool-for-redis-memcached/.
[6] Dell, “PowerEdge R760 Rack Server,” accessed June 7, 2023, https://www.dell.com/en-us/shop/productdetailstxn/poweredge-r760.
[7] Dell, “PowerEdge R760 Rack Server.”
[8] Dell.
[9] Broadcom, “BCM57508 – 200GbE,” accessed June 5, 2023, https://www.broadcom.com/products/ethernet-connectivity/network-adapters/bcm57508-200g-ic.
[10] Broadcom, “BCM57508 – 200GbE.”
[11] Broadcom.
[12] Timothy Prickett Morgan, “The Interesting Years Ahead for Servers,” accessed June 11, 2023, https://www.nextplatform.com/2023/01/04/the-interesting-years-ahead-for-servers/.
[13] Timothy Prickett Morgan, “The Interesting Years Ahead for Servers.”
[14] OliverWyman, “Machine Learning: Competitive Advantage and Strategic Imperative,” accessed June 11, 2023, https://www.oliverwyman.com/our-expertise/insights/2016/nov/Machine-Learning-Competitive-Advantage-and-Strategic-Imperative.html.
[15] Mary Zhang, “United States Data Centers: Top 10 Locations,” accessed July 27, 2023, https://dgtlinfra.com/united-states-data-centers/.
[16] S&P Global Market Intelligence, “Sustainability is no longer a ‘nice to have’ goal for the data center industry,” accessed June 14, 2023, https://www.spglobal.com/marketintelligence/en/news-insights/research/sustainability-is-no-longer-a-nice-to-have-goal-for-the-data-center-industry.
[17] S&P Global Market Intelligence, “Sustainability is no longer a ‘nice to have’ goal for the data center industry.”
[18] Lori Aniti, “Wholesale U.S. electricity prices were volatile in 2022,” accessed June 14, 2023, https://www.eia.gov/todayinenergy/detail.php?id=55139.
[19] InfoQ, “The Problem of Power Consumption in Servers,” accessed June 14, 2023, https://www.infoq.com/articles/power-consumption-servers/.
[20] Marcus Law, “Energy efficiency predictions for data centres in 2023,” accessed June 14, 2023, https://datacentremagazine.com/articles/efficiency-to-loom-large-for-data-centre-industry-in-2023.
[21] Michael Copley, “Data centers, backbone of the digital economy, face water scarcity and climate risk,” accessed June 14, 2023, https://www.npr.org/2022/08/30/1119938708/data-centers-backbone-of-the-digital-economy-face-water-scarcity-and-climate-ris.
[22] IDC, “Optimizing Performance with Frequent Server Replacements for Enterprise,” accessed June 12, 2023, https://www.delltechnologies.com/asset/en-us/products/servers/industry-market/server-infrastructure-resiliency-enterprise-whitepaper.pdf.
[23] Deloitte, “Milliseconds make Millions,” accessed June 12, 2023, https://www2.deloitte.com/content/dam/Deloitte/ie/Documents/Consulting/Milliseconds_Make_Millions_report.pdf.
[24] GVR, “Machine Learning Industry Data Book,” accessed June 14, 2023, https://www.grandviewresearch.com/sector-report/machine-learning-industry-data-book.
[25] Adil Mohommad, Mehdi Raissi, Kyuho Lee, Chapheng Fizzarotti, “IMF Blog: Volatile Commodity Prices Reduce Growth and Amplify Swings in Inflation,” accessed May 17, 2023, https://www.imf.org/en/Blogs/Articles/2023/03/28/volatile-commodity-prices-reduce-growth-and-amplify-swings-in-inflation.
[26] Mehdi Benatiya Andaloussi, Chiara Maggi, “IMF Blog: World Economic Outlook Shows Economies Facing High Uncertainty,” accessed June 21, 2023, https://www.imf.org/en/Blogs/Articles/2023/04/19/world-economic-outlook-shows-economies-facing-high-uncertainty.
[27] Robert Curtis et al, The Future of Server Cooling,” accessed June 8, 2023, https://infohub.delltechnologies.com/p/the-future-of-server-cooling-part-2-new-it-hardware-features-and-power-trends-1/.
[28] Kazi Main Uddin Ahmed, Manuel Alvarez, Math Bollen, “A Review of Data Centers Energy Consumption And Reliability Modeling,” accessed June 14, 2023, https://www.researchgate.net/publication/355862079_A_Review_of_Data_Centers_Energy_Consumption_And_Reliability_Modeling#pf8.
[29] Kazi Main Uddin Ahmed, Manuel Alvarez, Math Bollen, “A Review of Data Centers Energy Consumption And Reliability Modeling.”
[30] Patrick Kennedy, “Deep Dive into Lowering Server Power Consumption,” accessed June 8, 2023, https://www.servethehome.com/deep-dive-into-lowering-server-power-consumption-intel-inspur-hpe-dell-emc/.
[31] Broadcom, “Broadcom Eternet NICs,” accessed July 25, 2023, https://docs.broadcom.com/doc/PCIe-NIC-Ethernet-Adapters-Specification-Sheet.
[32] Allied Telesis, “Broadcom NetXtreme II® Network Adapter User Guide,” accessed July 25, 2023, https://www.alliedtelesis.com/sites/default/files/documents/installation-guides/brcm_nx2_server_t7-10.pdf.
[33] Datacenters.com, “Data Center Cooling: Future of Cooling Systems, Methods and Technologies,” accessed June 6, 2023, https://www.datacenters.com/news/data-center-cooling-future-of-cooling-systems-methods-and-technologies.
[34] Brien Posey, “Data center temperature and humidity guidelines,” accessed June 20, 2023, https://www.techtarget.com/searchdatacenter/tip/Data-center-temperature-and-humidity-guidelines.
[35] LDP Associates, “What are the Different Types of Data Center Cooling?,” accessed June 21, 2023, https://www.ldpassociates.com/what-are-the-different-types-of-data-center-cooling/.
[36] Matt Ogle and Hasnain Shabbir, “Multi Vector Cooling 2.0 for Next-Generation PowerEdge Servers,” accessed September 8, 2023, https://infohub.delltechnologies.com/p/multi-vector-cooling-2-0-for-next-generation-poweredge-servers/.
[37] Dell, “The Future of Server Cooling - Part 2: New IT hardware Features and Power Trends,” accessed July 25, 2023, https://infohub.delltechnologies.com/p/the-future-of-server-cooling-part-2-new-it-hardware-features-and-power-trends-1/.
Author: Principled Technologies