Introduction

The complexity of today’s infrastructure along with recent government regulations is driving major changes in infrastructure deployment. One such change is the transition from Internet Protocol version 4 (IPv4) to Internet Protocol version 6 (IPv6).    

With the rapid growth of the Internet and the increasing number of connected devices, IPv4 addresses are becoming scarce. This scarcity is referred to as address exhaustion. As a result, service providers have started charging a premium price for continued use of IPv4 and in some cases leasing the network addresses. This practice is encouraging the transition to IPv6.

 Address exhaustion particularly affects vertical industries such as telecommunications where the need for network addresses continues to grow. At the close of 2021, mobile service subscriptions reached 5.3 billion individuals, equivalent to 67 percent of the world’s population. From now until 2025, there will be more than 400 million new mobile subscribers[1].

While IPv4 allows for about 4.3 billion unique IP addresses, IPv6 expands this number to an almost limitless and astonishing number of possible addresses using 128-bit addresses (2128), allowing 340 undecillion, or approximately 3.4 x 1038, unique IP addresses. To illustrate the size of this number, if every square meter of the earth’s surface was assigned an IPv6 address, there would be enough addresses to cover the entire surface of the earth more than seven billion times. Therefore, we do not anticipate running out of IPv6 addresses anytime soon.   

Many organizations, including communication solution providers, are upgrading their network infrastructure to support IPv6.

Security and performance benefits of IPv6   

In addition to providing more network addresses, IPv6 provides many other benefits over IPv4. IPv6 provides customers with better end-to-end connectivity, simplified network management, and improved security:  

• Improved network performance—IPv6 provides numerous benefits that can improve network performance. For example, the reduced need for fragmentation of packets helps reduce latency and improve network performance. Additionally, IPv6 supports larger packets that help reduce overhead and improve network throughput.
• Simplified network management—IPv6 simplifies network management through multiple features, including:
  • Route aggregation—IPv6 can be deployed using a hierarchical address allocation method. This method facilitates route aggregation across the Internet, which limits the growth of routing tables.
  • Autoconfiguration—IPv6 devices can independently autoconfigure themselves when connected to other IPv6 devices. This action simplifies network configuration. IPv6 includes multiple autoconfiguration options, including support for stateless address autoconfiguration (SLAAC) and Dynamic Host Configuration Protocol (DHCP) v6, which can help simplify managing an address. In addition, it can add security by preventing attacks such as DHCP spoofing.
• Enhanced security—IPv6 provides enhanced security features that are not available in IPv4. For example, IPv6 has integrated support for Internet Protocol Security (IPsec), and when enabled it provides end-to-end encryption and authentication.

Government mandates accelerate the adoption of IPv6

Some governments and regulatory bodies have mandated the use of IPv6 in various sectors, such as telecommunications, government networks, and critical infrastructure.  

In 2020, the US government issued OMB M-21-07 directing all federal agencies to enable IPv6-only networks and services starting in 2023, with the goal of 80 percent completion by 2025. The directive also acknowledges that IPv6 offers significant benefits such as improved network performance, enhanced security, and future-proofing. The latest National Cybersecurity Strategy Paper from March 2023 specifically states that steps must be taken to mitigate the slow adoption of IPv6.

The United States government has strongly advocated for IPv6 adoption and uses the USGv6 program for strategic planning and acquisition policies. The program requires OEMs and product vendors to test their products according to the USGv6-r1 specifications at accredited test labs.

USGv6 validated RFC 2460 at Layer 3, which had a denial-of-service vulnerability.  USGv6r1 provides many improvements over USGv6. These improvements include addressing the denial-of-service vulnerability by validating RFC8200/8201, and IPv6-only support within the application.  By testing on Dell hardware, Dell Technologies also validates Layer 2 NIC compliance for devices that provide IP off-loading functionality.   USGv6-r1 went into effect as of November 2022.

The drive to adopt IPv6 is not just restricted to North America; task force-like groups are emerging worldwide. To help with the global adoption, the IPv6 Forum, a worldwide consortium focused on providing technical guidance for the deployment of IPv6, launched a single worldwide IPv6 Ready Logo Program. This conformance and interoperability testing program is intended to increase user confidence by demonstrating that IPv6 is now available and ready to use. India and Malaysia also have IPv6 certification programs for telecommunication equipment compliance. The specifics of these programs, including their focus, certification authority, requirements, and target audience vary depending on the guidelines and objectives set forth by the respective governments.

Table 1. Worldwide IPv6 certification programs

Dell’s industry-first certification

To uphold these standards and help organizations achieve their adoption goals, Dell PowerEdge servers now offer IPv6-only support. This support enables federal agencies and critical infrastructures to comply with the government’s directive and take advantage of the many benefits of IPv6.

Dell Technologies is proud to be the first company to provide USGv6r1 capabilities with our PowerEdge servers and Unity-XT storage products. These capabilities are a significant milestone for Dell Technologies and the industry. We are excited to see the positive impact on our customers’ networks.

Dell Technologies provides key features with both our PowerEdge servers and our Unity-XT storage products, offering a fully capable solution to Dell customers from the operating system, base management controller (BMC), and storage.   

• The Dell PowerEdge server is the first server in the industry to be USGv6r1- and IPv6 Ready Logo 5.1.2-compliant while running the
  •  Red Hat Enterprise Linux 8.4 and greater operating system
  • Applicable versions of the Windows 2019 operating system
  • Applicable versions of the Windows 2022 operating systems
• Dell PowerEdge iDRAC9 with 5.10.00.00 firmware is the first BMC to be “IPv6-only” compliant and validated on the USGv6R1 register, and Ready Logo 5.1.2 compliant.   
• Unity-XT is the first storage product to meet the USGv6r1 profile capability requirement IPv6-Only Functional v1.1.

Conclusion

Although IPv6 has been available for more than two decades, it is still a relatively new technology. Some customers might not be ready to transition. However, our responsibility as a technology leader is to push the industry forward and to offer our customers the latest and most advanced technologies. In addition to the benefits of IPv6-only support, Dell PowerEdge servers offer exceptional performance, reliability, and security features. With PowerEdge servers, Dell customers can be confident that they are getting the best of both worlds: the latest and most advanced technology combined with the exceptional quality and performance for which Dell Technologies is known. 

Summary

To combat and reduce the threat surface in the pre-boot environment, a broad transitioning is happening industry-wide from legacy BIOS boot to Unified Extensible Firmware Interface (UEFI) boot. UEFI changes the interface and data structures to interact with I/O device firmware and operating systems. The primary intent of UEFI is to eliminate shortcomings in the legacy BIOS boot environment, enabling system firmware to continue scaling with industry trends. System administrators are using UEFI boot throughout their data centers for cyber resilience and secure end-to-end boot.

Threat surface in the pre-boot environment

As security breaches are becoming more frequent, system administrators must employ a wider variety of defenses. Cyber threats not only affect traditional areas of security focus, such as network, operating system, and applications; they affect firmware as well. Attackers find this pre-boot environment lucrative and use firmware rootkits to hide malicious code, called malware, in device or system firmware.  

Firmware is software that is embedded in a piece of hardware. While server components are often viewed strictly as hardware, many components have firmware, such as network interface cards, storage controllers, graphics cards, and more. The firmware acts as the device’s operating system, providing control, monitoring, and data manipulation functions. Firmware runs before the operating system environments come into existence.    

Malware can control system firmware and then gain full access to the system. Pre-boot malware avoids operating system privilege levels, escapes detection by operating system anti-malware tools, and even survives reinstallation of the operating system. If an attacker injects malware into the pre-boot environment, administrators might find it difficult to remove, if they detect it at all.

Industry transition to UEFI boot

To combat and reduce the threat surface in the pre-boot environment, many vendors and customers are embracing UEFI boot and have stopped certifying operating systems and applications with legacy BIOS boot mode.

• Microsoft requires UEFI boot on Windows Server 2022 and beyond.
• VMware only certifies UEFI boot in their latest ESXi offerings.
• Intel has stopped supporting legacy BIOS boot with certain platforms such as the Intel® Xeon® E-2300 series processors.
• Many new technologies, such as PCIe Gen 5 and NVMe, have eliminated legacy BIOS boot, and the modern data center is transitioning with them.  

UEFI benefits

UEFI was designed to overcome many limitations of legacy BIOS boot.

UEFI supports drive sizes up to 9 zettabytes, whereas BIOS supports only 2.2 terabytes.

UEFI can provide faster boot time by allowing parallel execution for sections of the boot flow.

 UEFI has discrete driver support, while BIOS has drivers stored in its ROM and lacks modularity, so updating BIOS firmware can be more difficult.

UEFI offers improved security, including Secure Boot. Secure Boot prevents the computer from running unauthorized and unverified code during boot, which helps prevent rootkit and bootkit attacks.

UEFI is easier to deploy and manage. The UI supports mouse-based navigation due to UEFI’s ability to run in 32-bit and 64-bit modes. BIOS runs in 16-bit mode and only allows keyboard-based navigation.

UEFI operations

UEFI does not change the traditional purposes of the system BIOS. To a large extent, UEFI performs the same initialization, boot, configuration, and management tasks as a traditional BIOS. However, UEFI does change the interfaces and data structures that the BIOS uses to interact with I/O device firmware and operating systems. The interface consists of data tables that contain platform-related information, plus boot and runtime service calls that are available to the operating system and its loader. Together, these provide a standard, modern environment for booting an operating system and running pre-boot applications. The primary intent of UEFI is to eliminate shortcomings in the traditional BIOS environment, enabling system firmware to continue scaling with industry trends.

Figure 1.  Levels of UEFI boot

UEFI Secure Boot

UEFI Secure Boot is a technology that eliminates a major security void that might occur during a handoff between the UEFI firmware and the operating system.   

Users configure a Secure Boot policy consisting of X.509 certificates and hash values for both authorized and unauthorized entities. The system firmware enforces this policy when determining whether to run pre-boot software including I/O device firmware and operating system loaders. When enabled, UEFI Secure Boot prevents unsigned or compromised UEFI device drivers from being loaded, displays an error message, and does not allow the device to function. You must disable Secure Boot to load the unsigned device drivers.

Since mid-2017, you can enable or disable the Secure Boot feature on Dell PowerEdge servers through various interfaces. The Secure Boot Management on Dell PowerEdge Servers white paper provides more details about Secure Boot and how to configure it on PowerEdge servers.

Figure 2.  UEFI Secure Boot working principle

Dell Secure Boot customization

For customers wanting to avoid standard keys because of some of the risks they present, Dell Technologies provides complete customization capabilities for UEFI Secure Boot. This gives system owners an option to eliminate reliance on industry keys and third-party certificate authorities. In Figure 1, the yellow boxes have one CA certificate that authorizes multiple versions of firmware, whereas the green boxes signify a customized certificate for a specific version of firmware. Dell PowerEdge offers tools to enable capture of firmware hashes and create customized certificates so system administrators can optimize the effectiveness of their UEFI Secure Boot policies according to their security requirements. 

In a recent Cybersecurity Technical Report, the National Security Agency highlighted the need for enabling UEFI Secure Boot and the benefits of using customization to realize the highest level of server security available today. The report showcases the fully customized Secure Boot capabilities of PowerEdge servers as the example of how to achieve this highest level of boot security.

Impact to environments with servers configured for legacy BIOS boot

For customers whose environments include servers configured for legacy BIOS boot, the primary impact of switching to UEFI boot mode is likely to their deployment and maintenance model. Customers who use PXE servers to deploy and perform routine maintenance need to plan for adaption of the PXE server. For more information about this process, see Boot Mode Considerations: BIOS vs. UEFI.

Another impact is the use of NVMe drives that natively boot UEFI. However, they can be used as data drives in both legacy BIOS and UEFI boot modes.

Conclusion

Cyber attacks are becoming more numerous, frequent, and difficult to detect. Strengthening your organization’s security posture by implementing the latest security approaches positions your organization to respond to today’s cyber threat environment. The security features and boot mechanisms available only when a system is configured for UEFI boot mode is driving the industry-wide transition to UEFI boot.  

To achieve additional boot security, UEFI-enabled systems include Secure Boot as a setup option, providing additional security checks during the boot process. 

References   

Dell documents

• Secure Boot Management on Dell PowerEdge Servers
• Boot Mode Considerations: BIOS vs. UEFI
• Dell PowerEdge UEFI Boot: Enhanced Security to Combat Persistent Firmware Threats

NSA documents

• UEFI Defensive Practices Guidance
• Boot Security Modes and Recommendations
• UEFI Lockdown Quick Guidance
• UEFI Secure Boot Customization

UEFI Forum documents

• UEFI Secure Boot in Modern Computer Security Solutions

Integrated Dell Remote Access Controller (iDRAC) is a baseboard management controller (BMC) built into Dell PowerEdge servers. iDRAC allows IT administrators to monitor, manage, update, troubleshoot, and remediate Dell servers from any location without the use of agents and out-of-band. It consists of both hardware and software that provides extensive features compared to a basic baseboard management controller.  

Key features of iDRAC

iDRAC is designed to make you more productive as a system administrator and improve the overall availability of Dell servers. iDRAC alerts you to system issues, helps you to perform remote management, and reduces the need for physical access to the system.

Ease of use

• Remote management: Server management can be performed remotely, reducing the need for an administrator to physically visit the server. By providing secure access to remote servers, administrators can perform critical management functions while maintaining server and network security. This remote capability is essential to keeping distributed and scaled-out IT environments running smoothly. Using the GUI, an administrator can perform firmware maintenance and configuration of BIOS, iDRAC, RAID, and NICs; deploy operating systems; and install drivers.
• Agent-free monitoring: iDRAC is not dependent on the host operating system and does not spend CPU cycles on agent execution, intensive inventory collection, and so on.
• Thermal management: iDRAC’s Thermal Manage feature provides key thermal telemetry and associated controls that allow customers to monitor the thermal radiation dynamics and run their environment efficiently.
• Virtual power cycle: With servers increasingly being managed remotely, a means of performing the virtual equivalent of pulling out the power cord and pushing it back in is a necessary capability to occasionally ”unstick” the operating system. With the PowerEdge iDRAC9 virtual power cycle feature, IT admins have access to console or agent-based routines to restore or reset power states in minutes rather than hours.  

Security features

iDRAC offers security features that adhere to and are certified against well-known NIST, Common Criteria, and FIPS-140-2 standards.

• Automatic certificate renewal and enrollment: This feature makes it easy for users to secure network connections using TLS/SSL certificates. The iDRAC web server has a self-signed TLS/SSL certificate by default. The self-signed certificate can be replaced with a custom certificate, a custom signing certificate, or a certificate signed by a well-known certificate authority (CA). Automated certificate upload can be accomplished by using Redfish scripts. iDRAC9 automatic certificate enrollment and renewal automatically ensures that SSL/TLS certificates are in place and up to date for both bare-metal and previously installed systems. Automatic   certificate enrollment and renewal requires the iDRAC9 Datacenter license.
• Secure supply chain: The iDRAC boot process uses its own independent silicon-based Root of Trust that verifies the iDRAC firmware image. The iDRAC Root of Trust also provides a critical trust anchor for authenticating the signatures of Dell firmware update packages (DUPs).
• Authentication: iDRAC offers a simple two-factor authentication option to enhance login security for local users. RSA SecurID can be used as another means of authenticating a user on a system.

Scalable data analytics with telemetry streaming

Using analytics tools, IT managers can more proactively manage systems by analyzing trends and discovering relationships between seemingly unrelated events and operations. iDRAC9 telemetry streaming with over 180 metrics/sensors can provide data on server status with no performance impact on the main server. Telemetry streaming’s big performance advantage is in reducing the overhead needed to get the complete data stream from a remote device. Advantages of iDRAC telemetry streaming include:

• Better scalability: Polling requires a lot of scripting work and CPU cycles to aggregate data and suffers from scaling issues when we are talking about hundreds or thousands of servers. Streaming data, in contrast, can be pushed directly into popular analytics tools such as Prometheus, ELK stack, InfluxDB, and Splunk without the overhead and network loading associated polling.
• More accuracy: Polling can also lead to data loss or “gaps” in sampling for time series analysis; it is usually only a snapshot of current states, not the complete picture over time. You might miss critical peaks or excursions in data.
• Less delay: Data can be severely delayed in time due to needing multiple commands to get a complete set of data and the inability to poll simultaneously from a central management host. Streaming more accurately preserves the time-series context of data samples.

Resources

You can explore the following resources to learn more about iDRAC. Also, you can see for yourself the capabilities of PowerEdge iDRAC in our virtual lab setting.

Tech notes

• Telemetry streaming
• Thermal management
• Improved iDRAC9 Security using TLS 1.3 over HTTPS
• iDRAC9 Virtual Power Cycle
• iDRAC9 System Lockdown: Preventing Unintended Server Changes
• Automatic SSL/TLS certificate enrollment

Benchmark studies by industry analysts

• Deployment with zero touch provisioning
• Automated renewal of SSL certificates
• Telemetry streaming
• Splunk integration with iDRAC telemetry

Videos

• Large scale iDRAC telemetry and integration with Splunk
• Advanced Thermal Management with iDRAC9
• Automatic Certificate Enrollment with iDRAC9
• System lockdown to prevent unwanted drift in server configurations

Other resources:

• Explainer video covering what’s new in the GUI of iDRAC9 v4.0
• Deep dive demo of Server Configuration Profile feature of iDRAC9
• Deep-dive webinar on Telemetry Streaming feature for large-scale server management

Summary

Between customers’ sustainability initiatives to reduce carbon emissions, and demands to control energy consumption and costs, the ability to report, analyze and action server power usage data has become a key initiative. This DfD tech note explores the rich server power usage data available from Dell PowerEdge servers and the various methods to collect, report, analyze, and act upon it. 

What is server power consumption?

A wide variety of server power information is offered by the iDRAC. The amount and frequency of information varies by iDRAC version and licensed features and the choice of optional tools and consoles.

One-to-one and one-to-many

There are multiple ways to view power consumption data from the iDRAC, depending on needs and preferences. One way is to open the web interface GUI. Another way is using scripts, either Racadm or Redfish, to retrieve the data. iDRAC can also send data to the OpenManage Enterprise Power Manager Plugin. OpenManage Enterprise can also forward this information to CloudIQ for PowerEdge. For those customers looking for the ultimate solution, iDRAC9 can stream these power statistics as telemetry data to analytics solutions such as Splunk or ELK Stack for real-time in-depth analysis.

Figure 1. PowerEdge management stack, with power management and data reporting highlighted

PowerEdge server power data

Embedded with every Dell PowerEdge server, the integrated Dell Remote Access Controller (iDRAC) enables secure and remote server access for out-of-band and agent-free server management tasks. Features include BIOS configuration, OS deployment, firmware updates, health monitoring, and maintenance. One key set of data that iDRAC provides is power usage. IT admins have used iDRAC data to view and react to power issues for over 10 years. The iDRAC engineering teams have continued to expand the capabilities within the iDRAC UI as well as the information available to “one to many” consoles such as OpenManage Enterprise. iDRAC9 with Datacenter feature set enabled extends the solution even further with telemetry streaming.

iDRAC

iDRAC monitors the power consumption, processes, and reports continuously at the individual server level. The browser user interface displays the following power values:

• Power consumption warning and critical thresholds
• Cumulative power, peak power, and peak amperage values
• Power consumption over the last hour, last day, or last week
• Average, minimum, and maximum power consumption with historical peak values and peak timestamps
• Peak headroom and instantaneous headroom values (for rack and tower servers)

iDRAC9 provides a graphical view of these power metrics such as the power consumption example shown here.

Figure 2. iDRAC9 GUI power consumption data

iDRAC9 connects to all critical server components and, in conjunction with the Datacenter license, can collect over 180 server metrics in near-real-time. These metrics include granular, time-stamped data for critical functions such as processor and memory utilization, network card, power, thermal, and more. iDRAC9 can stream this telemetry data in real time.

Figure 3.  iDRAC power telemetry data collected by Splunk 

Get Server Power – RACADM CLI Examples

The RACADM command-line provides a basic scriptable interface that enables you to retrieve server power either locally or remotely. In addition to the CLI interface, iDRAC also supports the Redfish RESTful API. Example Powershell and Python scripts that can be used to collect power data can be download from the Dell area in github.com. The RACADM CLI can be access from the following interfaces:

• Local - Supports running RACADM commands from the managed server's operating system (Linux/Windows). To run local RACADM commands, install the OpenManage DRAC Tools software on the managed server.
• SSH or Telnet (also known as Firmware RACADM) - Firmware RACADM is accessible by logging into iDRAC using SSH or Telnet.
• Remote - Supports running RACADM commands from a remote management station such as a laptop or desktop running Windows or Linux. To run remote RACADM commands, install the OpenManage DRAC Tools software on the management station.

Here are some examples using the remote iDRAC9 SSH CLI method, post authentication.

• Instantaneous server power usage:

• Server power stats:

OpenManage Enterprise Power Manager

The Power Manager Plugin for OpenManage Enterprise uses the power data securely collected from iDRACs to observe, alert, report, and, if required, place power caps on servers. For ease of management, servers can be logically grouped together, such as in a rack, a row, or in custom grouping, such as a workload. Using this data, customers can drive data center efficiency in several ways, such as by easily identifying idle servers for repurposing or retirement. Using built in reports or creating a custom report, customers can identify server racks not using their full available power capacity to deploy new hardware without needing additional power. Customers can mitigate risk by detecting when groups of servers are nearing their power capacity during specific timeframes. Using automated policies, customers can maximize power available to business-critical applications by reducing noncritical consumption by using scheduled or permanent power capping.

Important in today’s climate concerns are reports on carbon emissions based on server usage. Power Manager provides reports on the carbon emissions for individual servers as well as racks and custom groups of servers. This information can be used to identify areas of concern and to show progress in carbon emission reductions based on power policies, removal of idle servers, and other initiatives such as consolidation and refresh.

The power data is displayed by applets integrated into OpenManage Enterprise. (See examples in the following figure.) There are also several predefined reports built into the report library designed around power usage. Power Manager automates actions driven by specific power or thermal events, including running scripts, applying power caps, and forwarding alerts. Power Manager collects this power data and stores it for up to 365 days.

Figure 4. View of a rack group alert threshold graphic for power and thermal

Figure 5. Rack view showing max/min/avg power for the last six hours

CloudIQ for PowerEdge – Reporting Server Power

Another method to visualize and report the power data is by CloudIQ. Utilizing the OpenManage Enterprise CloudIQ Plugin, customers can connect their PowerEdge servers to the Dell hosted CloudIQ secure portal. This is a cloud based software-as-a-service portal, hosted in the Dell data centers, that provides powerful analytic, health, and performance monitoring for servers. CloudIQ can consolidate multiple OpenManage Enterprise instances, providing a truly global view of an organization’s server estate. Within CloudIQ, power data can be graphed and reported on over time. These graphs can easily be exported or emailed as PDFs and the raw data exported as CSV for further reviews. In fact, in addition to collecting power metrics, CloudIQ can track and collect over 50 server metrics for users to review. CloudIQ also interfaces with other elements of Dell’s infrastructure, including storage and networking, giving customers the ability to correlate data, events, and trends across multiple technologies. CloudIQ is offered at no additional cost for all PowerEdge servers with ProSupport or higher contracts.

When power data is collected in CloudIQ, advanced AI algorithms process this data and automatically flag whether the server power usage behavior is outside normal parameters, based on historic data from that particular server.

Fiure 6. individual server power data with historical seasonality – no anomaly

Multiple servers can be put onto the same graph, making it easy to identify any rogue behavior by individual servers.

Figure 7. Multi server power usage report

The visualization of this data can be displayed from just hours to a whole year, with the ability to zoom in on a particular time.

Summary

Dell EMC PowerEdge servers stand out for offering the ability to remotely invoke an A/C power cycle to the Baseboard Management Controller. With distributed and scaled-out IT environments, the means of restoring or resetting power states in as little time as possible takes on added importance.

Introduction

On those occasions when it’s necessary for an IT admin to reboot a server, whether due to a faulty hardware component or an operating system ‘stuck’ in an unresponsive state, it may be necessary to drain all power to the server. This step is rare but could be the essential means to drain auxiliary power from capacitors to recover a device in a hung state and reboot the physical device’s firmware stack.

 Since it is increasingly unlikely that a server room is located ‘down the hall’, and more likely across town within a ‘lights out’ co-location datacenter, the means of restoring or resetting power states in as little time as possible takes on added importance.

iDRAC9 enables remote power cycles

With the integrated Dell Remote Access Controller (iDRAC), standard on all Dell EMC PowerEdge servers, IT administrators can mimic a power cycle and restore the system without having to go to the datacenter, find the server in the hot aisle, and pull the plug. The following solutions will work for either AC or DC power supplies.

Invoking Virtual A/C Power Cycle

Dell EMC PowerEdge servers with iDRAC9 offer 2 options for invoking a virtual A/C (vAC) power cycle:

• Use of iDRAC9 out-of-band capabilities
• Use of an iDRAC Service Module (iSM) installed on Windows, Linux, or ESXi

Both options eliminate the need for physical presence, to locate the correct server in a hot aisle, and pull out the power cord before plugging it back in.

The path chosen is likely predicated on situation particulars:

• Using iDRAC, assuming no operating system dependencies:
  • Set “Full Power Cycle” using GUI/Redfish/RACADM
  • ‘Power Cycle’ – perform a power cycle of the server via iDRAC
  • To note, the virtual A/C power cycle is always available and can be performed regardless of the host state; indeed, it may be required if the host operating system is not responding properly
  • Of further note, this process applies to rack/tower systems, whereas for modular systems, it’s best to use the “virtual reseat” of the server option.
• iSM – sending commands to an agent through the operating system or hypervisor:

• Two commands are issued, one to activate the vAC, and one to perform a graceful power-down of the host
• May be necessary whenever the iDRAC is in an unresponsive state
• Requires PowerEdge servers with iDRAC9

Invoking a remote virtual A/C power cycle

With iDRAC, via the:

• GUI – navigate to Configuration > BIOS Settings > Miscellaneous Settings > Power Cycle Request

RACADM

• racadm set BIOS.MiscSettings.PowerCycleRequest FullPowerCycle
• racadm jobqueue create BIOS.Setup.1-1
• reboot host when ready.

Redfish

• PATCH / redfish/v1/Systems/System.Embedded.1/Bios/Settings with

{

“Attributes”: {

“PowerCycleRequest”: “FullPowerCycle:

},

"@Redfish.SettingsApplyTime": {

"@odata.type": "#Settings.v1_1_0.PreferredApplyTime", "ApplyTime": "OnReset"

}

}

When the patch command has successfully completed, a 202 “Accepted” status message will be returned along with the Task URI for newly created job.

• POST /redfish/v1/Systems/System.Embedded.1/Actions/ComputerSystem.Reset

{

"ResetType":"On" /// If Powered Off

}

or

{

"ResetType":"GracefulRestart" /// If already Powered On

}

This will restart the Host and start the Task/Job, wait for it to complete.

• iSM
  • For Windows operating system – Shortcut menus are available for the FullPowerCycle Activate (request), FullPowerCycle Cancel and FullPowerCycle get status operations.
    • To request FullPowerCycle on your system, type Invoke-FullPowerCycle –status request cmdlets in power shell console
    • To get the status of the Full Power Cycle on your system, type Invoke-FullPowerCycle –status Get cmdlets in power shell console
    • To cancel the Full Power Cycle on your system, type Invoke-FullPowerCycle –status cancel cmdlets in power shell console

• For Linux operating system –
  • To request Full Power Cycle on your system, type /opt/dell/svradmin/iSM/bin/Invoke-FullPowerCycle request
  • To get the status of the Full Power Cycle on your system, type /opt/dell/svradmin/iSM/bin/Invoke-FullPowerCycle get-status
  • To cancel the Full Power Cycle on your system, type /opt/dell/svradmin/iSM/bin/Invoke-FullPowerCycle cancel

Note: After running the command, a host power cycle (cold boot) is necessary for FullPowerCycle to take effect.

Conclusion

 With servers increasingly managed remotely, a means of performing the virtual equivalent of pulling out the power cord and pushing it back in is a necessary capability in order to occasionally ‘unstick’ the operating system. With the Dell EMC PowerEdge iDRAC9 virtual power cycle feature, IT admins have access to console or agent-based routines to restore or reset power states in minutes rather than hours. This remote capability is essential to keeping distributed and scaled- out IT environments running smoothly.

Resources

iDRAC9 whitepapers and videos www.dell.com/support/idrac

iDRAC Manuals and User Guides www.dell.com/idracmanuals

iDRAC Service Module

www.dell.com/idracmanuals (select iDRAC Service Module)

Summary

iDRAC9 telemetry enhancements include the ability to create user- defined custom reports and balance volume of streamed telemetry across more than one collection point. iDRAC9 data streamed to an external ingress collector, from which tools like Splunk or ELK Stack can be used to aggregate data, examine trends, issue alerts, and generate timely reports.

Introduction

The iDRAC9 firmware v4.40.10 in conjunction with the Datacenter license, now includes feature enhancements to the telemetry streaming function. These include the ability to create user-defined custom reports and direct data streams to more than one collection point.

Embedded with every PowerEdge server, the integrated Dell Remote Access Controller 9 (iDRAC9) enables secure and remote server access, regardless of operating system state or presence of hypervisor, and makes possible a range of server management tasks, including configuration, OS deployment, firmware updates, health monitoring and maintenance.

The iDRAC9, while providing out-of-band and agent-free systems management, connects to all critical server component and collects over 180 server metrics in near-real-time. These metrics include granular, time-stamped data for critical functions such as processor and memory utilization, network card, power, thermal, memory, and graphics processing, and more; they enable consistency and scaling as infrastructure needs grow.

iDRAC9 data is streamed to an external ingress collector, from which tools like Splunk or ELK Stack can be used to aggregate data, examine trends, issue alerts, and generate timely reports. Data collected from iDRAC9 by server administrators can be used to make better data center performance decisions and prioritize proactive maintenance.

Customized Reporting

Building on prior capabilities, which included exposed time-series sensor data and JSON-enabled streaming telemetry data, version 4.40.10 of the iDRAC9 firmware has moved the DMTF Redfish schema-based reporting beyond default reports and values, to include the creation of user-defined custom reports. This flexibility helps to potentially cut down the size of data sets and reports, whether by changing the collection time interval, using additional aggregation functions within reports (beyond average/maximum/minimum), eliminating unwanted metrics, using 24 custom report definitions (in addition to 24 existing report definitions), or limiting report content to a subset of the maximum 2,400 values per report. 

Support for Multiple Consoles

New iDRAC9 features also include, in response to customer feedback, the ability to send iDRAC9-streamed telemetry from one or many Dell EMC PowerEdge servers, to more than one collection console, for use by one or many organizations charged with overseeing data center operations. A total of eight separate collection consoles can be specified, which allows for reducing the rate and volume of telemetry data flowing to any one particular collector, and avoiding any “thundering herd” effect when formerly thousands of iDRAC9 servers could potentially fire off data at a particular collector on a non-randomized schedule. This feature improvement also allows for variations in data sampling rates and reporting schedules, tied to custom reports that drive requirements for sampling interval, metrics collected, and configuration parameters set. Through better distribution of streamed telemetry at the collector level, the greater the number of iDRAC9 servers that can be supported.

All changes to all reports are normally global, regardless of whether a report is a legacy report or a custom report, as all collectors see the changes, regardless of which particular collector initiated the change. By using specific report definition names, however, a particular collector can lay claim to that particular report definition.

New reports are created using functions supported by HTTP, including PATCH, POST, PUT, and DELETE, whereby a web server accepts enclosed data or a request to make partial changes or deletions to an existing resource. ‘Pre-canned’ reports included with iDRAC9 can be changed using the PATCH function. They cannot be deleted, however, using DELETE, as this merely resets the report back to factory default values. Standard DMTF Redfish semantics apply to all of these operations, as does Report URI, used for monitoring security policies. Report definitions can be deployed using the Server Configuration Profile feature (SCP). SCP enables changes to configuration, firmware and redeployment of the operating system through a single XML or JSON template; The SCP template can then be applied to multiple servers.

Conclusion

As data centers grow in importance, servers proliferate, and differences between poorly-run and well-run facilities become readily apparent and thus consequential, iDRAC9, standard with all PowerEdge servers, provides an effective means of monitoring, analyzing, and acting upon data streamed from 180 or more monitored server performance indicators. The addition of feature enhancements to the latest iDRAC release make it now possible to create custom reports and balance the volume of streamed telemetry across more than one collection point.

These tools and more underscore how Dell EMC PowerEdge servers are compelling compute solutions. The inclusion of custom reports and support for multiple collectors, ease-of-monitoring, managing, updating, troubleshooting, and remediation of server performance, make for seamless and integrated server data collection, a key enabler of any well-run datacenter.

Summary

Enabling system lockdown mode is part of Dell Technologies’ cyber resilient architecture of Protect, Detect and Recover. System Lockdown helps prevent change or “drift” in system firmware images and critical server configuration settings. Dell Technologies is the only vendor to offer the ability to dynamically enable and disable system lockdown once your server is provisioned and in production without having to reboot.

Introduction

Running the latest firmware on datacenter servers helps keep up with security and performance improvements, maintain optimal operating parameters, and leverage new features. All are critical to the bottom line, to getting the most from your datacenter investment. When unplanned or unforeseen changes occur to server configurations, whether benign or malicious, these can propagate across a datacenter with a corresponding loss in productivity or extra cost. 

iDRAC9 System Lockdown Benefits

To prevent unintentional changes, the iDRAC9 Enterprise and Datacenter licenses now include a feature “System Lockdown,” a virtual lock for firmware and hardware configurations. Even those with full admin privileges are limited to read-only access—unless the lock is first disabled. This prevents server ‘drift’, the unintentional migration of firmware and configuration settings across servers.

The lock does, however, allow for continued access to key operations, such as power capping and power cycling, health monitoring and virtual console access, while keeping server workloads running. All hypervisor and OS functionality are also fully accessible.

When accessed via a web GUI, Redfish REST APIs, or RACADM command-line utility, systems administrators are prevented from making changes that could impact servers in production. Additionally, the lockdown status is evident via a padlock icon and greyed out settings in the iDRAC GUI.

Even before logging in, the admin is notified the system is in Lockdown mode.

iDRAC9 System Lockdown is Part of Dell’s Cyber Resilient Architecture

The lockdown mode is part of Dell’s PowerEdge cyber resilient architecture, with its emphasis on Protect, Detect and Recover. It protects by preventing firmware downgrades as a possible vector of attack, adding or removing users as a means of circumventing settings, or modifying lockout policies. System Lockdown enables detecting changes outside a maintenance window by creating alerts in the iDRAC lifecycle log that can be configured to send notifications, and it potentially cuts recovery time spent re-imaging or re-configuring servers.

System lockdown now offers native lockdown support in select NICs which prevents malware in the OS from installing firmware updates using altered versions of vendor tools. This also addresses concerns for cloud providers of end customers installing their own firmware versions on the server hardware they are using. As a result, subsequent users of a cloud server can be assured that the networking adaptor firmware is secure and version consistent.

System Lockdown Drives Datacenter Efficiencies

The system lockdown fits well with standard server maintenance window methodologies, the unlocking and locking of servers serving as ‘bookends’ at the start or end of maintenance work. Once operationalized, it helps drive good maintenance behavior, cuts unforced errors, and prevent server ‘drift’. 

In Conclusion

Enabled in iDRAC Enterprise and Datacenter licenses, the lockdown feature is another important tool available from Dell Technologies to manage and maximize your investment in your PowerEdge servers.

Summary

In the latest generation of Dell EMC PowerEdge Servers, iDRAC v4.0, has implemented a new automated security feature to keep your iDRAC SSL/TLS certificates current. The iDRAC’s Automatic Certificate feature automatically assures SSL/TLS certificates are in place and up-to-date for both bare-metal and previously installed systems.

Introduction

Dell EMC PowerEdge server’s Integrated Dell Remote Access Controller (iDRAC) v4.0 offers a new security feature, Secure Sockets Layer (SSL)/ Transport Layer Security (TLS) Automatic Certificate Enrollment that helps the Data Center Manager maintain security with less effort.

Data Center Managers need to be vigilant to make sure that their compute environment is protected from a range of threats and attacks. Monitoring and assuring that all security measures are current and in place is time- consuming and imperative to prevent unauthorized access and manipulation of your servers. 

iDRAC Web User Interface and SSL/TLS Certificates

The iDRAC enables remote system management and reduces the need for physical access to the system. The iDRAC Web User Interface can be reached with any supported browser and uses an SSL/TLS certificate to authenticate itself to web browsers and command-line utilities running on management stations thereby establishing an encrypted link.

If the Certificate Authority that issued the certificate is not trusted by the management station, warning messages will be displayed on the management station. Having an iDRAC SSL/TLS certificate in place ensures a validated and secure connection. 

Previously, creating and renewing iDRAC SSL/TLS certificates required a mostly manual, time-consuming effort. Monitoring approaching expiration dates and arranging for new certificates to be generated from a CA authority is just one aspect. IT admins then had to update scripts to deploy the certificates to embedded devices like the iDRAC.

iDRAC SCEP Client Support - Automatic Certificate Enrollment

iDRAC has added a client for Simple Certificate Enrollment Protocol (SCEP) support. SCEP is a protocol standard used for managing certificates to large numbers of network devices using an automatic enrollment process. The iDRAC can now integrate with SCEP-compatible servers like Microsoft Server’s NDES service to maintain SSL/TLS Certificates automatically. This feature can be used to enroll and refresh a soon-to-be-expired web server certificate.

 ACE- Automatic Certificate Enrollment

Automatic Certificate Enrollment will enroll and monitor the iDRAC web server SSL/TLS certificate. It enrolls to the specified Certificate Authority (CA) credentials provided. This can be done 1x1 in the iDRAC GUI, set via Server Configuration Profile, or scripted via tools such as Racadm.

iDRAC Integration with MS-NDES over SCEP

In Conclusion

Monitoring and assuring that all security measures are current and in place is both time-consuming and essential to prevent unauthorized access and manipulation of your servers. The Automatic Certificate Enrollment feature in iDRAC9 v4.0, is just another way Dell EMC is helping you to keep your data center secure.

Summary

Telemetry Streaming, a new feature in iDRAC9 v4.0 enabled by the new Datacenter License, can produce more high-value (comprehensive and accurate) data faster than with previous versions. There is a huge amount of untapped machine data in your IT infrastructure: use iDRAC9 Telemetry Streaming and analytics to leverage that data to optimize your server management and operations.

Introduction

With the advent of the new iDRAC9 v4.00.00.00 firmware release and the Datacenter license, IT managers can now integrate advanced telemetry about the server hardware operation into their existing analytics solutions. This telemetry is provided as granular, time-series data that can be streamed versus using inefficient, legacy polling methods. The advanced agent-free architecture in iDRAC9 provides over 180 data metrics (with more coming) related to server and peripherals operations that are precisely time-stamped and internally buffered to allow highly efficient data stream collection and processing with minimal network loading. This comprehensive telemetry can be fed to popular analytics tools to predict failure events, optimize server operation, and enhance cyber-resiliency.

Telemetry and Analytics

Telemetry has been around for decades and has been used in various business applications, from hospitals monitoring patients to oil and gas drilling systems to weather balloons transmitting meteorological data. The definition of Telemetry is an “automated communications process by which measurements are made, and other data collected at remote or inaccessible points are transmitted to receiving equipment for monitoring.”

Figure 1. Telemetry Monitoring in a Typical Data Center

In the era of “Big Data,” IT managers leverage a wide range of telemetry from their infrastructure in their monitoring tools, as shown in Figure 1. However, increasingly that telemetry is also used in AI-based analytics to gain operational insight into their datacenter operations. This is far more powerful than using simple alerting and monitoring techniques that typically only report health and status via SNMP alerts or IMPI traps.

Using analytics tools, IT managers can more proactively manage by analyzing trends and discovering insightful relationships between seemingly unrelated events and operations. A recent survey found that 61% of IT decision-makers considered data and analytics very important to their business growth strategy/digital transformation efforts.¹

Some of the use cases for data center analytics are:

• Predictive analytics: Customers can perform an in-depth analysis of server telemetry, including device parametric data to proactively replace failing devices. In one case, an IT team used analytics on telemetry from memory devices to develop an algorithm that predicted eventual failure. This allows proactive replacement of suspect devices during scheduled maintenance windows, significantly improving uptime and SLA quality.
• Optimized IT operations: You can perform time-series analysis of vital server metrics to gain insights into optimizing server operation, including tracking of power, temperature, CPU, and I/O performance, etc. One industry that makes extensive use of analytics is High-Frequency Trading, where every millisecond of compute counts in accelerating automated trades. Detailed telemetry is commonly used to discover ways to squeeze out more performance from servers, which becomes a key competitive advantage in this industry.
• Security: AI-based analytics can respond far faster to security events. You can enhance security AI and forensics by monitoring the say of unusual user login activity or physical intrusion events on your servers.

However, to perform effective analytics, you need data: lots and lots of it to feed Machine or Deep Learning techniques effectively. The larger the data set, the more accurate the analysis becomes as evidenced by the petabytes of data that social media uses in analytics of user attributes and buying behaviors.

The Streaming Advantage in iDRAC9

Telemetry streaming’s big performance advantage is in reducing the overhead needed to get the complete data stream from a remote device. Retrieving telemetry using polling can result in an enormous number of discrete commands being issued, which is very challenging in scaling across a large datacenter. With iDRAC9 Telemetry Streaming, you get time-series and detailed statistics reports delivered directly to a variety of analytics collection tools with higher efficiency by removing the need for issuing individual commands for each piece of data. The streaming configuration is flexible so users can modify the number of metrics they require, the report interval (30 seconds for example), and enable reports to be sent immediately upon detection of critical events in the server (like a PSU failure say).

In summary, the advantages of Streaming over Polling are:

• Better Scalability: Polling requires a lot of scripting work and CPU cycles to aggregate data and suffers from scaling issues when we are talking about 100’s or 1000’s of servers. Streaming data, in contrast, can be pushed directly into popular analytics tools like Prometheus, ELK stack, InfluxDB, Splunk without the overhead and network loading associated polling.
• More Accuracy: Polling can also lead to data loss or “gaps” in sampling for time series analysis; it is usually only a snapshot of current states, not the complete picture over time. You might miss critical peaks or excursions in data.
• Less Delay: Data can be severely delayed in time due to needing multiple commands to get a complete set of data and the inability to poll simultaneously from a central management host. Streaming more accurately preserves the time-series context of data samples.

Consequently, streaming is a far more efficient and accurate way to gather telemetry.

Telemetry Excellence with the iDRAC9 Datacenter License

iDRAC9 v4.0, with the Datacenter license, offers over 180 telemetry metrics on various server devices and sensors. These metrics also form the basis of our SupportAssist Collection Report, an incredibly useful tool that captures over 5,000 pieces of diagnostic data and log files for troubleshooting server issues. iDRAC9 Telemetry Streaming does all the heavy lifting for you by internally sampling and storing all the data points and then streaming them out in reports at a frequency that fits your needs. iDRAC9 can deliver almost 3 million metrics a day to transform the accuracy of analytics processing for your data center!

Telemetry can be delivered via the following methods:

• Redfish Server-Sent Events (SSE), a DMTF standard for streaming data2
• Redfish subscription for pushing events, another DMTF standard
• Remote Syslog, a protocol for pushing logs for centralized monitoring
• Non-streaming, scripted polling via the iDRAC9 RESTful API (though not as efficient as streaming as discussed earlier)

The data is formatted using JSON (JavaScript Object Notation) and can be easily adapted to connect many analytics solutions on the market, as shown in Figure 2.

Figure 2. Integrating iDRAC9 Telemetry Streaming with Popular Analytics Solutions

Types of Telemetry Data

A summary of the types of telemetry that iDRAC9 has are: 

New Telemetry Data with iDRAC9 4.0:

• Serial Data Log messages
• GPU Accelerator Inventory & Monitoring
• Advanced CPU Metrics
• Storage Drive SMART logs
• Advanced Memory Monitoring
• SFP+ Optical Transceiver Inventory & Monitoring

Existing Telemetry Data:

• Configuration – comprehensive settings for all devices (BIOS, iDRAC, NICs, RAID, etc.)
• Inventory: comprehensive server hardware and firmware reporting
• Performance: CPU, memory bandwidth and I/O usage (Compute Usage Per Second or CUPS)
• Performance and diagnostic statistics: PERC, NICs, Fiber Channel
• Sensors: voltage, temperature, power, connectivity status, intrusion detection
• Logs: SEL log, iDRAC diagnostics, Lifecycle Controller Log

Figure 3 illustrates an external analytics solution capturing and visualizing iDRAC9 Telemetry Streaming. In this case, CUPS performance data was streamed to InfluxDB for the data analysis, and Grafana then used for the visualization.

Figure 3. Example of iDRAC9 Telemetry for CUPS Performance Data

In Conclusion

Dell EMC continues to introduce innovations that help our customers automate the management of their IT infrastructure. iDRAC9 Telemetry Streaming represents a huge step forward in helping our customers leverage the extensive data available in their PowerEdge servers. Customers can easily stream this telemetry into their analytics tools and leverage advanced AI techniques to automate their IT systems management and operations further.

1. “2020 Global State of Enterprise Analytics”, published by MicroStrategy.
2. Server-Sent Events (SSE) is a server push technology (part of HTML5) enabling a client to receive automatic updates from a server via an HTTP/S internet connection.

Summary

The iDRAC is designed for secure local and remote server management and offers industry-leading security features. iDRAC9 5.10.00.00 supports TLS 1.3 over HTTPS, to encrypt data and authenticate connections for moving data over the internet. TLS 1.3 uses advanced encryption algorithms, fewer cipher suites, and more secure handshakes.

Features supported by iDRAC9 over HTTPS using TLS 1.3 include:

• iDRAC9 Web Server
• Firmware Updates
• Export SupportAssist
• Import/Export Server Configuration File
• Export Inventory
• Export Lifecycle Log

Introduction

Data Center Managers rely on remote server management to deploy, update, and monitor their servers to extend their reach without having physical access to them. Securing your remote connection with encryption and secure login credentials is one way to prevent malicious actors from gaining access to your server. A secure connection prevents the deletion of critical data, ability to apply malware, or alter the system configuration 

Embedded within every Dell PowerEdge server is a powerful leading-edge remote server management processor, the Integrated Dell Remote Access Controller (iDRAC). The iDRAC is designed for secure local and remote server management and offers industry-leading security features. iDRAC9 establishes an encrypted connection over HTTPS using an SSL/TLS certificate to authenticate to web browsers and command line utilities. iDRAC9 version 5.10.00.00, now supports TLS v1.3 over HTTPS.

Secure communications with SSL/TLS

The iDRAC Web User Interface can be reached with any supported browser. iDRAC uses an SSL/TLS certificate to authenticate itself to web browsers and command line utilities, establishing an encrypted link. Transport Layer Security (TLS) is one of the most widely used security protocols.

When a user goes to a website, their browser checks for a TLS certificate on the site. If a certificate is present, their browser performs a TLS handshake to check its validity and authenticate the server. Once a link has been established between the two servers, TLS encryption and SSL decryption enable secure data transport.

There are several options available to secure the network connection using an TLS/SSL certificate. iDRACs web server has a self-signed TLS/SSL certificate by default. The self-signed certificate can be replaced with a custom certificate, a custom signing certificate, or a certificate signed by a well- known Certificate Authority (CA). Automated certificate upload can be accomplished by using Redfish scripts. The iDRAC9 Automatic Certificate Enrollment and Renewal feature automatically assures SSL/TLS certificates are in place and up to date for both bare-metal and previously installed systems. The Automatic Certificate Enrollment and Renewal feature requires the iDRAC9 Datacenter license.

TLS 1.3

TLS 1.3 offers several advantages over TLS 1.2. TLS version 1.3 uses advanced encryption algorithms, fewer cipher suites and, faster and more secure handshakes. Enabling TLS 1.3 results in better network connection performance.

Many new operating systems and browsers support TLS 1.3. Web browsers and command-line utilities, such as RACADM and WS-Man, use this TLS/SSL certificate for server authentication and to establish an encrypted connection. If the HTTPS server is configured for TLS 1.3, the clients will automatically detect it and perform the operation over TLS 1.3.

iDRAC9 Web Server can be configured with options to support “TLS 1.3 only.” Use the “TLS 1.3 only” option when the HTTPS client can support it. Older browsers that do not support TLS 1.3 should be configures to “TLS 1.2 and Higher” or “TLS 1.1 and Higher.”

Once iDRAC is configured and the TLS/SSL certificate is installed on the management stations, SSL enabled clients can access iDRAC securely and without certificate warnings.

Conclusion

iDRAC9 continues to support that latest security standards to meet the needs of security focused customers. iDRAC9 5.10.00.00 TLS 1.3 support over HTTPS, enables you to use the most current security stance for remotely managing your PowerEdge servers.

Summary

The next generation of PowerEdge servers brings a new Power Supply layout that allows for improved system cooling and helps enable support for Gen4 PCIe cards. Purchase with confidence, knowing that these system improvements help ensure that the next generation PowerEdge server continues to deliver best-in- class features.

Split Power Supplies

 The layout of previous generations of Dell PowerEdge rack servers utilized two power supplies grouped on one side of the chassis. Dell’s next generation of PowerEdge servers improves the mechanical design with the two power supplies split – one on each side of the chassis. This new system and power supply layout offers several tangible benefits over the older system design.

Balanced Airflow

 In prior generations, the location of the inner power supply was near the CPU exhaust airflow. Due to the proximity to the CPU, the PSU was continually exposed to air that is heated to high temperatures from moving through the CPU heatsink. With each new CPU refresh, power continues to increase and PSU cooling becomes exponentially more challenging. Additionally, the PSU location compounded the thermal challenges because it was also an obstruction to airflow moving freely through the CPU heatsink.

The split power supply placement in the next generation of PowerEdge servers allows for both low temperature airflow for PSU cooling and less obstruction for cooling high power CPUs. The result is that system airflow is balanced across the width of the system providing more uniform airflow for CPU, Memory, and PCIe cards in the rear of the chassis.

 Support for Gen4 PCIe

One of the goals of the new architectures in the next generation of PowerEdge servers is to support faster I/O speeds, such as PCIe Gen 4, and beyond. PCIe Gen 4 doubles the lane speed to 16GT/s from the previous generation. A key element in PCIe performance is the length of PCIe traces. With the new system layout, a main goal was to shorten the overall PCIe trace lengths in the topology, including traces in the motherboard. By positioning PSU’s at both edges, the I/O traces to connectors can be shortened for both processors. This is the optimal physical layout for PCIe Gen 4 and will enable even faster speeds for future platforms. The shorter PCIe traces translate into better system costs and improved Signal Integrity for more reliable performance across a broad variety of customer applications.

Balanced Airflow Illustration

The illustration below shows the 14G Generation Server layout (left image) with PSUs located on one side of the chassis. In this layout it is evident that system airflow and PSU cooling are not optimized. In the 15th Generation layout on the right, the dual power are split, one on each side of the chassis. The split PSU layout helps to balance the system airflow, reduce PSU operating temperatures, and allows for PCIe Gen4 card support and thus an overall more optimal system design layout.

In Conclusion

PowerEdge servers continue to deliver best-in-class features. The new PowerEdge servers have the PSUs on both rear sides of the server, improving chassis airflow, overall thermal efficiency and allows for Gen4 PCIe card support.

Summary

Dell EMC offers a patented approach to complete customization capabilities for UEFI Secure Boot policies, giving system owners an option to eliminate reliance on industry keys and industry certificate authorities.

Dell EMC system management tools enable the removal of the UEFI CA certificate and the addition of custom Secure Boot policy entries. This facilitates precise authorization of operating system boot loaders, hypervisors, and I/O device firmware.

Introduction

While traditional datacenter security focuses on operating system, application, physical, and network levels, new evolving threats require specific attention to server firmware. Dell EMC servers include UEFI Secure Boot for firmware authentication and authorization. In addition to the standard UEFI Secure Boot customization capabilities for operating systems (OSs) and hypervisors (HVs), Dell EMC now offers full customization capabilities for I/O devices. Dell EMC recognizes the risks in relying on standard UEFI Secure Boot signing keys currently used widely in the industry. For customers wanting to avoid standard keys, because of some of the risks they present, Dell EMC provides complete customization capabilities for UEFI Secure Boot, giving system owners an option to eliminate reliance on industry keys and industry certificate authorities.

Firmware Threats

An area of interest to malware developers is creating attacks on the server pre-boot environment. Firmware is stored in protected non-volatile storage and runs in privileged memory, separate from the operating system and storage media. Because of this separation, infected firmware can escape detection and remediation by both the operating system and antivirus software. With direct access to hardware components via the firmware, those seeking to exploit a server can potentially compromise systems without the administrator’s knowledge, or detection by the operating system.When the system firmware is targeted, the compromise persists even if the hard drive is swapped out. Absent use of Secure Boot, which prevents execution of unauthorized pre-boot code, firmware rootkits can compromise device or system firmware, or in the case of bootkits, alter the code path used to boot up the operating system. They can even survive re-installation of the operating system.

Reducing Risk with Standard UEFI Secure Boot

Legacy Basic Input/Output System (BIOS) interfaces present high-risk vulnerabilities and are being replaced with the Unified Extensible Firmware Interface (UEFI). PowerEdge servers supporting UEFI Secure Boot, an industry-wide standard for security in the pre-boot environment, check the cryptographic signatures of UEFI drivers and other code loaded prior to running the OS. These include OS boot loaders and UEFI drivers for PCIe cards and mass storage devices.

Secure Boot is a feature added to the UEFI specification version 2.3.1. The system administrator defines the Secure Boot policy, which the BIOS applies to pre-boot code modules. The BIOS uses the policy to determine whether to trust code modules. If any module does not satisfy the policy, the system BIOS neither loads nor executes the module.

The Standard Secure Boot policy contains industry-standard certificates that authorize all the third-party pre- boot firmware supported, such as OS boot loaders, hypervisors (HVs), and I/O devices, for that system.

While the Standard Secure Boot policy provides a simple way to deploy UEFI Secure Boot, it still carries risk. The Standard policy authorizes many firmware binaries using only a few standard certificates. This broad authorization exposes the system to a broad set of potential vulnerabilities (such as the recent GRUB2 “BootHole” vulnerability, discussed further below).

Basic UEFI Secure Boot Customizations

Some system owners reduce the risks of the Standard Secure Boot policy by customizing the policy according to OS usage. For example, if a server is intended to boot only one OS, administrators remove certificates that authorize other OS(s). Other system owners, who develop custom OS components (kernels or boot loaders), sign the components with their own keys. In this case, the system owner replaces standard certificates in the policy with custom certificates. In this way, a Custom Secure Boot policy reduces risk by authorizing only specific OS boot loaders and HVs. Dell EMC servers support these basic customizations.

However, as shown in the figure above, such custom policies still include industry keys that trust a wide array of signed I/O device firmware. The UEFI CA certificate – part of the Standard Secure Boot policy – is used across the industry for authorizing I/O device firmware and “shim” (an early stage boot loader in Linux distributions). Removing the UEFI CA certificate is desirable, but this presents some challenges since obtaining and custom-signing I/O firmware is difficult.

Dell EMC’s Approach to Full Customization

Dell EMC provides complete customization capabilities for UEFI Secure Boot, eliminating reliance on industry keys and industry certificate authorities. These capabilities include remote management tools for authorizing specific I/O device firmware and pre-boot binaries. Using these tools, system owners can customize their policy easily by removing the UEFI CA certificate and adding custom entries instead. A recent report from the U.S. Government’s National Security Agency (NSA) documents the topic of increased server hardware security, specifically citing the use of PowerEdge UEFI Secure Boot Customization as a method that provides a significantly higher level of security along with the flexibility to support multiple operating systems.

Note that Dell EMC enables removal of the UEFI CA certificate regardless of whether the system owner chooses to use their own public key infrastructure (PKI). System owners who maintain their own PKI rely only on custom policy entries (see the fifth level in the figure above). Other system owners, who do not maintain their own PKI, may choose to trust the OS vendor’s keys and still use Dell EMC’s approach to authorize specific I/O device firmware (see the fourth level in the figure above). In this way, system owners without their own PKI can still mitigate the risk of the UEFI CA certificate.

 Although complete customization requires careful management, this solution is attractive for system owners who want precise control over firmware authorization. When vulnerabilities are discovered in industry-signed firmware or industry-signed bootloaders, Dell EMC’s precise authorization method can help eliminate exposure, as in the case of the GRUB2 BootHole vulnerability. 

GRUB2 BootHole Vulnerability

In July 2020, researchers disclosed a buffer overflow vulnerability in the GRUB2 Linux bootloader that enables arbitrary code execution in GRUB2. Dubbed “BootHole,” this vulnerability is significant because GRUB2 is normally launched by the “shim” early-stage bootloader, which is authorized by the UEFI CA certificate. Thus, attackers can modify the boot flow even when UEFI Secure Boot is enabled. The exploit is possible both on Linux and Windows systems.

Since Dell EMC servers offer complete customization of the Secure Boot policy – including removing the UEFI CA certificate – system owners can use customization to eliminate exposure to BootHole and similar classes of future vulnerabilities.

In Conclusion

Dell EMC UEFI Secure Boot Customization serves to minimize the attack surface in pre-boot firmware. Using custom Secure Boot policies, system owners fine-tune their firmware authorization rules, reducing exposure to potential vulnerabilities in industry-signed binaries.

Dell EMC PowerEdge is the first server vendor to provide advanced UEFI Secure Boot Customization, enabling customers to eliminate dependence on all third-party certificates and take full ownership of firmware execution. Using the precise authorization offered by Dell management tools, system administrators can optimize the effectiveness of their UEFI Secure Boot policy.

Summary

A core element of Dell Technologies’ security- enabled supply chain program is Secured Component Verification (SCV), now an integral part of the entire PowerEdge Server line. SCV enables end-users to validate that systems delivered are secure, that components and configurations set at time of manufacture conform to the specifications set by the customer, and remain so throughout the journey, from factory to data center.

Introduction

Dell Technologies has long known that a secure product begins with a secure supply chain and has had a robust Supply Chain Assurance Program for many years. As the threat landscape becomes more complex and sophisticated, so too system protection and control measures need to evolve to meet the challenge. Systems can be vulnerable to hardware intrusion or manipulation in the form of undetectable malware inserted during manufacturing or during transit from the factory. From the moment a server leaves the factory until it arrives at its destination, it is potentially exposed to security threats without the user even realizing what is happening, such as counterfeit components, malware and firmware tampering.

Organizations understand the importance of a secure supply chain and are making purchasing decisions with that in mind. Addressing this industry-wide concern for customers, Dell Technologies is adding new supply chain security offerings for the entire Dell EMC PowerEdge Server portfolio, strengthening the integrity of the hardware and expanding its comprehensive secure supply chain practices.

Dell Technologies Secure Supply Chain

Dell Technologies takes a multifaceted approach to protect its supply chain and to deliver solutions that customers can trust. Whether it’s a desktop, laptop, server, or a data storage array, product features are conceived, designed, prototyped, implemented, set into production, deployed, maintained and validated with supply chain security as a top priority. As a safeguard, Dell has ‘cybersecurity-hardened’ the entire server development lifecycle, from design and development, through manufacturing, to delivery and use in ways that span the entire PowerEdge portfolio.

Some of the Key measures in place to enable broad supply chain assurance are:

• chain of custody tracking and anti-tamper packaging
• 3rd party access restrictions and staff checks at point of manufacturing
• code signing and secure downloading
• chain of trust maintenance for critical components
• hardware intrusion detection, and recording of enclosure breaches during shipmen

Through trusted relationships, and high standards of responsibility and integrity for ourselves and across our supply chain network, we drive reliable manufacturing that our stakeholders can trust.

In total, these measures serve as important differentiators for Dell EMC customers, especially in the Federal, Banking/Finance, Healthcare and Retail sectors.

Dell Technologies Secured Component Verification

A core element of Dell’s security-enabled supply chain program is Secured Component Verification (SCV), now an integral part of the entire PowerEdge portfolio, and soon to come to other product lines.

Dell Technologies Secured Component Verification for PowerEdge provides verification of the as-built hardware configuration for PowerEdge servers. The verification enables customers to confidently deploy new servers in their datacenters knowing the hardware integrity is in-tact from the outset and that the chosen configuration will provide them with a solid foundation for their mission critical applications.

SCV enables IT administrators to validate the componentry of incoming systems to ensure that the configuration is identical to what has been manufactured, and that components and configurations set at time of manufacture, conform to the specifications set by the customer, and remain so throughout the journey, from factory to data center. Any component changes that occur after a device leaves the Dell factory, and before the verification is run, will show up as a mismatch in the resulting report. This allows customers to account for authorized changes and to identify unauthorized changes.

A cryptographic certificate is generated in the factory, capturing component data at the point of origin, that contains specific component data and corresponding unique identifiers. It is securely stored in the server and is later validated against the as-received configuration by the customer at point of arrival. This is achieved via an application that assesses the as-received componentry and associated unique ID’s. SCV conforms to Federal supply chain crypto requirements to meet future standards.

Validation through Application

When a customer purchases a server with an SCV license, it is built to order with validated components. At time of manufacture, each critical server part is analyzed, a unique ID is recorded, which includes a set of component data. The specific manufactured hardware configuration is captured within a cryptographic certificate that is bound to the unique server. This SCV certificate is signed by a Dell Certificate Authority and stored in iDRAC, to be retrieved by the customer or by the Dell SCV Validation application. The manufactured hardware configuration is cryptographically locked to the certificate and will accompany the server during transit to the customer.

When a customer brings a new server into their environment and powers it up, they can run the SCV Validation app to collect and compare the current hardware configuration against the hardware configuration collected at the time it was built in the Dell factory. The result is either a perfect match, or a list of components not in compliance with those used at time of build.

In Conclusion

The Dell Technologies Secured Component Verification (SCV) enables IT administrators to validate what Dell has manufactured and track any expected or unexpected hardware modifications that have occurred during the journey from the factory to datacenter.

Using SCV, IT operations and security teams gain assurance that just-delivered systems conform to component specifications, and that potential attack vectors have been much reduced. They can now spend more time focusing on supporting business outcomes and let Dell help them provide assurance and confidence with their server infrastructure.

Summary

Enabling system lockdown mode is part of Dell Technologies’ cyber resilient architecture of Protect, Detect and Recover. System Lockdown helps prevent change or “drift” in system firmware images and critical server configuration settings. Dell Technologies is the only vendor to offer the ability to dynamically enable and disable system lockdown once your server is provisioned and in production without having to reboot.

Introduction

Running the latest firmware on datacenter servers helps keep up with security and performance improvements, maintain optimal operating parameters, and leverage new features. All are critical to the bottom line, to getting the most from your datacenter investment.

 When unplanned or unforeseen changes occur to server configurations, whether benign or malicious, these can propagate across a datacenter with a corresponding loss in productivity or extra cost.

iDRAC9 System Lockdown Benefits

To prevent unintentional changes, the iDRAC9 Enterprise and Datacenter licenses now include a feature “System Lockdown,” a virtual lock for firmware and hardware configurations. Even those with full admin privileges are limited to read-only access—unless the lock is first disabled. This prevents server ‘drift’, the unintentional migration of firmware and configuration settings across servers.

 The lock does, however, allow for continued access to key operations, such as power capping and power cycling, health monitoring and virtual console access, while keeping server workloads running. All hypervisor and OS functionality are also fully accessible.  When accessed via a web GUI, Redfish REST APIs, or RACADM command-line utility, systems administrators are prevented from making changes that could impact servers in production. Additionally, the lockdown status is evident via a padlock icon and greyed out settings in the iDRAC GUI.

Even before logging in, the admin is notified the system is in Lockdown mode.

iDRAC9 System Lockdown is Part of Dell’s Cyber Resilient Architecture

The lockdown mode is part of Dell’s PowerEdge cyber resilient architecture, with its emphasis on Protect, Detect and Recover. It protects by preventing firmware downgrades as a possible vector of attack, adding or removing users as a means of circumventing settings, or modifying lockout policies. System Lockdown enables detecting changes outside a maintenance window by creating alerts in the iDRAC lifecycle log that can be configured to send notifications, and it potentially cuts recovery time spent re-imaging or re-configuring servers.

System lockdown now offers native lockdown support in select NICs which prevents malware in the OS from installing firmware updates using altered versions of vendor tools. This also addresses concerns for cloud providers of end customers installing their own firmware versions on the server hardware they are using. As a result, subsequent users of a cloud server can be assured that the networking adaptor firmware is secure and version consistent. 

System Lockdown Drives Datacenter Efficiencies

The system lockdown fits well with standard server maintenance window methodologies, the unlocking and locking of servers serving as ‘bookends’ at the start or end of maintenance work. Once operationalized, it helps drive good maintenance behavior, cuts unforced errors, and prevent server ‘drift’. 

In Conclusion

Enabled in iDRAC Enterprise and Datacenter licenses, the lockdown feature is another important tool available from Dell Technologies to manage and maximize your investment in your PowerEdge servers.

We've heard it said “Give a hacker a 0-day vulnerability, and they will have access for a day; teach a hacker to phish, and they will have access for life.” That made us smile. However, at Dell Technologies we take security very seriously with the mindset that security should be built in, not an add on. In our roles at Dell, we focus on the server management portfolio and we have created a number of tools to help organizations strengthen the security posture of PowerEdge servers.

Starting with CloudIQ, our cloud-based AI OPS infrastructure analytics offering, we incorporate a cybersecurity engine that includes a selection of click to enable security policies for PowerEdge servers, based on Dell best practices. We recently published two DfD (direct from development) papers: 

• Harden Your Server Cybersecurity With Dell CloudIQ DfD highlights the details of the security policy engine now included in CloudIQ. 
• Dell CloudIQ Cybersecurity For PowerEdge: The Benefits Of Automation DfD highlights the time saved using CloudIQ cyber security to police PowerEdge configurations. The results demonstrated that customers can easily create a 15 test policy and apply it to 1000 servers in under three minutes*, as compared to over 98 hours to perform these tasks manually.

*Projected outcomes based on Dell internal analysis of results of one and ten servers, customer results may vary.

Then looking on premise — OpenManage Enterprise (OME), Dell’s server management solution, scales up to 8000 nodes. OME provides full and rich server configuration drift detection and remediation  management of the server configuration profiles accessed from each individual server’s iDRAC. For an overview of that feature, and details about firmware versions and the firmware configuration process, see Improve Operational Efficiency Through OME Server Drift Management.

References

• Dell CloudIQ Cybersecurity For PowerEdge: The Benefits Of Automation (DfD)
• Harden Your Server Cybersecurity With Dell CloudIQ (DfD)
• OpenManage Enterprise: Security Built In (Blog)

Authors: Kim Kinahan and Mark Maclean, PowerEdge Technical Marketing Engineering

LinkedIn