Summary

Enabling system lockdown mode is part of Dell Technologies’ cyber resilient architecture of Protect, Detect and Recover. System Lockdown helps prevent change or “drift” in system firmware images and critical server configuration settings. Dell Technologies is the only vendor to offer the ability to dynamically enable and disable system lockdown once your server is provisioned and in production without having to reboot.

Introduction

Running the latest firmware on datacenter servers helps keep up with security and performance improvements, maintain optimal operating parameters, and leverage new features. All are critical to the bottom line, to getting the most from your datacenter investment.

 When unplanned or unforeseen changes occur to server configurations, whether benign or malicious, these can propagate across a datacenter with a corresponding loss in productivity or extra cost.

iDRAC9 System Lockdown Benefits

To prevent unintentional changes, the iDRAC9 Enterprise and Datacenter licenses now include a feature “System Lockdown,” a virtual lock for firmware and hardware configurations. Even those with full admin privileges are limited to read-only access—unless the lock is first disabled. This prevents server ‘drift’, the unintentional migration of firmware and configuration settings across servers.

 The lock does, however, allow for continued access to key operations, such as power capping and power cycling, health monitoring and virtual console access, while keeping server workloads running. All hypervisor and OS functionality are also fully accessible.  When accessed via a web GUI, Redfish REST APIs, or RACADM command-line utility, systems administrators are prevented from making changes that could impact servers in production. Additionally, the lockdown status is evident via a padlock icon and greyed out settings in the iDRAC GUI.

Even before logging in, the admin is notified the system is in Lockdown mode.

iDRAC9 System Lockdown is Part of Dell’s Cyber Resilient Architecture

The lockdown mode is part of Dell’s PowerEdge cyber resilient architecture, with its emphasis on Protect, Detect and Recover. It protects by preventing firmware downgrades as a possible vector of attack, adding or removing users as a means of circumventing settings, or modifying lockout policies. System Lockdown enables detecting changes outside a maintenance window by creating alerts in the iDRAC lifecycle log that can be configured to send notifications, and it potentially cuts recovery time spent re-imaging or re-configuring servers.

System lockdown now offers native lockdown support in select NICs which prevents malware in the OS from installing firmware updates using altered versions of vendor tools. This also addresses concerns for cloud providers of end customers installing their own firmware versions on the server hardware they are using. As a result, subsequent users of a cloud server can be assured that the networking adaptor firmware is secure and version consistent. 

System Lockdown Drives Datacenter Efficiencies

The system lockdown fits well with standard server maintenance window methodologies, the unlocking and locking of servers serving as ‘bookends’ at the start or end of maintenance work. Once operationalized, it helps drive good maintenance behavior, cuts unforced errors, and prevent server ‘drift’. 

In Conclusion

Enabled in iDRAC Enterprise and Datacenter licenses, the lockdown feature is another important tool available from Dell Technologies to manage and maximize your investment in your PowerEdge servers.

Summary

A core element of Dell Technologies’ security- enabled supply chain program is Secured Component Verification (SCV), now an integral part of the entire PowerEdge Server line. SCV enables end-users to validate that systems delivered are secure, that components and configurations set at time of manufacture conform to the specifications set by the customer, and remain so throughout the journey, from factory to data center.

Introduction

Dell Technologies has long known that a secure product begins with a secure supply chain and has had a robust Supply Chain Assurance Program for many years. As the threat landscape becomes more complex and sophisticated, so too system protection and control measures need to evolve to meet the challenge. Systems can be vulnerable to hardware intrusion or manipulation in the form of undetectable malware inserted during manufacturing or during transit from the factory. From the moment a server leaves the factory until it arrives at its destination, it is potentially exposed to security threats without the user even realizing what is happening, such as counterfeit components, malware and firmware tampering.

Organizations understand the importance of a secure supply chain and are making purchasing decisions with that in mind. Addressing this industry-wide concern for customers, Dell Technologies is adding new supply chain security offerings for the entire Dell EMC PowerEdge Server portfolio, strengthening the integrity of the hardware and expanding its comprehensive secure supply chain practices.

Dell Technologies Secure Supply Chain

Dell Technologies takes a multifaceted approach to protect its supply chain and to deliver solutions that customers can trust. Whether it’s a desktop, laptop, server, or a data storage array, product features are conceived, designed, prototyped, implemented, set into production, deployed, maintained and validated with supply chain security as a top priority. As a safeguard, Dell has ‘cybersecurity-hardened’ the entire server development lifecycle, from design and development, through manufacturing, to delivery and use in ways that span the entire PowerEdge portfolio.

Some of the Key measures in place to enable broad supply chain assurance are:

• chain of custody tracking and anti-tamper packaging
• 3rd party access restrictions and staff checks at point of manufacturing
• code signing and secure downloading
• chain of trust maintenance for critical components
• hardware intrusion detection, and recording of enclosure breaches during shipmen

Through trusted relationships, and high standards of responsibility and integrity for ourselves and across our supply chain network, we drive reliable manufacturing that our stakeholders can trust.

In total, these measures serve as important differentiators for Dell EMC customers, especially in the Federal, Banking/Finance, Healthcare and Retail sectors.

Dell Technologies Secured Component Verification

A core element of Dell’s security-enabled supply chain program is Secured Component Verification (SCV), now an integral part of the entire PowerEdge portfolio, and soon to come to other product lines.

Dell Technologies Secured Component Verification for PowerEdge provides verification of the as-built hardware configuration for PowerEdge servers. The verification enables customers to confidently deploy new servers in their datacenters knowing the hardware integrity is in-tact from the outset and that the chosen configuration will provide them with a solid foundation for their mission critical applications.

SCV enables IT administrators to validate the componentry of incoming systems to ensure that the configuration is identical to what has been manufactured, and that components and configurations set at time of manufacture, conform to the specifications set by the customer, and remain so throughout the journey, from factory to data center. Any component changes that occur after a device leaves the Dell factory, and before the verification is run, will show up as a mismatch in the resulting report. This allows customers to account for authorized changes and to identify unauthorized changes.

A cryptographic certificate is generated in the factory, capturing component data at the point of origin, that contains specific component data and corresponding unique identifiers. It is securely stored in the server and is later validated against the as-received configuration by the customer at point of arrival. This is achieved via an application that assesses the as-received componentry and associated unique ID’s. SCV conforms to Federal supply chain crypto requirements to meet future standards.

Validation through Application

When a customer purchases a server with an SCV license, it is built to order with validated components. At time of manufacture, each critical server part is analyzed, a unique ID is recorded, which includes a set of component data. The specific manufactured hardware configuration is captured within a cryptographic certificate that is bound to the unique server. This SCV certificate is signed by a Dell Certificate Authority and stored in iDRAC, to be retrieved by the customer or by the Dell SCV Validation application. The manufactured hardware configuration is cryptographically locked to the certificate and will accompany the server during transit to the customer.

When a customer brings a new server into their environment and powers it up, they can run the SCV Validation app to collect and compare the current hardware configuration against the hardware configuration collected at the time it was built in the Dell factory. The result is either a perfect match, or a list of components not in compliance with those used at time of build.

In Conclusion

The Dell Technologies Secured Component Verification (SCV) enables IT administrators to validate what Dell has manufactured and track any expected or unexpected hardware modifications that have occurred during the journey from the factory to datacenter.

Using SCV, IT operations and security teams gain assurance that just-delivered systems conform to component specifications, and that potential attack vectors have been much reduced. They can now spend more time focusing on supporting business outcomes and let Dell help them provide assurance and confidence with their server infrastructure.

Summary

Dell EMC offers a patented approach to complete customization capabilities for UEFI Secure Boot policies, giving system owners an option to eliminate reliance on industry keys and industry certificate authorities.

Dell EMC system management tools enable the removal of the UEFI CA certificate and the addition of custom Secure Boot policy entries. This facilitates precise authorization of operating system boot loaders, hypervisors, and I/O device firmware.

Introduction

While traditional datacenter security focuses on operating system, application, physical, and network levels, new evolving threats require specific attention to server firmware. Dell EMC servers include UEFI Secure Boot for firmware authentication and authorization. In addition to the standard UEFI Secure Boot customization capabilities for operating systems (OSs) and hypervisors (HVs), Dell EMC now offers full customization capabilities for I/O devices. Dell EMC recognizes the risks in relying on standard UEFI Secure Boot signing keys currently used widely in the industry. For customers wanting to avoid standard keys, because of some of the risks they present, Dell EMC provides complete customization capabilities for UEFI Secure Boot, giving system owners an option to eliminate reliance on industry keys and industry certificate authorities.

Firmware Threats

An area of interest to malware developers is creating attacks on the server pre-boot environment. Firmware is stored in protected non-volatile storage and runs in privileged memory, separate from the operating system and storage media. Because of this separation, infected firmware can escape detection and remediation by both the operating system and antivirus software. With direct access to hardware components via the firmware, those seeking to exploit a server can potentially compromise systems without the administrator’s knowledge, or detection by the operating system.When the system firmware is targeted, the compromise persists even if the hard drive is swapped out. Absent use of Secure Boot, which prevents execution of unauthorized pre-boot code, firmware rootkits can compromise device or system firmware, or in the case of bootkits, alter the code path used to boot up the operating system. They can even survive re-installation of the operating system.

Reducing Risk with Standard UEFI Secure Boot

Legacy Basic Input/Output System (BIOS) interfaces present high-risk vulnerabilities and are being replaced with the Unified Extensible Firmware Interface (UEFI). PowerEdge servers supporting UEFI Secure Boot, an industry-wide standard for security in the pre-boot environment, check the cryptographic signatures of UEFI drivers and other code loaded prior to running the OS. These include OS boot loaders and UEFI drivers for PCIe cards and mass storage devices.

Secure Boot is a feature added to the UEFI specification version 2.3.1. The system administrator defines the Secure Boot policy, which the BIOS applies to pre-boot code modules. The BIOS uses the policy to determine whether to trust code modules. If any module does not satisfy the policy, the system BIOS neither loads nor executes the module.

The Standard Secure Boot policy contains industry-standard certificates that authorize all the third-party pre- boot firmware supported, such as OS boot loaders, hypervisors (HVs), and I/O devices, for that system.

While the Standard Secure Boot policy provides a simple way to deploy UEFI Secure Boot, it still carries risk. The Standard policy authorizes many firmware binaries using only a few standard certificates. This broad authorization exposes the system to a broad set of potential vulnerabilities (such as the recent GRUB2 “BootHole” vulnerability, discussed further below).

Basic UEFI Secure Boot Customizations

Some system owners reduce the risks of the Standard Secure Boot policy by customizing the policy according to OS usage. For example, if a server is intended to boot only one OS, administrators remove certificates that authorize other OS(s). Other system owners, who develop custom OS components (kernels or boot loaders), sign the components with their own keys. In this case, the system owner replaces standard certificates in the policy with custom certificates. In this way, a Custom Secure Boot policy reduces risk by authorizing only specific OS boot loaders and HVs. Dell EMC servers support these basic customizations.

However, as shown in the figure above, such custom policies still include industry keys that trust a wide array of signed I/O device firmware. The UEFI CA certificate – part of the Standard Secure Boot policy – is used across the industry for authorizing I/O device firmware and “shim” (an early stage boot loader in Linux distributions). Removing the UEFI CA certificate is desirable, but this presents some challenges since obtaining and custom-signing I/O firmware is difficult.

Dell EMC’s Approach to Full Customization

Dell EMC provides complete customization capabilities for UEFI Secure Boot, eliminating reliance on industry keys and industry certificate authorities. These capabilities include remote management tools for authorizing specific I/O device firmware and pre-boot binaries. Using these tools, system owners can customize their policy easily by removing the UEFI CA certificate and adding custom entries instead. A recent report from the U.S. Government’s National Security Agency (NSA) documents the topic of increased server hardware security, specifically citing the use of PowerEdge UEFI Secure Boot Customization as a method that provides a significantly higher level of security along with the flexibility to support multiple operating systems.

Note that Dell EMC enables removal of the UEFI CA certificate regardless of whether the system owner chooses to use their own public key infrastructure (PKI). System owners who maintain their own PKI rely only on custom policy entries (see the fifth level in the figure above). Other system owners, who do not maintain their own PKI, may choose to trust the OS vendor’s keys and still use Dell EMC’s approach to authorize specific I/O device firmware (see the fourth level in the figure above). In this way, system owners without their own PKI can still mitigate the risk of the UEFI CA certificate.

 Although complete customization requires careful management, this solution is attractive for system owners who want precise control over firmware authorization. When vulnerabilities are discovered in industry-signed firmware or industry-signed bootloaders, Dell EMC’s precise authorization method can help eliminate exposure, as in the case of the GRUB2 BootHole vulnerability. 

GRUB2 BootHole Vulnerability

In July 2020, researchers disclosed a buffer overflow vulnerability in the GRUB2 Linux bootloader that enables arbitrary code execution in GRUB2. Dubbed “BootHole,” this vulnerability is significant because GRUB2 is normally launched by the “shim” early-stage bootloader, which is authorized by the UEFI CA certificate. Thus, attackers can modify the boot flow even when UEFI Secure Boot is enabled. The exploit is possible both on Linux and Windows systems.

Since Dell EMC servers offer complete customization of the Secure Boot policy – including removing the UEFI CA certificate – system owners can use customization to eliminate exposure to BootHole and similar classes of future vulnerabilities.

In Conclusion

Dell EMC UEFI Secure Boot Customization serves to minimize the attack surface in pre-boot firmware. Using custom Secure Boot policies, system owners fine-tune their firmware authorization rules, reducing exposure to potential vulnerabilities in industry-signed binaries.

Dell EMC PowerEdge is the first server vendor to provide advanced UEFI Secure Boot Customization, enabling customers to eliminate dependence on all third-party certificates and take full ownership of firmware execution. Using the precise authorization offered by Dell management tools, system administrators can optimize the effectiveness of their UEFI Secure Boot policy.

Summary

To combat and reduce the threat surface in the pre-boot environment, a broad transitioning is happening industry-wide from legacy BIOS boot to Unified Extensible Firmware Interface (UEFI) boot. UEFI changes the interface and data structures to interact with I/O device firmware and operating systems. The primary intent of UEFI is to eliminate shortcomings in the legacy BIOS boot environment, enabling system firmware to continue scaling with industry trends. System administrators are using UEFI boot throughout their data centers for cyber resilience and secure end-to-end boot.

Threat surface in the pre-boot environment

As security breaches are becoming more frequent, system administrators must employ a wider variety of defenses. Cyber threats not only affect traditional areas of security focus, such as network, operating system, and applications; they affect firmware as well. Attackers find this pre-boot environment lucrative and use firmware rootkits to hide malicious code, called malware, in device or system firmware.  

Firmware is software that is embedded in a piece of hardware. While server components are often viewed strictly as hardware, many components have firmware, such as network interface cards, storage controllers, graphics cards, and more. The firmware acts as the device’s operating system, providing control, monitoring, and data manipulation functions. Firmware runs before the operating system environments come into existence.    

Malware can control system firmware and then gain full access to the system. Pre-boot malware avoids operating system privilege levels, escapes detection by operating system anti-malware tools, and even survives reinstallation of the operating system. If an attacker injects malware into the pre-boot environment, administrators might find it difficult to remove, if they detect it at all.

Industry transition to UEFI boot

To combat and reduce the threat surface in the pre-boot environment, many vendors and customers are embracing UEFI boot and have stopped certifying operating systems and applications with legacy BIOS boot mode.

• Microsoft requires UEFI boot on Windows Server 2022 and beyond.
• VMware only certifies UEFI boot in their latest ESXi offerings.
• Intel has stopped supporting legacy BIOS boot with certain platforms such as the Intel® Xeon® E-2300 series processors.
• Many new technologies, such as PCIe Gen 5 and NVMe, have eliminated legacy BIOS boot, and the modern data center is transitioning with them.  

UEFI benefits

UEFI was designed to overcome many limitations of legacy BIOS boot.

UEFI supports drive sizes up to 9 zettabytes, whereas BIOS supports only 2.2 terabytes.

UEFI can provide faster boot time by allowing parallel execution for sections of the boot flow.

 UEFI has discrete driver support, while BIOS has drivers stored in its ROM and lacks modularity, so updating BIOS firmware can be more difficult.

UEFI offers improved security, including Secure Boot. Secure Boot prevents the computer from running unauthorized and unverified code during boot, which helps prevent rootkit and bootkit attacks.

UEFI is easier to deploy and manage. The UI supports mouse-based navigation due to UEFI’s ability to run in 32-bit and 64-bit modes. BIOS runs in 16-bit mode and only allows keyboard-based navigation.

UEFI operations

UEFI does not change the traditional purposes of the system BIOS. To a large extent, UEFI performs the same initialization, boot, configuration, and management tasks as a traditional BIOS. However, UEFI does change the interfaces and data structures that the BIOS uses to interact with I/O device firmware and operating systems. The interface consists of data tables that contain platform-related information, plus boot and runtime service calls that are available to the operating system and its loader. Together, these provide a standard, modern environment for booting an operating system and running pre-boot applications. The primary intent of UEFI is to eliminate shortcomings in the traditional BIOS environment, enabling system firmware to continue scaling with industry trends.

Figure 1.  Levels of UEFI boot

UEFI Secure Boot

UEFI Secure Boot is a technology that eliminates a major security void that might occur during a handoff between the UEFI firmware and the operating system.   

Users configure a Secure Boot policy consisting of X.509 certificates and hash values for both authorized and unauthorized entities. The system firmware enforces this policy when determining whether to run pre-boot software including I/O device firmware and operating system loaders. When enabled, UEFI Secure Boot prevents unsigned or compromised UEFI device drivers from being loaded, displays an error message, and does not allow the device to function. You must disable Secure Boot to load the unsigned device drivers.

Since mid-2017, you can enable or disable the Secure Boot feature on Dell PowerEdge servers through various interfaces. The Secure Boot Management on Dell PowerEdge Servers white paper provides more details about Secure Boot and how to configure it on PowerEdge servers.

Figure 2.  UEFI Secure Boot working principle

Dell Secure Boot customization

For customers wanting to avoid standard keys because of some of the risks they present, Dell Technologies provides complete customization capabilities for UEFI Secure Boot. This gives system owners an option to eliminate reliance on industry keys and third-party certificate authorities. In Figure 1, the yellow boxes have one CA certificate that authorizes multiple versions of firmware, whereas the green boxes signify a customized certificate for a specific version of firmware. Dell PowerEdge offers tools to enable capture of firmware hashes and create customized certificates so system administrators can optimize the effectiveness of their UEFI Secure Boot policies according to their security requirements. 

In a recent Cybersecurity Technical Report, the National Security Agency highlighted the need for enabling UEFI Secure Boot and the benefits of using customization to realize the highest level of server security available today. The report showcases the fully customized Secure Boot capabilities of PowerEdge servers as the example of how to achieve this highest level of boot security.

Impact to environments with servers configured for legacy BIOS boot

For customers whose environments include servers configured for legacy BIOS boot, the primary impact of switching to UEFI boot mode is likely to their deployment and maintenance model. Customers who use PXE servers to deploy and perform routine maintenance need to plan for adaption of the PXE server. For more information about this process, see Boot Mode Considerations: BIOS vs. UEFI.

Another impact is the use of NVMe drives that natively boot UEFI. However, they can be used as data drives in both legacy BIOS and UEFI boot modes.

Conclusion

Cyber attacks are becoming more numerous, frequent, and difficult to detect. Strengthening your organization’s security posture by implementing the latest security approaches positions your organization to respond to today’s cyber threat environment. The security features and boot mechanisms available only when a system is configured for UEFI boot mode is driving the industry-wide transition to UEFI boot.  

To achieve additional boot security, UEFI-enabled systems include Secure Boot as a setup option, providing additional security checks during the boot process. 

References   

Dell documents

• Secure Boot Management on Dell PowerEdge Servers
• Boot Mode Considerations: BIOS vs. UEFI
• Dell PowerEdge UEFI Boot: Enhanced Security to Combat Persistent Firmware Threats

NSA documents

• UEFI Defensive Practices Guidance
• Boot Security Modes and Recommendations
• UEFI Lockdown Quick Guidance
• UEFI Secure Boot Customization

UEFI Forum documents

• UEFI Secure Boot in Modern Computer Security Solutions