Summary

Enabling system lockdown mode is part of Dell Technologies’ cyber resilient architecture of Protect, Detect and Recover. System Lockdown helps prevent change or “drift” in system firmware images and critical server configuration settings. Dell Technologies is the only vendor to offer the ability to dynamically enable and disable system lockdown once your server is provisioned and in production without having to reboot.

Introduction

Running the latest firmware on datacenter servers helps keep up with security and performance improvements, maintain optimal operating parameters, and leverage new features. All are critical to the bottom line, to getting the most from your datacenter investment.

 When unplanned or unforeseen changes occur to server configurations, whether benign or malicious, these can propagate across a datacenter with a corresponding loss in productivity or extra cost.

iDRAC9 System Lockdown Benefits

To prevent unintentional changes, the iDRAC9 Enterprise and Datacenter licenses now include a feature “System Lockdown,” a virtual lock for firmware and hardware configurations. Even those with full admin privileges are limited to read-only access—unless the lock is first disabled. This prevents server ‘drift’, the unintentional migration of firmware and configuration settings across servers.

 The lock does, however, allow for continued access to key operations, such as power capping and power cycling, health monitoring and virtual console access, while keeping server workloads running. All hypervisor and OS functionality are also fully accessible.  When accessed via a web GUI, Redfish REST APIs, or RACADM command-line utility, systems administrators are prevented from making changes that could impact servers in production. Additionally, the lockdown status is evident via a padlock icon and greyed out settings in the iDRAC GUI.

Even before logging in, the admin is notified the system is in Lockdown mode.

iDRAC9 System Lockdown is Part of Dell’s Cyber Resilient Architecture

The lockdown mode is part of Dell’s PowerEdge cyber resilient architecture, with its emphasis on Protect, Detect and Recover. It protects by preventing firmware downgrades as a possible vector of attack, adding or removing users as a means of circumventing settings, or modifying lockout policies. System Lockdown enables detecting changes outside a maintenance window by creating alerts in the iDRAC lifecycle log that can be configured to send notifications, and it potentially cuts recovery time spent re-imaging or re-configuring servers.

System lockdown now offers native lockdown support in select NICs which prevents malware in the OS from installing firmware updates using altered versions of vendor tools. This also addresses concerns for cloud providers of end customers installing their own firmware versions on the server hardware they are using. As a result, subsequent users of a cloud server can be assured that the networking adaptor firmware is secure and version consistent. 

System Lockdown Drives Datacenter Efficiencies

The system lockdown fits well with standard server maintenance window methodologies, the unlocking and locking of servers serving as ‘bookends’ at the start or end of maintenance work. Once operationalized, it helps drive good maintenance behavior, cuts unforced errors, and prevent server ‘drift’. 

In Conclusion

Enabled in iDRAC Enterprise and Datacenter licenses, the lockdown feature is another important tool available from Dell Technologies to manage and maximize your investment in your PowerEdge servers.

Summary

A core element of Dell Technologies’ security- enabled supply chain program is Secured Component Verification (SCV), now an integral part of the entire PowerEdge Server line. SCV enables end-users to validate that systems delivered are secure, that components and configurations set at time of manufacture conform to the specifications set by the customer, and remain so throughout the journey, from factory to data center.

Introduction

Dell Technologies has long known that a secure product begins with a secure supply chain and has had a robust Supply Chain Assurance Program for many years. As the threat landscape becomes more complex and sophisticated, so too system protection and control measures need to evolve to meet the challenge. Systems can be vulnerable to hardware intrusion or manipulation in the form of undetectable malware inserted during manufacturing or during transit from the factory. From the moment a server leaves the factory until it arrives at its destination, it is potentially exposed to security threats without the user even realizing what is happening, such as counterfeit components, malware and firmware tampering.

Organizations understand the importance of a secure supply chain and are making purchasing decisions with that in mind. Addressing this industry-wide concern for customers, Dell Technologies is adding new supply chain security offerings for the entire Dell EMC PowerEdge Server portfolio, strengthening the integrity of the hardware and expanding its comprehensive secure supply chain practices.

Dell Technologies Secure Supply Chain

Dell Technologies takes a multifaceted approach to protect its supply chain and to deliver solutions that customers can trust. Whether it’s a desktop, laptop, server, or a data storage array, product features are conceived, designed, prototyped, implemented, set into production, deployed, maintained and validated with supply chain security as a top priority. As a safeguard, Dell has ‘cybersecurity-hardened’ the entire server development lifecycle, from design and development, through manufacturing, to delivery and use in ways that span the entire PowerEdge portfolio.

Some of the Key measures in place to enable broad supply chain assurance are:

• chain of custody tracking and anti-tamper packaging
• 3rd party access restrictions and staff checks at point of manufacturing
• code signing and secure downloading
• chain of trust maintenance for critical components
• hardware intrusion detection, and recording of enclosure breaches during shipmen

Through trusted relationships, and high standards of responsibility and integrity for ourselves and across our supply chain network, we drive reliable manufacturing that our stakeholders can trust.

In total, these measures serve as important differentiators for Dell EMC customers, especially in the Federal, Banking/Finance, Healthcare and Retail sectors.

Dell Technologies Secured Component Verification

A core element of Dell’s security-enabled supply chain program is Secured Component Verification (SCV), now an integral part of the entire PowerEdge portfolio, and soon to come to other product lines.

Dell Technologies Secured Component Verification for PowerEdge provides verification of the as-built hardware configuration for PowerEdge servers. The verification enables customers to confidently deploy new servers in their datacenters knowing the hardware integrity is in-tact from the outset and that the chosen configuration will provide them with a solid foundation for their mission critical applications.

SCV enables IT administrators to validate the componentry of incoming systems to ensure that the configuration is identical to what has been manufactured, and that components and configurations set at time of manufacture, conform to the specifications set by the customer, and remain so throughout the journey, from factory to data center. Any component changes that occur after a device leaves the Dell factory, and before the verification is run, will show up as a mismatch in the resulting report. This allows customers to account for authorized changes and to identify unauthorized changes.

A cryptographic certificate is generated in the factory, capturing component data at the point of origin, that contains specific component data and corresponding unique identifiers. It is securely stored in the server and is later validated against the as-received configuration by the customer at point of arrival. This is achieved via an application that assesses the as-received componentry and associated unique ID’s. SCV conforms to Federal supply chain crypto requirements to meet future standards.

Validation through Application

When a customer purchases a server with an SCV license, it is built to order with validated components. At time of manufacture, each critical server part is analyzed, a unique ID is recorded, which includes a set of component data. The specific manufactured hardware configuration is captured within a cryptographic certificate that is bound to the unique server. This SCV certificate is signed by a Dell Certificate Authority and stored in iDRAC, to be retrieved by the customer or by the Dell SCV Validation application. The manufactured hardware configuration is cryptographically locked to the certificate and will accompany the server during transit to the customer.

When a customer brings a new server into their environment and powers it up, they can run the SCV Validation app to collect and compare the current hardware configuration against the hardware configuration collected at the time it was built in the Dell factory. The result is either a perfect match, or a list of components not in compliance with those used at time of build.

In Conclusion

The Dell Technologies Secured Component Verification (SCV) enables IT administrators to validate what Dell has manufactured and track any expected or unexpected hardware modifications that have occurred during the journey from the factory to datacenter.

Using SCV, IT operations and security teams gain assurance that just-delivered systems conform to component specifications, and that potential attack vectors have been much reduced. They can now spend more time focusing on supporting business outcomes and let Dell help them provide assurance and confidence with their server infrastructure.

Summary

Dell EMC offers a patented approach to complete customization capabilities for UEFI Secure Boot policies, giving system owners an option to eliminate reliance on industry keys and industry certificate authorities.

Dell EMC system management tools enable the removal of the UEFI CA certificate and the addition of custom Secure Boot policy entries. This facilitates precise authorization of operating system boot loaders, hypervisors, and I/O device firmware.

Introduction

While traditional datacenter security focuses on operating system, application, physical, and network levels, new evolving threats require specific attention to server firmware. Dell EMC servers include UEFI Secure Boot for firmware authentication and authorization. In addition to the standard UEFI Secure Boot customization capabilities for operating systems (OSs) and hypervisors (HVs), Dell EMC now offers full customization capabilities for I/O devices. Dell EMC recognizes the risks in relying on standard UEFI Secure Boot signing keys currently used widely in the industry. For customers wanting to avoid standard keys, because of some of the risks they present, Dell EMC provides complete customization capabilities for UEFI Secure Boot, giving system owners an option to eliminate reliance on industry keys and industry certificate authorities.

Firmware Threats

An area of interest to malware developers is creating attacks on the server pre-boot environment. Firmware is stored in protected non-volatile storage and runs in privileged memory, separate from the operating system and storage media. Because of this separation, infected firmware can escape detection and remediation by both the operating system and antivirus software. With direct access to hardware components via the firmware, those seeking to exploit a server can potentially compromise systems without the administrator’s knowledge, or detection by the operating system.When the system firmware is targeted, the compromise persists even if the hard drive is swapped out. Absent use of Secure Boot, which prevents execution of unauthorized pre-boot code, firmware rootkits can compromise device or system firmware, or in the case of bootkits, alter the code path used to boot up the operating system. They can even survive re-installation of the operating system.

Reducing Risk with Standard UEFI Secure Boot

Legacy Basic Input/Output System (BIOS) interfaces present high-risk vulnerabilities and are being replaced with the Unified Extensible Firmware Interface (UEFI). PowerEdge servers supporting UEFI Secure Boot, an industry-wide standard for security in the pre-boot environment, check the cryptographic signatures of UEFI drivers and other code loaded prior to running the OS. These include OS boot loaders and UEFI drivers for PCIe cards and mass storage devices.

Secure Boot is a feature added to the UEFI specification version 2.3.1. The system administrator defines the Secure Boot policy, which the BIOS applies to pre-boot code modules. The BIOS uses the policy to determine whether to trust code modules. If any module does not satisfy the policy, the system BIOS neither loads nor executes the module.

The Standard Secure Boot policy contains industry-standard certificates that authorize all the third-party pre- boot firmware supported, such as OS boot loaders, hypervisors (HVs), and I/O devices, for that system.

While the Standard Secure Boot policy provides a simple way to deploy UEFI Secure Boot, it still carries risk. The Standard policy authorizes many firmware binaries using only a few standard certificates. This broad authorization exposes the system to a broad set of potential vulnerabilities (such as the recent GRUB2 “BootHole” vulnerability, discussed further below).

Basic UEFI Secure Boot Customizations

Some system owners reduce the risks of the Standard Secure Boot policy by customizing the policy according to OS usage. For example, if a server is intended to boot only one OS, administrators remove certificates that authorize other OS(s). Other system owners, who develop custom OS components (kernels or boot loaders), sign the components with their own keys. In this case, the system owner replaces standard certificates in the policy with custom certificates. In this way, a Custom Secure Boot policy reduces risk by authorizing only specific OS boot loaders and HVs. Dell EMC servers support these basic customizations.

However, as shown in the figure above, such custom policies still include industry keys that trust a wide array of signed I/O device firmware. The UEFI CA certificate – part of the Standard Secure Boot policy – is used across the industry for authorizing I/O device firmware and “shim” (an early stage boot loader in Linux distributions). Removing the UEFI CA certificate is desirable, but this presents some challenges since obtaining and custom-signing I/O firmware is difficult.

Dell EMC’s Approach to Full Customization

Dell EMC provides complete customization capabilities for UEFI Secure Boot, eliminating reliance on industry keys and industry certificate authorities. These capabilities include remote management tools for authorizing specific I/O device firmware and pre-boot binaries. Using these tools, system owners can customize their policy easily by removing the UEFI CA certificate and adding custom entries instead. A recent report from the U.S. Government’s National Security Agency (NSA) documents the topic of increased server hardware security, specifically citing the use of PowerEdge UEFI Secure Boot Customization as a method that provides a significantly higher level of security along with the flexibility to support multiple operating systems.

Note that Dell EMC enables removal of the UEFI CA certificate regardless of whether the system owner chooses to use their own public key infrastructure (PKI). System owners who maintain their own PKI rely only on custom policy entries (see the fifth level in the figure above). Other system owners, who do not maintain their own PKI, may choose to trust the OS vendor’s keys and still use Dell EMC’s approach to authorize specific I/O device firmware (see the fourth level in the figure above). In this way, system owners without their own PKI can still mitigate the risk of the UEFI CA certificate.

 Although complete customization requires careful management, this solution is attractive for system owners who want precise control over firmware authorization. When vulnerabilities are discovered in industry-signed firmware or industry-signed bootloaders, Dell EMC’s precise authorization method can help eliminate exposure, as in the case of the GRUB2 BootHole vulnerability. 

GRUB2 BootHole Vulnerability

In July 2020, researchers disclosed a buffer overflow vulnerability in the GRUB2 Linux bootloader that enables arbitrary code execution in GRUB2. Dubbed “BootHole,” this vulnerability is significant because GRUB2 is normally launched by the “shim” early-stage bootloader, which is authorized by the UEFI CA certificate. Thus, attackers can modify the boot flow even when UEFI Secure Boot is enabled. The exploit is possible both on Linux and Windows systems.

Since Dell EMC servers offer complete customization of the Secure Boot policy – including removing the UEFI CA certificate – system owners can use customization to eliminate exposure to BootHole and similar classes of future vulnerabilities.

In Conclusion

Dell EMC UEFI Secure Boot Customization serves to minimize the attack surface in pre-boot firmware. Using custom Secure Boot policies, system owners fine-tune their firmware authorization rules, reducing exposure to potential vulnerabilities in industry-signed binaries.

Dell EMC PowerEdge is the first server vendor to provide advanced UEFI Secure Boot Customization, enabling customers to eliminate dependence on all third-party certificates and take full ownership of firmware execution. Using the precise authorization offered by Dell management tools, system administrators can optimize the effectiveness of their UEFI Secure Boot policy.

Summary

There is a broad transitioning happening industry-wide from legacy boot to UEFI boot. UEFI boot mode offers enhanced security to combat persistent firmware threats. Specific security features and boot mechanisms are available only when the system is configured for UEFI boot mode.

The Cyber Threat Landscape is Evolving

As security breaches are becoming more frequent, system administrators must employ a wider variety of defenses. Cyber threats not only impact traditional areas of security focus, such as network, operating system (OS), and applications but the firmware is often overlooked. Attackers find this pre-boot environment lucrative and use firmware rootkits to hide malicious code, called malware, in device or system firmware.

Firmware is software that is embedded in a piece of hardware. While there is a tendency to view server components as strictly hardware, many components have firmware, such as network interface cards, storage controllers, graphic cards, and more. The firmware acts as the device’s operating system providing control, monitoring, and data manipulation functions. Firmware executes before the operating system environments come into existence.

Malware can control system firmware and then gain full access to the system. Pre-boot malware avoids OS privilege levels, escapes detection by OS anti-malware tools, and even survives re- installation of the OS. If an attacker injects malware into the pre-boot environment, administrators may find it difficult to remove, if they detect it at all.

Industry Transition to UEFI Boot

To combat and reduce the threat surface in the pre-boot environment, there is a broad transitioning happening industry- wide from legacy boot to UEFI boot. Many vendors and customers are embracing UEFI boot and have stopped their OS and application certification with legacy boot mode:

• Microsoft requires UEFI Boot on Windows Server 2022 and beyond.
• VMware only certifies UEFI in latest ESXi offerings.
• Intel® has stopped supporting legacy boot with certain platforms such as the latest Intel® Xeon® E-2300 series processors.
• Many new technologies have relegated legacy BIOS, such as PCI Express Gen 5 and NVMe, and the modern data center is transitioning with them.

 Boot Mode

Dell Technologies customers have the option to specify the boot mode (traditional BIOS boot mode or UEFI boot mode) they would like their servers configured to in the Dell Technologies factory. The System Administrator can also choose the boot mode before deploying the server to its operating environment. However, with the introduction of new PowerEdge servers, Intel® Xeon® E-Series and Xeon® D-Series based servers and going forward, only UEFI boot will be supported. Legacy BIOS boot will be deprecated across the portfolio on Dell 17th generation servers including Intel® Xeon-SP and AMD EPYC™ based systems.

The boot mode determines how the system BIOS interacts with adapter card firmware and operating system firmware. Traditionally, the system BIOS performs initialization, boot, system management and configuration tasks. The BIOS initializes the system’s processors, memory, bus controllers, and I/O devices. After initialization is complete, the BIOS passes control to the operating system (OS) software. The OS loader uses basic services provided by the system BIOS to locate and load OS modules into system memory. After booting the system, the BIOS and embedded management controllers execute system management algorithms, which monitor and optimize the condition of underlying hardware. BIOS configuration settings enable fine-tuning of the performance, power management, and reliability features of the system.

Most traditional BIOS implementations do not include mechanisms that verify the integrity of non-BIOS code modules (such as I/O device firmware or operating system loaders). A traditional BIOS may offer protection for the non-volatile memory where the BIOS code is stored, as well as defenses against unauthorized configuration changes. However, these implementations vary between vendors, and risks associated with each implementation may be difficult to assess.

UEFI Boot Mode

The Unified Extensible Firmware Interface (UEFI) does not change the traditional purposes of the system BIOS. To a large extent, a UEFI-compliant BIOS performs the same initialization, boot, configuration, and management tasks as a traditional BIOS. However, UEFI does change the interfaces and data structures the BIOS uses to interact with I/O device firmware and operating system software. The interface consists of data tables that contain platform-related information, plus boot and runtime service calls that are available to the operating system and its loader. Together, these provide a standard, modern environment for booting an operating system and running pre- boot applications. The primary intent of UEFI is to eliminate shortcomings in the traditional BIOS environment, enabling system firmware to continue scaling with industry trends.

Secure Boot Provides Additional Security Checks

To achieve additional boot security, secure boot is a setup option available with UEFI enabled systems, providing additional security checks on the BIOS when loading third-party drivers, OS boot loaders, etc.

 UEFI defines a mechanism, named Secure Boot, which verifies the integrity of each pre-boot code module and allows only authorized code modules to execute. Users configure a Secure Boot policy consisting of X.509 certificates and hash values for both authorized and unauthorized entities. The system BIOS enforces this policy when determining whether to execute pre-boot software including I/O device firmware and operating system loaders.

 UEFI Secure Boot is a technology that eliminates a major security void that may occur during a handoff between the UEFI firmware and UEFI operating system. In UEFI Secure Boot, each component in the chain is validated and authorized against a specific certificate before it is allowed to load or run. Secure Boot removes the legacy threat and provides software identity checking at every step in the boot–Platform firmware, Option ROMs, and OS Bootloader.

The Unified Extensible Firmware Interface (UEFI) Forum – an industry body that develops standards for pre-boot software – defines Secure Boot in the UEFI specification. Computer system vendors, expansion card vendors, and operation system providers collaborate on this specification to promote interoperability. As a portion of the UEFI specification, Secure Boot represents and industry-wide standard for security in the pre-boot environment. When enabled, UEFI Secure Boot prevents the unsigned or compromised UEFI device drivers from being loaded, displays an error message, and does not allow the device to function. You must disable Secure Boot to load the unsigned device drivers.

On Dell EMC PowerEdge servers, 14th generation and later, you can enable or disable the Secure Boot feature through various interfaces. The white paper Secure Boot Management on Dell EMC PowerEdge Servers provides more detail on Secure Boot concepts and how to configure it on Dell EMC PowerEdge servers.

Impact to Environments that Currently Deploy Servers with Legacy Boot

 For customers who have an existing environment with servers configured for legacy boot, the primary impact of switching to UEFI Boot Mode is likely their deployment and maintenance model.

 If you currently have an infrastructure environment where you use PXE servers to deploy and perform routine maintenance, you’ll need to plan for adaption of the PXE server. The whitepaper Boot Mode Considerations: BIOS vs UEFI has more information about this process.

 Another impact is the use of NVMe drives, they are only bootable in UEFI Boot Mode. However, they can be used as data drives in both boot modes.

Conclusion

Cyber-attacks are becoming more numerous, frequent, and more difficult to detect. Implementing a strong security posture by implementing the latest security approaches positions your organization to respond to today’s cyber threat environment. There is a broad industry-wide transition to UEFI boot because of the security features and boot mechanisms available only when the system is configured for UEFI boot mode.

Additional Resources

• Whitepaper: Boot Mode Considerations: BIOS vs UEFI - information on changes needed to support UEFI boot mode in an existing datacenter infrastructure
• Direct from Development: Dell EMC PowerEdge UEFI Secure Boot Customization: Reduce  Attack Surface with Complete Control of Certificates