PowerEdge R7625 Rack Server & Emulex: 64G Fibre Channel up to 4:1 Server Virtualization Consolidation
Fri, 29 Mar 2024 16:28:58 -0000
|Read Time: 0 minutes
Dell PowerEdge R7625 Rack Server & Emulex LPe36002 Host Bus Adapter
64G Fibre Channel up to 4:1 Server Virtualization Consolidation
Tolly Report #224105
Tolly test report demonstrating that Dell PowerEdge R7625 Rack Server outfitted with the Emulex LPe36002 64G Fibre Channel adapter can improve virtualization server performance up to 4x vs older generation technologies.
Executive Summary
New generation technology can be expected to improve performance. There are times, however, when multiple technology advances can combine to provide an outsized advantage. Such is the case when the Dell PowerEdge R7625 Rack Server with AMD EPYC processors is combined with the Broadcom Emulex LPe36002 64G Fibre Channel Host Bus Adapter.
Dell commissioned Tolly to benchmark the database performance of the Broadcom Emulex LPe36002 64G Fibre Channel dual-port host bus adapter (HBA) running in the Dell PowerEdge R7625 Rack Server and compare that to the same combined workload performance running in four separate, R740-class servers each outfitted with a 16G FC HBA as was standard with that server generation.
Tests showed that the new R7625 AMD EPYC platform's increased CPU power and improved memory performance/capacity provide an environment where the database application can push the Emulex 64G FC HBA to full line rate performance of 64GFC thus matching the combined application throughput of four R740-class Purley platform servers using 16G FC HBAs. See Figure 1.
The Bottom Line | |
Dell PowerEdge R7625 with AMD EPYC processors & Emulex LPe36002 64GFC HBA benefits over older generation servers with 16GFC HBAs: | |
1 | 1x R7625 with 64GFC HBA can achieve the same VM “Boot Storm” throughput as 4x R740-class servers with 16GFC HBAs |
2 | Per-VM startup time improvement of 76% |
Overview
The goal of this test was to illustrate, simply, that a single Dell PowerEdge R7625 Rack Server using a single port of a PCIe 4.0-based, dual-port Emulex 64GFC HBA can equal the I/O throughput of four individual, older generation, R740-class servers each using a single port of a 16GFC HBA.
The Dell PowerEdge R740-class servers use older, less powerful CPUs and use 16GFC HBAs that offer, at best, 25% of the 64GFC HBA’s throughput. The HBAs are constrained by the bandwidth of the PCIe 3.0 bus architecture which would limit the benefits of using the higher FC speed HBAs in the older servers.
The broader point is that this significant performance improvement means that, for server virtualization applications, a single Dell PowerEdge R7625 Rack Server can be used to replace and consolidate the workloads and operating expenses of up to four older servers.
Test Background & Results
Server virtualization is an important part of IT infrastructure for countless businesses and organizations worldwide. Efficient use of the underlying server hardware components is an important aspect of providing high quality end-user experience while controlling costs. Certain elements of server virtualization can place a tremendous load on I/O resources. In particular, “Boot Storms” can be impacted severely by lack of sufficient I/O bandwidth. The scenario was run separately on a single Dell PowerEdge R7625 server, powered by AMD EPYC, outfitted with a 64GFC HBA and then, again, simultaneously on four R740-class servers each outfitted with a 16GFC HBA.
“Boot Storm”
This is an informal term applied to situations where multiple VMs are started simultaneously. During the boot process all of the VMs use a workspace profile that will, upon startup, load a standard set of applications and read initial data from the data store simultaneously thus creating the "storm" of I/O requests.
The test was run using two different scenarios. In the first scenario, four of the older servers each booted six VMs simultaneously against the same Dell data store. In the second scenario, the single Dell PowerEdge R7625 booted 24 VMs simultaneously against the same data store.
Figure 1, on the cover page, summarizes the results of the “Boot Storm” tests in terms of storage I/O and startup (boot) time. The I/O throughput difference between the single 64GFC server and the four 16GFC servers is apparent. Where each of the older servers delivers throughput of ~1,600MB/s, the 64GFC server throughput was measured at ~6,400MB/s.
This increase in throughput on the 64GFC Dell PowerEdge R7625 server results in dramatically faster boot times for each of the 24VMs tested. As shown in the the figure, the average, per-VM boot time for VMs running on the R740-class systems was 31.54. The the average, per-VM boot time for VMs running on the R7625 system was 7.58s. This represents an improvement of 76%.
Test Setup & Methodology
The HBA under test used current production drivers that are publicly available. Default settings were used. Details of the test environment and systems under test are found in Tables 1-9. Figure 2 shows a composite test environment.
As tests involved basic functions of VMware, no detailed test methodology is required.
Table 1. 64G HBA Under Test
Vendor | Product Name | Bus Architecture | Firmware | Driver |
Broadcom | Emulex LPe36002 | PCIe 4.0 | 14.2.455.15 | 14.2.560.8 |
Table 2. R7625 Server Configuration
Vendor/System | Dell PowerEdge R7625 |
CPU | 2 socket AMD EPYC 9374F 32-core processor @ 3.8 GHz |
Number of CPUs | 128 logical processors. Profile: Performance, Logical Processors: Enabled, Sub Numa Clustering: Disabled |
Memory (RAM) | 384 GB |
OS | VMware ESXi-8 |
Table 3. VMware Configuration
VMware OS | RHEL 8.9 |
Storage/Controller | Storage volumes mapped to VM as thick provisioned, eagerly zeroed |
VM RAM | 15GB |
VM vCPU | 6 |
“Boot Storm” Settings | Total VMs: 24. R7625 ran 24 VMs, each R740 ran 6 VMs |
Table 4. Storage Configuration
Vendor/Device | Dell PowerStore 9200T v3.5.0.0 |
Ports | 8 x 32G FC |
Performance Policy | High |
Namespace/LUN | 8 x 32G Target ports per Namespace (single namespace) |
Namespaces | 24 namespaces, each 500GB
|
Network Fabric | Dell Connectrix DS7720B 64GFC Switch v9.0.1a
|
Table 5. 16G HBA Under Test
Vendor | Product Name | Bus Architecture | Firmware | Driver |
Broadcom | LPe31002 | PCIe 3.0 | 14.2.455.11 | 14.2.560.8 |
Table 6. R740 Class Server Configuration Host 1
CPU | 2 socket Intel(R) Xeon(R) Gold 6146 @ 3.2GHz |
Number of CPUs | 24 |
Memory (RAM) | 128 GB |
Table 7. R740 Class Server Configuration Host 2
CPU | 2 socket Intel(R) Xeon(R) Platinum 8176 @ 2.10GHz |
Number of CPUs | 56 |
Memory (RAM) | 128 GB |
Table 8. R740 Class Server Configuration Host 3
CPU | 2 socket Intel(R) Xeon(R) Platinum 8176 @ 2.10GHz |
Number of CPUs | 56 |
Memory (RAM) | 128 GB |
Table 9. R740 Class Server Configuration Host 4
CPU | 2 socket Intel(R) Xeon(R) Gold 6148 @ 2.40GHz |
Number of CPUs | 40 |
Memory (RAM) | 128 GB |
About AMD
For over 50 years, AMD has been at the forefront of driving innovation in high-performance computing, graphics, and visualization technologies. Their products are relied upon by billions of people, leading Fortune 500 businesses, and cutting-edge scientific research institutions worldwide. AMD's mission is to build exceptional products that accelerate next-generation computing experiences and power solutions for the world's most important challenges. Visit http://www.amd.com for more information about AMD.
Broadcom Emulex LPe36002
The Broadcom Emulex LPe36000-series Gen 7 Fibre Channel HBAs are designed for demanding mission-critical workloads and emerging applications. The family of adapters features Silicon Root of Trust security, designed to thwart firmware attacks aimed at enterprises and governments.
Gen 7 64G provides seamless backward compatibility to 32G and 16G networks.
Dell sells the LPe36002 64G HBA for the same price as the 32G model.
About Tolly
The Tolly Group companies have been delivering world-class IT services for over 30 years. Tolly is a leading global provider of third-party validation services for vendors of IT products, components and services.
You can reach the company by E-mail at sales@tolly.com, or by telephone at +1 561.391.5610.
Visit Tolly on the Internet at: http://www.tolly.com
Tolly Terms Of Usage
The Tolly Gro This document is provided, free-of-charge, to help you understand whether a given product, technology, or service merits additional investigation for your particular needs. Any decision to purchase a product must be based on your own assessment of suitability based on your needs. The document should never be used as a substitute for advice from a qualified IT or business professional. This evaluation was focused on illustrating specific features and/or performance of the product(s) and was conducted under controlled, laboratory conditions. Certain tests January have been tailored to reflect performance under ideal conditions; performance January vary under real-world conditions. Users should run tests based on their own real-world scenarios to validate performance for their own networks.
Reasonable efforts were made to ensure the accuracy of the data contained herein but errors and/or oversights can occur. The test/audit documented herein January also rely on various test tools the accuracy of which is beyond our control. Furthermore, the document relies on certain representations by the sponsor that are beyond our control to verify. Among these is that the software/hardware tested is production or production track and is, or will be, available in equivalent or better form to commercial customers. Accordingly, this document is provided "as is," and Tolly Enterprises, LLC (Tolly) gives no warranty, representation or undertaking, whether express or implied, and accepts no legal responsibility, whether direct or indirect, for the accuracy, completeness, usefulness, or suitability of any information contained herein. By reviewing this document, you agree that your use of any information contained herein is at your own risk, and you accept all risks and responsibility for losses, damages, costs, and other consequences resulting directly or indirectly from any information or material available on it. Tolly is not responsible for, and you agree to hold Tolly and its related affiliates harmless from any loss, harm, injury, or damage resulting from or arising out of your use of or reliance on any of the information provided herein.
Tolly makes no claim as to whether any product or company described herein is suitable for investment. You should obtain your own independent professional advice, whether legal, accounting or otherwise, before proceeding with any investment or project related to any information, products or companies described herein. When foreign translations exist, the English document is considered authoritative. To assure accuracy, only use documents downloaded directly from Tolly.com. No part of any document January be reproduced, in whole or in part, without the specific written permission of Tolly. All trademarks used in the document are owned by their respective owners. You agree not to use any trademark in or as the whole or part of your own trademarks in connection with any activities, products or services which are not ours, or in a manner which January be confusing, misleading, or deceptive or in a manner that disparages us or our information, projects or developments.