PowerEdge R7625 Rack Server & Emulex: 64G Fibre Channel Enables up to 4:1 Application Server Consolidation
Fri, 29 Mar 2024 16:19:02 -0000
|Read Time: 0 minutes
Dell PowerEdge R7625 Rack Server & Emulex LPe36002 Host Bus Adapter
64G Fibre Channel Enables up to 4:1 Application Server Consolidation
Tolly Report #224104
Tolly test report demonstrating that Dell PowerEdge R7625 Rack Server outfitted with the Emulex LPe36002 64G Fibre Channel adapter can improve application performance up to 4x vs older generation technologies.
Executive Summary
New generation technology can be expected to improve performance. There are times, however, when multiple technology advances can combine to provide an outsized advantage. Such is the case when the Dell PowerEdge R7625 Rack Server with AMD EPYC processors is combined with the Broadcom Emulex LPe36002 64G Fibre Channel Host Bus Adapter.
Dell commissioned Tolly to benchmark the database performance of the Broadcom Emulex LPe36002 64G Fibre Channel dual-port host bus adapter (HBA) running in the Dell PowerEdge R7625 Rack Server and compare that to the same combined workload performance running in four separate, R740-class servers each outfitted with a 16G FC HBA as was standard with that server generation.
Tests showed that the new R7625 AMD EPYC platform's increased CPU power and improved memory performance/capacity provide an environment where the database application can push the Emulex 64G FC HBA to full line rate performance of 64GFC thus matching the combined application throughput of four R740-class Purley platform servers using 16G FC HBAs. See Figure 1.
The Bottom Line | |
Dell PowerEdge R7625 with AMD EPYC processors & Emulex LPe36002 64G HBA benefits over older generation with 16G HBAs: | |
1 | 1x R7625 with 64GFC HBA can achieve same TPROC-H query throughput compared to 4x R740-class servers with 16GFC HBA |
2 | Consolidating Oracle DSS workloads from 4 R740 servers with 16GFC HBA to a single R7625 with 64GFC can significantly reduce I/O bound TPROC-H query time |
Overview
The goal of this test was to illustrate, simply, that a single Dell PowerEdge R7625 Rack Server using a single port of a PCIe 4.0-based, dual-port Emulex 64G FC can equal the I/O throughput of four individual, older generation, R740-class servers each using a single port of a 16G FC HBA.
The R740-class servers use older, less powerful CPUs and use 16G FC HBAs that offer, at best, 25% of the 64G FC HBA’s throughput. The HBAs are constrained by the bandwidth of the PCIe 3.0 bus architecture which would limit the benefits of using the higher FC speed HBAs in the older servers.
The broader point is that this significant performance improvement means that, for I/O-bound applications, a single Dell PowerEdge R7625 Rack Server can be used to replace and consolidate the workloads and operating expenses of up to four older servers.
Test Background & Results
The same test was run on all of the servers and consisted of running the TPROC-H analytics workload of HammerDB. The tests were run using the Oracle 19c database environment, but the results are generally applicable to any database or other input/output intensive workload.
The TPROC-H workload measures how long it takes to run a series of 22 different types of decision support queries. This type of workload is “read only” with no database updates taking place.
The test was run using two different scenarios. In the first scenario, four of the older servers ran the HammerDB benchmark simultaneously against the same Dell data store. In the second scenario, the single Dell PowerEdge R7625 ran the benchmark against the same data store.
Figure 1, above the horizontal dividing line, summarizes results of the first scenario. Because those servers were using 16G FC HBAs, 16G was the theoretical maximum for network I/O and, thus a potential bottleneck for each server. As each server finished the test, the reduced load on the target data store allowed subsequent server’s tests to run more quickly. The fastest completion time was 335 seconds and the slowest was 448 seconds with the average being 405.5 seconds.
Figure 1, below the horizontal dividing line, summarizes results of the second scenario. Here, a single Dell PowerEdge R7625 Rack Server outfitted with an Emulex 64G FC HBA was able to complete the same test in 99 seconds. This illustrates that the R7625 could take on the full load of four servers running this type of workload.
Figure 2 shows the results of the same two scenarios overlaid and measured in terms of disk I/O over the course of the tests. The red dots represent the combined disk I/O of all four older generation servers. The blue dots represent the single Dell PowerEdge R7625 Rack Server, powered by AMD EPYC processors. The disk throughput of the single R7625 at 64G matches or exceeds the combined throughput of the four 16G servers.
Figure 3, below, illustrates the networking flow of the four older generation servers, in blue, and the Dell PowerEdge R7625, in red, across the Broadcom Brocade 64G Fibre Channel switch.
Test Setup & Methodology
The HBA under test used current production drivers that are publicly available. Default settings were used. Details of the test environment and systems under test are found in Tables 1-10. Figure 3 shows a composite test environment.
Server systems were all VMware ESXi 8 hosts running ESXi-8.0U1-21495797 (8U2 GA). Storage volumes mapped to each VM were configured as thick provisioned, eagerly zeroed. PVSCSI controller was used.
Each VM was assigned 100GB of memory and 40 vCPUs. Each VM was running RHEL 8.8
Details of the HammerDB tests are found in the “Test Background & Results” section above.
Table 1. 64G HBA Under Test
Vendor | Product Name | Bus Architecture | Firmware | Driver |
Broadcom | Emulex LPe36002 | PCIe 4.0 | 14.2.455.15 | 14.2.560.8 |
Table 2. R7625 Server Configuration
Vendor/System | Dell PowerEdge R7625 |
CPU | 2 socket AMD EPYC 9374F 32-core processor @ 3.8 GHz |
Number of CPUs | 64 physical, 128 logical |
Memory (RAM) | 384 GB |
OS | VMware ESXi 8 |
Guest OS | RHEL 8.9 |
Table 3. Database Test Tool
Vendor | Open Source |
Application | HammerDB 4.7 |
TPROC-H settings | Degree of parallelism = 32 Scale factor = 30 Virtual users = 1 Ramp-up time: 2 minutes Run time: 5 minutes |
Table 4. Oracle Database Configuration
Database | Oracle Database 19c (19.3) |
Storage | Oracle Grid 19c, ASM disk group with external redundancy, 1 namespace for data |
Dataset Size | 40GB |
Database Settings | SGA = 12000 MB |
Table 5. Storage Configuration
Vendor/Device | Dell PowerStore 9200T v3.2.0.1 |
Ports | 8 x 32G FC |
Volumes | 2 x NVMe: 200 GB and 1 TB |
Performance Policy | High |
Namespace/LUN | 8 x 32G Target ports per Namespace |
Network Fabric | Dell Connectrix 64G FC Switch v9.0.1.a |
Table 6. 16G HBA Under Test
Vendor | Product Name | Bus Architecture | Firmware | Driver |
Broadcom | LPe31002 | PCIe 3.0 | 14.2.455.11 | 14.2.560.8 |
Table 7. R740 Class Server Configuration Host 1
CPU | 2 socket Intel(R) Xeon(R) Gold 6146 @ 3.2GHz |
Number of CPUs | 24 |
Memory (RAM) | 128 GB |
Table 8. R740 Class Server Configuration Host 2
CPU | 2 socket Intel(R) Xeon(R) Platinum 8176 @ 2.10GHz |
Number of CPUs | 56 |
Memory (RAM) | 128 GB |
Table 9. R740 Class Server Configuration Host 3
CPU | 2 socket Intel(R) Xeon(R) Platinum 8176 @ 2.10GHz |
Number of CPUs | 56 |
Memory (RAM) | 128 GB |
Table 10. R740 Class Server Configuration Host 4
CPU | 2 socket Intel(R) Xeon(R) Gold 6148 @ 2.40GHz |
Number of CPUs | 40 |
Memory (RAM) | 128 GB |
About AMD
For over 50 years, AMD has been at the forefront of driving innovation in high-performance computing, graphics, and visualization technologies. Their products are relied upon by billions of people, leading Fortune 500 businesses, and cutting-edge scientific research institutions worldwide. AMD's mission is to build exceptional products that accelerate next-generation computing experiences and power solutions for the world's most important challenges. Visit http://www.amd.com for more information about AMD.
Broadcom Emulex LPe36002
The Broadcom Emulex LPe36000-series Gen 7 Fibre Channel HBAs are designed for demanding mission-critical workloads and emerging applications. The family of adapters features Silicon Root of Trust security, designed to thwart firmware attacks aimed at enterprises and governments.
Gen 7 64G provides seamless backward compatibility to 32G and 16G networks.
Dell sells the LPe36002 64G HBA for the same price as the 32G model.
About Tolly
The Tolly Group companies have been delivering world-class IT services for over 30 years. Tolly is a leading global provider of third-party validation services for vendors of IT products, components and services.
You can reach the company by E-mail at sales@tolly.com, or by telephone at +1 561.391.5610.
Visit Tolly on the Internet at: http://www.tolly.com
Tolly Terms Of Usage
The Tolly Gro This document is provided, free-of-charge, to help you understand whether a given product, technology, or service merits additional investigation for your particular needs. Any decision to purchase a product must be based on your own assessment of suitability based on your needs. The document should never be used as a substitute for advice from a qualified IT or business professional. This evaluation was focused on illustrating specific features and/or performance of the product(s) and was conducted under controlled, laboratory conditions. Certain tests January have been tailored to reflect performance under ideal conditions; performance January vary under real-world conditions. Users should run tests based on their own real-world scenarios to validate performance for their own networks.
Reasonable efforts were made to ensure the accuracy of the data contained herein but errors and/or oversights can occur. The test/audit documented herein January also rely on various test tools the accuracy of which is beyond our control. Furthermore, the document relies on certain representations by the sponsor that are beyond our control to verify. Among these is that the software/hardware tested is production or production track and is, or will be, available in equivalent or better form to commercial customers. Accordingly, this document is provided "as is," and Tolly Enterprises, LLC (Tolly) gives no warranty, representation or undertaking, whether express or implied, and accepts no legal responsibility, whether direct or indirect, for the accuracy, completeness, usefulness, or suitability of any information contained herein. By reviewing this document, you agree that your use of any information contained herein is at your own risk, and you accept all risks and responsibility for losses, damages, costs, and other consequences resulting directly or indirectly from any information or material available on it. Tolly is not responsible for, and you agree to hold Tolly and its related affiliates harmless from any loss, harm, injury, or damage resulting from or arising out of your use of or reliance on any of the information provided herein.
Tolly makes no claim as to whether any product or company described herein is suitable for investment. You should obtain your own independent professional advice, whether legal, accounting or otherwise, before proceeding with any investment or project related to any information, products or companies described herein. When foreign translations exist, the English document is considered authoritative. To assure accuracy, only use documents downloaded directly from Tolly.com. No part of any document January be reproduced, in whole or in part, without the specific written permission of Tolly. All trademarks used in the document are owned by their respective owners. You agree not to use any trademark in or as the whole or part of your own trademarks in connection with any activities, products or services which are not ours, or in a manner which January be confusing, misleading, or deceptive or in a manner that disparages us or our information, projects or developments.