Home > Workload Solutions > High Performance Computing > White Papers > HPC Software-Defined Storage with PixStor > Overview
Table 1 has the list of main components for the solution. The first description column has the components available to customers with the solution, and the last column is the components actually used for characterizing the performance of the solution. The drives listed for data (12TB NLS) and metadata (960Gb SSD), are the ones used for performance characterization, but all supported HDDS and SSDs in the ME4 Support Matrix could be used for the solution.
When ME4 controllers are the bottleneck of the solution, then using faster drives (10K, 15K and SSDs) are not expected to provide more sequential throughput, but they can provide better random IOPs and may improve create/removal metadata operations. For full high-speed network redundancy, two high-speed switches must be used — the QM87000 for IB and SN3700 for GbE — therefore each switch should have one CX6 server network interface card connected from each server.
Software components listed in Table 1 describe versions during the initial testing and planned to be used at release time. However, these software versions may change over time in between official releases to include important bug fixes, support for new hardware components, or add important new features.
Finally, for completeness, the list of possible data HDDs and metadata SSDs was included, which is based on the drives supported as enumerated on the Dell EMC PowerVault ME4 support matrix, available online.
Solution Component |
At Release |
Test Bed |
|
Optional high-demand metadata storage subsystem |
From 1x up to 4x Dell EMC PowerVault ME4024 (4x ME4024 for Large config only) |
||
Raid storage controllers |
Duplex 12 Gbps SAS |
||
Capacity w/o expansion (with 12TB HDDS) |
Raw: 4032 TB (3667 TiB or 3.58 PiB) |
||
Capacity w/ expansion (12 TB HDDS) |
Raw: 8064 TB (7334 TiB or 7.16 PiB) |
||
Processor |
Gateway/Ngenea |
2x Intel Xeon Gold 6326 2.9 GHz, 16C/32T, 11.2GT/s, 24M Cache , Turbo, HT (185W) DDR4-3200 |
2x Intel Xeon Platinum 8352Y |
High-demand metadata |
|||
Storage node |
|||
Management node |
2x Intel Xeon Gold 6330 2GHz, 28C/56T, 11.2GT/s, 42M Cache, Turbo, HT (185W) DDR4-2933 |
||
Memory |
Gateway/Ngenea |
16 x 16GiB 3200 MT/s RDIMMs (256 GiB) |
|
High-demand metadata |
|||
Storage node |
|||
Management node |
|||
Operating system |
Red Hat Enterprise Linux 8.4 |
||
Kernel version |
4.18.0-305.12.1.el8_4.x86_64 |
||
PixStor software |
6.0.0.0 |
||
Spectrum scale (GPFS) |
5.1.1-2 |
||
OFED version |
MLNX_OFED_LINUX-5.4-1.0.3.0 |
||
High-performance NIC |
All: 2x Dell OEM ConnectX-6 Single Port HDR VPI InfiniBand PCIe Low Profile |
||
High-performance switch |
2x Mellanox QM8700 for IN HDR or 2x SN3700 for 200/200 GbE |
||
Local Disks (OS and analysis/monitoring) |
3x 480GB SSD SAS3 (RAID1 + HS) for OS |
||
Systems management |
iDRAC 9 Enterprise + Dell EMC OpenManage 10.0.1-4517 |