Home > Workload Solutions > High Performance Computing > White Papers > Dell Validated Design for HPC NG-Stor Storage - Joint Solution with Kalray > Solution introduction
In high-performance computing (HPC), all its different components must be balanced for optimal performance and to avoid bottlenecks. Technological evolution of compute node components like higher core-count CPUs, larger and faster memory, faster PCIe buses, and increasingly faster networks caused storage to become the bottleneck in many workloads. Parallel File Systems (PFS) can scale to meet greater performance demands. PFS provides concurrent access to a single file or a set of files from multiple nodes, efficiently and securely distributing data to multiple Logical Unit Numbers (LUNs) across several storage servers and storage arrays. These PFS normally use spinning disk media to provide the highest capacity at the lowest cost. However, the speed and latency of spinning media often cannot keep up with the demands of many modern HPC workloads. Recent rapid advances in storage technologies like Non-Volatile Memory Express (NVMe) SSDs, make better and faster options available to PFS storage systems. The use of this NVMe technology in the form of burst buffers, faster tiers, or fast scratch (local or distributed) can mitigate or eliminate spinning media issues.
A well-balanced storage system is required to achieve optimal performance. Its components such as back-end arrays, storage controllers, disk drives, IO cards, network adapters, and switches must be able to provide similar bandwidth and distribute the load among the processors and memory of the servers they use. The PFS and supporting software must be able to balance the load on the different storage system components, distribute data among them, monitor performance and the health of the components, and provide administration tools to manage the solution efficiently.
The Dell Validated Design for HPC NG-Stor[1] Storage is a highly available (HA) solution. It offers a cost-effective, high-capacity tier based on Dell PowerEdge R660, R760, and R7625 servers and PowerVault ME5084 storage arrays arranged in ME5 storage modules and NVMe nodes to address high-bandwidth demands. Each ME5 storage module can scale up its capacity to 13.12 PiB of formatted and available storage (raw capacity 14,784 TB, using 22 TB HDDs and ME484 arrays). The module can scale out to customer requirements by adding more modules up to the required capacity and performance. For extreme performance, the NVMe tier can scale out performance by adding pairs of HA nodes with the latest NVMe devices using replication among the server pairs. For situations that require high throughput or metadata demands, an NVMe pair can be used, scaling up the number of files with larger capacity devices, and scaling out performance by adding more NVMe metadata pairs.
The NG-Stor storage uses the well-established General Parallel File System (GPFS), also known as Spectrum Scale. It includes:
This storage solution is fully supported (hardware and software) with deployment services and offers a single namespace, multi-tiered, easy-to-use, high throughput, among other features.
[1] Kalray rebranded pixstor as NG-Stor and ngenea as NG-HUB.