Home > AI Solutions > Gen AI > White Papers > Dell PowerScale Storage Reference Architecture for NVIDIA DGX SuperPOD > Hardware details
The basic building block for NVIDIA DGX SuperPOD is the Scalable Unit (SU). A single SU consists of 32 DGX systems when using InfiniBand for compute fabric (also called East-West networking). Each SU communicates with the storage through the storage fabric (also called North-South networking). This document will focus on the storage fabric. Table 4 and Figure 1 show the different elements that compose the Storage Fabric of the DGX SuperPOD architecture with Dell Ethernet Storage.
The North-South networking between the DGX and storage happens over NVIDIA Spectrum-4 SN5600 switches in a leaf-spine configuration with 2 spine and 4 leaf switches.
A cluster of 16 PowerScale F710 storage nodes running OneFS 9.9. is used. That configuration provides enough network storage performance to meet NVIDIA DGX SuperPOD requirements.
PowerScale storage has a private backend network for PowerScale node-to-node communication. The F710 nodes are connected to this private network using 100 GbE and 2 Dell Z9264-ON switches for high availability.
PowerScale storage is mounted on the DGX cluster using the Dell Multipath Client Driver. Dell Multipath Client Driver allows NFS traffic from the DGX compute nodes make full use of the aggregate performance of the PowerScale storage.
Component | Quantity | Description |
NVIDIA DGX system | 32 | Compute systems |
NVIDIA Spectrum SN5600 Switch | 6 | Switches used for Ethernet storage networking: 2 spine switches 4 leaf switches |
Dell PowerScale F710 Storage nodes | 16 | Storage nodes |
Dell PowerSwitch Z9264-ON switches | 2 | Backend switches used for private PowerScale node to node communication |
MMS4X00-NS | 8 | NVIDIA twin port transceiver, 800Gbps,2xNDR, OSFP, 2xMPO12 APC, 1310nm SMF, up to 100m, finned |
EFALU-PA2S1Q-005M | 8 | 800gb/s to 4x 100Gb/s splitter cable 3rd party vendor SMF |
MMS1V70-CM | 32 | 100gb/s single mode 1-lane (DR1), QSFP28 optical transceiver |
470 ACLU | 32 | Active Optical PowerScale Backend Cable 100GBase QSFP28 (850nm, MMF, 3m) |
The solution components are for a 1 SU |
Note: OneFS 9.9 is the minimum version supported for DGX SuperPOD architecture with Dell Ethernet Storage.