The following tables list the system configurations and software stack used for validation in this design.
Newer versions of software might be available after the release of the software. We recommend that you update the cluster to the latest supported firmware and software releases.
Table 6. System configuration
Component | Configuration |
Management servers | 3 x PowerEdge R660 |
Compute server | 4 x PowerEdge XE9680 each with 8 AMD MI300X (see Table 3 for server configuration details) |
Frontend (north-south) network | 2 x PowerSwitch Z9432F-ON |
Backend (east-west/GPU) network | 1 x PowerSwitch Z9664F-ON |
Storage | PowerScale F710 |
Table 7. Software components and versions
Operating system | Ubuntu 22.04 LTS |
XE9680 BIOS | 2.0.3 |
iDRAC9 | 7.10.50.07 |
AMD ROCm | 6.1.2 |
AMD MI300X FW package | 24.10 |
Broadcom 57608 FW | 22.92.07.20 |
Broadcom 57608 driver | 229.2.49.0 |
Cluster management | Omnia 1.6 |
Kubernetes cluster | Kubernetes 1.26.12 |
SONiC version on the switches | 4.2.1 |
OpenMPI | 4.1.6 |
UCX | 1.15.0 |
The following validation steps use Docker containers. These steps can be easily extended to run on Kubernetes by building corresponding manifest files and running the containers as pods in the Kubernetes cluster.
