Selecting the appropriate server and network configuration for generative AI model customization is crucial to ensure adequate resources are allocated for model training. This section provides example configurations for both management and compute workloads and network architecture.
The following sections describe the management server configuration.
The following table provides the recommended minimum configuration for the management head node and the control plane node:
Head node and control plane nodes
3 x PowerEdge R660
1 x Intel Xeon Gold 6426Y 2.5G, 16C/32T
8 x 16 GB DDR5 4800 MT/s RDIMM
PERC H755 with rear load Brackets
4 x 960 GB SSD vSAS Read Intensive 12 Gbps 512e 2.5in Hot-Plug, AG Drive SED, 1DWPD
1 x Broadcom 5720 Dual Port 1 GbE Optional LOM
1 x NVIDIA ConnectX-6 Lx Dual Port 10/25 GbE SFP28, OCP NIC 3.0
Kubernetes/storage network (optional)
1 x NVIDIA ConnectX-6 Lx Dual Port 10/25 GbE SFP28 Adapter, PCIe
InfiniBand network (Optional)
1 x NVIDIA ConnectX-7 Single Port NDR OSFP PCIe, No Crypto, Full Height or
1 x NVIDIA ConnectX-6 Single Port HDR200 VPI InfiniBand Adapter PCIe
Consider the following recommendations for head and control plane node configuration:
PowerEdge XE9680 servers can be configured as worker nodes for Generative AI model customization. The following table provides a recommended configuration for a PowerEdge XE9680 GPU worker node:
PowerEdge XE9680 (minimum of 2)
2 x Intel Xeon Platinum 8468 2.1G, 48 C/96 T, 16 GT/s
16 x 64 GB RDIMM, 4800 MT/s Dual Rank
2 x 1.92 TB Enterprise NVMe Read Intensive AG Drive U.2 Gen4 with carrier
Broadcom 5720 Dual Port 1 GbE Optional LOM
1 x Intel E810-XXV Dual Port 10/25GbE SFP28, OCP NIC 3.0
2 x NVIDIA ConnectX-6 DX Dual Port 100 GbE QSFP56 Network Adapter
8 x NVIDIA H100 SXM
8 x NVIDIA ConnectX-7 Single Port NDR OSFP PCIe, No Crypto, Full Height or
8 x Mellanox ConnectX-6 Single Port HDR200 VPI InfiniBand Adapter PCIe
Internally, each XE9680 has four PCIe switches. Since two GPUs are connected to each PCIe switch, for maximum throughput and performance, each PCIe switch has been subdivided into two virtual switches. Therefore, for optimal GPU network performance, we recommend that each GPU has a dedicated network adapter. We recommend four InfiniBand adapters for the XE8640 and eight InfiniBand adapters for the XE9680.
While LLM tasks primarily rely on GPUs and do not significantly tax the CPU and memory, it is advisable to equip the system with high-performance CPUs and larger memory capacities. This provisioning ensures sufficient headroom for various data processing activities, machine learning operations, monitoring, and logging tasks. Our goal is to guarantee that the servers boast ample CPU and memory resources for these functions, preventing any potential disruptions to the critical AI operations carried out on the GPUs.
The following figure shows the network architecture. It shows the network connectivity for compute servers. The figure also shows three PowerEdge head nodes, which incorporate NVIDIA Base Command Manager Essentials and Kubernetes control plane nodes.
This validated design requires the following networks to manage the cluster and facilitate communication and coordination between different components and nodes in the cluster:
The following figure shows an example rack design for two PowerEdge XE9680 servers, equivalent to the configuration shown in Figure 5:
This rack was created by using the Dell Enterprise Infrastructure Planning Tool (with the illustrations of the switches enhanced). You can use the tool to configure your solution and receive weight, power requirement, airflow, and other details.
This example shows two servers in a single rack and requires two 17kW Power Distribution Units (PDUs). However, customers must carefully evaluate their own power and cooling requirements and their preference for rack layout, power distribution, airflow management, and cabling design.
Where significant growth is anticipated in the size of the deployment, customers should consider separate racks for compute, storage, and management nodes to allow sufficient capacity for that growth.
The following table provides the APC Rack and PDU recommendations for the Americas region. We recommend that you consult your Dell or APC representative to understand your unique data center requirements to provide an accurate PDU recommendation.
Table 5. Example Rack and PDU recommendations for the PowerEdge XE9680 server
Servers per cabinet
Rack U height
APC Rack Model
APC PDU model