Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Design Guide—Generative AI in the Enterprise – Model Customization
Introduction Customizing Large Language Models Solution Components
Overivew
Model customization using NVIDIA AI Enterprise
Dell PowerEdge servers and NVIDIA GPUs
Dell PowerScale Storage
Dell and NVIDIA networking
Solution Architecture Validation Results Performance Characterization Dell Professional Services for Generative AI Conclusion References Sample Scripts

Dell and NVIDIA networking

Dell and NVIDIA networking

  • Dell PowerSwitch

    The Dell PowerSwitch Z-series are high-performance, open, and scalable data center switches used for spine, core, and aggregation applications. 

    The PowerSwitch Z9432F-ON 100/400 GbE fixed switch consists of Dell Technologies’ latest disaggregated hardware and software data center networking solutions, providing state-of-the-art, high-density 100/400 GbE ports and a broad range of functionality to meet the growing demands of today’s data center environment. This innovative, next-generation open networking high-density aggregation switch offers optimum flexibility and cost-effectiveness for enterprise, midmarket, and cloud service providers with demanding compute and storage traffic environments.

    The Z9432F-ON switch is highly scalable with an optimized leaf/spine architecture that supports data centers of virtually any size, with up to over 100,000 servers. And it has exceptionally high density, providing 32 400GbE and 128 100GbE ports in a single switch.

    NVIDIA Quantum Networking Switches

    Deploying generative AI applications or training foundational AI models like Llama 2 can be computationally demanding, particularly for larger and more complex models. To address this demand, distributed computing is used to distribute the workload across multiple interconnected compute nodes, where the slowest node determines the runtime of a task. The network's role is critical in ensuring timely message delivery to all nodes, making tail latency significant, especially in large-scale data centers with competing workloads. Scalability and handling a growing number of nodes are essential for training large AI models and managing extensive data.

    Lossless networking, such as InfiniBand, ensures that data is transmitted without any loss or corruption, guaranteeing the accuracy of all data packets. Remote Direct Memory Access (RDMA) further enhances data transfer efficiency by allowing direct, high-speed, low-latency transfers between memory, GPUs, and storage, bypassing the traditional multistep data transfer process.

    InfiniBand Next Data Rate (NDR) and High Data Rate (HDR) are generations of high-speed interconnect technologies used in high-performance computing and data center environments. NDR offers a data transfer rate of 400 Gbps, while HDR provides 200 Gbps.

    NVIDIA Quantum InfiniBand switches high throughput, In-Network Computing, smart acceleration engines, flexibility, and a robust architecture to achieve high performance in generative AI computing. NVIDIA Quantum QM8700 InfiniBand Series support HDR, while Quantum QM9700 series supports NDR. This design was validated with both the series of switches.