Unlocking Possibilities: Dell’s XE9680 with Intel Gaudi3 Accelerator
Fri, 02 Aug 2024 16:45:12 -0000
|Read Time: 0 minutes
The demand for AI inferencing and training applications continues to surge, in line with IDC’s projection that IT infrastructure spending on AI semiconductors will reach $193.3 billion by the end of 2027, with a 5-year CAGR of 35.7%.
Dell’s latest addition to their portfolio, the Dell XE9680 with Intel Gaudi 3 accelerator, addresses customer challenges related to GPU options, open-source software stacks, networking, scalability, infrastructure costs, and efficiency. This powerful solution aims to help customers tackle complex problems.
XE9680 is powered by two Intel 4th or 5th Gen Xeon Scalable processors, including 32 DDR5 memory DIMM slots for high-speed memory capacity, providing support up to 16 EDSFF3 storage drives or 8xSAS/SATA/NVMe storage drives and eight PCIe Gen 5.0 slots for expanded connectivity and bandwidth. It has dedicated OSFP ports for direct GPU connectivity.
Figure 1: Dell PowerEdge XE9680
The Intel Gaudi3 AI accelerator brings competitive Gen AI performance with 64 custom and programmable tensor processor cores (TPCs), 128 GB of HBMe2 memory capacity, and 96 MB of onboard static random-access memory (SRAM). A key differentiator is the flexible on-chip networking option with 24x200GbE RDMA NIC for each GPU card supporting industry standard RoCE Ethernet ports. It also includes specialized media decoders designed for AI vision applications.
Figure 2: Intel Gaudi3 Accelerator
The Dell PowerEdge XE9680, powered by Intel’s Gaudi3, offers a robust platform for AI-ML workloads. It excels in training and inferencing for LLMs, vLLMs, Conversational AI, Chatbots, and video-to-text generation.
Stay tuned for more performance measurement results on the Dell PowerEdge XE9680 with Gaudi 3.