The following figure demonstrates the physical architecture design of the solution used to run the LLM-based application on Dell APEX Cloud Platform for Red Hat OpenShift.
The following sections describe the various hardware stacks involved in designing this solution.
Dell APEX Cloud Platform for Red Hat OpenShift compute layer setup is configured with four Dell APEX MC-760 nodes for running AI workloads. is a 2U, two-socket fully featured enterprise rack server, designed to optimize even the most demanding workloads like Artificial Intelligence and Machine Learning. It provides a balanced CPU and storage for demanding workloads, powered by Intel Xeon processors and Nvidia GPUs.
Dell APEX MC-760 Servers offer:
For more information about Dell APEX MC-760 servers, see the page. This solution includes Dell APEX MC-760 servers with Intel Xeon Platinum 8462Y+ processors which offer 32 cores per socket, operating at a speed of 2.80 GHz. Additionally, servers are equipped with NVIDIA A2 GPU which offer low power consumption, a small footprint, and high performance. These specifications provide the necessary horsepower to handle the AI workloads.
Dell APEX Cloud Platform for Red Hat OpenShift storage nodes are deployed as a separate cluster and used to provide persistent block storage for OpenShift containers. This separation enables the compute and storage nodes to scale independently. The storage cluster is based on Dell PowerFlex software-defined storage architecture.
Object storage systems, designed for scalable and cost-effective data management, provide an ideal solution for housing massive LLMs and large datasets. Object storage is used in this solution to store Llama 2 model, other datasets, and Red Hat OpenShift AI pipeline artifacts.
Two network switches are being used for data plane connectivity, and two 10 GbE Dell PowerSwitch network switches are being used for management plane connectivity. Dell PowerSwitch enhances performance in data center fabrics of all sizes with efficient and flexible switching solutions.
This node is used to run VMs, and Containers related to client application and tools required to manage the Red Hat OpenShift cluster, Object Storage, and LLM-based application. Also, infra VMs such as AD and DNS are running in this node.