Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Design Guide—Implementing a Digital Assistant with Red Hat OpenShift AI on Dell APEX Cloud Platform
Introduction Solution Concepts Solution Architecture and Requirements
Physical architecture
Logical architecture
Digital assistant design
Solution requirements
Solution Deployment Solution execution Summary References Open-Source License

Physical architecture

Home > AI Solutions > Artificial Intelligence > Guides > Design Guide—Implementing a Digital Assistant with Red Hat OpenShift AI on Dell APEX Cloud Platform > Physical architecture

Physical architecture

  • Overview


    The following figure demonstrates the physical architecture design of the solution used to run the LLM-based application on Dell APEX Cloud Platform for Red Hat OpenShift.

    Figure 4.   Dell APEX Cloud Platform for Red Hat OpenShift physical architecture

    The following sections describe the various hardware stacks involved in designing this solution.

    Dell servers

    Dell APEX Cloud Platform for Red Hat OpenShift compute layer setup is configured with four Dell APEX MC-760 nodes for running AI workloads. Dell APEX MC-760 server is a 2U, two-socket fully featured enterprise rack server, designed to optimize even the most demanding workloads like Artificial Intelligence and Machine Learning. It provides a balanced CPU and storage for demanding workloads, powered by Intel Xeon processors and Nvidia GPUs.

    Dell APEX MC-760 Servers offer:

    • Up to two Next Generation Intel Xeon Scalable processors with up to 56 cores for faster and more accurate processing performance.
    • Accelerate in-memory workloads with up to 32 DDR5 RDIMMS up to 4400 MT/sec (2DPC) or 4800 MT/sec for 1DPC (16 DDR5 RDIMMs max).
    • Support for GPUs including 2 x double-wide or 6 x single-wide for workloads requiring acceleration.

    For more information about Dell APEX MC-760 servers, see the Dell APEX Cloud Platform MC-760 Hardware Requirements and Specifications page. This solution includes Dell APEX MC-760 servers with Intel Xeon Platinum 8462Y+ processors which offer 32 cores per socket, operating at a speed of 2.80 GHz. Additionally, servers are equipped with NVIDIA A2 GPU which offer low power consumption, a small footprint, and high performance. These specifications provide the necessary horsepower to handle the AI workloads.

    Storage 

    Dell APEX Cloud Platform for Red Hat OpenShift storage nodes are deployed as a separate cluster and used to provide persistent block storage for OpenShift containers. This separation enables the compute and storage nodes to scale independently. The storage cluster is based on Dell PowerFlex software-defined storage architecture.

    Object storage systems, designed for scalable and cost-effective data management, provide an ideal solution for housing massive LLMs and large datasets. Object storage is used in this solution to store Llama 2 model, other datasets, and Red Hat OpenShift AI pipeline artifacts.

    Networking 

    Two 25 GbE Dell PowerSwitch network switches are being used for data plane connectivity, and two 10 GbE Dell PowerSwitch network switches are being used for management plane connectivity. Dell PowerSwitch enhances performance in data center fabrics of all sizes with efficient and flexible switching solutions.

    Bastion node

    This node is used to run VMs, and Containers related to client application and tools required to manage the Red Hat OpenShift cluster, Object Storage, and LLM-based application. Also, infra VMs such as AD and DNS are running in this node.