This design guide describes the process of deploying a digital assistant using Red Hat OpenShift AI on Dell APEX Cloud Platform for Red Hat OpenShift. This solution is designed to create a cloud native AI application, with the ease of deployment and manageability.
LLMs are highly sophisticated AI models that are designed to understand and generate human-like text, enabling a wide range of natural language processing applications. One limitation of LLMs is that once they are generated, they do not have access to information beyond the date that they were trained. Retrieval Augmented Generation (RAG) can extend the functionality of the LLMs by retrieving facts from an external knowledge base, in this case, from a Redis in-memory Vector database.
Digital assistants are designed to assist users by answering questions and processing simple tasks. Answers remain up to date and contain information unique to the organization by anchoring the model with relevant documentation.
In this solution, we have deployed a LLM based digital assistant that can answer user queries related to domain specific documents. As text-based searches are limited in obtaining the right data, a digital assistant can help retrieve more accurate and relevant results using semantic search and natural language processing.
The Red Hat OpenShift Container Platform provides a robust containerization platform and Kubernetes-based orchestration framework that enables organizations to build, deploy, and manage applications and databases efficiently across on-premises and multicloud environments.
Dell APEX Cloud Platform for Red Hat OpenShift is designed collaboratively with Dell Technologies and Red Hat to optimize and extend OpenShift deployments on-premises with an integrated operational experience. By combining Dell’s expertise in delivering robust infrastructure solutions with Red Hat’s industry-leading , this collaboration empowers organizations to start on a transformative journey towards modernization and innovation.
The APEX Cloud Platform for Red Hat uses separate storage nodes to provide persistent storage for the compute cluster. This separation enables the compute and storage nodes to scale independently.
Red Hat OpenShift AI offers organizations an efficient way to deploy an integrated set of common open-source and third-party tools to perform ML modeling. The ML models developed using Red Hat OpenShift AI are portable to deploy in production, on containers, on-premises, at the edge, or in the public cloud.
Utilizing object storage for LLMs and huge datasets is crucial for modern data-driven applications, due to their reliability and scalability. Object storage is leveraged in this solution to store Llama 2 model, relevant datasets, and artifacts.
The purpose of this document is to provide design guidance for deploying a digital assistant using Red Hat OpenShift AI on Dell APEX Cloud Platform for Red Hat OpenShift.
Note: The contents of this document are valid for the described software and hardware versions. For information about updated configurations for newer software and hardware versions, contact your Dell Technologies sales representative.
This design guide is intended for AI solution architects, data scientists, data engineers, IT infrastructure managers, and IT personnel who are interested in, or considering, implementing AI and ML deployments.
This document provides design guidance on deploying a digital assistant using open-source tools. It is the responsibility of the user to review and comply with the licensing terms and conditions for each tool mentioned in this document. Licensing information for each tool can be found on the respective tool's official website or in its documentation. See Appendix A to access the licensing page for some of the open-source tools used during validation of this solution.