Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Design Guide—Implementing a Digital Assistant with Red Hat OpenShift AI on Dell APEX Cloud Platform
Introduction Solution Concepts Solution Architecture and Requirements Solution Deployment
Introduction
Dell APEX Cloud Platform for Red Hat OpenShift
Object storage
Digital assistant deployment and configuration
Model serving environment
Data science project in Red Hat OpenShift AI
Project Workbench
LLM model preparation
Model deployment
Redis deployment
Document embeddings
Gradio deployment
Deployment summary
Solution execution Summary References Open-Source License

Gradio deployment

Home > AI Solutions > Artificial Intelligence > Guides > Design Guide—Implementing a Digital Assistant with Red Hat OpenShift AI on Dell APEX Cloud Platform > Gradio deployment

Gradio deployment

  • This section describes the procedure to deploy Gradio to provide a user interface for a digital assistant.

    Gradio UI for RAG using Caikit + TGIS and Redis (GitHub)

    This is a simple UI example for a RAG-based Chatbot using Gradio, Caikit + TGIS for LLM inference, and Redis as a vector database.

    Deployment requirements include:

    • A Caikit + TGIS deployment with a deployed LLM. This example is based on Llama2 but depending on your LLM you may need to adapt the prompt.
    • A Redis installation with a database and an index already populated with documents. See GitHub for deployment instructions.
    • An index in the Redis database that you have populated with documents. See GitHub for an example.

    Deployment on OpenShift

    A pre-built container image of the application is available at: quay.io/rh-aiservices-bu/gradio-caikit-rag-redis:latest

    The deployment folder includes the necessary files necessary to deploy the application:

    • cm_redis_schema.yaml: A ConfigMap containing the schema for the database you have created in Redis (see the ingestion Notebook).
    • cm_certificate.yaml: You must put your Caikit endpoint certificate and create this ConfigMap prior to the deployment.
    • deployment.yaml: You must provide the URL of your inference server in the placeholder on L54 and Redis information for L56 and L58. Please feel free to modify other parameters as you see fit.
    • service.yaml
    • route.yaml

    The different parameters you must pass as environment variables in the deployment are:

    INFERENCE_SERVER_URL - mandatory
REDIS_URL - mandatory
REDIS_INDEX - mandatory
MAX_NEW_TOKENS - optional, default: 512
MIN_NEW_TOKENS - optional, default: 100 

    Create a new project in OpenShift cluster using the following command:

    oc new-project dell-digital-assistant 

    Clone dell digital assistant GitHub repository and go to:

    dell-digital-assistant/05-UI/Gradio/caikit-rag-redis/ 
    git clone https://github.com/DellBizApps/dell-digital-assistent.git
cd dell-digital-assistent/05-UI/Gradio/caikit-rag-redis/
oc apply -f deployment/ 

    Note: You can also deploy the application in OpenShift dashboard by using the source to image feature of the OpenShift cluster by following the instructions available on the Red Hat Customer Portal.

    The deployment replicas are initially set to 0 to let you properly fill in the environment variables based on your setup. Change the values accordingly and scale the deployment replicas from 0 to 1.