This section describes the main steps for implementing the conversational AI application. It includes how to install all components needed, how to enable basic tests using container-based CLI, how to perform more advanced testing using Red Hat OpenShift AI, and how to deploy an application using Red Hat OpenShift builder. The Dell APEX Cloud Platform for Red Hat OpenShift makes installing components easy through the operator process. The OperatorHub marketplace is available in the user interface.

Before you start, verify that you have an available worker node with compatible GPU resources in your Dell APEX Cloud Platform for Red Hat OpenShift cluster. See NVIDIA RIVA support matrix for more details. Also, confirm your access as a cluster-admin.

Your first step is to install the operators required for this solution. From the administrator perspective in the Red Hat OpenShift console, go to OperatorHub and search for the following operators:

• Web Terminal operator
• Node Feature Discovery (NFD) operator
• NVIDIA GPU operator
• Red Hat OpenShift AI operator

Follow the on-screen installation process for each of them until you confirm that they are available in the installed operators' menu, as in Figure 4.

Note: The NFD operator is a prerequisite for the NVIDIA GPU operator.

Figure 4.   Web console showing installed operators

Next, install NVIDIA Riva. To proceed, make sure you have access to the NVIDIA NGC portal and generate an API key, following the steps indicated in the NGC user guide.

The NGC Helm Repository hosts a chart designed for the automated deployment of NVIDIA Riva to Kubernetes clusters, which needs to be adapted for OpenShift. The helm chart pulls container images and model artifacts from NGC, generates the Triton Inference Server model repository, starts the Riva Speech AI server, and exposes it as a service.

You can perform the deployment process in the Red Hat OpenShift web console terminal, as described in the detailed deployment guide prepared by the “AI on OpenShift” website. The high-level steps to perform the deployment of NVIDIA Riva in Red Hat OpenShift are:

• Download the helm chart from the NGC portal using your API key.
• Edit the deployment.yaml file for your Riva container with specific security settings for OpenShift.
• Modify the values.yaml file by adding your NGC credentials and specifying PVC (persistence volume claim) and storage class name.
• If applicable, customize your values.yaml by choosing specific models that you want to enable/disable in the deployment (commented-out models are disabled). For example, Conformer-CTC for automatic speech recognition, megatron for natural language processing, and vocoder HiFiGAN for speech synthesis are some model categories available.
• Create your project and install Riva with the helm chart.