Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Generative AI in the Enterprise with AMD Accelerators
Introduction Concepts Solution Architecture Validation Results
Overview
System configurations for validation
Building the base container
Inference validation
RAG validation
Fine-tuning validation
Summary
Performance Characterization Dell Professional Services and Consulting Conclusion References Configuration file for LLM fine-tuning

RAG validation

Home > AI Solutions > Gen AI > Guides > Generative AI in the Enterprise with AMD Accelerators > RAG validation

RAG validation

  • Retrieval-Augmented Generation (RAG) is a method that allows the use of custom data to augment the context of LLMs, shifting away from sole reliance on static training datasets. It addresses challenges such as LLMs' lack of knowledge about specific data, their tendency to provide outdated answers or hallucinate, and the inefficiency and high costs associated with training and fine-tuning. RAG is particularly useful for applications such as Q&A chatbots, search augmentation, and knowledge engines that ask questions on your data. The benefits of RAG include providing up-to-date and accurate responses, reducing inaccuracies and hallucinations, offering domain-specific responses, and being a cost-effective approach to customizing LLMs.

    The following steps show how to deploy a simple RAG application using LlamaIndex:

    1. Install LlamaIndex using the following command. See Custom installation from Pip for more information:

    pip install llama-index-core llama-index-readers-file llama-index-llms-ollama llama-index-embeddings-huggingface

    1. Follow the starter tutorial for deploying RAG. In your working directory, create a data/ folder and in the folder create a paul_graham_essay.txt file. Paste the content from this link into the file.
    2. Install Ollama and pull additional integrations:

    curl -fsSL https://ollama.com/install.sh | sh

    ollama pull llama3

    ollama pull nomic-embed-text

    pip install llama-index-llms-ollama

    pip install llama-index-embeddings-ollama

    1. In the same folder where you created the data/ folder, create the starter.py file and include the following code:

    from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings

    from llama_index.embeddings.ollama import OllamaEmbedding

    from llama_index.llms.ollama import Ollama

    documents = SimpleDirectoryReader("data").load_data()

    # nomic embedding model

    Settings.embed_model = OllamaEmbedding(model_name="nomic-embed-text")

    # ollama

    Settings.llm = Ollama(model="llama3", request_timeout=360.0)

    index = VectorStoreIndex.from_documents(

        documents,

    )

    query_engine = index.as_query_engine()

    response = query_engine.query("What did the author do growing up?")

    print(response)

    1. Use the following command to run and observe a similar output:

    $ python starter.py
    The author grew up writing short stories and programming on an IBM 1401 in his junior high school. He also worked on a TRS-80 microcomputer in about 1980, which he convinced his father to buy him after years of nagging.