The Dell Validated Design for Generative AI provides validated architectures, reducing the risks associated with designing and implementing custom solutions. The design validated inference, RAG, and fine-tuning techniques with Llama 3 models on the reference architecture, ensuring system reliability, optimal performance, scalability, and interoperability. The validation steps use Docker containers and can be extended to run on Kubernetes. The design also includes a “base container” with all relevant components for LLM tasks. The objective of inference validation is to deploy the Llama 3 70 B model using the vLLM library on KServe for real-time generative AI inference tasks. The design also validates RAG and fine-tuning techniques, including LoRA and SFT using the latest LLM tools and frameworks.
