Home > AI Solutions > Artificial Intelligence > White Papers > Model Customization for Code Creation with Red Hat OpenShift AI on Dell AI Optimized Infrastructure > Solution approach
The proposed approach to solving these business challenges includes hardware and software components as well as an overall process for continuous improvement.
Major process steps include training a base coding large language model on additional data then deploying the new model for testing and use as a code generation and explanation system.
The first step is choosing the model. Large language models such as those in the Code Llama family are desirable since they have been trained on vast libraries of code for infilling and new code generation purposes. The most widely used coding scenarios which follow standards have higher weightage when it comes to tune parameters in the model. This results in code recommendations which follow widely accepted industry standards.
The higher the number of tuned parameters the LLM consists of, the higher the accuracy it provides. Hence, for the best accuracy, the Code Llama 70b parameter version is recommended. However, if GPU and compute resources are limited and faster results are wanted at the cost of accuracy, 7b, 13b, or 34b models can be used.
Note: As the state of the art in large language models changes over time, part of the implementation should have capabilities to continually evaluate new models and technology to drive improvements in output.
For the hardware in this solution, a scalable, distributed, multi-tier multi-purpose configuration is proposed. This hardware division of labor includes the following major components and characteristics:
Red Hat OpenShift AI running on top of its OpenShift container platform provides a robust developer-friendly ecosystem when deployed on Dell Technologies AI optimized infrastructure. With this hardware and software, we can take advantage of multi-GPU training and inferencing by using newly released distributed workloads features of Red Hat OpenShift AI.
Distributed workloads provide the following benefits[1]:
The ongoing functions of the solution include training data and model storage and curation, training framework, and the infrastructure for testing and production inferencing. Finally, the overall continuous improvement process can be loosely defined as the following steps: