The goal of our validation is to ensure the reliability, optimal performance, scalability, and interoperability of the system. Model customization yields an LLM that incorporates domain-specific knowledge. We validated our design to ensure the functionality of model customization techniques available in the NeMo framework. Our goal in this validation was not to train a model to convergence. The following list provides the details of our validation setup:

• Foundational models─We validated 7B, 13B, and 70B Llama 2
• Model customization techniques─We use the following techniques:
  • Prompt engineering─The following section shows the prompt engineering results.
  • SFT, P-Tuning, and LoRA─The next sections show the results of these fine-tuning methods. Appendix A shows the Slurm scripts that we used to launch the jobs.
• Cluster configuration─We used both Slurm and Kubernetes clusters:
  • On the Slurm cluster, we validated both single-node and multinode (two nodes) clusters on all three model customizations.
  • On the Kubernetes cluster, we validated both single-node and multinode (two nodes) clusters for P-Tuning and LoRA model customization techniques. We only validated single-node training for SFT.

For multimode SFT employs distributed checkpointing, which necessitates worker nodes accessing folders created by their counterparts. We are working to determine the best way to make the volumes available to all the worker nodes

• Dataset─We used two datasets for this validation:
  • Dolly dataset from Databricks (databricks-dolly-15k) is an open-source dataset of instruction-following records generated by thousands of Databricks employees in several of the behavioral categories outlined in the InstructGPT paper. The categories include brainstorming, classification, closed QA, generation, information extraction, open QA, and summarization.
  • Alpaca is a dataset of 52,000 instructions and demonstrations generated by OpenAI's text-davinci-003 engine. This instruction data can be used to conduct instruction-tuning for language models and make the language model follow instructions better.

Time for training─ Usually, data scientists train a model until it reaches convergence, a point influenced by factors like the dataset, model complexity, and chosen hyperparameters. Our aim was not to achieve convergence for every scenario, as it is specific to our chosen dataset and parameters, offering limited insight into a customer's needs. To maintain a consistent metric across all scenarios, we conducted training jobs for a minimum of 50 steps.

Prompt engineering validation

The following table provides examples of prompt engineering results from our validation work and how the response from the model can be shaped using specific prompts:

Table 8. Examples of prompt engineering for describing the earth[1]

SFT and PEFT validation

The following table summarizes the scenarios we validated and the configuration that we used.

Table 9. Validated scenario configuration

In the preceding table, TP refers to Tensor Parallelism. It is a technique used to distribute the computations and memory across multiple GPUs in a single layer or operation in a neural network. It is particularly useful when dealing with models that are too large to fit in the memory of a single device. TP divides the computation graph into smaller tensors and distributes them across multiple devices, enabling parallel processing and reducing the time required for training.

PP refers to Pipeline Parallelism. It is a technique used to improve the efficiency of training by splitting the model into smaller, connected stages or segments. Each stage processes a portion of the data, and the results are passed sequentially from one stage to the next. This method allows for overlapping computation and communication, reducing idle time and improving overall training speed. PP is especially useful for very deep models with many layers, as it helps exploit parallelism in the forward and backward passes of the neural network.

To illustrate the validation, we provided a few example results. The appendix includes the Slurm batch file that initiates the model customization job. 

The following figure illustrates the training loss for Llama 2 70B on a single-node 8-GPU run. Note that the graphs illustrate the validation of model customization, and not necessarily the optional hyperparameter configuration (such as learning rate):

Figure 7. Training loss for Llama 2 70B

For the same experiments, the following figure shows how the learning rate adapted during model customization:

Figure 8. Learning rate adapted during model customization