Model customization yields an LLM that incorporates domain-specific knowledge. Our goal in this validation was not to train a model to convergence, but to demonstrate the functionality of various techniques and frameworks. The following list provides the details of our validation setup:

• Foundational models—We used Llama 3 8 B and 70 B for our validation.
• Model customization techniques—We used SFT and LoRA. The scripts and example outputs provided in this section are for LoRA. SFT implementation is similar to LoRA.
• Dataset—We used the Dolly dataset. The Databricks Dolly 15K Dataset is an open-source dataset of instruction-following records generated by thousands of Databricks employees in several of the behavioral categories outlined in the InstructGPT paper. The categories include brainstorming, classification, closed QA, generation, information extraction, open QA, and summarization.
• Framework – We used two frameworks for distributed training.
• torchtune, a new library from PyTorch for LLM fine-tuning
• Hugging Face Transformer Reinforcement Learning (TRL) and Hugging Face Accelerate with DeepSpeed

To run fine-tuning using torchtune, we perform the following steps:

1. Run the base container with the following parameters:

docker run -it --shm-size 250G --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --ipc host --network host --device=/dev/kfd --device=/dev/dri --group-add video -v /dev:/dev -v /sys:/sys --privileged -v /cm/shared/apps/waco/:/waco -v <powerscalef710 mount path for dataset and model> <base container location>

2. Build the torchtune configuration file for LoRA. See Appendix A for a sample configuration.
3. Run torchtune by using the following command:

tune run --nnodes=4 --nproc_per_node=8 --rdzv_id=456 --rdzv_backend=c10d --node-rank=1 --rdzv_endpoint=<node_name>:29603 lora_finetune_distributed --config <config_file location> output_dir=<output directory in PowerScale F710> checkpointer.output_dir=<checkpoint directory in PowerScale F710> dtype=bf16"

Training starts and then provides output similar to the following example:

INFO:torchtune.utils.logging:Model checkpoint of size 9.17 GB saved to /aipsf710-22/project-baylor/torchtune/lora-70b-bf16-sn/hf_model_0001_0.pt

INFO:torchtune.utils.logging:Model checkpoint of size 9.33 GB saved to /aipsf710-22/project-baylor/torchtune/lora-70b-bf16-sn/hf_model_0002_0.pt

INFO:torchtune.utils.logging:Model checkpoint of size 10.00 GB saved to /aipsf710-22/project-baylor/torchtune/lora-70b-bf16-sn/hf_model_0003_0.pt

INFO:torchtune.utils.logging:Model checkpoint of size 9.93 GB saved to/aipsf710-22/project-baylor/torchtune/lora-70b-bf16-sn/hf_model_0004_0.pt

1|939|Loss: 0.9668265581130981: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 939/939 [1:32:18<00:00,  5.65s/it]

Model checkpoints are created in the folder provided in the torchtune launch command.