Home > Servers > PowerEdge Cyber Security > White Papers > Securing AI workloads on Dell PowerEdge with Intel Xeon processors using Intel Trust Domain Extensions > Software design
The solution design leverages the Intel® Extension for PyTorch, a powerful tool that enhances PyTorch with the latest features and optimizations for superior performance on Intel® hardware. This extension is a critical component of our solution software, providing a significant performance boost. Optimizations take advantage of Intel® Advanced Vector Extensions 512 (Intel® AVX-512), Vector Neural Network Instructions (VNNI), and Intel® Advanced Matrix Extensions (Intel® AMX) on Intel® CPUs, as well as Intel® Xe Matrix Extensions (XMX) AI engines on Intel® discrete GPU.
The solution uses DeepSpeed, an open-source library for deep learning optimization, distributing large models, and scaling. DeepSpeed includes innovations in parallelism technology, custom inference kernels, communication optimizations, and heterogeneous memory technologies.
The large language model (LLM) LLama 2 7B was used in the solution as a representative Generative AI model that is popular and widely used in various use cases.
Llama 2 is a family of open-source, pre-trained, and fine-tuned LLMs released by Meta AI in 2023. It is designed for tasks like:
The platform software used in our solution includes:
The KVM is configured with 256 GB RAM and utilizes all available cores in a 2-socket platform, amounting to 64 cores. This ensures full resource allocation and performance when running the Llama 2 model.
The use of the KVM is justified by the integration of Intel® TDX technology into our solution. Intel® TDX provides hardware-based isolation on the Virtual Machine level to protect sensitive data and code from external threats. By securing the KVM, Intel® TDX effectively encrypts the memory utilized by the VM, enhancing the overall security of our solution.