Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Generative AI in the Enterprise
Introduction Generative AI background and concepts Business and technical challenges Benefits Use cases Dell and NVIDIA solution architecture Infrastructure component considerations for AI Dell infrastructure and software components NVIDIA infrastructure and software components
Introduction
NVIDIA accelerators
NVIDIA AI software
System Configurations Conclusion References

NVIDIA AI software

NVIDIA AI software

  • NVIDIA enterprise software solutions are designed to give IT admins, data scientists, architects, and designers access to the tools they need to easily manage and optimize their accelerated systems. 

    NVIDIA AI Enterprise

    NVIDIA AI Enterprise, the software layer of the NVIDIA AI platform, accelerates the data science pipeline and streamlines development and deployment of production AI including generative AI, computer vision, speech AI and more. This secure, stable, cloud-native platform of AI software includes over 100 frameworks, pretrained models, and tools that accelerate data processing, simplify model training and optimization, and streamline deployment.

    • Data preparation—Improve data processing time by up to 5 times while reducing operational costs by 4 times with the NVIDIA RAPIDS Accelerator for Apache Spark.
    • AI Training—Create custom, accurate models in hours, instead of months, using NVIDIA TAO Toolkit and pretrained models.
    • Optimization for inference—Accelerate application performance up to 40 times over CPU-only platforms during inference with NVIDIA TensorRT.
    • Deployment at scale—Simplify and optimize the deployment of AI models at scale and in production with NVIDIA Triton Inference Server.

    Available in the cloud, the data center and at the edge, NVIDIA AI Enterprise enables organizations to develop once and run anywhere. Since the full stack is maintained by NVIDIA, organizations can count on regular security reviews and patching, API stability, and access to NVIDIA AI experts and support teams to ensure business continuity and AI projects stay on track. 

     

    Figure 2.  NVIDIA AI Enterprise

    NVIDIA AI Enterprise includes NVIDIA NeMo, a framework to build, customize, and deploy generative AI models with billions of parameters. The NVIDIA NeMo framework provides an accelerated workflow for training with 3D parallelism techniques. It offers a choice of several customization techniques and is optimized for at-scale inference of large-scale models for language and image applications, with multi-GPU and multinode configurations. NVIDIA NeMo makes generative AI model development easy, cost-effective, and fast for enterprises.

    NVIDIA Base Command Manager

    NVIDIA BCM is NVIDIA’s cluster manager for AI infrastructure. It facilitates seamless operationalization of AI development at scale by providing features like operating system provisioning, firmware upgrades, network and storage configuration, multi-GPU and multinode job scheduling, and system monitoring, thereby maximizing the utilization and performance of the underlying hardware architecture.

    Figure 3.  NVIDIA Base Command Manager

    NVIDIA BCM supports automatic provisioning and management of changes in nodes throughout the cluster's lifetime.

    With an extensible and customizable framework, it has seamless integrations with the multiple HPC workload managers, including Slurm IBM Spectrum LSF, OpenPBS, Univa Grid Engine, and others. It offers extensive support for container technologies including Docker, Harbor, Kubernetes, and operators. It also has a robust health management framework covering metrics, health checks, and actions.