Tue, 14 May 2024 15:16:38 -0000
|Read Time: 0 minutes
This blog is part two of the blog series Dell Technologies PowerEdge R760xa with NVIDIA Omniverse:
The following related technical white paper is also available:
A Digital Twin Journey: Computer Vision AI Model Enhancement with Dell Technologies Integrated Solutions & NVIDIA Omniverse
Digital Twins are physically accurate virtual replicas of assets, processes, or environments. Digital Twins are becoming increasingly popular with organizations looking to benefit from timely, and better decision making. Having a digitized virtual twin of real-world resources might, for example, enable businesses to identify and prevent costly mistakes or anomalies before they occur.
The Digital Twin paradigm can be applied to a lot of real-world scenarios, both big and small!
See Building Digital Twins of the Earth in Omniverse.
Implementation of Digital Twin is a complex undertaking made all the more challenging by the fact there is no “one size fits all” approach.
See 5 Steps to Get Started with Digital Twins.
This article describes how Dell Technologies PowerEdge R760xa servers, in conjunction with the NVIDIA Omniverse platform, can be used in the development of physically accurate Digital Twin 3D workflows.
NVIDIA Omniverse is a platform of APIs, SDKs, and services that enable developers to easily integrate Universal Scene Description (OpenUSD) and RTX rendering technologies into existing software tools and simulation workflows for building AI systems and 3D Digital Twin use cases.
The Omniverse ecosystem is designed from the ground up to enable development of new tools and workflows from scratch with customizable sample foundation application templates or SDKs that are easy to modify.
For a sample configuration, refer to the Dell PowerEdge R760xa Technical Guide.
The PowerEdge R760xa server is positioned to meet the diverse needs of Digital Twin requirements such as 3D modelling, physics simulations, image rendering, computer vision, robotics, edge computing, AI training, and inferencing.
Launcher is an easy access GUI for enabling and learning about the Omniverse platform.
In Launcher, click the Exchange tab to download, install, and update Omniverse Apps and Connectors. Connectors extend the capabilities of Omniverse to integrate with a wide range of applications and services such as CAD, GIS, and VR.
USD Composer is a foundation app template built using NVIDIA Omniverse Kit. It takes advantage of the advanced 3D workflow capabilities of OpenUSD such as layers, variants, instancing, and animation caches. USD Composer lets you develop and compose physically accurate 3D scenes.
Note: See figure 7 depicting Assembly line scene generated with USD Composer.
For organizations looking to make more insightful and timely decisions by simulating their real-world resources—such as production lines, warehouse/factory designs, and computer vision AI anomaly detection capabilities—the generation of highly accurate 3D models that mirror real-world physics is crucial.
It’s not enough just to have 3D models that contain virtual assets that are visually accurate. For simulations to be insightful, 3D assets must also represent their real-world counterparts as closely as possible. NVIDIA is working to create a new 3D standard specification called SimReady assets. These assets are building blocks for virtual worlds.
Simulation platforms, such as Omniverse, can leverage SimReady assets (information and metadata) to make scenario modelling far more useful for research and training of a particular product or ecosystem.
See figures 6 and 7 below depicting the development of a basic automotive assembly line scene with both visual and SimReady assets.
The NVIDIA Omniverse RTX Renderer is a powerful rendering technology offering real-time and offline rendering capabilities.
RTX - Real-Time mode allows rendering more geometry than traditional rasterization methods while maintaining high fidelity. It is suitable for real-time applications.
In RTX - Interactive (Path Tracing) mode, the renderer uses a path tracing-based algorithm to achieve photorealistic results. This mode is ideal for scenarios where quality takes precedence over real-time performance with 4k or lower resolution. The balance of quality and performance depends on the given use case.
The NVIDIA-SMI command-line utility (figure 9) shows an example of multi-GPU resource utilization (four physical L40 GPUs) on a Dell PowerEdge 760xa server and virtualized Omniverse instance.
Number of GPUs | Purpose |
2 | RTX rendering |
1 | 3D scene simulation |
1 | Training computer vision model |
There is no single implementation approach for Digital Twins due to their diverse and complex nature. Requirements might evolve as development takes place, so flexibility is a key consideration when embarking on a Digital Twin venture.
Omniverse Enterprise is architected with interoperability in mind. Based on OpenUSD, the platform fundamentally transforms complex 3D workflows. You can easily connect your 3D product pipelines and unlock full design-fidelity visualization of your 3D datasets using purpose-built connector plugins for the most common 3D applications and data types.
The compatibility and collaboration capabilities of NVIDIA Omniverse and the flexibility and computational power of Dell Technologies PowerEdge Servers combine to enable a strong foundation for the development/use of Digital Twin workflows.
Virtual Workstation Interactive Collaboration with NVIDIA Omniverse
Tue, 14 May 2024 15:08:48 -0000
|Read Time: 0 minutes
This blog is part one of the blog series Dell Technologies PowerEdge R760xa with NVIDIA Omniverse:
The following related technical white paper is also available:
A Digital Twin Journey: Computer Vision AI Model Enhancement with Dell Technologies Integrated Solutions & NVIDIA Omniverse
Digital Twins (DT) and Artificial Intelligence (AI) are driving a massive increase in the volume of data organizations need to manage. Harnessing the insight potential from within this data is a constant challenge that drives the need for evermore performant and flexible solutions.
This article describes how hardware from Dell Technologies running NVIDIA Omniverse software can be deployed using GPU virtualization to provide more flexibility and performance for DT and AI applications.
A key challenge for IT administrators is providing optimized infrastructure hardware and software that can support the integration of complex new technologies such as AI and DT.
NVIDIA Omniverse offers an integrated ecosystem of solutions harnessing hardware acceleration plus software designed for DT workloads and 3D modeling collaboration.
The NVIDIA Omniverse platform offers developers a vast increase in creativity and efficiency potential. It is a scalable, multi-GPU, real-time reference development suite for 3D modeling and design collaboration based on the Pixar Universal Scene Description (USD) framework and NVIDIA RTX technology.
Designers, artists, and creators can use the power of Omniverse to accelerate their DT and high-fidelity 3D workflows. It provides real-time ray tracing and AI-enhanced graphics, quintessential for simulating the real world within a DT environment.
The PowerEdge R760xa server shines for both DT and AI applications. Coupled with either 4x NVIDIA L40 or L40S PCIe, 48 GB GPUs and enabled by Intel Xeon Scalable processors, this server provides the processing muscle for reliable, precise, and fast 3D Graphics and Compute centric workloads.
The PowerEdge R760xa server is positioned perfectly to meet the diverse needs of DT requirements such as 3D modeling, physics simulations, image rendering, computer vision, robotics, edge computing, AI training and Inferencing.
Omniverse installations come in two deployment flavors: Omniverse Workstation or Enterprise. This article concentrates on the deployment of Omniverse Enterprise on Dell PowerEdge R760xa servers.
Deploying Omniverse Enterprise as a virtualized instance enables a flexible infrastructure configuration that is tailored to individual requirements, such as splitting physical GPUs resources into vGPU partitions. This flexibility can prove immensely beneficial when DT or AI workload needs are likely to change during development.
NVIDIA’s Omniverse Install Guide references three key components, all of which can be served within the confines of a virtualized environment.
Component | Description |
---|---|
Licensing | Mechanism to procure and enable Omniverse software. |
Enterprise Nucleus | The central database and collaborative engine of Omniverse. Enables users to share and modify representations of virtual worlds. |
Launcher | The native client for downloading, installing, and updating Omniverse Apps, Extensions, and Connectors. |
Some prerequisites before you start:
NVIDIA’s Virtualized Deployment Guide outlines several foundational steps needed to create a virtualized Omniverse solution.
Virtualized Omniverse environments that are built on top of high-performant infrastructure like the Dell PowerEdge R760xa server create a foundation for building 3D, DT, and AI solutions.
Platform | Dell PowerEdge 760xa | |
CPU | 2x Intel(R) Xeon(R) Gold 6438M | |
GPU | 4x NVIDIA L40 | |
FP32(Tera Flops) | 90 | |
Memory (GB) | 48 GDDR6 w/EEC | |
Media Engines | 3 Video Encoder 3 Video Decoder 4 JPEG Decoder | |
Power (Watts) | 300 | |
Memory | 512 GB DDR5 | |
Software Stack | VMware ESXi, 8.0.1 Windows 10 Enterprise 10.0.19045 NVIDIA vGPU Grid Driver 16.1 Omniverse USD Composer 2023.2.0 Omniverse Launcher 1.8.11 Omniverse Nucleus 2023.1.0 |
The following figure shows a VMware vCenter Omniverse USD Composer Virtual Workstation configured with 4 x L40 vGPUs.
A sample 3D scene being rendered within the Omniverse USD Composer application is shown in the following figure.
The NVIDIA-SMI command-line utility shows 4 physical L40 GPUs configured in vGPU mode with Virtual Workstation vWS profile (Enabling both graphic and compute acceleration). Natively the USD Composer App consumes all available GPU resources to render the depicted 3D scene.
A more realistic virtualized Omniverse configuration might be, 1 to 2 GPUs assigned to rendering tasks with other GPUs being assigned to other 3D or DT tasks, such as PhysX simulations or AI model training.
Complex DT workloads encapsulate the integration of 3D models, simulations, and AI software components, each with their own unique system requirements. NVIDIA Omniverse is not a one-size-fits-all solution but rather a dynamic 3D ecosystem for collaboratively creating shared virtual worlds.
Often in development scenarios, system requirements may not be fully understood and thus the need for a flexible infrastructure solution. Omniverse can be easily configured and customized for various applications and customer needs as development evolves.
We found that virtualized Omniverse deployment allows for amazing flexibility to meet numerous workload requirements!
Virtual Workstation Interactive Collaboration with NVIDIA Omniverse
Tue, 14 May 2024 15:08:48 -0000
|Read Time: 0 minutes
This blog is part three of the blog series Dell Technologies PowerEdge R760xa with NVIDIA Omniverse:
Synthetic Data Generation (SDG) is the process of generating data for model development using analysis and tools to represent the patterns, relationships, and characteristics of real-world data.
The collection, curation, and annotation of real-world data is inherently time-consuming, expensive and may not be feasible in many circumstances. Synthetic data on the other hand might prove a suitable alternative, either in its own right or in conjunction with existing real-word data. Synthetic data has the added benefit of less risk of infringing on privacy or exposing sensitive information while providing data diversity.
Types of Synthetic data and use cases:
Synthetic text: Artificially generated text, such as text used to train language models.
Synthetic media: Artificially generated sound, image, and video, such as media used to train autonomous vehicles or computer vision models.
Synthetic tabular data: Artificially generated structured data (such as, spreadsheets and databases).
Synthetic data/datasets are increasingly being used to train AI algorithms and perform real-world modeling.
Gartner predicts that by 2030, most AI modeling data will be artificially generated by rules, statistical models, simulations, or other techniques.
Part one of this blog series discussed how to deploy a virtual instance of NVIDIAs Omniverse platform with Part two highlighting how 3D scenes that are built with Omniverse SimReady assets can be used in the development of physically accurate Digital Twin 3D workflows.
This article, Part 3, is the final installment in the series and describes how NVIDIA Omniverse™ Isaac Sim with SimReady Assets can produce photorealistic, physically accurate simulations and synthetic data, which can subsequently be used for AI training and integrated into Digital Twin development workflows.
Isaac Sim is an extensible robotics simulation toolkit for the NVIDIA Omniverse™ platform. With Isaac Sim, developers can train and optimize AI robots for a wide variety of tasks. It provides the tools and workflows needed to create physically accurate simulations and synthetic datasets.
Note: Isaac Sim exposes as a set of extensions (Omniverse Replicator) to provide synthetic data generation capabilities.
SimReady 3D assets and SDG are designed to be used in concert to create and randomize a variety of scenarios to meet specific training goals and do it safely as a virtual simulation. Unlike real-world datasets that must be manually annotated before use, SDG data annotated using semantic labeling can be used directly to train AI models.
NVIDIA TAO Toolkit is used for model training and inferencing. The TAO Toolkit is a powerful open-source AI toolkit that is designed to simplify the process of creating highly accurate, customized computer vision AI models. It leverages transfer learning, enabling the adaption of existing AI models for specific data. Built with TensorFlow and PyTorch, it streamlines the model training process.
TAO is part of NVIDIA AI Enterprise, an enterprise-ready AI software platform with security, stability, manageability, and support.
TAO supports most popular computer vision tasks such as:
This article concentrates on leveraging Dell PowerEdge R760xa servers with 4x L40 GPUs, a virtualized instance of NVIDIAs Omniverse Enterprise with Isaac Sim to synthetically generate a dataset of suitable images that can be used to train Computer Vision object detection AI models.
The following example explores the generation of synthetic data and how it can be used to train a Computer Vision AI model to detect “Pallet Jack” objects within a warehouse setting for logistics planning.
High Level Steps:
The following NVIDIA documentation provides a repository of collateral material (information, scripts, notebooks, and so forth) focused on:
Several python scripts are included to configure and run 3D simulations with Isaac Sim. The resulting SDG output consists of ~ 5,000 high quality annotated images of “Pallet Jack” objects within a warehouse scene.
One factor to consider when generating synthetic image data is diversity. Similar or repetitive images from the synthetic domain/scene will likely not help to improve AI model accuracy. Suitable domain randomization techniques that vary image generation maybe required, such as:
See Figure 6 & 7.
Data diversity techniques will likely vary on a use case or scene basis, for example indoor vs outdoor scenes may require different approaches before satisfactory data is generated.
A sample Jupyter notebook from the following (Omniverse repository) uses the TAO Toolkit from NVIDIA to download a pre-trained computer vision AI (DetectNet_v2/resnet18) model, which is then trained on the previously generated synthetic images.
Figure 8 shows computer vision AI model validation on synthetic images.
Finally, the synthetically trained computer vision AI object detection model is evaluated on real-world “pallet-jack” images during inferencing to assess the performance and accuracy obtained.
The example in Figure 9 shows computer vision object detection of “pallet-jacks” in real-world images
Note: See Logistic Objects in Context (LOCO) dataset containing real-world images within logistics settings.
Synthetic data is playing an increasingly important role in enhancing the capabilities of AI models. Synthetic data used either on its own or in augmenting existing datasets may be used to address certain shortcomings and impediments of real datasets in the training of AI models such as: insufficient data, diversity, privacy, rare scenarios, and cost of curation.
NVIDIA’s Omniverse Platform in conjunction with the Isaac Sim application enables 3D scenario modeling and simulation with corresponding synthetic data image generation which can then be used to train AI and or integrated into 3D data pipelines such as Digital Twins.
In this three-part blog series Dell Technologies PowerEdge 760xa with NVIDIA Omniverse Enterprise Platform, we explored,
For further information see Dell Technologies Technical White Paper: A Digital Twin Journey: Computer Vision AI Model Enhancement with Dell Technologies Integrated Solutions & NVIDIA Omniverse
Virtual Workstation Interactive Collaboration with NVIDIA Omniverse
Applying Digital Twin and AMR Technologies for Industrial Manufacturing