Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Digital Twin Journey: Computer Vision AI Model Enhancement with Dell Technologies Solutions & NVIDIA Omniverse
Introduction Business considerations Solution overview Partner overview Solution architecture overview Use case validation
Use case validation
3D simulations
Simple scene interaction with SimReady assets Complex scene interaction with Several SimReady assets
Synthetic data generation
Developing a Computer Vision AI object detection model with synthetic data
Training and validation Inferencing
Conclusions References

Synthetic data generation

Home > Workload Solutions > Computer Vision > White Papers > Digital Twin Journey: Computer Vision AI Model Enhancement with Dell Technologies Solutions & NVIDIA Omniverse > Synthetic data generation

Synthetic data generation

  • We investigated how to generate synthetic 3D data with NVIDIA Omniverse Isaac Sim. This test explored how to generate suitable synthetic data that is required to train a CV AI model to detect "Pallet Jack'" objects within a warehouse (logistics) setting.

    The Isaac Sim environment configurations are based on NVIDIA-Omniverse synthetic-data repository examples . For strategies to improve the quality of synthetic data such as domain randomization (texture and lighting changes, scene distractors), see How to Train Autonomous Mobile Robots to Detect Warehouse Pallet Jacks Using Synthetic Data documentation.

    High-level simulation and synthetic data generation steps:

    • Load warehouse scene
    • Load SimReady assets
    • Simulate domain randomization
    • Generate data
    Figure 14. Warehouse scene containing several instances of pallet jacks and other assets
    This figure shows a warehouse scene containing several instances of pallet jacks and other assets

    Results

    Several Python scripts are provided to configure both Omniverse Isaac Sim and Replicator to automatically generate scene simulations and associated metadata (labeling and annotations) required for subsequent AI model training.

    • Running these scripts generates approximately 5,000 pallet jack images and associated metadata.
    • The associated metadata is written in standard CV object detection KITTI format.
    • The images generated vary based on the domain randomization configurations.
    Figure 15. Examples of pallet jack images based on domain randomization scene distractor techniques
    This figure shows examples of pallet jack images based on domain randomization scene distractor techniques

    Findings

    • Generation of approximately 5,000 annotated images was seamless and took less than five minutes to complete.
    • Average CPU and GPU utilization of approximately 7% and approximately 33% respectively during this SDG test.
    • Setup of this predefined scene was slightly complex for first-time use by a novice.
    • Developing a new scene from scratch would be a significant undertaking. Domain randomization would require a suitable heuristic approach that includes different scenes. Indoor and outdoor scenes might require additional considerations.