Introduction 

Digital Twins (DT) and Artificial Intelligence (AI) are driving a massive increase in the volume of data organizations need to manage. Harnessing the insight potential from within this data is a constant challenge that drives the need for evermore performant and flexible solutions. 

This article describes how hardware from Dell Technologies running NVIDIA Omniverse software can be deployed using GPU virtualization to provide more flexibility and performance for DT and AI applications. 

The Technical Challenge 

A key challenge for IT administrators is providing optimized infrastructure hardware and software that can support the integration of complex new technologies such as AI and DT.  

NVIDIA Omniverse offers an integrated ecosystem of solutions harnessing hardware acceleration plus software designed for DT workloads and 3D modeling collaboration. 

Omniverse 

The NVIDIA Omniverse platform offers developers a vast increase in creativity and efficiency potential. It is a scalable, multi-GPU, real-time reference development suite for 3D modeling and design collaboration based on the Pixar Universal Scene Description (USD) framework and NVIDIA RTX technology. 

Designers, artists, and creators can use the power of Omniverse to accelerate their DT and high-fidelity 3D workflows. It provides real-time ray tracing and AI-enhanced graphics, quintessential for simulating the real world within a DT environment. 

Dell PowerEdge R760xa Server 

The PowerEdge R760xa server shines for both DT and AI applications. Coupled with either 4x NVIDIA L40 or L40S PCIe, 48 GB GPUs and enabled by Intel Xeon Scalable processors, this server provides the processing muscle for reliable, precise, and fast 3D Graphics and Compute centric workloads.  

The PowerEdge R760xa server is positioned perfectly to meet the diverse needs of DT requirements such as 3D modeling, physics simulations, image rendering, computer vision, robotics, edge computing, AI training and Inferencing.  

Figure 1: Front view of the Dell PowerEdge R760xa server

Figure 2: Top View of the Dell PowerEdge R760xa server

Laying the Foundation for A Digital Twin Environment:  

Omniverse installations come in two deployment flavors: Omniverse Workstation or Enterprise. This article concentrates on the deployment of Omniverse Enterprise on Dell PowerEdge R760xa servers.  

Deploying Omniverse Enterprise as a virtualized instance enables a flexible infrastructure configuration that is tailored to individual requirements, such as splitting physical GPUs resources into vGPU partitions. This flexibility can prove immensely beneficial when DT or AI workload needs are likely to change during development. 

NVIDIA’s Omniverse Install Guide references three key components, all of which can be served within the confines of a virtualized environment. 

Some prerequisites before you start: 

• NVIDIA Enterprise or Developer Account. 
• Suitable Graphics Capable GPU, such as NVIDIA Lovelace GPU series 
• NVIDIA GPU driver (≥471.11)
• Suitable OS—Linux or Windows  
• Note that each Launcher Application may have its own unique system requirements. 

Setting Up a Virtualized Omniverse 

NVIDIA’s Virtualized Deployment Guide outlines several foundational steps needed to create a virtualized Omniverse solution. 

• VMware vSphere ESXi Hypervisor 
• VMware vCenter
• NVIDIA vGPU Manager (VIB) 
• NVIDIA License System (NLS) 

Figure 3: Virtualized Omniverse Stack

Virtualized Omniverse environments that are built on top of high-performant infrastructure like the Dell PowerEdge R760xa server create a foundation for building 3D, DT, and AI solutions. 

Post-Deployment Configuration Example. 

The following figure shows a VMware vCenter Omniverse USD Composer Virtual Workstation configured with 4 x L40 vGPUs. 

Figure 4: Omniverse USD Composer Virtual Workstation configured with 4 x vGPUs

A sample 3D scene being rendered within the Omniverse USD Composer application is shown in the following figure. 

Figure 5: Omniverse USD Composer App using 4 x L40 GPUs

The NVIDIA-SMI command-line utility shows 4 physical L40 GPUs configured in vGPU mode with Virtual Workstation vWS profile (Enabling both graphic and compute acceleration). Natively the USD Composer App consumes all available GPU resources to render the depicted 3D scene. 

A more realistic virtualized Omniverse configuration might be, 1 to 2 GPUs assigned to rendering tasks with other GPUs being assigned to other 3D or DT tasks, such as PhysX simulations or AI model training. 

Conclusion 

Complex DT workloads encapsulate the integration of 3D models, simulations, and AI software components, each with their own unique system requirements. NVIDIA Omniverse is not a one-size-fits-all solution but rather a dynamic 3D ecosystem for collaboratively creating shared virtual worlds.  

Often in development scenarios, system requirements may not be fully understood and thus the need for a flexible infrastructure solution. Omniverse can be easily configured and customized for various applications and customer needs as development evolves.  

We found that virtualized Omniverse deployment allows for amazing flexibility to meet numerous workload requirements! 

References  

PowerEdge R760xa  

NVIDIA L40

NVIDIA L40S   

Virtual Workstation Interactive Collaboration with NVIDIA Omniverse 

Omniverse Documentation 

Omniverse Glossary of Terms 

Introduction

Long gone are the days when facilities managers and security personnel were required to be in a control room with their attention locked onto walls of video monitors. The development of lower-cost and more capable video cameras, more powerful data science computing platforms, and the need to reduce operations overhead have caused the deployment of video management systems (VMS) and computer vision analytics applications to skyrocket in the last ten years in all sectors of the economy. Modern computer vision applications can detect a wide range of events without constant human supervision, including overcrowding, unauthorized access, smoke detection, vehicle operation infractions, and more. Better situational awareness of their environments can help organizations achieve better outcomes for everyone involved.

Table 1 – Outcomes achievable with better situational awareness

The technical challenge

Computer vision analytics uses various techniques and algorithms, including object detection, classification, feature extraction, and more. The computation resources that are required for these tasks depend on the resolution of the source video, frame rates, and the complexity of both the scene and the types of analytics being processed. The diagram below shows a simplified  set of steps (pipeline) that is frequently implemented in a computer vision application.

Figure 1: Logical processing pipeline for computer vision

Inference is the step that most people are familiar with. A trained algorithm can distinguish between a passenger automobile and a delivery van, similar to the classic dogs versus cats example often used to explain computer vision. While the other steps are less familiar to the typical user of computer vision applications, they are critical to achieving good results and require dedicated graphics processing units (GPUs). For example, the Decode/Encode steps are tuned to leverage hardware that resides on the GPU to provide optimal performance.

Given the extensive portfolio of NVIDIA GPUs available today, organizations that are getting started with computer vision applications often need help understanding their options. We have tested the performance of computer vision analytics applications with various models of NVIDIA GPUs and collected the results. The remainder of this article provides background on the test results and our choice of model.

Choosing a GPU

The market for GPUs is broadly divided into data center, desktop, and mobility products. The workload that is placed on a GPU when training large image classification and detection models is almost exclusively performed on data center GPUs. Once these models are trained and delivered in a computer vision application, multiple CPU and GPU resource options can be available at run time. Small facilities, such as a small retailer with only a few cameras, can afford to deploy only a desktop computer with a low-power GPU for near real-time video analytics. In contrast, large organizations with hundreds to thousands of cameras need the power of data center-class GPUs.

However, all data center GPUs are not created equal. The table below compares selected characteristics for a sample of NVIDIA data center GPUs. The FP32 floating point calculations per second metric indicates the relative performance that a developer can expect on either model training or the inference stage of the typical pipeline used in a computer vision application, as discussed above.

The capability of the GPU for performing other pipeline elements required for high-performance computer vision tasks, including encoding/decoding, is best reflected by the Media Engines details.

First, consider the Media Engines row entry for the A30 GPU column. There is 1 JPEG decoder and 4 video decoders, but no video encoders. This configuration makes the A30 incompatible with the needs of many market-leading computer vision application vendors' products, even though it is a data center GPU.

Table 2:  NVIDA Ampere architecture GPU characteristics

Comparing the FP32 TFLOPS between the A30 and A40 shows that the A40 is a more capable GPU for training and pure inference tasks. During our testing, the computer vision applications quickly exhausted the available Media Engines on the A40. Selecting a GPU for computer vision requires matching the available resources needed for computer vision including media engines, available memory, and other computing capabilities that can be different across use cases.

Next, examining the Media Engines description for the A2 GPU column confirms that the product houses 1 video encoder and 2 video decoders. This card will meet the needs of most computer vision applications and is supported for data center use; however, the low number of encoders and decoders, memory, and floating point processing will limit the number of concurrent streams that can be processed. The low power consumption of the A2 increases the flexibility of choice of server for deployment, which is important for edge and near-edge scenarios.

Still focusing on the table above, compare all the characteristics of the A2 GPU column with the A16 GPU. Notice that there are four times the resources on the A16 versus the A2. This can be explained by looking at the diagram below. The A16 was constructed by putting four A2 “engines” on a single PCI card. Each of the boxes labeled GPU0-GPU3 contains all the memory, media engines and other processing capabilities that you would have available to a server that had a standard A2 GPU card installed. Also notice that the A16 requires approximately 4 times the power of an A2.

The table below shows the same metric comparison used in the discussion above for the newest NVIDIA GPU products based on the Ada Lovelace architecture. The L4 GPU offers 2 encoders and 4 decoders for a card that consumes just 72 W. Compared with the 1 encoder and 2 decoder configuration on the A2 at 40 to 60 W, the L4 should be capable of processing many more video streams for less power than two A2 cards. The L40 with 3 encoders and 3 decoders is expected to be the new computer vision application workhorse for organizations with hundreds to thousands of video streams. While the L40S has the same number of Media Engines and memory as the L40, it was designed to be an upgrade/replacement for the A100 Ampere architecture training and/or inference computing leader.

Conclusion

In total seven different NVIDIA GPU cards were discussed that are useful for CV workloads. From the Ampere family of cards we found that the A16 performed well for a wide variety of CV inference workloads. The A16 provides a good balance of video Decoders/Encoders, CUDA cores and memory for computer vision workloads.

For the newer Ada Lovlace family of cards, the L40 looks like a well-balanced card with great throughput potential. We are currently testing out this card in our lab and will provide a future blog on its performance for CV workloads.

References

A2 - https://www.nvidia.com/content/dam/en-zz/solutions/data-center/a2/pdf/a2-datasheet.pdf

A16 - https://images.nvidia.com/content/Solutions/data-center/vgpu-a16-datasheet.pdf

A30 - https://www.nvidia.com/en-us/data-center/products/a30-gpu/

A40 - https://images.nvidia.com/content/Solutions/data-center/a40/nvidia-a40-datasheet.pdf

L4 - https://www.nvidia.com/en-us/data-center/l4/

L40 - https://www.nvidia.com/en-us/data-center/l40/

L40S - https://www.nvidia.com/en-us/data-center/l40s/

Introduction

In today’s world, the deployment of security cameras is a common practice.  In some public facilities like airports, travelers can be in view of a security camera 100% of the time. The days of security guards watching banks of video panels being fed from hundreds of security cameras are quickly being replaced by computer vision systems powered by artificial intelligence (AI).  Today’s advanced analytics can be performed on many camera streams in real-time without a human in the loop. These systems enhance not only personal safety but also provide other benefits, including better passenger experience and enhanced shopping experiences.

Modern IP cameras are complex devices.  In addition to recording video streams at increasingly higher resolutions (4k is now common), they can also encode and send those streams over traditional internet protocol IP to downstream systems for additional analytic processing and eventually archiving.  Some cameras on the market today have enough onboard computing power and storage to evaluate AI models and perform analytics right on the camera.

The Problem

The development of IP-connected cameras provided great flexibility in deployment by eliminating the need for specialized cables.  IP cameras are so easy to plug into existing IT infrastructure that almost anyone can do it.  However, since most camera vendors use a modified version of an open-source Linux operating system, IT and security professionals realize there are hundreds or thousands of customized Linux servers mounted on walls and ceilings all over their facilities. Whether you are responsible for <10 cameras at a small retail outlet or >5000 at an airport facility, the question remains “How much exposure do all those cameras pose from cyber-attacks?”

The Research

To understand the potential risk posed by IP cameras, we assembled a lab environment with multiple camera models from different vendors. Some cameras were thought to be up to date with the latest firmware, and some were not. 

Working in collaboration with the Secureworks team and their suite of vulnerability and threat management tools, we assessed a strategy for detecting IP camera vulnerabilities   Our first choice was to implement their Secureworks Taegis™ VDR vulnerability scanning software to scan our lab IP network to discover any camera vulnerabilities. VDR provides a risk-based approach to managing vulnerabilities driven by automated & intelligent machine learning.

We planned to discover the cameras with older firmware and document their vulnerabilities.  Then we would have the engineers upgrade all firmware and software to the latest patches available and rescan to see if all the vulnerabilities were resolved.

Findings

Once the SecureWorks Edge agent was set up in the lab, we could easily add all the IP ranges that might be connected to our cameras. All the cameras on those networks were identified by SecureWorks VDR and automatically added to the VDR AWS cloud-based reporting console. 

Discovering Camera Vulnerabilities

The results of the scans were surprising.  Almost all discovered cameras had some Critical issues identified by the VDR scanning.  In one case, even after a camera was upgraded to the latest firmware available from the vendor, VDR found Critical software and configuration vulnerabilities shown below: 

One of the remaining critical issues was the result of an insecure FTP username/password that was not changed from the vendor’s default settings before the camera was put into service. These types of procedural lapses should not happen, but inadvertently they are bound to.  The password hardening mistake was easily caught by a VDR scan so that another common cybersecurity risk could be dealt with. This is an example of an issue not related to firmware but a combination of the need for vendors not to ship with a well-known FTP login and the responsibility of users to not forget to harden the login.

Another example of the types of Critical issues you can expect when dealing with IP cameras relates to discovering an outdated library dependency found on the camera. The library is required by the vendor software but was not updated when the latest camera firmware patches were applied.

Camera Administration Consoles

The VDR tool will also detect if a camera is exposing any HTTP sites/services and look for vulnerabilities there. Most IP cameras ship with an embedded HTTP server so administrators can access the cameras' functionality and perform maintenance.  Again, considering the number of deployed cameras, this represents a huge number of websites that may be susceptible to hacking.  Our testing found some examples of the type of issues that a camera’s web applications can expose:

The scan of this device found an older version of Apache webserver software and outdated SSL libraries in use for this cameras website and should be considered a critical vulnerability. 

Conclusion

In this article, we have tried to raise awareness of the significant Cyber Security risk that IP cameras pose to organizations, both large and small. Providing effective video recording and analysis capabilities is much more than simply mounting cameras on the wall and walking away. IT and security professionals must ask, “Who’s watching our IP cameras?  Each camera should be continuously patched to the latest version of firmware and software - and scanned with a tool like SecureWorks VDR. If vulnerabilities still exist after scanning and patching, it is critical to engage with your camera vendor to remediate the issues that may adversely impact your organization if neglected. Someone will be watching your IP cameras; let’s ensure they don’t conflict with your best interests.

Dell Technologies is at the forefront of delivering enterprise-class computer vision solutions.  Our extensive partner network and key industry stakeholders have allowed us to develop an award-winning process that takes customers from ideation to full-scale implementation faster and with less risk.  Our outcomes-based process for computer vision delivers:

• Increased operational efficiencies: Leverage all the data you’re capturing to deliver high-quality services and improve resource allocation.
• Optimized safety and security: Provide a safer, more real-time aware environment
• Enhanced experience: Provide a more positive, personalized, and engaging experience for customers and employees.
• Improved sustainability: Measure and lower your environmental impact.
• New revenue opportunities: Unlock more monetization opportunities from your data with more actionable insights

Where to go next...

Beyond the platform - How Dell Technologies is leading the industry with an outcomes-based process for computer vision

Dell Technologies Workload Solutions for Computer Vision

Secureworks

Virtualized Computer Vision for Smart Transportation with Genetec

Virtualized Computer Vision for Smart Transportation with Milestone

Introduction

Ansible is an astoundingly useful and convenient DevOps tool that helps streamline the process of managing remote hosts. However, it does have a learning curve and requires at least some technical knowledge to use efficiently given that it is a CLI (Command Line Interface) tool. Fortunately, there are several modern, feature-complete User Interface options for managing and running an Ansible instance on a remote server that can be controlled directly from a web browser. This, along with their open-source nature, makes the process of using Ansible and running playbooks much more intuitive and convenient; even for experienced team members that are familiar with using Ansible from the command line. This blog describes our evaluation of the most relevant aspects of each UI, including their features and accessibility.

AWX: The premiere open-source UI

Figure 1: AWX Dashboard

AWX is the most well-known and feature complete UI for Ansible. It provides a sleek and intuitive interface that neatly organizes the configuration options by category and allows for the use of Role-Based Access Control. This gives users the option to regulate who can see or modify certain settings and files. Key amongst these are job templates, which serve as a set of parameters and definitions that allow the same job to be executed many times. Additionally, the built-in dashboard provides a visually pleasing yet extensive overview of past jobs as well as their outcome, along with other relevant information about the AWX configuration. Last, but certainly not least, AWX allows for secure and encrypted storage of credentials and vault passwords, allowing them to be shared between team members safely and effortlessly.

Ansible Semaphore: Easy to start, easy to use

Figure 2: Ansible Semaphore UI

Compared to AWX, Ansible Semaphore is more simplistic in every sense, with its straightforward installation process and streamlined UI coming at the cost of features that the other UI options we evaluated have. For example, Ansible Semaphore does not support high availability, meaning that it cannot automatically recover from component failure and can result in longer downtimes. However, this tool can easily be setup to pull ansible playbooks from GitHub, store credentials for GitHub/your machine, and run playbooks through a simple task template. Inside of a template you can specify hosts to run on (inventory), variables (environment), and extra command line arguments. That being said, Sempahore’s best feature is quite possibly its dashboard. Designed on Google’s Material UI, Ansible Semaphore’s dashboard is very easy to navigate and has a simplistic look to showcase the critical information for each run.

Rundeck Community Edition: More than just Ansible

Figure 3: Rundeck UI

Rundeck Community Edition gives users the basic functionality that is needed to execute playbooks inside a UI and, just like Ansible Semaphore, it is very easy to install and get up and running. Rundeck is a general automation tool so you can do more than just execute ansible playbooks, but the dashboard is not quite as easy to use as Ansible Semaphore and it is not as visually appealing.  Some features of Rundeck CE include creating multi-step jobs, running shell commands, and executing local commands.  While the community edition boasts many features beyond just running Ansible playbooks, the most desired features such as high availability and certified enterprise plugins are reserved for the enterprise or cloud editions.

Custom Coding the UI: Full Control

If none of the aforementioned solutions seem ideal to you, or they do not appropriately address your requirements, you always have the option of designing and creating your own UI solution. Doing so will grant you an appropriately scaled solution that meets all of your needs and requirements while also allowing you to express your creativity and originality. For example, if you want to offer some niche features to the UI like a “revert operation” that will undo a previously run playbook or displaying the completion percentage of a job that is in progress, then a custom UI could be your best option. However, this approach requires an immense amount of effort compared to the other options we discussed to develop and properly maintain a secure solution. One approach we investigated was to build a robust REST API running on an ansible-capable remote host for the backend services and a web frontend running on the same host. The two components of the application can then use HTTP requests to communicate and run and/or modify the pertinent files locally on the server.

Conclusion

Figure 4: UI Comparison Summary

Leveraging an Ansible UI is a great way to easily extend the functionalities and capabilities of Ansible to non-CLI experts by making server management and automation more accessible. Namely, it provides a less error prone execution and a more consumable way of seeing job progress and output for all users. Every option described above has its pros and cons, and it’s important to factor in the setup/installation process of each option. Incidentally, despite AWX being our top choice due to its maturity and feature set, its installation process is notoriously difficult. Because of this, our team decided to make our own guide describing what made the installation work for us. If you are interested in learning more about AWX’s setup process, feel free to check out the installation tutorial blog created by our team by clicking on the link right here!

Introduction

As the use of virtualization (VMs and containers) expands rapidly in many organizations, automation is needed for virtual server management to address the tedious and repetitive tasks. Ansible is a powerful tool for automation, deployment, and configuration management that has historically required living on the command line interface (CLI). The open-source version of Ansible Tower is AWX - a web-based user interface (UI) for Ansible. When we wanted to explore how AWX works we quickly realized that the existing AWX installation guides need an overwhelming amount of trial and error to make work. This blog presents how to execute a reliable installation process and also explains the automation of the process that reduced our installation to just running a single command. Our comparison and selection of an Ansible UI from a list of 4 options is documented in this blog if you want to learn about that effort.

AWX overview

AWX is a UI solution that sits on top of the Ansible CLI supporting functionality such as visualization of host management and running job status including Ansible playbooks, specification of job parameters, and login authentication. Since AWX is an open-source version of an enterprise product, it has very limited official documentation. During our testing we encountered issues such as insufficient dependency specs, failure to pull Docker images, and inability to visualize our AWX instance. There are many different unofficial guides, but unfortunately, very few of them work reliably without the need for debugging. This blog documents a simple and reliable method for installing AWX.

Prerequisites

Our goal was to deploy AWX on a management system that can connect to a workload environment for VM automation. The only prerequisites you need to get started is to have Ansible installed in the management system and to have your Docker Hub login credentials available. It is crucial to store your Docker Hub username and password in a file named secret.enc under the vars folder of the playbook in following format:

Then, you should encrypt the file using a command similar to the one below using Ansible Vault.

Testing system details

Table 1: System Details

Components to be deployed by the Installation Playbook

Table 2: Components to be Deployed

Installation Process

Figure 1: High-level Overview of the Components in the Installation

The goal is to have a running instance of AWX accessible with a browser. With this design, the user only needs one command to run the playbook that installs AWX. This command asks for the sudo permission so the playbook can use elevated privileges whenever necessary. A vault password is also requested to use the encrypted Docker Hub credentials described above for a successful login into Docker. Minikube and Docker are automatically installed by the installation playbook. Minikube is the backbone of this installation process and provides the resources that the AWX instance is installed on. Docker ensures that the Minikube pods are ready for initializing AWX. 

Here is an outline of the background process for the Ansible playbook:

1. The playbook installs the necessary prerequisites. 

2. The playbook logs into and sets up Docker.

3. A Minikube instance is run with specified configurations. 

Figure 2: Creation of Minikube Instance

4. An image pull secret is created and patched to the service account based on the Docker Hub credentials for successful image pulls.

Figure 3: Creation of an Image Pull Secret

5. AWX operator is deployed and it runs as a pod[PD1] .

Figure 4: Deployment of AWX Operator

Figure 5: Running AWX Operator Pod[SM2] 

6. AWX instance is deployed with 4 pods for the instance and 1 pod for postgres.

Figure 6: Deployment of AWX Instance

Figure 7: Deployment File (ansible-awx.yml)

Figure 8: Running Pods for AWX Instance and Postgres[SM3] 

7. Expose the port for the AWX instance through port forwarding and display the IP address and login information for accessing the instance.

After running the Ansible install AWX playbook, the login information including username, password, and IP address with port for the AWX instance will be displayed as a part of the detailed output. 

Figure 9: An Example of the Playbook Output with Login Information

Then, you can access the dashboard for AWX using your host’s IP address and port 32483 with login credentials provided from the above output. 

Figure 10: AWX Dashboard After a Successful Installation and Login

Common errors and solutions

A few errors that you may encounter during the installation process:

• ImagePullBackOff: Kubernetes fails to pull container images from Docker Hub. It is important to make sure that you are logged into Docker Hub successfully using Ansible Vault. You can also login manually using docker login, but it is less secure.
• Certificate and connection related errors: Ensure that VM resources are sufficient for running Minikube with predefined specifications. If multiple users are working on the same server with several Kubernetes clusters, such errors may also occur due to resource limitations.

Conclusion

This blog introduces a quicker and more convenient way to reliably install AWX. With a simple goal of having a running AWX instance on a server, this blog demonstrates a straightforward solution to achieve that goal while many other existing guides need much more customizations and configurations for the successful execution of an AWX deployment.