The Cognex computer vision system within this Dell Validated Design provides functions such as product classification, component defect detection, and Optical Character Recognition (OCR) capabilities to aid in production quality and process. Using image processing and AI, the Cognex application builds a machine learning model that can be used with industrial cameras to identify product quality issues and integrate with other systems in the manufacturing environment (for example, PLCs, Process Control, MES).

The following sections provide more in-depth explanations for the topics covered here.

The major components of a machine vision system include lighting, lens, image sensor, vision processing, and communications. For evaluating and validating the performance on edge compute platforms, the focus was on the vision processing component. Specifically, vision processing consists of algorithms that review the image and extract required information, run the necessary inspection, and make a decision. The Cognex vision system as deployed within this DVD consists of three applications: VisionPro, VisionPro Deep Learning, and Cognex Designer.

Edge compute-based vision platform—Edge compute-based systems readily interface with direct-connect cameras and are well supported with configurable machine vision application software. In addition, edge platforms provide custom code development options using familiar and well-supported languages such as Visual C/C++, Visual Basic, and Java, as well as graphical programming environments. In the evaluation, each of the Cognex applications was installed on various Dell edge platforms and performance tested for scalability and sizing.

VisionPro—VisionPro is Cognex’s primary vision software for designing and deploying vision applications on PC-based platforms. It can perform a wide range of functions including geometric object location and inspection, measurement, alignment, as well as specialized 1D and 2D functions.

VisionPro Deep Learning—VisionPro ViDi Deep Learning is a deep learning-based image analysis software designed for factory automation. Its algorithms are optimized specifically for machine vision, with a graphical user interface that simplifies neural network deployment.

Cognex Designer—Designer incorporates VisionPro and Deep Learning, allowing developers to build complete vision applications and take full advantage of the powerful VisionPro tool library.

Image Acquisition—Cognex acquisition technology supports all types of image capture: analog, digital, color, monochrome, area scan, line scan, high resolution, multi-channel, and multiplexed. In addition, Cognex supports hundreds of industrial cameras and video formats covering the complete range of acquisition requirements typically used in machine vision.

I/O Communications—Communications with systems outside of the vision system is an integral component of an integrated production environment. VisionPro and Cognex Designer allow creation of an application that can communicate with a compatible Programmable Logic Controller (PLC) across an industrial network. Customized I/O can also be developed using scripting tools.

Although architectures may vary depending on use-case specifics, the components that make up a vision system remain consistent, with how they are implemented and optimized being an important factor. The following architecture describes how the different vision components were architected:

Use Cases—Several defined use cases were evaluated specific to the Cognex vision system based on their relevance to AI/ML in the manufacturing process. Those use cases are as follows:

• Anomaly and defect detection—Identification of defects, irregularities, and other manufacturing flaws using 2D and 3D vision inspection.
• Pattern detection and classification—Use of object location tools and geometric pattern-matching technology to locate patterns regardless of rotation or scale, as well as use of classification tools to verify characteristics of completed assemblies.
• Complex text and OCR—Use of font-trainable optical character recognition (OCR) and verification (OCV) tools with auto-tune capability to read alphanumeric characters, barcodes, and data matrix markings on product surfaces.

ViDi training phase—As with traditional rule-based vision tools, how the ViDi deep learning tools are trained is the largest determinant of their runtime performance. Unlike rule-based vision tools, successful deep learning training is largely a function of the number and representativeness of the images that you collect for your image training set and the consistency with which you label them.

From an edge compute platform perspective, the training phase is the most resource-intensive stage. Neural networks form the basis of deep learning and are designed to run in parallel, with each task running independently of the other. GPUs are optimized for parallel processing of large data sets, and complex mathematical data is used to train neural networks. It is highly recommended to run deep learning training for model development on an edge platform that is equipped with a GPU.

ViDi runtime phase—The runtime phase of the Cognex ViDi deep learning tools is simpler than the training phase. The runtime components are comprised of sampling, neural network processing, and results formation. Although a GPU is not required for the runtime phase, when ViDi Deep Learning is used in ML model generation, performance benefits greatly when a GPU is present.

ViDi tools—The Cognex ViDi tools are a collection of machine vision tools which address programming challenges using deep learning. While the tools share a common engine, they differ in what they are looking for in images. More specifically, each tool has a different focus analyzing either single points, individual regions, or complete images. Below are the four tools used in ViDi deep learning:

• Blue Locate tool—The Cognex blue Locate tool is used to identify and locate specific features or groups of features in an image. The output of the tool can be used to provide positional data for other downstream ViDi tools. When using the tool, you provide a training set, and then identify the features within the images.
• Green Classify tool—The green Classify tool is used to identify and classify an object, or the entire scene, in an image. It can also be used to sort objects for further analysis. Once a green Classify tool has been trained, it will assign a tag to the image, which the tool uses to assign a class to an image. The tag is represented by a label and is given a percentage declaring the confidence the tool has for the classification it has assigned.
• Red Analyze tool—The red Analyze tool is used to perform two types of detection, anomaly detection, or defect detection, and segmentation, depending on the mode of operation, Unsupervised or Supervised. The reason for the different modes of operation is due to the subtle differences that can exist between the two types of defect detection:
  • Unsupervised Mode—In Unsupervised Mode, the red Analyze tool is taught the appearance of the good parts— and only the good parts (including any and all acceptable variations)—so that it finds anomalies from the learned, normal appearance. As such, the tool tries to find a compact description of the object to be inspected. The tool's ability to find anomalies, both in terms of selectivity and specificity, depends largely on the compactness of the model. If, for instance, the part comes in different types and different orientations, the compactness gets reduced, and ambiguities between part types and orientations may prevent the tool from finding certain anomalies.
  • Supervised Mode—In Supervised Mode, the red Analyze tool is taught the appearance of defects. It does not form a model of the inspected part, and as a consequence, it is much less dependent on part configuration, type, or the conditions during image acquisition. However, the red Analyze tool in Supervised Mode will need to form an explicit model of the different types of defects. Thus, it needs both good and bad samples to train. In particular, it requires a representative collection of defects.
• Blue Read tool—The Cognex ViDi blue Read tool is used to perform OCR within an image. From cleanly printed characters to strongly deformed characters on very noisy backgrounds, the blue Read tool can identify and read characters by learning from annotated images. To train the blue Read tool, the user must provide images where the targeted characters are marked.

ViDi tool chaining—Each of the Cognex ViDi tools has a unique output, and that output can be used as an input to another tool in what is referred to as tool chaining. In the following figure, a blue Locate tool has a node model, made up of three features (H, B, and T), which provides a location (this is similar to using a pattern tool in a traditional vision application to create a fixture). The location output by the blue Locate tool is used by the red Analyze tool to orient and create its view, allowing the detection analysis to be performed in the same area of the image, based on the node model output by the blue Locate tool.

Finally, the green Classify tool is used to classify the output of the red Analyze tool, based on the classification criteria established for the green Classify tool.

• Integration options
  • ViDi Deep Learning integration with Cognex Designer is enabled through the ViDi plug-in that is loaded during Designer startup. This provides Designer access to the full suite of deep learning tools (such as red Analyze), while allowing a tool-chaining integration with the other VisionPro tools, scripting, and HMI/dashboarding capabilities.
  • Microsoft Visual Studio may be used with the ViDi Deep Learning libraries to develop a customized integration with other third-party applications to meet the design needs for the specific vision system deployment.

Cognex Designer is an integrated development environment for developing 2D and 3D machine vision applications. The software allows machine vision developers to create flexible solutions to complex machine vision challenges.

Cognex Designer provides multi-camera support, drag-and-drop application and HMI development environments, high-speed discrete input/output and factory floor networking protocol support, and advanced scripting and user management features.

Designer Project—Developing a project in Designer involves the following:

1. Add the cameras and communications devices that will provide the data for your application.
2. Tasks are added to process the data and are configured in a Task. This is the area of the application where you will define the vision processing capabilities and the inputs and outputs to the system:
   • Task Blocks execute from left to right and provide results for other blocks. Connect all of the functional blocks by connecting their input and output pins.
3. Define the flow of data:
   • Utilize the Tag Manager to exchange information and data throughout the application. Tags will automatically route the data from Tasks to a Database or I/O.
   • Using the Scripting tools, create custom scripts to interact with the data and tools in the project.
   • Create Databases for storing data from the project.
4. Create the human-machine interface (HMI) for operator interaction with the application during runtime. This is done using Pages, which are used to convey the produced data for operator interaction.
   • Design the HMI interface using Page User Inputs, Page Graphics, Page Data Display, Page Vision Controls, and Displays elements.
5. Put the project in Test Mode to ensure that everything is functioning properly.

   

   

Industrial communications—The industrial communications protocols (also known as factory floor protocols) supported by VisionPro and Cognex Designer allow the creation of an application that can communicate with a compatible Programmable Logic Controller (PLC) across an industrial network.

VisionPro and Designer support the following protocols:

• PROFINET (Siemens PLCs)
• EtherNet/IP (Rockwell PLCs)
• SLMP (Mitsubishi PLCs)

The Cognex implementation of the factory floor protocols is based on the Cognex Network Data Model (NDM), a protocol-independent abstraction that allows your application to work with any supported protocol without modifications.

Designer project deployment—The final step of project development is to deploy the application. Deployment allows the application to be run by end users without requiring access to the Cognex Designer Integrated Development Environment (IDE). This runtime application is typically installed on an edge compute platform at Purdue Level 2 of the factory floor.

Once the application development is complete, the project can be deployed and structured so that the application cannot be modified. You can then launch the deployed application by going to the deployment folder and running the Launcher.exe file, or by using the shortcut on the desktop. Both the Cognex VisionPro application and the ViDi Deep Learning application must be installed on the end-user workstation for the Cognex Designer application to properly execute.

Scripting—Cognex Designer provides a powerful scripting environment using Microsoft C# .NET. You can write standard C# .NET code and execute it within the Cognex Designer environment.

• Cognex ViDi 4.1.3
• VisionPro 9.8
• Cognex Designer 4.3

VisionPro Deep Learning, Cognex Designer, and the VisionPro runtime are supported on compute platforms running Windows 10 Professional (64-bit).

• System Memory (RAM)—The minimum amount of RAM is equal to the sum of all GPU memory. For example, if you have two NVIDIA GeForce RTX 3080 Ti GPUs, which each have 10 GB of memory, the PC should have 20 GB of RAM.
• Graphics Processing Unit (GPU)—A GPU greatly improves the speed and performance while training ViDi tools. GPU selection and performance guides are highlighted below. The Cognex applications only support NVIDIA GPUs.
• USB 2.0—USB port for a permanent Cognex Security Dongle (containing the training license) connection via USB 2.0.

• CPU—When selecting a CPU, a higher CPU clock speed rate and multiple core processors result in superior runtime tool execution. If your application relies on the blue Locate tool, it is more sensitive to clock speed rates, particularly with complex model matching applications.
• System memory (RAM)—Per above, when selecting the system RAM, specify the greater of:
  • The sum of all GPU memory. For example, if you have four NVIDIA GeForce RTX 3080 Ti GPUs, which each have 10 GB of memory, then the PC should have 40 GB of RAM.
  • One and a half times the typical workspace size. For example, if your typical workspace is 20 GB, the PC should have a minimum of 30 GB of RAM.
• System storage—It is recommended to provide a minimum amount of free space that is two times the typical workspace size times the number of concurrent users.
• Graphics Processing Unit (GPU)—When selecting an NVIDIA GPU, a unit with a higher core clock frequency, CUDA cores, and Tensor cores results in faster computational speeds.

• CPU—When selecting a CPU, a higher CPU clock speed rate and multiple core processors result in faster runtime tool execution. If your application relies on the blue Locate tool, it is more sensitive to clock speed rates, particularly with complex model matching applications.
• Graphics Processing Unit (GPU)—Technically, a GPU is not required on runtime deployments, but it greatly improves the speed and performance while executing ViDi tools. Thus, it is highly recommended to use a GPU-enabled platform when deep learning models are used in production.
• System memory (RAM)—The minimum memory requirements for the deployment PC are dependent on the size of the application to be run, plus any other applications that need to be running in the background. The memory sized for the XR4000 deployment PC used in validation testing was 16 GB.

The following are the Dell Technologies compute platforms that were evaluated with the Cognex VisionPro applications in this document. The recommended hardware configurations and their performance variances are highlighted below, with detailed scaling and sizing for each application and specified platform contained in the Sizing and Scaling Guidance chapter of this document:

• Development environment for ViDi Deep Learning and Cognex Designer
  • PowerEdge (XR11/12) with GPU (small)
    • 48 GB RAM
    • 8 vCPU
    • NVIDIA T4 GPU
  • VxRail
    • 48 GB RAM
    • 8 vCPU
    • NVIDIA T4 GPU
• VisionPro runtime
  • EGW-5200 Dell Edge Gateway (VPro without ViDi)
    • 8 CPU
    • 32 GB RAM
  • XR4000 with A2 GPU (VPro with ViDi)
    • 8 vCPU
    • 16 GB RAM

A number of elements affect the performance of the vision applications, and becoming aware of the trade-offs between cost and design can aid in the development of a more robust solution. Throughput refers to the total number of images that can be processed per unit time. If your application can process multiple streams concurrently using different threads, it may be able to improve system throughput, although individual tool processing would be slower.

The parameters below can be used as part of a performance toolkit:

• Application design
  • Use a small number of tools per stream. For example, an Analyze tool runs at the same speed, regardless of how many defects it finds, and a Classify tool can classify into 2 classes or 2000 classes at the same speed.
  • Avoid image conversions—During tool operation, the image must be sampled for processing by the neural network. This sampling requires a raster (uncompressed) format image such as a bitmap. Performing this conversion takes time. Attempt to configure the application using a single-channel grey-scale image.
  • Reducing the amount of processed data improves processing speed by reducing the total amount of data processed. This can be achieved by:
    • Using a smaller ROI
    • Using a mask
    • Using as few image channels as possible
  • Multi-threading
    • On systems with multiple GPUs, processing multiple streams concurrently allows tools to execute in parallel, increasing throughput.
    • On single-GPU systems, you can configure the system to allow multiple processes to make use of the same GPU. This allows a higher GPU occupancy and can improve throughput, although tool execution time will increase.
• Tool parameters
  • Feature size—At runtime, ViDi tools need to sample the entire input image. The feature size determines the number of samples required for a given image size. The larger the feature size, the fewer the samples.
  • Sampling density—Similar to feature size, the sampling density determines the number of samples required for a given image size. An increase in speed may result in a lower sampling density, so users must consider the trade-off of missing features or defects.
• NVIDIA GPU selection and configuration
  • NVIDIA device type
    • The number of CUDA cores is directly related to high-precision processing speed and training.
    • The number of standard Tensor cores is directly related to processing speed and training speed.
    • The number of Turing Tensor cores is related to processing speed in low precision mode only. These cores do not affect standard precision processing or training speed.
  • NVIDIA driver model
    • Consumer-grade gaming-oriented NVIDIA devices only support the WDDM device driver model. This driver is intended to support graphics display, not computation. Professional-grade NVIDIA cards support the TCC driver mode, which provides better performance and stability.
    • It is recommended to select a Quadro, Tesla, or Titan-branded NVIDIA card.
  • Optimized memory
    • ViDi optimized memory, which is enabled by default, improves performance by overriding the standard NVIDIA GPU memory management system.
    • Ensure the card has at least 4 GB of GPU memory.
• Multiple GPUs
  • Except under very narrow circumstances, use of multiple GPUs in a single system does not reduce ViDi tool training or processing time. But multiple GPUs can provide the following:
    • Increase system throughput when your application uses multiple threads to concurrently process images.
    • Increase training productivity by allowing you to train multiple tools at the same time.
  • There is one circumstance under which multiple GPUs can be used to reduce tool processing time. If configuring the system in MultipleDevicesPerTool mode, then all installed GPUs are treated as a single GPU during processing. This means that only one tool can be processed at a time for the entire server.
• Training time
  • Reducing tool training time does not affect performance at runtime, but it can improve the productivity of your development team.
  • ViDi training uses a mixture of CPU and GPU resources. When considering training specifically, there are three phases: computing image statistics, building the model, and then processing the image set with the newly trained model. The model building phase of training usually takes the longest, and it is an iterative process. Each iteration requires that the tool generate training data from all of the training images. If the images are in a non-BMP format, they need to be converted to BMP for each iteration.
    • Tool training is always single-threaded and single GPU. You cannot make training faster using multiple GPUs.
    • Using multiple GPUs can improve your productivity in that you can train multiple tools concurrently.

For more information, see the following link: Performance Guide for ViDi

The sections above cover the high-level details of what should be considered, planned, and configured to successfully design a Cognex vision system. For specific scaling and sizing of the components in this chapter, see Cognex sizing and scaling guidance. Best practices and sample use-cases for computer vision are contained in the Accelerated time to value with Cognex section.