The ability to automate manufacturing-specific tasks such as parts component assembly verification, inspection of products for defects, and validation of label identification can all enhance the overall product quality in an industrial environment. These capabilities are brought to bear in the Dell Validated Design for Manufacturing Edge using the Cognex vision system. This chapter will provide a more in-depth understanding of those capabilities as well as highlight some best practices to attaining a successful outcome.
Cameras and image capture
In factory automation, deep learning-based software like VisionPro ViDi can perform judgment-based part location, inspection, classification, and character recognition challenges more effectively than humans or traditional machine vision solutions.
This is possible using digital industrial cameras to perform high-resolution image captures for analysis by the machine vision software. Camera selection can vary based on the intended purpose and application of the vision system. Cognex Designer and VisionPro ViDi support hundreds of manufacturer cameras. These can be identified with the support matrix. For more information, see VisionPro Camera Support from Cognex.
For connectivity and networking between the camera and the edge platform running the vision software, there are two predominant interfaces: USB and GigE. USB leverages the USB interfaces on the edge platform to both power the camera and for I/O communication. GigE cameras rely on the GigE Vision protocol for setup and communications with the VisionPro software.
GigE Vision is an interface standard for industrial image processing by digital cameras that originated in the field of machine vision. The use of Gigabit-Ethernet standards allows industrial cameras to integrate seamlessly into existing network systems. GigE Vision allows for fast image transfer, using low-cost standard cables over very long lengths, with support for multi-camera setups.
The Cognex GigE Vision Configuration Tool is a utility within the Cognex Designer application for easy setup and camera configuration.
Another critical aspect of image capture in an industrial environment is lighting. Failure to properly illuminate a target can result in the loss of information and productivity. A lighting technique involves a light source and its placement with respect to the part and the camera. There are a host of lighting techniques that compensate for the disparate illumination’s environments present on a factory floor (ceiling lights, sky lights). Cognex provides an interactive tool at the following link to assist with illumination techniques:
See Lighting Advisor on the Cognex website for information about minimizing ambient light and image capture disruption.
Defect detection using ViDi deep learning
An element of the computer vision development for consideration is which application is best suited for a specific task. ViDi Deep Learning has tools to perform many functions. Leveraging the VisionPro toolset within Cognex Designer to perform certain classifications without the need for ViDi’s Machine Learning can conserve resources (for example, the Blob Tool).
The following figures illustrate a Cognex Designer project using classic VisonPro tools in combination with ViDi Deep Learning to detect anomalies in printed circuit boards (PCBs) on a manufacturing line.
Image ingestion occurs with a camera input (or image file) on the left, which feeds a classic VisionPro Toolblock. VisionPro uses a Blob Tool to inspect for the presence of a RESET button on the PCB. Figure 128. Cognex Designer The result of that inspection is a boolean output relayed to a ScriptBlock that updates the TagManager database within Designer. This indicates whether the RESET switch is present or missing. Figure 129. VisionPro Tool Block – button inspection
In parallel to the switch inspection is a ViDi Deep Learning ToolBlock, as shown in the following figure. In this ViDi subtask, a Machine Learning model is developed to perform two main functions:
Perform a USB connector inspection for defects
Perform an inspection of connector pins to determine if any are bent or missing
Figure 130. ViDi Deep Learning SubTask
The ViDi red Analyze tool is used for each of these inspections. Operating in Unsupervised Mode, the Analyze tool is trained on known “Good” images to develop a machine learning model that can detect anomalies in both the USB connector and the pin connector, as shown in the following figures.
Figure 131. ViDi Deep Learning ToolBlock – USB connector Figure 132. ViDi Deep Learning ToolBlock – pins
The first of the previous figures depicts the model for the USB_Connector after it is trained. A red square highlights defective USB connectors identified in images, and a green square highlights a connector that has no defects. Similarly, the next figure shows a pin connector that is highlighted with a green square, meaning it is determined to have no defects.
Once the tool has been trained, results are displayed in graphs and a Confusion Matrix within the subtask. These measurements can be used to help predict outcomes and tune the model to within acceptable results for a given application.
Cognex Designer has a built-in capability for a runtime operator interface to monitor and control the vision application. The following figure shows the runtime HMI for the PCB project.
Figure 133. Cognex Designer PCB HMI
Deciphering complex text using ViDi OCR
Another feature of the ViDi Deep Learning is it’s ability to decipher complex text. This might be alphanumeric characters imprinted on a product (for example, serial number and lot number). The tool's strength is in its ability to work on difficult projects with low contrast, low resolution, or deformed characters. The tool can read characters that traditional machine vision tools have difficulty reading, particularly deformed or curved characters on noisy backgrounds.
In the following figure, a ViDi Read Tool is used to identify numbers imprinted on a product’s surface. The next two figures show the Cognex Designer project with image acquisition, the ViDi Read ToolBlock, and the Output ScriptBlock.
Figure 134. OCR using ViDi Project
Figure 135. ViDi OCR Read ToolBlock
The Read tool is a pre-trained tool, which provides it with a generic baseline of characters for reading performance, without the need for training. However, when using the tool, a training set of images is provided and the region around the characters to be read is set.
The tool automatically identifies and reads the characters, then produces markings of the characters, which the user can accept as labels for further training. After at least one instance of the characters is labeled, the tool can be trained. Test images that were not used during the training phase can then be used to evaluate model performance.
Figure 136. ViDi Read Tool
Following model training, a Confusion Matrix is displayed with a graphical representation of the accuracy scores for each character (labeled as feature), as shown in the previous figure.
The output of the results are then fed to the ScriptBlock for use in export utilities or HMI display to an operator.
Figure 137. Cognex Designer HMI
Bottling line detection using ViDi Classify
Object classification is another tool within ViDi Deep Learning that can classify types of defects using image analysis. The following example represents a computer vision system for a bottling line. The Classify tool is used to detect two separate defects:
Bottle cap is missing or damaged.
Bottle is overfilled or underfilled.
The following figure shows a Cognex Designer project. Image ingestion starts on the left and feeds into the ViDi Classify ToolBlock. The resulting images and inferences are sent to the ScriptBlock for use in outputs and the HMI Operator Dashboard. Figure 138. Cognex Designer bottle Classify project
The ViDi ToolBlock in the following figure contains two Classify tools: one for checking the fill-level of the bottle, and the other to verify that the cap is properly fit on top. The output of each Classify tool is the image file and a parameter indicating if the bottle is overfilled, underfilled, or cap off. Figure 139. ViDi Classify ToolBlock
The following figure shows the details of the ViDi Classify training model - specifically for checking the bottle fill level. An image data set was provided which contained images that were overfilled, underfilled, and normal. Each image was labeled as such, and the Classify model was trained on the images. The resulting Confusion Matrix indicates the level of accuracy of the model after operating on a test image data set. Figure 140. ViDi Deep Learning Classify training – fill-line
The following figure shows the ViDi Classify tool that performs the cap inspection. Figure 141. ViDi Deep Learning Classify ToolBlock – cap
The following figure shows the ScriptBlock for correlating the ToolBlock output and relaying that information to a database, file, or Operator HMI. Figure 142. Cognex Designer output ScriptBlock
The following figure shows the HMI that was built for the deployment runtime. It has the ability to record bad images, along with inference data (for example, overfilled or cap off), for use in export to other process elements. The event board also provides an operator interface to allow feedback on the image analysis on the production line. Figure 143. Cognex Designer – bottling line HMI
Summary
The Cognex vision system can provide insights into the manufacturing process as it relates to product assembly verification, defect detection, and reading of complex text, as the previous examples have shown. Determining where to utilize the technology for the greatest economic benefit can be just as important. The following link provides guidance on the cost savings that can be attained, depending on the vision application:
Additionally, the development life cycle of a vision system should not be overlooked. The following figure provides a flow of the process and the elements involved in designing and deploying a successful project. Having a well thought-out plan and being able to convey the overall development process to stakeholders are part of the path to success.
The ViDi ToolBlock in the following figure contains two Classify tools: one for checking the fill-level of the bottle, and the other to verify that the cap is properly fit on top. The output of each Classify tool is the image file and a parameter indicating if the bottle is overfilled, underfilled, or cap off.
The following figure shows the details of the ViDi Classify training model - specifically for checking the bottle fill level. An image data set was provided which contained images that were overfilled, underfilled, and normal. Each image was labeled as such, and the Classify model was trained on the images. The resulting Confusion Matrix indicates the level of accuracy of the model after operating on a test image data set.
The following figure shows the ViDi Classify tool that performs the cap inspection.
The following figure shows the ScriptBlock for correlating the ToolBlock output and relaying that information to a database, file, or Operator HMI.
The following figure shows the HMI that was built for the deployment runtime. It has the ability to record bad images, along with inference data (for example, overfilled or cap off), for use in export to other process elements. The event board also provides an operator interface to allow feedback on the image analysis on the production line.
