TRADITIONAL MACHINE VISION OR A DEEP LEARNING?
A lot of conventional industrial control tasks, such as detection and counting of items, checking for correct assembly, detection of defects, reading of identification markings and labels, are quite successfully addressed with traditional machine vision systems based on deterministic algorithms. Such algorithms are reliable and efficient for the manufacturing products with strict standards, for the productions where it is possible to fix the position of the controlled objects in relation to camera, there is fixed dedicated lighting, etc.
At the same time, however, machine vision systems based on deterministic algorithms are ineffective for the tasks that have large variation in controlled objects and their deviations from the engineering sample prototype due to presence of some abnormality. These abnormalities may lead to functionality breakdown of controlled object – i.e. they can lead to substantial product defects; or be within the specified quality norms (the examples of such abnormalities include anomalies on a surface of metal-roll, paper or film). The use of artificial intelligence, and particularly the deep machine learning methods are necessary for addressing these tasks.
Image 1: Traditional machine vision |
versus deep learning. Performed tasks |
Only experts with sufficient knowledge and experience can make informed choice between the use of traditional machine vision technologies and deep learning (image 1). Through a close cooperation between system engineers at the production facility and the system developers it is possible to make well inform decision for the system that would suit the production facility the most. In addition, a close cooperation between an enterprise and machine vision system manufacturer opens a possibility for efficient conducting of additional tests and controlled experiments which mutually benefits both involved parties.
FACTORS THAT NEED TO BE CONSIDERED WHEN IMPLEMENTING A PROJECT BASED ON DEEP LEARNING
Implementation of the projects based on deep learning, however, has a number of peculiarities. First of all, the realisation should be formed that this is a not a one-day project. After such common steps for machine vision projects as a choice of optical scheme, collection and mark-up of data, there is a stage of choosing neural network archetype and a following process of iterations of system learning. Most often it is impossible to predict the type of quality results that would be reached in the real-world production environment (e.g. mistakes of missing and/or false positive alerts or critical/non-critical classification mistakes). As a result, there are scientific and technological risks at initial stage of the projects based on deep learning. These risks should be shared by both developer of the system and a client.
However, based on past experience, not all enterprises are ready to invest into innovative developments and pilot projects. Instead, they prefer to receive a final product that is ready for immediate use. Based on experience, such approach of clients to the projects that use artificial intelligence to deliver unique and customised final solutions is ineffective. Unwillingness of enterprise management to invest into the project development stage may lead to long production cycle or even an inability to deliver the final solution at all.
A few years ago, our company has developed two advanced solutions for automatic sorting of diamonds based on color and shape for ALROSA company. The system controls raw natural diamonds of 1-5 mm in a diameter that enter the control area at a rate of 20 diamonds per second. The machine vision cameras collect the visual data of diamond while it is being transported and when the diamond is in a free-fall. The system sends the collected images to a server where the software analyses properties of the diamonds and classifies them among different classes (image 2).
Image 2: Collection of data for learning of diamond |
classification model by color |
All these steps occur almost instantaneously while the diamond is in a free fall and travels the overall distance of 30 cm, after which the system separates it and the diamond is being transported into designated container. Overall, our company spend four years on this project – from initial stage of conducting experiments for R&D, and creation and testing of pilot system to delivery of the final system. The development of the system took considerable time, however the client had clear understanding that development of such unique system should be performed in separate stages to minimise potential risks for both involved parties.
LOCAL IMPLEMENTATION OF CONTROL SYSTEM OR TOTAL AUTOMATION OF PRODUCTION?
When considering an implementation of machine vision project at the production facility, the clients first try to estimate an economic efficiency of such project. If the motivation for the project is a replacement of production line operator, the cost of the system is most often compared against the variable costs for operator. In reality, it is quite complicated to reliably identify an actual effect from system implementation. For example, if we consider a system for quality control of manufactured products, we should account for not only such indicators as the operator wage, decrease of defects or number of returned or recalled products, but also an increase in the customer brand loyalty and overall improvement in the perceived image of the company.
Assessment of the time that it will take the project to payback its original investment is another topic for consideration. When trying to estimate this period, the clients most often view it as acceptable for local implementation of control system. At the same time, however, when considering an comprehensive automation of entire production facility that have several control points, and where the overall machine vision system is able to collect the data for the manufactured products, control their quality and aggregate all collected data into single record system with subsequent data analysis by higher-level system, then the potential return from system implementation should be considered in a completely different way. Overall, the enterprises should consider a movement away from addressing individual tasks to comprehensive automation of production based on the machine vision technologies.
RECOMMENDATIONS
Based on the experience of developing and implementing more than one hundred industrial control systems for different industries (image 3), we have compiled a number of practical recommendations for industrial enterprises, which we constantly sharing during discussions with our clients. These recommendations include general directions mentioned earlier (i.e. use of comprehensive approach, close collaboration with system engineers at production facility, automation of tasks formalization and data collection with the use of prototype stage equipment for machine vision, investment into the development of new solutions in a form of pilot projects and R&D), and specific steps to take for each of these directions.
Image 3: Hot-rolled coils position control at the production |
line implemented by Severstal PJSC |
How the enterprise should address the task of a large-scale machine vision system implementation? First of all, the tasks for visual industrial control of the manufactured products must be identified for a particular production facility. Most often, the algorithm is the following:
The second stage involves training of the personnel and development of IT infrastructure. The recommended minimum for engineering team is 3 people: production designer, engineer for automation system for control of the technological processes at the enterprise, and IT specialist. For the task formalization, data collection and conducting natural experiments the enterprise would require to purchase a minimum set of machine vision equipment (which includes 2-3 types of cameras, 2-3 types of lenses, 2-3 types of light sources, the software with algorithms for analysis of images, etc.). In addition, the facility may need to upgrade a local network and specialised network equipment.
Based on description and collected materials of formalized tasks, the preliminary expert evaluation is performed for complexity and feasibility of control task realisation. Then, all task at the production facility are divided into several groups in accordance with the evaluation results:
Once the results are obtained, the tasks from this last category would be allocated to the one of three above groups.
The natural experiments are conducted by the client engineering team with the support from specialists from the developer company. With the use of machine vision equipment (cameras, lenses, lighting), the images are taken for the control items (presented both with and without defects) in the real-world conditions or ones that are close to the existing manufacturing environment, with different optical schemes and lighting conditions. Obtained photo and video materials undergo the evaluation with the use of developed libraries for image analysis and recognition.
Based on the results from natural experiments, the fundamental possibility for addressing intended tasks with machine vision methods is determined. The requirements are specified and limitations are defined. Approximate costs for equipment and machine vision development services are evaluated.
In addition, the client assesses economic effect from addressing certain tasks. If there is no possibility to assess individual task, the assessment is performed for the aggregate effect from implementation of control and tracking system for the given production area, facility, etc. The economic effect is then compared with the cost of system, and the client makes a decision about overall implementation of a solution for particular task.
PERSPECTIVE OF THE DEVELOPER
Everything that has been mentioned in this article has specific practical basis. This article presents detailed format of interaction with industrial enterprises from such different industries as metallurgy, mechanical engineering, diamond mining and others. Today we observe that our approach for automation is shared by major Russian enterprises that turn to our company for addressing their manufacturing tasks.
Dr Vladimir Tsarev
General Director at Mallenom Systems LLC
Published in December 2020 issue of Control Engineering journal