CN114998357B - Industrial detection method, system, terminal and medium based on multi-information analysis - Google Patents
Industrial detection method, system, terminal and medium based on multi-information analysis Download PDFInfo
- Publication number
- CN114998357B CN114998357B CN202210942231.9A CN202210942231A CN114998357B CN 114998357 B CN114998357 B CN 114998357B CN 202210942231 A CN202210942231 A CN 202210942231A CN 114998357 B CN114998357 B CN 114998357B
- Authority
- CN
- China
- Prior art keywords
- industrial
- scene
- target
- detected
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
An industrial detection method, a system, a terminal and a medium based on multi-information analysis relate to the technical field of industrial detection, and solve the problems that the existing industrial detection method based on deep learning is not strong in universality and needs a large amount of training data to ensure the detection accuracy, and the process is as follows: training a convolutional neural network based on an industrial target class library and an integrated industrial scene data set with multi-label class labels to obtain an industrial target identification model; inputting data of an industrial scene to be detected into an industrial target recognition model and outputting a recognition result of a general industrial scene; performing fusion analysis on the identification result of the general industrial scene and the information of the industrial scene to be detected to obtain the identification result of the industrial scene to be detected; and screening non-interest targets and distinguishing similar targets to obtain a final recognition result. The invention does not need a large amount of data and has weak scene dependency.
Description
Technical Field
The invention relates to the technical field of industrial detection, in particular to an industrial detection method, an industrial detection system, an industrial detection terminal and an industrial detection medium based on multi-information analysis.
Background
In the current industrial production line, a large number of tasks including quality inspection, process control, error tracking and the like depend on human eyes to identify targets, so that the labor cost and the possibility of misoperation of the production line are greatly increased. To solve this problem, the visual inspection method is a low-cost and highly automated solution. The vision-based industrial target detection is realized in a complex industrial scene, the target is an object involved in industrial detection, and a required and specific target (including but not limited to a bolt, a hole, a frame, a workpiece and the like) is positioned and identified through a vision sensor. By accurately positioning and classifying the required targets, the operations of flaw sample detection, error state tracking, precise control, abnormal state monitoring and the like can be realized.
With the development of DL (deep learning) technology, a large number of deep CNNs (convolutional neural networks) have been developed for 2D object detection. The existing visual-based industrial target detection is to directly utilize the existing deep CNN network to realize industrial target detection, and the process comprises the following steps: 1) A 2D vision sensor is utilized to collect a large amount of picture data in an industrial field; 2) Marking the collected pictures, and determining the coordinate information and the category of a target required in each picture; 3) Constructing a data set by using the collected pictures, and training the existing deep CNN network; 4) And carrying out field deployment on the trained deep CNN network, and realizing field target detection according to requirements. However, this method has the following disadvantages:
1) A large amount of data is needed to train to obtain a CNN network with high precision, and when the training data is insufficient, the error rate is large;
2) The scene dependency is strong, when the scene is changed, the CNN network needs to be retrained, and the research and development cost is high.
Disclosure of Invention
In order to solve the problems that the existing industrial detection method based on DL is not strong in universality and needs a large amount of training data to ensure the detection accuracy, the invention provides an industrial detection method, a system, a terminal and a medium based on multi-information analysis.
The technical scheme adopted by the invention for solving the technical problem is as follows:
an industrial detection method based on multi-information analysis comprises the following steps:
step 1, training a convolutional neural network based on an industrial target class library and an integrated industrial scene data set with multi-label class labels to obtain an industrial target identification model; the multi-label type labeling comprises target positioning information labeling, target surface appearance size information labeling and target substance attribute information labeling; the industrial target category library comprises target positioning information, target surface appearance size information and target attribute information; the industrial target category library comprises target defect categories, and the multi-label category labels comprise target defect category labels;
step 2, inputting data of the industrial scene to be detected into the industrial target recognition model obtained in the step 1, and outputting a recognition result of the general industrial scene corresponding to the industrial scene to be detected;
step 3, carrying out fusion analysis on the identification result obtained in the step 2 and the information of the industrial scene to be detected to obtain the identification result of the industrial scene to be detected; the information of the industrial scene to be detected comprises: scene information of the industrial scene to be detected, task information of the industrial scene to be detected, target information of the industrial scene to be detected and relation information between a target and a target of the industrial scene to be detected;
and 4, screening out non-interest targets of the industrial scene to be detected, and distinguishing similar targets according to the relation information between the targets to obtain a final identification result of the industrial scene to be detected.
An industrial detection system based on multi-information analysis, comprising:
the training module is used for training the convolutional neural network based on an industrial target class library and an integrated industrial scene data set with multi-label class labels to obtain an industrial target recognition model; the industrial target recognition model can perform target recognition on the data of the industrial scene to be detected input into the industrial target recognition model and output a recognition result of the general industrial scene corresponding to the industrial scene to be detected; the multi-label type labeling comprises target positioning information labeling, target surface appearance size information labeling and target substance attribute information labeling; the industrial target category library comprises target positioning information, target surface appearance size information and target attribute information; the industrial target class library comprises target defect classes, and the multi-label class labels comprise target defect class labels;
the identification module is used for inputting the data of the industrial scene to be detected into the industrial target identification model for target identification, and the industrial target identification model outputs the identification result of the general industrial scene corresponding to the industrial scene to be detected;
the fusion analysis module is used for carrying out fusion analysis according to the identification result of the general industrial scene corresponding to the industrial scene to be detected and the information of the industrial scene to be detected to obtain the identification result of the industrial scene to be detected; the information of the industrial scene to be detected comprises: scene information of the industrial scene to be detected, task information of the industrial scene to be detected, target information of the industrial scene to be detected and relation information between targets of the industrial scene to be detected;
and the screening module is used for screening out the non-interesting targets of the industrial scene to be detected in the identification result of the industrial scene to be detected and distinguishing the similar targets in the identification result of the industrial scene to be detected according to the relationship information between the targets.
A terminal comprising a memory, a processor, and an industrial inspection program stored in the memory and executable on the processor, the industrial inspection program, when executed by the processor, implementing the steps of the method for industrial inspection based on multi-information analysis.
A computer readable storage medium storing an industrial inspection program, which when executed by a processor implements the steps of a multi-information analysis-based industrial inspection method.
The invention has the beneficial effects that:
1. the industrial detection method, the system, the terminal and the medium based on the multi-information analysis obtain a universal industrial target identification model through an industrial target class library and training of an industrial target identification model based on an integrated industrial scene data set with multi-label class labels, do not need to carry out independent training aiming at each scene, do not need a large amount of training data on the whole of various application scenes, realize reduction of data requirements by matching with the industrial target class library on the basis of the integrated industrial scene data set, can realize industrial target detection under a new scene, do not need or only need to carry out transfer training on the network by using limited new scene data, and can realize target detection and identification under the new scene through multi-information fusion analysis to obtain accurate positioning and classes. The scene dependency is not strong, and the CNN network does not need to be retrained when the scene is changed. The invention has the advantages of universality and small requirement on the scale of the training data set.
2. Compared with the traditional target detection method which only carries out single-label prediction on a single type of target, the industrial detection method based on multi-information analysis simultaneously predicts a plurality of types of targets, provides abundant additional information, can accurately classify similar targets, does not need additional manual analysis, and enables detection results to be more visual and accurate.
3. Through analyzing the multi-information, even if the multi-information cannot be directly identified through vision, the multi-information can be positioned and identified through indirect information, so that the research and development cost and the requirement on data are greatly reduced.
Drawings
Fig. 1 is a schematic diagram of an implementation process of the industrial detection method based on multi-information analysis according to the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention, taken in conjunction with the accompanying drawings and detailed description, is set forth below.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
The industrial detection method based on multi-information analysis, as shown in fig. 1, includes the following steps:
step 1, training a convolutional neural network to obtain an industrial target recognition model based on an industrial target class library and an integrated industrial scene data set with multi-label class labels.
In various different industrial scenes, image data of various different industrial scenes are acquired by using an image sensor, the specificity of the industrial scenes is limited, the data scale is limited or is limited, and an integrated industrial scene data set is established by using the limited data. The integrated industrial scene data set includes, but is not limited to, image-like data, and may also include measurement data, spectra, etc., such as measurement data may be used to describe a specific distance between two targets, spectra at different optical bands may be used to describe material characteristics, etc. The integrated industrial scene data set comprises scene information, task information and target information. Scene information includes, but is not limited to: scene layout information, operating condition information (status information of production devices and facility production operations), and the like, and the task information includes but is not limited to: process requirement information, and the like. The target information comprises target surface appearance size information, target positioning information, target attribute information, relationship information between targets, and relationship information between targets comprises dependency relationship information between targets and position information between targets. In the present embodiment, the target attribute information is target material information. The target information may further include target defect information, that is, the target surface topography size information includes target defect information, and each target may have its corresponding defect classification.
According to the analysis of various industrial scenes (which may include but are not limited to the working scenes corresponding to the integrated industrial scene data set), in the embodiment, the working scenes corresponding to the integrated industrial scene data set are analyzed, the general rules in each industrial scene are extracted, and the industrial object class library is established, wherein the general rules are likely to appear in different scenes, such as bolts, frames, holes and the like, so that the industrial object class library with certain universality can be established. The industrial object class library is a collection of objects and object classes, and includes objects and classes of the objects that are present in a variety of industrial scenarios. The industry standard category library in this embodiment is accumulated from a large number of different industry data, the sources of which include not only public/public data sets, but also those collected by the vision system in different industry type scenarios. The industrial target category library comprises target positioning information, target surface appearance size information and target attribute information, wherein the target positioning information, the target surface appearance size information and the target attribute information all belong to target categories. In this embodiment, the industrial object class library includes object defect information.
In this embodiment, taking the machine industry scenario as an example, the industry object class library includes, but is not limited to, a name of a processing device (a name of a device used in a production process), a tool for production (including a name of a hand tool used in a production process, such as a wrench and a pliers), a worker, a name of a part (such as a gear, a shaft, a bolt, a screw, and a bolt), a material of the part (such as aluminum, steel, various alloys, plastics, and ceramics), a specification model of the part (such as a bolt specification having a specification of M8, M10, and M12), a degree of completeness of the part (such as a perfect part, a crack problem of the part, and a defect problem of the part), a defect type of the part (different parts have different defect classifications, such as a fatal defect, an important defect, a minor defect, such as an appearance defect, a dimension defect, and a position of a performance defect, such as a point defect, a line defect, a surface defect, and a position of the part where the defect depends on the part), a type of the worker, and a dependency relationship between the parts (such as a screwing sequence or a link relationship). The name of the processing equipment, the tool for production, the name of the part and the name of the worker belong to a target, and the specification and the model of the part, the completeness of the part, the defect type of the part and the work type of the worker belong to a target category.
And carrying out multi-label category labeling on the integrated industrial scene data set according to but not limited to an industrial target category library to obtain the integrated industrial scene data set for training the deep CNN. The method specifically comprises the following steps: and performing target labeling and target multi-label type labeling on the data in the integrated industrial scene data set according to the industrial target type library and the label type of the integrated industrial scene data set in the corresponding industrial scene to obtain a labeled integrated industrial scene data set. The multi-label category label is that one target can correspond to multiple label categories, the label categories in the multiple label categories can be in a subordination relationship or a parallel relationship, and usually, the multiple label categories are in a network relationship. From these tags, the object can get a more detailed description with respect to a single tag. The multi-label type labeling comprises target positioning information labeling, target surface appearance size information labeling and target substance attribute information labeling, and preferably, the multi-label type labeling comprises labeling each information in an industrial target type library. Specifically, the multi-label category marking comprises marking according to names, part specifications and models and part completeness. The industrial scenes comprise a scene of producing and processing automobile hubs, a scene of producing and processing automobile shells, a scene of producing and processing kitchen steam cookers and the like. The label category in the industrial scene may include an industrial scene, a name and specification of a finished product/semi-finished product of the industrial scene, a name and specification of a processing device of the industrial scene, a name and specification of a production tool of the industrial scene, a name of a part of the industrial scene, a specification model of the part of the industrial scene, a part health degree of the industrial scene, and the like.
And training the deep convolutional neural network by utilizing the industrial target class library and the labeled integrated industrial scene data set, and obtaining an industrial target recognition model through training. And obtaining the recognition result of the general industrial scene by using the industrial target recognition model, wherein the recognition result comprises the recognized target and the multi-label category of the recognized target.
In this embodiment, the object class in the industrial object class library includes an object defect class, and the labeling of the integrated industrial scene data set includes labeling of the object defect class.
And 2, inputting data of the industrial scene to be detected into the industrial target recognition model obtained in the step 1, and outputting a recognition result of the general industrial scene corresponding to the industrial scene to be detected by the industrial target recognition model.
The method comprises the steps of collecting real-time data of a certain industrial scene to be detected, wherein the real-time data is usually pictures but not limited to pictures or only pictures, inputting the real-time data of the certain industrial scene to be detected into an industrial target recognition model, outputting a recognition result of a general industrial scene corresponding to the industrial scene to be detected by the industrial target recognition model, wherein the recognition result of the general industrial scene corresponding to the industrial scene to be detected is a multi-label category comprising all targets possibly existing in the industrial scene and targets, and outputting the multi-label category, namely complete description information of the targets. In fig. 1, objects 1, 2, and N only represent different objects, objects 1, 2, and N only represent object numbers, N is a natural number, and object N schematically represents the nth object.
And 3, performing fusion analysis according to the identification result of the general industrial scene of the industrial scene to be detected and the information of the industrial scene to be detected to obtain the identification result of the industrial scene to be detected.
The industrial scene identification result to be detected comprises an object identified by the industrial scene to be detected and a label category of the object identified by the industrial scene to be detected (the label category at this time is not limited to be single, two or more, and is determined according to specific situations). The information of the industrial scene to be detected includes but is not limited to: the method comprises the steps of obtaining scene information of an industrial scene to be detected, task information of the industrial scene to be detected, target information of the industrial scene to be detected and relation information between targets of the industrial scene to be detected. The fusion analysis can also utilize prior knowledge.
And 4, screening out non-interesting targets of the industrial scene to be detected in the identification result obtained in the step 3, and distinguishing similar targets in the identification result obtained in the step 3 according to the relation information between the targets to obtain the final identification result of the industrial scene to be detected.
Screening out interest targets which are not under the industrial scene to be detected, which specifically comprises the following steps: and screening out the interest targets which are not under the industrial scene to be detected according to the information and the prior knowledge of the industrial scene to be detected. And (3) further analyzing similar targets (for example, two bolts with close sizes are similar targets) according to the information of the industrial scene to be detected to distinguish the similar targets, specifically, calculating information such as relative positions between different targets according to the relationship information between the targets, and further classifying the similar targets finely to distinguish the similar targets, so that more accurate recognition result output is realized, and the recognition result in the step (3) is optimized and adjusted. And the final identification result of the industrial scene to be detected comprises the final identified target of the industrial scene to be detected and the label category of the final identified target of the industrial scene to be detected.
The target in the final identification result of the industrial scene to be detected is more accurate relative to the target in the step 2, and the label category of the target in the final identification result of the industrial scene to be detected is more accurate relative to the multi-label category of the target in the step 2.
The industrial detection method based on multi-information analysis obtains a universal industrial target recognition model through training the industrial target recognition model, does not need to carry out independent training aiming at each scene, does not need a large amount of training data on the whole of various application scenes, realizes reduction of data requirements by matching with an industrial target class library on the basis of integrating an industrial scene data set, can realize industrial target detection under a new scene, does not need or only needs limited new scene data to carry out migration training on a network, can realize target detection and recognition under the new scene through multi-information fusion analysis, obtains accurate positioning and class, can carry out more accurate classification aiming at similar targets, does not need additional human analysis, and enables detection results to be more visual and accurate; compared with the traditional target detection method which only carries out single-label prediction on a single type of target, the industrial detection method based on multi-information analysis simultaneously predicts a plurality of types of targets, provides abundant additional information and can accurately classify similar targets. Through analyzing the multi-information, even if the multi-information cannot be directly identified through vision, the multi-information can be positioned and identified through indirect information, so that the research and development cost and the requirement on data are greatly reduced. The industrial detection method can be used for a general industrial scene, even if the scene changes or objects (such as new types of defective parts) which are not seen appear, the objects can be identified and positioned based on an industrial object identification model trained by an industrial object class library, and the product universality is greatly improved. Therefore, the method has both universality and small requirements on the scale of the training data set. The invention has wide application range, can be used in a plurality of industrial fields including automobile industry, mechanical industry, precision electronic industry and the like, and can also be used for target detection tasks of various complex scenes such as airport, subway safety inspection, medical treatment and the like. Meanwhile, based on the invention, the objects which are not discovered manually and not determined can be described comprehensively and qualitatively, so that the intelligent and automatic monitoring of the industrial field can be realized.
In addition, the invention can cover most industrial scenes by analyzing a large amount of industrial scenes and developing and establishing an industrial target class library by self, thereby improving the universality of the detection method. The industrial target category library and the target categories in the integrated industrial scene data set comprise target defect categories, so that similar targets can be accurately distinguished, in the prior art, whether the targets have defects or not can only be marked, and further identification cannot be performed, for example, the targets have gap defects, but specific defects cannot be identified due to the fact that the appearance of the gaps is similar. By the method, the appearance of the gap can be defined descriptively, accurately and qualitatively by virtue of a large amount of sample information with multi-label information, and one or more types of new related labels and samples are formed iteratively along with the acquisition and supplement of similar samples.
Furthermore, the invention carries out comprehensive analysis by utilizing the recognition results of multiple targets, further adjusts and optimizes the target recognition results by utilizing the relative position information, the geometric information, the scene information and the task information, and outputs accurate positioning and category information under the task scene.
The following description will be made in an application based on the method taking the rack bolt detection as an example.
The industrial scene to be detected is rack bolt installation, and rack bolt detection is carried out by utilizing an industrial detection method based on multi-information analysis.
The method comprises the following steps: the bolt torque of different positions of frame is different, and the workman utilizes the spanner to screw up the operation to the bolt, need judge the classification and the sequence number of screwing up the bolt, and then confirms required moment of torsion and monitors whether the moment of torsion sets up the mistake.
The difficulty of the prior art is as follows: the appearance of the bolts is consistent, the traditional target detection method cannot judge the specific types and the serial numbers of the bolts, and the specific torque is difficult to set for each bolt.
The industrial detection method based on multi-information analysis can identify the rack and the bolts, including the specific category and the specific position (serial number during installation) of each bolt, and comprises the following specific processes:
a. firstly, real-time operation data are collected on a workshop site by using an image sensor, and an integrated industrial scene data set is established.
b. Analyzing and marking the integrated industrial scene data set according to the self-built industrial target class library, wherein the bolts need to be marked, and off-site information such as wrenches, racks, workers and the like can be marked.
c. And training the target detection depth CNN by using the labeled integrated industrial scene data set to obtain the trained depth CNN, namely obtaining an industrial target recognition model.
d. The trained industrial target recognition model is placed on a bolt detection site, a site detection picture is input in real time, the positions of a plurality of information such as bolts, wrenches, racks, workers and the like can be marked by the trained industrial target recognition model, and accurate classification is carried out.
e. According to the task information, firstly, the type of the bolt needs to be judged, and the specific type of the bolt can be judged according to the relative position of the bolt and the rack and the type of the rack; judging the currently operated bolt: and calculating the relative distance between the wrench and each bolt, wherein the closest bolt is the current operating bolt.
f. Aiming at new objects (such as new racks) which do not appear in the integrated industrial scene data set, the industrial detection method based on the multi-information fusion analysis can still detect the bolts and can also obtain the class description of the new racks.
The invention provides an industrial detection system based on multi-information analysis, which comprises a training module, an identification module, a fusion analysis module and a screening module. The training module is used for training the convolutional neural network based on an industrial target class library and an integrated industrial scene data set with multi-label class labels to obtain an industrial target identification model; the industrial target recognition model can perform target recognition on the image of the industrial scene to be detected input into the industrial target recognition model, and outputs a recognition result of the general industrial scene corresponding to the industrial scene to be detected. And the identification module is used for substituting the picture of the industrial scene to be detected into the industrial target identification model to carry out target identification and outputting an identification result of the general industrial scene corresponding to the industrial scene to be detected. And the fusion analysis module is used for performing fusion analysis according to the identification result of the general industrial scene corresponding to the industrial scene to be detected and the information of the industrial scene to be detected, and specifically obtaining the target identified by the industrial scene to be detected and the label category of the target identified by the industrial scene to be detected. And the screening module is used for screening out the non-interesting targets of the industrial scene to be detected in the identification result of the industrial scene to be detected, and distinguishing similar targets in the identification result of the industrial scene to be detected according to the relationship information between the targets to obtain the final identification result of the industrial scene to be detected. The multi-label type labeling comprises target positioning information labeling, target surface appearance size information labeling and target substance attribute information labeling; the industrial target class library comprises target positioning information, target surface appearance size information and target attribute information; the industrial target category library comprises target defect categories, and the multi-label category labels comprise target defect category labels; the information of the industrial scene to be detected comprises: the method comprises the steps of obtaining scene information of an industrial scene to be detected, task information of the industrial scene to be detected, target information of the industrial scene to be detected and relation information between targets of the industrial scene to be detected.
It should be noted that the implementation principle and the implementation mode of an industrial detection system based on multi-information analysis are consistent with the industrial detection method based on multi-information analysis, and therefore, the details are not described below.
In another aspect, the present invention provides a terminal (not shown in the drawings), where the terminal includes a memory, a processor, and an industrial inspection program stored in the memory and capable of running on the processor, and when the industrial inspection program is executed by the processor, the terminal implements the steps of the industrial inspection method based on multi-information analysis according to any one of the above embodiments. The processor may be a Central Processing Unit (CPU), may be a graphics unit processor or Graphics Processor (GPU), and may be other general purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The memory may include various types of storage units such as system memory, read Only Memory (ROM), permanent storage.
The present invention further provides a computer-readable storage medium (not shown in the drawings), wherein the computer-readable storage medium stores an industrial detection program, and the industrial detection program, when executed by a processor, implements the steps of the industrial detection method based on multi-information analysis according to any of the above embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative modules and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The invention is not limited solely to that described in the specification and embodiments, and additional advantages and modifications will readily occur to those skilled in the art, so that the invention is not limited to the specific details, representative apparatus, and illustrative examples shown and described herein, without departing from the spirit and scope of the general concept as defined by the appended claims and their equivalents.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (6)
1. An industrial detection method based on multi-information analysis is characterized by comprising the following steps:
step 1, analyzing a working scene corresponding to an integrated industrial scene data set, extracting general rules in each industrial scene, and establishing an industrial target class library; the industrial object category library comprises objects and categories of the objects in various industrial scenes; training a convolutional neural network based on an industrial target class library and an integrated industrial scene data set with multi-label class labels to obtain a universal industrial target identification model; the multi-label type marking comprises target positioning information marking, target surface appearance size information marking, target substance attribute information marking and target defect type marking; the industrial target category library comprises target positioning information, target surface appearance size information, target attribute information and target defect categories;
step 2, inputting data of the industrial scene to be detected into the industrial target recognition model obtained in the step 1, and outputting a recognition result of the general industrial scene corresponding to the industrial scene to be detected; the identification result of the general industrial scene corresponding to the industrial scene to be detected comprises all possible targets in the industrial scene and multi-label categories of the targets;
step 3, carrying out fusion analysis on the identification result obtained in the step 2 and the information of the industrial scene to be detected to obtain the identification result of the industrial scene to be detected; the information of the industrial scene to be detected comprises: scene information of the industrial scene to be detected, task information of the industrial scene to be detected, target information of the industrial scene to be detected and relation information between a target and a target of the industrial scene to be detected;
and 4, screening out non-interesting targets of the industrial scene to be detected, and distinguishing similar targets according to the relation information between the targets to obtain the final recognition result of the industrial scene to be detected.
2. The industrial detection method based on multi-information analysis, as claimed in claim 1, wherein step 1 is preceded by step 0.1, and step 0.1 is: the method comprises the steps of acquiring data under various industrial scenes to obtain an integrated industrial scene data set, establishing an industrial target class library and carrying out multi-label class labeling on the integrated industrial scene data set.
3. The industrial detection method based on multi-information analysis according to claim 2, wherein the step 0.1 and the step 1 are specifically as follows: the method comprises the steps of acquiring data under various industrial scenes to obtain an integrated industrial scene data set, establishing an industrial target class library suitable for various industrial scenes according to the integrated industrial scene data set, carrying out multi-label class labeling on the integrated industrial scene data set according to the industrial target class library and label classes under the industrial scenes corresponding to the integrated industrial scene data set to obtain a labeled integrated industrial scene data set, and training a deep convolutional neural network according to the industrial target class library and the labeled integrated industrial scene data set to obtain an industrial target identification model.
4. An industrial detection system based on multi-information analysis, comprising:
the training module is used for training the convolutional neural network based on an industrial target class library and an integrated industrial scene data set with multi-label class labels to obtain a universal industrial target identification model, wherein a working scene corresponding to the integrated industrial scene data set is analyzed, universal rules in each industrial scene are extracted, and the industrial target class library is established; the industrial object class library comprises objects and classes of the objects in various industrial scenes; the industrial target recognition model can perform target recognition on the data of the industrial scene to be detected input into the industrial target recognition model and output a recognition result of the general industrial scene corresponding to the industrial scene to be detected; the multi-label type labeling comprises target positioning information labeling, target surface appearance size information labeling, target substance attribute information labeling and target defect type labeling; the industrial target category library comprises target positioning information, target surface appearance size information, target attribute information and target defect categories;
the identification module is used for inputting the data of the industrial scene to be detected into the industrial target identification model for target identification, and the industrial target identification model outputs the identification result of the general industrial scene corresponding to the industrial scene to be detected; the identification result of the general industrial scene corresponding to the industrial scene to be detected comprises all possible targets in the industrial scene and multi-label categories of the targets; the information of the industrial scene to be detected comprises:
the fusion analysis module is used for carrying out fusion analysis according to the identification result of the general industrial scene corresponding to the industrial scene to be detected and the information of the industrial scene to be detected to obtain the identification result of the industrial scene to be detected; scene information of the industrial scene to be detected, task information of the industrial scene to be detected, target information of the industrial scene to be detected and relation information between targets of the industrial scene to be detected;
and the screening module is used for screening out non-interest targets of the industrial scene to be detected in the identification result of the industrial scene to be detected and distinguishing similar targets in the identification result of the industrial scene to be detected according to the relationship information between the targets.
5. A terminal, characterized in that the terminal comprises a memory, a processor and an industrial detection program stored in the memory and executable on the processor, the industrial detection program, when executed by the processor, implementing the steps of a multi-information analysis based industrial detection method according to any one of claims 1-3.
6. A computer-readable storage medium, characterized in that the computer-readable storage medium stores an industrial inspection program, which when executed by a processor implements the steps of a multi-information analysis-based industrial inspection method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210942231.9A CN114998357B (en) | 2022-08-08 | 2022-08-08 | Industrial detection method, system, terminal and medium based on multi-information analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210942231.9A CN114998357B (en) | 2022-08-08 | 2022-08-08 | Industrial detection method, system, terminal and medium based on multi-information analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114998357A CN114998357A (en) | 2022-09-02 |
| CN114998357B true CN114998357B (en) | 2022-11-15 |
Family
ID=83023145
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210942231.9A Active CN114998357B (en) | 2022-08-08 | 2022-08-08 | Industrial detection method, system, terminal and medium based on multi-information analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114998357B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115438592B (en) * | 2022-11-08 | 2023-01-24 | 成都中科合迅科技有限公司 | Industrial research and development design data modeling method based on system engineering |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110400315A (en) * | 2019-08-01 | 2019-11-01 | 北京迈格威科技有限公司 | A kind of defect inspection method, apparatus and system |
| CN110781805A (en) * | 2019-10-23 | 2020-02-11 | 上海极链网络科技有限公司 | Target object detection method, device, computing equipment and medium |
| CN112730425A (en) * | 2020-12-19 | 2021-04-30 | 滁州蜂云视觉科技有限公司 | Industrial scene oriented product surface defect automatic identification method |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7324682B2 (en) * | 2004-03-25 | 2008-01-29 | Mitutoyo Corporation | System and method for excluding extraneous features from inspection operations performed by a machine vision inspection system |
| CN108764208B (en) * | 2018-06-08 | 2021-06-08 | Oppo广东移动通信有限公司 | Image processing method and device, storage medium and electronic equipment |
| CN111444891A (en) * | 2020-04-30 | 2020-07-24 | 天津大学 | Unmanned rolling machine operation scene perception system and method based on airborne vision |
| CN112966697B (en) * | 2021-03-17 | 2022-03-11 | 西安电子科技大学广州研究院 | Target detection method, device and equipment based on scene semantics and storage medium |
| CN113326796B (en) * | 2021-06-17 | 2022-11-29 | 北京百度网讯科技有限公司 | Object detection method, model training method and device and electronic equipment |
| CN113903081B (en) * | 2021-09-29 | 2024-10-18 | 北京许继电气有限公司 | Hydropower plant image visual identification artificial intelligence alarm method and device |
-
2022
- 2022-08-08 CN CN202210942231.9A patent/CN114998357B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110400315A (en) * | 2019-08-01 | 2019-11-01 | 北京迈格威科技有限公司 | A kind of defect inspection method, apparatus and system |
| CN110781805A (en) * | 2019-10-23 | 2020-02-11 | 上海极链网络科技有限公司 | Target object detection method, device, computing equipment and medium |
| CN112730425A (en) * | 2020-12-19 | 2021-04-30 | 滁州蜂云视觉科技有限公司 | Industrial scene oriented product surface defect automatic identification method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114998357A (en) | 2022-09-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210390678A1 (en) | Method for monitoring manufacture of assembly units | |
| CN111179251B (en) | Defect detection system and method based on twin neural network and by utilizing template comparison | |
| US10835930B2 (en) | Sorting system | |
| US11982628B2 (en) | System and method for detecting defects on imaged items | |
| CN107657603A (en) | A kind of industrial appearance detecting method based on intelligent vision | |
| CN104063873A (en) | Shaft sleeve part surface defect on-line detection method based on compressed sensing | |
| US20220284699A1 (en) | System and method of object detection using ai deep learning models | |
| WO2017107533A1 (en) | Electronic component sample labeling method and device | |
| CN114998357B (en) | Industrial detection method, system, terminal and medium based on multi-information analysis | |
| JP6795562B2 (en) | Inspection equipment and machine learning method | |
| Rosandich et al. | Intelligent visual inspection | |
| CN117114420B (en) | Image recognition-based industrial and trade safety accident risk management and control system and method | |
| CN118376864A (en) | System and method for diagnosing faults of tubing mechanism based on semiconductor punching equipment | |
| Herlambang et al. | Single-digit time: Toward a quick change-over process with the SMED method using the vision system | |
| CN117314829A (en) | Industrial part quality inspection method and system based on computer vision | |
| Reyna et al. | Product Digital Quality Inspection using Machine Vision Systems–A Categorical Review | |
| Nahar et al. | Computer aided system for inspection of assembled PCB | |
| CN115200513A (en) | Coaxiality jumping detection, analysis and control system for rotating body clamp | |
| Sahoo et al. | A robotic assistance machine vision technique for an effective inspection and analysis | |
| Sahoo et al. | An effective quality inspection system using image processing techniques | |
| Tan et al. | Research on YOLOv8 Application in Bolt and Nut Detection for Robotic Arm Vision | |
| Hunger et al. | Data acqusition challenges in AI-driven surface inspection: a proven solution proposal on coated sheet metal parts | |
| Shaloo et al. | Flexible automation of quality inspection in parts assembly using CNN-based machine learning | |
| CN111724352B (en) | Patch LED flaw labeling method based on kernel density estimation | |
| Kiran et al. | Inception V3 Model-Based Approach for Detecting Defects on Steel Surfaces |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |