WO2023236372A1

WO2023236372A1 - Surface defect detection method based on image recognition

Info

Publication number: WO2023236372A1
Application number: PCT/CN2022/116053
Authority: WO
Inventors: 刘晓升; 王宜怀; 罗喜召; 马小虎; 韦雪婷
Original assignee: 苏州大学
Priority date: 2022-06-09
Filing date: 2022-08-31
Publication date: 2023-12-14
Also published as: CN114882010A

Abstract

A surface defect detection method based on image recognition, characterized by: preprocessing collected image data to convert into a plurality of pieces of associated image data so as to form an image data set; training a first preset neural network model and a second preset neural network model by using the image data set, recognizing and analyzing a target image by means of the two neural network models to determine object state information comprised in a picture of the target image and surface structure defect state information of an object existing in the target image, and marking the target image according to the surface structure defect state information. According to the surface defect detection method, the collected image data is preprocessed to convert into the associated image data, such that the type of the image data in the image data set can be fully enriched, and comprehensive and effective training of the neural network models is achieved. The method can be suitable for images obtained under different photographing conditions, such that the accuracy and reliability of object surface defect detection are improved.

Description

Surface defect detection method based on image recognition

Technical field

The invention relates to the field of image recognition, and in particular to a surface defect detection method based on image recognition.

Background technique

At present, in industrial production, product detection algorithms have been commonly used to analyze and process pictures corresponding to industrial production products to obtain the surface structure status of the product, thereby judging the quality of the product.

technical problem

Existing product detection algorithms can only analyze pictures of specific states, that is, they put forward higher requirements for the photography of industrial production products, and cannot be promoted and applied to any industrial production occasions, thus failing to improve the detection of industrial production products. The accuracy of surface defect detection reduces the quality and stability of industrial production.

Technical solutions

The purpose of the present invention is to provide a surface defect detection method based on picture recognition, which preprocesses the collected picture data and converts it into a number of associated picture data to form a picture data set; and uses the picture data set to perform the first preprocessing Assume that the neural network model and the second preset neural network model are trained, and the target picture is recognized and analyzed through the two neural network models to determine the object status information contained in the target picture and the surface structure defect status information of the object present in the target picture. , and then mark the target picture according to the surface structure defect status information; the above surface defect detection method preprocesses the collected picture data and converts it into associated picture data, which can fully enrich the types of picture data in the picture data collection and realize the neural network Comprehensive and effective training of the model enables the above method to be applied to pictures obtained under different shooting conditions, effectively extended to different picture recognition situations, and improves the accuracy and reliability of object surface defect detection.

The purpose of the present invention is achieved through the following technical solutions:

A surface defect detection method based on image recognition, including the following steps:

Step S1, collect a predetermined amount of picture data, preprocess each picture data, thereby converting each picture data into a number of associated picture data; then form a picture data set by forming a number of associated picture data corresponding to all the picture data;

Step S2: Use the picture data set to train the first preset neural network model; input the target picture into the first preset neural network model for recognition and analysis to determine the object state information contained in the target picture;

Step S3: Determine whether the target picture is a valid picture according to the object status information contained in the frame of the target picture; and use the picture data set to train the second preset neural network model;

Step S4: Input the target picture judged to be a valid picture into the second preset neural network model for identification and analysis to determine the surface structure defect status information of the object present in the target picture; and then according to the surface structure defect status information to mark the target image.

In one embodiment, in step S1, collecting a predetermined amount of picture data, preprocessing each picture data, thereby converting each picture data into several associated picture data specifically includes:

Collect no less than 200 image data, each of which has different image brightness, contrast and chroma;

At least one of flipping preprocessing, scaling preprocessing and shearing preprocessing is performed on each image data, thereby converting each image data into several associated image data.

In one embodiment, in step S1, flipping preprocessing on each picture data specifically includes:

Using a certain pixel in the image corresponding to the image data as the rotation center point, the image is rotated at several random angles, thereby converting the image data into several associated image data;

or,

In step S1, scaling preprocessing of each image data specifically includes:

The image corresponding to the image data is scaled by several random scaling factors, thereby converting the image data into several associated image data;

or,

In the step S1, the shearing preprocessing of each picture data specifically includes:

A number of random amplitude shearing processes are performed along different boundaries of the image corresponding to the image data, thereby converting the image data into a number of associated image data.

In one embodiment, in step S1, forming a picture data set from a number of associated picture data corresponding to all picture data specifically includes:

All associated image data corresponding to each image data are randomly arranged and combined to form a corresponding image data set.

In one embodiment, in step S2, using the picture data set to train the first preset neural network model specifically includes:

Randomly select a predetermined number of associated image data from the image data set, and input the selected associated image data into the first preset neural network model for training, thereby determining model convergence of the trained first preset neural network model. degree;

If the convergence degree of the model meets the predetermined convergence condition, then the training of the first preset neural network model is completed; otherwise, a predetermined number of associated picture data is randomly selected from the picture data set to train the first preset neural network model. The neural network model is trained again until the model convergence degree meets predetermined convergence conditions.

In one embodiment, in step S2, the target picture is input into the first preset neural network model for recognition and analysis, and it is determined that the object state information contained in the frame of the target picture specifically includes:

The target picture is input into the first preset neural network model that has completed training for recognition and analysis, and the type of object contained in the picture of the target picture and the total pixel area of the corresponding object in the picture are determined.

In one embodiment, in step S3, judging whether the target picture is a valid picture according to the object status information contained in the frame of the target picture specifically includes:

According to the object type, determine whether the target picture contains an object of a predetermined type; if not, determine that the target picture does not belong to a valid picture;

If included, it is determined whether the total pixel area of the corresponding object in the picture is greater than the preset area threshold; if so, it is determined that the target picture is a valid picture; if not, it is determined that the target picture is not a valid picture.

In one embodiment, in step S3, using the picture data set to train the second preset neural network model specifically includes:

Randomly select a predetermined number of associated image data from the image data set, and input the selected associated image data into the second preset neural network model for training, thereby determining model convergence of the trained second preset neural network model. degree;

If the convergence degree of the model meets the predetermined convergence condition, then the training of the second preset neural network model is completed; otherwise, a predetermined number of associated picture data is randomly selected from the picture data set to train the second preset neural network model. The neural network model is trained again until the model convergence degree meets predetermined convergence conditions.

In one embodiment, in step S4, the target picture judged to be a valid picture is input to the second preset neural network model for identification and analysis to determine the surface structure defects of the object present in the target picture. Status information specifically includes:

The target picture judged to be a valid picture is input into the second preset neural network model for identification and analysis, and the location coordinates of the surface defect of the object present in the target picture and the shape and size of the surface defect are determined.

In one embodiment, in step S4, marking the target image according to the surface structure defect status information specifically includes:

In the target image, pixel sharpening is performed on the picture area corresponding to the surface defect, and the location coordinates of the surface defect and the shape and size related information of the surface defect are added to the picture area corresponding to the surface defect.

beneficial effects

Compared with the prior art, the present invention has the following beneficial effects:

The surface defect detection method based on image recognition provided by this application preprocesses the collected image data and converts it to obtain a number of associated image data to form a image data set; and uses the image data set to compare the first preset neural network model and the third Two preset neural network models are trained to identify and analyze the target picture through two neural network models, determine the object status information contained in the target picture and the surface structure defect status information of the object present in the target picture, and then based on the surface structure defects status information to mark the target image; the above surface defect detection method preprocesses the collected image data and converts it into associated image data, which can fully enrich the types of image data in the image data collection and achieve comprehensive and effective training of the neural network model. This enables the above method to be applied to pictures obtained under different shooting conditions, effectively extended to different picture recognition situations, and improves the accuracy and reliability of object surface defect detection.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts. in:

Figure 1 is a schematic flow chart of the surface defect detection method based on image recognition provided by this application.

Embodiments of the invention

In order to make the above objects, features and advantages of the present application more obvious and easy to understand, the specific implementation modes of the present application will be described in detail below with reference to the accompanying drawings. It can be understood that the specific embodiments described here are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for convenience of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

The terms "including" and "having" and any variations thereof in this application are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes Other steps or units inherent to such processes, methods, products or devices.

Reference herein to "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those skilled in the art understand, both explicitly and implicitly, that the embodiments described herein may be combined with other embodiments.

Please refer to FIG. 1 . An embodiment of the present application provides a surface defect detection method based on image recognition. The surface defect detection method based on image recognition includes the following steps:

Step S2, use the picture data set to train the first preset neural network model; input the target picture into the first preset neural network model for recognition and analysis to determine the object state information contained in the target picture;

Step S3: Determine whether the target picture is a valid picture based on the object status information contained in the frame of the target picture; and use the picture data set to train the second preset neural network model;

Step S4: Input the target picture that is judged to be a valid picture into the second preset neural network model for identification and analysis to determine the surface structure defect status information of the object present in the target picture; then, based on the surface structure defect status information, the target Pictures are tagged.

The above-mentioned surface defect detection method preprocesses the collected picture data and converts it into a number of related picture data to form a picture data set; and uses the picture data set to train the first preset neural network model and the second preset neural network model. , identify and analyze the target picture through two neural network models, determine the object status information contained in the target picture and the surface structure defect status information of the objects present in the target picture, and then conduct the target picture based on the surface structure defect status information. mark; the above surface defect detection method preprocesses the collected image data and converts it into associated image data, which can fully enrich the types of image data in the image data collection and achieve comprehensive and effective training of the neural network model, making the above method applicable to different The pictures obtained under the shooting conditions can be effectively extended to different picture recognition situations to improve the accuracy and reliability of object surface defect detection.

Optionally, in step S1, a predetermined amount of picture data is collected, and each picture data is preprocessed, thereby converting each picture data into several associated picture data, specifically including:

Through the above method, based on the collected image data, each image data is separately subjected to flipping preprocessing, scaling preprocessing and shearing preprocessing, so that each image data can derive multiple associated image data respectively, thereby maximizing Enrich the type and content of image data to the maximum extent to ensure comprehensive and reliable subsequent training of neural network models.

Optionally, in step S1, performing flip preprocessing on each image data specifically includes:

or,

In step S1, scaling preprocessing of each image data specifically includes:

or,

In step S1, the shearing preprocessing of each image data specifically includes:

Through the above method, the image corresponding to the image data is flipped, scaled, and cut, so that the same image data can be converted into multiple associated image data in different content forms through simple image processing operations, thereby improving the conversion of associated image data. Efficiency and ensuring diverse image types in image data collections.

Optionally, in step S1, forming a picture data set from several associated picture data corresponding to all picture data specifically includes:

Through the above method, after randomly arranging and combining all associated image data corresponding to each image data, it can effectively avoid excessive aggregation of associated image data originating from the same image data in the image data set, which is not conducive to subsequent training of the neural network model. reliability.

Optionally, in step S2, using the picture data set to train the first preset neural network model specifically includes:

Randomly select a predetermined number of associated image data from the image data set, and input the selected associated image data into the first preset neural network model for training, thereby determining the degree of model convergence of the trained first preset neural network model;

If the convergence degree of the model meets the predetermined convergence conditions, the training of the first preset neural network model is completed; otherwise, a predetermined number of associated picture data is randomly selected from the picture data set to train the first preset neural network model again. Training is performed until the model convergence degree meets predetermined convergence conditions.

In the above manner, the first preset neural network model is trained at least once using the picture data set as the training data source, so that the first preset neural network model can reliably identify and detect objects in the target picture. Among them, the first preset neural network model may be but is not limited to Inception v4 model.

Optionally, in step S2, the target picture is input into the first preset neural network model for recognition and analysis, and it is determined that the object state information contained in the target picture specifically includes:

Through the above method, the first preset neural network model can perform outline and texture recognition of objects that always exist in the target picture, thereby determining the types of objects contained in the target picture and the total pixel area of the corresponding objects in the picture, and realizing All objects present in the target picture are comprehensively and accurately recognized and detected.

Optionally, in step S3, judging whether the target picture is a valid picture according to the object status information contained in the frame of the target picture specifically includes:

According to the object type, determine whether the target picture contains a predetermined type of object; if not, determine that the target picture does not belong to a valid picture;

If included, determine whether the total pixel area of the corresponding object in the picture is greater than the preset area threshold; if so, determine that the target picture is a valid picture; if not, determine that the target picture is not a valid picture.

Through the above method, when the target picture contains a predetermined type of object and the total pixel area of the corresponding object in the picture is greater than the preset area threshold, it indicates that the object existing in the target picture has sufficient pixel area for surface defect detection, which can Ensure the reliability of subsequent object surface defect detection.

Optionally, in step S3, using the picture data set to train the second preset neural network model specifically includes:

Randomly select a predetermined number of associated image data from the image data set, and input the selected associated image data into the second preset neural network model for training, thereby determining the degree of model convergence of the trained second preset neural network model;

If the convergence degree of the model meets the predetermined convergence conditions, the training of the second preset neural network model is completed; otherwise, a predetermined number of associated image data is randomly selected from the image data set to train the second preset neural network model again. Training is performed until the model convergence degree meets predetermined convergence conditions.

In the above manner, the second preset neural network model is trained at least once using the picture data set as the training data source, so that the second preset neural network model can reliably identify and detect objects in the target picture. Among them, the second preset neural network model can be but is not limited to YOLO v5 model.

Optionally, in step S4, the target picture judged to be a valid picture is input into the second preset neural network model for identification and analysis. Determining the surface structure defect status information of the object present in the target picture specifically includes:

The target picture judged to be a valid picture is input into the second preset neural network model for identification and analysis, and the position coordinates of the surface defect of the object present in the target picture and the shape and size of the surface defect are determined.

Through the above method, the second preset neural network model can extract surface areas with differences in chromaticity, brightness, contrast, etc. on the surface of the object in the target picture, so as to determine the defect areas present on the surface of the object, and further extract The obtained surface area is identified and analyzed at the pixel level to determine the location coordinates of the surface defects of the object present in the target picture, as well as the shape and size of the surface defects.

Optionally, in step S4, marking the target image according to the surface structure defect status information specifically includes:

In the target image, the pixel sharpening process is performed on the picture area corresponding to the surface defect, and the position coordinates of the surface defect and the shape and size related information of the surface defect are added to the picture area corresponding to the surface defect.

Through the above method, pixel sharpening processing is performed on the image area corresponding to the surface defect in the target image, and the pixel enhanced characterization of the area where the surface defect is located can be performed; in addition. Add the location coordinates of the surface defect and the associated information on the shape and size of the surface defect to the picture area corresponding to the surface defect, so that the actual state of the surface defect can be determined intuitively and accurately.

The above is only a specific embodiment of the present invention, and any other improvements made based on the concept of the present invention are deemed to be within the protection scope of the present invention.

Claims

A surface defect detection method based on image recognition, which is characterized by including the following steps:

Step S1, collect a predetermined amount of picture data, preprocess each picture data, thereby converting each picture data into a number of associated picture data; then form a picture data set by forming a number of associated picture data corresponding to all the picture data;

Step S2: Use the picture data set to train the first preset neural network model; input the target picture into the first preset neural network model for recognition and analysis to determine the object state information contained in the target picture;

Step S3: Determine whether the target picture is a valid picture according to the object status information contained in the frame of the target picture; and use the picture data set to train the second preset neural network model;

Step S4: Input the target picture judged to be a valid picture into the second preset neural network model for identification and analysis to determine the surface structure defect status information of the object present in the target picture; and then according to the surface structure defect status information to mark the target image.
The surface defect detection method based on image recognition according to claim 1, characterized in that:

In the step S1, a predetermined amount of picture data is collected, each picture data is preprocessed, and each picture data is converted into several associated picture data, which specifically includes:

Collect no less than 200 image data, each of which has different image brightness, contrast and chroma;

At least one of flipping preprocessing, scaling preprocessing and shearing preprocessing is performed on each image data, thereby converting each image data into several associated image data.
The surface defect detection method based on image recognition according to claim 2, characterized in that:

In step S1, flipping preprocessing of each picture data specifically includes:

Using a certain pixel in the image corresponding to the image data as the rotation center point, the image is rotated at several random angles, thereby converting the image data into several associated image data;

or,

In step S1, scaling preprocessing of each image data specifically includes:

The image corresponding to the image data is scaled by several random scaling factors, thereby converting the image data into several associated image data;

or,

In the step S1, the shearing preprocessing of each picture data specifically includes:

A number of random amplitude shearing processes are performed along different boundaries of the image corresponding to the image data, thereby converting the image data into a number of associated image data.
The surface defect detection method based on image recognition according to claim 3, characterized in that:

In the step S1, forming a picture data set from several associated picture data corresponding to all picture data specifically includes:

All associated image data corresponding to each image data are randomly arranged and combined to form a corresponding image data set.
The surface defect detection method based on image recognition according to claim 4, characterized in that:

In step S2, using the picture data set to train the first preset neural network model specifically includes:

Randomly select a predetermined number of associated image data from the image data set, and input the selected associated image data into the first preset neural network model for training, thereby determining model convergence of the trained first preset neural network model. degree;

If the convergence degree of the model meets the predetermined convergence condition, then the training of the first preset neural network model is completed; otherwise, a predetermined number of associated picture data is randomly selected from the picture data set to train the first preset neural network model. The neural network model is trained again until the model convergence degree meets predetermined convergence conditions.
The surface defect detection method based on image recognition according to claim 5, characterized in that:

In the step S2, the target picture is input into the first preset neural network model for recognition and analysis, and it is determined that the object state information contained in the frame of the target picture specifically includes:

The target picture is input into the first preset neural network model that has completed training for recognition and analysis, and the type of object contained in the picture of the target picture and the total pixel area of the corresponding object in the picture are determined.
The surface defect detection method based on image recognition according to claim 6, characterized in that:

In step S3, judging whether the target picture is a valid picture according to the object status information contained in the frame of the target picture specifically includes:

According to the object type, determine whether the target picture contains an object of a predetermined type; if not, determine that the target picture does not belong to a valid picture;

If included, it is determined whether the total pixel area of the corresponding object in the picture is greater than the preset area threshold; if so, it is determined that the target picture is a valid picture; if not, it is determined that the target picture is not a valid picture.
The surface defect detection method based on image recognition according to claim 7, characterized in that:

In step S3, using the picture data set to train the second preset neural network model specifically includes:

Randomly select a predetermined number of associated image data from the image data set, and input the selected associated image data into the second preset neural network model for training, thereby determining model convergence of the trained second preset neural network model. degree;

If the convergence degree of the model meets the predetermined convergence condition, then the training of the second preset neural network model is completed; otherwise, a predetermined number of associated picture data is randomly selected from the picture data set to train the second preset neural network model. The neural network model is trained again until the model convergence degree meets predetermined convergence conditions.
The surface defect detection method based on image recognition according to claim 8, characterized in that:

In the step S4, the target picture judged to be a valid picture is input into the second preset neural network model for identification and analysis. Determining the surface structure defect status information of the object present in the target picture specifically includes:

The target picture judged to be a valid picture is input into the second preset neural network model for identification and analysis, and the location coordinates of the surface defect of the object present in the target picture and the shape and size of the surface defect are determined.
The surface defect detection method based on image recognition according to claim 9, characterized in that:

In step S4, marking the target image according to the surface structure defect status information specifically includes:

In the target image, pixel sharpening is performed on the picture area corresponding to the surface defect, and the location coordinates of the surface defect and the shape and size related information of the surface defect are added to the picture area corresponding to the surface defect.