CN111738290B - Image detection method, model construction and training method, device, equipment and medium - Google Patents
Image detection method, model construction and training method, device, equipment and medium Download PDFInfo
- Publication number
- CN111738290B CN111738290B CN202010407166.0A CN202010407166A CN111738290B CN 111738290 B CN111738290 B CN 111738290B CN 202010407166 A CN202010407166 A CN 202010407166A CN 111738290 B CN111738290 B CN 111738290B
- Authority
- CN
- China
- Prior art keywords
- image
- compliance
- detection model
- model
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 499
- 238000012549 training Methods 0.000 title claims abstract description 125
- 238000000034 method Methods 0.000 title claims abstract description 116
- 238000010276 construction Methods 0.000 title abstract description 20
- 238000000605 extraction Methods 0.000 claims description 58
- 238000010801 machine learning Methods 0.000 claims description 35
- 230000004913 activation Effects 0.000 claims description 10
- 238000011176 pooling Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 description 24
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 238000011161 development Methods 0.000 description 5
- 238000012552 review Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000010224 classification analysis Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000001373 regressive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Probability & Statistics with Applications (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses an image detection method, a model construction and training method, a device, equipment and a medium. The method comprises the following steps: acquiring an image to be detected; inputting the image to be detected into a compliance image detection model to obtain a first detection probability that the image to be detected is a compliance image, wherein the compliance image detection model is obtained by training a target image detection model based on a training sample formed by the compliance image and an non-compliance image; and determining whether the image to be detected is a compliant image according to the first detection probability. Through the technical scheme, more accurate and more comprehensive image detection is realized.
Description
Technical Field
The embodiment of the invention relates to an image processing technology, in particular to an image detection method, a model construction and training method, a device, equipment and a medium.
Background
All internet companies that involve original content (UGC), such as live video, social applications, and electronic commerce applications, require the detection and filtering of non-compliant images according to national regulations.
The existing image detection is generally exhaustive and non-conforming to the specified image types, such as watermark images containing watermarks, yellow images, riot images, public character images, advertisement images and the like, and utilizes an image recognition algorithm corresponding to each image type to recognize whether a certain image is a non-conforming image (i.e. a non-conforming image). If yes, filtering; if not, the detection is passed.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art: the set image types which are not in accordance with the regulations are limited, and the expansibility of the newly added illegal contents is insufficient, so that some newly added images which are not in accordance with the regulations cannot be identified, thereby reducing the accuracy of image detection.
Disclosure of Invention
The embodiment of the invention provides an image detection method, a model construction and training method, a device, equipment and a medium, so as to realize more accurate and more comprehensive image detection.
In a first aspect, an embodiment of the present invention provides an image detection method, including:
acquiring an image to be detected;
inputting the image to be detected into a compliance image detection model to obtain a first detection probability that the image to be detected is a compliance image, wherein the compliance image detection model is obtained by training a target image detection model based on a training sample formed by the compliance image and an non-compliance image;
And determining whether the image to be detected is a compliant image according to the first detection probability.
In a second aspect, an embodiment of the present invention further provides a training method for an image detection model, including:
classifying the collected images to generate an initial compliance image set and an initial non-compliance image set;
training a target image detection model based on the initial compliance image set and the initial non-compliance image set to generate a compliance image detection model, wherein the compliance image detection model is used for detecting whether an input image is a compliance image.
In a third aspect, an embodiment of the present invention further provides a method for constructing an image detection model, including:
constructing a feature extraction sub-network according to a target detection model based on a two-step method;
constructing a regression classification sub-network according to a target detection model based on a one-step method;
constructing a target image detection model by the feature extraction sub-network and the regression classification sub-network;
the characteristic extraction sub-network is used for extracting characteristics of an input image to obtain a characteristic layer; the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image; the input image is a compliant image and/or a non-compliant image.
In a fourth aspect, an embodiment of the present invention further provides an image detection apparatus, including:
the image acquisition module is used for acquiring an image to be detected;
the first detection probability obtaining module is used for inputting the image to be detected into a compliance image detection model to obtain the first detection probability that the image to be detected is a compliance image, wherein the compliance image detection model is obtained by training a target image detection model based on a training sample formed by the compliance image and a non-compliance image;
and the compliance image detection module is used for determining whether the image to be detected is a compliance image according to the first detection probability.
In a fifth aspect, an embodiment of the present invention further provides a training device for an image detection model, where the device includes:
the sample classification module is used for classifying the collected images to generate an initial compliance image set and an initial non-compliance image set;
and the model training module is used for training the target image detection model based on the initial compliance image set and the initial non-compliance image set to generate a compliance image detection model, wherein the compliance image detection model is used for detecting whether an input image is a compliance image.
In a sixth aspect, an embodiment of the present invention further provides a device for constructing an image detection model, where the device includes:
the feature extraction sub-network construction module is used for constructing a feature extraction sub-network according to a target detection model based on a two-step method;
the regression classification sub-network construction module is used for constructing a regression classification sub-network according to the target detection model based on the one-step method;
the target image detection model construction module is used for constructing a target image detection model by the feature extraction sub-network and the regression classification sub-network;
the characteristic extraction sub-network is used for extracting characteristics of an input image to obtain a characteristic layer; the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image; the input image is a compliant image and/or a non-compliant image.
In a seventh aspect, an embodiment of the present invention further provides an electronic device, including:
one or more processors;
storage means for storing one or more programs,
when the one or more programs are executed by the one or more processors, the one or more processors implement the image detection method, the training method of the image detection model, or the building method of the image detection model provided by any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements the image detection method, the training method of the image detection model, or the building method of the image detection model provided in any embodiment of the present invention.
The embodiment of the invention obtains the image to be detected; inputting an image to be detected into a compliance image detection model to obtain a first detection probability that the image to be detected is a compliance image, wherein the compliance image detection model is obtained by training a target image detection model based on a training sample formed by the compliance image and a non-compliance image; and determining whether the image to be detected is a compliant image according to the first detection probability. The method and the device realize the purpose of detecting whether the image is a compliant image or not by utilizing the compliant image detection model obtained by training the compliant image and the non-compliant image, achieve the purpose of forward detection of the image by the feature abstraction and the classification of the machine learning model, ensure that the image detection is not limited by an image type blacklist, avoid the problem of wrong image detection caused by incomplete image type blacklist and incapability of correctly identifying the non-compliant image, and improve the accuracy and the comprehensiveness of the image detection.
Drawings
FIG. 1 is a flowchart of an image detection method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of an image detection method according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram of a model structure of a target image detection model in a third embodiment of the present invention;
FIG. 4 is a schematic diagram of a model comparison result based on the model detection accuracy and the detection rate in the third embodiment of the present invention;
FIG. 5 is a flowchart of a training method of an image detection model in a fourth embodiment of the present invention;
FIG. 6 is a flowchart of a method for constructing an image detection model in a fifth embodiment of the present invention;
fig. 7 is a schematic structural diagram of an image detection device in a sixth embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a training device for an image detection model in a seventh embodiment of the present invention;
fig. 9 is a schematic structural diagram of an apparatus for constructing an image detection model in an eighth embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to a ninth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting thereof. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Example 1
The image detection method provided by the embodiment can be suitable for detecting the compliant image and the non-compliant image in the network environment. The method may be performed by an image detection device, which may be implemented in software and/or hardware, which may be integrated in an electronic device with image processing functions, such as a tablet, a desktop, a server or a server cluster, etc.
Referring to fig. 1, the image detection method of the present embodiment specifically includes the following steps:
s110, acquiring an image to be detected.
The image to be detected can be acquired in real time or can be read from a storage medium.
S120, inputting the image to be detected into a compliance image detection model, and obtaining a first detection probability that the image to be detected is the compliance image.
The compliance image detection model refers to a model for detecting whether an image is a compliance image. A compliant image is an image that meets the specifications of the relevant image distribution in the network environment. Accordingly, the non-compliant image is an image that does not meet the specifications issued by the related image in the network environment, such as a yellow-related image, a public character image, a riot image, a watermark image containing watermark information, and the like. The compliance image detection model is obtained by training a target image detection model, which belongs to a machine learning model, based on a training sample composed of a compliance image and an non-compliance image. The target image detection model here is a model capable of detecting whether an image is compliant, and belongs to a machine learning model of a target detection class. The model structure of the target image detection model may be the model structure of the machine learning model of the existing target detection or the model structure of the modified machine learning model (see description of the third embodiment). In order to improve the detection accuracy and comprehensiveness of the model on the compliant image, the embodiment of the invention adopts the compliant image and the non-compliant image to construct a training sample set so that the machine learning model can extract various image features through the characteristic abstract capability to learn and generate the compliant image detection model.
In the related art, the idea of reverse detection is adopted when the image detection is carried out, namely, the image types of the non-compliant images are exhausted in advance, and then whether the image to be detected belongs to the exemplified image type is judged, so that whether the image to be detected is the non-compliant image is determined. However, the manner of exhausting the non-compliant image types cannot timely adapt to the variation of the image release rule, and the definition of the same non-compliant image type cannot be unified, for example, the definition scale of the yellow-related image is different, for example, the watermark types in the watermark image are various, and the like, so that the image detection accuracy and the comprehensiveness according to the non-compliant image types are limited. Based on the above, the embodiment of the invention provides a forward detection idea, namely actively excavating the characteristics of the compliance image conforming to the regulations, and constructing a compliance image detection model. In specific implementation, the image to be detected is input into a compliance image detection model, and a probability value (namely a first detection probability) of the image to be detected being the compliance image is output through model operation processing.
S130, determining whether the image to be detected is a compliant image according to the first detection probability.
The larger the probability value of the first detection probability output by the model, the greater the probability that the image to be detected is a compliant image is, so whether the image to be detected is a compliant image can be determined according to the first detection probability. For example, a probability threshold (referred to as a first preset probability threshold) for determining whether the image to be detected is a compliant image may be set. And if the first detection probability is larger than a first preset probability threshold value, determining that the image to be detected is a compliant image. And if the first detection probability does not exceed the first preset probability threshold, determining that the image to be detected is a suspected image without determining whether the image is compliant or not. The suspected image may continue to be detected for non-compliant images or may be submitted directly to manual review.
The first preset probability threshold may be determined according to the training degree of the compliance image detection model, for example, the more the types and the number of training samples, the higher the training degree is considered, and the first preset probability threshold may be set to a larger value.
According to the technical scheme, an image to be detected is obtained; inputting an image to be detected into a compliance image detection model to obtain a first detection probability that the image to be detected is a compliance image, wherein the compliance image detection model is obtained by training a target image detection model based on a training sample formed by the compliance image and a non-compliance image, and the target image detection model belongs to a machine learning model; and determining whether the image to be detected is a compliant image according to the first detection probability. The method and the device realize the purpose of detecting whether the image is a compliant image or not by utilizing the compliant image detection model obtained by training the compliant image and the non-compliant image, achieve the purpose of forward detection of the image by the feature abstraction and the classification of the machine learning model, ensure that the image detection is not limited by an image type blacklist, avoid the problem of wrong image detection caused by incomplete image type blacklist and incapability of correctly identifying the non-compliant image, and improve the accuracy and the comprehensiveness of the image detection.
Example two
In this embodiment, the step of detecting the non-compliant image is added based on the first embodiment. Wherein, the explanation of the same or corresponding terms as the above embodiments is not repeated herein. Referring to fig. 2, the image detection method provided in this embodiment includes:
s210, acquiring an image to be detected.
S220, inputting the image to be detected into the non-compliant image detection model, and obtaining a second detection probability that the image to be detected is the non-compliant image.
The non-compliant image detection model is a model for detecting whether an image is a non-compliant image. The non-compliance image detection model is obtained by training a set machine learning model based on an image sample of a preset non-compliance image type. The preset non-compliant image type is an image type of a preset non-compliant image. The set machine learning model refers to a predetermined machine learning model of the target detection class. Illustratively, the machine learning model and the target image detection model are set to have the same model structure. Thus, the compliance image detection model and the non-compliance image detection model can be obtained by training a machine learning model with the same model structure, but the input image and the output result of the two are different in the training process. For example, the input image in the training process of the compliance image detection model is a compliance image and a non-compliance image, and the output result is the probability that the input image is the compliance image; and the input image in the training process of the non-compliance image detection model is an image of a preset non-compliance image type, and the output result is the probability that the input image is the non-compliance image. The method has the advantages that the machine learning model based on the same model structure is used for training to obtain various different image detection models, the development difficulty of model construction and training can be reduced, the development cost is saved, and the model generation speed is improved.
Considering that the types of the non-compliant images are relatively fixed, in the embodiment, the non-compliant image detection model is used for detecting the non-compliant image of the image to be detected. And inputting the image to be detected into an unconformity image detection model, and obtaining a probability value (namely a second detection probability) that the image to be detected is the unconformity image through model operation processing.
And S230, if the image to be detected is not the non-compliant image according to the second detection probability, inputting the image to be detected into a compliant image detection model to obtain a first detection probability that the image to be detected is the compliant image.
The larger the probability value of the second detection probability output by the model, the greater the probability that the image to be detected is an irregular image is, so that whether the image to be detected is an irregular image can be determined according to the second detection probability. For example, a probability threshold (referred to as a second preset probability threshold) for determining whether or not the image to be detected is a non-compliant image is set. If the second detection probability is larger than a second preset probability threshold, determining that the image to be detected is a non-compliant image, and ending the detection flow of the image to be detected. And if the second detection probability does not exceed the second preset probability threshold, determining that the image to be detected is a suspected image without determining whether the image is compliant or not. At this time, the image to be detected is input into a compliance image detection model, and the compliance image is detected, so that a first detection probability is obtained.
Similarly, the second preset probability threshold may be determined according to the training degree of the non-compliance image detection model, for example, the more the types and the number of training samples, the higher the training degree is considered, and the second preset probability threshold may be set to a larger value.
Illustratively, the preset non-compliant image type includes at least one of a watermark image, a yellow-related image, a public character image, and a riot image. The more types of the preset non-compliance image types are, the more comprehensive the detection of the non-compliance image is, and the more comprehensive and accurate the detection of the whole image is. The detection of the public character image belongs to face recognition detection, and a face database containing the public character face image is needed in the detection process. For the detection of watermark image types, considering the characteristics of multiple watermark patterns, unfixed watermark positions and sizes, complex background and the like, a machine learning model with the same model structure as that of a target image detection model can be used for model training and generation so as to improve the detection accuracy and efficiency of watermark images.
It should be noted that, when there are multiple types of preset non-compliant image types, the non-compliant image detection model may be a model for one type of preset non-compliant image type, so that the number of non-compliant image detection models is consistent with the number of preset non-compliant image types, so that the training degree of the model of each preset non-compliant image type can be improved, and thus the detection accuracy of the corresponding image type is improved. The non-compliance image detection model can be a model aiming at various preset non-compliance image types, so that the images of the various preset non-compliance image types are needed to be used as training samples to train the model, the model training difficulty is increased, the model detection precision is reduced, the number of the models can be reduced, the generation speed of the non-compliance image detection model is improved, and the detection of the non-compliance image is not needed to be carried out after all the models are completely built. The type selection of the non-compliant image detection model may be determined based on business requirements. In addition, no matter what type of model generation mode is selected to generate the non-compliance image detection model, incremental learning can be continuously performed on the non-compliance image detection model so as to continuously improve the actual prediction accuracy of the model.
S240, determining whether the image to be detected is a compliant image according to the first detection probability.
And if the image to be detected is determined to be a compliant image through the first detection probability, ending the detection flow of the image to be detected. If the image to be detected is determined to be a suspected image through the first detection probability, the image to be detected needs to be manually judged to be a compliant image or a non-compliant image. Further, if the number of the suspected images submitted to the manual review is multiple, the suspected images may be arranged in a descending order according to the first detection probability of each suspected image, and then the ordered suspected images are sent to the manual review, so that the manual review efficiency may be further improved.
According to the technical scheme, the image to be detected is input into the non-compliant image detection model, so that the second detection probability that the image to be detected is the non-compliant image is obtained; and if the image to be detected is not the non-compliant image according to the second detection probability, the step of inputting the image to be detected into a compliant image detection model to obtain a first detection probability that the image to be detected is the compliant image is executed. The method and the device realize the combination of the image detection of the compliance image and the non-compliance image detection, further improve the accuracy and the comprehensiveness of the image detection, reduce the probability of judging the image as the suspected image, reduce the number of the suspected images needing manual checking, reduce the workload of the manual checking, and further improve the image detection efficiency.
Example III
The present embodiment further optimizes the model structure of the target image detection model on the basis of the above embodiments. Wherein, the explanation of the same or corresponding terms as the above embodiments is not repeated herein. Referring to fig. 3, the model structure of the target image detection model provided in the present embodiment includes:
a feature extraction sub-network 310, configured to perform feature extraction on an input image to obtain a feature layer; the feature extraction subnetwork 310 is constructed based on a two-step target detection model;
the regression classification sub-network 320 is configured to perform classification regression on the feature layer obtained by the feature extraction sub-network 310 to obtain a detection result of the input image; the regression classification sub-network is constructed based on a one-step target detection model.
The target detection model based on the two-step method refers to a target detection model belonging to the two-step method, which performs detection and classification of a region of interest separately, and can be a region convolution neural network model (Region Convolutional Neural Networks, R-CNN), a Fast region convolution neural network model Fast R-CNN or a Faster region convolution neural network model Fast R-CNN, and the like. The one-step-based object detection model refers to an object detection model belonging to a one-stage/one-shot method, which uses a network to detect and classify a region of interest simultaneously, and can be a deformable component model (Deformable Parts Model, DPM), a one-step multi-frame detection model (Single Shot MultiBox Detector, SSD), an end-to-end object detection model (You Only Look Once, YOLO) series model, or the like.
The object detection model in the related art cannot well balance the model detection precision and the model detection real-time, but the image detection in the network environment needs to be performed quickly and accurately when the user issues the image so as to avoid affecting the use experience of the user. Based on this, a new model structure for object detection is proposed in the embodiment of the present invention. The model structure includes a feature extraction sub-network 310 and a regression classification sub-network 320. Considering that the detection accuracy of the candidate region of the target detection model based on the two-step method is high, the feature extraction sub-network 310 is constructed based on the model structure of the target detection model based on the two-step method in this embodiment, and is used for receiving the input image, and performing feature extraction operation on the input image to obtain the feature layer corresponding to the input image. Considering that the regression classification speed of the target detection model based on the one-step method is high, the regression classification sub-network 320 is constructed based on the model structure of the target detection model based on the one-step method in this embodiment, and is used for performing regression classification analysis on the feature layer obtained by the feature extraction sub-network 310 to obtain the detection result of the input image. Based on the new model structure, unbalance of positive samples and negative samples (background) is not needed to be considered too much in the model training stage, model training difficulty is reduced, and good model detection instantaneity can be achieved while high model detection accuracy is ensured.
Illustratively, the input image is a compliant image and/or a non-compliant image. The novel model structure is suitable for target detection, and both the compliance image detection and the non-compliance image detection belong to the target detection, so that the model structure can be used for training of the compliance image detection model and training of the non-compliance image detection model. When the model structure is used for the training of the compliance image detection model, the input image is a compliance image and a non-compliance image, and the output result is the probability that the input image is the compliance image; when the model structure is used for the non-compliance image detection model training, the input image is a non-compliance image of a preset non-compliance image type, and the output result is the probability that the input image is the non-compliance image. Therefore, the detection precision and the detection instantaneity of the compliance image detection model and the non-compliance image detection model can be improved simultaneously.
Illustratively, the feature extraction subnetwork 310 includes an input layer, a convolution layer, an activation layer, a pooling layer, and a first fully-connected layer in a two-step-based object detection model; and/or regression classification sub-network 320 includes a feature classification layer in the one-step-based object detection model.
The feature extraction subnetwork 310 is constructed using the input layer, convolution layer, activation layer, pooling layer, and first full-connection layer in a two-step-based object detection model for generating candidate regions. One convolution layer, one activation layer and one pooling layer connected in series are called a group, and one or more groups, preferably the first full connection layer and all layers before the first full connection layer in the target detection model based on the two-step method, can be contained in the feature extraction sub-network. Thus, candidate regions with higher precision and different scales and aspect ratios can be generated at different positions in the input image, and the precision and speed of the new model structure are further improved. In addition, the regression classification sub-network 320 may be constructed using feature classification layers in a one-step-based object detection model. The feature classification layer in the regression classification sub-network is preferably consistent with that in the one-step-based object detection model. Therefore, the extraction features corresponding to different candidate areas can be obtained once, and can be used for subsequent regression classification analysis, so that the precision and the speed of a new model structure are further improved. The construction and the realization of the model structure can be completed based on the development of an open source software library of machine learning, for example, the programming realization of the model structure is carried out by utilizing a symbolic mathematical system Tensorflow based on the programming of a data flow diagram, so that the development flow is simplified, the development difficulty is reduced, and the algorithm realization efficiency of a target image detection model is improved.
Based on the obtained new model structure, a series of candidate areas with different scales and length-width ratios are generated for the input image through unsupervised/supervised learning in the feature extraction sub-network, and then whether the candidate areas contain targets and which type of targets are judged through a CNN classifier of the regression classification sub-network. For the convolutional layer of the CNN classifier, the input image size may not be fixed, but it is required that the input size remains consistent from the fully connected layer later. Therefore, when the images with any size output by the feature extraction sub-network are input into the CNN classifier until the first full-connection layer, the feature images of all layers can be obtained by forward operation once, and then the regressive objects are the position information and the category information of the object to be detected, and regression can be carried out on the feature images with different layers according to the requirement of the object size.
Illustratively, the one-step-based object detection model is a model having an operation speed that is greater than an operation speed of the two-step-based object detection model. In order to further increase the operation speed of the new model structure, the operation speed of the selected one-step-based target detection model is required to be greater than that of the two-step-based target detection model.
Illustratively, the two-step-based target detection model is a Faster regional convolutional neural network fast R-CNN model, and the one-step-based target detection model is a one-step multi-frame detection SSD model.
Referring to fig. 4, the mean average accuracy mAP of the model is focused when selecting a two-step-based target detection model, while accounting for the model detection rate (Fps); and when a target detection model based on a one-step method is selected, the model detection rate (Fps) of the model is focused, and meanwhile, the mean average precision mAP is considered. Therefore, when selecting a target detection model based on a one-step method, selecting an SSD model with the fastest detection rate Fps and relatively higher average mean value average precision mAP in DPM, YOLO and SSD; when a target detection model based on a two-step method is selected, a Fast R-CNN model with highest average mean value average precision mAP and relatively Fast detection rate Fps in R-CNN, fast R-CNN and Fast R-CNN is selected. Thus, the novel model structure formed by combining the model structures of the fast R-CNN model and the SSD model can have higher detection precision and higher detection instantaneity.
According to the technical scheme, a new model structure comprising a feature extraction sub-network and a regression classification sub-network is designed, wherein the feature extraction sub-network is constructed based on a target detection model of a two-step method and is used for extracting features of an input image to obtain a feature layer; the regression classification sub-network is constructed based on a one-step target detection model and is used for carrying out classification regression on the feature layer obtained by the feature extraction sub-network 310 to obtain a detection result of the input image. The novel model structure is realized, unbalance of positive samples and negative samples (background) is not needed to be considered too much in the model training stage, model training difficulty is reduced, and good model detection instantaneity can be obtained while high model detection precision is ensured.
Example IV
The training method of the image detection model provided by the embodiment can be suitable for training whether the image is a compliant image detection model for compliant image detection. The method may be performed by a training device of the image detection model, which may be implemented in software and/or hardware, which may be integrated in an electronic device with big data computing capabilities, such as a desktop computer, a server or a supercomputer, etc. Wherein, the explanation of the same or corresponding terms as the above embodiments is not repeated herein.
Referring to fig. 5, the training method of the image detection model of the present embodiment specifically includes the following steps:
s410, classifying the collected images to generate an initial compliance image set and an initial non-compliance image set.
In the training process of the compliance image detection model, a large number of images are firstly collected, and the images comprise compliance images and non-compliance images. In order to optimize the model training process, the collected images are classified according to the compliance images and the non-compliance images, and an initial compliance image set only containing the compliance images and an initial non-compliance image set only containing the non-compliance images are generated.
Illustratively, classifying the collected images to generate an initial set of compliant images and an initial set of non-compliant images includes: performing non-compliance image recognition on the collected images based on a non-compliance image detection model, and generating an initial compliance image set, a suspected image set and an initial non-compliance image set, wherein the non-compliance image detection model is obtained by training a machine learning model based on an image sample of a preset non-compliance image type; and adding each image in the suspected image set into the initial compliance image set or the initial non-compliance image set according to the feedback result of whether each image in the suspected image set is compliant.
In order to improve the image classification accuracy and the classification speed, in the embodiment, an existing non-compliance image detection model is adopted to detect all collected images, and an image with the model detection determined as a non-compliance image is added into an initial non-compliance image set. And simultaneously, setting a third preset probability threshold, judging an image with the second detection probability smaller than the third preset probability threshold output by the model as a compliance image, and adding the compliance image into the initial compliance image set. And the image with the second detection probability between the second preset probability threshold and the third preset probability threshold is judged to be a suspected image, and a suspected image set is added. And (3) submitting the suspected image sets to manual verification, and adding the suspected image sets into the corresponding image sets according to the verification results of compliance or non-compliance of each suspected image by manual verification. The advantage of setting up like this is that increase training sample quantity on the basis of guaranteeing classification more accurate, improves model training degree to improve the detection precision of model.
S420, training a target image detection model based on the initial compliance image set and the initial non-compliance image set to generate a compliance image detection model, wherein the target image detection model belongs to a machine learning model, and the compliance image detection model is used for detecting whether an input image is a compliance image.
And training the target image detection model by taking the images in the initial compliance image set and the initial non-compliance image set as training inputs of the models, and outputting training results of the corresponding input images. And then, comparing the training result with the prior information of the compliance or non-compliance corresponding to the input image, and calculating the model deviation of the training. And then, the model deviation is subjected to error reflection by using the loss function to update the model parameters (weights) in the target image detection model. The process is repeated until the model deviation falls within the allowable range (i.e. the model convergence condition is reached), and the model training process can be ended, so as to generate the compliance image detection model.
Illustratively, training the target image detection model based on the initial set of compliant images and the initial set of non-compliant images includes: performing person classification and scene classification on the initial compliance image set and the initial non-compliance image set respectively to generate a person compliance image set, a scene compliance image set, a person non-compliance image set and a scene non-compliance image set; training the target image detection model based on the character compliance image set, the scene compliance image set, the character non-compliance image set and the scene non-compliance image set to generate a compliance image detection model.
In this embodiment, each training image is classified into a fine class, for example, each image in the initial compliance image set and the initial non-compliance image set is classified into a person (including clothing) class and a scene class by using an image recognition method such as principal component analysis, and finally a person compliance image set indicating that the image belongs to a person compliance, a scene compliance image set indicating that the image belongs to a scene compliance, a person non-compliance image set indicating that the image belongs to a person non-compliance, and a scene non-compliance image set indicating that the image belongs to a scene non-compliance are generated. When training the target image detection model by using the images in the image set of the fine classification, the accuracy of model training can be improved, and the detection accuracy of the compliance image detection model can be further improved.
In order to improve the accuracy and speed of the compliance image detection model, the target image detection model in this embodiment may adopt the new model structure in the third embodiment, specifically as follows:
the target image detection model includes:
the feature extraction sub-network is used for extracting features of an input image to obtain a feature layer, wherein the input image is a compliant image and/or a non-compliant image;
the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image; the feature extraction sub-network is constructed based on a target detection model of a two-step method, and the regression classification sub-network is constructed based on a target detection model of a one-step method.
Further, the feature extraction sub-network comprises an input layer, a convolution layer, an activation layer and a pooling layer in the target detection model based on the two-step method; and/or the regression classification sub-network includes a full connection layer and a feature classification layer in the one-step-based object detection model.
Further, the one-step-based target detection model is a model having an operation speed greater than that of the two-step-based target detection model.
Further, the target detection model based on the two-step method is a Faster regional convolutional neural network Faster R-CNN model, and the target detection model based on the one-step method is a one-step multi-frame detection SSD model.
It should be noted that, incremental learning may be continuously performed on the composite image detection model to continuously improve the actual prediction accuracy of the model.
According to the technical scheme, the collected images are classified to generate an initial compliance image set and an initial non-compliance image set; training a target image detection model based on the initial compliance image set and the initial non-compliance image set to generate a compliance image detection model, wherein the target image detection model belongs to a machine learning model, and the compliance image detection model is used for detecting whether an input image is a compliance image. The method and the device realize training of the target image detection model through the initial compliance image set and the initial non-compliance image set, generate the compliance image detection model, and improve the training degree of the compliance image detection model by utilizing more comprehensive image types and more image numbers, thereby improving the detection precision of the compliance image detection model.
Example five
The method for constructing the image detection model provided by the embodiment can be suitable for constructing the target image detection model. The method may be performed by a means of construction of an image detection model, which may be implemented in software and/or hardware, which may be integrated in an electronic device with machine learning model processing capabilities, such as a desktop computer, a server or a supercomputer, etc.
Referring to fig. 6, the method for constructing an image detection model of the present embodiment specifically includes the following steps:
s510, constructing a feature extraction sub-network according to a target detection model based on a two-step method; the characteristic extraction sub-network is used for extracting characteristics of the input image to obtain a characteristic layer; the input image is a compliant image and/or a non-compliant image.
Illustratively, S510 includes: and constructing a feature extraction sub-network according to an input layer, a convolution layer, an activation layer, a pooling layer and a first full-connection layer in the target detection model based on the two-step method.
S520, constructing a regression classification sub-network according to a target detection model based on a one-step method; the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image.
Illustratively, S520 includes: and constructing a regression classification sub-network according to the feature classification layer in the target detection model based on the one-step method.
S530, constructing a target image detection model by the feature extraction sub-network and the regression classification sub-network.
Illustratively, the one-step-based object detection model is a model having an operation speed that is greater than an operation speed of the two-step-based object detection model.
Illustratively, the two-step-based target detection model is a Faster regional convolutional neural network fast R-CNN model, and the one-step-based target detection model is a one-step multi-frame detection SSD model.
The above-described model structure of the image detection model and the explanation thereof can be referred to the explanation of the third embodiment.
According to the technical scheme, a characteristic extraction sub-network is constructed according to a target detection model based on a two-step method; constructing a regression classification sub-network according to a target detection model based on a one-step method; constructing a target image detection model by the feature extraction sub-network and the regression classification sub-network; the characteristic extraction sub-network is used for extracting characteristics of the input image to obtain a characteristic layer; the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image; the input image is a compliant image and/or a non-compliant image. The method realizes the construction of the target image detection model of a new model structure, so that the target image detection model does not need to take unbalance of positive samples and negative samples (background) into consideration in the model training stage, the model training difficulty is reduced, and good model detection instantaneity can be obtained while high model detection precision is ensured.
Example six
The present embodiment provides an image detection apparatus, referring to fig. 7, which specifically includes:
an image acquisition module 710, configured to acquire an image to be detected;
the first detection probability obtaining module 720 is configured to input an image to be detected into a compliance image detection model, and obtain a first detection probability that the image to be detected is a compliance image, where the compliance image detection model is obtained by training a target image detection model based on a training sample composed of a compliance image and an non-compliance image, and the target image detection model belongs to a machine learning model;
and the compliance image detection module 730 is configured to determine whether the image to be detected is a compliance image according to the first detection probability.
Optionally, on the basis of the above device, the device further includes a non-compliance image detection module, configured to:
before inputting the image to be detected into the compliance image detection model to obtain the first detection probability that the image to be detected is the compliance image, the method further comprises the following steps:
inputting an image to be detected into an non-compliance image detection model to obtain a second detection probability that the image to be detected is the non-compliance image, wherein the non-compliance image detection model is obtained by training a set machine learning model based on an image sample of a preset non-compliance image type;
And if the image to be detected is not the non-compliant image according to the second detection probability, the step of inputting the image to be detected into a compliant image detection model to obtain a first detection probability that the image to be detected is the compliant image is executed.
Alternatively, the machine learning model and the target image detection model are set to have the same model structure.
Further, the model structure includes:
the feature extraction sub-network is used for extracting features of an input image to obtain a feature layer, wherein the input image is a compliant image and/or a non-compliant image;
the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image;
the feature extraction sub-network is constructed based on a target detection model of a two-step method, and the regression classification sub-network is constructed based on a target detection model of a one-step method.
Optionally, the feature extraction sub-network comprises an input layer, a convolution layer, an activation layer, a pooling layer and a first full-connection layer in the two-step-based target detection model; and/or
The regression classification sub-network includes a feature classification layer in a one-step-based object detection model.
Alternatively, the one-step-based object detection model is a model having an operation speed greater than that of the two-step-based object detection model.
Further, the target detection model based on the two-step method is a Faster regional convolutional neural network Faster R-CNN model, and the target detection model based on the one-step method is a one-step multi-frame detection SSD model.
Optionally, the preset non-compliance image type includes at least one of a watermark image, a yellow-related image, a public character image, and a riot image.
According to the image detection device provided by the embodiment of the invention, whether the image is the compliant image or not is detected by utilizing the compliant image detection model obtained by training the compliant image and the non-compliant image, so that the purpose of forward detection of the image through the feature abstraction and classification of the machine learning model is achieved, the image detection is not limited by the image type blacklist, the problem of image detection errors caused by the fact that the non-compliant image cannot be correctly identified due to incomplete image type blacklist is avoided, and the accuracy and the comprehensiveness of the image detection can be improved.
The image detection device provided by the embodiment of the invention can execute the image detection method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example seven
The present embodiment provides a training device for an image detection model, referring to fig. 8, the device specifically includes:
A sample classification module 810 for classifying the collected images to generate an initial set of compliant images and an initial set of non-compliant images;
the model training module 820 is configured to train the target image detection model based on the initial set of compliant images and the initial set of non-compliant images, and generate a compliant image detection model, where the target image detection model belongs to a machine learning model, and the compliant image detection model is configured to detect whether the input image is a compliant image.
Optionally, the model training module 820 is specifically configured to:
performing person classification and scene classification on the initial compliance image set and the initial non-compliance image set respectively to generate a person compliance image set, a scene compliance image set, a person non-compliance image set and a scene non-compliance image set;
training the target image detection model based on the character compliance image set, the scene compliance image set, the character non-compliance image set and the scene non-compliance image set to generate a compliance image detection model.
Optionally, the sample classification module 810 is specifically configured to:
performing non-compliance image recognition on the collected images based on a non-compliance image detection model, and generating an initial compliance image set, a suspected image set and an initial non-compliance image set, wherein the non-compliance image detection model is obtained by training a machine learning model based on an image sample of a preset non-compliance image type;
And adding each image in the suspected image set into the initial compliance image set or the initial non-compliance image set according to the feedback result of whether each image in the suspected image set is compliant.
Optionally, the target image detection model includes:
the feature extraction sub-network is used for extracting features of an input image to obtain a feature layer, wherein the input image is a compliant image and/or a non-compliant image;
the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image;
the feature extraction sub-network is constructed based on a target detection model of a two-step method, and the regression classification sub-network is constructed based on a target detection model of a one-step method.
Optionally, the feature extraction sub-network comprises an input layer, a convolution layer, an activation layer, a pooling layer and a first full-connection layer in the two-step-based target detection model; and/or
The regression classification sub-network includes a feature classification layer in a one-step-based object detection model.
Alternatively, the one-step-based object detection model is a model having an operation speed greater than that of the two-step-based object detection model.
Further, the target detection model based on the two-step method is a Faster regional convolutional neural network Faster R-CNN model, and the target detection model based on the one-step method is a one-step multi-frame detection SSD model.
According to the training device for the image detection model, provided by the embodiment of the invention, the training of the target image detection model through the initial compliance image set and the initial non-compliance image set is realized, the compliance image detection model is generated, and the training degree of the compliance image detection model is improved by using more comprehensive image types and more image numbers, so that the detection precision of the compliance image detection model is improved.
The training device for the image detection model provided by the embodiment of the invention can execute the training method for the image detection model provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example eight
The present embodiment provides a device for constructing an image detection model, referring to fig. 9, the device specifically includes:
a feature extraction sub-network construction module 910, configured to construct a feature extraction sub-network according to a two-step-based object detection model;
the regression classification sub-network construction module 920 is configured to construct a regression classification sub-network according to the one-step-based target detection model;
the target image detection model construction module 930 is configured to construct a target image detection model from the feature extraction sub-network and the regression classification sub-network;
The characteristic extraction sub-network is used for extracting characteristics of the input image to obtain a characteristic layer; the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image; the input image is a compliant image and/or a non-compliant image. Optionally, the feature extraction sub-network construction module 910 is specifically configured to:
and constructing a feature extraction sub-network according to an input layer, a convolution layer, an activation layer, a pooling layer and a first full-connection layer in the target detection model based on the two-step method.
Optionally, the regression-classification-subnetwork construction module 920 is specifically configured to:
and constructing a regression classification sub-network according to the feature classification layer in the target detection model based on the one-step method.
Alternatively, the one-step-based object detection model is a model having an operation speed greater than that of the two-step-based object detection model.
Optionally, the two-step-based target detection model is a Faster regional convolutional neural network fast R-CNN model, and the one-step-based target detection model is a one-step multi-frame detection SSD model.
By the aid of the image detection model construction device, a new model structure target image detection model is constructed, so that the target image detection model does not need to take unbalance between positive samples and negative samples (background) into consideration in a model training stage, model training difficulty is reduced, high model detection accuracy is ensured, and good model detection instantaneity can be achieved.
The image detection model construction device provided by the embodiment of the invention can execute the image detection model construction method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
It should be noted that, in the embodiments of the above devices, each unit and module included are only divided according to the functional logic, but not limited to the above division, so long as the corresponding functions can be implemented; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present invention.
Example nine
Referring to fig. 10, the present embodiment provides an electronic device 1000, which includes: one or more processors 1020; the storage 1010 is configured to store one or more programs, where the one or more programs are executed by the one or more processors 1020, so that the one or more processors 1020 implement the image detection method provided by the embodiment of the present invention, and the method includes:
acquiring an image to be detected;
inputting an image to be detected into a compliance image detection model to obtain a first detection probability that the image to be detected is a compliance image, wherein the compliance image detection model is obtained by training a target image detection model based on a training sample formed by the compliance image and a non-compliance image, and the target image detection model belongs to a machine learning model;
And determining whether the image to be detected is a compliant image according to the first detection probability.
Of course, those skilled in the art will appreciate that the processor 1020 may also implement the technical solution of the image detection method provided in any embodiment of the present invention.
The electronic device 1000 shown in fig. 10 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 10, the electronic device 1000 is embodied in the form of a general purpose computing device. Components of electronic device 1000 may include, but are not limited to: one or more processors 1020, a memory device 1010, and a bus 1050 that connects the different system components (including the memory device 1010 and the processor 1020).
Bus 1050 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Electronic device 1000 typically includes many types of computer system readable media. Such media can be any available media that is accessible by the electronic device 1000 and includes both volatile and nonvolatile media, removable and non-removable media.
The storage 1010 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 1011 and/or cache memory 1012. Electronic device 1000 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, the storage system 1013 may be configured to read from and write to non-removable, non-volatile magnetic media (not shown in FIG. 10, commonly referred to as a "hard disk drive"). Although not shown in fig. 10, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In such cases, each drive may be coupled to bus 1050 via one or more data medium interfaces. The storage 1010 may include at least one program product having a set (e.g., at least one) of program modules configured to carry out the functions of the embodiments of the invention.
A program/utility 1014 having a set (at least one) of program modules 1015 may be stored, for example, in storage 1010, such program modules 1015 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment. Program modules 1015 generally perform the functions and/or methods of any of the embodiments described herein.
The electronic device 1000 may also be in communication with one or more external devices 1060 (e.g., keyboard, pointing device, display 1070, etc.), with one or more devices that enable a user to interact with the electronic device 1000, and/or with any device (e.g., network card, modem, etc.) that enables the electronic device 1000 to communicate with one or more other computing devices. Such communication may occur through an input/output interface (I/O interface) 1030. Also, the electronic device 1000 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet, through the network adapter 1040. As shown in fig. 10, the network adapter 1040 communicates with other modules of the electronic device 1000 via a bus 1050. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with the electronic device 1000, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
The processor 1020 executes various functional applications and data processing by running a program stored in the storage 1010, for example, to implement the image detection method provided by the embodiment of the present invention.
The embodiment of the invention also provides another electronic device, which comprises: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by one or more processors, the one or more processors realize the training method of the image detection model provided by the embodiment of the invention, and the training method comprises the following steps:
classifying the collected images to generate an initial compliance image set and an initial non-compliance image set;
training a target image detection model based on the initial compliance image set and the initial non-compliance image set to generate a compliance image detection model, wherein the target image detection model belongs to a machine learning model, and the compliance image detection model is used for detecting whether an input image is a compliance image.
Of course, those skilled in the art will understand that the processor may also implement the technical solution of the training method of the image detection model provided in any embodiment of the present invention. The hardware structure and function of the electronic device can be explained with reference to the contents of the ninth embodiment.
The embodiment of the invention also provides another electronic device, which comprises: one or more processors; the storage device is used for storing one or more programs, and when the one or more programs are executed by one or more processors, the one or more processors realize the method for constructing the image detection model provided by the embodiment of the invention, and the method comprises the following steps:
constructing a feature extraction sub-network according to a target detection model based on a two-step method;
constructing a regression classification sub-network according to a target detection model based on a one-step method;
constructing a target image detection model by the feature extraction sub-network and the regression classification sub-network;
the characteristic extraction sub-network is used for extracting characteristics of the input image to obtain a characteristic layer; the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image; the input image is a compliant image and/or a non-compliant image.
Of course, it will be understood by those skilled in the art that the processor may also implement the technical scheme of the method for constructing an image detection model provided by any embodiment of the present invention. The hardware structure and function of the electronic device can be explained with reference to the contents of the ninth embodiment.
Examples ten
The present embodiment provides a storage medium containing computer executable instructions which, when executed by a computer processor, are for performing an image detection method comprising:
acquiring an image to be detected;
inputting an image to be detected into a compliance image detection model to obtain a first detection probability that the image to be detected is a compliance image, wherein the compliance image detection model is obtained by training a target image detection model based on a training sample formed by the compliance image and a non-compliance image, and the target image detection model belongs to a machine learning model;
and determining whether the image to be detected is a compliant image according to the first detection probability.
Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the above method operations, and may also perform the related operations in the image detection method provided in any embodiment of the present invention.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
Embodiments of the present invention also provide another computer-readable storage medium, which when executed by a computer processor, is configured to perform a training method for an image detection model, the method comprising:
classifying the collected images to generate an initial compliance image set and an initial non-compliance image set;
training a target image detection model based on the initial compliance image set and the initial non-compliance image set to generate a compliance image detection model, wherein the target image detection model belongs to a machine learning model, and the compliance image detection model is used for detecting whether an input image is a compliance image.
Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the above method operations, and may also perform the related operations in the training method of the image detection model provided in any embodiment of the present invention. The description of the storage medium can be explained with reference to the contents of the tenth embodiment.
The embodiment of the invention also provides another computer readable storage medium, which is used for executing a method for constructing an image detection model when being executed by a computer processor, the method comprises the following steps:
Constructing a feature extraction sub-network according to a target detection model based on a two-step method;
constructing a regression classification sub-network according to a target detection model based on a one-step method;
constructing a target image detection model by the feature extraction sub-network and the regression classification sub-network;
the characteristic extraction sub-network is used for extracting characteristics of the input image to obtain a characteristic layer; the regression classification sub-network is used for carrying out classification regression on the feature layer to obtain a detection result of the input image; the input image is a compliant image and/or a non-compliant image.
Of course, the storage medium containing the computer executable instructions provided in the embodiments of the present invention is not limited to the above method operations, and may also perform the related operations in the method for constructing the image detection model provided in any embodiment of the present invention. The description of the storage medium can be explained with reference to the contents of the tenth embodiment.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (14)
1. An image detection method, comprising:
acquiring an image to be detected;
inputting the image to be detected into an non-compliance image detection model to obtain a second detection probability that the image to be detected is a non-compliance image, wherein the non-compliance image detection model is obtained by training a set machine learning model based on an image sample of a preset non-compliance image type; the non-compliance image detection model is a model for detecting whether the image to be detected is a non-compliance image;
if the image to be detected is not the non-compliant image according to the second detection probability, inputting the image to be detected into a compliant image detection model to obtain a first detection probability that the image to be detected is the compliant image, wherein the compliant image detection model is obtained by training a target image detection model based on a training sample consisting of the compliant image and the non-compliant image; the compliance image detection model is a model for detecting whether the image to be detected is a compliance image;
and determining whether the image to be detected is a compliant image according to the first detection probability.
2. The method of claim 1, wherein the set machine learning model and the target image detection model have the same model structure.
3. The method of claim 2, wherein the model structure comprises:
the feature extraction sub-network is used for extracting features of an input image to obtain a feature layer, wherein the input image is a compliant image and/or a non-compliant image;
the regression classification sub-network is used for carrying out classification regression on the characteristic layer to obtain a detection result of the input image;
the feature extraction sub-network is constructed based on a target detection model of a two-step method, and the regression classification sub-network is constructed based on a target detection model of a one-step method.
4. A method according to claim 3, wherein the feature extraction subnetwork comprises an input layer, a convolution layer, an activation layer, a pooling layer and a first fully connected layer in a two-step based object detection model; and/or
The regression classification sub-network includes a feature classification layer in a one-step-based object detection model.
5. The method of claim 3, wherein the one-step-based object detection model is a model having an operational speed greater than an operational speed of the two-step-based object detection model.
6. A training method of an image detection model based on the image detection method of claim 1, characterized in that the training method comprises:
Classifying the collected images to generate an initial compliance image set and an initial non-compliance image set;
training a target image detection model based on the initial compliance image set and the initial non-compliance image set to generate a compliance image detection model, wherein the compliance image detection model is used for detecting whether an input image is a compliance image or not;
training a machine learning model based on an image sample of a preset non-compliance image type to obtain a non-compliance image detection model, wherein the non-compliance image detection model is used for detecting the collected images, and adding the images which are detected and determined to be non-compliance images into the initial non-compliance image set.
7. The method of claim 6, wherein training a target image detection model based on the initial set of compliant images and the initial set of non-compliant images comprises:
performing person classification and scene classification on the initial compliance image set and the initial non-compliance image set respectively to generate a person compliance image set, a scene compliance image set, a person non-compliance image set and a scene non-compliance image set;
Training the target image detection model based on the person compliance image set, the scene compliance image set, the person non-compliance image set and the scene non-compliance image set to generate the compliance image detection model.
8. The method of claim 6, wherein classifying the collected images to generate an initial set of compliant images and an initial set of non-compliant images comprises:
performing non-compliance image recognition on the collected images based on the non-compliance image detection model to generate an initial compliance image set, a suspected image set and an initial non-compliance image set;
and adding each image in the suspected image set into the initial compliance image set or the initial non-compliance image set according to a feedback result of whether each image in the suspected image set is compliant.
9. The method of claim 6, wherein the target image detection model comprises:
the feature extraction sub-network is used for extracting features of an input image to obtain a feature layer, wherein the input image is a compliant image and/or a non-compliant image;
the regression classification sub-network is used for carrying out classification regression on the characteristic layer to obtain a detection result of the input image;
The feature extraction sub-network is constructed based on a target detection model of a two-step method, and the regression classification sub-network is constructed based on a target detection model of a one-step method.
10. The method of claim 9, wherein the feature extraction subnetwork comprises an input layer, a convolution layer, an activation layer, a pooling layer, and a first fully connected layer in a two-step-based object detection model; and/or
The regression classification sub-network includes a feature classification layer in a one-step-based object detection model.
11. An image detection apparatus, comprising:
the image acquisition module is used for acquiring an image to be detected;
the second detection probability obtaining module is used for inputting the image to be detected into an unconformity image detection model to obtain a second detection probability that the image to be detected is an unconformity image, wherein the unconformity image detection model is obtained by training a set machine learning model based on an image sample of a preset unconformity image type; the non-compliance image detection model is a model for detecting whether the image to be detected is a non-compliance image;
the first detection probability obtaining module is used for inputting the image to be detected into a compliance image detection model to obtain the first detection probability that the image to be detected is a compliance image if the image to be detected is judged not to be the non-compliance image according to the second detection probability, wherein the compliance image detection model is obtained by training a target image detection model based on a training sample formed by the compliance image and the non-compliance image; the compliance image detection model is a model for detecting whether the image to be detected is a compliance image;
And the compliance image detection module is used for determining whether the image to be detected is a compliance image according to the first detection probability.
12. Training device for an image detection model, based on the image detection device according to claim 11, characterized in that it comprises:
the sample classification module is used for classifying the collected images to generate an initial compliance image set and an initial non-compliance image set;
the model training module is used for training a target image detection model based on the initial compliance image set and the initial non-compliance image set to generate a compliance image detection model, wherein the compliance image detection model is used for detecting whether an input image is a compliance image or not;
training a machine learning model based on an image sample of a preset non-compliance image type to obtain a non-compliance image detection model, wherein the non-compliance image detection model is used for detecting the collected images, and adding the images which are detected and determined to be non-compliance images into the initial non-compliance image set.
13. An electronic device, the electronic device comprising:
one or more processors;
storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the image detection method of any of claims 1-5 or the training method of the image detection model of any of claims 6-10.
14. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the image detection method according to any one of claims 1-5 or the training method of the image detection model according to any one of claims 6-10.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010407166.0A CN111738290B (en) | 2020-05-14 | 2020-05-14 | Image detection method, model construction and training method, device, equipment and medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010407166.0A CN111738290B (en) | 2020-05-14 | 2020-05-14 | Image detection method, model construction and training method, device, equipment and medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111738290A CN111738290A (en) | 2020-10-02 |
CN111738290B true CN111738290B (en) | 2024-04-09 |
Family
ID=72647468
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010407166.0A Active CN111738290B (en) | 2020-05-14 | 2020-05-14 | Image detection method, model construction and training method, device, equipment and medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111738290B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113743439A (en) * | 2020-11-13 | 2021-12-03 | 北京沃东天骏信息技术有限公司 | Target detection method, device and storage medium |
CN115086721B (en) * | 2022-08-22 | 2022-10-25 | 深圳市稻兴实业有限公司 | Ultra-high-definition live system service supervision system based on data analysis |
Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105095911A (en) * | 2015-07-31 | 2015-11-25 | 小米科技有限责任公司 | Sensitive picture identification method and apparatus, and server |
CN105989330A (en) * | 2015-02-03 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Picture detection method and apparatus |
CN107122806A (en) * | 2017-05-16 | 2017-09-01 | 北京京东尚科信息技术有限公司 | A kind of nude picture detection method and device |
CN107609598A (en) * | 2017-09-27 | 2018-01-19 | 武汉斗鱼网络科技有限公司 | Image authentication model training method, device and readable storage medium storing program for executing |
AU2017101803A4 (en) * | 2017-12-24 | 2018-02-15 | Chen, Mufei MS | Deep learning based image classification of dangerous goods of gun type |
CN107871314A (en) * | 2016-09-23 | 2018-04-03 | 商汤集团有限公司 | A kind of sensitive image discrimination method and device |
CN109101523A (en) * | 2018-06-14 | 2018-12-28 | 北京搜狗科技发展有限公司 | A kind of image processing method, device and electronic equipment |
CN109583501A (en) * | 2018-11-30 | 2019-04-05 | 广州市百果园信息技术有限公司 | Picture classification, the generation method of Classification and Identification model, device, equipment and medium |
CN109886319A (en) * | 2019-01-30 | 2019-06-14 | 杭州芯影科技有限公司 | A kind of concealment object detecting method and system based on millimeter wave safety check image |
CN109977848A (en) * | 2019-03-22 | 2019-07-05 | 广州新视展投资咨询有限公司 | Training method and device, the computer equipment and readable medium of pornographic detection model |
CN110163143A (en) * | 2019-05-17 | 2019-08-23 | 国网河北省电力有限公司沧州供电分公司 | Unlawful practice recognition methods, device and terminal device |
CN110321927A (en) * | 2019-06-03 | 2019-10-11 | 深圳中兴网信科技有限公司 | Generation method, computer equipment and the readable storage medium storing program for executing of violation detection model |
CN111046956A (en) * | 2019-12-13 | 2020-04-21 | 苏州科达科技股份有限公司 | Occlusion image detection method and device, electronic equipment and storage medium |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9355406B2 (en) * | 2013-07-18 | 2016-05-31 | GumGum, Inc. | Systems and methods for determining image safety |
-
2020
- 2020-05-14 CN CN202010407166.0A patent/CN111738290B/en active Active
Patent Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105989330A (en) * | 2015-02-03 | 2016-10-05 | 阿里巴巴集团控股有限公司 | Picture detection method and apparatus |
EP3125153A1 (en) * | 2015-07-31 | 2017-02-01 | Xiaomi Inc. | Method, device and server for sensitive picture recognition |
CN105095911A (en) * | 2015-07-31 | 2015-11-25 | 小米科技有限责任公司 | Sensitive picture identification method and apparatus, and server |
CN107871314A (en) * | 2016-09-23 | 2018-04-03 | 商汤集团有限公司 | A kind of sensitive image discrimination method and device |
CN107122806A (en) * | 2017-05-16 | 2017-09-01 | 北京京东尚科信息技术有限公司 | A kind of nude picture detection method and device |
CN107609598A (en) * | 2017-09-27 | 2018-01-19 | 武汉斗鱼网络科技有限公司 | Image authentication model training method, device and readable storage medium storing program for executing |
AU2017101803A4 (en) * | 2017-12-24 | 2018-02-15 | Chen, Mufei MS | Deep learning based image classification of dangerous goods of gun type |
CN109101523A (en) * | 2018-06-14 | 2018-12-28 | 北京搜狗科技发展有限公司 | A kind of image processing method, device and electronic equipment |
CN109583501A (en) * | 2018-11-30 | 2019-04-05 | 广州市百果园信息技术有限公司 | Picture classification, the generation method of Classification and Identification model, device, equipment and medium |
CN109886319A (en) * | 2019-01-30 | 2019-06-14 | 杭州芯影科技有限公司 | A kind of concealment object detecting method and system based on millimeter wave safety check image |
CN109977848A (en) * | 2019-03-22 | 2019-07-05 | 广州新视展投资咨询有限公司 | Training method and device, the computer equipment and readable medium of pornographic detection model |
CN110163143A (en) * | 2019-05-17 | 2019-08-23 | 国网河北省电力有限公司沧州供电分公司 | Unlawful practice recognition methods, device and terminal device |
CN110321927A (en) * | 2019-06-03 | 2019-10-11 | 深圳中兴网信科技有限公司 | Generation method, computer equipment and the readable storage medium storing program for executing of violation detection model |
CN111046956A (en) * | 2019-12-13 | 2020-04-21 | 苏州科达科技股份有限公司 | Occlusion image detection method and device, electronic equipment and storage medium |
Non-Patent Citations (4)
Title |
---|
Fudong Nian et al.Pornographic image detection utilizing deep convolutional neural networks.《Neurocomputing》.2016,283-293. * |
KaiLong Zhou et al.Convolutional Neural Networks based Pornographic Image Classification.《2016 IEEE Second International Conference on Multimedia Big Data》.2016,206-209. * |
余明扬 等.基于卷积神经网络的色情图像检测.《计算机应用与软件》.2018,第35卷(第1期),232-236,275. * |
王景中 ; 杨源 ; 何云华 ; .基于多分类和ResNet的不良图片识别框架.《计算机系统应用》.2018,第27卷(第09期),100-106. * |
Also Published As
Publication number | Publication date |
---|---|
CN111738290A (en) | 2020-10-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107992596B (en) | Text clustering method, text clustering device, server and storage medium | |
CN107633265B (en) | Data processing method and device for optimizing credit evaluation model | |
WO2022179138A1 (en) | Image processing method and apparatus, and computer device and storage medium | |
WO2023056723A1 (en) | Fault diagnosis method and apparatus, and electronic device and storage medium | |
CN115240145A (en) | Method and system for detecting illegal operation behaviors based on scene recognition | |
CN112883990A (en) | Data classification method and device, computer storage medium and electronic equipment | |
CN111738290B (en) | Image detection method, model construction and training method, device, equipment and medium | |
CN114418124A (en) | Method, device, equipment and storage medium for generating graph neural network model | |
CN111400695B (en) | Equipment fingerprint generation method, device, equipment and medium | |
CN112381458A (en) | Project evaluation method, project evaluation device, equipment and storage medium | |
CN114692778B (en) | Multi-mode sample set generation method, training method and device for intelligent inspection | |
CN114078008A (en) | Abnormal behavior detection method, device, equipment and computer readable storage medium | |
CN112579781B (en) | Text classification method, device, electronic equipment and medium | |
CN114037912A (en) | Method and device for detecting change of remote sensing image and computer readable storage medium | |
CN113591998A (en) | Method, device, equipment and storage medium for training and using classification model | |
RU2715024C1 (en) | Method of trained recurrent neural network debugging | |
CN117875320A (en) | Data processing method, device, equipment and storage medium based on artificial intelligence | |
CN116245630A (en) | Anti-fraud detection method and device, electronic equipment and medium | |
CN110826616A (en) | Information processing method and device, electronic equipment and storage medium | |
WO2023060954A1 (en) | Data processing method and apparatus, data quality inspection method and apparatus, and readable storage medium | |
CN112487394A (en) | Method, system, device and medium for identifying graph reasoning verification code | |
CN112632284A (en) | Information extraction method and system for unlabeled text data set | |
CN116069673B (en) | Simulation application operation control method, device, electronic equipment and medium | |
CN117478434B (en) | Edge node network traffic data processing method, device, equipment and medium | |
CN117058432B (en) | Image duplicate checking method and device, electronic equipment and readable storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |