CN112819715B - Data recovery method, network training method, related device and electronic equipment - Google Patents
Data recovery method, network training method, related device and electronic equipment Download PDFInfo
- Publication number
- CN112819715B CN112819715B CN202110127932.2A CN202110127932A CN112819715B CN 112819715 B CN112819715 B CN 112819715B CN 202110127932 A CN202110127932 A CN 202110127932A CN 112819715 B CN112819715 B CN 112819715B
- Authority
- CN
- China
- Prior art keywords
- feature
- person
- image data
- attribute
- character
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000012549 training Methods 0.000 title claims abstract description 42
- 238000011084 recovery Methods 0.000 title claims abstract description 22
- 238000013528 artificial neural network Methods 0.000 claims abstract description 81
- 238000000605 extraction Methods 0.000 claims abstract description 63
- 238000012545 processing Methods 0.000 claims abstract description 53
- 238000002372 labelling Methods 0.000 claims description 25
- 238000013473 artificial intelligence Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000013135 deep learning Methods 0.000 abstract description 3
- 238000004590 computer program Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 10
- 230000009467 reduction Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000011282 treatment Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000011176 pooling Methods 0.000 description 4
- 230000003796 beauty Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 239000002537 cosmetic Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002087 whitening effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000005808 skin problem Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a data recovery method, a network training method, a related device and electronic equipment, and relates to the field of artificial intelligence such as computer vision, deep learning and the like. The specific implementation scheme is as follows: acquiring first person image data, wherein the first person image data is person image data subjected to image processing; performing feature extraction on the first human image data based on a target neural network to obtain first human features, wherein the first human features are the human features of the first human image data before the image processing; second person image data is acquired that matches the first person feature. According to the data recovery method and device, the problem that data recovery accuracy is low in the data recovery technology is solved, and the accuracy of data recovery is improved.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and deep learning, and specifically relates to a data recovery method, a network training method, a related device and electronic equipment.
Background
With the rapid development of image processing technology, the character image uploaded in the current character testing scene may be a cosmetic, beauty or processed image, which brings many problems to skin detection and artificial intelligence face detection in the medical industry, identity recognition in the financial industry, and the like. For example, in the medical industry, if the character image uploaded by the user is an image shot in the cosmetic state of the user or an image automatically beautified by the camera, the result detected by the system at this time has no skin problem.
The original appearance of the user in the person image can be restored by a data restoration technology, and in the related art, the stretching, the peeling and other treatments are generally performed on the face in the person image to restore the original appearance of the user in the person image.
Disclosure of Invention
The disclosure provides a data recovery method, a network training method, a related device and electronic equipment.
According to a first aspect of the present disclosure, there is provided a data reduction method, including:
acquiring first person image data, wherein the first person image data is person image data subjected to image processing;
performing feature extraction on the first human image data based on a target neural network to obtain first human features, wherein the first human features are the human features of the first human image data before the image processing;
Second person image data is acquired that matches the first person feature.
According to a second aspect of the present disclosure, there is provided a network training method, comprising:
acquiring training sample data, wherein the training sample data comprises third character image data and target image data, and the third character image data is character image data subjected to image processing;
performing feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is the person feature of the third person image data before the image processing;
acquiring fourth person image data matched with the second person feature;
determining first difference information of the fourth person image data and target image data;
and updating parameters of the target neural network based on the first difference information.
According to a third aspect of the present disclosure, there is provided a data restoring apparatus comprising:
the first acquisition module is used for acquiring first person image data, wherein the first person image data is person image data subjected to image processing;
the first feature extraction module is used for carrying out feature extraction on the first human image data based on a target neural network to obtain first human features, wherein the first human features are the human features of the first human image data before the image processing;
And the second acquisition module is used for acquiring second person image data matched with the first person characteristic.
According to a fourth aspect of the present disclosure, there is provided a network training apparatus comprising:
a third acquisition module configured to acquire training sample data including third person image data and target image data, the third person image data being image-processed person image data;
the second feature extraction module is used for carrying out feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is the person feature of the third person image data before the image processing;
a fourth acquisition module for acquiring fourth person image data matching the second person feature;
a first determining module configured to determine first difference information of the fourth person image data and the target image data;
and the updating module is used for updating the parameters of the target neural network based on the first difference information.
According to a fifth aspect of the present disclosure, there is provided an electronic device comprising:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform any one of the methods of the first aspect or to perform any one of the methods of the second aspect.
According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform any one of the methods of the first aspect, or to perform any one of the methods of the second aspect.
According to a seventh aspect of the present disclosure, there is provided a computer program product capable of performing any one of the methods of the first aspect or of performing any one of the methods of the second aspect when the computer program product is run on an electronic device.
According to the data recovery method and device, the problem that the data recovery accuracy is low in the data recovery technology is solved, and the accuracy of data recovery is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is a flow chart of a data reduction method according to a first embodiment of the present application;
FIG. 2 is a schematic diagram of a framework for data reduction of first person image data based on a target neural network;
FIG. 3 is a flow chart of a network training method according to a second embodiment of the present application;
fig. 4 is a schematic structural diagram of a data restoring apparatus according to a third embodiment of the present application;
fig. 5 is a schematic structural diagram of a network training device according to a fourth embodiment of the present application;
fig. 6 illustrates a schematic block diagram of an example electronic device 600 that may be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
First embodiment
As shown in fig. 1, the present application provides a data recovery method, which includes the following steps:
step S101: first person image data, which is image-processed person image data, is acquired.
In this embodiment, the data reduction method relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and deep learning, and can be widely applied to various scenes such as skin detection in medical industry, identity recognition in artificial intelligence side face and financial industry, and the like.
In actual use, the data recovery method of the embodiment of the present application may be executed by the data recovery device of the embodiment of the present application. The data recovery device of the embodiment of the application may be configured in any electronic device to execute the data recovery method of the embodiment of the application. The electronic device may be a server or a terminal, which is not particularly limited herein.
The first person image data may be image data of a target image, the target image may include image data of a person figure, and the image data of the person figure may include at least face image data of the person figure. Of course, the first person image data may further include image data such as style templates of the person image, a frame covering the headwear, and additional elements added to the person image. In the following embodiments, the first face image data will be described by taking face image data as an example.
The first person image data may include person attribute image data, which may be a color RGB image data stream, and person line image data, which may be a line image data stream. The line image data may be image data obtained by performing a wind extraction on an RGB image.
In addition, the first person image data is person image data subjected to image processing, and the image processing may include face peeling processing, face liquefying processing, face whitening processing, wearing headwear frame processing, style conversion processing, and the like. The image processing software may be used to process the person image, or the image processing function of the camera may be used to process the person image during shooting.
The first person image data may be obtained in various ways, for example, a person image captured by a camera, a face of a person image in the person image may be a face in a cosmetic state, and in this scenario, the person image may be regarded as an image subjected to face whitening treatment. The figure image can also be an image subjected to beauty treatment in the shooting process, and the beauty treatment can be face skin grinding treatment, face liquefying treatment, head ornament frame wearing treatment and the like. Correspondingly, the data of the character image can be acquired, and the data of the character image is the first character image data.
For another example, a person image may be downloaded from a network or a person image stored in advance in the acquiring device, where the person image may be an image processed by an image, and correspondingly, data of the person image is acquired, and the data of the person image is the first person image data.
Step S102: and carrying out feature extraction on the first human image data based on a target neural network to obtain first human features, wherein the first human features are the human features of the first human image data before the image processing.
In this step, the target neural network may be a convolutional neural network, and the convolutional neural network may be used to perform feature extraction on the first person image data to obtain a first person feature, where the first person feature may be a person feature of the first person image data before the image processing.
For example, if the first person image data is person image data subjected to face beautifying, the first person feature is a person feature of the first person image data after face beautifying, i.e. after face beautifying. If the first person image data is person image data subjected to style conversion processing, the first person feature is a person feature of the first person image data in a normal shooting style.
The first character feature may include a character attribute feature, which may be a feature extracted based on character attribute image data, and a character line feature, which may be a feature extracted based on character line image data.
Character attribute features may include features such as contrast, brightness, and shape of the face, and character line features may include link features of the character line, such as whether the links are consistent.
The target neural network may include at least two convolution layer modules, which are respectively configured to perform feature extraction on face attribute image data to obtain face attribute features, and perform feature extraction on face line image data to obtain face line features.
In order to obtain the first person feature by using the target neural network, training is required to be performed on the target neural network to train an initial value of the feature, that is, train the person feature of the person image data subjected to image processing before the image processing, so as to restore the original appearance of the face in the person image.
Step S103: second person image data is acquired that matches the first person feature.
The second person image data may be image data of the first person image data before image processing, which is data of the person image at the time of performing the image pixel true value before image processing.
The second person image data matched with the first person feature may be obtained in two ways, and the first person image data matched with the first person feature may be obtained from a database, where person attribute image data corresponding to different person attribute features are stored.
Specifically, the character attribute image data corresponding to the character attribute feature may be acquired from the database, and the second character image data may be constructed based on the character line feature and the character attribute image data.
And secondly, the figure of the character attribute feature can be up-sampled to obtain character attribute image data, and the second character image data can be constructed based on the character line feature and the character attribute image data.
In this embodiment, feature extraction is performed on the first person image data through a target neural network to obtain a first person feature, where the first person feature is a person feature of the first person image data before the image processing; and acquiring second person image data matching the first person feature. Therefore, the character characteristics of the character image data subjected to the image processing before the image processing can be obtained based on the target neural network, the original appearance of the face in the character image can be restored based on the character characteristics, and compared with the restoration processing such as stretching, peeling and the like of the face in the character image, the accuracy of data restoration can be improved. Accordingly, the detection accuracy can be improved by performing skin detection, face detection, identity recognition, or the like on the character image based on the second character image data.
Optionally, the first person image data includes first person attribute image data and first person line image data, and the step S102 specifically includes:
performing feature extraction on the first person attribute image data based on a target neural network to obtain first person attribute features, and performing feature extraction on the first person line image data based on the target neural network to obtain first person line features;
wherein the first person feature comprises the first person attribute feature and a first person line feature.
In this embodiment, the first person image data may include first person attribute image data and first person line image data, and the target neural network may include two convolution layer modules, see fig. 2, and fig. 2 is a schematic diagram of a frame for performing data restoration on the first person image data based on the target neural network, and as shown in fig. 2, the target neural network includes an RGB convolution layer module and a line convolution layer module.
The first person attribute image data may be feature extracted based on the RGB convolution layer module to obtain a first person attribute feature, and the first person line image data may be feature extracted based on the line convolution layer module to obtain a first person line feature.
The first person line image data may be image data obtained by performing wind extraction on the first person attribute image data.
The first person feature may include a first person attribute feature and a first person line feature, the first person attribute feature may include features such as contrast, brightness, shape, etc. of a face, and the first person line feature may include a link feature of a person line, such as whether the links are consistent.
In this embodiment, the first person attribute feature and the first person line feature are extracted by the two convolution layer modules based on the target neural network, so that the line connection position in the first person line data can be identified when the line connection position is inconsistent, and the line connection position is restored, so that the accuracy of data restoration can be further improved.
Optionally, the feature extraction is performed on the first person attribute image data based on the target neural network to obtain a first person attribute feature, and the feature extraction is performed on the first person line image data based on the target neural network to obtain a first person line feature, which includes:
performing feature extraction on the first person attribute image data based on a target neural network to obtain a first feature; extracting features of the first human line image data based on the target neural network to obtain second features;
Fusing the first feature and the second feature to obtain a target feature;
and classifying the target features to obtain first person attribute features and first person line features.
In this embodiment, as shown in fig. 2, the target neural network further includes a candidate network (Region Proposal Network, RPN) layer and a Rol pooling layer, where the dimensions of the first feature may include a plurality of features that may respectively represent dimensions such as contrast, brightness, shape, and the like of the face, the first feature may be processed by the RPN layer and then input to the Rol pooling layer, and the second feature may also be input to the Rol pooling layer, where the Rol pooling layer may fuse the first feature and the second feature and classify the features to obtain the first person attribute feature and the first person line feature.
Thereafter, as shown in fig. 2, person attribute image data corresponding to the first person attribute feature may be acquired, and the second person image data may be constructed based on the first person line feature and the person attribute image data.
In this embodiment, the first feature and the second feature are fused to obtain the target feature, and then the target feature is classified to obtain the first person attribute feature and the first person line feature, so that the feature extraction accuracy can be improved.
Second embodiment
As shown in fig. 3, the present application provides a network training method, which includes the following steps:
step S301: training sample data is acquired, the training sample data including third person image data and target image data, the third person image data being image-processed person image data.
Step S302: performing feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is the person feature of the third person image data before the image processing;
step S303: acquiring fourth person image data matched with the second person feature;
step S304: determining first difference information of the fourth person image data and target image data;
step S305: and updating parameters of the target neural network based on the first difference information.
The object of the present embodiment is to train a target neural network to train initial values of features, that is, train features of a person before the image processing of image-processed person image data, so as to restore the original appearance of a face in a person image.
The training sample data may include a plurality of third person image data and target image data corresponding to the third person image data, and the target image data may be image data of the third person image data before image processing. For example, for the character image of the character image a, it may include an original character image and a character image in which the original character image has undergone image processing, the original character image may not undergo any image processing, the target image data may be data of the original character image, and the third character image data may be data of the character image in which the original character image has undergone image processing.
And extracting the characteristics of the third character image data based on a target neural network to obtain a second character characteristic, wherein the second character characteristic can be the character characteristic of the third character image data before the image processing.
And acquiring fourth person image data matched with the second person feature, wherein the acquisition mode of the fourth person image data is similar to that of the second person image data, and details thereof are not repeated here.
Thereafter, the fourth person image data may be compared with the target image data to determine first difference information of the fourth person image data and the target image data, wherein the first difference information of the fourth person image data and the target image data may be determined using an existing or new distance calculation method.
Finally, the parameters of the target neural network may be updated based on the first difference information until the first difference information is minimized, at which time the target neural network may be trained.
In this embodiment, the initial value of the feature is trained by training the target neural network, that is, the feature of the character before the image processing of the image-processed character image data is trained, so as to restore the original appearance of the face in the character image.
Optionally, the training sample data further includes a person labeling feature of the target image data, and before the step S305, the method further includes:
determining second difference information of the second character feature and the character labeling feature;
the step S305 specifically includes:
and updating parameters of the target neural network based on the first difference information and the second difference information.
In this embodiment, the training sample data may further include a person labeling feature of the target image data, and when the training sample data is prepared, an existing or new feature extraction network may be used to perform feature extraction on the target image data to obtain the person labeling feature of the target image data, where the person labeling feature characterizes the person feature in the target image data.
In order to improve the accuracy and speed of the target neural network training, existing or new distance calculation methods may be used to determine second difference information of the second character feature and the character annotation feature.
Thereafter, parameters of the target neural network may be updated based on the first and second difference information. Specifically, a sum of the first difference information and the second difference information may be calculated, and based on the sum, parameters of the target neural network are updated until the sum reaches a minimum, at which time the training of the target neural network is completed.
In this embodiment, the accuracy and speed of training the target neural network may be improved by determining the second difference information of the second character feature and the character labeling feature, and updating the parameters of the target neural network based on the first difference information and the second difference information.
Optionally, the third person image data includes second person attribute image data and second person line image data, and the step S302 specifically includes:
performing feature extraction on the second character attribute image data based on a target neural network to obtain second character attribute features, and performing feature extraction on the second character line image data based on the target neural network to obtain second character line features;
wherein the second persona feature includes the second persona attribute feature and a second persona line feature.
In this embodiment, the target neural network may include an RGB convolution layer module and a line convolution layer module.
The feature extraction may be performed on the second personage attribute image data based on the RGB convolution layer module to obtain a second personage attribute feature, and the feature extraction may be performed on the second personage line image data based on the line convolution layer module to obtain a second personage line feature.
The second character line image data may be image data obtained by performing a wind extraction on the second character attribute image data.
The second character feature may include a second character attribute feature and a second character line feature, the second character attribute feature may include features such as contrast, brightness, shape, etc. of a face, and the second character line feature may include a link feature of a character line, such as whether the links are consistent.
In this embodiment, the second character attribute feature and the second character line feature are extracted by the two convolution layer modules based on the target neural network, so that the line connection position in the second character line data can be identified when the line connection position is inconsistent, an initial value model of the character line can be trained, and the accuracy of data recovery is further improved.
Optionally, the dimension of the second character attribute feature is N, the character labeling feature includes a character attribute labeling feature and a character line labeling feature, the dimension of the character attribute labeling feature is M, M and N are positive integers greater than 1, and N is equal to M; the determining the second difference information of the second character feature and the character annotation feature includes:
Determining a first loss value of the feature of each dimension in the second character attribute feature and the feature corresponding to the dimension in the character attribute labeling feature aiming at the feature of each dimension in the second character attribute feature;
determining a second loss value of the second character line feature and the character line annotation feature;
wherein the second difference information includes the first loss value and a second loss value.
In this embodiment, for each dimension feature in the second person attribute feature, a different loss function may be adopted based on the data distribution feature of each dimension feature, to determine a first loss value of the dimension feature in the second person attribute feature and a feature corresponding to the dimension in the person attribute labeling feature.
Specifically, for the additional elements added to the character image and the feature of the scene dimension, the following formula (1) may be adopted to determine the first loss value of the feature of the dimension in the second character attribute feature and the feature corresponding to the dimension in the character attribute labeling feature.
Wherein, in the above formula (1),representing a first loss value,/->May represent the feature of the second character attribute feature for the additional element added to the character image and the dimension of the scene, Φ l (x) Can represent the additional elements added to the character image and the feature of the scene dimension in the character attribute labeling feature, and +.>And x represents data of the second character attribute feature and the character attribute annotation feature for the additional element added on the character image and the feature of the scene dimension, respectively, and l represents a low-dimensional loss function.
For the feature of the dimension of the style template, the following formula (2) may be adopted, and a first loss value of the feature of the dimension in the second character attribute feature and a first loss value of the feature corresponding to the dimension in the character attribute labeling feature are determined.
Wherein, in the above formula (2),represents a first loss value, d c Representing a vector function>Representing characteristics of the second character attribute characteristics aiming at the dimension of the style template, ψ i (Φ l (y NN (i) ) represent features for the dimensions of the wind template in the character attribute labeling features, M being the number. />And Y represents the data of the feature aiming at the dimension of the style template in the second character attribute feature and the character attribute labeling feature respectively, and l represents the low-dimensional loss function.
For the feature of the face brightness dimension, the following formula (3) may be adopted, and a first loss value of the feature of the dimension in the second person attribute feature and a first loss value of the feature corresponding to the dimension in the person attribute labeling feature are determined.
Wherein, in the above formula (3),representing a first loss value,/->Can represent the feature of the second person attribute feature for the face brightness dimension, Γ (x) can represent the feature of the person attribute annotation feature for the face brightness dimension, and +.>And x represents data of the feature for the face brightness dimension in the second person attribute feature and the person attribute annotation feature, respectively.
Further, a second loss value for the second character line feature and the character line annotation feature may be determined based on an existing or new loss function. The second difference information includes a first loss value and a second loss value, and the second difference information may specifically be a sum of the first loss value and the second loss value obtained by the calculation.
In this embodiment, by determining, based on the data distribution characteristics of the features of each dimension, the first loss value of the feature of the dimension in the second person attribute feature and the first loss value of the feature corresponding to the dimension in the person attribute labeling feature by using different loss functions, so as to obtain second difference information, so that the accuracy and speed of training the target neural network can be further improved.
Third embodiment
As shown in fig. 4, the present application provides a data recovery apparatus 400, including:
A first acquiring module 401, configured to acquire first person image data, where the first person image data is person image data subjected to image processing;
a first feature extraction module 402, configured to perform feature extraction on the first person image data based on a target neural network, to obtain a first person feature, where the first person feature is a person feature of the first person image data before the image processing;
a second obtaining module 403, configured to obtain second person image data that matches the first person feature.
Optionally, the first person image data includes first person attribute image data and first person line image data, and the first feature extraction module 402 includes:
the feature extraction unit is used for carrying out feature extraction on the first person attribute image data based on a target neural network to obtain first person attribute features, and carrying out feature extraction on the first person line image data based on the target neural network to obtain first person line features;
wherein the first person feature comprises the first person attribute feature and a first person line feature.
Optionally, the feature extraction unit is specifically configured to perform feature extraction on the first person attribute image data based on a target neural network to obtain a first feature; extracting features of the first human line image data based on the target neural network to obtain second features; fusing the first feature and the second feature to obtain a target feature; and classifying the target features to obtain first person attribute features and first person line features.
The data reduction device 400 provided in the present application can implement each process implemented by the embodiment of the data reduction method, and can achieve the same beneficial effects, so that repetition is avoided, and no further description is provided herein.
Fourth embodiment
As shown in fig. 5, the present application provides a network training apparatus 500, including:
a third obtaining module 501, configured to obtain training sample data, where the training sample data includes third person image data and target image data, and the third person image data is image-processed person image data;
a second feature extraction module 502, configured to perform feature extraction on the third person image data based on a target neural network, to obtain a second person feature, where the second person feature is a person feature of the third person image data before the image processing;
a fourth obtaining module 503, configured to obtain fourth person image data that matches the second person feature;
a first determining module 504 configured to determine first difference information between the fourth person image data and the target image data;
an updating module 505, configured to update parameters of the target neural network based on the first difference information.
Optionally, the training sample data further includes a person annotation feature of the target image data, and the apparatus further includes:
the second determining module is used for determining second difference information of the second character features and the character labeling features;
the updating module is specifically configured to update parameters of the target neural network based on the first difference information and the second difference information.
Optionally, the third person image data includes second person attribute image data and second person line image data, and the second feature extraction module 502 is specifically configured to perform feature extraction on the second person attribute image data based on a target neural network to obtain a second person attribute feature, and perform feature extraction on the second person line image data based on the target neural network to obtain a second person line feature; wherein the second persona feature includes the second persona attribute feature and a second persona line feature.
Optionally, the dimension of the second person attribute feature is N, the person attribute feature includes a person attribute feature and a person line feature, the dimension of the person attribute feature is M, M and N are positive integers greater than 1, and N is equal to M; the second determining module is specifically configured to determine, for each dimension feature in the second person attribute feature, a first loss value of the dimension feature in the second person attribute feature and a first loss value of a feature corresponding to the dimension in the person attribute labeling feature; determining a second loss value of the second character line feature and the character line annotation feature; wherein the second difference information includes the first loss value and a second loss value.
The network training device 500 provided in the present application can implement each process implemented by the embodiment of the network training method described above, and can achieve the same beneficial effects, so as to avoid repetition, and is not described herein again.
According to embodiments of the present application, there is also provided an electronic device, a readable storage medium and a computer program product.
Fig. 6 illustrates a schematic block diagram of an example electronic device 600 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, ROM602, and RAM603 are connected to each other by a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Various components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, mouse, etc.; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a data reduction method or a network training method. For example, in some embodiments, the data reduction method or the network training method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM602 and/or the communication unit 609. When the computer program is loaded into the RAM603 and executed by the computing unit 601, one or more steps of the data reduction method described above, or one or more steps of the network training method described above, may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the data reduction method or the network training method by any other suitable method (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out the methods of the present disclosure can be written in any combination of one or more editing languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (12)
1. A data recovery method, comprising:
acquiring first person image data, wherein the first person image data is person image data subjected to image processing;
performing feature extraction on the first human image data based on a target neural network to obtain first human features, wherein the first human features are the human features of the first human image data before the image processing;
acquiring second person image data matched with the first person feature;
The first person image data includes first person attribute image data and first person line image data, the feature extraction is performed on the first person image data based on a target neural network to obtain a first person feature, including:
performing feature extraction on the first person attribute image data based on a target neural network to obtain first person attribute features, and performing feature extraction on the first person line image data based on the target neural network to obtain first person line features;
wherein the first person feature comprises the first person attribute feature and a first person line feature.
2. The method of claim 1, wherein the feature extracting the first person attribute image data based on the target neural network to obtain a first person attribute feature, and the feature extracting the first person line image data based on the target neural network to obtain a first person line feature, includes:
performing feature extraction on the first person attribute image data based on a target neural network to obtain a first feature; extracting features of the first human line image data based on the target neural network to obtain second features;
Fusing the first feature and the second feature to obtain a target feature;
and classifying the target features to obtain first person attribute features and first person line features.
3. A method of network training, comprising:
acquiring training sample data, wherein the training sample data comprises third character image data and target image data, and the third character image data is character image data subjected to image processing;
performing feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is the person feature of the third person image data before the image processing;
acquiring fourth person image data matched with the second person feature;
determining first difference information of the fourth person image data and target image data;
updating parameters of the target neural network based on the first difference information;
the third person image data includes second person attribute image data and second person line image data, and the feature extraction is performed on the third person image data based on the target neural network to obtain a second person feature, including:
Performing feature extraction on the second character attribute image data based on a target neural network to obtain second character attribute features, and performing feature extraction on the second character line image data based on the target neural network to obtain second character line features;
wherein the second persona feature includes the second persona attribute feature and a second persona line feature.
4. The method of claim 3, wherein the training sample data further includes a person annotation feature of the target image data, the method further comprising, prior to updating parameters of the target neural network based on the first difference information:
determining second difference information of the second character feature and the character labeling feature;
the updating the parameters of the target neural network based on the first difference information includes:
and updating parameters of the target neural network based on the first difference information and the second difference information.
5. The method of claim 4, wherein the second persona attribute feature has a dimension N, the persona annotation feature comprises a persona attribute annotation feature and a persona line annotation feature, the persona attribute annotation feature has dimensions M, M and N are positive integers greater than 1, and N is equal to M; the determining the second difference information of the second character feature and the character annotation feature includes:
Determining a first loss value of the feature of each dimension in the second character attribute feature and the feature corresponding to the dimension in the character attribute labeling feature aiming at the feature of each dimension in the second character attribute feature;
determining a second loss value of the second character line feature and the character line annotation feature;
wherein the second difference information includes the first loss value and a second loss value.
6. A data recovery apparatus comprising:
the first acquisition module is used for acquiring first person image data, wherein the first person image data is person image data subjected to image processing;
the first feature extraction module is used for carrying out feature extraction on the first human image data based on a target neural network to obtain first human features, wherein the first human features are the human features of the first human image data before the image processing;
a second acquisition module for acquiring second person image data matched with the first person feature;
the first person image data includes first person attribute image data and first person line image data, and the first feature extraction module includes:
The feature extraction unit is used for carrying out feature extraction on the first person attribute image data based on a target neural network to obtain first person attribute features, and carrying out feature extraction on the first person line image data based on the target neural network to obtain first person line features;
wherein the first person feature comprises the first person attribute feature and a first person line feature.
7. The device according to claim 6, wherein the feature extraction unit is specifically configured to perform feature extraction on the first person attribute image data based on a target neural network, so as to obtain a first feature; extracting features of the first human line image data based on the target neural network to obtain second features; fusing the first feature and the second feature to obtain a target feature; and classifying the target features to obtain first person attribute features and first person line features.
8. A network training device, comprising:
a third acquisition module configured to acquire training sample data including third person image data and target image data, the third person image data being image-processed person image data;
The second feature extraction module is used for carrying out feature extraction on the third person image data based on a target neural network to obtain a second person feature, wherein the second person feature is the person feature of the third person image data before the image processing;
a fourth acquisition module for acquiring fourth person image data matching the second person feature;
a first determining module configured to determine first difference information of the fourth person image data and the target image data;
an updating module, configured to update parameters of the target neural network based on the first difference information;
the third character image data comprises second character attribute image data and second character line image data, and the second feature extraction module is specifically configured to perform feature extraction on the second character attribute image data based on a target neural network to obtain second character attribute features, and perform feature extraction on the second character line image data based on the target neural network to obtain second character line features; wherein the second persona feature includes the second persona attribute feature and a second persona line feature.
9. The apparatus of claim 8, wherein the training sample data further comprises a person annotation feature of the target image data, the apparatus further comprising:
the second determining module is used for determining second difference information of the second character features and the character labeling features;
the updating module is specifically configured to update parameters of the target neural network based on the first difference information and the second difference information.
10. The apparatus of claim 9, wherein the second persona attribute feature has a dimension N, the persona annotation feature comprises a persona attribute annotation feature and a persona line annotation feature, the persona attribute annotation feature has dimensions M, M and N are positive integers greater than 1, and N is equal to M; the second determining module is specifically configured to determine, for each dimension feature in the second person attribute feature, a first loss value of the dimension feature in the second person attribute feature and a first loss value of a feature corresponding to the dimension in the person attribute labeling feature; determining a second loss value of the second character line feature and the character line annotation feature; wherein the second difference information includes the first loss value and a second loss value.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-2 or to perform the method of any one of claims 3-5.
12. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-2 or to perform the method of any one of claims 3-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110127932.2A CN112819715B (en) | 2021-01-29 | 2021-01-29 | Data recovery method, network training method, related device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110127932.2A CN112819715B (en) | 2021-01-29 | 2021-01-29 | Data recovery method, network training method, related device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112819715A CN112819715A (en) | 2021-05-18 |
| CN112819715B true CN112819715B (en) | 2024-04-05 |
Family
ID=75860310
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110127932.2A Active CN112819715B (en) | 2021-01-29 | 2021-01-29 | Data recovery method, network training method, related device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112819715B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596830A (en) * | 2018-04-28 | 2018-09-28 | 国信优易数据有限公司 | A kind of image Style Transfer model training method and image Style Transfer method |
| CN111275784A (en) * | 2020-01-20 | 2020-06-12 | 北京百度网讯科技有限公司 | Method and device for generating image |
| AU2020101122A4 (en) * | 2020-06-25 | 2020-07-30 | A, Tamilselvi MS | Cnn model based approach towards breast cancer features detection from ultra sound images |
| CN111753770A (en) * | 2020-06-29 | 2020-10-09 | 北京百度网讯科技有限公司 | Person attribute identification method and device, electronic equipment and storage medium |
| CN112232425A (en) * | 2020-10-21 | 2021-01-15 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
-
2021
- 2021-01-29 CN CN202110127932.2A patent/CN112819715B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596830A (en) * | 2018-04-28 | 2018-09-28 | 国信优易数据有限公司 | A kind of image Style Transfer model training method and image Style Transfer method |
| CN111275784A (en) * | 2020-01-20 | 2020-06-12 | 北京百度网讯科技有限公司 | Method and device for generating image |
| AU2020101122A4 (en) * | 2020-06-25 | 2020-07-30 | A, Tamilselvi MS | Cnn model based approach towards breast cancer features detection from ultra sound images |
| CN111753770A (en) * | 2020-06-29 | 2020-10-09 | 北京百度网讯科技有限公司 | Person attribute identification method and device, electronic equipment and storage medium |
| CN112232425A (en) * | 2020-10-21 | 2021-01-15 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, storage medium and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度神经网络的图像超分辨率重构算法研究;杨瞻远;《中国优秀硕士学位论文全文数据库》;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112819715A (en) | 2021-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113343826B (en) | Training method of human face living body detection model, human face living body detection method and human face living body detection device | |
| CN113435408A (en) | Face living body detection method and device, electronic equipment and storage medium | |
| CN113221771A (en) | Living body face recognition method, living body face recognition device, living body face recognition equipment, storage medium and program product | |
| CN113177449B (en) | Face recognition method, device, computer equipment and storage medium | |
| CN113221767B (en) | Method for training living body face recognition model and recognizing living body face and related device | |
| CN114723888B (en) | Three-dimensional hair model generation method, device, equipment, storage medium and product | |
| WO2022227765A1 (en) | Method for generating image inpainting model, and device, medium and program product | |
| CN112802037A (en) | Portrait extraction method, device, electronic equipment and storage medium | |
| CN115393488B (en) | Method and device for driving virtual character expression, electronic equipment and storage medium | |
| CN114140320B (en) | Image migration method and training method and device of image migration model | |
| CN113962845B (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| CN113269719A (en) | Model training method, image processing method, device, equipment and storage medium | |
| CN114120413A (en) | Model training method, image synthesis method, device, equipment and program product | |
| CN113379750A (en) | Semi-supervised learning method of semantic segmentation model, related device and product | |
| CN113657248A (en) | Training method and device for face recognition model and computer program product | |
| CN112819715B (en) | Data recovery method, network training method, related device and electronic equipment | |
| CN113177466A (en) | Identity recognition method and device based on face image, electronic equipment and medium | |
| CN117152352A (en) | Image processing method, deep learning model training method and device | |
| CN114926322B (en) | Image generation method, device, electronic equipment and storage medium | |
| CN114445898B (en) | Face living body detection method, device, equipment, storage medium and program product | |
| CN114387651B (en) | Face recognition method, device, equipment and storage medium | |
| CN114494782B (en) | Image processing method, model training method, related device and electronic equipment | |
| CN116524475A (en) | Method and device for generating recommended dressing, vehicle, electronic equipment and storage medium | |
| CN112927319B (en) | Model training method, image processing method, device, equipment and storage medium | |
| CN116052288A (en) | Living body detection model training method, living body detection device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |