CN110032927B - Face recognition method - Google Patents
Face recognition method Download PDFInfo
- Publication number
- CN110032927B CN110032927B CN201910146378.5A CN201910146378A CN110032927B CN 110032927 B CN110032927 B CN 110032927B CN 201910146378 A CN201910146378 A CN 201910146378A CN 110032927 B CN110032927 B CN 110032927B
- Authority
- CN
- China
- Prior art keywords
- image
- model
- dimensional
- texture
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 50
- 238000005286 illumination Methods 0.000 claims abstract description 47
- 238000010606 normalization Methods 0.000 claims abstract description 29
- 230000009466 transformation Effects 0.000 claims abstract description 19
- 238000013507 mapping Methods 0.000 claims description 12
- 239000013598 vector Substances 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000004020 luminiscence type Methods 0.000 claims description 2
- 238000002347 injection Methods 0.000 claims 1
- 239000007924 injection Substances 0.000 claims 1
- 239000000243 solution Substances 0.000 claims 1
- 230000001131 transforming effect Effects 0.000 claims 1
- 230000008859 change Effects 0.000 abstract description 3
- 238000003909 pattern recognition Methods 0.000 abstract description 3
- 239000011159 matrix material Substances 0.000 description 15
- 230000001815 facial effect Effects 0.000 description 14
- 230000006870 function Effects 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 8
- 239000000523 sample Substances 0.000 description 5
- 238000013459 approach Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000011524 similarity measure Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000000137 annealing Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 238000002922 simulated annealing Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a face recognition method, and belongs to the technical field of image processing and pattern recognition. The face recognition method of the invention comprises the following steps: constructing 3-dimensional face model texture data by using the 3-dimensional model and the 2-dimensional face image data; performing projection transformation on the 3-dimensional face model texture data to obtain a 2-dimensional texture image; carrying out parameterization treatment on the 2-dimensional texture image to obtain a UV image; carrying out illumination normalization processing on the UV image; and identifying the UV image subjected to the normalization processing of the contrast degree to obtain a similarity score. The face recognition method of the invention provides a new 3-2-dimensional (3D-2D) recognition framework, the framework is registered by using 3D data, and only 2D data is needed for recognition, and the recognition method has higher recognition rate and more discernability and robustness to condition change.
Description
Technical Field
The invention belongs to the technical field of image processing and pattern recognition, and particularly relates to a face recognition method.
Background
Face Recognition (FR) has been a key topic in computer vision, pattern recognition and machine learning research, which extends the perceptive, behavioral and social principles. In parallel, FR technology has been continuously evolving in terms of sensors, algorithms, databases, and evaluation frameworks, etc., causing its growing interest to be driven in part by difficult tasks and challenges (i.e., complex, intra-class object recognition problems), and in part by various applications involving identity management. Challenges of current research include: (i) separating the intrinsic from the extrinsic appearance changes; (ii) develop a discriminant representation and a similarity measure; (iii) finding performance invariants across heterogeneous data and conditions. Application-oriented, faces are becoming a powerful biometric technology, a high-level semantic for content-based indexing and retrieval, and a natural and rich communication model for human-computer interaction. Existing frameworks for face recognition vary according to some methods (e.g., data driven/model/perception based) or facial data domain (e.g., image/point cloud/depth map).
The existing 2D-2D face recognition technology has defects in terms of accuracy, recognition passing rate, recognition speed and the like, and is affected by objective conditions such as illumination, angle, definition and the like, so that a judgment result is affected, and methods for improving the accuracy of comparison are provided at present, but a plurality of limitations exist in the methods in the prior art.
Disclosure of Invention
The invention provides a face recognition method which aims to overcome the defects of the existing 2D-2D face recognition technology in terms of accuracy, recognition passing rate, recognition speed and the like.
In order to solve the technical problems, the invention provides a face recognition method, which comprises the following steps:
constructing 3-dimensional face model texture data by using the 3-dimensional model and the 2-dimensional face image data;
performing projection transformation on the 3-dimensional face model texture data to obtain a 2-dimensional texture image;
Carrying out parameterization treatment on the 2-dimensional texture image to obtain a UV image;
carrying out illumination normalization processing on the UV image;
And identifying the UV image subjected to the normalization processing of the contrast degree to obtain a similarity score.
According to an embodiment of the present invention, the step of identifying the UV image after the normalization process to obtain a similarity score further includes:
normalizing the similarity score.
According to another embodiment of the present invention, the 3-dimensional model is an AFM build model, and the surface parameters of the AFM build model are an internal reflection function:
Wherein the method comprises the steps of Representing a 3-dimensional vector R 3, M representing an image, and U representing a 2-dimensional image network.
According to another embodiment of the present invention, the step of performing projective transformation on the 3-dimensional face model texture data to obtain a 2-dimensional texture image includes:
Performing linear mapping on the 3-dimensional face model texture data under a perspective projection model to obtain a 2-dimensional texture image;
and carrying out re-projection error minimization processing on the 2-dimensional texture image.
According to another embodiment of the present invention, the step of parameterizing the 2-dimensional texture image to obtain a UV image includes:
performing model parameterization on the 2-dimensional texture image to obtain a UV image;
And removing the pseudo value points in the UV image.
According to another embodiment of the present invention, the step of performing illumination normalization processing on the UV image includes:
determining a skin reflection model of the UV image;
Constructing a texture illuminance model of the UV image according to the skin reflection model;
And carrying out illumination normalization processing on the UV image by using the texture illumination model.
On the other hand, the invention also provides a face recognition device, which comprises:
The 3-dimensional model construction module is used for constructing 3-dimensional face model texture data by utilizing the 3-dimensional model and the 2-dimensional face image data;
the projection conversion module is used for carrying out projection conversion on the 3-dimensional face model texture data to obtain a 2-dimensional texture image;
the parameterization processing module is used for carrying out parameterization processing on the 2-dimensional texture image to obtain a UV image;
the illumination processing module is used for carrying out illumination normalization processing on the UV image;
And the identification module is used for identifying the UV image subjected to the illumination normalization processing to obtain a similarity score.
According to an embodiment of the present invention, the projective transformation module includes:
the perspective projection unit is used for carrying out linear mapping on the 3-dimensional face model texture data under the perspective projection model to obtain a 2-dimensional texture image;
And the re-projection error minimization unit is used for carrying out re-projection error minimization processing on the 2-dimensional texture image.
According to another embodiment of the present invention, the parameterization module includes:
the parameterization processing unit is used for carrying out model parameterization processing on the 2-dimensional texture image to obtain a UV image;
and the removing unit is used for removing the pseudo value points in the UV image.
According to another embodiment of the present invention, the illumination processing module includes:
A skin reflection model analysis unit for analyzing and determining a skin reflection model of the UV image;
A texture illuminance model construction unit for constructing a texture illuminance model of the UV image from the skin reflection model;
and the illumination normalization unit is used for carrying out illumination normalization processing on the UV image by utilizing the texture illumination model.
The invention has the beneficial effects that:
The face recognition method of the invention provides a new 3-2-dimensional (3D-2D) recognition framework, the framework is registered by using 3D data, and only 2D data is needed for recognition, and the recognition method has higher recognition rate and more discernability and robustness to condition change.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.
FIG. 1 is a flow chart of one embodiment of a face recognition method of the present invention;
FIG. 2 is a flow chart of one embodiment of step 200 of a face recognition method of the present invention;
FIG. 3 is a flow chart of one embodiment of a step 300 of a face recognition method of the present invention;
FIG. 4 is a flow chart of one embodiment of step 400 of a face recognition method of the present invention;
Fig. 5 is a schematic structural view of an embodiment of a face recognition device of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, an embodiment of the present invention provides a face recognition method, including:
step 100: constructing 3-dimensional face model texture data by using the 3-dimensional model and the 2-dimensional face image data;
The 3-dimensional face model texture data is 3-dimensional control key face model data corresponding to a 2-dimensional face image obtained by using a 3-dimensional model.
Step 200: performing projection transformation on the 3-dimensional face model texture data to obtain a 2-dimensional texture image;
The 2-dimensional texture image here is a two-dimensional image without color information.
Step 300: carrying out parameterization treatment on the 2-dimensional texture image to obtain a UV image;
The UV image herein is a two-dimensional image containing color and other image information.
Step 400: carrying out illumination normalization processing on the UV image;
step 500: and identifying the UV image subjected to the normalization processing of the contrast degree to obtain a similarity score.
As an illustration, step 500 of the embodiment of the present invention shown in fig. 1 further comprises:
step 600: normalizing the similarity score.
Optionally, in step 100 of this embodiment, the 3-dimensional model is an AFM build model, and the surface parameter of the AFM build model is an internal reflection function:
Wherein the method comprises the steps of Representing a 3-dimensional vector R 3, M representing an image, and U representing a 2-dimensional image network.
The facial data pattern and recording framework in step 100 of this embodiment is provided by an AFM build model. The original polygon or surface M is mapped to a regularly sampled fixed-size 2-dimensional (2D) grid U, called a geometric image. Any model suitable for 3-dimensional (3D) data will inherit this predefined parameterization and the same geometric image grid. By associating texture values with model points, texture images can be constructed on the UV coordinate space, providing a universal frame of reference for local facial features. Texture images in UV space are aligned by construction (due to geometric images) and recorded (due to regularity of the grid) and can be compared using local similarity measures. For a 3D model with recorded 2D texture data, image values for each UV coordinate are obtained from the texture element corresponding to the closest 3D point, and automatic 3D-2D registration is performed on the gallery and the probe texture using the same method.
As another illustration, as shown in fig. 2, step 200 of an embodiment of the present invention includes:
step 201: performing linear mapping on the 3-dimensional face model texture data under a perspective projection model to obtain a 2-dimensional texture image;
step 202: and carrying out re-projection error minimization processing on the 2-dimensional texture image.
The specific step 200 first achieves registration of the fitted 3D model to the image plane by estimating a perspective projection transformation, which involves a 3D rigid transformation (rotation and translation) and a 2D projection. In the most general case, both modalities may exhibit a contoured face. During registration, known camera parameters and simultaneously acquired 3D and 2D data may be used for transformation. When the 2D probe needs to register with the 3D gallery model, recognition may involve uncontrolled face rotations and different identities, correcting the relative pose differences between modalities by explicitly estimating a given 3D-2D projection.
Prior to model fitting, the 3D pose is considered by rigid alignment of the raw data with the AFM. The similarity transformation a in R 3 is estimated by iterating the nearest point algorithm so that the model points are aligned with the closest 3D surface points. Robustness of extreme poses can be obtained by initializing algorithms based on corresponding prealignment, 3D data mapped to canonical model posesIs a kind of medium. After fitting, the deformation model vertices may be mapped to the original space:
wherein, Is a column arrangement matrix of homogeneous coordinates midpoints, a is a 4 x 4 matrix. Such bijective mapping between the original 3D data and the model pose may use points in the 3D surfaceTo establish and 2D image pointsCorresponding relation of (3). Through point correspondence setPerspective projection is estimated to obtain the relative pose of the 3D and 2D surfaces.
Perspective projection estimation: the basic assumption is a 2D facial image generated by observing some of the facial meshes. Points from the 3D surface are mapped linearly on the 2D image plane, ignoring nonlinear camera lens distortion by perspective projection consisting of extrinsic (or pose) components and intrinsic (or camera) components. The linear mapping P ε R 3×4 from 3D to 2D is given under the full perspective projection model:
Where K is a3 x 3 matrix of internal parameters and E is a3 x 4 matrix of external parameters. Can be relative to translation vector The rotation matrix R and the scale s are further written, which is ambiguous for perspective models. Point(s)Mapping to pointsAll in homogeneous coordinates. Solving for the P-quantity to estimate a set of 3D points by linear transformationEntries of the 3X 4 projection matrix mapped to 2D image positions X:
Wherein, X= (X 1,...,xl) and Is a column arrangement matrix of points in homogeneous coordinates. The system involves 11 degrees of freedom, since for any scalar s, any matrix sP will result in ambiguity for the same projection set on the image. Establishing a set of 3D-2D point correspondence sets by 2D and 3D labeled facial marker localizationUsing a point reference set of l=9, this results in an approximation of an overdetermined system within some small error, equal in (4).
Then, landmark re-projection error minimization is carried out: the estimation of the projective transformation (4) is formulated to minimize the 3D-2D landmark re-projection error, which is the error of approximating a 2D point by projection of the 3D point of P. P is estimated by solving a least squares approximation problem at all reference points using the square difference:
In the formula, the objective function is parameterized with respect to the projection matrix P (i.e., the set of variables { (P) j } j=1,..12), rather than the individual camera and pose parameters. An iterative optimization procedure is used to solve the minimization problem, in this case a Levenberg-Marquadt (LM) algorithm is used, initialized with a direct linear transformation algorithm, which gives an efficient and close approximation of the exact point correspondence P. P gives the exact point correspondence. For invariance to the similarity transformation, the point set is normalized with respect to the position (origin) and the average distance (scale) from its centroid.
The minimization P is a coupled estimate of pose and camera parameters corresponding to an unknown base setting matching arbitrary 3D and 2D data. The estimation can be further decomposed into an internal parameter matrix K and an external matrix E, which represents the relative orientation of the camera frame (equation 3). However, in order to obtain a texture image using a 3D model, the projection matrix P is sufficient, since the decomposition in the respective parameter matrix is usually not unique.
The accuracy of the projection depends on the number of 3D and 2D landmarks, positioning and corresponding accuracy. In order to register the 3D model to an image of the same object, the algorithm can handle various head pose changes under unknown imaging conditions. Iterative refinement of the initial approximation solution provides robustness for the less corresponding estimates from; when five or four landmark points are used, a visually consistent texture is generated for the near frontal pose. The 3D model points and mesh are projected onto the 2D image by visualization.
As another illustration, as shown in fig. 3, step 300 of an embodiment of the present invention includes:
step 301: performing model parameterization on the 2-dimensional texture image to obtain a UV image;
step 302: and removing the pseudo value points in the UV image.
Specifically, step 300 in this embodiment is: given a 3D model and 2D image pair (M, I) and an estimated projective transformation P, a texture image T (U), U ε U is derived from the image value I (X), X ε X and registered model verticesGenerating or lifting. The process is similar to extracting UV images from texture images co-registered to the model: by UV parameterization, texture values in the geometric image are assigned by image values at the location of the re-projected model vertices.
The projection of the model points is obtained from a cascade of two transformations, the three-dimensional transformation in the equation mapping the deformation model to the 3D data coordinate system and minimizing the perspective projection P:
Wherein X and The matrix of model vertices in image and 3D space, respectively. At a point corresponding to a certain modelThe value of T is obtained using the value of image I in X e X:
wherein, h: model parameterization in the M→U equation. Specifying that u does not correspond to some by triangulating interpolation from projection models T of (c).
Due to the 3D pose and the 2D pose, two types of self-occlusion can affect the texture generated: model surface occlusion (invisible 3D triangle) along the camera viewpoint direction and region occlusion (invisible face region) on the 2D image plane. In occluded facial regions, the re-projected mesh is not area-inplane, and mapping of surface triangulation to image areas will result in overlapping 2D triangles. The same image value will be assigned to the texture point that corresponds to the visible and occlusion triangle area.
By calculating the visibility map, false value points due to occlusion are excluded from subsequent processing. To determine 2 the visibility of D is that, in the case of a visual, D can see the nature of the material used for the coating, tracking on an image plane depth value of the projection point. The value from the target image at x is assigned by equation 1. Corresponds to a 3D point having a minimum depth value (closest to the camera point)Is a point u of (2). The visibility graph is an index function:
Wherein x i,xj is represented by the equation Given the sum of the re-projectionsIs the depth coordinate unit vector in R 3. In contrast, the invisible plot is an indication function of point u, competing for the same image value as other image values of smaller depth. Additional 3D visibility map can be estimated by excluding 3D points with transformed surface normals in UV, whereinOpposite to the direction of the viewpoint: if it isThenWhere E is the extrinsic matrix of the equation 3D-2D projection calculation.
As another illustration, as shown in fig. 4, step 400 of an embodiment of the present invention includes:
Step 401: determining a skin reflection model of the UV image;
Step 402: constructing a texture illuminance model of the UV image according to the skin reflection model;
Step 403: and carrying out illumination normalization processing on the UV image by using the texture illumination model.
Specifically, step 400 in this embodiment is: under a normalized pair of texture illumination conditions, the illumination transmission is estimated by applying an optimized, non-explicit albedo. A secondary illumination algorithm is proposed to operate on textures in the UV space of the AFM, minimizing their element-by-element illumination differences. Analysis was performed using A Skin Reflectance Model (ASRM) in the form of a hybrid bi-directional reflectance distribution function (BRDF). This approach has fewer constraints and limitations than existing approaches: (i) do not assume that the source face image (i.e., viewpoint, registration) is on, because the UV representation inherits through 3D-2D registration; (ii) it is not assumed that the light is made or specularly reflected in the presence of a light source (number, distance or direction); (iii) rely on minimal input and no light calibration information (i.e., a pair probegallery of texture images and 3D fitting model); (iv) From different examples, local light variation is necessary; (v) it involves an optimization step that minimizes joint errors, rather than solving two separate inverse problems to estimate the albedo of each texture independently; (vi) it is bi-directional (i.e. roles of source and target textures may be interchanged); (vii) The method adopts the same 3D model for normalizing gesture and texture lifting and for obtaining the estimation of the surface normal.
Texture lighting model: texture image T (u) in UV space,Applied to unknown face albedo B (u):
T(u)＝Ls(u)+(Ld(u)+La(u))B(u) (9)
Where L d(u),Ls (u) is the diffuse and specular component (assuming white specular highlights) and L a (u) is the ambient lighting component. A pair of texture images may be normalized in two different ways: the albedo component is estimated (luminescence) or the illumination parameter is delivered (re-illumination). Solving equation (9) requires estimating the lighting assembly because the lighting B (u) = (T (u) -L s(u))/(Ld(u)+La (u)). In this work, it is advocated to use textures for re-illumination without prior estimation of albedo.
Analysis of skin reflectance model: subsurface scattering is ignored using analysis of the diffuse and specular components of the BRDF. Diffuse reflectance L d is a model using the basic lambertian BRDF that assumes an equally bright surface in all directions. For a single light source with intensity L, surface pointsIs proportional to the angle θ (u) between the surface normal and the direction of the incident light: l d (u) =lcos (θ (u)). Specular reflection is explained by Phan-Brdf, which simulates the specular reflection intensity at a surface point by L s(u)＝Lcosη (phi (u)), where phi (u) is the angle between the view vector and the reflected light and eta is a parameter that controls the highlight size. Variations in the specular characteristics of different face regions are accommodated by AFM annotation-based specular mapping.
The model in equation (9) may be written with respect to the parameters of the analytical model. Texture is modeled in an independent channel of RGB or Hue Saturation Intensity (HSI) space and light color is assumed to be white, which is a reasonable approximation of facial images obtained under indoor or controlled conditions. In addition, the multi-point light source is polymerized by the SUMM. Their respective BRFD functions:
Optimizing illumination transfer: the parameters of the illumination model are the plurality of light sources per light source and color channel Parameters of position and reflectivity components, e.gTo minimize the difference in lighting conditions between the two textures, an optimization problem was formulated for the two sets of light (on spheres located around the centroid of the model); one for removing illumination from the source texture and the other for adding illumination of the target texture. The scheme is based on minimizing the approximation error between textures of the same albedo but different lighting conditions:
Where L a,Ld and L s are the reflected components of the source texture T, given in equation 1. The reflected component of the target texture T of L a,Ld and L s; the minimization is defined in terms of the position of the complex vectors lambda and T of the optical parameters. The error function can also be interpreted as the average intensity variation of the face albedo B (u) in the two textures. In practice, the error function is the euclidean norm of the color vector value Eq. On the RGB channels, and when the visibility map V (u), the union of the equations for V' (u) is applied is:
the minimization of equation 11 may be sought by a global optimization method. Simulated annealing and adaptive exponential annealing schemes are used. Consider a plurality of light spots, three for simulating illuminance on a light source. To improve performance in low light conditions, color textures are converted to the HSI color space and RGB in the equation is converted to the equation. Replaced by a weighted average, the intensity weighted twice the hue and saturation. This approach improves the synthetic appearance of dark facial regions and increases the similarity score. The gallery serves as the target texture for the source and probe, both represented by UV under the same facial model.
Another characteristic of the re-illumination is bi-directional in the sense that the re-illumination transfer can occur in any direction between source-target and the roles of probe and gallery texture can be interchanged in the cost function. Visual inspection showed that the redesigned texture appeared natural, facial features were preserved, and no significant artifacts were produced by the relighting process.
In step 500 of this embodiment, a pair-wise comparison of the probe and gallery lifting texture is obtained by global similarity score based on the correlation coefficient of the image gradient direction, which is very insensitive to severe mismatch of the two images. It is particularly suitable for measuring the similarity of changing facial data, not only due to different acquisition conditions, but also due to significant changes in the appearance of the individual.
In addition, in this embodiment, the similarity score may be normalized using a standard value, for example: the standard Z-score normalizes the scores for 1-N, normalizes and scales the N-N normalized metrics multidimensional using the distances extracted from the gallery data.
The face recognition method of the embodiment of the invention provides a novel 3D-2D recognition framework, the framework utilizes 3D data for registration, only needs 2D data for recognition, and can be easily applied to the 2D-3D condition, a 3D-based algorithm shows very high recognition rate, and a 3D model-based face signature has more discrimination and robustness to condition change. In contrast to asymmetric or heterogeneous recognition methods that map features across different modalities, the developed 3D-2D framework (referred to as UR 2D) relies on modality synergy, where 3D models are used for registration, alignment and back-light normalization of 2D images and texture data. UR2D uses 3D shape information to re-illuminate (using surface normal information) and score calculations, as compared to previous methods for 3D-2D registration and fitting. In contrast to existing multi-modal 2d+3d approaches, UR2D integrates facial data across modalities and across registration/recognition stages in a subject-specific manner. Furthermore, unlike existing 3D-aided 2D recognition methods, the present method uses 2D images to infer a 3D gallery model. UR2D is constructed from an actual 3D facial data fitting model based on a personalized gallery model.
On the other hand, as shown in fig. 5, an embodiment of the present invention further provides a face recognition device, including:
the 3-dimensional model construction module 10 is used for constructing 3-dimensional face model texture data by utilizing the 3-dimensional model and the 2-dimensional face image data;
The projective transformation module 20 is configured to projectively transform the 3-dimensional face model texture data to obtain a 2-dimensional texture image;
A parameterization module 30, configured to parameterize the 2-dimensional texture image to obtain a UV image;
an illuminance processing module 40, configured to perform illumination normalization processing on the UV image;
And the identification module 50 is used for identifying the UV image subjected to the degree normalization processing to obtain a similarity score.
Optionally, the face recognition device in this embodiment further includes a normalization processing module, configured to normalize the similarity score.
As an illustration, the projective transformation module 20 according to the embodiment of the present invention includes:
the perspective projection unit is used for carrying out linear mapping on the 3-dimensional face model texture data under the perspective projection model to obtain a 2-dimensional texture image;
And the re-projection error minimization unit is used for carrying out re-projection error minimization processing on the 2-dimensional texture image.
As another example, the parameterization processing module 30 of the present embodiment includes:
the parameterization processing unit is used for carrying out model parameterization processing on the 2-dimensional texture image to obtain a UV image;
and the removing unit is used for removing the pseudo value points in the UV image.
As another example, the illumination processing module 40 according to an embodiment of the present invention includes:
A skin reflection model analysis unit for analyzing and determining a skin reflection model of the UV image;
A texture illuminance model construction unit for constructing a texture illuminance model of the UV image from the skin reflection model;
and the illumination normalization unit is used for carrying out illumination normalization processing on the UV image by utilizing the texture illumination model.
The embodiment is a device corresponding to the face recognition method, and since the face recognition method has the technical effects described above, the corresponding face recognition device also has corresponding technical effects, and will not be described in detail herein.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Claims (8)
1. A face recognition method, comprising:
constructing 3-dimensional face model texture data by using the 3-dimensional model and the 2-dimensional face image data;
performing projection transformation on the 3-dimensional face model texture data to obtain a 2-dimensional texture image;
Carrying out parameterization treatment on the 2-dimensional texture image to obtain a UV image;
carrying out illumination normalization processing on the UV image;
The step of carrying out illumination normalization processing on the UV image comprises the following steps:
determining a skin reflection model of the UV image;
Constructing a texture illuminance model of the UV image according to the skin reflection model;
performing illumination normalization processing on the UV image by using the texture illumination model;
texture illumination model: texture image T (u) in UV space, Applied to unknown face albedo B (u):
T(u)＝Ls(u)+(Ld(u)+La(u))B(u)；
Where L d (u) is a diffuse component, L s (u) is a specular component, and L a (u) is an ambient lighting component; a pair of texture images may be normalized in two different ways: estimating an albedo component or delivering illumination parameters, wherein the illumination component needs to be estimated because of luminescence B (u) = (T (u) -L s(u))/(Ld(u)+La (u)) in the solution equation;
And identifying the UV image subjected to the normalization processing of the contrast degree to obtain a similarity score.
2. The face recognition method according to claim 1, wherein the step of identifying the UV image after the normalization processing to obtain a similarity score further comprises:
normalizing the similarity score.
3. The face recognition method of claim 1, wherein the 3-dimensional model is an AFM build model, and the surface parameters of the AFM build model are an internal injection function:
Wherein the method comprises the steps of Representing a 3-dimensional vector R 3, M representing an image, and U representing a 2-dimensional image network.
4. The method of claim 2, wherein the step of projectively transforming the 3-dimensional face model texture data to obtain a 2-dimensional texture image comprises:
Performing linear mapping on the 3-dimensional face model texture data under a perspective projection model to obtain a 2-dimensional texture image;
and carrying out re-projection error minimization processing on the 2-dimensional texture image.
5. A method of face recognition according to claim 3, wherein the step of parameterizing the 2-dimensional texture image to obtain a UV image comprises:
performing model parameterization on the 2-dimensional texture image to obtain a UV image;
And removing the pseudo value points in the UV image.
6. A face recognition device, comprising:
The 3-dimensional model construction module is used for constructing 3-dimensional face model texture data by utilizing the 3-dimensional model and the 2-dimensional face image data;
the projection conversion module is used for carrying out projection conversion on the 3-dimensional face model texture data to obtain a 2-dimensional texture image;
the parameterization processing module is used for carrying out parameterization processing on the 2-dimensional texture image to obtain a UV image;
The illumination processing module is used for carrying out illumination normalization processing on the UV image;
The illuminance processing module includes:
A skin reflection model analysis unit for analyzing and determining a skin reflection model of the UV image;
A texture illuminance model construction unit for constructing a texture illuminance model of the UV image from the skin reflection model;
The illumination normalization unit is used for carrying out illumination normalization processing on the UV image by utilizing the texture illumination model;
And the identification module is used for identifying the UV image subjected to the illumination normalization processing to obtain a similarity score.
7. The face recognition device of claim 6, wherein the projective transformation module comprises:
the perspective projection unit is used for carrying out linear mapping on the 3-dimensional face model texture data under the perspective projection model to obtain a 2-dimensional texture image;
And the re-projection error minimization unit is used for carrying out re-projection error minimization processing on the 2-dimensional texture image.
8. The face recognition device of claim 7, wherein the parameterization module comprises:
the parameterization processing unit is used for carrying out model parameterization processing on the 2-dimensional texture image to obtain a UV image;
and the removing unit is used for removing the pseudo value points in the UV image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910146378.5A CN110032927B (en) | 2019-02-27 | 2019-02-27 | Face recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910146378.5A CN110032927B (en) | 2019-02-27 | 2019-02-27 | Face recognition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110032927A CN110032927A (en) | 2019-07-19 |
| CN110032927B true CN110032927B (en) | 2024-08-02 |
Family
ID=67235021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910146378.5A Active CN110032927B (en) | 2019-02-27 | 2019-02-27 | Face recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110032927B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110580715B (en) * | 2019-08-06 | 2022-02-01 | 武汉大学 | Image alignment method based on illumination constraint and grid deformation |
| CN112528902B (en) * | 2020-12-17 | 2022-05-24 | 四川大学 | Video monitoring dynamic face recognition method and device based on 3D face model |
| CN113505717B (en) * | 2021-07-17 | 2022-05-31 | 桂林理工大学 | Online passing system based on face and facial feature recognition technology |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652172A (en) * | 2020-06-09 | 2020-09-11 | 安徽省徽腾智能交通科技有限公司 | Method for improving face recognition efficiency |
| CN115761844A (en) * | 2022-11-03 | 2023-03-07 | 江西方兴科技股份有限公司 | 3D-2D face recognition method |
Family Cites Families (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070080967A1 (en) * | 2005-10-11 | 2007-04-12 | Animetrics Inc. | Generation of normalized 2D imagery and ID systems via 2D to 3D lifting of multifeatured objects |
| WO2008104549A2 (en) * | 2007-02-28 | 2008-09-04 | Fotonation Vision Limited | Separating directional lighting variability in statistical face modelling based on texture space decomposition |
| CN101159015B (en) * | 2007-11-08 | 2010-12-08 | 清华大学 | Two-dimensional human face image recognizing method |
| KR100992151B1 (en) * | 2008-02-28 | 2010-11-04 | 성균관대학교산학협력단 | Method and apparatus for illumination normalization of face image |
| CN101561874B (en) * | 2008-07-17 | 2011-10-26 | 清华大学 | Method for recognizing face images |
| CN101320484B (en) * | 2008-07-17 | 2012-01-04 | 清华大学 | Three-dimensional human face recognition method based on human face full-automatic positioning |
| CN101777120A (en) * | 2010-01-28 | 2010-07-14 | 山东大学 | Face recognition image processing method based on sequence characteristics |
| CN101916454B (en) * | 2010-04-08 | 2013-03-27 | 董洪伟 | Method for reconstructing high-resolution human face based on grid deformation and continuous optimization |
| US8648959B2 (en) * | 2010-11-11 | 2014-02-11 | DigitalOptics Corporation Europe Limited | Rapid auto-focus using classifier chains, MEMS and/or multiple object focusing |
| CN103049896B (en) * | 2012-12-27 | 2015-09-16 | 浙江大学 | The geometric data of three-dimensional model and data texturing autoregistration algorithm |
| CN104376594B (en) * | 2014-11-25 | 2017-09-29 | 福建天晴数码有限公司 | Three-dimensional face modeling method and device |
| KR20160098581A (en) * | 2015-02-09 | 2016-08-19 | 홍익대학교 산학협력단 | Method for certification using face recognition an speaker verification |
| US10521649B2 (en) * | 2015-02-16 | 2019-12-31 | University Of Surrey | Three dimensional modelling |
| CN106469465A (en) * | 2016-08-31 | 2017-03-01 | 深圳市唯特视科技有限公司 | A kind of three-dimensional facial reconstruction method based on gray scale and depth information |
| CN106529486A (en) * | 2016-11-18 | 2017-03-22 | 深圳市唯特视科技有限公司 | Racial recognition method based on three-dimensional deformed face model |
| CN107247916A (en) * | 2017-04-19 | 2017-10-13 | 广东工业大学 | A kind of three-dimensional face identification method based on Kinect |
| CN107507263B (en) * | 2017-07-14 | 2020-11-24 | 西安电子科技大学 | Texture generation method and system based on image |
| CN107909640B (en) * | 2017-11-06 | 2020-07-28 | 清华大学 | Face relighting method and device based on deep learning |
| CN108109198A (en) * | 2017-12-18 | 2018-06-01 | 深圳市唯特视科技有限公司 | A kind of three-dimensional expression method for reconstructing returned based on cascade |
-
2019
- 2019-02-27 CN CN201910146378.5A patent/CN110032927B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111652172A (en) * | 2020-06-09 | 2020-09-11 | 安徽省徽腾智能交通科技有限公司 | Method for improving face recognition efficiency |
| CN115761844A (en) * | 2022-11-03 | 2023-03-07 | 江西方兴科技股份有限公司 | 3D-2D face recognition method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110032927A (en) | 2019-07-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dame et al. | Dense reconstruction using 3D object shape priors | |
| Rad et al. | Bb8: A scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth | |
| JP6681729B2 (en) | Method for determining 3D pose of object and 3D location of landmark point of object, and system for determining 3D pose of object and 3D location of landmark of object | |
| Marton et al. | General 3D modelling of novel objects from a single view | |
| US9189855B2 (en) | Three dimensional close interactions | |
| JP6216508B2 (en) | Method for recognition and pose determination of 3D objects in 3D scenes | |
| JP5455873B2 (en) | Method for determining the posture of an object in a scene | |
| CN110335234B (en) | Three-dimensional change detection method based on antique LiDAR point cloud | |
| CN111768447B (en) | Monocular camera object pose estimation method and system based on template matching | |
| CN110032927B (en) | Face recognition method | |
| CN108229416A (en) | Robot SLAM methods based on semantic segmentation technology | |
| Toderici et al. | Bidirectional relighting for 3D-aided 2D face recognition | |
| CN110751097A (en) | Semi-supervised three-dimensional point cloud gesture key point detection method | |
| Jiang et al. | REG-Net: Improving 6DoF object pose estimation with 2D keypoint long-short-range-aware registration | |
| JP4379459B2 (en) | Object collation method, object collation apparatus, and recording medium recording the program | |
| Li et al. | Accurate and fast 3D head pose estimation with noisy RGBD images | |
| Chen et al. | A comparative analysis between active structured light and multi-view stereo vision technique for 3D reconstruction of face model surface | |
| Wang et al. | Joint head pose and facial landmark regression from depth images | |
| Xie et al. | Fine registration of 3D point clouds fusing structural and photometric information using an RGB-D camera | |
| Chang et al. | 3d pose estimation and segmentation using specular cues | |
| CN115761844A (en) | 3D-2D face recognition method | |
| Zins et al. | Level set-based camera pose estimation from multiple 2D/3D ellipse-ellipsoid correspondences | |
| Guo et al. | A hybrid framework based on warped hierarchical tree for pose estimation of texture-less objects | |
| Liu | Semantic mapping: a semantics-based approach to virtual content placement for immersive environments | |
| Matez-Bandera et al. | Sigma-FP: Robot Mapping of 3D Floor Plans With an RGB-D Camera Under Uncertainty |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240909 Address after: Room B1705, Huazi Pentium Building, 2816 Yixian Road, Baoshan District, Shanghai, 200441 Patentee after: Shiyuan (Shanghai) Transportation Technology Co.,Ltd. Country or region after: China Address before: 200433 Room 401, No. 48, Lane 580, Zhengli Road, Yangpu District, Shanghai Patentee before: SHIYUAN (SHANGHAI) INTELLIGENT TECHNOLOGY Co.,Ltd. Country or region before: China |
|
| TR01 | Transfer of patent right |