WO2020211347A1

WO2020211347A1 - Facial recognition-based image modification method and apparatus, and computer device

Info

Publication number: WO2020211347A1
Application number: PCT/CN2019/117660
Authority: WO
Inventors: 李影
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-04-16
Filing date: 2019-11-12
Publication date: 2020-10-22
Also published as: CN110163814B; CN110163814A

Abstract

Disclosed in the present application are a facial recognition-based image modification method and apparatus, and a computer device. The method comprises: extracting a face image in a video after receiving an image modification instruction sent by a user, wherein the instruction comprises an object to be loaded on the face image; calculating a deflection angle of the face image, and calling a 3D model corresponding to the object in a database; projecting the 3D model according to the deflection angle to obtain a replacement image; and loading the replacement image on the face image. In the present application, during replacement of a face image, according to the deflection angle of the face in the image or the video, the angle of a pre-replacement face is also replaced with the same deflection angle as the face in the video, so that the image obtained after replacement is more realistic.

Description

Method, device and computer equipment for modifying pictures based on face recognition

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on April 16, 2019, the application number is 201910305175.6, and the invention title is "Method, Apparatus, and Computer Equipment for Modifying Images Based on Face Recognition", and its entire contents Incorporated in this application by reference.

Technical field

This application relates to the field of artificial intelligence technology, in particular to a method, device and computer equipment for modifying pictures based on face recognition.

Background technique

At present, there are many image processing software on the market. The image processing software can automatically recognize the face of the person in the picture to be processed, and then replace other faces that the user needs to replace with the face on the picture to be processed. However, when the face of the person in the picture is sideways, the other face information that the user needs to replace is all upright, resulting in inconsistent angles between the face and the body in the processed picture.

technical problem

The main purpose of this application is to provide a method, device and computer equipment for modifying a picture based on face recognition that can modify the picture according to the deflection angle of the face in the picture.

Technical solutions

In order to achieve the above-mentioned object of the invention, this application proposes a method for modifying pictures based on face recognition, including:

After receiving the instruction to modify the picture issued by the user, extract the face image in the video, and the instruction includes the object to be loaded on the face image;

According to the formula: y=18.75x ² -135x+116.25, calculate the deflection angle of the face image, and call the 3D model corresponding to the object in the database, where y represents the deflection angle to the left, and x represents the face The ratio of the size of the left organ in the image to the size of the corresponding right organ;

Project the 3D model according to the deflection angle to obtain a replacement picture;

Loading the replacement picture on the face image.

This application also provides a device for modifying pictures based on face recognition, including:

The extraction module is configured to extract the face image in the video after receiving the instruction to modify the picture sent by the user, and the instruction includes the object to be loaded on the face image;

The calculation angle module is used to calculate the deflection angle of the face image according to the formula: y=18.75x ² -135x+116.25, and call the 3D model corresponding to the object in the database, where y represents the deflection to the left Angle, x represents the ratio of the left organ size to the corresponding right organ size in the face image;

A projection module, configured to project the 3D model according to the deflection angle to obtain a replacement picture;

The loading module is used to load the replacement picture on the face image.

The present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of any one of the foregoing methods when the computer program is executed.

The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the above methods are implemented.

Beneficial effect

The method, device and computer equipment for modifying pictures based on face recognition of the present application, when replacing a face picture, according to the deflection angle of the face in the picture or video, the angle of the pre-replaced face is also replaced with The deflection angle of the face in the video is the same, which makes the replaced picture more realistic.

Description of the drawings

FIG. 1 is a schematic flowchart of a method for modifying a picture based on face recognition according to an embodiment of the application;

2 is a schematic block diagram of the structure of an apparatus for modifying pictures based on face recognition according to an embodiment of the application;

FIG. 3 is a schematic block diagram of the structure of a computer device according to an embodiment of the application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The best mode of the invention

1, an embodiment of the present application provides a method for modifying a picture based on face recognition, including the steps:

S1. After receiving an instruction to modify a picture from a user, extract a face image in the video, where the instruction includes an object to be loaded on the face image;

S2. According to the formula: y=18.75x ² -135x+116.25, calculate the deflection angle of the face image, and call the 3D model corresponding to the object in the database, where y represents the deflection angle to the left, and x represents The ratio of the left organ size to the corresponding right organ size in the face image;

S3. Project the 3D model according to the deflection angle to obtain a replacement picture;

S4. Load the replacement picture on the face image.

As described in step S1 above, the execution body of this embodiment is applied to an image processing APP on a mobile phone, and the execution body is the processor of the mobile phone. When the APP loads the video file, the processor receives the modified picture sent by the user, reads the video file, scans the video file to form an image, or directly extracts the frame pictures in the video, and then extracts the face in the video Image, extract the face image in the video in real time according to the video playback. The above-mentioned video files include video files that are included in the mobile phone, video files played in the browser, dynamic pictures, and video files generated by online video chats. The step for the user to issue an instruction to modify the picture is to receive the instruction to modify the picture from the user on the APP interface, and load multiple objects in the database to be loaded on the face image on the interface for the user to select, and to receive the user’s choice After the object, generate instructions to modify the picture. There are multiple objects stored in the database, and each object is displayed in the form of an image or text or a combination of the two, so that the user can quickly understand the specific information of the object and select the object the user wants. For example, there are ten objects in the database, and each object is the face of an entertainment star. When the APP interface receives the user's instruction to modify the picture, it loads these ten objects on the interface at the same time, allowing the user to select one of the face images as the object loaded on the above face image; then the APP packs the object Enter the instruction to form an instruction issued by the user to modify the picture.

As described in step S2 above, after the processor extracts the face image in the video, it calculates the deflection angle of the face according to the face feature points. After the processor extracts the face image in the video, it can determine the deflection angle of the face image by using the facial features information. The deflection angle is the deflection angle set for the face image relative to the preset reference position. If the preset reference position is the face image when the user is looking straight ahead, when the face in the video turns left or right or tilts up or When looking down, the deflection angle is relative to the rotation angle of head-up view. When the face image is deflected to different degrees, the collected facial features have different relative positions, and the size of some symmetric facial features will change. For example, if a person deflects to the left, the left ear is farther away from the camera, and the corresponding left ear in the video is smaller than the right ear. Therefore, the relative position of the facial features and the size of some symmetrical facial features can be used to calculate the deflection angle of the face image. In a specific embodiment, after extracting the face picture in the video, the two eyes in the face picture are identified, and the width dimensions of the two eyes are calculated respectively, and then the ratio of the width dimensions of the two eyes is calculated. Calculate the deflection angle. For example, in a captured image, the width of the left eye of a person is 3000, and the width of the right eye is 4000. The above-mentioned width unit is the smallest size unit obtained after cutting the image. Then calculate the ratio of the left eye to the right eye to be 3:4, that is, the width of the left eye is 3/4 of the width of the right eye, and then output to the preset formula y=f(x) to calculate the deflection angle to the left, In the above formula, y represents the deflection angle of deflection to the left, and x represents the ratio of the width of the left eye to the width of the right eye 3/4. The final calculated y is an angle between negative 90 degrees and positive 90 degrees, that is, the deflection angle of leftward deflection. If the obtained value is negative, it means the absolute value of the rightward deflection angle. After a basic test in a fixed pixel, the specific formula is: y=18.75x ² -135x+116.25.

At the same time, when the user issues an instruction to modify the picture, first select a loaded object for loading on the face image in the video. Among them, the user can select an object to be loaded in the database that comes with the mobile phone. Multiple objects and 3D models corresponding to each object are stored in the database. When the user needs to replace the face image in the video or add a picture on the APP, select an object in the database in the APP. After the processor receives the instruction from the user, it calls the user selection according to the object in the instruction The 3D model corresponding to the object. A 3D model refers to a three-dimensional image corresponding to an object stored in the user's face, animal avatar, glasses and hat waiting to be loaded on the face image. The three-dimensional image is formed by the user by taking a 360-degree photograph of an object or person in advance Three-dimensional images can also be downloaded from the Internet.

As described in step S3 above, after the processor calls the 3D model, it obtains the calculated deflection angle, and then projects the 3D model from the deflection angle to obtain a two-dimensional picture of the 3D model projected at the deflection angle. That is, the above replacement picture.

As described in step S4, the processor then loads the replacement picture on the face image in the video, so that the replacement picture is overlaid on the face image, which achieves the purpose of replacing the face in the video.

In this embodiment, because the video is played continuously, the calculation of the deflection angle of the face image is also calculated in real time after receiving the user's instruction, and the corresponding loaded replacement image is also loaded in real time.

In one embodiment, the aforementioned 3D model is a human face 3D model, and the aforementioned step of projecting the 3D model according to the deflection angle to obtain a replacement picture includes:

S31. After preprocessing the face image, it is input into a trained expression recognition model for recognition, so as to obtain the expression of the face image;

S32. Acquire the first feature point of the human face 3D model and the adjustment matrix corresponding to the expression, wherein the correspondence between the expression and the adjustment matrix is preset;

S33. Multiply the first feature point by the adjustment matrix to obtain the second feature point;

S34. Map the second feature point on the 3D face model, and control the 3D face model to simulate the expression;

S35. After rotating the face 3D model that simulates the expression by the degree of the deflection angle, map it to the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.

In this embodiment, the 3D model refers to a 3D model of a human face. After the processor extracts the face image in the video, it preprocesses the face image. The preprocessing is the processing before the feature extraction, segmentation and matching of the image. The main purpose of image preprocessing is to eliminate irrelevant images in the image. Information, restore useful real information, enhance the detectability of relevant information and simplify data to the greatest extent, thereby improving the reliability of feature extraction, image segmentation, matching and recognition. Then, the preprocessed face image is input to the trained training model for expression recognition, and the expression recognition result of the face image sequence is obtained; wherein, the input end to the output end of the training model are sequentially performed by the convolutional neural network The model, the long and short-term memory cyclic neural network model, the first pooling layer and the logistic regression model are constructed, and the training model is obtained by training a collection of continuous frame images with annotated expression categories. After obtaining the expression of the face image, the processor controls the 3D face model to simulate an expression consistent with the expression of the aforementioned face image, and the method of simulating the corresponding expression is: first obtain the first feature point of the aforementioned face 3D model , And then multiply the first feature point by the adjustment matrix corresponding to the above expression to obtain the second feature point with the expression, and then map the second feature point on the above-mentioned face 3D model, then the person with the above expression is obtained 3D model of face. Each expression corresponds to an adjustment matrix, and the characteristic matrix corresponding to each expression is obtained by the staff after a lot of calculation and debugging. In the further specific adjustment process, the processor first divides the various parts of the face 3D model into regions, such as face region, nose region, mouth region, eyebrow region, etc., and can continue to subdivide each region into multiple sub-regions. Area, and then adjust the type according to the expression type and the preset area. After obtaining the expression, obtain the area or sub-area corresponding to the expression, then obtain the feature points of the area or sub-area corresponding to the expression, and then multiply the feature points by the expression The corresponding region adjustment matrix is used to obtain feature points with expressions, and then the feature points with expressions are mapped to corresponding regions or sub-regions to obtain a 3D face model with the above expressions. Realize the 3D model of the human face to simulate the same expression as the above-mentioned facial image expression. Then, according to the deflection angle of the aforementioned image, the deflection angle is the y value calculated in the formula, and the replacement picture with the aforementioned expression of the 3D face model is obtained by projection. For example, when it is recognized that the expression is a smile, in the corresponding relationship between the expression type and the area adjustment method, the adjustment area corresponding to the smile expression type is the mouth area and the adjustment matrix corresponding to the mouth area, and then the above 3D model The mouth area in is multiplied by the above adjustment matrix to obtain a 3D face model with a smiling expression. Then rotate the 3D face model with the smile expression at the above deflection angle, and then project the rotated 3D face model with the smile expression from the direction facing the original 3D face model on the initial On the plane behind the 3D face model, a replacement picture with a smiling expression is finally obtained. Specifically, the 3D face model with the above expression is centered on the axis of symmetry of the 3D face model, and the angle of the y value calculated above is rotated, and then a camera is simulated, from the initial 3D face model (not rotated) Take a picture of the opposite place, and the resulting picture is the above replacement picture.

S36. Input the face image into a preset organ recognition model to obtain the first organ in the face image;

S37. Build a matrix with the same number of rows and columns according to the pigment difference between the first organ and the second organ under the preset normal state to obtain the first matrix and the second matrix respectively;

S38. Subtract the second matrix from the first matrix to obtain a matrix difference;

S39. Calculate the rank of the matrix difference;

S310. Invoke a preset correspondence relationship between a rank and a facial action, and obtain the facial action corresponding to the first organ according to the rank of the matrix difference;

S311: Control each organ of the human face 3D model to simulate the facial action;

S312: After rotating the face 3D model that simulates the facial motion by the degree of the deflection angle, map the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.

In this embodiment, the extracted face image in the video is preprocessed and input into a trained organ recognition model. The organ recognition model segments the aforementioned face image and recognizes each of the face images. Organs. The image area containing the various organs in the above face image is defined as the first organ. The first organ includes multiple organs that can move on the face. Specifically, the first organ includes eyes, mouth, nose, eyebrows, and cheeks. muscle. Each first organ is compared with the preset second organ state under the normal state one by one, and the action of each organ is determined according to the comparison result. The specific process of judging the actions of the organs is to perform image analysis on the first organ and the corresponding second organ, and establish a matrix with the same number of rows and columns for the first organ and the second organ according to the pigment difference, and then subtract the matrix of the first organ. The matrix of the second organ is removed to obtain the matrix difference, and then the rank of the matrix difference is calculated. According to the result of the rank, the facial movement of the organ is determined from the correspondence between the rank of the organ and the facial movement. In addition, if the result of the rank is less than a certain value, it is determined that the organ is in a normal state, and the corresponding organ in the aforementioned 3D face model does not need to simulate other facial movements. Then, the various organs of the human face 3D model are identified, and it is confirmed that the first organ has an abnormal facial movement in the above-mentioned organ. Then multiply each organ of the face 3D model by the action matrix of the facial action corresponding to the organ to obtain the facial action of each organ of the face model, and make the face 3D model make facial actions with the above changes The same action. Then, according to the deflection angle of the image, a two-dimensional picture of the 3D face model with the same facial action is obtained by projection, and then the two-dimensional picture is loaded at the position of the face image in the video. The steps of projecting the 3D face model with facial action to obtain the replacement picture are the same as the method of S35.

The above-mentioned organ recognition model is based on a deep neural network model training. The staff first collects a number of pictures containing the face of a person, and annotates each organ on the face of each person in the picture, and then the picture And each annotation is input into the deep neural network model for training, and the organ recognition model is obtained.

In one embodiment, the above step of loading the replacement picture on the face image includes:

S41: Acquire the size of the display screen in the attribute information of the terminal and the proportion of the face image in the display screen in the playback interface;

S42. Simulate loading the face image on the playback interface, and calculate the size of the face image in the playback interface;

S43. Calculate the ratio between the size of the face image and the preset standard size;

S44. Perform scaling processing on the replacement picture according to the scale;

S45. Load the replacement picture after the scaling process on the face image.

In this embodiment, the processor calculates the size of the aforementioned face image according to the video information. Specifically, it obtains the attribute information of the mobile phone where the APP is located, and obtains the size of the display screen. Proportion, which simulates loading a face image on the playback interface, and then calculates the size of the face image in the playback interface. Then compare the size with the preset standard size to get the ratio between the two. The standard size is set manually. The setting process is to place a standard face at a specified distance from the specified camera and then collect the size of the face. The 3D model corresponding to the object in the database is also specified according to the specified camera. Captured after the distance. Calculate the ratio of the size of the face image to the standard size, and then scale the replacement image according to the above ratio, so that the size of the replacement image matches the size of the face image, and the size of the replacement image is the same as that of the face image. The size is close, so that when the replacement image is loaded on the face image, the size is appropriate and looks more coordinated. Then load the zoomed replacement picture on the face image.

In one embodiment, the step of extracting the face image in the video after receiving the instruction to modify the picture from the user includes:

S11. After receiving the instruction to modify the picture sent by the user, collect the frame picture in the video;

S12. Input the frame picture into a preset face recognition model, and output the face image.

In this embodiment, after the processor receives the replacement or addition instruction issued by the user, it obtains the video played in the specified APP, then reads the detailed information of the video, intercepts the frame picture played at the current moment of the video, and then the frame The picture is input into the preset face recognition model, and the recognition is performed according to the face recognition model, and the face image is obtained by outputting.

In an embodiment, before the step of inputting the frame picture into the preset face recognition model and outputting the face image, the method includes:

S101: Input a plurality of pictures containing human face images in a training set into a preset neural network model for training, and obtain a neural network model for recognizing human faces in the pictures as the face recognition model.

In this embodiment, the staff selects a number of pictures containing human face images in advance to form a training set, and then inputs these pictures into a preset neural network model for training. The neural network model is automatically based on the gray in the face image. The trajectory formed by the degree is calculated and optimized to obtain the feature coefficients of the face picture, even if the neural network model can be used to identify whether there is a face image in the picture. In the subsequent extraction of the face image in the video, the frame picture is processed to obtain the grayscale of the picture, and then the grayscale trajectory is calculated whether the feature coefficient obtained by the above training is included, and if it is, the human face image in the video is judged. According to the gray trajectory corresponding to the feature coefficient, the face image is extracted.

In one embodiment, after the above step of loading the replacement picture on the face image, it includes:

S5. Obtain attribute information of the object;

S6. Determine whether the attribute information is face information;

S7. If yes, delete the face image.

In this embodiment, the object is stored in the database, and the corresponding information of each object is also stored in the database. The attributes of each object include whether it is face information. If it is face information, the replacement picture corresponds to the person’s face information. The face information is in an irregular shape, and the replacement pictures formed rarely cover the above On the face image, therefore, it is necessary to delete the face image in the video to make the overall replacement video information more coordinated. The processor reads the attribute information of the object, and judges whether the attribute information is face information. If it is face information, delete the face image. In a specific embodiment, the attribute information of the object includes two types of facial information and accessory information, where the facial information includes human faces, dog faces, and cat faces, and the accessory information includes glasses, earrings, and hats. If it is determined that the object is not facial information, only the replacement picture is loaded on the face image to form the effect of wearing accessories to the characters in the video, which gives the user a better experience.

In summary, the method for modifying a picture based on face recognition of the present application, when replacing a face picture, according to the deflection angle of the face in the picture or video, the angle of the pre-replaced face is also replaced with The deflection angle of the face in the video is the same, which makes the replaced picture more realistic.

2, an embodiment of the present application also provides an apparatus for modifying pictures based on face recognition, including:

The extraction module 1 is configured to extract a face image in a video after receiving an instruction to modify a picture issued by a user, and the instruction includes an object to be loaded on the face image;

The angle calculation module 2 is used to calculate the deflection angle of the face image according to the formula: y=18.75x ² -135x+116.25, and call the 3D model corresponding to the object in the database, where y represents the left deflection Deflection angle, x represents the ratio of the size of the left organ in the face image to the size of the corresponding right organ;

The projection module 3 is used to project the 3D model according to the deflection angle to obtain a replacement picture;

The loading module 4 is used to load the replacement picture on the face image.

3, an embodiment of the present application also provides a computer device. The computer device may be a server, and its internal structure may be as shown in FIG. 3. The computer equipment includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the computer designed processor is used to provide calculation and control capabilities. The memory of the computer device includes a storage medium and an internal memory. The storage medium may be a non-volatile storage medium or a volatile storage medium, and the storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the operation of the operating system and computer programs in the storage medium. The database of the computer equipment is used to store data such as 3D models. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to realize a method of modifying pictures based on face recognition.

Those skilled in the art can understand that the structure shown in FIG. 3 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.

An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, a method for modifying a picture based on face recognition is implemented.

A person of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program. The computer program can be stored in a computer readable storage medium. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

It should be noted that in this article, the terms "including", "including" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements not only includes those elements, It also includes other elements that are not explicitly listed, or elements inherent to the process, device, article, or method. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, device, article or method that includes the element.

The above are only the preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of this application description and drawings, or directly or indirectly applied to other related The technical field is equally included in the scope of patent protection of this application.

Claims

A method for modifying pictures based on face recognition, which is characterized in that it includes:

After receiving the instruction to modify the picture issued by the user, extract the face image in the video, and the instruction includes the object to be loaded on the face image;

According to the formula: y=18.75x 2 -135x+116.25, calculate the deflection angle of the face image, and call the 3D model corresponding to the object in the database, where y represents the deflection angle to the left, and x represents the face The ratio of the size of the left organ in the image to the size of the corresponding right organ;

Project the 3D model according to the deflection angle to obtain a replacement picture;

Loading the replacement picture on the face image.
The method for modifying a picture based on face recognition according to claim 1, wherein the 3D model is a face 3D model, and the step of projecting the 3D model according to the deflection angle to obtain a replacement picture ,include:

After the face image is preprocessed, it is input into a trained expression recognition model for recognition, so as to obtain the expression of the face image;

Acquiring a first feature point of the human face 3D model and an adjustment matrix corresponding to the expression, wherein the corresponding relationship between the expression and the adjustment matrix is preset;

Multiply the first feature point by the adjustment matrix to obtain the second feature point;

Mapping the second feature point on the face 3D model, and controlling the face 3D model to simulate the expression;

After the face 3D model that simulates the expression is rotated by the degree of the deflection angle, it is mapped to the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.
The method for modifying a picture based on face recognition according to claim 1, wherein the 3D model is a face 3D model, and the step of projecting the 3D model according to the deflection angle to obtain a replacement picture ,include:

Inputting the face image into a preset organ recognition model to obtain the first organ in the face image;

Establishing a matrix with the same number of rows and columns according to the pigment difference between the first organ and the preset second organ under normal conditions to obtain the first matrix and the second matrix respectively;

Subtract the second matrix from the first matrix to obtain a matrix difference;

Calculating the rank of the matrix difference;

Calling a preset correspondence relationship between a rank and a facial movement, and acquiring the facial movement corresponding to the first organ according to the rank of the matrix difference;

Controlling each organ of the human face 3D model to simulate the facial action;

After the face 3D model that simulates the facial motion is rotated by the degree of the deflection angle, it is mapped to the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.
The method for modifying a picture based on face recognition according to claim 1, wherein the step of loading the replacement picture on the face image comprises:

Acquiring the size of the display screen in the attribute information of the terminal and the ratio of the face image to the display screen in the playback interface;

Simulate loading the face image on the playback interface, and calculate the size of the face image in the playback interface;

Calculating the ratio between the size of the face image and the preset standard size;

Performing zoom processing on the replacement picture according to the scale;

The replacement picture after the scaling process is loaded on the face image.
The method for modifying a picture based on face recognition according to claim 1, wherein the step of extracting the face image in the video after receiving the instruction to modify the picture from the user includes:

After receiving the user's instruction to replace the face, collect frame pictures in the video;

Input the frame picture into a preset face recognition model, and output the face image.
The method for modifying a picture based on face recognition according to claim 5, wherein before the step of inputting the frame picture into a preset face recognition model, and outputting the face image, include:

A plurality of pictures containing human face images in the training set are input to a preset neural network model for training, and a neural network model for recognizing the human face in the picture is obtained as the face recognition model.
The method for modifying a picture based on face recognition according to claim 1, wherein after the step of loading the replacement picture on the face image, the method comprises:

Acquiring attribute information of the object;

Judging whether the attribute information is face information;

If yes, delete the face image.
A device for modifying pictures based on face recognition, characterized in that it comprises:

The extraction module is configured to extract the face image in the video after receiving the instruction to modify the picture sent by the user, and the instruction includes the object to be loaded on the face image;

The calculation angle module is used to calculate the deflection angle of the face image according to the formula: y=18.75x 2 -135x+116.25, and call the 3D model corresponding to the object in the database, where y represents the deflection to the left Angle, x represents the ratio of the left organ size to the corresponding right organ size in the face image;

A projection module, configured to project the 3D model according to the deflection angle to obtain a replacement picture;

The loading module is used to load the replacement picture on the face image.
8. The device for modifying pictures based on face recognition according to claim 8, wherein the 3D model is a face 3D model, and the projection module comprises:

An expression acquiring unit, configured to preprocess the face image and input it into a trained expression recognition model for recognition, so as to acquire the expression of the face image;

The feature point unit is used to obtain the first feature point of the human face 3D model and the adjustment matrix corresponding to the expression, wherein the corresponding relationship between the expression and the adjustment matrix is preset;

The multiplication unit is configured to multiply the first feature point by the adjustment matrix to obtain the second feature point;

The emoticon simulation unit is configured to map the second feature point on the face 3D model, and control the face 3D model to simulate the emoticon;

The first projection unit is configured to rotate the face 3D model that simulates the expression by the degree of the deflection angle, and then map the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.
The device for modifying pictures based on face recognition of claim 8, wherein the D model is a 3D face model, and the projection module comprises:

The obtaining organ unit is used to input the face image into a preset organ recognition model to obtain the first organ in the face image;

A matrix establishment unit, configured to establish a matrix with the same number of rows and columns according to the pigment difference between the first organ and the second organ under a preset normal state, to obtain the first matrix and the second matrix respectively;

The subtraction unit is configured to subtract the second matrix from the first matrix to obtain a matrix difference;

A calculating rank unit for calculating the rank of the matrix difference;

An obtaining action unit, configured to call a preset correspondence relationship between rank and facial action, and obtain the facial action corresponding to the first organ according to the rank of the matrix difference;

The simulation action unit is used to control the various organs of the human face 3D model to simulate the facial action;

The second projection unit is configured to rotate the face 3D model that simulates the facial action by the degree of the deflection angle, and then map the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.
The device for modifying pictures based on face recognition according to claim 8, wherein the loading module comprises:

The obtaining size unit is used to obtain the size of the display screen in the attribute information of the terminal and the proportion of the face image in the display screen in the playback interface;

The size calculation unit is used to simulate loading the face image on the playback interface and calculate the size of the face image in the playback interface;

A calculating ratio unit for calculating the ratio between the size of the face image and the preset standard size;

A scaling unit, configured to perform scaling processing on the replacement picture according to the scale;

The loading unit is configured to load the zoomed replacement picture on the face image.
The device for modifying pictures based on face recognition according to claim 8, wherein the extraction module comprises:

The collection unit is used to collect the frame picture in the video after receiving the instruction to modify the picture sent by the user;

The output unit is configured to input the frame picture into a preset face recognition model, and output the face image.
The device for modifying a picture based on face recognition of claim 12, wherein the device for modifying a picture based on face recognition further comprises:

The training module is used for inputting a plurality of pictures containing human face images in the training set to a preset neural network model for training, and obtaining a neural network model for recognizing the human face in the picture as the face recognition model.
The device for modifying pictures based on face recognition according to claim 8, wherein the device for modifying pictures based on face recognition further comprises:

An obtaining module, used to obtain attribute information of the object;

The judgment module is used to judge whether the attribute information is face information;

The deleting module is used to delete the face image if the attribute information is face information.
A computer device includes a memory and a processor, the memory stores a computer program, and is characterized in that, when the processor executes the computer program, a method for modifying a picture based on face recognition is implemented, and the method includes the steps :

After receiving the instruction to modify the picture issued by the user, extract the face image in the video, and the instruction includes the object to be loaded on the face image;

According to the formula: y=18.75x 2 -135x+116.25, calculate the deflection angle of the face image, and call the 3D model corresponding to the object in the database, where y represents the deflection angle to the left, and x represents the face The ratio of the size of the left organ in the image to the size of the corresponding right organ;

Project the 3D model according to the deflection angle to obtain a replacement picture;

Loading the replacement picture on the face image.
15. The computer device of claim 15, wherein the 3D model is a human face 3D model, and the step of projecting the 3D model according to the deflection angle to obtain a replacement picture comprises:

After the face image is preprocessed, it is input into a trained expression recognition model for recognition, so as to obtain the expression of the face image;

Acquiring a first feature point of the human face 3D model and an adjustment matrix corresponding to the expression, wherein the corresponding relationship between the expression and the adjustment matrix is preset;

Multiply the first feature point by the adjustment matrix to obtain the second feature point;

Mapping the second feature point on the face 3D model, and controlling the face 3D model to simulate the expression;

After the face 3D model that simulates the expression is rotated by the degree of the deflection angle, it is mapped to the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.
15. The computer device according to claim 15, wherein the 3D model is a human face 3D model, and the step of projecting the 3D model according to the deflection angle to obtain a replacement picture comprises:

Inputting the face image into a preset organ recognition model to obtain the first organ in the face image;

Establishing a matrix with the same number of rows and columns according to the pigment difference between the first organ and the preset second organ under normal conditions to obtain the first matrix and the second matrix respectively;

Subtract the second matrix from the first matrix to obtain a matrix difference;

Calculating the rank of the matrix difference;

Calling a preset correspondence relationship between a rank and a facial movement, and acquiring the facial movement corresponding to the first organ according to the rank of the matrix difference;

Controlling each organ of the human face 3D model to simulate the facial action;

After the face 3D model that simulates the facial motion is rotated by the degree of the deflection angle, it is mapped to the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.
A computer-readable storage medium having a computer program stored thereon is characterized in that, when the computer program is executed by a processor, a method for modifying pictures based on face recognition is realized, the method comprising the steps of: After the instruction to modify the picture, extract the face image in the video, and the instruction includes the object to be loaded on the face image;

According to the formula: y=18.75x 2 -135x+116.25, calculate the deflection angle of the face image, and call the 3D model corresponding to the object in the database, where y represents the deflection angle to the left, and x represents the face The ratio of the size of the left organ in the image to the size of the corresponding right organ;

Project the 3D model according to the deflection angle to obtain a replacement picture;

Loading the replacement picture on the face image.
18. The computer-readable storage medium of claim 18, wherein the 3D model is a human face 3D model, and the step of projecting the 3D model according to the deflection angle to obtain a replacement picture comprises:

After the face image is preprocessed, it is input into a trained expression recognition model for recognition, so as to obtain the expression of the face image;

Acquiring a first feature point of the human face 3D model and an adjustment matrix corresponding to the expression, wherein the corresponding relationship between the expression and the adjustment matrix is preset;

Multiply the first feature point by the adjustment matrix to obtain the second feature point;

Mapping the second feature point on the face 3D model, and controlling the face 3D model to simulate the expression;

After the face 3D model that simulates the expression is rotated by the degree of the deflection angle, it is mapped to the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.
18. The computer-readable storage medium of claim 18, wherein the 3D model is a human face 3D model, and the step of projecting the 3D model according to the deflection angle to obtain a replacement picture comprises:

Inputting the face image into a preset organ recognition model to obtain the first organ in the face image;

Establishing a matrix with the same number of rows and columns according to the pigment difference between the first organ and the preset second organ under normal conditions to obtain the first matrix and the second matrix respectively;

Subtract the second matrix from the first matrix to obtain a matrix difference;

Calculating the rank of the matrix difference;

Calling a preset correspondence relationship between a rank and a facial movement, and acquiring the facial movement corresponding to the first organ according to the rank of the matrix difference;

Controlling each organ of the human face 3D model to simulate the facial action;

After the face 3D model that simulates the facial motion is rotated by the degree of the deflection angle, it is mapped to the back of the face 3D model when it is not rotated for projection to obtain a replacement picture.