CN105894446A - Automatic face outline modification method for video - Google Patents

Abstract

The invention discloses an automatic face outline modification method for a video. The method comprises the following steps: 1) detecting a face frame in the whole video to be processed, and detecting a bounding box and feature points of the face; 2) dividing a face area of interest into multiple triangles according to the positions of the bounding box and the feature points, and calculating a target feature point of each feature point; 3) calculating an affine transform matrix of each triangle having coordinates of three original feature points and target feature points obtained from the three original feature points in step 2) according to the coordinates of the six feature points, and performing affine transform on each point in the triangle formed by the three original feature points according to the affine transform matrix to generate a new face image; and 4) replacing the old face image with the new face image, thereby generating a result video. The method can be used for processing the face in the video, thereby achieving the face-lifting effect on the face in the video, namely realizing nonlinear edition of video contents.

Description

A kind of automatic face profile method of modifying in video

Technical field:

The invention belongs to video, technical field of image processing, the automatic face wheel being specifically related in a kind of video Wide method of modifying.

Background technology:

Universal along with social networks and smart mobile phone, people increasingly get used to the life of videograph oneself Live, such as have a meal, travel or concert etc..People can first compare when social networks issues photo Sheet carries out landscaping treatment, and is impossible when people want the process being similar to video when.Existing Some video editing methods are all the linear editings to video, are difficult to meet popular requirement, thus propose one The non-linear editing method to video content of planting seems the most necessary.

Summary of the invention:

It is an object of the invention to overcome the deficiencies in the prior art, it is provided that the automatic face wheel in a kind of video Wide method of modifying, its face being able to ensure that in the video after processing by the method can have as U.S. figure software Thinning face effect, thus meet the popular process requirement that video is shot the video particularly certainly.

For reaching above-mentioned purpose, the present invention adopts the following technical scheme that and is achieved:

Automatic face profile method of modifying in a kind of video, comprises the following steps:

1) there is the frame of face in detecting whole pending video, detect bounding box and the feature of face Point；

2) according to bounding box and characteristic point position, human face region interested is divided into multiple triangle, And calculate the target characteristic point of each characteristic point；

3) to each triangle, there are the coordinate of three former characteristic points and the former characteristic point of these three in step 2) In the target characteristic point that obtains, calculate the affine transformation square of this triangle according to the coordinate of these six characteristic points Battle array, does affine transformation according to each point in the triangle that three former characteristic points are formed by affine transformation matrix, Produce the facial image made new advances；

4) replace old facial image with new facial image, i.e. generate result video.

The present invention is further improved by, step 1) in, the characteristic point of face refer to face profile, eyes, Characteristic point on nose and lip.

The present invention is further improved by, step 2) in, human face region interested is face profile peripheral region Territory.

The present invention is further improved by, step 3) in, characterize with R matrix and rotate and scaling, L matrix Characterize translation, to a triangle, have:

$R=\left[\begin{array}{cc}{r}_{00}& r_{01}\\ r_{10}& r_{11}\end{array}\right]$

$L=\left[\begin{array}{c}{l}_{00}\\ l_{01}\end{array}\right]$

(x, y) is certain apex coordinate before affine transformation, after (x ', y ') is affine transformation on diabolo three summits The position of representative points, has:

$\left[\begin{array}{c}{x}^{\′}\\ y^{\′}\end{array}\right]=\left[\begin{array}{c}{r}_{00}x+r_{01}y+l_{00}\\ r_{10}x+r_{11}y+l_{01}\end{array}\right]$

6 amounts characterizing affine transformation in this R matrix and L matrix are calculated by these three summit, according to The coordinate of these six characteristic points calculates the affine transformation matrix of this triangle.

The present invention is further improved by, step 3) in, every according in affine transformation matrix diabolo Before one point does affine transformation, first to a triangle ABC, utilize s=CB × AB and W=(MA × MB) & (MB × MC) & (MC × MA) judges that a some M is whether in triangle；

As s > 0 time, represent triangle ABC order be counterclockwise, at this moment w > 0 represent some M three In dihedral；As s, < when 0, representing that the order of triangle ABC is clockwise, at this moment w < 0 represents some M In triangle；

I.e. to arbitrary M point in triangle, diabolo ABC, have:

((MA × MB) & (MB × MC) & (MC × MA)) (CB × AB) ＞ 0

These points are done affine by all of point in finding out triangle according to above-mentioned formula according to affine transformation matrix Conversion, the image after being converted, it is and produces the facial image made new advances.

Compared with prior art, the invention has the beneficial effects as follows: the face mask district of face in amendment frame of video Territory is by being only divided into little delta-shaped region by face mask region interested, then to each triangle Region carries out affine transformation and obtains new face image to modify the face in video.

Video editing of the present invention refers to editor rather than the row again of simple frame of video of the picture to video Row.

Wherein, existing video editing method, simply from one section of video, editing goes out a bit of or by several sections The frame of video rearranging method of one video of Video Composition.Editor described in the present invention refers to video pictures Editor, respond well, advantages.Further, traditional video editing method cannot meet popular right The requirement of video editing, this invention have employed the method for video content editing, it is ensured that video editing effective Property.

Accompanying drawing illustrates:

Fig. 1 is the explanation schematic diagram of the present invention.

Fig. 2 is for calculating target characteristic point process schematic diagram；Wherein, Fig. 2 (a) is face's central point and feature Point moving direction schematic diagram, Fig. 2 (b) is characterized the schematic diagram of D in displacement 4%*D.

Fig. 3 is for calculating radiation matrix process schematic；Wherein, Fig. 3 (a) is three before calculating affine transformation Angular domain, Fig. 3 (b) is the delta-shaped region after calculating affine transformation.

Fig. 4 is to judge a little the whether method schematic diagram in triangle；Wherein, Fig. 4 (a) is to work as triangle Whether summit order judges when be counterclockwise a little at the method schematic diagram of triangle interior, and Fig. 4 (b) is ought three Whether angled peak order judges when being clockwise a little at the method schematic diagram of triangle interior.

Fig. 5 is the flow chart of the present invention.

Detailed description of the invention:

The present invention is described in more detail with embodiment below in conjunction with the accompanying drawings.

Automatic face profile method of modifying in a kind of video of the present invention, is achieved by the steps of:

1. calculate target characteristic point

Seeing accompanying drawing 2, the effect of thinning face to be obtained, the characteristic point position of the labelling face mask after thinning face is inevitable Change.Before calculating these target characteristic point positions, first find face's central point.Face's central point On one line being scheduled on the point in the middle of two eyes and the point in the middle of face, we select the conduct of 4 points here Alternative, they distances to face centre and the distance to eye center are 10:0 than successively, 8:2,6:4 and x:y, Wherein last point is taken at the position of nose.Experiment display, selects second point as the result of central point The most natural.

The target characteristic point of face mask is on face's central point line with face mask point, simply length Being reduced to original 96%, the point obtained is exactly the position of target characteristic point.

2. calculate affine transformation matrix

See accompanying drawing 3, region interested around face mask is divided into multiple little delta-shaped region, for often One little delta-shaped region calculates affine transformation matrix.What affine transformation matrix represented is the connection between two width figures System, generally represents with the matrix of 2*3, refers to that a width figure obtains the second width by rotation, zooming and panning Figure.One arbitrary affine transformation can be expressed as being multiplied by a matrix and then add a vector, this Matrix can by the characteristic point on each three summit of delta-shaped region and corresponding target characteristic point 6 Know and a little try to achieve.

Characterizing with R matrix and rotate and scaling, L matrix characterizes translation, to a triangle, has:

$R=\left[\begin{array}{cc}{r}_{00}& r_{01}\\ r_{10}& r_{11}\end{array}\right]$

$L=\left[\begin{array}{c}{l}_{00}\\ l_{01}\end{array}\right]$

(x, y) is certain apex coordinate before affine transformation, after (x ', y ') is affine transformation on diabolo three summits The position of representative points, has:

$\left[\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right]=\left[\begin{array}{c}{r}_{00}x+r_{01}y+l_{00}\\ r_{10}x+r_{11}y+l_{01}\end{array}\right]$

These 6 amounts characterizing affine transformation can be calculated by such three opposite vertexes.

3. do a little affine transformation in pair each triangle

See accompanying drawing 4, to a triangle ABC, utilize s=CB × AB and W=(MA × MB) & (MB × MC) & (MC × MA) judges that a some M is whether in triangle.

As s > 0 time, represent triangle ABC order be counterclockwise, at this moment w > 0 represent some M three In dihedral；As s, < when 0, representing that the order of triangle ABC is clockwise, at this moment w < 0 represents some M In triangle.I.e. to arbitrary M point in triangle, diabolo ABC, have:

((MA × MB) & (MB × MC) & (MC × MA)) (CB × AB) ＞ 0

To sum up, all of point in can finding out triangle, to these o'clocks according to the 2 affine transformation squares calculated Battle array does affine transformation, the image after i.e. can being converted, it is simply that the image after thinning face, like this single-frame images Interior implementation process, as it is shown in figure 1, replace original image with the image after thinning face again, has just obtained thin Video after face editor, to the specific implementation process of whole video as shown in Figure 5.

Claims (5)

1. the automatic face profile method of modifying in a video, it is characterised in that comprise the following steps:

1) there is the frame of face in detecting whole pending video, detect bounding box and the feature of face Point；

2) according to bounding box and characteristic point position, human face region interested is divided into multiple triangle, And calculate the target characteristic point of each characteristic point；

3) to each triangle, there are the coordinate of three former characteristic points and the former characteristic point of these three in step 2) In the target characteristic point that obtains, calculate the affine transformation square of this triangle according to the coordinate of these six characteristic points Battle array, does affine transformation according to each point in the triangle that three former characteristic points are formed by affine transformation matrix, Produce the facial image made new advances；

4) replace old facial image with new facial image, i.e. generate result video.

Automatic face profile method of modifying in a kind of video the most according to claim 1, its feature exists In, step 1) in, the characteristic point of face refers to the characteristic point on face profile, eyes, nose and lip.

Automatic face profile method of modifying in a kind of video the most according to claim 1, its feature exists In, step 2) in, human face region interested is face profile peripheral region.

Automatic face profile method of modifying in a kind of video the most according to claim 1, its feature exists In, step 3) in, characterizing with R matrix and rotate and scaling, L matrix characterizes translation, to a triangle, Have:

$R=\left[\begin{array}{cc}{r}_{00}& r_{01}\\ r_{10}& r_{11}\end{array}\right]$

$L=\left[\begin{array}{c}{l}_{00}\\ l_{01}\end{array}\right]$

(x, y) is certain apex coordinate before affine transformation, after (x ', y ') is affine transformation on diabolo three summits The position of representative points, has:

$\left[\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right]=\left[\begin{array}{c}{r}_{00}x+r_{01}y+l_{00}\\ r_{10}x+r_{11}y+l_{01}\end{array}\right]$

6 amounts characterizing affine transformation in this R matrix and L matrix are calculated by these three summit, according to The coordinate of these six characteristic points calculates the affine transformation matrix of this triangle.

Automatic face profile method of modifying in a kind of video the most according to claim 1, its feature exists In, step 3) in, before doing affine transformation according to each point in affine transformation matrix diabolo, first To a triangle ABC, s=CB × AB and w=(MA × MB) & (MB × MC) & (MC × MA) is utilized to judge a point Whether M is in triangle；

As s > 0 time, represent triangle ABC order be counterclockwise, at this moment w > 0 represent some M three In dihedral；As s, < when 0, representing that the order of triangle ABC is clockwise, at this moment w < 0 represents some M In triangle；

I.e. to arbitrary M point in triangle, diabolo ABC, have:

((MA × MB) & (MB × MC) & (MC × MA)) (CB × AB) ＞ 0

These points are done affine by all of point in finding out triangle according to above-mentioned formula according to affine transformation matrix Conversion, the image after being converted, it is and produces the facial image made new advances.