JP3459950B2 - Face detection and face tracking method and apparatus - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face detection and face tracking method and apparatus for detecting or tracking a human face at high speed based on a color image input from a color image input means such as a video camera. Is.

[0002]

2. Description of the Related Art A method for detecting or tracking a face by processing a digital image with a computer has been studied. As a research on face detection using monochrome images, for example, "Finding and locating faces in a scene using multiple pyramids for personal identification" by Kosugi
(The Institute of Electronics, Information and Communication Engineers, Vol.J77-D-II, No.4, pp672-
681, 1994). For research on face detection using color images, see Face Detection by Q. Chen et al.
fuzzy pattern matching '' (Proceedings of 5th Interna
tional Conference on Computer Vision, pp 591-596).

[0003]

However, face detection using the monochrome image described above has a problem that face detection tends to be unstable because the face color cannot be detected. On the other hand, in the face detection using the conventional color image, since the matching is performed using the face template, it is necessary to prepare many face templates in advance, and in actual detection, many positions are detected. In the above, since the correlation calculation between the face template and the actual image has to be performed, the time required for detection is usually longer than 2 seconds,
It was not suitable for real-time detection.

Further, in the method using the face template, the detection accuracy may be affected by the brightness of the digital image, the size of the face, the hair style, the presence or absence of wearing glasses, and the like. If the face orientation is horizontal other than the front, etc., if the head is tilted even if the face is facing the front, or if the digital image contains multiple faces, In some cases, accurate detection could not be performed.

[0005]

SUMMARY OF THE INVENTION The present invention solves the above problems and provides a face detection and face tracking method and apparatus capable of detecting or tracking a human face at high speed and accurately based on a color image. The purpose is to provide. for that reason,
According to the face detection method of claim 1 of the present invention, a color image input from the color image input means and represented by the RGB color system is x.
y Skin color sample image prepared in advance by converting to y color system
For the color x in the xy color system
According Gaussian function representing the skin color distribution extracting threshold following dark area having a brightness of the color image extracts a <br/> skin color region from the color image, the skin color area
And a window whose area is approximately equal to each other and whose centers of gravity coincide with each other,
Is divided into upper and lower parts, and the upper half is further divided into left and right parts
Split into upper left window, upper right window, and lower window, above left
The dark areas are displayed in the upper window, the upper right window, and the lower window, respectively.
Area exists and the center of gravity of the dark area is the upper left window,
When the dark area is located in the upper right window and the lower window, and the dark area is detected in the skin color area at a constant area ratio, it is determined to be a human face. To do.

A face tracking method according to a second aspect of the invention is characterized in that the movement of a human face is tracked by continuously performing face detection by the method according to the first aspect in a plurality of frames of the color image. Is.

In the face tracking method according to a third aspect of the present invention, when the face tracking is performed by the method according to the second aspect, the face detection in each frame is performed only in an area near the area in which the face is detected in the previous frame. It is a feature.

A recording medium according to a fourth aspect is a computer-readable recording medium recording a program for executing the face detecting method or the face tracking method according to any one of the first to third aspects.

A face detecting apparatus according to a fifth aspect of the present invention is a color image input from a color image input means and represented by an RGB color system.
Is converted to the xy color system and the skin color sample prepared in advance
For the color x in the xy color system determined from the image,
The above color image according to the Gaussian function that represents the distribution of all skin colors
From the skin color area detecting means for extracting a skin color area from, a dark area detecting means for extracting a dark area whose brightness is below a certain threshold from the color image, and an area of the skin color area detected by the skin color area detecting means. The flesh color area analyzing means to be obtained, the dark area analyzing means to obtain the area of the dark area detected by the dark area detecting means, and the flesh color area and the area are substantially equal to each other.
And set a window whose center of gravity is the same, and divide the window into two up and down.
In addition, the upper half is further divided into upper and lower parts,
Divided into upper window, upper right window, and lower window, above left window
The dark areas in the window, the upper right window, and the lower window, respectively.
Is present and the center of gravity of the dark area is the upper left window and upper window, respectively.
In the upper right window and the lower window , a dark area exists in the skin color area based on the analysis results of the skin color area analysis means and the dark area analysis means, and its area It is characterized by comprising a face determining means for determining that the face is a human face when the ratio is within a certain range.

According to a sixth aspect of the present invention, in the structure of the fifth aspect, the skin color area analyzing means and the dark area analyzing means respectively obtain the barycenters of the skin color area and the dark area, respectively.
The face determining means sets a rectangular window centered on the center of gravity of the flesh color area and having an area substantially equal to the area of the flesh color area on the flesh color area, and this rectangular window is an upper window and a lower window consisting of an upper left window and an upper right window. And the center of gravity of the dark area is included in each of the upper left window, the upper right window, and the lower window, and the area ratio of the dark area to the skin color area in the upper left window, the upper right window, and the lower window is within a certain range. It is characterized in that it is determined to be the face of a person when it is inside.

That is, since the dark area in the human face is mainly around the eyes (particularly, black eyes and eyebrows) and around the mouth (mouths and lips), the above-mentioned upper left window for the left and right eyes. In addition to setting the upper right window and the upper right window, a lower window is set for the mouth, and it is determined whether or not dark areas are included in the upper left window, the upper right window and the lower window at a constant area ratio. Is.

According to a seventh aspect of the present invention, in the configuration of the sixth aspect, when at least one of the upper left window and the upper right window does not include the center of gravity of a dark area, the face determination means opens the upper left window and the upper right window. The determination is performed by sliding the skin color region in the left-right direction. That is, when a person's face is turned sideways diagonally in the color image, the center of gravity of the dark areas corresponding to the left and right eyes may shift to the left and right. This is handled by sliding the skin color area to the left and right.

The face tracking device according to claim 8 is the face tracking device according to any one of claims 5 to 7.
In any of the above configurations, the movement of the face is tracked by continuously performing face detection by the face detection apparatus according to any one of claims 5 to 7 in a plurality of frames.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. The face detection and tracking device of this embodiment is a human interface, database search, building monitoring, security, video conference, face matching system, virtual (artificial) reality, game, education, movie production, entertainment, image communication, etc. Can be used in various fields.

As shown in FIG. 1, a face detection and tracking device 1
Is a flesh color area detecting means 3 for detecting a flesh color area in the color image input from the color image input means 2, a dark area detecting means 4 for detecting a dark area in the color image,
Skin color region analysis unit 5 for obtaining the area and center of gravity of the skin color region detected by the skin color region detection unit 3, and dark region detection unit 4
Whether or not a human face is present in the color image based on the analysis results of the dark area analysis means 6 for obtaining the area and the center of gravity of the dark area detected in step 6, and the skin color area analysis means 5 and the dark area analysis means 6. And a face determination means 7 for determining whether or not.

FIG. 2 is a block diagram showing a more detailed structure of the face detection and tracking device 1 and the color image input means 2 of FIG. The color image input unit 2 includes an image pickup unit 21 and an A / D conversion unit 22 provided when the image pickup unit 21 outputs an analog signal. When face tracking is performed by the face detection and tracking device 1, for example, CCD (Charge Coupled Dev)
ice) A full-color moving image picked up by the image pickup means 21 including a video camera is converted into a digital signal by the A / D converter 22, and then sent to the skin color area detection means 3 of the face detection and tracking device 1. On the other hand, when the face detection and tracking device 1 performs face detection, for example, a color still image (digital signal) captured by the image capturing unit 21 including a digital camera is sent to the skin color region detecting unit 3 as it is.

The skin color area detecting means 3 comprises a first converting means 31,
The second conversion unit 32 and the skin color pixel detection unit 33 are included. The color image input from the image pickup means 21 such as the CCD video camera or the digital camera is usually N
Although it is expressed in the RGB (red, green, blue) color system of the TSC (National Television System Committee) system, in the present embodiment, this RGB color system is first converted into CIE (international) by the first conversion means 31. (Certification Committee) Convert to XYZ color system. The conversion formula (1) from the RGB color system to the XYZ color system is shown below.

[0018]

[Equation 1]

Next, the XYZ color system is converted into the xy color system by the second conversion means 32 in the skin color area detection means 3. That is, the brightness (brightness) changes only when X, Y, and Z are doubled and tripled, respectively. Therefore, if only colors are expressed regardless of brightness, the ratio of X, Y, and Z is calculated. You only need to know The conversion formula (2) from X, Y, Z to x, y is as follows. As described later, the brightness is XYZ.
It is represented by Y in the color system.

[0020]

[Equation 2]

After the conversion into the xy color system by the second conversion means 32, the skin color pixel detection means 33 detects the area of the skin color pixels. Here, although the skin color (face color) has individual differences, it is considered that the same race follows a normal distribution. Therefore, the present inventor investigated the skin color distribution situation of Oriental people. That is, face images for several people were acquired using a video camera, and only the skin color portion was manually cut out to prepare a skin color sample image. The two-dimensional normal distribution function is expressed by the following equation (3) in equation (3).

[0022]

[Equation 3]

In Equation 3, bold x is a color of the xy color system, bold μ is the center of the skin color distribution, and bold Σ is the covariance matrix of the skin color distribution. Bold x, bold μ and The bold letters Σ are expressed by equations (4), (5), and (6) in the equation 4, respectively.

[0024]

[Equation 4]

Using the xy color information of the obtained skin color sample image, and letting the number of pixels of the skin color sample image be n,
The center μ (bold type) and the covariance matrix Σ (bold type) can be calculated by the equations (7) and (8) in the equation 5, respectively.

[0026]

[Equation 5]

The Mahalanobis distance D in the exponential part of the normal distribution function is expressed by the equation (9) in the equation 6, and this Mahalanobis distance D is used for extracting the skin color area from the color image. When the value of the Mahalanobis distance D is less than a certain threshold value, the flesh color pixel detection unit 33 determines that the color represented by bold x is a flesh color, and if D is equal to or more than the threshold value, the color is not a flesh color. to decide.

[0028]

[Equation 6]

Further, the dark area detecting means 4 binarizes the luminance (brightness) of Y of the XYZ color system sent from the first converting means 31 and compares this with a threshold value to determine the luminance. If it is less than or equal to the threshold value, it is determined to be a dark pixel. As described above, the skin color area analysis unit 5 and the dark area analysis unit 6 obtain the areas and the centers of gravity of the skin color area and the dark area, and transmit these data to the face determination unit 7.

The face determining means 7 determines that the flesh color area is a human face when the flesh color area includes a dark area in a certain area ratio. That is, the dark areas in a human face are mainly around the left and right eyes (black eyes, eyebrows, etc.) and around the mouth (mouth, lips, etc.), and the left and right eyes are the upper part of the face,
The mouth is located at the bottom of the face. Therefore, in the present embodiment, whether or not the skin color area is a human face is determined by whether or not there are dark areas corresponding to the eyes and the mouth above and below the skin color area detected in the color image. To judge.

More specifically, as shown in part A of the face determining means 7 in FIG. 2, the face determining means 7 has a square window M so as to overlap with the skin color region H detected in the color image. To set. In that case, the area of the window M is made substantially equal to the area of the skin color region H, and the center of gravity of the window M and the center of gravity of the skin color region H are matched. The window M is vertically divided into two parts, and the upper half is further divided into left and right parts into an upper left window M1, an upper right window M2 and a lower window M3. Then, the upper left window M1 and the upper right window M2
And lower windows M3 have dark areas respectively, and the centers of gravity of these dark areas are respectively located in the upper left window M1, the upper right window M2 and the lower window M3, and the skin color area of the area of these dark areas is present. If the ratio of H to the area is within a certain range, it is determined to be a human face.

That is, the dark area in the skin color area H in the upper left window M1 and the upper right window M2 is 1 with respect to the skin color area H.
If it exists in the area ratio of / 900 to 1/40, it is determined that these dark areas are around the left eye and the right eye (especially the black eye and the eyebrow), and the dark area in the skin color area H in the lower window M3. Is present at an area ratio of 1/200 to 1/5, it is determined that the dark region is around the mouth, and the dark regions in the upper left window M1, the upper right window M2, and the lower window M3 are respectively the above-mentioned. When it exists in proportion, it is determined that the skin color region H is a human face.

Further, as shown in the portion B or the portion C in the face determining means 7 in FIG. 2, when the human face is facing diagonally forward in the color image, the eye position in the face is left and right. In this case, the upper left window M1 and the upper right window M2
To the left and right, and then slide it to the upper left window M
1 and the upper right window M2, it is determined whether or not each of the dark areas is included in a certain area ratio, and the upper left window M1 and the upper right window M2 are determined.
If a dark area is included in the area within a certain area ratio, it is determined to be a human face.

When face detection and tracking device 1 performs face detection based on a color still image, it can be detected by the above procedure, but when face tracking based on a color moving image is performed, color image input means 2 is used. Face tracking may be performed by performing face detection in each of the plurality of frames input to the skin color region detection unit 3 and the dark region detection unit 4 in the above procedure. In that case, the movement of the position of the face between the adjacent frames is relatively small. Therefore, as shown in FIG. 3, the face detection in each frame is performed in the area F1 in which the skin color area is detected in the previous frame. By performing only in the neighboring rectangular area F, the time required for face detection can be further shortened. Note that F2 in FIG. 3 indicates the skin color region detected in the current frame.

FIG. 4 is a flow chart showing a procedure of face detection or face tracking by the face detection and tracking apparatus 1. Since it overlaps with the above description, it will not be described in detail, but S1 to S9.
Each step of is executed sequentially. In S9, the position of the face is displayed. If the application is, for example, building monitoring, the face of the detected person is displayed on the monitor for monitoring, and if the application is a video conference, it is displayed for the conference. It is what is displayed on the monitor.

When the face detection and tracking device 1 of the present embodiment performs face tracking while displaying the position of the face on a monitor or the like, the image pickup means 21, for example, a video camera,
It is preferable to use one that can automatically control the shooting direction and zoom. That is, the video camera should be able to rotate (pan) and swing vertically (tilt) in the horizontal plane, and change the shooting direction of the video camera according to the movement of the face position in the color image. By controlling the face in the color image always near the center of the color image by changing the zoom according to the change in the area of the skin color region in the color image, It is preferable that the area is always within a certain range.

The inventor conducted an experiment to detect a human face using the face detection and tracking device 1. In the experiment, a CCD video camera was used as the color image input means 2 and a personal computer (Gateway 2000) was used as the face detection and tracking device 1. The color image board used was Linx DT3153. The input image was full color and the image size was 320 x 240 pixels.

In the above experiment, the face is directed diagonally right or left, the distance between the face and the video camera is changed, a plurality of faces are photographed simultaneously, glasses are worn, and the face is exposed under various conditions. I detected it, but the face was not detected,
Only 2 out of 41 people. The cause was that these two had hair on their eyes. Further, the time required for face detection is about 0.2 seconds, and the time required for face tracking is about 0.1 seconds, which is significantly shorter than in the past.

It should be noted that the computer program for executing the face detection and tracking method described in the present embodiment by using a personal computer or the like is recorded in various recording media such as a CD-ROM and sold or used. Are naturally included within the scope of implementation of the present invention.

[0040]

As described above, according to the first aspect of the present invention.
The face detection method of R
Converts color images represented in GB color system into xy color system
Then, it is determined from the skin color sample image prepared in advance, and
Note that the xy color coordinate system represents the skin color distribution for color x.
The skin color area is extracted from the color image according to the Us function and the dark area whose brightness is lower than a certain threshold is extracted from the color image, and the area is almost equal to the skin color area.
And set a window whose center of gravity is the same, and divide the window into two up and down.
At the same time, the upper half is further divided into two parts, left and right,
Divided into upper right window and lower window, above left window, above right window
The dark area is present in the window and in the lower window, and
The center of gravity of the dark area is the upper left window, the upper right window, and
And the lower window, and when the dark area is detected in the skin color area at a constant area ratio, it is determined that the face is a human face. Therefore, a conventional color image is used. Unlike the case of face detection, it is not necessary to repeatedly perform collation calculation with a face template prepared in advance at many positions, and the time required for detection can be significantly shortened, so face detection in real time is possible. In addition, when a dark area (around the eyes, around the mouth, etc.) is included in the skin color (face color) area at a certain area ratio, it is determined to be a face. The accuracy is sufficiently high, and the detection accuracy does not decrease depending on the size and orientation of the face, whether or not the head is tilted, etc., and it is possible to detect even if multiple faces are included in the color image. Is.
Further, if a certain width is given to the area ratio of the dark area to the skin color area, the face can be accurately detected regardless of whether or not the eyeglasses are worn.

In addition, a dark area (around the eyes,
Since it is determined that there is something around the mouth, etc., and information about the hair located outside the skin color area is not used for face detection, face detection is performed without being affected by the amount of hair or hair style. be able to. Furthermore, a Gaussian function is used to detect the flesh-colored area, and if the distance to the center of the Gaussian function is less than a certain threshold value, it is determined to be flesh-colored, and the dark area is also detected as brightness (luminance). Since it is performed based on whether or not the threshold value is less than or equal to a certain threshold, face detection can be accurately performed without being affected by the brightness of a color image used for detection, individual differences in face color, and the like.

In the face tracking method according to the second aspect, the movement of the human face is tracked by continuously performing the face detection by the method according to the first aspect in a plurality of frames of the color image. Face tracking inside can be done accurately in real time.

In the face tracking method according to the third aspect, when performing the face tracking by the method according to the second aspect, the face detection in each frame is performed only in the area near the area where the face is detected in the previous frame. Since the movement of the position of the face is usually small between the adjacent frames, the face can be detected by performing detection in the vicinity of the area where the face was detected in the previous frame in the new frame. Therefore, by limiting the detection area to the vicinity of the area where the face is detected in the previous frame, the time required for detection can be further shortened, which is suitable for face tracking in real time. .

A recording medium according to a fourth aspect is a computer-readable recording medium (CD-ROM or the like) recording a program for executing the face detecting method or the face tracking method according to any one of the first to third aspects. The face detection method or the face tracking method according to the present invention can be executed by installing the program from the recording medium to the computer.

A face detecting apparatus according to a fifth aspect of the present invention is a color image input from a color image input means and represented by an RGB color system.
Is converted to the xy color system and the skin color sample prepared in advance
For the color x in the xy color system determined from the image,
The above color image according to the Gaussian function that represents the distribution of all skin colors
From the skin color area detecting means for extracting a skin color area from, a dark area detecting means for extracting a dark area whose brightness is below a certain threshold from the color image, and an area of the skin color area detected by the skin color area detecting means. The flesh color area analyzing means to be obtained, the dark area analyzing means to obtain the area of the dark area detected by the dark area detecting means, and the flesh color area and the area are substantially equal to each other.
And set a window whose center of gravity is the same, and divide the window into two up and down.
In addition, the upper half is further divided into upper and lower parts,
Divided into upper window, upper right window, and lower window, above left window
The dark areas in the window, the upper right window, and the lower window, respectively.
Is present and the center of gravity of the dark area is the upper left window and upper window, respectively.
In the upper right window and the lower window , a dark area exists in the skin color area based on the analysis results of the skin color area analysis means and the dark area analysis means, and its area Since it is provided with a face determination unit that determines that the face is a human face when the ratio is within a certain range, the time required for detection can be significantly increased as compared with the face detection method using the conventional face template. Since it can be shortened, face detection can be performed in real time, and the detection accuracy is sufficiently high.It can be detected depending on the size and orientation of the face, whether or not the neck is tilted, whether glasses are worn, etc. The accuracy does not decrease,
It is possible to detect even when a plurality of faces are included in the color image.

In addition, a dark area (around the eyes,
Since it is determined that there is something around the mouth, etc., and information about the hair located outside the skin color area is not used for face detection, face detection is performed without being affected by the amount of hair or hair style. If the distance to the center of the Gaussian function is less than a certain threshold, it is determined that the color is flesh color, and the dark area is also bright. Since it is performed based on whether or not the threshold value is less than or equal to a certain threshold, face detection can be accurately performed without being affected by the brightness of a color image used for detection, individual differences in face color, and the like.

According to a sixth aspect of the present invention, in the structure of the fifth aspect, the flesh color area analyzing means and the dark area analyzing means determine the barycenters of the flesh color area and the dark area, respectively.
The face determining means sets a rectangular window centered on the center of gravity of the flesh color area and having an area substantially equal to the area of the flesh color area on the flesh color area, and this rectangular window is an upper window and a lower window consisting of an upper left window and an upper right window. And the center of gravity of the dark area is included in each of the upper left window, the upper right window, and the lower window, and the area ratio of the dark area to the skin color area in the upper left window, the upper right window, and the lower window is within a certain range. When it is inside, it is determined that it is the human face, and it is the dark areas in the human face, around the left and right eyes (black eyes and eyebrows) and around the mouth (mouth and lips).
The upper left window, the upper right window and the lower window are set for and, respectively, and the human face is determined based on whether or not the dark areas are included in the upper left window, the upper right window and the lower window at a constant area ratio. Since it is determined whether or not there is a face, face detection can be accurately performed based on the presence or absence of these eyes and mouth.

According to a seventh aspect of the present invention, in the configuration of the sixth aspect, when at least one of the upper left window and the upper right window does not include the center of gravity of a dark area, the face determination means opens the upper left window and the upper right window. Since the determination is made by sliding the skin color region in the left-right direction, for example, the face of a person is diagonally sideways in the color image, or the neck is tilted to correspond to the left and right eyes. Even if the center of gravity of the dark area is shifted to the left and right of the face, it is possible to detect the face by sliding the upper left window and the upper right window to the left and right with respect to the skin color area. The person's face can be accurately detected without being affected by the inclination or the like.

A face tracking device according to claim 8 is the face tracking device according to any one of claims 5 to 7.
In any one of the above configurations, the face movement is tracked by continuously performing the face detection by the face detection apparatus according to any one of claims 5 to 7 in a plurality of frames. Can be done accurately in real time. In this case, the position of the face in the moving image is constantly changed by automatically changing the direction (imaging direction) of the image pickup unit in the color image input unit according to the movement of the position of the person's face in the moving image. It is preferable to control so as to be located near the center of the moving image.

[Brief description of drawings]

FIG. 1 is a schematic block diagram showing a face detection and tracking device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a more detailed configuration of the face detection and tracking device.

FIG. 3 is an explanatory diagram showing a face detection range in each frame when face tracking is performed by the face detection and tracking device.

FIG. 4 is a flowchart showing a processing procedure by the face detection and tracking device.

[Explanation of symbols]

1 Face detection and tracking device 2 Color image input means 3 Skin color area detection means 4 Dark area detection means 5 Skin color area analysis means 6 Dark area analysis means 7 Face determination means

─────────────────────────────────────────────────── --Continued from the front page Application for application of Article 30, Paragraph 1 of the Patent Act Yes, published in Nikkei Sangyo Shimbun, published on February 17, 1998. (56) References JP-A-6-153047 (JP, A) JP-A-6-303601 (JP, A) JP-A-7-311833 (JP, A) JP-A-9-81732 (JP, A) Kaihei 7-44706 (JP, A) JP 7-29014 (JP, A) JP 9-44685 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 7 / 00-7/60 G06T 1/00

Claims (8)

(57) [Claims] 1. An RG input from a color image input means.
Converts a color image represented by B color system to xy color system
Then, it is determined from the skin color sample image prepared in advance, and
Note that the xy color coordinate system represents the skin color distribution for color x.
Accordance mouse function extracts the threshold or less in the dark areas with the brightness of the color image extracts a skin color region from the color image, the skin color area and the area substantially equal and the center of gravity coincides window
Set the window and divide the window into upper and lower parts and expose the upper half.
It is divided into two parts, left and right, and divided into an upper left window, an upper right window, and a lower window.
Split into the upper left window, the upper right window, and the lower window, respectively.
There is a dark area and the center of gravity of the dark area is on the left side.
Located in the upper window, the upper right window, and the lower window,
Moreover , when the dark area is detected in the skin color area at a constant area ratio, it is determined that the face is a human face. 2. A face tracking method characterized by tracking the movement of a person's face by continuously performing face detection by the method according to claim 1 in a plurality of frames of the color image. 3. The face tracking method according to claim 2, wherein face detection in each frame is performed only in a region near a region where a face is detected in a previous frame. 4. A computer-readable recording medium in which a program for executing the face detecting method or the face tracking method according to claim 1 is recorded. 5. An RG input from a color image input means.
Converts a color image represented by B color system to xy color system
Then, it is determined from the skin color sample image prepared in advance, and
Note that the xy color coordinate system represents the skin color distribution for color x.
A flesh color region detecting means for extracting a flesh color region from the color image in accordance with a Us function, a dark region detecting means for extracting a dark region having a brightness equal to or lower than a threshold value from the color image, and the flesh color region detecting means. Skin color area analysis means for obtaining the area of the skin color area , dark area analysis means for obtaining the area of the dark area detected by the dark area detection means, and a window whose area is substantially equal to that of the skin color area
Is set, the window is divided into upper and lower parts, and the upper half is
It is further divided into two parts, the upper left window, the upper right window, and
And the lower window, the upper left window, the upper right window, and
The dark areas are present in the lower window and are dark.
The center of gravity of the area is the upper left window, the upper right window, and the lower
When a dark region exists in the flesh color region based on the analysis result of the flesh color region analysis unit and the dark region analysis unit and the area ratio is within a certain range, A face detection apparatus comprising: a face determination unit that determines a face. 6. The skin color area analyzing means and the dark area analyzing means respectively obtain the center of gravity of the skin color area and the dark area, respectively, while the face determining means has an area centered on the center of gravity of the skin color area and substantially equal to the area of the skin color area. Rectangular window is set on the flesh-colored area, and the rectangular window is divided into an upper window and a lower window consisting of an upper left window and an upper right window, and the dark areas are respectively provided in the upper left window, the upper right window and the lower window. 6. It is determined to be a human face when the center of gravity is included and the area ratio of the dark area to the skin color area in each of the upper left window, the upper right window and the lower window is within a certain range. Face detection device described. 7. When the center of gravity of a dark area is not included in at least one of the upper left window or the upper right window, the face determining means slides the upper left window and the upper right window in the left-right direction with respect to the skin color area to perform the determination. The face detection device according to claim 6, wherein 8. A face tracking device for tracking the movement of a face by continuously performing face detection by the face detecting device according to claim 5 in a plurality of frames.