JP2014068280A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce malfunction of an imaging apparatus caused by accuracy shortage of eye line detection, in the imaging apparatus which controls imaging on the basis of a result of the detection of the eye line of a subject.SOLUTION: The imaging apparatus comprises: face detection means for detecting the face of a subject; face angle detection means for detecting the angle of the face detected by the face detection means with respect to the imaging apparatus; and eye line detection means for detecting the direction of an eye line relating to each eye of the face detected by the face detection means. If the angle of the face detected by the face angle detection means with respect to the imaging apparatus is larger than a predetermined angle θ1, the detection of the eye line by the eye line detection means is not performed. If the angle of the face detected by the face angle detection means with respect to the imaging apparatus is larger than a predetermined angle θ2, conditions of the detection of the eye lien by the eye line detection means are changed and the eye line is detected.

Description

  The present invention relates to an imaging apparatus including an imaging element and a control method thereof.

  In an image obtained by photographing a person as a subject, an image photographed in a state where the subject's line of sight faces the imaging device is more preferred. However, it is often difficult to take a picture of the camera, such as taking a picture of a child who does not stay still.

  Patent Document 1 discloses a technique for extracting and recording an image of a subject taken from a plurality of captured images while looking at the camera.

JP 2007-96440 A

  Patent Document 1 discloses a method for determining whether a face image of a subject is a camera eye image even when the face of the subject faces obliquely sideways. In Patent Document 1, the eye line is determined without distinguishing the left and right eyes of the subject's eyes.

  As shown in FIG. 1, as the face angle θ increases in the horizontal direction from the front where the subject's face is directly facing the imaging device, the image extracted from the subject's eyes is in a state in which it is collapsed in the horizontal direction. Further, the eyes on the side farther from the imaging device as it is turned sideways collapse in the sideways direction even at a smaller angle than the eyes on the near side. For this reason, there is a problem in that the determination accuracy of the black eye position in the horizontal direction is lowered and erroneous detection is likely to occur. Furthermore, in a subject with a high nose and deep carving, there is a concern that the eyes on the far side are easily hidden, and the line of sight cannot be determined.

  Therefore, an object of the present invention is to perform imaging control by performing eye-gaze detection and eye-gaze determination with higher reliability even in a state where the subject is in a sideways direction with respect to the image-gearing apparatus. An object of the present invention is to provide an imaging apparatus capable of reducing malfunctions and performing highly practical shooting control.

  In order to solve the above-described problems, an imaging apparatus according to the present invention includes a face detection unit that detects a face of a subject, a face angle detection unit that detects an angle of the face detected by the face detection unit, and the face detection. An eye-gaze detecting unit that detects a gaze direction for each eye of the face detected by the means, wherein the angle of the face detected by the face angle detecting unit is greater than a predetermined angle θ1 If the angle of the face detected by the face angle detection means is larger than a predetermined angle θ2, the eye detection condition in the eye detection means is changed. Eye line detection is performed.

According to the imaging device of the present invention,
In an imaging device that performs shooting control based on the detection result of the subject's line of sight, it is highly practical by reducing the malfunction of the imaging device due to insufficient accuracy of eye-gaze detection by performing shooting control with more reliable eye-gaze detection and eye-gaze determination. It is possible to perform advanced shooting control.

It is a figure which shows how the eye | texture | subject of the to-be-photographed object which looks at the slant side. It is a figure which shows schematic structure of the imaging device of this embodiment. It is an imaging operation | movement flowchart of the imaging device of this embodiment. The setting of the weighting coefficient to the eye-gaze detection result with respect to the face angle θ is shown.

  Hereinafter, the present invention will be described in detail with reference to the drawings.

  FIG. 2 is a diagram showing a schematic configuration of an imaging apparatus to which the embodiment of the present invention is applied.

  The optical system 201 images a subject by forming incident light on a solid-state image sensor. The optical system control unit 202 controls exposure, zoom, focus, optical blur correction, and the like of the optical system 201. The solid-state image sensor 203 converts the optical image formed into an electric signal. The imaging system control unit 204 is a control system for driving the solid-state imaging device. The analog signal processing unit 205 performs processing such as clamping and gaining on the output of the solid-state imaging device. An analog / digital (A / D) conversion unit 206 performs digital conversion on an analog signal and outputs it as image data.

  The digital signal processing unit 207 generates an output image from the A / D converted digital signal and performs development processing. The internal storage unit 208 temporarily stores image data when an output image is generated by the digital signal processing unit, image data for EVF operation for displaying a live image, and the like. Further, it is also used for temporarily storing various calculation data by the face detection unit 212 and the eye-line detection unit 214 described later. An interface (I / F) unit 209 is connected to an external storage device for finally storing the generated image data. The image display unit 210 has a function (EVF operation) as an electronic viewfinder that displays a generated image as a live image.

  The face detection unit 212 detects a person's face from the captured image, and further detects the position of the eye in the detected face image. A known method may be used for the face detection method. Here, it is assumed that face detection is performed by template matching of face feature points, and position information of the face, face size, and eye, nose, and mouth are obtained. Further, the template of the feature points of the face includes a side face, and it is possible to detect a face of the side face.

  The face angle detection unit 213 performs angle detection with respect to the imaging apparatus for each face detected by the face detection unit 212. As shown in FIG. 1, the face angle is based on the direction facing the imaging apparatus. The face angle detection method is realized by preparing a plurality of templates having different face angles for the face feature point templates, searching for the template with the highest similarity, and referring to the corresponding face angle information. The

  The eye line detection unit 214 detects the eye line from the image of the eye part detected by the face detection unit 212 by a method described later.

  The system control unit 211 controls the entire system such as controlling the imaging system control unit 204 in order to perform desired photographing from the face detection result and the eye detection result.

  FIG. 3 is a shooting operation flowchart of the imaging apparatus to which the embodiment of the present invention is applied. FIG. 4 is a diagram showing the setting of the weighting coefficient for the eye-gaze detection result with respect to the face angle θ.

  In step S <b> 301, the face detection unit 212 detects the face and detects the position of the detected face eye from the image signal generated by the digital signal processing unit 207.

  In step S302, the face that is a candidate for the main subject is narrowed down based on the position and size information of the face detected in step S301. Specifically, the face is selected in a preferential order from a face that is larger than a predetermined size and close to the center of the screen. Here, it is not always necessary to narrow down to one face.

  In step S303, if there is a target face of the main subject candidate narrowed down in step S302, the process proceeds to step S304, and if not, the process proceeds to step S311.

  In step S304, the face angle detection unit 213 detects the face angle of the target face of the main subject candidate narrowed down in step S302.

  In step S305, it is checked whether there is a target face of the main subject candidate whose face angle detected in step S304 is a predetermined value θ1 or less (or −θ1 or more), and the face angle detected in step S304 is a predetermined value θ1 or less (or −θ1 or more). If there is a target face that is a candidate for the main subject, the process proceeds to step S306; otherwise, the process proceeds to step S313.

  In step S306, for each target face of the main subject candidate whose face angle detected in step S304 is a predetermined value θ1 or less (or −θ1 or more), eye-gaze detection conditions corresponding to the detected face angles are set. In step S307, eye line detection processing is performed. The eye line detection is performed by template matching in the eye area detected in step S301, and reliability information for the eye line detection result is simultaneously obtained from the degree of similarity with the template.

  Here, FIG. 1 illustrates a case where the face is front-facing and the face is slightly sideways with respect to the imaging apparatus, and the line of sight is facing the imaging apparatus. FIGS. 1C and 1D are eye regions. Is an enlarged version. As shown in FIG. 1D, when the subject's face is slightly sideways with respect to the imaging device, the width of the eye far from the imaging device is smaller than the width of the near eye (Wnear> Wfar). ) The width of the white eye is also smaller. For this reason, templates for the right eye and the left eye at each face angle with the line of sight facing the imaging device are prepared as eye templates used in the line of sight detection in step S307.

  Further, as described above, the eye width is smaller in the eyes farther from the imaging device even when the face angle is the same, so that the detection accuracy of eye-gaze detection in the eyes farther from the imaging device is reduced in detection by template matching. Concerns arise. Accordingly, in step S306, the eye-gaze detection condition in step S307 is changed depending on whether the face angle detected in step S304 is equal to or smaller than the predetermined value θ2 (θ1> θ2). For example, the contrast of the eye image cut out for detection is adjusted in order to clarify the boundary between black eyes and white eyes. In step S308, which will be described later, a detection accuracy measure is taken by weighting the detection result.

  In step S308, as shown in FIG. 4, the eye-gaze detection result in step S307 is weighted according to the face angle detected in step S304. Specifically, for each of the left and right eyes, the reliability information obtained in the eye detection in step S307 is multiplied by the weighting and then added to obtain the eye detection evaluation value for each target face of the main subject candidate. calculate.

  This is set to avoid the above-described decrease in eye-gaze detection accuracy, and in the face angle range (−θ3 <θ <θ3) that is substantially in the front direction, the weights for the left and right eye-gaze detection results are set to be equal. As the angle becomes horizontal, setting is made so that the detection result closer to the imaging device is more important. As described above, eye-gaze detection is not performed at an angle larger than θ1.

  In step S309, it is checked whether there is a target face of the main subject candidate whose eye-eye detection evaluation value calculated in step S308 is equal to or greater than a predetermined value Ths. If there is a target face of the main subject candidate that is equal to or greater than the predetermined value Ths, step S310 is performed. Otherwise, go to step S313.

  In step S310, for the target face of the main subject candidate, review the ranking of the main subject face by adding the eye-gaze detection evaluation value information calculated in step S308 to the position and size information of the face detected in step S301. I do.

  In step S311, since there is no face that is a candidate for the main subject, AF is performed at the center of the screen, and normal imaging operations such as center-weighted exposure control are performed.

  In step S312, based on the face information of the main subject candidates ordered in step S302, for example, an imaging operation at the time of face detection is performed such as adjusting the focus and exposure of the main subject face.

  In step S313, based on the main subject face information reviewed in step S310, a predetermined imaging operation at the time of eye line detection is performed. For example, the eye is focused on a certain eye, the aperture is set close to the open position so that the depth of field is very shallow, and shooting is performed so that the eye is more impressive.

  As described above, according to the imaging apparatus to which the present invention is applied, the imaging operation is switched based on the reliable eye-gaze detection result even in a slightly horizontal subject face. Is possible.

201 Optical system 202 Optical system control unit 203 Imaging device 204 Imaging system control unit 205 Analog signal processing unit 206 A / D conversion unit 207 Digital signal processing unit 208 Internal Storage unit 209 I / F
210... Image display unit 211... System control unit 212 .. Face detection unit 213 .. Face angle detection unit 214.

Claims (3)

Face detecting means for detecting the face of the subject, face angle detecting means for detecting an angle of the face detected by the face detecting means with respect to the imaging device, and detecting the eye direction for each eye of the face detected by the face detecting means An eye-gaze detecting means,
When the angle of the face detected by the face angle detection means with respect to the imaging device is larger than a predetermined angle θ1, the eye detection by the eye detection means is not performed,
In addition, when the angle of the face detected by the face angle detection unit with respect to the imaging device is larger than a predetermined angle θ2, the eye detection is performed by changing the eye detection condition in the eye detection unit. The imaging apparatus according to claim 1, wherein the θ1 is set to be larger than the θ2. When the face angle detection result is smaller than the predetermined angle ± θ3, the eye detection result in the eye located closer to the imaging device is set to be more weighted than the eye detection result detected by the eye detection means. The imaging apparatus according to claim 1, wherein a weight of the eye detection result in an eye located on a side farther from the imaging apparatus is set to be small.