JP2016099432A - Focus detection device, and method, program and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a focus detection device capable of detecting a phase difference with small error and high precision.SOLUTION: A focus detection device comprises: a first calculation unit which calculates a first phase difference between a first image signal based upon luminous flux passed through a partial region of an exit pupil of a photographic optical system and a third signal based upon luminous flux passed through the entire region of the exit pupil; a second calculation unit which calculates a second phase difference between a second image signal based upon luminous flux passed through a partial region different from the partial region of the exit pupil and a fourth image signal based upon the luminous flux passed through the entire region of the exit pupil; a third calculation unit which calculates a defocusing amount of the photographic optical system using at least one of the first phase difference and second phase difference; and a switching unit which calculates the defocusing amount of the photographic optical system using both the first phase difference and second phase different or calculates the defocusing amount of the photographic optical system using one of the first phase difference and second phase difference in calculating the defocusing amount of the photographic optical system by the third calculation part.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a focus detection apparatus and method, and more particularly, to a focus detection apparatus and method for performing phase difference detection type focus detection based on an output of an image sensor.

  As one of methods for detecting the focus state of the photographing lens, Patent Document 1 discloses an apparatus that performs pupil division type focus detection using a two-dimensional image sensor in which a microlens is formed in each pixel. In this apparatus, the photoelectric conversion unit of each pixel constituting the imaging element is divided into a plurality of parts, and the divided photoelectric conversion unit receives light beams that have passed through different regions of the pupil of the photographing lens via the microlens. It is configured. A correlation calculation is performed on the pair of output signals of the photoelectric conversion unit that has received the light beams that have passed through different areas of the pupil of the photographic lens to calculate a phase difference that is a shift amount, and defocusing is performed from the phase difference. The amount can be calculated.

  Patent Document 2 discloses a configuration in which a part of pixels constituting an image sensor is configured as a focus detection pixel in order to similarly perform focus detection of the pupil division method. In contrast to Patent Document 1, it is necessary to correct the output of the focus detection pixel as a photographed image, but since there are few signals to be read out as a focus detection signal, it can be configured inexpensively in terms of the configuration of the image sensor and the subsequent arithmetic processing. There is a feature. In the focus detection method disclosed in Patent Document 2, a pair of focus detection signals for performing pupil division is obtained as outputs of different pixels.

JP 2008-52009 A Japanese Patent No. 3592147

  However, in the technique disclosed in Patent Document 2 described above, the phase difference between a pair of output signals often includes errors and cannot be detected accurately. For example, when focus detection pixels are arranged as shown in FIG. 2 of Patent Document 2, the positions of subjects corresponding to a pair of output signals are different. Therefore, the similarity between the pair of output signals may be low, and in such a case, focus detection cannot be performed with high accuracy.

  In addition, when focus detection pixels are arranged as shown in FIG. 6 of Patent Document 2, the sampling pitch of each of the pair of output signals is widened. Therefore, in the frequency characteristics of the optical image that is the subject, the frequency in the high frequency band Ingredients cannot be acquired. Therefore, different aliasing noises are generated in the pair of output signals, and a focus detection error occurs.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a focus detection apparatus capable of detecting a phase difference with high accuracy and less error.

  The focus detection apparatus according to the present invention includes a first image signal based on a light beam that has passed through a partial area of an exit pupil of a photographing optical system, and a third image signal based on a light beam that has passed through the entire area of the exit pupil. A first calculation means for calculating a first phase difference between the first pupil and the second image signal based on a light beam that has passed through a partial area different from the partial area of the exit pupil; The second calculation means for calculating a second phase difference with the fourth image signal based on the light flux that has passed through the entire region, and at least one of the first phase difference and the second phase difference is used to In the calculation of the defocus amount of the photographic optical system by the third calculation means for calculating the defocus amount of the photographic optical system and the third calculation means, the first phase difference and the second phase difference are calculated. The defocus amount of the photographing optical system is calculated using both, or the first phase is calculated. Characterized in that it comprises a switching means for switching whether to calculate the defocus amount of the photographing optical system by using either of the second phase difference between.

  According to the present invention, it is possible to provide a focus detection apparatus capable of detecting a phase difference with high accuracy and less error.

1 is a block diagram illustrating an example of a functional configuration of a camera system as an example of an imaging apparatus including a focus adjustment device according to an embodiment. FIG. 3 is a diagram illustrating a configuration example of an image sensor according to the first embodiment. The figure which shows the relationship between the photoelectric conversion area | region and exit pupil in 1st Embodiment. 4 is a diagram illustrating an example of a relationship between a focus detection area and pixels used for AF signals in the embodiment. 6 is a flowchart illustrating a focus adjustment operation in the embodiment. 6 is a flowchart illustrating a defocus amount calculation method according to the first embodiment. FIG. 6 is a diagram illustrating a configuration example of an image sensor according to a second embodiment. The figure which shows the relationship between the photoelectric conversion area | region and exit pupil in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram illustrating a configuration of an image pickup apparatus having a focus adjustment device (focus detection device), which is a camera system including a camera capable of exchanging a plurality of photographing lenses and the photographing lenses. In FIG. 1, the camera system including the focus adjustment apparatus of the present embodiment is configured to include a camera 100 and a photographing lens 300 that is attached to the camera 100 in an exchangeable manner. First, the configuration of the camera 100 will be described.

  The camera 100 corresponds to a camera system in which a plurality of types of photographing lenses 300 are present, and it is possible to mount lenses of the same type but having different manufacturing numbers. Furthermore, a photographic lens 300 having a different focal length or open F number, or a photographic lens 300 having a zoom function can be mounted, and can be replaced regardless of the same or different photographic lens.

  In the camera 100, the light beam that has passed through the photographing lens 300 passes through the lens mount 106, is reflected upward by the main mirror 130, and enters the optical viewfinder 104. The optical viewfinder 104 allows the photographer to take a picture while observing the subject as an optical image. In the optical viewfinder 104, some functions of the display unit 54, for example, focus display, camera shake warning display, aperture value display, exposure correction display, and the like are installed.

  A part of the main mirror 130 is constituted by a semi-transmissive half mirror, and a part of the light beam incident on the main mirror 130 passes through the half mirror part, and is reflected downward by the sub mirror 131 to the focus detection device 105. Incident. The focus detection apparatus 105 employs a phase difference detection AF mechanism having a secondary imaging optical system and a line sensor. The obtained optical image is converted into an electric signal and sent to an AF unit (autofocus unit) 42. . The AF unit 42 performs a phase difference detection calculation from this electric signal, and obtains the defocus amount and direction of the photographing lens 300. Based on the calculation result, the system control unit 50 controls the focus control unit 342 (described later) of the photographing lens 300, such as a focus adjustment process. In this embodiment, the AF detection unit 42 also corrects the focus detection result.

  When the focus adjustment processing of the photographic lens 300 is completed and still image shooting is performed, when an electronic viewfinder display is performed, or when moving image shooting is performed, the main mirror 130 and the sub mirror 131 are captured by the quick return mechanism (not shown). Evacuate outside. Thus, the light beam that passes through the photographing lens 300 and enters the camera 100 enters the image sensor 14 that converts an optical image into an electric signal via the shutter 12 for controlling the exposure amount. After completion of these photographing operations, the main mirror 130 and the sub mirror 131 return to the positions as shown in the figure.

  The imaging device 14 is a CCD or CMOS image sensor, has a configuration in which a plurality of pixels are two-dimensionally arranged, photoelectrically converts an optical image of a subject for each pixel, and outputs an electrical signal. The electrical signal photoelectrically converted by the image sensor 14 is sent to the A / D converter 16, and the analog signal output is converted into a digital signal (image data). The timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26. The timing generation circuit 18 is controlled by the memory control unit 22 and the system control unit 50. The image processing unit 20 applies predetermined processing such as pixel interpolation processing, white balance processing, and color conversion processing to the image data from the A / D converter 16 or the image data from the memory control unit 22.

  In the imaging device 14 according to the present embodiment, some pixels are configured as focus detection pixels, and even when the main mirror 130 and the sub mirror 131 are retracted out of the optical path by the quick return mechanism, focus detection by the phase difference detection method is performed. Is possible. Of the image data obtained by the image sensor 14, pixel data used for generating a focus detection signal is converted into focus detection data by the image processing unit 20. Thereafter, the focus detection data is sent to the AF unit 42 via the system control unit 50, and the AF unit 42 adjusts the focus of the photographing lens 300 based on the focus detection data.

  Note that contrast-based AF is also possible in which the image processing unit 20 calculates a contrast evaluation value from image data captured by the image sensor 14 and the system control unit 50 performs focusing on the focus control unit 342 of the imaging lens 300. It is. As described above, the camera 100 according to the present embodiment has a phase difference from the image data obtained by the image sensor 14 even when the main mirror 130 and the sub mirror 131 are retracted from the optical path as in live view display or moving image shooting. Both detection AF and contrast AF are possible. The camera 100 according to the present embodiment can perform phase difference detection AF using the focus detection device 105 in normal still image shooting that does not use live view display in which the main mirror 130 and the sub mirror 131 are in the optical path. Therefore, focus adjustment is possible in any state during still image shooting, live view display, and moving image shooting. Furthermore, since phase difference detection AF can be performed, high-speed focusing is possible.

  The memory control unit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing unit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression unit 32. Then, the data of the A / D converter 16 is sent via the image processing unit 20 and the memory control unit 22 or the data of the A / D converter 16 is sent directly via the memory control unit 22 to the image display memory 24 or the memory 30. Is written to. Display image data written in the image display memory 24 is displayed on an image display unit 28 including a liquid crystal monitor or the like via a D / A converter 26. An electronic viewfinder function (live view display) can be realized by sequentially displaying a moving image captured by the image sensor 14 on the image display unit 28. The image display unit 28 can turn on / off the display according to an instruction from the system control unit 50. When the display is turned off, the power consumption of the camera 100 can be greatly reduced.

  The memory 30 is used for temporary storage of captured still images and moving images, and has a storage capacity sufficient to store a predetermined number of still images and moving images for a predetermined time. Thereby, even in the case of continuous shooting or panoramic shooting, a large amount of image writing can be performed on the memory 30 at high speed. The memory 30 can also be used as a work area for the system control unit 50. The compression / decompression unit 32 has a function of compressing / decompressing image data by adaptive discrete cosine transform (ADCT) or the like, reads an image stored in the memory 30, performs compression processing or decompression processing, and finishes the processed image data Is written back to the memory 30.

  The shutter control unit 36 controls the shutter 12 based on the photometric information from the photometric unit 46 in cooperation with the aperture control unit 344 that controls the aperture 312 of the photographing lens 300. The interface unit 38 and the connector 122 electrically connect the camera 100 and the photographing lens 300. These have functions of transmitting control signals, status signals, data signals, and the like between the camera 100 and the photographing lens 300 and supplying currents of various voltages. Moreover, it is good also as a structure which transmits not only electrical communication but optical communication, audio | voice communication, etc.

  The photometry unit 46 performs automatic exposure control (AE) processing. The light flux that has passed through the photographic lens 300 is incident on the photometric unit 46 via the lens mount 106, the main mirror 130, and a photometric lens (not shown), whereby the luminance of the subject optical image can be measured. The photometry unit 46 can determine the exposure condition by using a program diagram in which the subject brightness and the exposure condition are associated with each other. The photometry unit 46 also has a light control processing function in cooperation with the flash 48. The system control unit 50 can also perform AE control on the shutter control unit 36 and the aperture control unit 344 of the photographic lens 300 based on the calculation result obtained by calculating the image data of the image sensor 14 by the image processing unit 20. It is. The flash 48 also has an AF auxiliary light projecting function and a flash light control function.

  The system control unit 50 includes a programmable processor such as a CPU or MPU, and controls the operation of the entire camera system by executing a program stored in advance. The nonvolatile memory 52 stores constants, variables, programs, etc. for operating the system control unit 50. The display unit 54 is, for example, a liquid crystal display device that displays an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program by the system control unit 50. One or a plurality of display units 54 are installed at positions in the vicinity of the operation unit of the camera 100 that are easily visible, and are configured by a combination of an LCD, an LED, and the like, for example. Among the display contents of the display unit 54, what is displayed on the LCD or the like includes information on the number of shots such as the number of recorded sheets and the number of remaining shots, information on shooting conditions such as shutter speed, aperture value, exposure correction, and flash. There is. In addition, the remaining battery level, date / time, and the like are also displayed. Further, as described above, a part of the display unit 54 is installed in the optical viewfinder 104.

  The nonvolatile memory 56 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used. Reference numerals 60, 62, 64, 66, 68, and 70 are operation units for inputting various operation instructions of the system control unit 50, such as a switch, a dial, a touch panel, pointing by gaze detection, a voice recognition device, or the like. Consists of multiple combinations.

  The mode dial 60 can switch and set each function mode such as power-off, auto shooting mode, manual shooting mode, playback mode, and PC connection mode. A shutter switch SW1 62 is turned on when a shutter button (not shown) is half-pressed, and instructs to start operations such as AF processing, AE processing, AWB processing, and EF processing. The shutter switch SW2 64 is turned on when the shutter button is fully pressed, and instructs the start of a series of processing related to photographing. A series of processing related to photographing is exposure processing, development processing, recording processing, and the like. In the exposure process, the signal read from the image sensor 14 is written as image data in the memory 30 via the A / D converter 16 and the memory control unit 22. In the development process, development is performed using computations in the image processing unit 20 and the memory control unit 22. In the recording process, image data is read from the memory 30, compressed by the compression / decompression unit 32, and written as image data on the recording medium 200 or 210.

  The image display ON / OFF switch 66 can set ON / OFF of the image display unit 28. With this function, when photographing using the optical viewfinder 104, power supply can be saved by cutting off the current supply to the image display unit 28 including a liquid crystal monitor or the like. The quick review ON / OFF switch 68 sets a quick review function for automatically reproducing captured image data immediately after shooting. The operation unit 70 includes various buttons and a touch panel. The various buttons include a menu button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, an exposure correction button, and the like.

  The power supply control unit 80 is configured by a battery detection circuit, a DC / DC converter, a switch circuit that switches blocks to be energized, and the like. The presence / absence of a battery, the type of battery, and the remaining battery level are detected, the DC / DC converter is controlled based on the detection result and the instruction of the system control unit 50, and a necessary voltage is included for a necessary period and a recording medium. Supply to each part. The connectors 82 and 84 connect the camera 100 with a power supply unit 86 including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a lithium ion battery, or an AC adapter.

  The interfaces 90 and 94 have a connection function with a recording medium such as a memory card or a hard disk, and the connectors 92 and 96 make a physical connection with a recording medium such as a memory card or a hard disk. The recording medium attachment / detachment detection unit 98 detects whether or not a recording medium is attached to the connector 92 or 96. Although the present embodiment has been described as having two systems of interfaces and connectors for attaching the recording medium, the interface and connectors may have a single system or a plurality of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard. Furthermore, by connecting various communication cards such as a LAN card to the interface and connector, it is possible to transfer image data and management information attached to the image data to and from other peripheral devices such as a computer and a printer.

  The communication unit 110 has various communication functions such as wired communication and wireless communication. The connector 112 connects the camera 100 to other devices via the communication unit 110, and is an antenna in the case of wireless communication. The recording media 200 and 210 are a memory card, a hard disk, or the like. The recording media 200 and 210 include recording units 202 and 212 configured by a semiconductor memory, a magnetic disk, and the like, interfaces 204 and 214 with the camera 100, and connectors 206 and 216 for connecting with the camera 100.

  Next, the photographing lens 300 will be described. The taking lens 300 is mechanically and electrically coupled to the camera 100 by engaging the lens mount 306 with the lens mount 106 of the camera 100. Electrical coupling is realized by the connector 122 and the connector 322 provided on the lens mount 106 and the lens mount 306. The lens 311 includes a focus lens for adjusting the focusing distance of the photographic lens 300, and the focus control unit 342 adjusts the focus of the photographic lens 300 by driving the focus lens along the optical axis. The diaphragm 312 adjusts the amount and angle of subject light incident on the camera 100.

  The connector 322 and the interface 338 electrically connect the photographing lens 300 to the connector 122 of the camera 100. The connector 322 has a function of transmitting a control signal, a status signal, a data signal, and the like between the camera 100 and the photographing lens 300 and supplying currents of various voltages. The connector 322 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.

  The zoom control unit 340 drives the variable power lens of the lens 311 and adjusts the focal length (view angle) of the photographing lens 300. If the taking lens 300 is a single focus lens, the zoom control unit 340 does not exist. The aperture control unit 344 controls the aperture 312 in cooperation with the shutter control unit 36 that controls the shutter 12 based on the photometry information from the photometry unit 46.

  The lens system control unit 346 has a programmable processor such as a CPU or an MPU, for example, and controls the entire operation of the photographing lens 300 by executing a program stored in advance. The lens system control unit 346 has a memory function for storing constants, variables, programs, and the like for operating the photographing lens. The nonvolatile memory 348 stores identification information such as a number unique to the photographing lens, management information, function information such as an open aperture value, a minimum aperture value, and a focal length, and current and past setting values.

  In the present embodiment, lens frame information corresponding to the state of the photographing lens 300 is also stored. This lens frame information is information on the radius of the frame opening that determines the light beam passing through the photographing lens and information on the distance from the image sensor 14 to the frame opening. The diaphragm 312 is included in a frame that determines a light beam that passes through the photographing lens, and an opening of a lens frame component that holds the lens corresponds to the frame. In addition, since the frame for determining the light flux that passes through the photographing lens differs depending on the focus position and zoom position of the lens 311, a plurality of pieces of lens frame information are prepared corresponding to the focus position and zoom position of the lens 311. When the camera 100 performs focus detection using the focus detection means, optimal lens frame information corresponding to the focus position and zoom position of the lens 311 is selected and sent to the camera 100 through the connector 322.

  The above is the configuration of the camera system according to this embodiment including the camera 100 and the photographing lens 300.

  Next, the focus detection operation of the phase difference detection method using the image sensor 14 will be described.

  FIG. 2A is a diagram schematically showing an example of the pixel arrangement of the image sensor 14 in the present embodiment. Among the pixel groups arranged two-dimensionally in the CMOS image sensor, the vertical (Y-axis direction) is 6 rows. A state in which a range of 8 rows (in the X-axis direction) is observed from the photographing lens 300 side is shown. The image sensor 14 has a Bayer array color filter. The even-numbered pixels have green and red color filters in order from the left, and the odd-numbered pixels have blue in order from the left. Green color filters are alternately provided. However, in the image sensor 14 of the present embodiment, a green color filter is provided instead of the original blue color filter for the pixel having the photoelectric conversion unit for focus detection. In the following description, a pixel provided with a blue (or green, red) color filter may be referred to as a blue pixel (or green pixel, red pixel).

  Each pixel is provided with an on-chip microlens 211i, and the rectangle in the on-chip microlens 211i schematically shows the light receiving area of the photoelectric conversion unit. The focus detection photoelectric conversion units 311a and 311b are arranged laterally offset from the center of the pixel. In the following description, a pixel provided with focus detection photoelectric conversion units 311a and 311b may be referred to as a focus detection pixel. The focus detection photoelectric conversion units 311a and 311b are arranged in green pixels provided in place of the original blue pixels. This is because the output of the blue pixel has the lowest influence on the image quality. Note that the present invention does not depend on the color filter pattern of the image sensor. As described above, since the image sensor 14 of this embodiment includes one photoelectric conversion unit for each pixel including the focus detection pixel, one photoelectric conversion signal is read from one pixel.

  Here, generation of an image signal used for focus detection by the phase difference detection method will be described. In this embodiment, four types of image signals are generated. As will be described later, in the present embodiment, the exit pupil of the imaging optical system (imaging lens 300) is divided using the microlens 211i and the photoelectric conversion units 311a and 311b having different bias positions. Then, among the outputs of the pixels 211 arranged in the same pixel row (X-axis direction), an output obtained by connecting the outputs of the plurality of photoelectric conversion units 311a and connecting the outputs of the A image and the plurality of photoelectric conversion units 311b is connected. A B image is obtained by knitting together. As shown in FIG. 2A, the A image and the B image can be obtained from a plurality of blue pixel positions (green pixels) adjacent to each other at a two-pixel pitch in the X-axis direction.

  Further, the photoelectric conversion units 311c that are the plurality of green pixels in the first row in FIG. 2A are adjacent to the photoelectric conversion unit 311a in the X-axis direction in FIG. A GA image is formed by connecting the outputs of 311c. In addition, the photoelectric conversion units 311c that are the plurality of green pixels in the fifth row in FIG. 2A are adjacent to the photoelectric conversion unit 311b in the X-axis direction in FIG. A GB image is formed by connecting the outputs of 311c and knitting them. The photoelectric conversion units 311a and 311b output a signal based on the light beam that has passed through a partial region of the exit pupil of the imaging optical system (imaging lens 300), whereas the photoelectric conversion unit 311c has an imaging optical system (imaging lens). 300), a signal based on the light flux that has passed through the entire area of the exit pupil is output. Thus, the A color image (output of the photoelectric conversion unit 311a), the B image (output of the photoelectric conversion unit 311b), the GA image (output of the photoelectric conversion unit 311c), and the GB image (output of the photoelectric conversion unit 311c) are the same color pixels. By obtaining from the group, it is possible to detect the phase difference with high accuracy.

  Note that the positions and the number of pixels used for generating the A image, the B image, the GA image, and the GB image are determined according to the focus detection area.

  Although details will be described later, since the A image and the GA image generated in this way are images obtained from pixels in the same row, the A image to be originally obtained from the relative image shift amount between the A image and the GA image. And 1/2 of the image shift amount of the B image can be accurately obtained. Similarly, since the B image and the GB image are images obtained from pixels in the same row, the remaining image displacement amount of the A image and the B image that is originally obtained from the relative image displacement amount between the B image and the GB image. Of 1/2 is accurately obtained. Then, by adding these together and performing a correlation calculation, it is possible to detect a defocus amount, that is, a defocus amount in a predetermined area of the A image and the B image. In the present embodiment, a normal pixel signal is obtained from a pixel having a photoelectric conversion region 311c whose position is not deviated from the center of the pixel (which may be referred to as a photographic pixel in the following description). When a captured image is generated, a normal pixel signal at a position corresponding to a focus detection pixel is generated (complemented) using the output of surrounding pixels. Note that, when generating a normal pixel signal, the output of the target focus detection pixel may or may not be used.

  Hereinafter, the plurality of pixels provided with the photoelectric conversion unit 311a used for generating the A image (first image signal) are referred to as a first pixel group, and are used for generating the B image (second image signal). A plurality of pixels provided with the photoelectric conversion unit 311b is referred to as a second pixel group. A plurality of pixels provided with the photoelectric conversion unit 311c, which are used for generating a GA image (third image signal), are referred to as a third pixel group, and are used for generating a GB image (fourth image signal). A plurality of pixels provided with the photoelectric conversion unit 311c is referred to as a fourth pixel group.

  In the present embodiment, the third pixel group and the fourth pixel group are pixel groups adjacent to the first pixel group and the second pixel group in the X-axis direction. However, the third pixel group and the fourth pixel group may be pixel groups adjacent to the first pixel group and the second pixel group in the Y-axis direction. Alternatively, a GA image and a GB image may be generated using pixel values obtained from other pixels. For example, the GA image may be generated from the pixel value calculated as the average value of a plurality of adjacent (for example, four) pixels for each pixel of the first pixel group.

  FIG. 2B is a diagram illustrating a configuration example of a readout circuit of the image sensor 14 of the present embodiment. The image sensor 14 has a horizontal scanning circuit 151 and a vertical scanning circuit 153, and a horizontal scanning line 252 and a vertical scanning line 254 are wired at the boundary of each pixel. Signals generated by the photoelectric conversion units 311a, 311b, and 311c are read out through the horizontal scanning line 252 and the vertical scanning line 254.

  FIG. 3 is a diagram for explaining a conjugate relationship between the exit pupil plane of the imaging lens 300 and the photoelectric conversion units 311a and 311b of the pixel 211 disposed in the vicinity of the center of the image plane of the imaging device 14. The exit pupil planes of the photoelectric conversion units 311a and 311b and the photographing lens 300 in the image sensor 14 are designed to have a conjugate relationship by the on-chip microlens 211i. In general, the exit pupil plane of the photographing lens 300 substantially coincides with the plane on which the iris diaphragm for adjusting the light amount is provided.

  On the other hand, the photographing lens 300 of the present embodiment is a zoom lens having a zooming function. Some zoom lenses change the size of the exit pupil and the distance from the image plane to the exit pupil (exit pupil distance) when a zooming operation is performed. FIG. 3 shows a state where the focal length of the photographic lens 300 is at the center between the wide-angle end and the telephoto end. Using the exit pupil distance Zep in this state as a standard value, the optimum design of the eccentric parameter according to the shape of the on-chip microlens and the image height is performed.

  In FIG. 3, the photographic lens 300 includes a first lens group 101, a lens barrel member 101b that holds the first lens group, a third lens group 105, and a lens barrel member 105b that holds the third lens group. . The photographic lens 300 includes an aperture 102, an aperture plate 102a that defines an aperture diameter when the aperture is open, and an aperture blade 102b for adjusting the aperture diameter when the aperture is closed. In FIG. 3, reference numerals 101b, 102a, 102b, and 105b that act as limiting members for the light flux passing through the photographing lens 300 indicate optical virtual images when observed from the image plane. Further, the synthetic aperture in the vicinity of the stop 102 is defined as the exit pupil of the photographing lens 300, and has an exit pupil distance Zep.

  In the lowest layer of the pixel 211, a photoelectric conversion unit 311a (FIG. 3A), a photoelectric conversion unit 311b (FIG. 3B), or a photoelectric conversion unit 311c (not shown) is arranged. On the upper layers of the photoelectric conversion units 311a to 311c, wiring layers 211e to 211g, a color filter 211h, and an on-chip microlens 211i are provided. The photoelectric conversion units 311a to 311c are projected onto the exit pupil plane of the photographing lens 300 by the on-chip microlens 211i. In other words, the exit pupil is projected onto the surfaces of the photoelectric conversion units 311a to 311c via the on-chip microlens 211i.

  FIG. 3C shows projection images EP1a and EP1b of the photoelectric conversion units 311a and 311b on the exit pupil plane. The projected image EP1c for the photoelectric conversion unit 311c is substantially equal to the sum of EP1a and EP1b. In the present embodiment, the projection image EP1a is referred to as a first pupil region, the projection image EP1b is referred to as a second pupil region, and the projection image EP1c is referred to as a third pupil region.

  3A and 3B, the outermost part of the light beam passing through the photographing lens 300 is indicated by L. The outermost L of the light beam is regulated by the aperture plate 102a of the diaphragm, and the vignetting of the projection images EP1a and EP1b hardly occurs in the photographing lens 300. FIG. 3C shows a circle TL formed on the exit surface by the outermost L of the light beam in FIGS. 3A and 3B. Since most of the projection images EP1a and EP1b of the photoelectric conversion units 311a and 311b are present inside the circle TL, it can be seen that almost no vignetting occurs. Since the outermost L of the light beam is defined by the aperture plate 102a of the stop, it can be paraphrased as TL = 102a. At this time, the vignetting state of the projection images EP1a to EP1b is symmetric with respect to the optical axis at the center of the image plane, and the received light amounts of the photoelectric conversion units 311a and 311b are equal. As described above, the imaging device 14 according to the present embodiment has not only an imaging function but also a function as a device that generates a signal used for focus detection by the phase difference detection method.

  FIG. 4A is a diagram illustrating an example of the focus detection area 401 set in the shooting range 400. When focus detection is performed using the output of the pixels of the image sensor 14, the output of the pixels included in the area of the image sensor 14 corresponding to the focus detection area 401 is output in both the contrast detection method and the phase difference detection method. Use. Therefore, it can be said that the focus detection area 401 is set in the image sensor 14. Hereinafter, the focus detection area 401 will be described as a pixel area of the image sensor 14 for ease of explanation and understanding.

  Here, it is assumed that the photoelectric conversion units 311a to 311c are provided in the pixels in the focus detection area 401 according to the rules shown in FIG. Since a focus detection pixel having photoelectric conversion units 311a and 311b deviated in the horizontal (X-axis) direction from the center of the pixel is used, the phase difference of the image signal is detected based on the contrast difference in the horizontal direction of the image in the focus detection area 401. To do.

  The phase difference detected here is generated by the difference in the traveling angle of the pair of light beams, and the phase difference per unit defocus amount is within the region on the exit pupil plane of the light beam that generates the pair of image signals. It is proportional to the center of gravity interval. As described above, the projection image EP1c for the photoelectric conversion unit 311c is substantially equal to the sum of the projection images EP1a and EP1b. Therefore, the barycentric position of the projection image EP1c exists at the center of the pair of barycentric positions of the projection images EP1a and EP1b. Therefore, the phase difference between the pair of image signals (A image and B image) obtained from the photoelectric conversion units 311a and 311b is the same as the pair of image signals (A image (A image (B)) obtained from the photoelectric conversion units 311a (311b) and 311c. B image) and GA image (GB image)) are approximately twice the phase difference.

  Since the projection image EP1c is common to the GA image and the GB image, the light beam for generating the GA image and the light beam for generating the GB image have the same barycentric position on the exit surface. Accordingly, the sum of the phase difference between the A image and the GA image obtained from the outputs of the photoelectric conversion units 311a and 311c and the phase difference between the B image and the GB image obtained from the outputs of the photoelectric conversion units 311b and 311c is the photoelectric conversion unit 311a. , 311b is approximately equal to the phase difference between the A and B images obtained from the output.

  FIG. 4B is a diagram showing pixels used for generating an AF image signal out of the pixels included in the focus detection area 401 and which image signal is generated by the output of each pixel. It is. In FIG. 4B, for each pixel group (first to fourth pixel groups) that generate the same type of image signal, the j-th pixel on the i-th row is referred to as “image signal type” (i, j) (where i and j are integers from 1 to N). For example, in the first pixel group that generates the A image, the first pixel in the first row is represented as A (1,1). Note that the color coding of the photoelectric conversion units in FIG. 4B is intended to facilitate understanding of pixel groups that generate the same type of image signal.

  FIG. 4B shows a case where the pixels used for generating the AF signal among the pixels in the focus detection area 401 are 2 rows and 2 N columns. Not exclusively. The number of rows may be two or more, and the number of columns may be appropriately set within a range in which a phase difference can generally be detected. Note that the number of columns may be dynamically increased when the phase difference cannot be detected or when it is determined that the accuracy is low.

  Next, the focus adjustment operation in the camera 100 will be described using the flowchart shown in FIG. Note that the processing shown in FIG. 5 is a state in which the main mirror 130 and the sub mirror 131 are retracted from the optical path (mirror-up), more specifically, during live view display (when displaying a moving image for display) or during moving image recording (for recording). This is a process that is performed at the time of video shooting. Although the description is given here assuming that the phase difference detection type automatic focus detection using the output of the image sensor 14 is performed, as described above, the contrast detection type automatic focus detection can also be performed.

  In step S501, the system control unit 50 determines whether a focus detection start instruction has been input by operating the SW1 (62), the operation unit 70, and the like. If it is determined that the focus detection start instruction has been input, the process proceeds to step S502. If it is not determined that it is, it waits. Note that the system control unit 50 is not limited to the input of the focus detection start instruction, and the process may proceed to S502 with the start of live view display or moving image recording as a trigger.

  In step S <b> 502, the system control unit 50 acquires various lens information such as the lens frame information and the focus lens position of the photographing lens 300 from the lens system control unit 346 via the interface units 38 and 338 and the connectors 122 and 322.

  In step S503, the system control unit 50 generates AF image signals (A image, B image, GA image, and GB image) from the pixel data in the focus detection area of the frame image data that is sequentially read out. To the image processing unit 20. The AF image signal is sent to the AF unit 42, and processing for correcting a difference in signal level caused by the size of the photoelectric conversion unit being different between the focus detection pixel and the imaging pixel is performed.

  In S <b> 504, the AF unit 42 calculates a shift amount of the image by applying a known correlation calculation or the like to the two pairs of image signals of the A image and the GA image, and the B image and the GB image, and converts it to a defocus amount. . Details of this processing will be described later. The AF unit 42 outputs the defocus amount to the system control unit 50.

  In S505, the system control unit 50 calculates the lens driving amount of the photographing lens 300 based on the defocus amount obtained from the AF unit 42 in S504.

  In step S <b> 506, the system control unit 50 transmits information on the lens driving amount and the driving direction to the focus control unit 342 of the photographing lens 300 via the interface units 38 and 338 and the connectors 122 and 322. The focus control unit 342 drives the focus lens based on the received lens driving amount and driving direction information. Thereby, the focus of the photographic lens 300 is adjusted. Note that the operation of FIG. 5 may be continuously performed even when moving image data after the next frame is read.

  Next, the defocus amount calculation processing performed by the AF unit 42 in S504 of FIG. 5 will be further described with reference to the flowchart shown in FIG. In S5041, the AF unit 42 as the first calculating unit performs a correlation operation between the A image and the GA image generated from the same pixel row (m-th row). The correlation amount COR1 (k) used for the correlation calculation can be calculated by the following equation (1), for example.

  The variable k used in the equation (1) is a shift amount at the time of correlation calculation, and is an integer from −kmax to kmax. After the AF unit 42 obtains the correlation amount COR1 (k) for each shift amount k, the shift amount k that maximizes the correlation between the A image and the GA image, that is, the value of the shift amount k that minimizes the correlation amount COR1. Ask for. Note that the shift amount k at the time of calculating the correlation amount COR1 (k) is an integer, but when obtaining the shift amount k that minimizes the correlation amount COR1 (k), in order to improve the accuracy of the defocus amount, Interpolation processing is performed as appropriate to obtain a value (real value) in units of subpixels.

  In the present embodiment, the shift amount dk at which the sign of the difference value of the correlation amount COR1 changes is calculated as the shift amount k that minimizes the correlation amount COR1 (k).

  First, the AF unit 42 calculates the difference value DCOR1 of the correlation amount according to the following equation (2).

DCOR1 (k) = COR1 (k) -COR1 (k-1) ... (2)
Then, the AF unit 42 obtains the shift amount dk1 in which the sign of the difference amount changes using the correlation amount difference value DCOR1. If the value of k immediately before the sign of the difference amount changes is k1, and the value of k where the sign changes is k2 (k2 = k1 + 1), the AF unit 42 calculates the shift amount dk1 according to the following equation (3). .

dk1 = k1 + | DCOR1 (k1) | / | DCOR1 (k1) -DCOR1 (k2) | ... (3)
As described above, the AF unit 42 serving as the first calculating unit calculates the shift amount dk1 that maximizes the correlation amount between the A image and the GA image in units of subpixels, and ends the process of S5041. The method for calculating the phase difference between the two one-dimensional image signals is not limited to that described here, and any known method can be used.

  In S5042, the AF unit 42 as the second calculation unit calculates the shift amount dk2 that maximizes the correlation for the B image and the GB image generated from the same pixel row (m + 1) by the same method as in S5041. .

  In step S5043, the reliability of the shift amounts dk1 and dk2 which are two correlation calculation results is calculated. The reliability of the correlation calculation result is the degree of coincidence of the signals of the two images. When the degree of coincidence of the two images is good, the reliability of the correlation calculation result is generally high. Therefore, a threshold is set for the degree of coincidence between the two images, and the degree of coincidence between the two images of the image signal used to obtain the two correlation calculation results is determined.

  However, the index for determining the reliability of the correlation calculation result is not limited to the degree of coincidence of the two images. For example, the S level described in JP 2007-52072 A may be used as an index. In calculating the S level, not only the degree of coincidence of the two images but also the amount of correlation change and sharpness are used. Alternatively, primary contrast (sum of absolute values of differences between adjacent output signals) or secondary contrast (sum of squares of adjacent output signals) may be used. Also, a defocus amount that is synonymous with the shift amount may be used as an index for determining reliability, or a threshold value of an evaluation value such as the degree of coincidence of the two images may be changed according to the defocus amount. This is because the reliability of the shift amount decreases as the defocus amount increases. Further, as an index for determining reliability, a value correlated with the movement of the subject or camera shake may be used as the index. Subject movement and camera shake can be used to reduce the reliability of the shift amount.

  Further, the saturation state of the output signals of the two images may be used as a reliability determination method. An output level for determining saturation is set, and it is determined that the focus detection result calculated using a pixel group in which the number of pixel signals determined to be saturated is greater than a predetermined threshold is low in reliability.

  In determining the reliability of the shift amounts dk1 and dk2, which are two correlation calculation results, different determination thresholds may be used for each. When the focus detection area is arranged at a location away from the center in the imaging range, a difference occurs in the amount of light received by the first pixel group and the second pixel group due to vignetting of the imaging optical system. In general, a signal obtained from a pixel group having a small amount of received light has a reduced S / N, and the reliability of a correlation calculation result performed using the obtained signal output becomes low. Therefore, two focus detections are made based on the position of the focus detection region, which is a condition for causing a light amount difference, the F value of the photographing optical system, the position of the exit pupil in the optical axis direction, the position of the photoelectric conversion unit in the optical axis direction What is necessary is just to set a different threshold value with respect to a result.

  Next, in step S5044, it is determined whether or not the reliability of the two correlation calculation results dk1 and dk2 is high. If it is determined that both are highly reliable, the process advances to step S5045 to determine whether or not the absolute value of the difference (dk1-dk2) between the two correlation calculation results is large. The determination performed here is that two pairs of signal outputs (A image and GA image, B image and GB image) are at different positions within the shooting range, so that focus detection can be performed on subjects present at different distances. There is sex. In such a situation, the correlation calculation result dk1 obtained from the A image and the GA image and the correlation calculation result dk2 obtained from the B image and the GB image are highly reliable, but the absolute value of the difference between the correlation calculation results is large. There is a case. In step S5045, in preparation for such a case, it is determined whether the absolute value of the difference between the two correlation calculation results is large.

  If the difference between the two correlation calculation results is small, the process proceeds to step S5047, where the sum dk_sum of the two types of shift amounts dk1 and dk2 calculated so far is calculated, and the sensitivity stored in advance in the nonvolatile memory 56 is calculated. By multiplying, the shift amount is converted into the defocus amount. When the calculation of the defocus amount DEF ends, the defocus amount calculation subroutine ends.

  On the other hand, if it is determined in step S5044 that at least one of the two correlation calculation results has low reliability (No in S5044), the process proceeds to step S5046. In step S5046, it is determined whether one of the two correlation calculation results is highly reliable. The reliability determination method and the threshold value may be set in the same manner as in step S5044. When it is determined in step S5046 that the reliability of one correlation calculation result is high (Yes in S5046) and in step S5045, it is determined that the absolute value of the difference between the two correlation calculation results is large (Yes in S5045). ), The process proceeds to step S5048.

  In step S5048, the defocus amount is calculated using one of the two correlation calculation results determined to have high reliability. The shift amount dk1 obtained as the correlation calculation result is approximately the difference between the centroid position of the light beam that has passed through the first pupil region on the image sensor and the centroid position of the light beam that has passed through the third pupil region on the image sensor. Proportional. Similarly, the shift amount dk2 is approximately proportional to the difference between the barycentric position of the light beam that has passed through the second pupil region on the image sensor and the barycentric position of the light beam that has passed through the third pupil region on the image sensor. From this and the contents described with reference to FIG. 3, the shift amounts dk1 and dk2 are approximately the same. Therefore, when the defocus amount is calculated using one correlation calculation result, the shift amount dk1 or the shift amount dk2 is doubled and then multiplied by the sensitivity stored in advance in the nonvolatile memory 56. To convert the shift amount to the defocus amount.

  The coefficient by which one shift amount is multiplied is not limited to twice. For example, when the position in the optical axis direction of the exit pupil of the photographing lens 300 and the position in the optical axis direction of the projection image of the photoelectric conversion unit of the image sensor are greatly different, the two shift amounts as described above are approximately the same. It ’s not good. In such a case, two types of sensitivity to be stored in advance in the nonvolatile memory 56 may be prepared, one used for the shift amount dk1 and the other used for the shift amount dk2. Even when only one kind of sensitivity is stored, a value corresponding to the sum of two shift amounts may be calculated by multiplying one shift amount by a value based on the ratio of the two shift amounts. .

  When only one correlation calculation result is used, the reliability of the focus detection result is lower than when two correlation calculation results are used. On the other hand, generally, when detecting a large defocus amount, a focus detection error is large. Therefore, the coefficient to be multiplied may be changed according to the magnitude of the shift amount calculated as the correlation calculation result. When only one correlation calculation result is used, the reliability of focus detection is lowered. Therefore, when the shift amount is large, the calculated defocus amount may be reduced by multiplying by a value smaller than twice. . Thereby, it is possible to prevent passing through the in-focus position during the focus adjustment operation, and a quick focus adjustment operation can be performed.

  It is also conceivable to increase the reliability of the correlation calculation result by the number of signal additions. When only one correlation calculation result is used, the correlation calculation result may be calculated using more pixel outputs than when two correlation calculation results are used. In FIG. 4B, the example in which the signal output obtained from the pixel in 2 rows and 2 columns is used as the AF signal has been described. However, when more focus detection pixels are arranged, the AF signal is used. It is also possible to increase the number of rows used as and add output signals for each pixel group. Further, the reliability of the correlation calculation result may be improved by adding AF signals obtained at different timings.

  Next, when it is determined in step S5046 that both correlation calculation results are low in reliability (No in S5046), the process proceeds to step S5049, where it is determined that focus detection is impossible, and the defocus amount calculation subroutine is terminated.

  In this embodiment, the A image and the B image, which are signals obtained by photoelectric conversion of light beams passing through different regions on the exit pupil of the photographing optical system, are arranged apart from each other in the direction orthogonal to the phase difference detection direction. For this reason, the pixels that generate the A image and the B image differ in the position at which the optical image as the subject is sampled, and the similarity between the two images is not guaranteed. When obtaining the phase difference between the two signals as described above by calculating the correlation amount, the two signals need to have a high degree of coincidence in order to detect the phase difference with high accuracy. In the present embodiment, the phase difference between the A image and the GA image is calculated using a GA image that can sample substantially the same position as the A image on the optical image that is the subject. Further, similarly, the phase difference of the GB image that can sample the same position as the B image is calculated. Then, using the sum of these two phase difference calculation results, the phase difference between the A image and the B image can be calculated with high accuracy.

  Then, when the reliability of the two phase difference calculation results is high and the difference between the results is small, the phase difference between the A image and the B image is calculated by adding the two phase difference calculation results. For this phase difference, a larger phase difference is detected. Therefore, the influence of noise included in the phase difference detection result can be reduced, and highly accurate phase difference detection can be performed.

  If only one of the phase difference calculation results is highly reliable, focus detection is performed using the highly reliable result. As a result, the frequency at which focus detection becomes impossible can be reduced. Further, by performing an appropriate process on the phase difference calculation result used for focus detection, focus adjustment can be performed while maintaining focus detection accuracy.

  By configuring in this way, the degree of freedom of arrangement of the pixels (first pixel group) for obtaining the A image signal and the pixels (second pixel group) for obtaining the B image signal is increased, in other words, separated. Can be placed in position. This leads to the realization of an AF pixel arrangement that is easy to correct when generating a signal for imaging, and high image quality can be realized.

  In the present embodiment, the phase difference dk1 (first phase difference) is calculated using the correlation amount (first correlation amount) obtained from the A image and the GA image, and the correlation amount obtained from the B image and the GB image ( The phase difference dk2 (second phase difference) was calculated using the second correlation amount. Although the defocus amount is calculated using the sum of these phase differences dk1 and dk2, the method of calculating the defocus amount is not limited to this. For example, the sum of the correlation amount obtained from the A image and the GA image (first correlation amount) and the correlation amount obtained from the B image and the GB image (second correlation amount) is calculated, and the sum of the two correlation amounts is calculated. In the same manner as described above, the phase difference may be calculated. In this case, the amount of phase difference detected from the A image and the B image is reduced, but the difference in correlation amount can be increased, so that the detection accuracy of the shift amount is improved.

  Further, when calculating the defocus amount from the shift amount, the sum of the phase differences dk1 and dk2 is multiplied by the sensitivity. However, the sensitivity for the phase difference dk1 and the phase difference dk2 are previously stored in the nonvolatile memory 56. You may remember the sensitivity of. Although the storage capacity of sensitivity information is increased, more accurate focus detection can be performed.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. The main difference from the first embodiment is the pixel arrangement of the image sensor. The imaging device of the first embodiment has a configuration in which shooting pixels and two types of focus detection pixels are arranged, and one pixel has one photoelectric conversion unit. As described in the first embodiment, when the present invention is applied to an image pickup apparatus using such an image pickup device, the focus detection accuracy can be improved. However, the present invention can also be applied to an imaging apparatus using an imaging device in which two photoelectric conversion units are provided in all pixels and output signals of A and B images can be obtained from all the pixels.

  The configuration of the imaging apparatus described in the first embodiment (FIG. 1), the focus detection region (FIG. 4A), the focus adjustment operation and the defocus amount calculation processing (FIGS. 5 and 6) are as follows. Since it is common also in embodiment, description is abbreviate | omitted.

  The configuration of the image sensor 14 in the present embodiment will be described with reference to FIGS. 7 to 8, the same components as those in FIGS. 2 to 3 are denoted by the same reference numerals, and redundant description is omitted.

  7A is similar to FIG. 2A, among the pixel groups arranged two-dimensionally on the image sensor 14 in the present embodiment, 6 rows in the vertical (Y-axis direction) and 8 columns in the horizontal (X-axis direction). This range is observed from the photographic lens 300 side. However, in the image sensor 14 of the present embodiment, the arrangement of the color filters is the same as the Bayer arrangement. That is, unlike the first embodiment, a blue color filter is provided for all the pixels in the blue position. In this embodiment, since an A (B) image and a GA (GB) image are obtained from the same pixel, there is no need to change the color of the filter.

  In the present embodiment, all the pixels 211 have photoelectric conversion units 211a and 211b that are divided into two in the X-axis direction. The output signal of one photoelectric conversion region and the sum of the output signals of both photoelectric conversion regions are Are separately readable. A signal corresponding to the output signal of the other photoelectric conversion region can be obtained as a difference between the sum of the output signals of both photoelectric conversion regions and the output signal of one photoelectric conversion region. The output signal of the divided photoelectric conversion region can be used for focus detection of a phase difference detection method by a method described later, and also used for generating a 3D (3-Dimensional) image composed of a pair of parallax images. You can also. On the other hand, the sum of the output signals of both photoelectric conversion regions can be used as an output signal of a normal photographing pixel.

  Here, generation of an image signal used for focus detection by the phase difference detection method will be described. In the present embodiment, the exit pupil of the photographing lens 300 is divided by the microlens 211i in FIG. 7A and the divided photoelectric conversion units 211a and 211b. Then, the A image obtained by connecting the outputs of the photoelectric conversion units 211a in the plurality of pixels 211 arranged in the same pixel row (X-axis direction) within the focus detection region, and the output of the photoelectric conversion unit 211b are connected. A B image is obtained by knitting. As described above, the image sensor of this embodiment cannot directly read the output of one of the two photoelectric conversion regions. Therefore, an image signal that requires an output signal of the photoelectric conversion region that cannot be directly read out can be obtained as a difference between the sum of the output signals of the two photoelectric conversion regions and the output signal of the photoelectric conversion region that can be directly read out.

  In the present embodiment, the GA image and the GB image are generated from the sum of the output signals of the two photoelectric conversion regions read from the pixels used for generating the A image and the B image.

  By detecting the relative image shift amount between the A image and the GA image generated in this way and the relative image shift amount between the B image and the GB image by correlation calculation, the defocus amount of the focus detection region, that is, defocusing. The amount can be detected. The basic method is as described in the first embodiment.

  Hereinafter, a plurality of pixels provided with the photoelectric conversion unit 211a used for generating an A image (first image signal) are referred to as a first pixel group, and are used for generating a B image (second image signal). A plurality of pixels provided with the photoelectric conversion unit 211b is referred to as a second pixel group. In the present embodiment, the first pixel group is also used for generating a GA image (third image signal), and the second pixel group is also used for generating a GB image (fourth image signal).

  FIG. 7B is a diagram illustrating a configuration example of a readout circuit in the image sensor 14 of the present embodiment. The imaging device 14 includes a horizontal scanning circuit 151 and a vertical scanning circuit 153, and horizontal scanning lines 152a and 152b and vertical scanning lines 154a and 154b are wired at the boundaries between the pixels. One output of the photoelectric conversion unit and both addition outputs are read out to the outside through these scanning lines.

  In the present embodiment, it is assumed that the outputs of the A image and the GA image are read from the pixels in the odd rows, and the outputs of the B image and the GB image are read from the pixels in the even rows.

  FIG. 8 is a diagram for explaining a conjugate relationship between the exit pupil plane of the photographing lens 300 and the photoelectric conversion units 211 a and 211 b of the pixel 211 disposed in the vicinity of the center of the image plane of the image sensor 14. The photoelectric conversion units 211a and 211b in the pixel 211 and the exit pupil plane of the photographing lens 300 are designed to have a conjugate relationship by the on-chip microlens 211i. The configuration of this embodiment is the same as that of the first embodiment except that each pixel has both of the configurations shown in FIGS. 3A and 3B, and thus a duplicate description is omitted. To do.

  Next, the focus detection method of the phase difference detection method using the output of the image sensor 14 in this embodiment will be described. Also in the present embodiment, as in the first embodiment, the phase difference is detected for each combination of the A image and the GA image, and the B image and the GB image. In this embodiment, a GA image (shooting signal) that is the sum of the outputs of the photoelectric conversion units 211a and 211b and an A image that is the output of the photoelectric conversion unit 211a are read from the odd-numbered pixel rows. Further, a GB image (shooting signal) that is the sum of the outputs of the photoelectric conversion units 211a and 211b and a B image that is the output of the photoelectric conversion unit 211b are read from the even pixel rows.

  The B image in the odd-numbered pixel row and the A image in the even-numbered pixel row can be calculated as the difference between the GA image and the A image and the difference between the GB image and the B image, respectively. / N is lower than when reading directly. Therefore, it is better not to use the image signal obtained as the difference in order to detect the phase difference with high accuracy. Therefore, in this embodiment, the phase difference is detected using the output of one photoelectric conversion unit that can be read and the sum of the outputs of both photoelectric conversion units.

  Unlike the case of the first embodiment, the imaging signal and one image signal for AF are obtained from the same pixel. Therefore, as shown in FIG. 4C, from the 2 rows and N columns of pixels arranged in the focus detection area 401, A image (A (i, j)), B image (B (i, j)) GA image (GA (i, j)) and GB image (GB (i, j)) (1 ≦ i ≦ 2, 1 ≦ j ≦ N) can be obtained. Similarly to the first embodiment, the shift amounts dk1 and dk2 are obtained for each combination of the A image and the GA image, and the B image and the GB image, and the defocus amount is obtained based on the sum dk_sum.

  According to the present embodiment, when an image sensor having a configuration in which the photoelectric conversion area of each pixel is divided is used, the phase difference can be detected with high accuracy while reducing the processing load compared to the individual reading of each photoelectric conversion area. Thus, focus detection accuracy can be improved.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

14: Image sensor, 20: Image processing unit, 42: AF unit, 50: System control unit, 100: Camera, 300: Shooting lens, 346: Lens system control unit

Claims (16)

The first phase difference between the first image signal based on the light beam that has passed through a partial area of the exit pupil of the imaging optical system and the third image signal based on the light beam that has passed through the entire area of the exit pupil is calculated. First calculating means for
A second image signal based on a light beam that has passed through a partial area different from the partial area of the exit pupil and a second image signal based on a light beam that has passed through the entire area of the exit pupil. Second calculating means for calculating the phase difference of
Third calculating means for calculating a defocus amount of the photographing optical system using at least one of the first phase difference and the second phase difference;
In the calculation of the defocus amount of the photographing optical system by the third calculation unit, the defocus amount of the photographing optical system is calculated using both the first phase difference and the second phase difference. Switching means for switching whether to calculate the defocus amount of the photographing optical system using either the first phase difference or the second phase difference;
A focus detection apparatus comprising:   And determining means for determining reliability of the first phase difference and the second phase difference, wherein the switching means is configured to determine whether the first phase difference and the second phase difference are based on the reliability. Whether to calculate the defocus amount of the photographing optical system using both of the above, or to calculate the defocus amount of the photographing optical system using either the first phase difference or the second phase difference The focus detection apparatus according to claim 1, wherein the focus detection apparatus is switched.   The switching means determines the phase difference of which the reliability is determined to be high when one of the first phase difference and the second phase difference is determined to be low reliability by the determination means. The focus detection apparatus according to claim 2, wherein the focus detection apparatus is switched so as to calculate a defocus amount of the photographing optical system.   The third calculating means uses the value obtained by almost doubling the phase difference of the first phase difference and the second phase difference, which is determined to be highly reliable, as the imaging optical system. The focus detection apparatus according to claim 3, wherein the defocus amount is calculated.   The switching means determines the first phase difference and the second phase difference when the determination means determines that the reliability of both the first phase difference and the second phase difference is high. The focus detection apparatus according to claim 2, wherein switching is performed so as to calculate a defocus amount of the photographing optical system using both of the two.   The third calculation unit calculates a defocus amount of the photographing optical system based on a value obtained by adding the first phase difference and the second phase difference. Focus detection device.   The determination means is characterized in that the threshold value for determining the reliability is different between a case where the reliability of the first phase difference is determined and a case where the reliability of the second phase difference is determined. The focus detection apparatus according to claim 2.   The determination means determines a reliability threshold value for determining the reliability according to the amount of defocus, and determines the reliability of the first phase difference and the reliability of the second phase difference. The focus detection apparatus according to claim 7, wherein the focus detection apparatus is different from the case where the focus detection is performed.   The determination means determines a reliability threshold value for determining the reliability of the first phase difference according to a position of the focus detection area within the imaging range, and determines the reliability of the second phase difference. The focus detection device according to claim 7, wherein the focus detection device is different from that in the case of determining the sex.   The determination means has low reliability of one of the first phase difference and the second phase difference when the difference between the first phase difference and the second phase difference is larger than a predetermined threshold. The focus detection device according to claim 2, wherein   The determination unit determines that the reliability of the first phase difference is low when the degree of coincidence between the first image signal and the third image signal is low, and the second image signal and the second image signal 4. The focus detection apparatus according to claim 2, wherein when the degree of coincidence of the four image signals is low, it is determined that the reliability of the second phase difference is low.   The apparatus further includes a driving unit that drives a focus lens of the photographing optical system, and the driving unit calculates a defocus amount of the photographing optical system using either the first phase difference or the second phase difference. In this case, the drive amount of the focus lens is made smaller than when the defocus amount of the photographing optical system is calculated using both the first phase difference and the second phase difference. The focus detection apparatus according to claim 1, wherein:   The first calculation means calculates the first phase difference based on a value obtained by adding the first image signal for a plurality of pixels and a value obtained by adding the third image signal for a plurality of pixels. The second calculating means is configured to determine the second phase difference based on a value obtained by adding the second image signal for a plurality of pixels and a value obtained by adding the third image signal for a plurality of pixels. And calculating the defocus amount of the photographing optical system using either the first phase difference or the second phase difference, the first calculation means and the second calculation The focus detection apparatus according to claim 1, wherein the unit increases the number of the plurality of pixels to be added. The first phase difference between the first image signal based on the light beam that has passed through a partial area of the exit pupil of the imaging optical system and the third image signal based on the light beam that has passed through the entire area of the exit pupil is calculated. A first calculating step,
A second image signal based on a light beam that has passed through a partial area different from the partial area of the exit pupil and a second image signal based on a light beam that has passed through the entire area of the exit pupil. A second calculation step of calculating the phase difference of
A third calculation step of calculating a defocus amount of the photographing optical system using at least one of the first phase difference and the second phase difference;
In calculating the defocus amount of the photographing optical system in the third calculating step, the defocus amount of the photographing optical system is calculated using both the first phase difference and the second phase difference. A switching step for switching whether to calculate a defocus amount of the photographing optical system using either the first phase difference or the second phase difference;
A focus detection method characterized by comprising:   The program for making a computer perform each process of the focus detection method of Claim 14.   The computer-readable storage medium which memorize | stored the program for making a computer perform each process of the focus detection method of Claim 14.

Images