CN118743241A - Image pickup apparatus and control method thereof - Google Patents

Abstract

The present invention provides an image pickup apparatus capable of obtaining an image having a good S/N ratio while suppressing deterioration of a signal readout speed. The image pickup apparatus includes: a pixel section in which a plurality of pixels are arranged in a matrix, each pixel including a photoelectric conversion section, a charge-voltage conversion section for converting charge of a signal transmitted from the photoelectric conversion section into voltage, and an expansion section for expanding capacitance of the charge-voltage conversion section; a switching unit for switching connection between the expansion section and the charge-voltage conversion section; and a readout unit that amplifies signals of the same pixel at a plurality of types of amplification rates and reads out the amplified signals, wherein, when the readout unit amplifies signals of the same pixel at the plurality of types of amplification rates and reads out the amplified signals, the switching unit configures settings for switching connection between the expansion section and the charge-voltage conversion section to be the same for a readout operation at the plurality of types of amplification rates.

Description

Image pickup apparatus and control method thereof

Technical Field

The present invention relates to an image pickup apparatus and a control method thereof.

Background

In recent years, as image sensors for digital still cameras, digital video cameras, and the like, low-power-consumption image sensors suitable for high-speed readout are widely used. In the image sensor, a signal is read by transferring charges of photodiodes to floating diffusions (hereinafter referred to as "FD") and converting them into voltages. At this time, if the capacitance of the FD is small, the amount of charge to be processed is also small. On the other hand, if the capacitance of FD is large, the gain of conversion into voltage is small, and noise appears large.

In contrast, in order to be able to select an appropriate FD capacitance, patent document 1 proposes an image sensor having a function of switching the FD capacitance.

Further, patent document 2 proposes an image sensor that amplifies electric signals generated by photoelectric conversion elements under the same exposure at different amplification ratios, and reads out the amplified signals, thereby improving the S/N ratio.

In an image pickup apparatus including such an image sensor, the S/N ratio can be improved by appropriately selecting two amplified electric signals or two analog-to-digital (AD) converted digital signals according to brightness and combining the two amplified electric signals or the two analog-to-digital (AD) converted digital signals into a single image.

CITATION LIST

Patent literature

Patent document 1: U.S. patent 7427790

Patent document 2: japanese patent laid-open No. 2021-168460

Disclosure of Invention

Problems to be solved by the invention

However, the image sensor having a function of switching the capacitance of the FD has a problem that, when the electric signals generated by the photoelectric conversion elements under the same exposure are amplified at different amplification ratios and the amplified signals are output, the capacitance of the FD may be switched to deteriorate the signal readout speed.

The present invention has been made in view of the above-described problems, and provides an image pickup apparatus that can obtain an image with a good S/N ratio while suppressing degradation of a signal readout speed when amplifying electric signals generated by photoelectric conversion elements under the same exposure at different amplification ratios and outputting the amplified signals.

Solution for solving the problem

According to the present invention, an image pickup apparatus is characterized by comprising: a pixel section in which a plurality of pixels are arranged in a matrix, each pixel including a photoelectric conversion section, a charge-voltage conversion section configured to convert charge of a signal transmitted from the photoelectric conversion section into voltage, and an expansion section configured to expand capacitance of the charge-voltage conversion section; a switching unit configured to switch a connection between the extension portion and the charge-voltage conversion portion; and a readout unit configured to amplify signals of the same pixels at a plurality of types of amplification rates and to read out the amplified signals, wherein when the readout unit amplifies signals of the same pixels at the plurality of types of amplification rates and reads out the amplified signals, the switching unit configures settings for switching connection between the extension section and the charge-voltage conversion section to be the same for a readout operation at the plurality of types of amplification rates.

ADVANTAGEOUS EFFECTS OF INVENTION

According to the present invention, when amplifying electric signals generated by photoelectric conversion elements under the same exposure at different amplification ratios and outputting the amplified signals, an image having a good S/N ratio can be obtained while suppressing deterioration of the signal readout speed.

Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings. Note that the same reference numbers will be used throughout the drawings to refer to the same or like components.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a block diagram showing a schematic configuration of an image pickup apparatus according to an embodiment of the present invention.

Fig. 2 is a diagram showing a configuration of an image sensor.

Fig. 3 is a diagram showing a circuit configuration of a pixel.

Fig. 4 is a circuit diagram showing a circuit configuration of the column circuit.

Fig. 5 is a diagram showing an example of the total gain when one image capturing is performed instead of the HDR capturing.

Fig. 6 is a diagram showing an example of the total gain when HDR shooting is performed.

Fig. 7 is a diagram showing input/output characteristics of the image sensor.

Fig. 8 is a diagram showing input/output characteristics of the image sensor.

Fig. 9 is a diagram showing input/output characteristics of the image sensor.

Detailed Description

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following examples are not intended to limit the scope of the claimed invention. In the embodiments, a plurality of features are described, but the invention requiring all such features is not limited thereto, and a plurality of such features may be appropriately combined. In addition, in the drawings, the same reference numerals are given to the same or similar configurations, and redundant description thereof is omitted.

Fig. 1 is a block diagram showing a schematic configuration of an image capturing apparatus 100 as an embodiment of the image capturing apparatus of the present invention.

In fig. 1, a photographing lens 101 is an interchangeable lens unit attachable to a main body of an image pickup apparatus 100 or a lens part built in the main body, and is constituted by a plurality of lens groups including a focus lens, a zoom lens, and the like, and a diaphragm mechanism, and the like.

The image sensor 102 is a CMOS image sensor including a plurality of pixels, and can be driven by at least two driving methods. One method is a driving method of photoelectrically converting each pixel of an optical image of an object formed by the photographing lens 101 in one exposure (same exposure) to generate electric charges based on the amount of incident light, and outputting an image signal obtained by amplifying the signal of the pixel with the same gain. Another method is a driving method of outputting a plurality of image signals obtained by amplifying signals of pixels obtained under one exposure (same exposure) with a plurality of types of gains.

Further, the image sensor 102 has an electronic shutter function such as a rolling shutter to adjust the amount of light incident on each pixel, and can control the exposure time of the subject image.

The image acquisition unit 103 temporarily holds the image signal output from the image sensor 102, and performs photometry processing using the held image signal.

The image processing unit 104 performs various types of signal processing such as noise reduction processing, gamma processing, color signal processing, and exposure compensation processing on the image signal held in the image acquisition unit 103, and outputs the processed image signal.

Furthermore, the image processing unit 104 generates a High Dynamic Range (HDR) image using a suitable synthesis method. For example, there are the following methods: the image signal amplified with high gain (amplification ratio) is used for an image portion having a predetermined signal level or lower, and the image signal amplified with low gain (amplification ratio) is used for an image portion (bright highlight-out image) having a signal level exceeding the predetermined level, and these image signals are synthesized. Note that in a normal image for synthesizing signals in a dark portion of an image, random noise in the dark portion is preferably suppressed.

The image recording unit 105 records the image signal processed by the image processing unit 104 in a storage device or a storage medium. For example, a memory device mountable to the main body of the image pickup apparatus 100 is used as a storage device or a storage medium.

By operating the operation unit 106, the user can input various instructions to the image capturing apparatus 100. The operation unit 106 includes operation members such as a release button, a mode switching dial, and a zoom lever, and a touch panel. User input via the operation unit 106 is given to the system control unit 110. The setting of the HDR photographing is also given to the system control unit 110 by the user input via the operation unit 106.

The storage unit 107 is a storage unit that stores, for example, contents of instructions given to a user of the image capturing apparatus 100, and is constituted of an electrically erasable and recordable nonvolatile memory.

The display unit 108 may display a photographed image and information thereof at the time of photographing, a user interface operated using the operation unit 106, and the like, and is constituted of, for example, a TFT-LCD. The display unit 108 may also be constituted by a touch panel, and may also be input on the display unit 108 by a user input given through an operation of the operation unit 106.

The system control unit 110 controls the image sensor control unit 111 and the lens control unit 112 based on the image signal and the photometry result held in the image acquisition unit 103 and the user input given through the operation unit 106.

The image sensor control unit 111 performs drive control of the image sensor 102 in accordance with a control signal from the system control unit 110. The lens control unit 112 performs drive control of the photographing lens 101 according to a control signal from the system control unit 110.

Fig. 2 is a block diagram showing the configuration of the image sensor 102 of the present embodiment.

The pixel region (pixel section) 208 is configured such that a plurality of unit pixels 200 are arranged in a matrix. For convenience of description, the present embodiment shows a configuration in which n pixels (n is a natural number of 2 or more) are arranged in the horizontal direction and 4 pixels are arranged in the vertical direction, but in reality, the matrix has a configuration in which a large number of pixels are arranged in the respective horizontal and vertical directions.

Each unit pixel 200 is provided with an optical filter of one of a plurality of colors, and acquires a video signal according to the color of the optical filter. In fig. 2, "RGB" means a unit pixel, that is, a unit pixel provided with a red filter is an R pixel, a unit pixel provided with a green filter is a G pixel, and a unit pixel provided with a blue filter is a B pixel. In this way, the unit pixels each having the optical filter of one of the three colors are arranged according to the bayer layout.

The driving pulse is transmitted from the vertical scanning circuit 203 to the pixels in the respective rows through the corresponding common driving signal line 202. Note that one drive signal line 202 is shown for each row for convenience of description, but a plurality of drive signal lines may be provided for each row.

In the unit pixels 200 in the same column, pixels in odd rows are connected to a column output line 201. Further, pixels in even rows are connected to the column output line 211. When the switch 209 is turned off, an image signal from an odd-numbered row is input to only the column circuit 204 via the column output line 201. When the switch 209 is turned on, an image signal from an odd-numbered row is input to both the column circuit 204 and the column circuit 210.

The same applies to the image signals from the pixels in the even-numbered rows, and when the switch 209 is turned off, the image signals are input only to the column circuit 210 via the column output line 211. When the switch 209 is turned on, an image signal from an even-numbered row is input to both the column circuit 204 and the column circuit 210.

When the switch 209 is turned off, the image signals from the pixels in the even lines and the image signals from the pixels in the odd lines can be read out simultaneously, and when the switch 209 is turned on, the image signals from the pixels in the even lines and the image signals from the pixels in the odd lines are read out sequentially. Therefore, when the switch 209 is changed from off to on, the speed of reading out the image signal decreases. In the present embodiment, the switch 209 is turned off and used when one image capturing (capturing of a single image) is performed, and the switch 209 is turned on and used when two images are acquired for HDR capturing.

The column circuits 204 and 210 are connected to the vertical scanning circuit 203 via column circuit signal lines 205 for transmitting column gain setting signals and other control signals, and perform processing for applying gains to signals from the respective unit pixels in accordance with instructions from the image sensor control unit 111. When the switch 209 is turned off, the same gain is set for the column circuit 204 and the column circuit 210. On the other hand, when the switch 209 is turned on, different gains are set for the column circuit 204 and the column circuit 210.

In the present embodiment, when one image capturing is performed, control is performed such that the switch 209 is turned off, and the same gain is set for the column circuit 204 and the column circuit 210, so that an output signal amplified with the same gain (amplification ratio) is obtained for the pixels in the even-numbered rows and the pixels in the odd-numbered rows. Further, when HDR shooting is performed, control is performed such that the switch 209 is turned on, and different gains required for HDR shooting are set for the column circuit 204 and the column circuit 210, so that output signals amplified with different gains (amplification ratios) are obtained for pixels in even rows and pixels in odd rows.

Further, the column circuit 204 and the column circuit 210 perform a/D conversion processing to output digital signals to the horizontal transfer circuit 206. The horizontal transfer circuit 206 outputs an input signal to the image acquisition unit 103.

Fig. 3 is a diagram showing a circuit configuration of each unit pixel 200 of the image sensor 102. In fig. 3, one of the plurality of unit pixels 200 constituting the pixel region 208 is representatively illustrated as a dotted rectangle.

The unit pixel 200 is connected to other circuits via a drive signal line 202 and a column output line 201. The column output line 201 is connected to the current source 303 and the column circuit 204, and is also connected to a plurality of unit pixels 200 (vertical pixel lines) arranged in the same column so as to transmit pixel signals. The driving signal line 202 is connected to the vertical scanning circuit 203, and is also connected to a plurality of unit pixels 200 (horizontal pixel lines) arranged in the same row.

As a result of the vertical scanning circuit 203 simultaneously controlling the unit pixels 200 in the same row via the driving signal line 202, signal readout from the unit pixels 200 and reset thereof are performed. The driving signal lines 202 each include a transfer control line pTX, an FD extension control line pFDext, a reset control line pRS, and a selection control line pSEL, which will be described later.

The photoelectric conversion element (photoelectric conversion portion) PD is a photodiode that converts incident light into electric charges and accumulates the converted electric charges. In the photoelectric conversion element PD, the P side of the PN junction is grounded, and the N side of the PN junction is connected to the source of the transfer transistor (transfer switch) TX.

The gate of the transfer transistor TX is connected to the transfer control line pTX, and the drain of the transfer transistor TX is connected to a Floating Diffusion (FD) capacitor CFD. The transfer transistor TX controls charge transfer from the photoelectric conversion element PD to the FD capacitance CFD.

One side of the FD capacitance CFD (charge-voltage conversion section) is grounded and charges are accumulated while the charges transferred from the photoelectric conversion element PD are converted into voltages. Hereinafter, a connection point between the drain of the transfer transistor TX and the other side (non-ground side) of the FD capacitance CFD is referred to as an FD node 300.

The FD extension transistor (FD extension) FDext is a MOS transistor, a gate of which is connected to the FD extension control line pFDext, a source of which is connected to the FD capacitance CFD, and a drain of which is connected to the reset transistor (reset switch) T2.

The gate of the reset transistor T2 is connected to the reset control line pRS, the drain of the reset transistor T2 is connected to the power supply voltage Vdd, and the source of the reset transistor T2 is connected to the FD extension transistor FDext.

When the FD extension transistor FDext and the reset transistor T2 are set to on states, the potential of the FD node 300 is reset to the power supply voltage Vdd. On the other hand, when both the FD extension transistor FDext and the reset transistor T2 are in an off state, the charge transferred from the photoelectric conversion element PD is converted into a voltage by the FD capacitance CFD.

When the FD extension transistor FDext is in an on state and the reset transistor T2 is in an off state, the FD extension transistor FDext functions as an accumulating portion (i.e., an accumulating capacitance) capable of accumulating electric charges. The accumulation capacitance is hereinafter referred to as "FD extension capacitance Cex". In this case, since the accumulated capacitance of the FD extension transistor FDext and the FD capacitance CFD are parallel to the substrate ground, the capacitance seen from the FD node 300 is the capacitance CFDadd obtained by adding the FD extension capacitance Cex and the FD capacitance CFD.

Therefore, at the FD node 300, the charge transferred from the photoelectric conversion element PD is converted into a voltage using the sum capacitor CFDadd, where the sum capacitor CFDadd is the sum of the FD capacitor CFD and the FD extension capacitor Cex.

The driving transistor (amplifying section) Tdrv is a transistor constituting an in-pixel amplifier, and has a gate connected to the FD capacitance CFD, a drain connected to the power supply voltage Vdd, and a source connected to the selection transistor SEL. Accordingly, the driving transistor Tdrv outputs a voltage corresponding to the voltage of the FD capacitance CFD.

The gate of the selection transistor SEL is connected to the selection control line pSEL, and the source of the selection transistor SEL is connected to the column output line 201. The selection transistor SEL outputs the output of the driving transistor Tdrv as an output signal (pixel signal) of the unit pixel 200 to the column output line 201.

The current source 303 constitutes a source follower circuit serving as an in-pixel amplifier together with a driving transistor Tdrv for driving the unit pixel 200 in the column to which the column output line 201 is connected.

In the present embodiment, transistors other than the driving transistor Tdrv and the current source 303 function as switches, and are configured to be turned on (turned on) when a signal on a control line connected to the gate thereof is high, and to be turned off (turned off) when the signal is low.

How to use the FD extension transistor FDext when capturing one image without HDR capturing will be described below.

As described above, when one image is photographed, in fig. 2, the switch 209 is turned off, and the same gain is set for the column circuit 204 and the column circuit 210. If a first gain having a relatively low amplification rate is set for the column circuit 204 and the column circuit 210, the FD extension transistor FDext is turned on so that a larger amount of charge can be accumulated to secure a target dynamic range.

On the other hand, if a gain that can ensure the target dynamic range is set, the FD extension transistor FDext is turned off to reduce noise. For example, if a second gain having an amplification factor larger than the first gain and capable of ensuring the target dynamic range, or a third gain having an amplification factor larger than the second gain is set, the FD extension transistor FDext is turned off.

Thus, when HDR photographing is not performed, the FD extension transistor FDext is switched on and off according to the gain of the same value set for the column circuit 204 and the column circuit 210.

Fig. 4 is a circuit diagram showing a circuit configuration of the column circuit 204. Since the column circuit 210 has the same circuit configuration as that of the column circuit 204, the configuration of the column circuit 204 will be described below as a representative example.

The column circuit 204 includes a column amplifier 700, a comparator 701, a counter circuit 702, a latch circuit 703, and an arithmetic circuit 704. The RAMP signal generating section 706 (hereinafter referred to as "RAMP") is a circuit that generates a RAMP signal that changes with respect to time. The column amplifier 700 is an amplifier that amplifies an output signal (pixel signal) of the unit pixel 200 output to the column output line 201.

The comparator 701 compares the ramp signal generated by the ramp signal generating section 706 with the output of the column amplifier 700, and outputs an inverted signal at a timing at which the ramp signal which changes with respect to time coincides with the pixel output.

The counter circuit 702 performs a counting operation based on a clock supplied from the connected counter control line pCNT. The counter circuit 702 starts a counting operation at a timing when the comparator 701 starts comparing the pixel signal with the ramp signal, and outputs a count value at a timing when the output of the comparator 701 is inverted.

The latch circuit 703 temporarily holds the count value output from the counter circuit 702, and outputs the held count value based on control via the connected latch control line pLTC.

The arithmetic circuit 704 stores the count value output from the latch circuit 703 as a digital signal of a pixel based on control via the connected arithmetic control line pCAL. Further, the arithmetic circuit 704 outputs a digital signal of the stored pixel. The comparator 701, the counter circuit 702, the latch circuit 703, the arithmetic circuit 704, and the RAMP 706 constitute an a/D converter.

The gain setting in the column circuit 204 will be described below. The same applies to the column circuit 210. The column amplifier 700 of the column circuit 204 may amplify the output signal (pixel signal) of the unit pixel 200 output to the column output line 201 with a plurality of different gains. The column amplifier 700 amplifies the pixel signal with a gain set in a manner described later, and the amplified pixel signal is input to the comparator 701.

Note that the pixel signal can be amplified with different gains at the time of a/D conversion by changing the time variation of the ramp signal not only by the column amplifier 700 but also by the downstream a/D converter. In other words, if the time variation of the ramp signal is slow, the inversion of the output of the comparator 701 will be delayed, which causes the count value to increase. Since the time-varying speed of the RAMP signal corresponds to the amplification rate of the output signal of the unit pixel 200, this is called a RAMP gain, and the RAMP gain can be switched by changing the time-varying level.

With reference to fig. 5, an example of the total gain of the amplifiers other than the column circuit 204 when one image capturing is performed instead of the HDR capturing will be described below.

In the present embodiment, an example is described in which the second total gain is 8 times as large as the first total gain. Further, an example is described in which the FD extension capacitance Cex is 3 when the FD capacitance CFD is 1.

In this case, the capacitance CFDadd (which is the sum of the FD capacitance CFD and the FD extension capacitance Cex) is 4, and the conversion gain of charge-voltage conversion (hereinafter referred to as "FD gain") in this case is twice as large as that of the standard. The FD extension transistor FDext is turned on when the first total gain is set, and the capacitance CFDadd is set to 4, and the FD extension transistor FDext is turned off when the second total gain is set. As a result, the gain difference of the second total gain with respect to the first total gain is equal to 4 times due to the FD capacitance.

Although the column circuit 204 and the column circuit 210 are configured so that the pixel signals can be amplified by the column amplifier 700 or the a/D converter with different gains, in the present embodiment, it is assumed that the RAMP gain of the a/D converter is equal to 1 time regardless of whether the first total gain or the second total gain is set. Note that the a/D converter may have different RAMP gains between the first total gain and the second total gain.

Assuming that the first total gain (1-fold total gain) is set, the gains of the column amplifiers 700 of the column circuit 204 and the column circuit 210 are each equal to 1-fold. It is also assumed that when the second total gain (8 times the total gain) is set, the gains of the column amplifiers 700 of the column circuits 204 and 210 are both equal to 2 times, so that the second total gain is 8 times greater than the first total gain together with the above-described gain difference of 4 times due to FD capacitance.

Thus, the second total gain is 8 times greater than the first total gain (FD gain is 4 times, gain of column amplifier 700 is 2 times, which results in a total gain of 8 times).

How to use the FD extension transistor FDext when acquiring two images for HDR photographing will be described below.

As described above, when HDR shooting is performed, in fig. 2, control is performed such that the switch 209 is turned on, and different gains required for HDR shooting are set for the column circuit 204 and the column circuit 210, so that output signals amplified with a plurality of gains are obtained.

Here, regarding the amplification rates of the column circuit 204 and the column circuit 210, an amplification rate at which the target dynamic range cannot be ensured unless the FD extension transistor FDext is turned on is defined as a first gain. Further, the amplification ratio that is larger than the first gain and that ensures the target dynamic range even if the FD extension transistor FDext is turned off is defined as the second gain. Even a larger amplification than the second gain is defined as the third gain. The case where one of these first gain to third gain is applied to the column circuit 204 and the column circuit 210 will be described below.

In one image capturing without HDR capturing, the FD extension transistor FDext is set on for the first gain, and the FD extension transistor FDext is set off for the second gain and the third gain.

In HDR photographing, a case is considered in which a first gain is set for the column circuit 204 and a second gain is set for the column circuit 210. In this case, if the FD extension transistor FDext tries to turn on and off as one image is captured, the output signal of the same unit pixel 200 cannot be read out to the column circuit 204 and the column circuit 210 at the same time.

Accordingly, the pixel signal when the FD extension transistor FDext is turned on is read out to the column circuit 204, and then the pixel signal when the FD extension transistor FDext is turned off is read out to the column circuit 210, which causes an increase in the readout time.

Therefore, in the present embodiment, in HDR photographing in which an electric signal generated by a photoelectric conversion element under the same exposure is amplified with a plurality of different gains (amplification ratios) and the amplified signal is read out, the same operation setting of the FD extension transistor FDext is set. Further, the pixel signals are read out to the column circuit 204 and the column circuit 210 at the same time, thereby avoiding a decrease in the readout speed.

Note that when the second gain is set for the column circuit 204 and the third gain is set for the column circuit 210, the FD extension transistor FDext is set to off as in one image capturing. Therefore, the setting of the FD extension transistor FDext does not change between HDR photographing and one image photographing.

With reference to fig. 6, an example of the total gain of the amplifiers other than the column circuit 204 when HDR shooting is performed will be described below.

In the present embodiment, an example is described in which two images whose total gains differ from each other by 8 times are simultaneously photographed during HDR photographing.

Referring to the first total gain of 1 time, the second total gain is set to 8 times, the third total gain is set to 64 times, and images having the first total gain (1 time) and the second total gain (8 times) are simultaneously taken. Alternatively, images with the second total gain (8 times) and the third total gain (64 times) are simultaneously taken. Thereby, two images whose total gains differ from each other by 8 times can be obtained. Note here that an example is described in which the FD extension capacitance Cex is 3 when the FD capacitance CFD is 1.

When images having a first total gain (1 time) and a second total gain (8 times) are simultaneously photographed, the FD extension transistor FDext is turned on for both images, and the FD gain is set to 1 time.

Although the column circuit 204 and the column circuit 210 are configured so that the pixel signals can be amplified by the column amplifier 700 or the a/D converter with different gains, in the present embodiment, it is assumed that the RAMP gain of the a/D converter is equal to 1 time regardless of whether the first total gain, the second total gain, or the third total gain is set. Note that different RAMP gains of the a/D converter may be set for the first total gain, the second total gain, and the third total gain.

Further, when the first total gain (1 time) is set, the gain of the column amplifier 700 of the column circuit 204 is set to 1 time. When the second total gain (8 times) is set, the gain of the column amplifier 700 of the column circuit 210 is set to 8 times.

Thereby, images having the first total gain (1 time) and the second total gain (8 times) can be simultaneously photographed.

A case where images having the second total gain and the third total gain are simultaneously photographed will be described below.

When images having the second total gain (8 times) and the third total gain (64 times) are simultaneously photographed, the FD extension transistor FDext is turned off for both images, and the FD gain is set to 4 times.

Further, when the second total gain (8 times) is set, the gain of the column amplifier 700 of the column circuit 204 is set to 2 times. When the third total gain (64 times) is set, the gain of the column amplifier 700 of the column circuit 210 is set to 16 times.

Thereby, images having the second total gain (8 times) and the third total gain (64 times) can be simultaneously photographed.

The following will describe the setting of the FD extension transistor FDext in HDR photographing and the relationship between the exposure determination method and the synthesis method of two images. The following describes the setting of the S/N ratio priority of the high luminance, the setting of the S/N ratio priority of the low luminance, and the setting of the S/N ratio priority of both the high luminance and the low luminance.

First, the setting of the S/N ratio priority of high luminance is described with reference to fig. 7.

In the HDR photographing with higher S/N ratio priority, the FD extension transistor FDext is turned on. Further, the aperture opening amount and the exposure time of the image sensor 102 are set by the lens control unit 112 so that the image signal amplified by the column circuit 210, which is set with the second gain having a relatively large magnification, has correct exposure. In this case, the image signal amplified by the column circuit 204, which is set with the first gain having a relatively small amplification ratio, has underexposure.

In fig. 7, the horizontal axis indicates the light amount of the subject incident on the image sensor, i.e., input luminance, and the vertical axis indicates the output value of the column circuit. By setting the FD extension transistor FDext on, a larger amount of charge can be accumulated in the FD unit, and thus an object with high input brightness can be photographed because the gain set for the column circuit 204 is lower than the gain set for the column circuit 210.

The image signal amplified by the column circuit 204 and the image amplified by the column circuit 210 are synthesized by the image processing unit 104 based on the image signals in the respective ranges enclosed by the ellipses. By combining the low-luminance portion having the correctly exposed image of the image signal amplified by the column circuit 210 and the high-luminance portion of the underexposed image obtained by gamma correction of the image signal amplified by the column circuit 204, the dynamic range on the high-luminance side can be extended and the S/N ratio can be improved.

Then, the setting of the S/N ratio priority of the low luminance is described with reference to fig. 8.

In the HDR photographing with a low luminance S/N ratio priority, the FD extension transistor FDext is turned on. Further, the aperture opening amount and the exposure time of the image sensor 102 are set by the lens control unit 112 so that the image signal amplified by the column circuit 204, which is set with the first gain having a relatively small magnification, has correct exposure. In this case, the image signal amplified by the column circuit 210, which is set with the second gain having a relatively large amplification ratio, has overexposure.

By setting the FD extension transistor FDext on, a larger amount of charge can be accumulated in the FD unit, and thus an object with high input brightness can be photographed because the gain set for the column circuit 204 is lower than the gain set for the column circuit 210.

The image signal amplified by the column circuit 204 and the image signal amplified by the column circuit 210 are synthesized by the image processing unit 104 using pixel signals in the respective ranges surrounded by ellipses. By performing correction for reducing the gain of an overexposed image having an image signal amplified by the column circuit 210 so that it has an exposure corresponding to the correct exposure, an image with less noise can be obtained. By combining the low noise image and the high luminance portion of the correctly exposed image having the image signal amplified by the column circuit 204, the S/N ratio on the low luminance side can be improved.

Then, the setting of the S/N ratio preference of both the high luminance and the low luminance is described with reference to fig. 9.

In HDR photographing in which the S/N ratio of both high luminance and low luminance is improved, the FD extension transistor FDext is turned off.

Further, the aperture opening amount and the exposure time of the image sensor 102 are set by the lens control unit 112 so that the image signal amplified by the column circuit 210, which is set with the second gain having a relatively large magnification, has correct exposure. In this case, the image signal amplified by the column circuit 204, which is set with the first gain having a relatively small amplification ratio, has underexposure.

By setting the FD extension transistor FDext to off, an image with less noise than when the FD extension transistor FDext is set to on can be obtained. On the other hand, an underexposed image having the image signal amplified by the column circuit 204 cannot indicate an object having higher luminance than the case described in fig. 4 because the accumulated charge amount decreases.

However, by combining a low-luminance portion having a correctly exposed image of the image signal amplified by the column circuit 210 with a high-luminance portion of an underexposed image obtained by gamma-correcting the image signal amplified by the column circuit 204, an image with an improved S/N ratio for both low luminance and high luminance can be captured.

Note that this embodiment describes the following example: when HDR shooting is performed, the gains of the column amplifiers 700 of the column circuit 204 and the column circuit 210 have a gain difference of 8 times, but the gain of the RAMP 706 may have the gain difference.

A method for controlling an image pickup apparatus to increase the S/N ratio of high luminance or low luminance and the S/N ratio of both high luminance and low luminance has been described. Note that the following configuration is also possible: the brightness of the S/N ratio is determined to be desirably increased based on the brightness of the image captured by the image capturing apparatus, and the control is switched.

Other embodiments

The present invention can be achieved by supplying a program for realizing one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium and causing one or more processors in a computer of the system or apparatus to read out and execute the process of the program. The invention may also be implemented by circuitry (e.g., an ASIC) for implementing one or more functions.

The present invention is not limited to the above-described embodiments, and various changes and modifications may be made within the spirit and scope of the present invention. Accordingly, to apprise the public of the scope of the present invention, the following claims are made.

The present application claims priority from japanese patent applications 2022-028386 filed on 25 nd 2 nd of 2022 and japanese patent application 2022-159716 filed on 3 nd 10 th of 2022, which are incorporated herein by reference.

Claims (13)

1. An image pickup apparatus characterized by comprising:

A pixel section in which a plurality of pixels are arranged in a matrix, each pixel including a photoelectric conversion section, a charge-voltage conversion section configured to convert charge of a signal transmitted from the photoelectric conversion section into voltage, and an expansion section configured to expand capacitance of the charge-voltage conversion section;

a switching unit configured to switch a connection between the extension portion and the charge-voltage conversion portion; and

A readout unit configured to amplify signals of the same pixels at a plurality of types of amplification ratios and to read out the amplified signals,

Wherein when the readout unit amplifies signals of the same pixel at a plurality of types of amplification ratios and reads out the amplified signals, the switching unit configures settings for switching connection between the extension section and the charge-voltage conversion section to be the same for readout operations at the plurality of types of amplification ratios.

2. The image pickup apparatus according to claim 1, wherein,

When the readout unit amplifies a signal of the same pixel at a single amplification rate and reads out the amplified signal, the switching unit switches the connection between the extension portion and the charge-voltage conversion portion according to the amplification rate.

3. The image pickup apparatus according to claim 2, wherein,

When the readout unit amplifies signals of the same pixel at a plurality of types of amplification rates and reads out the amplified signals, the switching unit configures a setting for switching connection between the extension section and the charge-voltage conversion section to be different from a setting when the readout unit amplifies signals of the same pixel at a single amplification rate and reads out the amplified signals.

4. The image pickup apparatus according to any one of claims 1 to 3, wherein,

When the readout unit amplifies signals of the same pixel at a plurality of types of amplification ratios and reads out the amplified signals, the switching unit is configured so that the expansion section and the charge-voltage conversion section are connected to each other.

5. The image pickup apparatus according to any one of claims 1 to 3, wherein,

When the readout unit amplifies signals of the same pixel at a plurality of types of amplification ratios and reads out the amplified signals, the switching unit is configured so that the extension portion and the charge-voltage conversion portion are not connected to each other.

6. The image pickup apparatus according to any one of claims 1 to 3, wherein,

When the readout unit amplifies signals of the same pixel at a plurality of types of amplification ratios and reads out the amplified signals, the switching unit changes a setting for switching connection between the expansion section and the charge-voltage conversion section according to brightness of an image to be photographed.

7. The image pickup apparatus according to claim 6, wherein,

When the readout unit amplifies signals of the same pixel at a plurality of types of amplification ratios and reads out the amplified signals, if the S/N ratio of high brightness of an image to be photographed is prioritized, the switching unit is configured so that the expansion section and the charge-voltage conversion section are connected to each other.

8. The image pickup apparatus according to claim 7, wherein,

If the S/N ratio of the high brightness of the image to be photographed is prioritized, the setting is configured such that the extension portion and the charge-voltage conversion portion are connected to each other, and the pixel portion is exposed in such a manner that the image having the largest magnification among the plurality of types of magnifications has correct exposure.

9. The image pickup apparatus according to claim 6, wherein,

When the readout unit amplifies signals of the same pixel at a plurality of types of amplification ratios and reads out the amplified signals, if the S/N ratio of low luminance of an image to be photographed is prioritized, the switching unit is configured so that the expansion section and the charge-voltage conversion section are connected to each other.

10. The image pickup apparatus according to claim 9, wherein,

If the S/N ratio of the low luminance of the image to be photographed is prioritized, the arrangement is such that the expansion section and the charge-voltage conversion section are connected to each other, and the pixel section is exposed in such a manner that the image having the smallest magnification among the plurality of types of magnifications has the correct exposure.

11. The image pickup apparatus according to claim 6, wherein,

When the readout unit amplifies signals of the same pixel at a plurality of types of amplification ratios and reads out the amplified signals, if the S/N ratio of both high luminance and low luminance of an image to be photographed is to be increased, the switching unit is configured so that the expansion section and the charge-voltage conversion section are not connected to each other.

12. The image pickup apparatus according to claim 11, wherein,

If the S/N ratio of both high luminance and low luminance of an image to be photographed is to be increased, the arrangement is such that the extension portion and the charge-voltage conversion portion are not connected to each other, and the pixel portion is exposed in such a manner that an image having the largest magnification among a plurality of types of magnifications has correct exposure.

13. A control method of an image pickup apparatus, the image pickup apparatus comprising a pixel section in which a plurality of pixels are arranged in a matrix, each pixel comprising: a photoelectric conversion section; a charge-voltage conversion section configured to convert charges of a signal transmitted from the photoelectric conversion section into voltages; and an expansion section configured to expand a capacitance of the charge-voltage conversion section, the control method being characterized by comprising:

a switching step of switching connection between the extension section and the charge-voltage conversion section; and

A readout step of amplifying signals of the same pixels at a plurality of types of amplification ratios and reading out the amplified signals,

Wherein, in the switching step, when signals of the same pixel are amplified at a plurality of types of amplification ratios and read out in the reading out step, settings for switching connection between the extension section and the charge-voltage conversion section are configured to be the same for reading out operations at the plurality of types of amplification ratios.