TWI617198B - Imaging systems with clear filter pixels - Google Patents

Abstract

An image sensor may have an array of image sensor pixels arranged in a unit unit of a color filter, each unit having a red image pixel for generating a red image signal, a blue image pixel for generating a blue image signal, and Two transparent image sensor pixels that produce a white image signal. The image sensor can be coupled to a processing circuit that performs filtering operations on the red image signals, the blue image signals, and the white image signals to increase noise correlation in the image signals. A color correction matrix is applied to the noise amplification of the image signals. The processing circuit can extract a green image signal from the white image signal. The processing circuit can calculate a certain value including a linear combination of the red image signals, the blue image signals, the white image signals, and the green image signals. The scaling value can be applied to the red image signals, the blue image signals, and the green image signals to produce a corrected image signal having improved image quality.

Description

Imaging system with transparent filter pixels

The present invention is directed to imaging devices, and more particularly to imaging devices having transparent image pixels.

Image sensors are commonly used in electronic devices such as cellular phones, cameras, and computers to capture images. In a typical arrangement, the electronic device has an array of image pixels arranged in a column of pixels and rows of pixels. A circuit is typically coupled to each pixel row for reading image signals from the image pixels. Conventional imaging systems employ a single image sensor in which the visible spectrum is sampled by red, green, and blue (RGB) image pixels arranged in a Bayer mosaic pattern. The Bayer mosaic pattern consists of repeating units of two by two image pixels, two of which are diagonally opposite each other, and the other corners are red and blue. However, the Bayer pattern cannot easily achieve further miniaturization of the image sensor via a smaller image pixel size because of the limitations in the signal-to-noise ratio (SNR) in the image signal captured from the image pixels. One way to improve SNR is to increase the available image signal by increasing the exposure at low light levels, where SNR limits image quality. One conventional method uses a subtraction filter in which, for example, red, green, and blue image pixels are replaced by cyan, magenta, and yellow image pixels. However, such signals typically must be converted to RGB or some equivalent output image signal color to be capable of driving most conventional image displays. This transformation typically involves the use of a color correction matrix (CCM) to modify the captured image signal, which amplifies the noise so that the effects of increased exposure are compromised. Accordingly, there will be a need for an imaging device that provides improved means of capturing and processing image signals.

This application claims US Patent Application Serial No. 13/736, filed on Jan. 8, 2013. U.S. Provisional Patent Application No. 768/612, filed on March 19, 2012, Right 819, The entire contents of these patent applications are hereby incorporated herein by reference.  Such as a digital camera, computer, Electronic devices for cellular phones and other electronic devices include image sensors that collect incident light to capture images. The image sensor can include an array of image pixels. The pixels in the image sensor may include photosensitive elements such as photodiodes that convert incident light into image signals. The image sensor can have any number of pixels (eg, Hundreds or thousands or more). A typical image sensor can have, for example, hundreds or thousands or millions of pixels (eg, Megapixel). The image sensor can include a control circuit, Such as a circuit for operating image pixels, And a readout circuit for reading an image signal corresponding to the charge generated by the photosensitive element. The readout circuitry can include selectable readout circuitry coupled to each row of pixels, It can be enabled or disabled to reduce power consumption in the device and improve pixel readout operations.  1 is a diagram of an illustrative electronic device that captures an image using an image sensor. The electronic device 10 of FIG. 1 can be, for example, a camera. Honeycomb phone, A portable electronic device that is a camera or other imaging device that captures digital image data. The camera module 12 can be used to convert incident light into digital image data. Camera module 12 can include one or more lenses 14 And one or more corresponding image sensors 16. During the image capture operation, Light from the scene can be focused by lens 14 onto image sensor 16. Image sensor 16 may include circuitry for converting analog pixel data into corresponding digital image data to be provided to processing circuitry 18. As needed, The camera module 12 can be provided with an array of lenses 14 and an array of corresponding image sensors 16.  Processing circuitry 18 may include one or more integrated circuitry (eg, Image processing circuit, microprocessor, Storage devices such as random access memory and non-volatile memory, and many more), And components that are separate from the camera module 12 and/or components that form part of the camera module 12 (eg, The implementation is performed by forming a circuit including an integral circuit of the image sensor 16 or a portion of the integrated circuit associated with the image sensor 16 in the module 12. Image data that has been captured by camera module 12 can be processed and stored using processing circuitry 18. The processed image data may optionally be provided to an external device using a wired and/or wireless communication path coupled to processing circuitry 18 (eg, Computer or other device).  as shown in picture 2, Image sensor 16 may include a pixel array 200 including image sensor pixels 190 (sometimes referred to as image pixels 190) and control and processing circuitry 122. Array 200 can contain, for example, hundreds or thousands of columns and rows of image sensor pixels 190. Control circuit 122 can be coupled to column decoder circuit 124 and row decoder circuit 126. Column decoder circuit 124 can receive the column address from control circuit 122, And will be reset, Column selection, A corresponding column control signal for the transfer and read control signals is supplied to pixel 190 via control path 128. One or more conductive lines, such as row line 40, may be coupled to each of pixels 190 in array 200. The line 40 can be used to read an image signal from the pixel 190. And used to bias the signal (for example, A bias current or a bias voltage is supplied to the pixel 190. During the pixel readout operation, The column of pixels in array 200 can be selected using column decoder circuit 124, Image data associated with image pixels 190 in the pixel column can be read along row line 40.  Row decoder circuit 126 can include a sample and hold circuit, Amplifier circuit, Analog/digital conversion circuit, Bias circuit Line memory, a latch circuit for selectively enabling or disabling a row circuit, Or one or more rows of pixels coupled to array 200 for operating pixel 190 and for reading other signals from pixel 190. Row decoder circuit 126 can be used to selectively provide power to the row circuitry on a selected subset of row lines 40. Signal processing circuitry such as associated with row decoder circuit 126 (eg, Sampling and holding circuit, The readout circuitry of the analog/digital conversion circuitry can be used to supply digital image data to processor 18 (FIG. 1) on path 210 for pixels in the selected pixel row.  Image sensor pixels, such as image pixels 190, are conventionally provided with a color filter array. It allows a single image sensor to use the corresponding red, arranged in a Bayer mosaic pattern, Green and blue image sensor pixels are red, Green and blue (RGB) light is sampled. The Bayer mosaic pattern consists of repeating unit cells of two by two image pixels. Two of the green image pixels are diagonally opposite each other and adjacent to the red image pixels diagonally opposite to the blue image pixels. however, The limitations of the signal-to-noise ratio (SNR) associated with the Bayer mosaic pattern make it difficult to reduce the size of image sensors such as image sensor 16. therefore, Image sensors capable of providing improved means of capturing images may be required.  In a suitable example of this article sometimes discussed as an example, The green pixels in the Bayer pattern are replaced by transparent image pixels. As shown in Figure 3. As shown in Figure 3, The unit cell 192 of the image pixel 190 can be formed by two transparent image pixels (sometimes referred to herein as white (W) image pixels). The two transparent image pixels are diagonally opposite each other and adjacent to the red (R) image pixels diagonally opposite to the blue (B) image pixels. The white image pixels 190 in the unit cell 192 can be formed using a visually transparent color filter. The visually transparent color filter emits light in the visible spectrum (eg, White pixels 190 can capture white light). The transparent image pixel 190 can have a material that forms a transparent color filter and/or a material that forms an image sensor pixel (eg, 矽) Define the natural sensitivity. The sensitivity of the transparent image pixels 190 can optionally be adjusted to achieve better color reproduction and/or noise characteristics by using a light absorber such as a pigment. Unit unit 192 can be repeated on image pixel array 200 to form red, Mosaic of white and blue image pixels 190. In this way, The red image pixel can generate a red image signal in response to the red light. The blue image pixel can generate a blue image signal in response to the blue light. And the white image pixel can generate a white image signal in response to the white light. The white image signal can also be echoed by the white image pixel, Produced by any suitable combination of blue and/or green light.  Unit unit 192 of Figure 3 is merely illustrative. As needed, Image pixels of any color adjacent to diagonally opposite white image pixels in unit cell 192 can be formed. For example, Unit unit 194 can be defined by two white image pixels 190. The two white image pixels 190 are formed opposite to each other and adjacent to the red image pixels diagonally opposite to the green (G) image pixels. As shown in Figure 4. In yet another suitable arrangement, Unit unit 196 can be defined by two white image pixels 190. The two white image pixels 190 are formed opposite to each other and adjacent to the blue image pixels diagonally opposite to the green image pixels. As shown in Figure 5.  White image pixels W can be facilitated by having a narrower color filter (eg, An image pixel that emits a filter of light on a subset of the visible spectrum (eg, The green image pixels) increase the signal-to-noise ratio (SNR) of the image signal captured by the image pixels 190 compared to the additional light. The white image pixel W can especially improve the SNR under low light conditions. SNR can sometimes limit the image quality of an image in low light conditions. Since it has white image pixels (for example, The image signals collected by the image pixel array 200 as shown in FIGS. 3 to 5 can be converted to be used for driving most image displays (for example, Display screen, Circuits and software compatible with monitors, etc. Green and blue image signals. This conversion typically involves modifying the captured image signal using a color correction matrix (CCM). If you don’t pay attention, The color correction operation may undesirably amplify the noise.  In a suitable arrangement, The noise generated by the CCM can be implemented by applying strong de-noise before applying the CCM to the aggregated image signal (eg, Chromaticity goes to noise) to reduce. The chrominance denoising can be performed by processing circuitry 18 (Fig. 1) by applying a chrominance filter to the image signals that are gathered by image pixels 190. Chroma filters can be used to increase image signals from image pixels of different colors (for example, red, Correlation between white and blue image signals). Increasing the noise correlation between image signals from image pixels of different colors can reduce the noise amplification performed by the CCM. This results in improved final image quality. In another arrangement, The noise amplified by the CCM can be compensated by applying a so-called "dot filter" to the captured image signal. Point filters can use high fidelity white image signals to enhance the red color produced by CCM, The quality of green and blue image signals. As needed, Image sensor 16 may implement both chroma denoising and point filters to reduce the noise amplification by the CCM to produce improved brightness performance in the final image.  6 is shown that may be performed by a processing circuit, such as processing circuit 18 of FIG. 1, to process a filtered pixel array, such as by pixel array 200 (eg, Flowchart of illustrative steps for aggregating image signals for a pixel array without green image pixels. The steps of FIG. 6 can be performed, for example, by processing circuitry 18 to reduce noise in image signals captured using unit cells having white image pixels such as those shown in FIGS. 3 through 5.  At step 100, Image sensor 16 can capture image signals from the scene. The image signal captured by image sensor 16 may include a white image signal that is generated in response to light that is collected by the white pixels. As needed, The image signal may also include a red image signal, One or more of a blue image signal or a green image signal, This depends on the configuration of the image pixels used (for example, If the unit unit 192 of Figure 3 is used, The image signal may include red, White and blue image signals; If the unit unit 194 of Figure 4 is used, The image signal may include red, White and green image signals, and many more). In the example of Figure 6, Capturing red (R'), White (W') and blue (B') image signals. The red image signal may have a first spectral response value (integrated signal power level as a function of the frequency of the light received by the red image sensor pixel), The blue image signal can have a second spectral response value. And the white image signal can have, for example, a third spectral response value that is greater than seventy-five percent of the sum of the first spectral response value and the second spectral response value (eg, In the case of the standard CIE illuminant E, The white image signal has a broad sensitivity to the equal energy radiator in the visible spectrum). The image signal may have an image value corresponding to the light captured by each image pixel 190 (eg, The red image signal may include a red image value, The blue image signal may include a blue image value. and many more). The captured image signal can be communicated to processing circuitry 18 for image processing.  At step 102, A white balance operation can be performed on the captured image signal. In the example of Figure 6, Can generate white balance red image signal (R), White balance white image signal (W) and white balance blue image signal (B).  At step 104, The processing circuit 18 can demosaize the white balance image signal. And applying a chroma filter to extract red from the white balance image signal, White and blue image data. A chroma filter can be applied to perform chrominance denoising on the white balance image signal. Processing circuitry 18 may, for example, demosaize the image signal, At the same time, The chroma filter is applied sequentially or in an interlaced manner. The process of applying chroma filters and demosaicing to image signals can be referred to herein as "chroma demosaicing." A chroma filter increases the noise correlation between image signals of each color (for example, red, The noise fluctuations in the white and blue channels can be increased or decreased together in a related manner). For example, Processing circuit 18 can be red, The associated noise between the white and green image signals increases to red and 70% or more of all noise associated with white and green image signals.  By adding noise correlation, The processing circuit 18 reduces the amount of noise amplification that occurs when the CCM is applied to the image signal. Chromatically demosaicing the image signal allows for the determination of the missing color image signal from the available color image signal (eg, An image signal of a color that is not produced by an image pixel). In this example, It is possible that the green image signal is lost from the collected image signal. This is because the green color filter is not used in the unit cell 192 (Fig. 3). Can use white, Red and blue image signals (for example, The green image signal is determined by performing a subtraction operation. In general, The available color image signals can be used to determine the primary color additive (for example, red, Any of green and blue). May need to produce red, Green and blue image signals, Regardless of the color filter used on the image pixel array 200, This is because the display system usually uses red, Green and blue pixels to display images.  At step 106, The processing circuit 18 can apply a color correction matrix (CCM) to the red image data, White image data and blue image data. The CCM may, for example, extract green image data from confession image data to generate red, Green and blue imagery. For example, CCM converts image data to standard red, Standard green and standard blue image data (sometimes collectively referred to as linear sRGB image data or simply sRGB image data). In another suitable arrangement, CCM can extract green image data from red and/or blue image data. As needed, A gamma correction process can be performed on the linear sRGB image data. After gamma correction, The sRGB image data can be used to display using an image display device. In some cases, Additional noise reduction may be required (for example, Further mitigating the application of CCM to red by applying a point filter to the sRGB image data) Noise amplification caused by white and blue image data. The processing circuit 18 can store white image data. For further processing of the sRGB image data during optional step 108.  At optional step 108, The processing circuit 18 can apply a point filter to the image data (for example, Applied to apply CCM to red, sRGB image data generated after white and blue image data). The point filter operates on the sRGB image data to produce corrected sRGB data. Point filters can be used to further reduce the application of CCM to red, Noise amplification caused by white and blue image data. When using the display system display, The corrected sRGB data thereby provides better image quality than the sRGB data prior to applying the dot filter (eg, Better brightness performance).  7 shows a flow chart of illustrative steps that may be performed by processing circuitry 18 to demosaic and filter image signals received from image pixel array 200. The steps of FIG. 7 may be performed, for example, by processing circuitry 18 to focus the image pixels 190, White and blue image signals perform chroma de-mosaic, Producing red, Full noise related in white and blue image data. The steps of FIG. 7 can be performed, for example, as part of step 104 of FIG.  At step 110, Processing circuitry 18 may demomic the white image signal to produce white image data (eg, White image value for each image pixel). In another suitable arrangement, White image values can be generated for combinations of available image pixels 190. White image values can be used to calculate different values for red and blue image signals to increase red, Correlation between white and blue image signals.  At step 112, Processing circuitry 18 may generate a red delta value by subtracting the white image value from the red image value for each pixel. Processing circuitry 18 may generate a blue delta value by subtracting the white image value from the blue image value. The red difference value can be calculated, for example, for each red image pixel. And the blue delta value can be calculated for each blue image pixel of the image pixel array 200.  At step 114, Processing circuit 18 may filter the red and blue values using a chrominance filter. A weighted average of the differences calculated on the kernel of image pixels 190 can be performed, for example (eg, The chrominance filter is applied to the red and blue differences by performing a weighted average of the group of differences calculated in step 112. The core of the image pixel may be defined as a subset of the image pixels in the image pixel array 200 on which the chroma filtering is being performed (eg, The core may include some or all of the image pixels in the image pixel array 200. For example, When using a 5 pixel by 5 pixel core, A weighted average of the differences is calculated for a 5 pixel by 5 pixel subset of image pixels 190 in image pixel array 200 when chroma filtering is performed (eg, A weighted average of the differences is calculated for a given image pixel 190 using the difference at the 25 surrounding image pixels in image pixel array 200. In general, A core having any desired size can be used.  At step 116, The white image value may be added to the chroma-filtered red difference value and the chroma-filtered blue difference value to respectively generate the chroma-filtered red image value and the chroma-filtered blue image value.  At step 118, The processing circuit 18 may demosamate the chroma filtered red image values and the chroma filtered blue image values to generate red image data and blue image data with increased correlated noise (eg, Chromatographic demosaicing red and blue image data has been performed). The de-mosaic white image data and the chroma-de-mosaic red and blue image data can then be manipulated using CCM to produce a standard red, as described above in connection with step 106 of FIG. Standard green and standard blue (sRGB) image data.  Figure 7 is merely illustrative. As needed, Processing circuitry 18 may demosaze chroma-filtered red and blue image values before generating red and blue differences (eg, Processing circuit 18 may perform step 118) prior to step 112.  If the chrominance filtering of the difference is performed on a sufficiently large core of the image pixels 190, The minimum noise from the red and blue image signals can be after chroma filtering (for example, After performing step 114), it remains in the red and blue difference values. For example, If the core has a size of 15 pixels by 15 pixels or more, The chrominance filter reduces the noise in the difference between the red and blue chrominance filters to a negligible level. As needed, The core of the image pixel 190 may include image pixels located in the plurality of image pixel arrays 200, Image pixels located in the plurality of image sensors 16 And/or use during multiple timeframes (for example, Image pixels that allow time to go to noise. When the white image value is added to the chroma filtered difference, The noise in the white image value can dominate the noise in the difference. In this way, The noise in the red and blue image data generated at step 116 can be substantially equal to the noise in the white image data. The noise in the red and blue image data can be highly correlated. This produces a reduced noise amplification caused by CCM. When a Bayer pattern is used for the image pixel array 200, This process produces less noise amplification caused by CCM.  At step 106 (Fig. 6), CCM can be red, White and blue image data are manipulated to produce linear sRGB data. For example, CCM can extract information from white image data to generate standard green data. White image data (for example, The demosaiced white image data generated at step 104 can be retained after manipulation of the image data using CCM. The sRGB image data can be represented in a three-dimensional space such as a luminance-chroma-hue (LCH) space. In the LCH space, The brightness channel (L) can be related to the brightness of the image captured by the image sensor 16. And the chroma channel (C) can be related to the color saturation of the image. And the hue channel can be associated with a particular color of the image (for example, red, purple, yellow, Green, etc.) related. The perception of noise and sharpness in the displayed image can be affected by noise and signal changes in the luminance channel. The SNR in the image data can be converted into LCH data by converting the sRGB data. Instead of the luminance value in the luminance channel, the white image value, which is better correlated with the overall image brightness due to the wide spectrum of the white image signal, And improve the LCH data back to sRGB data. In this way, The noise amplification caused by CCM can be suppressed in the luminance channel. Among them, the noise is particularly noticeable to the observer when observing the displayed image.  As described above in connection with optional step 108 of FIG. 6, A point filter can be applied to the linear sRGB data to produce corrected sRGB data using white image data. The dot filter can operate on a single image pixel 190 without information from neighboring image pixels 190. While chroma de-mosaic can be applied to multiple image pixels when applied to image signals at a single image pixel 190 (eg, Image signal of the image pixel (for example, Difference). For example, A point filter can be used for each image pixel versus standard red value, Standard green value and standard blue value operation. To perform a point filter operation on the sRGB data, The processing circuit 18 can use red image data, White image data and blue image data (for example, The original (original) luminance signal is calculated by applying the image data before the CCM. The original brightness signal can be white image data, A linear combination of red image data and blue image data (for example, Weighted sum). As needed, White image data can be weighted more heavily in linear combinations than red and blue image data. Processing circuit 18 can be calculated as standard red, The implied luminance signal of a linear combination of standard green and standard blue image data (for example, After applying CCM to the image data). As needed, The weights used in calculating the linear combination of the implied luminance signals can be substantially similar to the weights used to calculate the original luminance signals. These weights can be adjusted to modify the "strength" of the point filter (for example, The degree to which the point filter transforms or corrects the sRGB data).  Processing circuit 18 may, in the simplest case, generate a scaling value by dividing the original luminance signal by the implied luminance signal (eg, The scaling factor to be applied to the color corrected image values). As needed, The scaling factor can include the numerator and the denominator. The numerator and/or denominator of the scaled value may comprise a weighted sum of the original luminance signal and the implied luminance signal. The scaling value can include an adjustable weighting parameter that can be varied to adjust the strength of the point filter (eg, These weighting parameters can be varied to adjust the intensity of the point filter from zero to full intensity). To apply a point filter to sRGB data (for example, Applied to standard red, Green and blue image data), Processing circuitry 18 may multiply the sRGB data by a scaling value to produce corrected sRGB data. For example, The processing circuit 18 can multiply the standard red image data by a calibration value. Multiply the standard green image data by the calibration value. and many more. As needed, The corrected sRGB data can have a self-application point filter (for example, Approximate preserved hue and chroma channels after converting the corrected sRGB data to the LCH space. The corrected sRGB data may have improved noise and/or sharpness due to the fidelity of the inheritance of the white image signal.  In the simplest case, The original luminance signal can be approximated by white image data. 8 shows that it can be performed by processing circuit 18 to apply CCM to red, A flow chart (as an example) of an illustrative step of applying a point filter (in the simplest case) to the sRGB data after the white and blue image data. Processing circuitry 18 may apply a point filter to the sRGB data, for example, for each image pixel 190 in image pixel array 200. The steps of FIG. 8 may be performed, for example, as part of step 108 of FIG.  At step 130, Processing circuit 18 can be combined by red, green, Blue image data (for example, The applied luminance value is generated for a given image pixel 190 after application of the CCM (eg, The brightness value in the LCH space). The suggested brightness value can be calculated, for example, as red, A linear combination of green and blue imagery.  At step 132, Processing circuitry 18 may generate the scaling value by dividing the white image value by the implied luminance value. As needed, The scaling factor can be generated by dividing the white image value by the weighted sum of the implied luminance value and the white image value. The scaling factor can include an adjustable weighting parameter that can be varied to adjust the strength of the point filter (eg, These weighting parameters can be varied to adjust the intensity of the point filter from zero to full intensity). The scaling value can be, for example, an operator that operates on sRGB data.  At step 134, Processing circuitry 18 may multiply the sRGB data by a scaling value to produce corrected sRGB data (eg, Corrected standard red, Green and blue image data). For example, The processing circuit 18 can multiply the standard red image data by a calibration value. Multiply the standard green image data by the calibration value. and many more. The corrected sRGB data can be provided to the image display as needed. The corrected sRGB data may have improved noise and/or sharpness compared to the sRGB data prior to applying the point filter.  The examples of Figures 6 through 8 are merely illustrative. Any desired color filter can be used in conjunction with the white filter shown in Figures 3 through 5 to obtain a color image signal. Any combination of desired color filters can be used (for example, Red filter, Green filter, Cyan filter, Infrared filter, UV filter, Blue filter, Yellow filter, Magenta filter, Any combination of purple filters, etc.). As needed, Any other suitable three-dimensional space can be used to perform the point filter operation.  As needed, Any number of image pixel arrays 200 formed on any number of image sensors 16 can be used to capture images. Each image pixel array used can be used, for example, for image signals of different colors. For example, The first image pixel array may have a transparent filter for generating a white image signal. The second image pixel array can have a red filter for generating a red image signal. And the third image pixel array can have a blue filter for generating a blue image signal. Image signals from each of these arrays may be chroma-de-mosaic and/or operated using dot filters. Each image pixel array can be formed on a different image sensor such as image sensor 16 in device 10 as desired (eg, A plurality of image sensors 16 may be formed in device 10. This embodiment may, for example, allow for shorter camera focal lengths and thinner camera modules.  Figure 9 shows a typical processor system 300 in a simplified form, Such as a digital camera, It includes an imaging device 2000 (for example, An imaging device 2000 such as the imaging sensor 16 of Figures 1 through 8 employing a transparent color filter and techniques for the operations described above. Processor system 300 is an illustrative system having digital circuitry that can include imaging device 2000. Without limitation, This system can include a computer system, Static or video camera system, scanner, Machine vision, Vehicle navigation, Video call, Supervision system, Autofocus system, Star tracker system, Motion detection system, Image stabilization system, And other systems that use imaging devices.  Processor system 300 typically includes: Lens 396, It is used to focus an image on the pixel array 200 of the device 2000 when the shutter release button 397 is pressed; Central Processing Unit (CPU) 395, Such as a microprocessor, It controls the camera and one or more image streaming functions, It communicates with one or more input/output (I/O) devices 391 via bus 393. Imaging device 2000 also communicates with CPU 395 via bus 393. System 300 also includes random access memory (RAM) 392, And may include removable memory 394, Such as flash memory, It also communicates with the CPU 395 via the bus 393. Imaging device 2000 can be combined with a CPU with or without a single integrated circuit or memory storage on a different wafer. Although bus 393 is illustrated as a single bus, But it can be used to interconnect system components with one or more bus bars or bridges or other communication paths.  Various embodiments have been described, It describes an image processing technique having a transparent image pixel filter and noise for reducing image signals generated by the image signal (for example, Chroma de-mosaic, Image sensor using point filter, etc.).  An image sensor can have an array of image sensor pixels. The image sensor pixels include red image pixels that generate red image signals in response to red light, a blue image pixel that produces a blue image signal in response to blue light, And responding to at least red light, Green and blue light (for example, White light) A transparent image sensor pixel that produces a white image signal. Image pixels may be arranged in an array of pixel unit cells. Each pixel unit includes a plurality of image pixels of different colors. An image sensor can be coupled to the processing circuit, The processing circuit is red, Blue and white image signals perform filtering operations to increase red, The blue and white image signals are associated with noise. The processing circuit can perform a filtering operation for a given image pixel, for example, by generating a weighted sum of image signals produced by at least 25 image pixels in the image pixel array. The weighted sum can include adjustable weights (eg, Based on the weight of the observed image feature adjustment). A weighted sum can be generated for image signals captured during multiple time frames or captured from multiple image sensors. By generating a weighted sum of multiple time frames, The processing circuit can reduce the size of the core of the image pixel, At the same time, the image signal noise is successfully reduced.  This example is merely illustrative. In general, The array of image sensor pixels can include image sensor pixels of any desired color (eg, Respond to image sensor pixels of any color of light). For example, The array of image sensor pixels can include a first group of image sensor pixels responsive to light of the first color, Responding to a second group of image sensor pixels of the second color of light, And responding to the light of the third color (for example, red, The third group of image sensor pixels for blue and white light. The first image signal can have a first spectral response level (eg, Integrated signal power level as a function of the frequency of the light received by the first group of image sensor pixels, The second image signal can have a second spectral response level (eg, Integrated signal power level as a function of the frequency of light received by the second group of image sensor pixels, And the third image signal can have a third spectral response level (eg, Integrated signal power level as a function of the frequency of the light received by the third group of image sensor pixels. The third image signal can have a spectral response level that is greater than the first spectral response level and the second spectral response level (eg, The third spectral response level may be greater than 75 percent of the sum of the first spectral response level and the second spectral response level. In other words, The third image signal can be captured in response to an optical frequency that is wider than the first image signal and the second image signal.  The processing circuit can use the first image signal as needed, The second image signal and the third image signal to generate an estimated brightness value (eg, The brightness value in the LCH space). The processing circuit can be configured by Converting the second image signal and the third image signal into the derived three color space (for example, Linear sRGB space, CIE space, XYZ space, Bayer space, etc.) to generate a transformed first image signal, a second image signal and a third image signal. The processing circuit can be, for example, by the first image signal, Performing a linear combination of the second image signal and the third image signal to generate the transformed first image signal, a second image signal and a third image signal. The processing circuit can combine the transformed first image signal, The transformed second image signal and the transformed third image signal are used to generate derived luminance values (eg, The brightness value in the LCH space). The processing circuit can compare the derived luminance value with the estimated luminance value, And modifying the transformed first image signal, The transformed second image signal and the transformed third image signal, Having the derived luminance value close to the estimated luminance value (eg, The derived luminance value is sufficiently matched to the estimated luminance value).  The processing circuit may use an image sensor having a processing circuit to process the first image signal including the first color, The image data of the second image signal and the white image signal of the second color of the first color. The processing circuit can use the white image signal to produce a third image signal that is different from the first color and the third color of the second color. The processing circuit can transmit the first image signal, Combining the second image signal and the third image signal to form a derived brightness value, And can be from the first color image signal, The second color image signal and the white image signal calculate the estimated brightness value. The processing circuit can be configured by using a white image signal and a first image signal, The second image signal and the third image signal are combined to form a derived luminance value.  The processing circuit can use the derived luminance value and the estimated luminance value to modify the first image signal, a second image signal and a third image signal. For example, The processing circuit can calculate the calibration value based on the derived luminance value and the estimated luminance value. And the first image signal, The second image signal and the third image signal are multiplied by the generated calibration value. The processing circuit can combine the first image signal, The second image signal and the third image signal are used to calculate the first image signal by using a weighting factor, A linear combination of the second image signal and the third image signal to form a derived luminance value.  The processing circuitry may perform infinite impulse response (IIR) filtering on the captured image signal as needed. The processing circuit can adjust the filter applied to the captured image signal by the characteristics of the image signal captured based on the image pixel 190 (eg, The IIR filtering is performed as described in connection with Figures 6-8. Performing IIR filtering increases the efficiency with which the processing circuitry processes the captured image signals.  The processing circuit can be red, The blue and white image signals perform a white balance operation. The processing circuitry can apply a color correction matrix to the white image signal to extract image signals of different colors, such as green image signals, from each of the white image signals. The processing circuit can combine the red image signal, Blue image signal, a green image signal and a white image signal to form a brightness value (eg, By calculating red, blue, Linear combination or weighted sum of green and white image signals). The processing circuit divides the white image signal by the luminance value to produce a scaling value. Processing circuitry can be done by red, The green and blue image signals are multiplied by the calibration value to modify the red, Green and blue image signals. The scaling value can act as a pair of red, Point filter for green and blue image signal operation. As needed, Image pixels of any color can be combined with white image signals. As needed, Processing circuitry can be used to array from multiple image pixels, The image signal array on the plurality of image sensors and/or the image signals of the image signals captured during the plurality of time frames perform such operations.  Transparent image pixels and associated filtering techniques can be implemented in a system. The system also includes a central processing unit, Memory, Input-output circuit, And imaging device, The imaging device further includes a pixel array, a lens for focusing light onto a pixel array, And data conversion circuit.  According to an embodiment, Providing an imaging system, It includes: Image sensor with image sensor pixel array, Wherein the image sensor pixel array includes red image sensor pixels configured to generate a red image signal in response to red light, Blue image sensor pixels configured to respond to blue light to produce a blue image signal, And configured to respond to at least red light, Transparent image sensor pixels that generate white image signals in green and blue light; And processing circuitry, It is configured to align red image signals, The blue image signal and the white image signal perform a filtering operation, This increase with the red image signal, The blue image signal is associated with the noise associated with the white image signal.  According to another embodiment, The processing circuit is configured to communicate with the red image signal, The noise correlation associated with the blue image signal and the white image signal is increased to be greater than the red image signal, 70% of all noise associated with blue image signals and white image signals.  According to another embodiment, The processing circuit is configured to perform a filtering operation by generating a weighted sum of image signals produced by at least 25 image sensor pixels for each image sensor pixel of a given color.  According to another embodiment, The processing circuit is configured to align the red image signal, The blue image signal and the white image signal perform a white balance operation.  According to another embodiment, The processing circuit is configured to apply a color correction matrix to the white image signal, The color correction matrix extracts a green image signal from each white image signal.  According to another embodiment, The image sensor further includes an additional image sensor pixel array.  According to another embodiment, The imaging system further includes an additional image sensor having at least one image sensor pixel array.  According to an embodiment, Providing a method of processing image data using an image sensor having a processing circuit, The image data includes a first image signal of a first color, a second image signal different from the second color of the first color, And white image signals, The method includes: Using processing circuitry, Generating, by the white image signal, a third image signal different from the first color and the third color of the second color; Using processing circuitry, Combining the first image signal, a second image signal and a third image signal to form a derived brightness value; Using processing circuitry, Based on the first image signal, Calculating the estimated brightness value by the second image signal and the white image signal; And using the processing circuit, Modifying the first image signal using the derived luminance value and the estimated luminance value, a second image signal and a third image signal.  According to another embodiment, The method further includes utilizing a processing circuit, Calculating the calibration value based on the derived luminance value and the estimated luminance value, Wherein the first image signal is modified, The second image signal and the third image signal include a first image signal, The second image signal and the third image signal are multiplied by the generated calibration value.  According to another embodiment, Combining the first image signal, The second image signal and the third image signal to form the derived luminance value include calculating a first image signal using a weighting factor, A linear combination of the second image signal and the third image signal.  According to another embodiment, Combining the first image signal, The second image signal and the third image signal to form the derived brightness value further includes a white image signal and the first image signal, The second image signal and the third image signal are combined to form a derived luminance value.  According to another embodiment, The use of the white image signal to generate the third image signal of the third color comprises extracting the third image signal from the white image signal using the color correction matrix.  According to another embodiment, The first image signal includes a red image signal, The second image signal includes a blue image signal, And the third image signal includes a green image signal.  According to another embodiment, The first image signal includes a red image signal, The second image signal includes a green image signal, And the third image signal includes a blue image signal.  According to another embodiment, The first image signal includes a blue image signal, The second image signal includes a green image signal, And the third image signal includes a red image signal.  According to an embodiment, Providing a system, It includes: Central processing unit, Memory, Input-output circuits and imaging devices, Wherein the imaging device comprises: Pixel array a lens that focuses an image on a pixel array; Image sensor with image pixel array, Wherein the image pixel array includes red image pixels configured to generate a red image signal in response to red light, Blue image pixels configured to respond to blue light to produce a blue image signal, And configured to respond to at least red light, Green and blue light produce transparent image pixels of white image signals; And processing circuitry, It is configured to generate a white image signal and at least a red image signal, The scaling value of the ratio of the linear combination of the blue image signal and the green image signal, And modifying the red image signal using the generated calibration value, Blue image signal and green image signal.  According to another embodiment, The image pixel array includes a plurality of pixel unit cells. Each pixel unit includes one of the red image pixels, One of the blue image pixels, And transparent image pixels.  According to another embodiment, The processing circuit is further configured to align the red image signal, The blue image signal and the white image signal perform a filtering operation to increase the red image signal, The blue image signal is associated with the noise associated with the white image signal.  According to another embodiment, The processing circuit is further configured to produce a difference between the white image signal and the red image signal and between the white image signal and the blue image signal.  According to another embodiment, The processing circuit is further configured to use the generated difference to the red image signal, The blue image signal and the white image signal perform a filtering operation.  According to an embodiment, Providing a method of processing an image signal using an image sensor, The image sensor includes an image sensor pixel array and a processing circuit. Wherein the image sensor pixels include red image sensor pixels, Blue image sensor pixels and transparent image sensor pixels, And the method includes: Using red image sensor pixels, Producing a red image signal in response to red light; Using blue image sensor pixels, Producing a blue image signal in response to blue light; Using transparent image sensor pixels, Responding to at least red light, Green light and blue light produce white image signals; And using the processing circuit, For red image signals, The blue image signal and the white image signal perform a filtering operation, This increase with the red image signal, The blue image signal is associated with the noise associated with the white image signal.  According to another embodiment, Performing filtering operations includes combining red image signals, The noise correlation associated with the blue image signal and the white image signal is increased to be greater than the red image signal, 70% of all noise associated with blue image signals and white image signals.  According to another embodiment, Performing the filtering operation includes generating a weighted sum of image signals produced by at least 25 image sensor pixels for each image sensor pixel of a given color.  According to an embodiment, Providing a method of processing an image signal using an image sensor, The image sensor includes an image sensor pixel array and a processing circuit. Wherein the image sensor pixel array includes a first group of image sensor pixels responsive to light of the first color, Responding to a second group of image sensor pixels of the second color of light, And a third group of image sensor pixels responsive to the third color of light, And the method includes: Using the first group of image sensor pixels, Generating a first image signal in response to light of the first color; Using the second group of image sensor pixels, Generating a second image signal in response to the light of the second color; Using a third group of image sensor pixels, Generating a third image signal in response to at least the first and second colors of light; And using the processing circuit, For the first image signal, The second image signal and the third image signal perform a filtering operation, This increase is related to the first image signal, The second image signal and the third image signal are associated with the noise.  According to another embodiment, The first image signal has a first spectral response level. The second image signal has a second spectral response level. And generating, in response to the at least first and second colors of light, the third image signal comprises generating a third image signal having a third spectral response level greater than 75 percent of the sum of the first signal response level and the second signal response level .  According to another embodiment, Generating the first image signal includes generating a red image signal in response to the red light, Generating a second image signal includes generating a blue image signal in response to the blue light, And generating the third image signal includes generating a white image signal in response to at least red light and blue light.  According to another embodiment, The red image signal has a first spectral response level. The blue image signal has a second spectral response level. And generating the white image signal includes generating a white image signal having a third spectral response level of 75 percent greater than the sum of the first spectral response level and the second spectral response level.  According to another embodiment, For the first image signal, Performing a filtering operation on the second image signal and the white image signal includes: The second image signal and the third image signal perform an infinite impulse response filter.  According to an embodiment, Providing a method of processing an image signal using an image sensor, Wherein the image sensor comprises a processing circuit, a first group of image sensor pixels that generate a first image signal in response to light of the first color, Generating a second group of image sensor pixels of the second image signal in response to the light of the second color, And a third group of image sensor pixels that generate a third image signal in response to the light of the third color, Wherein the first image signal has a first spectral response, Wherein the second image signal has a second spectral response, And wherein the third image signal has a third spectral response greater than the first spectral response and the second spectral response, And the method includes: Using processing circuitry, Using the first image signal, The second image signal and the third image signal generate the estimated brightness value; Using processing circuitry, By using the first image signal, Converting the second image signal and the third image signal into the derived three color space to generate the transformed first image signal, The transformed second image signal and the transformed third image signal; Using processing circuitry, By combining the transformed first image signal, The second image signal and the third image signal generate a derived brightness value; And using the processing circuit, Modifying the transformed first image signal, The second image signal and the third image signal cause the derived luminance value to be close to the estimated luminance value.  According to another embodiment, The first image signal, Converting the second image signal and the third image signal into the derived three color space includes including the first image signal, The second image signal and the third image signal are transformed into a standard red-green-blue space.  According to another embodiment, Generating a transformed first image signal, The second image signal and the third image signal include generating a first image signal, A linear combination of the second image signal and the third image signal.  According to another embodiment, The first image signal includes a red image signal captured by the first group of image sensor pixels in response to the red light. The second image signal includes a blue image signal captured by the second group of image sensor pixels in response to the blue light. The third image sensor signal includes a white image signal captured by a third group of image sensor pixels in response to at least blue light and red light. And generating a first image signal, The linear combination of the second image signal and the third image signal includes using red, The blue and white image signals produce a linear combination.  According to an embodiment, Providing an imaging system, It includes an image sensor having an image sensor pixel array. Wherein the image sensor pixel array includes a first group of image sensor pixels configured to generate a first image signal in response to light of the first color, Generating a second group of image sensor pixels of the second image signal in response to the light of the second color, And a third group of image sensor pixels configured to generate a third image signal in response to the light of at least the first color and the second color; And processing circuitry, It is configured to operate on the first image signal, The second image signal and the third image signal perform a filtering operation, This increase is related to the first image signal, The second image signal and the third image signal are associated with the noise.  According to another embodiment, The first image signal has a first spectral response level. The second image signal has a second spectral response level. And the third image signal has a third spectral response level that is greater than 75 percent of the sum of the first spectral response level and the second spectral response level.  According to another embodiment, The first group of image sensor pixels includes red image sensor pixels configured to generate a red image signal in response to red light. A second group of image sensor pixels includes blue image sensor pixels configured to generate blue image signals in response to blue light, And the third group of image sensor pixels includes transparent image sensor pixels configured to generate white image signals in response to at least red and blue light.  According to another embodiment, The first group of image sensor pixels includes red image sensor pixels configured to generate a red image signal in response to red light. A second group of image sensor pixels includes green image sensor pixels configured to generate a green image signal in response to green light. And the third group of image sensor pixels includes transparent image sensor pixels configured to generate white image signals in response to at least red and green light.  The foregoing merely illustrates the principles of the invention that may be practiced in other embodiments.

10‧‧‧Electronic devices

12‧‧‧ camera module

14‧‧‧ lens

16‧‧‧Image Sensor

18‧‧‧Processing Circuit

40‧‧‧Line

122‧‧‧Control and processing circuits

124‧‧‧ column decoder circuit

126‧‧‧ row decoder circuit

128‧‧‧Control path

190‧‧•Image Sensor Pixels/Image Pixels

192‧‧‧unit unit

194‧‧‧Unit unit

196‧‧‧unit unit

200‧‧‧pixel array

300‧‧‧ processor system

391‧‧‧Input/Output (I/O) devices

392‧‧‧ Random access memory

393‧‧‧ Busbar

394‧‧‧Removable memory

395‧‧‧Central Processing Unit

396‧‧‧ lens

397‧‧‧Shutter release button

2000‧‧‧ imaging device

1 is a diagram of an illustrative electronic circuit having an imaging system in accordance with an embodiment of the present invention. 2 is a diagram of an illustrative pixel array and associated control circuitry for reading pixel data from image pixels along a row line in an image sensor, in accordance with an embodiment of the present invention. 3 through 5 are diagrams of illustrative pixel unit cells having transparent filter pixels in accordance with an embodiment of the present invention. 6 is a flow diagram of illustrative steps that may be performed by processing circuitry in an imaging system to process image signals received from a filtered pixel array, in accordance with an embodiment of the present invention. 7 is a flow diagram of illustrative steps that may be performed by processing circuitry in an imaging system to demosaize and filter image signals received from a filtered pixel array, in accordance with an embodiment of the present invention. 8 is a flow diagram of illustrative steps that may be performed by processing circuitry in an imaging system to apply a point filter to an image signal received from a filtered pixel array, in accordance with an embodiment of the present invention. 9 is a block diagram of a processor system employing the embodiment of FIG. 1 in accordance with an embodiment of the present invention.

Claims (20)

An imaging system comprising: an image sensor pixel array, wherein the image sensor pixel array comprises a first image sensor pixel configured to generate a first color first image signal, configured a second image sensor pixel for generating a second image signal of a second color and a third image sensor pixel configured to generate a third image signal of a third color; and a processing circuit, wherein the processing The circuit is configured to generate a first difference based on the first image signal and the third image signal, generate a second difference based on the second image signal and the third image signal, Applying a chrominance filter to the first difference and the second difference to generate a filtered difference and adding the third image signal to the filtered difference, and in the first image signal, And performing a filtering operation on the second image signal and the third image signals. The imaging system of claim 1, wherein the processing circuit is configured to generate a weighted sum of one of the image signals produced by the at least 25 image sensor pixels for each image sensor pixel of a given color, These filtering operations are performed. The imaging system of claim 1, wherein the processing circuit is configured to be in the first image signal, the second image signal, and the third image before generating the first difference and the second difference A white balance operation is performed on the signal. The imaging system of claim 3, wherein the processing circuit is configured to apply a color correction matrix to the third image signal after generating the filtered difference values, wherein the color correction matrix is from the third image The signal extracts a fourth image signal of one of the fourth colors. The imaging system of claim 1, wherein the third image sensor pixels have a larger spectral response than the first image sensor pixels and the second image sensor pixels. The imaging system of claim 1, wherein the filtering operations increase noise correlation between the first image signals, the second image signals, and the third image signals. The imaging system of claim 6, wherein the filtering operations together increase or decrease noise fluctuations in the first image signal, the second image signals, and the third image signals in an associated manner. A method for processing image data using an imaging system having a processing circuit, the image data comprising a first image signal of a first color, a second image signal of a second color, and a third image signal of a third color, The method includes: receiving, by the processing circuit, the first image signal, the second image signal, and the third image signal, the first image signal, the second image signal, and the third image signal system And generating, by the processing circuit, a first difference value based on the first image signal and the third image signals; using the processing circuit based on the second image signals and the The third image signal generates a second difference; the processing circuit applies a chroma component to the first difference and the second difference to generate a filtered difference; and using the processing circuit to perform the same A third image signal is added to the filtered difference values to generate a first filtered value and a second filtered value. The method of claim 8, further comprising: using the processing circuit, demosaicing the first filtered values and the second filtered values to generate a first chroma de-mosaic value and a second chroma de-mosaic value. The method of claim 9, wherein the first chrominance demosaicing values and the second chrominance demosaicing values have greater noise correlation than the first image signals and the second image signals. the amount. The method of claim 9, further comprising: demosaicing the third image signals with the processing circuit to generate a demosaiced third image signal. The method of claim 11, further comprising: applying, by the processing circuit, a color correction matrix to the first chrominance demosaicing values, the second chrominance demosaicing values, and the demosaiced third image Signals to produce standard red, standard green, and standard blue (sRGB) image data. The method of claim 11, further comprising: generating, by the processing circuit, luminance-chrominance based on the first chrominance demosaicing value, the second chrominance demosaicing value, and the demosaiced third image signal - Hue (LCH) image data. The method of claim 12, further comprising: converting, by the processing circuit, the (sRGB) data to luminance-chrominance-tone (LCH) data having a luminance value; using the processing circuit, the LCH data The brightness value is replaced with the third image signal; and the processing circuit is used to convert the LCH data back to the sRGB data after replacing the brightness value in the LCH data with the third image signal. The method of claim 8, wherein applying the chrominance filter to the first difference and the second difference comprises: performing the nucleus on one of the image sensor pixels in the array The first difference and one of the second differences are weighted averaged. The method of claim 15, wherein the core comprises at least 25 adjacent image sensor pixels in the array. An imaging system includes: a first image sensor pixel that collects a first image signal of a first color; a second image sensor pixel that collects a second image signal of a second color; a sensor pixel that collects a third image signal of a third color; a processing circuit, wherein the processing circuit is configured to generate a fourth image signal of a fourth color using the third image signal, the processing circuit is grouped Combining the first image signals, the second image signals, and the fourth image signals to generate an derived luminance value, the processing circuit being configured to be based on the first image signals, the second The image signal and the third image signals calculate an estimated brightness value, and the processing circuit is configured to modify the first image signal, the second image signals, and the like using the derived brightness value and the estimated brightness value The fourth image signal. The imaging system of claim 17, wherein the processing circuit is configured to calculate a scaling value based on the derived luminance value and the estimated luminance value, and the processing circuit is configured to The image signal, the second image signal, and the fourth image signal are multiplied by the generated calibration value to modify the first image signal, the second image signal, and the fourth image signal. The imaging system of claim 17, wherein the first image sensor pixels have a first spectral response, the second image sensor pixels have a second spectral response, and the third image sensors The pixel has a third spectral response that is greater than 75 percent of the sum of the first spectral response and the second spectral response. The imaging system of claim 17, wherein the processing circuit is configured to calculate a linear combination of the first image signal, the second image signal, and the fourth image signal by using a weighting factor The first image signal, the second image signals, and the fourth image signals are combined to form the derived luminance value.