JP2000350221A - Digital camera and image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a digital camera which has good characteristics over all the sensitivity, moire and coloration of a highlighted part. SOLUTION: A complementary color filter 14 consisting of color elements of Mg, Ye, and Cy is mounted on the photodetection surface of a CCD imager 16. A pixel signal of a complementary color generated by photoelectric conversion is read out by a timing generator 20 and written to a RAM 26 through an A/D converter 18. A memory controller 28 interpolates the pixel signal stored in the RAM 26 and thus obtains two complementary colors having pixel absence. The interpolated pixel signal is further processing through the RGB conversion of an arithmetic circuit 32. The output of the arithmetic circuit 32 is processed by a signal processing circuit 34 and recorded on a recording medium 36. Filter elements of Mg, Ye and Cy are equal in sensitivity and higher than filter elements of R, G and B. Consequently, characteristics for all the sensitivity, moire, and coloration of a highlighted part are superior to those of a conventional color filter.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital camera, and more particularly to, for example, a digital camera for photographing a subject image through a single-plate color filter having a plurality of color elements.

[0002] The present invention also relates to an image pickup apparatus applied to such a digital camera.

[0003]

2. Description of the Related Art As a single-plate type color filter used in a digital camera, a primary color filter in which R (red), G (green) and B (blue) color elements are arranged as shown in FIG. was there. The color elements of Ye (yellow), Cy (cyan), Mg (magenta), and G (green) are arranged as shown in FIG. 6B or FIG. 6C, or W (white), G (Green), Cy (cyan) and Ye (yellow) are shown in FIG.
There was also a complementary color filter arranged as shown in (D).

[0004]

However, the sensitivities of the respective color elements do not always match. That is, assuming that the sensitivities of R, G and B are "1", the sensitivities of Ye, Cy and Mg are "2" and W is "3". Therefore, any of the above-described color filters is inferior in sensitivity, moiré, or coloring of a highlight portion to other color filters.

That is, in the primary color filter shown in FIG.
Since the sensitivities of the respective color elements coincide with each other, moiré and coloring of the highlight portion do not occur, but the sensitivity is lower than that of other complementary color elements. On the other hand, the complementary color filters shown in FIGS. 6B to 6D are superior in sensitivity to the primary color filter shown in FIG. 6A. Coloring occurs.

SUMMARY OF THE INVENTION It is therefore a primary object of the present invention to provide a digital camera having good characteristics in all of sensitivity, moiré and coloring of a highlight portion.

Another object of the present invention is to provide an image pickup apparatus having good characteristics in all of sensitivity, moiré and coloring of a highlight portion.

[0008]

According to a first aspect of the present invention, there is provided a color filter including only complementary color elements, an image sensor having the color filter mounted on a light receiving surface, and an interpolation process for a complementary color signal output from the image sensor. The digital camera includes an interpolation unit and a conversion unit that converts a complementary color signal output from the interpolation unit into a primary color signal.

A second aspect of the present invention is an imaging apparatus including a color filter including only complementary color elements, and an image sensor having the color filter mounted on a light receiving surface.

[0010]

According to the first aspect of the invention, a color filter including only complementary color elements is mounted on the light receiving surface of the image sensor. The interpolation means performs interpolation processing on the complementary color signal output from such an image sensor, and the conversion means converts the complementary color signal output from the interpolation means into a primary color signal.

In one embodiment of the present invention, the color filter has all kinds of color elements on two vertically adjacent lines, and one color element corresponds to one pixel of the image sensor. Here, the color elements include Ye, Cy and Mg,
When focusing on a square block composed of four pixels, any one type of color element is arranged in two pixels on one diagonal of the block, and the remaining two types of color elements are two pixels on the other diagonal. Respectively.

In another embodiment of the present invention, one frame of the complementary color signal output from the image sensor is stored in the memory, and the arithmetic means performs a predetermined operation on the complementary color signal written in the memory. As a result, the complementary color component in which each pixel is insufficient is interpolated.

In another embodiment of the present invention, the driving means drives the image sensor by a progressive scan method or an interlace scan method. Such a driving unit is activated by the activation unit in response to a photographing instruction by the instruction unit.

In the second invention, a color filter consisting of only complementary color elements is mounted on the light receiving surface of the image sensor. Here, the color filter has all kinds of color elements on two lines adjacent in the vertical direction, and one color element corresponds to one pixel of the image sensor. The color elements include Ye, Cy and Mg, and if attention is paid to a square block composed of four pixels, any one type of color element is arranged at two pixels on one diagonal line of the block, and the remaining two types of colors are arranged. The element is arranged at each of two pixels on the other diagonal line.

[0015]

According to these inventions, the color filter comprises only complementary color elements. Since the sensitivities of the complementary color elements match each other, coloring of the moiré and highlight portions does not occur. The sensitivity of the complementary color element is higher than the sensitivity of the primary color element. From the above, good characteristics can be obtained in all of the sensitivity, moiré, and coloring of the highlight portion.

The above objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

[0017]

Referring to FIG. 1, a digital camera 10 of this embodiment includes a lens 12, and a light image incident from the lens 12 is transmitted through a single-plate type complementary color filter 14 to a CC.
The D imager 16 is irradiated. The complementary color filter 14
As shown in FIG. 2, it has color elements (filter elements) of Ye, Cy and Mg. Each color element individually corresponds to a plurality of light receiving elements formed on the front surface of the CCD imager 16. That is, one color element corresponds to one pixel.

The arrangement of each color element will be specifically described. In the horizontal direction, Mg and Cy are alternately arranged for each pixel on odd lines, and Ye and Mg are alternately arranged for each pixel on even lines. You. In the vertical direction, Mg and Ye are alternately arranged in odd columns for each pixel, and Cy and Mg are alternately arranged in even columns. Therefore, when attention is paid to a square block composed of four pixels, one color element of Cy and one color element of Ye are arranged on one diagonal line, and two color elements of Mg are arranged on the other diagonal line. An imaging device is formed by such a complementary color filter 14 and the CCD imager 16.

When the operator operates the shutter button 22, the system controller 24 outputs a control signal for photographing processing to a signal processing system. The timing generator 20 is activated by such a control signal,
The CD imager 16 is driven by a progressive scan method. The pixel signal (complementary color signal) for one frame photoelectrically converted by the CCD imager 16 is output in a progressive scan manner in response to the timing signal output from the timing generator 20. As a result, a pixel signal having a Mg component and a Cy component is output on an odd line, and a pixel signal having a Ye component and a Mg component is output on an even line.

The A / D converter 18 is a CCD imager 1
The pixel signal of the complementary color system output from 6 is converted into digital data, that is, pixel data, and input to the RAM 26. The input pixel data is written to the memory area 26a by a progressive scan method by the memory controller 28 that operates according to the timing signal from the timing generator 20. In the memory area 26a, pixel data for one frame is stored in a bitmap format.

Since each pixel has only one of the complementary color components of Ye, Cy and Mg, the memory controller 28 processes the flowchart shown in FIG. 3 when reading out pixel data from the memory area 26a. Are interpolated between two complementary color components for which the number of pixels is insufficient.

Referring to FIG. 3, memory controller 28
First, in step S1, the count value x of the counter 28a indicating the address in the row direction and the count value y of the counter 28b indicating the address in the column direction are set to "2", and then (x-1, y) in step S3. -1), (x, y
-1), (x + 1, y-1), (x-1, y), (x,
y), (x + 1, y), (x-1, y + 1), (x, y +
Pixel data is read from each of the addresses (1) and (x + 1, y + 1). That is, pixel data is read from nine addresses centered on the addresses indicated by the current count values x and y.

Subsequently, in steps S5 to S9, the memory controller 28 determines the count values x and y, and proceeds to any one of steps S11 to S17 according to the determination result. Specifically, if both the count values x and y are even numbers, steps S5 to S11
If the count value x is odd and the count value y is even, the process proceeds from step S7 to step S13. If the count value x is even and the count value y is odd, step S9
Then, the process proceeds to step S15, and if both the count values x and y are odd, the process proceeds from step S9 to step S17.

In step S11, an interpolation process is performed according to equation (1), and in step S13, an interpolation process is performed according to equation (2). In step S15, the interpolation processing is performed according to equation 3, and in step S17, the interpolation processing is performed according to equation 4.

[0025]

## EQU1 ## Mg '(x, y) = D (x, y) Ye' (x, y) = [D (x-1, y) + D (x + 1, y)] / 2 Cy '( x, y) = [D (x, y-1) + D (x, y + 1)] / 2

[0026]

(2) Mg '(x, y) = [D (x, y-1) + D (x-1, y) + D (x +
1, y) + D (x, y + 1)] / 4 Ye '(x, y) = D (x, y) Cy' (x, y) = [D (x-1, y-1) + D (x + 1, y-1) + D (x-1, y +
1) + D (x + 1, y + 1)] / 4

[0027]

(3) Mg '(x, y) = [D (x, y-1) + D (x-1, y) + D (x +
1, y) + D (x, y + 1)] / 4 Ye '(x, y) = [D (x-1, y-1) + D (x + 1, y-1) + D (x -1, y +
1) + D (x + 1, y + 1)] / 4 Cy '(x, y) = D (x, y)

[0028]

## EQU4 ## Mg '(x, y) = D (x, y) Ye' (x, y) = [D (x, y-1) + D (x, y + 1)] / 2 Cy '( x, y) = [D (x-1, y) + D (x + 1, y)] / 2 Referring to FIG. 2, for example, the pixel of (x, y) = (2, 2) is Mg Color components only. For this reason, the operation of Formula 1 is performed, and the Ye ′ component and the Cy ′ component at (x, y) = (2, 2) are obtained from the Ye component and the Cy component in the peripheral pixels. (X, y) = (3,
The pixel of 2) also has only the Ye color component. Therefore, by calculating Equation 2, the surrounding Mg component and Cy are calculated.
The Mg ′ component and the Cy ′ component at (x, y) = (3, 2) are obtained from the components. Similarly, when (x, y) = (2, 3) having only the Cy component, the Mg ′ component and the Cy ′ component are obtained by Expression 3, and (x, y) = (3, 3) having only the Mg component. In (4), the Ye ′ component and the Cy ′ component are obtained from Expression 4.

In this way, a complementary color component lacking the pixel data D (x, y) stored at the address (x, y) is obtained. Note that, in order to distinguish between the complementary color component before the interpolation processing and the complementary color component after the interpolation processing, a dash “′” is added to the complementary color component after the interpolation processing.

In step S19, the color components Mg ', Ye' and C
y ′ is output in a serial manner. Memory controller 2
8 then determines in step S21 whether the count value x is a predetermined value, that is, the number of horizontal pixels minus one. If "NO" here, the count value x is incremented in step S23 to process the block of 9 pixels shifted by one column in the row direction. Then, the process returns to step S3.

On the other hand, if "YES" in the step S21, the memory controller 28 proceeds to a step S25,
It is determined whether or not the count value y is a predetermined value, that is, the number of vertical pixels minus one. If "NO", the count value x is returned to "2" and the count value y is incremented in step S27 in order to shift one line in the column direction and process the block of 9 pixels located at the left end. Then, the process returns to step S3. Step S2
If "YES" in 5, the process ends.

Y serially output from RAM 26
Complementary color data of e ', Cy' and Mg 'are serial /
The signal is input to the parallel conversion circuit 30. The serial / parallel conversion circuit 30 performs serial / parallel conversion on the input complementary color data for every three components. Therefore, the Ye ′ component, the Cy ′ component, and the Mg ′ component corresponding to the same pixel are output from the serial / parallel conversion circuit 30 at the same time. The arithmetic circuit 32 performs a matrix operation on the Ye ′ component, Cy ′, and Mg ′ components output from the serial / parallel conversion circuit 30 to generate pixel data having primary color components (R component, G component, and B component). I do. That is, the input complementary color data is subjected to RGB conversion to generate primary color pixel data. The generated pixel data is
The signal processing circuit 34 performs processing such as white balance correction, YUV conversion, and JPEG compression. The output from the signal processing circuit 34 is recorded on a recording medium 36 thereafter.

When the shutter button 22 is pressed, the above-described photographing processing is performed, whereby a subject image for one screen is recorded on the recording medium 36.

The conventional color filter shown in FIGS. 6A to 6D and the color filter shown in FIG. 2 differ in sensitivity, moiré and coloring of a highlight portion as follows. When the sensitivities of R, G and B are set to “1”,
The sensitivity of Mg, Ye and Cy is “2”, and the sensitivity of W is “3”. Equation 5 holds between the respective color components.

[0035]

## EQU5 ## Ye = R + G Cy = G + B Mg = R + BW = R + G + BR = G = B The color filters shown in FIG. 6 (A) to FIG.
A plurality of blocks indicated by thick lines in FIG.
The average values Sa to Sd of the sensitivities in such a block are represented by Expression 6, which is the sensitivity of the color filter shown in FIGS. 6A to 6D.

[0036]

[Equation 6] Sa = (1 + 1 + 1 + 1) / 4 = 1 Sb = (2 + 2 + 2 + 1 + 2 + 2 + 1 + 2) / 8 = 7/4 Sc = (2 + 2 + 1 + 2) / 4 = 7/4 Sd = (3 + 1 + 2 + 2) / 4 = 2 Moiré is two adjacent in the vertical or horizontal direction
This is caused by a difference in sensitivity between pixels. For this reason, FIG.
Considering the sensitivity difference between the color filters shown in FIG. 6D, the following result is obtained.

The sensitivity difference in the block shown by the thick line in FIG. 6A is expressed by the following equation (7) in both the vertical and horizontal directions.
It is represented by From this, it can be seen that moire does not occur in the color filter shown in FIG.

[0038]

| (R + G) − (G + B) | = | (1 + 1) − (1 + 1) | = 0 Also shown by thick lines in FIGS. 6 (B) and 6 (C). For the block, the sensitivity difference in the vertical direction is represented by Equation 8,
The sensitivity difference in the horizontal direction is represented by Expression 9. Than this,
It can be seen that 4: 3 modulation is performed in any direction, and when a subject image whose luminance changes in the horizontal direction and the vertical direction is captured, luminance moire occurs.

[0039]

(Ye + Cy) − (Mg + G) | = | (2 + 2) −
(2 + 1) | = 1

[0040]

(Ye + G) − (Cy + Mg) | = | (2 + 1) − (2
+ 2) | = 1 or | (Ye + Mg) − (Cy + G) | = | (2 + 2) − (2 + 1) |
= 1 Further, in the case of the block indicated by the thick line in FIG. 6D, the sensitivity difference in the vertical direction is represented by Expression 10, and the sensitivity difference in the horizontal direction is represented by Expression 11. From this, it can be seen that, even when a subject image whose luminance changes in the vertical direction is photographed, luminance moire does not occur, but when a subject image whose luminance changes in the horizontal direction is photographed, luminance moire occurs.

[0041]

(W + G) − (Ye + Cy) | = | (3 + 1) −
(2 + 2) | = 0

[0042]

(W + Cy) − (Ye + G) | = | (3 + 2) −
(2 + 1) | = 2 Next, the coloring of the highlighted portion will be considered. Equation (5) is equivalent to Equation (12).
According to FIG. 7, the relationship shown in FIG. 7 is established between the amount of incident light and the output of the image sensor.

[0043]

R = G = BW = 3G Ye = Cy = Mg = 2G According to FIG. 7, R = G = B holds for any amount of incident light. Therefore, when the color filter shown in FIG. 6A is used, no coloring occurs in the highlight portion.
On the other hand, the inclination of the straight line differs between Ye, Cy and Mg and G. Therefore, FIG. 6B or FIG.
When the color filters shown in (1) and (2) are used, the color balance is lost when the amount of incident light is in the range of, and a highlight / green phenomenon occurs. Since the inclination of the straight line is different between W and Ye, Cy, and Mg, the highlight / green phenomenon occurs in the range of-even with the color filter shown in FIG.

From the above description, when the color filter shown in FIG. 6A is used, moire and highlight portions are not colored, but the sensitivity is lower than other color filters. When the color filters shown in FIGS. 6B and 6C are used, the sensitivity is 1.75 of the color filter shown in FIG.
Although it is doubled, coloring of moiré and highlight portions occurs. The sensitivity of the color filter shown in FIG. 6D is 2.0 times that of the color filter shown in FIG.
Moiré and highlights are colored. That is,
Any of the color filters shown in FIGS.
Good characteristics cannot be obtained in all of sensitivity, moiré and coloring of a highlight portion.

On the other hand, the complementary color filter 14 of this embodiment shown in FIG. 2 has the following characteristics. First, considering the sensitivity, the average value Se of the sensitivities of the color elements included in the blocks indicated by the thick lines in FIG. Further, the average value Sa to the sensitivity of the conventional color filter is
Equation 14 holds true as compared with Sd, and it can be seen from this that the sensitivity of the complementary color filter 14 is the highest.

[0046]

[Expression 13] Se = (2 + 2 + 2 + 2) / 4 = 2

[0047]

[Equation 14] Sa: Sb: Sc: Sd: Se = 4: 7: 7: 8: 8 Next, considering the luminance moiré, the sensitivity difference in the vertical direction of the block indicated by the thick line is expressed by Equation 15.
The sensitivity difference in the horizontal direction is represented by Expression 16. In other words, the sensitivity of each color element matches each other,
Even when a subject image whose luminance changes in the vertical direction and the horizontal direction is photographed, luminance moire does not occur.

[0048]

| (Mg + Cy) − (Ye + Mg) | = | (2 + 2) −
(2 + 2) | = 0

[0049]

(Mg + Ye) − (Cy + Mg) | = | (2 + 2) −
(2 + 2) | = 0 Further, since the inclinations of Ye, Cy, and Mg are equal in the characteristic of FIG. 7, coloring of the highlighted portion does not occur.

As described above, the color filter applied to this embodiment is superior to the conventional color filter in all of the characteristics of sensitivity, moiré and coloring of a highlight portion.
Therefore, the quality of the captured image can be improved.

Such an effect is obtained when the color filters are Ye, C
The color elements need not be arranged as shown in FIG. 2 as long as they are constituted only by the y and Mg color elements. That is, the color elements of Ye, Cy and Mg may be arranged as shown in FIG. 4 or may be arranged as shown in FIG. According to FIG. 4, one color element of Mg and Cy is provided on one diagonal line of a block shown by a thick line,
Two Ye color elements are provided on the other diagonal line. Further, according to FIG. 5, one color element of Mg and one color element of Ye are provided on one diagonal line of the same block, and two color elements of Cy are provided on the other diagonal line.

However, when the color filter shown in FIG. 4 is used, in the steps S11 to S17 of FIG.
It is necessary to calculate Expression 20, and when using the color filter shown in FIG. 5, it is necessary to calculate Expressions 21 to 24 in the same steps S11 to S17.

[0053]

Ye '(x, y) = D (x, y) Mg' (x, y) = [D (x-1, y) + D (x + 1, y)] / 2 Cy '( x, y) = [D (x, y-1) + D (x, y + 1)] / 2

[0054]

Ye '(x, y) = [D (x, y-1) + D (x-1, y) + D (x
+1, y) + D (x, y + 1)] / 4 Mg '(x, y) = D (x, y) Cy' (x, y) = [D (x-1, y-1) + D (x + 1, y-1) + D (x-1, y +
1) + D (x + 1, y + 1)] / 4

[0055]

(19) Ye '(x, y) = [D (x, y-1) + D (x-1, y) + D (x
+1, y) + D (x, y + 1)] / 4 Mg '(x, y) = [D (x-1, y-1) + D (x + 1, y-1) + D ( x-1, y +
1) + D (x + 1, y + 1)] / 4 Cy '(x, y) = D (x, y)

[0056]

Ye ′ (x, y) = D (x, y) Mg ′ (x, y) = [D (x, y−1) + D (x, y + 1)] / 2 Cy ′ ( x, y) = [D (x-1, y) + D (x + 1, y)] / 2

[0057]

## EQU21 ## Cy '(x, y) = D (x, y) Ye' (x, y) = [D (x-1, y) + D (x + 1, y)] / 2 Mg '( x, y) = [D (x, y-1) + D (x, y + 1)] / 2

[0058]

## EQU22 ## Cy '(x, y) = [D (x, y-1) + D (x-1, y) + D (x
+1, y) + D (x, y + 1)] / 4 Ye '(x, y) = D (x, y) Mg' (x, y) = [D (x-1, y-1) + D (x + 1, y-1) + D (x-1, y +
1) + D (x + 1, y + 1)] / 4

[0059]

## EQU23 ## Cy '(x, y) = [D (x, y-1) + D (x-1, y) + D (x
+1, y) + D (x, y + 1)] / 4 Ye '(x, y) = [D (x-1, y-1) + D (x + 1, y-1) + D ( x-1, y +
1) + D (x + 1, y + 1)] / 4 Mg '(x, y) = D (x, y)

[0060]

## EQU24 ## Cy '(x, y) = D (x, y) Ye' (x, y) = [D (x, y-1) + D (x, y + 1)] / 2 Mg '( x, y) = [D (x-1, y) + D (x + 1, y)] / 2 The color filters shown in FIGS. 2, 4 and 5 also output pixel signals in an interlaced scan mode. The present invention can also be applied to a digital camera having a CCD imager. In such a digital camera, the TG 20 shown in FIG. 1 drives the CCD imager 16 in an interlaced scan mode, and the memory controller 28 converts the pixel data output from the A / D converter 18 into a memory area 26a in an interlaced scan mode. Can be realized by writing to Since pixel data for one frame is stored in the memory area 26a in a bitmap format, subsequent processing may be the same as described above.

Although this embodiment has been described assuming a digital still camera for recording a still image, the present invention can be applied to a video movie for recording a moving image. That is, the digital camera of the present invention includes a video movie in addition to the digital still camera.

In this embodiment, a CCD imager is used as an image sensor. However, it goes without saying that a CMOS sensor may be used instead of the CCD imager.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of the present invention.

FIG. 2 is an illustrative view showing one example of a color filter applied to the embodiment in FIG. 1;

FIG. 3 is a flowchart showing a part of the operation of the embodiment in FIG. 1;

FIG. 4 is an illustrative view showing another example of the color filter applied to the embodiment in FIG. 1;

FIG. 5 is an illustrative view showing another example of the color filter applied to the embodiment in FIG. 1;

6A is an illustrative view showing one example of a conventional primary color filter, FIG. 6B is an illustrative view showing one example of a conventional complementary color filter, and FIG. 6C is another example of a conventional complementary color filter; And (D) is an illustrative view showing another example of the conventional complementary color filter.

FIG. 7 is a graph showing characteristics of each filter element.

[Explanation of symbols]

 10 Digital camera 14 Complementary color filter 16 CCD imager 20 Timing generator 26 RAM 28 Memory controller

Claims (9)

[Claims] A color filter including only complementary color elements; an image sensor having the color filter mounted on a light receiving surface; an interpolation unit for performing an interpolation process on a complementary color signal output from the image sensor; A digital camera, comprising: conversion means for converting an output complementary color signal into a primary color signal. 2. The digital camera according to claim 1, wherein said color filter has all kinds of color elements in two lines vertically adjacent to each other, and one color element corresponds to one pixel of said image sensor. 3. The color element includes Ye, Cy, and Mg. When attention is paid to a square block composed of four pixels, any one type of color element is arranged at two pixels on one diagonal line of the block. 3. The digital camera according to claim 2, wherein the remaining two types of color elements are arranged in two pixels on the other diagonal line of the block. 4. The memory according to claim 1, wherein said interpolation means stores a complementary color signal for one frame output from said image sensor.
4. The digital camera according to claim 1, further comprising an arithmetic unit for performing a predetermined arithmetic operation on the complementary color signal written in said memory. 5. The image processing apparatus according to claim 1, further comprising a driving unit that drives the image sensor by a progressive scan method.
A digital camera according to any one of claims 1 to 4. 6. The digital camera according to claim 1, further comprising driving means for driving said image sensor by an interlaced scanning method. 7. An imaging apparatus comprising: a color filter including only complementary color elements; and an image sensor in which the color filter is mounted on a light receiving surface. 8. The color filter according to claim 1, wherein the color filters are two adjacent color filters.
The imaging device according to claim 7, wherein the line has all kinds of color elements, and one color element corresponds to one pixel of the image sensor. 9. The color element includes Ye, Cy, and Mg. When attention is paid to a square block composed of four pixels, any one of the color elements is arranged at two pixels on one diagonal line of the block. 9. The imaging apparatus according to claim 8, wherein the remaining two types of color elements are arranged in two pixels on the other diagonal line of the block.