JP2006024999A - Pixel interpolation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide technology of enhancing pixel interpolation accuracy. <P>SOLUTION: Correlation values in a vertical direction, a horizontal direction, an oblique A direction and an oblique B direction of a pixel signal of the RGB Bayer arrangement are obtained. The correlation values are calculated by using a G signal. The correlation values in the vertical direction and the horizontal direction are made a first set and the correlation values in the oblique A, B directions are made a second set, and the first or second set whose deviation in the correlation values is greater is selected. Then a direction with higher correlation is selected in the set with greater deviation in the correlation values, and pixels are interpolated in the selected direction. Alternatively, pixels are interpolated by applying weighting in response to the ratio of the degree of correlation to the two directions belonging to the set with a greater deviation. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a pixel interpolation method used for a digital camera or the like.

  A CCD image sensor used in a digital camera or the like photoelectrically converts light received through a color filter and outputs pixel data. Examples of the color filter include an RGB filter and a YMCK color filter. In the single-plate color filter, pixel data of one color is output for one pixel. For example, in the case of an RGB color filter, pixel data of an R (red) component, a G (green) component, or a B (blue) component is output for one pixel.

  For this reason, for pixel data output from a CCD image sensor having a single-plate color filter, only single-color image data is output for each pixel. Therefore, interpolation processing is performed for image data of other color components. ing. Various algorithms are used for this interpolation processing.

  However, since the interpolation process is a process for estimating the pixel data of the target pixel from the surrounding pixel data, an incorrect interpolation may be performed depending on the content of the image. Therefore, by calculating the difference between the G signal in the vertical direction and the difference in the G signal in the horizontal direction, the degree of correlation in the vertical direction and the horizontal direction is obtained, and pixel interpolation is performed according to this degree of correlation. There is a technology to make it happen.

  Further, the following patent document discloses a technique for performing pixel interpolation by evaluating a correlation degree in an oblique direction in addition to a vertical direction.

JP-A-5-153562 JP 2001-94951 A JP 2003-230109 A

  In the above method, the interpolation accuracy of the pixels is improved by evaluating the correlation degree in the oblique direction, but the interpolation method using the correlation degree in the vertical and horizontal directions is simply expanded as it is. In some cases, interpolation is performed.

  In view of the above problems, an object of the present invention is to provide a technique for improving interpolation accuracy in a pixel interpolation technique using a plurality of correlation degrees.

  In order to solve the above-described problem, the invention according to claim 1 includes a first step of inputting pixel data of a predetermined color space, a pixel correlation degree in the first direction using peripheral pixels of the target pixel, The pixel correlation in a second direction orthogonal to the first direction, the pixel correlation in a third direction different from the first and second directions, and a first orthogonal to the third direction. A second step of obtaining the correlation degree of the pixels in the four directions, and a third step of evaluating the correlation degree of the peripheral region of the pixel of interest using the four correlation degrees in the first to fourth directions. And a fourth step of performing pixel interpolation of the pixel of interest with respect to the direction evaluated as having a high degree of correlation in the third step.

  According to a second aspect of the present invention, in the pixel interpolation method according to the first aspect, in the third step, the first direction and the second direction are a first set, and the third direction is When the fourth direction is the second set, a step of selecting a set having a large correlation bias among the first set and the second set as a set having a high correlation degree. And the fourth step includes a fifth step of performing pixel interpolation with respect to a direction included in the set having a large correlation degree bias.

  According to a third aspect of the present invention, in the pixel interpolation method according to the second aspect, in the five steps, pixel interpolation is performed only in a direction having a high degree of correlation among two directions included in the group having a large correlation degree bias. The process of performing is included.

  According to a fourth aspect of the present invention, in the pixel interpolation method according to the second aspect, in the five steps, weighting is performed in accordance with the ratio of the correlation degree for the two directions included in the group having the large correlation degree bias. And a step of performing pixel interpolation.

  According to a fifth aspect of the present invention, the first step of inputting pixel data in a predetermined color space, and the pixel correlation in the first direction using the peripheral pixels of the target pixel, and the orthogonality to the first direction The pixel correlation in the second direction, the pixel correlation in the third direction different from the first and second directions, and the pixel correlation in the fourth direction orthogonal to the third direction. The pixel interpolation is performed by performing weighting according to the ratio of the correlation degree in the first to fourth directions based on the second step for obtaining the degree and the correlation degree obtained in the second step. Performing the process.

  According to a sixth aspect of the present invention, in the pixel interpolation method according to any one of the first to fifth aspects, the pixel data of the predetermined color space includes pixels of each color component included in the predetermined color space. Including arranged data.

  According to a seventh aspect of the present invention, in the pixel interpolation method according to any one of the first to sixth aspects, the first direction is a vertical direction or a horizontal direction, and the second direction is relative to the vertical direction. It is characterized by a direction having an inclination of 45 degrees.

  According to an eighth aspect of the present invention, in the pixel interpolation method according to any one of the first to seventh aspects, in the second step of the first or second aspect, the pixel data is pixel data in an RGB color space. In some cases, the degree of correlation is obtained for the G signal, or when the pixel data is pixel data in a predetermined color space including the luminance signal.

  Since the present invention obtains a plurality of sets of correlations in two directions intersecting at right angles using pixel data of the peripheral region of the pixel of interest, and executes the interpolation process using these correlations as a criterion, more accurate interpolation process Can be performed.

  In addition, since the present invention considers the degree of correlation in a plurality of oblique directions, even in the case where the number of pixels in which the pixels are arranged in an oblique direction, such as octagons and dodecagons, increases in the future. It is valid.

  Hereinafter, an embodiment when the pixel interpolation method of the present invention is applied to a digital camera will be described with reference to the drawings. FIG. 1 is a functional block diagram of a digital camera 1 according to the embodiment. The digital camera 1 includes a CCD 2 that is an image sensor, an A / D conversion circuit 3, an image processing circuit 4, a memory 6, and an LCD 7. The image processing circuit 4 includes a color interpolation circuit 5 to which the pixel interpolation method of the present invention is applied.

  In this embodiment, the CCD 2 includes an RGB Bayer array color filter array. Accordingly, the light incident from the subject is decomposed into light of each RGB color by the color filter of the RGB Bayer array. The RGB light components are photoelectrically converted in each pixel of the CCD 2 and accumulated as charge information. The CCD 2 outputs the accumulated charge information as an analog image signal. The analog image signal is digitally converted by the A / D conversion circuit 3, and the A / D conversion circuit 3 outputs the RGB Bayer array image signal as a digital signal. As shown in FIG. 2, the image signal output from the A / D conversion circuit 3 includes a horizontal line in which an R (red) signal and a G (green) signal are output alternately, a G (green) signal, and a B ( (Blue) is a signal that is alternately output to horizontal lines from which signals are output alternately.

  In this embodiment, the image signal output from the CCD 2 is an RGB Bayer array image signal. However, the pixel interpolation method of the present invention is output from an imaging sensor having a complementary color filter array. The present invention can be applied to various image signals such as image signals.

  The Bayer array image signal output from the A / D conversion circuit 3 is input to the image processing circuit 4, and color interpolation processing is performed in the color interpolation circuit 5. In this color interpolation process, the pixel interpolation method that is a feature of the present invention is executed. Thereby, one pixel is converted into an image signal having signals of three RGB color components. Then, after other correction processing, compression processing, and the like are performed, the image signal is stored in the memory 6. Alternatively, it is output to the LCD 7.

{Embodiment 1}: Processing for pixel array of general RGB Bayer array A pixel interpolation method according to Embodiment 1 of the present invention will be described. In the first embodiment, the pixel signal output from the CCD 2 is a pixel signal of a general RGB Bayer array as shown in FIG.

  In FIG. 2, the two-digit numbers following the symbols R (red), G (green), and B (blue) representing the color components are the number of rows in the pixel array in the first digit and the number of columns in the pixel array in the second digit. Is shown. In this example, even-numbered lines are lines in which R and G are alternately read out such as R → G → R → G..., And odd-numbered lines are G → B → G → B. In this manner, the G and B pixels are read out alternately.

  FIG. 3 shows four directions for evaluating the degree of correlation for the pixel array shown in FIG. In the present embodiment, two sets in two directions orthogonal to each other are used. The first set is a set in the vertical direction and the horizontal direction orthogonal to each other, and the second set is a set in the diagonal A direction and the diagonal B direction orthogonal to each other. In this embodiment, the diagonal A direction is a direction inclined 45 degrees to the left from the vertical direction, and the diagonal B direction is a direction inclined 45 degrees to the right from the vertical direction.

  2 and 4-9, the circle drawn with a thick solid line is a G signal, the circle drawn with a thin solid line is an R signal, and the circle drawn with a broken line shows a B signal. Yes. In each drawing, the notation such as R00 and G01 is used as a name for identifying a pixel. In the equations 1 to 20, the notation such as R00 and G01 indicates the pixel value of each pixel. And

<Correlation value calculation method>
4A to 4D show a correlation value calculation method when the target pixel is a G signal (here, G22). In this embodiment, the G signal included in the 5 × 5 peripheral area centered on the pixel of interest is used as the correlation value calculation target area, but the range of the peripheral area is not particularly limited. In addition, a 7 × 7 area or the like may be used.

  FIG. 4A shows a method of calculating the correlation value (Cvertical) in the vertical direction, and the calculation formula is expressed by Formula 1.

  That is, using the pixel values of the seven G signals (G11, G31, G02, G22, G42, G13, G33), four sets of vertical pixel difference absolute values are calculated, and the average value of these is calculated as a correlation value (Cvertical ). The smaller the correlation value (Cvertical), the higher the degree of correlation in the vertical direction.

  FIG. 4B shows a method of calculating the horizontal correlation value (Chorizontal), and the calculation formula thereof is expressed by the following equation (2).

  That is, using the pixel values of seven G signals (G11, G13, G20, G22, G24, G31, G33), four sets of horizontal pixel difference absolute values are calculated, and the average value of these is calculated as a correlation value (Chorizontal ). The smaller the correlation value (Chorizontal), the higher the degree of correlation in the horizontal direction.

  FIG. 4C shows a method of calculating the correlation value (CdiagonalA) in the diagonal A direction, and the calculation formula is Equation 3.

  That is, using the pixel values of the five G signals (G00, G11, G22, G33, G44), four sets of pixel difference absolute values in the diagonal A direction are calculated, and the average value of these is set as the correlation value (CdiagonalA). It is. The smaller the correlation value (CdiagonalA), the higher the degree of correlation in the diagonal A direction.

  FIG. 4D shows a method of calculating the correlation value (CdiagonalB) in the oblique B direction, and the calculation formula is Equation 4.

  That is, using the pixel values of the five G signals (G04, G13, G22, G31, G40), four sets of pixel difference absolute values in the diagonal B direction are calculated, and the average value of these is set as the correlation value (CdiagonalB). It is. The smaller the correlation value (CdiagonalB) is, the higher the degree of correlation is.

  FIGS. 5A to 5D show a correlation value calculation method when the target pixel is an R signal (here, R22). In this embodiment, the G signal included in the 5 × 5 peripheral area centered on the target pixel is used as a target pixel for correlation value calculation, but the range of the peripheral area is not particularly limited. In addition, a 3 × 3 region or a 7 × 7 region may be used.

  FIG. 5A shows a method for calculating a correlation value (Cvertical) in the vertical direction.

  That is, using the pixel values of the eight G signals (G01, G21, G41, G12, G32, G03, G23, G43), five sets of absolute pixel difference values in the vertical direction are calculated, and the average value of these is calculated as the correlation value. (Cvertical). The smaller the correlation value (Cvertical) is, the higher the degree of correlation is.

  FIG. 5B shows a method of calculating the horizontal correlation value (Chorizontal), and the calculation formula thereof is expressed by Equation (6).

  That is, using the pixel values of eight G signals (G10, G12, G14, G21, G23, G30, G32, G34), five sets of horizontal pixel difference absolute values are calculated, and the average value of these is calculated as a correlation value. (Chorizontal). The smaller the correlation value (Chorizontal), the higher the degree of correlation.

  FIG. 5C shows a method of calculating the correlation value (CdiagonalA) in the diagonal A direction, and the calculation formula thereof is expressed by Equation 7.

  That is, using the pixel values of the eight G signals (G10, G21, G32, G43, G01, G12, G23, G34), 6 sets of pixel difference absolute values in the diagonal A direction are calculated, and their average values are correlated. The value (CdiagonalA) is used. The smaller the correlation value (CdiagonalA), the higher the degree of correlation.

  FIG. 5D shows a method of calculating the correlation value (CdiagonalB) in the oblique B direction, and the calculation formula thereof is expressed by Equation 8.

  That is, using the pixel values of the eight G signals (G03, G12, G21, G30, G14, G23, G32, G41), six sets of absolute pixel difference values in the diagonal B direction are calculated, and their average values are correlated. The value (CdiagonalB) is used. The smaller the correlation value (CdiagonalB) is, the higher the degree of correlation is.

  The pixel interpolation method when the B signal is the target pixel is the same as that when the R signal is the target pixel. That is, in FIG. 5, the R signal and the B signal are interchanged, and similarly, the correlation values in the horizontal, vertical, diagonal A, and diagonal B directions are calculated by using the equations (5) to (8). It is possible to evaluate the degree of correlation.

  Next, three methods for evaluating the correlation degree in the four directions from the correlation values in the four directions obtained by the above-described method and performing pixel interpolation will be described.

<First pixel interpolation method>
First, the first pixel interpolation method will be described. In the first method, among the vertical direction, the horizontal direction, the oblique A direction, and the oblique B direction, a pair composed of the vertical direction and the horizontal direction is defined as a first pair, and a pair composed of the oblique A direction and the oblique B direction. As the second group, the degree of correlation is evaluated, and a group having a large correlation degree bias is selected.

  That is, the absolute difference value | Cvertical−Chorizontal | of the correlation value in the vertical direction (Cvertical) and the correlation value in the horizontal direction (Chorizontal) is calculated in the first group, and the correlation value in the diagonal A direction ( The absolute difference value | CdiagonalA−CdiagonalB | of the correlation value (CdiagonalB) in the diagonal B direction is calculated. Then, the magnitudes of the two absolute differences are evaluated, and the group having the large difference absolute value is selected as the group having the large correlation degree bias.

  For example, if correlation value (Cvertical) = 20, correlation value (Chorizontal) = 10, correlation value in diagonal A direction (CdiagonalA) = 10, correlation value in diagonal B direction (CdiagonalB) = 5, | 20 −10 |> | 10−5 |, and the first set having a large correlation degree bias is selected.

  When a group having a large correlation degree is determined, a direction having a large correlation degree is then selected from the group having a large correlation degree bias. In the example shown above, the first set is selected. Among them, the correlation value in the vertical direction (Cvertical) = 20 and the correlation value in the horizontal direction (Chorizontal) = 10. A direction with a higher degree, that is, a horizontal direction with a lower correlation value is selected.

  In this way, a group with a large correlation degree bias is selected, and further, when a direction with a large correlation degree is selected from a group with a large correlation degree bias, pixel interpolation is performed for the direction with the large correlation degree. Do it. Specifically, as shown in FIG. 4, if the G signal is a target pixel, the R signal or the B signal is subjected to pixel interpolation using the same color pixel in the selected direction for this target pixel. . As shown in FIG. 5, if the R signal is the target pixel, the G signal or the B signal is subjected to pixel interpolation using the same color pixel in the selected direction for the target pixel. Similarly, if the target pixel is B, the G signal or the R signal is subjected to pixel interpolation using the same color pixel in the selected direction for this target pixel.

  Further, when there are pixels of color components to be interpolated in the line in the direction to be interpolated, the pixel interpolation process can be executed by calculating an average value or performing linear interpolation. However, depending on the pixel arrangement, there may be no color component pixel to be interpolated in the line in the direction of interpolation. In such a case, a method of estimating a pixel value of a pixel to be interpolated from a change rate (Laplacian) in a direction perpendicular to a line in a direction to be interpolated may be taken. However, the interpolation method is not particularly limited as long as the evaluation of the correlation degree in the present invention is taken into consideration.

  In the above example, that is, when the correlation value (Cvertical) = 20, the correlation value (Chorizontal) = 10, the correlation value in the diagonal A direction (CdiagonalA) = 10, and the correlation value in the diagonal B direction (CdiagonalB) = 5 Since the first set is selected and the horizontal direction is selected, when the G signal is the target pixel, the R signal of the target pixel is interpolated using the horizontal R signal, and the horizontal direction is selected. The B signal of the target pixel is pixel-interpolated using the B signal (there is no B signal on the horizontal line, and as shown above, the B signal estimated in consideration of the rate of change in the vertical direction). . Similarly, when the R signal is the target pixel, the B signal in the horizontal direction (there is no B signal on the horizontal line. Signal) is used to interpolate the B signal of the target pixel, and the horizontal G signal is used to interpolate the G signal of the target pixel. If the B signal is the target pixel, the horizontal R signal ( Since there is no R signal on the horizontal line, as shown above, the R signal of the pixel of interest is interpolated using the R signal estimated in consideration of the rate of change in the vertical direction, and the horizontal G signal Is used to interpolate the G signal of the pixel of interest.

<Second pixel interpolation method>
Next, the second pixel interpolation method will be described. Similarly to the first pixel interpolation method described above, also in the second method, a group having a large correlation degree bias is selected from the first group and the second group. In the second method, the correlation degree in two directions included in the selected group having a large correlation degree is used. Specifically, the two directions included in the selected group having a high degree of correlation are weighted at a ratio according to the degree of correlation, and pixel interpolation is performed in consideration of pixel changes in the two directions.

  In the above example, in the selected first group with a large degree of correlation, correlation value (Cvertical) = 20, correlation value (Chorizontal) = 10, and the ratio of correlation values is 2/3: 1. Therefore, the correlation ratio is 1/3: 2/3. Here, assuming that the pixel value obtained by performing pixel interpolation in the vertical direction for the target pixel is P1, and the pixel value obtained by performing pixel interpolation in the horizontal direction is P2, “P1 × 1/3 + P2” weighted according to the correlation ratio The value “× 2/3” is set as the pixel value of the target pixel. Thereby, it is possible to execute an interpolation process with high accuracy in consideration of the degree of correlation in two vertical and horizontal directions.

  In general, the correlation values of the two directions X and Y included in the selected group with a large correlation degree are represented by CX and CY (the calculation method of CX and CY is expressed by Formula 1 to Formula 8. (Similar to the method)), and the pixel values obtained by performing pixel interpolation in the X and Y directions are PX and PY, the ratio of the correlation degree between the two directions X and Y is expressed by Equation (9).

  Therefore, the pixel interpolation value weighted according to the ratio of the correlation degree between the two directions X and Y is expressed by the following equation (10).

  In the above example, the correlation ratio is used as a weighting coefficient as it is, and the simple proportional distribution is performed according to the correlation. However, the functions shown in FIGS. 10A and 10B are used. A weighting coefficient may be calculated. In the figure, the horizontal axis indicates the ratio of the degree of correlation (corresponding to the weighting coefficient when performing simple proportional distribution), and the vertical axis indicates the value of the weighting coefficient determined by each function.

  As shown in the figure, the function for obtaining the weighting coefficient is not a simple linear function, but is set so that the inclination thereof varies depending on the correlation ratio region. That is, different functions are applied depending on the correlation ratio. The gradient of the function in each area shown in FIG. 10 is an example, but the function in each area can be freely set in this way.

  When the function as shown in FIG. 10 is used, the function value may be stored in the ROM in advance as a look-up table (LUT), or the calculation formula of the function may be executed as software processing. .

<Third pixel interpolation method>
Next, the third pixel interpolation method will be described. This method is a method in which pixel interpolation is performed by performing weighting according to the ratio of the correlation degree in all four directions in accordance with the correlation degree in the four directions obtained by the above method.

  In the above example, correlation value (Cvertical) = 20, correlation value (Chorizontal) = 10, correlation value (CdiagonalA) = 10, correlation value (CdiagonalB) = 5, and the ratio of correlation values is 20/45: Since 10/45: 10/45: 5/45, the correlation ratio is 25/45: 35/45: 35/45: 40/45. This correlation ratio calculation method is an example, and other examples will be described later.

  Then, the pixel value obtained by performing the pixel interpolation in the vertical direction for the target pixel is P1, the pixel value obtained by performing the pixel interpolation in the horizontal direction is P2, the pixel value obtained by performing the pixel interpolation in the oblique A direction is P3, and the pixel value in the oblique B direction Assuming that the interpolated pixel value is P4, the pixel value “P1 × 25/45 + P2 × 35/45 + P3 × 35/45 + P4 × 40/45” weighted according to the correlation ratio is set as the pixel value of the target pixel. It is. Thereby, it is possible to execute highly accurate interpolation processing in consideration of the correlation degree in the four directions of vertical, horizontal, diagonal A, and diagonal B.

  In general, the calculation method of CX, CY, CZ, CΩ (CX, CY, CZ, CΩ) in the X, Y, Z, Ω direction is the same as the method shown in Equation 1 to Equation 8. )), And the pixel values subjected to pixel interpolation in the X, Y, Z, and Ω directions are PX, PY, PZ, and PΩ, the ratio of the correlation degree with respect to the four directions X, Y, Z, and Ω is It becomes like the formula.

  Therefore, the pixel interpolation value weighted according to the ratio of the correlation degree in the four directions X, Y, Z, and Ω is expressed by the following equation (12).

  As a method for calculating the weighting coefficient, a function as shown in FIG. 10 can be used also in the third method.

  Further, as another method of obtaining the correlation ratio, an inverse ratio of correlation values may be used. That is, correlation value (Cvertical) = 20, correlation value (Chorizontal) = 10, correlation value (CdiagonalA) = 10, correlation value (CdiagonalB) = 5, and the ratio of correlation values is 20: 10: 10: 5. Therefore, the ratio of the correlation degree is the inverse ratio of these, 1/20: 1/10: 1/10: 1/5, that is, 1/9: 2/9: 2/9: 4/9.

  Or you may make it consider a correlation degree deviation rate as another method of calculating | requiring a correlation degree ratio. That is, the ratio of the correlation degree in each group is 10/30: 20/30 and 5/15: 10/15, and the correlation degree deviation rate for each group is | 20-10 |: | 10- 5 | That is, 2/3: 1/3. Therefore, by multiplying the correlation ratio of each group by the correlation divergence ratio, the correlation ratio is 10/30 × 2/3: 20/30 × 2 / 3: 5/15 × 1/3: 10/15 × 1/3, that is, 2/9: 4/9: 1/9: 2/9. In this method, the nonlinear characteristic shown in FIG. 10 can be applied to each of the correlation coefficient and the correlation deviation rate.

{Embodiment 2}: Processing for a pixel array obtained by rotating an RGB Bayer array by 45 degrees A pixel interpolation method according to Embodiment 2 of the present invention will be described. The CCD 2 in the second embodiment has a pixel array as shown in FIG. In other words, the pixel array is obtained by rotating the general RGB Bayer array shown in FIG. 2 by 45 degrees. For example, a honeycomb CCD is known as a CCD having such a pixel arrangement.

  The pixel signals output from the CCD 2 are pixel signals arranged as shown in FIG. In this example, R → B → R → B..., R (red) and B (blue) are alternately read for even rows (even lines), and G → G → for odd rows (odd lines). G (green) pixels are continuously read out as G → G. Thus, since the arrangement of the output signals is different from that in the first embodiment, a method different from the method described in the first embodiment is used.

  Also in this embodiment, as shown in FIG. 3, two sets of correlation directions orthogonal to each other are used.

<Correlation value calculation method>
FIGS. 8A to 8D show a correlation value calculation method when the pixel of interest is a G signal (here, G22). In this embodiment, the G signal included in the 3 × 3 peripheral region centered on the pixel of interest is used as a target pixel for correlation value calculation, but the range of the peripheral region is not particularly limited. In addition, a G signal in a 5 × 5 region or a 7 × 7 region may be used.

  FIG. 8A shows a method of calculating the correlation value (Cvertical) in the vertical direction, and the calculation formula thereof is expressed by Equation 13.

  That is, using the pixel values of the three G signals (G02, G22, G42), two sets of pixel difference absolute values in the vertical direction are calculated, and the average value thereof is used as the correlation value (Cvertical). The smaller the correlation value (Cvertical) is, the higher the degree of correlation is.

  FIG. 8B shows a method of calculating the horizontal correlation value (Chorizontal), and the calculation formula thereof is expressed by Equation (14).

  That is, using the pixel values of the three G signals (G32, G22, G23), two sets of pixel difference absolute values in the horizontal direction are calculated, and the average value of these is used as the correlation value (Chorizontal). The smaller the correlation value (Chorizontal), the higher the degree of correlation.

  FIG. 8C shows a method of calculating the correlation value (CdiagonalA) in the diagonal A direction, and the calculation formula is Expression 15.

  That is, using the pixel values of the three G signals (G01, G22, G43), two sets of pixel difference absolute values in the diagonal A direction are calculated, and the average value of these is set as the correlation value (CdiagonalA). The smaller the correlation value (CdiagonalA), the higher the degree of correlation.

  FIG. 8D shows a method of calculating the correlation value (CdiagonalB) in the diagonal B direction, and the calculation formula is expressed by Equation 16.

  That is, using the pixel values of the three G signals (G03, G22, G41), two sets of pixel difference absolute values in the oblique B direction are calculated, and the average value of these is set as the correlation value (CdiagonalB). The smaller the correlation value (CdiagonalB) is, the higher the degree of correlation is.

  FIGS. 9A to 9D show a correlation value calculation method when the pixel of interest is a B signal (here, B22). In this embodiment, the G signal in the 2 × 4 peripheral area centered on the target pixel is used as the target pixel for correlation value calculation, but the range of the peripheral area is not particularly limited.

  FIG. 9A shows a method of calculating the correlation value (Cvertical) in the vertical direction, and the calculation formula is Expression 17.

  That is, using the pixel values of the eight G signals (G10, G11, G12, G13, G30, G31, G32, G33), four sets of absolute pixel difference values in the vertical direction are calculated, and the average value of these is calculated as the correlation value. (Cvertical). The smaller the correlation value (Cvertical) is, the higher the degree of correlation is.

  FIG. 9B shows a method of calculating the horizontal correlation value (Chorizontal), and the calculation formula thereof is expressed by Equation 18.

  That is, using the pixel values of the eight G signals (G10, G11, G12, G13, G30, G31, G32, G33), six sets of pixel difference absolute values in the horizontal direction are calculated, and the average value of these is calculated as the correlation value. (Chorizontal). The smaller the correlation value (Chorizontal), the higher the degree of correlation.

  FIG. 9C shows a method of calculating the correlation value (CdiagonalA) in the diagonal A direction, and the calculation formula thereof is Expression 19.

  That is, using the pixel values of the six G signals (G10, G11, G12, G31, G32, G33), three sets of absolute pixel difference values in the diagonal A direction are calculated, and the average value of these is calculated as the correlation value (CdiagonalA). It is. The smaller the correlation value (CdiagonalA), the higher the degree of correlation.

  FIG. 9D shows a method of calculating the correlation value (CdiagonalB) in the diagonal B direction, and the calculation formula thereof is expressed by Equation 20.

  That is, using the pixel values of the six G signals (G11, G12, G13, G30, G31, G32), three sets of pixel difference absolute values in the diagonal B direction are calculated, and the average value of these is calculated as the correlation value (CdiagonalB). It is. The smaller the correlation value (CdiagonalB) is, the higher the degree of correlation is.

  The pixel interpolation method when the R signal is the target pixel is the same as that when the B signal is the target pixel. That is, in FIG. 9, the correlation values in the horizontal, vertical, diagonal A, and diagonal B directions can be calculated by exchanging the B signal and the R signal and using Equation 17 to Equation 20 in the same manner. .

  When the four correlation values (Cvertical, Chorizontal, Cdiagonal A, Cdiagonal B) are calculated in the horizontal, vertical, diagonal A, and diagonal B directions by the above method, the three pixel interpolation methods described in the first embodiment Pixel interpolation can be performed by a similar method. That is, in the first method, a group having a large correlation degree bias is selected, and a direction having a high correlation degree is selected from a group having a large correlation degree bias. Then, the pixel interpolation is performed in the direction. The second method is a method in which a pair having a large correlation degree bias is selected, and pixel interpolation is performed by weighting the two directions included in the pair having a large correlation degree bias according to the correlation degree ratio. . Further, the third method is a method of performing pixel interpolation by performing weighting according to the correlation degree ratio in four directions.

{Modification}
In the above embodiment, the diagonal A direction and the diagonal B direction are each selected as a group having an inclination of 45 degrees from the vertical direction, but this is an example. As long as the diagonal A direction and the diagonal B direction are orthogonal to each other, another angle may be selected as the inclination with respect to the vertical direction.

  In the above embodiment, the correlation value is calculated for the G signal. However, when the input pixel data is data including a luminance signal, for example, the correlation value is obtained using the luminance signal. It is preferable to do so. However, the pixel component for which the correlation value is obtained in the pixel interpolation method of the present invention is not limited to the G signal and the luminance signal, and the correlation value may be obtained using other colors.

  Further, when evaluating the correlation value, evaluation with a threshold value may be performed. As described above, in this embodiment, correlation values are calculated in the vertical, horizontal, diagonal A, and diagonal B directions. Then, each correlation value is compared and evaluated with a predetermined threshold value. For example, in the third pixel interpolation method in which pixel interpolation is performed in four directions, pixel interpolation may be performed only in a direction having a correlation value lower than the threshold (that is, only in a direction having a high degree of correlation). . Also, if the correlation value is above the threshold value in any direction (that is, if the degree of correlation is not high in any direction), select whether to use median interpolation or average value interpolation. The interpolation method may be switched accordingly.

1 is a block diagram of a digital camera according to an embodiment. It is a figure which shows a general RGB Bayer arrangement | sequence. It is a figure which shows four directions which calculate a correlation degree. It is a figure which shows the correlation calculation method of 4 directions in case an attention pixel is made into G signal. It is a figure which shows the correlation calculation method of 4 directions in case an attention pixel is made into R signal. It is a figure which shows the array which the RGB Bayer array rotated 45 degree | times. It is a figure which shows the output arrangement | sequence of the pixel signal shown in FIG. It is a figure which shows the correlation calculation method of 4 directions in case an attention pixel is made into G signal. It is a figure which shows the correlation calculation method of 4 directions in case an attention pixel is made into B signal. It is a figure which shows the function which calculates a weighting coefficient.

Explanation of symbols

4 Image processing circuit 5 Color interpolation circuit

Claims (8)

A first step of inputting pixel data of a predetermined color space;
Using the peripheral pixels of the pixel of interest, the pixel correlation in the first direction, the pixel correlation in the second direction orthogonal to the first direction, and the first and second directions are different A second step of determining a correlation degree of pixels in a third direction and a correlation degree of pixels in a fourth direction orthogonal to the third direction;
A third step of evaluating the degree of correlation of the peripheral region of the pixel of interest using the four degrees of correlation in the first to fourth directions;
A fourth step of performing pixel interpolation of the pixel of interest with respect to a direction in which the degree of correlation is evaluated to be high in the third step;
A pixel interpolation method comprising: The pixel interpolation method according to claim 1,
The third step includes
When the first direction and the second direction are a first set, and the third direction and the fourth direction are a second set, the first set and the second direction Selecting a group with a large correlation degree bias as a group in a direction with a high degree of correlation,
Including
The fourth step includes
A fifth step of performing pixel interpolation with respect to a direction included in the group having a large correlation degree bias;
A pixel interpolation method comprising: The pixel interpolation method according to claim 2,
The five steps are:
A step of performing pixel interpolation only in a direction having a high degree of correlation among two directions included in a set having a large correlation degree,
A pixel interpolation method comprising: The pixel interpolation method according to claim 2,
The five steps are:
A step of performing pixel interpolation by performing weighting according to a correlation degree ratio for two directions included in a set with a large correlation degree bias;
A pixel interpolation method comprising: A first step of inputting pixel data of a predetermined color space;
Using the peripheral pixels of the pixel of interest, the pixel correlation in the first direction, the pixel correlation in the second direction orthogonal to the first direction, and the first and second directions are different A second step of determining a correlation degree of pixels in a third direction and a correlation degree of pixels in a fourth direction orthogonal to the third direction;
A step of performing pixel interpolation by performing weighting according to a ratio of the correlation degree with respect to the first to fourth directions based on the correlation degree obtained in the second step;
A pixel interpolation method comprising: The pixel interpolation method according to any one of claims 1 to 5,
The pixel data of the predetermined color space is
Data in which pixels of each color component included in the predetermined color space are arranged in a Bayer array,
A pixel interpolation method comprising: The pixel interpolation method according to any one of claims 1 to 6,
The pixel interpolation method, wherein the first direction is a vertical direction or a horizontal direction, and the second direction is a direction having an inclination of 45 degrees with respect to the vertical direction. The pixel interpolation method according to any one of claims 1 to 7,
The second step in claim 1 or claim 2 is for the G signal when the pixel data is pixel data in the RGB color space, or the pixel data in a predetermined color space including a luminance signal. In some cases, a pixel interpolation method is characterized in that a correlation is obtained for a luminance signal.

Images