JP4990240B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program for removing noise mixed in an input image such as a digital camera.

In an image input device such as a digital camera or image scanner equipped with an image sensor such as a CCD, and an image output device such as a display for displaying an image input from this type of image input device or a color printer for printing, an image input system or Noise is mixed in image data in the image transmission system. This type of noise is generally removed by image processing within the apparatus. Of these image input / output devices, a conventional method for removing noise mixed in a digital camera will be described.
The simplest and most general method for removing noise is to form a processing window consisting of a plurality of pixels centered on a pixel to be image processed (hereinafter referred to as a target pixel), and signals of peripheral pixels in the processing window. There is a technique of applying a low-pass filter by weighting the level value and the signal level value of the pixel of interest and performing a convolution operation. However, when this method is used, there is a problem that the resolution of the image is lowered because a low-pass filter is uniformly applied to significant edges existing in an image other than noise.
As another method, the edge of the processing window is detected by applying an operator such as Prewitt or Sobel, and if it is determined that the pixel of interest is a significant edge in the image, the strength of the low-pass filter is reduced. Alternatively, a technique has been proposed in which noise is removed while suppressing a decrease in image resolution by applying a low-pass filter with a direction that avoids pixels constituting edges existing in the vicinity (see, for example, Patent Document 1). ).

JP 2002-185795 A

  The image processing method for removing noise described in Patent Document 1 misjudged noise that is continuous in a specific direction as an edge, or misidentified as noise without being able to detect a pattern present in a subject as an edge. Therefore, there is a problem that the edge and noise cannot be separated 100%, the resolution is lowered, and noise reduction cannot be performed sufficiently.

  The present invention has been made to solve the above-described problems, and an object thereof is to obtain an image processing apparatus and an image processing program that enable noise reduction without reducing the resolution.

  The image processing apparatus according to the present invention calculates a plurality of low-pass filter values in different one-dimensional directions, each including a target pixel, for a processing window composed of a plurality of pixels centered on the target pixel of an input two-dimensional image. A one-dimensional low-pass filter, a two-dimensional low-pass filter for calculating a low-pass filter value in a two-dimensional direction including the target pixel with respect to the processing window, and an absolute difference between the low-pass filter value in each one-dimensional direction and the target pixel value Calculate the edge correction coefficient corresponding to the detected edge level based on the edge level detection means that calculates each value and detects the minimum value of the absolute value of each difference as the edge level of the pixel of interest and the preset edge correction curve NR coefficient calculating means for performing weighting and adding the target pixel value and the two-dimensional low-pass filter value using the edge correction coefficient Those having a NR processing means for.

  According to the present invention, since the edge level can be adjusted without separating the original edge level and noise, noise reduction can be performed without lowering the resolution, and a high-quality image can be obtained. There is an effect that can be done.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a functional configuration of an image processing apparatus according to Embodiment 1 of the present invention.
In the figure, an image processing apparatus includes a plurality of one-dimensional low-pass filters including a 0-degree direction low-pass filter (hereinafter referred to as a low-pass filter) 11, a 45-degree direction LPF 12, a 90-degree direction LPF 13, and a 135-degree direction LPF 14. A dimension LPF 15, an edge level detection unit 20, an NR coefficient calculation unit 21, and an NR processing unit 22 are provided as basic configurations. The processing by the image processing apparatus according to the first embodiment can be configured as a program executed on a computer (including a microcomputer).

Next, the operation will be described.
An image constituting one screen is divided into a plurality of processing windows each having a pixel configuration including horizontal M pixels (for example, 17 pixels) and vertical N pixels (for example, 17 pixels). The following processing is performed.
The one-dimensional LPFs 11 to 14 are centered on the target pixel of the input image data, and have different one-dimensional directions (0 degree, 45 degree, 90 degree, 135 degree) LPF value is calculated.
FIG. 2 shows a method for performing filtering in the 0 degree direction with respect to the processing window. The pixel at the center is the target pixel Pxy. In the 0 degree direction LPF 11, processing is performed on pixels in the 0 degree direction including the target pixel Pxy. In FIG. 2, the pixels that are shaded in the horizontal direction are processed. Also, here, the input image to be processed is assumed to be an image input from a general single-panel color digital camera, and among the four shooting colors of R, Gr, Gb, and B in the Bayer array An example in which only the same color is processed is shown. Similarly, FIG. 3 shows a filtering method using 45 degree direction LPF 12, FIG. 4 shows 90 degree direction LPF 13, and FIG. 5 shows 135 degree direction LPF 14. In each one-dimensional LPF, for example, by calculating a simple average of a pixel group in a one-dimensional direction including the target pixel, the average pixel values in each direction, lpf000, lpf045, lpf090, and lpf135 are calculated as LPF values in the one-dimensional direction. To do.

Further, the two-dimensional LPF 15 calculates the LPF value in the two-dimensional direction with respect to the processing window of the input image data at the same timing.
FIG. 6 shows an LPF calculation processing method for the processing window. Here, as the LPF value in the two-dimensional direction, all average values lpf17 in the processing window composed of 17 pixels × 17 pixels are calculated according to the equation (1).

  Next, the edge level detection unit 20 calculates absolute values of differences between the one-dimensional LPF values lpf000 to lpf135 calculated by the one-dimensional LPFs 11 to 14 and the signal level Pxy of the target pixel of the input image data, respectively. Among them, the absolute difference value lpfmin that is the smallest is detected as the edge level of the target pixel.

The NR coefficient calculation unit 21 calculates an edge correction coefficient corresponding to the edge level detected by the edge level detection unit 20 based on a preset edge correction curve.
As illustrated in FIG. 7, the edge correction curve defines the characteristic that the edge correction coefficient changes according to the detected edge level. In this example, the range of the edge correction coefficient is defined in 65 levels from 0 to 64, and the degree of edge correction is increased as the detected edge level lpfmin is increased, and the correction is weakened as the detected edge level lpfmin is decreased. Show. Therefore, the edge correction coefficient dx1 is uniquely determined according to the edge level in the target pixel. In FIG. 7, DMINX, DMAXX, DMINY, and DMAXY for defining correction characteristics are given in advance, and an edge correction curve is defined in advance according to shooting conditions in the image input apparatus.
Furthermore, when the edge correction coefficient dx1 is calculated from the edge correction curve, the NR coefficient calculation unit 21 calculates an edge correction coefficient dx2 that is paired with the edge correction coefficient dx1 using the equation (2).
dx2 = 64−dx1 (2)
These edge correction coefficients dx1 and dx2 are NR coefficients used for noise reduction in the subsequent NR processing unit 22.

In the NR processing unit 22, the target pixel value of the input image data and the LPF value in the two-dimensional direction are weighted and added using the edge correction coefficient calculated by the NR coefficient calculating unit 21.
Here, the LPF value lpf17 in the two-dimensional direction is weighted with the edge correction coefficient dx1 coefficient, and the target pixel value Pxy is weighted with the edge correction coefficient dx2. Then, noise reduction processing is performed by adding both in accordance with equation (3) to obtain a pixel value Pxy ′ after noise correction.
Pxy ′ = (dx1 × lpf17 + dx2 × Pxy) / 64 (3)
The above-described series of processing has been described for only the same color among the four shooting colors R, Gr, Gb, and B in the processing window, but the same processing is performed for each of the other colors. In addition, the same processing is performed for all processing windows constituting one screen.
By performing the noise reduction process as described above, the noise level of the entire screen can be made uniform without degrading the resolution, so that a high-quality noise reduction effect can be obtained.

FIG. 8 is an explanatory diagram showing image signals before and after the noise reduction process.
FIG. 8A shows an example in which an image signal input to an image input apparatus to which the present invention is applied is shown as a one-dimensional signal for explanation. In the figure, the signal indicated by a broken line indicates a change in the signal level inherent to the subject, but is input as a solid line signal due to the influence of noise mixed in the analog circuit of the image input apparatus. FIG. 8B shows an example of an image signal after the noise reduction processing is performed on the signal of FIG. 8A using the correction characteristics of FIG. From this, it can be confirmed that the noise signal is satisfactorily suppressed and the edges originally included in the image are retained. This is because the edge level detection unit 20 calculates the absolute value of the difference between the LPF value in the one-dimensional direction from 0 degrees to 135 degrees and the target pixel, and sets the minimum value as the edge level of the target pixel. This is because the LPF in the ridge line direction of the edge is selected as the target of the absolute difference value, and the edge correction exceeding the differential value is not applied.

  As described above, according to the first embodiment, a plurality of LPF values in different one-dimensional directions including a target pixel are calculated for a processing window including a plurality of pixels centered on the target pixel. The LPF value in the two-dimensional direction including the target pixel is calculated for the processing window, the absolute value of the difference between the LPF value in each one-dimensional direction and the target pixel value is calculated, and the minimum absolute value of each difference is calculated. Detected as the edge level of the target pixel, calculates an edge correction coefficient corresponding to the detected edge level based on a preset edge correction curve, and uses the calculated edge correction coefficient to calculate the target pixel value and the LPF value in the two-dimensional direction The noise reduction is performed by weighted addition. Therefore, since the edge level can be adjusted without separating the original edge level and noise, noise reduction can be performed without reducing the resolution, and a high-quality image can be obtained. In addition, the detection of the edge level of the pixel of interest is configured to perform level suppression without distinguishing between the edge originally present in the image and the noise added by the imaging system, so the edge level in the image is detected using a Sobel filter or the like. Compared to a configuration in which noise is differentiated and noise reduction is performed only on noise, an effect of enabling output of an image having a uniform graininess and a low noise level can be obtained.

In the above example, an example in which the edge correction coefficient is calculated using an edge correction curve that is uniform for all shooting colors for an image shot using a single-plate color sensor is not limited thereto. A configuration in which curves having different sensitivities in accordance with the characteristics of the image sensor may be used for each color component of the input image. As a result, there is an effect that uniform noise reduction processing can be performed for all color components.
In the above example, the configuration is shown in which all pixels to be calculated are used when calculating the LPF value in each one-dimensional direction and when calculating the average value LPF of all the pixels in the processing window. Instead, only a pixel value having a signal level that is relatively close to the target pixel (that is, within a predetermined level difference) may be selected and used as the calculation target pixel. As a result, when a region that clearly differs from the pixel of interest in the processing window is included in the processing window, it is possible to perform an operation that excludes the feature of the different region, and noise reduction with higher resolution becomes possible.

In the above example, when the edge level is detected by the edge level detection unit 20, the one that has the smallest difference absolute value between the LPF value in each one-dimensional direction and the pixel of interest is defined as the edge level. Using the smallest (maximum edge direction candidates) and the largest for the LPF values from the 0 degree direction to the 135 degree direction, the smallest one is true if it is orthogonal to the smallest one. For example, the edge level may be detected from a plurality of edge directions. In this case, the selection of the absolute value of the edge level in the wrong direction due to the influence of random noise is reduced, and a more accurate edge level can be calculated.
In the above example, the method of calculating the edge correction coefficient calculated by the NR coefficient calculation unit 21 in 65 stages is described. However, the present invention is not limited to this, and the edge correction coefficient may be calculated in an arbitrary stage depending on factors such as the bit width of the input image. Also good.

In the above example, the LPF value in each one-dimensional direction and the LPF value in the two-dimensional direction are calculated as a simple average value of the calculation target pixel. You may comprise so that a weighted average may be used according to distance. As a result, it is possible to perform calculation with respect to the feature amount of the pixel close to the target pixel, and it is possible to detect the edge level more accurately particularly under the condition of relatively little noise.
In the above example, the NR processing unit 22 performs weighted addition of the target pixel value Pxy and the LPF value lpf17 in the two-dimensional direction based on the edge correction coefficient. However, the present invention is not limited to this, and lpfmin is used instead of lpf17. The LPF value (any one of lpf000 to lpf135) in the direction defining the angle may be used. In this case, weighted addition is performed in a narrower range than when lpf17 is used, and this is effective in that the resolution can be completely maintained particularly when the input image has relatively little noise.

Further, in the above example, in the average value calculation process according to the equation (1) for obtaining the LPF value in the two-dimensional direction, a method for calculating the same color pixels in the processing window composed of 17 × 17 pixels is described. Instead, other window sizes may be used based on the realized device cost, the S / N ratio of the input image signal, and the like. In this case, the number of dividers may be reduced by selecting the number of pixels so that the denominator of equation (1) is a power of 2.
In the above example, the case where division is used in the NR processing using equation (3) has been described. However, this is not limited to this, and division is realized by bit shift operation when the number of stages of weighted addition is a power of 2. Thus, the divider can be eliminated when the circuit is realized, while the execution speed of the operation can be improved when the circuit is realized by a program.

Embodiment 2. FIG.
FIG. 9 is a block diagram showing a functional configuration of an image processing apparatus according to Embodiment 2 of the present invention. In the figure, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted in principle. The second embodiment is different from the first embodiment in that a brightness correction unit 21 is provided between the two-dimensional LPF 15 and the NR coefficient calculation unit 21.
Next, the operation will be described.
The brightness correction unit 23 calculates a brightness correction coefficient corresponding to the LPF value in the two-dimensional direction of the two-dimensional LPF 15 based on a preset brightness correction curve. Here, for example, a curve as shown in FIG. 10 is used as the brightness correction curve. In this example, the dark correction stage is defined as 129, and 64 is defined as no brightness correction. NR1HX and NR1HY are control points set in advance to define the correction curve. In particular, in the case of the brightness correction curve of FIG. 10, the edge correction coefficient in the dark part of the image can be weakened. When brightness correction is performed using this correction curve, the brightness correction coefficient hy is obtained using the LPF value in the two-dimensional direction as the brightness around the pixel of interest.

As described in the first embodiment, the NR coefficient calculation unit 21 calculates an edge correction coefficient for the edge level detected by the edge level detection unit 20. The edge correction coefficient is used as the brightness correction unit 23. Correction is performed based on the brightness correction coefficient calculated in step (1). Here, the edge correction coefficient dx1 ′ after lightness correction is calculated by the expression (4) in which the lightness correction coefficient is applied to the edge correction coefficient dx1, and dx2 ′ is calculated by the expression (2).
dx1 ′ = dx1 × hy / 64 (4)
The edge correction coefficients dx1 ′ and dx2 ′ thus corrected for brightness can be varied according to the brightness correction coefficient hy, that is, according to the brightness of the image area.

  As described above, according to the second embodiment, by adding the brightness correction unit 23 to the configuration of the first embodiment, the edge correction levels of the bright and dark portions in the image can be made variable. For example, when the NR coefficient is adjusted in accordance with a bright region of the input image, it is possible to reduce correction unevenness due to the brightness of the input image, such as a dark portion with a low signal level being blurred.

In the example of the second embodiment, the configuration in which the brightness correction can be switched in 129 stages is shown. However, the present invention is not limited to this, and the brightness correction can be switched in any stage according to the input image and the image characteristics of the apparatus. You may make it do. Further, although brightness correction is performed using division in the equation (4), this is not restrictive, and the number of brightness correction steps may be raised to a power of two. In that case, it is possible to replace it with a bit shift operation, and in the case of circuitization, the number of dividers can be reduced, so the circuit scale can be simplified, and in the case of program processing, the processing speed can be increased. enable.
Further, in the example of the second embodiment, the example in which the brightness correction is uniformly performed for all the photographed colors with respect to the image photographed using the single plate color sensor is shown, but this is not the only case. The brightness correction coefficient may be defined for each color according to the characteristics of the image sensor having different sensitivities. Thereby, there is an effect that uniform noise reduction processing can be performed for all color components.

It is a block diagram which shows the function structure of the image processing apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the method of the filter process by 0 degree direction LPF which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the method of the filter process by 45 degree | times direction LPF which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the method of the filter process by 90 degree | times direction LPF which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the method of the filter process by 135 degree | times direction LPF which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the method of the filter process by the two-dimensional LPF which concerns on Embodiment 1 of this invention. It is explanatory drawing which illustrates the edge correction curve used with the NR coefficient calculation part which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the image signal before and behind the noise reduction process which concerns on Embodiment 1 of this invention. It is a block diagram which shows the function structure of the image processing apparatus by Embodiment 2 of this invention. It is explanatory drawing which illustrates the brightness correction curve used with the brightness correction part which concerns on Embodiment 2 of this invention.

Explanation of symbols

  11-14 One-dimensional LPF (low-pass filter), 15 Two-dimensional LPF (low-pass filter), 20 Edge level detection unit, 21 NR coefficient calculation unit, 22 NR processing unit, 23 Brightness correction unit.

Claims (7)

A plurality of one-dimensional low-pass filters for calculating a low-pass filter value in a different one-dimensional direction each including a target pixel for a processing window including a plurality of pixels centered on the target pixel of an input two-dimensional image;
A two-dimensional low-pass filter that calculates a low-pass filter value in a two-dimensional direction including the target pixel with respect to the processing window;
An edge level detecting means for calculating an absolute value of a difference between each low-pass filter value in one dimensional direction and a target pixel value, and detecting a minimum value of the absolute value of each difference as an edge level of the target pixel;
NR coefficient calculating means for calculating an edge correction coefficient corresponding to the detected edge level based on a preset edge correction curve;
An image processing apparatus comprising NR processing means for weighting and adding a target pixel value and a two-dimensional low-pass filter value using an edge correction coefficient.   Each of the one-dimensional low-pass filter and the two-dimensional low-pass filter selects only a pixel value within a predetermined level difference from the signal level value of the target pixel for the processing window, and calculates the low-pass filter value. The image processing apparatus according to claim 1.   3. The image processing apparatus according to claim 1, wherein the NR coefficient calculating means uses an edge correction curve having different sensitivities according to characteristics for each color component of the input image. A brightness correction means for calculating a brightness correction coefficient corresponding to a low-pass filter value in a two-dimensional direction based on a preset brightness correction curve;
The NR coefficient calculating means corrects the edge correction coefficient based on the brightness correction coefficient,
The NR processing means weights and adds the target pixel value and the two-dimensional low-pass filter value using the edge correction coefficient whose brightness is corrected. The image processing apparatus according to item.   5. The image processing apparatus according to claim 4, wherein the brightness correction means uses a brightness correction curve having different sensitivities according to characteristics for each color component of the input image. A process of calculating a plurality of low-pass filter values in different one-dimensional directions including the target pixel for a processing window including a plurality of pixels centered on the target pixel of the two-dimensional image;
A process of calculating a low-pass filter value in a two-dimensional direction including the target pixel for the same processing window;
A process of calculating an absolute value of a difference between each low-pass filter value in each one-dimensional direction and a target pixel value, and detecting a minimum value of the absolute value of each difference as an edge level of the target pixel;
A process of calculating an edge correction coefficient corresponding to the detected edge level based on a preset edge correction curve;
An image processing program for executing, on a computer, a process of weighting and adding a target pixel value and a two-dimensional low-pass filter value using an edge correction coefficient. A process of calculating a plurality of low-pass filter values in different one-dimensional directions including the target pixel for a processing window including a plurality of pixels centered on the target pixel of the two-dimensional image;
A process of calculating a low-pass filter value in a two-dimensional direction including the target pixel for the same processing window;
A process of calculating an absolute value of a difference between each low-pass filter value in each one-dimensional direction and a target pixel value, and detecting a minimum value of the absolute value of each difference as an edge level of the target pixel;
A process of calculating an edge correction coefficient corresponding to the detected edge level based on a preset edge correction curve;
A process for calculating a brightness correction coefficient corresponding to a low-pass filter value in a two-dimensional direction based on a preset brightness correction curve;
Processing for correcting the edge correction coefficient based on the lightness correction coefficient;
An image processing program for executing on a computer weighted addition of a target pixel value and a two-dimensional low-pass filter value using the brightness-corrected edge correction coefficient.

Images