JP6184277B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus that outputs flatness at a pixel of interest.

By determining the edge direction of each pixel in the image, it is possible to adaptively perform image processing based on the structural attributes of the image. Therefore, a technique for determining the presence and direction of an edge prior to noise removal Is known (for example, see Patent Document 1).
In Patent Document 1, the presence / absence of an edge and the direction of an edge are discriminated based on a plurality of evaluation values obtained by calculating a sum of absolute differences between a target pixel and a plurality of peripheral pixels centered on the target pixel in a plurality of directions. Technology is disclosed.

JP 2008-293425 A

  However, in high-sensitivity shot images, low-contrast edges and textures are very similar to the amplitude of noise, and the sum of absolute differences between the pixel of interest and a plurality of surrounding pixels in the area where the edges exist, and the flat part containing noise There is no clear difference in the sum of absolute differences between the pixel of interest and a plurality of peripheral pixels in a region where there is no clear edge or texture. For this reason, in Patent Document 1, the accuracy of flatness determination for determining whether or not a predetermined area is flat in a high-sensitivity photographed image and the flatness that is the degree of flatness is lowered.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to improve the accuracy of flatness determination without being affected by the amplitude due to noise.

In order to achieve the above object, the present invention provides the following means.
According to one aspect of the present invention, a plurality of boundary lines passing through the target pixel are set for a predetermined region including the target pixel of the input image and a plurality of peripheral pixels including the target pixel, and each boundary line is set. On the other hand, a divided region setting means for setting the first region and the second region divided by the boundary line, and a difference in luminance level between the first region and the second region for each boundary line. and the index value calculating means for calculating an index value that indicates, calculates the maximum value of the index value, and a flatness calculating means for outputting as the flatness of the pixel of interest, is the index value calculating means, the boundary line by line, and the average value of the signal value of the first area to calculate a difference value between the average value of the signal value of the second region, to provide an image processing apparatus you and the index value said difference value.

  According to this aspect, when the input image is input, the divided region setting unit has a plurality of boundaries passing through the target pixel with respect to a predetermined region including the target pixel of the input image and a plurality of peripheral pixels including the target pixel. A line is set, and two areas of a first area and a second area divided by the boundary line are set for each boundary line. Then, the index value calculation means calculates an index value indicating a difference in luminance level between the first area and the second area for each boundary line. Then, the flatness calculation unit calculates the maximum value from the plurality of calculated index values and outputs it as the flatness of the target pixel. Thus, when the target pixel exists on the edge, the edge can be detected by the boundary lines in a plurality of directions without being affected by the amplitude due to noise, and the flatness determination accuracy of the target pixel can be improved.

In the above aspect, the image processing device includes interpolation image generation means for performing a pixel value mixing process between the color components for each predetermined region of the image signal of the Bayer array and generating an interpolation image obtained by interpolating all the pixels. The interpolated image is preferably used as the input image.
In this way, the index value can be calculated without considering the color component information in the calculation, and the subsequent processing can be simplified.

In the above aspect, it is preferable that the number of pixels included in the first region and the number of pixels included in the second region are the same.
By doing in this way, the precision of the flatness determination of a focused pixel can be improved more.

In the aspect described above, it is preferable that the first region and the second region have the same shape.
By doing in this way, the precision of the flatness determination of a focused pixel can be improved more.

In the above aspect, the index value calculation means calculates a difference value between the average value of the signal values of the first region and the average value of the signal values of the second region for each boundary line, and calculates the difference value. you and the index value.
In this way, by calculating the average value of the signal values of each divided area, the influence of noise is also averaged, and there are also minute edges compared to noise existing in areas containing low-contrast structures. Since it becomes possible to detect, a region with a low contrast structure is less likely to be erroneously determined as a flat portion.

In another aspect of the present invention, a plurality of boundary lines passing through the target pixel are set for a predetermined region including the target pixel of the input image and a plurality of peripheral pixels including the target pixel, and each boundary line is set. A divided region setting means for setting a first region and a second region divided by the boundary line, and a difference in luminance level between the first region and the second region for each boundary line An index value calculating means for calculating an index value indicating a flatness calculating means for calculating a maximum value of the index values and outputting it as a flatness of the target pixel, wherein the index value calculating means comprises: each boundary, and the pixels of the first area and each pixel of the second area is associated, based on the weighted average of the difference values between corresponding pixels to provide an image processing apparatus for calculating the index value .
By doing so, it is possible to further improve the accuracy of determining the flatness of the pixel of interest without averaging the information on the structures of the divided areas.

  According to the present invention, it is possible to improve the accuracy of flatness determination without being affected by the amplitude due to noise.

1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to a first embodiment of the present invention. FIG. 4 is an explanatory diagram schematically showing a processing area for an input image in the image processing apparatus according to the first embodiment of the present invention. 3 is a diagram illustrating an example of a boundary line set by a divided region setting unit and a region divided by the boundary line in the image processing apparatus according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a boundary line set by a divided region setting unit and a region divided by the boundary line in the image processing apparatus according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a boundary line set by a divided region setting unit and a region divided by the boundary line in the image processing apparatus according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating an example of a boundary line set by a divided region setting unit and a region divided by the boundary line in the image processing apparatus according to the first embodiment of the present invention. FIG. FIG. 4 is a diagram illustrating an example in which pixels in a region divided according to a boundary line set by a divided region setting unit are associated in the image processing apparatus according to the first embodiment of the present invention. 5 is a flowchart illustrating an operation when outputting the flatness of a target pixel in an input image in the image processing apparatus according to the first embodiment of the present invention. It is a block diagram which shows schematic structure of the image processing apparatus which concerns on the modification of the 1st Embodiment of this invention. It is a figure which shows the example of the filter in the interpolation image generation part of the image processing apparatus which concerns on the modification of the 1st Embodiment of this invention. It is a figure which shows the example of the filter in the interpolation image generation part of the image processing apparatus which concerns on the modification of the 1st Embodiment of this invention. It is a block diagram which shows schematic structure of the image processing apparatus which concerns on the 2nd Embodiment of this invention.

(First embodiment)
The image processing apparatus according to the first embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the image processing apparatus includes a divided region setting unit 11 that sets two divided regions for a predetermined region, and an index value calculating unit that calculates an index value indicating a difference in luminance level between the divided regions. 12 and a flatness calculator 13 for outputting the flatness of the target pixel.

The divided region setting unit 11 sets a plurality of boundary lines passing through the target pixel for a predetermined region including the target pixel of the input image and a plurality of peripheral pixels including the target pixel, and for each boundary line The first area and the second area divided by the boundary line are set.
More specifically, as shown in FIG. 2, the divided region setting unit 11 first sets a processing region A, which is a predetermined region including the target pixel, for the input image I. In FIG. 2, a rectangular area of 9 pixels × 9 pixels centered on the target pixel is set as the processing area A for the input image. However, the present invention is not limited to this, and a circular or elliptical shape may be used as the processing area. it can.

  Subsequently, as illustrated in FIG. 3, the divided region setting unit 11 sets a boundary line for the set processing region A, and sets a first region and a second region divided by the boundary line. To do. That is, the divided region setting unit 11 sets a plurality of boundary lines that pass through the target pixel. In the example of FIG. 3, the processing region A passes through the target pixel and passes through the processing area A in the vertical boundary line L (FIG. 3A), the horizontal boundary line W (FIG. 3B), and the diagonal boundary lines D1 and D2 (FIG. C) and (D)) are generated to generate control information related to the four boundary lines, and control information related to the first area and the second area is output for each boundary line.

  In the example of FIG. 3, the division interval is one pixel, but the division interval is not limited to this, and the division may be performed with a distance apart as shown in FIG. Further, as shown in FIG. 5, the boundary line to be set is not necessarily a straight line, and the dividing direction is not limited to four directions, and irregular division is also conceivable.

  The index value calculation unit 12 calculates an index value indicating a difference in luminance level between the first region and the second region set for each boundary line by the divided region setting unit 11. More specifically, as shown in FIG. 6, the index value calculation unit 12 is based on the signal information related to the input image I and each control information generated by the divided region setting unit 11. The difference between the average values of the two areas is calculated as an index value.

That is, as shown in FIG. 6A, one area obtained by dividing the processing area A by the vertical boundary line L is a first area P1L, and the other area is a second area P2L. When the number of pixels in each region is N, the average values averageP1L and averageP2L can be obtained from the following equations (1) and (2).
Then, a difference difL between averageP1 and averageP2 is obtained according to equation (3), and this is used as an index value for the vertical boundary line L of the processing area A.

  Similarly, as shown in FIGS. 6B to 6D, the other boundary lines, that is, the horizontal boundary line W and the diagonal boundary lines D1 and D2 are also set for each boundary line. Differences difW, difD1, and difD2 between the average values of the first region and the second region are calculated as index values (Equations (4) to (12)), respectively.

In the above-described embodiment, the difference between the average values of the respective regions is used as the index value for the processing region. However, the present invention is not limited to this, and the addition average of the absolute value differences between the divided regions or between the regions. An addition average of the squares of the differences can be used as an index value.
When the addition average of the absolute value differences between the divided areas is used as the index value, as shown in FIG. 7A, the pixels included in the first area P1L divided by the vertical boundary line L and the first The pixels included in the second region P2L are associated with each other, and a weighted average of absolute value differences between the associated pixels is used as an index value. The weighted average of absolute value differences between the associated pixels is calculated according to the following equation (13).

As shown in FIGS. 7B to 7D, the first region set for each boundary line also for other boundary lines, that is, the horizontal boundary line W and the diagonal boundary lines D1 and D2. And the pixels included in the second region are associated with each other, and the weighted averages sW, sD1, and sD2 of the absolute value differences between the associated pixels are calculated as index values for each boundary line (formula ( 14) to (16)).

  When the addition average of the squares of differences between the regions is used as the index value, as shown in FIG. 7A, the pixels included in the first region P1L divided by the vertical boundary line L and the second region P2L Are associated with each other (in FIG. 7, each pixel is associated with each other along the direction of the arrow between both regions), and the average of the squares of the difference between the associated pixels Is an index value. The addition average of the square of differences between the associated pixels is calculated according to the following equation (17).

なお、各領域に含まれる画素の対応付けについて、図７において例示したが、差分を算出する際の画素の対応付けはこれに限られない。例えば、線対称となる位置の画素同士を対応付けることができるほか、点対称となる位置の画素同士を対応付けても良い。 As shown in FIGS. 7B to 7D, the first region set for each boundary line also for other boundary lines, that is, the horizontal boundary line W and the diagonal boundary lines D1 and D2. And the pixels included in the second region are associated with each other, and the addition average sW, sD1, sD2 of the difference squares of the associated pixels is calculated as an index value for each boundary line (formula (18 ) To (20)).

Note that the association of pixels included in each region is illustrated in FIG. 7, but the association of pixels when calculating the difference is not limited to this. For example, pixels at positions that are line symmetric can be associated with each other, and pixels at positions that are point symmetric may be associated with each other.

The flatness calculation unit 13 calculates the maximum value among the plurality of index values calculated by the index value calculation unit 12, and outputs the maximum value as flatness indicating the degree of flatness of the target pixel. In this specification, the flatness is calculated for each pixel of the input image and indicates the degree to which the region including the target pixel is flat. When the flatness of the region including the target pixel is low, it can be determined that there is no edge (or few) around the target pixel. On the other hand, when the flatness is high , the degree of flatness of the target pixel is low, and it can be determined that there is a pattern such as a fine texture or an edge.

  Next, the operation of the image processing apparatus configured as described above will be described with reference to the flowchart of FIG. In the following description, a 9 × 9 pixel rectangular area centered on the target pixel is set as the processing area A for the input image, and four boundary lines (FIG. 3 and the like) are set for the processing area A. An example in which the vertical direction L, the horizontal direction W, and the diagonal (diagonal direction 2 directions) are set will be described. As shown in FIG. 3, the first region and the second region divided by each boundary line have the same shape, and the pixels included in the first region and the pixels included in the second region Are the same number.

  In order to output the flatness of each pixel in the input image by the image processing apparatus according to the present embodiment, a region to be processed in the input image is set in step S11. Specifically, the divided region setting unit 11 sets a rectangular region of 9 pixels × 9 pixels centered on the target pixel as a processing region. In the next step S12, a plurality of boundary lines passing through the target pixel are set, and a first area and a second area divided by the boundary lines are set for each boundary line. In the present embodiment, as shown in FIG. 3, four system boundaries are set in the vertical direction L, the horizontal direction W, and the diagonal (diagonal) two directions.

  In the next step S13, an index value indicating a difference in luminance level between the first area and the second area divided by the set boundary line is calculated. Here, for each boundary line, the difference between the average value of the signal values of the pixels included in the first area and the average value of the signal values of the pixels included in the second area is calculated as an index value for the boundary line. . The index values are calculated for all the boundary lines. If it is determined in step S14 that the index values for all the boundary lines have been calculated, the process proceeds to the next step S15.

  Subsequently, in step S15, the maximum value is calculated from the index values calculated in the previous step S13, and in step S16, this maximum value is output as the flatness indicating the flatness of the target pixel.

  As described above, according to the present embodiment, a plurality of boundary lines passing through the target pixel are set for a predetermined region including the target pixel of the input image and a plurality of peripheral pixels including the target pixel. Two areas of the first area and the second area to be divided are set for each boundary line. Then, for each boundary line, an index value indicating a difference in luminance level between the first area and the second area is calculated, and the flatness calculation unit calculates a maximum value from the calculated index values. Output as the flatness of the pixel of interest. Thus, when the target pixel exists on the edge, the edge can be detected by the boundary lines in a plurality of directions without being affected by the amplitude due to noise, and the flatness determination accuracy of the target pixel can be improved.

  When the input image is an image signal captured by an image sensor that lacks a specific color component in each pixel, such as an image signal of a Bayer array, a predetermined filter is applied to such an image signal. It is preferable that the interpolation processing is performed using them. That is, as shown in FIG. 9, an interpolation image generation unit 14 for interpolating an image signal is separately provided, and pixel values between color components are mixed for each predetermined region for an image signal lacking a specific color component. Processing is performed to generate an interpolated image obtained by interpolating all pixels.

Specifically, for example, an interpolation image as an input image used for subsequent processing is generated by performing filter processing using, for example, a filter illustrated in FIGS. 10 and 11 on a specific color signal. The filter coefficient and the filter size can be set as appropriate.
Then, using the interpolated image as an input image, the divided region setting unit 11 sets a processing region, a boundary line, and a divided region, and the index value calculating unit 12 calculates an index value. By doing so, the index value can be calculated without considering the color component information at the time of calculation, so that the subsequent processing can be simplified.

  In this way, by generating an interpolated image for an image signal lacking a specific color component and using this as an input image, the index value can be calculated without considering color component information at the time of calculation. As a result, subsequent processing can be simplified.

(Second Embodiment)
In the above-described embodiment, the example in which the luminance image or the pixel value mixing process between the color components is performed and the interpolation image obtained by interpolating all the pixels is processed as the input image has been described. In the present embodiment, in the case of an image signal captured by an imaging device that lacks a specific color component in each pixel, such as an image signal in a Bayer array, the specific color component is generated without generating an interpolation image. An example of processing a missing image signal will be described.

  As shown in FIG. 12, the image processing apparatus according to the present embodiment includes a color component determination unit 15 that determines the color component of each pixel in the image signal. In the following description, the same components as those in the first embodiment described above are denoted by the same reference numerals, and the description thereof is omitted.

  The divided region setting unit 11 has a plurality of boundary lines passing through the target pixel with respect to a predetermined region including the target pixel and a plurality of peripheral pixels including the target pixel in the image signal lacking a specific color component as the input image. Are set, a first region and a second region divided by the boundary line are set for each boundary line, and control information about these is output.

The color component determination unit 15 determines the color component of the input image signal and outputs it as control information. In the present embodiment, the color component of the image signal is RGB.
The index value calculation unit 12 determines the luminance between the first region and the second region set for each boundary line based on the image signal and the control information generated by the divided region setting unit 11 and the color component determination unit 15. An index value indicating a level difference is calculated. Hereinafter, as shown in FIG. 3, the processing area A has a vertical boundary line L (FIG. 3A), a horizontal boundary line W (FIG. 3B), and diagonal boundary lines D1, D2 (FIG. 3C, (D)) is an example in which the image is divided by four boundary lines to be divided and the difference sum average obtained by adding the differences of the average values calculated for each color component is added as an index value. To do.

The index value calculation unit 12 sets one area divided by the boundary line divided into the vertical boundary line L as the first area P1L and the other area as the second area P2L, and sets the color components R, The average value for each G and B is obtained. Next, the difference of the average value for every R, G, B between each area | region is calculated | required. An index is obtained by adding the average values. Incidentally, _{N R} number the number of the R component, _{N Gr} number the number of Gr component, _{N Gb} number the number of Gb component, the number of the B component and _{N B} number.
The average values averageP1L _R , averageP1L _Gr , averageP1L _Gb , and averageP1L _B of the R component, Gr component, Gb component, and B component of the P1L region are calculated according to the following formulas (21) to (24).

Similarly, R component of P2L region, Gr component, Gb component, the average value AverageP2L _R and B _{components, averageP2L Gr, averageP2L Gb, averageP2L} B, the calculated difference of the average value for each color component from the average value of the two regions difL _R , difL _Gr , difL _Gb , and difL _B are calculated (see formulas (25) to (28)).

A difference sum average is calculated by adding the differences difL _R , difL _Gr , difL _Gb , and difL _B of the average values for each color component (equation (29)), and this is used as an index value for the vertical boundary L.

Similarly, as shown in FIG. 3, the other boundary lines, that is, the horizontal boundary line W and the diagonal boundary lines D1 and D2, are also set to the first region and the second region set for each boundary line. The difference sum average for each color component from the area is calculated as an index value.
The flatness calculation unit 13 calculates the maximum value among the plurality of index values calculated by the index value calculation unit 12, and outputs the maximum value as flatness indicating the degree of flatness of the target pixel.

In addition to the difference sum average for each color component, an addition average of absolute value differences for each color component between regions or an addition average of difference squares can also be used as an index value.
In the case of the addition average of absolute value differences, it is calculated as follows. When one region divided by the boundary line divided into the vertical boundary line L is a first region P1L and the other region is a second region P2L, it is between the first region P1L and the second region P2L. Then, the average of the absolute value differences between the same colors is obtained. For example, the R component is calculated according to the following equation (30).

This calculation is performed for all color components, and the addition averages dif_averagePL _R , dif_averagePL _Gr , dif_averagePL _Gb , and dif_averagePL _B of the absolute value differences of _R , _Gr , _Gb , and _B are calculated.
Then, the sum of the average of the absolute value differences is calculated, and dif_sum that is the average of the absolute value differences is calculated as an index value for the boundary line.

dif_sum = dif_averagePL _R + dif_averagePL _Gr + dif_averagePL _Gb + dif_averagePL _B

  In the case of the addition average of the squares of the differences, it is calculated as follows. When one region divided by the boundary line divided into the vertical boundary line L is a first region P1L and the other region is a second region P2L, it is between the first region P1L and the second region P2L. The average of the difference squares of the same color is obtained. For example, the R component is calculated according to the following equation (31).

This calculation is performed for all color components, and the addition averages dif_sqrt_averagePL _R , dif_sqrt_averagePL _Gr , dif_sqrt_averagePL _Gb , and dif_sqrt_averagePL _B of the absolute value differences of _R , _Gr , _Gb , and _B are calculated.

Then, the sum of the averages of these difference squares is calculated, and dif_sqrt_sum, which is the sum of the difference squares, is calculated as an index value for the boundary line.
dif_sqrt_sum_ = dif_sqrt_averagePL _R + dif_sqrt_averagePL _Gr + dif_sqrt_averagePL _Gb + dif_sqrt_averagePL _B

  As described above, the flatness is calculated directly from the image signal without performing an interpolation process such as a process of mixing pixel values between the color components for the image signal lacking a specific color component. The flatness with high accuracy can be calculated without degrading information regarding the distribution of pixel values of the color components.

DESCRIPTION OF SYMBOLS 11 Division area setting part 12 Index value calculation part 13 Flatness calculation part 14 Interpolation image generation part 15 Color component determination part

Claims (5)

A plurality of boundary lines passing through the target pixel are set for a predetermined region including the target pixel of the input image and a plurality of peripheral pixels including the target pixel, and each boundary line is divided by the boundary line. Divided region setting means for setting the first region and the second region,
Index value calculation means for calculating an index value indicating a difference in luminance level between the first area and the second area for each boundary line;
Flatness calculation means for calculating a maximum value of the index values and outputting as a flatness of the pixel of interest ,
The index value calculation means calculates a difference value between the average value of the signal values of the first region and the average value of the signal values of the second region for each boundary line, and uses the difference value as the index value. Image processing apparatus.   Interpolation image generating means for generating an interpolated image obtained by interpolating all the pixels by performing a pixel value mixing process for each color component for each predetermined region of the image signal for the Bayer array image signal. The image processing apparatus according to claim 1, wherein the input image is the input image.   The image processing apparatus according to claim 1, wherein the number of pixels included in the first area and the number of pixels included in the second area are the same.   The image processing apparatus according to claim 3, wherein the first area and the second area have the same shape. A plurality of boundary lines passing through the target pixel are set for a predetermined region including the target pixel of the input image and a plurality of peripheral pixels including the target pixel, and each boundary line is divided by the boundary line. Divided region setting means for setting the first region and the second region,
Index value calculation means for calculating an index value indicating a difference in luminance level between the first area and the second area for each boundary line;
Flatness calculation means for calculating a maximum value of the index values and outputting as a flatness of the pixel of interest,
Calculating the index value calculating means, for each boundary line, each pixel of the first area and the pixels of the second area is associated, the index value based on the weighted average of the difference values between corresponding pixels It is that images processing apparatus.

Images