JP2870299B2 - Image signal processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing apparatus, and more particularly to an image signal processing apparatus for removing noise and enhancing edges.

[0002]

2. Description of the Related Art Elimination of noise in an image plays an important role as a first step in various image processing techniques and image recognition techniques.

[0003] There are many known algorithms for removing noise (see Television and Image Information Engineering Handbook, Ohmsha, edited by the Institute of Television Engineers of Japan). The most commonly used method is to replace the value of each pixel with a weighted average of the values of the pixels in nearby small areas. In such an averaging process, the noise component is attenuated, but the signal component is also blurred.
The Wiener filter method, which is known as an algorithm for performing image restoration based on the statistical properties of the noise component and the signal component, is, in a sense, an optimal noise removal method and is an excellent method. If applied to the data itself, it will simply result in a weighted averaging process, and the signal components will also be blurred. Although a method using a median filter is effective for suppressing the occurrence of blurring due to smoothing, it also removes signal components such as thin lines as noise.

[0004] Detecting the contours and edges of an object is also an important process in an image recognition system or the like. As an algorithm for edge detection, methods such as a differential operation, a second-order differential operation, and a Sobol operator are known. However, since these methods can be regarded as a kind of high-frequency pass filter, they are weak against high-frequency noise. Therefore, this technique also requires noise removal processing, and at that time, there is a problem how to avoid blurring of signal components.

[0005]

The conventional method of replacing each pixel value with a weighted average of the values of pixels in a nearby small area and the method of a Wiener filter both cause blurring of signal components. There were drawbacks. In addition, the method using the median filter also removes signal components such as thin lines as noise, and has a problem as a method for avoiding signal blur.

An object of the present invention, such improved <br/> disadvantages of the prior art, the processing unit of the image signal can be divided <br/> removed by the noise without causing blurring of the signal Is to provide.

[0007]

According to the present invention , an input image signal is divided into small regions including a plurality of pixels while allowing overlap, and a Fourier transform is applied to each of the small regions to calculate a Fourier amplitude. Calculating means, and the spatial distribution and frequency of the Fourier amplitude from the Fourier amplitude calculated by the amplitude calculating means.
Ru comprising a noise removing means for removing a noise component based on the distribution, and an image restoring means for calculating the removed restored image noise components subjected to Fourier inverse transform using Fourier amplitude obtained by the noise removal means The image signal
Processing device Ru obtained.

[0008]

[0009]

[0010]

According to the present invention, the input image signal is divided into several small areas, and the Fourier transform and its amplitude are calculated for each small area. Next, the Fourier amplitude is passed through a noise removal filter to attenuate the noise component, and then Fourier inverse transform is performed. If a noise reduction filter is formed on the input image signal itself and noise reduction processing is performed as in the conventional method, this usually results in simple smoothing processing, so that the high-frequency components of the signal are also attenuated. This results in blurring. On the other hand, in the present invention, since the noise removal filter processing is performed on the Fourier amplitude calculated for each small region, the noise component can be attenuated without attenuating the high frequency component, and therefore, the signal component is blurred. It is possible to remove the noise component without any noise.

[0011] In the present invention, the input image signal is divided into a number of small regions, Fourier transform and amplitude only critical spatial frequency Kc or more components which have been determined pre Me for each small region as a target Is calculated. Critical spatial frequency Kc mentioned above are pre Me set according to the degree of or how finely extracted edge portions. The edge portion of the image has high-frequency spatial frequency components, but does not have high-frequency components in a region where the intensity is constant, so that the amplitude of the Fourier transform calculated in this way has sharp changes in the intensity of edges and the like. It takes a non-zero value only in the region where it is. However, since the above operation can be regarded as a high-frequency pass filter, a value other than zero is obtained in a portion where high-frequency noise exists other than the edge portion. Then, this Fourier amplitude is passed through a noise removal filter to attenuate the noise component, and then Fourier inverse transform is performed to perform threshold processing.
An image is obtained in which noise is attenuated and edges are enhanced.

According to the present invention, the Fourier amplitude calculated for each small area is subjected to a noise removal filter process, so that the high-frequency components are not attenuated. Therefore, the edges are emphasized without blurring, and the high-frequency noise is removed. It becomes possible.

[0013]

Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of the present invention.

The image information input to the input / pre-processing unit 11 is subjected to pre-processing such as A / D conversion if necessary, and sent to the Fourier transform calculation unit 12. Fourier transform calculator 1
2 performs the following processing on this image information using a processor or the like.

Assume that the number of pixels of the input image is N × N.
First, this input image is divided into n2 × n1 × n1 × n1 pixels.
It is divided into n2 small areas. Here, when dividing an image so that there is no overlap with each other, N = n1 × n
2, but may be divided so as to overlap each other. Next, the Fourier transform and its amplitude are calculated for each of the divided small areas. A Fourier transform power spectrum which is the square of the amplitude may be calculated instead of the amplitude. Here, each small area is subscripted (X,
Y) (X, Y = 1 to n2), and one of the pixels in each small area is represented by a subscript (x, y) (x, y = 1 to n1).
The value of the pixel at the position (x, y) in the small area represented by (X, Y) is represented by I (x, y; X, Y). Then, the above-described Fourier transform and its power spectrum are calculated by the following equations.

[0016]

Here, when calculating the Fourier transform, a window function such as a Gaussian window may be used in order to reduce the discontinuity at the boundary of each small area according to a well-known method. In addition, a fast algorithm such as a fast Fourier transform can be used for this calculation. Hereinafter, the Fourier amplitude P is referred to as a local Fourier amplitude.

The noise removing unit 13 removes noise by performing a filtering process using a Wiener filter on the local Fourier amplitude P. As is well known, the Wiener filter minimizes the square of the error between the true signal and the estimated value of the signal obtained as the output of the filter when the autocorrelation functions of the signal component and the noise component are known. It is a filter that has been proven to be optimal in the sense of Specifically, the local Fourier amplitude P is represented by (n1 × n2) ²
When considered as a dimensional vector, the Wiener filter M is determined as follows. ^{M = E [PP t] /} [E [PP t] + E [nn t]] where E [] denotes statistical average operation. E [PP ^t ]
Represents a correlation matrix of a signal component, and E [nn ^t ] represents a correlation matrix of a noise component. P ^t and n ^t are transposed vectors. Also, matrix division means multiplication by an inverse matrix or a generalized inverse matrix. Here, it is also possible to use a covariance matrix by subtracting an average value in advance instead of the correlation matrix. The filtering process is given by P '= M · P. In the normal case, the correlation between small regions far from each other is zero, so as is often done,
Assuming that the estimated P ′ is determined only from the values in nearby small areas,
It is also possible to reduce the amount of calculation by approximating the calculation for obtaining the filter.

In this embodiment, the noise removal operation using the Wiener filter is performed not on the input image signal itself as in the conventional method but on the local Fourier amplitude calculated from the input image signal. This is an important point. That is, when a Wiener filter is formed on the input image signal itself and noise attenuation processing is performed as in the conventional method, this usually results in a simple smoothing processing over a range of about the correlation distance of the signal. As a result, the high frequency components of the signal are also attenuated, thereby causing blurring. On the other hand, in the present invention, since the local Fourier amplitude is subjected to the Wiener filter processing, the noise component can be attenuated without attenuating the high frequency component, and therefore, the noise component can be removed without causing a blur in the signal component. It becomes possible.

The image restoration unit 14 performs an inverse transform of the processing in the Fourier transform unit 12 from the local Fourier transform in which the noise has been attenuated in this way, and calculates and outputs a restored image in which the noise has been attenuated. As is well known, in this case, not only the amplitude but also the phase information is required to perform the inverse Fourier transform. This phase information is received from the local Fourier transform F calculated by the Fourier transform unit 12. If necessary, the image restoration unit performs threshold processing, and sets a negative value to 0.
Or perform cut-off processing for a value that is too large.

In the present embodiment, the filtering processing by the Wiener filter is performed as the processing in the noise removing unit. However, this part can be replaced by a neural network having a learning function. In this case, since the statistical properties of the signal component and the noise component can be extracted by learning, an effect of reducing the design labor can be obtained.

It is well known that when a multilayer neural network is trained by the back propagation method, a learned pattern can be output as an output pattern when an input pattern is input if the learning conditions are satisfied. ing. Therefore, a multi-layer neural network is used as the noise removing unit, and a signal P + n with a noise component is added.
Is input and learning is performed so as to output the signal P from which noise has been removed as a teacher pattern, and thereby, it is also possible to remove noise components.

It is known that the relationship between an input pattern and an output pattern corresponding thereto can be learned by performing orthogonal learning using a two-layer neural network. Therefore, it is also possible to use a neural network that performs orthogonal learning as the noise removing unit. When orthogonal learning is performed so that the signal P + n with noise is input and the signal P from which noise is removed is output, assuming that the noise component is independent of the signal component and that the average value is 0, the learning is performed. Taking sufficiently large constant coupling matrix of the neural network is ^{M = E [PP t] /} [E [PP t] + E [nn t]] Shunichi to converge has been demonstrated (Amari Author, neural networks Mathematics, Sangyo Tosho, 1978). Here, E [] represents a statistical averaging operation. E [PP ^t ] represents a correlation matrix of a signal component, and E [nn ^t ] represents a correlation matrix of a noise component. P ^t and n ^t are transposed vectors. Also, matrix division means multiplication by an inverse matrix or a generalized inverse matrix. Therefore, it is understood that this neural network can construct a Wiener filter by learning by using linear elements. In this case, there are advantages in that the number of elements is reduced and the learning process is simplified as compared with a multilayer neural network.

Further, it is also possible to use a two-layer neural network for performing correlation learning, which has a simpler learning process, as the noise removing unit. When the self-recall that outputs the signal P by itself as the input is learned by the correlation learning, the coupling matrix of the neural network converges to M = E [PP ^t ]. When the correlation matrix trained in this way is used, the correlation matrix of the noise component represents the unit matrix by 1, and E [n
Assuming that n ^t ] = σ ² 1, for example, a Wiener filter can be constituted by a circuit as shown in FIG. FIG. 3 is a block diagram in which a noise removing unit is configured using a neural network that performs correlation learning. In FIG. 3, elements 32-1 to 32-N change the output Z according to the following equation. dZ / dt =-(1 / τ) (σ ² Z + P′-P) Here, P is an input signal, and P ′ is given by P ′ = M · Z. When the circuit of FIG. 3 reaches a steady state, σ ² Z + P′−P = 0 holds. Therefore, σ ² Z + M · Z−P = 0 is satisfied. When this is solved, Z = P / (σ ² + M), and eventually P ′ = M · Z = (M / (σ ² + M)) · P. , P ′, an output equivalent to the output of the Wiener filter can be obtained. In this case, the learning process is further simplified compared to a neural network that performs orthogonal learning, and when the noise level σ ² is unknown, the process can be performed while changing the noise level σ ² as a parameter. There is.

FIG. 2 is a block diagram showing a second embodiment of the present invention. The image information input to the input / preprocessing unit 21 is subjected to preprocessing such as A / D conversion as necessary, and is sent to the Fourier transform calculation unit 22. The Fourier transform unit 22 first divides the input image into small areas, as in the first embodiment of the present invention shown in FIG. Next, for each of the small areas divided in this way, only the components equal to or higher than the critical spatial frequency Kc previously input from the parameter input unit 23,
The Fourier transform and its amplitude are calculated as in the embodiment of FIG. The value of the critical spatial frequency Kc is set in accordance with the purpose of how much a gentle edge should be emphasized.
Assuming that the maximum spatial frequency is Kmax, in order to emphasize up to an edge portion that changes over about m pixels, a critical space is required.
The inter-frequency Kc may be set to about Kmax / m. Since intensity of the image is its Fourier transform in certain areas do not contain high frequency components, thus the local Fourier transform F 及 beauty amplitude P that is calculated has a value only in the region where the intensity is changing rapidly, Therefore, the edge is emphasized. Hereinafter, similarly to the embodiment of FIG. 1, the noise removing unit 24 performs filtering on the local Fourier amplitude P by a Wiener filter to remove noise, and the image restoring unit 25 performs the inverse processing of the processing in the Fourier transform unit 22. performs conversion, noise and calculates and outputs restored image edges are emphasized attenuated.

[0026]

As described above, according to the present invention, the input image
The signal is divided into small areas containing multiple pixels, allowing
Perform Fourier transform for each small area to calculate Fourier amplitude
Means for calculating the amplitude, and the flow calculated by the means for calculating the amplitude.
From the Fourier amplitude to the spatial and frequency distribution of the Fourier amplitude
Noise removing means for removing a noise component based on the noise component;
Fourier inverse transformation using the Fourier amplitude obtained by the removal means
To calculate a restored image with noise components removed
By providing a restoring means, it has the effect that the Ru can remove noise components in the image signal without causing blurring of the signal components.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a block diagram showing a second embodiment of the present invention.

FIG. 3 is a block diagram illustrating a configuration of a noise removing unit using a neural network that performs correlation learning.

[Explanation of symbols]

 11, 21 Input / preprocessing unit 12, 22 Fourier transform calculation unit 13, 24 Noise removal unit 14, 25 Image restoration unit 23 Parameter input unit 31 Correlation matrix 32-1, -32-N Analog arithmetic element

Claims (1)

(57) [Claims] 1. An amplitude calculating means for dividing an input image signal into small areas including a plurality of pixels while permitting overlap and performing a Fourier transform on each of the small areas to calculate a Fourier amplitude. Fourier amplitude from the calculated Fourier amplitude
A noise removing means for removing a noise component based on the spatial distribution of the width and the frequency distribution, and an image for performing a Fourier inverse transform using the Fourier amplitude obtained by the noise removing means and calculating a restored image from which the noise component has been removed An image signal processing device comprising: a restoration unit.