JPH07147670A - Image data interpolating device - Google Patents

Abstract

PURPOSE:To provide an image data interpolating device which can secure the satisfactory picture quality of interpolation without deteriorating the output signal when the value of the pixel that is not included in the input signal is interpolated. CONSTITUTION:The input digital video signal of the interlaced scanning is supplied to a movement detecting circuit 5, and a moving or stationary state of a pixel to be interpolated is decided. When a moving state is decided, a level distribution pattern of plural pixels existing near the decided pixel in terms of time and space is detected by a class code generating circuit 6a. Then a class code is generated. In the same way, a class code is also generated for the pixel whose stationary state is decided. The optimum prediction coefficients acquired previously by learning are stored in the memories 4a and 4b with those class codes used as the addresses. An interpolation arithmetic circuit 3 generates the interpolation value on an interpolation scanning line by the primary linear connection secured between the prediction coefficients read out of the memories and the value of plural pixels.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data interpolating device which can be applied to a scanning line interpolating device for interpolating a horizontal scanning line missing in, for example, interlaced scanning.

[0002]

2. Description of the Related Art Current standard television signals (for example, NTSC system) have problems such as line flicker due to interlaced scanning. In order to solve this problem, there has already been proposed a scanning line interpolating device which interpolates a missing scanning line on the receiver side and converts it into a progressive scanning television signal. FIG. 5 is an example of a conventional motion adaptive scanning line interpolation device.

The input terminal indicated by 21 has, for example, 13.5.
An interlaced-scan digital video signal is supplied, which is sampled at a frequency of MHz and 1 sample (1 pixel) is quantized to 8 bits. This input signal is supplied to the intra-field interpolation circuit 22, the inter-field interpolation circuit 23, and the motion detection circuit 24, respectively. The intra-field interpolation circuit 22 performs interpolation with signals in the same field. For example, the average value of two samples located on the upper and lower scanning lines is used as the interpolation value. Inter-field interpolation circuit 23
Is a circuit that interpolates at the pixel at the same position as the interpolation pixel on the scanning line at the same position as the interpolation scanning line of the previous field in terms of time.

The motion detection circuit 24 detects the amount of motion for each pixel. The motion detection circuit 24 uses the motion coefficients k and 1-k.
To occur. The motion coefficient k is (k =
1), and the coefficient becomes smaller as the amount of movement increases.

The output signal of the intra-field interpolation circuit 22 is supplied to the multiplication circuit 25, and the output signal of the inter-field interpolation circuit 23 is supplied to the multiplication circuit 26. The multiplication circuit 25 multiplies the coefficient by 1-k, and the multiplication circuit 26 multiplies k.
Is multiplied by the coefficient of. These multiplication circuits 25 and 26
Are supplied to the adder circuit 27, and the adder circuit 27 generates an interpolated output signal in which the output signals of the interpolators 22 and 23 are mixed.

[0006]

The conventional interpolating circuit uses an intra-field interpolated signal and an inter-field interpolated signal by weighting according to the motion amount which is empirically determined according to the characteristics of the image signal assumed from the motion amount. It mixes with the signal. However, when the assumed image signal characteristic and the actual image signal characteristic are different, there is a problem that not only good interpolation cannot be performed but also an output signal is deteriorated such as an afterimage. Furthermore, when the intra-field interpolation and the inter-field interpolation are used, the ability of the interpolator is not sufficient, and the degree of improvement in image quality due to the conversion from interlaced scanning to progressive scanning is insufficient.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an image data interpolating device capable of obtaining a good interpolated image quality without causing deterioration of an output signal.

[0008]

SUMMARY OF THE INVENTION The present invention relates to an image data interpolating device for interpolating pixel data that does not exist in an input image signal, and a motion detecting circuit for detecting the amount of motion of the input image signal. A prediction coefficient obtained by learning in advance is stored for each motion amount, and a coefficient storage unit for outputting a prediction coefficient according to the detected motion amount information from the motion detection circuit, and a prediction from the coefficient storage unit An interpolation device for image data, comprising: an interpolation value generating means for generating an interpolation value of pixel data that does not exist in an input image signal using a coefficient.

[0009]

The prediction coefficient for generating the optimum interpolated value is determined for each motion amount in advance by learning, and this is stored in the memory. The motion amount of the actual input image data is detected, the motion amount corresponding to this motion amount is read from the memory, and the linear linear combination of the read coefficient and the spatially and temporally neighboring pixel data of the interpolation target pixel is performed. , Interpolation values are generated.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a scanning line interpolation device will be described below. In FIG. 1, 1
Is an input terminal for interlaced scanning digital video signals. Specifically, a transmission signal by broadcasting or the like, a reproduction signal from a VTR or the like is supplied to the input terminal 1. Reference numeral 2 is a time series conversion circuit for converting an input signal into a block structure signal.

The output signal of the time series conversion circuit 2 is supplied to the interpolation calculation circuit 3 and the motion detection circuit 5. The interpolation calculation circuit 3 is connected to memories 4a and 4b in which prediction coefficients acquired by learning in advance are stored as described later. The prediction coefficient is a coefficient for accurately estimating the interpolated value on the skipped horizontal scanning line.

As will be described later, the motion detection circuit 5 detects a motion amount from the sum of absolute values of inter-frame differences for each block, for example. The inter-frame difference is obtained by subtracting pixel data at the same position between the current frame and the temporally previous frame. This interframe difference is converted into an absolute value, and the sum totaled for one block is compared with a predetermined threshold value to determine whether the pixel of interest in the current frame is a moving pixel or a still pixel.

The output signal of the motion detection circuit 5 is supplied to the class code generation circuits 6a and 6b. Is the target of interpolation,
The pattern of the level distribution of a plurality of pixels near the target pixel, that is, the class is determined by the class code generation circuits 6a and 6b. A class code is created that points to this class.

The class code is supplied to the memories 4a and 4b as addresses, and the prediction coefficient corresponding to the class is read from the memories 4a and 4b, respectively. The memories 4a and 4b are not limited to separate memories, but a memory area of one memory may be divided and a memory area may be selected according to the result of motion / still determination. The prediction coefficient is supplied to the interpolation calculation circuit 3, and the interpolation value of the pixel of interest is formed. The interpolation calculation circuit 3 outputs the interpolated value of the pixel on the scanning line skipped to the output terminal 7. The existing scanning line data (input data) and interpolation scanning line data can be combined by a frame memory (not shown).

FIG. 2 is for explaining the class classification and the interpolation calculation, and shows a part of each of the current field (nth field) and the previous field ((n-1) field). When generating the pixel value y on the non-existing scan line Ln + 1 between the scan lines Ln and Ln + 2 of the current field, the pixel values shown are used to correspond to the level distribution. Class classification and interpolation operations are performed.

As an example, the values x ₂ and x _{5 of} two pixels located on the interpolation scanning line in the current field and located above and below the interpolation pixel, and the values x ₁ and x of pixels on both sides of the pixel x _{2. 3} and the pixel values x ₄ and x on both sides of the pixel x _5.
6 , the pixel value x ₈ located on the scanning line Ln + 1 in the previous field and located at the same position as the pixel of interest, and the pixel values x ₇ and x _{9 on} both sides thereof are used. The interpolation value y of the pixel of interest is generated by the interpolation calculation circuit 3 by the following linear linear combination when the prediction coefficients are represented by w ₁ to w ₉ .

Y = w ₁ x ₁ + w ₂ x ₂ + w ₃ x ₃ + w ₄
x ₄ + w ₅ x ₅ + w ₆ x ₆ + w ₇ x ₇ + w ₈ x ₈ + w ₉
x ₉ (1)

Such interpolation processing is performed for each of the moving pixel and the still pixel in response to the result determined by the motion detecting circuit 5. In the class code generation circuits 6a and 6b, the level distribution pattern is detected by referring to, for example, the same nine pixels used in the interpolation calculation. Here, the pixel referred to for class classification and the pixel used for the interpolation calculation are the same, but they are not necessarily required. If the value of each pixel is 8 bits, the number of classes will be very large, and the hardware scale will increase due to an increase in memory capacity.
The number of bits in these pixels is compressed.

As an example, ADRC (Adaptive Dynamic
The value of the pixel referred to by the range coding is compressed. ADRC is a technique for adaptively removing redundancy in the level direction by utilizing local correlation of images. That is, within the dynamic range of 0 to 255 possessed by the 8-bit original data, the intra-block dynamic range required for requantization for each block composed of a plurality of spatially and temporally neighboring pixels is significantly small. It is noted that the number of quantization bits of each pixel is significantly reduced from the original 8 bits.

More specifically, 1-bit ADRC can be used. That is, the maximum value and the minimum value of the block including the reference pixels x _{1 to} x ₉ described above are detected, the dynamic range that is the difference between the maximum value and the minimum value is detected, and the value of the reference pixel is divided by the dynamic range. , Its quotient is compared with 0.5, and those with 0.5 or more are coded as `1 ', and those with less than 0.5 are coded as` 0'. Therefore, a 9-bit class code is generated. ADRC that generates an output of a bit number other than 1 bit may be adopted.

Not only ADRC but also DPCM (Differentia
l pulse code modulation), BTC (Block Trancation
Coding), VQ (vector quantization) and other compression encoding encoders can be used as the class code generating circuits 6a and 6b. Further, although the reference pixels are converted into data having the same number of bits, the number of allocated bits may be changed in consideration of the distance between the pixel of interest and the reference pixel. That is, the number of allocated bits of the reference pixel closer to the pixel of interest is made larger than the number of bits of the farther pixel.

An example of motion detection will be described with reference to FIG. The nth frame is composed of the nth field and the (n-1) th field of FIG. The motion detection circuit 5, 9 pixels at the same position in the one block formed from the above nine pixels in the current frame (n th frame) (x ₁ ~x _9), the previous frame ((n-1) frame) The absolute value sum ΣΔF of the frame differences between one block consisting of (x ₁ ′ to x ₉ ′) is obtained. That is, ΣΔF = | x ₁ −x ₁ ′ | + | x ₂ −x ₂ ′ | + ...
_{··· + | x 9 -x 9 '} | (2)

This sum of absolute values ΣΔF is compared with a predetermined threshold value. When it is larger than the threshold value, the central pixel (x ₈ ) in the block is judged to be a moving pixel. It is determined as a still pixel. Then, as described above, the class code generation circuit 6a forms a class code corresponding to the level distribution pattern for the moving pixel, and the class code generation circuit 6b similarly corresponds to the level distribution pattern for the still pixel. To form.

As described above, in one embodiment of the present invention, the motion detection and the class classification based on the level distribution pattern are used together. However, only the detected motion amount,
You may make it classify. For example, the detected motion amount is divided into 256 classes, and an 8-bit class code is generated. The coefficient corresponding to each class is acquired by learning. Furthermore, as the sum of absolute values of frame differences, not only the previous frame but also the sum of absolute values of frame differences of subsequent frames may be used together. Further, not limited to the method of detecting the motion amount from the absolute value of the frame difference, various known motion amount detection methods such as the block matching method can be used.

Prediction coefficients acquired by learning in advance are stored in the memories 4a and 4b. FIG. 4 is a flowchart showing the operation when learning is performed by software processing. The control of the learning process is started from step 11, and in the learning data formation of step 12, learning data corresponding to a known image is formed. Specifically, as described above, a plurality of pixels in the arrangements of FIGS. 2 and 3 can be used. At the end of the data in step 13, if the processing of all the input data, for example, one frame of data has been completed, the process proceeds to step 16 to determine the prediction coefficient, and if not, the process proceeds to step 14 for motion detection and class determination. Shifts.

The motion detection and class determination in step 14 are
As described above, the motion / stillness of the pixel of interest is determined, and
This is a step of detecting a pattern of level distribution centered on the pixel of interest. In the normal equation addition of the next step 15, a normal equation described later is created.

After the processing of all the data is completed from the end of the data in step 13, the control is transferred to step 16,
In the determination of the prediction coefficient of No. 6, the coefficient is determined by solving the equation (10) described later using the matrix solution method. The prediction coefficient store in step 17 stores the prediction coefficient in the memory, and then step 1
At 8, the control of the learning process ends.

Step 15 in FIG. 4 (normal equation generation)
The process of step 16 (determination of prediction coefficient) will be described in more detail. Let the true value of the pixel of interest be y and its predicted value be y
′ And the values of the surrounding pixels are x ₁ to x _n ,
Setting the coefficient for each class w ₁ linear combination of n taps by _{~w n y'= w 1 x 1} + w 2 x 2 + ‥‥ + w n x n (3). Before learning, w _i is an undetermined coefficient.

As mentioned above, learning is done for each class,
When the number of data is m, according to the equation (3), y _j ′ = w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (4) (where j = 1, 2, ...

When m> n, w _{1 to} w _n are not uniquely determined, so the elements of the error vector E are e _j = y _j − (w ₁ x _j1 + w ₂ x _j2 + ... + w _n x _jn (5) (where j = 1, 2, ..., M), and the coefficient that minimizes the following equation (6) is obtained.

[0031]

[Equation 1]

This is a so-called least squares method. Here, the partial differential coefficient by w _i of the equation (6) is obtained.

[0033]

[Equation 2]

Since each w _i may be determined so that the equation (7) is set to 0,

[0035]

[Equation 3]

Using a matrix as

[0037]

[Equation 4]

[0038] This equation is generally called a normal equation. If this equation is solved for w _i using a general matrix solution method such as a sweeping method, the prediction coefficient w _i is obtained, and the prediction result w _i is stored in memory using the motion / still decision result and the class code as an address. Store it.

Although FIG. 4 shows a software configuration for learning, learning can also be performed by a hardware configuration. Further, the interpolation value is not limited to the linear linear combination using the prediction coefficient, but the data value itself may be created in advance by learning, and this value may be used as the interpolation value. Further, the present invention can be applied not only to the scanning line interpolation but also to the case of interpolating the values of non-existing pixels such as interpolating the values of pixels thinned out by subsampling.

[0040]

According to the present invention, the optimum prediction coefficient for each motion amount is previously obtained by learning, the prediction coefficient is stored in the memory, and this coefficient is used during interpolation, so that the output signal is deteriorated. It is possible to obtain a good interpolation image quality without causing

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment in which the present invention is applied to a scanning line interpolation device.

FIG. 2 is a schematic diagram showing an array of pixels used for class classification and interpolation in the present invention.

FIG. 3 is a schematic diagram for explaining an operation of detecting a motion amount in the present invention.

FIG. 4 is a flowchart when learning for obtaining a prediction coefficient is performed by software processing.

FIG. 5 is a block diagram of a conventional scanning line interpolation device.

[Explanation of symbols]

 3 interpolation calculation circuit 4a, 4b memory storing prediction coefficients 5 motion detection circuit

Claims (4)

[Claims] 1. An image data interpolating device for interpolating pixel data that does not exist in an input image signal, a motion detecting means for detecting a motion amount of the input image signal, and a prediction obtained by learning in advance. A coefficient is stored for each motion amount, and a coefficient storage unit for outputting the prediction coefficient according to the detected motion amount information from the motion detection unit, and a prediction coefficient from the coefficient storage unit are used. And an interpolation value generating means for generating an interpolation value of pixel data not present in the input image signal. 2. The image data interpolating device according to claim 1, wherein the motion detecting means detects the amount of motion based on a difference between frames. 3. The image data interpolating apparatus according to claim 1, wherein the input image signal is an interlaced scanning signal, and interlaced horizontal scanning lines in a field are interpolated to output a progressive scanning signal. An image data interpolating device. 4. The image data interpolating device according to claim 1, wherein the target pixel is interpolated by a level distribution pattern of a plurality of reference pixels spatially and / or temporally adjacent to the target pixel. An interpolating device for image data, further comprising a class classification means for determining a class.