JP2894140B2 - Image coding method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for encoding an image, and more particularly to an encoding method using a correlation between frames of a moving image or the like and between fields by zooming or the like in an image sequence. The present invention relates to an image encoding method that is efficient when an encoded image exists.

[0002]

2. Description of the Related Art Conventionally, as a moving image coding method, a sum and difference image between frames or between fields is divided into N × M (size 8 × 8) blocks, and the data of the blocks is divided into N × M blocks. To an orthogonal transform (eg, DCT:
There is a method in which a transform such as discrete cosine transform) is performed, and the obtained transform coefficient is quantized and coded. FIG.
0 shows a block diagram of a conventional encoding device as described above. Hereinafter, a conventional encoding device will be described with reference to FIG.

In FIG. 10, reference numeral 1000 denotes an image input unit;
Reference numerals 1 and 1002 denote frame memories for temporarily storing one frame of image data. The first frame of the image data input from the image input unit 1000 is a frame memory 1001.
The second frame is stored in 1002. An adder 1003 obtains a result of adding the image data stored in the frame memory 1001 and the image data stored in the frame memory 1002. Reference numeral 1004 denotes a subtractor, which stores image data stored in the frame memory 1001 and a frame memory.
A difference value of the image data stored in 1002 is obtained.

[0004] A selector 1005 switches the output of the adder 1003 and the output of the subtractor 1004. Reference numeral 1006 denotes a DCT (discrete cosine transform) unit, which outputs N
It is divided into × M blocks (size 8 × 8) and subjected to DCT. Reference numeral 1007 denotes a quantization unit that performs quantization processing on the block DCT-processed by the DCT unit 1006. Reference numeral 1009 denotes a variable length coding unit which performs entropy coding on the output of the quantization unit 1007.

[0005] Further, when there is an image enlarged or reduced by zooming in an image sequence, the correlation between frames and fields is reduced over the entire image, and the coding efficiency is reduced. Forming a frame group composed of a plurality of frames, setting a reference field for each input field belonging to the frame group, and enlarging / reducing each input field using a motion vector of each input field with respect to the reference field. An image is generated and used as a field to be coded, thereby increasing the coding efficiency. If there is an image enlarged with respect to the reference field due to zooming in the image sequence, a reduced image is generated by thinning out the pixel lines in the horizontal or vertical direction as a pre-encoding process to generate a reduced image as the encoded field. At the time of decoding, the decoded field obtained by decoding is interpolated from adjacent vertical or horizontal lines with pixels decimated by pre-coding processing as post-decoding processing to generate an enlarged image of the decoded field. I do.

[0006] When an image reduced in size relative to a reference field due to zooming in an image sequence, an enlarged image is generated by inserting pixel lines in the horizontal or vertical direction as a pre-encoding process. An encoded field is generated, and an enlarged image of the decoded field is generated by thinning out the pixel lines inserted in the pre-coding process as the post-coding process from the decoded field obtained by the decoding at the time of decoding.

[0007]

However, in the above-mentioned conventional encoding method, zooming is performed in an image sequence.
If there is an enlarged image for the reference field,
As a pre-encoding process, in the horizontal or vertical direction, a reduced image is generated by thinning out pixel lines and used as a field to be coded, and at the time of decoding, an already decoded field obtained by decoding is processed as a post-decoding process Interpolation of the pixel lines thinned out in the pre-encoding process from adjacent horizontal or vertical lines to generate an enlarged image of the already-decoded field causes deterioration at the center of the image due to the interpolation process of the post-decoding process. There was a problem that it occurred.

In view of the above, the present invention has been made in consideration of the above-described problems in the encoding using the correlation between frames and fields between moving images and the like, particularly when there is enlargement / reduction of an image due to zooming. It is an object of the present invention to increase the correlation between fields, suppress deterioration occurring at the center of an image during reproduction even in the case of enlarged zoom, and improve coding efficiency.

[0009]

In order to achieve the above object,
In the first invention, when there is an image enlargement / reduction by zooming in an image sequence, one reference field is set for each input field belonging to the same frame group composed of n frames, A motion vector of each field with respect to the reference field is detected, and when the input field with respect to the reference field is an enlarged image, pixels that are not adjacent to each other in the horizontal and vertical directions are thinned out using the motion vector, and a reduced image of the input field is reduced. Is generated, and the reduced image is used as a field to be coded.

[0010] In the second invention, in the post-decoding process, when the input field belonging to the same frame group is an enlarged image with respect to the reference field, the encoded field is added to the decoded field obtained by the decoding process. The pixels thinned out by the pre-processing are interpolated using peripheral pixels in the same field as reference pixels.

Further, in the third invention, in the post-decoding process, if the input field belonging to the same frame group is an enlarged image with respect to the reference field, a code is added to the already-decoded field obtained by the decoding process. Pixels thinned out by the pre-processing are interpolated using peripheral pixels of the same field and peripheral pixels of other fields constituting the same frame as reference pixels.

[0012]

According to the first aspect of the invention, if an image sequence includes an image enlarged or reduced by zooming in the image sequence, if the input field for the reference field is an enlarged image, the movement of the input field relative to the reference field A reduced image of the input field is generated by thinning out pixels that are not adjacent in both the horizontal and vertical directions using a vector, so that horizontal and vertical edges are not completely lost.

In the second and third inventions, in the post-decoding process, when the input field belonging to the same frame group is an enlarged image with respect to the reference field, the decoded field obtained by the decoding process is The pixels decimated by the pre-encoding process are interpolated by using pixels having higher correlation, such as peripheral pixels of the same field or peripheral pixels of the same field and peripheral fields of another field constituting the same frame, as reference pixels. Therefore, image quality degradation due to interpolation can be further suppressed.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An image coding method according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an image encoding apparatus according to an embodiment of the present invention.

In FIG. 1, reference numeral 100 denotes an image input unit to which field data is input. Reference numeral 101 denotes a motion vector detection unit which detects a motion vector corresponding to reference field data. Reference numeral 102 denotes a zoom coefficient calculation unit that calculates a zoom coefficient based on the vector obtained by the motion vector detection unit 101. Reference numeral 103 denotes a field memory, which is a memory for one field that temporarily stores field data input from the image input unit 100.
Reference numeral 105 denotes a sampling address control unit which controls the sampling address in the enlargement / reduction processing unit 104 based on the zoom coefficient obtained by the zoom coefficient calculation unit 102. An enlargement / reduction processing unit 104 generates an enlarged image or reduced image of the field data stored in the field memory 103 in accordance with the address control of the sampling address control unit 105. Reference numeral 106 denotes a switch unit, which switches between field data read from the field memory 103 by the reference field control signal 107 and an enlarged image or a reduced image generated by the enlargement / reduction processing unit 104 and outputs the switched image to the encoding unit 108.

Referring to FIG. 1, FIG. 4, FIG. 6, and FIG.
The detailed operation of the block diagram of FIG. FIG. 3 shows a frame group composed of two frames (four fields).
It is composed of fields # 0 to # 3. Processing when field # 0 is used as a reference field in this frame group will be described below. In this case, it is assumed that the zoom is an enlarged zoom in this case.

First, as an input image, a reference field #
0 is input from the image input unit 100. At this time, the reference field control signal 107 is asserted. If the reference field control signal 107 is asserted, the motion vector detection unit 101 temporarily stores the data of the reference field # 0. In this case, it is not necessary to store all the data of the reference field # 0, and only the representative point may be stored (for example, JP-A-2-5689). In the same cycle, the data of the reference field # 0 is written to the field memory 103.

In the next cycle, the switch unit 106 uses the reference field control signal 107 to control the field memory 1
The reference field # 0 read from No. 03 is selected,
Output to encoding section 108. Further, field data # 1 is input from the image input unit 100 in the same cycle. In motion vector detecting section 101, reference field #
A motion vector is detected from 0 and field # 1, and a zoom coefficient of field # 1 with respect to reference field # 0 is calculated by zoom coefficient calculation section 102 based on the vector value.

FIG. 4 shows the relationship between the zoom coefficient and the motion vector. FIG. 4A is a diagram showing a motion vector of a target field (in this case, field # 1) with respect to a reference field (in this case, field # 0). In FIG. 4, H indicates the horizontal direction, V indicates the vertical direction, the upper left is the reference point, and a circle indicates the center of the zoom. Since FIG. 4 is an enlarged zoom, the motion vector extends toward the center of the zoom. FIG. 4B shows a state in which the horizontal motion vector in FIG. 4A is approximated by a linear function. The motion vector Vh (x) at a position that is x pixels away from the reference point in the horizontal direction is a horizontal zoom coefficient (ah, bh).
And Vh (x) = ah * x + bh. Similarly, FIG. 4C shows a state in which the vertical motion vector in FIG. 4A is approximated by a linear function. The motion vector Vv (y) at a position vertically y pixels away from the reference point is the vertical zoom coefficient (av,
Vv (y) = av * y + bv using bv).

As described above, the zoom coefficient calculation unit 102 calculates the horizontal zoom coefficients (ah01, bh01) and the vertical zoom coefficients (av01, bv01) from the motion vector of the field # 1 with respect to the reference field # 0. I do.

In the sampling address control unit 105,
According to the zoom coefficient obtained by the zoom coefficient calculation unit 102,
An address is generated and given to the enlargement / reduction processing unit 104.
Here, pixel thinning will be described. FIG. 6 shows an example of thinning out horizontal lines as an example of thinning. FIG.
Represents the state of the field data, and the numerals represent the pixels. FIG. 6A is a diagram showing a state of line thinning out in which all pixels on the same line are thinned out. Lines indicated by arrows are thinning lines.
1,. . . Pixels 18 and 19 are thinned out. FIG. 6 (b)
FIG. 4 is a diagram showing a state of a field after pixels are thinned out.

In FIG. 6, a hatched portion 61 indicates a line in which no pixel exists due to the thinning process. Encoding is performed using this field data as an encoded field. In this case, in order to suppress the deterioration of the coding efficiency, a process of interpolating a line in which a pixel no longer exists from neighboring pixels is performed. For example, a process of replacing pixels with pixels (30, 31,..., 38, 39) on one line is performed. However, in this method, all the pixels (10, 11,..., 18, 19) on the same line are lost. Therefore, if an edge exists on the same pixel line, the edge is completely lost. Unlike the thinning method of thinning out all pixels on the same line, the sampling address control unit 105 of the present embodiment employs a zoom coefficient calculating unit 102
A sampling address for thinning out pixels that are not adjacent to each other in the horizontal direction and the vertical direction is generated from the zoom coefficient obtained in (1), and is provided to the enlargement / reduction processing unit 104.

FIG. 7 shows one embodiment of the thinning-out state of the present invention. FIG. 7A shows the thinning-out state of the present embodiment. As shown in FIG. 7A, oblique pixels that are not adjacent to each other are thinned out, that is, in this case, 10, 21, 12, 2
Pixels of 3, 14, 25, 16, 27, 18, and 29 are thinned out. In this way, by thinning out non-adjacent pixels, it is possible to prevent all pixels on the same line from being lost. FIG. 7B illustrates a state of a field after pixels are thinned out.
In the figure, a hatched portion 71 indicates a line in which no pixel exists due to the thinning process. Encoding is performed using this field data as an encoded field.

In this case, in order to suppress the deterioration of the coding efficiency, a process of interpolating the line where the pixel no longer exists from the neighboring pixels is performed. For example, a process of replacing pixels with pixels (30, 31,..., 38, 39) on one line is performed. The address generated by the sampling address control unit 105 is not limited to the sampling pattern shown in FIG. 7A, and may be any sampling pattern that stores pixels that are adjacent in both the horizontal and vertical directions.

As described above, the sampling address control unit 1
In accordance with 05, a reduction image of the field # 1 is generated by the enlargement / reduction processing unit 104 with respect to the data of the field # 1 temporarily stored in the field memory 103, and is output to the encoding unit 108 as a field to be encoded. Similarly, for field # 2 and thereafter, a reduced image for reference field # 0 is generated and output to encoding section 108.

Next, the decoding process will be described. FIG. 2 is a block diagram showing a decoding apparatus according to one embodiment of the present invention. In FIG. 2, reference numeral 200 denotes a decoded data input unit for inputting decoded data, 201 denotes a field memory for temporarily storing a field input from the decoded data input unit 200, and 202 denotes a zoom coefficient input unit 207. A zoom coefficient memory unit 204 for temporarily storing the input zoom coefficient is a sampling address control unit, and controls the sampling address in the enlargement / reduction processing unit 203 based on the zoom coefficient read from the zoom coefficient memory unit 202.

An enlargement / reduction processing unit 203 generates an enlarged image or reduced image of the field data stored in the field memory 201 according to the address control of the sampling address control unit 204. Reference numeral 205 denotes a switch, which switches and outputs the field data read from the field memory 201 based on the reference field control signal from the zoom coefficient memory and the enlarged image or the reduced image generated by the enlargement / reduction processing unit 203.

The detailed operation of the block diagram of FIG. 2 will be described with reference to FIGS. 3, 6, and 7. For the frame group composed of field # 0 to field # 3 in FIG. 3, the fields to be coded generated at the time of the pre-coding process are referred to as fields # 0 'to # 3', respectively, and are obtained by coding and decoding. Decoded field to field #
0 ″ to # 3 ″, and the field obtained in the post-decoding process is set to field # 0 ″ ′ for the decoded field.
~ # 3 '''. First, field # 0 ″ is input from the decoded data input unit 200 as decoded data. The zoom coefficient memory unit 202 has a zoom coefficient input unit 2
07 to the zoom coefficient ((ah0
0, bh00), (av00, bv00)) = ((0,
0), (0, 0)).

The zoom coefficient memory unit 202 asserts a reference field control signal and supplies it to the switch unit 205.
Data input from the decoded data input unit 200 in the same cycle is written to the field memory 201. In the next cycle, field # from field memory 201
0 '' is read out, and the output image 20
As 6, the reference field # 0 '' is output.

In the same cycle, the decoded data input unit 2
00, field # 1 '' is input. The zoom coefficient memory unit 202 negates the reference field control signal and supplies the negated reference field control signal to the switch unit 205.
Is written in field # 1 ''. In the sampling address control unit 204, the zoom coefficient memory unit 20
2, the zoom coefficient ((ah01, bh01), (av01, bv01)) of the field # 1 '' read out, an address is generated, and the enlargement / reduction processing unit 20
Give to 3. The enlargement / reduction processing unit 203 generates and outputs an enlarged image.

FIGS. 6C and 6D show how an enlarged image is generated in a line thinning method for thinning all pixels on the same line. FIG. 6C shows the decoded field data. The hatched portion 62 is a pixel having no data due to the thinning-out process by the zoom coefficient during the encoding process.
This portion is a pixel interpolated by processing such as interpolation from peripheral pixels in the same field. FIG.
(D) is an enlarged image of the decoded field (FIG. 6 (c)) obtained by interpolating pixels at the pixel positions thinned out during the pre-encoding processing by the zoom coefficient. Pixels 10 to 19 are pixels interpolated by peripheral pixels, for example, upper and lower pixels (marked by ○).

However, since pixels serving as reference pixels do not exist on the same line, image quality deterioration due to interpolation becomes large, for example, when an edge exists on an interpolation line. Unlike the line thinning method of thinning out all the pixels on the same line, the sampling address control unit 204 of this embodiment generates an address from the zoom coefficient read from the zoom coefficient memory unit 202 and enlarges / reduces the address. This is given to the processing unit 203.

FIG. 7 is an embodiment showing the manner of generating an enlarged image according to the present invention. FIG. 7 (c) shows the decoded field data. The pixel in which data no longer exists, and this portion is an interpolated pixel by a process such as interpolation from peripheral pixels in the same field. FIG. 7D
FIG. 7C is an enlarged image of the decoded field in FIG. 7C obtained by interpolating pixels at pixel positions thinned out during the encoding pre-processing by the zoom coefficient. Pixels 10, 2
1, 12, 23, 14, 25, 16, 27, 18, 29
Is a pixel interpolated by peripheral pixels, for example, upper and lower pixels (marked by ○). Since the pixels to be referred to also exist on the same line, it is possible to suppress image quality deterioration due to interpolation.

There is also a method of suppressing deterioration by using pixels of another field constituting the same frame as reference pixels. This can be realized by adopting a configuration having a field memory capable of storing two fields in the enlargement / reduction processing unit 203. FIG. 9 shows how pixels between fields are used as reference pixels for interpolation. FIG. 9A shows a frame, where a0, b0, c0, d0, and e0 indicate line data of an odd field, and a1, b1, c1, and d1 indicate line data of an even field. Further, it is assumed that the hatched portion and the white portion indicate the boundary of the edge. At this time, consider a case where the b0 line of the odd field is thinned out by the thinning processing to generate a reduced image.

FIG. 9B shows an encoded image obtained by thinning out the b0 line. FIG. 9C obtained by interpolating the thinned b0 line from the peripheral pixels of the same field in FIG.
The enlarged image of (b) is shown. The b0 'line is the interpolated line. When configured as a frame like this,
A new edge such as the b0 ′ line is generated between the a1 line and the b1 line.

FIG. 9D shows an enlarged image of FIG. 9B obtained by interpolating the decimated b0 line from the peripheral pixels of the same field and the peripheral pixels of another field constituting the same frame. The b0 'line is the interpolated line.
When configured as a frame in this way, a new edge such as the b0 'line is generated between the a1 line and the b1 line, but deterioration is suppressed as compared with the case where only pixels in the same field are referred to. it can.

As described above, the enlargement / reduction processing unit 203 generates an enlarged image field # 1 ″ ′ of the decoding field # 1 ″. The switch unit 205 outputs the field # 1 ″ ′ generated by the enlarged / reduced image generation unit 203 as the output image 206. Field # 2 ''
Similarly, the enlarged image field # 2 "" and the field # 3 "" for the reference field # 0 are generated and output from the output image 206 as an output image.

Next, the case of reduced zoom will be described with reference to FIG. Basically, it is the same as in the case of the above-described enlarged zoom. FIG. 5 shows a zoom coefficient in the case of a reduced zoom,
FIG. 4 shows a relationship diagram of motion vectors. FIG. 5A is a diagram showing a motion vector of a target field (in this case, field # 1) with respect to a reference field (in this case, field # 0). In the figure, H indicates the horizontal direction, V indicates the vertical direction, the upper left is the reference point, and ○ indicates the center of the zoom. Since the figure shows a reduced zoom, the motion vectors extend radially outward from the center of the zoom. FIG.
FIG. 5B shows a state in which the horizontal motion vector in FIG. 5A is approximated by a linear function. The motion vector Vh (x) at a position that is x pixels away from the reference point in the horizontal direction can be expressed as Vh (x) = ah * x + bh using the horizontal zoom coefficient (ah, bh).

Similarly, FIG. 5C shows a state in which the vertical motion vector in FIG. 5A is approximated by a linear function. The motion vector Vv (y) at a position horizontally y pixels away from the reference point is a horizontal zoom coefficient (av, bv)
And Vv (y) = av * y + bv. Such a reference field # 0
, An enlarged image is generated from the motion vector of field # 1 using the horizontal zoom coefficients (ah01, bh01) and the vertical zoom coefficients (av01, bv01).

FIG. 8 shows how an enlarged image is generated. FIG. 8A shows input field data. FIG.
(B) is an image after enlargement. In the same figure, the hatched portion is a pixel generated by the interpolation process, and this portion is replaced by a pixel at the same position in the reference field, or is interpolated as a reference pixel by using a peripheral pixel of the same field as a reference pixel. It is necessary to perform processing for suppressing the deterioration of the coding efficiency. At the time of decoding, a process of thinning out the pixels inserted by interpolation at the time of pre-encoding processing is performed on the decoded data.

FIG. 8C shows the decoded field data. The hatched portion is a pixel inserted by the zoom coefficient at the time of the pre-coding process, and this portion is replaced by a pixel at the same position in the reference field. , Or a pixel interpolated by a process such as interpolation from peripheral pixels in the same field. FIG. 8D is a reduced image of the decoded field in FIG. 8C obtained by thinning out the pixels inserted at the time of the pre-coding process by the zoom coefficient. The hatched portions are pixels for which no data exists due to the thinning process. With respect to this pixel, peripheral pixels of the same field are set as reference pixels, and processing such as interpolation is performed.
It is necessary to perform processing for suppressing image quality deterioration.

[0042]

As described above, according to the present invention, according to the present invention, (1) In an encoding method utilizing correlation between frames or between fields of an image, enlargement by zooming in the image sequence is performed in the image sequence. When there is a reduced image, especially when the input field for the reference field is an enlarged image, the input field is reduced by thinning out pixels that are not adjacent in both the horizontal and vertical directions using the motion vector of the input field for the reference field. Since the reduced image is generated, the encoding efficiency can be further improved without completely losing the horizontal and vertical edges.

(2) In the post-decoding process, when the input fields belonging to the same frame group are an enlarged image with respect to the reference field, the decoded field obtained by the decoding process is processed by the pre-encoding process. Since the thinned-out pixels are interpolated using pixels having higher correlation as reference pixels, it is possible to suppress image quality deterioration due to the interpolation.

[Brief description of the drawings]

FIG. 1 is a block diagram of an encoding device according to an embodiment of the present invention.

FIG. 2 is a block diagram of a decoding device according to an embodiment of the present invention.

FIG. 3 is a diagram showing a state of field data belonging to the same frame group;

FIG. 4 is a diagram showing a relationship between a zoom coefficient and a motion vector.

FIG. 5 is a diagram showing a relationship between a zoom coefficient and a motion vector.

FIG. 6 is a diagram showing a state of field data by a pre-encoding process and a post-decoding process at the time of enlargement zoom;

FIG. 7 is a diagram showing a state of field data by a pre-encoding process and a post-decoding process at the time of enlargement zoom;

FIG. 8 is a diagram illustrating a state of field data by a pre-encoding process and a post-decoding process at the time of reduction zoom.

FIG. 9 is a diagram showing a state of frame data by a pre-encoding process and a post-decoding process at the time of enlargement zoom;

FIG. 10 is a block diagram of a conventional encoding device.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 image input unit 101 motion vector detection unit 102 zoom coefficient calculation unit 103 field memory 104 enlargement / reduction processing unit 105 sampling address control unit 106 switch unit 107 encoding unit

────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Yuji Fujiwara 1006 Kazuma Kadoma, Kadoma City, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. (56) References JP-A-3-166885 (JP, A) (58) Field (Int.Cl. ⁶ , DB name) H04N ^7/ 24-7/68

Claims (3)

(57) [Claims] 1. A pre-encoding process in an encoding method using correlation between frames or between fields of an image, wherein when an image sequence includes an image enlarged / reduced by zooming, the number of images is n. (N: natural number) is formed, and for each input field belonging to the frame group, an m-th field (m: 0 ≦
m ≦ 2n-1, integer), a motion vector of each input field with respect to the reference field is detected, and an enlarged / reduced image of each input field is generated using the motion vector. Is a field to be coded (input image in the encoding process), and when the input field for the reference field is an enlarged image, the motion vector is used and neither the horizontal direction nor the vertical direction is adjacent. An image encoding method, wherein pixels are thinned to generate a reduced image of an input field, and the reduced image is used as a field to be coded. 2. A post-decoding process for the pre-encoding process, wherein, when an input field belonging to the same frame group is an enlarged image with respect to a reference field, an already decoded field obtained by the decoding process is And interpolating pixels decimated by the pre-encoding process using peripheral pixels of the same field as reference pixels.
Image coding method as described in the above. 3. A post-decoding process for the pre-encoding process, wherein, if an input field belonging to the same frame group is an enlarged image with respect to a reference field, the decoded field obtained by the decoding process is 2. The image encoding apparatus according to claim 1, wherein the pixels decimated by the pre-encoding process are interpolated using peripheral pixels of the same field and peripheral pixels of another field forming the same frame as reference pixels. Method.