KR20110043407A - Video coding and decoding method and apparatus - Google Patents

Abstract

PURPOSE: A video encoding/decoding apparatus and method thereof which uses a prediction unit are provided to improve the compression efficiency and reconstitution efficiency. CONSTITUTION: An image coding unit(140) generates a prediction sub block by predicting the sub block. The image coding unit substrates the sub block and the prediction sub block and generates a residual sub block. The image coding unit generates a conversion factor which quantizes a residual sub block according to the size of the sub block. The image coding unit generates image data which encodes the quantized conversion factor. An image decoding unit decodes the image data. An image decoding unit adds the prediction sub block and the residual sub block.

Description

Video Coding and Decoding Method and Apparatus

The present invention relates to an image encoding / decoding apparatus and method. More specifically, by dividing the expanded macroblock into subblocks of various sizes and shapes, and encoding the image using different prediction units and transformation units according to the size and shape of the divided subblocks to improve image compression efficiency. A method and apparatus are disclosed. The present invention also relates to an apparatus and method for improving the reconstruction efficiency of an image by receiving a bitstream from such an image encoding apparatus and correspondingly decoding the image.

Standardized techniques for compressing video data currently include H.261, H.263, H.264, MPEG-2, and MPEG-4. According to the existing video compression technology, each image is encoded by dividing each image into fixed-size macroblocks consisting of a rectangular region of pixels of size 16x16 of luminance and pixels of size 8x8 of chrominance. Luma and chrominance components of each macroblock are predicted spatially and temporally, and the predicted residuals are transform and quantization, entropy coding, etc. Compressed through the process of.

The encoding apparatus according to the H.264 / AVC compression standard can intra-prediction encode by dividing each macroblock into smaller blocks of 16x16 size, 8x8 size, and 4x4 size, and 4 kinds of 16x16 pixel blocks. Intra prediction encoding is performed using one of prediction modes, and one of nine prediction modes for 8x8 pixel blocks and 4x4 pixel blocks. In the case of inter prediction encoding, each macroblock may be divided into 16x16 size, 16x8 size, 8x16 size, 8x8 size, 8x4 size, 4x8 size, and 4x4 pixel blocks to be inter prediction encoded. Transform is applied in units of 8x8 size or 4x4 pixel block, and scalar quantization is used as quantization of transform coefficients.

However, 16x16 macroblocks are generally suitable for small QCIF and CIF sized images, and are not suitable for macroblocks for high resolution image compression such as 4Kx2K video. In addition, the conventional video compression technique encodes an image using a fixed size macroblock (although H.264 / AVC divides and encodes a macroblock into smaller block units, the macroblock size is fixed). In the case of encoding a high resolution image, there is a problem that efficient encoding is difficult.

In addition, since the decoding of the image is performed corresponding to the encoding of the image, it is difficult to expect high-efficiency decoding in a state where the compression efficiency for encoding of the high resolution image is deteriorated.

In order to solve the above-described problem, the present invention extends a macroblock to various sizes, divides the expanded macroblock into subblocks of various sizes and shapes, and different prediction units and transform units according to the size of the divided subblocks. The main purpose is to improve the compression efficiency and the reconstruction efficiency by encoding and decoding an image using the image.

In order to achieve the above object, according to the present invention, in the image encoding / decoding apparatus, a prediction subblock is generated by predicting a subblock in which a macroblock is divided, and a residual subblock is generated by subtracting the subblock and the prediction subblock. A video encoder for transforming and quantizing the remaining subblocks using a transform selected according to the size of the subblock to generate quantized transform coefficients, and encoding the quantized transform coefficients to generate encoded image data; And decoding the encoded image data to restore partition type information, motion information, transform type information, and quantized transform coefficients, and inversely quantize and inverse transform the quantized transform coefficients using transforms identified by the restored transform type information. Reconstructing the remaining subblocks, predicting the subblocks identified according to the reconstructed partition type information by using the reconstructed motion information, generating a prediction subblock, and adding the prediction subblocks and the remaining subblocks to reconstruct the subblocks. An image encoding / decoding apparatus comprising an image decoder is provided.

An image encoding apparatus for achieving the above object includes a predictor for predicting a subblock in which a macroblock is divided to generate a predicted subblock; A subtractor for subtracting the subblock and the predictive subblock to generate a residual subblock; A transform and quantizer for transforming and quantizing the remaining subblocks using a transform selected according to the size of the subblock to produce quantized transform coefficients; And an encoder for encoding the quantized transform coefficients to generate encoded image data.

)의

화소 블록이 4개의 블록으로 분할되는 경우에 매크로블록 레이어 K+1의 서브블록들을 사용할 수 있다.Here, the predictor is a macroblock layer K (wherein, N is a macroblock having a size of NxN (where N is an integer of 16 or more)).

)of

When the pixel block is divided into four blocks, subblocks of the macroblock layer K + 1 may be used.

In addition, the predictor may predict a subblock by dividing the macroblock having the size of NxN (where N is an integer greater than or equal to 16) pixel blocks in units of 16x16 pixel blocks when N is larger than 16.

Also, when the macroblock is an intra macroblock, the predictor predicts the subblock in units of mxn pixels, wherein m and n are one of 4, 8, and 16, which are not equal to each other, or larger than 16, and N (N is N × N pixels). It is preferably an integer less than or equal to the pixel unit of the macroblock having the size of the block.

The macroblock may include at least one of 64x64, 32x64, 64x32, 32x32, 16x32, and 32x16 pixel blocks.

In addition, the transform and quantizer uses a 4x4 transform if the size of the subblock is one of 4x4, 4x8, and 8x4 when the macroblock is an intra macroblock, and an 8x8 transform if the size of the subblock is one of 8x8, 8x16, and 16x8. If the size of the subblock is 16x16 or more, 16x16 transform may be used.

Also, when the extended macroblock is an intra macroblock, the transform and quantizer may generate transform type data by encoding a transform type indicating a type of a transform to be selected.

In addition, the transform and quantizer divides the remaining subblocks into 16x16 pixel residual blocks, divides the divided 16x16 pixel residual blocks into 16 4x4 pixel residual blocks, and converts each 4x4 pixel residual block by using 4x4 transform. 16 transform blocks may be generated, and a DC component block composed of DC components of the 16 transform blocks may be transformed using a 4 × 4 transform.

In addition, the 4x4 transform for transforming the 4x4 pixel residual block and the 4x4 transform for transforming the DC component block may use different transform schemes.

Also, when the macroblock is an inter macroblock, the transform and quantizer may select one of 4x4 transform, 8x8 transform, 8x16 transform, 16x8 transform, and 16x16 transform to transform the remaining subblocks.

Also, the transform and quantizer converts the remaining subblocks using 4x4 transform when the subblock size is smaller than 8x8, and encodes the residual subblocks during 4x4 transform and 8x8 transform when the subblock size is 8x8. The smallest transform is used, and when the size of the subblock is 8x16, the remaining subblocks are transformed using the transform having the lowest encoding cost among the 4x4 transform, 8x8 transform, and 8x16 transform, and the subblock size is 16x8. In case of, the residual block of the subblock is transformed using the transform having the lowest encoding cost among 4x4 transform, 8x8 transform, and 16x8 transform.If the size of the subblock is 16x16 or more, the residual block of the subblock is 4x4 transform, 8x8 transform. And a transform having the lowest encoding cost among the 16x16 transforms.

Also, the transform and quantizer converts the remaining subblocks using 4x4 transform when the subblock size is smaller than 8x8, and encodes the residual subblocks during 4x4 transform and 8x8 transform when the subblock size is 8x8. The smallest transform is used, and when the size of the subblock is 8x16, the residual subblock is transformed using the transform having the lowest encoding cost among the 4x4 transform, the 8x8 transform, and the 8x16 transform, and the subblock has a size of 16x8. In case of, the residual block of the subblock is transformed using the transform having the lowest encoding cost among 4x4 transform, 8x8 transform, and 16x8 transform.If the size of the subblock is 16x16 or more, the residual block of the subblock is 4x4 transform, 8x8 transform. And a transform having the lowest encoding cost among the 16x16 transforms.

Also, when the extended macroblock is an inter macroblock, the transform and quantizer may generate transform type data by encoding a transform type indicating the type of transform to be selected.

An image decoding apparatus according to an embodiment of the present invention for achieving the above object comprises: a decoder for decoding coded data to restore partition type information, motion information, transform type information, and quantized transform coefficients; An inverse quantizer and an inverse transformer for restoring the remaining subblocks by inverse quantization and inverse transformation of the quantized transform coefficients using a transform identified by the transformed type information to be recovered; A predictor for predicting a subblock identified according to the reconstructed partition type information by using the reconstructed motion information to generate a predicted subblock; And an adder for reconstructing the subblock by adding the prediction subblock and the residual subblock.

Here, the inverse quantizer and the inverse transformer use a 4x4 transform if the size of the subblock is one of 4x4, 4x8, and 8x4, and the 8x8 transform if the size of the subblock is one of 8x8, 8x16, and 16x8, and the subblock If the size of is 16x16 or more, 16x16 conversion can be used.

An image decoding apparatus according to an embodiment of the present invention for achieving the above object comprises: a decoder for decoding coded data to restore partition type information, motion information, transform type information, and quantized transform coefficients; An inverse quantizer and an inverse transformer that inverse quantizes and inverse transforms quantized transform coefficients using a transform identified by the transformed type information to be recovered to recover a residual subblock; A predictor for predicting a subblock identified according to the reconstructed partition type information using the reconstructed motion information to generate a predicted subblock; And an adder for reconstructing the subblock by adding the prediction subblock and the residual subblock.

Here, the inverse quantizer and the inverse transformer may use one of a 4x4 transform, an 8x8 transform, an 8x16 transform, a 16x8 transform, and a 16x16 transform.

Preferably, the image decoding apparatus may further include a filter for performing deblocking filtering on a boundary between a left side and an upper side of the block according to the selected transform in the reconstructed subblock.

Here, the decoder may restore the SKIP_N flag indicating whether the macroblock is a SKIP macroblock by decoding the SKIP_N flag data extracted from the encoded data.

In addition, the decoder may decode the macroblock type flag data extracted from the encoded data to restore a macroblock type flag indicating whether the macroblock is an intra macroblock or an inter macroblock.

Here, the image decoding apparatus may be identified by the division type number for each division layer.

The apparatus for decoding an image decodes the CBP data extracted from the encoded data and restores the CBPX flag and the CBP according to the size of the subblock, wherein the CBPX flag indicates whether there is a non-zero quantized transform coefficient in the XxX pixel block. Flag and CBP is a flag indicating whether there is a non-zero quantized transform coefficient in a 16x16 pixel block.

A video encoding / decoding method for achieving the above object includes generating a prediction subblock by predicting a subblock in which a macroblock is divided, generating a residual subblock by subtracting the subblock and the prediction subblock, and remaining residual subblock. An image encoding step of transforming and quantizing a block using a transform selected according to the size of the subblock to generate a quantized transform coefficient, and generating encoded image data by encoding a quantized transform coefficient; And decoding the encoded image data to restore partition type information, motion information, transform type information, and quantized transform coefficients, and inversely quantize and inverse transform the quantized transform coefficients using transforms identified by the restored transform type information. Reconstruct the remaining subblocks, predict the subblocks identified according to the reconstructed partition type information using the reconstructed motion information, and generate a prediction subblock, and add the prediction subblocks and the residual subblocks to reconstruct the subblocks. And a video decoding step.

In accordance with another aspect of the present invention, there is provided a method of encoding an image, the method comprising: generating a prediction subblock by predicting a subblock divided from a macroblock; Generating a residual subblock by subtracting the subblock and the prediction subblock; Transforming and quantizing the remaining subblocks using a transform selected according to the size of the subblock to generate quantized transform coefficients; And generating encoded image data by encoding the quantized transform coefficients.

)의

화소 블록이 4개의 블록으로 분할되는 경우에 매크로블록 레이어 K+1의 서브블록들을 이용할 수 있다.Here, the prediction subblock generation step may include a macroblock layer K (where:

)of

When the pixel block is divided into four blocks, subblocks of the macroblock layer K + 1 may be used.

The predicting subblock generation step may predict a subblock by dividing the macroblock having the size of an NxN pixel block (where N is an integer greater than or equal to 16) into units of 16x16 pixel blocks when N is greater than 16.

In addition, when the macroblock is an intra macroblock, the predictive subblock generation step predicts a subblock in units of mxn pixels, wherein m and n are one of 4, 8, and 16, which are not equal to each other, or larger than 16, and N (N is a pixel unit of a macroblock having the size of an N × N pixel block).

In the prediction subblock generation step, the subblock may be predicted according to a prediction direction according to a prediction mode having the best encoding efficiency among three intra prediction modes.

In addition, when the macroblock is an intra macroblock, the quantized transform coefficient generation step uses 4x4 transform if the subblock size is one of 4x4, 4x8, and 8x4, and the subblock has one of 8x8, 8x16, and 16x8. In this case, 8x8 transform is used, and if the size of the subblock is 16x16 or more, 16x16 transform can be used.

In addition, the quantized transform coefficient generation step divides the remaining subblocks into 16x16 pixel residual blocks, divides the divided 16x16 pixel residual blocks into 16 4x4 pixel residual blocks, and uses each of the 4x4 pixel residual blocks using 4x4 transform. 16 transform blocks are generated by transforming the DC component block, and the DC component block composed of the DC components of the 16 transform blocks can be transformed using 4 × 4 transform.

In addition, the 4x4 transform for transforming the 4x4 pixel residual block and the 4x4 transform for transforming the DC component block may use different transform schemes.

In the quantized transform coefficient generation step, when the macroblock is an inter macroblock, the residual subblock may be transformed by selecting one of 4x4 transform, 8x8 transform, 8x16 transform, 16x8 transform, and 16x16 transform.

In the quantized transform coefficient generation step, when the size of the subblock is smaller than 8x8, the residual subblock is transformed using 4x4 transform. When the size of the subblock is 8x8, the residual subblock is 4x4 transformed and 8x8 transformed. The transform having the lowest encoding cost is transformed.If the size of the subblock is 8x16, the remaining subblocks are transformed using the transform having the lowest encoding cost among the 4x4, 8x8, and 8x16 transforms. If the size is 16x8, the residual block of the subblock is transformed using the 4x4 transform, the 8x8 transform, and the transform having the smallest encoding cost among the 16x8 transform.If the size of the subblock is 16x16 or more, the residual block of the subblock is 4x4 transform. Among the 8x8 and 16x16 transforms, the transform having the lowest encoding cost may be used.

Preferably, the video encoding method may further include generating transform type data by encoding a transform type indicating a type of transform to be selected when the macroblock is an inter macroblock.

In addition, the image encoding method may include recovering a residual subblock by inverse quantization and inverse transformation on the quantized transform coefficients; Reconstructing the subblock by adding the reconstructed residual subblock and the prediction subblock; And filtering the reconstructed subblock, wherein the subblock filtering may perform deblocking filtering on the left and upper boundaries of the block according to the selected transform in the reconstructed subblock.

The image encoding method may further include generating split type data by encoding split type information of a macroblock.

Here, the split type information is identified by the split type number for each split layer, and the step of generating split type data encodes the split type information by sequentially encoding the split type number for each split layer according to the split type number decoding order. It is desirable to.

The above-described image encoding method may further include generating CBP data by encoding the CBPX flag and the CBP according to the size of the subblock, wherein the CBPX flag has a non-zero quantized transform coefficient in the XxX pixel block. And CBP is preferably a flag indicating whether there is a non-zero quantized transform coefficient in a 16x16 pixel block.

The image encoding method may further include generating SKIP_N flag data by encoding a SKIP_N flag indicating whether the macroblock is a SKIP macroblock.

The video encoding method may further include generating macroblock type flag data by encoding a macroblock type flag indicating whether the macroblock is an intra macroblock or an inter macroblock.

The video encoding method may further include generating encoded data including the encoded image data. When the picture including the macroblock is an intra picture, the encoded data may include a partition type field, an intra prediction mode field, and a CBPX. It may include one or more of a flag / CBP field and a conversion coefficient field.

The video encoding method may further include generating encoded data including the encoded video data. When the picture including the macroblock is an inter picture, the encoded data may include a SKIP_N flag field, a macroblock type field, and a division. And at least one of an intra prediction mode field, a CBPX flag / CBP field, and a transform coefficient field if the macroblock is an intra macroblock, and if the macroblock is an inter macroblock. Preferably, the apparatus further includes one or more of a SKIPX flag field, a differential motion vector / reference picture index field, a CBPX flag field, a transform type field, a CBP field, and a transform coefficient field.

An image decoding method according to an embodiment of the present invention for achieving the above object comprises the steps of: decoding coded data to reconstruct partition type information, intra prediction mode and quantized transform coefficients; Inverse quantizing and inverse transforming the quantized transform coefficients using a transform selected according to the size of the subblock identified according to the reconstructed partition type information to recover the remaining subblocks; Generating a predicted subblock by predicting a subblock identified according to the reconstructed partition type information using the reconstructed intra prediction mode; And reconstructing the subblock by adding the prediction subblock and the residual subblock.

Here, the transform selected according to the size of the subblock uses a 4x4 transform if the size of the subblock is one of 4x4, 4x8 and 8x4, and uses an 8x8 transform if the size of the subblock is one of 8x8, 8x16 and 16x8. If the size of the subblock is 16x16 or more, 16x16 transform may be used.

According to another aspect of the present invention, there is provided a method of decoding an image, the method including: restoring partition type information, motion information, transform type information, and quantized transform coefficients by decoding encoded data; Inverse quantizing and inverse transforming the quantized transform coefficients using the transform identified by the transformed type information to be recovered to recover the remaining subblocks; Generating a prediction subblock by dividing the subblock into identified subblocks according to the reconstructed partition type information and predicting the subblock using the reconstructed motion information; And reconstructing the subblock by adding the prediction subblock and the residual subblock.

Here, the transform identified by the transform type information may be one of a 4x4 transform, an 8x8 transform, an 8x16 transform, a 16x8 transform, and a 16x16 transform.

The above-described image decoding method may further include performing deblocking filtering on a boundary between a left side and an upper side of the block according to the selected transform in the reconstructed subblock.

The video decoding method may further include decoding the SKIP_N flag data extracted from the encoded data to restore the SKIP_N flag indicating whether the macroblock is a SKIP macroblock.

The image decoding method may further include reconstructing a macroblock type flag indicating whether the macroblock is an intra macroblock or an inter macroblock by decoding the macroblock type flag data extracted from the encoded data.

Here, the division type information is preferably identified by the division type number for each division layer.

The above-described image decoding method further includes reconstructing the CBPX flag and the CBP according to the size of the subblock by decoding the CBP data extracted from the encoded data, wherein the CBPX flag is a non-zero quantized transform coefficient in the XxX pixel block. Is a flag indicating whether or not there is a CBP is a flag indicating whether there is a non-zero quantized transform coefficient in the 16x16 pixel block.

As described above, according to the present invention, compression efficiency is achieved by dividing an extended macroblock into subblocks of various sizes and shapes, and encoding and decoding an image using different prediction units and transformation units according to the sizes of the divided subblocks. And the recovery efficiency can be improved.

1 is a block diagram schematically illustrating a video encoding apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating encoding and decoding when an input image is padded according to an embodiment of the present invention. FIG. 2 illustrates an input image and positions of an extended macroblock to be currently encoded.
FIG. 3 is an enlarged view of an extended macroblock that is currently encoded in FIG. 2.
4 is a diagram illustrating an example of a process of dividing a macroblock into subblocks of various sizes for intra prediction encoding and inter prediction encoding according to an embodiment of the present invention.
5 is a diagram illustrating an example of a process of dividing a macroblock into subblocks of various sizes for intra prediction encoding and inter prediction encoding according to another embodiment of the present invention.
6 is an exemplary diagram for describing a method of encoding an intra picture and related data to be transmitted through encoded data, according to an embodiment of the present invention.
7 is an exemplary diagram illustrating nine prediction directions and prediction mode numbers according to intra prediction modes of luminance components according to an embodiment of the present invention.
8 is an exemplary diagram illustrating four prediction directions and prediction mode numbers according to intra prediction modes of luminance components according to an embodiment of the present invention.
9 is an exemplary diagram illustrating three prediction directions and prediction mode numbers according to intra prediction modes of luminance components according to an embodiment of the present invention.
10 is an exemplary diagram illustrating prediction directions and prediction mode numbers according to four intra prediction modes of a color difference component according to an embodiment of the present invention.
11 is an exemplary diagram illustrating various subblock types for an extended macroblock of 64x64 pixels according to an embodiment of the present invention.
12 is an exemplary diagram for describing an encoding method of an inter picture and related data to be transmitted through encoded data according to an embodiment of the present invention.
13 is an exemplary view for explaining a process of determining a predicted motion vector according to an embodiment of the present invention.
14 is an exemplary diagram for describing a process of determining a predicted motion vector of an 8x16 pixel block and a 16x8 pixel block according to an embodiment of the present invention.
FIG. 15 is an exemplary diagram illustrating types of usable transforms and units in which transform types are encoded according to subblock sizes according to an embodiment of the present invention.
16 is an exemplary diagram illustrating a block boundary to perform deblocking filtering according to an embodiment of the present invention.
17 is an exemplary diagram illustrating syntax of an intra picture according to an embodiment of the present invention.
18 is an exemplary view showing a number for each division type according to an embodiment of the present invention.
FIG. 19 is a diagram illustrating a split type number coding sequence in the case of dividing an expanded macroblock into subblocks of various sizes according to an embodiment of the present invention.
20 is an exemplary diagram sequentially illustrating a process of dividing a macroblock into divided layers according to an embodiment of the present invention.
21 is an exemplary diagram for describing a process of sequentially encoding a split type number for each layer of a macroblock.
22 is an exemplary diagram illustrating a coding order of an intra prediction mode according to an embodiment of the present invention.
FIG. 23 is an exemplary diagram for explaining an example of a process of encoding an intra prediction mode of a macroblock according to an embodiment of the present invention.
24 is an exemplary diagram for describing a process of encoding a CBPX_flag and a CBP for an intra macroblock according to an embodiment of the present invention.
25 is an exemplary view showing the structure of a CBP according to an embodiment of the present invention.
FIG. 26 is an exemplary diagram illustrating an example of a sequence of encoding a CBPX flag and a CBP according to an embodiment of the present invention. FIG.
27 is an exemplary diagram illustrating another example of a sequence of encoding a CBPX flag and a CBP for an intra macroblock according to an embodiment of the present invention.
28 is an exemplary diagram illustrating a coding order of quantized transform coefficients for each partition type according to an embodiment of the present invention.
29 is an exemplary diagram illustrating a syntax structure of encoded data according to an embodiment of the present invention.
30 is a diagram illustrating an example of syntax used in a P-picture (or P-slice) according to an embodiment of the present invention.
FIG. 31 is a diagram illustrating an example of syntax used in a B-picture (or B-slice) according to an embodiment of the present invention.
32 is an exemplary diagram for describing an encoding process of a SKIPX flag according to an embodiment of the present invention.
33 and 34 are exemplary diagrams for describing a CBPX_flag, a transform type, and a CBP encoding process for an inter macroblock according to an embodiment of the present invention.
35 is a diagram illustrating an example of ctx_inc for encoding / decoding skipX_flag according to an embodiment of the present invention.
36 illustrates an example of ctx_inc for encoding / decoding interX_flag according to an embodiment of the present invention.
37 illustrates an example of ctx_inc for encoding / decoding directX_flag according to an embodiment of the present invention.
38 and 39 are diagrams for describing a process of encoding split type information using a tree structure according to an embodiment of the present invention.
40 is a diagram illustrating an example of encoding of a partition type according to an embodiment of the present invention.
41 is a diagram illustrating another example of split type encoding according to an embodiment of the present invention.
42 illustrates an example of ctx_inc for encoding / decoding B_pdir according to an embodiment of the present invention.
43 is a diagram illustrating an example of ctx_inc for encoding / decoding a transform type when two types of transforms are available according to an embodiment of the present invention.
44 is a diagram illustrating an example of ctx_inc for encoding / decoding a transform type when there are three types of transforms available according to an embodiment of the present invention.
45 is a diagram illustrating another example of ctx_inc for encoding / decoding a transform type when there are three types of transforms available according to an embodiment of the present invention.
46 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected or connected to that other component, but between components It will be understood that may be "connected", "coupled" or "connected".

According to an embodiment of the present invention, a video encoding apparatus (Video Encoding Apparatus) and a video decoding apparatus (Video Decoding Apparatus) are a personal computer (PC), a notebook computer, a personal digital assistant (PDA). , A portable multimedia player (PMP: Portable Multimedia Player), a PlayStation Portable (PSP: PlayStation Portable), a mobile communication terminal (Mobile Communication Terminal), a smart phone (Smart Phone), etc., and communicates with various devices or wired and wireless communication networks. The present invention refers to various devices including a communication device such as a communication modem, a memory for storing various programs and data for encoding or decoding an image, a microprocessor for executing and operating a program, and the like.

Also, an image encoded in a bitstream by an image encoding apparatus may be connected to a wired / wireless communication network such as an internet, a local area wireless communication network, a wireless LAN network, a WiBro network, a mobile communication network, a wired telephone network, or the like using a cable or a universal serial bus. The device may be transmitted to an image decoding apparatus through a communication interface such as a universal serial bus (USB), and decoded by the image decoding apparatus to be restored and reproduced as an image.

1 is a block diagram schematically illustrating a video encoding apparatus according to an embodiment of the present invention.

The image encoding apparatus 100 according to an embodiment of the present invention includes a predictor 110, a subtractor 120, a transformer and a quantizer 130, an encoder 140, an inverse quantizer and an inverse transformer 150, and an adder ( 160, a filter 170, and a picture buffer 180.

The predictor 110 may include an intra predictor 112 and an inter predictor 114, and the inter predictor 114 may further include a motion estimator 116 and a motion compensator 118.

The input image may be input to the image encoding apparatus 100 of FIG. 1 on a frame basis or a field basis, or may be divided into macroblocks having NxN pixels (where N is an integer of 16 or more) and input to the image encoding apparatus 100. Can be. In other words, when the input image is a high resolution image such as a 4Kx2K image, encoding may be performed by dividing the macroblock into N macroblocks having integers larger than 16 to improve image compression efficiency. Here, a macroblock consisting of integers with N greater than 16 is called an extended macroback (EMB). For example, the extended macroblock may be formed of a square pixel block having a size of 64x64, 32x32, or the like. Note that the macroblock described below is a concept including an extended macroblock and a macroblock of a general 16 × 16 pixel block.

On the other hand, when image compression is performed using an extended macroblock having an integer greater than 16, N is padded to be a multiple of 16 pixels for the input image when the input image is not a multiple of 16 pixels. Can be performed.

FIG. 2 is a diagram illustrating image encoding and decoding when an input image is padded according to an embodiment of the present invention. FIG. 3 is an enlarged view of an extended macroblock that is an object to be encoded in FIG. . If the location of the input macro padded with multiples of 16 pixels and the extended macroblock to be encoded currently is as shown in FIG. 2, the encoding of the current extended macroblock is a pixel belonging to the padded input image to be a multiple of 16 pixels. It is preferable to use only values. That is, in FIG. 3, the padded region in the current extended macroblock is preferably not used for encoding. It is also preferable to use only pixel values belonging to the input image padded so that the data encoded in the bitstream is a multiple of 16 pixels. That is, in FIG. 3, the padded region in the current extended macroblock is preferably not used for encoding. It is also preferable to use only pixel values belonging to the input image padded so that the data encoded in the bitstream is a multiple of 16 pixels. If the input image is not a multiple of the macroblock size, in the embodiment according to the present invention, the input image is padded to be the nearest multiple of 16 (in this case, an image padded with a multiple of 16 is a macroblock as shown in FIG. 3). The encoding mode may be determined using only regions padded with multiples of 16). Here, even if the image area padded with multiples of 16 in the macroblock is smaller than the size of the macroblock, all available block modes in the macroblock may be applied, but it is preferable to use only padded regions with multiples of 16 when determining the mode. The reason is that when the size of the macroblock is larger than 16, the padded area is larger than the actual image area to prevent erroneous mode determination.

The video encoding and decoding method described below may be applied in the same manner as the encoding and decoding on the corresponding video when the input video covers a part of the macroblock as shown in FIGS. 2 and 3.

When the input image is a 4: 2: 0 format image, the macroblock includes a luminance block having NxN pixels and a color difference block having two pixels. The improvement of the compression efficiency of the high resolution image by such an extended macroblock is achieved by the specific embodiment described below.

The extended macroblock according to an embodiment of the present invention may be internally divided into smaller subblocks as shown in FIG. 4 to perform intra prediction encoding or inter prediction encoding.

)의 서브블록 중 제일 큰 서브블록이 4 개의 블록으로 분할된 경우에만 매크로블록 레이어 K+1의 서브블록들을 사용할 수 있다. 즉, 도 5의 경우와 같이, 매크로블록 레이어 0의 64x64 서브블록이 4개의 서브블록으로 분할된 경우에만 매크로블록 레이어 1의 서브블록들을 사용할 수 있으며, 매크로블록 레이어 1의 32x32 서브블록이 4개의 16x16 서브블록으로 분할된 경우에만 매크로블록 레이어 2의 서브블록들을 사용할 수 있다.That is, in the case of using a macroblock having N equal to or greater than 16 for encoding a high resolution image, the expanded macroblock may be divided into smaller pixel block units to perform encoding, and each pixel block is divided into smaller subblocks. Encoding can be performed. In addition, when the length of one side of the extended macroblock or the divided pixel block is larger than 16 pixels, division into a pixel block having a rectangular shape such as 32x64, 64x32, 16x32, or 32x16 is omitted as shown in FIG. After dividing the expanded macroblock into square pixel blocks, encoding may be performed in units of 16 × 16 pixel blocks. 4 and 5 exemplarily show each subblock having a 4 × 4 minimum block size for the extended macroblock. For example, when the extended macroblock is a 64x64 pixel block, the macroblock layer 0 of FIG. 4 belongs to a subblock of 64x64 pixel blocks, 64x32 pixel blocks, 32x64 pixel blocks, and 32x32 pixel blocks. 1 may belong to a 32x32 pixel block, a 32x16 pixel block, a 16x32 pixel block, and a 16x16 pixel block, which are subblocks. In addition, as shown in FIG. 5, the division of rectangular subblocks such as 64x32 pixel blocks, 32x64 pixel blocks, 32x16 pixel blocks, and 16x32 pixel blocks may be omitted for subblocks larger than 16x16 pixel blocks. In this case, 64x64 pixel blocks and 32x32 pixel blocks, which are subblocks, belong to macroblock layer 0, and 32x32 and 16x16 pixel blocks, which are subblocks, belong to macroblock layer 1. Here, macroblock layer K (where

The subblocks of the macroblock layer K + 1 may be used only when the largest subblock of subband is divided into four blocks. That is, as shown in FIG. 5, the subblocks of the macroblock layer 1 may be used only when the 64x64 subblock of the macroblock layer 0 is divided into four subblocks, and the 32x32 subblocks of the macroblock layer 1 have four subblocks. Subblocks of the macroblock layer 2 may be used only when divided into 16 × 16 subblocks.

The image encoding apparatus 100 calculates encoding efficiency for each subblock when the macroblock is divided into subblocks and encoded, and converts the subblock when the highest encoding efficiency is the final intra prediction block or the inter prediction block. Decide The coding efficiency may be measured based on a rate-distortion optimization (RDO) technique as shown in Equation (1).

Here, Distortion represents an error value between a block to be encoded and a reconstruction block, is a constant determined by a quantization parameter, and Bit represents the number of bits required to encode a block to be encoded.

The mode of a block can be divided into the process of determining the optimal prediction mode (intra prediction mode, motion data) of each block for all types of possible blocks, and then determining the block type, and usually in determining the prediction mode (intra prediction mode). In the case of selecting a motion vector / reference picture, SAD (Sum of Absolute Difference) or SATD (Sum of Absolute Transformed Difference) is used. Bits of transform coefficients are not considered in Bit. This can be summarized as in Equation 2.

In Equation 2, SAD refers to the difference between the original image and the predicted image, plus all the absolute values, and SATD adds the absolute value to the residual block after the transformation is performed.

In determining the block type, it is preferable to calculate Rdcost strictly using Equation 3.

Distortion of Equation 3 may be represented by Equation 4.

Equation (2) is less than the equation (3), but since the distortion is almost the same in the case of a motionless or uncomplicated region, it is difficult to determine the optimal mode using the equation (2). On the other hand, Equation 3 can calculate more accurate coding efficiency than Equation 2, but has a disadvantage in that a large amount of calculation is required. Therefore, in the embodiment of the present invention, it is preferable to select and use Equation 2 or Equation 3 according to the importance of data (whether or not used as a reference picture) or the mode to be encoded. For example, Equation 3 may be used when determining an intra prediction mode and when determining a motion vector / reference picture of a block. However, exceptionally, the prediction directions (L0 prediction, L1 prediction, bidirectional prediction) and the motion vector / reference picture determination process of the pixel block of 16x16 or larger size of the B picture are performed according to Equation 3 according to each prediction direction. The prediction direction may be determined using Equation 2. Equation 3 may be used to determine the block type. In addition, when the size of the subblock is larger than or equal to the set pixel block, the encoding cost may be determined according to Equation 3, and when the size of the subblock is smaller than the set pixel block, the encoding cost may be determined according to Equation 2.

For example, assuming that the selection criterion of encoding cost is set to 16x16 pixelblocks, the encoding mode that a 16x16 pixelblock may have is a 16x16 block using a 16x16 SKIP mode, a 16x16 Direct mode, or a L0 prediction mode (for example, There may be motion information and residual data), 16x16 block using L1 prediction mode (e.g. there may be motion information and residual data), 16x16 block using both L0 and L1 prediction mode (e.g. , Motion information and residual data), 16x16 intra blocks, and the like. It is preferable to use a precise encoding cost for encoding modes of pixel blocks of 16 × 16 or more. If the block type has a size of 16x16 or more, it is mainly because of the flat area such as the background, so it is difficult to determine the optimum mode by the approximated equation. Therefore, when using blocks of 16x16 or more, the coding cost is selectively used according to the block size and mode when determining the mode in the macroblock. In addition, the parameters used in the equation for calculating the encoding cost may be considered differently from the parameters used for mode determination between small blocks and the parameters used for mode determination between large blocks.

The minimum block size is determined by the maximum layer value MaxLayer, which is the maximum usable layer value. For example, in the case of a macroblock of N × N pixels, the size of the minimum block is determined as N / (2 ^MaxLayer ).

Each component of the image encoding apparatus 100 will be described together with related drawings in a process to be described later.

When encoding an image, the image encoding apparatus 100 classifies a picture type in picture units constituting a video and predictively encodes the picture type according to the picture type. For example, when any picture is an intra picture, the types of macroblocks in the intra picture are all intra macroblocks, and each macroblock is intra prediction encoded. In addition, when any picture is an inter picture, the macroblock type of the macroblocks in the interpicture may be an inter macroblock or an intra macroblock, and may be intra prediction encoded or inter prediction encoded according to the macroblock type. The information about the picture type may be inserted as it is or encoded and inserted into a picture header or a sequence header of encoded data. The image decoding apparatus to be described later may extract or restore information about a picture type from a picture header or a sequence header and decode the same. The picture type of the picture may be determined.

Hereinafter, an encoding method and encoded data generated through the encoding method according to an embodiment of the present invention will be described according to a picture type.

6 is an exemplary diagram for describing a method of encoding an intra picture and related data to be transmitted through encoded data, according to an embodiment of the present invention.

The intra predictor 112 of the predictor 110 generates a reference block by using neighboring pixels of a block to be currently encoded (hereinafter referred to as a 'current block') and compares the reference block with the current block to determine an intra prediction mode. . Herein, the neighboring pixel refers to pixels in a block around the current block, and includes a neighboring pixel adjacent to the current block in blocks adjacent to the current block.

When intra prediction is performed by dividing the extended macroblock into 4x4 pixel sub-blocks for the luminance component, FIG. 7 illustrates nine prediction directions and prediction mode numbers according to the intra prediction mode of the luminance component. For reference, among the nine prediction directions (the prediction directions according to the prediction modes 0 to 8), one of the most suitable prediction directions is selected for each current block in units of 4x4 pixels, and the current block is intra predicted using the selected prediction direction. Encode The mean value of prediction mode 2 is a method of predicting all 4x4 pixels of the current block by obtaining an average value of a total of eight adjacent pixels of four adjacent pixels of the left block of the current block and four adjacent pixels of the upper block of the current block. .

In the case of a block located at the left boundary of the picture and a block located at the upper boundary of the picture, the left block and the upper block are located outside the picture, respectively. In this case, the use of the prediction direction is limited because the block outside the picture boundary cannot be referenced. For example, the prediction direction of the prediction mode 0, the prediction mode 3, the prediction mode 4, the prediction mode 5, the prediction mode 6, and the prediction mode 7 referring to the pixels of the upper block cannot be used in the block located at the top of the picture. Exceptionally, in the DC mode, prediction of the current block is performed by referring to only pixels that do not deviate from the boundary of the picture. If the left block and the upper block that are adjacent to the current block are not available, the DC value uses 128.

The prediction direction of the selected current block is compared with the prediction direction of the block having the smaller prediction mode number among the left block and the upper block of the current block, and when the two prediction directions are the same, the prediction direction of the current block estimated from the neighboring block and the selected current A prediction mode flag (for example, represented by 1 bit) indicating whether the prediction directions of the blocks are the same is encoded to indicate that the prediction direction is the same as the estimated direction.

 If the prediction direction of the current block estimated from the neighboring block and the prediction direction of the selected current block are different, the prediction mode flag is encoded to indicate that the prediction direction of the selected current block is different from the estimated direction of the estimated current block. Prediction mode information for indicating which prediction mode among the eight prediction modes minus the prediction mode corresponding to the prediction direction of the current block estimated in the nine prediction modes is the prediction direction of the selected current block (for example, Can be represented by 3 bits).

When intra prediction is performed by dividing the extended macroblock into subblocks of 8x8 pixel units for the luminance component, intra prediction of 8x8 pixel units for the luminance component is illustrated in FIG. 7 as in the intra prediction of 4x4 pixels. The same nine prediction directions are used, and the method of calculating the prediction pixels is the same as that of intra prediction coding in units of 4x4 pixels except for differences in block sizes (4x4 pixels and 8x8 pixels).

When intra prediction is performed by dividing an extended macroblock into subblocks of 16x16 pixel units for a luminance component, a prediction direction having the best coding efficiency is selected from four prediction directions as shown in FIG. According to the prediction direction, a 16x16 pixel block is predictively encoded from 16 neighboring pixels in a 16x16 pixel block located above a 16x16 current block and 16 neighboring pixels in a 16x16 pixel block located to the left of the current block. .

In FIG. 8, in the case of plane prediction in the prediction mode 3, interpolation between adjacent pixels in an upper block of the current block and adjacent pixels in a left block of the current block is predicted by interpolation. In the mean value prediction corresponding to the prediction mode 2, the mean value predicts an average value of 16 adjacent pixels in the upper block of the current block and 16 adjacent pixels in the left block as 16 × 16 pixels of the current block. However, when the current block is located at the top of the picture, the average value of 16 adjacent pixels in the left block of the current block is used as a prediction value. When the current block is located at the leftmost point in the picture, the top of the current block is used. The average value of 16 adjacent pixels in the block is used as a prediction value. If the left block and the top block of the current block are not available, 128 is used as the prediction value. The prediction mode of the 16x16 pixel block is not predictively encoded, and the number of the selected prediction mode is simply encoded as prediction mode information (e.g., represented by 2 bits).

When intra prediction is performed by dividing an extended macroblock into sub blocks excluding the 4x4 pixel block, 8x8 pixel block, or 16x16 pixel block, the intra prediction mode of the luminance component has three prediction directions and predictions as shown in FIG. 9. Can be represented by the mode number. In the case of performing intra prediction by dividing the sub block into 4x4 pixel blocks, 8x8 pixel blocks, or 16x16 pixel blocks, the size of the subblock is mxn, and it is selected from three prediction directions as shown in FIG. An mxn pixel block is intra-predictively encoded from m neighboring pixels in the upper block of the current block and n neighboring pixels in the left block according to the prediction direction having the highest coding efficiency.

In the mean value prediction corresponding to prediction mode 2, the current block is predictively encoded using the average value of m neighboring pixels in the upper block of the current block and n neighboring pixels in the left block. However, when the current block is located at the top of the picture, the average value of n neighboring pixels of the left block of the current block is used as the prediction value of the current block, and when the current block is located at the leftmost side of the picture, the upper block An average value of m neighboring pixels in the frame is used as a prediction value of the current block. If the left block and the upper block of the current block are not available, 128 is used as a prediction value of the current block.

When the prediction direction of the selected current block and the prediction direction of the block having the smaller prediction mode number among the left block and the upper block of the current block are the same direction, the prediction direction of the current block estimated from the neighboring blocks of the current block and the selected current Information about a prediction direction that is the same as the prediction direction of the current block in which a prediction mode flag (for example, one bit may be indicated) indicating whether or not the prediction directions of the blocks are the same is encoded. Since the number of prediction modes that an mxn pixel block can have is 0 to 2, when the prediction mode number of the current block predicted using the left block and the upper block of the current block is 3 or more, the prediction direction of the current block is the prediction mode number. 2 (DC mode) can be used. For example, when the prediction mode number of the current block predicted from the neighboring block is 4, a prediction mode flag (eg, 1 bit) indicating whether the prediction mode number of the current block is the same as the prediction direction of the current block selected. Coded).

Here, the intra prediction for the extended macroblock is a pixel block unit of 16x16 or less, that is, a 4x4 pixel block, an 8x8 pixel block, a 16x16 pixel block, or an mxn (where m ≠ n and m and n are less than 16) pixels. It is preferable to perform block by block.

화소 블록 단위로 수행될 수 있다. 예측 방향은 도 10에 나타낸 바와 같이, 평균치 예측, 수평 예측, 수직 예측, 평면 예측 4 가지의 예측 방향이 사용될 수 있다. 바람직하게는, 색차 성분의 인트라 예측은 8x8 화소 블록 단위로 수행된다.Intra prediction of chrominance components

It may be performed in pixel block units. As shown in FIG. 10, four prediction directions may be used: average prediction, horizontal prediction, vertical prediction, and plane prediction. Preferably, intra prediction of the chrominance component is performed in units of 8x8 pixel blocks.

Referring to FIG. 10 exemplarily illustrating a prediction direction and a prediction mode number according to four intra prediction modes of a chrominance component according to an embodiment of the present invention, planar prediction corresponding to prediction mode 3 and prediction mode 0 correspond to each other. The predictive value calculation method for predicting the mean value is also the same as the intra prediction encoding method in units of 16x16 pixels of the luminance component except for the difference in block size (16x16 pixels and 8x8 pixels). The prediction mode of the color difference signal may be selected independently of the prediction mode of the luminance signal. There are two types of color difference signals, U and V, but the prediction method is the same. The prediction mode of the chrominance signal is one each of U and V. The prediction mode of each chrominance signal is represented by using two bits instead of the predictive coding.

Referring back to FIG. 6, the intra predictor 112 predicts the current block in nine prediction directions for 4x4 pixel units in the same manner as described above with reference to FIGS. 7 to 10, such as predictive encoding and rate-distortion optimization. Intra prediction mode, which is determined by the method, predicts the current block with 9 prediction directions for 8x8 pixel units, and intra prediction mode, which is determined in the same way as predictive encoding and rate-distortion optimization, with 4 prediction directions for 16x16 pixel units. Intra prediction mode that is determined by predicting the current block in the same manner as the predictive encoding high-distortion optimization, and predicted by the predictive encoding high rate-distortion optimization by predicting the current block in three prediction directions for mxn pixel units. Intra prediction mode is one intra prediction mode that is determined by using rate-distortion optimization. Determine as prediction mode. The rate-distortion optimization equation used at this time is preferably in accordance with the above equation (3). In this way, if the intra prediction mode is determined, the size of the subblock is also determined accordingly.

According to the intra prediction mode determined as described above, the intra predictor 112 predicts the current block to generate a predicted block, and the subtractor 120 subtracts the current block and the predictive block to residual blocks. Block). The transformer and quantizer 130 transform and quantize the residual block to generate quantized transform coefficients, and the encoder 140 entropy encodes the quantized transform coefficients to generate encoded data.

The converter and quantizer 130 performs 4x4 transform on the remaining blocks of the current block in which intra prediction in units of 4x4, 4x8, and 8x4 pixels in luminance components is selected, and the current block in which intra prediction in units of 8x8, 8x16 and 16x8 pixels is selected. An 8x8 transform may be performed on the residual block of. In addition, the converter and the quantizer 130 may perform 16x16 transform on the residual block of the current block in which intra prediction of 16x16 pixels or more is selected. In this case, since the unit of intra prediction and the size of the subblock are the same, the type of transform may be determined according to the block size of the subblock.

Herein, a residual block having 16x16 pixels to which each 16x16 transform is performed (hereinafter, referred to as a 16x16 pixel residual block) is further divided into a residual block having 16 4x4 pixels (hereinafter referred to as a 4x4 pixel residual block). 4x4 transform may be performed on each 4x4 pixel residual block. Subsequently, only 16 DC components of a transform block (hereinafter, referred to as a 4x4 pixel transform block) having each 4x4 pixel generated by 4x4 pixel residual block 4x4 transform are collected to have a 4x4 transform coefficient (hereinafter, 'DC'). Component block '), and the 4x4 transform is performed once more. In this case, a transform different from the transform when the residual block is transformed may be used to transform the DC component block. That is, a 4x4 Discrete Cosine Transform (DCT) is performed on the 4x4 pixel residual block, and a 4x4 Hadamard transform may be used on the DC component block. For example, a 16x32 pixel residual block for the current block in which intra prediction is selected in units of 16x32 pixels is divided into two 16x16 pixel residual blocks, and 4x4 conversion is performed in 4x4 pixel units for each of the divided 16x16 pixel residual blocks. Thereafter, DC components of the 4x4 transform blocks in the 16x16 pixel residual block are gathered to form a DC component block, and 4x4 transform is performed once more on the DC component block.

If the location of the extended macroblock, which is a 32x32 pixel block, spans the padded image and the padded region in multiples of 16 pixels as shown in FIG. 2, if the extended macroblock is an intra mode, as shown in FIG. Similarly, the encoding mode is determined only for the actual image region (16x16 pixel block), and the division type, the intra prediction mode, the CBP, and the transform coefficient of the corresponding region are encoded.

11 is an exemplary diagram illustrating a block type of a sub block for an extended macroblock of 64x64 pixels according to an embodiment of the present invention.

Assuming that the extended macroblock is a 64x64 pixel block and the optimal block type after intra prediction is determined as shown in FIG. 11, 4x4 transform is performed for blocks 0 through 7 and 8x8 for blocks 8 through 12 The transform is performed, and for blocks 13 to 18, a 4x4 transform is performed in 4x4 pixel units within a 16x16 pixel block, and the transform is again performed on a DC component block including DC components of each of the 4x4 transform blocks within each 16x16 pixel block. Is performed.

In the case of the chrominance signal, 4x4 transform is performed as in the case of intra prediction of 16x16 pixel unit or more, and then 4x4 transform is performed once more for the DC component block. However, the residual block of the macroblock of the chrominance component is divided into a residual block having 8x8 pixels (hereinafter referred to as an '8x8 pixel residual block'), and 4x4 transform is performed on four 4x4 pixel residual blocks in each 8x8 pixel residual block. . DC components of four 4x4 transform blocks in an 8x8 pixel residual block are gathered to form a DC component block having a 2x2 transform coefficient, and 2x2 transform is performed on the DC component block. In this case, the 4x4 transform for the 8x8 pixel residual block and the 2x2 transform for the DC component block may be different.

Alternatively, instead of performing a transform according to the above-described transform type after the optimal block type is determined for the extended macroblock, a transform type having good efficiency according to the size of the sub-block used for intra prediction in units of 16x16 pixel blocks. You can also select and use. That is, the transform and quantizer 130 performs one or more transforms and quantizations in units of 16x16 pixel blocks, selects the most efficient transform type, and generates transformed and quantized transform coefficients according to the selected transform type. can do. In this case, it is preferable that the encoder 140 generates transform type data by generating and encoding a transform type for identifying the type of the transform selected for each block. Transform type data is included in the encoded data. However, different transforms cannot be used in a 16x16 pixel block.

In more detail, when a 16x16 pixel block is divided into 8x8 pixel blocks and one or more 8x8 pixel blocks of four 8x8 pixel blocks are divided into subblocks smaller than 8x8 pixels, 4x4 conversion is performed on the 16x16 pixel block. Only can be used, in which case the transform type is not encoded. In addition, when a 16x16 pixel block is divided into four 8x8 pixel blocks, a 4x4 transform and an 8x8 transform are performed on the 16x16 pixel block to select a transform having a low encoding cost, and a transform type (for example, a type of a transform selected) , May be represented by 1 bit). In addition, when a 16x16 pixel block is divided into two 8x16 pixel subblocks, a 4x4 transform, an 8x8 transform, and an 8x16 transform are performed on the remaining blocks of the 16x16 pixel subblock to select the type of the transform having the lowest encoding cost. When the pixel subblock is divided into two 16x8 pixel subblocks, the 4x4 transform, the 8x8 transform, and the 16x8 transform are performed on the remaining blocks of the 16x16 pixel subblock to select the type of transform having the lowest encoding cost. A transformation type (for example, may be represented by 1 bit or 2 bits) indicating the type may be encoded. For a subblock of 16x16 pixels or more, 4x4 transform, 8x8 transform, and 16x16 transform are performed, and the transform having the lowest encoding cost is selected, and a transform type (for example, represented by 1 bit or 2 bits) indicating the type of the selected transform is encoded. Can be.

12 is an exemplary diagram for describing an encoding method of an inter picture and related data to be transmitted through encoded data according to an embodiment of the present invention.

In the case of a P-picture or a P-slice, all block types of FIG. 4 may be used for motion estimation, and within an extended macroblock, an intra mode or an inter mode may be selected in units of 16 × 16 pixel blocks. That is, both intra mode and inter mode may be used in an extended macroblock of a P-picture or a P-slice.

In the case of a B-picture or B-slice, up to two arbitrary reference pictures can be selected, which are called L0 prediction and L1 prediction, respectively, and these two reference pictures play different roles. Usually, L0 prediction (List 0 Prediction, mainly used for forward prediction) is used as forward prediction, and L1 prediction (List 1 Prediction, mainly used for backward prediction) is used as backward prediction. Each of the more detailed inter picture encoding methods for the P-picture and the B-picture is described below.

When the extended macroblock is a 32x32 pixel block, whether to perform encoding in units of 32x32 pixel blocks or encoding in units of 16x16 pixel blocks may be indicated through an extended macroblock flag (extended_mb_flga). For example, when the extended macroblock flag is 1, encoding may be performed in units of 32x32 pixel blocks, and when the extended macroblock flag is 0, encoding may be performed in units of 16x16 pixel blocks. When the extended macroblock flag is 0, the extended macroblock may be divided into four 16x16 pixelblocks to determine whether it is an intra mode or an inter mode for each 16x16 pixelblock. In addition, each 16x16 pixelblock unit may be divided into smaller subblocks to perform encoding.

Referring back to FIG. 1, the motion estimator 116 of the inter predictor 114 of the predictor 110 finds a block most similar to the current block that is the current encoding target in the current picture, that is, a reference block within the reference picture, for the current block. A motion vector (MV) indicating a relative position of the reference block is output. This process is called motion estimation, and motion estimation generates a motion vector by comparing subblocks in a macroblock with one or more reference pictures in the picture buffer 180. The 8x8 pixel block may use different reference pictures, but the subblocks in the 8x8 pixel block use the same reference picture.

The motion vector is encoded by the encoder 140 and included in the encoded data. The encoder 140 uses a median of motion vectors of neighboring blocks of the current block as a predicted motion vector (PMV). Only the differential motion vector (DMV), which is a difference vector between the predicted motion vector and the motion vector of the current block, is encoded to generate motion information data. In this case, the encoder 140 may further encode not only a differential motion vector but also a reference picture index. That is, the motion information may include a differential motion vector and a reference picture index, and the encoder 140 may encode the motion information to generate motion information data and include the motion information data in the encoded data.

13 is an exemplary view for explaining a process of determining a predicted motion vector according to an embodiment of the present invention.

13A illustrates the motion vector of the neighboring block used to determine the motion vector and the predictive motion vector of the current block when both the current block and the neighboring block are 64x64 pixel blocks. When both the current block and the neighboring block are blocks of the same size, the median of the motion vectors of the neighboring block may be determined as the predicted motion vector as in the conventional method.

13B illustrates the motion vector of the neighboring block used to determine the motion vector and the predictive motion vector of the current block when the sizes of the current block and the neighboring block are different. If the size of the current block and the neighboring block is different, it is necessary to select which neighboring block to determine the prediction motion vector.

According to an embodiment of the present invention, as shown in 13B, when the neighboring block on the left side of the current block is divided into several blocks, the motion vector mvA of the block A located on the upper right side is used for prediction, and The motion vector mvB of the lowermost left block B is used for prediction when the neighboring blocks at the top are divided into several, and the block at the bottom left of the upper right when the neighboring blocks are divided into several The motion vector mvC of C is used for prediction. However, in an exceptional case, when both the blocks B and C are located outside the boundary of the picture, the prediction motion vector may not be determined using the block B and the block C. Decide In addition, when only one block among blocks A, B, and C having the same reference picture number is used, the motion vector of the block is used to determine a predicted motion vector.

14 is an exemplary diagram for describing a process of determining a predicted motion vector of an 8x16 pixel block and a 16x8 pixel block according to an embodiment of the present invention.

In addition, when the current block is an 8x16 pixel block or a 16x8 pixel block, the median value is not used. In other words, when the current block is an 8x16 pixel block, block A is used as the neighboring block on the left side and block C is used to determine the predictive motion vector as the neighboring block on the right side. If the current block is a 16x8 pixel block, the upper neighboring block uses block B and the lower neighboring block uses block A to determine the predicted motion vector.

In addition, in one embodiment of the present invention, the SKIP mode is used for a macroblock that is a pixel block of 16x16 or more. When the macroblock, which is a 16x16 pixel block or more, is a SKIP macroblock, motion estimation and compensation are performed using the predicted motion vector. However, when one or more of the motion vectors used to determine the predictive motion vector is 0, the predictive motion vector is determined as 0. For example, when the motion vectors of neighboring blocks A, B, and C are mvA (0, 10), mvB (1, 10), and mvC (2, 10), the predicted motion vector of the current block is (0). , 10).

If the location of the extended macroblock, which is a 32x32 pixel block, spans the padded image and the padded region in multiples of 16 pixels as shown in FIG. 2, if the extended macroblock is in the SKIP mode, as shown in FIG. Similarly, a predictive motion vector is obtained using the left block and the upper block of an actual image block (16x16 pixel block), and motion prediction and compensation are performed on the actual image block using the predicted motion vector. If the optimal encoding mode of the extended macroblock that is the current encoding target is the SKIP mode, skip32_flag is decoded and the next extended macroblock is encoded.

Referring back to FIG. 1, the subtractor 120 subtracts the current block and the reference block indicated by the motion vector of the current block estimated by the motion estimator 116 to generate a residual block, and the transformer and quantizer 130 subtracts the subtractor. The residual block generated by 120 is transformed and quantized, and the encoder 140 generates encoded data by entropy encoding the quantized transform coefficients. Here, the transformer and quantizer 130 performs one or more transforms and quantizations according to the size of the current block, selects the most efficient transform type, and generates transform and quantized transform coefficients according to the selected transform type. do. The encoder 140 generates transform type data by generating and encoding a transform type for identifying a type of a transform selected for each block. Transform type data is included in the encoded data. However, different transforms cannot be used in a 16x16 pixel block.

Similar to the transform method described in the intra prediction block, 16x16 pixel subblocks are divided into 8x8 pixel blocks and 16x16 pixels when one or more 8x8 pixel subblocks of four 8x8 pixel subblocks are divided into subblocks smaller than 8x8 pixels. Only 4x4 transforms may be used for the remaining blocks of the subblock, in which case the transform type is not encoded. When a 16x16 pixel subblock is divided into four 8x8 pixel subblocks, a 4x4 transform and an 8x8 transform are performed on the remaining blocks of the 16x16 pixel subblock to select a transform having a low encoding cost, and a transform type indicating the type of the selected transform. (E.g., represented by 1 bit) is encoded.

When the 16x16 pixel subblock is divided into two 8x16 pixel subblocks, the 4x4 transform, the 8x8 transform, and the 8x16 transform are performed on the remaining blocks of the 16x16 pixel subblock to select the type of transform having the lowest encoding cost. When the subblock is divided into two 16x8 pixel subblocks, the 4x4 transform, the 8x8 transform, and the 16x8 transform are performed on the remaining blocks of the 16x16 pixel subblock, and the type of transform having the lowest encoding cost is selected. A transform type (which may be represented by 1 bit or 2 bits, for example) is encoded. For a subblock of 16x16 pixels or more, 4x4 transform, 8x8 transform, and 16x16 transform are performed, and the transform having the lowest encoding cost is selected, and a transform type (for example, represented by 1 bit or 2 bits) indicating the type of the selected transform is encoded. do.

When there are two types of transforms that can be selected, such as when a 16x16 pixel block is divided into four 8x8 pixel subblocks, encoding bits used for transform type encoding may be configured as shown in Table 1 below.

In addition, when a 16x16 pixel block is divided into two 8x16 pixel blocks, two 16x8 pixel blocks, or three kinds of selectable transforms such as 16x16 or more subblocks, the encoding bit used for transform type encoding is It may be configured as shown in Table 2.

FIG. 15 is an exemplary diagram illustrating types of usable transforms and units in which transform types are encoded according to subblock sizes according to an embodiment of the present invention.

15 illustrates an example in which an extended macroblock is 64x64 pixels and an extended macroblock is divided into subblocks of various sizes. As described above, since the first 16x16 pixel block 1510 of the macroblock is divided into 4x4 pixel subblocks smaller than the 8x8 pixel block, the only available transform is a 4x4 transform, in which case the transform type is not encoded. Since the second 16x16 pixel block 1520 is divided into four 8x8 pixel subblocks, one of the 4x4 transform or the 8x8 transform having the lowest encoding cost is selected, and a transform type indicating the type of the selected transform is encoded. Since the third 16x16 pixel block 1530 is divided into two 8x16 pixel subblocks, one of the 4x4 transform, the 8x8 transform, or the 8x16 transform has a smaller encoding cost, and a transform type indicating the type of the selected transform is encoded. Since the fourth 16x16 pixel block 1540 is divided into two 16x8 pixel subblocks, one of the 4x4 transform, the 8x8 transform, or the 16x8 transform having the smallest encoding cost is selected, and a transform type indicating the type of the selected transform is encoded. The fifth 16x32 pixel block (1550), the sixth 16x32 pixel block (1560), the seventh 32x32 pixel block (1570), and the eighth 32x32 pixel block (1580) are all larger than the 16x16 pixel block, so they are either 4x4 transform, 8x8 transform, or 16x16. Among the transforms, one having a small encoding cost is selected, and a transform type indicating the type of the selected transform is encoded. For blocks larger than 16x16 pixels, a conversion type may be selected in units of 16x16 pixel blocks. In this case, a plurality of transform types are encoded for a 16x16 pixel block in which an encoded block pattern in a pixel block larger than a 16x16 pixel block such as a 32x32 pixel block is not zero.

Referring back to FIG. 1, the residual block transformed and quantized by the transformer and quantizer 130 is inversely quantized and inverse transformed by the inverse quantizer and inverse transformer 150 to restore the residual block, and the adder 160 The current block is reconstructed by adding the reconstructed residual block and the predicted block predicted by the predictor 110.

The filter 170 performs filtering using a deblocking filter, etc., on the pictures created by accumulating the current blocks restored by the adder 160, and the filtered pictures are stored in the picture buffer 180 to refer to the next picture. It is used as a picture.

16 is an exemplary diagram illustrating a block boundary to perform deblocking filtering according to an embodiment of the present invention.

16A to 16E represent block boundaries for deblocking filtering in 16x16 pixel blocks using 4x4 transform, 8x8 transform, 8x16 transform, 16x8 transform, and 16x16 transform, respectively. 16x16 pixel blocks using the 4x4 transform shown in 16A are deblocked filtered at the block boundaries on the left and top of each 4x4 pixel block, and 16x16 pixel blocks using the 8x8 transform shown in 16B are used to the left and top of each 8x8 block. Deblocking filtering is performed at the block boundary of the 16x16 pixel block using the 8x16 transform shown in 16C, and deblocking filtering is performed at the left and top block boundaries of each 8x16 pixel block, and the 16x8 transform shown in 16D is used. 16x16 pixel blocks are deblocked filtered at the left and top block boundaries of each 16x8 pixel block, and 16x16 pixel blocks using the 16x16 transform shown in 16E are deblocked filtered at the left and top block boundaries of the 16x16 pixel block. Is performed.

17 is an exemplary diagram illustrating syntax of an intra picture according to an embodiment of the present invention.

When the shape of the sub block for the extended macroblock includes a rectangular type (for example, a 32x16 pixel block, a 16x32 pixel block, etc.) as shown in FIG. 4, the image encoding apparatus 100 may be configured in the image decoding apparatus. Since the type information of the divided sub-blocks must be transmitted, the encoded data of the intra picture includes a partition type, an intra prediction mode, a coded block pattern X-flag / coded block pattern, and a CBPX_flag / CBP. It may consist of fields such as Transform Coefficient. The encoder 140 of the image encoding apparatus 100 encodes the segmentation type information, the intra prediction mode, the CBPX_flag / CBP, the transform coefficient, and the like, and assigns them to each field to generate encoded data.

However, the syntax of the encoded data for the intra picture shown in FIG. 17 is merely exemplary, and the syntax of the encoded data for the intra picture is not necessarily configured as shown in FIG. 17. That is, it should be appreciated that some or all of the syntax of the encoded data for the intra picture may be changed differently from that shown in FIG. 17, and some syntax may be omitted.

Alternatively, when intra prediction encoding is performed in units of 16x16 pixel blocks after the subblock shape of the extended macroblock is divided into square pixel blocks as illustrated in FIG. 5, encoding for the CBPX_flag is omitted. The increase / decrease information Δpq for the quantization parameter may be encoded. In this case, the increase / decrease information on the quantization parameter may be encoded in extended macroblock units, and the CBP information may be encoded in units of 16 × 16 pixel blocks.

In FIG. 17, split type data, which is data in which split type information of a macroblock indicating a macroblock is divided into subblocks, is allocated to a split type field. The partition type information may be generated using the partition type number shown by way of example in FIG. 18.

화소 블록이 더 이상 분할되지 않는 경우에는 매크로블록 레이어 K의

화소 블록에 분할 타입 번호 0이 할당된다. 매크로블록 레이어 K의

화소 블록이 두 개의

화소 블록으로 분할된 경우에는 매크로블록 레이어 K의

화소 블록에 분할 타입 번호 1이 할당된다. 매크로블록 레이어 K의

화소 블록이 두 개의

화소 블록으로 분할된 경우에는 매크로블록 레이어 K의

화소 블록에 분할 타입 번호 2가 할당된다. 매크로블록 레이어 K의

화소 블록이 4 개의

로 분할되는 경우에는 매크로블록 레이어 K의

화소 블록에 분할 타입 번호 3이 할당된다. 도 18에서 각 매크로블록 레이어 K의

화소 블록 내의 분할된 서브블록에 표시된 0, 1, 2, 3 등의 숫자는 각 서브블록을 식별하기 위한 파티션 번호(Partition Number)이다. Referring to FIG. 18, macroblock layer K

If the pixel block is no longer divided, the macroblock layer K

The division type number 0 is assigned to the pixel block. Of the macroblock layer K

Two pixel blocks

If divided into pixel blocks, macroblock layer K

The division type number 1 is assigned to the pixel block. Of the macroblock layer K

Two pixel blocks

If divided into pixel blocks, macroblock layer K

The division type number 2 is assigned to the pixel block. Of the macroblock layer K

Four pixel blocks

Is divided into macroblock layer K

The division type number 3 is assigned to the pixel block. Of each macroblock layer K in FIG.

Numbers such as 0, 1, 2, 3, etc., displayed on the divided subblocks in the pixel block are partition numbers for identifying each subblock.

In FIG. 18, the number of each partition type is assigned in the order of the smallest to the largest order. However, the partition type number of FIG. 18 is merely an example and may be allocated from the larger number to the smaller number in order of occurrence. . In addition, the partition type number may be allocated according to the occurrence probability of each partition type, or the partition type number may be adaptively changed. In addition, each divided block may be divided into smaller blocks. For example, when a 32x32 pixel block is divided into four 16x16 pixel blocks by partition type number 3, each 16x16 pixel block may be divided into smaller subblocks using the partition type-specific number of FIG. In addition, when the 16x16 subblock is divided into smaller blocks, a partition type number of the upper pixel block, that is, a partition type number different from the partition type number of the 32x32 pixel block may be used. When encoding is performed by dividing the extended macroblock into units of 16x16 pixel blocks, the subblocks (for example, 64x32 pixel blocks, 32x64 pixel blocks, 16x32 pixel blocks, 32x16 pixel blocks, etc.) larger than 16x16 pixel blocks are divided. Encoding of the type number may be omitted.

FIG. 19 is a diagram illustrating a split type number coding sequence in the case of dividing an expanded macroblock into subblocks of various sizes according to an embodiment of the present invention.

In FIG. 19, the extended macroblock is a 64x64 pixel block and the maximum number of divided layers is four. When the extended macroblock is partitioned as shown in FIG. 19, according to an embodiment of the present invention, the partition type information of the macroblock may be sequentially encoded according to the partition type number encoding order in order to encode the partition type information of the macroblock. .

A process of dividing the macroblocks shown in FIG. 19 for each of the divided layers may be shown as shown in FIG. 20. Referring to FIG. 20, in macroblock layer 0 (L0), a 64x64 pixel subblock is divided into four 32x32 pixel subblocks, and in layer 1 (L1), L1-P0 (partition number 0 of macroblock layer 1). The subblock of L1-P3 (partition number 3 of macroblock layer 1) is further divided into four 16x16 pixel subblocks, and the subblock of L1-P1 (partition number 1 of macroblock layer 1) and L1- The subblocks of P2 (partition number 2 of layer 1) are divided into 16x32 pixel subblocks and 32x16 pixel subblocks, respectively. Since the subblocks of L1-P1 and L1-P2 are divided into two subblocks, and are no longer divided, the partition number for each layer is not represented. In layer 2 (L2), the subblock of L2-P0 (partition number 0 of macroblock layer 2) is further divided into four 8x8 pixel subblocks and L2-P3 (partition number 3 of macroblock layer 2) is It is divided into 16x8 pixel subblocks. Again, in macroblock layer 3 (L3), the subblocks of L3-P0 (partition number 0 of macroblock layer 3) and the subblocks of L3-P1 (partition number 1 of macroblock layer 3) are each 4 4x4 pixels. It is divided into subblocks. In this case, when the extended macroblock is encoded in units of 16x16 pixel blocks, the L1-P1 blocks and the L1-P2 blocks are respectively divided into L1-P3 blocks instead of being divided into 16x32 pixel subblocks and 32x16 pixel subblocks, respectively. Encoding may be performed in units of four 16 × 16 pixel blocks.

21 is an exemplary diagram for describing a process of sequentially encoding a split type number for each layer of a macroblock.

If the partition type information of the macroblock shown in FIG. 19 is encoded, it may be encoded as shown in FIG. 21. In Fig. 19, numerals written in '□' indicate an order of encoding division type numbers of respective subblocks. If the split type number for each layer for the extended macroblock is sequentially encoded according to the split type number encoding order, the split type number for each layer may be sequentially encoded in the order shown in FIG. 21.

First, since the 64x64 pixel subblocks L0-P0 of the macroblock layer 0 are divided into four 32x32 pixel subblocks, division type number 3 is encoded. Since the first 32x32 pixel subblocks L1-P0 of the four 32x32 pixel subblocks in the 64x64 pixel subblock are also divided into four 16x16 pixel subblocks, the partition type number 3 is encoded. Since the first 16x16 pixel subblocks L2-P0 of the four 16x16 pixel subblocks in the first 32x32 pixel subblocks L1-P0 of Layer 1 are also divided into four 8x8 pixel subblocks, the partition type 3 is encoded and The four 8x8 pixel blocks (L3-P0, L3-P1, L3-P2, L3-P3) in the 16x16 pixel subblocks L2-P0 are no longer divided into small subblocks, so the partition type number {3, 3, 0 and 0} are encoded respectively. Since subblocks of the layer 3 cannot be divided into smaller subblocks, the partition type numbers of the subblocks belonging to the layer 3 are not encoded.

Since the partition type numbers of the subblocks in the macroblock layer 3 are all encoded, the partition type numbers of the second 16x16 pixel subblock L2-P1 and the third 16x16 pixel subblock L2-P2 of the macroblock layer 2 are encoded. Since both are no longer divided into small blocks, partition type number 0 is encoded. The fourth 16x16 pixel subblock L2-P3 is divided into 16x8 pixel subblocks, but since the partition type number is not 3, only the partition type number 1 is encoded. Since the partition type numbers of the four subblocks in the layer 2 are all encoded, the partition type numbers of the second 32x32 pixel subblocks L1-P1 of the layer 1 are encoded. ) Is divided into 16x32 pixel subblocks, and since each divided subblock is no longer divided into small subblocks, the partition type number 2 is encoded. In this manner, the third 32x32 pixel subblock L1-P2 of macroblock layer 1, the fourth 32x32 pixel subblock L1-P3 of macroblock layer 1, and the four 16x16 pixel subblocks L2- below it. When the division type numbers of P0, L2-P1, L2-P2, and L2-P3 are sequentially encoded, {1, 3, 0, 0, 0, 0} is encoded.

When the partition type information of the macroblock shown in FIG. 19 is encoded according to the above-described encoding method, as shown in FIG. 21, the partition type number {3, 3, 3, 3, 3, 0, 0, 0, 0, 0, 1, 2, 1, 3, 0, 0, 0, 0} is encoded. In this case, the split type number may be encoded into a binary bit string using lossless compression coding such as binary arithmetic coding or Huffman coding.

이하인 경우에는 표 3을 이용하고 레이어 번호가

보다 큰 경우에는 표 4를 이용하여 부호화할 수 있다. 예를 들어 도 11의 서브블록(L1-P0)의 분할 타입 번호 3은 표 3을 참고하면 이진수 '01'로 표현될 수 있으므로 이진수 '0'과 '1'을 각각 산술 부호화하여 분할 타입 번호 3을 부호화할 수 있다. 그리고 서브블록(L2-P0)에 속하는 서브블록(L3-P2)의 분할 타입 번호 0은 표 4를 참고하면 이진수 '1'로 표현될 수 있으므로 이진수 '1'을 산술 부호화하여 분할 타입 번호 3을 부호화할 수 있다.For example, when binary arithmetic coding is used, each partition type number may use a different binary value according to the layer number of the partition type number to be currently encoded. Layer number

If it is below, use Table 3 and the layer number is

If larger, it can be encoded using Table 4. For example, since the partition type number 3 of the subblocks L1 to P0 of FIG. 11 may be represented by the binary number '01', referring to Table 3, the partition type number 3 may be arithmetic encoded by the binary numbers '0' and '1', respectively. Can be encoded. Since the partition type number 0 of the subblock L3-P2 belonging to the subblock L2-P0 may be represented by the binary number '1' referring to Table 4, the partition type number 3 is arithmetic encoded by the binary number '1'. Can be encoded.

Referring back to FIG. 17, when split type data is encoded by encoding split type information of a macroblock for an intra picture and is allocated to a split type field, an intra prediction mode is data in which information about an intra prediction mode is encoded in an intra prediction mode field. The data is allocated.

22 is an exemplary diagram illustrating a coding order of an intra prediction mode according to an embodiment of the present invention.

FIG. 22 illustrates a case in which the expanded macroblock is a 64x64 pixel block and the number of maximum division layers is 4 as shown in FIG. 19. When the extended macroblock is divided as shown in FIG. 22, the intra prediction mode for each subblock of the extended macroblock is sequentially encoded according to the encoding order of the intra prediction mode shown in FIG. 22 to generate intra prediction mode data. can do. In other words, the number indicated in each subblock indicates the encoding order of the intra prediction mode for encoding the intra prediction mode. When encoding an extended macroblock of 64x64 pixels in units of 16x16 pixel blocks, a 32x32 pixel block corresponding to a 15th block and a 16th block, or a 17th block and an 18th block may be a 16x32 pixel block or a 32x16 pixel block. Instead of dividing into pixel blocks, encoding may be performed by dividing into four 16x16 pixel block units as in the 19th to 22nd blocks.

If the subblock of the luminance component of the extended macroblock is not a 16x16 pixel block, a prediction mode flag indicating whether the prediction direction of the current block estimated from the neighboring block and the prediction direction of the selected current block is the same is encoded, and the prediction mode If the prediction direction of the block whose flag is the current encoding target and the prediction direction of the current block estimated from the neighboring block are not the same, the prediction mode indicating that the prediction direction of the current block and the prediction direction of the current block estimated from the neighboring block are not the same. The prediction mode number of the prediction mode indicating the prediction direction among the remaining prediction directions except the prediction direction of the current block estimated from the neighboring block among the various intra prediction directions according to the size of the flag and the corresponding subblock, the intra prediction mode of the corresponding block. Encoded as The intra-prediction mode data is produced. In this case, when intra prediction is performed by dividing the expanded macroblock into subblocks other than 4x4 pixel blocks, 8x8 pixel blocks, or 16x16 pixel blocks, the prediction mode of the current block predicted using the left block and the upper block of the current block. If the number is 3 or more, the prediction direction of the current block may use prediction mode number 2 (DC mode) of FIG. 9.

If the prediction direction of the current block estimated from the neighboring block and the prediction direction of the current block are the same, the prediction mode flag indicating that the prediction direction of the current block and the prediction direction of the current block estimated from the neighboring block is the same is intra prediction mode. As a result, intra prediction mode data is generated. In the prediction mode of the macroblock of the chrominance component or the subblock of the luminance component is an intra prediction mode, intra prediction mode data is generated by encoding the number of the prediction mode of the block as an intra prediction mode. In this way, the intra prediction mode data for each subblock of the macroblock generated by encoding in the order shown in FIG. 22 is allocated to the intra prediction mode field shown in FIG.

FIG. 23 is an exemplary diagram for explaining an example of a process of encoding an intra prediction mode of a macroblock according to an embodiment of the present invention.

In 23A, the current macroblock to be encoded is a 64x64 pixel block, and is divided into subblocks of various sizes and shapes as shown. The neighboring block on the left side of the current macroblock is intra predicted in units of 64x64 pixels, and thus the neighboring block on the left side. The prediction mode number of 0 is 0 (vertical mode), and the upper neighboring block of the current macroblock is intra predicted in units of 64x64 pixels, and the prediction mode number of the left neighboring block is 1 (horizontal mode). The number displayed in each subblock in the current extended macroblock indicates a prediction mode number of each subblock.

The prediction mode number of the first 4x4 pixel subblock of the current extended macroblock is 2 (DC mode), and the prediction mode number of the corresponding subblock estimated using the neighboring block on the left side and the upper neighboring block is 0. That is, since 0 of the prediction mode number 0 of the left neighboring block and the prediction mode number 1 of the upper neighboring block is smaller, the prediction mode number of the first 4x4 pixel subblock is estimated to be zero. Since the prediction mode of the first 4x4 pixel subblock is different from the prediction mode of the corresponding subblock estimated from the neighboring block, the prediction indicating that the prediction direction of the corresponding subblock and the prediction direction of the corresponding subblock estimated from the neighboring block are not the same. The mode flag and prediction mode number 2, which is the prediction mode number of the subblock, are encoded as an intra prediction mode of the subblock. Since the prediction mode number of the second 4x4 pixel subblock is 1 and the prediction mode number of the corresponding subblock estimated from the neighboring block is 1, the prediction direction of the corresponding subblock and the prediction direction of the corresponding subblock estimated from the neighboring block are the same. Only the prediction mode flag indicating that the prediction directions are the same is encoded as the intra prediction mode of the corresponding subblock.

Next, the intra prediction mode of the next subblock is encoded according to the encoding order of the intra prediction mode shown in FIG. 22. The prediction mode number of the third 4x4 pixel subblock is 0 and the prediction of the corresponding subblock estimated from the neighboring block is performed. Since the mode number is 0, the prediction direction of the subblock estimated from the neighboring block and the prediction direction of the corresponding subblock are the same, so that only the prediction mode flag indicating the same prediction direction is encoded as the intra prediction mode of the subblock. In the case of the eleventh 16x16 pixel subblock, the prediction flag is not encoded, and prediction mode number 1 of the subblock is encoded as the intra prediction mode of the subblock. FIG. 23 illustrates a case in which the 15th and 16th blocks are divided into 16x32 pixel blocks and encoded in an intra prediction mode, and the 17th and 18th blocks are divided into 32x16 pixel blocks and encoded in an intra prediction mode. However, when intra prediction coding is performed on an extended macroblock in units of 16x16 pixel blocks, a rectangular block such as a 15th block and a 16th block or a 17th block and an 18th block may be used for a subblock larger than the 16x16 pixel block. Pixel block division is omitted.

23B exemplarily shows intra prediction mode data included in an intra prediction mode field when encoding an intra prediction mode of a current macroblock as shown in 23A. The prediction mode flag and the prediction mode number are encoded for the first 4x4 subblock, and only the prediction mode flag is encoded for the second, third, and fourth 4x4 subblocks. Mode data is assigned to the intra prediction mode field.

When intra prediction encoding an extended macroblock, as shown in FIG. 4, when intra prediction encoding is performed by dividing into a rectangular subblock larger than a 16x16 pixel block (for example, 16x32, 32x16, etc.), the subblock is performed. Coded block pattern flag (CBPX_flag: Coded Block Pattern X_flag, hereinafter referred to as 'CBPX flag') and CBP (CBP: Coded Block Pattern, hereinafter referred to as 'CBP') by considering the size of CBPX_flag / CBP Can be assigned to a field. The CBP data allocated to the CBPX_flag and the CBP field indicates whether a non-zero quantized transform coefficient is included in a subblock of the macroblock. However, when performing intra prediction encoding by dividing the extended macroblock into units of 16x16 pixel blocks as shown in FIG. 5, the encoding of the block pattern flag is omitted and the increase / decrease information (Δpq) for the quantization parameter may be encoded. have. In this case, the increase / decrease information on the quantization parameter may be encoded in the extended macroblock unit, and the CBP data may be encoded in the 16 × 16 pixel block unit.

24 is an exemplary diagram for describing a process of encoding a CBPX_flag and a CBP for an intra macroblock according to an embodiment of the present invention.

24 illustrates an example of encoding a CBPX_flag and a CBP when an extended macroblock is a 64x64 pixel block. The encoder 140 of the image encoding apparatus 100 determines whether the size of the subblock of the extended macroblock is one of 64x64, 64x32, and 32x64 (S2410), and the size of the subblock is 64x64, 64x32, or 32x64. If not, it is determined whether the size of the subblock is one of 32x32, 32x16, and 16x32. In this case, when intra prediction is performed by dividing the extended macroblock into units of 16x16 pixels, in step S2410, it is determined whether the size of the subblock is a 64x64 pixel block, that is, whether intra prediction coding is performed in units of the extended macroblock. In S2420, when the size of the subblock is not a 64x64 pixel block, it may be determined whether the size of the subblock is a 32x32 pixel block. As a result of the determination in step S2420, if the size of the subblock is one of 32x32, 32x16, and 16x32 or 32x32 pixel block, it is determined whether there is a non-zero transform coefficient in the subblock (S2430), and the non-zero transform coefficient is If present, the CBP32 flag (for example, may be represented by 1 bit such as '1') indicating that a non-zero coefficient to be encoded is encoded in the corresponding subblock (S2440), and a 16x16 pixel block in the subblock is encoded. CBP is encoded as a unit (S2450), and if there is no non-zero transform coefficient in the subblock, the CBP32 flag indicating that there is no non-zero transform coefficient in the subblock (for example, 1 bit such as '0') It can be encoded (S2460).

Here, the CBP64 flag is a flag indicating the presence or absence of a coefficient to be encoded that is not zero in a 64x64 pixel block. Similarly, the CBP32 flag is a flag indicating the presence or absence of a non-zero coefficient to be encoded in a 32x32 pixel block. Such a CBPX flag (X is 32, 64, etc.) indicates whether the luminance component blocks have a non-zero transformation coefficient since the transform type does not need to be transmitted when the residual block of the luminance component does not have a nonzero transformation coefficient. Used to. The CBP is encoded in units of 16x16 pixel blocks, and indicates whether there is a non-zero transform coefficient for each 8x8 pixel block in the 16x16 pixel block using 1 bit per 8x8 pixel block, and two 2x2 chrominance component DC blocks for the color difference component. Indicates whether there is a non-zero transform coefficient using 1 bit, and indicates whether there are non-zero transform coefficients in two 8x8 color difference component AC blocks using 1 bit.

On the other hand, in step S2420, if the size of the subblock is not one of 32x32, 32x16 and 16x32 or is not a 32x32 pixel block, the process proceeds to step S2450 to encode the CBP. In this case, when the size of the sub block is not a 32x32, 32x16, or 16x32 pixel block, it may be determined whether the size of the subblock is a 16x16, 16x8, or 8x16 pixel block. If the subblock is a 16x16, 16x8, or 8x16 pixel block, if there is a non-zero coding coefficient in the subblock, the CBP16 flag (for example, may be represented by 1 bit such as '1') is encoded. In other cases, the CBP may be encoded in units of 16 × 16 pixel blocks without using the CBP flag.

Further, in step S2410, if the size of the subblock is one of 64x64, 64x32, and 32x64 or is a 64x64 pixel block, it is determined whether there is a non-zero transform coefficient in the subblock (S2470), and the non-zero transform in the subblock. If there is a coefficient, a CBP64 flag (for example, may be represented by 1 bit such as '1') indicating that there is a non-zero transform coefficient is encoded (S2480), and a non-zero coefficient to be encoded in the subblock is encoded. If none, a CBP64 flag (for example, may be represented by 1 bit such as '0') indicating that there is no coefficient to encode other than 0 (S2490). Here, the CBP64 flag is a flag indicating whether a non-zero quantized transform coefficient exists in a 64x64 pixel block.

If intra prediction coding is performed on an extended macroblock of 64x64 pixels in units of 16x16 pixel blocks, as described above, the CBP64 flag and the CBP32 flag may be omitted, and the increase / decrease information Δpq of the quantization parameter may be encoded. can do. In this case, it is preferable to encode the increase / decrease information for the quantization parameter in an extended macroblock unit.

In the case of encoding the CBP flag, the conversion type is encoded when the CBP flag is not zero. Exceptionally, when the 16x16 pixel block is divided into four 8x8 pixel blocks, the CBP is encoded, and when the CBP is not 0, the transform type applied in units of 16x16 pixel blocks is encoded.

After encoding the transform type, the CBP is encoded according to the transform type. If 16x16 transform is used, only the CBP 2-bits of the chrominance component are encoded. If 8x16 or 16x8 transform is used, CBP 2-bit indicating whether two 8x16 or 16x8 pixel blocks within the 16x16 pixel block have a non-zero transform coefficient. Encode However, in exceptional cases, when the CBP bit of the first divided block of the two divided blocks is 0, the CBP bits of the second divided block may not be encoded.

25 is an exemplary view showing the structure of a CBP according to an embodiment of the present invention.

FIG. 25 exemplarily illustrates a structure of a CBP when an input image has a 4: 2: 0 format. If there is no nonzero transform coefficient in the first 8x8 pixel block (Y0) of the luminance component, the least significant bit (LSB) of the CBP is zero, and the nonzero quantized transform in the second 8x8 pixel block (Y1). If there is a coefficient, the second least significant bit of the CBP is one. In this way, the CBP for the third 8x8 pixel block (Y2) and the fourth 8x8 pixel block (Y3) are allocated, and nonzero in the two 2x2 chrominance component DC blocks (U (DC) and V (DC)). Since there is a quantized transform coefficient, the bit for chrominance DC is 1, and there are no nonzero quantized transform coefficients in the two 8x8 chrominance component AC blocks (U-AC and V-AC), so Chroma AC Bit is zero.

FIG. 26 is an exemplary diagram illustrating an example of a sequence of encoding a CBPX flag and a CBP according to an embodiment of the present invention. FIG.

FIG. 26 exemplarily illustrates a coding sequence of a CBP64 flag, a CBP32 flag, and a CBP when an extended macroblock is a 64x64 pixel block and a size of a subblock currently being encoded is 64x64, 64x32, or 32x64. In FIG. 26, the dashed-dotted arrow indicates the order of progression when the CBPX flag is 0, the dashed arrow indicates the order of progression when the CBPX flag is 1, and the solid arrow indicates the order of progressing without any condition. .

If all of the quantized transform coefficients are 0 in the 64x64 pixel block, which is an extended macroblock, the CBP64 flag is coded 0, and if the at least one quantized transform coefficient is not 0, the CBP64 flag is coded 1. If the CBP64 flag is 0, since there are no more CBPs to encode, the CBPX flag and CBP of the next macroblock are encoded. If the CBP64 flag is 1, the CBP32 flag of each 32x32 pixel subblock in the 64x64 pixel block is encoded. If the CBP32 flag is 1, the CBP of four 16x16 pixel blocks in the 32x32 pixel subblock is encoded.

27 is an exemplary diagram illustrating another example of a sequence of encoding a CBPX flag and a CBP for an intra macroblock according to an embodiment of the present invention.

In FIG. 27, when the encoding target is a 32x32 pixel block and the size of the subblock currently encoding target is 32x32, 32x16, or 16x32, the encoding order of the CBP32 flag and the CBP is exemplarily illustrated. In FIG. 27, the dashed-dotted arrow indicates the order of progression when the CBPX flag is 0, the dashed arrow indicates the order of progression when the CBPX flag is 1, and the solid arrow indicates the order of progressing without any condition. .

Referring back to FIG. 1, the encoder 140 of the image encoding apparatus 100 may only encode blocks having non-zero quantized transform coefficients after encoding a CBPX (X: integer greater than or equal to 32) flag and CBP. The encoded quantized transform coefficients are encoded to generate encoded image data. If the size of the block to be encoded is larger than the size of the transform, the quantized transform coefficients of the block are encoded in a raster scan order in the block.

28 is an exemplary diagram illustrating a coding order of quantized transform coefficients for each partition type according to an embodiment of the present invention.

FIG. 28 exemplarily illustrates a procedure of encoding quantized transform coefficients when an encoding target block is a 32x32 pixel block. For blocks larger than 16x16 pixels, 16x16 transform (4x4 transforms of 16 4x4 blocks in 16x16 blocks and 4x4 transforms are performed for 16 DC coefficients. Therefore, the conversion is performed in units of 16x16 pixels. Since 16x16 conversion is performed), four 16x16 transforms are performed in the case of a 32x32 pixel block. In the case of a 32x16 pixel block, two 16x16 transforms are performed for each 32x16 pixel block. In the case of a 16x32 pixel block, two 16x16 transforms are performed for each 16x32 pixel block. In the case of a 16x16 pixel block, 16x16 conversion is performed for each 16x16 pixel block.

29 is an exemplary diagram illustrating a syntax structure of encoded data for an inter picture according to an embodiment of the present invention.

FIG. 29 exemplarily illustrates a syntax structure of encoded data generated by encoding according to an embodiment of the present invention when the input image is an inter picture. As shown, the encoded data for the inter picture according to an embodiment of the present invention includes one or more fields of a SKIP_N flag, a macroblock type, and a partition type, and is intra prediction when the extended macroblock is an intra macroblock. It further includes one or more fields of Mode, CBPX Flag / CBP, and Transform Coefficient, and if the extended macroblock is an inter macroblock, SKIPX flag, differential motion vector / reference picture index, CBPX flag, transform type, CBP, and transform It may be configured to further include one or more fields of the coefficients.

However, the syntax of the encoded data for the inter picture shown in FIG. 29 is merely exemplary, and the syntax of the encoded data for the inter picture is not necessarily configured as shown in FIG. 29. That is, it should be appreciated that some or all of the syntax of the encoded data for the inter picture may be changed differently from those shown in FIG. 29, and some syntax may be omitted.

Referring back to FIG. 1, the encoder 140 of the image encoding apparatus 100 encodes a SKIPN flag indicating whether a macroblock currently being encoded is a SKIP macroblock. Where N represents the size of the macroblock. For example, if the macroblock is a 64x64 pixel block, the SKIP64 flag is encoded to indicate whether the 64x64 pixel macroblock is a SKIP macroblock, and if the macroblock is a 32x32 pixel block, the SKIP32 flag to indicate whether the 32x32 pixel macroblock is a SKIP macroblock. Is encoded, and a SKIP16 flag indicating whether the 16x16 pixel macroblock is a SKIP macroblock is encoded.

If the macroblock is a SKIP macroblock, since the current macroblock has no data to be encoded, the next macroblock is encoded. If the macroblock is not a SKIP macroblock, a macroblock type flag indicating whether the macroblock is an intra macroblock or an inter macroblock is transmitted. For example, when the input image to be encoded is a P picture, the available macroblock types are I-macroblocks and P-macroblocks, and when the input image to be encoded is the B picture, the macroblock type is usable. Are I- and B-macroblocks.

Since the split type field and the fields such as intra prediction mode, CBPX flag / CBP, and transform coefficient for the intra picture have been described above with reference to FIG. 17, the following descriptions describe SKIPX flag, differential motion vector / reference picture index, and CBPX flag for inter picture. Fields such as, transform type, CBP, transform coefficient, and the like are described.

In the case of a P-picture or a P-slice, all block types of FIG. 4 may be used for motion estimation, and may be selected to be intra mode or inter mode in units of 16 × 16 pixel blocks within an extended macroblock. That is, both intra mode and inter mode may be used in an extended macroblock of a P-picture or a P-slice.

When the object to be encoded is a 32x32 pixel block, whether to perform encoding in units of 32x32 pixel blocks or encoding in units of 16x16 pixel blocks may be indicated by an extended macroblock flag (extended_mb_flag). For example, when the extended macroblock flag is 1, encoding may be performed in units of 32x32 pixel blocks, and when the extended macroblock flag is 0, encoding may be performed in units of 16x16 pixel blocks. When the extended macroblock flag is 0, the extended macroblock may be divided into four 16x16 pixelblocks to determine whether it is an intra mode or an inter mode for each 16x16 pixelblock. In addition, each 16x16 pixelblock unit may be divided into smaller subblocks to perform encoding.

30 is a diagram illustrating an example of syntax used in a P-picture (or P-slice) according to an embodiment of the present invention. For simplicity, it is assumed that the expanded macroblock is a 32x32 pixel block.

In case of context-based adaptive binary arithmetic coding (CABAC) in entropy coding, skipX_flag and interX_flag may have different context tables according to X values. For example, skip32_flag and skip16_flag may have different context tables. Detailed description thereof will be described later.

The skipX_flag is a flag indicating whether or not the current block to be encoded is in the SKIP mode. As shown in Table 5, the skipX_flag may indicate whether or not the SKIP mode is in one bit. For example, when the encoding bit of skipX_flag is 1, this may indicate that the current block is in the SKIP mode. Here, X means a number greater than or equal to 16 within the range of the extended macroblock.

If the current block is not in the SKIP mode, interX_flag is encoded. Here, interX_flag is a flag indicating whether the current block to be encoded is intra mode or inter mode, and as shown in Table 6, whether the divided pixel blocks in the macroblock extended to 1 or 2 bits are intra mode or inter mode. It may indicate whether the mode.

For example, when interX_flag is '00', this means that all the divided subblocks in the extended macroblock are in intra mode. In this case, the 32x32 pixel block, which is an extended macroblock, is divided into four 16x16 pixel blocks, and each 16x16 pixel block is encoded by the above-described intra prediction encoding method. Therefore, when all subblocks in the extended macroblock are intra modes, the block type of the divided subblock, the intra prediction mode of the luminance component and the chrominance component, the CBP, the quantization parameter, the transform coefficient, and the like may be encoded.

As shown in FIG. 30, when skip32_flag is 0 and inter32_flag is 1 for a 32x32 pixel block which is an extended macroblock, this means that all of the divided subblocks in the extended macroblock are inter mode. transmit extended_mb_flag '. In this case, when extended_mb_flag is 1, this means that motion compensation is performed in units of extended macroblocks. Since one motion vector is encoded, one MVD / ref_idx is encoded. Here, MVD / rex_idx means motion vector information / reference index included in an extended macroblock. When extended_mb_flag is 0, this means that the extended macroblock is divided into 16x16 pixel blocks and encoded, and skip16_flag indicating whether each 16x16 pixel block is in SKIP mode for each 16x16 pixel block. In this case, the block type and MVD / ref_idx are encoded only for the 16x16 pixel block, not the SKIP mode, for each 16x16 pixel block. skip16_flag can be encoded using Table 5. In this case, when skip32_flag and skip16_flag are context-based adaptive binary arithmetic coding (CABAC), as described above, each syntax may have a different context table.

If skip32_flag is 0 and inter32_flag is 01 for the extended macroblock, this means that both inter mode and intra mode are used in the extended macroblock. In this case, 'extended_mb_flag' is not necessary, and since the extended macroblock is divided into 16x16 pixel blocks and encoded, it can be encoded using Table 7 indicating whether each 16x16 pixel block is intra mode or inter mode. have.

In this case, when the intra mode and the inter mode exist together in the extended macroblock, each of the 16x16 pixel blocks is mostly an inter mode or all intra modes in each 16x16 pixel block. The case where the sub block of the intra mode and the sub block of the inter mode exist at the same time can be ignored. Therefore, after encoding the division type of each 16x16 pixel block, subblocks in the 16x16 pixel block in which inter16_flag is 0, that is, the intra mode, encode the luminance component and the chrominance component in the intra prediction mode. In addition, skip16_flag is encoded by using Table 5 for 16 × 16 pixel blocks having inter16_flag of 1, that is, inter mode. Here, the reason of encoding skip16_flag after encoding the partition type is that the intra mode and the inter mode are present in the extended macroblock at the same time, which means that the texture exists in the inter mode. The mode is likely to be divided into small blocks, and therefore, it is preferable to transmit the split type first and to encode skip16_flag only for pixel blocks having a block size of 16x16. However, the encoding order is not limited to this, and skip16_flag may be encoded first, and then the partition type may be encoded.

Subsequent encoding of CBPX_flag, transform type, CBP / quantization parameter, transform coefficient, etc. is the same as described above, and thus description thereof is omitted. However, in case of intra picture / slice or intra EMB, CBPX_flag may not be encoded.

If the extended macroblock is a 64x64 pixel block, the skipX_flag is encoded only for subblocks having a size of 16x16 or more after the split type information of the current block to be encoded is encoded, similarly to the case where the extended macroblock is a 32x32 pixel block. do. In this case, X represents the size of a subblock such as 16 or 32. For example, when the macroblock is a 64x64 pixel block and is not in the SKIP mode and the size of the subblock of the extended macroblock is one of 64x32, 32x64, and 32x32, skip32_flag is encoded for each subblock. When the size of the subblock of the extended macroblock that is 64x64 pixels is 64x32 or 32x64, two skip32_flag are encoded. If the size of the subblock is smaller than 32x32, skip32_flag is not encoded. When the size of the subblock is one of 32x16, 16x32, and 16x16, skip16_flag of each subblock is encoded.

FIG. 31 is a diagram illustrating an example of syntax used in a B-picture (or B-slice) according to an embodiment of the present invention.

In a B-picture, up to two arbitrary reference pictures can be selected. The two sheets are called L0 prediction and L1 prediction, respectively, and each reference picture plays a different role. In general, L0 prediction is used for forward prediction and L1 prediction is used for backward prediction. In other words, the B-picture can use a picture before the current picture to be encoded and a picture after the picture to be encoded as a reference picture, and can use the two reference pictures at the same time (bidirectional prediction) to perform motion estimation and compensation. . As the block type used for the motion estimation, all block types of FIG. 2 may be used, and whether the intra mode or the inter mode is selected in units of 16 × 16 pixel blocks within an extended macroblock in the same manner as the P-picture method. In the embodiment according to the present invention, two pictures are not limited to the picture before the current picture and the picture after the picture, and any two pictures can be used as the reference picture regardless of before and after. The motion picture / compensation (bidirectional prediction) may be performed using the reference picture.

Coding modes that can be used in a B-picture according to an embodiment of the present invention include intra mode, SKIP mode, direct mode, L0 prediction, L1 prediction, and bidirectional.

The intra mode may be encoded using the same method as the above-described intra encoding method, and may select an intra mode in units of extended macroblocks or an intra mode in units of 16 × 16 pixel blocks within the extended macroblocks.

The SKIP mode transmits only a flag indicating that the current block to be encoded is a SKIP mode. In this case, the SKIP mode may be used for blocks of 16x16 pixels or more such as 16x16 pixel blocks and 32x32 pixel blocks, and the motion compensation is performed using the predicted B_pred_dir, the predicted motion vector MVD, the predicted ref_idx, and the reference obtained through the motion compensation. The block becomes a restored block. Here, B_pred_dir is a prediction direction of the B-picture, and means both directions of L0 and L1.

화소블록 단위로 움직임 추정 및 보상을 수행한다. 예를 들어, 8x8 다이렉트 모드의 경우, 4x4 화소 블록 단위로 움직임 추정 및 보상을 수행한다.The direct mode encodes only the flag indicating the direct mode and data related to transform coefficient encoding (for example, CBPX_flag / CBP, quantization parameter, transform type, transform coefficient, etc.). In other words, the direct mode is an encoding mode in which motion information is omitted, and the direct mode may be used for blocks of 8x8 pixel blocks or more, such as 8x8, 16x16, 32x32, and the like. In the case of NxN direct mode, motion estimation and compensation may be performed.

Motion estimation and compensation are performed in pixel block units. For example, in the 8x8 direct mode, motion estimation and compensation are performed in units of 4x4 pixel blocks.

L0 prediction means one direction prediction using only L0 motion information, and L1 prediction means one direction prediction using only L1 motion information. Bidirectional prediction means prediction using two motion information of L0 and L1 using a mean value of two pictures as a reference picture.

Motion estimation methods for B-pictures include motion estimation / compensation in direct prediction mode, motion estimation / compensation in SKIP mode, motion estimation / compensation in one-way prediction (L0 prediction or L1 prediction), and motion estimation in bidirectional prediction. / Compensation method.

First, the motion estimation / compensation method of the direct prediction mode is described. In the direct mode, the motion estimation / compensation method of the temporal direct prediction mode using the motion information correlation in the time direction, and the spatial direction (horizontal and vertical two-dimensional space) There are two methods of the motion estimation / compensation method in the spatial direct prediction mode using the motion information correlation of. The temporal direct prediction mode or the spatial direct prediction mode is used by selecting one slice unit rather than a macroblock unit or a block unit. In the direct prediction mode, a reference picture having the smallest reference number of L1 prediction is important and is called an anchor picture. In the display order, the nearest reference picture in the reverse direction of the target picture becomes an anchor picture, and a block at the same spatial location as the target block in the anchor picture is called an anchor block.

In the direct temporal prediction mode, first, the motion information of the anchor block is examined and the L0 motion vector of the anchor block is mvCol (MV of the Co-located block). If there is no L0 motion vector of the anchor block and it has an L1 motion vector, the motion vector of L1 is used as mvCol. The L0 reference picture in temporal direct prediction mode is a picture referred to by mvCol, and the L1 reference picture in temporal direct prediction mode is an anchor picture. When the anchor block does not have intra-picture coded motion information, the size of the motion vector is 0, and the reference picture number is 0 for the L0 reference picture in the temporal direct prediction mode. Further, the motion vector mvL0 of L0 and the motion vector mvL1 of L1 are obtained from the motion vector mvCol. Assuming that the moving speed between the reference picture, the target picture, and the anchor picture is constant, the sizes of mvL0 and mvL1 are proportional to the intervals of the reference picture, the target picture, and the anchor picture, and are thus obtained by proportional allocation. That is, mvL0 can be obtained by using Equation 5 using the interval tb between the reference picture and the target picture and the interval td between the reference picture and the anchor picture.

In addition, since mvL1 is in the same reverse direction as mvCol, it can be obtained using Equation 6.

Even in the spatial direct prediction mode, the anchor picture which is the reference picture of the smallest reference number in the L1 prediction and the anchor block at the same spatial location as the anchor picture are used to obtain the motion vector. In the spatial direct prediction mode, the motion vector of the target block is predicted from the motion vector encoded in the target picture. Even in the spatial direct prediction mode, the motion block information can be obtained by introducing a part of the concept of temporal direct prediction mode, i) the motion vector size of the anchor block is ± 1 pixel or less both horizontally and vertically, and ii) 0, iii) L0 motion vector mvL0 and L1 motion vector mvL1 are set to 0 when the reference picture with the minimum reference number at L0 satisfies the condition that both are short-term reference pictures, and the picture with the lowest reference number in L0 prediction is L0. The picture having the smallest reference number in the prediction picture and L1 is referred to as the L1 prediction picture. When such conditions are satisfied, the motion vectors mvL0 and mvL1 of the target block, and the L0 picture and the L1 picture, which are reference pictures, are obtained from the encoded motion vectors in the target picture.

In the L0 prediction, the median value of the horizontal and vertical components is calculated using the L0 prediction motion vectors mvA, mvB, and mvC located at the left, top, and left upper corners, and the value is mvL0. Similarly, in the L1 prediction, the median values of the horizontal and vertical components are calculated using the L1 prediction motion vectors mvA, mvB, and mvC of the left, top, and right upper corners, and the values are mvL1. If the neighboring block is a unidirectional prediction or an intra coded L0 prediction, or there is no motion vector of the L1 prediction, it is replaced with a zero vector. When the reference number is obtained, the minimum reference number of the motion vectors mvA, mvB, and mvC of the left, upper, and upper right corners is calculated, and the reference number of the NxN block is calculated. The process of obtaining this reference number is also performed independently of the L0 prediction and the L1 prediction. In addition, when the neighboring block is unidirectional prediction or when there is no motion vector of L0 or L1 prediction that is intra prediction coded, the block is excluded when obtaining a reference number. It should be noted that when performing the direct prediction mode of the NxN pixel block, the process of obtaining the mvL0, mvL1, L0 reference picture and the L1 reference picture is performed in units of NxN pixel blocks, but the prediction motion vector (mvL0) obtained through the neighboring block is performed. , mvL1) or 0 vector is determined by block unit.

Next, the motion estimation / compensation method of the SKIP mode will be described.

Although the SKIP mode performs motion prediction and compensation using the predicted motion vector, it is the same as the SKIP mode of the P-picture, but the motion vector used for the SKIP mode in the B-picture and the predicted motion vector of the Direct mode of the reference picture, Same as the reference picture. That is, the motion estimation / compensation method of the SKIP mode is the same as the motion estimation / compensation method of the aforementioned direct prediction mode. The difference between the direct mode and the SKIP mode is that the SKIP mode has no motion information and no conversion coefficient, and the direct mode has no motion information only.

Since the motion estimation / compensation method of one direction prediction (L0 prediction or L1 prediction) is the same as the motion estimation and compensation method of the P-picture except that the reference picture is one of L0 and L1, the description thereof is omitted.

Finally, a motion estimation / compensation method of bidirectional prediction will be described.

L0 reference pictures and mvL0 are obtained by using the motion estimation / compensation method of the above-described P-picture for pixel blocks having a size of 8x8 or more in a B-picture having motion information, and L0 reference pictures and mvL1 are performed. L1 prediction is performed. Also, the reference block indicated by mvL0 in the L0 reference picture obtained through the L0 prediction process and the average block of the reference block indicated by mvL1 in the L1 reference picture obtained through the L1 prediction process are used as reference blocks for mvL0 and mvL1. Bidirectional prediction can be performed. In addition, L1 prediction is performed on the motion information of L0 prediction to select the optimal mvL1 and L1 reference pictures for the mvL0 / L0 reference picture, and the reference block indicated by mvL1 in the selected L1 reference picture and the L0 prediction are obtained. Bidirectional prediction may be performed using an average block of the reference block indicated by mvL0 in the L0 reference picture as a reference block. In addition, by performing L0 prediction on the motion information of the L1 prediction, an optimal mvL0 and L0 reference picture is selected for the mvL1 / L1 reference picture, and the reference block indicated by mvL0 in the selected L0 reference picture and L1 prediction are obtained. Bidirectional prediction may be performed using an average block of the reference block indicated by mvL1 in the L1 reference picture as a reference block. Preferably, one bidirectional prediction mode having the best encoding efficiency is selected by comparing the encoding efficiency of the bidirectional prediction mode with respect to pixel blocks having a size of 8x8 or more in the B-picture having the motion information. For the pixel block having a size of 16x16 or larger, the motion information of the above-described bidirectional prediction modes is found, and then the transform, quantization, and entropy transform coefficient encoding are performed to determine the optimal encoding mode, and the actual coding bits and prediction errors required for the corresponding mode are determined. It is desirable to determine the bidirectional prediction mode using (Distortion).

For the L0 prediction, the L1 prediction, and the bidirectional prediction, each pixel block of 8x8 or more may use a different reference picture, but subblocks in the 8x8 pixelblock use the same reference picture.

Referring to FIG. 1, the subtractor 120 generates a residual block by subtracting a current block and a reference block indicated by the motion vector of the current block estimated by the motion estimator 116, and the generated residual block is a transformer and a quantizer ( After being transformed and quantized by 130, the quantized transform coefficients are entropy coded by the encoder 140 to generate encoded data. Here, since the conversion method for the B-picture is the same as the above-described conversion method for the P-picture, a detailed description thereof will be omitted.

FIG. 31 shows data to be transmitted through a bitstream of a B-picture (or B-slice) assuming that the expanded macroblock is a 32x32 pixel block. In general, the transmission syntax of the B-picture (or B-slice) is similar to the transmission syntax of the P-picture (or B-slice) described with reference to FIG. 30, and only a part added to the syntax of the P-picture is described here.

The directX_flag is a flag indicating whether the extended macroblock, the 16x16 pixel block or more, and the 8x8 pixel block are in the direct mode among the interblocks not in the SKIP mode.

In the direct mode, motion information is not encoded, and information related to transform coefficients is encoded. In addition, when the extended macroblock is not in the direct mode, data of motion information (for example, B_pdir, MVD, ref_idx, etc.) is encoded. Here, B_pdir is information indicating the prediction direction (L0 prediction, L1 prediction, bidirectional prediction) of the current block. The B_pdir is encoded only for inter blocks, not in SKIP or direct mode, and has three modes as shown in Table 9 below.

In the bidirectional prediction, both the motion information (mvL0, ref_idx_L0) for the L0 prediction and the motion information (mvL1, ref_idx_L1) for the L1 prediction are encoded, and only the motion information (mvL0, ref_idx_L0) for the L0 prediction is encoded in the L0 prediction. do. In addition, in the case of L1 prediction, only motion information mvL1 and ref_idx_L1 for L1 prediction is encoded.

32 is an exemplary diagram for describing an encoding process of a SKIPX flag according to an embodiment of the present invention.

In FIG. 32, when a macroblock is a 64x64 pixel block, a split type of a macroblock divided into subblocks of various sizes and shapes and an encoding order of each subblock are illustrated. 32, the SKIP16 flag is encoded for the 11th 16x16 pixel subblock, the SKIP16 flag of the 12th 16x16 pixel subblock is encoded, and the SKIP16 flag of the 15th 16x32 pixel subblock is encoded. The SKIP16 flag of the 16th 16x32 pixel subblock is encoded, the SKIP32 flag of the 17th 32x32 pixel subblock is encoded, and the SKIP32 flag of the 18th 32x32 pixel subblock is encoded in order.

For subblocks of a macroblock that are not SKIP macroblocks, refer to a differential motion vector (DMV), which is a difference vector between a motion vector of the subblock and a predicted motion vector estimated using the neighboring blocks of the subblock. Motion information including the picture index is encoded.

33 and 34 are exemplary diagrams for describing a CBPX_flag, a transform type, and a CBP encoding process for an inter macroblock according to an embodiment of the present invention.

33 and 34 illustrate a process of encoding a CBPX_flag, a transform type, and a CBP when the macroblock is a 64x64 pixel block. The encoder 140 of the image encoding apparatus 100 determines whether the size of the subblock of the extended macroblock is one of 64x64, 64x32, and 32x64 (S3302), so that the size of the subblock is 64x64, 64x32, and 32x64. If not, it is determined whether the size of the subblock is one of 32x32, 32x16, and 16x32.

As a result of the determination in step S3304, if the size of the subblock is one of 32x32, 32x16, and 16x32, it is determined whether there is a non-zero quantized transform coefficient in the corresponding subblock (S3306), and the non-zero quantized transform coefficient is determined. If present, the CBP32 flag (for example, may be represented by 1 bit such as '1') indicating that there is a non-zero quantized transform coefficient in the corresponding subblock (S3308).

After encoding the CBP32 flag in step S3308, it is determined whether there is any non-zero quantized transform coefficient in the 16x16 pixel block in the subblock (S3310), and if there is a non-zero quantized transform coefficient, 0 in the 16x16 pixel block. The CBP16 flag indicating that there is a quantized transform coefficient is encoded (S3312).

After encoding the CBPX flag as described above, the transform type is encoded only for blocks having a non-zero quantized transform coefficient (S3316). In this case, if the size of the subblock is 16x16 or more, the transform type is encoded for each subblock. If the size of the subblock is smaller than 16x16, one transform flag is encoded for each 16x16. This is because the conversion is selected in units of 16x16 pixels. For example, if a 64x64 pixel macroblock is divided into two 64x32 pixel subblocks and the CBP64 flag is nonzero, two CBP32 flags in each 64x32 pixel subblock encode a transform type for a 64x32 pixel subblock that is not zero. .

Here, when the size of the subblock is 16x16 or more and the CBPX flags belonging to the subblock are not all zeros, the transformation type of each subblock is encoded using Table 3.

In addition, when the CBP16 flag is not 0 and the 16x16 pixel block is divided into two 8x16 pixel blocks or 16x8 pixel blocks, the conversion type is encoded in units of 16x16 pixel blocks using Table 4.

If the CBP16 flag is not 0 and the 16x16 pixel block is divided into four 8x8 pixel blocks, if 4x4 transform is selected, 0 is encoded as the transform type; if 8x8 transform is selected, 1 is encoded as the transform type; otherwise Do not encode the conversion type.

After encoding the transform type in step S3316, the CBP is encoded according to the transform type. To this end, it is determined whether the transform type is a 16x16 transform (S3318). In the case of the 16x16 transform, only the CBP 2 bits of the chroma component are encoded (S3320). If the 16x16 transform is not, the transform type is an 8x16 transform. Alternatively, it is determined whether it is a 16x8 transform (S3322), and in the case of an 8x16 transform or a 16x8 transform, the CBP 2 bits of the luma component and the CBP 2 bits of the chrominance component are encoded (S3324), and the 8x16 or 16x8 transform is If not, it is determined whether the transform type is an 8x8 transform or a 4x4 transform (S3326). In the case of an 8x8 transform or a 4x4 transform, the CBP 4 bits of the luminance component and the CBP 2 bits of the chrominance component are encoded (S3328). In the steps S3320, S3324, and S3328, an example of encoding the CBP of the luminance component and the CBP of the chrominance component when the input video is in the 4: 2: 0 format has been described. Therefore, when the format of the input video is changed, the luminance component is changed accordingly. The method of encoding the CBP and the CBP of the chrominance component may vary.

On the other hand, when there is no non-zero quantized transform coefficient in the subblock as a result of the determination in step S3306, the CBP32 flag indicating that there is no non-zero quantized transform coefficient in the subblock is encoded (S3330). Since there are no nonzero quantized transform coefficients in the subblock, the transform type and the CBP are not encoded.

In addition, if the size of the subblock is one of 64x64, 64x32, and 32x64 as a result of the determination in step S3302, it is determined whether there is a non-zero quantized transform coefficient in the subblock (S3330), and the non-zero quantized transform. If there is a coefficient, a CBP64 flag indicating that there is a non-zero quantized transform coefficient in the corresponding subblock is encoded (S3334), and the flow proceeds to step S3306, and if there is no non-zero quantized transform coefficient, it is in the corresponding subblock. The CBP64 flag indicating that there is no non-quantized transform coefficient is encoded (S3336). Since there are no nonzero quantized transform coefficients in the subblock, the transform type and the CBP are not encoded.

As such, after encoding the CBPX (where X is an integer greater than or equal to 32) flag, transform type, and CBP, the encoder 140 of the image encoding apparatus 100 quantizes only blocks having a non-zero quantized transform coefficient. The encoded transform coefficients are encoded to generate encoded image data, and are assigned to the transform coefficient field. The encoding method of the quantized transform coefficients is the same as the encoding method of the quantized transform coefficients for the intra macroblock described above.

In the embodiment according to the present invention, CABAC (Context-based Adaptive Variable Length Code) may be used as a variable length coding method, and CABAC coding includes a binary arithmetic coding unit and a context calculating unit.

The binary arithmetic encoder is a binarizer that converts binary signals (signals of 0 and 1), and the context computation unit maintains a plurality of probability of generating binary signals to be encoded (context table) and binary according to the current encoding target and surroundings. The probability of occurrence of the signal is changed and transmitted to the binary arithmetic encoder. At this time, the context table is initialized at the start of slice encoding / decoding and updated in the slice. Each syntax has a different table (probability) according to ctxIdx (context number), and ctxIdx is obtained by equation (7).

[Equation 7]

ctxIdx = ctxIdx_offset + ctx_inc

Here, ctx_offset is determined by picture type and syntax type. At this time, although it is preferable to use different tables according to the X value, interX_flag may use the same table. ctx_inc is a value determined according to surrounding conditions when encoding the target syntax, and examples of ctx_inc for each syntax are shown in FIGS. 35 to 37. For example, in the case of skipX_flag, as shown in FIG. 35, ctx_inc = 0 indicates that both the upper and left blocks are not in SKIP mode, and ctx_inc = 1 indicates that the upper or left block is in the SKIP mode. In this case, ctx_inc = 2 may represent a case where both the upper block and the left block are in SKIP mode. The context table can be used differently for each ctx_inc. Similarly, in the case of interX_flag, as shown in FIG. 36, an example of prediction type of a context table for an extended macroblock that is 32x32 pixel blocks when ctx_inc is 0 to 5 is 16x16 pixels when ctx_inc is 7 to 9 An example of a prediction type of a context table for a block is shown. The context table may be used differently according to each ctx_inc. Similarly, ctx_inc for encoding / decoding of directX_flag is as shown in FIG. 37. As described in Table 8, the encoding bit of directX_flag may be represented by 1 bit, and the upper block and the left block of the target block to be encoded are not in the direct mode, the both are in the direct mode, and the upper block or the left block. The context table can be determined differently according to the case where only one block is in direct mode. Also, in the case of ctx_inc for encoding / decoding of extended_flag, similarly to FIG. 37, whether encoding is performed in units of macroblocks in which neighboring blocks of the target block to be encoded are extended (in this case, extended_mb_flag may be set to binary bit 1). Alternatively, the context table may be used differently depending on whether the encoding is performed by being divided into 16 × 16 pixel blocks (in this case, extended_mb_flag may be set to binary bit 1). As such, in the case of interX_flag, a different context table may be used according to a value of X, a picture type, whether it is used as a reference picture, and data of neighboring blocks.

38 and 39 are exemplary diagrams for describing a process of encoding split type information according to an embodiment of the present invention.

FIG. 38 illustrates a partition type value for each 16x16 pixel block in an extended macroblock to be encoded in order to encode partition type information using a tree structure.

In 38A, Mx (a, b) represents a partition type value of a 16x16 pixel block corresponding to the position of (a, b) in the extended macroblock to be encoded. That is, Mx (0,0) represents the position of (0,0) in the extended macroblock to be encoded, that is, the partition type value of the first 16x16 pixel block in the raster scan direction in the extended macroblock to be encoded. Mx (0,1) represents the position of (0,1) in the extended macroblock to be encoded, that is, the partition type value of the second 16x16 pixel block in the raster scan direction in the block to be encoded.

The division type values for the 16x16 pixel block shown in 38A are grouped into a certain area unit, for example, an area unit including KxL division type values, and the minimum value among the division type values in the grouped area is selected for the grouped area. Assign as a partition type value. If the split type value for the grouped region is Mx-1 (c, d), then c = a / K and d = b / L. Here, Mx-1 is an upper node of Mx, and the largest value may be selected by selecting a value of the upper node. In addition, a method of selecting an upper node value may be encoded and used adaptively.

38B groups the partition type values for the 16x16 pixelblocks shown in 38A into area units including 2x2 partition type values, and assigns the minimum value of the partition type values in each group to the partition type values for the grouped areas. The results are shown. That is, the division type values of the pixel blocks including the 16x16 pixel blocks (0,0), (0,1), (1,0), and (1,1) shown in 38A are grouped and grouped into 2x2 area units. Select and group the minimum value among the partition type values M _X (0,0), M _X (0,1), M _X (1,0), and M _X (1,1) of each of the 16x16 pixel blocks included in the included region. The partition type value M _{X - 1} (0,0) for the allocated area. Again, the division type values of the pixel blocks including the 16x16 pixelblocks (0,2), (0,3), (1,2), and (1,3) are grouped into 2x2 area units, and within the grouped area. The minimum value among the division type values M _X (0,2), M _X (0,3), M _X (1,2), and M _X (1,3) of each of the 16x16 pixel blocks included in the grouped region. A partition type value M _{x -1} (0,1) was assigned. In the same manner, the same process is performed on the remaining 16x16 pixel blocks. The process of allocating the partition type values of the 16x16 pixel blocks shown in 38A to the partition type values of the grouped areas shown in 38B is performed by M _{X -2} , M _{X -3} . Repeatedly performing up to M ₁ layer can assign the segmentation type value of the grouped region like 36C, and repeating up to M ₀ layer again grouped to have the segmentation type value of one grouped region like 38D. can do.

In FIG. 38, the division type values for adjacent 16x16 pixel blocks are grouped into 2x2 area units, but the present disclosure is not limited thereto, and 16x16 pixel blocks that are adjacent to each other in various sizes such as 4x2 and 2x4 areas. Subdivision type values may be assigned by grouping the subdivided type values with respect to the subdivided type values, or by grouping the subdivided type values with respect to non-adjacent 16 × 16 pixel blocks in a predetermined area unit.

As shown in FIG. 38, the allocation of the partition type values of the 16x16 pixel blocks of the block to be encoded as the partition type values of the grouped regions may be represented in a tree structure as illustrated in FIG. 39.

39 exemplarily illustrates a split type value for regions grouped by layers in a tree structure. Assuming that the split type value for the layered region for each layer is a node value, the node values of the tree structure illustrated in FIG. 39 may be encoded by encoding a difference value with a node value of an upper node. have.

A method of encoding a difference value between node values and a node value of an upper node encodes binary bit 0 by the difference value and encodes binary bit 1 at the end. If the difference between the node value of the node to be currently encoded and the node value of the higher node is 0, binary bit 1 is encoded. When encoding binary bits of 0 and 1, for example, an arithmetic coding method may be used, and a different context may be used for each layer.

As described above, when encoding a split type value using a tree structure, that is, a node value, a node value (hereinafter, referred to as a 'top node value') of a top node has a difference value between 0 and 0 as described above. It can be encoded using binary bits. Alternatively, in the case of allocating the partition type number described with reference to FIG. 18 when the large partition type number is assigned in the order of high frequency of occurrence, the highest node value uses a binary bit of 0 and 1 as a difference value from the largest partition type number. Can be encoded. Alternatively, in the case of allocating the partition type numbers described with reference to FIG. 18, when the partition type numbers are assigned in the order of high frequency of occurrence, the highest node value uses a binary bit of 0 and 1 as a difference value from the smallest partition type number. Can be encoded.

The remaining node values except for the highest node may be encoded using binary bits 0 and 1 between a node value of a node to be encoded and a node value of an upper node of the corresponding node. That is, each node value may be encoded by encoding the number of binary bits 0 as much as the difference value, and finally encoding the binary bit 1. If the difference between the node value of the node to be encoded and the node value of the higher node is 0, binary bit 1 is encoded. In addition, on the contrary, each node value may be encoded by encoding as many binary bits 1 as the difference value and encoding binary bit 0 at the end, and when the difference value is 0, the binary bit 0 may be encoded. .

However, when the upper node value is the maximum value that the split type number may have when encoding each node value, the node values of the lower nodes of the upper node are not encoded. For example, if the node value of node M ₁ (0,0) is 3, M ₂ (0,0), M ₂ (0,1), and M ₂ which are subordinate nodes of node M ₁ (0,0) The node values of the (1,0) and M ₂ (1,1) nodes are not encoded. That is, M ₁ (0,0) is the minimum of M ₂ (0,0), M ₂ (0,1), M ₂ (1,0), and M ₂ (1,1), so M ₂ (0,0) ), M ₂ (0,1), M ₂ (1,0), and M ₂ (1,1) all have a value of 3 or more. However, since the maximum value of the split type value in FIG. 18 is 3, M ₂ (0,0), M ₂ (0,1), M ₂ (1,0), and M ₂ (1,1) are different from 3. There is no need to encode because it cannot have a value.

Also, when encoding the difference value between the node to be encoded and the higher node, if the node value of the current node to be encoded is the maximum value that the partition type number can have, only the number of binary bits 0 as many as the difference value is encoded. Does not encode binary bit 1 at the end. For example, node values M ₁ (0,0) of the upper node of the current node to be encoded and node values M ₂ (0,0), M ₂ (0,1), M ₂ ( 1,0), assuming that M ₂ (1,1) is 1, 2, 3, 3, 2, respectively, the node values M ₂ (0,0) and M ₂ (1,1) encode binary bits 01 The node values M ₂ (0,1) and M ₂ (1,0) encode 00 instead of binary bit 001.

In addition, in the case of encoding the node value of the last node among the nodes having the same higher node, the node value of the last node is not encoded when the node values of the nodes except the last node are all larger than the node value of the upper node. For example, the node value M ₁ (0,0) of the higher node of the node to be encoded and the node value M ₂ (0,0), M ₂ (0,1), M ₂ (1, 0), assuming that M ₂ (1,1) is 1, 2, 3, 3, 1, respectively, node values M ₂ (0,0), M ₂ (0,1), M ₂ (1,0) Since both of these are larger than the node value M ₁ (0,0), the node value M ₂ (1,1) of the last node is not encoded.

Meanwhile, the remaining node values except for the top node may be encoded by encoding the difference between the node value of the node to be encoded and the node value of the upper node of the corresponding node using binary bits 0 and 1 as described above. Alternatively, it may be encoded by encoding a difference value between the node value of each node and the partition type value having the highest frequency of occurrence of the partition type. Here, the split type value having the highest frequency may or may not use a fixed value. If it is not a fixed value, it may be encoded and transmitted to the decoder or not. If not, the mode with the highest frequency of occurrence may be used by accumulating the statistics of the encoded blocks before the current block. In another embodiment of encoding the partition type information using a tree structure, when the partition type values of the grouped regions of the 38B are determined by grouping the partition type values of the subblocks shown in 38A, a partition type number having a high frequency in order of occurrence. If is assigned, the maximum value among the values of the grouped area may be used as the split type value of the grouped area.

In an embodiment of the present invention, when the extended macroblock is divided into 16x16 pixel blocks by using a split type value as shown in FIG. 18 (that is, when extended_mb_flag = 0), the split type is encoded using the aforementioned method. / Decrypt In this case, the value of the lowest node is a partition type value of the 16x16 pixel block in the extended macroblock, and the maximum value of the values of the lower nodes may be used as the representative value of the upper node. For example, as shown in FIG. 40, when an extended macroblock of a 32x32 pixel block belongs to a 16-padded image and is divided into 16x16 pixel blocks, the encoding of the most significant node is a partition type because the representative value of the most significant node is 0. The binary bit '000' is coded to represent 3, which is the difference between the maximum value 3 and the representative value 0. As another example, as shown in FIG. 41, an extended macroblock of a 32x32 pixel block belongs to an image padded with 16, three 16x16 pixel blocks are divided into 8x8 pixel blocks or less, and one 16x16 pixel block is encoded as a 16x16 pixel block. In the case of coding with, since the representative value of the most significant node is 3, the binary bit 1 representing the difference between the representative value 3 of the most significant node and the maximum value 3 of the split type is encoded, and the difference between the values of the lower nodes and the upper node is encoded. If the value is 0, the binary bit '1' is encoded. If the difference is 3, the binary bit '000' is encoded.

As described in FIGS. 6 to 10, encoding of the intra prediction mode encodes a 1-bit flag indicating whether the intra prediction mode of the 4x4 and 8x8 pixel blocks of the luminance component is the same as the prediction intra mode predicted using the neighboring block, When the prediction intra mode is different, 3 bits are used to indicate which of the eight prediction modes minus the prediction direction among the nine prediction modes. In addition, the 16x16 pixel block is represented by using two bits of which of the four prediction modes is used. In addition, the intra prediction mode of the remaining mxn pixel blocks except for the 4x4 pixel block, the 8x8 pixel block, and the 16x16 pixel block encodes a 1-bit flag indicating whether the prediction intra mode is the same as that predicted using the neighboring block, and when the intra prediction mode is different from the prediction intra mode. 1 bit is used to indicate which of the two prediction modes except the predicted prediction direction among the three prediction modes. In addition, the 16x16 pixel block of the color difference component indicates which of four prediction modes is used using two bits. Here, it is preferable to use different context tables for the 4x4, 8x8, 16x16, and mxn pixel blocks of the luminance component and the chrominance component.

42 shows an example of ctx_inc for encoding / decoding of bidirectional prediction. As described above, ctx_inc is a value determined according to surrounding conditions when encoding target syntax, and the encoding bits of B_pdir indicating bidirectional prediction are as described in Table 9. When the neighboring block that is the L0 prediction is 0, the neighboring block that is the L1 prediction is 1, and the neighboring block that is the bidirectional prediction is 2, the target syntax may be determined as illustrated in FIG. 42 according to the surrounding situation. In this case, when the left or the upper block is not available, the L0 prediction (0) may be used in the prediction direction of the unavailable block. It is preferable to use a different context table for each ctx_inc as illustrated in FIG. 43.

Ctx_inc for encoding / decoding of CBPX_flag may determine a context table similarly to the case of FIG. 37. For example, if CBPX_flag is not encoded when an intra picture or an extended macroblock is an intra mode, and a case where 0 inter mode is 1 when the upper block or the left block of the current block to be encoded is intra mode is 1, The target syntax can determine and use the context table according to the surrounding situation.

43 is a diagram illustrating an example of ctx_inc for encoding / decoding of a transform type. 43 shows an example in which there are two types of transforms that can be used. As described in Table 1, when a 16x16 pixel block is divided into four 8x8 pixel subblocks, there are two types of selectable transforms, 4x4 transform and 8x8 transform, so that the target syntax includes both the top block and the left block of the current block. In the case of 4x4 transform, all 8x8 transform, or one block is 4x4 transform and the other block is 8x8 transform, the context table may be determined and used differently.

44 and 45 are diagrams illustrating another example of ctx_inc for encoding / decoding a transform type when three types of transforms are available, respectively. FIG. 44 is a diagram illustrating selecting a context table used for first bit encoding. 45 illustrates an example of ctx_inc. FIG. 45 illustrates an example of ctx_inc for selecting a context table used for second bit encoding.

As described above, when the pixel block smaller than the 8x8 pixel block is divided into 4x4 pixel blocks, 4x8 pixel blocks, or 8x4 pixel blocks, 4x4 transform is performed, and the transform having the smaller encoding cost among the 8x8 pixel block 4x4 transform and the 8x8 transform is performed. The 8x16 pixel block performs the transform with the lowest coding cost during the 4x4 transform, the 8x8 transform, and the 8x16 transform, and the 16x8 pixel block performs the transform with the lower encoding cost during the 4x4 transform, the 8x8 transform, and the 16x8 transform, and the 16x16 pixel. Since a block larger than a block performs a transform having a small encoding cost among 4x4 transform, 8x8 transform, and 16x16 transform, as shown in FIG. 44, in the first bit encoding, whether the upper block and the left block of the current block are 4x4 transformed blocks or the left block. Encoding of the transform type according to whether only one of the block and the upper block is a 4x4 transformed block or not all 4x4 transformed blocks You can use different ctx_inc for luxury. Similarly, in the second bit encoding, as shown in FIG. 45, ctx_inc for encoding / decoding of a transform type may be different depending on whether at least one of the upper and left blocks of the current block is an 8x8 transformed block. .

46 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention.

The image decoding apparatus 4400 according to an embodiment of the present invention includes a decoder 4410, an inverse quantizer and an inverse transformer 4420, a predictor 4430, an adder 4440, a filter 4450, and a picture buffer. 4440.

The decoder 4410 decodes the encoded data encoded by the image encoding apparatus 100 using various encoding or decoding techniques such as entropy decoding. The decoder 4410 has different data to decode depending on the type of picture to be decoded. The information about the picture type may be extracted from the picture header or the sequence header of the encoded data, or may be extracted and decoded and restored. In this case, when the extended macroblock to be decoded currently spans the image boundary as shown in FIG. 2 (the actual size of the input image is transmitted as a sequence parameter), a part belonging to the image in the extended macroblock is obtained and Text data and prediction modes can be obtained by performing entropy decoding, and intra prediction, motion compensation, and inverse transform / inverse quantization are performed to decode a part belonging to an actual image.

When the picture to be decoded is an intra picture, the type and order of data to be decoded in the image encoded data by the decoder 4610 are as illustrated in FIG. 17. That is, the decoder 4410 restores partition type information by decoding the partition type data extracted from the input bitstream, partitions the macroblock using the partition type identified by the recovered partition type information, and intra prediction. By decoding the mode data, the intra prediction mode is restored by the number of subblocks divided in the macroblock, and the CBP data allocated to the CBPX flag / CBP in the same or similar order as shown in FIG. 24 according to the size of each subblock. Decrypt to restore the CBPX flag or CBP. Subsequently, the decoder 4610 restores the transform coefficients by decoding transform coefficient data only for subblocks having non-zero transform coefficients indicated by CBP. In this case, when the image encoding apparatus 100 performs intra prediction encoding on the extended macroblock in units of 16 × 16 pixel blocks, decoding of the CBPX flag may be omitted. In this case, the quantization parameter encoded in the extended macroblock unit may be omitted. Decryption may be performed.

However, the syntax of the video encoded data shown in FIG. 17 is merely an example, and the syntax structure of the video encoded data according to an embodiment of the present invention is not limited to the structure shown in FIG. 17. In addition, the coding order of the CBPX flag or CBP shown in FIG. 24 is merely an example, and the order of reconstruction of the CBPX flag or CBP according to an embodiment of the present invention is not limited to the order shown in FIG. 24.

Here, how the decoder 4610 reconstructs the intra prediction mode depends on the size of the subblock. That is, when the size of the subblock is not 16x16, such as 4x4 or 8x8, the decoder 4610 decodes 1 bit of intra prediction mode data extracted from the bitstream, restores the prediction mode flag, and restores the prediction mode flag. When the prediction direction according to the intra prediction mode of the corresponding subblock is the same as the prediction direction according to the intra prediction mode of the corresponding subblock estimated from the neighboring block, the intra prediction mode of the corresponding subblock estimated from the neighboring block is Reconstruct the intra prediction mode as the intra prediction mode of the current block by reconstructing as the intra prediction mode of the block, decoding the intra prediction mode of the next subblock, and decoding the next 3 bits of the intra prediction mode data if it is not equal. do. In addition, when the size of the subblock is 16x16, the decoder 4610 reconstructs the intra prediction mode of the current block by decoding two bits of the intra prediction mode data extracted from the bitstream. The intra prediction mode reconstructed as described above is input to the intra predictor 4452 of the predictor 4630.

In addition, when the extended macroblock, which is a 32x32 pixel block, is an intra mode, after determining a block belonging to an image padded by a multiple of 16 pixels in the expanded macroblock, the partition type of the corresponding block may be entropy decoded. In the case of FIG. 3, only a first 16x16 pixel block corresponds to a block belonging to a padded image in an extended macroblock, which is a 32x32 pixel block, and thus entropy-decodes the partition type of the first 16x16 pixel block. If it is assumed that the corresponding 16x16 pixel block is divided into two 16x8 pixel blocks, two intra prediction modes are entropy decoded, and one CBP and 16x16 transform coefficients for the 16x16 pixel block are decoded. The 16x16 pixel block is decoded using the subblock type obtained through entropy decoding, an intra prediction mode, and a transform coefficient.

When the picture to be decoded is an inter picture, the type and order of data to be decoded in the image encoded data by the decoder 4610 are as shown in FIG. 29. That is, the decoder 4610 decodes SKIPN flag data extracted from the bitstream and restores the SKIPN flag indicating whether the macroblock to be decoded is the SKIP macroblock. When the macroblock identified by the SKIPN flag to be recovered is not a SKIP macroblock, the decoder 4610 decodes the macroblock type flag data and the partition type data extracted from the bitstream to decode the macroblock type flag and the partition type information. Restore

In addition, when the extended macroblock is a 32x32 pixel block, after determining a block belonging to a padded image in multiples of 16 pixels in the extended macroblock, the extended macroblock is decoded and the extended macroblock to be currently decoded corresponds to a SKIP mode. By using the neighboring blocks on the left and top of the block, a prediction motion vector may be obtained and motion compensation may be performed. In the case of FIG. 3, the block belonging to the padded image in the extended macroblock, which is a 32x32 pixel block, corresponds to only the first 16x16 pixel block, and is the first case when the extended macroblock to be decoded and currently decoded skip32_flag is SKIP mode. The prediction motion vector may be obtained using the neighboring blocks at the left and the top of the first 16 × 16 pixel block, and motion compensation may be performed.

When the macroblock type identified by the macroblock type flag to be restored is an intra macroblock, the same as described above. When the macroblock type is an inter macroblock, the decoder 4610 may determine a subblock having a size of 16x16 or more. MVD for subblocks (not non-SKIP) that are smaller than 16x16 or subblocks other than the SKIP block identified by the SKIPX flag being recovered by decoding the SKIPX flag data by decoding only the blocks. Decode / ref_idx data to reconstruct the motion vector and the reference picture index which are motion information. In addition, the decoder 4610 decodes blocks other than SKIP in the same or similar order as shown in FIGS. 33 and 33 to restore the CBPX flag, the transformation type, and the CBP, and zeros identified by the restored CBP. Reconstruct the quantized transform coefficients only for subblocks having coefficients other than. The reconstructed motion information is input to an inter predictor 4452 (which can be implemented as a motion compensator), and the reconstructed CBPX flag, CBP, transform type, quantized transform coefficient, etc. is a target macroblock or a target macroblock to be decoded. A subquantizer and an inverse transformer are input to the inverse quantizer 4620 to recover the remaining blocks of the lock.

The inverse quantizer and inverse transformer 4620 inverse quantizes and inverse transforms the quantized transform coefficients delivered from the decoder 4610 to restore a residual block. In this case, the inverse quantizer and inverse transformer 4620 may inverse quantize and inverse transform the quantized transform coefficients using the CBPX flag, the CBP, and the transform type transmitted from the decoder 4610.

The predictor 4630 generates a predictive block by predicting a target macroblock or subblock thereof to be decoded. To this end, the intra predictor 4452 generates the predictive block of the target macroblock or its subblocks by intra prediction of the target macroblock or subblock thereof to be decoded using the intra prediction mode reconstructed by the decoder 4610. The inter predictor 4474 generates a prediction block of the target macroblock or its subblocks by inter-prediction of the target macroblock or its subblocks to be decoded using the motion information reconstructed by the decoder 4610.

The adder 4640 reconstructs the target macroblock to be decoded by adding the predicted block generated by the predictor 4630 and the remaining blocks reconstructed by the inverse quantizer and the inverse transformer 4620.

The target macroblock reconstructed by the adder 4640 is deblocked filtered by the filter 4650, accumulated in picture units, and output as a reconstructed image, and is stored in the picture buffer 4660 to be later inter predicted 4452. It can be used to predict motion compensation in.

The operations of the intra predictor 4452, the inter predictor 4462, the inverse quantizer, and the inverse transformer 4620 may be performed by using the intra predictor 112, the inter predictor 114, the inter predictor 114 of the image encoding apparatus 100 described above with reference to FIG. 1. Since the inverse quantizer and the inverse transformer 150 are the same as or similar to the operating method, detailed description thereof will be omitted.

The video decoding apparatus 4600 may be configured as follows according to whether the extended macroblock is an inter macroblock or an intra macroblock.

For example, when the macroblock is an intra macroblock, the decoder 4610 decodes the encoded data to restore partition type information, an intra prediction mode, and a quantized transform coefficient, and the inverse quantizer and inverse transformer 4620 may be used. The quantized transform coefficients are inversely quantized and inversely transformed using a transform selected according to the size of the subblock identified according to the reconstructed partition type information, and the predictor 4630 reconstructs the remaining subblocks according to the reconstructed partition type information. A predicted subblock is generated by predicting the identified subblock using the reconstructed intra prediction mode, and the adder 4640 adds the predicted subblock and the remaining subblock to reconstruct the subblock.

As another example, when the macroblock is an inter macroblock, the decoder 4610 decodes the encoded data to restore partition type information, motion information, transform type information, and quantized transform coefficients, and inverse quantizer and inverse transform 4460. ) Reconstructs the remaining subblocks by inverse quantization and inverse transformation of the quantized transform coefficients using the transform identified by the transform type information to be recovered, and the predictor 4630 reconstructs the subblock identified according to the recovered partition type information. The predicted subblock is generated by using the predicted motion information, and the adder 4640 reconstructs the subblock by adding the predicted subblock and the remaining subblock.

According to an image decoding method according to an embodiment of the present invention, when the macroblock to be decoded is an intra macroblock, the image decoding apparatus 4600 decodes the encoded data to divide partition type information, an intra prediction mode, and a quantized transform coefficient. Restore the remaining subblocks by inverse quantization and inverse transformation of the quantized transform coefficients using a transform selected according to the size of the subblock identified according to the partition type information to be restored, and identify the partition according to the partition type information to be recovered. A predicted subblock is generated by predicting the subblock using the reconstructed intra prediction mode, and the subblock is reconstructed by adding the predicted subblock and the remaining subblock.

In this case, the transform selected according to the size of the subblock uses a 4x4 transform if the size of the subblock is one of 4x4, 4x8, and 8x4, and uses an 8x8 transform if the size of the subblock is one of 8x8, 8x16, and 16x8. If the size of the subblock is 16x16 or more, 16x16 transform can be used.

In addition, according to an image decoding method according to an embodiment of the present invention, when the macroblock to be decoded is an inter macroblock, the image decoding apparatus 3000 decodes the encoded data to thereby split type information, motion information, and transform type information. And reconstructing the quantized transform coefficients, inversely quantizing and inversely transforming the quantized transform coefficients by using the transform identified by the transformed type information to be recovered, and restoring the remaining subblocks, and the subblocks identified according to the recovered partition type information. The prediction subblocks are generated by partitioning the subblocks and predicting the subblocks using the reconstructed motion information, and reconstructing the subblocks by adding the prediction subblocks and the remaining subblocks.

In this case, the transform identified by the transform type information may be one of a 4x4 transform, an 8x8 transform, an 8x16 transform, a 16x8 transform, and a 16x16 transform. Also, the image decoding apparatus 4600 may filter the reconstructed subblock, and may perform deblocking filtering on the left and upper boundaries of the block according to the selected transform in the reconstructed subblock.

The image decoding apparatus 4600 may further reconstruct the SKIP_N flag indicating whether the macroblock is a SKIP macroblock by decoding the SKIP_N flag data extracted from the encoded data. The video decoding apparatus 4600 may perform a decoding process only on the macroblock identified as not the SKIP macroblock by the reconstructed SKIP_N flag.

The image decoding apparatus 4600 may further reconstruct the macroblock type flag indicating whether the macroblock is an intra macroblock or an inter macroblock by decoding the macroblock type flag data extracted from the encoded data. The image decoding apparatus 4600 determines whether the macroblock to be decoded is an intra macroblock or an inter macroblock based on the reconstructed macroblock type flag, and accordingly, whether to perform intra prediction decoding or inter prediction decoding. Can be determined.

On the other hand, regardless of the type of the macroblock, the split type information may be identified by the split type number for each split layer, and the image decoding apparatus 4600 may decode CBP data extracted from the encoded data and according to the size of the subblock. You can restore additional CBPX flags and CBP. Here, the CBPX flag may be a flag indicating whether there is a non-zero quantized transform coefficient in the XxX pixel block, and the CBP may be a flag indicating whether there is a non-zero quantized transform coefficient in the 16x16 pixel block.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. In other words, within the scope of the present invention, all of the components may be selectively operated in combination with one or more. In addition, although all of the components may be implemented in one independent hardware, each or all of the components may be selectively combined to perform some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like.

In addition, the terms "comprise", "comprise" or "having" described above mean that the corresponding component may be inherent unless specifically stated otherwise, and thus excludes other components. It should be construed that it may further include other components instead. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. Terms used generally, such as terms defined in a dictionary, should be interpreted to coincide with the contextual meaning of the related art, and shall not be interpreted in an ideal or excessively formal sense unless explicitly defined in the present invention.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

100: video encoding apparatus
110: predictor 120: subtractor
130: converter and quantizer 140: encoder
150: inverse quantizer and inverse converter 160: adder
170: filter 180: picture buffer
4600: video decoding device
4610: decoder 4620: inverse quantizer and inverse transformer
4630: Predictor 4632: Intra Predictor
4634: inter predictor 4640: subtractor
4650: filter 4660: picture buffer

Claims (52)

In the video encoding / decoding device,
A prediction is performed by predicting a subblock in which a macroblock is divided to generate a prediction subblock, subtracting the subblock and the prediction subblock to generate a residual subblock, and converting the residual subblock according to the size of the subblock. An image encoder for generating quantized transform coefficients by transforming and quantizing the quantized transform coefficients, and generating encoded image data by encoding the quantized transform coefficients; And
Decode the encoded image data to restore partition type information, motion information, transform type information, and quantized transform coefficients, and dequantize the quantized transform coefficients using transforms identified by the restored transform type information. Reconstruct the residual subblocks by inverse transform, generate a prediction subblock by predicting the subblocks identified according to the reconstructed partition type information using the reconstructed motion information, and add the prediction subblocks and the residual subblocks. An image decoder to reconstruct the subblock
Image encoding / decoding apparatus comprising a. In the apparatus for encoding a video,
A predictor for predicting a subblock in which the macroblock is divided to generate a predicted subblock;
A subtractor for generating a residual subblock by subtracting the subblock and the prediction subblock;
A transform and quantizer for transforming and quantizing the residual subblocks using a transform selected according to the size of the subblock to generate quantized transform coefficients; And
An encoder that encodes the quantized transform coefficients to generate encoded image data
An image encoding apparatus comprising a. The method of claim 2, wherein the predictor is
For the macroblock having the size of NxN (where N is an integer of 16 or more), the macroblock layer K (

)of

And a subblock of the macroblock layer K + 1 when the pixel block is divided into four blocks. The method of claim 2, wherein the predictor is
And predicting a subblock by dividing the macroblock having a size of NxN (where N is an integer greater than or equal to 16) pixel blocks in units of 16x16 pixel blocks when N is larger than 16. The method of claim 2,
When the macroblock is an intra macroblock, the subblock is predicted in units of mxn pixels, wherein m and n are one of 4, 8, and 16, which are not equal to each other, or greater than 16, and N (N is an N × N pixel block. And a pixel unit of a macroblock having a size of?). The method of claim 2,
And the macroblock includes at least one of 64x64, 32x64, 64x32, 32x32, 16x32, and 32x16 pixel blocks. The method of claim 2, wherein the transform and quantizer,
When the macroblock is an intra macroblock, if the size of the subblock is one of 4x4, 4x8, and 8x4, 4x4 transform is used; if the size of the subblock is one of 8x8, 8x16, and 16x8, 8x8 transform is used. And 16x16 transform if the size of the subblock is 16x16 or larger. The method of claim 2, wherein the transform and quantizer,
And when the extended macroblock is an intra macroblock, transform type data is generated by encoding a transform type indicating the type of the selected transform. The method of claim 2, wherein the transform and quantizer,
The remaining subblocks are divided into 16x16 pixel residual blocks, the divided 16x16 pixel residual blocks are further divided into 16 4x4 pixel residual blocks, and each 4x4 pixel residual block is transformed using 4x4 transform to convert 16 transform blocks. And a DC component block composed of DC components of the 16 transform blocks using 4x4 transform. The method of claim 9,
And a 4x4 transform for transforming the 4x4 pixel residual block and a 4x4 transform for transforming the DC component block use different transform schemes. The method of claim 2, wherein the transform and quantizer,
And converting the residual subblock by selecting one of 4x4 transform, 8x8 transform, 8x16 transform, 16x8 transform, and 16x16 transform when the macroblock is an inter macroblock. The method of claim 2, wherein the transform and quantizer,
When the size of the subblock is smaller than 8x8, the residual subblock is transformed using 4x4 transform. When the size of the subblock is 8x8, the encoding cost of the residual subblock is smallest among 4x4 transform and 8x8 transform. If the size of the subblock is 8x16, the residual subblock is transformed using a transform having the lowest coding cost among 4x4, 8x8, and 8x16 transforms, and the size of the subblock is 16x8. In case of, the residual block of the subblock is transformed using a transform having the smallest coding cost among 4x4 transform, 8x8 transform, and 16x8 transform. And the 8x8 transform and the 16x16 transform are transformed using the transform having the lowest encoding cost. The method of claim 2, wherein the transform and quantizer,
When the size of the subblock is smaller than 8x8, the residual subblock is transformed using 4x4 transform. When the size of the subblock is 8x8, the encoding cost of the residual subblock is smallest among 4x4 transform and 8x8 transform. If the size of the subblock is 8x16, the residual subblock is transformed using a transform having the lowest coding cost among 4x4, 8x8, and 8x16 transforms, and the size of the subblock is 16x8. In case of, the residual block of the subblock is transformed using a transform having the smallest coding cost among 4x4 transform, 8x8 transform, and 16x8 transform. And the 8x8 transform and the 16x16 transform are transformed using the transform having the lowest encoding cost. The method of claim 2, wherein the transform and quantizer,
And when the extended macroblock is an inter macroblock, transform type data is generated by encoding a transform type indicating the type of the selected transform. In the apparatus for decoding an image,
A decoder which decodes the encoded data and restores partition type information, motion information, transform type information, and quantized transform coefficients;
An inverse quantizer and an inverse transformer for restoring residual subblocks by inverse quantization and inverse transformation of the quantized transform coefficients using a transform identified by the recovered transform type information;
A predictor for predicting a subblock identified according to the reconstructed partition type information using the reconstructed motion information to generate a predicted subblock; And
An adder for reconstructing the subblock by adding the prediction subblock and the residual subblock
Video decoding apparatus comprising a. The method of claim 15, wherein the inverse quantizer and inverse transformer,
If the size of the subblock is one of 4x4, 4x8 and 8x4, 4x4 transform is used.If the size of the subblock is one of 8x8, 8x16 and 16x8, 8x8 transform is used. If the size of the subblock is 16x16 or more, 16x16 transform is used. And a video decoding apparatus. In the apparatus for decoding an image,
A decoder which decodes the encoded data to restore partition type information, motion information, transform type information, and quantized transform coefficients;
An inverse quantizer and an inverse transformer for reconstructing the remaining subblocks by inverse quantization and inverse transformation of the quantized transform coefficients using a transform identified by the recovered transform type information;
A predictor for predicting a subblock identified according to the reconstructed partition type information by using the reconstructed motion information to generate a predicted subblock; And
An adder for reconstructing the subblock by adding the prediction subblock and the residual subblock
Video decoding apparatus comprising a. The method of claim 17, wherein the inverse quantizer and inverse transformer,
And a 4x4 transform, an 8x8 transform, an 8x16 transform, a 16x8 transform, and a 16x16 transform. The method of claim 17,
A filter for performing deblocking filtering on a boundary between a left side and an upper side of the block according to the selected transform in the reconstructed subblock;
The video decoding apparatus further comprises. The method of claim 17, wherein the decoder,
And decoding the SKIP_N flag data extracted from the encoded data to restore a SKIP_N flag indicating whether the macroblock is a SKIP macroblock. The method of claim 17, wherein the decoder,
And decoding a macroblock type flag data extracted from the encoded data to restore a macroblock type flag indicating whether the macroblock is an intra macroblock or an inter macroblock. The method of claim 17,
A video decoding apparatus characterized by being identified by a partition type number for each partition layer. The method of claim 17,
CBPX flag and CBP according to the size of the subblock are decoded by decoding CBP data extracted from the encoded data, wherein the CBPX flag is a flag indicating whether there is a non-zero quantized transform coefficient in an XxX pixel block and the CBP Is a flag indicating whether there is a non-zero quantized transform coefficient in a 16x16 pixel block. In the video encoding / decoding method,
A prediction is performed by predicting a subblock in which a macroblock is divided to generate a prediction subblock, subtracting the subblock and the prediction subblock to generate a residual subblock, and converting the residual subblock according to the size of the subblock. Converting and quantizing the quantized transform coefficient to generate quantized transform coefficients, and encoding the quantized transform coefficients to generate encoded image data; And
Decode the encoded image data to restore partition type information, motion information, transform type information, and quantized transform coefficients, and dequantize the quantized transform coefficients using transforms identified by the restored transform type information. Reconstruct the residual subblocks by inverse transform, generate a prediction subblock by predicting the subblocks identified according to the reconstructed partition type information using the reconstructed motion information, and add the prediction subblocks and the residual subblocks. Image decoding step of reconstructing the subblock
Image encoding / decoding method comprising a. In the method of encoding an image,
Predicting a subblock divided from the macroblock to generate a predictive subblock;
Generating a residual subblock by subtracting the subblock and the prediction subblock;
Transforming and quantizing the residual subblock using a transform selected according to the size of the subblock to generate quantized transform coefficients; And
Generating encoded image data by encoding the quantized transform coefficients.
Image encoding method comprising a. 26. The method of claim 25, wherein generating the prediction subblocks comprises:
For the macroblock having the size of NxN (where N is an integer of 16 or more), the macroblock layer K (where,

)of

And the subblocks of the macroblock layer K + 1 when the pixel block is divided into four blocks. 26. The method of claim 25, wherein generating the prediction subblocks comprises:
And predicting a subblock by dividing the macroblock having a size of NxN (where N is an integer greater than or equal to 16) pixel blocks in units of 16x16 pixel blocks when N is larger than 16. 26. The method of claim 25, wherein generating the prediction subblocks comprises:
When the macroblock is an intra macroblock, the subblock is predicted in units of mxn pixels, wherein m and n are one of 4, 8, and 16, which are not equal to each other, or greater than 16, and N (N is an N × N pixel block. And a pixel unit of a macroblock having a size of?). 26. The method of claim 25, wherein generating the prediction subblocks comprises:
And predicting the subblock according to a prediction direction according to a prediction mode having the best encoding efficiency among three intra prediction modes. The method of claim 25, wherein the quantized transform coefficient generation step,
When the macroblock is an intra macroblock, if the size of the subblock is one of 4x4, 4x8, and 8x4, 4x4 transform is used; if the size of the subblock is one of 8x8, 8x16, and 16x8, 8x8 transform is used. And 16x16 transform if the size of the subblock is 16x16 or larger. The method of claim 25, wherein the quantized transform coefficient generation step,
The remaining subblocks are divided into 16x16 pixel residual blocks, the divided 16x16 pixel residual blocks are further divided into 16 4x4 pixel residual blocks, and each 4x4 pixel residual block is transformed using 4x4 transform to convert 16 transform blocks. And converting a DC component block composed of DC components of the 16 transform blocks using 4x4 transform. 32. The method of claim 31,
And a 4x4 transform that transforms the 4x4 pixel residual block and a 4x4 transform that transforms the DC component block using different transform schemes. The method of claim 25, wherein the quantized transform coefficient generation step,
And when the macroblock is an inter macroblock, transforming the residual subblock by selecting one of 4x4 transform, 8x8 transform, 8x16 transform, 16x8 transform, and 16x16 transform. The method of claim 25, wherein the quantized transform coefficient generation step,
When the size of the subblock is smaller than 8x8, the residual subblock is transformed using 4x4 transform. When the size of the subblock is 8x8, the encoding cost of the residual subblock is smallest among 4x4 transform and 8x8 transform. If the size of the subblock is 8x16, the residual subblock is transformed using a transform having the lowest coding cost among 4x4, 8x8, and 8x16 transforms, and the size of the subblock is 16x8. In case of, the residual block of the subblock is transformed using a transform having the smallest coding cost among 4x4 transform, 8x8 transform, and 16x8 transform. And the 8x8 transform and the 16x16 transform are transformed using the transform having the lowest encoding cost. The method of claim 25,
If the macroblock is an inter macroblock, generating transform type data by encoding a transform type indicating a type of the selected transform;
The video encoding method further comprises. The method of claim 25,
Inverse quantization and inverse transformation of the quantized transform coefficients to recover a residual subblock;
Reconstructing a subblock by adding the reconstructed residual subblock and the prediction subblock; And
Filtering the reconstructed subblock
The method of claim 1, wherein the filtering of the subblocks comprises performing deblocking filtering on a boundary between a left side and an upper side of a block according to the selected transform in the reconstructed subblock. The method of claim 25,
Generating partition type data by encoding partition type information of the macroblock;
The video encoding method further comprises. The method of claim 25, wherein the split type information,
Identified by the division type number per division layer,
The generating of the partition type data may include encoding the partition type information by sequentially encoding the partition type number for each partition layer according to a partition type number encoding order. The method of claim 25,
Generating CBP data by encoding a CBPX flag and a CBP according to the size of the subblock;
Include more,
The CBPX flag is a flag indicating whether there is a non-zero quantized transform coefficient in an XxX pixel block, and the CBP is a flag indicating whether there is a non-zero quantized transform coefficient in a 16x16 pixel block. . The method of claim 25,
Generating SKIP_N flag data by encoding a SKIP_N flag indicating whether the macroblock is a SKIP macroblock;
The video encoding method further comprises. The method of claim 25,
Generating macroblock type flag data by encoding a macroblock type flag indicating whether the macroblock is an intra macroblock or an inter macroblock;
The video encoding method further comprises. The method of claim 25,
Generating encoded data including the encoded image data
Include more,
When the picture including the macroblock is an intra picture, the encoded data further includes one or more of a division type field, an intra prediction mode field, a CBPX flag / CBP field, and a transform coefficient field. . The method of claim 25,
Generating encoded data including the encoded image data
Include more,
When the picture including the macroblock is an inter picture, the encoded data includes at least one of a SKIP_N flag field, a macroblock type field, and a split type field, and when the macroblock is an intra macroblock, an intra prediction mode field. And at least one of a CBPX flag / CBP field and a transform coefficient field, and if the macroblock is an inter macroblock, a SKIPX flag field, a differential motion vector / reference picture index field, a CBPX flag field, a transform type field, And at least one of a CBP field and a transform coefficient field. In the method of decoding an image,
Decoding encoded data to recover partition type information, an intra prediction mode, and a quantized transform coefficient;
Inversely quantizing and inversely transforming the quantized transform coefficients using a transform selected according to a size of a subblock identified according to the partition type information to be restored to recover a residual subblock;
Generating a prediction subblock by predicting a subblock identified according to the reconstructed partition type information using the reconstructed intra prediction mode; And
Reconstructing the subblock by adding the prediction subblock and the residual subblock
Image decoding method comprising a. The method of claim 44,
The transform selected according to the size of the subblock,
If the size of the subblock is one of 4x4, 4x8 and 8x4, 4x4 transform is used.If the size of the subblock is one of 8x8, 8x16 and 16x8, 8x8 transform is used. If the size of the subblock is 16x16 or more, 16x16 transform is used. Image decoding method characterized in that using. In the method of decoding an image,
Decoding encoded data to recover partition type information, motion information, transform type information, and quantized transform coefficients;
Inversely quantizing and inversely transforming the quantized transform coefficients using a transform identified by the recovered transform type information to recover a residual subblock;
Generating a prediction subblock by dividing the subblock into identified subblocks according to the reconstructed partition type information and predicting the subblock using the reconstructed motion information; And
Reconstructing the subblock by adding the prediction subblock and the residual subblock
Image decoding method comprising a. 47. The method of claim 46, wherein the transform identified by the transform type information is:
And a 4x4 transform, an 8x8 transform, an 8x16 transform, a 16x8 transform, and a 16x16 transform. The method of claim 46,
Performing deblocking filtering on a boundary between a left side and an upper side of the block according to the selected transform in the reconstructed subblock;
The video decoding method further comprising. The method of claim 46,
And restoring a SKIP_N flag indicating whether the macroblock is a SKIP macroblock by decoding SKIP_N flag data extracted from the encoded data. The method of claim 46,
Decoding the macroblock type flag data extracted from the encoded data, and restoring a macroblock type flag indicating whether the macroblock is an intra macroblock or an inter macroblock. . 47. The method of claim 44 or 46, wherein the partition type information is
A video decoding method characterized by being identified by a partition type number for each partition layer. 47. The method of claim 44 or 46,
Restoring a CBPX flag and a CBP according to the size of the subblock by decoding the CBP data extracted from the encoded data;
Include more,
The CBPX flag is a flag indicating whether there is a non-zero quantized transform coefficient in an XxX pixel block, and the CBP is a flag indicating whether there is a non-zero quantized transform coefficient in a 16x16 pixel block. .

Images