KR101076876B1 - Block-based depth map coding method and apparatus and 3D video coding method using the method - Google Patents

Abstract

A method and apparatus for coding a block-based depth map and a 3D video coding method using the same are provided. In the decoding method of the depth map according to an embodiment of the present invention, the depth map is reconstructed by performing a bitplane decoding technique on a block basis of a predetermined size with respect to the input bitstream. For example, based on the decoded coding mode information, decoding may be performed by selecting a bitplane decoding technique or an existing DC-based decoding technique in units of blocks. The bitplane decoding technique may include adaptively performing an XOR operation on a bitplane block basis. For example, the XOR operation may be adaptively performed in units of bitplane blocks according to the value of the XOR operation information decoded from the bitstream.

MVC, 3D Video Coding, depth map, exclusive OR operation

Description

Block-based depth map coding method and apparatus, and 3D video coding method using same {Block-based depth map coding method and apparatus and 3D video coding method using the method}

The present invention relates to video coding, and more particularly, to a method and apparatus for encoding / decoding a depth map of an image, and a 3D video coding method using the same. will be.

As interest in realistic media has increased, studies on it have been actively conducted. Realistic media refers to media that can be seen, heard and felt as in the real world even in a virtual environment where space and time are limited. Users can feel the realism and immersion like the real world even in the virtual environment through realistic media. Realistic media is expected to be used not only in the fields of broadcasting and telecommunications but also in advertising, exhibition, education and medical fields.

One example of sensory media is a multi-view image. The multi-view image refers to a plurality of images acquired from various viewpoints for the same subject. Using a multi-view image, the user can observe the same subject from various angles. Representative video codings related to multi-view video include multi-view video coding (MVC) and 3D video coding.

The MVC is for efficiently encoding / decoding multiple view images received from a plurality of cameras (eg, four or eight). The MVC provides the user with only the image of the same viewpoint as that of the camera used to acquire the multiview image. Therefore, in order to provide images of more viewpoints by using image data encoded with MVC, images must be photographed using more cameras. However, as the number of cameras increases, the amount of video data increases accordingly. Since there is a certain limit in the transmission bandwidth or storage capacity of existing transmission media (broadcasting, communication, storage media, etc.), it is provided to a user using MVC. The point in time can be a constraint. In addition, MVC cannot provide a user with a viewpoint image different from that of a camera.

3D video coding is intended to compensate for this limitation of MVC. 3D video coding supports the generation of an image of a viewpoint (eg, N viewpoints) of the camera used to acquire an image as well as other viewpoints, that is, a virtual viewpoint. To this end, in 3D video coding, a depth map is also coded in addition to images of N viewpoints. An image of a virtual view may be generated by performing view interpolation using images acquired from a camera located at an adjacent view and depth maps thereof. Such 3D video coding may support three-dimensional display systems such as free view-point television (FTV) systems.

1 is a diagram illustrating an example of rendering in an FTV system to which 3D video coding is applied. The FTV system illustrated in FIG. 1 uses N (N = 5) cameras, and images of images (images of a viewpoint represented by a black camera) acquired by N (N = 5) cameras at the time of coding Information and depth information of each image are encoded. The FTV system decodes the encoded image information and the depth information, and then uses the decoded image information and the depth information of each camera, and not only the viewpoint of the camera but also images of a viewpoint different from that of the camera. An image may be rendered at any point in time in the region indicated by the fan-shaped shade.

As such, in 3D video coding applied to an FTV system, it is not necessary to acquire images of all viewpoints to be provided to a user with a camera. Thus, in providing images of as many views as possible, 3D video coding is relatively free from the limitations of transmission bandwidth or storage capacity compared to MVC. When 3D video coding is used, an image of any viewpoint desired by a user can be provided without particular limitation.

However, in the MVC, only a process of encoding / decoding only image information of an image at the time of input is required. On the other hand, in 3D video coding, depth maps must be encoded / decoded in addition to image information of an image. That is, compared to MVC, 3D video coding further requires generating a depth map, encoding / decoding a depth map, and generating a virtual view image using the depth map. . Most of the current researches related to 3D video coding are focused on the generation of depth maps and the generation of virtual images, and much research is being done on the encoding / decoding of depth maps. not. This is based on the recognition that the encoding / decoding process of the depth map is sufficient to use existing techniques (hereinafter, referred to as 'image information coding technique') for coding image information (luminance, color difference, etc.). .

The depth map is intended to represent the distance between the camera and the object at the present time. The distance between the camera and the object may vary depending on the position of the object, that is, the spatial position in which the object is placed in the image. Therefore, like the image information, the depth information map may be expressed in pixel units. For example, the depth map may be generated by expressing the distance with a constant number of bits at the same resolution as the image information of the current image.

However, since the actual range to the object can vary greatly depending on the frame, the distance to the object from each pixel, that is, the depth information, is expressed as a relative value rather than an absolute value. For example, the shortest distance Z _near and the longest distance Z _far in the same frame may be obtained, and then depth information may be represented as a relative value using the same. For example, when the depth information is represented by 8 bits, the pixel with the shortest distance (Z _near ) is represented by '255', and the pixel with the longest distance (Z _far ) is represented by '0'. In the case of pixels having an object having an intermediate distance, the pixels may be proportionally represented by a predetermined value between '0' and '255'.

As such, the depth map represents distance information based on the distance between the real object and the camera. On the other hand, data that is the object of encoding / decoding in conventional video coding is image information such as luminance, color difference, or RGB value. Depth information and image information show similar characteristics in that the same subject may have the same distance, brightness, color, etc., but there are many cases in which the same subjects have low relations with each other. For example, an object having completely different image information such as luminance or color difference, or two different parts of the same object may have similar depth information. On the contrary, the depth information may be different even though image information such as luminance and color difference is similar.

However, existing video coding techniques have been developed to efficiently compress in consideration of the characteristics of image information. If the existing video coding techniques are applied to the encoding / decoding of the depth map as it is, it is not easy to efficiently encode the depth map having characteristics different from those of the image information. In addition, when an image of a viewpoint different from that of a camera is generated using the depth map, the accuracy of the depth map may greatly affect the image quality of the image. Therefore, in order to maximize the advantages of 3D video coding as opposed to MVC, an efficient depth map encoding / decoding technique considering characteristics unique to depth maps needs to be developed.

One object of the present invention is to provide a method and apparatus for efficiently encoding and decoding a depth map, and a 3D video coding method using the same.

Another object of the present invention is to provide a method and apparatus for encoding / decoding a depth map capable of improving the quality of a depth map image, and a 3D video coding method using the same.

Another object of the present invention is to provide a method and apparatus for encoding / decoding a depth map in consideration of characteristics unique to depth information different from image information, and a 3D video coding method using the same.

Another problem to be solved by the present invention is a method and apparatus for encoding / decoding a depth map capable of improving the quality of an interpolated view image by improving coding efficiency of a depth map in 3D video coding, and It provides a three-dimensional video coding method used.

The decoding method of the depth map according to an embodiment of the present invention for resolving the above problem is to reconstruct the depth map by performing a bitplane decoding technique on a block basis of a predetermined size with respect to the input bitstream. For example, the block-based bitplane decoding technique includes decoding coding mode information for each depth map block from a bitstream, and only when the decoded coding mode information indicates a bitplane decoding technique. The bitplane decoding technique may be performed. The bitplane decoding technique may include an XOR operation step of adaptively performing an exclusive OR operation in units of bitplane blocks. More specifically, the step of decoding the XOR operation information from the bitstream of the depth map, wherein the XOR operation step instructs the value of the decoded XOR operation information to perform the XOR operation on the corresponding bitplane block May be performed. In this case, the method may further include configuring a depth map block by combining the bitplane block that has been subjected to the XOR operation step and / or the bitplane block that has not been subjected to the XOR operation step.

According to another aspect of the present invention, there is provided a decoding method of a depth map according to another embodiment of the present invention. ) To perform the operation. For example, the decoding method includes a bitplane decoding step for recovering the bitplane block from the input bitstream, an XOR operation for adaptively performing an exclusive OR operation on the restored bitplane block. And a bitplane combining step for combining the restored bitplane block on which the XOR operation was performed and / or the restored bitplane block on which the XOR operation was not performed.

According to another aspect of the present invention, there is provided a method of decoding a depth map, the method comprising: decoding coding mode information for each depth map block from a bitstream, and according to the decoded coding mode information. And reconstructing the depth map using the block-based bitplane decoding technique or the block-based DC-based decoding technique.

Decoding of the depth map according to another embodiment of the present invention for solving the above problems is a method of decoding a depth map block represented by N bits, decoding according to the block mode information for each bitplane block Restoring n (≤N) bitplane blocks, and adaptively XORing the reference bitplane block to the n recovered bitplane blocks according to exclusive OR operations information for each bitplane block. Performing an operation, and combining the n recovered bitplane blocks on which the XOR operation is performed and / or on which the XOR operation is not performed.

According to another aspect of the present invention, there is provided a decoding method of a depth map according to a method of decoding block mode information from an input bitstream and a same level block decoding technique or binary image according to the block mode information. Reconstructing the current bitplane block using a decoding technique.

According to the embodiment of the present invention, the coding efficiency of the depth map can be improved in 3D video coding. More specifically, it is possible to improve the image quality of the reconstructed image, especially the depth map near the object boundary, or to reduce the bit rate using the depth map. Therefore, by using the depth map coding method according to an embodiment of the present invention, the image quality of the original image captured by the camera in 3D video coding as well as the image interpolated using the original image can be improved.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The terms used are terms selected in consideration of functions in the embodiments, and the meaning of the terms may vary depending on the intention or custom of the user or operator. Therefore, the meaning of the terms used in the embodiments to be described later, according to the definition if specifically defined herein, and if there is no specific definition should be interpreted to mean generally recognized by those skilled in the art.

Before describing an encoding / decoding apparatus and method according to an embodiment of the present invention, a characteristic unique to a depth map will be described.

FIG. 2 illustrates an example of a real image and a depth map image thereof, and FIG. 2A illustrates an image sequence of 'Breakdancers', which is one of the test sequences of MVC as an example of the actual image, and FIG. ) Is the depth map image of this 'Breakdancers' video sequence. In the depth map image of FIG. 2B, 8-bit depth information of each pixel is represented, and the closer to the camera, the greater the value (bright value).

An example of a method of obtaining the actual distance Z from each pixel using the depth map represented by such 8 bits may be expressed by Equation 1 below.

Here, v is a depth value of the corresponding pixel, and Z _far and Z _near are parameters defined in MPEG-C Part 3, respectively, to the part closest to the distance from the farthest distance to the actual distance in the video. Indicates the distance of. Therefore, the value of the depth information displayed on the depth map is 2 ^N (where N represents the number of bits representing the depth map between the farthest and the closest regions in the real world shown in the image, FIG. 1). In the case of (b), N is 8).

3A and 3B are graphs for illustrating the level distribution of image information and the level distribution of depth information in the image of FIG. 2, respectively. FIG. 3A is a three-dimensional graph representing luminance, that is, brightness, which is an example of image information of each pixel in the actual image of FIG. 2A, and FIG. 3B is a diagram of each pixel in the depth map image of FIG. 2B. Three-dimensional graph representing the level of depth information. Referring to FIGS. 3A and 3B, it can be seen that, in general, the change in brightness is rather rapid in the same image, but the change in depth information level is relatively slow. However, there is a limit to generalizing such characteristic difference between the image information and the depth information level, and there may be an exception depending on the type of the image.

4A and 4B are diagrams for illustrating an example of characteristics inherent in depth information different from image information, and FIG. 4A is a depth information graph representing a depth map block including a boundary of an object in three dimensions. 4B is a depth information graph representing a depth map block in a background portion in three dimensions. Referring to FIG. 4A, although there is a sudden change in depth information at a boundary portion of an object in a block including an object boundary, depth information is provided at neighboring pixels adjacent to each other inside or outside of the object (background). It can be seen that is distributed in nearly similar values. On the other hand, referring to Figure 4b, in the block containing only the inside of the object or the background portion, the depth information of each pixel is distributed between 46 ~ 54, but it seems that there is no significant change, the depth information is slightly depending on the position of the pixel You can see that there is a difference.

This characteristic of the depth information can be more clearly understood by dividing the depth map image into bit plane units. 5 (a) and 5 (b) are representations of the actual image of FIG. 2 (a) and the depth map image of FIG. 2 (b), respectively, in bit plane units. Most Significant Bit (MSB), Bit 6 (MSB-1), Bit 5 (MSB-2), Bit 4 (MSB-3), Bit 3 (MSB-4), Bit 2 (MSB-5), and Bit 1 ( MSB-6), and bitplanes for Bit 0 or Least Significant Bit (LSB). Overall, the bit plane of the depth map image (Fig. 4 (b)) is considerably monotonous compared to the bit plane (Fig. 4 (a)) of the actual image (e.g., the image expressed in brightness of the image information). This will be described in more detail below.

Referring to FIG. 5A, it can be seen that the bitplane for the most significant bit MSB in the actual image (image information) has a monotonous shape, but becomes more complicated toward the bitplane of the lower bit. In particular, it can be seen that the lower three bitplanes, including the least significant bit (LSB), are all quite complex in the form of white noise. Referring to FIG. 5B, although the bitplane becomes more complicated in the depth map image from the most significant bit (MSB) to the least significant bit (LSB), it is similar to the bitplane of the real image. It can be seen that the image is monotonous compared to the bit plane of the image. In addition, in the depth map image, there is a morphological correlation between the bitplanes of some upper bits and the bitplanes of some lower bits, which can be clearly confirmed through FIG. 5C.

FIG. 5C illustrates gray-coding of the depth map image in units of bit planes. Graycoding refers to a transformation technique that changes only 1-bit when expressing consecutive values. It can be expressed as applying graycode or performing an exclusive OR (XOR) operation between bitplanes. have. In graycoding, for example, a bitplane and an XOR operation are performed on a bit one step below the current level, or a bitplane and XOR operation is performed on a bit one step higher, thereby converting the bit for the current level. It can represent a bitplane (or a graycoded bitplane or an XOR-operated bitplane). In the following, the latter case will be described as an example, but it will be apparent to those skilled in the art that the following embodiments of the present invention can be equally applied to the former case.

Also in FIG. 5C, bit 7 or most significant bit (MSB), bit 6 (MSB-1), bit 5 (MSB-2), bit 4 (MSB-3), and bit 3 (MSB-4), respectively, from the top. , Gray coded bitplanes for bit 2 (MSB-5), bit 1 (MSB-6), and bit 0 or least significant bit (LSB). Referring to FIG. 5C, it can be seen that the gray coded bitplanes of the depth map are more monotonous than the original bitplane (see FIG. 5B).

The characteristic unique to the depth map can be found in more detail by examining the depth map image in units of blocks. 6 (a) is a part of the depth map of FIG. 2 (b), and FIGS. 6 (b), 6 (c) and 6 (d) are each arbitrarily selected from the depth map of FIG. 6 (a). Bitplane blocks (from the most significant bit (MSB) plane block to the least significant bit (LSB) plane block) for the selected block (e.g., 16x16 block) and the bitplane blocks greyed out these bitplane blocks. to be. Here, the selected blocks are all blocks including the object boundary portion or the object boundary, but are not necessarily limited thereto.

Looking at the bitplane blocks selected in each of (b), (c), and (d) of FIG. 6, all or some of the binary images of each bitplane block are completely coincident with one another or inverted with each other. It can be seen that. The characteristics of the depth map may not necessarily appear in all blocks, but may appear only in some blocks, and may occur frequently in blocks near an object boundary. In this case, the depth map is likely to coincide with some blocks, especially depth map blocks comprising the boundary of the object or the boundary of the object, in which all or part of the depth map coincides completely or inverted with each other.

The characteristics of the depth map can be more clearly confirmed by performing gray coding on the bitplane block. As shown in each of (b), (c), and (d) of FIG. 6, among the gray-coded bitplane blocks (other bitplane blocks except the MSB plane block), the entire block has the same value (0). There are many blocks with (0) or 255 (1)). This is because of the characteristics of the depth map described above, when the binary image of the current bitplane completely matches the binary image of the higher bitplane, all pixels of the gray coded bitplane block become '0 (0)', This is because the pixels of the gray coded bitplane block are all 255 (1) when invertedly matched.

However, when the gray code is applied to the depth map image, the binary image of the bitplane (the binary image of the gray-coded bitplane) for the specific bit becomes more monotonous than the bitplane before the graycode is applied. The opposite can also happen. That is, a binary image of a bitplane for some bits may have a more complex form of a gray coded bitplane. This is because the bitplane binary image for the bit of the corresponding level has little correlation with the bitplane binary image for the bit of the adjacent level.

FIG. 7 illustrates an example in which the bit plane for a bit of a corresponding level has little correlation with the bit plane for a bit of an adjacent level. FIG. 7A is a binary image of a bitplane block for bit 2 (MSB-5) in FIG. 6B, and FIG. 7B is bit 2 (MSB-) in FIG. 6B. 5) is a binary image of a gray coded bit play block. Referring to (a) and (b) of FIG. 7, it can be seen that in the case of bitplanes for some bits, applying a gray code further complicates a binary image.

The unique characteristics of the depth map described above are summarized as follows.

(1) The depth map has a fairly gentle change between pixels. When the depth map is divided into bitplane units, and each separated bitplane binary image is compared with a bitplane binary image of an actual image (image information), the bitplane binary image of the depth map is a bitplane binary image. It has a fairly monotonous shape compared to the image.

(2) In a block-wise bitplane binary image for a depth map, some blocks, for example, when the block includes or is adjacent to a boundary of an object, are divided between binary images of some levels of bitplanes. Frequently it is a perfect match or an inverted match. However, even in this case, such a correlation does not appear between all levels of the bitplane binary images, and other levels of the bitplane binary image may be more complicated when gray coding is performed.

Based on the characteristics of the depth map, the coding method of the depth map according to an embodiment of the present invention performs encoding / decoding on the depth map in the following manner.

First, the bitplane encoding scheme is selectively or adaptively applied to the coding of the depth map. For example, in a depth map image (frame or picture), some blocks apply a bitplane encoding scheme, while some blocks use a conventional technique for coding image information that is not a bitplane encoding scheme (hereinafter referred to as 'bitplane'). In order to distinguish it from the encoding (encoding) technique, it is called a "discrete cosine transform (DCT) based encoding (encoding) technique." Therefore, although the term "DCT-based encoding technique" is used herein, However, this should not be construed as being limited to an encoding technique that essentially includes a DCT process, and other encoding may be used in a transform process. In this case, the coding efficiency of the DCT-based encoding method and the bitplane encoding method is compared, and then encoding may be performed by selectively applying an encoding method having a better coding efficiency for each block, which will be described later. Obviously, this embodiment of the present invention does not exclude applying only the bitplane encoding scheme to the depth map image.

For the block to be encoded by applying the bitplane encoding scheme, the bitplane is encoded after adaptively performing an XOR operation (gray coding) according to the bitplane. More specifically, bitplanes for bits of some levels are encoded without performing XOR operations, while bitplanes for bits of other levels are performed after performing XOR operations first. Of course, it is obvious that the bitplane encoding technique using this adaptive XOR operation does not exclude performing the XOR operation at all or all the XOR operations on the bitplanes for all levels of bits.

8 is a flowchart illustrating an example of a bitplane encoding method for bitplanes of a depth map using an adaptive XOR operation according to an embodiment of the present invention, which is an example of an encoding method for an intra picture. to be.

Referring to FIG. 8, first, bitplane separation is performed on the input depth map block (10). As a result, a plurality of (eg, eight) bitplane blocks are obtained, from the bitplane block for the MSB to the bitplay block for the LSB.

In operation 11, an XOR operation is adaptively performed on each bitplane block. More specifically, the XOR operation is performed only on the bitplane block for the bit of a specific level, and the XOR operation is not performed on the bit of the remaining level. Which level of bits to perform XOR operation can be determined in consideration of the coding efficiency. For example, compare the amount of bits that occur when coding a bitplane block without performing an XOR operation and the amount of bits that occur when coding a bitplane block after performing an XOR operation. Only when the amount of generated bits is smaller than in the former case, the XOR operation may be performed on the bitplane blocks of the corresponding level. Information indicating whether or not to perform an XOR operation (for example, can be represented by using an 'XOR flag (xor_flag)', which is an example) is included in the bitstream as additional information about each bitplane block of each level. It is sent to the decoding stage.

Performing an XOR operation between the current bitplane and the reference bitplane means performing an XOR operation between the corresponding pixels of the two bitplanes. In this case, a bitplane block of a higher level of the current bitplane or a bitplane block of a lower level may be used as the reference bitplane. Alternatively, a bitplane block for a specific level of bits selected from a bitplane block other than the current bitplane is used as a reference bitplane, or a combination is performed by performing an XOR operation on the current bitplane block and a plurality of specific bitplane blocks. The bitplane block may be used as the reference bitplane.

Subsequently, in the bitplane coding method using the adaptive XOR operation according to the embodiment of the present invention, bitplane coding is sequentially performed on a bitplane block on which an XOR operation is performed or not. For example, bitplane coding may be performed in order from an MSB bitplane block to an LSB bitplane block. In this case, it is not necessary to perform bitplane coding for all the bitplane blocks, and if it is necessary to adjust the amount of bits generated, only perform bitplane coding on the bitplane blocks for the bits of the upper part of the MSB bitplane block. Alternatively, the bitplane coding may be performed after downsampling on each bitplane block. However, the present embodiment is not limited thereto, and after performing step 10, a step after step 11 may be arbitrarily performed only for the bitplane blocks for the upper part bits, thereby adjusting the amount of bits generated by bitplane coding. have.

In a bitplane coding process for an intra picture, for example, encoding may be adaptively performed according to whether all binary image values in a corresponding bitplane block are the same. To this end, it is first determined whether all binary image values in the bitplane block are the same (12). As a result of the determination, when all binary image values in the block are the same, for example, if all are '0 (0)' or all are '255 (1)', 'block mode information' of the bitplane indicating this is indicated. Only a value indicating (eg, 'all_0' mode or 'all_1' mode) is encoded (13). On the other hand, if all binary image values in the block are not the same, a predetermined 'binary image compression method', for example, an intra context-based arithmetic encoding (intraCAE) technique, may be used. Encoding may be performed (14). In this case, the mode of the current bitplane block can be set to 'intraCAE mode'.

9 is a flowchart illustrating an example of a bitplane encoding method for bitplanes of a depth map using an adaptive XOR operation according to another embodiment of the present invention, which is an example of an encoding method for an inter picture. to be.

Referring to FIG. 9, as in FIG. 10 (steps 10 and 11), bitplane separation is performed on the input depth map block (20). As a result, a plurality of (eg, eight) bitplane blocks are obtained, from the bitplane block for the MSB to the bitplay block for the LSB. In operation 21, the XOR operation is adaptively performed on all or some of the bitplane blocks. Which level of bits to perform XOR operation can be determined in consideration of the coding efficiency. Information indicating whether to perform an XOR operation (for example, can be represented by using an 'XOR flag (xor_flag)', which is an example) is included in the bitstream as additional information about the bitplane block of each level. And transmitted to the decoding end.

Subsequently, bitplane coding is sequentially performed on the bitplane blocks on which the XOR operation is performed or not. For example, bitplane coding may be performed in order from an MSB bitplane block to an LSB bitplane block. In this case, it is not necessary to perform bitplane coding for all the bitplane blocks, and if it is necessary to adjust the amount of bits generated, only perform bitplane coding on the bitplane blocks for the bits of the upper part of the MSB bitplane block. Alternatively, the bitplane coding may be performed after downsampling on each bitplane block.

In the bitplane coding process for an inter picture, encoding can be efficiently performed using the following method. However, the steps 22 to 30 to be described later are exemplary, and the present embodiment is not limited thereto. For example, in the flowchart illustrated in FIG. 9, step 22 of determining a mode using the predicted value MVp of the motion vector may be omitted, and step 24 may be performed immediately after step 21.

First, an error Aerr is obtained between a reference bitplane block and a current bitplane block corresponding to the prediction value MVp of the motion vector MV of the current bitplane block, and then the above error Aerr is a predetermined allowable error ( Berr) it is determined whether the range (S22). There is no limitation on the method of obtaining the predicted value MVp of the motion vector MV of the current bitplane block. There is no particular limitation on the value of the tolerance. As a result of the determination, when the error Aerr is equal to or smaller than the tolerance Berr, the bitplane block is set to a predetermined block mode (for example, 'No Update Without MV' mode) indicating this ( 23, only this block mode information (a predetermined value indicating the 'No Update Without MV' mode) is encoded (31).

On the other hand, if the error Aerr is greater than the allowable error Berr as a result of the determination in step 23, encoding may be adaptively performed according to whether all binary image values in the current bitplane block are the same. To this end, it is determined whether all binary image values in the bitplane block are the same (24). As a result of the determination, when all binary image values in the block are the same, for example, if all are '0 (0)' or all are '255 (1)', a predetermined block mode (eg, for indicating this bitplane block) is indicated. It is set to the 'all_0' mode or the 'all_1' mode (25), and only this block mode information (a predetermined value indicating the 'all_0' mode or the 'all_1' mode) is encoded (31).

On the contrary, when the determination result in step 24 indicates that all pixel values in the current bitplane block are not the same, motion prediction for the current bitplane block is performed (26). After calculating the error Cerr between the reference bitplane block corresponding to the motion vector MV calculated in the motion prediction step 26 and the current bitplane block, this error Cerr is a predetermined allowable error Berr. It is determined whether it is within the range (27). This embodiment uses the same value as the tolerance used in step 22 as the tolerance (Berr), but this is exemplary. As a result of the determination, when the error Cerr is equal to or smaller than the allowable error Berr, the bitplane block is set to a predetermined block mode (for example, a 'No Update With MV' mode) indicating this ( 28, only this block mode information (a predetermined value indicating the 'No Update With MV' mode) and the motion vector MV are encoded (30).

On the other hand, when the error Cerr is greater than the allowable error Berr, encoding is performed by using a predetermined binary image compression method, for example, a context-based arithmetic encoding (CAE) technique, for the error Cerr for the current bitplane block. May be performed (29). When encoding using the CAE encoding technique, either one of the intraCAE and the interCAE techniques is used, or the CAE technique having the better coding efficiency among the two CAE techniques. May be adaptively performed. In the latter case, the encoding is performed by using the intra CAE technique and the inter CAE technique, respectively, and then the encoding is performed by using a CAE technique having a smaller bit amount resulting from the encoding, and the block mode of the current bitplane block. The furnace can be set to 'intraCAE' mode or 'interCAE' mode.

Next, an apparatus for encoding a depth map according to an embodiment of the present invention using the characteristics of the above-described depth map and an encoding method in the encoder will be described.

FIG. 10 is a block diagram illustrating a configuration of an apparatus for encoding a depth map, that is, a depth map encoder, according to an embodiment of the present invention. In the depth map encoder according to the present embodiment (a bit plane encoding unit as well as a DCT-based encoding unit), the input depth map is encoded in a block unit (M ㅧ N block) of a predetermined size (M ㅧ N). To perform. There is no particular limitation on the size of the M ㅧ N block, for example, a macroblock having a size of 16 ㅧ 16 blocks, a larger block, or a smaller block (eg, 8 ㅧ 8 blocks or 4 ㅧ). 4 blocks, etc.).

Referring to FIG. 10, the depth map encoder 100 according to the present embodiment includes a DCT based encoding unit A and a bitplane encoding unit B. FIG. The DCT-based encoding unit A is an example of an apparatus for encoding a depth map in units of blocks by the same encoding scheme as that for encoding image information. As shown, the DCT based encoding unit A may have the same configuration as the encoding unit of H.264 / Advanced Video Coding (AVC), but the embodiment is not limited thereto. For example, in addition to H.264 / AVC, the DCT-based encoding unit (A) is an encoding unit in MPEG-1, MPEG-2, MPEG-4 Part 2 Visual, VC (Video Coding) -1, etc., H.264 / AVC The encoding unit may be an encoding unit of MVC or SVC (Scalable Video Coding) to which the encoding unit is applied, or an encoding unit according to a new video coding technique that is currently made or will be made in the future.

On the other hand, the bitplane encoding unit B divides the depth map block into bitplane blocks, adaptively applies an XOR operation to each separated bitplane block, and then performs encoding on a bitplane block basis. It is an example of a device. Therefore, the embodiment of the present invention should not be interpreted as being limited to the configuration of the illustrated bitplane encoding unit B. For example, some components of the bitplane encoding unit B (eg, the bit rate controller 142) may be arbitrary components or integrated into other components. Each of the components constituting the bitplane encoding unit B may also be replaced, integrated, or separated into other blocks according to a viewpoint. The specific structure of this bitplane encoding unit B and its encoding method will be described later.

According to the present embodiment, the depth map encoder 100 encodes the encoding unit having a higher encoding efficiency among the DCT-based encoding unit A and the bitplane encoding unit B, and then outputs the encoded data as a bitstream. do. In this bitstream, together with the encoded depth information map data, information indicating which encoding unit is encoded (hereinafter, referred to as 'coding mode information') may be represented using, for example, bitplane_coding_flag. ) Is also included. Here, the coding mode information is information indicating that the coded depth map data is coded in the DCT based coding mode or information indicating that the coded depth map data is coded, and there is no particular limitation on the method of expressing the coded depth map data. For example, in the case of expressing coding mode information using bitplane_coding_flag, if the value is '0', it may indicate a DCT based coding mode, and if it is '1', it may indicate a bitplane coding mode and vice versa. It is possible.

Therefore, the depth map encoder 100 according to the present embodiment includes a unit (a mode selection unit such as an electric circuit or a software program) for selecting such a coding mode. FIG. 11 shows a DCT-based coding (encoding in A unit of FIG. 10) mode and a bitplane coding (encoding in B unit of FIG. 10) mode in a mode selection unit (not shown) of the depth map encoder 100. A flowchart illustrating an example of how to select a mode with better coding efficiency.

Referring to FIG. 11, the depth map encoder 100 first performs a cost _A when encoding the M_N block in DCT based coding mode and a cost when encoding in the bitplane coding mode. (Cost _B ) are respectively calculated (40). Here, the method of calculating the costs Cost _A and Cost _B is not particularly limited. For example, the cost (Cost _A , Cost _B ) can be calculated using the 'Rate-Distortion (RD) optimization technique' used in H.264 / AVC. Equation 2

Here, SSD _i (Sum of Squared Differences) refers to the sum of the squares of the prediction errors between the original image and the reconstructed image, and λ _MODE is a value generated using a quantization parameter (QP). A Lagrange constant represents a Lagrange constant, and R _i represents an amount of bits generated when actual coding is performed in a corresponding macroblock mode.

In operation 41, the DCT-based coding mode cost Cost _A and the bitplane coding mode cost Cost _B are calculated. As a result of the comparison, when the former Cost _A is smaller, a DCT based coding mode is selected (42). On the contrary, when the latter Cost _B is smaller, a bitplane coding mode is selected (43). Therefore, when the DCT-based coding mode cost Cost _A is smaller than the bitplane coding mode cost Cost _B , the coding mode information of the input M ㅧ N block (or a block having a predetermined size as a unit of coding) is DCT. When the bitplane coding mode cost Cost _B is smaller, the coding mode information of the input M_N block (or a predetermined size block that is a unit of coding) becomes the bitplane coding mode. As described above, such coding mode information is included in the bitstream together with the depth map data encoded in the corresponding mode and output.

10, the depth map encoder 100 according to an embodiment of the present invention includes an encoding unit according to H.264 / AVC as an example of the DCT-based encoding unit (A). Hereinafter, the encoding process in the H.264 / AVC encoding unit will be described as an example of the encoding process in the DCT-based encoding unit A. However, this means that the encoding unit according to another video codec will be the DCT-based encoding unit A. It does not exclude that it can be as described above.

Data input to the DCT-based encoding unit A may be input as a depth map (eg, image format 4: 0: 0) in units of M ㅧ N size blocks, for example, 16 × 16 pixels. In the DCT-based encoding unit A, encoding is performed in an intra mode or an inter mode. In the former case, the internal switch of the DCT-based encoding unit A is connected intra, whereas in the latter case, the switch is interconnected. The DCT based encoding unit A according to H.264 / AVC first generates a prediction block for the input macroblock. The difference between the input macroblock and the prediction block is obtained, and then residual data consisting of the differences is encoded.

(1) The method of generating the prediction block depends on whether it is intra mode or inter mode. In the intra mode, the intra predictor 106 generates a predictive block by performing spatial prediction using values of neighboring pixels already coded in the current macroblock. On the other hand, in the inter mode, the motion predictor 102 first searches for a region that best matches the currently input block in the reference image stored in the reference image buffer of the spiritual buffer 130 to obtain a motion vector. We perform the motion prediction process to find. The motion compensator 104 generates a prediction block by performing a motion compensation process of bringing a prediction block from a reference image stored in a reference image buffer using the motion vector.

(2) When the prediction block is generated, the first adder 108 calculates a difference between the prediction block and the current block to generate a residual block. The transform unit 110 that receives the generated residual block performs a transform process and outputs a transform coefficient. The transform coefficient is input to the quantization unit 112, and the quantization unit 112 outputs a quantized coefficient by performing a quantization process according to a quantization parameter (QP). For the quantized coefficients, an entropy coding process according to a probability distribution is performed in the entropy coding unit 114.

(3) Encoded data output by performing entropy coding in the entropy coding unit 114 is input to a multiplexer 160, and the multiplexer 160 converts the encoded data together with other information / data into a bitstream. Output

(4) Meanwhile, the DCT based encoding unit A according to H.264 / AVC re-decodes the encoded current video and stores it in the video buffer 130. This is because the current picture should be used as a reference picture in encoding a picture input later. To this end, the inverse quantization unit 116 performs an inverse quantization process on the quantized coefficients, and the inverse transformer 118 performs an inverse transformation process on the inverse quantization coefficients generated as a result of the inverse quantization process to generate a residual block. do. The generated residual block is added to the prediction block in the second adder 120, and as a result, the reconstructed block is output. The reconstructed block output from the second adder 120 may be used as an intra prediction process in the intra predictor 106 or as a reference block in the bitplane encoding unit B. FIG. In addition, the reconstructed block is input to the deblocking filter 122 to remove blocking artifacts generated during the encoding process and then stored in the reconstructed image buffer of the image buffer 130.

10, the depth map encoder 100 according to an embodiment of the present invention includes a bitplane encoding unit B for performing encoding in bitplane units using an adaptive XOR operation. . The data input to the bitplane encoding unit B is a depth map (eg, image format 4: 0: 0), and is input in units of macroblocks having an M ㅧ N size block, for example, 16 × 16 pixels. Can be. In the bitplane encoding unit B, after the separation 144 into bitplane blocks, encoding may be performed according to the bitplane encoding method using the adaptive XOR operation described above with reference to FIG. 8 or 9, and The process for bit rate adjustment 142 may be further performed. The procedure in the bitplane encoding unit B is described below.

(1) First, a unit for processing data in the bitplane encoding unit B may be variously applied. For example, the input M_N block may be divided into small blocks to form subblocks, and then encoding may be adaptively performed for each subblock. Alternatively, encoding may be performed after combining the input M ㅧ N block into a larger block or by dividing the combined block into sub-blocks.

(2) The bit rate adjusting unit 142 is for adjusting the amount of encoded data output from the bitplane encoding unit B, that is, the bit rate. The bit rate controller 142 operates only when the amount of encoded data to be output is required, and thus is an optional component. Alternatively, a method such as downsampling at the time of encoding in the bitplane encoding unit 148 or appropriately using the encoding algorithm used in the bitplane encoding unit 148 without separately adjusting the bit rate in the bit rate adjusting unit 142. You can also adjust the bit rate by using.

The method of adjusting the bit rate in the bit rate controller 142 is not particularly limited. For example, when the depth map is expressed in N bits, the bit rate controller 142 limits the bit rate by limiting the bit plane block to be encoded to only some bits (for example, the upper n (≤N) bits). I can regulate it. To this end, the bit rate controller 142 may delete the lower (N-n) bit from the depth map block represented by the input N bits and output the depth map block represented by the upper n bits.

In this case, how many bits the bitplane to be encoded is to be determined arbitrarily or based on other information affecting the bit rate, for example, based on the quantization parameter (QP) value in the DCT-based encoding unit (A). The decision may be made. In addition, the bit rate controller 142 may restore information about the lower (Nn) bits to be erased (for example, whether to restore all of them to '0 (0)' or all of them to '255 (1)' when decoding). Alternatively, information on a method of padding using another bit value) may be additionally generated and output to the multiplexer 160.

12 is a flowchart illustrating an example of a method of determining a bit rate based on the quantization parameter QP in the bit rate controller 142. The method used in H.264 / AVC is used as the quantization parameter (QP) for determining the number of bit planes to be coded in FIG. 12, but the present embodiment is not limited thereto.

Referring to FIG. 12, it is first determined whether a quantization parameter (QP) value is a predetermined value, for example, 22 or less (50). As a result of the determination, when the value of the quantization parameter QP is 22 or less, the bitplane coding unit B may include all or part of the bitplanes (for example, the top 7 bitplanes MSB, MSB-1, MSB-2, and MSB). -3, MSB-4, MSB-5, MSB-6 bitplane)) only block (51). If the value of the quantization parameter QP is larger than 22, it is determined whether the value of the quantization parameter QP is a predetermined value, for example, 32 or less (52). As a result of the determination, when the value of the quantization parameter QP is greater than 23 and less than or equal to 32, the number of bit planes smaller than step 31 (for example, the upper six bit planes MSB, MSB-1, MSB-2, MSB-3, MSB-4, MSB-5 bitplane)) only blocks (53). On the other hand, if the value of the quantization parameter (QP) is greater than 32, fewer number of bitplanes (e.g., the top five bitplanes MSB, MSB-1, MSB-2, MSB-3, MSB-4) than step 33 Bit-plane)) only blocks (54).

(2) The bitplane separation unit 144 receives the depth information map block represented by n bits, and divides it into n bitplane blocks. The bitplane separator 144 then outputs the n bitplane blocks separated to the XOR operator 146 and / or the bitplane encoder 148. Although the bitplane separator 144 is shown separately from the bit rate controller 142 in FIG. 10, it should be noted that this is for convenience of description only. For example, the bit rate controller 142 and the bitplane separator 144 combine the input depth map block or input into one component into N bitplane blocks in the bitplane separator 144 first. After separation, the number of bitplane blocks output from the bit rate controller 142 to the XOR operator 146 and / or the bitplane encoder 148 may be limited to n.

(3) The XOR operator 146 performs an XOR operation on all or some of the bitplane blocks among the n bitplane blocks output from the bitplane separator 144. For example, as described with reference to FIG. 8 or 9, which bitplane block among the n bitplane blocks is to be performed by XOR operation and the amount of bits generated when coding the original bitplane block. It is possible to determine by comparing the amount of bits that occur when coding the bitplane block after performing, but the present embodiment is not limited thereto. The XOR operator 146 generates predetermined information (eg, XOR operation information such as 'XOR flag') indicating whether the XOR operation is performed on the corresponding bitplane block, and transmits the generated information to the multiplexer 160. Is included in the bitstream and transmitted to the decoding end.

13 is a flowchart illustrating an example of a method of determining whether to perform an XOR operation in the XOR operator 146. Referring to FIG. 13, first, an amount of bits (Amount _A ) generated by bit-plane coding a currently input bitplane block as it is, and bits generated by bit-plane coding after performing an XOR operation with a reference bitplane Obtain the amount AOB _B (60). Then, the two generated bit amounts AOB _A and AOB _B are compared (61). As a result of the comparison, only when the amount of bits (Amount A Bit _A ) generated by bit-plane coding the bit-plane block as it is, the XOR operation is performed on the bit-plane block (62). Therefore, when the bit amount AOB _B generated by bit-plane coding after performing the XOR operation with the reference bit plane is larger, the corresponding bitplane block is directly transmitted to the bitplane encoding unit 148 without performing the XOR operation. Is entered. In the case where both are the same, whether to perform an XOR operation may be arbitrarily determined.

When the XOR operation is performed by the XOR operation unit 146, a reference bit may be used as a reference bitplane or a bitplane block at a lower level may be used as a reference bitplane. There is no particular limitation on how the plane is determined. For example, if the reference bitplane is a bitplane block of one level higher level, the XOR operation unit 146 may include a bitplane block to be currently coded and a bitplane block of one level higher level (that is, a bitplane coded immediately before). XOR operation can be performed for each pixel (block).

The bitplane encoding unit 148 encodes the current bitplane block by using a predetermined encoding algorithm. Here, the current bitplane block refers to a bitplane block to be coded in the depth map block, and is a bitplane block in which an XOR operation is performed in the XOR operator 146 or a bitplane separation without passing through the XOR operator 146. It may be a bitplane block directly input from the unit 144. The bitplane encoding unit 148 may perform encoding by another method according to whether the current video is an intra picture or an inter picture, or may perform encoding using only one method regardless of the type of the intra picture or the inter picture. It may be. The former case will be described below.

If the current video is an intra picture, encoding may be performed according to the method described above with reference to FIG. 8. More specifically, the bitplane encoding unit 148 has a case in which all pixel values of the current bitplane block are '0 (0)' or '255 (1)' and '0 (0)' and '255 (1)'. In this case, the encoding can be performed separately. For example, in the former case, only bitplane block mode information may be coded. On the other hand, in the latter case, the bitplane encoding unit 148 may perform encoding on the current bitplane block according to a predetermined encoding algorithm suitable for encoding a binary image. The result encoded by the bitplane encoding unit 148 (encoded data and bitplane block mode information) is output to the multiplexer 160.

14 is a block diagram illustrating an example of a detailed configuration of a bitplane encoding unit 148 for an intra picture. The bitplane encoding unit 148 of FIG. 14 may be an example of an apparatus for implementing the encoding method described with reference to FIG. 8. The method for encoding the bitplane block of the intra picture in the bitplane encoding unit 148 according to FIG. 14 is binary shape coding of MPEG-4 Part-2 Visual (ISO / IEC 14496-2), which is an international video standard. Coding) is applied to the method, which will be described in detail below. However, the configuration of the bitplane encoding unit 148 and the encoding algorithm for the intra picture used in the bitplane encoding unit 148 should not be interpreted as being limited thereto.

Referring to FIG. 14, the mode determiner 1480 is to determine a mode of an input current bitplane block. For example, if the pixel values of the current bitplane block are all '255 (1)', the mode of the current bitplane block is set to predetermined block mode information indicating this, for example, 'all_1 mode', and the pixel values are all If it is '0 (0)', it may be set to predetermined block mode information indicating this, for example, 'all_0 mode'. In this case, only block mode information for the current bitplane block is encoded, input to the multiplexer 160, and included in the bitstream.

If the pixel values of the current bitplane block are not all the same value, the block mode information of the current bitplane block may be set to a value indicating a predetermined 'binary image coding technique' used to encode the corresponding bitplane block. Can be. For example, if you are coding a current bitplane block that contains a mixture of '0 (0)' and '255 (1)' using Context-based Arithmetic Encoding (CAE), The block mode information of the block may be set to a value indicating 'intraCAE mode'. In this case, the current bitplane block is encoded using the CAE method in the CAE encoding unit 1486, and the CAE encoded result (encoded data) is input to the multiplexer 160 together with the block mode information and included in the bitstream. do.

The CAE encoding unit 1486 may perform encoding using a known CAE encoding method. Alternatively, the CAE encoding unit 1486 may perform encoding by appropriately applying a known CAE encoding method in consideration of the characteristics of the bitplane of the depth map. A reference bitplane is required for CAE encoding. The reference bitplane is generated from a bitplane block of a reference picture that has passed through the bitplane separator 1462 and / or adaptively the XOR operator 1484. The reference picture may be an image before it is decoded by the DCT based encoding unit A (see FIG. 10) and input to the deblocking filter 122 or may be an image decoded by the bitplane coding unit B (see FIG. 10). The reference image is input to the bitplane separation unit 1462 and separated into a plurality of bitplane blocks in block units. Also, among the separated bitplane blocks, a bitplane block having the same level as the current bitplane block is adapted by the XOR operator 1484 according to whether the current bitplane block is a block in which an XOR operation is performed by the XOR operator 146. In general, the XOR operation is performed and then input to the CAE encoding unit 1486 as a bitplane block constituting the reference bitplane.

FIG. 15 is a block diagram illustrating an example of a detailed configuration of a bitplane encoding unit 148 for an inter picture. The bitplane encoding unit 148 of FIG. 15 may be an example of an apparatus for implementing the encoding method described with reference to FIG. 9. In addition, a method of encoding a bitplane block of an inter picture in the bitplane encoding unit 148 according to FIG. 15 also includes binary shape coding of MPEG-4 Part-2 Visual (ISO / IEC 14496-2), which is an international video standard. Coding) is applied to the method, which will be described in detail below. However, the configuration of the bitplane encoding unit 148 and the encoding algorithm for the inter picture used in the bitplane encoding unit 148 should not be interpreted as being limited thereto.

Referring to FIG. 15, the mode determiner 148a is to determine a mode of an input current bitplane block. For example, as described with reference to FIG. 9, a current inputted using a bitplane block of a reconstructed image and / or a reference image that has been adaptively passed through the bitplane separation unit 148b and the XOR operation unit 148c may be used. The mode of the bitplane block can be determined by 'No Update Without MV' mode, 'all_1' mode, 'all_0' mode, 'No Update with MV' mode, 'intraCAE encoding' mode, and / or 'interCAE encoding' mode. have.

If the mode of the current bitplane block is determined as the 'No Update Without MV' mode, the 'all_1' mode, or the 'all_0' mode, only the block mode information may be encoded and input to the multiplexer 160. In this case, the mode determination unit 148a may perform the algorithm described in steps 22 to 25 of FIG. 9 to determine the mode of the current bitplane block.

When the mode of the current bitplane block is determined as the 'No Update with MV' mode, the block mode information and the motion vector MV may be encoded and input to the multiplexer 160. In this case, step 26 of FIG. 9 may be performed by the motion predictor 148d, and steps 27 and 28 may be performed by the motion compensator 148e and the mode determiner 148a.

In addition, when the mode of the current bitplane block is determined as the 'intraCAE encoding' mode or the 'interCAE encoding' mode, the block mode information is encoded and input to the multiplexer 160, but the data of the current bitplane block is transmitted to the CAE encoding unit. The data may be input to 148f and encoded using the intra CAE method or the inter CAE method, and then input to the multiplexer 160. In this case, determining the 'intraCAE encoding' mode or the 'interCAE encoding' mode may be performed by the mode determining unit 148a or by the CAE encoding unit 148f. In addition, the CAE encoding unit 148f may perform encoding by using a known intra CAE method or an inter CAE method as it is, or by appropriately applying the same. In order to perform intra CAE encoding and / or inter CAE encoding in the CAE encoding unit 148f, a reference bit plane is required, which is a bit plane separation unit 148b and / or adaptively an XOR operator 148c. Is generated from a bitplane block of a reconstructed picture (or reference picture) which has undergone the processing.

FIG. 16 illustrates an example of a method of configuring a reference bitplane used in CAE encoding by the CAE encoding unit 1486 of FIG. 14 or the CAE encoding unit 148f of FIG. 15. In FIG. 16, the current bitplane block is a block indicated by solid lines among the five blocks. Referring to the lower right of FIG. 16, it can be seen that the current bitplane block is an MSB-3 bitplane block, which is a block in which an XOR operation is performed by the XOR operator 146 (see FIG. 10). However, it will be apparent to those skilled in the art that the reference bitplane can be configured by applying the method shown in FIG. 16 even if the current bitplane block is a bitplane block of another level or a block that has not performed an XOR operation.

As shown in FIG. 16, in order to perform CAE encoding on the MSB-3 bitplane block in the CAE encoding unit 1486 or 148f, the same level as the current bitplane block in the blocks adjacent to the current block (MSB-3). Bit) of the reference bitplane (four blocks indicated by dashed lines among the five middle blocks in FIG. 16). To this end, the bitplane separation unit 1462 or 148b separates the blocks adjacent to the current block, that is, the reference blocks, into the bitplane blocks in the reference picture. Since the current bitplane (MSB-3 bitplane) block to be coded is a block in which the XOR operation is performed in the XOR operation unit 146, the XOR operation unit 1484 or 148c also performs an XOR operation on the MSB-3 bitplane block of the reference block. The operation is performed. As described above, the MSB-3 bitplane blocks of each of the reference blocks on which the XOR operation is performed in the XOR operator 1484 or 148c form the reference bitplane.

In the above, the method of encoding the bitplane blocks using the CAE encoding algorithm in the bitplane encoding unit 148 has been described. However, embodiments of the present invention are not limited thereto, and the bitplane encoding unit 148 may encode n bitplane blocks by using another encoding algorithm. For example, the bitplane encoding unit 148 uses a known Run Length Coding (RLC) method and a Variable Length Coding (VLC) method, or known context-based adaptive binary arithmetic. Coding (Context-based Adaptive Binary Arithmetic Coding (CABAC)) may be applied, or the bitplane may be encoded by using a known joint bi-level image processing group (JBIG) method or quad tree method.

10, the information and data output from the bitplane encoding unit 148 are output to the bitstream through the multiplexer 160, and again in the bitplane block unit for encoding of the depth map. Decoding is done. To this end, the output of the bitplane encoding unit 148 is input to the bitplane decoding unit 150 to finally pass through the XOR operator 152, the bitplane combiner 154, and the depth map configuration unit 156. The reconstructed depth map block is output. This will be described in more detail below.

In the bitplane decoding unit 150, a process opposite to that of the bitplane encoding unit 148 described above is performed. More specifically, the bitplane decoding unit 150 reconstructs the bitplane block by using the encoded data and / or mode information output from the bitplane encoding unit 148. The reconstructed bitplane block may be a bitplane block in which an XOR operation is performed in the encoding step or a bitplane block in which the XOR operation is not performed. The former is output to the XOR operation unit 152, but the latter is a bitplane combining unit 154. Will be displayed.

In order to generate a reconstructed bitplane block, the bitplane decoding unit 150 has a configuration corresponding to the bitplane encoding unit 148.

For example, if the bitplane encoding unit 148 includes a configuration as shown in FIG. 14, the bitplane decoding unit 150 also has an MPEG-4 Part-2 Visual, as shown in FIG. 17. It may include a configuration for applying the binary shape decoding method for the intra picture of (ISO / IEC 14496-2). 17 is a block diagram showing an example of a detailed configuration of the bitplane decoding unit 150.

Referring to FIG. 17, when the mode of the bitplane block to be decoded is 'all_0 mode' or 'all_1 mode', all pixel values in the block are equal to '0 (0)' in the same level block decoding unit 1500. Or create a bitplane block of '255 (1)'. On the other hand, when the mode of the bitplane block to be decoded is the 'intraCAE mode', the CAE decoding unit 1506 outputs the reconstructed bitplane block by performing binary arithmetic decoding (CAE decoding) using the reference bitplane. . The reference bitplane may be made through the same process as described with reference to FIG. 14. More specifically, the input reference image is divided into a plurality of bitplane blocks in the bitplane separation unit 1502, and the separated bitplane blocks constitute a reference bitplane used in CAE decoding in the CAE decoding unit 1506 as it is. Alternatively, a bitplane block in which an XOR operation is performed by the XOR operator 1504 with respect to the separated bitplane block may configure the reference bitplane. In this case, whether the bitplane block separated by the bitplane separation unit 1502 passes through the XOR operation unit 1504 (that is, whether the XOR operation is performed) is determined by whether the current bitplane block to be decoded is XOR. The operation unit 146 determines whether or not the block in which the XOR operation is performed.

If the bitplane encoding unit 148 includes the configuration as shown in FIG. 15, the bitplane decoding unit 150 also uses the inter picture of MPEG-4 Part-2 Visual (ISO / IEC 14496-2). It may include a configuration for applying a binary shape decoding method for. An example of the detailed configuration of such a bitplane decoding unit will be described later (see FIG. 19).

10, the XOR operator 152 performs an XOR operation on the reconstructed bitplane block. In this case, instead of performing the XOR operation on all reconstructed bitplane blocks, the XOR operation is performed only on the bitplane block in which the XOR operation unit 146 performs the XOR operation. Whether the current reconstructed bitplane block is a block in which the XOR operation is performed in the XOR operation unit 146 may be known from XOR operation information indicating the XOR operation, for example, an XOR flag.

The bitplane combiner 154 restores the depth map block composed of n bits from the reconstructed bitplane blocks. The reconstructed bitplane block may be a reconstructed bitplane block directly output from the bitplane decoding unit 150 and / or a bitplane block output after the XOR operation is performed by the XOR operation unit 152.

In addition, the depth map configuration unit 156 generates an N-bit depth map block from the restored n-bit depth map block. The bitplane block for the lower (Nn) bits added to the n-bit depth map to configure the N-bit depth map block may be a bitplane block in which pixel values are all composed of '0 (0)'. . That is, by setting all the lower (Nn) bits of each pixel added to constitute the depth map block to have a value of '0', the N-bit depth map block can be restored from the n-bit depth map block. . As described above, the depth map block reconstructed by the depth map configuration unit 156 is used as a reference image for intra prediction in the DCT-based encoding unit A or is input to the deblocking filter unit 122 to deblocking filtering. The process may be performed and then stored in the reconstructed image buffer of the image buffer 130 and / or used to configure the reference bitplane in the bitplane encoding unit 148 and the bitplane decoding unit 150.

FIG. 18 is a block diagram illustrating an apparatus for decoding a depth map, that is, a depth map decoder according to an embodiment of the present invention. The depth map decoder 200 according to the present embodiment (the DCT-based decoding unit D as well as the bitplane decoding unit C) also corresponds to the depth map encoder 100 described with reference to FIG. 10. Therefore, the configuration of the depth map decoder 200 may also be changed to correspond to the configuration of the depth map encoder 100. The depth map decoder 200 also performs decoding in units of blocks of a predetermined size (M ㅧ N block). Here, there is no particular limitation on the size of the M ㅧ N block. Hereinafter, an apparatus and method for decoding a block-based depth map according to an embodiment of the present invention will be described with reference to FIG. 18.

Referring to FIG. 18, the apparatus for decoding a block-based depth map according to the present embodiment, that is, the depth map decoder 200 includes a bitplane decoding unit C and a DCT based decoding unit D. Referring to FIG. The bitplane decoding unit C corresponds to the bitplane encoding unit B of FIG. 10, and reconstructs a depth map in units of bitplane blocks from a bitstream input from a demultiplexer 210 and adapts the bitmap block. In particular, the depth map block is reconstructed by combining the reconstructed bitplane blocks in which the XOR operation is performed and / or in which the XOR operation is not performed. The detailed configuration and operation of the bitplane decoding unit C will be described later.

On the other hand, the DCT-based decoding unit (D) corresponds to the DCT-based encoding unit (A) of FIG. 10, and decodes the depth map in units of blocks by the same decoding technique as the existing technique of decoding image information. To this end, the DCT based decoding unit D may have the same configuration as that of the decoding unit of H.264 / Advances Video Coding (AVC). However, the present embodiment is not limited thereto, and the DCT-based decoding unit D also corresponds to the configuration of the DCT-based encoding unit A, and also MPEG-1, MPEG-2, MPEG-4 Part 2 Visual, and VC (Video Coding). Decoding unit such as) -1 or a decoding unit according to a new video coding technique to be defined in the future.

Referring to FIG. 18, the depth map decoder 200 reconstructs depth information by performing a decoding process on any one of a bitplane decoding unit C and a DCT based decoding unit D according to coding mode information. Print a map block. The coding mode information is included in the bitstream. The demultiplexer 210 first decodes the coding mode information from the input bitstream, and then decodes the bitstream according to the coding mode information. To the base decoding unit D. For example, when the coding mode information indicates 'bitplane coding', the coded data output from the demultiplexer 210 is decoded in the bitplane decoding unit C. same.

(1) First, the bitplane decoding unit 212 restores n bitplane blocks by receiving encoded data from the demultiplexer 210 and performing a predetermined decoding algorithm. Herein, n bitplane blocks indicate the number of bitplane blocks that are actually coded in order to obtain a desired bit rate in the encoding step among the N bitplanes. The bitplane decoding unit 212 has a configuration corresponding to the bitplane encoding unit 148 (see FIG. 10).

For example, the bitplane decoding unit 212 may include a decoding unit having the same configuration as the bitplane decoding unit 150 shown in FIG. 17. The bitplane decoding unit 212 having such a configuration may reconstruct the bitplane block of the block constituting the intra picture by using a CAE decoding technique and / or a same level block decoding technique for the intra picture. In addition, the bitplane decoding unit 212 may also include a decoding unit for reconstructing the bitplay block of the block constituting the inter picture. FIG. 19 illustrates an example of a configuration of the bitplane decoding unit 212 for the inter picture. A block diagram is shown.

Referring to FIG. 19, the received bitstream is demultiplexed by the demultiplexer 210, and as a result, the mode information, the motion vector (MV), the CAE-encoded bitstream, etc. of the current bitframe block are decoded. Is entered. The bitplane decoding unit 212 restores the bitplane blocks by decoding based on mode information among the input data.

If the mode information is 'No Update Without MV' mode, the motion compensator 2126 performs motion compensation using the motion vector prediction value MVp as a motion vector, and as a result, the motion-compensated bitplane block, that is, motion The reference bitplane block corresponding to the vector prediction value MVp is output to the reconstructed bitplane block. The motion vector prediction value MVp may be determined using various methods, and may be the same method as that used by the bitplane encoding unit 148 (see FIG. 10) to determine the motion vector prediction value MVp.

If the mode information is the 'all_0 mode' or the 'all_1 mode', the same level block decoding unit 2120 may determine a bitplane block in which all pixel values in the block are '0 (0)' or '255 (1)'. Restore it and print it out. When the mode information is 'No Update with MV' mode, the motion compensator 2126 performs motion compensation using the motion vector MV input from the demultiplexer 210, and as a result, the motion compensated bit. The plane block, that is, the reference bitplane block corresponding to the motion vector MV, is output to the reconstructed bitplane block. In addition, when the mode information is an 'intraCAE' mode or an 'interCAE' mode, the CAE decoding unit 2128 may output a reconstructed bitplane block by performing binary arithmetic decoding (eg, intraCAE decoding or interCAE decoding). .

A reference bitplane for performing motion compensation or CAE decoding may be made through the same process as described above. More specifically, the input reference image (or reconstructed image) is separated into a plurality of bitplane blocks in the bitplane separator 2122. In addition, among the separated bitplane blocks, a bitplane block having the same level as the current bitplane block is input to the motion compensation unit 2126 or the CAE decoding unit 2128 without being subjected to an XOR operation and used as a reference bitplane. Alternatively, a bitplane block in which an XOR operation is performed by the XOR operator 2124 with respect to the bitplane block may be used as a reference bitplane. In this case, whether or not the bitplane block separated by the bitplane separator 2122 passes through the XOR operator 2124 (that is, whether the XOR operation is performed) is determined by encoding the current bitplane block to be decoded. It may be determined according to whether it is encoded after the XOR operation is performed in the process or encoded without performing the XOR operation (eg, the value of the XOR flag).

(2) Subsequently, referring to FIG. 18, the XOR operator 214 adaptively performs an XOR operation on the bitplane block reconstructed by the bitplane decoder 212. For example, the XOR operator 214 may or may not perform an XOR operation on the corresponding bitplane block according to the decoded XOR operation information, for example, the value of the XOR flag. The XOR operator 214 may perform an XOR operation using a bitplane block for a bit higher than the current bitplane block as a reference bitplane, but is not limited thereto. That is, the reference bitplane for the XOR operation in the XOR operator 214 may be determined in the same manner as the reference bitplane used when the XOR operator 146 (see FIG. 10) performs the XOR operation.

(3) The bitplane combiner 216 combines the bitplane block output from the bitplane decoding unit 212 and / or the bitplane block output from the XOR operator 214 so that each pixel is represented by n bits. Create an information map block.

(4) The depth map configuration unit 218 generates the depth map block of N bits of the actual depth map from the n-bit depth map block output from the bitplane combiner 216. . For example, the depth map constructing unit 218 adds (Nn) lower bits to each pixel of the n-bit depth map block, that is, (Nn) bitplane blocks to bits for the lower level bits. By adding as a plane block, an N-bit depth map block can be constructed. In this case, the lower bit added may be any predetermined value, for example all '0' (ie, each of the (Nn) bitplane blocks added has a value of all pixels '0'), It is not limited only to this.

Alternatively, the depth map configuration unit 218 may use predetermined information included in the bitstream, that is, information about (N-n) bitplane blocks to be added. For example, the pixel values of the (Nn) bit plane blocks added according to the information indicated by the above information are all set to '0 (0)', all to '255 (1)', or adjacent bit plane blocks. The pixel value may be determined by a method such as a padding method.

(5) Finally, the depth map block represented by N bits output from the depth map configuration unit 218 is used as a reference block for intra prediction in the intra prediction unit 228 of the DCT-based decoding unit (D), or And / or in the bitplane decoding unit 212 of the bitplane decoding unit C, may be used as a reference block for CAE decoding. After the N-bit depth map block passes through the deblocking filter unit 232, the depth map decoder 200 is stored in the reconstructed image buffer of the image buffer 240 and / or as a reconstructed depth map block 200. Can be output from

18, when the coding mode information indicates 'DCT-based coding', the coded data output from the demultiplexer 210 is decoded in the DCT-based decoding unit D. The specific method is as follows. The DCT-based decoding unit D first generates a prediction block, and then adds the prediction block to the residual block obtained by decoding the input bitstream, thereby generating a reconstructed depth map block. Briefly described as follows.

(1) Generation of the prediction block is performed according to the prediction mode of the current block. For example, when the prediction mode of the current block is the intra prediction mode, the intra prediction unit 228 generates the prediction block by performing spatial prediction using the already decoded neighboring pixel values of the current block. On the other hand, when the prediction mode of the current block is the inter prediction mode, the inter prediction unit 230 searches for a corresponding region in the reference image stored in the reference image buffer of the image buffer 240 using the motion vector to compensate for the motion. By performing the prediction block generation.

(2) The entropy decoding unit 220 outputs quantized coefficients by performing entropy decoding on the bitstream input from the demultiplexer 210 according to the probability distribution.

(3) The inverse quantizer 222 outputs the transformed coefficient by performing an inverse quantization process on the quantized coefficients output from the entropy decoding unit 220, and the inverse transformer 224 outputs the transformed coefficients. The residual block is generated by performing an inverse transform process. The adder 226 sums the residual block output from the inverse transformer 224 and the prediction block input from the intra predictor 228 or the motion compensator 230 and outputs the reconstructed block. The reconstructed block is input as a reference block for performing bitplane decoding in the bitplane decoding unit C and / or the deblocking filtering unit 232 performs a deblocking filtering process and then the image buffer 240. Stored in the reconstructed image buffer and / or output as a reconstructed depth map block.

In order to verify the effect of the coding method of the block-based depth map according to the embodiment of the present invention described above, the method according to the embodiment of the present invention using JM (Joint Model) 13.2, which is H.264 / AVC reference software And simulation were performed according to the conventional method (encoding / decoding method according to H.264 / AVC), and the experimental conditions are shown in Table 1.

Resolution and Frame Rate 1024x768, 15 Hz Frames 100 frames Video Format YUV 4: 0: 0 Quantization Parameter 22, 27, 32, 37 Predictive structure I-P-P-P- Entropy Coding Method CAVLC

 20 and 21 are rate-distortion curves of peak signal-to-noise ratio (PSNR) versus bit rate for an existing method using H.264 / AVC and a method according to an embodiment of the present invention, respectively. FIG. 20 illustrates a 'Breakdancers' image sequence, which is one of test image sequences, and FIG. 21 illustrates a 'Ballet' image sequence, which is one of test image sequences. 20 and 21, it can be seen that the result of the method according to the embodiment of the present invention is significantly improved in performance than the result of the conventional method. More specifically, in the depth map of the "Breakdancers" image sequence, the performance was improved by about 1.2 dB when the performance was compared using the BD-PSNR method, which indicates the improvement of the average PSNR, and the average bit-rate reduction was reduced. By comparing the performance using the BD-rate method, the bit rate was reduced by about 16.8%. In the depth map of the "Ballet" video sequence, the performance was improved by about 1.3 dB when using the BD-PSNR method, and the bit rate was reduced by about 18.7% when the performance was compared using the BD-rate method. .

FIG. 22 is a diagram illustrating image quality when a block-based depth map coding method is used according to an embodiment of the present invention. FIG. 22A illustrates an original depth map showing a part of an image sequence of 'Breakdancers'. FIG. 22B is a depth map image obtained by encoding and decoding (a) using an existing method (H.264 / AVC), and FIG. 22 (c) illustrates (a) according to the present invention. A depth map image encoded, decoded, and reconstructed according to an embodiment of the present invention. Referring to FIG. 22, it can be easily confirmed that the image quality deteriorates near the boundary of the reconstructed image, in particular, when the existing method is used as it is. On the other hand, according to the embodiment of the present invention, it can be seen that the image quality of the object boundary portion is significantly improved compared to the conventional method.

The above description is only an embodiment of the present invention, and the technical idea of the present invention should not be construed as being limited by this embodiment. The technical idea of the present invention should be specified only by the invention described in the claims. Therefore, it will be apparent to those skilled in the art that the above-described embodiments may be implemented in various forms without departing from the spirit of the present invention.

1 is a diagram illustrating an example of rendering in an FTV system to which 3D video coding is applied.

FIG. 2A is a 'Breakdancers' video sequence, which is one of MVC test sequences, which is an example of an actual video, and FIG. 2B is a depth map image of the 'Breakdancers' video sequence.

3A and 3B are graphs for illustrating the level distribution of image information and the level distribution of depth information in the image of FIG. 2, respectively.

4A is a depth graph of a depth map block including a boundary of an object, and FIG. 4B is a depth graph of a depth map block in a background part.

5A and 5B are diagrams representing the real image of FIG. 2A and the depth map image of FIG. 2B, respectively, in bit plane units, and FIG. This is shown in units of bit planes by performing gray coding on the depth map image of b).

6 (a) is a part of the depth map of FIG. 2 (b), and FIGS. 6 (b), 6 (c) and 6 (d) are each arbitrarily selected from the depth map of FIG. The figure shows the selected block and the bitplane blocks greyed out these bitplane blocks.

FIG. 7 illustrates an example of a bitplane block having little correlation with bitplanes for bits of adjacent levels.

8 is a flowchart illustrating an example of an encoding method for a bitplane of a depth map using an adaptive XOR operation according to an embodiment of the present invention.

9 is a flowchart illustrating an example of an encoding method for an inter picture according to another embodiment of the present invention.

10 is a block diagram showing the configuration of a depth map encoder according to an embodiment of the present invention.

FIG. 11 is a flowchart illustrating an example of selecting a mode having a higher coding efficiency among a DCT based coding mode and a bitplane coding mode in the depth map encoder illustrated in FIG. 10.

FIG. 12 is a flowchart illustrating an example of a method of determining a bit rate in a bit rate controller of the depth map encoder of FIG. 10.

FIG. 13 is a flowchart illustrating an example of a method of determining whether to perform an XOR operation in the XOR calculator of the depth map encoder of FIG. 10.

FIG. 14 is a block diagram illustrating an example of a detailed configuration of a bitplane encoding unit in the depth map encoder of FIG. 10.

FIG. 15 is a block diagram illustrating an example of a detailed configuration of a bitplane encoding unit of FIG. 10 for an inter picture.

FIG. 16 is a diagram illustrating an example of a method of configuring a reference bitplane in the CAE encoding unit of the bitplane encoding unit of FIG. 14 or FIG. 15.

FIG. 17 is a block diagram illustrating an example of a detailed configuration of a bitplane decoding unit in the depth map encoder of FIG. 10.

18 is a block diagram illustrating a configuration of a depth map decoder according to an embodiment of the present invention.

19 is a block diagram illustrating an example of a configuration of a bitplane decoding unit of FIG. 18 for an inter picture.

20 is a rate-distortion curve of a peak signal-to-noise ratio (PSNR) versus a bit rate in an existing method using H.264 / AVC and a method according to an embodiment of the present invention. RD) curve) is an example of the experimental results.

21 is another example of an experimental result showing a rate-distortion curve of a PSNR versus a bit rate in an existing method using H.264 / AVC and a method according to an embodiment of the present invention.

(A) of FIG. 22 is an original depth map image showing a part of a 'Breakdancers' image sequence, and FIG. 22 (b) encodes and decodes (a) by the conventional method (H.264 / AVC). It is a reconstructed depth map image, and FIG. 22C is a depth map image obtained by encoding and decoding (a) according to an embodiment of the present invention.

Claims (44)

delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete In the method for decoding a depth map block represented by N bits, Restoring n (≤N) bitplane blocks by performing decoding according to block mode information for each bitplane block; Adaptively performing an XOR operation with the reference bitplane block on the n recovered bitplane blocks according to XOR operation information for each bitplane block; And Combining the n recovered bitplane blocks in which the XOR operation is performed and / or in which the XOR operation is not performed. The method of claim 26, If n is less than N, depth information further includes a reconstruction step for reconstructing the depth map block represented by N bits by adding (Nn) lower level bits for each pixel of the combined bitplane block. Method of decoding the map. The method of claim 27, And the block mode information is one of an 'all_0 mode', an 'all_1 mode', and a 'binary image coding mode'. The method of claim 27, The block mode information is a decoding method of a depth map, which is one of 'No Update Without MV mode', 'all_0 mode', 'all_1 mode', 'No Update with MV mode', and 'binary image coding mode'. The method of claim 28 or 29, The binary image coding mode is a method of decoding a depth map representing a context-based arithmetic encoding (CAE) technique. 31. The method of claim 30, The reference bitplane for performing the CAE scheme is the same as the current bitplane block among M bitplane blocks generated by performing bitplane separation on each of the reference depth map blocks adjacent to the current depth map block. A method of decoding a depth map generated using bitplane blocks output by performing an XOR operation adaptively on a bitplane block for a bit of a level. The method of claim 31, wherein And performing an XOR operation in generating the reference bitplane only when the XOR operation information of the current bitplane block indicates to perform an XOR operation. The method of claim 31, wherein And the reference depth map block is a depth map block reconstructed in the reconstruction step or a depth map block reconstructed using a DC-based decoding technique. The method of claim 27 or 28, The binary image coding mode includes a combination of a run length coding method and a variable length coding method, a context-based adaptive binary arithmetic coding (CABAC) technique, and a joint bi-level image processing group (JBIG). A method of decoding a depth map representing any one of a technique, and a quad tree technique. The method of claim 26, And the reference bitplane block is a reconstructed bitplane block for bits one level higher than the current bitplane block. delete delete delete delete delete delete delete delete delete

Images