JP2006246241A - Scalable transcoding method, deice, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To convert an image coding format of a luminance color difference signal to an arbitrary image coding format. <P>SOLUTION: Band-divided data by a wavelet filter is processed in simple operations such as rearrangement, deletion, etc., and is converted to the image coding format of the luminance color difference signal of different subsampling. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a scalable transcoding method, apparatus, and program, and more particularly, to a scalable transcoding method, apparatus, and program related to color image coding for efficiently transmitting and storing moving images.

  As encoding methods for digital images, methods such as JPEG, JPEG2000, MPEG-2, and MPEG-4 are known. The general encoding procedure and decoding procedure of these encoding systems are shown in FIG.

On the encoding side, RGB data is converted into an arbitrary image format in accordance with a user's device or the like as preprocessing for inputting data into the encoder, and encoding is performed on the arbitrary image format. On the decoding side, the opposite process is performed, and RGB data is decoded from the encoded data. The format conversion in FIG. 22 is performed by the operation shown in FIG. 23, and is composed of a color conversion unit and a subsample unit. Although the operation of the color conversion portion have been proposed various ones, generally expression ITU Recommendation _{601 (1) (K R =} 0.299, K B = 0.114) is used, R (red) , G (green) and B (blue) are converted into images of luminance component Y and color difference components Cb and Cr. The color-converted YCbCr data has the following characteristics.

  1. It is easy to maintain compatibility with a monochrome display.

  2. Compared with RGB, YCbCr has less correlation between components, and thus the coding efficiency is improved.

  3. This is suitable for audio-visual characteristics that are less sensitive to color difference components than luminance components.

  The color difference component is often sampled due to the above three characteristics, and the sub-sampling ratio of 4: 2: 2, 4: 2: 0 (4: 1: 1) is displayed by the sampler according to the quality requirement of the user. Converted to format. The correspondence relationship between each sampling rate, luminance signal, and color difference signal is as shown in FIGS. 24, 25, and 26. Hereinafter, a 4: 4: 4 image is represented as YCbCr444, a 4: 2: 2 image is represented as YCbCr422, 4: The 2: 0 image is denoted as YCbCr420.

YCbCr444，YCbCr422，YCbCr420の各符号化データ（この場合JPEG2000ファイルやMPEGファイル等）は、他のサブサンプリング率の画像と互換性は無く、図２７に示す例のように、YCbCr422ファイルをYCbCr420のデコーダで復号することはできない。YCbCr422の符号化データにするには復号処理とサブサンプリング率変換処理並びに、再符号化処理が必要となる。これらの処理は演算コストが多く処理に時間がかかる他、変換処理のための画像劣化も生じる。

Each encoded data of YCbCr444, YCbCr422, and YCbCr420 (in this case, JPEG2000 file, MPEG file, etc.) is not compatible with other sub-sampling rate images. Cannot be decrypted. Decoding processing, sub-sampling rate conversion processing, and re-encoding processing are required to make YCbCr422 encoded data. These processes are computationally expensive and take time to process, and also cause image degradation due to the conversion process.

  The sub-sampling rate conversion processing is equivalent to converting the image size, and the scalable function of spatial resolution is also provided with MPEG2, MPEG-4, and JPEG2000. In MPEG-2 and MPEG-4, the spatial scalability function is made possible by layering data, and there is a method in which this method is applied to image format conversion.

This method introduces a coding mechanism for the odd-numbered and even-numbered scan lines for the color difference signal component in MPEG2 and MPEG4, so that codes with different subsampling rates can be obtained simply by truncating the bitstream. This is a method for creating digitized data (see, for example, Non-Patent Document 1). In the JPEG2000 encoding system, four priorities are assigned, ie, layer, spatial resolution level, position, and component, and the arrangement can be arbitrarily selected by the user in five patterns. As a result, the data structure of JPEG2000 has scalability, and resolution conversion is possible by truncating arbitrary data. However, the data processing method in which arbitrary data is truncated is not included in the JPEG2000 standard and cannot be decoded depending on the decoder specifications.
L.Yuan, G.Shen, F.Wu, S.Li, W.Gao, “Color Space Compatible Coding Framework for YUV422 Video Coding” ICASP 2004

  The above-described conventional method of Non-Patent Document 1 is a method that can create YCbCr420 from YCbCr422 by a simple operation, and it is desired to deal with a wide range of equipment from broadcasting equipment to consumer equipment. However, the method of Non-Patent Document 1 cannot be decoded by a general-purpose decoder currently used.

  The present invention has been made in view of the above points, and provides a method, apparatus, and program related to color image encoding that can be reproduced by a general-purpose JPEG2000 decoder and can generate an arbitrary sub-sampled image by a simple operation. For the purpose.

The present invention (Claim 1) is directed to image coding of luminance color difference signals.
When data divided by the wavelet filter of the encoding device is input to the scalable transcoding device, the data is rearranged or discarded.
The luminance color difference signal of different sub-sampling is converted into an image encoding format.

The present invention (Claim 2), when converting to a different sub-sampling image encoding format,
When the data band-divided by the wavelet filter of the encoding device is input to the scalable transcoding device, subband coefficient truncation processing, partial wavelet processing and rearrangement of the encoded data of the frequency band of an arbitrary component are performed, Convert to a different sub-sampling image encoding format.

According to the present invention (Claim 3), in JPEG2000 encoding,
When encoded data is input from the encoding device to the scalable transcoding device, subband coefficient truncation and rearrangement of the encoded data in the frequency band of an arbitrary component are performed,
Rewrite the header of the corresponding part and convert it to JPEG2000 encoding format.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (Claim 4) is a scalable transcoding device for performing image coding of a luminance color difference signal,
Means for rearranging the data when the band-divided data is input by the wavelet filter of the encoding device 200;
Or a means of discarding part of the entered data;
Transcoding means 110 comprising means for converting the luminance color difference signal of different sub-sampling into an image encoding format and outputting it to a decoding device.

The present invention (Claim 5) is a scalable transcoding device for converting to a different sub-sampling image encoding format,
When the band-divided data is input by the wavelet filter of the encoding device, subband coefficient truncation processing, partial wavelet processing, and rearrangement of the encoded data in the frequency band of any component are performed, and images of different subsampling are performed. Transcoding means comprising: means for converting into an encoding format and outputting to a decoding device.

The present invention (Claim 6) is a scalable transcoding device that performs JPEG2000 encoding,
When JPEG2000 data is input from the encoding device to the scalable transcoding device, means for subband coefficient truncation in the frequency band of arbitrary components and rearrangement of the encoded data;
Transcoding means comprising: means for rewriting the header of the corresponding part, converting to a JPEG2000 encoding format, and outputting to the decoding device.

The present invention (Claim 7) is a scalable transcoding program for converting an image encoding format into an arbitrary image encoding format,
A program for causing a computer to execute processing for realizing the scalable transcoding method according to claim 1.

  As described above, according to the present invention, the image format can be converted into a scalable manner with a simple operation, and data conversion according to the image display device and application can be performed.

  The generated encoded data can be played back by a general-purpose JPEG2000 decoder, and the processing time required for conversion can be shortened to eliminate redundant processing compared to conventional image format conversion. It can be increased by avoiding aliasing by using bandwidth limitation.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 2 is a system configuration diagram in one embodiment of the present invention.

  The system shown in the figure includes an encoding device 200, a transcoder 100, and a decoding device 300.

  The encoding apparatus 200 includes a wavelet transform unit 210 having a wavelet filter and an encoding unit 220 that encodes an image and generates a single compressed encoded data.

  The transcoder 100 is a scalable transcoder, which extracts a bit stream of a part of compressed data necessary for display format conversion, and in some cases, codes with different subsampling by a simple operation such as a rearrangement operation of the compressed data. The encoded data is output to the encoding device 300.

  The decoding apparatus 300 includes a decoding unit 310 that decodes encoded data acquired from the transcoder 100, and an inverse wavelet transform unit 320 that performs inverse wavelet transform on the decoded data.

  Various transcoders will be described below.

[First Embodiment]
In this embodiment, a transcoder capable of converting from a YCbCr444 display format to a YCbCr420 display format will be described.

  FIG. 3 is a basic configuration diagram of the transcoder according to the first embodiment of the present invention.

  The transcoder 100 shown in the figure has a data discarding unit 10 and a header rewriting unit 11. JPEG2000 data is input to the transcoder 100.

  Assume that the JPEG2000 data input to the transcoder 100 is as follows.

Display format: YCbCr444
Number of DWT levels: 2
Precinct: off
Progression order: RLCP
Layer: 1
FIG. 4 shows a JPEG200 data structure RLCP in the first embodiment of the present invention, and FIG. 5 shows Mallet division in the first embodiment of the present invention.

  Considering now that the image of YCbCr444 is converted to the image of YCbCr420, it becomes a problem of how to create an image with a 1/4 times the area of the color difference signal.

  In this embodiment, a method of using LL2, HL2, LH2, and HH2 in FIG. 6 as an area 1/4 image will be described.

  First, in the encoded image YCbCr444 input to the transcoder 100, the highest frequency band of the color difference component (CbCr) is discarded by the data discarding unit 10. As can be seen from FIG. 4, this is equivalent to the work of simply truncating the last data structure.

  Next, the header rewriting unit 11 rewrites the information of the corresponding part. In this case, there are three header portions to be rewritten, and XRsiz (2, 3) and YRsiz (2, 3) of the SIZ marker segment are rewritten, a component code style marker is added, and Psot of the SOT marker segment is rewritten.

  The YCbCr 420 output from the transcoder 100 has a feature that the number of DWT levels of color components is one less than the data before transcoding.

  FIG. 7 is a diagram showing an effect in the first embodiment of the present invention, and shows PSNR-Entropy characteristics for each color difference component. For the experiment, digital cinema test material StEM (https://www.dcimovies.com/) was used. In the conventional method, after the YCbCr444 image is decoded, the YCbCr420 image is created by the down-sampling method, and the second encoding is performed. In the conventional method, the best case of quality is assumed and work such as quantization is not performed in the first encoding.

[Second Embodiment]
Also in this embodiment, a transcoder capable of converting from the YCbCr444 display format to the YCbCr420 display format will be described.

  FIG. 8 is a basic configuration diagram of a transcoder according to the second embodiment of the present invention.

  The transcoder 100 shown in the figure includes an inverse entropy encoding unit 12, a data discarding unit 13, an entropy encoding unit 14, and a header rewriting unit 15.

  In the following, it is assumed that the JPEG2000 data input to the transcoder 100 is as follows, as in the first embodiment.

Display format: YCbCr444
Number of DWT levels: 2
Precinct: off
Progression order: RLCP
Layer: 1
As in the first embodiment described above, when converting an image of YCbCr444 into an image of YCbCr420, it becomes a problem of how to create an image having an area 1/4 times the color difference signal. In this embodiment, a method of using an area quarter image by using the downsampling method shown in FIG. 9 will be described.

  First, the YCbCr444 image input to the transcoder 100 is returned to wavelet coefficients by the inverse entropy encoding unit 12. Next, the data is discarded by the data discard unit 13 in the horizontal and vertical directions for half a minute. Next, the data is encoded by the entropy encoding unit 14, and the header rewriting unit 15 rewrites the corresponding portion of information. In this case, there are two header portions to be rewritten, the SIR marker segment XRsiz (2, 3) and YRsiz (2, 3) are rewritten, and the SOT marker segment Psot is rewritten.

  10 and 11 show the results of carrying out this embodiment. In the present embodiment, it is expected that the effect of aliasing is large and affects the reproduced image. For this reason, we used “baboon” and “barbara”, which are considered to have large image degradation due to aliasing because there are many high-frequency components in the experiment. In addition, assuming the best case of quality, work such as quantization was not performed in the first encoding.

[Third embodiment]
In the present embodiment, it is considered that a YCbCr444 image is converted into a YCbCr422 image by the same method as in the second embodiment. The configuration of the transcoder 100 is the same as that of the second embodiment.

  As a third embodiment, a method of creating an area 1/2 image (FIG. 12B) by using the downsampling method for FIG. 12 will be described.

  First, the YCbCr444 image input to the transcoder 100 is returned to wavelet coefficients by the inverse entropy encoding unit 12. Next, the data is discarded by the data discarding unit 13 for half a minute. Next, the information is encoded by the entropy encoding unit 14 and the corresponding portion of information is rewritten by the header rewriting unit 15. In this case, there are two header portions to be rewritten, the SIR marker segment XRsiz (2, 3) and YRsiz (2, 3) are rewritten, and the SOT marker segment Psot is rewritten.

  FIG. 13 and FIG. 14 show the results of performing the third embodiment. In the present embodiment, it is expected that the effect of aliasing is large and affects the reproduced image. Therefore, we used “baboon” and “barbara”, which are considered to have a large image degradation due to aliasing because there are many high-frequency components in the experiment. In the present embodiment, the best case of quality is assumed and work such as quantization is not performed in the first encoding.

[Fourth embodiment]
In this embodiment, a scalable transcoder that can convert from a YCbCr444 display format to a YCbCr422 display format is used.

  FIG. 15 is a basic configuration diagram of a transcoder according to the fourth embodiment of the present invention.

  The transcoder 100 shown in the figure includes an inverse entropy encoding unit 17, a data discarding unit 18, a partial wavelet unit 19, a partial inverse wavelet unit 20, an entropy encoding unit 21, and a header rewriting unit 22.

  In the following, it is assumed that JPEG2000 data input to the transcoder 100 is as follows.

Display format: YcbCr444
Number of DWT levels: 2
Precinct: off
Progression order: RLCP
Layer: 1
Considering now that a YCbCr444 image is converted to a YCbCr422 image, it becomes a problem of how to create an image having a half the area of the color difference signal. In the third embodiment, a half image is created by using the downsampling method. However, in this embodiment, LL2, HL2, LH2, HH2, and LH1 in FIG. A method for creating the second image will be described.

  First, the encoded image YCbCr444 input to the transcoder 100 is returned to the wavelet coefficient by the inverse entropy encoding unit 17. Next, the data discarding unit 18 discards the HL1 and HH1 components of the color difference component (CbCr). Next, the partial wavelet unit 19 creates LH1 to LH1 'and HH1', LL2 to LL2 'and HL2', and LH2 to LH2 'and HH2'. Next, the partial inverse wavelet unit 20 creates HL1 'from HL2 and HH2. Next, encoding is performed by the entropy encoding unit 21, and a corresponding portion is rewritten by the header rewriting unit 22. In this case, there are two header portions to be rewritten, the SIR marker segment XRsiz (2, 3) and YRsiz (2, 3) are rewritten, and the SOT marker segment Psot is rewritten.

  FIG. 16 shows the PSNR-Entropy characteristic for each color difference component as an effect when the fourth embodiment is performed. For the experiment, digital cinema test material StEM (https://www.dcimovies.com/) was used. In the conventional method, after the YCbCr444 image is decoded, the YCbCr422 image is created by the down-sampling method and encoded twice. Note that the conventional method and Propose1 are the cases where the best case of quality is assumed, and the work such as quantization is not performed at the first encoding, and Propose2 is the amount of information by the quantization at the first encoding. It is decreasing.

[Fifth embodiment]
In this embodiment, a scalable transcoder capable of converting from a YCbCr444 display format to a YCbCr422 display format is used.

  FIG. 17 is a basic configuration diagram of a transcoder according to the fifth embodiment of the present invention.

  The difference from the fourth embodiment described above is that the partial wavelet transform unit 19 and the partial inverse wavelet transform unit 20 are not provided. Instead, the operation of the wavelet transform unit 19 and the partial inverse wavelet transform unit 20 is replaced by the method shown in FIGS.

  In FIG. 18, pseudo wavelet transformation is performed by dividing the vertical line into two. In FIG. 19, the inverse wavelet transform is performed in a pseudo manner by combining one for each horizontal line. Other processes are the same as those in the fourth embodiment.

  It should be noted that the image quality is improved by using LL2 instead of LH2 (that is, creating LL1 'by the upsampling method) instead of creating LL1' from LL2 and LH2 for important parts such as LL1 ' It is thought to improve.

  The result at the time of performing this Embodiment of FIG. 20, FIG. 21 is shown. In the fifth embodiment, it is expected that the effect of aliasing is large and affects the reproduced image. For this reason, we used “baboon” and “barbara”, which are considered to have large image degradation due to aliasing because there are many high-frequency components in the experiment. The reproduction frequency of the color difference signal was not created from LL2 and LH2 by the method shown in FIG. 21, but LL2 was used instead of LH2.

  The operation of each component of the transcoder shown in FIGS. 2, 6, 15, and 17 in each of the above embodiments is constructed as a program and installed in a computer used as a transcoder, or a network It is possible to circulate through.

  In addition, the built program can be stored in a hard disk drive connected to a computer used as a transcoder, a portable storage medium such as a flexible disk or CD-ROM, and installed or distributed to the computer. It is.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  The present invention can be applied to a technique related to encoding and decoding for transmitting and storing moving images.

It is a principle block diagram of this invention. 1 is a system configuration diagram according to an embodiment of the present invention. 1 is a basic configuration diagram of a transcoder in a first embodiment of the present invention. It is a JPEG2000 data structure RLCP in the first embodiment of the present invention. It is a figure which shows the Mallat division | segmentation in the 1st Embodiment of this invention. It is a figure which shows the resolution of the color difference signal in the 1st Embodiment of this invention. It is a figure which shows the effect of the 1st Embodiment of this invention. It is a basic block diagram of the transcoder in the 2nd Embodiment of this invention. It is a figure which shows the resolution of the color difference signal in the 2nd Embodiment of this invention. It is a result (baboon) of the 2nd Embodiment of this invention. It is a result (barbara) of the 2nd Embodiment of this invention. It is a figure which shows the resolution of the color difference signal in the 3rd Embodiment of this invention. It is a result (baboon) of the 3rd Embodiment of this invention. It is a result (barbara) of the 3rd embodiment of the present invention. It is a basic block diagram of the transcoder in the 4th Embodiment of this invention. It is a figure which shows the effect in the 4th Embodiment of this invention. It is a basic block diagram of the transcoder in the 5th Embodiment of this invention. It is a figure for demonstrating the pseudo wavelet transformation in the 5th Embodiment of this invention. It is a figure for demonstrating pseudo-inverse wavelet transformation in the 5th Embodiment of this invention. It is a result (baboon) of the 5th Embodiment of this invention. It is a result (barbara) in the 5th embodiment of the present invention. This is a general encoding / decoding procedure. This is a general format conversion method. The spatial correspondence is 4: 2: 0. 4: 2: 2 spatial correspondence. 4: 4: 4 spatial correspondence. General system compatibility.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Data discarding part 11 Header rewriting part 12 Reverse entropy encoding part 13 Data discarding part 14 Entropy encoding part 15 Header rewriting part 17 Reverse entropy encoding part 18 Data discarding part 19 Partial wavelet part 20 Partial inverse wavelet part 21 Entropy encoding Unit 22 header rewriting unit 24 inverse entropy coding unit 25 data discarding unit 26 data switching unit 27 entropy coding unit 28 header rewriting unit 100 scalable transcoding device 110 transcoding means 200 coding device 210 wavelet transform unit 220 coding unit 300 Decoding device 310 Decoding unit 320 Inverse wavelet transform unit

Claims (7)

In image coding of luminance color difference signals,
When data divided by the wavelet filter of the encoding device is input to the scalable transcoding device, the data is rearranged or discarded.
A scalable transcoding method comprising converting luminance color difference signals of different subsampling into an image encoding format. When converting to a different subsampling image encoding format,
When the data band-divided by the wavelet filter of the encoding device is input to the scalable transcoding device, subband coefficient truncation processing, partial wavelet processing and rearrangement of the encoded data of the frequency band of an arbitrary component are performed, A scalable transcoding method comprising converting to an image encoding format of different subsampling. In JPEG2000 encoding,
When encoded data is input from the encoding device to the scalable transcoding device, subband coefficient truncation and rearrangement of the encoded data in the frequency band of an arbitrary component are performed,
A scalable transcoding method, wherein a header of a corresponding part is rewritten and converted into a JPEG2000 encoding format. A scalable transcoding device for performing image coding of luminance color difference signals,
When data divided by the wavelet filter of the encoding device is input, means for rearranging the data;
Or a means of discarding part of the entered data;
A scalable transcoding device comprising: transcoding means comprising: means for converting luminance color difference signals of different sub-sampling into image coding formats and outputting the same to a decoding device; A scalable transcoding device for converting into different sub-sampling image coding formats,
When band-divided data is input by the wavelet filter of the encoding device, subband coefficient truncation processing, partial wavelet processing, and rearrangement of encoded data are performed in a frequency band of an arbitrary component, and images with different subsampling are performed. A scalable transcoding device comprising: transcoding means comprising: means for converting into an encoding format and outputting to a decoding device; A scalable transcoding device that performs JPEG2000 encoding,
When JPEG2000 data is input from the encoding device to the scalable transcoding device, means for subband coefficient truncation in the frequency band of arbitrary components and rearrangement of the encoded data;
A scalable transcoding device comprising: transcoding means comprising: means for rewriting a header of a corresponding part, converting to a JPEG2000 encoding format, and outputting to a decoding device. A scalable transcoding program for converting an image encoding format into an arbitrary image encoding format,
A scalable transcoding program that causes a computer to execute processing for realizing the scalable transcoding method according to claim 1.

Images