GB2329544A

GB2329544A - Image comparison in transform-quantised space

Info

Publication number: GB2329544A
Application number: GB9720048A
Authority: GB
Inventors: Baruh Hason; Reuven Livne
Original assignee: Motorola Israel Ltd
Current assignee: Motorola Solutions Israel Ltd
Priority date: 1997-09-19
Filing date: 1997-09-19
Publication date: 1999-03-24
Anticipated expiration: 2017-09-19
Also published as: GB9720048D0; GB2329544B

Abstract

A video compression codec performs a comparison between a block of the present frame and the most recently encoded corresponding block from a previous frame for transmission of a 'skip' signal if the blocks are similar, wherein both present and previous blocks have been subjected to transform coding and the comparison is adapted to compare only a reduced set of N frequency coefficients such as those of lowest frequency or highest energy. Comparison between blocks of different frames can be carried out in transform space only if no motion estimation was involved in the transformation. The transform coding may be DCT, Fourier or Kurhunen-Loeve coding. Transformed blocks from the present frame to be transmitted are quantised and entropy coded. Entropy coding may be any lossless encoding such as run-length coding or Huffman coding. The invention has the advantage of not requiring any decoding of a transformed signal to provide a comparison.

Description

Video codec and method for encoding video frames Technical Field The invention relates to the field of video codecs. In particular, the invention concerns the comparison of image frames in a video codec.

Background Video codecs are used to prepare video image frames for transmission In the ainder of this description, videos will be used to mean motion video. 'Image' wiz be used to describe a single image.

Modern video codecs compare consecutive video frames and encode only the difference between the present frame and the previous frame. In a video transmission system, such selective encoding reduces the amount of information which must be transmitted. This encoding is referred to as ecoiipression' Inthe rest of this document, Discrete Cosine Transform (DOT) based video codecs will be considered unless stated otherwise. In these codecs the frame is divided into blocks of fixed size. These blocks consist generally of 8x8 or 16x16 pixels. Each block is compressed separately. Video standards such as MPEG I, MPEG-II, H.261 and others are based on this principle.

A video codec which carries out video compression is commonly termed an encoder. Similarly, a video codec which carries out video de-compression is commonly termed a decoder.

Simple codecs use an on-off technique for the encoding. This technique involves comparing corresponding blocks of consecutive images and if they are similar then the latest block is not encoded. Instead the system generates and transmits information to the effect that this block is a duplicate of the corresponding block from the previous frame. If however the latest block is sufficiently different from the corresponding block of the earlier image, then the block concerned is encoded as is.

This 'on-off differential video compression is illustrated in figure 1.

Figure 1 shows the generic block diagram of a video compressor based on DCT and implementing on-off differential encoding technique. Each block from the present frame is compared to the most recently encoded corresponding block from a previous frame. The most recently encoded block from a particular position in the frame and its frame will be referred to as the 'previous block' and 'previous frame' respectively. These will only be from the immediately preceding frame when the corresponding block in that frame was in fact encoded.

The comparison performed by the arrangement of figure 1 is performed in 'image space'. This means that the comparison is performed between data concerning the latest image frame, which has not yet been processed for transmission, and data from the previous image frame, which has both been processed for transmission and then also processed back into its original format. The decision whether or not to encode a block of a video frame is therefore made by comparing two portions of images themselves.

If the present block and the previous block are similar, then the output of the encoder is a signal indicating that no encoding of the present block will take place. This has been shown in figure 1 as a 'Skip' block. This signal may consist of a single byte of data. Obviously transmission of this byte in place of the code for an entire block reduces the requirements, particularly for bandwidth, on circuitry and transmission paths downstream of the codec.

However, if corresponding blocks in the present and previously encoded image are not similar, then the present block must be encoded. The present block is DCT transformed by the element marked DCT, the resultant coefficients are qaantised by element Q, and the quantised coefficients are compressed using an entropy compressor, element E. This entropy compressor may in fact include any lossless encoding steps such as run-length coding, Huffinan coding etc.

The comparison carried out by the 'compare' block in figure 1 is made in order to decide whether or not to encode a particular block of the latest image. This comparison must be done in image space when motion estimation compensation is used in the compressor. This is the reason why the arrangement of figure 1 converts the block of the previous compressed frame back into image space. The elements IDCT and Q1are employed to do this.

Element IDIOT provides an inverse DCT transform of the coefficients which are input to it. Element Q-l is a de-quantiser. Together elements IDCT and Q-l constitute a partial decoder.

There are however encoding arrangements where motion compression can be omitted. In these cases it is not necessary to perform the comparison of image Mocks in image space. Instead the comparison can be performed in DCT coefficient space, also referred to as the DCT 'domain'. To perform such a comparison, it is not necessary to perform an inverse DCT operation on the coefficients of the previous block. An arrangement for on-off differential video compression in the DCT domain is shown in figure 2.

The manor differences between figures 1 and 2 are that: f) In the arrangement of figure 2, the compressed previous image block is in ent space when the comparison is made between the latest and therevious blocks; (ii) In the arrangement of figure 2, all blocks pass the DCT; (iii) The arrangement of figure 2 does not include an IDCT element for performing an inverse DCT transformation in the decoding branch.

Function of the comparators The function of the comparator of figure 1 is to compute the energy of the differences between a block from the present frame and the corresponding block from the previous frame, in image space. If the energy difference is not computed, then the 'compare' block must calculate the difference in a similar variable between the two blocks. The comparator in figure 2 performs the same computations in DCT coefficients space.

Summary of the Invention A video compressor in accordance with the invention comprises means for comparing the present and previous video frames in post-transformation space, whereby the means for comparing are adapted to compare only a reduced set of N frequency coefficients.

In accordance with a preferred embodiment of the invention, the means for comparing the present and previous video frames are arranged to compare only the coefficients which include the highest energy and/or the coefficients of lowest frequency. There may be means provided to apply an energy compacting transform to the blocks of the video frames to produce the coefficients. Finally, the coefficients may result from applying to the blocks of the video frames either a DCT, a Fourier or a Kurhunen-Loeve transform.

A method of performing video compression in accordance with the present invention comprises comparing the present and previous video frames in posttransformation space, the comparison using only a reduced set of N frequency coefficients.

In accordance with a preferred embodiment of the invention, the comparison is made only for the coefficients of the present and previous video frames which include the highest energy and/or those of lowest frequency. An energy coznptmg' transform may be applied to the blocks of the video frames te produce the coefficients. Finally,the coefficients may result from applying to the blocks of the video frames either a DCT, a Fourier, or a Kurhunen-Loeve transform.

Brief description of the drawings pre I shows a prior art arrangement for performing 'on-off9 differential video cocsion in image space; Figure 2 shows an arrangement for performing 'on-off differential video compression in the DCT domain; Figure 3 shows an example of an ordering of coefficients, some of which are used in the apparatus and method of the invention.

Detailed description of the preferred embodiment Video compression performed by an apparatus such as that in figure 2 involves a comparison of DCT coefficients. Assume that the comparator compares the DCT co-coefficients which relate to a block which measures 8x8 pixels. Assume also that the comparison measure chosen is the sum of the absolute differences (SAD) between the coefficients of the present and the previous images.

The sum of the absolute differences for the 8x8 block of pixel elements is:

(Equation 1) Here: The symbols i and j are indices indicating the index number ofthe particular pixel; The symbol c,i(t) represents the DCT coefficient for the particular pixel of the present frame; The symbol qij is the quantisation step used; The symbol ejj(t-1) is the pre-dequantisation value of the co-efficient for the previous frame.

Equation 1 represents therefore the sum of the given coefficients over all 64 DCT coefficients of the image block for which the comparison is being made.

Clearly 64 calculations are necessary to produce the Sum of Absolute Differences for this block.

The invention relates to an arrangement where the sum is not performed in this way. The sum can be performed according to the invention by an apparatus analogous to that shown in Figure 2, but where the 'Compare' block is modified to perform a different calculation than that shown in Equation 1 above.

To understand the arrangement according to the invention, consider the following sum:

(Equation 2) Here: The symbol k is a single index number indicating the particular pixel; The symbol ck(t) represents the DCT coefficient for the particular pixel of the present frame; The symbol qk is the quantisation step used; The symbol e,(t-l) is the pre-dequantisation value of the co-efficient for the previous frame.

The index k takes the place of the two indices i and j used for each pixel in Equation 1.

The index k can be defined in the manner shown in figure 3. Here the sixty four pixels of a single block of the image have been arranged in a particular order.

The path indicated by the line sneaking through the figure indicates a method of selecting pixels arranged in this order. This a 'zig-zag' selection. Each square in figure 3 represents a pixel, and each successive pixel has a successively higher index number k. Therefore the pixel represented by the square in the upper left of the figure has index k=1. The last square along the indicated path, that in the bottom right of the figure, has index k=64.

Consider the case of the comparison carried out by the box marked 'compare' in figure 2 being a computation of energy. The method in accordance with the invention relies on being able to make this comparison for each image block with fewer than all 64 of the coefficients for that block. In the case of the energy comparison, the energy is concentrated in the coefficients of lowest frequency. This is because the DCT transform is an energy compacting trasform Thus the invention in this case relies on making the comparison between DCT coefficients of the present and previous blocks for only the icients of lowest frequency.

In the case of the 'compare' block comparing energy values therefore, figure 3 represents the pixels of a given block arranged in ascending order of frequency.

Thecreation of figure 3 can more easily be understood by referring to figure 2.

After compression of a block by elements DCT and Q, there are 64 co efiis for the block. In accordance with the invention, these co-efficients can be ordered in a sequence such that the co-efficients representing the lowest frequencies are at the beginning of the sequence. Those which represent the highest frequencies are at the end of the sequence. Figure 3 shows this sequence with co-efficients of lowest frequency toward the upper left of the figure, and those of higher frequency towards the lower right.

It should be noted that figure 3 actually shows one example in accordance with the invention. The ordering shown in figure 3 is 'zig-zag' ordering. In general, the invention envisages mapping the DCT coefficients of an image block into a one-dimensional array such that the low frequency coefficients get low indices. This facilitates the comparison of only the first few, low frequency coefficients when comparing two image blocks.

In an even more general form, the invention extends to mapping the coefficients relating to an image block into any sequence which is ordered such that the coefficients with the greatest information content for the comparison to be performed by the on-off differential encoder can be selected out. The comparison is then made on these components only. This invention is applicable analogously to 'on-off video compressors using other forms of transform than DCT.

It is important to note the function of the 'compare' element of figure 2. This makes the decision whether or not the present block is sufficiently different from the most recently encoded block to warrant encoding of the present block. The 'compare' element thus makes a 'yes' or 'no' decision. It does not perform the actual encoding of the present block. Therefore no degradation of the encoding itself takes place through performing the comparison step for only a limited number of DCT coefficients.

The apparatus required to carry out the present invention is analogous to that illustrated in figure 2. The apparatus can be implemented as a digital signal processor. It is important that the apparatus be adapted to select the coefficients of lowest frequency, for example using zig-zag ordering as shown in figure 3.

Example In order to better understand the difference between the invention and the prior art arrangement functioning according to Equation 1, the following numerical example should be considered: The prior art arrangement carries out a comparison between the coefficients for 64 different values of the ij combination. This would correspond to performing the sum in Equation 2 for all coefficients, i.e. setting N=64.

Howeezer, the invention contemplates performing the sum of Equation 2 for N < 64. N can be chosen as low as 15, or even 10, without significantly affecting the result of the comparison.

This means that the sum of Equation 2 would be performed, for instance, on the DOT coefficients 1-15 in figure 3. Thus the sum would be performed on the fifteen DCT coefficients of the present block which are of lowest frequency, but which carry the majority of the information about the energy.

The advantages of the invention are apparent to the skilled person in the light of the disclosure above. In particular: (i) For any N < 64, the number of calculations necessary to perform the sum in Equation 2 is less than the number for that in Equation 1. In fact, the number of operations necessary to compute the SAD decreases linearly with the number N. Thus the calculation can be performed more quickly and/or using less calculation power.

(ii) The memory space necessary to save the previous compressed image decreases linearly with the number N. This is because the 'compare' element needs only N coefficients from the previous frame to reach its decision. Taking the example of N = 15, the reduction in the number of computation steps and memory space is more than 75%.

Claims

1. A video compressor (DCT; Q; E), comprising: means for comparing the present and previous video frames in posttransformation space (DCT; Q-l); whereby the means for comparing are adapted to compare only a reduced set (N) of frequency coefficients.

2. A video compressor (DCT; Q; E) according to claim 1, wherein the means for comparing the present and previous video frames are arranged to compare only the coefficients (N) which include the highest energy.

3. A video compressor (DCT; Q; E) according to claim 1 or claim 2, wherein the means for comparing the present and previous video frames are arranged to compare only the coefficients (N) of lowest frequency.

4. A video compressor (DCT; Q; E) according to any of claims 1-3, wherein means are provided to apply an energy compacting transform (DCT) to the blocks of the video frames to produce the coefficients.

5. A video compressor (Q; E) according to any of claims 1-4, wherein the coefficients result from applying to the blocks of the video frames either: a DCT transform (DCT); a Fourier transform; or a Kurhunen-Loeve transform.

6. A method of performing video compression (DCT; Q; E), comprising: comparing the present and previous video frames in post-transformation space (DCT;Q-l); the comparison using only a reduced set (N) of frequency coefficients.

7. A method of performing video compression (DCT; Q; E) according to claim 6, wherein the comparison is made only for the coefficients (N) of the present and previous video frames which include the highest energy.

8. A method of performing video compression (DCT; Q; E) according to claim 6 or claim 7, wherein the comparison is made only for the coefficients (N) of the present and previous video frames of lowest frequency.

9. A method of performing video compression (DCT; Q; E) according to any of claims 68 > wherein an energy compacting transform (DCT) is applied to the blocks of the video frames to produce the coefficients (N).

10 A method of performing video compression (Q; E) according to any of claims 6-9, wherein the coefficients result from applying to the blocks of the video frames eiter: a DCT transform (DCT); or a Fourier transform; or a Kurhuner-Loeve transform.