JP5786498B2 - Image coding apparatus, image coding method, and image coding program - Google Patents

Description

  The present invention relates to a moving image encoding technique using motion compensated prediction, and more particularly to an image encoding device, an image encoding method, and an image encoding program used for motion compensated prediction.

  MPEG-4 AVC / H. In moving picture coding represented by H.264 (hereinafter referred to as AVC) and the like, as information compression using correlation in the time direction, a picture to be coded which is a picture signal to be coded is already coded and decoded. Motion-compensated prediction that uses the local decoded signal as a reference picture, detects a motion amount (hereinafter referred to as a motion vector) between the target picture and the reference picture in a predetermined coding processing unit, and generates a prediction signal. Used.

  In AVC, both one-way prediction in which a prediction signal is generated in a single direction using one motion vector from one reference picture in motion compensation prediction and two motion vectors from two reference pictures are used. Bidirectional prediction that generates a prediction signal in the direction is used. A method of changing the size of a block to be processed (hereinafter referred to as a prediction target block) within a 16 × 16 pixel two-dimensional block as a predetermined encoding processing unit, or among a plurality of reference pictures By applying this method to the method of selecting a reference picture used for prediction, the information amount of the prediction signal is reduced. On the encoding side, information specifying the prediction mode information and the reference image is selected and transmitted, and on the decoding side, motion compensation prediction processing is performed according to the transmitted prediction mode information and information specifying the reference image.

  For the motion vector, the motion vector of the encoded block adjacent to the processing target block is set as the prediction motion vector (hereinafter referred to as prediction vector), the difference between the motion vector of the processing target block and the prediction vector is obtained, and the difference vector is encoded. The compression efficiency is improved by transmitting as a vector.

  Further, in AVC, a direct motion compensated prediction that realizes motion compensated prediction without transmitting an encoded vector by using a motion vector used for encoding a block of a reference picture at the same position as the prediction target block. It has been known.

  In addition, as a method of not transmitting other encoded vectors, motion compensation prediction that realizes motion compensation prediction using motion information of a block adjacent to a processing target block is known (see, for example, Patent Document 1). .

JP-A-10-276439

  The direct motion compensated prediction described above pays attention to the continuity of motion in the temporal direction in the block of the reference picture located at the same position as the prediction target block, and the motion compensated prediction that does not transmit the coded vector described above is predicted as the prediction target block. By focusing on the continuity of motion in the spatial direction in the encoded block adjacent to the target block, the motion information of other blocks is used as it is. As a result, encoding efficiency is improved without encoding the difference vector as an encoded vector.

  However, when the continuity of motion is not sufficiently maintained or when the motion vector in the motion information of other blocks does not indicate an accurate motion, etc. In this case, a predicted image using the motion information in which the shift has occurred is generated. In that case, there is a difficult aspect that a motion compensated prediction image with high accuracy cannot be generated and the encoding efficiency is not improved.

  In addition, when motion compensation is not performed without encoding a differential motion vector, conventional motion compensated prediction that encodes a differential motion vector is used, but when performing conventional motion compensated prediction, a prediction mode is used. It is necessary to encode information, designation information of the reference image, and a motion vector used for the designated reference image as a difference vector from the prediction vector. In this case, there is a difficult aspect that the amount of codes for specifying the motion compensation prediction method increases and the encoding efficiency is not sufficiently improved.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a technique for improving the coding efficiency when using motion compensation prediction.

In order to solve the above problems, an image encoding device according to an aspect of the present invention is an image encoding device that detects and encodes a motion vector in units of blocks obtained by dividing each picture of a moving image. as the motion information of the target block, when the prediction direction in the case of bidirectional prediction, and for modifying one of the motion vectors included in the motion information of the reference blocks adjacent to the encoding target block, before hexane irradiation A prediction mode determination unit (109) capable of selecting a mode in which a block index and a difference vector between the motion vector of the coding target block and the one motion vector included in the motion information of the reference block are used as a coding target. ), A prediction signal generated based on the motion information supplied from the prediction mode determination unit (109), and the encoding target block A prediction error signal encoding unit (103) that encodes a prediction error signal, which is a difference from the image signal, and a motion information encoding unit (encoding motion information supplied from the prediction mode determination unit (109)) 110), and a multiplexing unit (112) that multiplexes the encoded data supplied from the prediction error signal encoding unit (103) and the encoded data supplied from the motion information encoding unit (110). And comprising. The one of the motion vectors included in the motion information of the reference block to be subjected to the differential vector is determined from motion information of the reference block candidate is the motion vector determined by the prediction direction.

Another aspect of the present invention is an image encoding method. This method is an image coding method in which a motion vector is detected and coded in units of blocks obtained by dividing each picture of a moving image, and when the prediction direction is bi-directional prediction as motion information of an encoding target block. there are, and in the case of modifying one of the motion vectors included in the motion information of the reference blocks adjacent to the encoding target block, the index of the previous hexane irradiation block, and the reference motion vector of the encoding target block Selecting a mode in which a difference vector from the one motion vector included in the block motion information is used as an encoding target, a prediction signal generated based on the motion information, and the encoding target block A step of encoding a prediction error signal that is a difference from the image signal, a step of encoding the motion information, and the prediction error Comprising the No. of the encoded data, and a step of multiplexing the encoded data of the motion information. The one of the motion vectors included in the motion information of the reference block to be subjected to the differential vector is determined from motion information of the reference block candidate is the motion vector determined by the prediction direction.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to improve the coding efficiency when using motion compensated prediction.

It is a figure which shows the structure of the moving image encoder which concerns on Embodiment 1 of this invention. It is a figure which shows an example of an encoding target image. It is a figure which shows the detailed definition of prediction block size. 4A to 4D are diagrams for explaining the prediction direction of motion compensation prediction. It is a flowchart which shows the flow of the operation | movement of the encoding process in the moving image encoder which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the moving image decoding apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows the flow of operation | movement of the decoding process in the moving image decoding apparatus which concerns on Embodiment 1 of this invention. FIGS. 8A and 8B are diagrams for explaining three prediction modes for encoding motion information used in motion compensated prediction according to Embodiment 1 of the present invention. It is a figure for demonstrating regarding the relationship of the prediction direction of motion compensation, the information which defines a reference image, and the number of motion vectors to encode in the intra-frame prediction which does not implement motion compensation prediction and motion compensation prediction. FIG. 3 is a diagram illustrating a detailed configuration of a mode determination unit in the moving image encoding device according to the first embodiment. It is a figure which shows the structure of a prediction vector calculation part. It is a figure which shows the structure of a joint motion information calculation part. It is a figure which shows the structure of a joint motion correction motion compensation prediction production | generation part. FIG. 11 is a flowchart for explaining an operation of motion compensation prediction mode / prediction signal generation, which is step S502 of FIG. 5, which operates via the prediction mode determination unit of FIG. It is a flowchart for demonstrating the detailed operation | movement of combined motion information candidate list generation in step S1400 of FIG. It is a figure which shows the space candidate block group used for a space joint motion information candidate list generation. It is a flowchart for demonstrating the detailed operation | movement of a space joint motion information candidate list generation. It is a figure which shows the time candidate block group used for a time joint movement information candidate list generation. It is a flowchart for demonstrating the detailed operation | movement of time coupling | bonding motion information candidate list generation. It is a figure for demonstrating the calculation method of the motion vector value mvL0t and mvL1t registered with respect to the reference | standard motion vector value ColMv with respect to time coupling | bonding motion information with respect to L0 prediction and L1 prediction. It is a flowchart for demonstrating the detailed operation | movement of combined prediction mode evaluation value generation in step S1401 of FIG. It is a flowchart for demonstrating the detailed operation | movement of combined motion correction prediction mode evaluation value production | generation in step S1402 of FIG. FIGS. 23A to 23C are diagrams illustrating the concept of scaling processing for combined motion information in the combined motion modified prediction mode. It is a figure which shows the number of the motion vectors generated by scaling which exist with respect to each joint motion information. FIGS. 25A to 25C are conceptual diagrams showing the differential motion vector transmission reference image determination algorithm in the first embodiment. It is a flowchart for demonstrating the detailed operation | movement of step S2203 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S2601 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S2702 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S1403 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S2901 of FIG. It is a flowchart for demonstrating the detailed operation | movement of spatial motion information utilization prediction vector candidate list generation. It is a flowchart for demonstrating the detailed operation | movement of temporal motion information utilization prediction vector candidate list generation. It is a flowchart for demonstrating the detailed operation | movement of step S1404 of FIG. It is a flowchart for demonstrating the detailed operation | movement of step S504 of FIG. 6 is a diagram illustrating an example of a motion prediction flag according to Embodiment 1. FIG. FIG. 35 is a diagram illustrating an example of syntax of an encoded stream generated by motion information encoded data generation processing illustrated in the flowchart of FIG. 34. It is a figure which shows the detailed structure of the motion information decoding part in the moving image decoding apparatus of Embodiment 1 shown in FIG. It is a flowchart for demonstrating the detailed operation | movement of step S701 of FIG. FIG. 39 is a flowchart for explaining detailed operation of step S3802 of FIG. 38. FIG. FIG. 39 is a flowchart for explaining detailed operation of step S3807 of FIG. 38. FIG. FIG. 39 is a flowchart for explaining detailed operation of step S3811 of FIG. 38. FIG. It is a flowchart for demonstrating the detailed operation | movement of step S3810 of FIG. 10 is a diagram showing a configuration of a motion prediction mode in Embodiment 2. FIG. FIG. 11 is a diagram illustrating the syntax of an encoded stream generated by motion information encoded data generation processing in the second embodiment. 10 is a flowchart for explaining motion information encoded data generation processing in the moving image encoding apparatus according to the second embodiment. 11 is a flowchart for explaining motion information decoding processing in the video decoding device in the second embodiment. FIG. 11 is a diagram illustrating a detailed configuration in a combined motion corrected motion compensated prediction generation unit in FIG. 10 according to Embodiment 3. 15 is a flowchart for illustrating a combined motion correction prediction mode evaluation value generation process in the third embodiment. FIGS. 49A to 49C are conceptual diagrams showing the distance priority algorithm in the fourth embodiment. 15 is a flowchart for explaining reference image determination processing for transmitting bidirectional motion vector values and difference vectors in the fourth embodiment. 51 (a) to 51 (c) specify reference images for transmitting a difference vector when combined motion information is space BiPred, space UniPred, and time BiPred in the fifth embodiment. It is a figure which shows the conceptual diagram of a process structure. In Embodiment 5, it is a figure which shows the relationship of the motion information transmitted in each prediction mode. 20 is a flowchart for describing reference image determination processing for transmitting a bidirectional motion vector value and a difference vector in the fifth embodiment. FIG. 25 is a diagram illustrating an example of an encoding syntax in the fifth embodiment. 10 is a flowchart for explaining motion information encoded data generation processing in the moving image encoding apparatus according to the fifth embodiment. 29 is a flowchart for explaining combined motion corrected predicted motion information decoding processing in the video decoding device in the fifth embodiment. FIGS. 57A and 57B are conceptual diagrams showing the state of the difference vector between the space BiPred and the space UniPred in the configuration in which two reference images are always sent in the fifth embodiment. It is a figure which shows an example of the encoding syntax employable when taking the structure which always transmits a difference vector with respect to two reference images. FIG. 25 is a diagram illustrating a configuration of a prediction mode determination unit in the video encoding device according to the sixth embodiment. [Fig. 38] Fig. 38 is a diagram illustrating a configuration of a motion information decoding unit in the video decoding device according to the sixth configuration. 218 is a diagram illustrating an example of coding syntax in Embodiment 6. [FIG. FIG. 38 is a diagram illustrating an example in which combined information applied in each prediction mode in the sixth embodiment is different. 63 (a) and 63 (b) are conceptual diagrams showing the effect of the difference vector on the two reference images in the seventh embodiment. FIG. 25 is a diagram illustrating an example of a motion vector search method based on a combined motion vector value of two reference images in the seventh embodiment. FIG. 38 is a flowchart for describing combined motion correction prediction mode evaluation value generation processing in the seventh embodiment. FIG. 38 is a flowchart for describing combined motion corrected predicted motion information decoding processing in the seventh embodiment. It is a conceptual diagram which shows another structure which applies the effect | action of the difference vector in Embodiment 7 to both reference images. 218 is a diagram illustrating the definition of a motion compensation prediction mode in Embodiment 8. [FIG. FIG. 25 is a diagram illustrating a configuration of a prediction mode determination unit in the video encoding device according to the eighth embodiment. 29 is a flowchart for explaining motion compensation prediction mode / prediction signal generation processing in the video encoding device in the eighth embodiment. 29 is a flowchart for explaining motion information encoded data generation processing in the moving image encoding apparatus according to the eighth embodiment. 218 is a diagram illustrating an example of coding syntax regarding motion information in Embodiment 8. [FIG. FIG. 38 is a diagram illustrating a configuration of a motion information decoding unit in the moving picture decoding apparatus according to the eighth embodiment. FIG. 38 is a flowchart for describing motion information decoding processing in the video decoding device in the eighth embodiment. FIG. 75 is a flowchart for explaining a detailed operation of step S3810 of FIG. 74. FIG. 76 (a) to 76 (c) are conceptual diagrams showing a difference vector transmission method in a combined motion modified prediction mode of another configuration according to the eighth embodiment. FIG. 38 is a flowchart for describing combined motion modified prediction mode evaluation value generation processing in the video encoding device in the ninth embodiment. FIG. 38 is a flowchart for describing combined motion modified prediction mode evaluation value generation processing of another configuration in the ninth embodiment. FIG. 38 is a flowchart for describing joint motion corrected predicted motion information decoding processing in the video decoding device in Embodiment 9. It is a figure which shows the Truncated Unary code sequence when the number of combined motion information candidates is 5.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Preferred embodiments of a moving image encoding apparatus, a moving image encoding method, a moving image encoding program, a moving image decoding apparatus, a moving image decoding method, and a moving image decoding program according to an embodiment of the present invention will be described with reference to the drawings. A form is demonstrated in detail. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

(Embodiment 1)
[Overall configuration of video encoding apparatus]
FIG. 1 is a diagram showing a configuration of a moving picture coding apparatus according to Embodiment 1 of the present invention. Hereinafter, the operation of each unit will be described. The moving picture coding apparatus according to Embodiment 1 includes an input terminal 100, a subtraction unit 101, an orthogonal transformation / quantization unit 102, a prediction error coding unit 103, an inverse quantization / inverse transformation unit 104, an addition unit 105, and a decoding An image memory 106, a motion vector detection unit 107, a motion compensation prediction unit 108, a prediction mode determination unit 109, a motion information encoding unit 110, a motion information memory 111, a multiplexing unit 112, and an output terminal 113 are provided.

  The image signal of the prediction block to be encoded is extracted from the image signal input from the input terminal 100 based on the position information and the prediction block size of the prediction block, and the image signal of the prediction block includes the subtraction unit 101, the motion vector, This is supplied to the detection unit 107 and the prediction mode determination unit 109.

  FIG. 2 is a diagram illustrating an example of an encoding target image. Regarding the prediction block size according to the first embodiment, as shown in FIG. 2, the encoding target image is encoded in units of 64 × 64 pixel encoding blocks (CU), and the prediction block is further divided into encoding blocks. Unit. The maximum prediction block size is 64 × 64 pixels, which is the same as the encoded block, and the minimum prediction block size is 4 × 4 pixels. The coding block is divided into prediction blocks: non-division (2N × 2N), horizontal / vertical division (N × N), horizontal only division (2N × N), and vertical division only (N × 2N) is possible. Only in the case of horizontal and vertical division, the further divided prediction block can be divided into prediction blocks hierarchically as encoded blocks, and the hierarchy is expressed by the number of CU divisions.

  FIG. 3 is a diagram showing a detailed definition of the predicted block size. There are 13 predicted block sizes from 64 pixels × 64 pixels, which is the maximum predicted block size with 0 CU divisions, to 4 pixels × 4 pixels, which is the minimum predicted block size with 3 CU partitions. Will exist.

  Regarding the division | segmentation structure of the prediction block which concerns on Embodiment 1 of this invention, it is not limited to this combination. In addition, the selection of the prediction block size in the moving image encoding apparatus can adaptively select a structure with better encoding efficiency in units of encoding blocks, but the first embodiment is in units of prediction blocks. Since attention is paid to inter-frame prediction and inter-frame motion information encoding, the components and description relating to selection of the optimal prediction block size are omitted. Regarding the subsequent operation of the video encoding apparatus, the operation performed in units of the selected prediction block size will be described.

  Returning to FIG. 1, the subtraction unit 101 subtracts the image signal supplied from the input terminal 100 and the prediction signal supplied from the prediction mode determination unit 109 to calculate a prediction error signal, and orthogonally transforms / quantizes the prediction error signal. To the conversion unit 102.

  The orthogonal transform / quantization unit 102 performs orthogonal transform and quantization on the prediction error signal supplied from the subtraction unit 101, and the quantized prediction error signal is subjected to a prediction error encoding unit 103 and an inverse quantization / inverse conversion unit. 104 is supplied.

  The prediction error encoding unit 103 entropy-encodes the quantized prediction error signal supplied from the orthogonal transform / quantization unit 102, generates a code string for the prediction error signal, and supplies the code sequence to the multiplexing unit 112 .

  The inverse quantization / inverse transform unit 104 performs processing such as inverse quantization and inverse orthogonal transform on the quantized prediction error signal supplied from the orthogonal transform / quantization unit 102, and outputs the decoded prediction error signal. Generated and supplied to the adding unit 105.

  The addition unit 105 adds the decoded prediction error signal supplied from the inverse quantization / inverse conversion unit 104 and the prediction signal supplied from the prediction mode determination unit 109, generates a decoded image signal, and outputs the decoded image signal. The decoded image memory 116 is supplied.

  The decoded image memory 106 stores the decoded image signal supplied from the adding unit 105. In addition, a decoded image for which decoding of the entire image has been completed is stored as a reference image by a predetermined number of images, and the reference image signal is supplied to the motion vector detection unit 107 and the motion compensation prediction unit 108.

  The motion vector detection unit 107 receives the prediction block image signal supplied from the input terminal 100 and the reference image signal stored in the decoded image memory 106, detects the motion vector for each reference image, and detects the motion vector. The value is supplied to the prediction mode determination unit 109.

  A general motion vector detection method calculates an error evaluation value for an image signal corresponding to a reference image moved by a predetermined movement amount from the same position as the image signal, and moves the movement amount that minimizes the error evaluation value. Let it be a vector. As the error evaluation value, a sum of absolute differences SAD (Sum of Absolute Difference) for each pixel, a sum of square error values SSE (Sum of Square Error) for each pixel, or the like is used. Furthermore, the code amount related to the coding of the motion vector can also be included in the error evaluation value.

  The motion compensated prediction unit 108 predicts the reference image indicated by the reference image designation information in the decoded image memory 106 by a motion vector value according to the reference image designation information designated by the prediction mode determination unit 109 and the motion vector value. An image signal at a position moved from the same position as the image signal is acquired to generate a prediction signal.

  When the prediction mode specified by the prediction mode determination unit 109 is unidirectional prediction, a prediction signal acquired from one reference image is used as a motion compensation prediction signal, and when the prediction mode is bidirectional prediction, two references are made. A weighted average of the prediction signals acquired from the images is used as a motion compensation prediction signal, and the motion compensation prediction signal is supplied to the prediction mode determination unit 109. Here, the ratio of the weighted average of bidirectional prediction is 1: 1.

  4A to 4D are diagrams for explaining the prediction direction of motion compensation prediction. The process of performing prediction from a single reference image is defined as unidirectional prediction, and in the case of unidirectional, one of the reference images registered in the two reference image management lists of the L0 direction and the L1 direction is used.

  FIG. 4A illustrates a case where the reference image (RefL0Pic) in one direction and in the L0 direction is at a time before the encoding target image (CurPic). FIG. 4B shows a case where the reference image in one direction and in the L0 direction is at a time after the encoding target image. Similarly, the one-way prediction can be performed by replacing the reference image in the L0 direction in FIGS. 4A and 4B with the reference image in the L1 direction (RefL1Pic).

  The process of performing prediction from two reference images is defined as bidirectional prediction, and in the case of bidirectional, the BI direction is expressed using both the L0 direction and the L1 direction. FIG. 4C shows a case where the reference image in the L0 direction is at a time before the encoding target image and the reference image in the L1 direction is at a time after the encoding target image. . FIG. 4D shows a case in which the reference image in the L0 direction and the reference image in the L1 direction are at a time before the encoding target image. As described above, the relationship between the prediction direction of L0 / L1 and time can be used without being limited to L0 being the past direction and L1 being the future direction.

  In the first embodiment, POC (Picture Order Count) is used as information indicating the time of the reference image and the encoding target image. POC is a counter indicating the display order of images defined by AVC. When the image display order is increased by 1, the POC is also increased by 1. Therefore, the time difference (distance) between images can be acquired from the POC difference between images.

  Returning to FIG. 1, the prediction mode determination unit 109 detects the motion vector value detected for each reference image input from the motion vector detection unit 107 and the motion information stored in the motion information memory 111 (prediction direction, motion Based on the vector value and the reference image designation information), the reference image designation information and the motion vector value used for each of the motion compensation prediction modes defined in the first embodiment are set in the motion compensation prediction unit 108. . Based on the set value, an optimal motion compensation prediction mode is determined using the motion compensation prediction signal supplied from the motion compensation prediction unit 108 and the image signal of the prediction block supplied from the input terminal 100.

  The prediction mode determination unit 109 supplies information specifying the determined prediction mode and the prediction direction, the motion vector, and the reference image designation information according to the prediction mode to the motion information encoding unit 110, and determines the determined prediction mode and A prediction direction, a motion vector value, and reference image designation information for the prediction mode are supplied to the motion information memory 111, and a prediction signal corresponding to the determined prediction mode is supplied to the subtraction unit 101 and the addition unit 105.

  In the moving image encoding apparatus, intra-frame prediction is performed in which prediction is performed using an encoded image in the same screen in order to encode a reference image as a reference. Embodiment 1 focuses on inter-screen prediction. Therefore, the components related to the intra prediction are omitted. A detailed configuration of the prediction mode determination unit 109 will be described later.

  The motion information encoding unit 110 encodes the prediction mode supplied from the prediction mode determination unit 109 and information specifying the prediction direction, the motion vector, and the reference image designation information according to the prediction mode according to a predetermined syntax structure. Thus, a code string of motion information is generated and supplied to the multiplexing unit 112. A detailed configuration of the motion information encoding unit 110 will be described later.

  The motion information memory 111 stores motion information (prediction direction, motion vector, and reference image index) supplied from the prediction mode determination unit 109 for a predetermined image on the basis of the minimum prediction block size unit. The motion information of the adjacent block of the prediction block to be processed is set as a spatial candidate block group, and the motion information of the block on the ColPic at the same position as the prediction block to be processed and its neighboring blocks is set as the time candidate block group.

  ColPic is a decoded image different from the prediction block to be processed, and is stored in the decoded image memory 106 as a reference image. In Embodiment 1, ColPic is a reference image decoded immediately before. In the first embodiment, ColPic is the reference image decoded immediately before, but the reference image immediately before in display order or the reference image immediately after in display order may be used, and the reference image used for ColPic is included in the encoded stream. Direct specification is also possible.

  The motion information memory 111 supplies the motion information of the spatial candidate block group and the temporal candidate block group to the prediction mode determination unit 109 as motion information of the candidate block group. The multiplexing unit 112 multiplexes the prediction error encoded sequence supplied from the prediction error encoding unit 103 and the motion information encoded sequence supplied from the motion information encoding unit 110 to encode the encoded bit stream. And the encoded bit stream is output to the recording medium / transmission path via the output terminal 113.

  The configuration of the moving picture encoding apparatus shown in FIG. 1 can also be realized by hardware such as an information processing apparatus including a CPU (Central Processing Unit), a frame memory, and a hard disk.

  FIG. 5 is a flowchart showing the flow of the encoding process in the video encoding apparatus according to Embodiment 1 of the present invention. A prediction block image to be processed is acquired from the input terminal 100 for each prediction block unit (S500). The motion vector detection unit 107 calculates a motion vector value for each reference image from the prediction block image to be processed and a plurality of reference images stored in the decoded image memory 106 (S501).

  Subsequently, the prediction mode determination unit 109 uses each of the motion vectors supplied from the motion vector detection unit 107 and the motion information stored in the motion information memory 111 to each of the motion compensation prediction modes defined in the first embodiment. Is obtained using the motion compensation prediction unit 108, an optimal prediction mode is selected, and a prediction signal is generated (S502). Details of the processing in step S502 will be described later.

  Subsequently, the subtraction unit 101 calculates a difference between the prediction block image to be processed and the prediction signal supplied from the prediction mode determination unit 109 as a prediction error signal (S503). The motion information encoding unit 110 encodes the prediction mode supplied from the prediction mode determination unit 109 and information specifying the prediction direction, the motion vector, and the reference image designation information according to the prediction mode according to a predetermined syntax structure. Then, encoded data of motion information is generated (S504). Details of the processing in step S504 will be described later.

  Subsequently, the prediction error encoding unit 103 entropy encodes the quantized prediction error signal generated by the orthogonal transform / quantization unit 102 to generate encoded data of the prediction error (S505). The multiplexing unit 112 multiplexes the motion information encoded data supplied from the motion information encoding unit 110 and the prediction error encoded data supplied from the prediction error encoding unit 103 to generate an encoded bitstream. (S506).

  The adding unit 105 adds the decoded prediction error signal supplied from the inverse quantization / inverse transform unit 104 and the prediction signal supplied from the prediction mode determination unit 109 to generate a decoded image signal (S507). The generated decoded image signal is supplied to and stored in the decoded image memory 106 by the adding unit 105, and is used for motion compensation prediction processing of an encoded image to be encoded later (S508). The motion information memory 111 stores the motion information (prediction direction, motion vector, and reference image designation information) supplied from the motion vector detection unit 107 in units of the minimum predicted block size (S509).

[Overall configuration of video decoding apparatus]
FIG. 6 is a diagram showing a configuration of the moving picture decoding apparatus according to Embodiment 1 of the present invention. Hereinafter, the operation of each unit will be described. The video decoding apparatus according to Embodiment 1 includes an input terminal 600, a demultiplexing unit 601, a prediction difference information decoding unit 602, an inverse quantization / inverse transformation unit 603, an addition unit 604, a decoded image memory 605, and a motion information decoding unit. 606, a motion information memory 607, a motion compensation prediction unit 608, and an output terminal 609.

  The encoded bit stream is supplied from the input terminal 600 to the demultiplexing unit 601. The demultiplexing unit 601 specifies the code sequence of the supplied encoded bitstream, the encoded sequence of the prediction error information, the prediction mode, the prediction direction according to the prediction mode, the motion vector, and the reference image designation information. Are separated into motion information encoded sequences. The encoded sequence of the prediction error information is supplied to the prediction difference information decoding unit 602, and the encoded sequence of the motion information is supplied to the motion information decoding unit 606.

  The prediction difference information decoding unit 602 decodes the encoded sequence of the prediction error information supplied from the demultiplexing unit 601 and generates a quantized prediction error signal. The prediction difference information decoding unit 602 supplies the generated quantized prediction error signal to the inverse quantization / inverse transform unit 603.

  The inverse quantization / inverse transform unit 603 performs a process such as inverse quantization or inverse orthogonal transform on the quantized prediction error signal supplied from the prediction difference information decoding unit 602 to generate a prediction error signal, and performs decoding prediction. The error signal is supplied to the adding unit 604.

  The adder 604 adds the decoded prediction error signal supplied from the inverse quantization / inverse transform unit 603 and the prediction signal supplied from the motion compensation prediction unit 608 to generate a decoded image signal, and decodes the decoded image signal This is supplied to the image memory 605.

  The decoded image memory 605 has the same function as the decoded image memory 106 in the moving image encoding apparatus in FIG. 1, stores the decoded image signal supplied from the addition unit 604, and stores the reference image signal in the motion compensation prediction unit 608. Supply. Also, the decoded image memory 605 supplies the stored decoded image signal to the output terminal 609 in accordance with the image display order in accordance with the reproduction time.

  The motion information decoding unit 606 obtains information specifying the prediction mode and the prediction direction, the motion vector, and the reference image designation information corresponding to the prediction mode from the motion information encoded sequence supplied from the demultiplexing unit 601. Decrypt as Based on the decoded motion information and the motion information of the candidate block group supplied from the motion information memory 607, the prediction direction, motion vector, and reference image designation information used for motion compensation prediction are reproduced and supplied to the motion compensation prediction unit 608. In addition, the motion information decoding unit 606 supplies the reproduced motion information to the motion information memory 607. A detailed configuration of the motion information decoding unit 606 will be described later.

  The motion information memory 607 has the same function as that of the motion information memory 111 in the video encoding apparatus of FIG. 1, and reproduces the reproduced motion information supplied from the motion information decoding unit 606 for a predetermined image on the basis of the minimum prediction block size unit. Remember. Also, the motion information memory 607 supplies the motion information of the space candidate block group and the time candidate block group to the motion information decoding unit 606 as motion information of the candidate block group.

  The motion compensation prediction unit 608 has the same function as that of the motion compensation prediction unit 108 in the video encoding apparatus in FIG. 1, and based on the motion information supplied from the motion information decoding unit 606, a reference image in the decoded image memory 605. A prediction signal is generated by acquiring an image signal at a position obtained by moving the reference image indicated by the designation information from the same position as the image signal of the prediction block by the motion vector value. If the prediction direction of motion compensation prediction is bidirectional, an average of the prediction signals in each prediction direction is generated as a prediction signal, and the prediction signal is supplied to the adding unit 604.

  The output terminal 609 reproduces a decoded image signal by outputting the decoded image signal supplied from the decoded image memory 605 to a display medium such as a display.

  The configuration of the video decoding device shown in FIG. 6 can also be realized by hardware such as an information processing device including a CPU, a frame memory, a hard disk, etc., similarly to the configuration of the video encoding device shown in FIG. is there.

  FIG. 7 is a flowchart showing a flow of decoding processing in the video decoding apparatus according to Embodiment 1 of the present invention. The demultiplexing unit 601 separates the coded bit stream supplied from the input terminal 600 into a coded sequence of prediction error information and a coded sequence of motion information (S700). The separated coded sequence of motion information is supplied to the motion information decoding unit 606, and the motion information of the decoding target block is decoded using the motion information of the candidate block group supplied from the motion information memory 607 (S701).

  The separated coded sequence of prediction error information is supplied to the prediction difference information decoding unit 602, decoded as a quantized prediction error signal, and dequantized or inverse orthogonal transformed by an inverse quantization / inverse transformation unit 603. By performing the above process, a decoded prediction error signal is generated (S702). Details of the processing in step S702 will be described later.

  The motion information decoding unit 606 supplies the motion information of the decoding target block to the motion compensation prediction unit 608, and the motion compensation prediction unit 608 performs motion compensation prediction according to the motion information to calculate a prediction signal (S703). The adding unit 604 adds the decoded prediction error signal supplied from the inverse quantization / inverse transform unit 603 and the prediction signal supplied from the motion compensation prediction unit 608 to generate a decoded image signal (S704).

  The decoded image signal supplied from the adding unit 604 is stored in the decoded image memory 605 (S705), and the motion information of the decoding target block supplied from the motion information decoding unit 606 is stored in the motion information memory 607 ( S706). This completes the decoding process in units of prediction blocks.

[Detailed Function Description of Embodiment 1]
Operation of prediction mode determination unit 109 and motion information encoding unit 110 of the video encoding apparatus according to Embodiment 1 of the present invention, processing in steps S502 and S504 in the flowchart of FIG. 5, and embodiment of the present invention The operation of the motion information decoding unit 606 in the moving image decoding apparatus according to 1 and the detailed operation of the processing in step S702 in the flowchart of FIG. 7 will be described below.

[Definition of Motion Compensation Prediction Mode in Embodiment 1]
FIGS. 8A and 8B are diagrams for explaining three prediction modes for encoding motion information used in motion compensated prediction according to Embodiment 1 of the present invention. In the first prediction mode, the prediction target block directly encodes its own motion information using the continuity of motion in the temporal direction and the spatial direction in the prediction target block and the encoded block adjacent to the prediction target block. In this method, the motion information of spatially and temporally adjacent blocks is used for encoding, which is called a joint prediction mode (merge mode).

  In the combined prediction mode, motion information that is selectively combined from a plurality of adjacent block candidates can be defined, and the motion information is encoded with information (index) that specifies the adjacent block to be used. The obtained motion information is used as it is for motion compensation prediction. Further, in the joint prediction mode, a Skip mode is defined in which the prediction signal predicted in the joint prediction mode is a decoded picture without encoding prediction transmission of the prediction difference information, and a decoded image is obtained with information having only the combined motion information. Can be reproduced. The motion information transmitted in the Skip mode is designation information that defines adjacent blocks, as in the combined prediction mode.

  The second prediction mode is a technique for individually coding all the components of motion information and transmitting motion information with little prediction error to the prediction block, and is called a motion detection prediction mode. In the motion detection prediction mode, information for specifying a reference image (reference image index) and information for specifying a motion vector are encoded separately, as in the case of encoding motion information in conventional motion compensation prediction. The

  In the motion detection prediction mode, the prediction mode indicates whether to use unidirectional prediction or bidirectional prediction. In the case of unidirectional prediction, information for specifying a reference image for one reference image, and motion vector prediction Encode the difference vector with the vector. In the case of bidirectional prediction, information for specifying reference images for two reference images and a motion vector are individually encoded. The prediction vector for the motion vector is generated from the motion information of the adjacent block similarly to the AVC. However, similarly to the combined prediction mode, the motion vector used for the prediction vector can be selected from a plurality of adjacent block candidates, and the motion vector is the prediction vector. The information is transmitted by encoding two pieces of information (index) for designating adjacent blocks to be used for and a difference vector.

  In the third prediction mode, a part of the motion information generated from the adjacent block is encoded by additionally encoding a difference vector value for correcting the motion vector value with respect to the motion information of the adjacent block. This is a technique of correcting motion vectors and encoding them as motion information, and is called a combined motion correction prediction mode (merge MVD mode). Similar to the combined prediction mode, the motion information transmitted in the combined motion correction prediction mode is a difference vector value for the motion vector to be corrected in addition to the information specifying the adjacent block. In the combined motion correction prediction mode in the first embodiment, one difference vector value is always transmitted, and the prediction direction of motion compensation prediction is bidirectional prediction.

  The configuration having the combined motion correction prediction mode is a factor that improves the coding efficiency over the conventional configuration that does not have the combined motion correction prediction mode. That is, in the combined motion correction prediction mode, a difference vector for correcting a motion vector is encoded and added to the information encoded in the combined prediction mode. As a result, when the continuity of motion is not sufficiently maintained, or when the motion vector in the motion information of other blocks does not indicate an accurate motion, the motion vector is generated with respect to the motion information generated from the adjacent block. By correcting only the motion information, motion information for generating a motion compensated prediction signal with a small prediction residual can be expressed with a small amount of information.

  FIG. 9 is a diagram for explaining a relationship between a motion compensation prediction direction, information defining a reference image, and the number of motion vectors to be encoded in motion compensation prediction and intra prediction without performing motion compensation prediction. .

  In the case of the unidirectional prediction mode (UniPred) in which the prediction direction is unidirectional among the motion detection prediction modes, one encoded vector, one reference image index, and one prediction vector index are transmitted as motion information to be encoded. Is done.

  Similarly, in the bidirectional prediction mode (BiPred) in which the prediction direction is bidirectional, two encoded vectors, two reference image indexes, and two prediction vector indexes are transmitted as motion information to be encoded.

  In the joint prediction mode (Merge), only one joint motion information index is transmitted as motion information to be encoded, and unidirectional or bidirectional motion compensation prediction is performed based on the motion information.

  In the combined motion correction prediction mode (MergeMvd), one combined motion information index and one encoded vector are transmitted as motion information to be encoded, and bidirectional motion compensation is performed according to the motion information and the motion vector correction information. Prediction is made.

  In the intra mode (Intra), which is a predictive coding mode in which motion compensation prediction is not performed, motion information is not coded.

  When temporal and spatial correlation with adjacent motion information is very high, the combined prediction mode is selected as an effective motion compensation prediction mode. When the correlation is low or only a part of the motion information can be effectively used as the motion information of the prediction block, the motion detection prediction mode is selected as an effective motion compensation prediction mode. If the correlation is high but there is a slight shift in the motion, or if the motion information of the adjacent block does not show accurate motion, or most of the motion information is valid, but only some motion information When the correlation is not sufficient, the combined motion correction prediction mode is selected as an effective motion compensation prediction mode. Thus, motion compensated prediction with high coding efficiency can be selected according to the situation.

[Detailed Operation Description of Prediction Mode Determination Unit in Moving Picture Encoding Device in Embodiment 1]
FIG. 10 is a diagram illustrating a detailed configuration of the prediction mode determination unit 109 in the video encoding device according to the first embodiment. The prediction mode determination unit 109 has a function of determining an optimal motion compensation prediction mode.

  The prediction mode determination unit 109 includes a unidirectional motion compensation prediction generation unit 1000, a bidirectional motion compensation prediction generation unit 1001, a prediction error calculation unit 1002, a prediction vector calculation unit 1003, a difference vector calculation unit 1004, and a motion information code amount calculation unit 1005. A prediction mode evaluation unit 1006, a combined motion information calculation unit 1007, a combined motion compensation prediction generation unit 1008, and a combined motion correction motion compensation prediction generation unit 1009.

  The motion vector values input from the motion vector detection unit 107 to the prediction mode determination unit 109 in FIG. 1 are converted into one-way motion compensation prediction generation unit 1000, bidirectional motion compensation prediction generation unit 1001, and combined motion correction motion compensation. The motion information supplied to the prediction generation unit 1009 and input from the motion information memory 111 is supplied to the prediction vector calculation unit 1003 and the combined motion information calculation unit 1007.

  In addition, the motion compensation prediction unit 108 receives motion from the unidirectional motion compensation prediction generation unit 1000, the bidirectional motion compensation prediction generation unit 1001, the combined motion compensation prediction generation unit 1008, and the combined motion correction motion compensation prediction generation unit 1009. Reference image designation information and motion vectors used for compensated prediction are output. Further, the motion compensation prediction image generated by the motion compensation prediction unit 108 is supplied to the prediction error calculation unit 1002. The prediction error calculation unit 1002 is further supplied with an image signal of a prediction block to be encoded from the input terminal 100.

  Also, the motion information encoding unit 110 is supplied with motion information to be encoded and the determined prediction mode information from the prediction mode evaluation unit 1006, the motion information is supplied to the motion information memory 111, and the motion compensated prediction signal is subtracted. To the unit 101 and the addition unit 105.

  The unidirectional motion compensation prediction generation unit 1000 receives a motion vector value calculated for each reference image that can be used for unidirectional prediction, supplies reference image designation information to the prediction vector calculation unit 1003, and specifies a reference image designation. Information and a motion vector are output to the motion compensation prediction unit 108.

  Similarly, the bi-directional motion compensation prediction generation unit 1001 receives the motion vector value calculated for each reference image that can be used for bi-directional prediction, and the reference vector designation information for each reference image is the prediction vector calculation unit 1003. The reference image designation information and the motion vector for each reference image are output to the motion compensation prediction unit 108.

  The prediction error calculation unit 1002 calculates a prediction error evaluation value from the input motion compensated prediction image and the prediction block image to be processed. As the calculation for calculating the error evaluation value, the sum SAD of the absolute difference value for each pixel, the sum SSE of the square error value for each pixel, and the like can be used as in the error evaluation value in motion vector detection. Furthermore, a more accurate error evaluation value can be calculated by taking into account the amount of distortion components generated in the decoded image by performing orthogonal transform / quantization performed when encoding the prediction residual. . In this case, the prediction error calculation unit 1002 can be realized by having the functions of the subtraction unit 101, the orthogonal transformation / quantization unit 102, the inverse quantization / inverse transformation unit 104, and the addition unit 105 in FIG. The prediction error calculation unit 1002 supplies the prediction error evaluation value calculated in each prediction mode and the motion compensation prediction signal to the prediction mode evaluation unit 1006.

  The prediction vector calculation unit 1003 is supplied with reference image designation information from the unidirectional motion compensation prediction generation unit 1000 and the bidirectional motion compensation prediction generation unit 1001, and is a candidate block group in the motion information of adjacent blocks supplied from the motion information memory 111. Thus, the motion vector value for the designated reference image is specified. Then, a plurality of prediction vectors are generated together with the prediction vector candidate list, and supplied to the difference vector calculation unit 1004 together with the reference image designation information.

  FIG. 11 is a diagram illustrating a configuration of the prediction vector calculation unit 1003. The prediction vector calculation unit 1003 includes a prediction vector candidate list generation unit 1100 and a prediction vector candidate list deletion unit 1101, creates prediction vector candidates in a predetermined order from the candidate block group, and uses the same motion vector value from among them. By deleting the candidates to be shown, only valid prediction vectors are registered as prediction vector candidates. The detailed operation of the prediction vector calculation unit 1003 will be described later.

  Returning to FIG. 10, the difference vector calculation unit 1004 supplies the prediction vector candidates supplied from the prediction vector calculation unit 1003 from the one-way motion compensation prediction generation unit 1000 and the bidirectional motion compensation prediction generation unit 1001. The difference with the motion vector value to be calculated is calculated, and the difference vector value is calculated. The difference vector value and the prediction vector index, which is the designation information for the prediction vector candidate, are the difference vectors that can represent the motion vector value with the least amount of information when encoding the prediction vector index. Therefore, the prediction vector index and the difference vector value for the prediction vector having the smallest information amount are supplied to the motion information code amount calculation unit 1005 together with the reference image designation information.

  The motion information code amount calculation unit 1005 is supplied from the difference vector calculation unit 1004, based on the difference vector values in the unidirectional prediction mode and the bidirectional prediction mode, the reference image designation information, the prediction vector index, and the prediction mode. A code amount required for motion information is calculated. In addition, the motion information code amount calculation unit 1005 receives information indicating a combined motion information index and a prediction mode that needs to be transmitted in the combined prediction mode from the combined motion compensation prediction generation unit 1008, and performs motion in the combined prediction mode. The amount of code required for information is calculated.

  Similarly, the motion information code amount calculation unit 1005 indicates a combined motion information index, a difference vector value, and a prediction mode that need to be transmitted in the combined motion correction prediction prediction mode from the combined motion correction motion compensation prediction generation unit 1009. Information is received, and the amount of code required for motion information in the combined motion correction prediction mode is calculated. The motion information code amount calculation unit 1005 supplies the motion information calculated in each prediction mode and the code amount required for the motion information to the prediction mode evaluation unit 1006.

  The prediction mode evaluation unit 1006 uses the prediction error evaluation value of each prediction mode supplied from the prediction error calculation unit 1002 and the motion information code amount of each prediction mode supplied from the motion information code amount calculation unit 1005. An overall motion compensation prediction error evaluation value of the prediction mode is calculated, a prediction mode having the smallest evaluation value is selected, and the motion information encoding unit 110 and the motion information memory store the selected prediction mode and motion information for the selected prediction mode. To 111. Similarly, the prediction mode evaluation unit 1006 selects a prediction signal in the selected prediction mode for the motion compensated prediction signal supplied from the prediction error calculation unit 1002 and outputs it to the subtraction unit 101 and the addition unit 105.

  The combined motion information calculation unit 1007 uses a candidate block group in the motion information of the adjacent blocks supplied from the motion information memory 111, a prediction direction indicating one-way or two-way, reference image designation information, motion A plurality of pieces of motion information are generated together with the combined motion information candidate list as motion information composed of vector values, and supplied to the combined motion compensation prediction generation unit 1008 and the combined motion correction motion compensation prediction generation unit 1009.

  FIG. 12 is a diagram illustrating a configuration of the combined motion information calculation unit 1007. The combined motion information calculation unit 1007 includes a combined motion information candidate list generation unit 1200 and a combined motion information candidate list deletion unit 1201. The combined motion information calculation unit 1007 creates motion information candidates in a predetermined order from the candidate block group, and among them, each element of motion information (prediction direction, reference image designation information, and motion vector value) is the same. By deleting the candidate indicating the value of, only valid motion information is registered as a combined motion information candidate. The detailed operation of the combined motion information calculation unit 1007 will be described later.

  Returning to FIG. 10, the combined motion compensation prediction generation unit 1008 determines the prediction direction based on the motion information for each registered combined motion information candidate from the combined motion information candidate list supplied from the combined motion information calculation unit 1007. In accordance with the reference image designation information and motion vector value of one reference image (one direction) or two reference images (bidirectional) to the motion compensation prediction unit 108 to generate a motion compensation prediction image, Are provided to the motion information code amount calculation unit 1005. The detailed operation of the combined motion compensation prediction generation unit 1008 will be described later.

  In the configuration of FIG. 10, the prediction mode evaluation in each combined motion information index is performed by the prediction mode evaluation unit 1006. The prediction error evaluation value and the motion information code amount are calculated by the prediction error calculation unit 1002 and the motion information code amount calculation. In other words, the combined motion compensation prediction generation unit 1008 receives the information from the unit 1005 and, after the combined motion index of the optimal combined motion compensated prediction is determined, evaluates the optimal prediction mode including other prediction modes. Is possible.

  The combined motion corrected motion compensation prediction generation unit 1009 calculates bidirectional motion information for the combined motion information candidate list and registered combined motion information candidates supplied from the combined motion information calculation unit 1007. Then, the reference image whose motion vector value is to be corrected is identified, the motion vector value detected by the motion vector detection unit 107 is input to the reference image designation information of the identified reference image, and the difference vector value is obtained. calculate. Then, a motion vector value used for motion compensated prediction including the bi-directional reference image designation information and the corrected motion vector value is designated to the motion compensated prediction unit 108 to generate a motion compensated predictive image, and combined motion The information index and the difference vector value to be transmitted are supplied to the motion information code amount calculation unit 1005.

  FIG. 13 is a diagram illustrating a configuration of the combined motion corrected motion compensation prediction generation unit 1009. The combined motion corrected motion compensation prediction generation unit 1009 includes a reference reference image / motion correction reference image selection unit 1300, a motion correction reference image motion vector acquisition unit 1301, a combined motion information correction motion compensation prediction generation unit 1302, and a difference vector calculation unit 1303. including.

  The reference reference image / motion correction reference image selection unit 1300 has a function of calculating a bidirectional motion information for a registered combined motion information candidate and specifying a reference image whose motion vector value is corrected. The motion correction reference image motion vector acquisition unit 1301 has a function of inputting a motion vector value for the reference image designation information of the identified reference image. The combined motion information modified motion compensation prediction generation unit 1302 has a function of outputting a motion vector value used for motion compensation prediction and reference image designation information to the motion compensation prediction unit 108. The difference vector calculation unit 1303 has a function of calculating a difference vector value between the motion vector value of the motion information calculated by the standard reference image / motion correction reference image selection unit 1300 and the motion vector value input from the motion vector detection unit 107. Have. The detailed operation of the combined motion corrected motion compensation prediction generation unit 1009 will be described later.

  FIG. 14 is a flowchart for explaining the operation of motion compensation prediction mode / prediction signal generation, which is step S502 of FIG. 5, which operates via the prediction mode determination unit 109 of FIG. First, a combined motion information candidate list is generated (S1400), and a combined prediction mode evaluation value is generated (S1401).

  Subsequently, a combined motion correction prediction mode evaluation value is generated using the combined motion information candidate list generated in step S1400 (S1402), a unidirectional prediction mode evaluation value is generated (S1403), and a bidirectional prediction mode evaluation is performed. A value is generated (S1404), and an optimum prediction mode is selected by comparing the generated evaluation values (S1405). However, the order of evaluation value generation in step S1401, step S1402, step S1403, and step S1404 is not limited to this order.

  A prediction signal is output in accordance with the selected prediction mode (S1406), and motion information is output in accordance with the selected prediction mode (S1407), thereby completing the motion compensation prediction mode / prediction signal generation process for each prediction block. Detailed operations of step S1400, step S1401, step S1402, step S1404, and step S1404 will be described later.

  FIG. 15 is a flowchart for explaining the detailed operation of generating the combined motion information candidate list in step S1400 of FIG. This operation shows the detailed operation of the configuration in the combined motion information calculation unit 1007 in FIG. The combined motion information candidate list generation unit 1200 in FIG. 12 performs spatial combined motion from a candidate block group obtained by excluding candidate blocks outside the region and candidate blocks in the intra mode from the spatial candidate block group supplied from the motion information memory 111. An information candidate list is generated (S1500). Detailed operations for generating the spatially coupled motion information candidate list will be described later.

  The combined motion information candidate list generation unit 1200 subsequently generates temporal combined motion information from a candidate block group obtained by excluding candidate blocks outside the region and candidate blocks in the intra mode from the temporal candidate block group supplied from the motion information memory 111. A candidate list is generated (S1501). Detailed operation of the time combination motion information candidate list generation will be described later.

  Subsequently, in the combined motion information candidate list deleting unit 1201, combined motion information having overlapping motion information from the combined motion information candidate list obtained by integrating the generated spatial combined motion information candidate list and the temporal combined motion information candidate list. If there are a plurality of candidates, the motion information candidate list is updated by deleting one of the combined motion information candidates (S1502).

  Finally, when there is no valid motion information in the adjacent block in the above process and there is no combined motion information candidate (S1503: YES), fixed motion information is added to the combined motion information candidate list (S1504), End the process. As the fixed motion information in the first embodiment, the prediction direction is bidirectional, the reference image designation information (index) is 0, and the two motion vector values are both (0, 0). The fixed motion information in the first embodiment is not limited to the above setting, and the same motion information can be reproduced in the moving picture decoding apparatus by being set by a method that can be implicitly specified. In the above process, when valid motion information exists in adjacent blocks and there is a combined motion information candidate (S1503: NO), step S1504 is skipped and the process ends.

  The candidate block group of motion information supplied from the motion information memory 111 to the combined motion information calculation unit 1007 includes a spatial candidate block group and a temporal candidate block group. First, generation of a spatially coupled motion information candidate list will be described.

  FIG. 16 is a diagram illustrating a spatial candidate block group used for generating a spatially coupled motion information candidate list. The spatial candidate block group indicates a block of the same image adjacent to the prediction target block of the encoding target image. The block group is managed in units of the minimum predicted block size, and the blocks at positions A1 to A4, B1 to B4, C, D, and E as shown in FIG. 16 are adjacent block groups. The position of the candidate block is managed in units of the minimum prediction block size, but if the prediction block size of the adjacent block is larger than the minimum prediction block size, the same motion information for all candidate blocks within the prediction block size Is stored. In the first embodiment, among these adjacent block groups, the spatial candidate block group is assumed to be four blocks of block A1, block B1, block C, and block D.

  FIG. 17 is a flowchart for explaining the detailed operation of generating the spatially coupled motion information candidate list. The following processing is repeated for the four candidate blocks included in the space candidate block group, block A1, block B1, block C, and block D (S1700 to S1703).

  First, the validity of the candidate block is checked (S1701). If the candidate block is not outside the area and is not in the intra mode (S1701: YES), the candidate block is valid. If the candidate block is valid, the motion information of the candidate block is added to the spatially combined motion information candidate list (S1702).

  Here, it is assumed that the spatial combination motion information candidate list includes motion information of four or less candidate blocks, but the spatial candidate block group is at least one or more processed blocks adjacent to the prediction block to be processed. The number of spatially coupled motion information candidate lists may be changed depending on the effectiveness of the candidate block, and the present invention is not limited to this. If the candidate block is out of the region or in the intra mode (S1701: NO), step S1702 is skipped, and the validity / invalidity determination of the next candidate block is performed.

  Subsequently, generation of a time combination motion information candidate list will be described. FIG. 18 is a diagram illustrating a time candidate block group used for generating a time combination motion information candidate list. The temporal candidate block group indicates blocks at the same position and the periphery of the prediction target block in the decoded image ColPic different from the prediction target block. The blocks at positions A1 to A4, B1 to B4, C, D, E, F1 to F4, G1 to G4, H, and I1 to I16 in FIG. 18 are temporally adjacent block groups. In the first embodiment, among these temporally adjacent block groups, the temporal candidate block group is assumed to be two blocks, block H and block I6.

  FIG. 19 is a flowchart for explaining the detailed operation of generating the time combination motion information candidate list. Regarding the block H and block I6 which are two candidate blocks included in the time candidate block group (step 1900, step S1905), the validity of the candidate block is checked (S1901). If the candidate block is valid (S1901: YES), the processing of step S1902 to step S1904 is performed, the generated motion information is registered in the time combination motion information candidate list, and the processing ends. When the candidate block indicates a position outside the screen area, or when the candidate block is an intra prediction block (S1901: NO), the candidate block is not valid and the next candidate block is determined to be valid / invalid.

  If the candidate block is valid (S1901: YES), the reference image selection candidate to be registered in the combined motion information candidate is determined based on the motion information of the candidate block (S1902). In the first embodiment, the reference image in the L0 direction is the reference image that is the closest to the processing target image among the reference images in the L0 direction, and the reference image in the L1 direction is the processing target image among the reference images in the L1 direction. The reference image is the closest distance.

  The determination method of the reference image selection candidate here is not limited to this as long as the reference image in the L0 direction and the reference image in the L1 direction can be determined. The reference image intended at the time of encoding can be determined by determining the reference image by the same method in the encoding process and the decoding process. Other determination methods include, for example, a method of selecting a reference image whose reference image index is 0 in the L0 direction reference image and the L1 direction reference image, and the L0 reference image and the L1 reference image used by the spatially adjacent blocks. Thus, it is possible to use a method of selecting the most frequently used reference image as a reference image in the prediction target block or a method of specifying a reference image in each direction in the encoded stream.

  Next, the motion vector value to be registered in the combined motion information candidate is determined based on the motion information of the candidate block (S1903). As the temporally combined motion information in the first embodiment, bidirectional motion information is calculated based on motion vector values that are effective prediction directions in motion information of candidate blocks. When the prediction direction of the candidate block is one direction of L0 prediction or L1 prediction, motion information in the direction used for prediction (L0 direction or L1 direction) is selected, and the reference image designation information and motion vector value are selected. A reference value for generating bidirectional motion information.

  When the prediction direction of the candidate block is bidirectional, motion information of either L0 prediction or L1 prediction is selected as a reference value. The selection method of the reference value is, for example, selecting motion information that exists in the same prediction direction as ColPic, selecting a candidate block having a shorter inter-image distance from ColPic in each of the L0 prediction and L1 prediction reference images. For example, it is possible to select a direction in which the motion vectors of the L0 prediction and the L1 prediction of the candidate block intersect with the encoding process target image.

  When the motion vector value used as a reference for generating bidirectional motion information is determined, a motion vector value to be registered in the combined motion information candidate is calculated.

FIG. 20 is a diagram for explaining a calculation method of motion vector values mvL0t and mvL1t registered for L0 prediction and L1 prediction with respect to the reference motion vector value ColMv for temporally coupled motion information. The distance between images between ColPic for the reference motion vector value ColMv and the reference image that is the target of the motion vector used as a reference for the candidate block is referred to as ColDist. The inter-image distances between the L0 prediction and L1 prediction reference images and the processing target image are CurrL0Dist and CururL1Dist. A motion vector obtained by scaling ColMv with a distance ratio of ColDist to CurrL0Dist and CurrL1Dist is set as a motion vector to be registered. Specifically, the motion vector values mvL0t and mvL1t to be registered are calculated by the following formulas 1 and 2.
mvL0t = mvCol × CurrL0Dist / ColDist (Formula 1)
mvL1t = mvCol × CurrL1Dist / ColDist (Formula 2)
It becomes.

  Returning to FIG. 19, the bi-directional reference image selection information (index) and the motion vector value generated in this way are added to the combined motion information candidates (S1904), and the time combined motion information candidate list creation process ends. To do.

  FIG. 21 is a flowchart for explaining the detailed operation of the combined prediction mode evaluation value generation in step S1401 of FIG. This operation is a detailed operation of the configuration using the combined motion compensation prediction generation unit 1008 of FIG.

  First, the prediction error evaluation value is set to the maximum value, and the combined motion information index that minimizes the prediction error is initialized (for example, a value outside the list such as −1) (S2100). If the number of combined motion information candidate lists generated by the combined motion information candidate list generation process is num_of_index, the following processing is repeated for combined motion information candidates from i = 0 to num_of_index-1 (S2101 to S2109).

  First, the motion information stored in the index i is acquired from the combined motion information candidate list (S2102). Subsequently, a motion information code amount is calculated (S2103). In the joint prediction mode, since only the joint motion information index is encoded, only the joint motion information index becomes the motion information code amount. In the first embodiment, a Truncated Unary code string is used as the code string of the combined motion information index.

  Subsequently, when the prediction direction of the motion information is unidirectional (S2104: YES), the reference image designation information and the motion vector for one reference image are set in the motion compensation prediction unit 108 in FIG. A compensated prediction block is generated (S2105). When the motion information is not unidirectional (S2104: NO), reference image designation information and motion vectors for two reference images are set in the motion compensation prediction unit 108, and a bidirectional motion compensation prediction block is generated (S2106).

  Subsequently, a prediction error evaluation value is calculated from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block (S2107), and when the prediction error evaluation value is the minimum value, the evaluation value is updated. The prediction error minimum index is updated (S2108).

  As a result of comparison of the prediction error evaluation values for all the combined motion information candidates, the selected prediction error minimum index is output together with the prediction error minimum value and the motion compensated prediction block as a combined motion information index used in the combined prediction mode. (S2110), the combined prediction mode evaluation value generation process is terminated.

  FIG. 22 is a flowchart for explaining the detailed operation of the combined motion correction prediction mode evaluation value generation in step S1402 of FIG. This operation shows a detailed operation of the configuration using the combined motion corrected motion compensation prediction generation unit 1009 of FIG.

  First, the prediction error evaluation value is set to the maximum value, and the combined motion information index that minimizes the prediction error is initialized (for example, a value outside the list such as −1) (S2200). If the number of combined motion information candidate lists generated by the combined motion information candidate list generation process is num_of_index, the following processing is repeated for combined motion information candidates from i = 0 to num_of_index-1 (S2201 to S2210).

  First, the motion information stored in the index i is acquired from the combined motion information candidate list (S2202). Subsequently, a motion-corrected reference image serving as a reference image for transmitting the motion vector value for bidirectional prediction, reference image designation information, and the difference vector is determined from the acquired motion information (S2203). The detailed operation of step S2203 will be described later.

  Next, the motion vector value detected for the motion correction reference image is input from the motion vector detection unit 107 of FIG. 1 to the motion correction reference image determined in step S2203 (S2204). In order to use the input motion vector value for motion compensation prediction, a difference vector with the motion vector value of the motion correction reference image in the motion information calculated from the combined motion information candidate is generated (S2205).

  Subsequently, a motion information code amount is calculated (S2206). In the combined motion correction prediction mode, necessary motion information is transmitted by encoding the combined motion information index and the difference vector value. For this reason, the code amount of the combined motion information index and the code amount of the difference vector value are added to obtain the motion information code amount in the combined motion correction prediction mode.

  In the combined corrected motion prediction mode, the motion information obtained by replacing only the motion vector corrected for the motion corrected reference image with respect to the motion vector of the bidirectional prediction generated from the combined motion information and the reference image selection information is shown in FIG. Is set in the motion compensated prediction unit 108 in step S2207 to generate a bidirectional motion compensated prediction block.

  A prediction error evaluation value is calculated from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block (S2208). When the prediction error evaluation value is the minimum value, the evaluation value is updated and the prediction error is calculated. The minimum index is updated (S2209).

  As a result of comparison of the prediction error evaluation values for all the combined motion information candidates, the selected prediction error minimum index is output together with the prediction error minimum value and the motion compensated prediction block as a combined motion information index used in the combined prediction mode. (S2211) The combined motion correction prediction mode evaluation value generation process ends.

  Next, before describing the detailed operation of step S2203, the scaling process for the combined motion information in the combined motion correction prediction mode will be described. FIGS. 23A to 23C are diagrams illustrating the concept of scaling processing for combined motion information in the combined motion modified prediction mode. As the combined motion information, the space BiPred in which the prediction direction calculated from the spatial adjacent block shown in FIG. 23A is bidirectional and the prediction direction calculated from the spatial adjacent block shown in FIG. There is a space UniPred and a time BiPred calculated from the time concatenation block shown in FIG.

  The scaling processing for the combined motion information in the first embodiment uses the motion vector applied at the time of calculating the combined motion information as it is in the space BiPred and the time BiPred in which bidirectional motion information can be acquired, and for the time UniPred, The motion vector value defined in the direction is scaled by the distance ratio between the encoding target image, the L0 image, and the L1 image, thereby generating motion information in a direction that has not been used.

  FIG. 24 is a diagram illustrating the number of scaled motion vectors that exist for each piece of combined motion information. The space UniPred is 1 in the above scaling process, the space BiPred is 0 because no scaling process is performed when calculating the combined motion information, and the time BiPred is subjected to the scaling process when calculating two motion vector values when calculating the combined motion information. Because it is, it becomes two.

  When the motion vector value subjected to the above scaling processing is compared with the combined motion information in which the motion vector value used in the adjacent block is directly adopted as the motion vector value, the influence of the calculation error due to scaling and the temporal change of the motion amount There is an influence when it is not constant, and there is a high possibility that the accuracy of the motion information has shifted. In addition, since the motion vector value used for the adjacent block has a larger screen change as the reference image is temporally separated from the encoding target, the accuracy of the value decreases.

  In the first embodiment, using the characteristics at the time of calculation of the motion vector registered in the combined motion information, a motion vector that needs to be corrected by sending a difference vector is transmitted as additional information. And specify only the difference vector. Thereby, motion information with high prediction accuracy of motion compensation prediction can be realized by transmission of a small amount of motion information.

  In the first embodiment, a reference image having a motion vector value calculated by scaling is preferentially set as a motion correction reference image for transmitting a difference vector. When the two motion vectors are both scaled or are not scaled together, a reference image that is separated in time from the prediction target image is set as a motion-corrected reference image that transmits the difference vector.

  FIGS. 25A to 25C are conceptual diagrams showing the differential motion vector transmission reference image determination algorithm in the first embodiment. In the space BiPred shown in FIG. 25A, since the scaled motion vector is zero, a difference vector is added to the motion vector of the reference image far in time. In the space UniPred shown in FIG. 25B, since there is one scaled motion vector, a difference vector is added to the scaled motion vector. At time BiPred shown in FIG. 25C, since there are two scaled motion vectors, a difference vector is added to the motion vector of the reference image far in time.

  FIG. 26 is a flowchart for explaining detailed operation of step S2203 of FIG. In step S2203 of FIG. 22, a reference image transmission process for transmitting the difference vector is performed.

  First, when the prediction direction of the acquired combined motion information is unidirectional (S2600: YES), it is combined motion information derived in the space UniPred, and the second reference for performing bidirectional motion compensation prediction Processing for determining the motion vector values of the image and the second reference image is performed (S2601). The detailed operation of step S2601 will be described later.

  Subsequently, when the scaled motion vector is only one unidirectional motion vector among the motion vector values of the bidirectional reference image (S2602: YES), the reference image having the scaled motion vector value is corrected for motion. The reference image is set (S2603).

  When the scaled motion vector is not one motion vector in one direction (S2602: NO), POC values indicating time information of the encoding target image, the L0 reference image, and the L1 reference image are acquired, and POCcur, POC_L0, and POC_L1 are obtained. (S2604).

The temporal distance between the encoding target image and the two reference images is compared by the following expression 3.
abs (POCcur-POC_L0)> abs (POCcur-POC_L1)
... (Formula 3)

  When Expression 3 is satisfied (S2605: YES), the L0 reference image is determined to be a distant reference image, and the L0 reference image is set as a motion corrected reference image (S2606). When Expression 3 is not satisfied (S2605: NO), the L1 reference image is set as a motion-corrected reference image (S2607). This completes the process of determining the reference image for transmitting the bidirectional motion vector value and the difference vector.

  FIG. 27 is a flowchart for explaining the detailed operation of step S2601 of FIG. First, when the motion information stored in the combined motion information is L0 prediction (S2700: YES), a reference image index for L1 prediction is determined as the second reference image (S2701). When it is not L0 prediction (S2700: NO), a reference image index for L0 prediction is determined as the second reference image (S2702). The reference image index determination process in step S2702 will be described later.

  Subsequently, POC values of the encoding target image, the reference image stored in the combined motion information, and the second reference image are acquired and set to POCcur, POC_Lx, POC_Ly (S2703). Using the POC value, the motion vector value stored in the combined motion information is scaled by the distance ratio between the two reference images and the encoding target image (S2704).

  When the motion information stored in the combined motion information is L0 prediction (S2705: YES), the motion vector value generated by scaling is set as the reference motion vector value for the reference image of L1 prediction (S2706). If it is not the L0 prediction (S2705: NO), the motion vector value generated by scaling is set as the reference motion vector value for the reference image for the L0 prediction (S2707), and the process is terminated.

  FIG. 28 is a flowchart for explaining the detailed operation of step S2702 of FIG. In step S2702 of FIG. 27, a process of determining a reference image index is performed.

  First, the reference image stored in the combined motion information is acquired as the standard reference image, and the reference image list in the prediction direction for setting the second reference image is acquired as the second reference image candidate list (S2800).

  Next, it is checked whether or not a reference image that exists at a time position in the opposite direction to the standard reference image across the encoding target image exists in the second reference image candidate list (S2801). If it exists (S2801: YES), the reference image existing at the time position closest to the encoding target image is determined as the reference image index for the second reference image (S2802). If it does not exist (S2801: NO), the reference image existing at the time position closest to the encoding target is set as the second reference image candidate (S2803), and the reference reference image and the second reference image candidate are the same image. (S2804).

  If the reference reference image and the second reference image candidate are not the same image (S2804: NO), the second reference image candidate is determined as a reference image index for the second reference image (S2805). When the reference reference image and the second reference image candidate are the same image (S2804: YES), it is checked whether another reference image exists in the second reference image candidate list (S2806). If it exists (S2806: YES), a reference image closest to the next encoding target image at the time position is acquired and determined as a reference image index for the second reference image (S2807). If no other reference image exists (S2806: NO), the process proceeds to step S2805, and the second reference image candidate is determined as a reference image index for the second reference image. As described above, the reference image index for the second reference image in all cases is determined, and the process ends.

  The determination algorithm for the second reference image in the first embodiment includes both an interpolation type (interpolation type) in which the motion compensation prediction process is configured with reference images that exist in opposite directions across the encoding target image in bidirectional prediction. Directional prediction can average the effects of motion and temporal changes of objects compared to bi-directional prediction of interpolation (extrapolation type) consisting of reference images that exist in the same direction without interposing the encoding target image. In addition, the reference image closer to the encoding target image is less affected by the motion and the time change of the object and the motion compensation prediction accuracy is better, and the prediction image generated by superimposing the same reference image The bi-directional prediction image generated from two different reference images is a deterministic algorithm based on three findings that the prediction image has a higher prediction accuracy when the motion is correctly captured. This method has an advantage that a reference image suitable for bidirectional prediction can be set as the second reference image.

  However, regarding the above-described determination process of the reference image index, it is only necessary to determine the reference image index of the second reference image, and it is possible to use a different determination algorithm. The reference image intended at the time of encoding can be determined by determining the reference image by the same method in the encoding process and the decoding process. As another determination method, when a motion vector in the motion information of another block does not indicate an accurate motion, a minute change is added to the reference image by using a prediction image generated by overlapping the same reference image. Emphasizing the effect that a motion compensated prediction signal can be added as a new candidate, step S2804, which is a process for checking whether the reference reference image and the second reference image candidate are the same image in the flowchart of FIG. 28, is eliminated. Even if the images are the same, a method of determining them as the second reference image candidates can be used.

  As another determination method, a method of simply selecting a reference image closest to the encoding target image from among reference images in the prediction direction determined as the second reference image, or the reference image index is always 0. Time, such as a method for obtaining the indicated reference image, or a method for selecting the reference image used most frequently as the reference image in the prediction target block in the L0 reference image and the L1 reference image used by the spatially adjacent block It is also possible to use determination means used in the reference image determination process in BiPred. In addition, the algorithm shown in FIG. 28 can be used for reference image determination processing at time BiPred.

  FIG. 29 is a flowchart for explaining the detailed operation of step S1403 of FIG. In step S1403 of FIG. 14, a one-way prediction mode evaluation value generation process is performed.

  First, reference image designation information (index) and motion vector values for unidirectional prediction are acquired (S2900). Subsequently, a prediction vector candidate list is generated (S2901), an optimal prediction vector is selected from the prediction vectors, and a difference vector is generated (S2902). It is desirable to select the optimal prediction vector with the least amount of code when the difference vector between the prediction vector and the motion vector to be transmitted is actually encoded. However, the horizontal and vertical components of the difference vector are simply selected. The calculation may be simplified by a method such as selecting one having a small absolute sum. Detailed operation of step S2901 will be described later.

  Subsequently, a motion information code amount is calculated (S2903). In the case of the unidirectional prediction mode, as motion information to be encoded, there are three elements of reference image designation information, a difference vector value, and a prediction vector index for one reference image, and the total amount of each encoded code amount is Calculated as the motion information code amount. As a predictive vector index code string generation method according to Embodiment 1, a Trunked Unary code string is used in the same manner as the combined motion information index code string.

  Subsequently, reference image designation information and a motion vector for one reference image are set in the motion compensated prediction unit 108 in FIG. 1 to generate a unidirectional motion compensated prediction block (S2904). Further, a prediction error evaluation value is calculated from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block (S2905), and a reference image designation that is the prediction error evaluation value and motion information for one reference image. Information, a difference vector value, and a prediction vector index are output with a motion compensation prediction block (S2906), and a one-way prediction mode evaluation value generation process is complete | finished.

  FIG. 30 is a flowchart for explaining the detailed operation of step S2901 of FIG. In step S2901 in FIG. 29, a prediction vector candidate list is generated. This operation shows the detailed operation of the configuration in the prediction vector calculation unit 1003 of FIG.

  The prediction vector candidate list generation unit 1100 in FIG. 11 uses the spatial motion information from the candidate block group excluding the candidate block outside the region from the spatial candidate block group supplied from the motion information memory 111 and the candidate block in the intra mode. A prediction vector candidate list is generated (S3000). The detailed operation of generating the spatial motion information use prediction vector candidate list will be described later.

  The prediction vector candidate list generation unit 1100 subsequently uses the temporal motion information use prediction vector from the candidate block group obtained by excluding the candidate block outside the region and the candidate block in the intra mode from the temporal candidate block group supplied from the motion information memory 111. A candidate list is generated (S3001). The detailed operation of generating the temporal motion information use prediction vector candidate list will be described later.

  Subsequently, the prediction vector candidate list deletion unit 1101 performs prediction with overlapping motion vectors from the prediction vector candidate list obtained by integrating the generated spatial motion information use prediction vector candidate list and temporal motion information use prediction vector candidate list. If there are a plurality of vector candidates, the prediction vector candidate list is updated by deleting one of the prediction vector candidates (S3002).

  Finally, when there is no valid motion information in the adjacent block and no prediction vector candidate exists in the above processing (S3003: YES), the fixed vector value (0, 0) is added to the prediction vector candidate list (S3004). ), The process is terminated.

  FIG. 31 is a flowchart for explaining a detailed operation of generating a spatial motion information use prediction vector candidate list. The candidate block group of motion information supplied from the motion information memory 111 to the prediction vector calculation unit 1003 includes a spatial candidate block group and a temporal candidate block group. The concepts of the spatial candidate block group and the temporal candidate block group are the same as those of the motion information candidate blocks for the combined motion information shown in FIGS.

  In the first embodiment, the space candidate block group is assumed to be four blocks of block A1, block B1, block C, and block D in FIG. In Embodiment 1, the number and position of candidate blocks are not limited, and the number and position of candidate blocks used in the combined motion information and the number and position of candidate blocks used in the prediction vector match even if they match. It doesn't have to be.

  First, the following processing is repeated for the four candidate blocks included in the space candidate block group, block A1, block B1, block C, and block D (S3100 to S3103).

  First, the validity of the motion vector value in the reference image designated in the candidate block is checked (S3101). The motion vector value becomes invalid not only when the candidate block is not out of the region and not in the intra mode, but also when the reference image designated by the unidirectional prediction is not used or when the reference index is different. However, in the first embodiment, when the reference image designated by the unidirectional prediction is not used or when the reference index is different, additional motion vector values are obtained by scaling the motion vector values of other reference images. It is also possible to perform processing and handle it as an effective motion vector, and the motion vector value validity / invalidity determination method is not limited to the method of the first embodiment.

  Subsequently, if the motion vector value is valid (S3101: YES), the motion vector value of the candidate block is added to the prediction vector candidate list (S3102). If the motion vector value is invalid (S3101: NO), step S3102 is skipped, and valid / invalid determination of the motion vector value of the next candidate block is performed.

  FIG. 32 is a flowchart for explaining a detailed operation of generating a temporal motion information use prediction vector candidate list. In the first embodiment, among the temporally adjacent block groups in FIG. 18, the temporal candidate block group is assumed to be two blocks, block H and block I6. Also in the generation of a temporal motion information prediction vector candidate list, the number and position of candidate blocks are not limited, and can be set separately from the number and position of candidate blocks used in combined motion information.

  Regarding the block H and the block I6 that are two candidate blocks included in the time candidate block group (step 3200, step S3204), the validity of the candidate block is checked (S3201). If the candidate block is valid (S3201: YES), the processes of steps S3202 to S3203 are performed, the generated motion vector values are registered in the prediction vector candidate list, and the process ends. When the candidate block indicates a position outside the screen area, or when the candidate block is an intra prediction block, the candidate block is not valid (S3201: NO), and the validity / invalidity determination of the next candidate block is performed.

  If the candidate block is valid (S3201: YES), the motion vector value to be registered in the prediction vector candidate is determined based on the motion information of the candidate block (S3202). The confirmed motion vector value is added to the prediction vector candidate list (S3203), and generation of the temporal motion information use prediction vector candidate list ends.

  FIG. 33 is a flowchart for explaining the detailed operation of step S1404 of FIG. In step S1404 of FIG. 14, a bidirectional prediction mode evaluation value generation process is performed.

  First, reference image designation information (index) and motion vector values for L0 prediction are acquired (S3300). Subsequently, an L0 prediction vector candidate list is generated (S3301), an optimal prediction vector in L0 prediction is selected from the prediction vectors, and a difference vector is generated (S3302). These processes are the same as the processes from step 2900 to step S2902 in the one-way prediction mode evaluation value generation process in the flowchart of FIG.

  Subsequently, reference image designation information (index) and motion vector value for L1 prediction are acquired (S3303). Subsequently, an L1 prediction vector candidate list is generated (S3304), an optimal prediction vector in L1 prediction is selected from the prediction vectors, and a difference vector is generated (S3305). These processes are also the same as the processes from step 2900 to step S2902 in the one-way prediction mode evaluation value generation process in the flowchart of FIG.

  Subsequently, a motion information code amount is calculated (S3306). In the bidirectional prediction mode, the motion information to be encoded is a total of six elements of reference image designation information, difference vector value, and prediction vector index for the two reference images L0 and L1, and is encoded respectively. The total code amount is calculated as the motion information code amount. For this reason, the amount of code of motion information to be encoded is larger than in the combined prediction mode and the combined motion correction prediction mode.

  Subsequently, reference image designation information and motion vectors for the two reference images are set in the motion compensated prediction unit 108 in FIG. 1 to generate a bidirectional motion compensated prediction block (S3307). Further, a prediction error evaluation value is calculated from the prediction error and the motion information code amount of the motion compensated prediction block and the prediction target block (S3308), and the prediction error evaluation value and motion information for two reference images L0 and L1. The two reference image designation information, the two difference vector values, and the two prediction vector indexes are output together with the motion compensation prediction block (S3309), and the bidirectional prediction mode evaluation value generation process is terminated.

  The above processing is the detailed operation of the prediction mode determination unit 109 in the video encoding apparatus in the first embodiment.

[Detailed Description of Operation of Motion Information Encoding Unit in Moving Picture Encoding Device in Embodiment 1]
FIG. 34 is a flowchart for explaining the detailed operation of step S504 of FIG. The process of step S504 in FIG. 5 shows a process of encoding motion information for each motion compensation prediction mode in the motion information encoding unit 110 of the video encoding apparatus of the first embodiment.

  The motion information encoding unit 110 is supplied with information indicating the prediction mode in the motion compensated prediction mode determined by the prediction mode determination unit 109 and information necessary for expressing the motion information of each prediction mode. The information encoding unit 110 starts a motion information encoded data generation process.

  First, when the prediction mode is the combined prediction mode (S3400: YES), the process proceeds to step S3401, and when the prediction error signal is not encoded in the target prediction block (Skip mode) (S3401: YES), The Skip flag is encoded (S3402). Otherwise (S3401: NO), the merge flag is encoded as 1 (S3403).

  After the Skip flag or merge flag is encoded, if the combined motion information candidate list is greater than 1 (S3404: YES), the combined motion information index is encoded (S3405), and the process ends. When the combined motion information candidate list is 1 (S3404: NO), since combined motion information can be specified, the combined motion information index is not transmitted.

  Next, when the prediction mode is not the combined prediction mode (S3400: NO), the merge flag is encoded as 0 (S3406), and the motion prediction flag is encoded (S3407). The motion prediction flag is encoded as information indicating whether the prediction direction is unidirectional prediction or bidirectional prediction.

  FIG. 35 is a diagram illustrating an example of a motion prediction flag in the first embodiment. As the motion prediction flag (inter_pred_flag), a value indicating one direction (Pred_LC) is 0, and a value indicating bidirectional (Pred_BI) is 1. Regarding the combined motion correction prediction mode, the structure of motion compensation prediction is bidirectional prediction, and a motion prediction flag is sent as Pred_BI.

  Subsequently, when the prediction direction indicated by the motion prediction flag is unidirectional prediction (S3408: YES), motion information transmission in the unidirectional prediction mode is performed. First, reference image designation information (index) is encoded (S3409), and then a difference vector is encoded (S3410). If the predicted vector candidate list is greater than 1 (S3411: YES), the predicted vector index is encoded (S3412), and the process is terminated. When the prediction vector candidate list is 1 (S3411: NO), since the prediction vector can be specified, the index is not transmitted.

  On the other hand, when the prediction direction indicated by the motion prediction flag is not unidirectional prediction (S3408: NO), when the prediction mode is the combined motion correction prediction mode (S3413: YES), motion information transmission in the combined motion correction prediction mode is performed. Is done.

  First, the merge Mvd flag for specifying the combined motion correction prediction mode is encoded with 1 (S3414). Subsequently, when the combined motion information candidate list is larger than 1 (S3415: YES), the combined motion information index is encoded (S3416). When the combined motion information candidate list is 1 (S3415: NO), the combined motion information index is not transmitted. Finally, the difference vector value between the motion vector value of the motion correction reference image specified in the combined motion correction prediction mode and the motion vector before correction of the motion correction reference image of the combined motion information is encoded (S3417), and processing Exit.

  When the prediction mode is not the combined motion correction prediction mode (S3413: NO), the prediction mode is the bidirectional prediction mode, and motion information transmission in the bidirectional prediction mode is performed. First, the merge Mvd flag is encoded with 0 to indicate that it is not the combined motion correction prediction mode (S3418). Subsequently, L0 reference image designation information, which is motion information of L0 prediction, is encoded (S3419), and an L0 difference vector value is encoded (S3420). When the prediction vector candidate list for the L0 prediction is larger than 1 (S3421: YES), the L0 prediction vector index is encoded (S3422). When the prediction vector candidate list for L0 prediction is 1 (S3421: NO), the L0 prediction vector index is not transmitted.

  Subsequently, L1 reference image designation information, which is motion information of L1 prediction, is encoded (S3423), and an L1 difference vector value is encoded (S3424). When the prediction vector candidate list for L1 prediction is larger than 1 (S3425: YES), the L1 prediction vector index is encoded (S3426). When the prediction vector candidate list for L1 prediction is 1 (S3425: NO), the L1 prediction vector index is not transmitted. With the above processing, the generation of motion information encoded data is completed.

  FIG. 36 shows an example of the syntax of the encoded stream generated by the motion information encoded data generation process shown in the flowchart of FIG. In the coding syntax of FIG. 36, the transmission of the Skip flag is selected in units of coding blocks in a layer higher than the coding of motion information in units of prediction blocks. Therefore, it is determined (S3401) whether or not the prediction error signal in FIG. 34 is not encoded (Skip mode) in units of higher-order encoded blocks.

[Detailed Operation Description of Motion Information Decoding Unit in Moving Picture Decoding Device in Embodiment 1]
FIG. 37 is a diagram showing a detailed configuration of the motion information decoding unit 606 in the moving picture decoding apparatus according to Embodiment 1 shown in FIG. The motion information decoding unit 606 includes a motion information bitstream decoding unit 3700, a prediction vector calculation unit 3701, a vector addition unit 3702, a unidirectional motion compensation prediction decoding unit 3703, a bidirectional motion compensation prediction decoding unit 3704, and a combined motion information calculation unit 3705. A combined motion compensated prediction decoding unit 3706 and a combined motion corrected motion compensated prediction decoding unit 3707.

  For the motion information decoding unit 606 in FIG. 6, the motion information bitstream input from the demultiplexing unit 601 is supplied to the motion information bitstream decoding unit 3700, and the motion information input from the motion information memory 607 is predicted. The vector calculation unit 3701 and the combined motion information calculation unit 3705 are supplied.

  In addition, the motion compensation prediction unit 608 receives motion from the one-way motion compensation prediction decoding unit 3703, the bidirectional motion compensation prediction decoding unit 3704, the joint motion compensation prediction decoding unit 3706, and the joint motion correction motion compensation prediction decoding unit 3707. Reference image designation information and motion vectors used for compensated prediction are output, and decoded motion information including information indicating the prediction direction is stored in the motion information memory 607.

  The motion information bitstream decoding unit 3700 generates motion information corresponding to the transmitted prediction mode and the prediction mode by decoding the motion information bitstream input from the demultiplexing unit 601 according to the encoding syntax. To do. Among the generated motion information, the combined motion information index is supplied to the combined motion compensation prediction decoding unit 3706 or the combined motion modified motion compensation prediction decoding unit 3707 according to the prediction mode, and the reference image designation information is the prediction vector calculation unit. 3701, the prediction vector index is supplied to the vector addition unit 3702, and the difference vector value is supplied to the vector addition unit 3702 or the combined motion modified motion compensated prediction decoding unit 3707 according to the prediction mode.

  The prediction vector calculation unit 3701 predicts a reference image that is a target of motion compensation prediction from the motion information of the adjacent block supplied from the motion information memory 607 and the reference image designation information supplied from the motion information bitstream decoding unit 3700. A vector candidate list is generated and supplied to the vector addition unit 3702 together with the reference image designation information. Regarding the operation of the prediction vector calculation unit 3701, the same operation as that of the prediction vector calculation unit 1003 in the moving picture encoding apparatus in FIG. 10 is performed, and the same candidate list as the prediction vector candidate list at the time of encoding is generated.

  The vector addition unit 3702 indicates the prediction vector index from the prediction vector candidate list and reference image designation information supplied from the prediction vector calculation unit 3701 and the prediction vector index and difference vector supplied from the motion information bitstream decoding unit 3700. By adding the prediction vector value and the difference vector value registered at the set position, the motion vector value for the reference image to be motion compensated prediction is reproduced. The reproduced motion vector value is supplied to the unidirectional motion compensated prediction decoding unit 3703 or the bidirectional motion compensated prediction decoding unit 3704 according to the prediction mode together with the reference image designation information.

  The one-way motion compensation prediction decoding unit 3703 is supplied with the reproduced motion vector value and reference image designation information for one reference image from the vector addition unit 3702, and the motion compensation prediction unit 608 receives the motion vector value and the reference image designation information. To generate a motion compensated prediction signal.

  The bidirectional motion compensation prediction decoding unit 3704 is supplied with the reproduced motion vector values and reference image designation information for the two bidirectional reference images from the vector addition unit 3702, and performs motion compensation on the motion vector values and the reference image designation information. By setting the prediction unit 608, a motion compensated prediction signal is generated.

  The combined motion information calculation unit 3705 generates a combined motion information candidate list from the motion information of adjacent blocks supplied from the motion information memory 607, and combines the combined motion information candidate list and the combined motion information candidate that is a component in the list. The reference image designation information and the motion vector value are supplied to the joint motion compensation prediction decoding unit 3706 or the joint motion corrected motion compensation prediction decoding unit 3707 according to the prediction mode.

  Regarding the operation of the combined motion information calculation unit 3705, the same operation as that of the combined motion information calculation unit 1007 in the moving image encoding apparatus of FIG. 10 is performed, and the same candidate list as the combined motion information candidate list at the time of encoding is generated. Is done.

  The combined motion compensated prediction decoding unit 3706 includes a combined motion information candidate list supplied from the combined motion information calculation unit 3705, reference image designation information of a combined motion information candidate that is a component in the list, a motion vector value, and a motion information bit. The reference image designation information and the motion vector value in the combined motion information candidate list indicated by the combined motion information index are reproduced from the combined motion information index supplied from the stream decoding unit 3700, and set in the motion compensation prediction unit 608. Generate a motion compensated prediction signal.

  The combined motion corrected motion compensated prediction decoding unit 3707 includes a combined motion information candidate list supplied from the combined motion information calculation unit 3705, reference image designation information of a combined motion information candidate that is a component in the list, a motion vector value, and a motion. The motion information for the combined motion correction prediction mode is calculated from the combined motion information index supplied from the information bitstream decoding unit 3700, and the motion correction reference image is determined. Then, by adding the difference vector value to the motion vector value of the motion-corrected reference image, the reference image designation information and motion vector value for bidirectional prediction are reproduced and set in the motion compensated prediction unit 608. A compensated prediction signal is generated.

  FIG. 38 is a flowchart for explaining the detailed operation of step S701 in FIG. The motion information decoding process in step S701 in FIG. 7 is performed by the motion information bitstream decoding unit 3700, the prediction vector calculation unit 3701, the combined motion information calculation unit 3705, and the combined motion correction motion compensation prediction decoding unit 3707.

  The motion information decoding process is a process of decoding motion information from the encoded bit stream encoded with the syntax structure of FIG. First, the Skip flag is decoded in a predetermined unit of the encoded block (S3800). Thereafter, processing is performed in units of prediction blocks. When the Skip flag indicates the Skip mode (S3801: YES), joint prediction motion information decoding is performed (S3802). Detailed processing in step S3802 will be described later.

  If it is not the Skip mode (S3801: NO), the merge flag is decoded (S3803). When the merge flag indicates 1 (S3804: YES), the process proceeds to joint prediction motion information decoding in step S3802.

  If the merge flag is not 1 (S3804: NO), the motion prediction flag is decoded (S3805). When the motion prediction flag is in the unidirectional prediction mode (S3806: YES), that is, when inter_pred_flag in FIG. 35 indicates Pred_LC, unidirectional prediction motion information decoding is performed (S3807). Detailed operation of step S3807 will be described later.

  When the motion prediction flag does not indicate the one-way prediction mode (S3806: NO), the merge Mvd flag is decoded (S3808). When the merge Mvd flag is 1 (S3809: YES), joint motion modified prediction motion information decoding is performed (S3810), and when the merge Mvd flag is not 1 (S3809: NO), bidirectional prediction motion information decoding is performed. (S3811). Detailed operations of step S3810 and step S3811 will be described later.

  FIG. 39 is a flowchart for explaining the detailed operation of step S3802 of FIG. In step S3802 of FIG. 38, joint prediction motion information decoding processing is performed.

  First, the combined prediction mode is set as the prediction mode (S3900), and a combined motion information candidate list is generated (S3901). The process of step S3901 is the same process as step S1401 which is the combined motion information candidate list generation in the moving image encoding device in the flowchart of FIG.

  When the combined motion information candidate list is larger than 1 (S3902: YES), that is, when there are a plurality of combined motion information candidates, the combined motion information index is decoded (S3903), and the combined motion information candidate list is one. (S3902: NO), 0 is set to the combined motion information index (S3904).

  Subsequently, the motion information stored at the position indicated by the combined motion information index is acquired from the combined motion information candidate list (S3905). The motion information to be acquired includes a prediction direction indicating unidirectional / bidirectional, reference image designation information, and a motion vector value. The generated motion information is stored as motion information in the joint prediction mode (S3906), and is supplied to the joint motion compensation prediction decoding unit 3706.

  FIG. 40 is a flowchart for explaining the detailed operation of step S3807 of FIG. In step S3807 of FIG. 38, a one-way predicted motion information decoding process is performed.

  First, the reference image designation information is decoded (S4000), and the difference vector value is decoded (S4001). Next, a prediction vector candidate list is generated (S4002). When the prediction vector candidate list is larger than 1 (S4003: YES), the prediction vector index is decoded (S4004), and when the prediction vector candidate list is 1 (S4003: NO), 0 is set to the prediction vector index (S4005).

  Here, in step S4002, processing similar to that in step S2901 in the flowchart of FIG. 29 in the video encoding device is performed. Next, the motion vector value stored at the position indicated by the prediction vector index is acquired from the prediction vector candidate list (S40006). The motion vector is reproduced by adding the decoded difference vector value and the motion vector value (S4007). As the generated motion information, reference image designation information and motion vector values for one reference image are stored as motion information in the unidirectional prediction mode (S4008) and supplied to the unidirectional motion compensated prediction decoding unit 3703.

  FIG. 41 is a flowchart for explaining the detailed operation of step S3811 of FIG. In step S3811 of FIG. 38, bidirectional prediction motion information decoding processing is performed.

  First, reference image designation information for L0 prediction is decoded (S4100), and a difference vector value is decoded (S4101). Next, a prediction vector candidate list for L0 prediction is generated (S4102). When the prediction vector candidate list is larger than 1 (S4103: YES), a prediction vector index for L0 prediction is decoded (S4104). When the prediction vector candidate list is 1 (S4103: NO), 0 is set to the prediction vector index of L0 prediction (S4105). In step S4102, processing similar to that in step S3301 in the flowchart of FIG. 33 in the video encoding device is performed.

  Next, the motion vector value stored at the position indicated by the prediction vector index of the L0 prediction is acquired from the prediction vector candidate list (S4106). A motion vector is reproduced by adding the decoded difference vector value and motion vector value (S4107).

  Subsequently, a similar motion information decoding process is performed on the reference image for L1 prediction. The reference image designation information for L1 prediction is decoded (S4108), and the difference vector value is decoded (S4109). A prediction vector candidate list for L1 prediction is generated (S4110), and if the prediction vector candidate list is larger than 1 (S4111: YES), a prediction vector index for L1 prediction is decoded (S4112). When the prediction vector candidate list is 1 (S4111: NO), 0 is set to the prediction vector index of L1 prediction (S4113). In step S4113, processing similar to that in step S3304 in the flowchart of FIG. 33 in the video encoding device is performed.

  Next, the motion vector value stored at the position indicated by the prediction vector index of the L1 prediction is acquired from the prediction vector candidate list (S4114). A motion vector is reproduced by adding the decoded difference vector value and the motion vector value (S4115). As the generated motion information, reference image designation information and motion vector values for two reference images are stored as motion information in the bidirectional prediction mode (S4116) and supplied to the bidirectional motion compensated prediction decoding unit 3704.

  FIG. 42 is a flowchart for explaining the detailed operation of step S3810 of FIG. In step S3810 of FIG. 38, a combined motion corrected predicted motion information decoding process is performed.

  First, the combined motion correction prediction mode is set as the prediction mode (S4200), and a combined motion information candidate list is generated (S4201). In step S4201, the same processing as in step S1401 of the flowchart of FIG. 14 in the moving image encoding apparatus is performed.

  When the combined motion information candidate list is larger than 1 (S4202: YES), that is, when there are a plurality of combined motion information candidates, the combined motion information index is decoded (S4203), and the combined motion information candidate list is one. (S4202: NO), 0 is set to the combined motion information index (S4204).

  Subsequently, the motion information stored at the position indicated by the combined motion information index is acquired from the combined motion information candidate list (S4205). The motion information to be acquired includes a prediction direction indicating unidirectional / bidirectional, reference image designation information, and a motion vector value.

  Next, based on the acquired combined motion information index and motion information, the combined motion corrected motion compensated prediction decoding unit 3707 performs reference image determination processing for transmitting the bidirectional motion vector value and the difference vector (S4206). In step S4206, the same processing as in step S2203 in the flowchart of FIG. 22 in the moving image encoding apparatus is performed. For more detailed processing contents, the processing of the flowcharts shown in FIGS. 26, 27, and 28 is performed to determine the reference image that transmits the same bidirectional motion vector value and difference vector as the moving image encoding apparatus. Is called.

  Next, with respect to the motion correction reference image that is the reference image for transmitting the determined difference vector, the difference vector for the motion vector value before correction stored as motion information is decoded (S4207), and the motion vector before correction is obtained. The motion vector value of the motion correction reference image is calculated by adding the difference vector to the value (S4208).

  With respect to the motion information stored at the position indicated by the combined motion information index, the prediction direction is set to bidirectional, and the reference image specification that specifies the second reference image when the motion information is unidirectional prediction The information and the updated motion vector value in the motion correction reference image are stored as motion information (S4209), and output from the combined motion correction motion compensation prediction decoding unit 3707 to the motion compensation prediction unit 608 and motion information memory 607 in FIG. Is done.

  In the moving picture coding apparatus and the moving picture decoding apparatus in Embodiment 1, the motion information of more adjacent blocks is used as selection candidates for the motion compensated prediction in AVC which is the conventional moving picture coding method. Thus, the combined motion correction is a new prediction mode in which the surrounding motion information is registered as a candidate that can be used more effectively and the motion vector value correction information is encoded as a difference vector with respect to the surrounding motion information. Prediction mode was used. As a result, the motion information is highly correlated but the continuity of the motion is not sufficiently maintained, or the motion vector in the motion information of other blocks is not accurate, due to factors such as distortion when the prediction residual is encoded. For example, a motion compensated prediction signal that can correct an appropriate motion vector with a small amount of information and has a small prediction residual with a small amount of additional information with respect to motion information generated from adjacent blocks. An encoding device and a decoding device to be generated can be realized.

  In the video encoding device and video decoding device according to Embodiment 1, the setting of the reference image for correcting the motion vector is combined from the prediction mode stored in the surrounding motion information and the surrounding motion information. Based on the acquisition method when acquiring the motion information, the reliability of the motion vector in the combined motion information is evaluated, and the reference image that needs to be corrected is specified. Accordingly, there is an effect that it is possible to correct an appropriate motion vector without adding designation information to be transmitted at the time of encoding.

  In the video encoding device and video decoding device according to Embodiment 1, the reference reference image and the reference motion vector of the combined motion modified prediction mode are determined and calculated in common with the combined prediction mode. Accordingly, it is possible to provide a combined motion correction prediction mode that can encode a small amount of additional information while minimizing new processing for the combined motion correction prediction mode.

  Further, in the moving image encoding device and the moving image decoding device according to the first embodiment, it is possible to increase the combined motion correction prediction mode in which bidirectional prediction is performed with a small load and code amount, and the subsequent encoding / decoding is performed. Since it is possible to increase the motion information of adjacent blocks referred to by the block, it is possible to improve the coding efficiency.

(Embodiment 2)
Next, the second embodiment of the present invention will be described. The second embodiment differs from the first embodiment in the encoding process when encoding the combined motion modified prediction mode. FIG. 43 is a diagram showing a configuration of a motion prediction mode in the second embodiment. The motion prediction flag (inter_pred_flag) encoded as information indicating whether the prediction direction is unidirectional prediction or bidirectional prediction in the first embodiment is shown in FIG. 43 in the second embodiment. In addition, as the motion prediction mode (inter_pred_mode), encoding is performed with a value indicating the combined motion correction prediction mode (Pred_MergeMvd) being 0, a value indicating one direction (Pred_LC) being 1, and a value indicating bidirectional (Pred_BI) being 2. . For this reason, it is not necessary to transmit the merge Mvd flag in Embodiment 1, and two flags for identifying the three prediction modes are integrated into one mode information.

  When the syntax of Embodiment 2 is used, entropy coding that can be encoded with an information amount corresponding to the occurrence frequency of arithmetic coding or the like when the occurrence frequency of the prediction mode is largely biased is used. It is possible to reduce the code amount.

  FIG. 44 is a diagram illustrating the syntax of the encoded stream generated by the motion information encoded data generation process in the second embodiment. The moving picture encoding apparatus and moving picture decoding apparatus according to the second embodiment are described in the flowchart of the motion information encoded data generation process shown in FIG. 34 and the flowchart of the motion information decoding process shown in FIG. Only a part of the processing is different from the moving picture coding apparatus and the moving picture decoding apparatus in the first embodiment.

  FIG. 45 is a flowchart for explaining motion information encoded data generation processing in the moving image encoding apparatus according to the second embodiment. Only the parts different from the operation shown in the flowchart of FIG. 34 will be described below.

  In the flowchart of FIG. 45, the process of step S3407, which is the process of encoding the motion prediction flag in the flowchart shown in FIG. 34, and the process of steps S3414 and S3418, which are the processes of encoding the merge Mvd flag, are eliminated. In step S4500, which is the encoding process, a processing configuration is newly added.

  When the prediction mode is not the combined prediction mode (S3400: NO), the merge flag is encoded with 0 (S3401), the motion prediction mode is encoded (S4500), and it is determined whether or not it is the unidirectional prediction mode (S3408).

  When the prediction mode is the combined motion correction prediction mode (S3413: YES), the process of determining the size of the integrated motion information candidate list (S3415) without going through the process of encoding the merge Mvd mode (S3414). Even when the prediction mode is not the combined motion correction prediction mode (S3413: NO), the L0 reference image designation information encoding process (S3420) is not performed without the process of encoding the merge Mvd mode (S3414). The processing configuration is directly connected.

  FIG. 46 is a flowchart for explaining motion information decoding processing in the video decoding device in the second embodiment. Only the parts different from the operation shown in the flowchart of FIG. 38 will be described below.

  In the flowchart of FIG. 46, step S3805, which is the motion prediction flag decoding process in the flowchart shown in FIG. 38, step S3808, which is the merge Mvd flag decoding process, and whether the merge Mvd flag is 1 or not. The process of step S3809, which is a determination process, is eliminated, and step S4600, which is a decoding process of motion prediction mode, and step S4601, which is a process of determining whether the motion prediction mode is a combined motion correction prediction mode, are newly added. Take processing configuration.

  When the merge flag is not 1 (S3804: NO), the motion prediction mode is decoded (S4600), and it is determined whether or not the unidirectional prediction mode is set (S3806). If it is not the one-way prediction mode (S3806: NO), it is determined whether or not the motion prediction mode is the combined motion correction prediction mode (S4601). If it is the combined motion correction prediction mode (S4601: YES), the combination is determined. A motion-corrected predicted motion information decoding process is performed (S3810). If the combined motion-corrected prediction mode is not set (S4601: NO), a bidirectionally-predicted motion information decoding process is performed (S3811).

(Embodiment 3)
Next, the third embodiment of the present invention will be described. In the third embodiment, compared to the first embodiment, a motion vector value suitable for encoding motion information in the combined motion corrected motion compensated prediction mode is obtained by using a reference image for motion correction and a reference motion vector. After the value is determined, motion compensation prediction with higher prediction efficiency can be realized by detecting a motion vector for the surrounding motion.

  When the moving image encoding device and the moving image decoding device in Embodiment 3 are compared with the moving image encoding device and the moving image decoding device in Embodiment 1, the moving image decoding device is the same. A part of the configuration of the encoding device is different. Specifically, the block configuration in FIG. 13, which is a detailed configuration in the combined motion corrected motion compensated prediction generation unit 1009 in FIG. 10, and specific processing thereof will be described with respect to the video encoding apparatus in the first embodiment. The combined motion modified prediction mode evaluation value generation process shown in the flowchart of FIG. 22 has a different configuration.

  FIG. 47 is a diagram illustrating a detailed configuration in the combined motion corrected motion compensated prediction generation unit 1009 in FIG. 10 according to the third embodiment. The combined motion correction motion compensation prediction generation unit 1009 includes a reference reference image / motion correction reference image selection unit 1300, a motion correction reference image motion vector detection unit 4700, a combined motion information correction motion compensation prediction generation unit 1302, and a difference vector calculation unit 1303. including.

  The reference reference image / motion correction reference image selection unit 1300, the combined motion information correction motion compensation prediction generation unit 1302, and the difference vector calculation unit 1303 perform the same operations as those in the first embodiment shown in FIG. The function of the motion correction reference image motion vector acquisition unit 1301 is replaced with the function of the motion correction reference image motion vector detection unit 4700.

  The motion correction reference image motion vector detection unit 4700 receives the image signal of the block to be encoded and the reference image signal stored in the decoded image memory 106 from the input terminal 100, and the motion correction reference image motion vector detection unit 4700 , Using the reference reference image designation information / standard reference image motion vector value and the motion correction reference image designation information / motion correction reference image prediction vector value supplied from the standard reference image / motion correction reference image selection unit 1300 A motion vector for the corrected reference image is detected.

  Since the third embodiment has the above function, an input from the decoded image memory 106 is added to the prediction mode determination unit 109 in FIG. 1, and an input image signal is input to the combined motion corrected motion compensation prediction generation unit 1009 in FIG. And an input from the decoded image memory 106 are added.

  In the detection of a motion vector for a motion-corrected reference image, the reference image corresponding to a reference image that has been moved by the motion vector value from the same position as the image signal with the determined motion vector value for the standard reference image out of the two reference images. Get the image signal. For a motion-corrected reference image, it corresponds to a reference image that is moved by a motion amount from the same position as the image signal, with respect to a predetermined range of motion centered on a motion-corrected reference image prediction vector value set as motion information. An image signal is acquired, and an image signal (an image signal corresponding to bidirectional prediction) obtained by averaging the two image signals is generated. Then, an error evaluation value with respect to the prediction block image to be encoded is calculated, and the motion amount of the motion correction reference image that minimizes the error evaluation value is set as a motion correction reference image motion vector.

  Since the search range of the motion vector of the motion correction reference image is directly used as the range of the difference vector value to be transmitted, a motion vector suitable for encoding in the combined motion correction prediction mode may be a small search range. For example, by performing a search within a range of about ± 2 pixels horizontally and vertically, it is possible to detect a motion vector effective for the combined motion correction prediction mode. By converting the code amount of the difference vector and adding it to the error evaluation value, more accurate motion vector detection can be performed.

  The motion vector value for the motion correction reference image generated by the motion correction reference image motion vector detection unit 4700 is supplied to the combined motion information correction motion compensation prediction generation unit 1302 and the difference vector calculation unit 1303.

  FIG. 48 is a flowchart for explaining the combined motion correction / prediction mode evaluation value generation process in the third embodiment. 48 is different from FIG. 22 which is a flowchart of the combined motion correction prediction mode evaluation value generation process in the first embodiment in that step S2204 which is a process of acquiring the motion vector value of the motion correction reference image is eliminated. Further, step S4800, which is a process for correcting the motion vector based on the combined motion vector value of the motion correction reference image, is added.

  In step S4800, the motion vector is searched based on the combined motion vector value of the motion corrected reference image, and the detected corrected reference image motion vector value is stored as a corrected motion vector.

  In the third embodiment, in accordance with the characteristics of the combined motion correction prediction mode, in a minute range centered on the motion vector value of the motion correction reference image generated from the motion information serving as a reference for generating the difference vector, A prediction error is determined using a reference image, and a motion vector value used for motion compensation prediction is determined. Thereby, the optimal motion compensation prediction signal in the combined motion correction prediction mode can be used as a prediction mode candidate, and the coding efficiency can be improved.

  In the third embodiment, as another method in step S4800, which is a process for searching for a motion vector, the motion vector detection unit 107 in FIG. 1 in the first embodiment detects the motion correction reference image before step S4800. It is also possible to perform step S2204, which is a process for inputting a motion vector value, and use a method for searching for a motion vector based on the input motion vector value instead of step S4800.

  The another method described above can detect a motion vector to be corrected of a motion correction reference image that is greatly different from the motion information of the adjacent block. In this case, the combined motion correction prediction mode is the bidirectional prediction mode. On the other hand, by not transmitting the bi-directional reference image designation information and the index of the prediction vector candidate, the effect of encoding with a small amount of motion information can be exhibited.

  Another method described above is used in combination with the method of searching for a motion vector based on the combined motion vector value of the motion-corrected reference image in step S4800, and a motion vector value with a small prediction error of the detected motion vector value is obtained as a result. It is also possible to use it, and it is possible to detect an appropriate motion vector value as a motion vector to be corrected in the motion-corrected reference image for both slightly different motion and greatly different motion from the motion information of adjacent blocks. Become.

(Embodiment 4)
Next, the fourth embodiment of the present invention will be described. The fourth embodiment differs from the first embodiment in the algorithm for motion correction reference image determination. In the fourth embodiment, a method for determining the motion-corrected reference image is based on the temporal distance between the encoding target image and the reference image.

  FIGS. 49A to 49C are conceptual diagrams showing the distance priority algorithm in the fourth embodiment. As shown in FIGS. 49A to 49C, when the combined motion information is the space BiPred, the space UniPred, and the time BiPred, the derived L0 reference image and L1 reference image are all displayed. The reference image that is distant in time from the encoding target image is determined to have low motion vector reliability, and is set as a corrected motion reference image that transmits the difference vector.

  In the case of a moving image sequence with high continuity of motion, the reliability of a motion vector from a reference image closer to the encoding target image becomes higher, so by correcting the motion vector value for a far reference image, A suitable motion compensated prediction signal can be generated.

  In the fourth embodiment, the reference image determination process for transmitting the bidirectional motion vector value and the difference vector shown in FIG. 26, which is applied to each of the moving image encoding device and the moving image decoding device in the first embodiment. These flowcharts are different processes.

  FIG. 50 is a flowchart for describing reference image determination processing for transmitting bidirectional motion vector values and difference vectors in the fourth embodiment. The flowchart in FIG. 50 is one-way compared to the flowchart in FIG. 26 in Embodiment 1 and step S2602, which is a process for determining whether or not the scaled motion vector is one-way (one). In this case (S2602: YES), step S2603, which is a process of setting a reference image having a scaled motion vector as a corrected reference image, is eliminated.

(Embodiment 5)
Next, the fifth embodiment of the present invention will be described. In the fifth embodiment, in contrast to the first embodiment, all reference images for which motion correction is performed in the combined motion correction motion compensated prediction mode are set for motion vectors generated using scaling, and two scaling motions are set. In the case of having a vector, a configuration is adopted in which two difference vectors are encoded.

  In the fifth embodiment, the motion vector generated by the scaling process based on the determination criterion that the accuracy of the motion vector generated by the scaling process is lower than the motion vector when the motion information of the spatially adjacent block is directly applied. The difference vector is transmitted to all the reference images having.

  51 (a) to 51 (c) specify reference images for transmitting a difference vector when combined motion information is space BiPred, space UniPred, and time BiPred in the fifth embodiment. A conceptual diagram of processing composition is shown. In the fifth embodiment, similarly to the first embodiment, a scaled motion vector is generated as motion information according to the type of combined motion information as shown in FIG.

  When the combined motion information is space BiPred as shown in FIG. 51 (a), the number of scaling vectors is 0. Therefore, as in the first embodiment, the reference image for transmitting the difference vector is derived from the encoded image. A reference image with a long time distance is selected.

  When the combined motion information is space UniPred as shown in FIG. 51 (b), since the number of scaling vectors is 1, one reference image having a scaled motion vector is obtained as in the first embodiment. , Specified as a reference image for transmitting the difference vector.

  When the combined motion information is time BiPred as shown in FIG. 51 (c), since the number of scaling vectors is 2, unlike the first embodiment, two reference images having scaled motion vectors are Designated as a reference image for transmitting a difference vector.

  FIG. 52 is a diagram illustrating a relationship of motion information to be transmitted in each prediction mode in the fifth embodiment. In the fifth embodiment, in the combined motion correction prediction mode, in the case of spatial UniPred and spatial BiPred, motion information is encoded by transmitting one combined motion information index and one difference vector. . In the case of time BiPred, motion information is encoded by transmitting one combined motion information index and two difference vectors.

  The moving picture encoding apparatus and moving picture decoding apparatus in Embodiment 5 are compared with the moving picture encoding apparatus and moving picture decoding apparatus in Embodiment 1, and the bidirectional motion vector values and difference vectors shown in FIG. FIG. 34 shows a flowchart of reference image determination processing to be transmitted, a flowchart of motion information encoded data generation processing by the moving image encoding device shown in FIG. The processing of the flowchart is partially different.

  FIG. 53 is a flowchart for describing reference image determination processing for transmitting bidirectional motion vector values and difference vectors in the fifth embodiment. The reference image determination process for transmitting the bidirectional motion vector value and the difference vector is a process for determining whether or not the number of scaled motion vectors is 1 with respect to the flowchart of FIG. 26 in the first embodiment. And a new Scaled_Merge_ref_num, which is a parameter for managing the number of scaled motion vectors, is provided to make a condition determination.

  A process of setting the number of scaled motion vectors among the motion vector values of the bi-directional reference image to Scaled_Merge_ref_num is performed (S5300). When Scaled_Merge_ref_num is 1 (S5301: YES), a reference image having a scaled motion vector is set as a motion-corrected reference image (S2603). When Scaled_Merge_ref_num is 2 (S5302: YES), both reference images are set as motion corrected reference images (S5303). When Scaled_Merge_ref_num is 0 (S5302: NO), the process proceeds to step S2604 for setting a reference image according to the distance, as in the first embodiment.

  FIG. 54 is a diagram illustrating an example of an encoding syntax in the fifth embodiment. Based on the Scaled_Merge_ref_num generated in the motion-correction prediction motion information decoding process, it is configured to switch between transmitting two (four elements in total, horizontal and vertical) difference vectors or transmitting one (two elements in total). Yes.

  FIG. 55 is a flowchart for explaining motion information encoded data generation processing in the video encoding device in the fifth embodiment. In the motion information encoded data generation process, when the combined motion information candidate list is larger than 1 in the flowchart of FIG. 34 in the first embodiment (S3415: YES), the combined motion compensation index is encoded (S3416). I do.

  When Scaled_Merge_ref_num is 2 (S5500: YES), the L0 difference vector value is encoded (S5501), and the L1 difference vector value is further encoded (S5502).

  FIG. 56 is a flowchart for explaining the combined motion corrected predicted motion information decoding process in the video decoding device in the fifth embodiment. The combined motion corrected predicted motion information decoding process is the Scaled_Merge_ref_num after the reference image determination process (S4206) for transmitting the bidirectional motion vector value and the difference vector shown in FIG. 53 with respect to the flowchart of FIG. 42 in the first embodiment. Is 2 (S5600: YES), the L0 difference vector value is decoded (S5601) and added to the motion vector value calculated from the motion information of the L0 reference image to calculate the motion vector value of the L0 reference image. (S5602). Further, the L1 difference vector value is decoded (S5603) and added to the motion vector value calculated from the motion information of the L1 reference image to calculate the motion vector value of the L1 reference image (S5604). It becomes composition.

  Further, as another configuration in the fifth embodiment, a configuration in which a difference vector is always transmitted for two reference images can be employed. FIGS. 57A and 57B are conceptual diagrams of the state of the difference vector between the space BiPred and the space UniPred in the configuration in which two reference images are always transmitted in the fifth embodiment. FIG. 57 (a) shows an example of transmitting two difference vectors in the space BiPred, and FIG. 57 (b) shows an example of transmitting two difference vectors in the space UniPred.

  FIG. 58 is a diagram illustrating an example of an encoding syntax that can be employed when a configuration is employed in which a difference vector is always transmitted for two reference images. When merge_mvd_flag which is the combined motion correction prediction mode is 1, two difference vectors (a total of four elements) are always encoded.

(Embodiment 6)
Next, the sixth embodiment of the present invention will be described. The sixth embodiment is different from the first embodiment in that the combined motion information candidate list used in the combined prediction mode and the combined motion information candidate list used in the combined motion modified prediction mode are generated by different algorithms.

  In Embodiment 1, in the combined motion correction prediction mode, when the difference vector value is (0, 0) and the spatial BiPRed and the temporal BiPred, the same motion compensated prediction image as that in the combined prediction mode is generated. The In the sixth embodiment, overlapping motion information is separated at the time of generating a combined motion information candidate list, or motion information is applied from adjacent blocks at different positions, so that the combined prediction mode and the combined motion correction prediction mode are the same. There is an advantage that duplication of the motion information transmission function is suppressed and more motion information can be selected.

  The difference in configuration between the moving image encoding device and the moving image decoding device in the sixth embodiment and the moving image encoding device and the moving image decoding device in the first embodiment is the same as that in the moving image encoding device shown in FIG. The configuration of the prediction mode determination unit 109 and the configuration of the motion information decoding unit 606 in the video decoding device shown in FIG.

  FIG. 59 is a diagram illustrating a configuration of the prediction mode determination unit 109 in the video encoding device according to the sixth embodiment. In the configuration of FIG. 59, the data flow from the combined motion information calculation unit 1007 input to the combined motion correction motion compensation prediction generation unit 1009 in Embodiment 1 is deleted, and the combined motion correction motion information calculation unit 5900 is added. Is done.

  The combined motion correction motion information calculation unit 5900 uses a candidate block group in the motion information of adjacent blocks supplied from the motion information memory 111 in FIG. 1, a prediction direction indicating whether it is unidirectional or bidirectional, reference A plurality of pieces of motion information are generated together with the combined motion corrected motion information candidate list as motion information composed of image designation information and motion vector values, and supplied to the combined motion corrected motion compensation prediction generation unit 1009.

  The combined motion corrected motion information calculation unit 5900 can generate different motion information and motion information candidate lists by acquiring motion information candidates different from the combined motion information calculation unit 1007. As an example, the spatial direction prediction block shown in FIG. 16, the method of changing the number of acquisition candidates from the temporal direction prediction block shown in FIG. 18, a block E, a block B 4, a block A 4, etc. at different positions A method of adding to a candidate of a candidate block group, a method of changing the order of addition of a candidate block in the spatial direction and a candidate block in the temporal direction to the list, a method of adding to the list by limiting to specific motion information, etc. It is done.

  FIG. 60 is a diagram illustrating a configuration of the motion information decoding unit 606 in the video decoding device according to the sixth embodiment. In the configuration of FIG. 60, the data flow from the combined motion information calculation unit 3705 that has been input to the combined motion correction motion compensated prediction decoding unit 3707 in Embodiment 1 is deleted, and the combined motion correction motion information calculation unit 6000 is added. Is done.

  The combined motion corrected motion information calculation unit 6000 performs the same processing as the combined motion corrected motion information calculation unit 5900 of the video encoding device in FIG. 59, and the combined motion corrected motion information calculation unit 6000 stores the motion information memory in FIG. The candidate block group in the motion information of the adjacent block supplied from 607 is used as a plurality of motion information composed of a prediction direction indicating whether it is unidirectional or bidirectional, reference image designation information, and motion vector values. Are generated together with the combined motion corrected motion information candidate list and supplied to the combined motion corrected motion compensated prediction decoding unit 3707.

  FIG. 61 is a diagram illustrating an example of the coding syntax in the sixth embodiment. As information different from merge_idx that is index information in the combined prediction mode, a structure is obtained in which merge_mvd_idx that is index information in the combined motion correction prediction mode is acquired and managed.

  As another configuration of the sixth embodiment, a configuration in which applicable motion information candidates are different in the Skip mode and the non-Skip mode in the combined prediction mode is also conceivable. FIG. 62 is a diagram illustrating an example in which the combination information applied in each prediction mode in the sixth embodiment is different. As shown in FIG. 62, only candidate motion information of the spatial BiPred is applied in the Skip mode, and candidate motion information of the spatial UniPred and the spatial BiPred is applied in the combined prediction mode (Merge) of the non-Skip mode. In the modified prediction mode (MergeMvd), candidate motion information of the space UniPred and time BiPred is applied.

  When changing candidates of motion information applicable in the Skip mode and the non-Skip mode, the candidate index in the Skip mode in FIG. 61 is different from the combined prediction mode, so merge_idx is replaced with skip_idx.

  In the combined motion modified prediction mode, the time BiPred for generating scaled motion information that is more effective by the difference vector transmission and the space UniPred for generating new motion information by scaling are enabled. By enabling the space UniPred and the space BiPred in which motion information with high continuity is easily generated, and in the Skip mode, by enabling only the space BiPred that can generate a predicted image with higher quality when the motion continuity is high. Thus, it is possible to adopt a configuration capable of generating an effective motion compensation prediction signal while suppressing duplication of functions of the combined prediction mode and the combined motion correction prediction mode and reducing the amount of index information depending on the number of candidates.

(Embodiment 7)
Next, the seventh embodiment of the present invention will be described. The seventh embodiment adopts a configuration in which one difference vector value transmitted in the combined motion modified prediction mode is applied to two reference images, as compared with the first embodiment.

  63 (a) and 63 (b) are conceptual diagrams showing the effect of the difference vector on the two reference images in the seventh embodiment. As shown in FIGS. 63A and 63B, one difference vector is transmitted, and the difference vector value is caused to act as a correction vector value for both reference images for bidirectional prediction.

  In the first embodiment, the difference vector value is the difference between the motion vector value generated from the combined motion information and the motion vector value used for motion compensated prediction with respect to one reference image selected as the motion corrected reference image. Was showing. The difference vector in the first embodiment has a shift in a part of the motion vector in the combined motion information, but there is a correct motion vector (which can reflect the motion based on the combined motion information as it is). This is correction information in a reliable state.

  The difference vector in the seventh embodiment is correction information for the entire combined motion information in a state in which the temporal correlation (temporal continuity) of the motion information is kept high, and is particularly a time when the combined motion information is time BiPred. In correction of motion information generated based on the correlation, effective correction information is transmitted with a small amount of information, and coding efficiency can be improved.

  The effect of the difference vector in the seventh embodiment is that, in the reference reference image for sending the difference vector, the motion vector value generated from the combined motion information and the motion used for the motion compensation prediction are the same as the difference vector in the first embodiment. Indicates the difference between vector values. In the other reference image of bi-directional prediction that is not a standard reference image for sending a difference vector, two methods can be used for the operation.

  One is that, as shown in FIG. 63 (a), the value obtained by scaling the difference vector is the difference between the motion vector value generated from the combined motion information in the other reference image and the motion vector value used for motion compensation prediction. This is a (mirror type) technique. The other is, as shown in FIG. 63 (b), without performing scaling, the motion vector value and the motion compensated prediction in which the difference vector value is generated from the combined motion information in the other reference image, similarly to the standard reference image. This is a method of indicating the difference between the motion vector values used in (Translation type).

  In the following description of the seventh embodiment, the mirror type difference vector operation method is used, but the mirror type and the translation type method are adaptively switched depending on the characteristics of the moving image, the prediction structure at the time of encoding, and the like. It is also possible.

  The difference in configuration between the video encoding device in the seventh embodiment and the video encoding device in the first configuration is a detailed configuration in the combined motion corrected motion compensation prediction generation unit 1009 in FIG. It is a combined motion correction prediction mode evaluation value generation process shown in FIG. 22 which is a flowchart for explaining the configuration and its specific process. The difference in configuration between the moving picture decoding apparatus in the seventh embodiment and the moving picture decoding apparatus in the first embodiment is the processing flowchart of the combined motion corrected predicted motion information decoding process shown in FIG.

  The block configuration in FIG. 13 in the seventh embodiment has the same configuration as that in FIG. 47 which is a detailed configuration in the combined motion corrected motion compensation prediction generation unit 1009 in FIG. 10 in the third embodiment. FIG. 64 is a diagram illustrating an example of a motion vector search method based on a combined motion vector value of two reference images in the seventh embodiment.

  The motion correction reference image motion vector detection unit 4700 performs the combined motion as a difference vector for the motion correction reference image as it is as a difference vector and as a scaled difference vector for the reference reference image with respect to a detection range as a difference vector candidate. An image signal obtained by acting on a motion vector value for each reference image generated from information is generated by averaging, and an error evaluation value with a predicted block image to be encoded is calculated, The motion amount of the motion correction reference image that minimizes the error evaluation value is set as a motion correction reference image motion vector.

  In the description of the seventh embodiment, the difference vector value of the motion-corrected reference image is set as the difference vector value to be transmitted in order to obtain the common processing with the other embodiments. A method of setting the difference vector value to the difference vector value to be transmitted can be taken, and the present invention is not limited to this. That is, in the configuration for transmitting the mirror type difference vector according to the seventh embodiment, since the difference motion vector may be transmitted based on which reference image, the code amount of the difference vector is usually reduced. A reference image closer to the target image is set as a motion correction reference image, and a differential motion vector for the motion correction reference image is transmitted. On the other hand (the distance from the encoding target image is the same or far away from the encoding target image), a scaling process with an absolute value of the magnification of 1 or more is performed on the transmitted differential motion vector, and the same or higher than the transmitted differential motion vector. Get the magnitude difference motion vector.

  FIG. 65 is a flowchart for explaining the combined motion corrected prediction mode evaluation value generation process in the seventh embodiment. Compared to FIG. 22 which is the flowchart of the combined motion correction prediction mode evaluation value generation process in the first embodiment, step S2204 which is a process for acquiring the motion vector value of the motion correction reference image is eliminated, and instead of two reference images. Step S6500, which is a process for correcting the motion vector based on the combined motion vector value, is added.

  In step S6500, based on the combined motion vector values of the standard reference image and the motion-corrected reference image, the motion amount scaled with respect to the corrected motion amount of the motion-corrected reference image becomes the corrected motion amount of the standard reference image. In this manner, the motion vector is searched, and the motion vector value obtained by adding the corrected motion amount to each reference image is stored.

  FIG. 66 is a flowchart for explaining the combined motion corrected predicted motion information decoding process in the seventh embodiment. Compared to FIG. 42, which is the flowchart of the combined motion correction predicted motion information decoding process in the first embodiment, step S4208, which is a process for calculating the motion vector value of the motion correction reference image, is eliminated. Step S6600, which is a process for calculating the motion vector value of the reference image (standard reference image), is added.

  The motion vector value is calculated by adding the decoded difference vector value to the motion vector value generated from the motion information for the motion corrected reference image and adding the scaled value of the difference vector value to the motion vector value for the reference reference image. Thus, the motion vector values of the two reference images are calculated.

  FIG. 67 is a conceptual diagram showing another configuration in which the action of the difference vector in the seventh embodiment is applied to both reference images. A method of generating a motion vector value of the other reference image by scaling the decoded motion vector value by adding the difference vector to the motion vector of the combined motion information of the target reference image is also conceivable. A prediction signal substantially equivalent to the motion compensation prediction signal can be generated.

  This method can be realized by integrating the combined motion information candidate generation process and the combined motion modified prediction motion information decoding process when the combined motion information is time BiPred. In this method, since the scaling process is performed only once after adding the differential motion vector to the motion vector of the combined motion information, the deterioration due to the calculation accuracy of the scaling hardly occurs, and a more accurate motion compensated prediction signal is generated. Can be generated.

  In addition, it is possible to adopt a configuration in which the embodiment 7 and other embodiments are switched according to the characteristics of the combined motion information as an extended configuration, which is adopted under the condition where the effectiveness of the embodiment 7 such as the time BiPred is high, By applying the first embodiment under other conditions, it is possible to transmit motion information more efficiently. For example, the high effectiveness condition of the seventh embodiment is a case where the distance between the encoding target image and the standard reference image is the same as the distance between the encoding target image and the motion-corrected reference image. In this case, since the difference vector for the other reference image can be calculated without scaling the difference vector to be transmitted (the absolute value of the scaling factor is 1), deterioration due to the calculation accuracy of scaling does not occur.

(Embodiment 8)
Next, an eighth embodiment of the present invention will be described. In the eighth embodiment, compared to the first embodiment, the combined prediction mode and the combined motion correction prediction mode are integrated, and the difference vector value in the combined motion correction prediction mode is encoded on the syntax structure for encoding the combined prediction mode. Take the configuration to transmit.

  In the combined motion correction prediction mode in the first embodiment, when the difference vector value is (0, 0) and the spatial BiPRed and the temporal BiPred, the same motion compensated prediction image as that in the combined prediction mode is generated. Redundancy (duplication of functions) is reduced by integrating the combined prediction mode and the combined motion modified prediction mode, and when using the combined motion information, the difference vector value is always transmitted in a mode other than the Skip mode described later, and One motion compensation prediction mode and its encoding syntax can be reduced in weight.

  FIG. 68 is a diagram showing the definition of the motion compensation prediction mode in the eighth embodiment. The motion compensated prediction mode is composed of two modes, a motion detection prediction mode and a combined motion modified prediction mode (merged MVD mode), and two prediction methods of unidirectional prediction and bidirectional prediction are defined as motion detection prediction modes. .

  In the first embodiment, the Skip mode defined as the combined prediction mode in which the prediction difference information is not encoded and transmitted is the combined motion modified prediction mode in which the prediction difference information and the difference vector information are not encoded and transmitted in the eighth embodiment. In the Skip mode, only the combined motion information index is transmitted as in the first embodiment.

  The difference in configuration between the video encoding device in the eighth embodiment and the video encoding device in the first configuration is that the configuration of the prediction mode determination unit 109 shown in FIG. 10 and the motion compensation shown in FIG. This is a prediction mode / prediction signal generation process and a motion information encoded data generation process shown in FIG. Also, the difference in configuration between the video decoding device in the eighth embodiment and the video decoding device in the first configuration is that the configuration of the motion information decoding unit 606 shown in FIG. 37 and the motion information shown in FIG. It is a flowchart of a decoding process and the joint motion correction prediction motion information decoding process shown by FIG.

  FIG. 69 is a diagram illustrating a configuration of the prediction mode determination unit 109 in the video encoding device according to the eighth embodiment. The configuration of the prediction mode determination unit 109 in the eighth embodiment is a configuration in which the combined motion compensation prediction generation unit 1008 and its operation are eliminated from the configuration in the first embodiment shown in FIG.

  FIG. 70 is a flowchart for explaining motion compensated prediction mode / prediction signal generation processing in the video encoding device in the eighth embodiment. The motion compensation prediction mode / prediction signal generation process in the eighth embodiment has a configuration in which step S1401 which is a combined prediction mode evaluation value generation process is eliminated from the process in the first embodiment shown in the flowchart of FIG.

  FIG. 71 is a flowchart for explaining motion information encoded data generation processing in the video encoding device in the eighth embodiment. In the motion information encoded data generation process in the eighth embodiment, the processes in step S3400, step S3401, step S3413, step S3414, and step S3418 are eliminated from the process in the first embodiment shown in the flowchart of FIG. Step S7100 for determining whether or not the prediction mode is the combined motion correction prediction mode and step S7101 for determining whether or not the prediction error signal is encoded and the difference vector is 0 are added. take.

  First, when the prediction mode is the combined motion corrected prediction mode (S7100: YES), when the prediction error signal is not encoded and the difference vector is 0 (S7101: YES), steps S3402, S3404, and The skip condition motion information encoding process configured in step S3405 is performed. When the prediction error signal is encoded or the difference vector is 0 (S7101: NO), the merge flag is encoded as 1 (S3403), and step S3415, which is the motion information encoding process in the combined motion correction prediction mode, step S3415 Processing of S3416 and step S3417 is performed. Here, it should be noted that the merge flag of the eighth embodiment is used to determine whether or not it is the combined motion correction prediction mode, and does not indicate whether or not it is the combined prediction mode as in the first embodiment.

  When the prediction mode is not the combined motion correction prediction mode (S7100: NO), the merge flag is encoded as 0 (S3406), and the motion prediction flag is encoded (S3407). When it is the unidirectional prediction mode (S3408: YES), Steps S3409, S3410, S3411, and S3412 which are motion information encoding processes in the unidirectional prediction mode are performed, and when it is not the unidirectional prediction mode (S3408: NO), the motion information encoding process in the bidirectional prediction mode from step S3419 to step S3426 is performed.

  FIG. 72 is a diagram illustrating an example of an encoding syntax related to motion information in the eighth embodiment. In the example shown in FIG. 72, in the case of the combined motion correction prediction mode specified by merge_flag and the combined motion correction prediction not in the Skip mode, a difference vector is transmitted.

  FIG. 73 is a diagram illustrating a configuration of the motion information decoding unit 606 in the video decoding device according to the eighth embodiment. The configuration of the motion information decoding unit 606 in the eighth embodiment is a configuration in which the combined motion compensated prediction decoding unit 3706 and its operation are eliminated from the configuration in the first embodiment shown in FIG.

  FIG. 74 is a flowchart for explaining motion information decoding processing in the moving picture decoding apparatus according to the eighth embodiment. The motion information decoding process in the eighth embodiment is different from the process in the first embodiment shown in FIG. 38 in step S3802 that is a joint prediction motion information decoding process, step S3808 that is a decoding process of the merge Mvd flag, Step S3809 for determining whether or not the Mvd flag is 1 is eliminated, and in the first embodiment, the step is a combined motion modified prediction motion information decoding process that has been performed when the merge Mvd flag is 1. When S3810 is the Skip mode (S3801: YES) and when the merge flag is 1 (S3804: YES), the processing is performed.

  FIG. 75 is a flowchart for explaining detailed operation of step S3810 of FIG. In step S3810 of FIG. 74, a combined motion corrected predicted motion information decoding process is performed.

  The combined motion modified prediction motion information decoding process in the moving picture decoding apparatus in the eighth embodiment is different from the combined prediction motion information decoding process in the first embodiment shown in FIG. After step S4205 which is a reference image determination process to be transmitted, step S7500 which is a process for determining whether or not the mode is the skip mode and step S7501 which is a process for setting the difference vector value to 0 are inserted.

  In the Skip mode (S7500: YES), the difference vector is not transmitted and motion vector correction is not performed. Therefore, 0 is set as the difference vector value (S7501), and step S4207, which is the difference vector decoding process, is not performed. Then, the process proceeds to step S4208, which is a process for calculating a motion vector value of the motion correction reference image.

  76 (a) to 76 (c) are conceptual diagrams showing a difference vector transmission method in a combined motion modified prediction mode of another configuration according to the eighth embodiment. As a second processing structure in the eighth embodiment, when the combined motion information is generated in the space UniPred, it is avoided that the condition for using the motion information as the one-way prediction is lost in the combined prediction mode in the first embodiment. However, as shown in FIG. 76 (b), in the case of the space UniPred, it is also possible to take a method of correcting the motion vector by transmitting the difference vector while maintaining the unidirectional prediction. While taking advantage of the joint prediction mode in the first embodiment, it is possible to adopt a motion compensated prediction structure that incorporates the advantage of the joint motion correction prediction mode. In the space UniPred in this case, the standard reference image is not set, and the reference image specified by the unidirectional prediction is specified as the motion-corrected reference image.

  As the third processing structure in the eighth embodiment, when the combined motion information is generated in the space UniPred, in the second reference image index determination process shown in FIG. In the case of existing in the two reference image candidate list, it is also possible to adopt a method of determining the reference image index for the same reference image as the standard reference image as the second reference image index and calculating motion information for bidirectional prediction. is there.

  When the same reference image as the standard reference image is set as the second reference image, and the difference vector value is (0, 0), the same motion compensated prediction as when the spatial UniPred motion information in the combined prediction mode is used. Since the signal can be generated, it is possible to avoid losing the condition for using the motion information as the one-way prediction as it is in the combined prediction mode in the first embodiment, similarly to the second processing structure.

  When the same reference image as the standard reference image is set as the second reference image, if a value other than (0, 0) is transmitted as the difference vector value, the motion represented by a slightly different motion of the same reference image By averaging the compensation prediction signal, a motion compensation prediction signal in which a minute change is added to the reference image can be added as a new candidate, and coding efficiency can be improved.

  Each configuration of the eighth embodiment can be applied in a configuration in which the respective configurations are combined. For example, in the case where the third processing structure is the basis, when the same reference image as the standard reference image cannot be set as the second reference image (when the same reference image as the standard reference image does not exist in the second reference image candidate list) ), A processing configuration in which another reference image is selected as the second reference image to generate motion information for bidirectional prediction can be applied.

  Further, in the case where the third processing structure is basically used, when the same reference image as the standard reference image cannot be set as the second reference image, the motion information is obtained as unidirectional prediction as shown in the second processing structure. A processing configuration that generates and encodes a difference vector with respect to a motion vector of a unidirectional prediction reference image is also applicable. In this case, as in the second processing structure, an effect that all the conditions for generating the same motion compensated prediction signal as in the combined prediction mode can be encoded as the combined motion modified prediction mode is added.

  These processing structures can be switched in a predetermined unit according to the encoding prediction structure and the characteristics of the image sequence, and motion compensation processing using motion information suitable for the characteristics of the image to be encoded is performed. realizable.

(Embodiment 9)
Next, the ninth embodiment of the present invention will be described. In the ninth embodiment, in contrast to the first embodiment, when the difference vector is (0, 0) in the combined motion modified prediction mode that can be expressed in the combined prediction mode, the difference motion is added to the motion vector value to be decoded. By defining differently from the case where the vector is other than (0, 0), a configuration is adopted in which new motion information candidates can be transmitted while reducing redundancy of motion information to be encoded.

  In the ninth embodiment, when the difference vector value is 0 in the combined motion modified prediction mode at the time of decoding, a (0, 0) vector that is a fixed value is decoded as the motion vector value. The motion vector value itself to be decoded may be a value that can be implicitly decoded. A fixed value other than (0, 0) is set, or a predetermined value is set for a motion vector value calculated as combined motion information. (The value having a certain amount of difference vector code amount), or a configuration for acquiring the motion vector value for the above condition using another motion information of the adjacent block is also a configuration for realizing the effect in the ninth embodiment. Is possible.

  The difference in configuration between the moving picture coding apparatus in the ninth embodiment and the moving picture coding apparatus in the first embodiment is the combined motion correction prediction mode evaluation value generation process shown in FIG. Also, the difference in configuration between the moving picture decoding apparatus according to the ninth embodiment and the moving picture decoding apparatus according to the first embodiment is the combined motion corrected predicted motion information decoding process shown in FIG.

  FIG. 77 is a flowchart for explaining combined motion corrected prediction mode evaluation value generation processing in the video encoding device in the ninth embodiment. In the combined motion correction prediction mode evaluation value generation process in the ninth embodiment, after step S2204, which is a process for acquiring the motion vector of the motion correction reference image, in contrast to the process in the first embodiment shown in FIG. Step S7700, which is a process for determining whether or not the motion vector of the motion-corrected reference image is (0, 0), step S7701, which is a process for setting (0, 0) as the difference vector, and the motion-corrected reference image Step S7702 which is a process for replacing a motion vector with a fixed value is added, and step S7703 which is a process for determining whether or not the difference vector is (0, 0) is added after step S2205 which is a difference vector value generation process. I take the.

  When the motion vector value for the motion-corrected image acquired from the motion vector detecting unit 107 in FIG. 1 is (0, 0) (S7700: YES), in the combined motion-corrected prediction mode in the ninth embodiment, the difference vector is ( Since the motion information can be encoded with a small code amount by setting to (0, 0), the difference vector value is set to (0, 0) (S7701), and the motion vector for the motion corrected image set from the combined motion information A fixed value (0, 0) is set as the value (S7702), and the subsequent bidirectional motion compensation prediction signal generation processing is performed.

  When the motion vector value for the motion corrected image is the same as the motion vector value for the motion corrected image set from the combined motion information, the difference vector value is (0, 0). In the case of (0, 0) (S7703: YES), a fixed value (0, 0) is set as the motion vector value for the motion-corrected image set from the combined motion information (S7702), and the subsequent bidirectional motion. By performing a compensated prediction signal generation process and evaluating a prediction error value, it is possible to perform mode determination using motion compensated prediction using a (0, 0) vector as a selection candidate.

  In the ninth embodiment, in the case of adopting a configuration that enables the search for motion vectors in the combined motion correction prediction mode evaluation value generation as in the moving image encoding apparatus of the third embodiment, the combined motion correction is performed. Prediction mode evaluation value generation processing can be performed. FIG. 78 is a flowchart for explaining combined motion modified prediction mode evaluation value generation processing of another configuration in the ninth embodiment. In contrast to the processing in the first embodiment shown in FIG. 22, step S2204, which is a process for acquiring the motion vector of the motion-corrected reference image, is eliminated, and instead, the difference vector based on the combined motion vector value of the motion-corrected reference image. Step S 7800, which is a process for detecting a motion vector except for the case of (0, 0), Step S 7801 for calculating a prediction error when the motion vector value of the motion corrected reference image is set to a fixed value (0, 0), correction Step S7802, which is a process of comparing the prediction error with the motion vector value using the fixed motion vector value and the prediction error at the fixed value (0, 0), and the combined motion vector value of the motion-corrected reference image is replaced with a fixed value. Is added to step S7803, which is a process of setting (0,0) to (0,0).

  In step S7800, since the condition for using the motion vector value of the modified reference image calculated from the combined motion information as it is is excluded, the motion vector search is performed with the difference vector (0, 0) at which the motion vector correction is zero. By not doing so, it is prevented from being registered as a motion vector value candidate.

  Thereafter, a prediction error evaluation value in a state where the motion vector value of the motion correction reference image is designated as a fixed value is calculated (S7801), and compared with the prediction error evaluation value when the motion vector value detected in step S7800 is used. If the prediction error in the fixed value is small (S7802: YES), the combined motion vector value of the motion corrected reference image is replaced with a fixed value, the difference vector value is set to 0 (S7803), and the motion information code amount The process proceeds to step S2206 which is a calculation process.

  By performing the above processing, it is possible to generate motion information that enables the best motion compensation prediction among the motion information that the combined motion correction prediction mode can take in the ninth embodiment, and to improve the encoding efficiency.

  FIG. 79 is a flowchart for explaining joint motion corrected predicted motion information decoding processing in the video decoding device in the ninth embodiment. The combined motion modified prediction motion information decoding process in the ninth embodiment is different from the process in the first embodiment shown in FIG. 42 in that a difference vector value is ( 0,0) and step 7901, which is a process for replacing the motion vector value of the motion-corrected reference image with a fixed value.

  In the moving picture decoding apparatus according to the ninth embodiment, when the difference vector is 0 as described above (S7900: YES), the motion correction reference is used to set the decoded motion vector value to a fixed value (0, 0). The motion vector value of the image is replaced with a fixed value (0, 0) (S7901). After step S7901, the motion vector value of the motion-corrected reference image is calculated (S4208), so that the fixed value becomes a motion value as a decoded value obtained by adding the difference vector (0, 0) to the fixed value (0, 0). Calculated as the motion vector value of the corrected reference image.

  In the ninth embodiment, fixed value replacement processing is performed for all combined motion information candidates in the combined motion correction prediction mode when the difference vector value is (0, 0). In the case of the space UniPred, The motion information in the combined prediction mode and the motion information in the combined motion correction prediction mode are different. In other words, in the case of the space UniPred, there is no overlap in function between the motion information in the joint prediction mode and the joint motion correction prediction mode, so it is necessary to replace the motion vector with a fixed value under the condition of the difference vector (0, 0). Rather, it is possible to use a method of switching whether or not to perform the replacement process according to the type of combined motion information, and it is possible to express more suitable motion information.

  In the embodiment described above, the description has been made on the assumption that a picture (B picture) in which bi-directional prediction can be used, but the embodiment of the present invention is also applied to a picture (P picture) that does not have adaptive bi-directional prediction. It is possible to apply. That is, when the embodiment of the present invention is applied to a P picture, motion compensation is always performed in unidirectional prediction in the combined motion-corrected prediction mode, and the same reference image as that specified in the combined prediction mode is used. On the other hand, the difference vector is added, and the motion vector is corrected.

  Thereby, compared with the case where the motion detection prediction mode is selected, the same prediction motion vector as that in the combined prediction mode is used, and the reference image designation information required in the motion detection prediction mode is unnecessary. For this reason, the combined motion correction prediction mode becomes a prediction mode having the property between the integrated prediction mode and the motion detection prediction mode, and the temporal and spatial correlation with the adjacent motion information is high, but a slight shift occurs in the motion. Encoding efficiency can be improved, for example, when the motion information of adjacent blocks does not indicate accurate motion.

  The moving image encoded stream output from the moving image encoding apparatus of the embodiment described above has a specific data format so that it can be decoded according to the encoding method used in the embodiment. Therefore, the moving picture decoding apparatus corresponding to the moving picture encoding apparatus can decode the encoded stream of this specific data format.

  When a wired or wireless network is used to exchange an encoded stream between a moving image encoding device and a moving image decoding device, the encoded stream is converted into a data format suitable for the transmission form of the communication path. It may be transmitted. In that case, a video transmission apparatus that converts the encoded stream output from the video encoding apparatus into encoded data in a data format suitable for the transmission form of the communication channel and transmits the encoded data to the network, and receives the encoded data from the network Then, a moving image receiving apparatus that restores the encoded stream and supplies the encoded stream to the moving image decoding apparatus is provided.

  The moving image transmitting apparatus is a memory that buffers the encoded stream output from the moving image encoding apparatus, a packet processing unit that packetizes the encoded stream, and transmission that transmits the packetized encoded data via the network. Part. The moving image receiving apparatus generates a coded stream by packetizing the received data, a receiving unit that receives the packetized coded data via a network, a memory that buffers the received coded data, and packet processing. And a packet processing unit provided to the video decoding device.

  Further, the above-described processing relating to encoding and decoding can be realized as a transmission, storage, and reception device using hardware, and is stored in a ROM (Read Only Memory), a flash memory, or the like. It can also be realized by firmware or software such as a computer. The firmware program and software program can be recorded on a computer-readable recording medium, provided from a server through a wired or wireless network, or provided as a data broadcast of terrestrial or satellite digital broadcasting Is also possible.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the description of the flowchart of FIG. 21 in the first embodiment, the example in which the Truncated Unary code string is used as the code string of the combined motion information index has been described. Here, a specific example of the Truncated Unary code string is given.

  FIG. 80 is a diagram illustrating a Truncated Unary code string when the number of combined motion information candidates is five. When the value of the combined motion information index is encoded using the Truncated Unary code string, the smaller the number of combined motion information candidates, the smaller the code bits assigned to the combined motion information index. For example, when the combined motion information index is 1, if the number of combined motion information candidates is 2, it is represented by 1 bit of “1”, but if the number of combined motion information candidates is 3, it is expressed as “10”. Expressed with 2 bits. Here, as described above, the Truncated Unary code string is used for encoding the combined motion information index, but other code string generation methods can be used, and the present invention is not limited to this.

  100 input terminals, 101 subtraction unit, 102 orthogonal transform / quantization unit, 103 prediction error coding unit, 104 inverse quantization / inverse transform unit, 105 addition unit, 106 decoded image memory, 107 motion vector detection unit, 108 motion compensation Prediction unit, 109 prediction mode determination unit, 110 motion information encoding unit, 111 motion information memory, 112 multiplexing unit, 113 output terminal, 600 input terminal, 601 demultiplexing unit, 602 prediction difference information decoding unit, 603 inverse quantization Inverse transformation unit, 604 addition unit, 605 decoded image memory, 606 motion information decoding unit, 607 motion information memory, 608 motion compensation prediction unit, 609 output terminal, 1000 unidirectional motion compensation prediction generation unit, 1001 bidirectional motion compensation prediction Generator, 1002 prediction error calculator, 100 Prediction vector calculation unit, 1004 difference vector calculation unit, 1005 motion information code amount calculation unit, 1006 prediction mode evaluation unit, 1007 joint motion information calculation unit, 1008 joint motion compensation prediction generation unit, 1009 joint motion correction motion compensation prediction generation unit, 1100 Prediction vector candidate list generation unit 1101 Prediction vector candidate list deletion unit 1200 Combined motion information candidate list generation unit 1201 Combined motion information candidate list deletion unit 1300 Reference reference image / motion correction reference image selection unit 1301 Motion correction reference Image motion vector acquisition unit, 1302 combined motion information modified motion compensation prediction generation unit, 1303 difference vector calculation unit, 3700 motion information bitstream decoding unit, 3701 prediction vector calculation unit, 3702 vector addition unit, 370 Unidirectional motion compensation prediction decoding unit, 3704 bidirectional motion compensation prediction decoding unit, 3705 joint motion information calculation unit, 3706 joint motion compensation prediction decoding unit, 3707 joint motion correction motion compensation prediction decoding unit, 4700 motion correction reference image motion vector detection Part, 5900, 6000 combined motion correction motion information calculation unit.

Claims (3)

An image encoding device that detects and encodes a motion vector in units of blocks obtained by dividing each picture of a moving image,
As the motion information of the encoding target block, when the prediction direction in the case of bidirectional prediction, and for modifying one of the motion vectors included in the motion information of the reference blocks adjacent to the encoding target block, before Symbol participation index irradiation block, and the encoding target blocks of the motion vector and the reference the one motion vector mode using difference vector as a coding target and selectable prediction mode determination unit included in the motion information of the block When,
A prediction error signal encoding unit that encodes a prediction error signal that is a difference between a prediction signal generated based on motion information supplied from the prediction mode determination unit and an image signal of the encoding target block;
A motion information encoding unit that encodes the motion information supplied from the prediction mode determination unit;
A multiplexing unit that multiplexes the encoded data supplied from the prediction error signal encoding unit and the encoded data supplied from the motion information encoding unit;
The one of the motion vectors included in the motion information of the reference block to be subjected to the differential vector is determined from motion information of the reference block candidates, characterized in that it is a motion vector determined by the prediction direction Image encoding device. An image encoding method for detecting and encoding a motion vector in units of blocks obtained by dividing each picture of a moving image,
As the motion information of the encoding target block, when the prediction direction in the case of bidirectional prediction, and for modifying one of the motion vectors included in the motion information of the reference blocks adjacent to the encoding target block, before Symbol selecting a mode to use the index of references blocks, and a difference vector between the one of the motion vector included in the coding target block motion vector and the motion information of the reference block as an encoding target,
Encoding a prediction error signal that is a difference between a prediction signal generated based on the motion information and an image signal of the encoding target block;
Encoding the motion information;
Multiplexing the encoded data of the prediction error signal and the encoded data of the motion information,
The one of the motion vectors included in the motion information of the reference block to be subjected to the differential vector is determined from motion information of the reference block candidates, characterized in that it is a motion vector determined by the prediction direction Image coding method. An image encoding program for detecting and encoding a motion vector in units of blocks obtained by dividing each picture of a moving image,
As the motion information of the encoding target block, when the prediction direction in the case of bidirectional prediction, and for modifying one of the motion vectors included in the motion information of the reference blocks adjacent to the encoding target block, before Symbol a process of selecting the mode to be used an index of references blocks, and a difference vector between the one of the motion vector included in the coding target block motion vector and the motion information of the reference block as an encoding target,
A process of encoding a prediction error signal that is a difference between a prediction signal generated based on the motion information and an image signal of the encoding target block;
A process of encoding the motion information;
Causing the computer to execute a process of multiplexing the encoded data of the prediction error signal and the encoded data of the motion information;
The one of the motion vectors included in the motion information of the reference block to be subjected to the differential vector is determined from motion information of the reference block candidates, characterized in that it is a motion vector determined by the prediction direction Image encoding program.

Images