JP2004274734A - Picture decoding apparatus, picture encoding apparatus, and method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for regulating the management of a picture memory when a picture size is revised. <P>SOLUTION: A picture decoding apparatus 1100 decodes a code sequence including moving picture streams with different picture sizes while referencing decoding finished pictures. The picture memory 203 is provided with a storage area to store a picture whose decoding is finished. A picture memory division method designation section 210 divides the storage area of the picture memory 203 into picture areas of prescribed sizes. A picture memory control section 206 consecutively stores all data of one picture which is newly decoded to the consecutive storage area comprising one picture area of the picture memory 203 or more. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to a moving picture decoding method and a coding method for performing inter-picture prediction with reference to a coded or decoded picture, and more particularly to a picture memory management method for storing decoded pictures. About.

  In recent years, the multimedia era, in which voices, images, and other pixel values are integrated, has been approached, and the traditional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, and telephones, to humans, has been developed. It has been taken up as an object. Generally, multimedia means not only characters, but also figures, sounds, and especially images, etc., that are simultaneously associated with each other. Is an essential condition.

  However, when the amount of information of each of the above information media is estimated as a digital information amount, the amount of information per character is 1 to 2 bytes in the case of characters, whereas 64 kbits per second in the case of voice (telephone quality). In addition, for a moving image, an information amount of 100 Mbits per second (current television reception quality) or more is required, and it is not realistic to handle the huge amount of information in the above-mentioned information medium in digital form. For example, a videophone has already been put into practical use by an integrated services digital network (ISDN) having a transmission speed of 64 kbps to 1.5 Mbps, but it is impossible to directly transmit a video of a video camera via the ISDN. It is possible.

  Therefore, information compression technology is required. For example, in the case of a videophone, H.264 standardized internationally by ITU (International Telecommunication Union Telecommunication Standardization Sector). 261 and H.E. H.263 video compression technology is used. In addition, according to the information compression technology of the MPEG-1 standard, it is possible to store image information together with audio information in a normal music CD (compact disc).

  Here, MPEG (Moving Picture Experts Group) is an international standard for digital compression of moving picture plane signals. MPEG-1 converts moving picture plane signals up to 1.5 Mbps, that is, information of television signals by about 1/100. It is a standard that compresses up to. In addition, since the transmission rate for the MPEG-1 standard is mainly limited to about 1.5 Mbps, the moving picture signal is 2 to 2 in the MPEG-2 standardized to meet the demand for higher image quality. It is compressed to 15Mbps.

  Furthermore, at present, MPEG-4, which has a higher compression ratio, has been standardized by a working group (ISO / IEC JTC1 / SC29 / WG11) that has been standardizing MPEG-1 and MPEG-2. MPEG-4 initially introduced a powerful error resilience technique that not only enables highly efficient encoding at a low bit rate, but also reduces subjective image quality degradation even if a transmission path error occurs. . Also, at present, H.264 / ISO / IEC and ITU have jointly developed H.264 as a next-generation screen coding system. H.264 standardization activities are underway.

  Generally, in coding of a moving image, the amount of information is compressed by reducing redundancy in the time direction and the space direction. Therefore, in the inter-picture prediction coding for the purpose of reducing temporal redundancy, motion detection and prediction image creation are performed in block units with reference to the forward or backward picture, and the obtained predicted image and the current Encoding is performed on the difference value from the picture.

  Here, the picture is a term representing one screen, and means a frame in a progressive image and a frame or a field in an interlaced image. Here, an interlaced image is an image in which one frame is composed of two fields at different times. In encoding and decoding of an interlaced image, one frame may be processed as a frame, processed as two fields, or processed as a frame structure or a field structure for each block in the frame. it can.

  Note that the picture described below is described in terms of a frame in a progressive image, but the description can be similarly applied to a frame or a field in an interlaced image.

  FIG. 28 is a block diagram illustrating a configuration of a conventional image encoding device 100. The image encoding apparatus 100 includes a picture memory 101, a prediction residual encoding unit 102, a code sequence generation unit 103, a prediction residual decoding unit 104, a picture memory 105, a motion vector detection unit 106, and a motion compensation encoding unit 107. , A motion vector storage unit 108, a picture memory control unit 109, a difference operation unit 112, an addition operation unit 113, a switch 114, and a switch 115.

  A moving image to be encoded is input to the picture memory 101 in units of pictures in the order in which display is performed, and the pictures are rearranged in the picture memory 101 in the order of encoding. Further, each picture is divided into blocks of, for example, horizontal 16 × vertical 16 pixels called macroblocks, and the subsequent processing is performed in block units.

  An input image signal read from the picture memory 101 is input to a difference calculation unit 112, and a difference image signal obtained by taking a difference from a prediction image signal output from the motion compensation encoding unit 107 is converted to a prediction residual code. Output to the conversion unit 102. The prediction residual encoding unit 102 performs image encoding processing such as frequency conversion and quantization, and outputs a residual signal. The residual signal is input to the prediction residual decoding unit 104, and performs image decoding processing such as inverse quantization and inverse frequency transform to output a residual decoded signal. The addition operation unit 113 generates a reconstructed image signal by adding the residual decoded signal and the predicted image signal, and the obtained reconstructed image signal is stored in the picture memory 105.

  On the other hand, the input image signal in macroblock units read from the picture memory 101 is also input to the motion vector detection unit 106. Here, one or a plurality of encoded pictures stored in the picture memory 105 are searched, and a motion vector indicating the position by detecting an image area closest to the input image signal is selected. And determine a reference picture index pointing to the obtained picture. The detection of the motion vector is performed for each block obtained by further dividing the macroblock, and the obtained motion vector is stored in the motion vector storage unit 108. The motion compensation coding unit 107 extracts an optimal image area from the coded pictures stored in the picture memory 105 and generates a predicted image using the motion vector and the reference picture index detected by the above processing.

The coded information such as the motion vector, the reference picture index, and the residual coded signal output by the above-described series of processing is subjected to variable-length coding in the code sequence generation unit 103, so that output is performed by this coding processing. Is obtained.
The flow of the processing described above is the operation in the case where the inter-picture prediction coding is performed. When performing intra-frame encoding, a predicted image is not generated by motion compensation, and a differential image signal is generated by generating a predicted image of an encoding target region from an encoded region in the same screen and taking a difference. . The difference image signal is converted into a residual coded signal in a prediction residual coder 102 and subjected to variable-length coding in a code sequence generator 103, as in the case of inter-picture prediction coding, and is output. Is obtained.

  FIG. 29 is a block diagram showing a configuration of a conventional image decoding device 200. The image decoding apparatus 200 includes a code string analysis unit 201, a prediction residual decoding unit 202, a picture memory 203, a motion compensation decoding unit 204, a motion vector storage unit 205, a picture memory control unit 206, an addition operation unit 211, It comprises a switch 212.

  First, the code sequence analysis unit 201 extracts various information such as a motion vector, a reference picture index, and a residual coded signal from the input code sequence. The motion vector extracted by the code sequence analyzer 201 is output to the motion vector storage unit 205, the reference picture index is output to the motion compensation decoding unit 204, and the residual coded signal is output to the prediction residual decoding unit 202.

The prediction residual decoding unit 202 performs image decoding processing such as inverse quantization and inverse frequency transform on the input residual encoded signal, and outputs a residual decoded signal. The addition operation unit 211 generates a reconstructed image signal by adding the residual decoded signal and the predicted image signal output from the motion compensation encoding unit 204, and stores the obtained reconstructed image signal in the picture memory 203. Store.
The motion compensation decoding unit 204 uses the motion vector input from the motion vector storage unit 205 and the reference picture index input from the code sequence analysis unit 201 to store one or more frames stored in the picture memory 203. An optimal image area for the predicted image is extracted from the decoded picture.

  The decoded picture generated by the above series of processing is output from the picture memory 203 as a display image signal in accordance with the display timing.

  The above processing flow is the operation in the case where the inter-picture prediction decoding is performed. However, the switch 212 is switched to the intra-picture prediction decoding. When performing intra prediction decoding, a predicted image is not generated by motion compensation. Instead, a predicted image of the decoding target area is generated from the decoded area in the same screen, and the reconstructed image signal is generated by performing addition with the residual decoded signal. As a result, the obtained reconstructed image signal is stored in the picture memory 203 and output as a display image signal in accordance with the display timing.

  30 (a) and 30 (b) are diagrams for explaining the reference relation of pictures in an input image. FIG. 30 (a) arranges pictures in display order, and FIG. 30 (b) arranges pictures in encoding order. It shows the appearance that it was. The number attached to the P of each picture indicates the display order of the picture. This number is assigned to each picture such that all the values are in ascending order when the pictures are arranged in the display order as shown in FIG. Arrows in the figure indicate a reference relationship between pictures in the inter prediction coding or inter prediction decoding, and the picture at the head of the arrow is one or two of the pictures at the tip of the arrow. The sheets can be arbitrarily referred.

  Taking P4 as an example, it can be seen that P4 performs encoding or decoding with reference to P0, P1, P2, and P6. The plurality of pictures used for the reference need to be coded or decoded before the target picture P4, and are stored in the picture memory as referenceable pictures. If the pictures are arranged in the encoding order as shown in FIG. 30B, it can be seen that all pictures referenced by P4 are located in front of P4. In the process of encoding and decoding, the picture P4 selects one or two optimal pictures from a plurality of pictures used for the reference for each block, and uses the selected picture as a reference picture. It is used for inter-picture prediction coding and inter-picture prediction decoding described below.

  Each picture stored in the picture memory is provided with information indicating whether the picture can be used for reference or not. Pictures that are not used for reference should be deleted from the picture memory immediately, but other pictures will be decoded until they are displayed, even if they are known not to be used for reference. You need to wait. For example, P3 in FIG. 30B has the second decoding order, whereas the display order as shown in FIG. 30A is the fourth. Therefore, P3 must be stored in the picture memory until P1 and P2 are decoded. The picture such as P3 is called a display waiting picture or a non-referenceable picture. On the other hand, a picture that can be used for reference is called a referenceable picture.

  In general, the image signal reconstructed in the addition operation unit 113 and the addition operation unit 211 of the image encoding device 100 and the image decoding device 200 is controlled by the picture memory control unit 109 and the picture memory control unit 206 as follows. Below, they are stored in the picture memory 109 and the picture memory 203. FIG. 31 is a flowchart showing the operation of controlling the storage of the reconstructed image signal in the picture memory 203 in the image decoding apparatus 200 shown in FIG. When the current picture is decoded (S401) and a new reconstructed image signal is generated, the picture memory control unit 206 allocates a sufficient free area in the picture memory 203 to store the generated reconstructed image signal. It is determined whether or not there is (S402). If there is not enough free space to store the generated reconstructed image signal, the picture memory control unit 206 deletes the oldest picture stored in the picture memory 203 in display order (S403). In this way, the picture memory control unit 206 repeats the steps S402 and S403, so that the picture memory control unit 206 has already stored the picture data until there is enough free space in the picture memory 203 to store a new reconstructed image signal. Pictures that are displayed are deleted from the oldest one in the display order. In this way, when there is enough free space in the picture memory 203 to store a new reconstructed image signal, the new reconstructed image signal is stored in the free space (S404). The free area means an area in which the corresponding area in the memory can be reused by another data. Also, “delete” represents a process that enables the memory area to be used for storing another data. When data is physically deleted, or data is overwritten without physically deleting the data. It is also possible to handle the case where only the permission is granted (see Non-Patent Document 1).

FIGS. 32 (a), 32 (b), 33 (a), 33 (b), 34 (a), and 34 (b) show that the picture memory is managed according to the procedure shown in FIG. FIG. 14 is a diagram showing an example of using a memory area when the memory area is used.
FIG. 32A shows a storage example in which the size of a picture to be decoded has been changed from small to large, and there is a free area in the picture memory 203 for storing the picture after the size change. FIG. The upper part of FIG. 32A shows a state where there is a sufficient free area (shaded area) for storing the decoded picture C in the picture memory 203. In this example, P2, P6, P4, and P5 in the picture sequence in FIG. 30 are stored in the picture memory, and P7 (= picture C) is being decoded. In this case, as shown in the lower part of FIG. 32A, the decoded picture C can be stored in the picture memory 203 from the beginning of the free area in the picture memory 203 as it is. That is, the picture already stored in the picture memory is not deleted.

  FIG. 32B shows a storage example in a case where the size of a picture to be decoded has been changed from small to large, and there is no free area in the picture memory 203 for storing the picture after the size change. FIG. In the figure, a picture indicated by a circle with a numeral attached to a picture is a picture to be deleted from the picture memory 203 in order to newly store a picture C. The upper part of FIG. 32B shows a state where there is not enough free space to store the decoded picture C in the picture memory 203. In this example, P3, P1, P2, P6, P4, and P5 in the picture sequence in FIG. 30 are stored in the picture memory, and P7 is currently being decoded (that is, C = P7). In this case, it is necessary to delete the picture before the picture size change stored in the picture memory 203 and secure an area for storing the newly decoded picture C. In this example, the picture C after the picture size change is larger in size than the picture before the picture size change, and in order to newly store the picture C, it is necessary to delete two pictures already stored. It turns out that there is. Next, the pictures to be deleted are selected in order from the oldest display order information possessed by each picture. In this example, it can be seen that the display order information of P1 is the oldest in the picture memory 203 and becomes newer in the order of P1, P2, P3,. The picture memory control unit 206 determines one or more pictures to be deleted by the above-described processing, outputs the determination result to the picture memory 203, and deletes the pictures. In the example shown in the upper part of FIG. 32B, P1 and P2 are deleted by the above-described processing, and the decoded picture C is accumulated as shown in the lower part of FIG. .

  FIG. 33A shows a case where the picture size of the current picture to be decoded is not changed and no picture having a picture size different from the current picture to be decoded exists in the picture memory. FIG. 14 is a diagram illustrating a storage example in a case where there is a free area for storing C in the picture memory 203; That is, this is an example that can occur when the picture size is not changed near the time of the picture C and the stream in which the picture size does not change is being decoded.

  In such a case, both the picture already stored in the picture memory 203 and the newly decoded picture C have the same picture size. Therefore, in order to store the newly decoded picture C, it is only necessary to delete one picture from the picture memory 203. Accordingly, the picture memory control unit 206 checks the free area in the picture memory 203, and if there is a free area for storing the picture C, writes the picture C into the free area, and the free area for storing the picture C If not, it is determined that one picture is deleted. The pictures stored in the picture memory 203 have information indicating the display order in addition to the picture data. The picture memory control unit 206 determines the picture to be deleted by examining the display order information, and notifies the picture memory 203 that the oldest picture in the display order information is to be deleted only as a determination result. As a result, the decoded picture C is stored in the generated free area.

  The upper part of FIG. 33A shows a state where there is enough free space in the picture memory 203 to store the newly decoded picture C. In this example, P2, P6, P4, and P5 in the picture sequence in FIG. 30 are stored in the picture memory, and P7 is currently being decoded (ie, C = P7). In this case, as shown in the lower part of FIG. 33A, the decoded picture C can be stored in the free area as it is. That is, the pictures already stored in the picture memory are not deleted.

  FIG. 33B shows a case where the picture size of the current picture to be decoded is not changed and no picture having a picture size different from the current picture to be decoded exists in the picture memory. FIG. 11 is a diagram illustrating a storage example in a case where there is no free area in the picture memory 203 for storing C. The upper part of FIG. 33 (b) shows a state where there is not enough free space for storing the newly decoded picture C in the picture memory. In this example, P3, P1, P2, P6, P4, and P5 in the picture sequence in FIG. 30 are stored in the picture memory, and P7 is currently being decoded (that is, C = P7). In this case, it is necessary to delete only one picture stored in the picture memory 203 to secure an area for storing the decoded picture C. As the picture to be deleted, the picture with the oldest display order information held by each picture is selected. In this example, it can be seen that the display order information of P1 is the oldest. The picture memory control unit 206 determines the picture to be deleted by the above-described processing, outputs the determination result to the picture memory 203, and deletes the picture. In the example in the upper part of FIG. 33B, P1 is deleted by the above-described processing, and the decoded picture C is accumulated in the empty area generated as shown in the lower part of FIG.

  FIG. 34A shows a case in which the picture size of the current picture to be decoded is not changed and a picture having a picture size different from the newly decoded picture C exists in the picture memory 203. FIG. 14 is a diagram illustrating a storage example in a case where there is a free area in the picture memory 203 to store a decoded picture C; That is, this is an example that can occur when a decoding target picture is decoded before a long time elapses after the picture size is changed. The picture memory control unit 206 checks the free area in the picture memory 203, does nothing if there is enough free area to store the newly decoded picture C, and if there is not enough free area, The size of the picture C newly decoded after the picture size change is compared with the size before the change, and the number of pictures to be deleted and the pictures to be deleted are determined. The picture memory control unit 206 deletes the required number of pictures in order from the oldest picture in the display order information, and stores the newly decoded picture C in the empty area created by the deletion. The picture to be deleted is specified by checking the order information. The picture to be deleted is transmitted from the picture memory control unit 206 to the picture memory 203. As a result, the newly decoded picture C is stored in the generated free area.

  The upper part of FIG. 34A shows a state in which there is enough free space in the picture memory 203 to store the newly decoded picture C. In this example, P4, P5, and P7 in the picture sequence of FIG. 30 are stored in the picture memory 203, and P8 is currently being decoded (that is, C = P8). In this case, as shown in the lower part of FIG. 34A, it is possible to accumulate the newly decoded picture C in the free area as it is. That is, the pictures already stored in the picture memory are not deleted.

FIG. 34B shows a case where the picture size of the current picture to be decoded is not changed and a picture having a picture size different from that of the newly decoded picture C exists in the picture memory 203. FIG. 11 is a diagram showing a storage example in a case where there is no free area in the picture memory 203 to store the decoded picture C in the picture memory 203. The upper part of FIG. 34B shows a state in which there is not enough free space in the picture memory 203 to store the newly decoded picture C. In this example, P2, P6, P4, P5, and P7 in the picture sequence of FIG. 30 are stored in the picture memory, and P8 is currently being decoded (that is, C = P8). In this case, it is necessary to delete the picture before the picture size change stored in the picture memory 203 and secure an area for storing the newly decoded picture C. In this example, the picture C after the picture size change is larger in size than the picture before the picture size change, and even if an area that is already an empty area for storing the picture C is used, two more pictures are used. It turns out that the picture needs to be deleted. Next, the pictures to be deleted are selected in order from the oldest display order information possessed by each picture. In this example, it can be seen that the display order information of P2 is the oldest, and becomes newer in the order of P2, P4, P5,. Accordingly, the picture memory control unit 206 determines one or a plurality of pictures to be deleted by the above processing, outputs the determination result to the picture memory 203, and deletes the picture. In the example in the upper part of FIG. 34B, P2 and P4 are deleted by the above processing, and the newly decoded picture C is accumulated in the generated free area as shown in the lower part of FIG.
"Joint Final Committee Draft (JFCD) of Joint Video Specification (ITU-T Rec. H.264 | ISO / IEC 14496-10 AVC)" 7.4.3.3 P.65, E.2.1 P.252.

  However, according to the decoding method described in the related art, since P2 and P4 to be deleted are stored in physically separated areas in the picture memory 203, they occur in the example shown in the lower part of FIG. As described above, a situation occurs in which a newly decoded picture is accumulated in a physically divided area. In particular, depending on the combination of the picture sizes of the pictures included in the coded stream, there is a possibility that one picture is extremely complicatedly divided into different areas. As described above, when a picture stored in a physically divided area is used as a reference picture, access to the picture memory becomes complicated, and the access speed to the picture memory and, consequently, the decoding speed are increased. May cause a decrease in

  In order to solve the above problem, the present invention proposes a picture memory management method that does not reduce the access efficiency to the picture memory when the picture size may change at the beginning or in the middle of a moving image stream. Aim.

  An image decoding apparatus according to the present invention is an image decoding apparatus that decodes a code string including moving image streams of different picture sizes with reference to decoded pictures, and stores decoded pictures. Memory, storage means for partitioning the storage area of the picture memory into partitioned areas of a predetermined size, and all data for one newly decoded picture stored in the picture memory. Storage means for continuously storing data in a continuous storage area composed of one or more of the divided areas.

  The image decoding apparatus may further determine whether the size of the picture has been changed by comparing the size of the newly decoded picture with the size of the previously decoded picture. A change determining unit, wherein, when it is determined that the size of the newly decoded picture has been changed, the size of the partition area is changed to the size of the newly decoded picture. The storage area may be partitioned such that

Further, the image decoding apparatus further includes reading means for reading, from the code string, partition information indicating a partitioning method of the storage area, wherein the memory partitioning means stores the storage area in accordance with the read partition information. It may be divided.
In addition, the present invention distributes encoded moving image content from a distribution server including such a moving image encoding device, and a distributed moving image is transmitted to a user side terminal device including a moving image decoding device. The present invention can be realized not only as a moving image content distribution system that reproduces image contents, but also as a single unit such as a distribution server and a user side terminal device that constitute these moving image content distribution systems, and a distribution server and a user. It can be realized as an encoding device and a decoding device provided in a side terminal device or the like, or can be realized as an encoding method and a decoding method in which the characteristic means provided in such a moving image content distribution system are steps. May be realized as a program for causing a computer to execute the steps. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

  This makes it possible to provide an image decoding apparatus that does not reduce the access efficiency to the picture memory when the picture size may change at the beginning or in the middle of the moving image stream.

  This makes it possible to realize a picture memory management method that does not reduce the access efficiency to the picture memory, even when the picture size of the moving image stream is likely to change in the code string.

  As a result, it is possible to realize a picture memory management method that does not significantly reduce the processing load on the image decoding device and does not decrease the access efficiency to the picture memory.

Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 27.
(Embodiment 1)
Conventionally, the management of a picture memory (storage and deletion of a picture in a memory area) in decoding has been performed on a picture basis. In contrast, the present embodiment is characterized in that the memory area of the picture memory in decoding is divided into one or a plurality of areas, and the divided area is used as one unit to manage the picture memory. . In the present embodiment, information for designating a method of dividing a picture memory is generated in an image encoding device and described in a code string. The divided area at this time is called a picture area.

  A moving picture coding method according to Embodiment 1 of the present invention will be described with reference to the block diagram shown in FIG. FIG. 1 is a block diagram illustrating a configuration of an image encoding device 800 according to the first embodiment. The configuration of the image encoding device 800 is almost the same as that of the conventional image encoding device 100 described with reference to FIG. The difference from the conventional image encoding device 100 is that a picture memory division information encoding unit 801 is newly added. The description of the same processing as that of the conventional technique is omitted here.

  Prior to encoding a stream to be encoded, the picture memory division information encoding unit 801 encodes information for designating a method of dividing the decoding-side picture memory. FIG. 2 is a diagram illustrating a data structure of a code string generated by the image coding device 800 according to the present embodiment. FIG. 2A is a diagram illustrating an example of a code string when encoding information of a maximum picture size that may be included in a stream. FIG. 2B shows a code for encoding the number of areas that can be secured in the picture memory when the picture memory is divided based on the maximum picture size that may be included in the stream. It is a figure showing an example of a column. In the present embodiment, as information specifying a method of dividing the picture memory on the decoding side, information of a maximum picture size that may be included in a stream is encoded, and a code string in FIG. Described as max_size of the picture common information area. As the maximum picture size here, for example, the maximum picture size that the image encoding device 800 can use for encoding is used. Alternatively, when the user performs encoding, a desired maximum picture size may be selected and determined from a plurality of presented picture sizes.

  If the picture to be coded is an IDR picture or a picture that allows a change in picture size in place of the IDR picture, a signal indicating a picture memory reset method is input from the picture memory control unit 109 to the code string generation unit 103. Note that an IDR picture refers to a picture having the following special properties. (1) Only intra prediction decoding is possible. (2) After decoding is completed, all pictures in the picture memory are reset. (3) The picture size can be changed. Therefore, a picture decoded after the IDR picture cannot refer to a picture decoded before the IDR picture. For example, P8 shown in FIG. 30A can refer only to P7. The IDR picture has a reset_flag in the header area of the slice data as shown in FIG. 2A, and the flag specifies a method of resetting the picture memory, which is one of the features of the IDR described above. Note that a slice indicates one divided region when one picture is divided into one or a plurality of regions. If reset_flag is 0, all the referable pictures in the picture memory are changed to non-referenceable pictures (pictures waiting for display). On the other hand, if reset_flag is 1, data of all pictures is deleted from the picture memory regardless of whether reference is possible or not. In other words, if reset_flag is 1, even if there is a picture that has been decoded earlier than the IDR picture and has not been displayed yet, the display of pictures after the IDR picture is performed without displaying those pictures. Will be In the present embodiment, the method of resetting the picture memory specified at this time is arbitrary. For example, when the operation of the picture memory on the decoding side is assumed, whether or not there is a picture not displayed in the picture memory Therefore, reset_flag is set to 0, that is, the picture data itself is not deleted just by making all pictures inaccessible. It is possible to arbitrarily select either one. The code sequence generation unit 103 encodes the input signal and generates a code sequence having a reset_flag in a slice header as shown in FIG.

As max_size, instead of describing information on the maximum picture size that may be included in the stream, the number of areas that can be secured in the picture memory for decoding as shown in FIG. It is also possible to describe min_area_num in the picture common information area of the code string. min_area_num indicates the number of picture areas that can be formed when the decoding-side picture memory is divided such that the size of one picture area becomes max_size, which is the maximum picture size that may be included in the stream. . In that case, the value of min_area_num is determined by Equation 1 below. The calculation result of the division in the expression is calculated as an integer by truncation processing.
(Min_area_num) = (size of picture memory on decoding side) / (max_size) (Equation 1)
It should be noted that the above-described picture size and picture memory size can be treated in the same manner regardless of the total number of pixels, the total number of macroblocks, or the total number of bits.

  It should be noted that the number of pictures that can be referred to when performing inter-screen predictive coding is set to be equal to or less than the min_area_num calculated by Expression 1 above, regardless of the picture size of the picture to be coded. This makes it possible to prevent a situation in which the reference picture does not exist in the picture memory even when the decoding side uses the picture memory by the above-described division method.

  FIG. 3 is a diagram showing an example of a code string in a case where additional information is described in a picture common information area in the code string shown in FIGS. 2A and 2B. As shown in FIG. 3, in addition to max_size and min_area_num, cnf_flag can be described in the picture common information area of the code string. The cnf_flag is a flag indicating whether all pictures included in the stream can be displayed without any problem even when the picture memory on the decoding side is divided and managed according to max_size and min_area_num. For example, a value of 1 indicates that display can be performed without any problem, and a value of 0 indicates that a picture that has not been displayed has already been lost due to the processing of this embodiment. Indicates that it may not be possible. Thus, by describing cnf_flag in the code string, it is possible to determine whether or not to apply the processing of the present embodiment by checking the value of this signal. For example, when cnf_flag is 0, it is known that there is a possibility that a picture that cannot be displayed may occur when the processing of the present embodiment is applied. By switching to, the memory area may be fragmented, but all pictures can be displayed without any problem.

  In FIGS. 2 and 3, max_size, min_area_num, and cnf_flag are described in the common picture information area. It can also be described in the data area. Further, it is also possible to describe in a sequence common information area referred to from the entire sequence. Also, it is possible to describe in the slice header.

  Next, a decoding method of a code string coded by the above-described coding method will be described with reference to FIGS. FIG. 4 is a block diagram illustrating a configuration of the image decoding apparatus 1100 according to the present embodiment. FIG. 5 is a flowchart illustrating a procedure of a decoding process in the image decoding device 1100 illustrated in FIG. FIG. 6 and FIG. 7 are schematic diagrams showing a management method of the picture memory 203 shown in FIG.

The configuration of the image decoding device 1100 shown in FIG. 4 is almost the same as that of the conventional image decoding device 200 described with reference to FIG. The difference from the conventional method is that a picture size change determination unit 207, a deleted picture area determination unit 209, and a picture memory division method designation unit 210 are added. The description of the same processing as that of the conventional technique is omitted here.
In the present embodiment, unlike the related art, management of the picture memory 203 is not performed in units of pictures, but an area called a picture area obtained by dividing the picture memory 203 into one or a plurality is performed as one unit of storage and deletion. It is characterized by the following. Therefore, prior to decoding the code string, the picture memory division method designating section 210 analyzes the picture memory dividing method from the signal in the code string and instructs the picture memory control section 206. In the present embodiment, as information for specifying the method of dividing the picture memory, information on the maximum picture size that may be included in the stream is defined as max_size included in the code string in FIG. It is described in the common information area. The picture memory division method designating section 210 determines the division method of the picture memory 203 by analyzing the signal. At this time, the number of divided regions is calculated by the following equation (2). The calculation result of the division in the expression is calculated as an integer by truncation processing.
(Number of picture areas) = (Size of picture memory 203 of image decoding apparatus 1100) / (max_size) (Equation 2)
When the code sequence of FIG. 2B is used instead of the code sequence of FIG. 2A, the number of picture regions is calculated by the following Expression 3.
(Number of picture areas) = (min_area_num) (Equation 3)
Note that the picture size and the size of the picture memory may be any of the total number of pixels, the total number of macroblocks, or the total number of bits, and can be handled similarly.

The picture size change determination unit 207 compares the picture size of the picture to be decoded with the picture size of the picture decoded before (or immediately before) and determines whether the picture size has been changed.
As shown in FIGS. 2A and 2B, reset_flag is encoded in the slice header in the code string. The code string analysis unit 201 analyzes the value of the reset_flag, and outputs information of the analysis result to the picture memory control unit 206. In the present embodiment, even if the picture size has been changed, the value of reset_flag may be both 0 or 1. Accordingly, the picture memory management method is divided according to the value of reset_flag as shown in the flowchart of FIG. In particular, when reset_flag is 0, there is a possibility that the picture before the picture size change remains in the picture memory. Therefore, which picture area must be deleted to accumulate the picture after the picture size change. Must be determined. The determination is made by the deleted picture area determination unit 209, and the determination result is output to the picture memory control unit 206 to manage the picture memory.

  Next, the details of the operations of the picture memory control unit 206, the picture size change determination unit 207, and the deleted picture area determination unit 209 in the decoding process will be described with reference to the flowchart in FIG. In the flowchart of FIG. 5, after the decoding of the target picture is performed, the determination of the change in the picture size and the determination of the value of the reset_flag are performed. However, the order is arbitrary, and is limited to the order shown in FIG. Not something.

  First, the flow of the decoding process executed when the picture size of the decoding target picture is changed (S1204) and reset_flag is 1 (S1205) will be described. A reset_flag of 1 is a value indicating that all picture data in the picture memory is deleted and all areas are left empty. That is, when this signal is input to the picture memory control unit 206, the picture memory control unit 206 deletes all the areas in the picture memory (S1206) and replaces the decoded picture with the changed picture size in the picture memory. (S1207).

  Next, the flow of the decoding process executed when the picture size of the decoding target picture is changed (S1204) and reset_flag is 0 (S1205) will be described. The reset_flag of 0 is a value indicating that all the picture data in the picture memory is not to be referred to, but the picture data itself is not deleted but is stored. The deleted picture area determination unit 209 checks a free area in the picture memory, and determines a picture area to be deleted when the free area is insufficient (S1208). The pictures stored in the picture memory 203 have display order information in addition to the picture data. By checking the display order information, the deletion picture area determination unit 209 specifies a picture area in which the oldest picture of the display order information is stored, and issues an instruction to delete the specified picture area to the picture memory control unit. The information is transmitted to the unit 206. As a result, the designated picture area is deleted, and the decoded picture is stored in the generated free area (S1210).

FIGS. 6A and 6B are diagrams showing a state in which a newly decoded picture is stored in the picture memory 203 according to the procedures of steps S1204 to S1210 in the flowchart of FIG.
FIG. 6A shows an example in which there is a sufficient free area in the picture memory 203 to store the decoded picture C to be decoded. In this example, P4 and P5 in the picture sequence of FIGS. 30A and 30B are stored in the picture memory, and P7 is currently being decoded (that is, C = P7). P4 and P5 are pictures having the same picture size as the divided picture area, that is, the pictures having the largest picture size among the pictures included in the sequence. On the other hand, the decoded picture C always has the same or smaller picture size as the picture area, but in the example of FIG. 6A, the pictures C to be decoded are P4 and P5. This shows a case where the picture size is smaller than the picture size. In this case, as shown in the lower part of FIG. 6A, the decoded picture C can be stored in the free area as it is. That is, the picture area in which the picture data is already stored is not deleted.

  The upper part of FIG. 6B shows an example in which there is not enough free space to store the decoded picture C in the picture memory. In this example, P6, P4, and P5 in the picture sequence of FIG. 30 are stored in the picture memory, and P7 is currently being decoded (that is, C = P7). In this case, it is necessary to delete the picture area having the picture data before the picture size change stored in the picture memory 203 to secure an area for storing the decoded picture C. In the present embodiment, the picture to be decoded always has the same picture size as the picture area or a smaller picture size. Therefore, when there is no free area in the picture memory 203, an area for storing decoded pictures can be secured by deleting one of the picture areas. As the picture area to be deleted, the one with the oldest display order information held by the stored pictures is selected. In this example, it can be seen that the display order information of P4 is the oldest. The deleted picture area determination unit 209 determines the picture area to be deleted by the above-described processing, outputs the determination result to the picture memory control unit 206, and deletes the picture area. In the example shown in the upper part of FIG. 6B, P4 is deleted by the above processing, and the picture C decoded as shown in the lower part of FIG. 6B is stored in a free area generated by deleting P4. .

  Finally, a description will be given of the flow of the decoding process executed when the picture size of the current picture to be decoded has not been changed, but has been changed before that, or has not been changed at all (S1204). . This embodiment is different from the related art in that the picture memory 203 is managed by exactly the same processing whether or not a picture having a picture size different from the decoding target picture C exists in the picture memory 203. Is possible. The deleted picture area determining unit 209 checks a free area in the picture memory 203, and determines a picture area to be deleted when the free area is insufficient. The pictures stored in the picture memory 203 have display order information in addition to the picture data. By checking the display order information, the deleted picture area determination unit 209 specifies a picture area in which the oldest picture of the display order information is stored, and issues an instruction to delete the specified picture area to the picture memory control unit 206. To communicate. The decoded picture is stored in the vacant area generated as a result.

FIGS. 7A and 7B are diagrams showing a state in which newly decoded pictures are stored in the picture memory 203 in accordance with the procedures of steps S1204 to S1213 in the flowchart of FIG.
FIG. 7A shows an example in a case where there is a sufficient free area for storing the decoded picture C in the picture memory. In this example, P5 and P7 in the picture sequence in FIG. 30 are stored in the picture memory, and P8 is currently being decoded (that is, C = P8). P5 is a picture having the same picture size as the picture area, that is, the picture having the largest picture size among the pictures included in the sequence. On the other hand, the decoded picture C always has the same picture size as the picture area or a smaller picture size. The example of FIG. 7A shows a case where the picture size is smaller than P5. In this case, as shown in the lower part of FIG. 7A, the decoded picture C can be stored in the free area as it is. However, in this case, the decoding target picture C is not stored from the beginning of the empty area, but is accumulated from the beginning of the picture area which is an empty area. Then, the picture area having the picture data already stored in the picture memory is not deleted.

  The upper part of FIG. 7B shows an example in which there is not enough free space in the picture memory 203 to store the decoded picture C. In this example, P4, P5, and P7 in the picture sequence of FIG. 30 are stored in the picture memory, and P8 is currently being decoded (that is, C = P8). In this case, it is necessary to delete the picture area having the picture data stored in the picture memory 203 to secure an area for storing the newly decoded picture C. In the present embodiment, the picture to be decoded has the same picture size as the picture area, which is an area obtained by dividing the picture memory, or a smaller picture size. Therefore, when there is no free area in the picture memory, an area for storing the decoded picture can be secured by deleting one of the picture areas. As the picture area to be deleted, the one with the oldest display order information held by the stored pictures is selected. In this example, it can be seen that the display order information of P4 is the oldest. The deletion picture area determination unit 209 determines the picture area to be deleted by the above-described processing, outputs the determination result to the picture memory control unit 206, and deletes the picture. In the example shown in the upper part of FIG. 7B, P4 is deleted by the above processing, and the decoded picture C is stored in the generated free area as shown in the lower part of FIG. 7B.

  In the above-described embodiment, when the picture area is deleted from the picture memory 203 in the decoding, the display order information of the stored pictures is performed in ascending order, but instead of the display order information, The case where the pictures are deleted in order from the oldest decoded picture can be handled in exactly the same way.

  As can be understood from the above description, in the present embodiment, even when the picture size is changed in the current picture C to be decoded or when the picture size is not changed in the current picture C to be decoded, the picture Since the management of the memory 203 can be realized, there is no need to distinguish between the two. In other words, even when the picture size is changed or not changed during encoding, the picture may not be displayed by a simple processing method of always deleting only one picture area. It is possible to manage the picture memory without inconsistency while leaving the picture in the picture memory as much as possible. Also, on the encoding device side, in addition to the conventional method, it is possible to realize the processing of the present embodiment only by encoding the information of the maximum picture size included in the stream and describing the information in the code sequence. Therefore, mounting of the image encoding device is easy. However, when the newly decoded picture (the picture after the current picture to be decoded C) has a small picture size, as occurs in the examples of FIGS. 6 and 7, the waste that is not used in each picture area is used. It is difficult to say that the picture memory has been effectively utilized because of the occurrence of a special area. However, since pictures of any picture size can be stored continuously in memory without being physically divided, memory access is simplified and decoding can be performed without reducing the speed. It becomes possible. Further, in ordinary image encoding and decoding, a configuration is often used in which processing is performed without changing the picture reference relationship (the number of pictures used for reference) even when the picture size is changed. You. Therefore, it can be said that decoding by an efficient processing method can be realized by the method described in the present embodiment.

(Embodiment 2)
In the first embodiment, the picture memory 203 on the decoding side is divided based on information on the maximum picture size that may be included in the stream, and the picture memory 203 is managed in units of this picture area. On the other hand, in the second embodiment, the picture memory 203 is managed based on the information on the minimum picture size that may be included in the stream.

A moving image encoding method according to Embodiment 2 of the present invention will be described with reference to the block diagram shown in FIG. The configuration of the image encoding device is the same as that of image encoding device 800 described in Embodiment 1. The description of the same components as those already described is omitted here.
Prior to encoding a stream to be encoded, the picture memory division information encoding unit 801 encodes information for designating a method of dividing the decoding-side picture memory. In the present embodiment, information of a minimum picture size that may be included in a stream is encoded as information specifying a method of dividing the picture memory 203 on the decoding side. FIG. 8 is a diagram illustrating a data structure of a code string generated by the image coding device 800 according to the second embodiment. FIG. 8A is a diagram illustrating an example of a code string when information of a minimum picture size that may be included in a stream is encoded. The picture memory division information coding unit 801 describes the information of the minimum picture size that may be included in the stream as min_size of the picture common information area of the code string in FIG. Here, the minimum picture size may be, for example, a minimum picture size that can be used by the image encoding device for encoding, or a plurality of pictures presented when the user performs encoding. One of the sizes may be selected and determined.

  If the picture to be coded is an IDR picture or a picture that can change the picture size in place of the IDR picture, a signal indicating a picture memory reset method is sent from the picture memory control 103 is input. In the present embodiment, the method of resetting the picture memory specified at this time is arbitrary. For example, when the operation of the picture memory on the decoding side is assumed, whether or not there is a picture not displayed in the picture memory Therefore, reset_flag is set to 0, that is, the picture data itself is not deleted just by making all pictures inaccessible. It is possible to arbitrarily select either one. The code sequence generation unit 103 codes the input signal, and generates a code sequence having a reset_flag in a slice header as shown in FIG. 8A.

  In addition, instead of describing the minimum picture size information that may be included in the stream as min_size, the size of each divided picture area when the picture memory 203 is divided in decoding is directly described as one_mem_size. There is also a way to do it. FIG. 8B is a diagram illustrating an example of a code string when encoding information on the size of a picture area obtained by dividing the picture memory 203. For example, as shown in FIG. 8B, the size of the divided picture area may be directly described as one_mem_size in the picture common information. As this one_mem_size, for example, the minimum picture size included in the stream may be used as it is, the greatest common divisor of all the picture sizes included in the stream may be used, or the decoding side memory may be used. The size of a group of areas that can be accessed most efficiently when an operation is assumed may be used.

  When determining one_mem_size, the value of one_mem_size is set so that one macroblock can be stored in the same area, or all macroblocks in one horizontal row of a screen (picture) can be stored in the same area. By rounding up, efficient memory management can be realized. This rounding up may be performed on the image decoding device side. For example, assume that pictures of three types of picture sizes as shown in FIG. 9 are encoded in one stream. FIG. 9A is a diagram illustrating an example of three types of picture sizes encoded in one stream. In the figure, the size of each picture is represented by how many vertical and horizontal lengths correspond to macroblocks (for example, 16 × 16 pixels). In FIG. 9A, picture A is a picture in which the number of macroblocks is 6 × 7, picture B is a picture in which the number of macroblocks is 8 × 10, and picture C is a macroblock. The picture is composed of 12 vertical 12 horizontal rows. FIG. 9B shows an example of a picture area dividing method for efficiently using the picture memory 203 when one stream includes pictures of the three picture sizes shown in FIG. 9A. FIG. In this case, if one_mem_size describes the size of an area in which data for 48 macroblocks can be stored, as shown in FIG. 9B, picture A is one picture area, picture B is two picture areas, Picture C can store data of one picture in three picture areas. At this time, it can be seen that the macroblocks for one row in the horizontal direction of each picture are allocated to one picture area without being divided on the way. Note that, in the example of FIG. 9, all pictures are equally divided, but even if the size of division is not uniform, it can be handled without any problem.

Instead of describing the size information of one picture area obtained by dividing the picture memory on the decoding side as one_mem_size, the number of divided picture areas may be described as area_num in the picture common information area of the code string. It is possible. FIG. 8C is a diagram illustrating an example of a code string when the number of picture regions obtained by dividing the picture memory 203 is encoded. For example, as shown in FIG. 8C, the number of divided picture areas may be directly described in the picture common information as area_num. In this case, since the size of the picture memory of the image decoding apparatus is constant according to the standard or the like, the value of area_num is easily determined by the following equation 4. The calculation result of the division in the expression is calculated as an integer by truncation processing.
(Area_num) = (size of picture memory of image decoding apparatus 1100) / (one_mem_size) (Equation 4)
It should be noted that the above-described picture size and picture memory size can be treated in the same manner regardless of the total number of pixels, the total number of macroblocks, or the total number of bits.

  FIGS. 10 (a), 10 (b) and 10 (c) show the regions of the picture common information in each code string shown in FIGS. 8 (a), 8 (b) and 8 (c). FIG. 7 is a diagram showing an example of a code string when additional information is described. The code strings shown in FIGS. 8A, 8B, and 8C are described as shown in FIGS. 10A, 10B, and 10C. You may. That is, in addition to min_size, one_mem_size, and area_num, cnf_flag can be described in the picture common information area of the code string. The cnf_flag is a flag indicating whether all pictures included in the stream can be displayed without any problem even when the decoding side picture memory is divided and managed according to min_size, one_mem_size, and area_num. For example, a value of 1 indicates that display can be performed without any problem, and a value of 0 indicates that a picture that has not been displayed has already been lost due to the processing of this embodiment. Indicates that it may not be possible.

  In FIGS. 8A, 8B, and 8C, and FIGS. 10A, 10B, and 10C, min_size, one_mem_size, area_num, and cnf_flag are described in the picture common information area. For example, it is also possible to describe in a data area where data that is not always necessary for decoding, but which assists the decoding process is collected. Further, it is also possible to describe in a sequence common information area referred to from the entire sequence. Also, it is possible to describe in the slice header.

Next, a method of decoding a code string encoded by the above-described encoding method will be described with reference to a block diagram of the image decoding apparatus 1100 shown in FIG. 4 and a flowchart of the decoding process shown in FIG. , And the schematic diagrams of the picture memories shown in FIG. 11 and FIG.
In the present embodiment, similar to the first embodiment, the management of the picture memory is not performed in units of pictures, but an area called a picture area divided into one or a plurality of picture memories is performed as one storage and deletion unit. It is characterized by the following. Therefore, prior to decoding the code string, the picture memory division method designating section 210 analyzes the picture memory dividing method from the signal in the code string and instructs the picture memory control section 206. In the present embodiment, as information specifying the method of dividing the picture memory, information on the minimum picture size that may be included in the stream is defined as min_size included in the code string of FIG. It is described in the common information area. The picture memory division method designating section 210 determines a picture memory division method by analyzing the signal. At this time, the number of divided regions is calculated by the following equation 5. The calculation result of the division in the expression is calculated as an integer by truncation processing.
(Number of picture areas) = (size of picture memory of image decoding apparatus 1100) / (min_size) (Equation 5)
When the code sequence of FIG. 8B is used instead of the code sequence of FIG. 8A, the number of picture regions is calculated by the following Expression 6.
(Number of picture areas) = (size of picture memory of image decoding apparatus 1100) / (one_mem_size) (Equation 6)
When the code sequence of FIG. 8C is used instead of the code sequence of FIG. 8A, the number of picture regions is calculated by the following equation (7).
(Number of picture areas) = (area_num) (Equation 7)
The above-described picture size and picture memory size can be treated in the same manner regardless of whether the size is represented by the total number of pixels, the total number of macroblocks, or the total number of bits. is there.

The picture size change determination unit 207 compares the picture size of the picture to be decoded with the picture size of the picture decoded before (or immediately before) and determines whether the picture size has been changed.
As shown in FIGS. 8A, 8B and 8C, reset_flag is encoded in the slice header in the code string. The code string analysis unit 201 analyzes the value of the reset_flag, and outputs information of the analysis result to the picture memory control unit 206. In the present embodiment, even if the picture size has been changed, the value of reset_flag may be both 0 or 1. Accordingly, the picture memory management method is divided according to the value of reset_flag as shown in the flowchart of FIG. In particular, when reset_flag is 0, there is a possibility that the picture before the picture size change remains in the picture memory. Therefore, which picture area must be deleted to accumulate the picture after the picture size change. Must be determined. The determination is made by the deletion picture area determination unit 209, and the determination result is output to the picture memory control unit 206, and the picture memory 203 is managed.

  Next, the details of the operations of the picture memory control unit 206, the picture size change determination unit 207, and the deleted picture area determination unit 209 in the decoding process will be described with reference to the flowchart in FIG. In the flowchart of FIG. 5, after the decoding of the target picture is performed, the determination of the change in the picture size and the determination of the value of the reset_flag are performed. However, the order is arbitrary, and is limited to the order shown in FIG. Not something.

  First, the flow of the decoding process executed when the picture size of the decoding target picture is changed (S1204) and reset_flag is 1 (S1205) will be described. A reset_flag of 1 is a value indicating that all picture data in the picture memory is deleted and all areas are left empty. That is, when this signal is input to the picture memory control unit 206, the picture memory control unit 206 deletes all the areas in the picture memory (S1206) and replaces the decoded picture with the changed picture size in the picture memory. (S1207).

  Next, the flow of the decoding process executed when the picture size of the decoding target picture is changed (S1204) and reset_flag is 0 (S1205) will be described. The reset_flag of 0 is a value indicating that all the picture data in the picture memory is not to be referred to, but the picture data itself is not deleted but is stored. The deleted picture area determination unit 209 checks the free area in the picture memory, and when the free area is not enough, compares the size after the picture size change with the size of one picture area obtained by dividing the picture memory, and A division method is determined, and the number of picture areas to be deleted and the picture area to be deleted are determined (S1208). The pictures stored in the picture memory 203 have display order information in addition to the picture data. By checking the display order information, the deletion picture area determination unit 209 specifies a picture area in which pictures with the oldest display order information are stored, and issues an instruction to delete in order from the specified picture area. To communicate. As a result, the designated picture area is deleted, and the decoded picture is stored in the generated free area (S1210).

FIGS. 11A and 11B are diagrams showing a state in which a newly decoded picture is stored in the picture memory 203 according to the procedure of steps S1204 to S1210 in the flowchart of FIG.
FIG. 11A shows an example in a case where there is a sufficient free area in the picture memory to store the newly decoded picture C. In this example, P2, P6, P4, and P5 in the picture sequence of FIG. 30 are stored in the picture memory, and P7 is currently being decoded (that is, C = P7). It can be seen that P2, P6, P4, and P5 are pictures having the same picture size as the picture area. On the other hand, the newly decoded picture C has a picture size larger than the size of one picture area, and shows an example in which the picture is divided into two and stored. In this case, as shown in the lower part of FIG. 11A, it is possible to store the newly decoded picture C as it is in the free area using two picture areas. That is, the picture area in which the picture data is already stored in the picture memory is not deleted.

  The upper part of FIG. 11B shows an example in which there is not enough free space to store the decoded picture C in the picture memory. In this example, P3, P1, P2, P6, P4, and P5 in the picture sequence in FIG. 30 are stored in the picture memory, and P7 is currently being decoded (that is, C = P7). In this case, it is necessary to secure a region for storing the decoded picture by deleting the picture region having the picture data before the size change. In this example, the picture C after the picture size change has a larger picture size than the size of one picture area, and it must be divided into two and stored using two picture areas. I understand. The picture area to be deleted is selected in ascending order of display order information of the pictures stored in the picture area. In this example, it can be seen that the display order information is older in the order of P1 and P2. The deleted picture area determining unit 209 determines a picture area to be deleted by the above processing, outputs a result of the determination to the picture memory control unit 206, and deletes the determined picture area. In the example shown in the upper part of FIG. 11B, P1 and P2 are deleted by the above-described picture area deletion processing, and the newly decoded picture C is stored in the generated free area as shown in the lower part of FIG. 11B. I have.

  Finally, the flow of the decoding process executed when the picture size of the decoding target picture C has not been changed, but has been changed before that, or has not been changed at all (S1204) will be described. I do. In the present embodiment, it is possible to manage the picture memory by exactly the same processing whether or not a picture having a picture size different from the picture to be decoded exists in the picture memory. The deleted picture area determination unit 209 checks the free area in the picture memory, and when the free area is not enough, compares the size after the picture size change with the size of one picture area obtained by dividing the picture memory, and The division method is determined, and the number of picture areas to be deleted and the picture areas to be deleted are determined. The pictures stored in the picture memory have display order information in addition to the picture data. By checking the display order information, the deleted picture area determination unit 209 specifies a picture area in which the oldest picture of the display order information is stored, and issues an instruction to delete the picture area in order from the specified picture area. The information is transmitted to the control unit 206. The newly decoded picture is stored in the free space generated as a result.

FIGS. 12A and 12B are diagrams showing a state in which newly decoded pictures are stored in the picture memory 203 in accordance with the procedures of steps S1204 to S1213 in the flowchart of FIG.
FIG. 12A shows an example in a case where there is sufficient free space in the picture memory to store the newly decoded picture C. In this example, P4, P5, and P7 in the picture sequence of FIG. 30 are stored in the picture memory, and P8 is currently being decoded (that is, C = P8). It can be seen that P4 and P5 are pictures having the same picture size as the picture area. On the other hand, the newly decoded picture C has a picture size larger than the size of one picture area, and shows an example in which the picture is divided into two and stored. In this case, as shown in the lower part of FIG. 12A, the decoded picture C can be directly stored in the free area using two picture areas. That is, the picture area having the picture data already stored in the picture memory 203 is not deleted.

  The upper part of FIG. 12B shows an example in which there is not enough free space in the picture memory 203 to store the decoded picture C to be decoded. In this example, P2, P6, P4, P5, and P7 in the picture sequence in FIG. 30 are stored in the picture memory, and P8 is currently being decoded (that is, C = P8). In this case, it is necessary to delete the picture area having the picture data and secure an area for storing the newly decoded picture C. In this example, since the picture size of the picture C after the picture size change is larger than the size of one picture area, the picture C is divided into two parts, and the picture C is not stored using the two picture areas. It turns out that it should not be. In addition, the area of the picture to be deleted is selected in ascending order of display order information of the stored picture. In this example, it can be seen that the display order information is older in the order of P2 and P4. The deleted picture area determining unit 209 determines the picture area to be deleted by the above-described processing, outputs the determination result to the picture memory control unit 206, and deletes the picture area. In the example of the upper part of FIG. 12B, P2 and P4 are deleted by the above-described determination process of the picture area to be deleted, and as shown in the lower part of FIG. Accumulated in the area.

  In the second embodiment, when the picture area is deleted from the picture memory 203 in the image decoding apparatus 1100, the display order information of the pictures stored in the picture area is deleted in ascending order. Instead of the information, the case where the picture area is deleted in the order of decoding of the stored pictures from the oldest can be handled in exactly the same manner.

  As can be understood from the above description, in the present embodiment, even when the picture size is changed in the current picture C to be decoded or when the picture size is not changed in the current picture C to be decoded, the picture Since the management of the memory 203 can be realized, there is no need to distinguish between the two. In other words, regardless of whether the picture size is changed or not when encoding is performed, it always determines how many areas the decoded picture needs, and deletes only the necessary areas. With such a processing method, it is possible to manage the picture memory without inconsistency while leaving the picture which may not be displayed in the picture memory as much as possible. In addition, on the encoding side, in addition to the conventional method, information on the minimum picture size included in the stream or the size of one picture area that divides the picture memory assuming the decoding side is encoded. It is possible to realize the processing according to the present embodiment only by converting it into a code string, and it is easy to create an image coding apparatus. Further, the size of dividing the decoded picture can be arbitrarily set in consideration of the efficiency of memory access, and decoding can be performed without reducing the speed related to memory access.

(Embodiment 3)
In the first embodiment, the information for specifying the method of dividing the picture memory is generated in the image coding device and described in the code string. On the other hand, in the third embodiment of the present invention, the method of dividing the picture memory is analyzed and determined by the image decoding device. Therefore, the moving picture encoding method according to Embodiment 3 of the present invention is the same as the conventional encoding method described with reference to FIG. 28, and a description thereof will not be repeated.

  FIG. 13 is a block diagram showing a configuration of an image decoding device 2000 according to Embodiment 3 of the present invention. Hereinafter, a moving image decoding method according to Embodiment 3 of the present invention will be described with reference to the block diagram shown in FIG. The configuration of the decoding process is almost the same as the method of the first embodiment described with reference to FIG. The difference from the method of the first embodiment is that an address dividing unit 210A is provided instead of the picture memory dividing method designating unit 210. Here, the description of the processing completely the same as that of the first embodiment is omitted.

  The picture size change determination unit 207 compares the picture size of the picture to be decoded with the picture size of the picture decoded before (or immediately before), and determines whether the picture size has been changed. When determining that the picture size has been changed, the picture size change determining unit 207 notifies the address dividing unit 210A of the changed picture size via the code string analyzing unit 201. The address dividing unit 210A divides the address space of the picture memory 203 so that the changed picture size notified by the picture size change determining unit 207 becomes the size of each picture area, and uses the divided addresses as units. It instructs the picture memory control unit 206 to manage 203.

At this time, the number of regions to be divided is calculated by the following equation (8). The calculation result of the division in the expression is calculated as an integer by truncation processing.
(Number of picture areas) = (size of picture memory 203 of image decoding apparatus 2000) / (picture size after changing picture size) (Equation 8)
The picture size and the size of the picture memory may be set to any of the total number of pixels, the total number of macroblocks, or the total number of bits, and can be handled in the same manner.

Next, the operations of the picture memory control unit 206, the picture size change determination unit 207, the deleted picture area determination unit 209, and the address division unit 210A in the decoding process will be described in detail with reference to FIG. FIG. 14 is a flowchart illustrating a procedure of a decoding process in the image decoding device 2000 illustrated in FIG.
First, when the picture size of the decoding target picture is changed (S2301), the address division unit 210A sends the picture size of the decoding target picture from the picture size change determination unit 207 via the code sequence analysis unit 201, that is, the size. The picture size after the change is obtained (S2302). The address dividing unit 210A instructs the picture memory control unit 206 to divide the address space of the picture memory 203 by the obtained picture size and manage the picture memory 203 in units of the divided picture area (S2303). If the picture size has not been changed in the decoding target picture (S2301), the processes in steps S2302 and S2303 are not performed.

  Next, the deleted picture area determination unit 209 checks a free area in the picture memory 203 (S2304), and if any one of the divided picture areas does not exist as a continuous free area, at least one Until, the pictures in the picture memory 203 are deleted in the order of display order (S2305). Specifically, the pictures stored in the picture memory 203 have information indicating the display order of the pictures in addition to the picture data. By checking the display order information, the deleted picture area determination unit 209 transmits an instruction to delete pictures in order from the oldest picture in the display order information to the picture memory control unit 206. The picture memory control unit 206 repeats the process of deleting the picture instructed by the deletion picture area determination unit 209 until one divided picture area becomes a free area, and as a result, When the minute picture area becomes an empty area, the newly decoded picture is stored in the empty picture area (S2306). If the one divided picture area is a free area from the beginning (S2304), the newly decoded picture is stored in that picture area as it is (S2306).

  FIGS. 15A and 15B are diagrams showing a state in which newly decoded pictures are stored in the picture memory 203 according to the procedure of the flowchart in FIG. FIGS. 15A and 15B show a case where the picture sizes after the current decoding target picture C are changed to larger ones. FIG. 15A shows an example of a case where there is a sufficient free area in the picture memory 203 to store the newly decoded picture C. In this example, it is assumed that P2, P4, P5, and P6 in the picture sequence shown in FIG. 30 are stored in the picture memory and P7 is currently being decoded (that is, C = P7). In this example, the picture size is changed from P7 which is the decoding target picture C, and the address of the picture memory 203 is divided by the picture size of P7 as shown in the lower part of FIG. In this case, as shown in the lower part of FIG. 15A, the newly decoded picture C can be stored in a free area as it is. A picture area having picture data already stored in the picture memory is not deleted.

  FIG. 15B shows an example in which there is not enough free space in the picture memory 203 to store the newly decoded picture C. In this example, P1, P2, P3, P4, P5, and P6 in the picture sequence in FIG. 30 are stored in the picture memory 203, and P7 is currently being decoded (that is, C = P7). I have. In this case, it is necessary to delete the area having the picture data stored in the picture memory 203 to secure an area for storing the newly decoded picture C. In the present embodiment, the picture C to be decoded always has the same picture size as the picture area which is an area obtained by dividing the picture memory. Therefore, when there is no free area in the picture memory 203, if a picture is deleted until one of the picture areas in the picture memory is completely free, the newly decoded picture can be stored. An area can be secured. The pictures to be deleted are selected in ascending order of display order information of the stored pictures. In this example, it can be seen that the display order information of P1 is the oldest. However, by simply deleting P1, none of the picture areas divided according to the picture size of the picture C to be decoded is one free area. At the stage where P2 and P3 are deleted after P1 according to the display order information, the second picture area from the head address of the picture memory 203 becomes one free area. The deletion picture area determination unit 209 determines a picture to be deleted by the above-described processing, and outputs a determination result to the picture memory control unit 206. The picture memory control unit 206 deletes a picture according to the determination result of the deleted picture area determination unit 209. In the example in the upper part of FIG. 15B, P1, P2, and P3 are deleted by the above processing, and the newly decoded picture C is stored in the generated free area as shown in the lower part of FIG.

  FIGS. 16A and 16B are diagrams showing a state in which newly decoded pictures are stored in the picture memory 203 according to the procedure of the flowchart in FIG. FIGS. 16A and 16B show a case where the picture sizes after the current decoding target picture C have been changed to smaller sizes. FIG. 16A shows an example of a case where there is a sufficient free area in the picture memory 203 to store the newly decoded picture C. In this example, it is assumed that P1, P2, and P3 in the picture sequence shown in FIG. 30 are stored in the picture memory and P4 is currently being decoded (that is, C = P4). In this example, the picture size is changed from the decoding target picture C P4, and the address of the picture memory 203 is divided by the P4 picture size as shown in the lower part of FIG. In this case, as shown in the lower part of FIG. 16A, the newly decoded picture C can be stored in one picture area in the free area without any excess or shortage. Then, the picture area having the picture data already stored in the picture memory is not deleted.

  FIG. 16B shows an example in which there is not enough free space in the picture memory 203 to store the newly decoded picture C. In this example, P1, P2, P3, and P4 in the picture sequence in FIG. 30 are stored in the picture memory 203, and P5 is currently being decoded (that is, C = P5). In this case, it is necessary to delete the area in which the picture data is already stored, and to secure an area in the picture memory 203 for storing the newly decoded picture C. In the present embodiment, the picture C to be decoded always has the same picture size as the picture area which is an area obtained by dividing the picture memory. Therefore, when there is no free area in the picture memory 203, if a picture is deleted until one of the picture areas in the picture memory is completely free, the newly decoded picture can be stored. An area can be secured. The pictures to be deleted are selected in ascending order of display order information of the stored pictures. In this example, it can be seen that the display order information of P1 is the oldest. Then, by deleting P1, one picture area becomes a free area. The deletion picture area determination unit 209 determines a picture to be deleted by the above-described processing, and outputs a determination result to the picture memory control unit 206. The picture memory control unit 206 deletes a picture according to the determination result of the deleted picture area determination unit 209. In the example in the upper part of FIG. 16B, P1 is deleted by the above processing, and the newly decoded picture C is stored in the generated free area as shown in the lower part of FIG. 16B.

In the above embodiment, when the picture area is deleted from the picture memory 203, the display order information of the stored pictures is deleted in ascending order, but instead of the display order information, decoding is performed. It is possible to handle in the same manner even when the deletion is performed in the order from the oldest picture.
As can be seen from the above description, in the present embodiment, the management of the picture memory 203 can be realized by exactly the same processing regardless of whether the picture size of the decoding target picture C is changed or not. There is no need to distinguish between the two. In other words, even when the picture size is changed or not changed during encoding, the picture stored in the picture area of only one area is always deleted until the picture area is completely free. With such a processing method, it is possible to manage the picture memory without inconsistency while leaving the picture that may not be displayed in the picture memory as much as possible. In addition, the size of each of the newly divided picture areas is the same as the size of the newly decoded picture (the picture after the decoding target picture C). Even if the size is changed to a small size, no unnecessary area is generated in each picture area. For this reason, in the image decoding apparatus 2000 of the present embodiment, the memory area of the picture memory 203 can be effectively used. At the same time, since pictures of any picture size can be continuously stored in memory without being physically divided, memory access is simplified and decoding is performed without reducing speed. Becomes possible. Further, in normal image encoding and decoding, a configuration is often used in which processing is performed without changing the picture reference relationship (the number of pictures used for reference) even when the picture size is changed. You. Therefore, it can be said that decoding by an efficient processing method can be realized by the method described in the present embodiment.

  However, in the present embodiment, the picture data is not stored from the beginning of the free area, but is stored from the beginning of the picture area in the free area. Therefore, as shown in FIG. There is a possibility that an area larger than one picture C to be decoded is deleted. Therefore, in order to accumulate the newly decoded picture C, if the pictures are deleted from the oldest one in the display order until one picture area becomes a free area, the display order information If the old pictures are not stored in the continuous area in the picture memory 203 in the order of the display order information, it cannot be denied that there is a possibility that the pictures in the display waiting state may be deleted.

  In the third embodiment, the pictures in the display order information are deleted in order from the oldest picture, and when one picture area becomes a continuous free area, the newly decoded picture C is stored in the picture area. did. However, the present invention is not limited to this, and the decoding target picture C may be stored by a different method. That is, regardless of the display order information of the pictures, the pictures that have already been stored are sequentially deleted from the top address until one top picture area of the picture memory 203 becomes a continuous free area. When the leading one picture area becomes a continuous free area, the newly decoded picture C is stored in that picture area. Next, when storing the decoded picture C, the next picture after the last deleted picture is stored until the second picture area from the top of the picture memory 203 becomes a continuous free area. Delete pictures in order. FIG. 17 is a flowchart illustrating a procedure of another decoding process in the image decoding device 2000 illustrated in FIG.

  17, the processing in steps S2601 to S2603 is the same as the processing in steps S2301 to S2303 shown in FIG. When the picture size of the picture to be decoded is changed by the processing here, the address space of the picture memory 203 is divided by the picture size after the size change, and the picture memory 203 is managed in units of the divided picture area. You.

  Next, the deleted picture area determination unit 209 checks a free area in the picture memory 203, and if there is a sufficient free area including a picture area for storing a newly decoded picture, the newly decoded picture Is stored in an empty picture area as it is (S2606). If no picture area for storing the newly decoded picture exists as a free area (S2604), the deleted picture area determination unit 209 determines the decoded picture in order from the top of the picture memory 203. Makes the picture area next to the last stored picture area an empty area (S2605), and stores a new decoded picture in the empty picture area (S2606). FIG. 18 is a flowchart showing a more detailed procedure of the process in step S2605 shown in FIG. More specifically, the picture memory control unit 206 indicates a specific address position in the picture memory 203 that is used only when there is not enough free space to store a new picture in the picture memory 203. A pointer is provided. The specific address position is, for example, the start address of each picture area. It is also assumed that the initial value of the pointer is “0”, and that the pointer = 0 indicates the head address of the picture area at the head of the picture memory 203. The pointer = (n-1) indicates the head address of the n-th picture area from the head of the picture memory 203 (where n is a natural number).

  When the deleted picture area determination unit 209 determines that there is no picture area for storing the newly decoded picture as an empty area (S2604), it is indicated by a pointer to the picture memory control unit 206. The pictures in the picture area are deleted in the order of the addresses (S2501 to S2503). For example, when the pointer = 0, the pictures in the picture memory 203 are sequentially deleted from the beginning of the memory until the picture area at the head of the picture memory 203 becomes a free area. When the picture area at the head of the picture memory 203 becomes a free area (S2502), the picture memory control unit 206 increments the pointer by “1” (S2504) and causes the pointer to point to the start address of the next picture area. Here, the picture memory control unit 206 checks whether or not the value of the pointer is equal to or greater than the number of all picture areas. If the value of the pointer is equal to or greater than the number of all picture areas (S2505), the value of the pointer is set to “0”. Then, the head address of the head picture area in the picture memory 203 is pointed. When the value of the pointer becomes equal to or greater than the number of all picture areas, it indicates that a new decoded picture has been accumulated in all picture areas in the picture memory 203.

  By the above processing, if there is no more free picture area for storing a new decoded picture in the picture memory 203, picture data is deleted from the picture area in order from the top of the picture memory 203, and A new decoded picture is stored in the picture area.

  FIGS. 19A and 19B are diagrams showing a state in which newly decoded pictures are stored in the picture memory 203 according to the procedures of the flowcharts in FIGS. FIGS. 19A and 19B show a case where the picture sizes after the current decoding target picture C are changed to larger sizes. FIG. 19A shows an example in which there is a sufficient free area in the picture memory 203 to store the newly decoded picture C. In this example, it is assumed that P2, P4, P5, and P6 in the picture sequence shown in FIG. 30 are stored in the picture memory and P7 is currently being decoded (that is, C = P7). In this example, the picture size is changed from P7 which is the decoding target picture C, and the address of the picture memory 203 is divided by the picture size of P7 as shown in the lower part of FIG. In this case, there is a sufficient free area at the beginning of the picture memory 203 as shown by the shaded area 2 in the upper part of FIG. 19 (a), and therefore, as shown in the lower part of FIG. Can be stored in the empty area as it is. Therefore, the deletion of the picture area having the picture data already stored in the picture memory is not performed.

  FIG. 19B shows an example in which there is not enough free space in the picture memory 203 to store the newly decoded picture C. In this example, P1, P2, P3, P4, P5, and P6 in the picture sequence in FIG. 30 are stored in the picture memory 203, and P7 is currently being decoded (that is, C = P7). I have. In this case, it is necessary to delete the area having the picture data stored in the picture memory 203 to secure an area for storing the newly decoded picture C. In the present embodiment, the picture C to be decoded always has the same picture size as the picture area which is an area obtained by dividing the picture memory. Therefore, when there is no free area in the picture memory 203, if a picture is deleted until one of the picture areas in the picture memory is completely free, the newly decoded picture can be stored. An area can be secured. Pictures to be deleted are selected in order of address from the top of the picture memory 203. In this example, at the stage where P4 and P1 are deleted in this order, the first picture area from the top of the picture memory 203 becomes one free area. The deletion picture area determination unit 209 determines a picture to be deleted by the above-described processing, and outputs a determination result to the picture memory control unit 206. The picture memory control unit 206 deletes a picture according to the determination result of the deleted picture area determination unit 209. In the example shown in the upper part of FIG. 19B, P4 and P1 are deleted by the above processing, and a newly decoded picture C is stored in the first picture area as shown in the lower part of FIG. 19B. When the next picture C to be decoded is further decoded, the newly decoded picture must be stored with the state in the picture memory 203 shown in the lower part of FIG. Disappears. In this case, P2 and P3 are deleted, and a new decoded picture is accumulated in the second picture area that has become a free area.

  FIGS. 20A and 20B are diagrams showing a state in which newly decoded pictures are stored in the picture memory 203 in accordance with the procedures of the flowcharts in FIGS. FIGS. 20A and 20B show a case where the picture sizes after the current decoding target picture C are changed to smaller sizes. FIG. 20A shows an example in a case where there is a sufficient free area in the picture memory 203 to store the newly decoded picture C. In this example, it is assumed that P1, P2, and P3 in the picture sequence in FIG. 30 are stored in the picture memory, and P4 is currently being decoded (that is, C = P4). In this example, the picture size is changed after P4, which is the decoding target picture C, and as shown in the lower part of FIG. Is divided. In this case, as shown in the lower part of FIG. 20A, the newly decoded picture C can be stored in one picture area in the free area without any excess or shortage. Then, the picture area having the picture data already stored in the picture memory is not deleted.

  FIG. 20B shows an example in which there is not enough free space in the picture memory 203 to store the newly decoded picture C. In this example, P1, P2, P3, and P4 in the picture sequence in FIG. 30 are stored in the picture memory 203, and P5 is currently being decoded (that is, C = P5). In this case, it is necessary to delete the area in which the picture data is already stored, and to secure an area in the picture memory 203 for storing the newly decoded picture C. In the present embodiment, the picture C to be decoded always has the same picture size as the picture area which is an area obtained by dividing the picture memory. Therefore, when there is no free area in the picture memory 203, if a picture is deleted until one of the picture areas in the picture memory is completely free, the newly decoded picture can be stored. An area can be secured. The picture to be deleted is selected from the top of the picture memory 203 in the order of the address where the picture data is stored. In this example, it can be seen that the address of P3 is the first. Then, by deleting P3, the first picture area of the picture memory 203 becomes a free area. The deletion picture area determination unit 209 determines a picture to be deleted by the above-described processing, and outputs a determination result to the picture memory control unit 206. The picture memory control unit 206 deletes a picture according to the determination result of the deleted picture area determination unit 209. In the example shown in the upper part of FIG. 20B, P3 is deleted by the above processing, and a newly decoded picture C is stored in the first picture area of the picture memory 203 as shown in the lower part of FIG. In this state, when the next decoding target picture C is decoded, P2 is deleted from the state at the bottom of FIG. 20B, and a new decoding is performed in the second picture area from the head of the picture memory 203. The stored picture is stored.

  In the example described with reference to FIGS. 17 to 20B, if there is a sufficient free area in the picture memory 203 to store a newly decoded picture, Although the newly decoded picture is stored in one picture area as an area, the present invention is not limited to this. For example, even if there is a sufficient free area in the picture memory 203, a picture area is selected in the order of address from the top of the picture memory 203, and the picture data stored in the selected picture area is selected. May be sequentially deleted in units of pictures until the picture areas become continuous free areas. As a result, the newly decoded picture is stored in the picture area which has become a free area. That is, as shown in the upper part of FIG. 20A, even if there is a sufficient free area for storing the decoding target picture C, the picture P1 stored at the top of the picture memory 203 is deleted. Alternatively, the decoding target picture C may be stored in the leading picture area of the picture memory 203 that has become a free area. In this way, whether or not there is enough free space in the picture memory 203 to store the newly decoded picture, Can be selected in the order of address, and newly decoded pictures can be stored in the selected picture area. Can be reduced.

  As can be understood from the above description, in the present embodiment, even when the picture size is changed or not changed when encoding is performed, the picture area of only one area always becomes a completely free area. By a simple processing method of deleting the picture stored up to this point, it is possible to manage the picture memory without inconsistency while leaving the picture which may not be displayed in the picture memory as much as possible. In addition, the size of each of the newly divided picture areas is the same as the size of the newly decoded picture (the picture after the decoding target picture C). Even if the size is changed to a small size, no unused area is generated in each picture area. For this reason, the image decoding device 2000 of the present embodiment can effectively utilize the memory area of the picture memory 203. At the same time, since pictures of any picture size can be continuously stored in memory without being physically divided, memory access is simplified and decoding is performed without reducing speed. Becomes possible. Further, in ordinary image encoding and decoding, a configuration is often used in which processing is performed without changing the picture reference relationship (the number of pictures used for reference) even when the picture size is changed. You. Therefore, it can be said that decoding by an efficient processing method can be realized by the method described in the present embodiment.

However, in the present embodiment, since the pictures are deleted in order from the beginning of the picture memory until a free area can be secured ignoring the display order, a new picture of display order information is particularly stored at the beginning of the picture memory. When stored in a close position, it cannot be denied that there is a possibility that a picture in a display waiting state may be deleted.
In the third embodiment, it is necessary to delete one or a plurality of pictures until one of the picture areas in the picture memory is completely vacant. Could be Therefore, cnf_flag may be used as a flag indicating whether all pictures included in the stream can be displayed without any problem. For example, a value of 1 indicates that all pictures can be displayed without any problem, and a value of 0 indicates that a picture that has not been displayed has already lost data by the processing of this embodiment. Indicates that display may not be possible. Thus, by describing cnf_flag in the code string, it is possible to determine whether or not to apply the processing of the present embodiment by checking the value of this signal. For example, when cnf_flag is 0, it is known that there is a possibility that a picture that cannot be displayed may occur when the processing of the present embodiment is applied. You can switch to As a result, the memory area may be subdivided, but all pictures can be displayed without any problem.

(Embodiment 4)
A moving picture coding method according to Embodiment 4 of the present invention will be described with reference to the block diagram shown in FIG. FIG. 21 is a block diagram illustrating a configuration of an image encoding device 2700 according to Embodiment 4. The configuration of the encoding process is almost the same as the conventional method described with reference to FIG. The difference from the conventional method is that a picture size change determination unit 110 is added. The description of the same processing as the conventional technique is omitted here. FIG. 22 is a diagram illustrating a data structure of a code string generated by the image coding device 2700 according to the fourth embodiment.

If the picture to be coded is an IDR picture or a picture that allows a change in picture size instead of the IDR picture, the picture size change determination unit 110 determines the picture size of the picture to be coded and the previously coded picture. To determine whether the picture size has been changed.
If the determination result input from the picture size change determination unit 110 indicates that the picture size has not been changed, the picture memory control unit 109 performs coding using the same method as in normal encoding. Perform the conversion. On the other hand, if the determination result indicates that the picture size has been changed, all pictures in the picture memory should be deleted regardless of whether they can be referenced or not, as a method of resetting the picture memory in decoding. Is output to the code sequence generation unit 103. The code sequence generation unit 103 sets the value of the flag so that reset_flag of the code sequence in FIG. 22 is 1, that is, indicates that all pictures in the picture memory are deleted (regardless of whether or not reference is possible). Encoding.

On the other hand, decoding of a code string coded by the above-described coding method can be performed using the same configuration as the block diagram used in the description of the conventional method in FIG. . The description of the same processing as the conventional technique is omitted here.
As shown in FIG. 22, reset_flag is encoded in the slice header in the code string. The code string analysis unit 201 analyzes the value of the reset_flag, and outputs information of the analysis result to the picture memory control unit 206. In a code string generated by the above-described coding method, reset_flag is always 1 when the picture size is changed, that is, all pictures in the picture memory are deleted regardless of whether they can be referred to or not. Instruct that Thus, when decoding, the picture memory can be reset according to the value of reset_flag without being conscious of whether the picture size has been changed, just like the conventional method. If the picture size has been changed, since the entire area of the picture memory becomes empty according to the instruction of the reset_flag, the picture after the picture size change is decoded and the decoded picture is stored in the picture memory as before. Also, pictures of different picture sizes do not coexist in the picture memory, and an arbitrary area can be used for storing decoded pictures.

  As described above, as described in the present embodiment, when a picture size is changed during encoding, a flag having a value indicating that all pictures in the picture memory are deleted is encoded. On the decoding side, it is possible to manage the picture memory without contradiction by performing the same processing as the conventional method.

(Embodiment 5)
Further, by recording a program for realizing the configuration of the moving picture encoding method and the moving picture decoding method described in each of Embodiments 1 to 4 on a recording medium such as a flexible disk, The processing described in each of the first to fifth embodiments can be easily performed in an independent computer system.

FIG. 23 is an explanatory diagram of a recording medium for storing a program for implementing the moving picture encoding method and the moving picture decoding method of Embodiments 1 to 4 by a computer system.
FIG. 23B shows the appearance, cross-sectional structure, and flexible disk as viewed from the front of the flexible disk, and FIG. 23A shows an example of the physical format of the flexible disk as the recording medium body. The flexible disk FD is built in the case F, and a plurality of tracks Tr are formed concentrically from the outer circumference toward the inner circumference on the surface of the disk, and each track is divided into 16 sectors Se in an angular direction. ing. Therefore, in the flexible disk storing the program, the moving image encoding method and the moving image decoding method as the program are recorded in the area allocated on the flexible disk FD.

  FIG. 23C shows a configuration for recording and reproducing the program on the flexible disk FD. When the above program is recorded on the flexible disk FD, the moving picture encoding method and the moving picture decoding method as the above program are written from the computer system Cs via the flexible disk drive. When the moving picture encoding method and the moving picture decoding method are to be constructed in a computer system using a program in a flexible disk, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.

  In the above description, the description has been made using a flexible disk as a recording medium. However, the same description can be made using an optical disk. Further, the recording medium is not limited to this, and any other recording medium such as a CD-ROM, a memory card, a ROM cassette, or the like can be similarly implemented.

(Embodiment 6)
Further, here, application examples of the moving picture coding method and the moving picture decoding method described in Embodiments 1 to 5 and a system using the same will be described.

FIG. 24 is a block diagram illustrating an overall configuration of a content supply system ex100 that realizes a content distribution service. A communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.
The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a camera on the Internet ex101 via the Internet service provider ex102 and the telephone network ex104, and the base stations ex107 to ex110. Each device such as a mobile phone ex115 with a tag is connected.

However, the content supply system ex100 is not limited to the combination as shown in FIG. 24, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.
The camera ex113 is a device capable of shooting moving images, such as a digital video camera. In addition, a mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system. Or PHS (Personal Handyphone System) or the like, and either may be used.

  The streaming server ex103 is connected from the camera ex113 to the base station ex109 and the telephone network ex104, and enables live distribution and the like based on the encoded data transmitted by the user using the camera ex113. The encoding process of the captured data may be performed by the camera ex113, or may be performed by a server or the like that performs the data transmission process. Also, moving image data captured by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera capable of shooting still images and moving images. In this case, encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed by the LSI ex117 included in the computer ex111 and the camera ex116. Note that software for encoding / decoding moving images may be incorporated in any storage medium (CD-ROM, flexible disk, hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Further, the moving image data may be transmitted by a mobile phone with camera ex115. The moving image data at this time is data that has been encoded by the LSI included in the mobile phone ex115.

  In the content supply system ex100, the content (for example, a video image of a live music) captured by the user with the camera ex113, the camera ex116, or the like is subjected to encoding processing in the same manner as in Embodiments 1 to 5, and the streaming server ex103. On the other hand, the streaming server ex103 stream-distributes the content data to the requesting client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like, which are capable of decoding the encoded data. In this way, the content providing system ex100 can receive and reproduce the encoded data on the client, and further, can receive, decode, and reproduce the encoded data on the client in real time, thereby realizing personal broadcasting. It is a system that becomes possible.

The encoding and decoding of each device constituting this system may be performed using the image encoding device or the image decoding device described in each of the first to fifth embodiments.
A mobile phone will be described as an example.

  FIG. 25 is a diagram illustrating the mobile phone ex115 that uses the moving picture encoding method and the moving picture decoding method described in Embodiments 1 to 5. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a video image of a CCD camera or the like, a camera unit ex203 capable of taking a still image, a video image captured by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display for displaying data obtained by decoding a received video or the like, a main unit including a group of operation keys ex204, an audio output unit ex208 such as a speaker for outputting audio, and audio input. Input unit ex205 such as a microphone for storing encoded or decoded data, such as data of captured moving images or still images, received mail data, moving image data or still image data, etc. Recording medium ex207, and a slot ex20 for allowing the recording medium ex207 to be attached to the mobile phone ex115. The it has. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a nonvolatile memory which can be electrically rewritten and erased, in a plastic case such as an SD card.

  Further, the mobile phone ex115 will be described with reference to FIG. The mobile phone ex115 controls a power supply circuit unit ex310, an operation input control unit ex304, an image encoding unit, and a main control unit ex311 that integrally controls each unit of a main unit including a display unit ex202 and operation keys ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronous bus ex313. Connected to each other.

The power supply circuit unit ex310 activates the camera-equipped digital mobile phone ex115 in an operable state by supplying power to each unit from the battery pack when the call end and the power key are turned on by a user operation. .
The mobile phone ex115 converts an audio signal collected by the audio input unit ex205 into digital audio data by the audio processing unit ex305 in the audio communication mode based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. This is spread-spectrum processed by a modulation / demodulation circuit unit ex306, subjected to digital-analog conversion processing and frequency conversion processing by a transmission / reception circuit unit ex301, and then transmitted via an antenna ex201. The mobile phone ex115 amplifies received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-to-digital conversion processing, performs spectrum despreading processing in the modulation / demodulation circuit unit ex306, and performs analog voice decoding in the voice processing unit ex305. After the data is converted, the data is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital / analog conversion processing and frequency conversion processing on the transmission / reception circuit unit ex301, and transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When the image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 includes the image encoding device described in the present invention, and uses the image data supplied from the camera unit ex203 in the image encoding devices described in the first to fifth embodiments. The data is converted into encoded image data by performing compression encoding by an encoding method, and is transmitted to the demultiplexing unit ex308. Further, at this time, the mobile phone ex115 simultaneously transmits the voice collected by the voice input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital voice data via the voice processing unit ex305.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and modulates and outputs the resulting multiplexed data. The signal is subjected to spread spectrum processing in ex306 and subjected to digital / analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmitted via the antenna ex201.

When data of a moving image file linked to a homepage or the like is received in the data communication mode, the data received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexed data is obtained. The demultiplexed data is sent to the demultiplexing unit ex308.
To decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into a bit stream of image data and a bit stream of audio data, and performs synchronization. The coded image data is supplied to the image decoding unit ex309 via the bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 is configured to include the image decoding device described in the present invention, and decodes a bit stream of image data according to the encoding method described in the first to fifth embodiments. By generating the reproduced moving image data by decoding in the method, the reproduced moving image data is supplied to the display unit ex202 via the LCD control unit ex302, whereby the moving image data included in the moving image file linked to the homepage is displayed. You. At this time, at the same time, the audio processing unit ex305 converts the audio data into analog audio data, and then supplies the analog audio data to the audio output unit ex208, whereby, for example, the audio data included in the moving image file linked to the homepage is reproduced. You.

  It should be noted that the present invention is not limited to the example of the above system, and digital broadcasting using satellites and terrestrial waves has recently become a hot topic. As shown in FIG. Either the device or the image decoding device can be incorporated. Specifically, at the broadcasting station ex409, the bit stream of the video information is transmitted to the communication or the broadcasting satellite ex410 via radio waves. The broadcast satellite ex410 receiving this transmits a broadcast radio wave, receives this radio wave with a home antenna ex406 having a satellite broadcast reception facility, and outputs the radio wave to a television (receiver) ex401 or a set-top box (STB) ex407. The device decodes the bit stream and reproduces it. In addition, the image decoding apparatus described in the first to fifth embodiments can be mounted on the reproducing apparatus ex403 that reads and decodes a bit stream recorded on a storage medium ex402 such as a CD or DVD that is a recording medium. It is. In this case, the reproduced video signal is displayed on the monitor ex404. A configuration is also conceivable in which an image decoding device is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on a monitor ex408 of the television. At this time, the image decoding device may be incorporated in the television instead of the set-top box. In addition, a car ex412 having an antenna ex411 can receive a signal from the satellite ex410 or a base station ex107 or the like, and can reproduce a moving image on a display device such as a car navigation ex413 included in the car ex412.

  Furthermore, an image signal can be encoded by the image encoding device described in the first to fifth embodiments and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder for recording an image signal on a DVD disc ex421 or a disc recorder for recording on a hard disk. Furthermore, it can be recorded on the SD card ex422. If the recorder ex420 includes the image decoding device described in the first to fifth embodiments, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

Note that the configuration of the car navigation ex413 can be, for example, a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 from the configuration illustrated in FIG. ) Ex401 etc. are also conceivable.
In addition, the terminal such as the above-mentioned mobile phone ex114 has three mounting formats, in addition to a transmitting / receiving terminal having both an encoder and a decoder, a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.

As described above, the moving picture coding method or the moving picture decoding method described in the first to fifth embodiments can be used for any of the devices and systems described above. The effects described in the first to fifth embodiments can be obtained.
Further, the present invention is not limited to the above embodiment, and various changes or modifications can be made without departing from the scope of the present invention.

INDUSTRIAL APPLICABILITY The image encoding device according to the present invention is useful as an image encoding device provided in a personal computer, a PDA, a digital broadcast station, a mobile phone, and the like having a communication function.
Further, the image decoding device according to the present invention is useful as an image decoding device provided in a personal computer having a communication function, a PDA, an STB for receiving a digital broadcast, a mobile phone, and the like.

FIG. 2 is a block diagram illustrating a configuration of an image encoding device according to the first embodiment. FIGS. 2A and 2B are diagrams illustrating a data structure of a code string generated by the image coding device 800 according to the present embodiment. FIG. 2A is a diagram illustrating an example of a code string when encoding information of a maximum picture size that may be included in a stream. FIG. 2B shows a code for encoding the number of areas that can be secured in the picture memory when the picture memory is divided based on the maximum picture size that may be included in the stream. It is a figure showing an example of a column. FIG. 3 is a diagram illustrating an example of a code string in a case where additional information is described in an area of picture common information in the code strings illustrated in FIGS. 2A and 2B. It is a block diagram showing the composition of the picture decoding device of this embodiment. 5 is a flowchart illustrating a procedure of a decoding process in the image decoding device illustrated in FIG. 4. FIGS. 6A and 6B are diagrams showing a state in which the newly decoded picture is stored in the picture memory according to the procedure of steps S1204 to S1210 in the flowchart of FIG. FIG. 6A shows an example in which there is a sufficient free area in the picture memory to store the decoded picture C to be decoded. FIG. 6B shows an example in which there is not enough free space in the picture memory to store the decoded picture C to be decoded. FIGS. 7A and 7B are diagrams showing a state in which a newly decoded picture is stored in the picture memory according to the procedures of steps S1204 to S1213 in the flowchart of FIG. FIG. 7A shows an example of a case where there is a sufficient free area for storing the decoded picture C in the picture memory. FIG. 7B shows an example in which there is not enough free space to store the decoded picture C in the picture memory. FIGS. 8A, 8B, and 8C are diagrams illustrating a data structure of a code string generated by the image coding device according to the second embodiment. FIG. 8A is a diagram illustrating an example of a code string when information of a minimum picture size that may be included in a stream is encoded. FIG. 8B is a diagram illustrating an example of a code sequence when encoding information on the size of a picture area obtained by dividing a picture memory. FIG. 8C is a diagram illustrating an example of a code string when the number of regions when the picture memory is divided is encoded. FIG. 9A is a diagram illustrating an example of three types of picture sizes encoded in one stream. FIG. 9B is a diagram showing an example of a picture area dividing method for efficiently using a picture memory when one stream includes pictures having the three picture sizes shown in FIG. 9A. It is. FIGS. 10 (a), 10 (b) and 10 (c) show the regions of the picture common information in each code string shown in FIGS. 8 (a), 8 (b) and 8 (c). It is a figure which shows an example of the code string in the case of describing additional information. FIGS. 11A and 11B are diagrams showing a state in which a newly decoded picture is stored in the picture memory according to the procedures of steps S1204 to S1210 in the flowchart of FIG. FIGS. 12A and 12B are diagrams showing a state in which a newly decoded picture is stored in the picture memory according to the procedures of steps S1204 to S1213 in the flowchart of FIG. FIG. 21 is a block diagram illustrating a configuration of an image decoding device according to Embodiment 3 of the present invention. 14 is a flowchart illustrating a procedure of a decoding process in the image decoding device illustrated in FIG. 13. FIGS. 15A and 15B are diagrams showing a state in which newly decoded pictures are stored in the picture memory according to the procedure of the flowchart in FIG. FIGS. 16A and 16B are diagrams showing a state in which a newly decoded picture is stored in the picture memory according to the procedure of the flowchart in FIG. 14 is a flowchart illustrating a procedure of another decoding process in the image decoding device illustrated in FIG. 13. 18 is a flowchart illustrating a more detailed procedure of a process in step S2605 illustrated in FIG. FIGS. 19A and 19B are diagrams showing a state in which newly decoded pictures are stored in the picture memory according to the procedures of the flowcharts in FIGS. FIGS. 20 (a) and 20 (b) are diagrams showing a state in which a newly decoded picture is stored in the picture memory according to the procedure of the flowcharts of FIGS. FIG. 14 is a block diagram illustrating a configuration of an image encoding device according to Embodiment 4. FIG. 14 is a diagram illustrating a data structure of a code string generated by the image coding device according to the fourth embodiment. FIG. 23A shows an example of a physical format of a flexible disk as a recording medium body. FIG. 23B shows the appearance, cross-sectional structure, and flexible disk as viewed from the front of the flexible disk. FIG. 23C shows a configuration for recording and reproducing the program on the flexible disk FD. It is a block diagram showing the whole composition of content supply system ex100 which realizes content distribution service. FIG. 21 is a diagram illustrating a mobile phone ex115 that uses the moving image encoding method and the moving image decoding method described in the above embodiment. FIG. 2 is a block diagram illustrating a configuration of a mobile phone. FIG. 1 is a diagram illustrating an example of a digital broadcasting system. FIG. 14 is a block diagram illustrating a configuration of a conventional image encoding device. FIG. 27 is a block diagram illustrating a configuration of a conventional image decoding device. FIGS. 30A and 30B are diagrams for explaining the reference relation of pictures in an input image. FIG. 30A shows a state where display order pictures are arranged. FIG. 30B shows a state in which pictures are arranged in coding order. 30 is a flowchart showing a storage control operation of a reconstructed image signal in a picture memory in the image decoding device shown in FIG. 29. FIG. 32A shows an example of storage in a case where the size of a picture to be decoded has been changed from small to large, and there is a free area in the picture memory for storing the picture after the size change. FIG. FIG. 32B shows an example of storage in a case where the size of a picture to be decoded has been changed from small to large, and there is no free space in the picture memory for storing the picture after the size change. FIG. FIG. 33 (a) shows a case where the picture size of the current picture to be decoded is not changed and no picture having a picture size different from the current picture to be decoded exists in the picture memory. FIG. 11 is a diagram illustrating a storage example in a case where there is a free area in the picture memory for storing. FIG. 33B shows the case where the picture size of the current picture to be decoded is not changed and no picture having a picture size different from the current picture to be decoded exists in the picture memory. FIG. 14 is a diagram showing a storage example when there is no free area in the picture memory for storing. FIG. 34A shows a case where the picture size of the current picture to be decoded is not changed and a picture having a picture size different from that of the newly decoded picture C exists in the picture memory. FIG. 11 is a diagram showing a storage example in a case where there is a vacant area in the picture memory for storing the selected picture C. FIG. 34B shows a case where the picture size of the current picture to be decoded is not changed and a picture having a picture size different from the newly decoded picture C exists in the picture memory. FIG. 14 is a diagram illustrating a storage example in a case where there is no free area in the picture memory for storing a converted picture C;

Explanation of reference numerals

800 image encoding device 101 picture memory 102 prediction residual encoding unit 103 code sequence generation unit 104 prediction residual decoding unit 105 picture memory 106 motion vector detection unit 107 motion compensation encoding unit 108 motion vector storage unit 109 picture memory control Unit 112 difference operation unit 113 addition operation unit 114 switch 115 switch 201 code string analysis unit 202 prediction residual decoding unit 203 picture memory 204 motion compensation decoding unit 205 motion vector storage unit 206 picture memory control unit 207 picture size change determination unit 209 Deletion picture region determination unit 210 Picture memory division method designating unit 210A Address division unit 211 Addition operation unit 212 Switch 801 Picture memory division information encoding unit 1100 Image decoding device 2000 Image decoding Equipment

Claims (33)

An image decoding apparatus for decoding a code string including moving image streams of different picture sizes with reference to decoded pictures,
A picture memory having a storage area for storing decoded pictures,
Memory partitioning means for partitioning the storage area of the picture memory into partitioned areas of a predetermined size;
Storage means for continuously storing all data of one newly decoded picture in a continuous storage area of the picture memory comprising one or more of the divided areas. apparatus. The image decoding apparatus further includes:
A size change determination unit that determines whether the size of the picture has been changed by comparing the size of the newly decoded picture with the size of the picture that was decoded immediately before,
The memory partitioning means is configured to, when it is determined that the size of the newly decoded picture has been changed, store the size of the partitioned area to be the size of the newly decoded picture. 2. The image decoding apparatus according to claim 1, wherein the image decoding apparatus divides an area. The storage unit includes a first free area detection unit that detects whether at least one of the divided areas is a free area,
The storage means, when at least one of the divided areas is detected as a free area, stores all data of one newly decoded picture in a continuous storage area including one of the detected divided areas. 3. The image decoding apparatus according to claim 2, wherein is stored continuously. The storage unit includes a first picture deletion unit that generates the empty area by deleting the decoded pictures already stored in the picture memory in order of display order from the oldest one,
The storage means, when one of the divided areas is not detected as a free area, causes the first picture deleting unit to delete the decoded picture until at least one of the partitioned areas is detected as a free area. 4. The image decoding device according to claim 3, wherein: The storage unit is further configured to, if the decoded picture already stored in the picture memory is deleted before being displayed, determine that the decoded picture cannot be displayed when displayed. The image decoding device according to claim 4, further comprising a possibility determination unit. The storage means includes a partitioned area selection unit that selects the partitioned area in order from the top of the picture memory,
3. The image according to claim 2, wherein the storage unit continuously stores all data of one newly decoded picture in a continuous storage area including the selected divided areas. Decryption device. A storage unit configured to detect whether or not the selected partitioned area is a free area;
A second picture deletion unit that makes the selected partitioned area a free area by deleting a decoded picture stored in the partitioned area when the selected partitioned area is not a free area,
7. The image according to claim 6, wherein the storage unit continuously stores all data of one newly decoded picture in a continuous storage area including the selected divided areas. Decryption device. A storing unit configured to, when one of the partitioned areas is not detected as a free area, select a partitioned area in order from the top of the picture memory;
A second free space detection unit that detects whether the selected partitioned area is a free space,
A second picture deletion unit that makes the selected partitioned area a free area by deleting a decoded picture stored in the partitioned area when the selected partitioned area is not a free area,
4. The image according to claim 3, wherein the storage unit continuously stores all data of one newly decoded picture in a continuous storage area including the selected divided areas. Decryption device. The image decoding apparatus further includes:
A reading unit that reads, from the code string, partition information indicating a partition method of the storage area,
The image decoding apparatus according to claim 1, wherein the memory partitioning unit partitions the storage area according to the read partition information. The partition information indicates a maximum picture size of a video stream included in the code string,
10. The image decoding apparatus according to claim 9, wherein the memory partitioning unit partitions the storage area such that the size of each of the partition areas has a picture size indicated in the partition information. The partition information indicates the number of partition areas when the size of the storage area is partitioned by the maximum picture size,
The image decoding device according to claim 9, wherein the memory partitioning unit partitions the storage area into the number of the partition areas. The partition information indicates a minimum picture size of a video stream included in the code sequence,
10. The image decoding apparatus according to claim 9, wherein the memory partitioning unit partitions the storage area such that the size of each of the partition areas has a picture size indicated in the partition information. The partition information indicates the number of partition areas when the size of the storage area is partitioned by the minimum picture size,
The image decoding device according to claim 9, wherein the memory partitioning unit partitions the storage area into the number of the partition areas. The partition information indicates the greatest common divisor of the picture size of the video stream included in the code sequence,
10. The image decoding apparatus according to claim 9, wherein the memory partitioning unit partitions the storage area such that the size of each of the partition areas has a picture size indicated in the partition information. 10. The image decoding apparatus according to claim 9, wherein the partition information is rounded up so that at least the same macroblock is stored in the same partition area. 10. The image decoding according to claim 9, wherein the partition information is rounded up so that all macroblocks for one column in the horizontal direction of each decoded picture are stored in one partition area. Device. 10. The image decoding apparatus according to claim 9, wherein the memory partitioning unit rounds up the size of the partitioned area so that at least the same macroblock is stored in the same partitioned area. The memory partitioning unit rounds up the size of the partitioned area so that all the macroblocks for one column in the horizontal direction of each decoded picture are stored in one partitioned area. 10. The image decoding device according to claim 9. The classification information further includes information indicating whether or not the decoded picture already stored in the picture memory may be deleted before being displayed. An image decoding apparatus according to claim 9. The storage unit includes a first free area detection unit that detects whether at least one of the partitioned areas in the picture memory is a free area,
The storage means is capable of continuously storing all the data of one newly decoded picture, and when a continuous storage area including one or more of the divided areas is detected as a free area. 10. The image decoding apparatus according to claim 9, wherein all the data of one newly decoded picture are continuously stored in a continuous storage area including the detected divided areas. The storage unit includes a first picture deletion unit that generates the empty area by deleting the decoded pictures already stored in the picture memory in order of display order from the oldest one,
The storage means, when a segmented area for continuously storing all the data of one newly decoded picture is not detected as a free area, stores all of the data of the newly decoded picture. 21. The image decoding apparatus according to claim 20, wherein the first picture deletion unit deletes the decoded picture until the divided area in which only data is continuously stored is detected as a free area. Device. An image encoding apparatus that encodes moving image streams of different picture sizes to generate a code string while referring to encoded pictures,
Picture size obtaining means for obtaining a picture size of a moving image stream included in the generated code sequence;
Division information generating means for generating division information indicating a division method of the picture memory on the image decoding device side based on the obtained picture size;
Encoding means for generating a code string including the generated section information. 23. The image encoding apparatus according to claim 22, wherein the partition information generating unit generates the partition information indicating a maximum picture size among the acquired picture sizes. 23. The image encoding apparatus according to claim 22, wherein the partition information generating unit generates the partition information indicating a minimum picture size among the acquired picture sizes. An image decoding method for storing a decoded picture in a picture memory, and decoding a code string including a video stream of a different picture size with reference to the decoded picture,
A memory partitioning step of partitioning a storage area of the picture memory into partition areas of a predetermined size;
Storing the entire data of one newly decoded picture in a continuous storage area of the picture memory, the storage area comprising one or more of the divided areas. Method. The image decoding method further includes:
A resizing determination step of determining whether the size of the picture has been changed by comparing the size of the newly decoded picture with the size of the previously decoded picture,
In the memory partitioning step, when it is determined that the size of the newly decoded picture has been changed, the storage is performed such that the size of the partition area becomes the size of the newly decoded picture. 26. The image decoding method according to claim 25, wherein the area is divided. The storing step includes a first empty area detecting step of detecting whether at least one of the divided areas is an empty area,
In the storing step, when at least one of the divided areas is detected as a free area, all data of one newly decoded picture is stored in a continuous storage area including one of the detected divided areas. 27. The image decoding method according to claim 26, further comprising: The storing step includes a first picture deleting step of generating the empty area by deleting the decoded pictures already stored in the picture memory in order of display order from the oldest one,
In the storing step, when one of the partitioned areas is not detected as a free area, the decoded picture is deleted in the first picture deleting step until at least one of the partitioned areas is detected as a free area. 28. The image decoding method according to claim 27, wherein: The storing step includes a sectioned area selecting step of sequentially selecting the sectioned area from the top of the picture memory,
27. The image according to claim 26, wherein, in the storing step, all data of one newly decoded picture is continuously stored in a continuous storage area including the selected divided areas. Decryption method. The storage step includes a second free area detection step of detecting whether the selected section area is a free area;
A second picture deletion step of making the selected partition area a free area by deleting a decoded picture stored in the partition area if the selected partition area is not a free area,
30. The image according to claim 29, wherein, in the storing step, all data of one newly decoded picture is continuously stored in a continuous storage area including the selected divided areas. Decryption method. An image encoding method for encoding a moving image stream of a different picture size to generate a code sequence while referring to an encoded picture,
A picture size obtaining step of obtaining a picture size of a moving image stream included in the generated code sequence;
A partition information generating step of generating partition information indicating a partition method of a picture memory on the image decoding apparatus side based on the obtained picture size;
An encoding step of generating a code string including the generated section information. A program for an image decoding device that stores a decoded picture in a picture memory, and decodes a code string including a video stream of a different picture size with reference to the decoded picture, A computer memory partitioning step of partitioning the storage area of the picture memory into partition areas of a predetermined size;
A storage step of continuously storing all data of one newly decoded picture in a continuous storage area of the picture memory, the storage area including one or more of the divided areas. A program for an image encoding device that encodes moving image streams of different picture sizes to generate a code sequence while referring to an already-encoded picture, the moving image being included in a code sequence generated by a computer. A picture size obtaining step for obtaining a picture size of the stream;
A partition information generating step of generating partition information indicating a partition method of a picture memory on the image decoding apparatus side based on the obtained picture size;
And an encoding step of generating a code string including the generated segment information.

Images