JP2009267537A - Multiplexing device for hierarchized elementary stream, demultiplexing device, multiplexing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To multiplex a hierarchized elementary stream so as to be normally decoded. <P>SOLUTION: A multiplexing device 502 includes: a packet generating part 503 for making an SVC stream 501, which is hierarchized into a basic hierarchy and extension hierarchies 1, 2, into packets, and for imparting identification information different by hierarchy to the packets; and a packet multiplexing position control part for exchanging at least one of the packets of the basic hierarchy with at least one of the packets of the extension hierarchies 1, 2 within the range of the access unit of the SVC stream 501, and for multiplexing the SVC stream 501. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a layered elementary stream multiplexing apparatus, demultiplexing apparatus, multiplexing method, and program.

  H. H.264 | MPEG-4 AVC (hereinafter referred to as “AVC”), which is an extension standard for scalable video coding (hereinafter referred to as “SVC”), is an image quality / resolution / frame rate for video data encoded by AVC. This is a method of encoding extension data that improves at least one of the above (see Non-Patent Document 1). When the AVC portion (hereinafter referred to as “AVC stream”) is separated from the bit stream including the extension by SVC (hereinafter referred to as “SVC stream”), it can be reproduced by a conventional AVC decoder. Thus, backward compatibility is considered in the SVC stream. In many applications, a video stream needs to be played back in synchronization with a stream such as audio. In order to realize this, multiplexing is performed in accordance with the MPEG-2 Systems standard (see Non-Patent Document 2). However, the MPEG-2 Systems standard does not define a method for multiplexing SVC streams.

  In the MPEG-2 Systems standard, a packet identifier (PID or stream_id) is used to identify each elementary stream to be multiplexed. Considering the MPEG-2 Systems standard, in order to multiplex an SVC stream including an AVC stream and one or more extension streams based on SVC according to the MPEG-2 Systems standard, it is considered that a packet identifier is assigned to each stream. It is done. As shown in FIG. 15, a 30 Mbps AVC stream (AVC_1) belonging to the base layer, a 7 Mbps first extension stream (SVC_1) belonging to the extension layer, and a 3 Mbps second extension stream (SVC_2) belonging to the extension layer. Suppose that a total of 40 Mbps SVC streams including the above are multiplexed. AVC_1, SVC_1, and SVC_2 have a data structure organized for each NAL unit (nal_unit). When multiplexing is performed without changing the order, as shown by reference numeral 101, each picture is multiplexed in a burst manner at 40 Mbps. Looking at only AVC_1, it can be seen that multiplexing is not performed during the period of SVC_1 and SVC_2, as indicated by reference numeral 102.

One of the purposes of the SVC standard is to ensure backward compatibility. When multiplexing is performed, if it is an AVC stream, a packet multiplexed at 30 Mbps is input. In this case, a decoder that reproduces only the AVC stream can be designed with the upper limit of the input bit rate being 30 Mbps. However, when multiplexed as an SVC stream, the AVC stream is input in bursts at 40 Mbps, and cannot be reproduced by this decoder. On the other hand, on the multiplexing side, as indicated by reference numeral 201 in FIG. 16, by interleaving each packet of AVC_1, SVC_1, and SVC_2 in a fine unit, a buffer defined in the MPEG-2 Systems standard (FIG. 17 transport buffers TB) can absorb the burstiness. In FIG. 17, when an AVC decoder is connected, only AVC_1 is separated by filtering with an identifier, and SVC_1 and SVC_2 are removed. In this case, it can be said that the data structure has backward compatibility.
SVC Verification Test Report, ISO / IEC JTC 1 / SC 29 / WG 11 N9577, Antalya, TR-January 2007 ISO / IEC 13818-1: 2007

  However, in FIG. 17, when the SVC decoder is connected, the AVC_1, SVC_1, and SVC_2 packets are interleaved as shown in FIG. Therefore, there is a problem in that when the elementary stream is written to the EB (SVC) of the SVC decoder in the order in which the multiplexed data is input, the SVC decoder cannot decode normally.

  The present invention has been made to solve the above-described problems, and it is an object of the present invention to multiplex a hierarchical elementary stream so that it can be normally decoded.

  In order to achieve the above object, a multiplexing apparatus according to the present invention packetizes an elementary stream layered into a base layer and at least one enhancement layer, and uses different identification information for each layer in the packet. A packet that multiplexes the elementary stream by exchanging at least one of the base layer packet and at least one of the enhancement layer packet within the range of the packet generator to be assigned and the access unit of the elementary stream. A multiplexing position control unit.

  According to the present invention, hierarchical elementary streams can be multiplexed so that they can be normally decoded.

  Hereinafter, embodiments of the present invention will be described.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of a multiplexing apparatus according to the first embodiment of the present invention. The multiplexing apparatus 502 according to the present embodiment includes a packet generation unit 503, a packet multiplexing position control unit 504, a dependency relationship information generation / multiplexing unit 505, an output changeover switch 509, and an input changeover switch 511. Prepare.

  The packet generation unit 503 divides the elementary stream of each layer of the SVC stream 501 into a plurality of packets (packetizes). The packet generation unit 503 adds different identification information (PID) for each layer to the header of each packet.

  The packet multiplexing position control unit 504 controls the timing at which the SVC stream 501 is multiplexed, and switches the base layer packet and the extended layer 1 and 2 packets.

  The dependency relationship information generation / multiplexing unit 505 generates dependency relationship information indicating the dependency relationship between the extended layer and each layer of the SVC stream 501. The dependency relationship information generating / multiplexing unit 505 packetizes the dependency relationship information. The extension hierarchy is dependent on the base hierarchy. However, there are cases where there is a dependency relationship between the extension layers and cases where there is no dependency relationship. For example, when one layer has a dependency relationship with the other layer, one layer cannot be decoded without the other layer. Note that the order of packets is not changed within the same hierarchy. The dependency relationship information can be described in, for example, data transmitted in a transport stream such as PSI (Program Specific Information) and SI (Service Information), data transmitted in a program stream such as descriptor, and the like.

  The output changeover switch 509 transmits a packet generated by the packet generation unit 503 and a packet of dependency relationship information generated by the dependency relationship information generation / multiplexing unit 505 to a control signal 507 from the packet multiplexing position control unit 504. Switch according to.

  The input changeover switch 511 switches the hierarchy of the SVC stream 501 input to the packet generation unit 503 according to the control signal 506 from the packet multiplexing position control unit 504. In FIG. 1, two enhancement layers are shown in the SVC stream 501, but the present invention is not limited to this, and may be one or more enhancement layers.

  FIG. 2 is a flowchart showing the operation of the multiplexing apparatus according to the first embodiment of the present invention.

  In step S601, the packet multiplexing position control unit 504 determines a packet to be generated. When generating a packet indicating the dependency relationship information, the dependency relationship information generating / multiplexing unit 505 is activated by the activation signal 507, and the process proceeds to step S602. When a base layer packet (base layer packet) is generated, the packet generation unit 503 is started by the start signal 506, and the process proceeds to step S603. When generating a packet for the extension layer 1 (packet for the extension layer 1), the packet generation unit 503 is activated by the activation signal 506, and the process proceeds to step S604. When generating a packet for the extension layer 2 (packet for the extension layer 2), the packet generation unit 503 is activated by the activation signal 506, and the process proceeds to step S605.

  In step S602, the dependency relationship information generating / multiplexing unit 505 generates a packet indicating the dependency relationship information.

  In step S603, the packet generation unit 503 generates a base layer packet.

  In step S604, the packet generation unit 503 generates an extension layer 1 packet.

  In step S605, the packet generation unit 503 generates an extension layer 2 packet.

  In step S606, the packet multiplexing position control unit 504 sends an activation signal 507 to control the output changeover switch 509. When a packet indicating the dependency relationship information is generated, the packet multiplexing position control unit 504 connects the output changeover switch 509 to the dependency relationship information generating / multiplexing unit 505. Packets indicating the dependency relationship information are multiplexed and output from the output 510. When the base layer packet, the extension layer 1 packet, or the extension layer 2 packet is generated, the packet multiplexing position control unit 504 connects the output changeover switch 509 to the packet generation unit 503. The base layer packet, the extended layer 1 packet, or the extended layer 2 packet is multiplexed and output from the output 510.

  In step S607, the packet multiplexing position control unit 504 determines whether or not multiplexing of the elementary streams of all layers in the access unit has been completed. If the multiplexing of elementary streams of all layers has not been completed (“N” in step S607), the process returns to step S601 to determine the next packet to be generated. When multiplexing of elementary streams of all layers is completed (“Y” in step S607), the multiplexing process is terminated.

  As illustrated in FIG. 3, the packet multiplexing position control unit 504 performs interleaving (replacement) between the base layer packet and the extended layer 1 and 2 packets in units of access units of the SVC stream 501. As shown in FIG. 4, when attention is paid to the packet of the base layer, it can be seen that all the base layer data in the access unit are gathered at the break of the access unit. Therefore, the separation device stores the packets of the extension layer 1 and the extension layer 2 in the buffer, and when the packets of the access units of all the layers are prepared, if the SVC stream is written from the buffer to the input buffer of the decoder, Can be obtained.

  FIG. 5 is a block diagram showing a configuration of a demultiplexer that demultiplexes data multiplexed by the multiplexer according to the first embodiment of the present invention.

  The separation device 901 includes an identification information interpretation unit 902, a dependency relationship extraction unit 903, a base layer packet separation unit 904, an enhancement layer 1 packet separation unit 905, an enhancement layer 2 packet separation unit 906, and an access unit. A completion detection unit 907, a first order change buffer 909, a second order change buffer 910, and an input changeover switch 914 are provided. Separation device 901 is communicably connected to an external hierarchical coding decoder 915.

  The identification information interpretation unit 902 interprets the identification information given to the header of the packet, and identifies the type of packet (for example, the type of program, the type of data such as video / audio) and the hierarchy.

  The dependency relationship extraction unit 903 extracts dependency relationship information described in data transmitted by a transport stream or the like. Hereinafter, description will be made assuming that transmission is performed using a transport stream.

  The base layer packet separator 904 separates the base layer packet from the SVC transport stream obtained by multiplexing the SVC stream by the multiplexing device of the present invention, and extracts the payload portion of the separated base layer packet, A base layer elementary stream is extracted. The base layer packet separation unit 904 writes the extracted base layer elementary stream into the input buffer 916 of the layer coding decoder 915.

  The enhancement layer 1 packet separation unit 905 extracts the enhancement layer 1 elementary stream by separating the enhancement layer 1 packet from the SVC transport stream and extracting the separated payload portion of the enhancement layer 1 packet. . The enhancement layer 1 packet separator 905 stores the extracted enhancement layer 1 elementary stream in the first order change buffer 909.

  The enhancement layer 2 packet separator 906 separates the enhancement layer 2 packet from the SVC transport stream, and extracts the extension layer 2 elementary stream by extracting the payload portion of the separated enhancement layer 2 packet. To do. The enhancement layer 2 packet separator 906 stores the extracted enhancement layer 2 elementary stream in the second order change buffer 910.

  The access unit completion detection unit 907 receives the completion signal 908 from the enhancement layer 1 packet separation unit 905 and the enhancement layer 2 packet separation unit 906, respectively. The access unit completion detection unit 907 detects whether or not elementary streams of all layers for the currently separated access unit have been acquired according to the received completion signal 908.

  Here, it is known in advance that the packet for extension layer 1 and the packet for extension layer 2 in each access unit are output after the base layer packet. Therefore, the access unit completion detection unit 907 does not need to receive a completion signal from the base layer packet separation unit 904. The access unit completion detection unit 907 may receive a completion signal from the base layer packet separation unit 904 as necessary.

  The first order changing buffer 909 stores the packets of the enhancement layer 1 elementary stream separated by the enhancement layer 1 packet separation unit 905 in the order of input.

The second order change buffer 910 stores the packets of the enhancement layer 2 elementary stream separated by the enhancement layer 2 packet separation unit 906 in the order of input.

  The input changeover switch 914 receives an output switch control signal 913 described later. The input changeover switch 914, according to the received output switch control signal 913, stores the enhancement layer 1 elementary stream and the enhancement layer 2 element stored in the first order change buffer 909 and the second order change buffer 910. The mental stream is written into the input buffer 916 of the hierarchical coding decoder 915. The input changeover switch 914 is connected to the base layer packet separator 904 before receiving the output switch control signal 913. Therefore, the input changeover switch 914 writes the base layer packet input to the base layer packet separator 904 into the input buffer 916 as it is.

  The SVC decoder 917 reads and decodes the base layer elementary stream, the enhancement layer 1 elementary stream, and the enhancement layer 2 elementary stream written to the input buffer 916.

  FIG. 6 is a flowchart showing the operation of the separation apparatus according to the first embodiment of the present invention.

  In step S1001, the identification information interpretation unit 902 interprets the identification information given to the header of the input multiplexed data (packet) and identifies the type and hierarchy of the packet. If a packet indicating dependency information is identified, the process proceeds to step S1002. If the base layer packet is identified, the process proceeds to step S1003. If the packet for enhancement layer 1 is identified, the process proceeds to step S1004. If an extension layer 2 packet is identified, the process advances to step S1005.

  In step S1002, the identification information interpretation unit 902 sends a switching signal 911 to the input switch 912. The input switch 912 is connected to the dependency relationship extraction unit 903 according to the switching signal 911. The identification information interpretation unit 902 extracts dependency relationship information from the SVC transport stream sent to the dependency relationship extraction unit 903.

  In step S <b> 1003, the identification information interpretation unit 902 sends a switching signal 911 to the input switch 912. The input switch 912 is connected to the base layer packet separator 904 according to the switching signal 911. The base layer packet separation unit 904 separates the base layer packet from the input SVC transport stream, and writes the payload portion of the separated base layer packet to the input buffer 916 via the input changeover switch 914.

  In step S1004, the identification information interpretation unit 902 sends a switching signal 911 to the input switch 912. The input switch 912 is connected to the extension layer 1 packet separator 905 according to the switching signal 911. The extension layer 1 packet separator 905 separates the extension layer 1 packet from the input SVC transport stream, and writes the payload portion of the separated base layer packet in the first order change buffer 909.

  In step S1005, the identification information interpretation unit 902 sends a switching signal 911 to the input switch 912. The input switch 912 is connected to the extension layer 2 packet separator 906 according to the switching signal 911. The enhancement layer 2 packet separation unit 906 separates the enhancement layer 2 packet from the input SVC transport stream, and writes the payload portion of the separated base layer packet in the second order change buffer 910.

  In step S1006, the access unit completion detection unit 907 determines whether or not the separation of all the enhancement layer elementary streams belonging to the access unit is finished when the separation of one packet is finished. When the access unit completion detection unit 907 receives the completion signal 908 from both the enhancement layer 1 packet separation unit 905 and the enhancement layer 2 packet separation unit 906, the access unit completion detection unit 907 determines that the separation of all the enhancement layer elementary streams has been completed. Judgment is made (“Y” in step S1006). Next, the access unit completion detection unit 907 outputs the output switch control signal 913 to the input changeover switch 914, and proceeds to step S1007. If the access unit completion detection unit 907 determines that the separation of all the enhancement layer elementary streams belonging to the access unit is not completed ("N" in step S1006), the process returns to step S1001.

  In step S1007, the input changeover switch 914 is connected to the first order change buffer 909 or the second order change buffer 910 in accordance with the output switch control signal 913. The elementary stream of each layer stored in the first order change buffer 909 or the second order change buffer 910 is stored in the input buffer 916 of the layer coding decoder 915.

  In step S1008, the identification information interpretation unit 902 determines whether or not all the multiplexed data has been separated. If the identification information interpretation unit 902 determines that all of the multiplexed data has been separated (“Y” in step S1008), the separation of the multiplexed data is finished. If the identification information interpretation unit 902 determines that the separation of all multiplexed data has not been completed (“N” in step S1008), the process returns to step S1001.

  As described above, in the present embodiment, at least one of the base layer packets and at least one of the enhancement layer packets are switched to multiplex the elementary streams within the range of the access units of the layered elementary streams. Turn into. As a result, a configuration that eliminates the need for a buffer for storing packets in the base layer on the separation device side is possible, and the amount of memory used can be suppressed.

(Second Embodiment)
FIG. 7 is a block diagram showing a multiplexing apparatus according to the second embodiment of the present invention. In each part of this embodiment, the same reference numerals are given to the same parts as those of the first embodiment of FIG.

  The multiplexing device 1101 according to the second embodiment is related to the first embodiment in that it includes a dependency information generation / multiplexing unit 1102 instead of the dependency relationship information generation / multiplexing unit 505 of FIG. This is different from the multiplexing device 502. Hereinafter, differences between the dependency information generation / multiplexing unit 1102 and other configurations associated therewith will be mainly described.

  The dependency relationship information generation / multiplexing unit 1102 generates dependency relationship information representing the dependency relationship between the layers of the SVC stream 501. The dependency relationship information generating / multiplexing unit 1102 packetizes the dependency relationship information. The dependency relationship information generation / multiplexing unit 1102 includes an order change information generation / multiplexing unit 1103. The order replacement information generation / multiplexing unit 1103 generates order replacement information (flag) indicating that the packet replacement has been performed when generating a packet of a preset hierarchy or a hierarchy that is dependent on the hierarchy. . The order change information generating / multiplexing unit 1103 packetizes the order change information and multiplexes it into the SVC stream 501. The order change information can be described in, for example, data transmitted in a transport stream such as PSI (Program Specific Information), SI (Service Information), or data transmitted in a program stream such as descriptor.

  The output changeover switch 509 includes a packet generated by the packet generation unit 503, a packet of dependency relationship information generated by the dependency relationship information generation / multiplexing unit 1102, and a relationship information generation / multiplexing unit 1103 The packet of the order change information is switched according to the control signal 507 from the packet multiplexing position control unit 504.

  FIG. 8 is a flowchart showing the operation of the multiplexing apparatus according to the second embodiment of the present invention. The operation of the multiplexing apparatus 1101 according to the second embodiment is different from the operation of the multiplexing apparatus according to the first embodiment of FIG. 2 in that step S1202 is included instead of step S602 of FIG. Hereinafter, differences between step S1202 and other steps accompanying this will be mainly described.

  In step S1201, the packet multiplexing position control unit 504 determines a packet to be generated. When generating a packet indicating the dependency relationship information and the order change information, the dependency relationship information generating / multiplexing unit 1102 is started according to the start signal 507 from the packet multiplexing position control unit 504.

  In step S1202, the dependency relationship information generating / multiplexing unit 1102 generates dependency relationship information. If the order change information generation / multiplexing unit 1103 determines that the layer in which the packet replacement is performed is included in either a preset layer or a layer having a dependency relationship with the layer, the packet order change is performed. The order change information (flag) indicating that there has been. The dependency relationship information and the order change information are multiplexed in the SVC stream 501.

  In step S1206, the packet multiplexing position control unit 504 sends an activation signal 507 to control the output changeover switch 509. When generating a packet indicating the dependency relationship information and the order change information, the packet multiplexing position control unit 504 connects the output changeover switch 509 to the dependency relationship information generating / multiplexing unit 1102. Packets indicating the dependency relationship information and the order change information are multiplexed and output from the output 510. When the base layer packet, the extension layer 1 packet, or the extension layer 2 packet is generated, the packet multiplexing position control unit 504 connects the output changeover switch 509 to the packet generation unit 503. The base layer packet, the extended layer 1 packet, or the extended layer 2 packet is multiplexed and output from the output 510.

  FIG. 9 is a diagram showing an arrangement of packets multiplexed so that the order is not changed during interleaving up to the base layer and the extended layer 1 which is a layer above it. The hierarchical coding decoder 915 that decodes the SVC transport stream up to the enhancement layer 1 may discard the packet of the enhancement layer 2. Therefore, the elementary streams to be separated are in the order of the original elementary streams as shown in FIG. Accordingly, even if the packets of the base layer and the extended layer 1 are separated and written to the input buffer 916 of the layer encoding decoder 915, the SVC stream is correctly ordered.

  FIG. 11 is a block diagram showing a demultiplexer that demultiplexes data multiplexed by the multiplexer according to the second embodiment of the present invention. In each part of the present embodiment, the same parts as those in the first embodiment in FIG.

  The separation device 1501 according to the second embodiment is different from the separation device 901 according to the first embodiment in that a dependency relationship / order replacement information extraction unit 1502 is provided instead of the dependency relationship extraction unit 903 of FIG. To do. Further, the separation device 1501 according to the second embodiment is different from the separation device 901 according to the first embodiment in that it includes a third order change buffer 1503.

  The dependency / order replacement information extraction unit 1502 extracts dependency information between identification information and each layer and order replacement information of each layer. The third order change buffer 1503 is used for elementary stream elementary streams. When the order change information indicates that the third order change buffer 1503 is not used, the separated elementary stream of each layer is not written to the third order change buffer 1503. As a result, the elementary stream of each layer is directly written in the input buffer 916.

  In the description of the operation so far, the device that separates and decodes all the layers of the basic layer, the extended layer 1, and the extended layer 2 has been described. However, it is also possible to realize a separation / decoding device that handles only a part of layers.

  In the case of a separation / decoding device that handles only up to the extension layer 1, the dependency / order change information extraction unit 1502 extracts only the order change information for the extension layer 1. The extension layer 2 packet separator 906 and the subsequent second order change buffer 910 and switch 1504 are not necessary. When the order change information for the extended hierarchy 1 indicates that there is a change, the buffer presence / absence selection switch 1504 for the basic hierarchy and the extended hierarchy 1 uses the first order change buffer 909 and the third order change buffer 1503. Select the output from. When the order replacement information for the extension layer 1 indicates no replacement, the buffer presence / absence selection switch 1504 for the base layer and the extension layer 1 performs the base layer use according to the order in which the multiplexed data packets are input. The output is selected from the packet separator 904 and the packet separator 905 for the enhancement layer 1. As a result, in a specific system, when it can be guaranteed that the order replacement information for the extended hierarchy 1 indicates no replacement, it is not necessary to allocate memory to the first order replacement buffer 909, and the memory usage amount Can be suppressed.

  In the case of the separation / decoding device that handles up to the extension layer 2, the dependency / order change information extraction unit 1502 extracts the order change information for the extension layer 1 and the extension layer 2. When the order replacement information for the extended hierarchy 1 indicates that there is a replacement, the order replacement information for the extension hierarchy 2 also indicates that there is a replacement. A buffer presence / absence selection switch 1504 for the base layer, extended layer 1 and extended layer 2 selects the output from the third order change buffer 1503. When the order replacement information for the extended hierarchy 1 indicates no replacement, the operation differs depending on whether the order for the extended hierarchy 2 is changed. When the order for the extension layer 2 is changed, the buffer selection switch 1504 for the base layer and the base layer 1 selects the output from the base layer packet separator 904 and the base layer 1 packet separator 905. . The buffer presence / absence selection switch 1504 for the extended hierarchy 2 selects the output from the second order change buffer 910. On the other hand, when there is no order change for the extended layer 2, the buffer presence / absence selection switch 1504 for the base layer, the extended layer 1 and the extended layer 2 follows the order in which the packets of multiplexed data are input. The outputs from the first order changing buffer 909, the second order changing buffer 910, and the third order changing buffer 1503 are selected. As a result, in a specific system, when it is guaranteed that the order replacement information for the extended hierarchy 1 and the extended hierarchy 2 indicates no replacement, it is not necessary to allocate a memory to the third order replacement buffer 1503. Thus, memory usage can be reduced.

  This embodiment will be described in comparison with the case of adding an order changing buffer RB for changing the order of packets shown in FIG. As shown in FIG. 12, a system with high flexibility can be configured by adding an order changing buffer RB for changing the order of packets to all of the basic layer, the extended layer 1 and the extended layer 2. However, in a specific application, at least some of the order replacement buffers RB are unnecessary, and the order replacement buffer RB may be wasted. For example, consider a system using an optical disk such as a DVD. The data reading speed from the optical disk is limited by physical characteristics. Therefore, in consideration of backward compatibility with a system that has already been commercialized, the sum of the basic hierarchy, the extended hierarchy 1 and the extended hierarchy 2 needs to be a certain value or less. For example, when there is a product that can reproduce only an AVC stream with an upper limit of 30 Mbps, the reading speed from the optical disc is 30 Mbps even when the SVC stream is used. For this reason, the AVC stream is, for example, 25 Mbps, and the remaining 5 Mbps is allocated to the extended portions (extended layer 1 and extended layer 2) by SVC. In this case, the problem as shown in FIG. 15 does not occur.

  In the current MPEG-2 Systems standard, there is no information for notifying the decoder side of the necessity of RB. If the MPEG-2 Systems standard is simply extended to allow multiplexing of SVC streams, a reordering buffer (RB) is always required in certain systems, even though RB is not required. . As a result, there is a risk of increasing the memory usage. Also, it is considered difficult to define conformance to the MPEG-2 Systems standard for an LSI that can be applied only to a system that does not require RB.

  On the other hand, in the present embodiment, since information indicating whether or not RB is necessary is multiplexed when the SVC stream is multiplexed, it is determined whether the multiplexed data has the problem shown in FIG. You can know on the side. Therefore, the SVC transport stream can be reproduced without allocating memory so as to secure RB under specific conditions. When a specific system is considered, even if there is no information for notifying the decoder side of the necessity of RB according to the MPEG-2 Systems standard, it is possible to ensure compatibility of data reproduction in the system. . However, the SVC decoder LSI used in the system is highly likely to be used for other purposes. Support for RB necessity information according to the MPEG-2 Systems standard is useful when confirming conformance of LSI standards. Moreover, it can be a standard that restricts the degree of freedom a little on the multiplexing side.

  FIG. 13 and FIG. 14 are diagrams illustrating the structure of data (descriptor) indicating information related to hierarchical coding included in packets generated by the dependency relationship information generation / multiplexing unit 1102 and the order change information generation / multiplexing unit 1103. It is. These descriptors describe the type of hierarchical coding (time, space, quality) and the dependency relationship of each layer. In the present invention, a 1-bit field called reordering_buffer_flag indicated by an underline indicates whether or not an order change buffer is necessary in all layers between the layer indicated by the descriptor and the basic layer. The name of the flag and the position of the 1-bit field in the descriptor are not limited to those shown in FIGS. 13 and 14 as long as they can indicate whether or not the order changing buffer that is the object of the present invention is necessary.

  Note that the configurations and methods in the above embodiments and modifications may be executed by a computer by a program.

  Further, the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

It is a block diagram which shows the structure of the multiplexing apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the multiplexing apparatus which concerns on the 1st Embodiment of this invention. It is a figure explaining the correspondence of the hierarchical elementary stream and the multiplexed data. It is a figure explaining the correspondence of the multiplexed data with the basic layer stream in the layered elementary stream. It is a block diagram which shows the structure of the isolation | separation apparatus which isolate | separates the data multiplexed by the multiplexing apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the separation apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the multiplexing apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the multiplexing apparatus which concerns on the 2nd Embodiment of this invention. It is a figure explaining the correspondence of the hierarchical elementary stream and the multiplexed data. It is a figure explaining the correspondence of the data multiplexed with the base layer stream in the layered elementary stream, and the stream for enhancement layer 1. It is a block diagram which shows the structure of the isolation | separation apparatus which isolate | separates the data multiplexed by the multiplexing apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the isolation | separation apparatus using the buffer for order change. It is a figure which shows an example of the data containing order replacement information. It is a figure which shows another example of the data containing order replacement information. It is a figure which shows the bit rate of the hierarchical elementary stream in the conventional multiplexing apparatus. It is a figure which shows the bit rate of the hierarchical elementary stream in the conventional multiplexing apparatus. It is a figure explaining the subject at the time of isolate | separating the elementary stream hierarchized by the conventional separation apparatus.

Explanation of symbols

501 SVC stream 502 multiplexer 503 packet generation unit

Claims (10)

Packetizing an elementary stream layered into a base layer and at least one enhancement layer, and giving different identification information to the packet for each layer;
Within the range of the access unit of the elementary stream, a packet multiplexing position control unit that multiplexes the elementary stream by replacing at least one of the base layer packet and at least one of the enhancement layer packet;
A multiplexing apparatus comprising:   The multiplexing apparatus according to claim 1, wherein the packet multiplexing position control unit replaces packets between the plurality of extension layers when there are a plurality of the extension layers.   2. A dependency information generation / multiplexing unit that generates dependency relationship information indicating a dependency relationship between the enhancement layer and each layer of the elementary stream, and multiplexes the dependency relationship information into the elementary stream. Alternatively, the multiplexing apparatus according to claim 2.   The dependency relationship information generation / multiplexing unit determines, based on the dependency relationship information, that the layer in which the packet is exchanged is included in either a preset layer or a layer that is dependent on the layer. In this case, an order change information generation / multiplexing unit that generates order change information indicating that the packet replacement has been performed and multiplexes the information to the elementary stream is included. The multiplexing device described. An identification information interpretation unit for interpreting the identification information of the packet of the elementary stream multiplexed by the packet multiplexing position control unit according to claim 1, and for identifying the layer of the packet;
An order change buffer for changing the order of packets identified by the identification information interpretation unit;
A base layer packet separation unit for writing the base layer part in the elementary stream included in the packet identified by the identification information interpretation unit to be a packet of the base layer into an input buffer of an external decoder;
An enhancement layer packet separation unit for writing the enhancement layer part in the elementary stream included in the packet identified as the packet of the enhancement layer by the identification information interpretation unit to the buffer for reordering;
A separation apparatus comprising: An access unit completion detection unit that detects that an elementary stream has been acquired in a predetermined layer and a dependency layer in the access unit of the elementary stream,
When the access layer completion detection unit detects that an elementary stream has been acquired in a preset layer and a dependency layer in the access unit, the extension layer packet separation unit 6. The separation apparatus according to claim 5, wherein the elementary stream of the enhancement layer separated in step 1 is written to an input buffer of the external decoder.   If the enhancement layer packet separation unit determines that there is an order change based on the order change information indicating that the packet has been changed, the extension layer packet separation unit writes the separated packet into the order change buffer, and 6. The separation apparatus according to claim 5, wherein when it is determined that the order is not changed based on the information, the separated packets are written in the input buffer of the external decoder according to the order in which the packets are input.   When the base layer packet separation unit determines that there is no order change based on the order change information indicating that the packet has been changed, the base layer packet separation unit converts the separated packet according to the order in which the packets are input to the external packet. 6. The separation apparatus according to claim 5, wherein the data is written to an input buffer of a decoder. Packetizing an elementary stream layered into a base layer and at least one enhancement layer, and giving different identification information to the packet for each layer;
Within the range of access units of the elementary stream, replacing at least one of the base layer packets and at least one of the enhancement layer packets to multiplex the elementary stream;
A multiplexing method comprising: On the computer,
Packetizing an elementary stream layered into a base layer and at least one enhancement layer, and giving different identification information to the packet for each layer;
Within the range of access units of the elementary stream, replacing at least one of the base layer packets and at least one of the enhancement layer packets to multiplex the elementary stream;
A program for running

Images