JP3510433B2 - Image processing device - Google Patents

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to MPEG2 (Movi
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing device that complies with ng Picture Experts Group 2) or the like and that performs high-speed motion compensation and decoding of large-scale moving image information.

[0002]

2. Description of the Related Art As an elemental technology of digital compression processing technology for moving image information represented by MPEG2, when decoding compressed moving image information, a so-called "motion" is described in which a motion (movement) of an image is described by a vector amount. There is a technology for constructing image information of a screen (frame) at a certain time position from image information of a screen at another time position in accordance with a "vector", which is called so-called "motion compensation".

As described above, as a video decoder for decoding moving image information compressed on the premise that motion compensation is performed at the time of decoding, an image processing apparatus having a configuration shown in FIG. 6 is known. There is. That is, the image processing apparatus shown in the figure is for performing motion compensation and decoding of image information compressed in accordance with MPEG2. The MPEG2 decoder 3 for decoding the compressed image information and the decoder 3 The memory 5 includes a dynamic RAM (Dynamic Random Access Memory) or the like for temporarily storing the decoded image information. The MPEG2 decoder 3 is embodied as an LSI (Large Scale Integrated circuit) chip or the like by using a method such as a standard cell.

In the conventional image processing apparatus thus configured, the image information of the screen at a certain time position decoded by the MPEG2 decoder 3 is temporarily stored in the memory 5. The image of a certain local area on the screen at another time position is considered to be the image of a certain local area on the screen at a certain time position recorded in the memory 5, and the image of this local area is moved. The image information stored in the memory 5 is read out according to the motion vector indicating the direction and the movement amount to perform motion compensation, and the image information of the screen at another time position is constructed. This motion compensation is sequentially performed over the entire screen in units of the above-mentioned "certain local area", but in the following description, "local area" which is a processing unit on the screen for performing motion compensation is referred to as "local area". Image block ".

[0005]

By the way, in general, M
The data amount of moving image information targeted by PEG2, etc.
The scale is larger than the still image information targeted by the EG or the like. In addition, as the screen size increases and the number of frames of an image per second increases, the amount of data increases significantly, and the MPE that constitutes the conventional device shown in FIG. 6 is formed.
The amount of data to be processed by the G2 decoder 3 and the memory 5 will remarkably increase.

However, a standard MPEG decoder LSI
Is an NTSC (National Television) with a resolution of about 480 pixels vertically, 720 pixels horizontally, and 30 frames per second.
sionSystem Committee) method and so on for decoding image information only. Therefore, one standard M
With a PEG decoder LSI only, 1080 pixels vertically and 1 horizontally
It is not possible to decode large-scale image information such as HDTV (High Definition TeleVision) system having a resolution of 920 pixels and 30 frames per second.

Therefore, when a large-scale image information of HDTV class is to be decoded by using a standard MPEG decoder LSI, a device having parallelized decoding processing is considered as shown in the configuration of FIG. To be That is, the image processing apparatus shown in the figure divides the screen into two areas, image areas A and B, and processes the image information corresponding to these image areas A and B by two systems of MPEG2 decoders 3a and 3b and a memory. 5a and 5b, the image information corresponding to these image areas A and B are processed in parallel and decoded, and then the image information obtained by decoding by the respective MPEG2 decoders 3a and 3b is combined by the combiner 6a. The decoded image information for one screen is configured.

However, in the MPEG2 or the like, as described above, when performing motion compensation, the memory is accessed according to the motion vector to read out the image information of the required image block, and the device shown in FIG. 7 is used. In the above configuration, the decoder 3 in charge of processing the image information of the image area A
A cannot access the memory 5b that stores the image information of the image area B. Therefore, for example, when the motion vector refers to the image information of the image block of the image region B when performing motion compensation for a certain image block belonging to the image region A, the image information specified by this motion vector is referred to. Motion compensation cannot be performed,
The movement of the image near the boundary between the image areas A and B becomes unnatural.

The present invention has been made in view of the above problem, and when moving image information is divided into a plurality of image areas on a screen and divided in parallel and decoded in parallel, the moving image information is displayed near the boundaries of the image areas. An object of the present invention is to provide an image processing device capable of performing appropriate motion compensation over the entire screen without making motion compensation impossible.

[0010]

[0011]

[Means for Solving the Problems] The present invention has the following configuration in order to solve the problems described above. That is, the image processing apparatus according to the first invention divides the screen into N image areas from the 1st to the Nth (N is an integer of 2 or more) vertically adjacent to each other and divides the image area. An image processing apparatus that performs motion compensation of the image information of a screen in parallel as a unit and decodes the image information, and associates the image information with the first to Nth image areas in association with the first to Nth image areas. Information dividing means for dividing into image information, and first to Nth decoding means for decoding the first to Nth image information into first to Nth decoding information, respectively.
And the maximum size of the motion vector in the vertical direction which is at least the size in the vertical direction so that each of the first to Nth image areas is adjacent in the vertical direction. Corresponding to the first to the 2N-th small image areas obtained by dividing the image into two, which are set to be larger than the above, and the first to the N-th decoding information are used as the first to the 2N-th reference information. A first to a second N storage unit that stores the data in a divided manner, and an information combining unit that combines the first to Nth decoding information in association with the first to Nth image areas, The first to Nth decoding means decodes the first to Nth image information into the first to Nth decoding information, respectively. Is an integer between 1 and N inclusive) By accessing any of the storage means from M-2 to 2M + 1 and referring to the reference information from 2M-2 to 2M + 1, the image information in which the relative positions of the first to Nth coincide are parallel. The image processing apparatus has a configuration for performing motion compensation.

[0012]

[0013]

[0014]

[0015]

The image processing apparatus according to the second invention is
The screen is divided into N image areas from the 1st to the Nth (N is an integer of 2 or more) vertically adjacent to each other, and the image information of the screen is parallelly motion-compensated in units of the image areas. And an image processing apparatus for decoding the image information, the first information dividing unit dividing the image information into luminance information and color difference information, and the first to Nth image areas in association with each other. A second information dividing unit for dividing the luminance information or the color difference information divided by the first information dividing unit into first to Nth image information; and the first to Nth image information for the first to Nth image information. 1st to Nth decryption information respectively
And the maximum size of the motion vector in the vertical direction which is at least the size in the vertical direction so that each of the first to Nth image areas is adjacent in the vertical direction. Corresponding to the first to the 2N-th small image areas obtained by dividing the image into two, which are set to be larger than the above, and the first to the N-th decoding information are used as the first to the 2N-th reference information. A first to a second N storage unit that stores the data in a divided manner, and an information combining unit that combines the first to Nth decoding information in association with the first to Nth image areas, The first to Nth decoding means decodes the first to Nth image information into the first to Nth decoding information, respectively. Is an integer between 1 and N inclusive) By accessing any of the storage means from M-2 to 2M + 1 and referring to the reference information from 2M-2 to 2M + 1, the image information in which the relative positions of the first to Nth coincide are respectively parallel. The image processing apparatus is characterized in that motion compensation is performed in the image processing apparatus.

Furthermore, the image processing apparatus according to the third invention divides the screen into N image regions from the first to the Nth (N is an integer of 2 or more) vertically adjacent to each other,
An image processing apparatus for parallelly motion-compensating and decoding image information of the screen in units of the image area, the first information dividing unit dividing the image information based on color components, and the first information dividing unit. Second information division means for dividing the image information divided by the first information division means into first to Nth image information in association with the first to Nth image areas; First to Nth decoding means for respectively decoding the Nth image information into the first to Nth decoding information,
Each of the first to Nth image areas is adjacent in the vertical direction, and the vertical size is at least the maximum value in the vertical direction of the motion vector in the motion compensation. Corresponding to the first to the 2N-th small image areas obtained by dividing the image into two, which are set to be larger than the above, and the first to the N-th decoding information are used as the first to the 2N-th reference information. A first to a second N storage unit that stores the data in a divided manner, and an information combining unit that combines the first to Nth decoding information in association with the first to Nth image areas, The first to Nth decoding means decodes the first to Nth image information into the first to Nth decoding information, respectively. Is an integer greater than or equal to 1 and less than or equal to N) is the second M-2. By accessing any of the 2M + 1th storage means and referring to the 2M-2nd to 2M + 1th reference information, motion compensation is performed in parallel for each of the image information whose first to Nth relative positions match. The image processing apparatus has a configuration for performing the operation.

[0018]

[0019]

[0020]

[0021]

The operation of the present invention thus constructed will be described below. That is, according to the image processing device of the first aspect of the present invention, the information dividing unit associates the image information with the first to Nth image areas of the vertically divided screen area. Each of the first to N-th decoding means that divides the image information of each of the first to N-th image areas divided by the information dividing means in parallel, It is given to each of the 2N storage means.
Here, the 1st to 2Nth storage means stores the image information decoded by the 1st to Nth decoding means from the first to the 2nd one obtained by further dividing each of the 1st to Nth image areas. Corresponding to the 2N-th small image area,
Are respectively stored as reference information. As a result, the image information for one image area decoded by the decoding means is divided and stored in a pair of consecutive odd-numbered and even-numbered storage means. When the first to N-th decoding means decode the image information divided by the information dividing means,
The Mth (M is an integer of 1 or more and N or less) decoding means is the 2nd Mth.
-2 to 2M + 1 storage means are accessed to refer to the 2M-2 to 2M + 1 reference information to perform motion compensation in parallel for the first to Nth image information. The information combining unit combines the first to Nth decoded information that has been motion-compensated and decoded by the first and second decoding units to reconstruct the image information.

[0023]

[0024]

[0025]

[0026]

Still further, according to the image processing apparatus of the second invention, the first information dividing means divides the image information into luminance information and color difference information, and the second information dividing means extends in the vertical direction. The first to Nth screens divided so that they are adjacent to each other
The luminance information or the color difference information divided by the first information dividing means in association with the th image area is further divided into the first to Nth image information. Each of the first to Nth decoding means parallelly decodes the first to Nth image information divided by the first and second information dividing means, and independently accessible first to second Nth. The storage means is divided and stored. That is, the 1st to 2Nth storage means stores the image information decoded by the 1st to Nth decoding means into the 1st to 1st image areas obtained by further dividing each of the 1st to Nth image areas. The first to second Nth reference information is stored in association with the 2Nth small image area. Therefore, the image information for one image area decoded by the decoding means is divided and stored in a pair of consecutive odd-numbered and even-numbered storage means. When each of the first to Nth decoding means decodes the first to Nth image information divided by the first and second information dividing means, the Mth (M is an integer of 1 or more and N or less) Of the second to (M-2) th to (2M + 1) th storage means to refer to the second to (M-2) th to the (2M + 1) th reference information to parallelize the first to Nth image information. Motion compensation. The information combining unit combines the first to Nth decoded information that has been motion-compensated and decoded by the first and second decoding units to reconstruct the image information. In this way, the image information for one image area is divided and stored in a pair of odd-numbered and even-numbered storage means, so that each decoding means accesses the storage means as a unit. In motion compensation of the “image area”, even if the motion vector extends over the “different image area”, the storage means is accessed according to this motion vector without interfering with the decoding operation in the “different image area”. Then, the motion compensation of the “certain image area” can be performed.

Furthermore, according to the image processing apparatus of the third invention, the first information dividing means divides the image information based on the color components, and the second information dividing means are vertically adjacent to each other. The image information divided by the first information dividing means is further divided into the first to Nth image information in association with the first to Nth image areas of the divided screen. Each of the first to Nth decoding means parallelly decodes the first to Nth image information divided by the first and second information dividing means, and independently accessible first to second Nth. The storage means is divided and stored. That is, the 1st to 2Nth storage means stores the image information decoded by the 1st to Nth decoding means into the 1st to 1st image areas obtained by further dividing each of the 1st to Nth image areas. The first to second Nth reference information is stored in association with the 2Nth small image area. Therefore, the image information for one image area decoded by the decoding means is divided and stored in a pair of consecutive odd-numbered and even-numbered storage means. And the first
When each of the Nth to Nth decoding means decodes the first to Nth image information divided by the first and second information dividing means, the Mth (M is an integer of 1 or more and N or less) decoding means , 2M-2 to 2M + 1 storage units are accessed to refer to the 2M-2 to 2M + 1 reference information to perform motion compensation in parallel for the 1st to Nth image information. To do. The information combining unit combines the first to Nth decoded information that has been motion-compensated and decoded by the first and second decoding units to reconstruct the image information. In this way, the image information for one image area is divided and stored in a pair of odd-numbered and even-numbered storage means, so that each decoding means accesses the storage means as a unit. In motion compensation of the “image area”, even if the motion vector extends over the “different image area”, the storage means is accessed according to this motion vector without interfering with the decoding operation in the “different image area”. Then, the motion compensation of the “certain image area” can be performed.

[0029]

[0030]

[0031]

[0032]

DETAILED DESCRIPTION OF THE INVENTION An image processing apparatus according to an embodiment of the present invention will be described in detail below with reference to FIGS. Here, FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to the first embodiment, and FIG. 2 is an explanatory diagram for explaining the motion compensation operation of the image processing apparatus according to the first embodiment. 3 is a block diagram when the number of screen divisions is expanded to three in the configuration of the apparatus of the first embodiment shown in FIG. 1, and FIG. 4 shows the configuration of the image processing apparatus according to the second embodiment. FIG. 5 is a block diagram shown in FIG. 5, and FIG. 5 is a block diagram when the screen is divided into two with respect to the luminance information in the configuration of the device according to the second embodiment shown in FIG. In each figure, the same elements or corresponding elements are designated by the same reference numerals.

(Regarding the First Embodiment) First, referring to FIG.
The image processing apparatus according to the first embodiment of the present invention will be described with reference to FIGS. The image processing apparatus according to the present embodiment is configured such that a plurality of decoders can refer to image information corresponding to each image area obtained by dividing a screen. When each decoder performs motion compensation, the decoder performs motion compensation. The image information of the image area other than the target image area can be referred to according to the motion vector. This enables motion compensation in accordance with the motion vector near the boundary of the image area.

That is, the image processing apparatus of the present embodiment shown in FIG. 1 parallelizes respective image information corresponding to image areas RA and RB obtained by vertically dividing the screen into two, as shown in FIG. 2 described later. A slice detector 1 (information division) for processing a bit stream of compressed image information, which detects a position on the screen and divides (slices) it in association with two upper and lower image areas RA and RB. And the image information corresponding to each of the upper and lower image areas RA and RB divided by the slice detector 1 are time-adjusted to match the operation timings of decoders 3A and 3B described later, and are output. FIFO (First-In Fi
(rst-Out) buffers 2A and 2B (the latter stage of the information dividing means) and the image information corresponding to the upper and lower image areas RA and RB time-adjusted by the buffers 2A and 2B are motion-compensated and decoded. Decoders 3A and 3B
(A plurality of decoding means), a selector 4A-2 (a part of the decoding means) for selecting one of the decoders 3A or 3B and connecting it to a memory 5A-2 described later, and either the decoder 3A or 3B. Selector 4B-1 (a part of the decoding means) that is connected to a memory 5B-1 to be described later, and a storage unit 5A (a plurality of units that stores the image information decoded by the decoder 3A as reference information for motion compensation). Part of the storage means)
A storage unit 5B (a part of a plurality of storage units) for storing the image information decoded by the decoder 3B, and the decoder 3
A combiner 6 (information combining means) configured to combine image information corresponding to the upper and lower image areas RA and RB decoded by motion compensation by A and 3B to form image information for one screen. It

Further, each of the storage units 5A and 5B is
It is configured to include independently accessible memories 5A-1 and 5A-2 (plurality of memory spaces) and memories 5B-1 and 5B-2 (plurality of memory spaces). Of these, memory 5
A-2 and 5B-1 are selectively connected to either decoder 3A or 3B via selectors 4A-2 and 4B-1, respectively. Conversely, memories 5A-2 and 5
B-1 is configured to be selectively accessible from both the decoders 3A and 3B. In addition, the memory 5A-1
And 5B-2 can be directly accessed only by the decoders 3A and 3B, respectively.

Here, the correspondence relationship between the image information stored in the storage units 5A and 5B as reference information at the time of motion compensation and each image area of the screen will be described. As shown in FIG. 2 showing the correspondence between the image information stored in each memory and the screen (image area), the storage units 5A and 5B divide the screen into, for example, two parts vertically adjacent to each other (up and down). The decoder 3A outputs the image information corresponding to the obtained image areas RA and RB.
And 3B, respectively, and stored as reference information.

Memory 5A-1 constituting this storage unit 5A
And 5A-2 are small areas RA1 and RA2 obtained by dividing the image area RA into two so as to be adjacent in the vertical direction.
In association with (small image area), the image information of the image area RA input from the decoder 3A is divided and stored, and similarly,
The memories 5B-1 and 5B-2 forming the storage unit 5B are
Small regions RB1 and R obtained by dividing the image region RB into two
The image information of the image area RB input from the decoder 3B is divided and stored in association with B2.

By the way, in MPEG2, the maximum value that a motion vector can take is defined, and the vertical size (height) of each of the small areas RA1 to RB2 shown in FIG. 2 can be taken by the motion vector of each image block. It is set larger than the maximum value. As a result, the motion vector of the image block belonging to a certain small area does not refer to the image information of the small areas separated by two or more, and as described later, the decoders 3A and 3B make the image areas RA and RB of In the process of processing image information in parallel, the same memory is not accessed at the same time (so-called memory batting). As described above, the relationship between the maximum value of the motion vector and the size of the small area is defined from the viewpoint of memory batting.

However, if the required image quality can be obtained, it is not always necessary to strictly comply with these relationships and to completely avoid the batting of the memory. The size of the small area may be made smaller than the maximum value that the motion vector can take, and a certain amount of memory batting may be allowed to save the memory storage capacity.

In the present embodiment, the small areas RA1 to RA1.
The size in the vertical direction (vertical direction) is defined in response to B2 being divided so as to be adjacent in the vertical direction. However, in the direction in which the small areas are adjacent, the size of each small area is the maximum value of the motion vector If the small areas are adjacent to each other in the horizontal direction, the size of the small areas in the horizontal direction (horizontal direction) may be larger than the maximum value of the motion vector.

Hereinafter, the motion compensation operation of the image processing apparatus of this embodiment having the above-described configuration will be described. As described above, in the case of decoding MPEG-compliant image information, motion compensation is sequentially performed according to a motion vector in image block units. That is, the decoder 3A shown in FIG.
First, the image block a belonging to the small area RA1 shown in FIG.
The motion compensation is started from the portion corresponding to 1, and in parallel with this, the decoder 3B determines that the block b1 belonging to the small region RB1.
Motion compensation is started from the portion corresponding to.

Here, the positions of the image blocks a1 to a5 belonging to the image region RA and the positions of the image blocks b1 to b5 belonging to the image region RB are such that the relative positions in the respective image regions coincide with each other. . Therefore, the image blocks a1 to a5 belonging to the image area RA and the image area R
The image blocks b1 to b5 belonging to B are separated from each other on the screen by twice the size of the small area in the vertical direction, so that the maximum value of the image does not exceed the size of the small area in the vertical direction. As will be described later, the motion vectors of the blocks a1 to a5 and the image blocks b1 to b5 do not refer to the same small area in the process of parallel motion compensation for each image information.

In this way, the correspondence between the image information stored in each of the storage units 5A and 5B and each image area of the screen is set, and the motion compensation is performed as follows. That is,
First, in the case of motion-compensating a portion corresponding to the image block a1 shown in FIG. 2, the motion vector of this image block a1 falls within the small area RA1 and the decoder 3A
Directly accesses the memory 5A-1 to access the image block a.
Motion compensation is performed according to the motion vector of 1.

On the other hand, in parallel with the processing of the portion corresponding to the image block a1, the motion compensation of the portion corresponding to the image block b1 is performed. In this case, the maximum vertical range that the motion vector of this image block b1 can take is
It spans both the small areas RA2 and RB1. Therefore, in this case, the selection states of the selectors 4A-2 and 4B-1 are set in the decoder 3B, and the decoder 3B selects either the memory 5A-2 or 5B-1 via the selector 4A-2 or 4B-1. Access and perform motion compensation.

Next, when motion compensation is performed on the portion corresponding to the block a2, the maximum vertical range that the motion vector of the image block a2 can take is still the small area RA1.
Since it is housed inside the
Access A-1 to perform motion compensation.

On the other hand, in parallel with the processing of the portion corresponding to the block a2, the motion compensation of the portion corresponding to the block b2 is performed. In this case, the maximum vertical range that the motion vector of this image block b2 can take is the small region RB.
It fits in one. Therefore, in this case, the selector 4
The selected state of B-1 is maintained in the decoder 3B, and the decoder 3B accesses the memory 5B-1 via the selector 4B-1 to perform motion compensation. In this case, the memory 5A
-2 is not accessed by either of the decoders 3A and 3B, the selector 4A-2 may have any selection state.

Next, when motion compensation is performed on the portion corresponding to the image block a3, the maximum vertical range that the motion vector of this image block a3 can take is the small areas RA1 and R.
It straddles both A2. Therefore, in this case, the selection state of the selector 4A-2 is changed to the decoder 3A,
The decoder 3A accesses either the memory 5A-1 or 5A-2 to perform motion compensation.

On the other hand, in parallel with the processing of the portion corresponding to the image block a3, the motion compensation of the portion corresponding to the image block b3 is performed. In this case, the maximum vertical range that the motion vector of this image block b3 can take is
It spans both the small regions RB1 and RB2. Therefore, in this case, the selection state of the selector 4B-1 is maintained in the decoder 3B, and the decoder 3B accesses either the memory 5B-1 or 5B-2 to perform motion compensation.

Next, when motion compensation is performed on the portion corresponding to the image block a4, the maximum vertical range that the motion vector of the image block a4 can take is within the small area RA2, so the selector 4A-2. The selected state is maintained by the decoder 3A, and the decoder 3A accesses the memory 5A-2 via the selector 4A-2 to perform motion compensation.

On the other hand, in parallel with the above-described processing of the portion corresponding to the image block a4, motion compensation of the portion corresponding to the image block b4 is performed. In this case, the maximum vertical range that the motion vector of this image block b4 can take is
Since it fits inside the small area RB2, the decoder 3B
Accesses the memory 5B-2 to perform motion compensation. In this case, the memory 5B-1 includes the decoders 3A and 3A.
Since neither of B is accessed, selector 4B-1
Any selection state may be used.

Finally, in the case of motion compensation of the portion corresponding to the image block a5, the maximum vertical range that the motion vector of the image block a5 can take extends over both the small areas RA2 and RB1. So in this case,
The selection state of the selector 4A-2 is maintained by the decoder 3A and the selection state of the selector 4B-1 is changed to the decoder 3A.
Motion compensation is performed by accessing either the memory 5A-2 or 5B-1 via B-1.

On the other hand, in parallel with the above-described processing of the portion corresponding to the image block a5, motion compensation of the portion corresponding to the image block b5 is performed. In this case, since the motion vector of this image block b5 is contained within the small area RB2, the decoder 3B accesses the memory 5B-2 to perform motion compensation.

In this way, in the process of performing the motion compensation of the image blocks a1 to a5 and the image blocks b1 to b5 in parallel, and performing the decoding process, the selector 4A- depending on the image block which is the target of the decoding process. The selection states of 2 and 4B-1 are set, and the decoders 3A and 3B do not access the same memory at the same time. In addition, for any image block on the screen, the image blocks a1 to a
5 or b1 to b5, the decoders 3A and 3B can store the image information (reference information) necessary for motion compensation according to the respective motion vectors without causing batting. Access in parallel.

Therefore, according to the image processing apparatus of the present embodiment, unlike the case of performing the motion compensation by simply referring to the divided image information as in the above-mentioned conventional apparatus, even at the boundary between the image areas RA and RB. It is possible to perform the motion compensation normally and perform the motion compensation in parallel for the image information corresponding to each image area, and it is possible to construct the image information in which the natural motion is reproduced.

Although the image processing apparatus of the present embodiment shown in FIG. 1 is configured to process image information in parallel in association with image areas RA and RB obtained by dividing the screen into two, it is necessary. The number of divisions (the number of parallel processing stages) may be increased according to the above. For example, when trying to decode image information in parallel by associating it with an image region obtained by dividing the screen into three, as shown in FIG. 3, in the configuration of the apparatus shown in FIG.
Further, the buffer 2C, the decoder 3C, the selector 4C-
1. It may be configured by adding each element of the storage unit 5C (memory 5C-1, 5C-2).

In this case, the slice detector 1 and the combiner 6 are constructed so that their functions are expanded to correspond to the newly added above-mentioned elements, and the storage units 5A to 5C are The storage capacity is adjusted (decreased) in accordance with the increase in the number of screen divisions.

In the case of the image processing apparatus of the present embodiment shown in FIGS. 1 and 3 described above, the storage unit (for example, the storage unit 5).
A) has two memories (for example, memories 5A-1 and 5A-).
2) and these two memories are used as the independently accessible memory space of the storage unit 5A, but each memory (5A-1 to 5C-2) may be treated as an independent storage unit in the first place. In this case, the image information decoded by one decoder is divided and stored in two storage units.

In the case of such a configuration, for example, the decoder 3B (Mth decoding means) shown in FIG. 3 has the memory 5A-2 (storage means of the second M-2) depending on the position of the image block to be processed. ) To memory 5C-1 (second M + 1
Any one of the storage means) is accessed to refer to the image information (reference information) stored in these memories to perform motion compensation.

Further, in the present embodiment, the storage unit (for example, the storage unit 5A) shown in FIGS. 1 and 3 is composed of two memories to realize a plurality of independently accessible memory spaces. Need only be independently accessible for each small area of each image area, and can be configured using a device such as a dual port memory that can simultaneously specify a plurality of addresses.

Furthermore, in the present embodiment, for example, the decoder 3A can access a maximum of three memories (memory 5A-1, 5A- depending on the selection state of each selector).
2, 5B-1), but each memory may be configured to be accessible by each decoder.

(Regarding the Second Embodiment) Next, referring to FIG.
An image processing apparatus according to the second embodiment of the present invention will be described with reference to FIGS. The image processing apparatus of the first embodiment described above divides image information in association with an image area that is a physical area on the screen and performs parallel processing. Is for dividing image information into parallel processing by paying attention to attributes (data structure) such as brightness and color difference.

In general, color image information includes luminance and color difference,
Alternatively, the components of RGB (Red-Green-Blue) and the like are decomposed and encoded / decoded. The image processing apparatus of the present embodiment is configured by focusing on this point, and separates the decoding processing of each component of the color image information of the luminance component (luminance information) and the color difference component (color difference information) from each other. , And the motion compensation of each of the luminance component and the color difference component is performed in parallel.

That is, the image processing apparatus of this embodiment shown in FIG. 4 inputs a bit stream of color image information,
A luminance decoder 30 and a color difference decoder 31 that decode the luminance component and the color difference component of the color image information, and a luminance that stores the luminance component and the color difference component of the image information decoded by the luminance decoder 30 and the color difference decoder 31, respectively. Memory 50 and color difference memory 51.

In the case of the apparatus of the present embodiment having such a configuration, the luminance decoder 30 and the color difference decoder 31 are
The bit stream of the color image information is input, the luminance component and the color difference component of the color image information are decoded, and stored in the luminance memory 50 and the color difference memory 51, respectively.

Here, in the case of performing motion compensation, the luminance decoder 30 and the color difference decoder 31 have the luminance memory 50.
And the color difference memory 51 are respectively accessed according to the motion vector to read out the image information of the required image block, and the luminance component and the color difference component of the image information subjected to motion compensation using this image information are connected to the subsequent stage side. It is given to a D / A converter (not shown).

As described above, in the apparatus according to the present embodiment, the color image information is divided into the luminance component and the color difference component and processed in parallel, so that the load of one decoder is not overloaded. It reduces the amount of data. In this embodiment, the device is configured to decode color image information including a luminance component and a color difference component. However, as described above, when configured as a device to decode RGB color image information, , The color image information is divided into color components of R (Red) component, G (Green) component, and B (Blue) component, and the image information is parallelly motion-compensated and decoded for each color component. It suffices to appropriately select the division criterion according to the type of image information to be processed.

(Regarding Third Embodiment) Next, referring to FIG.
An image processing apparatus according to the third embodiment of the present invention will be described using. As described above, the image processing apparatus of the second embodiment is configured to divide the image information into the luminance component and the color difference component and perform the parallel processing, but the image processing apparatus of the present embodiment shown in FIG. The image area obtained by dividing the screen in the process of processing the luminance component and the color difference component in parallel is configured by combining the device of the first embodiment and the device of the second embodiment. The processing of the luminance component or the color difference component is further parallelized in association with the above.

By the way, the number of divisions of the screen when processing the luminance component and the number of divisions of the screen when processing the color difference components do not necessarily have to be the same. That is, in MPEG and the like, the color difference component is thinned out with respect to the luminance component of the image information and the number of pixels of the image is reduced. Therefore, in general, it is difficult to decode the color difference component as compared with the luminance component. Does not require much processing power.

The device of this embodiment pays attention to this point,
It is configured by adjusting the distribution number of the decoders responsible for the processing of the luminance component and the color difference component in accordance with the data amount, and for decoding the luminance component having a large data amount, two decoders are further distributed. In addition to parallel processing, by allocating one decoder for decoding color difference components with a relatively small amount of data, the decoder is efficiently distributed by matching the data structure of image information. is there.

That is, the image processing apparatus shown in FIG. 5 uses the apparatus of the first embodiment shown in FIG. 1 as the processing system of the luminance component, and further, in order to process the color difference component, the apparatus shown in FIG. 4 is added with a color difference component processing system including a color difference decoder 31 and a color difference memory 51 shown in FIG.

However, the slice detector 1 which constitutes the apparatus shown in FIG. 5 divides the bit stream of the compressed image data into a luminance component and a color difference component, and the luminance component in the vertical direction of the screen. It is configured to be further divided into two in association with each image region obtained by dividing the image into two adjacent regions.

The decoders 3A, 3B and the decoder 31 shown in the figure are configured to decode the luminance component and the chrominance component of the image information input from the slice detector 1, of which the decoders 3A and 3B are included. Decodes each of the luminance components further divided into two in association with the image area. Further, the combiner 6 is configured to combine the luminance component and the color difference component of the image information decoded by the luminance decoders 3A and 3B and the color difference decoder 3C into the image information of one screen.

The apparatus of this embodiment having the above-described configuration divides the image information into the luminance component and the color difference component and performs the motion compensation decoding process in parallel, and the luminance component is distributed to each image area of the screen. This is realized as a case where the decoding process is parallelized by further associating with each other, and the distribution of the decoder is optimized according to the data structure of the image information.

In the present embodiment, two decoders are allocated to the processing of the luminance component and one decoder is allocated to the chrominance component, but the allocation of the decoders is not limited to this. The distribution is not limited, and the distribution may be appropriately determined according to the data amount of each component and the processing capacity of the decoder alone. Similarly, even when the image information is of RGB type, the distribution of the decoders may be determined according to the data amount of each color component and the like.

In the above description of the first and third embodiments, the screen division direction for parallel processing is the vertical direction, but the screen division direction is not limited to this, and the horizontal direction ( It may be a left-right direction or a two-dimensional direction, and further, these dividing directions may be combined appropriately, and the dividing direction may be appropriately determined as necessary.

[0076]

As is apparent from the above description, according to the present invention, the following effects can be obtained. That is, according to the image processing device of the first invention, when the motion compensation is executed in parallel by referring to the image information corresponding to the plurality of image regions obtained by dividing the screen, the image of a certain image region is displayed. Since the information can be referred to by a plurality of decoding means (decoders), even if the motion vector refers to reference information (image information) of another image area, the reference information required is referred to. The motion compensation can be performed in parallel, and the image can be decoded into image information in which the motion of the image is natural even near the boundary between the image regions. Moreover, since image information is processed in parallel, large-scale image information can be motion-compensated and decoded at high speed without increasing the performance of the decoding means alone.

[0077]

[0078]

Furthermore, according to the image processing apparatus of the second invention, the effects obtained by the image processing apparatus of the first invention can be obtained together, and a large-scale image composed of luminance and color difference components can be obtained. Information can be motion-compensated and decoded even faster.

[0080]

Furthermore, the image processing apparatus according to the third invention.
According to the above arrangement, the image processing apparatus according to the first aspect provides
It is possible to obtain the effect that
Decodes large-scale image information with even faster motion compensation
can do.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to a first embodiment.

FIG. 2 is an explanatory diagram illustrating a motion compensation operation of the image processing apparatus according to the first embodiment.

FIG. 3 is a block diagram showing a configuration when the number of screen divisions is expanded to three in the image processing apparatus according to the first embodiment.

FIG. 4 is a block diagram showing a configuration of an image processing apparatus according to a second embodiment.

FIG. 5 is a block diagram showing a configuration in a case where a screen is divided into two and parallel processing is performed on a luminance component in the image processing device according to the second embodiment.

FIG. 6 is a block diagram showing a conventional image processing device conforming to MPEG2.

FIG. 7 is an explanatory diagram for explaining an operation of a conventional image processing device that performs parallel processing of motion compensation.

[Explanation of symbols]

1 Slice detector 2A, 2B, 2C Buffer 3A, 3B, 3C Decoder 4A-2, 4B-1, 4B-2, 4C-1 Selector 5A-5C Storage part 5A-1, 5A-2, 5B-1, 5B -2,5C-1,
5C-2 memory 6 combiner 30 luminance decoder 31 color difference decoder 50 luminance memory 51 color difference memory

─────────────────────────────────────────────────── ─── Continuation of front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) H04N ^7/ 24-7/68

Claims (3)

(57) [Claims] 1. A screen is divided into N image areas from a first to an N-th (N is an integer of 2 or more) vertically adjacent to each other, and the image information of the screen is divided in units of the image areas. An image processing apparatus for performing motion compensation and decoding in parallel, wherein the information dividing unit divides the image information into first to Nth image information in association with the first to Nth image areas. A first to Nth decoding means for respectively decoding the first to Nth image information into first to Nth decoding information, and the first to Nth image areas respectively to the vertical direction. The first size obtained by being divided into two so that they are adjacent to each other in the direction and the size in the vertical direction is set to be larger than at least the maximum possible value in the vertical direction of the motion vector in the motion compensation. Corresponds to the 2nd to Nth small image areas , First to second N storage means for respectively storing the first to Nth decoded information by dividing the first to Nth reference information, and storing the first to Nth decoded information in the first To the Nth image area, the information combining means is configured to combine the first to Nth image areas, and the first to Nth decoding means connect the first to Nth image information to the first to Nth decoding information. In decoding each of the first to N-th decoding means, the M-th (M is an integer of 1 or more and N or less) decoding means of the first to N-th decoding means is second M-2 to second M +
By performing parallel motion compensation on the image information in which the first to Nth relative positions are coincident with each other by accessing any one of the first storage means and referring to the second M-2 to second M + 1 reference information. An image processing device characterized by: 2. A screen is divided into N image areas from the first to the Nth (N is an integer of 2 or more) vertically adjacent to each other, and the image information of the screen is divided in units of the image areas. An image processing apparatus for performing motion compensation and decoding in parallel, the first information dividing unit dividing the image information into luminance information and color difference information, and corresponding to the first to Nth image areas. In addition, a second information dividing unit that divides the luminance information or color difference information divided by the first information dividing unit into first to Nth image information, and the first to Nth image information First to Nth decoding means for respectively decoding the 1st to Nth decoding information, and the first to Nth image areas so as to be adjacent to each other in the vertical direction and have a size in the vertical direction. vertical motion vector of at least the motion compensated In association with the small image region from 1st obtained by halving was set larger than the maximum value that can be up direction of the winding <br/> up to the 2N-th, decoded from the first of the N First to second N storage means for dividing information into first to second N reference information and storing the divided information, respectively, and corresponding first to Nth decoding information to the first to Nth image areas. The first to Nth decoding means respectively decodes the first to Nth image information into the first to Nth decoding information. To Nth decoding means (M is an integer of 1 or more and N or less) of decoding means 2M-2 to 2M +.
By accessing any of the first storage means and referring to the second M-2 to second M + 1 reference information, motion compensation is performed in parallel for the image information in which the first to Nth relative positions match. An image processing device characterized by the above. 3. A screen is divided into N image areas from the 1st to the Nth (N is an integer of 2 or more) vertically adjacent to each other, and the image information of the screen is divided in units of the image areas. An image processing apparatus for performing motion compensation and decoding in parallel, in which a first information dividing unit that divides the image information based on color components is associated with the first to Nth image areas. , The image information divided by the first information dividing means is
To Nth image information, and first to Nth decoding means for respectively decoding the first to Nth image information into first to Nth decoding information, and Each of the first to Nth image areas is adjacent to each other in the vertical direction, and the size in the vertical direction is at least the maximum value in the vertical direction of the motion vector in the motion compensation. Also, the first to Nth decoding information is divided into the first to 2Nth reference information in association with the first to 2Nth small image areas obtained by dividing the image into a large area. First to second N storage means for respectively storing the information, and information combining means for combining the first to Nth decoding information in association with the first to Nth image areas, and The first to Nth decoding means are the first to Nth decoding means. When decoding each of the image information of the first to Nth decoding information, the Mth decoding means of the first to Nth decoding means (M is an integer of 1 or more and N or less) from the second M-2. Second M +
By accessing any of the first storage means and referring to the second M-2 to second M + 1 reference information, motion compensation is performed in parallel for the image information in which the first to Nth relative positions match. An image processing device characterized by the above.