JP2003319416A - Stereoscopic image encoding device and stereoscopic image decoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate encoded stereoscopic image data which are reduced in volume. <P>SOLUTION: A stereoscopic image encoding device encodes a plurality of image data corresponding to two points of view. The image encoding device includes an input right-image generating section 103 and an input left-image generating section 104 which generate four pieces of image information, by respectively dividing image data photographed by means of cameras 101 and 102 into image data at even- and odd-numbered positions in the horizontal direction and the two horizontally divided image data into image data at even- and odd-numbered positions in the vertical direction; first to n-th image coupling sections 105-107 which generate four pieces of coupling image information, by coupling the four pieces of image information correspondingly to the points of view; and first to n-th encoding sections 108-110 which generate four bit streams by encoding the four pieces of coupling image information. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

The present invention relates to a plurality of image data including a left image and a right image for displaying a stereoscopic image,
TECHNICAL FIELD The present invention relates to a technique for transmitting to a stereoscopic image decoding device via a communication path or a recording medium, and particularly to a technique for transmitting a small amount of image data to a plurality of stereoscopic image decoding devices having different image information necessary for stereoscopic display. Regarding

[0002]

2. Description of the Related Art A stereoscopic image that makes an object stereoscopically perceived by human eyes is generated by using a plurality of cameras, and the stereoscopic image information is stored in a recording medium or transmitted via a communication path to generate a stereoscopic image There is a processing device for displaying a stereoscopic image for displaying. There are mainly two methods for displaying a stereoscopic image in such a processing device.

First, a parallax barrier method, a method using a lenticular plate, or the like is provided with a slit plate in which a large number of vertically long slits are opened in front of a display such as a liquid crystal, and the viewer can see the display from the right side. The image for the right side of the image is configured so that the image for the left side on the display can be seen from the left side.

Japanese Laid-Open Patent Publication No. 11-18111 discloses a device for displaying a stereoscopic image based on such a principle. The stereoscopic image display device disclosed in this publication performs thinning processing on each pixel of the captured right image information and left image information in the horizontal direction, so that the right image information and the left image information are combined. Arrange them so that they appear alternately on the scanning lines. Accordingly, an optical system in which the right image is input to the viewer's right eye and the left image is input to the left eye is configured, and the viewer can recognize the stereoscopic image.

Second, the left image and the right image are alternately displayed on the display device in time, and the left and right images are distinguished by the goggles in which shutters such as a liquid crystal shutter are synchronized at the time of switching between them. In some cases, the afterimage on the retina gives a stereoscopic effect.

Japanese Unexamined Patent Publication No. 9-9293 discloses a device for displaying a stereoscopic image based on such a principle. The stereoscopic image display device disclosed in this publication thins out the captured horizontal lines of the right image and the left image on a line-by-line basis, associates each of the obtained images with a field image, and displays them on an interlaced monitor. Then, the left and right are switched and displayed at predetermined time intervals. As a result, the viewer can recognize the stereoscopic image by synchronizing the image displayed in this manner with the shutter of the goggles.

Since stereoscopic image information requires image information for both the left and right eyes, the amount of image information data is enormous. Therefore, it is essential to highly compress the stereoscopic image information in order to store and transmit it. As a compression method for stereoscopic images, the images of the left and right eyes are MPEG (Moving P
icture Experts Group) and JPEG (Joint Photograph)
Based on the ic Experts Group) standard, it is possible to consider a method of individually compressing the left and right sides. In addition, IEO / IEC MPEG
-2, a compression method of a stereoscopic image is standardized in a framework called a multi-view profile. MPEG-2 uses Temporal Scalability technology,
Only the left image information (or right image information) is encoded by the normal method, and the difference information between the corresponding right image information (or left image information) and the left image information is encoded separately.

[0008]

However, even in the two stereoscopic image coding methods exemplified above,
If the type of stereoscopic image representation method of the stereoscopic image display device is different, the horizontal resolution needs to be halved, or the vertical resolution needs to be halved.
Different types of image information are required in the display device. Therefore, conventionally, for each display method (for example, when using a lenticular plate, it is necessary to halve the resolution in the horizontal direction, whereas when using an electronic shutter, halve the resolution in the vertical direction). Different left and right image information is required (since it needs to be encoded).

Therefore, when the image information is simultaneously transmitted to at least two stereoscopic image display devices having different stereoscopic image display systems such as a stereoscopic image display device using a lenticular plate and a stereoscopic image display device using an electronic shutter. Even for the same stereoscopic image, it is necessary to separately create and send image information corresponding to each method. As a result, the size of the storage area of the image information in the device on the sending side becomes large, or the line capacity is wastedly consumed when transmitting using the communication path.

The present invention has been made in order to solve the above-mentioned problems, and is capable of reducing the amount of image information transmitted to a plurality of stereoscopic image decoding devices having different image information required for stereoscopic display. (EN) An encoding device and a stereoscopic image decoding device for decoding a bitstream encoded by the stereoscopic image encoding device.

[0011]

A stereoscopic image encoding apparatus according to a first aspect of the present invention encodes two or more image data corresponding to each of a plurality of viewpoints. This three-dimensional image encoding device uses n pieces of image information (n is 2) based on each image data.
Creating means for creating the above integer), image combining means for creating n pieces of combined image information by combining n pieces of image information corresponding to each viewpoint, and n pieces of combined images Encoding means for encoding the information to produce n bitstreams.

According to the first aspect of the invention, for example, the creating means creates, for example, four pieces of image information based on the image data of two viewpoints (left eye viewpoint and right eye viewpoint). The four pieces of image information are, for example, (A)
Image information in which the image data is extracted so that the resolution in the vertical and horizontal directions of the screen becomes 1/2, and (B) image information in which the image data is extracted so that the resolution in the vertical direction of the screen becomes 1/2. (C) The horizontal resolution of the screen is 1/2
And (D) image information in which the vertical and horizontal resolutions of the screen are not changed. The image combining means creates combined image information by combining such four pieces of image information in correspondence with the left and right viewpoints. The encoding means encodes the combined image information. Thereby, for example, in a stereoscopic image display device (horizontal resolution is half) using a lenticular plate, a bit stream obtained by encoding the image data of (A) and a bit stream obtained by encoding the image data of (B). And the stereoscopic image display device using an electronic shutter (the vertical resolution is half) encodes the bit stream in which the image data of (A) is encoded and the image information of (C). And a bitstream. By doing so, the bit stream in which the image data of (A) is encoded is transmitted once to the transmission line to which the stereoscopic image display device using the lenticular plate and the stereoscopic image display device using the electronic shutter are connected. Just enough. Also in the stereoscopic image encoding device, [a bit stream obtained by encoding image data of (A) + a bit stream obtained by encoding image data of (B)], and a bit obtained by encoding image data of ((A)) It is not necessary to store both of the stream + the bit stream obtained by encoding the image data of (C),
It is possible to reduce the area in which the bitstreams obtained by encoding the image data of (A) are redundantly stored. As a result, it is possible to provide a stereoscopic image encoding device that can reduce the amount of image information sent to a plurality of stereoscopic image decoding devices having different image information required for stereoscopic display.

A stereoscopic image encoding apparatus according to the second invention is
In addition to the configuration of the first aspect of the invention, the creating means corresponds to a device that decodes a bitstream and displays a stereoscopic image,
It includes means for creating n image information.

According to the second aspect of the present invention, the stereoscopic image encoding device is adapted to a stereoscopic image display device using a lenticular plate for transmitting a bit stream, a stereoscopic image display device using an electronic shutter, and the like, and has a horizontal resolution. 1 / n
It is possible to create n pieces of image information. As a result, the stereoscopic image encoding device can create and transmit a bitstream in which the image information required by the destination stereoscopic image display device is encoded.

A stereoscopic image coding apparatus according to the third invention is
In addition to the configuration of the first or second invention, a necessary bitstream is selected from n bitstreams coded by the coding means, corresponding to a device for decoding a bitstream to display a stereoscopic image. Further includes a delivery means for selecting and delivering to the device.

According to the third aspect of the invention, the transmitting means selects the bit stream in which the image information required by the destination stereoscopic image display device is encoded, in association with the destination stereoscopic image display device. , Send the selected bitstream. The stereoscopic image display device decodes the bit stream sent from the stereoscopic image encoding device to display a stereoscopic image.

A stereoscopic image coding apparatus according to a fourth invention is
In addition to the configuration of the first or second invention, a necessary bitstream is selected from n bitstreams coded by the coding means, corresponding to a device for decoding a bitstream to display a stereoscopic image. And further includes sending means for sending to the device a bitstream containing information representative of the selected bitstream.

According to the fourth aspect of the invention, the transmitting means selects the bit stream in which the image information required by the destination stereoscopic image display device is encoded, in association with the destination stereoscopic image display device. , A bitstream containing information representative of the selected bitstream. The stereoscopic image display device can decode information representing a bitstream selected from the bitstreams sent from the stereoscopic image encoding device, and can decode only necessary image information.

A stereoscopic image coding apparatus according to the fifth invention is
In addition to the configuration of any one of the first to fourth inventions, the creating means includes means for separating the image data into n pieces of image data according to the horizontal sample position, and the creating means separately separating in the horizontal direction. And a means for separating each of the n pieces of image data thus generated into m pieces of image data (m is an integer of 2 or more) according to the sample position in the vertical direction and the means for separating the image data in the horizontal and vertical directions. Means for creating image data as n × m pieces of image information.

According to the fifth invention, the creating means, for example, based on each of the image data of a plurality of viewpoints,
A number (n = 2, m = 2) of image information is created. These four pieces of image information are (A) image information obtained by extracting image data so that the vertical and horizontal resolutions of the screen are 1/2, and (B) the vertical resolution of the screen is 1/2. Image information extracted from the image data, (C) image information extracted from the image data so that the horizontal resolution of the screen is halved, and (D) the vertical and horizontal resolutions of the screen are changed. This is the image information that has not been made. Like this 4
The two pieces of image information are encoded, and the necessary bit stream is sent to a stereoscopic image display device using a lenticular plate or a stereoscopic image display device using an electronic shutter.

A stereoscopic image encoding apparatus according to the sixth invention is
In addition to the configuration of any one of the first to fourth inventions, the creating means includes a first sub-band dividing means for dividing the image data into a low band and a high band in the horizontal direction, A second sub-band dividing unit for dividing the image data into a low band and a high band in the vertical direction, a first sub-band dividing unit, and a second sub-band dividing unit are combined, and horizontal. Image data that is sub-band divided into the vertical low band and the vertical low band, image data that is sub-band divided into the horizontal low band and the vertical high band, the horizontal high band and the vertical low band To generate image data sub-band-divided into sub-bands, and image data sub-band-divided into a horizontal high band and a vertical high band, and generate the first sub-band dividing means and the second sub-band. The image data obtained the combination comprising a subband division unit repeatedly recursively and means for creating a n pieces of image information.

According to the sixth invention, by combining the first sub-band dividing means and the second sub-band dividing means, image data of horizontal low band and vertical low band, horizontal low band and Four pieces of image data of a vertical high band, image data of a horizontal high band and a vertical low band, and four image data of a horizontal high band and a vertical high band are generated. In this way, the low-frequency components of the image information can be concentrated and coded, so that a highly efficient coding process can be executed.

A stereoscopic image encoding apparatus according to the seventh invention is
In addition to the configuration of any one of the first to fourth inventions, the encoding means decodes the basic combined image information and a means for encoding the basic combined image information in the n pieces of combined image information. Means for encoding difference information between the basic combined image information and the combined image information.

According to the seventh invention, the encoding means encodes the basic combined image information and the difference information from the basic combined image information to generate a bit stream. Usually, since the difference from the basic combined image information is small, the data amount of the bit stream regarding the difference information can be reduced.

A stereoscopic image coding apparatus according to the eighth invention is
In addition to the configuration of any one of the first to fourth inventions, the creating means includes a first reducing means for reducing the image data in the horizontal direction and a first reducing means for reducing the image data in the vertical direction. 2 reduction means, image data reduced by the first reduction means, image data reduced by the second reduction means,
The image data reduced by the first reduction means and the second reduction means and the input image data are created as n pieces of image information (n = 4).

According to the eighth aspect of the invention, the bit stream obtained by encoding the image information obtained by reducing the image data in the horizontal direction and the image data in the vertical direction are made to correspond to the size of the screen of the stereoscopic image display device. Creating a bit stream that encodes image information and a bit stream that encodes image information that has been reduced in both the horizontal and vertical directions, in correspondence with the screen size of the stereoscopic image display device,
Only the required bitstream can be sent out.

A stereoscopic image encoding apparatus according to the ninth invention is
Two or more image data corresponding to each of the first viewpoint and the second viewpoint are encoded. This stereoscopic image encoding device is
N pieces of image information (n is an integer of 2 or more) are created based on the image data corresponding to the first viewpoint, and n pieces of image information are created based on the image data corresponding to the second viewpoint. Creating means, a first coding means for coding n pieces of image information corresponding to the first viewpoint to create n bit streams, and a k-th corresponding to the first viewpoint. It includes a second encoding unit for encoding difference information between (k = 1 to n) image information and the kth image information corresponding to the second viewpoint.

According to the ninth invention, the image information of the second viewpoint (for example, the viewpoint of the right eye) is based on the image information of the first viewpoint (for example, the viewpoint of the left eye), and the image information for the second viewpoint (for example, the viewpoint of the right eye) is the image of the first viewpoint. The difference information from the information is encoded. Usually, there is a strong correlation between image information with different viewpoints,
The difference is small, and the data amount of the bit stream for the difference information can be reduced.

A stereoscopic image decoding apparatus according to the tenth invention is
The n bit streams (n is an integer of 2 or more) are decoded to generate 2 or more image data corresponding to each of the plurality of viewpoints. This image decoding device includes a selection unit for selecting a necessary bitstream from n bitstreams, and a decoding unit for decoding the selected bitstream to create combined image information. An image separating unit for separating the created combined image information to create image information corresponding to each viewpoint, and creating output image information corresponding to each viewpoint based on the created image information. And a conversion means for converting the output image information so as to match the output format.

According to the tenth invention, for example, the selecting means has two viewpoints (a left eye viewpoint and a right eye viewpoint).
For example, a required bitstream is selected from four bitstreams. These four bit streams are, for example, (A) a bit stream in which image information extracted so that the vertical and horizontal resolutions of the screen are halved, and (B) a vertical direction of the screen. A bitstream obtained by encoding the extracted image information so that the resolution becomes 1/2, and a bitstream obtained by encoding the extracted image information so that the horizontal resolution of the screen (C) becomes 1/2. , (D) is a bit stream obtained by encoding image information in which the vertical and horizontal resolutions of the screen are not changed. From these bitstreams, a bitstream that encodes image information that matches the stereoscopic representation method of the stereoscopic image decoding device is selected. The selected bitstream is decoded by the decoding means, and is separated by the image separation means into image information corresponding to, for example, the left and right viewpoints. The creation means creates output image information corresponding to a plurality of viewpoints, and the conversion means matches the output format (for example, a format that defines the arrangement of the left image information and the right image information), and the output image information. To convert. As a result, the stereoscopic image decoding apparatus that has received the bit stream of (A), the bit stream of (B), and the bit stream of (C) is, for example, an apparatus using a lenticular plate (horizontal resolution). Is half), the bitstream encoded with the image data of (A) and the bitstream encoded with the image data of (B) are selected. If the device is, for example, a device using an electronic shutter (the vertical resolution is half), the bit stream obtained by encoding the image data of (A) and the image information of (C) are encoded. Selected bitstream is selected. By doing so, the data amount of the bit stream transmitted from the image encoding device to the stereoscopic image decoding device can be reduced. As a result, when the image information required for stereoscopic display is different, it is possible to provide a stereoscopic image decoding device corresponding to the stereoscopic image encoding device that can reduce the image information.

A stereoscopic image decoding apparatus according to the eleventh invention is
In addition to the structure of the tenth aspect of the invention, the creating means may include the image information selected by the selecting means and decoded by the decoding means,
It includes a means for creating combined output image information so as to satisfy a predetermined condition.

According to the eleventh invention, the creating means makes the selected plurality of image information correspond to a stereoscopic image display system using a lenticular plate, a stereoscopic image display system using an electronic shutter, and the like, for example, Output image information is created by alternately combining right image information and left image information so that the resolution satisfies a predetermined condition.

A stereoscopic image decoding apparatus according to the twelfth invention is
In addition to the structure of the tenth aspect of the invention, the creating means may include the image information selected by the selecting means and decoded by the decoding means,
It includes means for creating output image information combined so as to satisfy a predetermined resolution condition.

According to the twelfth aspect, the creating means creates the output image information by appropriately combining the image information so that the horizontal resolution and the vertical resolution match the output format of the stereoscopic image decoding apparatus. .

A stereoscopic image decoding apparatus according to the thirteenth invention is
In addition to the structure of the tenth invention, the creating means combines the plurality of image information selected by the selecting means in at least one of the horizontal direction and the vertical direction so that the plurality of image information are staggered. Includes means for creating output image information.

According to the thirteenth invention, a plurality of pieces of image information are provided in at least one of the horizontal direction and the vertical direction so that the horizontal resolution and the vertical resolution match the output format of the stereoscopic image decoding apparatus. Combined to create output image information whose horizontal and vertical resolutions match the output format.

A three-dimensional image decoding device according to the fourteenth invention is
In addition to the configuration of the tenth invention, the creating means includes a first subband combining means for horizontally subband combining, and a second subband combining means for vertically subband combining. And means for producing output image information by sub-band synthesizing by the first sub-band synthesizing means and the second sub-band synthesizing means so as to satisfy a predetermined condition.

According to the fourteenth invention, the first sub-band combining means and the second sub-band combining means are combined,
Image data of horizontal low band and vertical low band, image data of horizontal low band and vertical high band, image data of horizontal high band and vertical low band, horizontal high band In addition, four pieces of image data in the vertical high band are combined. This way,
Since the low-frequency components of the image information are concentrated and encoded, it is possible to provide a stereoscopic image decoding device corresponding to a stereoscopic image encoding device capable of executing highly efficient encoding processing.

A stereoscopic image decoding apparatus according to the fifteenth invention is
In addition to the configuration of the tenth invention, the decoding means includes means for decoding basic combined image information from the n bit streams, and means for decoding difference information corresponding to the combined image information, Means for creating n combined image information based on the decoded basic combined image information and the difference information.

According to the fifteenth aspect, the decoding means decodes the basic combined image information and the difference information from the basic combined image information to generate the image information. Usually, since the difference from the basic combined image information is small, the data amount of the bit stream for the difference information is reduced, and therefore the stereoscopic image decoding device corresponding to the stereoscopic image encoding device capable of executing the highly efficient encoding process. Can be provided.

A stereoscopic image decoding device according to the sixteenth invention is
In addition to the configuration of the tenth invention, the creating means includes a first interpolating means for interpolating the image data in the horizontal direction, and a second interpolating means for interpolating the image data in the vertical direction. A plurality of pieces of image information selected by the selection means and decoded by the decoding means are combined so as to satisfy a predetermined condition, and data is complemented by the first interpolation means and the second interpolation means to output an image. Means for creating information.

According to the sixteenth aspect, the creating means interpolates the image data obtained by decoding the bit stream in the horizontal direction or in the vertical direction based on the size of the screen for displaying the stereoscopic image. Or create the output image information. Since the image data before being interpolated has a reduced data amount and the bit stream encoded with the image data has a small data amount, a stereoscopic image corresponding to a stereoscopic image encoding device capable of performing highly efficient encoding processing. A decoding device can be provided.

A stereoscopic image decoding apparatus according to the seventeenth invention is
In addition to the configuration of the tenth invention, by selecting necessary image information from image information corresponding to a plurality of viewpoints,
It further includes an image creating means for creating either image information for displaying a stereoscopic image or image information for displaying a two-dimensional image.

According to the seventeenth invention, the image creating means comprises:
If the image display device does not have a stereoscopic image display circuit, the image information for two-dimensional image display is displayed. If the image display device has a stereoscopic image display circuit, the image information for stereoscopic image display is displayed. create. Thereby, even when the image display device does not have a stereoscopic image display circuit, a two-dimensional image can be displayed.

A stereoscopic image decoding apparatus according to the eighteenth invention is
The n number of bit streams (n is an integer of 2 or more) corresponding to each of the first viewpoint and the second viewpoint are decoded to
Two or more image data corresponding to each of the viewpoint and the second viewpoint. This image decoding device includes a first decoding means for decoding n bit streams corresponding to a first viewpoint to create n pieces of image information, and n bits corresponding to a second viewpoint. Second decoding means for decoding the stream to create n pieces of difference information, and k-th (k = 1 to n) image information corresponding to the first viewpoint decoded by the first decoding means. And creation for creating image information corresponding to the second viewpoint based on the k-th (k = 1 to n) difference information corresponding to the second viewpoint decoded by the second decoding means. And means.

According to the eighteenth invention, the image information of the second viewpoint (for example, the viewpoint of the right eye) is based on the image information of the first viewpoint (for example, the viewpoint of the left eye), and the image information for the second viewpoint (for example, the viewpoint of the right eye) is the image of the first viewpoint. The difference information from the information is encoded. Usually, there is a strong correlation between image information with different viewpoints,
Since the difference is small and the data amount of the bit stream for the difference information is reduced, it is possible to provide the stereoscopic image decoding device corresponding to the stereoscopic image encoding device capable of performing the highly efficient encoding process.

[0047]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<First Embodiment> FIG. 1 shows a control block diagram of a stereoscopic image encoding apparatus according to the present embodiment.
This stereoscopic image encoding device processes a right camera 101 and a left camera 102 for photographing a subject, image information representing the photographed image, and processes a plurality of image information (basic right image information 1, extended right image information 2 to 2). An input right image generation unit 103 that generates extended right image information n) and a plurality of image information (basic left image information 1, extended left image information 2 to extended left image information n) by processing the captured image information. The input left image generation unit 104 to be generated, the first image combination unit 105 to the nth image combination unit 107 that combine the corresponding right image information and left image information, and the first image combination unit 105 to the fifth image combination unit 105, respectively. The first encoding unit 108 to the n-th encoding unit 110 which encode the encoded information 1 to the encoded information n generated by the n image combining unit 107, respectively.

The right image information representing the right image and the left image information representing the left image captured by the right camera 101 and the left camera 102 are input to the input right image generating unit 103 and the input left image generating unit 104, respectively. .

The input right image generation unit 103 is provided for the right camera 10
N pieces of image information of the basic right image information 1 and the extended right image information 2 to the extended right image information n are generated from the right image information input from the first image combining unit 105 and the second image. It outputs to the combining unit 106 to the nth image combining unit 107.

The input left image generation unit 104 uses the left camera 10
N pieces of image information of the basic left image information 1 and the extended left image information 2 to the extended left image information n are generated from the left image information input from 2, and the first image combining unit 105 and the second image are generated. It outputs to the combining unit 106 to the nth image combining unit 107.

FIG. 2 shows a detailed control block diagram of the input right image generator 103. As shown in FIG. 2, the input right image generation unit 103 includes a horizontal separation unit 201 that processes the input right image information, a vertical separation unit 202 that processes the right image information output from the horizontal separation unit 201, and a vertical separation unit 202. Separation part 2
03 and.

The horizontal separation unit 201 generates two pieces of image information having half the horizontal resolution from the input right image information and inputs them to the vertical separation unit 202 and the vertical separation unit 203, respectively. As shown in FIG. 3, the horizontal separation unit 201 is
The vertical line of the input right image information is extracted line by line to divide the image information. The vertical separation unit 202 and the vertical separation unit 203 generate two images having half the vertical resolution from the right image information input from the horizontal separation unit 201. As shown in FIG. 4, the vertical separation unit 202 and the vertical separation unit 203 extract and divide the horizontal line of the input right image information for each line. In this way, four images each having half the horizontal and vertical resolutions are generated.

When such four images are generated, one of the image information 1 to the image information 4 shown in FIG. 2 is set as the basic right image information 1. When four pieces of image information are generated, any one of them may be the basic right image information 1, but it is necessary to select the image information corresponding to the basic left image information 1 from the image information 1 to the image information 4. is there. Further, the extended right image information 2 to the extended right image information 4 and the extended left image information 2 to the extended left image information 4 also need to correspond to each other.

The order of the horizontal separating section 201, the vertical separating section 202 and the vertical separating section 203 may be the reverse of the order shown in FIG. That is, the vertical separation process may be performed first, and then the horizontal separation process may be performed. Furthermore, although it has been described that four pieces of image information are extracted as described above, the present invention is not limited to this. For example, if the horizontal line thinning cycle is n (n is an integer of 2 or more), the remainder when dividing the line position by n is 0.
, An image in which the remainder obtained by dividing the line position by n is 1 and the image obtained by collecting the lines in which the remainder obtained by dividing the line position by n is n-1. In addition, by thinning out vertical lines from each image, m pieces (m is 2
It is possible to generate n × m image information by dividing the image information into the above integers.

Since the configuration of input left image generation unit 104 is such that the right image is the left image in input right image generation unit 103 described above, detailed description of input left image generation unit 104 will not be repeated.

The first image combining unit 105 outputs the basic right image information 1 and the basic left image information 1 input from the input right image generating unit 103 and the input left image generating unit 104 to the left and right as shown in FIG. Encoded information 1 in parallel and combined
As the output to the first encoding unit 108. Also, rearrange the left and right images as images at different times in the time direction,
You may output in right, left, left ... order.

The second image combining unit 109 to the n-th image combining unit 110 in the above-described first image combining unit 105 convert the basic right image information 1 into the extended right image information 2 to the extended right image information n. Since the basic left image information 1 is expanded left image information 2 to expanded left image information n, detailed description of the second image combining unit 109 to the nth image combining unit 110 will not be repeated.

The first coding unit 108 codes the input coded information 1 to generate a basic bit stream 1. The encoding method used by the first encoding unit 108 is, for example, MPEG-1, MPEG-2, MPE.
It is possible to use an encoding method standardized by ISO / IEC such as G-4. However, the encoding method is not limited to these.

Second coding section 109 to nth coding section 1
10 is an extension bitstream 2 to extension bitstream n of the generated basic bitstream 1, and therefore the second encoding unit 109 to the nth encoding unit 11
The detailed description of 0 will not be repeated here.

The basic bit stream 1 and the extended bit stream 2 to the extended bit stream n are stored in a recording medium or sent to a communication channel.

The operation of the stereoscopic image coding apparatus according to this embodiment having the above structure will be described. In the following description, it is assumed that n = 4.

The right image information and the left image information picked up by the right camera 101 and the left camera 102 are input right image generating section 103 and input left image generating section 104, respectively.
Entered in. Image information 1 whose horizontal and vertical resolutions have been halved by the input right image generator 103
~ Image information 4 is generated. One of them is set as the basic right image information 1. Similarly, the basic left image information 1 corresponding to the basic right image information 1 is set. The set basic right image information 1 and the set basic left image information 1 are corresponding image information.

The input right image generation unit 103 combines the basic right image information 1 with the basic right image information 1 so as to double the resolution in the horizontal direction, and the extended right image information 2 in the vertical direction. Extended right image information 3, which doubles the resolution,
The extended right image information 4 having the original resolution is generated. Similarly, the input right image generation unit 104 combines the basic left image information 1 with the basic left image information 1 so that the horizontal resolution is doubled and the extended left image information 2 is doubled.
Extended left image information 3 having double the vertical resolution and extended left image information 4 having the original resolution are generated.

The first image combining section 105 combines the basic right image information 1 and the basic left image information 1 into the encoded information 1
Is output to the first encoding unit 108. First encoding unit 1
08 encodes the coded information 1 by a predetermined method and outputs the basic bit stream 1. The first image combining unit 106 combines the extended right image information 2 and the extended left image information 2 to obtain the encoded information 2 as the second encoding unit 10.
Output to 9. The second encoding unit 109 uses the encoded information 2
Is encoded according to a predetermined standard, and the extended bit stream 2 is output. Similarly, the extension bitstream 3 is output from the third encoding unit, and the extension bitstream 4 is output from the fourth encoding unit.

As a result of such an operation, the four streams are structured as follows. Stream 1 (basic bitstream 1) stream 2 (extended bitstream 2. An image with double horizontal resolution can be decoded in combination with stream 1) stream 3 (extended bitstream 3. combined with stream 1 Image with double vertical resolution can be decoded) Stream 4 (extended bit stream 4. Image with original resolution can be decoded by combining with stream 1 to stream 3) Stereoscopic image encoding device is only in horizontal direction A stereoscopic image decoding device A which requires image information with half the resolution, a stereoscopic image decoding device B which requires image information with half the resolution only in the vertical direction,
These three streams (stream 1, stream 2 and stream 3) are sent to the communication path to which the stereoscopic image decoding apparatus C that requires image information having half the resolution in the horizontal and vertical directions is connected. In this case, the stream 1, the stream 2, and the stream 3 are separated, so that if the stream 1, the stream 2, and the stream 3 are transmitted once, the following problems do not occur.

That is, when the streams are not separated in this way, information corresponding to (stream 1 + stream 2) is supplied to the stereoscopic image decoding apparatus A which requires an image with half the resolution only in the horizontal direction in the vertical direction. Since information corresponding to (stream 1 + stream 3) is transmitted to the stereoscopic image decoding apparatus B that requires an image with half the resolution of, the result is that the stereoscopic image decoding apparatus A and the stereoscopic image decoding apparatus B are broadcast. If you want to send data to both,
Information corresponding to stream 1 will be transmitted twice.

On the other hand, the stereoscopic image coding apparatus according to the present embodiment does not need to send the stream 1 a plurality of times, and the information to be simultaneously sent to a plurality of stereoscopic image decoding apparatuses of different systems can be transmitted. The information common to the above method and the information unique to each stereoscopic image decoding device are separated so that the common information becomes one. As a result, the storage area or the required transmission band in the stereoscopic image encoding device can be reduced.

When transmitting to the communication channel, the basic bit stream and all extension bit streams may be transmitted, but the basic bit stream and the extension bit stream required by the stereoscopic image decoding apparatus are selected in advance and transmitted. By doing so, the required transmission band can be further reduced.

Also, the basic bit stream and a plurality of extension bit streams can be generated by performing the encoding process using the stereoscopic image encoding apparatus according to the present embodiment. The stereoscopic image decoding device can decode an image of a required size only by decoding an appropriate bitstream. At this time, in the stereoscopic image encoding device, for example, information for identifying the basic bitstream and the plurality of extension bitstreams is embedded in the bitstream, and the stereoscopic image decoding device refers to the embedded information. , So that you can select the required bitstream.

Further, the basic bit stream 1 and the extension bit stream 2 to the extension bit stream n generated by the first to nth encoding units and the file format at the time of accumulation and the communication transmitted through the communication path. As for the method, existing standard methods can be used, but each method is not limited to the standard method.

<Second Embodiment> The second embodiment of the present invention will be described below.
A stereoscopic image encoding device according to the embodiment will be described. FIG. 6 shows a control block diagram of the input right image generation unit and the input left image generation unit in the stereoscopic image encoding device according to the present embodiment. The stereoscopic image encoding device shown in FIG.
The input right image generation unit 103 and the input left image generation unit 1 in the stereoscopic image encoding device according to the above-described first embodiment.
04 is configured as shown in FIG. The configuration other than this is the same as that of the first embodiment described above. Therefore, detailed description thereof will not be repeated.

The input right image generation unit and the input left image generation unit will be described with reference to FIG. In addition, in the following description, since the left and right are common configurations, the left and right are not distinguished unless otherwise specified.

The input image generation unit includes an H-LPF 601 that executes low-pass filter processing in the horizontal direction, an H-HPF 602 that executes high-pass filter processing in the horizontal direction, and
V-LPF that performs low-pass filtering in the vertical direction
605 and V-LPF 609, and V-HPF 606 and V-HPF that perform high-pass filtering in the vertical direction.
610, a down-sampling unit 603 and a down-sampling unit 604 for thinning out pixels in the horizontal direction, and a down-sampling unit 607, a down-sampling unit 608, a down-sampling unit 611, and a down-sampling unit 612 for thinning out pixels in the vertical direction.

The H-LPF 601 executes horizontal low-pass filter processing on the input left image information or right image information. The H-HPF 602 executes horizontal high-pass filter processing on the input left image information or right image information.

The down-sampling unit 603 is an H-LP.
When the image information that has been low-pass filtered by F601 is input, pixel thinning is performed in the horizontal direction to generate image information with half the horizontal resolution. When the image information subjected to the high-pass filter processing by the H-HPF 602 is input, the down-sampling unit 604 thins out pixels in the horizontal direction to generate an image with half the horizontal resolution.

The V-LPF 605 executes low-pass filter processing in the vertical direction on the image information input from the downsampling unit 603 and having half the horizontal resolution. The V-HPF 606 includes a downsampling unit 6
The high-pass filter processing is performed in the vertical direction on the image information input from 03 with half the horizontal resolution.

The down-sampling unit 607 is a V-LPF6.
When the image information subjected to the low-pass filter processing by 05 is input, pixels are thinned out in the vertical direction to generate an image with half the resolution in the vertical direction. Down-sampling unit 6
When the image 08 subjected to the high-pass filter processing by the V-HPF 606 is input, it thins out pixels in the vertical direction to generate an image with half the vertical resolution.

The functions of the V-LPF 609, V-HPF 610, down-sampling section 611, and down-sampling section 612 are the same as those of the V-LPF 605, V-HPF 606, down-sampling section 607, and down-sampling section 608, respectively. Detailed description will not be repeated.

With such a configuration, the downsample unit 607 outputs image information 1, the downsample unit 608 outputs image information 2, and the downsample unit 611 outputs image information 3.
However, the image information 4 is output from the down-sampling unit 612. These image information 1 to image information 4 have half the resolution in the horizontal and vertical directions, respectively.

The V-LPF, H-LPF, V-HPF and H-LPF are, for example, QMF (Quadrature Mirror Filter) used in subband coding.

With such a configuration, of the generated image information 1 to image information 4, a large amount of information to be encoded is concentrated in the image information 1, so that the amount of code of the remaining image information 2 to image information 4 is large. Is reduced, resulting in higher overall coding efficiency.

In the present embodiment, four examples of bit streams to be output have been described. However, a plurality of input image generation units shown in FIG. 6 are combined, and each of image information 1 to image information 4 is Alternatively, it is possible to generate four or more bit streams by configuring a part of the input to the input image generation unit of the next stage.

As described above, when the stereoscopic image coding apparatus according to the present embodiment is used for coding, the low frequency components of the image information are concentrated in the image information 1, so that much information to be coded is concentrated. Therefore, it is possible to perform more efficient encoding.

<Third Embodiment> The third embodiment of the present invention will be described below.
A stereoscopic image encoding device according to the embodiment will be described. FIG. 7 shows a control block diagram of the stereoscopic image encoding device according to the present embodiment. In the stereoscopic image encoding device shown in FIG. 7, a first decoding unit is added to the stereoscopic image encoding device according to the first embodiment described above, and a decoded image output from the first decoding unit is They are different in that they are input to the second encoding unit to the nth encoding unit. For other configurations,
This is the same as the first embodiment described above. Therefore, detailed description thereof will not be repeated.

The basic bit stream output from the first decoding unit 701 is either stored in this stereoscopic image encoding apparatus or transmitted to the communication channel, and at the same time, the first decoding unit 70
Entered in 2. The first decoding unit 702 decodes the input basic bitstream and inputs the decoded image information to the second encoding unit 703 to the nth encoding unit 705, respectively.

The second encoding section 703 has the first decoding section 7
By using the decoded image information input by 02, the encoded information 2 output from the second image combining unit is encoded,
Output the extended bitstream 2. Third encoding unit 7
The 03 to nth encoding units 705 are also the second encoding units 703.
Therefore, detailed description will not be repeated here.
The decoded image information is information obtained by decoding the input basic bit stream, and is information necessary for all stereoscopic image systems. Therefore, the basic bit stream for decoding the decoded image information is sent to the stereoscopic image decoding device that employs all stereoscopic image systems.

In the stereoscopic image coding apparatus according to this embodiment, the basic right image information 1 and the basic left image information 1 output from the first decoding unit 702 and the extended right image information 2 are output.
-Encode the difference information between the extended right image information n and the extended left image information 2 to the extended left image information n. That is, the extended right image information 2 to the extended right image information n and the extended left image information 2
When encoding the extended left image information n, the difference information is encoded instead of encoding the image information itself. In this case, as the difference information, it is conceivable to use an image obtained by subtracting the corresponding pixel unit. Further, a method of calculating a motion vector between the input images, generating a motion compensation image, and subtracting the motion-compensated image in units of corresponding pixels to obtain a motion vector and a motion vector may be considered. As described above, since the basic bitstream is sent to the stereoscopic image decoding device that employs all stereoscopic image systems, the stereoscopic image decoding device
Basic right image information 1 obtained by decoding this basic bitstream
Further, based on the basic left image information 1 and the difference information, the extended right image information 2 to the extended right image information n and the extended left image information 2 to the extended left image information n can be obtained.

According to the three-dimensional image coding apparatus of this embodiment, the difference between the coding information and the decoded image is coded, so that efficient coding is possible.

The first decoding unit 701 of the stereoscopic image coding apparatus according to this embodiment uses the basic bit stream 1
However, if the first encoding unit includes a local decoding unit, the first decoding unit is not added and the decoding information obtained by the local decoding unit is obtained. The second
It may be possible to input the data from the encoding unit to the nth encoding unit.

<Fourth Embodiment> The fourth embodiment of the present invention will be described below.
A stereoscopic image encoding device according to the embodiment will be described. FIG. 8 shows a control block diagram of the input right image generation unit and the input left image generation unit in the stereoscopic image encoding device according to the present embodiment. The stereoscopic image encoding device shown in FIG.
The input right image generation unit and the input left image generation unit in the stereoscopic image encoding device according to the third embodiment described above are shown in FIG.
It is configured as follows. The configuration other than this is the same as that of the third embodiment described above. Therefore, detailed description thereof will not be repeated.

The input right image generation unit and the input left image generation unit will be described with reference to FIG. In addition, in the following description, since the left and right are common configurations, the left and right are not distinguished unless otherwise specified.

The input image generation unit includes an H-LPF 801 that performs low-pass filter processing in the horizontal direction, and V-LPF 802 and VL that perform low-pass filter processing in the vertical direction.
A PF 805, a down-sampling unit 803 for thinning out pixels in the horizontal direction, and a down-sampling unit 804 and a down-sampling unit 806 for thinning out pixels in the vertical direction are included.

The H-LPF 801 executes horizontal low-pass filter processing on the input left image information or right image information.

The down-sampling unit 803 uses the H-LP
When the image information that has undergone the low-pass filter processing in the horizontal direction is input in F801, pixel thinning is performed in the horizontal direction to generate image information having half the horizontal resolution. The generated image information is input to the V-LPF 805 and, at the same time, output as image information 2.

The V-LPF 805 has a down sampling section 8
When the image information in which the pixels are thinned out in the horizontal direction is input by 03, the low-pass filter processing in the vertical direction is executed.

The down-sampling unit 806 thins out pixels in the vertical direction with respect to the image that has been subjected to the low-pass filter processing in the vertical direction, and generates image information having half the resolution in the vertical direction. The generated image information is output as image information 1.

The V-LPF 802 executes vertical low-pass filter processing on the input left image information or right image information.

The down-sampling unit 804 uses the V-LP
When the image information that has been low-pass filtered by F802 is input, pixel thinning is performed in the vertical direction to generate image information with half the vertical resolution. The generated image information is output as image information 3.

The left image information or the right image information input to the input image generation section is output as the image information 4 as it is.

FIG. 9 shows the size of the image generated by the above-mentioned input image generating section. As shown in FIG. 9, the image size of the left image or the right image is X pixels in the horizontal direction and Y pixels in the vertical direction. Since such image information is created, the image sizes input from the image combining unit are different between the second encoding unit 703 to the nth encoding unit 706. Therefore, encoding is performed after the input image is complemented in advance and the sizes are adjusted.

According to the stereoscopic image coding apparatus of this embodiment, since a bitstream is generated for each required image size, the stereoscopic image decoding apparatus decodes one basic bitstream and one appropriate bitstream. It is possible to obtain an image of a required size only by itself and to simplify the configuration of the stereoscopic image decoding device. An appropriate bitstream may be selected by the stereoscopic image decoding device, or an appropriate bitstream may be selected when transmitting the bitstream to the stereoscopic image decoding device. When the stereoscopic image decoding device selects the basic bitstream and an appropriate bitstream, for example, the stereoscopic image encoding device embeds information for identifying the basic bitstream and a plurality of extension bitstreams into a bitstream, The image decoding apparatus may select the required bitstream by referring to this information.

<Fifth Embodiment> The fifth embodiment of the present invention will be described below.
A stereoscopic image encoding device according to the embodiment will be described. FIG. 10 shows a control block diagram of the stereoscopic image encoding device according to the present embodiment. The stereoscopic image encoding device shown in FIG. 10 includes a right camera 1001 and a left camera 1002 that capture a subject, an input right image generator 1003 and an input left image generator that process the captured images to generate a plurality of images. 1004, a first encoding unit 1012 to a second n encoding unit 1017 that encode the generated image, and a first decoding unit 100 that decodes the generated bit stream.
6, the third decoding unit 1008, ..., The (2n−1) th decoding unit 1010, the second subtraction unit 1007 that calculates the difference between the image information and the decoded image information, and the fourth subtraction unit 100.
, ..., The 2n-th subtraction unit 1011 is included. The configuration other than this is the same as that of the first embodiment described above. Therefore, detailed description thereof will not be repeated. Further, the right camera 1001, the left camera 1002, the input right image generation unit 1003, and the input left image generation unit 100.
4 is the same as that of the first embodiment. Therefore, detailed description thereof will not be repeated.

The first encoding unit 1012 encodes the input right image information 1 to generate the basic bit stream 1. The encoding system used here is, for example, IS such as MPEG-1, MPEG-2, MPEG-4.
It is possible to use an encoding method standardized by O / IEC. However, the encoding method is not limited to these.

The basic bit stream 1 generated by the first encoding unit 1012 is stored in a recording medium or sent to the communication channel, and at the same time, is input to the first decoding unit 1006. The first decoding unit 1006 decodes the input basic bitstream and generates decoded image information 1.

The second subtraction unit 1007 generates coding information 2 which is the difference between the generated decoded image information 1 and the left image information (n + 1) and inputs it to the second coding unit 1013.
As a method of subtraction, as described above, it is possible to perform subtraction in units of corresponding pixels. A method of calculating a motion vector between input images, generating a motion-compensated image, and subtracting the motion-compensated image in units of corresponding pixels to obtain a motion vector and a motion vector is also conceivable.

The second encoding unit 1013 encodes the input encoded information 2 to generate an extended bitstream 2. The encoding method used here is, for example, IS such as MPEG-1, MPEG-2, MPEG-4.
Although it is possible to use an encoding method standardized by O / IEC. It is not limited to these.

Third encoding section 1014 to 2n-th encoding section 1017, fourth subtraction section 1009 to 2n-th subtraction section 1
Since the configuration of 011 is the same as that of first encoding section 1012, second encoding section 1013 and second subtraction section 1007, detailed description thereof will not be repeated.

The basic bit stream 1 and the extended bit stream 2 to the extended bit stream 2n are stored in a recording medium or sent to a communication channel. It is possible to use the existing standard method as the file format at the time of storage and as the communication method sent out through the communication path. However, it is not limited to those standard methods.

According to the stereoscopic image coding apparatus of the present embodiment, a right image and a left image are separately coded and only the right or left image is transmitted, so that a display device that does not support stereoscopic display can be used. Images can be displayed.

<Sixth Embodiment> The sixth embodiment of the present invention will be described below.
A stereoscopic image decoding device according to the embodiment will be described. FIG. 11 shows a control block diagram of the stereoscopic image decoding device according to the present embodiment. The stereoscopic image decoding device illustrated in FIG. 11 is a device that decodes a bitstream encoded by the stereoscopic image encoding device according to the above-described first embodiment. However, the bitstream decoded by the stereoscopic image decoding apparatus according to the present embodiment is not limited to the bitstream encoded by the stereoscopic image encoding apparatus according to the first embodiment described above.

The stereoscopic image decoding apparatus according to this embodiment is
A bitstream selection unit 1101 that selects only a required bitstream from the input extended bitstream.
A first decoding unit 1102 to a k-th decoding unit 1104 for decoding the selected bitstream, a first separation unit 1106 to a k-th separation unit 1109 for separating left and right image information from the decoded image, An output right image conversion unit 1110 and an output left image conversion unit 1111 which generate image information necessary for display from the input image information, and an arrangement for rearranging data from the input left and right images into a format required by the display device. The replacement conversion unit 1112 is included.

The basic bit stream 1 stored in the recording medium or transmitted via the communication path is the first decoding unit 11
It is input to 02. The extension bitstream 2 to the extension bitstream n stored in the recording medium or transmitted via the communication path are input to the bitstream selection unit 1101.

The bitstream selection unit 1101 selects (k-1) pieces from the plurality of input bitstreams and inputs them to the second decoding unit 1103 to the kth decoding unit 1105, respectively. Here, k is the number of streams required by this stereoscopic image decoding device.

The first decoding section 1102 decodes the input basic bit stream 1 and outputs decoding information 1.
The output decoded information 1 is separated into basic right image information 1 and basic left image information 1 by the first image separation unit 1106, and the basic right image information 1 is output to the output right image conversion unit 1110.
The basic left image information 1 is input to the output left image conversion unit 1111.

The first image separating section 1106 has a function opposite to that of the first image combining section 105 shown in FIG.

The second decoding unit 1103 decodes each input extension stream and outputs the decoding information 2. The output decoding information 2 is the second image separation unit 110.
7 is separated into extended right image information 2 and extended left image information 2, and the extended right image information 2 is output right image conversion unit 1110.
Then, the extended left image information 2 is input to the output left image conversion unit 1111.

The second image separating section 1107 has a function opposite to that of the second image combining section 106 shown in FIG.

Third decoding section 1104 to kth decoding section 11
Reference numeral 05 denotes the first decoding unit 1102 and the third image separation unit 11
The 08th to kth image separation units 1109 are the first image separation unit 1
106 and the same configuration, respectively, and therefore detailed description thereof will not be repeated here.

The output right image converter 1110 generates a right image of a size required by the display device from the input basic right image information 1 and the extended right image information 2 to the extended right image information k. For example, when a right image or a left image that is twice the size of the basic image information 1 in the horizontal direction is generated, the thinning reverse processing shown in FIG. 2 is performed. That is, the input basic image information 1 and the difference image information that doubles the horizontal resolution are alternately overlapped in the horizontal direction to obtain a desired image.

When a right image or a left image having a size twice the size of the basic image information 1 in the vertical direction is required, the reverse process of the thinning process shown in FIG. 3 is performed. That is, the input basic image information 1 and the difference image information that doubles the vertical resolution are alternately overlapped in the vertical direction to obtain an image of a desired size.

Output left image conversion unit 1111 has the same function as the output right image conversion unit, and therefore detailed description thereof will not be repeated here.

The rearrangement processing unit 1112 converts the generated left image and right image to generate a format required by the display device. FIG. 12 shows a processing method when the display device uses the parallax barrier method or the lenticular plate arranging method. In this case, the input right image and left image are divided into vertical lines, and the vertical lines of the right image and the vertical lines of the left image are alternately arranged and output. In the case of a display device that alternately outputs the left and right images by the electronic shutter, the left image and the right image are alternately output in the time series direction.

With the stereoscopic image decoding apparatus according to this embodiment, it is possible to decode an image of a size required by the display apparatus by decoding only the basic bitstream and an appropriate extension bitstream.

<Seventh Embodiment> The seventh embodiment of the present invention will be described below.
A stereoscopic image decoding device according to the embodiment will be described. FIG. 13 shows a control block diagram of the stereoscopic image decoding device according to the present embodiment. The stereoscopic image decoding device shown in FIG. 13 is a device that decodes a bitstream encoded by the stereoscopic image encoding device according to the second embodiment described above. However, the bitstream decoded by the stereoscopic image decoding device according to the present embodiment is not limited to the bitstream encoded by the stereoscopic image coding device according to the second embodiment.

In the stereoscopic image decoding apparatus shown in FIG. 13, the output right image generation section 1110 and the output left image generation section 1111 of the stereoscopic image decoding apparatus according to the sixth embodiment described above are configured as shown in FIG. It is a thing. The other configuration is the same as that of the sixth embodiment described above. Therefore, detailed description thereof will not be repeated here.

The output right image generation unit and the output left image generation unit will be described with reference to FIG. In addition, in the following description, since the left and right are common configurations, the left and right are not distinguished unless otherwise specified.

The structure of the output image conversion unit depends on the image size required by the display device, as shown in FIGS.
There are three possible configurations (3). Of these, Figure 13
Since (2) and (3) are subsets of FIG. 13 (1), only FIG. 13 (1) will be described below.

The output image conversion unit shown in FIG. 13A is an H-LPF 130 that performs low-pass filter processing in the horizontal direction.
5 and H-LPF 1307, and H-HPF 1306 and H-HPF that perform high-pass filtering in the horizontal direction.
1308 and V that performs low-pass filter processing in the vertical direction
-LPF 1313, V-HPF 1314 that performs high-pass filtering in the vertical direction, up-sampling unit 1301 to up-sampling unit 1304 that up-samples pixels in the horizontal direction, up-sampling unit 1311 that up-samples pixels in the vertical direction, and up Sample part 13
12, combining unit 1309 for combining images, and combining unit 131
0 and the combining unit 1315.

The upsampling units 1301 to 1304 perform upsampling processing in the horizontal direction on the input basic right image information 1 and difference right image information 2 to difference right image information 4. As a result, an image having double the horizontal resolution is generated.

H-LPF1305, H-LPF1307
The H-HPF 1306 and the H-HPF 1308 input image information obtained by applying a low-pass filter and a high-pass filter to the upsampled input image, respectively, and input the image information to the synthesizing unit 1309 and the synthesizing unit 1310.

The synthesizer 1309 and the synthesizer 1310 are
Pixels of horizontal even position samples of the input image,
An image composed of pixels of odd-numbered samples in the horizontal direction of the other image is combined. Synthesis unit 1309 and synthesis unit 13
After up-sampling the output of 10 in the vertical direction by the up-sampling unit 1311 and the up-sampling unit 1312, a low-pass filter is applied by the V-LPF 1313,
In the PF 1314, the image information processed by applying the high pass filter is input to the combining unit 1315.

The synthesizing unit 1315 synthesizes and outputs an image consisting of pixels of even-numbered vertical position samples of the input image and pixels of odd-numbered vertical position samples of the other image to double the vertical resolution. To With such a configuration, the output image has double the resolution in both the horizontal and vertical directions.

By using the stereoscopic image decoding apparatus according to this embodiment, an image of a size required by the display apparatus can be obtained by only decoding the basic bit stream and an appropriate extended bit stream encoded with higher efficiency. Can be decrypted.

<Eighth Embodiment> The eighth embodiment of the present invention will be described below.
A stereoscopic image decoding device according to the embodiment will be described. FIG. 14 shows a control block diagram of the stereoscopic image decoding device according to the present embodiment. The stereoscopic image decoding device illustrated in FIG. 14 is a device that decodes a bitstream encoded by the stereoscopic image encoding device according to the above-described third embodiment. However, the bitstream decoded by the stereoscopic image decoding device according to the present embodiment is not limited to the bitstream coded by the stereoscopic image coding device according to the third embodiment.

The stereoscopic image decoding apparatus shown in FIG. 14 has the configuration shown in FIG. 15 for the first to kth decoding sections of the stereoscopic image decoding apparatus according to the sixth embodiment. . The other configuration is the same as that of the sixth embodiment described above. Therefore, detailed description thereof will not be repeated.

The second decoding unit 1401 shown in FIG. 14 uses the decoding information 1 decoded by the first decoding unit, decodes the extended bitstream selected by the bitstream selection unit 1101, and decodes the decoding information 2 Is generated and output to the second image separation unit 1107.

FIG. 15 is a control block diagram of the second decoding unit 1401. The second decoding unit 1401 includes a decoding processing unit 1501 and an addition unit 1502. Decryption processing unit 150
1 decodes the input extended bitstream to generate decoded image information. The addition unit 1502 adds the decoded image information and the separately input decoded information 1 to generate image information, and outputs the generated image information as difference image information 2.

Third decoding section 1402 to k-th decoding section 14
03 has the same configuration as the second decoding unit 1107,
Detailed description thereof will not be repeated here.

According to the stereoscopic image decoding device of the present embodiment, the display device needs only by decoding only the extended bitstream consisting of the difference between the basic bitstream and the image decoded from the appropriate basic bitstream. Images of size can be decoded.

<Ninth Embodiment> The ninth embodiment of the present invention will be described below.
A stereoscopic image decoding device according to the embodiment will be described. FIG. 16 shows a control block diagram of the stereoscopic image decoding device according to the present embodiment. The stereoscopic image decoding device illustrated in FIG. 16 is a device that decodes a bitstream encoded by the stereoscopic image encoding device according to the above-described fourth embodiment. However, the bitstream decoded by the stereoscopic image decoding device according to the present embodiment is not limited to the bitstream encoded by the stereoscopic image coding device according to the above-described fourth embodiment.

The stereoscopic image decoding apparatus according to this embodiment is
A bitstream selection unit 16 that selects only one appropriate extension bitstream from a plurality of extension bitstreams.
01 and the first decoding unit 1 for decoding the bitstream
102 and a second decoding unit 1602, and a first image separation unit 1 for extracting a right image and a left image from the decoded image
106, and a rearrangement processing unit 1112 for converting into a format required by the display device. Of these, the first decoding unit 1
102, first image separation unit 1106, rearrangement processing unit 1
112 is the same as in the sixth embodiment. Therefore, detailed description thereof will not be repeated here.

The bitstream selection unit 1601 selects only one appropriate bitstream from the input extension bitstream 2 to extension bitstream n, and inputs it to the second decoding unit 1602 as the selected bitstream.

The second decoding unit 1602 is the first decoding unit 1
The decoding information 1 decoded in 102 is used to generate the decoding information 2 obtained by decoding the selected bitstream.

FIG. 17 is a detailed control block diagram of second decoding section 1602. The second decoding unit 1602 includes a decoding processing unit 1701, an interpolation unit 1702, and an addition unit 1703.
Including and

The decoding processing section 1701 decodes the selected bitstream to generate a decoded image. Interpolator 1702
Generates an image having the same size as the decoded image by interpolating the pixels of the decoding information 1. The adding unit 1703 is used by the interpolation unit 170.
Decoding information 1 interpolated by 2 and decoding processing unit 1701
The decoded information 2 is generated by adding the image decoded by.

According to the stereoscopic image decoding apparatus of this embodiment, an image of a required size can be obtained only by selecting one of the basic bitstream and an appropriate bitstream, and the configuration of the apparatus can be simplified.

<Tenth Embodiment> A stereoscopic image decoding apparatus according to the tenth embodiment of the present invention will be described below. FIG. 18 shows a control block diagram of the stereoscopic image decoding device according to the present embodiment. The stereoscopic image decoding device illustrated in FIG. 18 is a device that decodes a bitstream encoded by the stereoscopic image encoding device according to the fifth embodiment described above. However, the bitstream decoded by the stereoscopic image decoding apparatus according to the present embodiment is not limited to the bitstream encoded by the stereoscopic image encoding apparatus according to the fifth embodiment described above.

The stereoscopic image decoding apparatus according to this embodiment is
A bitstream selection unit 1801 that selects only a necessary bitstream from the input extended bitstream
A first decoding unit 1802 to a 2k-th decoding unit 1807 that decodes the selected bit stream, and a (k + 1) -th addition unit 1808 to a 2k-th addition unit 1810 that add the decoded images. , Output right image conversion unit 1811 that generates image information necessary for display from input image information
And an output left image conversion unit 1812 and a rearrangement conversion unit 1813 that rearranges data from the input left and right images into a format required by the display device.

The first decoding unit 1802 generates the decoding information 1 when the basic bit stream 1 stored in the recording medium and transmitted through the communication line is input. The bitstream selection unit 1801 receives the extended bitstream 2 to the extended bitstream 2n stored in the recording medium and transmitted via the communication line (2n−).
1) (2k-1) pieces are selected from the extended bitstreams, and the second decoding section 1802 to the 2kth decoding section 180 are selected.
Input to 7 respectively. Here, 2k is the number of streams required by this stereoscopic image decoding apparatus.

The first decoding unit 1802 decodes the input basic bit stream 1 and outputs the decoding information 1,
The decoding information 1 is input to the output left image conversion unit 1811. The (k + 1) th decoding unit 1805 decodes the input extended bitstream and generates decoding information (k + 1).

The (k + 1) th addition section 1808 adds the decoded information 1 and the decoded information k + 1 and outputs the result to the output right image conversion section. Second decoding unit 1803 to kth decoding unit 18
04 is the first decoding unit 1802, and the (k + 2) th decoding unit 1806 to the 2kth decoding unit 1807 are the (k + 1) th decoding units.
Decoding unit 1805 and (k + 2) th addition unit 1809 to
The 2k-th addition unit 1810 is the (k + 1) -th addition unit 18
08, the detailed description thereof will not be repeated here.

Output right image conversion unit 1811, output left image conversion unit 1812 and rearrangement processing unit 1813 are the same as those in the sixth embodiment described above, and therefore detailed description thereof will not be repeated here. .

According to the stereoscopic image decoding apparatus of the present embodiment, the decoding apparatus for decoding the bit stream for the right image and the decoding apparatus for decoding the bit stream for the left image have different configurations, and thus are more versatile. It becomes possible to decode a stereoscopic image with high quality.

<Eleventh Embodiment> A three-dimensional image decoding apparatus according to the eleventh embodiment of the present invention will be described below. FIG. 19 shows a control block diagram of the stereoscopic image decoding device according to the present embodiment. The stereoscopic image decoding device illustrated in FIG. 19 is a device that decodes a bitstream encoded by the stereoscopic image encoding device according to the above-described first embodiment. However, the bitstream decoded by the stereoscopic image decoding apparatus according to the present embodiment is not limited to the bitstream encoded by the stereoscopic image encoding apparatus according to the first embodiment described above.

The stereoscopic image decoding apparatus according to this embodiment is
It has a configuration in which a 2D / 3D selection unit 1903 is added to the stereoscopic image decoding device shown in FIG. The other configuration is the same as that of the sixth embodiment described above. Therefore, detailed description thereof will not be repeated here.

The 2D / 3D selection unit 1902 shown in FIG.
When the display device displays a stereoscopic image, the output right image conversion unit 1901 and the output left image conversion unit 1902 output the image information as they are, and the rearrangement conversion unit 1904.
To enter. When the display device displays a two-dimensional image, the 2D / 3D selection unit 1902 outputs the output right image conversion unit 1901.
And only the image information output from any of the output left image conversion units 1902 is output.

According to the stereoscopic image decoding apparatus according to the present embodiment, since the 2D / 3D display information can be switched, even a decoding apparatus having no stereoscopic display function can display contents as a two-dimensional image. It becomes possible to do.

<Other Modifications> In the above-described embodiment, the stereoscopic image decoding apparatus shown in FIGS. 11, 14, 16, 18, and 19 has been described as including a bitstream selecting unit. It is not limited. For example, if the bitstream is selected and input to the stereoscopic image decoding device when the extended bitstream is transmitted, it is not necessary to provide the bitstream selection unit.

The stereoscopic image coding apparatus shown in FIGS. 1, 7 and 10 has been described as having a stereoscopic image composed of two left and right images, but the present invention is not limited to this. For example, by adding a camera and an input image generator,
It is also possible to use three or more multi-view stereoscopic image encoding devices. Similarly, although the stereoscopic image decoding apparatus shown in FIGS. 11, 14, 16, and 18 has been described as having a stereoscopic image composed of two left and right images, the present invention is not limited to this. For example, by adding an output image conversion unit, it is possible to provide a multi-view stereoscopic image decoding device of two or more eyes. According to such a multi-view stereoscopic image decoding device, when a viewer moves, it is possible to stereoscopically display a portion that cannot be seen until then.

Furthermore, in the above description, for the sake of convenience of description, it is assumed that a plurality of encoding units and a plurality of decoding units corresponding to each bit stream are provided, but the present invention is not limited to this. For example, a plurality of encoding units and a plurality of decoding units may be combined and implemented by one encoding unit and one decoding unit. In this case, for example, each bit stream may be encoded or decoded by time division processing.

The embodiments disclosed this time are to be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above description but by the claims, and is intended to include meanings equivalent to the claims and all modifications within the scope.

[Brief description of drawings]

FIG. 1 is a control block diagram of a stereoscopic image encoding device according to a first embodiment of the present invention.

FIG. 2 is a detailed control block diagram of the input image generation unit shown in FIG.

FIG. 3 is a diagram for explaining a horizontal thinning process in the input image generation unit shown in FIG.

FIG. 4 is a diagram illustrating vertical thinning processing in the input image generation unit illustrated in FIG. 1.

5 is a diagram for explaining a combining process of an image combining unit in FIG. 1. FIG.

FIG. 6 is a detailed control block diagram of an input image generation unit in the stereoscopic image encoding device according to the second embodiment of the present invention.

FIG. 7 is a control block diagram of a stereoscopic image encoding device according to a third embodiment of the present invention.

FIG. 8 is a detailed control block diagram of an input image generation unit in a stereoscopic image encoding device according to a fourth embodiment of the present invention.

9 is a diagram for explaining a generation process of an input image generation unit of FIG.

FIG. 10 is a control block diagram of a stereoscopic image encoding device according to a fifth embodiment of the present invention.

FIG. 11 is a control block diagram of a stereoscopic image decoding device according to a sixth embodiment of the present invention.

FIG. 12 is a diagram for explaining a rearrangement process of a rearrangement processing unit in FIG. 11.

FIG. 13 is a control block diagram of a stereoscopic image decoding device according to a seventh embodiment of the present invention.

FIG. 14 is a control block diagram of a stereoscopic image decoding device according to an eighth embodiment of the present invention.

FIG. 15 is a detailed control block diagram of second to kth decoding units shown in FIG.

FIG. 16 is a control block diagram of a stereoscopic image decoding device according to a ninth embodiment of the present invention.

17 is a detailed control block diagram of a second decoding unit shown in FIG.

FIG. 18 is a control block diagram of a stereoscopic image decoding device according to a tenth embodiment of the present invention.

FIG. 19 is a control block diagram of a stereoscopic image decoding device according to an eleventh embodiment of the present invention.

[Explanation of symbols]

101, 1001 right camera, 102, 1002 left camera, 103, 1003 input right image conversion unit, 104,
1004 input left image conversion unit, 105 first image combination unit, 106 second image combination unit, 107 nth image combination unit, 108, 701, 1012 first encoding unit, 1
09, 703, 1013 second encoding unit, 110, 7
05 nth encoding unit, 201 horizontal separation unit, 202,
203 vertical separation unit, 601, 801, 1305, 130
7 H-LPF, 602, 1306, 1308 H-H
PF, 603, 604, 607, 608, 611, 61
2, 803, 804, 806 Downsampling unit,
605, 609, 802, 805, 1313 V-LP
F, 606, 610, 1314 V-HPF, 702,
1006 first decoding unit, 1007 second subtraction unit, 1
Third decoding unit, 1009 Fourth subtraction unit, 10
10th (2n-1) decoding unit, 1011 2n subtraction unit, 1016 (2n-1) encoding unit, 1017
2n-th encoding unit 1101, 1601, 1801 bit stream selection unit, 1102, 1802 first decoding unit 1103, 1401, 1602, 1803 second
Decoding unit 1104, 1402 Third decoding unit 110
5, 1403, 1804 k th decoding unit, 1106 1st
Image separation unit, 1107 second image separation unit, 1108
Third image separation unit, 1109 kth image separation unit, 1109
110 output right image conversion unit, 1111 output left image conversion unit, 1112, 1813, 1904 rearrangement processing unit,
1301, 1302, 1303, 1304, 1311,
1312 up sample section, 1309, 1310, 13
15 combining unit, 1501, 1701 decoding processing unit, 15
02, 1703, 1808, 1809, 1810 adder, 1702 interpolator, 1805 (k + 1) th decoding unit, 1806 (k + 2) decoding unit, 1807 second
k decoding unit, 1811, 1812, 1901, 1902
Output image conversion unit, 1903 2D / 3D selection unit.

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5C059 KK38 LB04 LB05 LB15 MA00                       MA32 MA41 RB10 UA02 UA06                 5C061 AA03 AA07 AA11 AB04 AB08                       AB12                 5J064 AA02 BA15 BB04 BC02 BC11                       BC15 BD01

Claims (18)

[Claims] 1. A stereoscopic image encoding apparatus for encoding two or more image data corresponding to each of a plurality of viewpoints, wherein n pieces of image information (n is 2) based on each of the image data.
Creating means for creating the above integers), image combining means for combining the n pieces of image information corresponding to each of the viewpoints, and creating n pieces of combined image information; And a coding means for coding the combined image information of 1. to generate n bit streams. 2. The creating means includes means for creating the n pieces of image information corresponding to a device that decodes the bitstream and displays a stereoscopic image.
The three-dimensional image encoding device according to. 3. The three-dimensional image encoding device is required among n bit streams encoded by the encoding means, corresponding to a device for decoding the bit stream to display a three-dimensional image. Further comprising sending means for selecting a bitstream for sending to the device,
The stereoscopic image encoding device according to claim 1 or 2. 4. The stereoscopic image encoding device is required from the n bitstreams encoded by the encoding means, corresponding to a device for decoding the bitstream to display a stereoscopic image. The stereoscopic image encoding apparatus according to claim 1 or 2, further comprising: a sending means for selecting a bitstream and sending a bitstream including information representing the selected bitstream to the apparatus. 5. The creating means separates the image data into n pieces of image data according to horizontal sample positions, and the n pieces of image data created by separating the image data in the horizontal direction. A means for separating each into m pieces of image data (m is an integer of 2 or more) according to the vertical sampling position, and n × m pieces of image data created by separating the pieces in the horizontal and vertical directions 5. The stereoscopic image encoding device according to claim 1, further comprising means for creating the image information as the image information. 6. The first creating unit divides the image data into a low band and a high band in the horizontal direction, and the image data in the vertical direction. A second sub-band dividing means for dividing into a low band and a high band, and the first sub-band dividing means and the second sub-band dividing means are combined to form a horizontal low band and a vertical band. Image data sub-band divided into horizontal low band, horizontal low band and vertical high band image data, horizontal high band and vertical low band sub band Image data and image data sub-band-divided into a horizontal high band and a vertical high band are generated, and the first sub-band dividing means and the second sub-band are generated. The stereoscopic image encoding according to any one of claims 1 to 4, further comprising: means for creating image data obtained by recursively repeating a combination of subband dividing means as n pieces of image information. apparatus. 7. The encoding means includes means for encoding basic combined image information in n pieces of combined image information, basic combined image information obtained by decoding the basic combined image information, and the combined image information. 5. The stereoscopic image encoding device according to claim 1, further comprising: a unit for encoding difference information between and. 8. The creating unit includes a first reducing unit for horizontally reducing the image data, a second reducing unit for vertically reducing the image data, and the second reducing unit. Image data reduced by the first reducing means, image data reduced by the second reducing means, the first
And a means for creating the image data reduced by the second reducing means and the input image data as n pieces of image information (n = 4).
The stereoscopic image encoding device according to claim 1. 9. A stereoscopic image encoding apparatus for encoding two or more image data corresponding to each of a first viewpoint and a second viewpoint, wherein the stereoscopic image encoding device is based on the image data corresponding to the first viewpoint. creating means for creating n pieces of image information (n is an integer of 2 or more) and creating n pieces of image information based on the image data corresponding to the second viewpoint; and the first viewpoint. And a k-th (k = 1 to n) image corresponding to the first viewpoint, a first encoding unit for encoding n image information corresponding to A stereoscopic image encoding device comprising: information and a second encoding means for encoding difference information between the kth image information corresponding to the second viewpoint. 10. An image decoding apparatus which decodes n bit streams (n is an integer of 2 or more) to generate 2 or more image data corresponding to each of a plurality of viewpoints, wherein Selecting means for selecting a required bitstream from the bitstream; decoding means for decoding the selected bitstream to create combined image information; and creating the combined image information. Image separating means for separating and creating image information corresponding to each viewpoint, and creating means for creating output image information corresponding to each viewpoint based on the created image information And a conversion means for converting the output image information so as to match the output format. 11. The creating means includes means for creating output image information in which image information selected by the selecting means and decoded by the decoding means is combined so as to satisfy a predetermined condition. The stereoscopic image decoding device according to claim 10. 12. The creating means creates means for creating output image information in which the image information selected by the selecting means and decoded by the decoding means is combined so as to satisfy a predetermined resolution condition. The stereoscopic image decoding device according to claim 10, comprising: 13. The output image obtained by combining the plurality of pieces of image information selected by the selecting means in at least one of a horizontal direction and a vertical direction so that the plurality of pieces of image information are staggered. The stereoscopic image decoding device according to claim 10, further comprising means for creating information. 14. The creating means comprises: a first subband combining means for horizontally subband combining; a second subband combining means for vertically subband combining; 11. The unit according to claim 10, further comprising: a unit for performing subband synthesis by the first subband synthesis unit and the second subband synthesis unit to create output image information so as to satisfy the condition. Stereoscopic image decoding device. 15. The decoding means includes means for decoding basic combined image information from among n bit streams, means for decoding difference information corresponding to combined image information, and the decoded information. The stereoscopic image decoding apparatus according to claim 10, further comprising: a unit configured to create n pieces of combined image information based on the basic combined image information and the difference information. 16. The creating unit includes a first interpolating unit for horizontally interpolating the image data, a second interpolating unit for vertically interpolating the image data, and the selecting unit. A plurality of pieces of image information selected by the means and decoded by the decoding means are combined so as to satisfy a predetermined condition, and the data is complemented and output by the first interpolation means and the second interpolation means. The stereoscopic image decoding device according to claim 10, further comprising means for creating image information. 17. The stereoscopic image decoding device selects the necessary image information from among the image information corresponding to the plurality of viewpoints, whereby the stereoscopic image display image information and the two-dimensional image display image information are selected. The stereoscopic image decoding device according to claim 10, further comprising an image creating unit for creating any of the above. 18. n bit streams corresponding to each of the first viewpoint and the second viewpoint (n is an integer of 2 or more)
Is an image decoding apparatus that generates two or more image data corresponding to each of the first viewpoint and the second viewpoint by decoding n bit streams corresponding to the first viewpoint. A first decoding means for decoding and creating n pieces of image information; and a second decoding means for decoding n bit streams corresponding to the second viewpoint to create n pieces of difference information. Decoding means, k-th (k = 1 to n) image information corresponding to the first viewpoint decoded by the first decoding means, and the second
Creating means for creating image information corresponding to the second viewpoint based on the k-th (k = 1 to n) difference information corresponding to the second viewpoint decoded by the decoding means of A stereoscopic image decoding device including.