JP2014053782A - Stereoscopic image data processor and stereoscopic image data processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve a problem that a conventional stereoscopic image and parallax adjusting device of the stereoscopic image do not disclose a point that in which range a parallax angle of the stereoscopic image is to comfortably view the stereoscopic image when the whole screen of the stereoscopic image is viewed and a problem that it is unknown that in which parallax range the stereoscopic image can be viewed the most comfortably.SOLUTION: A stereoscopic image data processor (1) for parallax-adjusting stereoscopic image data being image data for stereoscopic vision includes: a stereoscopic image data input section (2) for inputting the stereoscopic image data; a parallax adjusting amount calculating section (6) for calculating a parallax adjusting amount in response to a parallax adjusting base angle being the parallax angle which is reference of parallax adjustment; and a parallax adjusting section (7) for parallax-adjusting the stereoscopic image data in response to the parallax adjusting amount and generating parallax adjustment stereoscopic image data. The parallax adjusting amount moves the parallax adjusting base angle to any value from -0.5 degree to -0.2 degree.

Description

  The present invention relates to a stereoscopic image data processing apparatus and a stereoscopic image data processing method.

  Conventionally, various methods for displaying a stereoscopic image have been proposed. Among them, what is generally used is a so-called “binocular type” that uses binocular parallax. When humans perceive the depth of an object placed in space, the position of the image projected to the left and right eyes, that is, binocular parallax, is used as one clue. This is a method of performing stereoscopic viewing by preparing a left-eye image and a right-eye image having binocular parallax and independently projecting them to the left and right eyes. At this time, a shift occurs between the left-eye image and the right-eye image according to the spatial position of the object. This is called “parallax”. That is, in the left-eye image and the right-eye image that are two-dimensional images, the three-dimensional spatial information is included as parallax. Different parallaxes indicate different three-dimensional positions. Therefore, by adjusting the parallax between the left and right images, the spatial position of the object projected on the image can be virtually adjusted, and as a result, the sense of depth can be manipulated. .

  The stereoscopic display using binocular parallax is different from the case where a human is actually looking at the subject, and may give a burden to the human. For this reason, Non-Patent Document 1 that compiles various research results on stereoscopic display and defines safety guidelines for generating stereoscopic image data using binocular parallax has been published.

  In stereoscopic display using binocular parallax, a subject is perceived at a location different from the display surface. On the other hand, the eye is in focus with the display surface. Therefore, if there is a difference between the perceived distance to the subject and the distance to the display surface, it causes fatigue and discomfort. Here, the difference between the convergence angle of the eye when the human eye looks at the display surface and the convergence angle of the eye with respect to the perceived position of the subject is referred to as a “parallax angle”. In the said nonpatent literature 1, it is supposed that the parallax angle of 1.0 degree or less is the range of the comfortable parallax angle.

  In addition, stereoscopic viewing is possible even with a parallax angle exceeding the range of a comfortable parallax angle. However, if the parallax angle is too large, the left and right images do not merge and appear double, making stereoscopic viewing impossible. Therefore, in the non-patent document 1, a parallax angle of 2.0 degrees is set as the fusion limit of the observer's stereoscopic vision.

  Also, if subjects with large disparity angles are mixed in one screen, the focus adjustment of the human eye cannot sufficiently follow the change in the disparity angle when a human moves his / her line of sight from one subject to another. is there. Therefore, it is desirable that the parallax angle within one screen be within a certain width. For this reason, in Non-Patent Document 1, the range of the parallax angle of a subject existing in one screen is called a “depth range”, and it is easy to see if this is suppressed to within 1 degree.

  Furthermore, since the human eye does not open outward, it is desirable to prevent parallax exceeding the distance between the binocular pupils on the display. Note in particular that children have a narrower interpupillary spacing. For this reason, in the said nonpatent literature 1, it shows that the interpupillary space | interval of a 6-year-old child is about 5 cm from the examination result of a pupil space | interval, and this is made into the representative value of a child in consideration of safety.

  Such stereoscopic images are currently used in some television broadcasts. Also, stereoscopic image package contents are sold. Basically, in these stereoscopic image data, the parallax angle of each subject in one screen is 1.0 ° or less and the depth range in one screen is 1.0 according to the conditions described in Non-Patent Document 1. It is produced to be within the degree.

  However, it is not always easy to create a stereoscopic image so as to always satisfy the condition of Non-Patent Document 1. A stereoscopic image by CG can be handled because the parallax is known in advance, but for example, in a scene where the subject moves from the back to the front in a live-action video, the range of the parallax angle of the subject or the parallax angle within one screen is continuously Change. Following this, it is difficult to adjust the camera so that the parallax angle satisfies the above conditions.

  In addition, stereoscopic images created by individuals are disclosed on the Internet or the like. These do not necessarily satisfy the conditions of Non-Patent Document 1.

  On the other hand, in a device that displays a stereoscopic image, a technique has been developed that automatically adjusts the parallax so that a human can safely see it.

  FIG. 33 is a configuration diagram of a conventional stereoscopic image display device described in Patent Document 1. In FIG. According to the description in Patent Literature 1, the parallax calculation unit 129 calculates a parallax map (parallax at each coordinate for the entire screen) from the left and right eye images. The gaze point calculation unit 130 calculates an average value ΔXave of the parallax of the entire screen based on the parallax map. The parallax control unit 131 moves the image in a direction in which the parallax is canceled by the average parallax value ΔXave. That is, the parallax of the stereoscopic image is adjusted so that the average value of the parallax of the stereoscopic image after parallax adjustment is zero. By doing so, an image part with a parallax of ΔXave is reproduced on the display display surface of the image display unit 132. By this adjustment, it is possible to display a stereoscopic image that is easy to see and less fatigued.

  Patent Document 1 describes that the shift amount in the parallax control unit 131 may be changed to ΔXave−α (α is an arbitrary value). In this way, an image part having a parallax of ΔXave can be set to a certain binocular parallax value α. The case where α = 0 indicates the image display surface position, and this position is on the near side or the rear side with respect to the image display surface depending on the value of α.

  As described above, the stereoscopic image indicated by the average value ΔXave of the parallax can be controlled on the display surface of the display or at a predetermined position, and the observer can always perceive the depth world of the subject in the widest range.

Patent No. 3089306

“3DC Safety Guidelines”, [online], April 20, 2010, 3D Consortium Safety Guidelines Subcommittee, [searched on September 15, 2010], Internet <URL: https://www.3dc.gr.jp /jp/scmt_wg_rep/3dc_guideJ_20100420.pdf>

  However, Non-Patent Document 1 merely defines a safe range by paying attention to extreme parallax angles such as the latest view and the farthest view in one screen. However, when an observer actually sees a stereoscopic image, he / she sees the entire screen including subjects having various parallaxes. As described above, Non-Patent Document 1 does not mention the range in which the parallax angle of the stereoscopic image can be comfortably viewed when viewing the entire screen. Even within the range of the conditions of Non-Patent Document 1, various parallax angle ranges can be set. Accordingly, there is a problem that even a stereoscopic image that satisfies the conditions of Non-Patent Document 1 cannot always be viewed comfortably.

  Moreover, although the stereoscopic image display device described in Patent Document 1 discloses that the parallax is always adjusted so that the observer can perceive the depth world of the subject in the widest range, the stereoscopic image is specifically displayed. There is no disclosure about what range of parallax can be comfortably viewed. That is, until now, there has been a problem that it is unclear which parallax range allows the most comfortable viewing of a stereoscopic image.

  The present invention has been made in view of such a problem, and an object of the present invention is to provide an optimal parallax angle range so that parallax adjustment can be performed so that the parallax is easy to see and comfortable as an entire stereoscopic image. A stereoscopic image data processing device and a stereoscopic image data processing method are provided.

  In order to solve the above problems, according to one aspect of the present invention, a stereoscopic image data processing device according to the present invention provides stereoscopic image data for performing parallax adjustment on stereoscopic image data that is image data for stereoscopic viewing. In the processing device, a stereoscopic image data input unit that inputs stereoscopic image data, a parallax adjustment amount calculation unit that calculates a parallax adjustment amount based on a parallax adjustment base angle that is a reference parallax angle for parallax adjustment, and a parallax adjustment amount A parallax adjustment unit configured to perform parallax adjustment on the stereoscopic image data and generate parallax adjustment stereoscopic image data, and the parallax adjustment amount may be set such that the parallax adjustment base angle is between −0.5 degrees and −0.2 degrees. It moves to either value of these, It is characterized by the above-mentioned.

  According to another aspect of the present invention, in the stereoscopic image data processing device according to the present invention, the parallax adjustment base angle may be 0 degrees.

  According to another aspect of the present invention, the stereoscopic image data processing device according to the present invention further includes a parallax calculation unit that calculates a parallax map from the stereoscopic image data, and the parallax adjustment amount calculation unit uses the parallax map. The parallax adjustment base angle may be calculated.

  According to another aspect of the present invention, in the stereoscopic image data processing device according to the present invention, the parallax adjustment base angle is a median value of the parallax angle value of the farthest view and the parallax angle value of the nearest scene in the parallax map. There may be.

  According to another aspect of the present invention, in the stereoscopic image data processing device according to the present invention, the parallax adjustment base angle may be a parallax angle having the highest appearance frequency in the parallax map.

  According to another aspect of the present invention, in the stereoscopic image data processing device according to the present invention, the parallax adjustment base angle may be an average value of parallax angles in a parallax map.

  According to another aspect of the present invention, in the stereoscopic image data processing device according to the present invention, the parallax adjustment base angle has a minimum sum of squares of differences from the respective parallax angles in the parallax map with respect to the stereoscopic image data. The determined parallax angle may be used.

  According to another aspect of the present invention, a stereoscopic image data processing method according to the present invention is a stereoscopic image data processing method for performing parallax adjustment on stereoscopic image data that is image data for stereoscopic viewing. Stereo image data input step for inputting image data, parallax adjustment amount determination step for calculating a parallax adjustment amount based on a parallax adjustment base angle that is a parallax angle used as a reference for parallax adjustment, and stereoscopic image data based on the parallax adjustment amount A parallax adjustment step for adjusting the parallax and generating parallax-adjusted stereoscopic image data, and the parallax adjustment amount is any of a parallax adjustment base angle from -0.5 degrees to -0.2 degrees It moves to a value.

  According to another aspect of the present invention, in the stereoscopic image data processing method according to the present invention, the parallax adjustment base angle may be 0 degrees.

  According to another aspect of the present invention, the stereoscopic image data processing method according to the present invention further includes a parallax calculation step of calculating a parallax map from the stereoscopic image data, and the parallax adjustment amount determination step uses a parallax map. Thus, the parallax adjustment base angle may be calculated.

  According to another aspect of the present invention, a stereoscopic image data processing apparatus according to the present invention is a stereoscopic image data processing apparatus that performs parallax adjustment on stereoscopic image data that is image data for stereoscopic viewing. A stereoscopic image data input unit that inputs a parallax, a parallax calculation unit that calculates a parallax map from the stereoscopic image data, and a parallax map to calculate the parallax angle of the nearest scene of the stereoscopic image data, and based on the parallax angle of the nearest scene A parallax adjustment amount calculation unit that calculates a parallax adjustment amount; and a parallax adjustment unit that performs parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generates parallax adjustment stereoscopic image data. The parallax angle of the scene is moved to any value from +0.3 degrees to 0.0 degrees.

  According to another aspect of the present invention, a stereoscopic image data processing method according to the present invention is a stereoscopic image data processing method for performing parallax adjustment on stereoscopic image data that is image data for stereoscopic viewing. A stereoscopic image data input step for inputting image data, a parallax calculation step for calculating a parallax map from the stereoscopic image data, and a parallax angle of the nearest scene of the stereoscopic image data are calculated using the parallax map. A parallax adjustment amount determining step for calculating a parallax adjustment amount based on the parallax, a parallax adjustment step for performing parallax adjustment on the stereoscopic image data based on the parallax adjustment amount, and generating parallax adjustment stereoscopic image data. Is characterized in that the parallax angle of the recent scene is moved to any value from +0.3 degrees to 0.0 degrees.

  According to another aspect of the present invention, a stereoscopic image data processing apparatus according to the present invention is a stereoscopic image data processing apparatus that performs parallax adjustment on stereoscopic image data that is image data for stereoscopic viewing. 3D image data input unit, a parallax calculation unit that calculates a parallax map from the stereoscopic image data, and the parallax map, the parallax angle of the farthest view of the stereoscopic image data is calculated, and based on the parallax angle of the farthest view A parallax adjustment amount calculation unit that calculates a parallax adjustment amount; and a parallax adjustment unit that performs parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generates parallax adjustment stereoscopic image data. The disparity angle of the distant view is moved to any value from -0.7 degrees to -1.0 degrees.

  According to another aspect of the present invention, a stereoscopic image data processing method according to the present invention is a stereoscopic image data processing method for performing parallax adjustment on stereoscopic image data that is image data for stereoscopic viewing. A stereoscopic image data input step for inputting image data, a parallax calculation step for calculating a parallax map from the stereoscopic image data, a parallax angle of the farthest view of the stereoscopic image data is calculated using the parallax map, and a parallax angle of the farthest view A parallax adjustment amount determining step for calculating a parallax adjustment amount based on the parallax, a parallax adjustment step for performing parallax adjustment on the stereoscopic image data based on the parallax adjustment amount, and generating parallax adjustment stereoscopic image data. Is characterized in that the parallax angle of the farthest view is moved to any value from -0.7 degrees to -1.0 degrees.

  According to another aspect of the present invention, a stereoscopic image data processing apparatus according to the present invention is a stereoscopic image data processing apparatus that performs parallax adjustment on stereoscopic image data that is image data for stereoscopic viewing. A stereoscopic image data input unit that inputs a parallax, a parallax calculation unit that calculates a parallax map from the stereoscopic image data, a parallax angle of the farthest view of the stereoscopic image data using the parallax map, A parallax adjustment amount calculation unit that calculates an adjustment amount, and a parallax adjustment unit that performs parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generates parallax adjustment stereoscopic image data. The width of the range of the parallax angle included in the parallax map does not exceed any value in the range of -0.1 degrees to 0.0 degrees, and all the parallax angles included in the parallax map are negative The farthest view And calculates the parallax adjustment amount for moving the angle difference to any value of -1.0 degrees -0.7 degrees.

  According to another aspect of the present invention, a stereoscopic image data processing method according to the present invention is a stereoscopic image data processing method for performing parallax adjustment on stereoscopic image data that is image data for stereoscopic viewing. A stereoscopic image data input step for inputting image data, a parallax calculation step for calculating a parallax map from the stereoscopic image data, a parallax angle of the farthest view of the stereoscopic image data using the parallax map, and a parallax angle of the farthest view A parallax adjustment amount determination step for calculating a parallax adjustment amount based on the parallax adjustment amount, and a parallax adjustment step for performing parallax adjustment on the stereoscopic image data based on the parallax adjustment amount to generate parallax adjustment stereoscopic image data. In the step, the width of the range of the parallax angle included in the parallax map does not exceed any value from −0.1 degrees to 0.0 degrees, and all the widths included in the parallax map If the difference angle is negative, and calculates the parallax adjustment amount for moving the parallax angle of the most distant one of the values of -1.0 degrees -0.7 degrees.

  According to the present invention, by providing an optimal range of parallax angles, it is possible to perform parallax adjustment so that a parallax that is easy to see as a whole stereoscopic image becomes comfortable.

It is a block diagram which shows the structure of the stereo image data display apparatus with which the stereo image data processing apparatus by embodiment of this invention is applied. It is a figure which shows the mode of imaging | photography of a stereo image data input part. It is a figure which shows stereo image data and its parallax. It is a figure explaining the convergence angle when a viewer looks at an image without parallax. It is a figure explaining the angle of convergence when a viewer views stereoscopic image data having parallax so that the subject can be seen in front of the screen. It is a figure explaining the angle of convergence when a viewer views stereoscopic image data having parallax so that a subject can be seen behind the screen. It is the figure which showed typically the parallax angle of the to-be-photographed object of stereo image data, and the apparent position of the to-be-photographed object. It is a figure explaining an example of the method of parallax adjustment. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is a figure explaining another example of the method of parallax adjustment. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is the flowchart which showed the operation | movement (stereoscopic image data processing method) of the stereo image data processing apparatus in embodiment of this invention. It is a flowchart which shows operation | movement of the parallax adjustment amount calculation part in embodiment of this invention. It is a flowchart about the operation | movement at the time of calculating the parallax adjustment amount by normal mode. It is the figure which showed typically the parallax angle of the to-be-photographed object of stereo image data, and the apparent position of the to-be-photographed object. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is the figure which showed typically the parallax angle of the to-be-photographed object of stereo image data, and the apparent position of the to-be-photographed object. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is a flowchart about the operation | movement at the time of calculating the parallax adjustment amount by an emphasized parallax adjustment mode. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is a flowchart about the operation | movement at the time of calculating the parallax adjustment amount by an emphasized parallax adjustment mode. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is a figure which shows stereo image data and its parallax. It is the figure which showed typically the parallax angle of the to-be-photographed object of stereo image data, and the apparent position of the to-be-photographed object. It is a figure explaining an example of the method of parallax adjustment. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is a flowchart about the process which corrects the parallax adjustment amount so that the parallax of the farthest view does not exceed 5 cm in the spreading direction. It is the figure which showed typically the parallax angle of the to-be-photographed object of the stereo image data after parallax adjustment, and the apparent position of the to-be-photographed object. It is the figure which looked at the mode of verification experiment of comfortable parallax from the top. It is the figure which showed typically the parallax angle range before parallax adjustment of the stereo image data in the subjective evaluation experiment, the parallax angle range after parallax adjustment, and the comfortable parallax angle range. It is a block diagram of the conventional stereo image display apparatus.

  Exemplary embodiments of a stereoscopic image data processing device and a stereoscopic image data processing method according to the present invention will be described below in detail with reference to the accompanying drawings. Here, it is assumed that the same reference numerals in different drawings are the same, and the description thereof is omitted.

  In the following description, “2D” is used as a term meaning two dimensions, and stereoscopic image data is referred to as stereoscopic image data, and normal two-dimensional image data is referred to as 2D image data. The “image” described in the present description indicates both a still image and a moving image. In addition, the fact that the stereoscopic image data appears to pop out before the screen is defined as “popping out”, and the appearance that the stereoscopic image data appears to be retracted from the screen is defined as “retraction”.

  First, the terms used in this specification will be described together.

  “Parallax” is the amount of shift between the left-eye image and the right-eye image for a certain subject in the stereoscopic image data composed of the left-eye image and the right-eye image. The parallax value may be signed. When adding a sign, if the parallax is positive, the position of the area displayed with the parallax will be a “pop-out” image in front of the screen, and if the parallax is negative, the position of the area displayed with the parallax Indicates that the video is “retracted” behind the screen.

  The amount of parallax can be represented by the number of pixels in stereoscopic image data, for example. Alternatively, the amount of parallax can also be expressed as a distance on the screen of a display device that displays stereoscopic image data. In this case, the amount of parallax varies depending on the size of the screen. Alternatively, the amount of parallax can also be expressed as a percentage of the number of pixels in the horizontal direction of the screen. If the number of pixels in the horizontal direction of the stereoscopic image data and the screen size of the display device are known, these values can be converted into each other. In this specification, it is expressed by the number of pixels.

  The “parallax angle” is a difference between the convergence angle of the eye when the human eye looks at the display surface and the convergence angle of the eye with respect to the perceived position of the subject. The parallax value and the parallax angle value can be converted into each other by calculation. The conversion method will be described later.

  A sign may also be attached to the value of the parallax angle. As with the parallax sign, when the parallax angle is positive, the position of the area displayed at that parallax angle is a “pop-out” image that is in front of the screen, and when the parallax angle is negative, it is displayed at that parallax angle. It is assumed that the position of the area to be displayed is a “retraction” image that is behind the screen.

  The “parallax map” is obtained by obtaining the parallax for each pixel of the stereoscopic image data or for each block of a predetermined size over the entire stereoscopic image data. Similarly, a parallax angle obtained for each pixel of stereoscopic image data or for each block of a predetermined size over the entire stereoscopic image data is referred to as a “parallax angle map”.

  The “farthest view” is an area where the parallax or the parallax angle is the smallest in the stereoscopic image data, and the “most recent view” has the largest parallax or parallax angle as opposed to the “farthest view”. This area is the area that appears to protrude the most.

  The “parallax range” is a range determined by the parallax of the farthest view and the parallax of the latest view of the stereoscopic image data existing on one screen. For example, the parallax range of the stereoscopic image data in which the parallax of the farthest view is −50 pixels and the parallax angle of the nearest scene is +50 pixels is −50 pixels to +50 pixels.

  The “parallax angle range” is a range determined by the parallax angle of the farthest view and the parallax angle of the nearest view. For example, the parallax angle range of stereoscopic image data in which the parallax angle of the farthest view is -1.0 degrees and the parallax angle of the nearest scene is +1.0 degrees is -1.0 degrees to +1.0 degrees.

  The “parallax range width” is the width of the parallax range, and is the absolute value of the difference between the parallax of the nearest scene and the parallax of the farthest view. For example, when the parallax range is −50 pixels to +50 pixels, the parallax range width is 100 pixels.

  Similarly, the “parallax angle range width” is the width of the parallax angle range, and is the absolute value of the difference between the parallax angle of the nearest scene and the parallax angle of the farthest view. For example, when the parallax angle range is −1.0 degree to +1.0 degree, the parallax angle range width is 2.0 degrees.

  As described above, “parallax” and “parallax angle” can be converted to each other and are not essentially different. Regarding matters related to the processing of image data, it is often easier to explain by using the term “parallax” that can be expressed in terms of the number of pixels. On the other hand, with regard to matters related to perception, such as comfort when a human sees stereoscopic image data, it is often more appropriate to discuss the parallax angle defined by the vergence angle of the eyes.

  In general, the fact that there is a difference between the left-eye image data and the right-eye image data of the stereoscopic image data itself is expressed as “parallax”.

  Based on such differences, in this specification, “parallax” and “parallax angle” are appropriately used depending on the situation.

  In the following description, for simplification of description, when obtaining the parallax, the parallax is calculated by searching corresponding points of the right-eye image data with respect to the left-eye image data. The parallax may be calculated by searching corresponding points of the image data for the left eye with respect to the data.

<1. Outline of Configuration of Stereoscopic Image Data Processing Device According to Embodiment of Present Invention>
FIG. 1 is a block diagram showing a configuration of a stereoscopic image data display device 75 to which the stereoscopic image data processing device 1 according to the embodiment of the present invention is applied. First, the outline of the configuration of the stereoscopic image data display device 75 will be described with reference to FIG.

  The stereoscopic image data display device 75 includes the stereoscopic image data processing device 1 and the image display unit 8. The stereoscopic image data processing apparatus 1 includes a stereoscopic image data input unit 2, a user instruction input unit 3, a parallax calculation unit 4, a parallax angle calculation unit 5, a parallax adjustment amount calculation unit 6, a parallax adjustment unit 7, It has. As a specific example of the stereoscopic image data processing device 1, for example, a television, a monitor, a mobile phone, a smartphone, a PC (Personal Computer), a portable game machine, an electronic photo frame, which can display stereoscopic image data, A digital still camera, a video camera, etc. are mentioned.

  The stereoscopic image data input unit 2 inputs stereoscopic image data and outputs the stereoscopic image data to the parallax calculation unit 4.

  The user instruction input unit 3 outputs the comfortable parallax adjustment mode M to the parallax adjustment amount calculation unit 6 in accordance with an instruction input from the user.

  The parallax calculation unit 4 sends the input stereoscopic image data to the parallax adjustment unit 7, calculates a parallax map of the stereoscopic image data, and inputs the parallax map to the parallax angle calculation unit 5.

  The parallax angle calculation unit 5 calculates a parallax angle map from the input parallax map and outputs the parallax angle map to the parallax adjustment amount calculation unit 6.

  The parallax adjustment amount calculation unit 6 calculates a parallax adjustment amount P used when performing parallax adjustment on the stereoscopic image data corresponding to the parallax angle map based on the input parallax angle map and the comfortable parallax adjustment mode M. , And input to the parallax adjustment unit 7.

  The parallax adjustment unit 7 outputs parallax adjustment stereoscopic image data generated by performing parallax adjustment processing on the input stereoscopic image data based on the parallax adjustment amount P, and inputs the parallax adjustment stereoscopic image data to the image display unit 8.

  The image display unit 8 receives the display signal of the parallax adjusted stereoscopic image data output from the parallax adjustment unit 7 and displays the stereoscopic image data on the screen included in the image display unit 8 based on the signal.

  The image display unit 8 alternately displays the image data for the left eye and the image data for the right eye on a liquid crystal display, a plasma display, a projector, etc. The image display device may be an image display device that is operated in synchronization with the display timing, and that is based on a stereoscopic display method generally called an active shutter method. In addition, the display is a polarizing glasses type image display device in which a special polarizing filter having different polarization for each line is attached to the display, and stereoscopic viewing is performed through polarized glasses that transmit only light of different polarization between the left and right eyes. May be. Furthermore, the image may be displayed by a liquid crystal display capable of autostereoscopic viewing such as a parallax barrier method or a lenticular method.

  In addition to a twin-lens type, a liquid crystal display using image data from a plurality of viewpoints may be used. For example, it is a liquid crystal display that displays a plurality of viewpoints at the same time to perform autostereoscopic viewing, or a two-lens stereoscopic display that tracks the viewer's eyes and switches the viewpoint according to the position of the viewer's eyes. Also good.

<2. Detailed description of components>
Next, among the components of the stereoscopic image data processing apparatus 1 according to the present embodiment, the stereoscopic image data input unit 2, the parallax calculation unit 4, the parallax angle calculation unit 5, and the parallax adjustment unit 7 will be described in more detail. The user instruction input unit 3 and the parallax adjustment amount calculation unit 6 will be described in <3. The operation of the stereoscopic image data processing apparatus 1 in the embodiment of the present invention (stereoscopic image data processing method)> will be described in detail.

<2-1. Stereoscopic image data input unit 2>
The stereoscopic image data input unit 2 can use at least one imaging device that captures and acquires stereoscopic image data, for example. In addition, a device that reads stereoscopic image data from an electronic medium such as a magneto-optical disk typified by a DVD or Blu-ray (registered trademark) Disk, a semiconductor memory typified by a USB memory or an SD (registered trademark) card, or the like. There may be. Further, it may be a television signal receiving device that receives broadcast waves of stereoscopic broadcasting, or a device that receives stereoscopic image data distributed from the Internet or other communication lines. Alternatively, an HDMI (registered trademark) (High-Definition Multimedia Interface) receiver that receives an image signal from an external device such as a Blu-ray (registered trademark) disc player may be used. That is, any device may be used as long as it is a device for inputting stereoscopic image data.

  The stereoscopic image data format is, for example, a top-and-bottom format (a format in which a left-eye image and a right-eye image are stored vertically as one frame image) or a side-by-side format (a left-eye image and a right-eye format). There are various formats such as a format in which images are stored as one-frame images so that the images are arranged horizontally) and a frame sequential format (a format in which a left image and a right image are input over time). Any of these formats may be used.

  Below, in order to demonstrate easily, the case where two cameras are used as the stereo image data input part 2 is demonstrated.

  FIG. 2 is a diagram illustrating a state in which the stereoscopic image data input unit 2 configured by two cameras captures stereoscopic image data of two left and right viewpoints. As shown in FIG. 2, the left camera 9 </ b> L that captures the left-eye image data and the right camera 9 </ b> R that captures the right-eye image data that are arranged at a predetermined distance in the horizontal direction are connected to the subject 10 in the foreground. A scene including the subject 11 in a distant view is photographed. The subject 10 is a car. The subject 11 is a scene in which clouds float on the waved sea and the sun is out.

  The predetermined distance between the cameras at this time is generally called a baseline length. For example, the base line length may be about 63 mm, which is the same as the distance between human eyes, and the base line length may be about 30 mm in order to reduce the overall parallax. Conversely, when it is desired to increase the parallax when shooting a distant view, for example, the baseline length may be set to several meters. In the embodiment of the present invention, the baseline length is 63 mm.

  The optical axes of the cameras may intersect or be parallel. In the embodiment of the present invention, they are parallel.

  In the above description, the number of cameras constituting the stereoscopic image data input unit 2 is two, but three or more cameras may be used. Further, the configuration may be such that stereoscopic image data is obtained by sliding and moving one camera and photographing from different positions.

<2-2. Parallax calculation unit 4>
First, the relationship between stereoscopic image data and parallax will be described.

  FIG. 3 is a diagram showing stereoscopic image data and its parallax. 3A shows the left-eye image data 12L, and FIG. 3B shows the left-eye image data 12R. As described above, the positions of the subject 10 and the subject 11 are shifted in the left-eye image data 12L and the right-eye image data 12R. This deviation is parallax. In addition, the subject 10 and the subject 11 have different ways of shifting the positions. In the right-eye image data 12R, the subject 10 is shifted to the left side, while the subject 11 is shifted to the right side compared to the left-eye image data 12L. This is because the parallax changes depending on the distance from the camera to the subject.

  In the left-eye image data 12L, an area that appears closest when stereoscopic display is set as the nearest point 13L, and an area that appears farthest is set as the farthest point 14L. In FIG. 3A, the door mirror part of the car is the closest point 13L, and the background sun part is the farthest point 14L. In the left-eye image data 11R, a point corresponding to the closest point 13L is set as the closest point 13R, and a point corresponding to the farthest point 14L is set as the farthest point 14R. In the following description, regardless of the left-eye image data 12L or the right-eye image data 12R, the nearest point is referred to as “nearest point 13”, and the farthest point is referred to as “farthest point 14”. Call.

  For such a stereoscopic image, the parallax is expressed as follows.

  The distance from the left end of the left-eye image data 12L to the nearest point 13L is dnL1, and the distance to the farthest point 14L is dfL1. The distance from the left end of the right-eye image data 12R to the nearest point 13R is dnR1, and the distance to the farthest point 14R is dfR1. Then, the parallax at the nearest point 12 is expressed as dnL1-dnR1. Further, the parallax at the farthest point 13 is expressed as dfL1-dfR1.

  The parallax calculation unit 4 calculates the parallax by the above method for each corresponding point searched by the corresponding point search process of the left-eye image and the right-eye image, and calculates a parallax map. The unit of the corresponding point search process is a block unit, but the size of the block is not particularly limited, and may be a pixel unit or the entire screen may be one block. Specifically, for example, the corresponding point search process performs stereo matching on the pixel data for the left-eye image data and the right-eye image data. The corresponding point search process may be any other method.

<2-3. Parallax angle calculator 5>
Next, the process in the parallax angle calculation part 5 is demonstrated using FIGS.

  FIG. 4 is a diagram for explaining the convergence angle when a viewer views an image without parallax. An image having no parallax is displayed on the screen 16. A viewer (not shown) faces the screen 16 and sees a point 17 that is one point of the image displayed on the screen 16 through the left eye 15L and the right eye 15R. Note that point 17 is located in front of the viewer.

  When the point 17 and the left eye 15L are connected by a straight line, the point where the straight line intersects the left eye 15L is defined as a point 18L, and when the point 17 and the right eye 15R are connected by a straight line, the straight line becomes the right eye 15R. The point that intersects with is designated as point 18R. Further, an intersection of a perpendicular line dropped from the point 17 with respect to a line segment connecting the points 18L and 18R is defined as a point 19. As described above, since the point 17 is in front of the viewer, the position of the point 19 is located at the center of the line segment connected by the points 18L and 18R. The angle formed by the line connecting point 17 and point 18L and the line connecting point 17 and point 18R is the convergence angle. The convergence angle in FIG. 4 is defined as a convergence angle α.

  Here, the line segment connecting the points 18L and 18R represents the binocular interval which is the distance between the left eye and the right eye. The length is T. A line segment connecting the points 17 and 19 represents the viewing distance from the viewer to the screen. Let that length be L. Then, the relationship as shown in Expression (1) is derived among the binocular interval T, the viewing distance L, and the convergence angle α.

  From Expression (1), the convergence angle α is expressed as Expression (2).

  FIG. 5 is a diagram for explaining the convergence angle when the viewer views stereoscopic image data having parallax so that the subject can be seen in front of the screen. Among the stereoscopic image data, the left eye 15L image has a subject at the point 20L, and the right eye 15R image has the subject at the point 20R. The distance between the point 20L and the point 20R is parallax. This is d1. A viewer (not shown) faces the screen 16. When the viewer views the displayed stereoscopic image through the left eye 15L and the right eye 15R, it appears as if the subject is at the position of the point 21 in front of the screen 16. It is assumed that a stereoscopic image is created so that the point 21 is positioned in front of the viewer.

  As in FIG. 4, points 18L, 18R, and 19 are taken. A line segment connecting the points 21 and 19 represents the distance from the viewer to the apparent subject position. Let that length be m. The viewing distance from the viewer to the screen is L as in FIG. The angle formed by the line segment connecting point 21 and point 18L and the line segment connecting point 21 and point 18R is the convergence angle. The convergence angle in FIG. 5 is defined as a convergence angle β1. Then, a relationship such as Expression (3) is derived among the binocular interval T, the viewing distance L, the distance m to the subject, and the convergence angle β1.

  From Expression (3), the convergence angle β1 is expressed as Expression (4).

  In addition, a relationship such as Equation (5) is derived from the similar relationship of triangles.

  From Expressions (4) and (5), the convergence angle β1 is expressed as Expression (6).

  Here, the parallax angle γ1 with respect to the parallax in the direction of jumping out from the screen is the convergence angle α when the viewer views the screen and the stereoscopic image data having the parallax in the direction of jumping out of the screen Therefore, the parallax angle γ1 is expressed as in Expression (7).

  FIG. 6 is a diagram for explaining the angle of convergence when the viewer views stereoscopic image data having parallax so that the subject can be seen behind the screen. Among the stereoscopic image data, the left eye 15L image has a subject at the point 22L, and the right eye 15R image has the subject at the point 22R. The distance between the point 22L and the point 22R is parallax. This is d2. A viewer (not shown) faces the screen 16. When the viewer views the displayed stereoscopic image through the left eye 15L and the right eye 15R, it appears as if the subject is at the position of the point 23 at the back of the screen 16. It is assumed that the stereoscopic image is created so that the point 23 is positioned in front of the viewer.

  As in FIG. 4, points 18L, 18R, and 19 are taken. A line segment connecting the points 23 and 19 represents the distance from the viewer to the apparent subject position. Let the length be n. The viewing distance from the viewer to the screen is L as in FIG. The angle formed by the line connecting point 23 and point 18L and the line connecting point 23 and point 18R is the convergence angle. The convergence angle in FIG. 6 is defined as a convergence angle β2. Then, a relationship such as Expression (8) is derived among the binocular interval T, the viewing distance L, the distance n to the subject, and the convergence angle β2.

  From Expression (8), the convergence angle β2 is expressed as Expression (9).

  Further, a relationship such as Equation (10) is derived from the similar relationship of triangles.

  From Expressions (9) and (10), the convergence angle β2 is expressed as Expression (11).

  Here, the parallax angle γ2 with respect to the parallax in the direction retracted from the screen is the convergence angle α when the viewer looks at the screen and the convergence when viewing the stereoscopic image data having the parallax in the direction retracted from the screen. Since this is the absolute value of the difference between the angles β2, the parallax angle γ2 is expressed as in Expression (12).

  The smaller the value of the parallax angle, the closer the stereoscopic image data is to the screen. On the contrary, as the value of the parallax angle increases, the position where the stereoscopic image data can be seen appears more popping out or more retracting.

  In the present embodiment, in order to make the explanation easy to understand, a sign is further added to the absolute parallax angle defined above as described above, and the parallax angle with respect to the parallax in the direction retracted from the screen is a negative value. The parallax angle with respect to the parallax in the direction of popping out more forward is a positive value. As a result, by referring to the sign of the value of the parallax angle, it can be clearly seen whether the position where the stereoscopic image data can be seen is protruding or retracting. For example, when the sign of the parallax angle is negative, the subject appears to be retracted deeper than the screen, and as the value decreases, it appears to be retracted deeper. Conversely, when the sign of the parallax angle is positive, the subject appears to pop out from the screen, and as the value increases, the subject appears to pop out.

  Furthermore, in the above description, the parallaxes d1 and d2 are both described as absolute values. However, like the parallax angle, d1 is a parallax in the pop-out direction, and a positive value, and d2 is a parallax in the retracting direction, A sign can be attached. Thus, if the sign of the parallax is taken into consideration, Expression 6 and Expression 11 are the same expression. Therefore, when the parallax angle including the code is γ and the parallax including the code is d, the parallax angle γ can be expressed as in Expression (13).

  Conversely, in order to obtain the corresponding parallax d from the parallax angle γ, formula (14) obtained by solving formula (13) for d is used.

  The parallax angle calculation unit 5 calculates the parallax angle map from the parallax map using the above-described Expression 13.

  Note that the relationship between the parallax d and the parallax angle γ is uniquely determined by the binocular interval T and the viewing distance L as shown in Equation 13. In addition, for a high-definition television, a distance that is three times the height of the screen is determined as the optimum viewing distance. That is, the relationship between the parallax d and the parallax angle γ can be uniquely determined by the binocular interval T and the screen size. Therefore, instead of actually converting using Equation 13, a conversion table may be prepared in advance, and the parallax may be converted into a parallax angle with reference to the value.

  Further, the conversion from the parallax map to the parallax angle map may be performed by the parallax calculation unit 4. In this case, the parallax angle calculation unit 5 is unnecessary, and the parallax angle map may be directly input to the parallax adjustment amount calculation unit 6.

  In the following, for simplification of description, a case where the parallax angle calculation unit 5 is provided and processing is performed using the parallax angle will be described.

<2-4. Parallax adjusting unit 7>
Next, the parallax adjustment processing in the parallax adjustment unit 7 will be described in more detail.

  First, as a premise of the parallax adjustment processing, the relationship between the parallax angle of the subject of the stereoscopic image data and the apparent position of the subject will be described.

  FIG. 7 is a diagram schematically showing the parallax angle of the subject of the stereoscopic image data and the apparent position of the subject. A state in which the viewer 25 is viewing the stereoscopic image data displayed on the screen 24 is shown. The stereoscopic image data displayed on the screen 24 is stereoscopic image data including the left-eye image data 12L and the right-eye image data 12R shown in FIG. 3, and the subject 10 is shown in the most recent view and the subject 11 is shown in the farthest view. Yes. Here, it is assumed that the parallax angle of the subject 10 is +0.5 degrees and the parallax angle of the subject 11 is −0.5 degrees.

  The horizontal axis in FIG. 7 is the parallax angle, and for convenience, the left side is positive and the right side is negative. If the subject's parallax angle is a positive value, the subject appears to jump out of the screen 24 toward the viewer 25, and if the subject's parallax angle is a negative value, the subject appears to be retracted from the screen 24. The relationship between the value of the parallax angle and the apparent position of the subject is schematically shown. More specifically, the subject 10 in the most recent view appears at the position of the dotted line 26, and the subject 11 in the farthest view appears at the position of the dotted line 27.

  Next, a specific method of parallax adjustment processing will be described.

  In the present embodiment, as a parallax adjustment method, a method of uniformly shifting the left-eye image data and the right-eye image data constituting the stereoscopic image data in the horizontal direction is used. In this specification, this method is referred to as “uniform shifting”.

  First, a method for parallax adjustment when stereoscopic image data is presented at a position farther from the viewer than before parallax adjustment will be described.

  FIG. 8 is a diagram illustrating a method for performing parallax adjustment by uniformly shifting the left-eye image data described in FIG. 3 to the left and uniformly shifting the right-eye image data to the right. Here, the shift amount of the left-eye image data and the right-eye image data is the same, and is P / 2. The value of P is set so that, for example, the parallax angle at the nearest point of the shifted stereoscopic image data is 0.0 degrees and the parallax angle at the farthest point is −1.0 degrees.

  FIG. 8A shows left-eye image data 28L generated by shifting the left-eye image data 12L described in FIG. 3A to the left. The nearest point 13L and the farthest point 14L in FIG. 3A are shifted to the nearest point 29L and the farthest point 30L in FIG. 8A, respectively. FIG. 8B shows right-eye image data 28R generated by shifting the right-eye image data 12R described in FIG. 3B to the right. The nearest point 13R and the farthest point 14R in FIG. 3B are shifted to the nearest point 29R and the farthest point 30R in FIG. 8B, respectively.

  In the following description, regardless of the left-eye image data 28L or the right-eye image data 28R, the nearest point is referred to as “nearest point 28”, and the farthest point is referred to as “farthest point 29”. Call it.

  In FIG. 8, the area that protrudes from the screen by shifting the image data described in FIG. 3, for example, the leftmost area of the image in FIG. 8A is shown as it is. However, this area is actually not displayed. . Conversely, a region where the image has been lost by shifting the image data described with reference to FIG. 3, for example, the right end region of the image in FIG. 8A is shown in black.

  Here, the distance from the left end of the left-eye image data 28L before shifting to the nearest point 29L after shifting is dnL2, and the distance to the farthest point 30L after shifting is dfL2. Further, the distance from the left end of the right-eye image data 28R before shifting to the nearest point 29R after shifting is dnR2, and the distance to the farthest point 30R after shifting is dfR2. Then, the parallax at the nearest point 28 is dnL2-dnR2, and the parallax at the farthest point 29 is dfL2-dfR2.

  Since the left-eye image data 28L is an image obtained by shifting the left-eye image data 12L of FIG. 3A to the left by P / 2, dnL2 = dnL1-P / 2 and dfL2 = dfL1-P / 2. . Similarly, since the right-eye image data 28R is an image obtained by shifting the right-eye image data 12R of FIG. 3B to the right by P / 2, dnR2 = dnR1 + P / 2 and dfR2 = dfR1 + P / 2. From here, the parallax of the closest point 28 is dnL2-dnR2 = dnL1-dnR1-P, and the parallax of the farthest point 29 is dfL2-dfR2 = dfL1-dfR1-P. That is, the parallax is decremented by P at both the nearest point 28 and the farthest point 29.

  In this way, by shifting the left-eye image data uniformly to the left and the right-eye image data uniformly to the right, the parallax of the stereoscopic image data is shifted in the negative direction, and the subject in the stereoscopic image is Move to a farther position.

  FIG. 9 is a diagram schematically showing the parallax angle of the subject and the apparent position of the subject in the stereoscopic image data after performing the parallax adjustment as shown in FIG. In FIG. 9, when the left-eye image data 28L and the right-eye image data 28R described in FIG. 8 are displayed on the screen 24 of the image display unit 8 and the viewer 25 views the screen 24 and performs a stereoscopic view, The subject 10 located at a position of the dotted line 31 with a parallax angle of 0.0 degrees and the subject 11 in the farthest view appear at the position of a dotted line 32 with a parallax angle of −1.0 degrees.

  Next, a method for parallax adjustment when stereoscopic image data is presented at a position closer to the viewer than before parallax adjustment will be described.

  FIG. 10 is a diagram for describing a method for performing parallax adjustment by uniformly shifting the left-eye image data described in FIG. 3 to the right and uniformly shifting the right-eye image data to the left. Here, the shift amount of the left-eye image data and the right-eye image data is the same, and is P / 2. The value of P is set so that, for example, the parallax angle at the nearest point of the shifted stereoscopic image data is +1.0 degree and the parallax angle at the farthest point is 0.0 degree.

  FIG. 10A shows left-eye image data 33L generated by shifting the left-eye image data 12L described in FIG. 3A to the right. The nearest point 13L and the farthest point 14L in FIG. 3A are shifted to the nearest point 34L and the farthest point 35L in FIG. 10A, respectively. FIG. 10B shows right-eye image data 33R generated by shifting the left-eye image data 11R described in FIG. 3B to the left. The nearest point 13R and the farthest point 14R in FIG. 3B are shifted to the nearest point 34R and the farthest point 35R in FIG. 10B, respectively.

  In the following description, regardless of the left-eye image data 33L or the right-eye image data 33R, the nearest point is referred to as “nearest point 33”, and similarly, the farthest point is referred to as “farthest point 34”. Call it.

  Further, FIG. 10 shows the area protruding from the screen by shifting the image data described in FIG. 3, for example, the rightmost area of the image in FIG. 10A, but this area is not displayed in practice. . Conversely, a region where the image has disappeared by shifting the image data described with reference to FIG. 3, for example, a leftmost region of the image in FIG. 10A is shown in black.

  Here, the parallax at the nearest point 33 is dnL3-dnR3 and the parallax at the farthest point 34 is dfL3-dfR3, as described above with reference to FIG.

  Since the left-eye image data 33L is an image obtained by shifting the left-eye image data 12L of FIG. 3A to the right by P / 2, dnL3 = dnL1 + P / 2 and dfL3 = dfL1 + P / 2. Similarly, since the right-eye image data 33R is an image obtained by shifting the right-eye image data 12R in FIG. 3B to the left by P / 2, dnR3 = dnR1-P / 2 and dfR3 = dfR1-P / 2. is there. From here, the parallax of the nearest point 33 is dnL3-dnR3 = dnL1-dnR1 + P, and the parallax of the farthest point 34 is dfL3-dfR3 = dfL1-dfR1 + P. That is, the parallax is added by P at the nearest point 33 and the farthest point 34.

  Thus, by shifting the left-eye image data uniformly to the right and the right-eye image data uniformly to the left, the parallax of the stereoscopic image data is shifted in the plus direction, and the subject in the stereoscopic image is determined by the viewer. Move closer.

  FIG. 11 is a diagram schematically showing the parallax angle of the subject and the apparent position of the subject in the stereoscopic image data after performing the parallax adjustment as shown in FIG. In FIG. 11, when the left-eye image data 33 </ b> L and the right-eye image data 33 </ b> R described in FIG. 10 are displayed on the screen 24 of the image display unit 8 and the viewer 25 performs a stereoscopic view by looking at the screen 24, The subject 10 located in the position of the subject is seen at the position of a dotted line 36 having a parallax angle of +1.0 degrees, and the subject 11 in the farthest view is seen at the position of a dotted line 37 having a parallax angle of 0.0 degrees.

  Note that when parallax adjustment is performed by uniform shifting, the parallax increases and decreases uniformly, but the amount of change varies depending on the parallax angle before adjustment. This is because the parallax and the parallax angle are connected by a non-linear function of tan-1 as shown in the equations 6 and 7 and the equations 11 and 12. Therefore, when the parallax adjustment is performed by uniform shifting, the parallax angle range width changes.

  However, tan-1 can be linearly approximated if the angle is small. When the parallax angle is about -2.0 degrees to +2.0 degrees as in the present invention, there is no problem with linear approximation. Therefore, in this description, it is assumed that there is no change in the parallax angle range width due to parallax adjustment, and it is assumed to be constant.

  In the above description, the shift amount of the left-eye image data and the right-eye image data is the same and is P / 2. However, as long as the sum of the left and right shift amounts is P, the parallax adjustment is performed similarly. However, for example, if only the left-eye image is shifted by P, the region that protrudes from the screen due to the uniform shift concentrates only on the left-eye image, resulting in poor balance. Accordingly, it is desirable that the shift amounts of the left-eye image data and the right-eye image data are equal or nearly equal.

  Furthermore, when the image is shifted as described above, a predetermined color such as black is used for a region on the side opposite to the shifting direction where there is no image to be displayed, that is, a region painted black in FIGS. Alternatively, the image may be filled with the above image, or may be filled with interpolation from surrounding pixels.

<3. Operation of Stereoscopic Image Data Processing Device 1 in the Embodiment of the Present Invention (Stereoscopic Image Data Processing Method)>
Next, the operation of the stereoscopic image data processing apparatus 1 in the embodiment of the present invention will be described.

<3-1. Overall operation>
FIG. 12 is a flowchart showing the operation of the stereoscopic image data processing apparatus 1.

  In step S1, the stereoscopic image data input unit 2 captures stereoscopic image data, outputs the captured stereoscopic image data to the parallax calculation unit 4, and proceeds to determination step S2.

  In determination step S <b> 2, the user instruction input unit 3 determines whether or not to perform comfortable parallax adjustment based on the first input of the user. If comfortable parallax adjustment is performed, that is, if Yes, the process proceeds to step S3, and if No, the process proceeds to step S8.

  Here, “comfortable parallax adjustment” refers to performing parallax adjustment so that the user can comfortably view stereoscopic image data. A plurality of methods for performing comfortable parallax adjustment are prepared, and these are referred to as “comfortable parallax adjustment modes”. In this embodiment, there are five different comfortable parallax adjustment modes: normal mode, pop-up mode, retract mode, extended pop-up mode, and extended retract mode. The pop-up mode and the retract mode are collectively referred to as “emphasized parallax adjustment mode”. The extended pop-up mode and the extended retract mode are collectively referred to as “extended parallax adjustment mode”. Detailed processing contents of each mode will be described later.

  In step S3, the user instruction input unit 3 determines which comfortable parallax adjustment mode to select according to the second input of the user, and outputs the selected comfortable parallax adjustment mode M to the parallax adjustment amount calculation unit 6. Proceed to step S4. Note that any one of the five types of modes is set as an initial setting value in advance, and when the user does not input the second instruction, the initial setting value is output as the comfortable parallax adjustment mode M.

  In step S4, the parallax calculation unit 4 calculates parallax using the input stereoscopic image data, and generates a parallax map. Then, the stereoscopic image data is output to the parallax adjustment unit 7 and the parallax map is output to the parallax angle calculation unit 5, and the process proceeds to step S5.

  In step S5, the parallax angle calculation unit 5 calculates a parallax angle from each parallax of the input parallax map, thereby converting the parallax angle map. The parallax angle map is output to the parallax adjustment amount calculation unit 6, and the process proceeds to step S6.

  In step S6, the parallax adjustment amount calculation unit 6 refers to the parallax angle map and the input comfortable parallax adjustment mode M, and the parallax adjustment amount P for adjusting the parallax of the input stereoscopic image data to a predetermined parallax. Is output to the parallax adjustment unit 7 and the process proceeds to step S7. Details of the operation of the parallax adjustment amount calculation unit 6 will be described later.

  In step S7, the parallax adjustment unit 7 performs parallax adjustment on the input stereoscopic image data using the input parallax adjustment amount P, and generates stereoscopic image data after parallax adjustment. The generated stereoscopic image data is output to the image display unit 8, and the process proceeds to step S8.

  In step S <b> 8, the image display unit 8 stereoscopically displays the input stereoscopic image data and ends the process.

<3-2. Processing in Step S6>
Next, the operation of the parallax adjustment amount calculation unit 6 in step S6 will be described in detail.

  FIG. 13 is a flowchart showing the operation of the parallax adjustment amount calculation unit 6 in step S6.

  In determination step S61, the parallax adjustment amount calculation unit 6 refers to the parallax angle map, and the parallax angle values included in the parallax angle map are all negative values and the parallax angle range width is equal to or less than T1 degrees. Determine whether or not. If the values of the parallax angles are all negative values and are equal to or less than T1 degrees, that is, Yes, the process proceeds to determination step S67.

  Here, the value of T1 exceeds 0 degree and is within +0.1 degree. In the following description, for example, the value of T1 is 0.1.

  If the parallax angle values of the stereoscopic image data are all negative values and the parallax angle range width is T1 degrees or less, the subject in the stereoscopic image appears to be retracted and the distance is not so different. It shows that. That is, it means that the stereoscopic image data includes only a distant view. Conversely, when there is a positive part in the parallax angle, it means that the subject in the stereoscopic image appears to pop out. Further, when the parallax angle range is wide, it means that there is a high possibility that the stereoscopic image data includes both a distant view and a close view. Since these three-dimensional images have different suitable parallax adjustment processes, the cases are classified in step S61.

  In determination step S62, the parallax adjustment amount calculation unit 6 refers to the input comfortable parallax adjustment mode M. If the comfortable parallax adjustment mode M is the normal mode, that is, Yes, the process proceeds to step S63. Proceed to S64.

  In step S63, the parallax adjustment amount calculation unit 6 calculates the parallax adjustment amount P by the processing in the normal mode, and proceeds to step S68.

  Next, the case where the determination in step 62 is No will be described. In the determination step S64, the parallax adjustment amount calculation unit 6 refers to the parallax map and determines whether the parallax angle range width exceeds 1.0 degree or whether the comfortable parallax adjustment mode M is the enhanced parallax adjustment mode. If Yes, the process proceeds to step S65. If both are No, the process proceeds to step S66. That is, the process proceeds to step S66 only when the parallax angle range width is 1.0 degrees or less in the extended parallax adjustment mode.

  Here, the “enhanced parallax adjustment mode” refers to emphasizing the sense that the subject of the stereoscopic image data pops out as much as possible within the parallax angle range in which the stereoscopic image data can be comfortably viewed. In this mode, parallax adjustment for emphasizing the feeling of being spread toward the back of the screen (feeling of retraction) is performed. As described above, the pop-out mode and the retract mode are the enhanced parallax adjustment mode.

  In addition, the “extended parallax adjustment mode” is a mode in which parallax adjustment is performed that emphasizes the feeling of popping out and retracting beyond the parallax angle range where the stereoscopic image data can be comfortably viewed to the permissible limit. As described above, the extended pop-up mode and the extended retract mode are the extended parallax adjustment modes.

  As described above, the extended parallax adjustment mode is parallax adjustment processing that sacrifices comfort. For this reason, the stereoscopic image data itself to be processed is limited to data having a small burden on the viewer's eyes, specifically, a parallax angle range width of 1.0 degrees or less. This is why the determination step S64 includes a condition regarding the parallax angle range width.

  In step S65, the parallax adjustment amount calculation unit 6 calculates the parallax adjustment amount P by processing in the enhanced parallax adjustment mode, and the process proceeds to step S68.

  On the other hand, in step S66, the parallax adjustment amount calculation unit 6 calculates the parallax adjustment amount P by the processing in the extended parallax adjustment mode, and the process proceeds to step S68.

  In step S67, the parallax adjustment amount calculation unit 6 refers to the input parallax map, calculates the parallax adjustment amount P such that the parallax angle of the farthest view is T4 degrees, and proceeds to step S68.

  In step S68, when the parallax adjustment amount calculation unit 6 performs parallax adjustment based on the calculated parallax adjustment amount P as the final confirmation of the parallax adjustment amount, the parallax of the farthest view is opened on the screen of the image display unit 8. The parallax adjustment amount P is corrected again so as not to exceed 5 cm in the scattering direction, and is output as the final parallax adjustment amount P. The “divergence direction” indicates a direction in which the viewer's eyes are directed outward from the front when performing stereoscopic viewing.

  With the above process, the process of step S6 is terminated. Next, some of the processes in step S6 will be described in more detail.

<3-3. Step S63: Calculation of Parallax Adjustment Amount P in Normal Mode>
FIG. 14 is a flowchart of the operation when the parallax adjustment amount calculation unit 6 calculates the parallax adjustment amount P in the normal mode in step S63.

  In step S101, the parallax adjustment amount calculation unit 6 refers to the input parallax map and determines a parallax angle as a reference for parallax adjustment. In the present invention, this parallax angle is referred to as a “parallax adjustment base angle”.

  Next, in step S102, the parallax adjustment amount calculation unit 6 calculates a parallax adjustment amount P that moves the parallax adjustment base angle to the second predetermined value T2 degrees.

  With the above process, the process of step S63 is completed.

<3-3-1. First example of parallax adjustment base angle determination method: 0 degree>
Various methods can be considered as the method for determining the parallax adjustment base angle in step S101. Here, a case where the parallax adjustment base angle is set to 0 degree will be described as a first example of the parallax adjustment base angle determination method.

  FIG. 15 illustrates the parallax angle of the subject of the stereoscopic image data and the apparent appearance of the subject when the parallax adjustment base angle is set to 0 degree with respect to the stereoscopic image data having the parallax angle range width of 1.0 degree. It is the figure which showed the position typically. In FIG. 15, it is assumed that the stereoscopic image data displayed on the screen 24 is the same as the stereoscopic image data described in FIG. Among these, the parallax angle is 0 degree, that is, the parallax angle at the position of the dotted line 38 is set as the parallax adjustment base angle.

  Subsequently, as described above, in step S102, the parallax adjustment amount calculation unit 6 calculates the parallax adjustment amount P that moves the parallax adjustment base angle to T2 degrees that is the second predetermined value. Here, T2 is preferably in the range of -0.2 degrees to -0.5 degrees, and more preferably in the range of -0.2 degrees to -0.3 degrees. T2 is the value of the parallax angle that can be seen most comfortably obtained by the inventors' experiment. In the following description, the value of T2 is assumed to be −0.25 degrees, for example. The basis for the value of T2 will be described later.

  FIG. 16 is a diagram schematically illustrating the parallax angle of the subject of the stereoscopic image data and the apparent position of the subject after the parallax adjustment is performed so that the parallax adjustment base angle illustrated in FIG. 15 is T2 degrees. is there. In FIG. 16, in order to present the parallax adjustment base angle that is the position of the dotted line 38 on the axis of the parallax angle 0 degree, that is, on the axis of the parallax angle, at the position of the dotted line 39 where the parallax angle is −0.25 degrees, It is necessary to adjust the parallax so that the angle is minus 0.25 degrees. The parallax adjustment amount calculation unit 6 outputs the parallax adjustment amount necessary at this time as the parallax adjustment amount P.

  Specifically, the parallax adjustment amount P is calculated as follows.

  Let the parallax adjustment base angle be γbase and T2 be γnew. These are all parallax angles. Also, the parallax corresponding to γbase is set to dbase, and the parallax corresponding to γnew is set to dnew. Since the parallax adjustment amount that adjusts γbase to γnew is P, P is expressed by Expression (15).

  Since dbase and dnew can be obtained from equation (14), P can be obtained from equation (16).

  As shown in FIG. 16, the object of the latest view at this time is presented at the position of the dotted line 40, the parallax angle is +0.25 degrees, and the object of the farthest view is at the position of the dotted line 41. Presented and its parallax angle is -0.75 degrees.

  In the above description, the parallax angle 0 degree corresponding to the screen position is set as the parallax adjustment base angle, and the parallax adjustment is performed so that the parallax angle of the parallax adjustment base angle is moved to T2 degrees.

  By the way, the viewer often watches the main subject in the image. Therefore, it is desirable that the parallax angle of the main subject is the parallax angle at which the subject can be most comfortably seen.

  As described in the section of the prior art, in the conventional stereoscopic image data, basically, the main subject is arranged at a parallax angle of 0 degrees with the least burden on the eyes, and the +0. In many cases, the image is produced so that the parallax angle falls within the range of 5 degrees to -0.5 degrees.

  These stereoscopic image data are set so that the parallax angle of the main subject is close to 0 degrees on the assumption that the parallax angle of 0 degrees is the least burdensome on the eyes, that is, the most easily viewable parallax angle. At the same time, in order to express a sense of depth centered on a certain position of the main subject, the parallax angle is set to 0 ° as the median value of the parallax angle range.

  However, according to experiments conducted by the inventors, which will be described in detail later, it can be most comfortably observed when the parallax angle of the subject is T2 degrees, and the parallax angle range of 0.5 degrees around the T2 degrees is the center. It became clear that stereoscopic image data can be seen most comfortably. Therefore, as described above, the parallax angle of the parallax adjustment base angle is set to 0 degree which is highly likely to be the parallax angle of the main subject, and the parallax angle of the parallax adjustment base angle is moved to T2 degrees. By performing the parallax adjustment, the parallax angle of the main subject in the conventional stereoscopic image is adjusted to the parallax angle that is most visible. Further, the entire stereoscopic image data is adjusted to the most comfortable parallax angle range. Thereby, the viewer can view stereoscopic image data more comfortably.

  Also, depending on the video, 2D images and stereoscopic images may be mixed. When viewing such a video, when the viewer recognizes that the stereoscopic display device is viewing stereoscopic image data on the stereoscopic display device and the 2D image is displayed, it is the same as the normal viewing of the 2D image. Since a flat image is displayed on the screen of the stereoscopic display device, there is a problem that there is a sense of incongruity.

  However, as described above, a normal 2D image scene is displayed while a stereoscopic image is displayed by using a parallax angle of 0 degrees as the parallax adjustment base angle and performing parallax adjustment so that the parallax angle is moved to T2 degrees. Even so, the 2D image is not presented as a planar image on the screen of the stereoscopic display device, but is presented at a position slightly behind the screen. Accordingly, the viewer recognizes the 2D image as stereoscopic image data of a type in which a planar image is displayed at the back, and as a result, the viewer feels that the stereoscopic image data is being continuously viewed. There is no sense of incongruity and you can watch comfortably.

  In general, stereoscopic images are produced so that 0 degrees is the center of the parallax angle range. However, as described above, the position of the parallax angle 0 degrees is set as the parallax adjustment base angle to adjust the parallax. By doing so, the center of the parallax angle range after parallax adjustment is T2 degrees. This means that when a normal 2D image scene is displayed while displaying a stereoscopic image, the center of the parallax angle range of the stereoscopic image and the parallax angle of the 2D image are displayed at the same T2 degrees. The impression of the change in stereoscopic effect felt by the viewer at the boundary between the stereoscopic image scene and the 2D image scene does not change before and after the parallax adjustment, and the viewer can view without any discomfort even when the parallax adjustment is performed. .

  Furthermore, in the above description, the range of the parallax angle in the basic example of the conventional stereoscopic image data is +0.5 degrees to −0.5 degrees, but includes a parallax angle of 0 degrees and includes a depth range within one screen. However, as long as the parallax angle is within 1 degree, a range other than +0.5 degrees to -0.5 degrees may be used. Even in this case, as described above, in the conventional stereoscopic image data, since the main subject is often set near the parallax angle of 0 degrees, the parallax adjustment is performed so that the parallax adjustment base angle becomes T2 degrees. By doing so, it is possible to adjust the parallax angle of the main subject that is highly likely to be watched by a person to the parallax angle that is most easily viewable.

  Note that, when the parallax angle 0 degree is used as the parallax adjustment base angle in this way, the parallax adjustment amount calculation unit 6 does not need to use a parallax map. Furthermore, the parallax calculation unit 4 and the parallax angle calculation unit 5 may be omitted. In this way, the present invention can be implemented with a simpler configuration.

<3-3-2. Second example of parallax adjustment base angle determination method: median value>
A second example of the parallax adjustment base angle determination method in step S101 will be described. Here, among the parallax angles of the entire screen of stereoscopic image data, the median value of the parallax angle of the farthest view and the parallax angle of the closest view is used as the parallax adjustment base angle.

  FIG. 17 shows the parallax angle of the subject of the stereoscopic image data when the parallax angle range width is 0.6 degrees and the parallax adjustment base angle is the median value of the parallax angle of the farthest view and the parallax angle of the closest view FIG. 3 is a diagram schematically illustrating the apparent position of the subject. In FIG. 17, stereoscopic image data including a subject in the closest view with a parallax angle of +0.5 degrees and a subject in the farthest view with a parallax angle of −0.1 degrees is displayed on the screen 24. At this time, the parallax adjustment base angle is the central value of the parallax angle of the subject in the nearest scene at the position of the dotted line 42 and the parallax angle of the subject in the farthest view at the position of the dotted line 43. That is, in FIG. 17, the parallax angle of the subject in the most recent view is +0.5 degrees, and the parallax angle of the subject in the farthest view is −0.1 degrees, so the dotted line 44 that becomes the parallax angle +0.2 degrees. The parallax angle at the position becomes the parallax adjustment base angle.

  Also in this case, the parallax adjustment amount P is calculated in the same manner as described in step S102.

  FIG. 18 shows the parallax angle of the subject in the stereoscopic image data after the parallax adjustment and the apparent position of the subject after performing the parallax adjustment so that the parallax adjustment base angle shown in FIG. 17 is moved to T2 degrees. It is the figure shown typically. The value of T2 is set to -0.25 degree. In FIG. 17, in order to present the parallax adjustment base angle from the position of the dotted line 44 where the parallax angle is +0.2 degrees to the position of the dotted line 45 where the parallax angle is −0.25 degrees, the parallax adjustment base angle is 0. It is necessary to adjust the parallax so as to be negative by 45 degrees. The parallax adjustment amount calculation unit 6 outputs the parallax adjustment amount necessary at this time as the parallax adjustment amount P.

  Note that, as shown in FIG. 17, the object of the closest view at this time is the position of the dotted line 46 where the parallax angle is +0.05 degrees, and the parallax angle of the object of the farthest view is -0.55 degrees. Each is presented in a position.

  In the above description, among the parallax angles of the entire screen, the parallax adjustment base angle is set to the median value of the parallax angle of the farthest view and the parallax angle of the closest view, and the parallax adjustment base angle is T2 degrees. The adjustment was explained.

  When the parallax adjustment base angle is set in this manner, the parallax adjustment base angle is set for stereoscopic image data in which the parallax angle is in the range of +0.5 degrees to -0.5 degrees centering on 0 degrees. The same effect as the first example is achieved.

  On the other hand, as described in the section of the prior art, there is also stereoscopic image data that does not satisfy the conditions of Non-Patent Document 1. Even in such stereoscopic image data, by adjusting the parallax so that the median value of the parallax in the stereoscopic image data is moved to T2 degree which is the most comfortable parallax angle, It is adjusted to a three-dimensional image in which the parallax exists evenly before and after. That is, the stereoscopic image as a whole is adjusted to the most comfortable parallax angle range. Thereby, the viewer can view stereoscopic image data more comfortably.

  In addition, the parallax angle of the main subject on the screen, that is, the area that is likely to be watched by the viewer is often considered to be located in the center of the parallax angle range. The parallax angle is adjusted to the most visible parallax angle. Thereby, the viewer can view stereoscopic image data more comfortably.

<3-3-3. Third example of parallax adjustment base angle determination method: parallax angle having the highest appearance frequency>
A third example of the parallax adjustment base angle determination method in step S101 will be described. Here, the parallax angle having the highest appearance frequency among the parallax angles of the entire screen is used as the parallax adjustment base angle. More specifically, a parallax histogram is generated from the parallax angle map, the parallax angle having the maximum frequency is obtained, and the parallax angle is used as the parallax adjustment base angle.

  When the parallax adjustment base angle is set in this manner, the parallax adjustment base angle is set for stereoscopic image data in which the parallax angle is in the range of +0.5 degrees to -0.5 degrees centering on 0 degrees. The same effect as the first example is achieved.

  Further, even when the stereoscopic image data is not such, the parallax angle having the highest appearance frequency is the main subject having the largest area in the stereoscopic image data, and the parallax of the region that is likely to be watched by the viewer It is considered to be a horn. By performing the parallax adjustment so that the parallax angle is moved to T2 degrees which is the most comfortable parallax angle, the gaze area is presented with the most comfortable parallax angle, and the viewer is more comfortable with the stereoscopic image data. Can see.

<3-3-4. Fourth example of parallax adjustment base angle determination method: average value>
A fourth example of the parallax adjustment base angle determination method in step S101 will be described. Here, the average value of the parallax angles of the entire screen is used as the parallax adjustment base angle. More specifically, an average value of all the parallaxes in the parallax map is obtained, and the parallax angle is used as the parallax adjustment base angle.

  When the parallax adjustment base angle is set in this manner, the parallax adjustment base angle is set for stereoscopic image data in which the parallax angle is in the range of +0.5 degrees to -0.5 degrees centering on 0 degrees. The same effect as the first example is achieved.

  Further, even when the stereoscopic image data is not such, the stereoscopic image data is obtained by performing the parallax adjustment so that the average value of the parallax angles of the entire screen is moved to T2 degrees which is the most comfortable parallax angle. Can be presented so that the parallax angle of each pixel becomes a comfortable parallax angle range width, and the viewer can view stereoscopic image data more comfortably.

  In particular, when the parallax angle range width of the stereoscopic image data is smaller than the comfortable parallax angle range width, the parallax angles of many pixels are closer to T2 degrees, which is the most comfortable parallax angle. ,effective.

<3-3-5. Fifth Example of Determination Method of Parallax Adjustment Base Angle: Parallax Angle Determined to Minimize Sum of Squares of Differences with Parallax Angles of Parallax Map>
A fifth example of the parallax adjustment base angle determination method in step S101 will be described. Here, as the parallax adjustment base angle, a parallax angle determined so as to minimize the sum of squares of differences with each parallax angle of the parallax map is used. Such a parallax angle represents an average parallax angle of stereoscopic image data.

  When the parallax adjustment base angle is set in this manner, the parallax adjustment base angle is set for stereoscopic image data in which the parallax angle is in the range of +0.5 degrees to -0.5 degrees centering on 0 degrees. The same effect as the first example is achieved.

  Further, even when the stereoscopic image data is not such, the parallax angle determined so that the sum of squares of differences with each parallax angle of the parallax map is minimized is moved to T2 degrees which is the most comfortable parallax angle. By performing the parallax adjustment in this manner, the parallax angle of each pixel of the stereoscopic image data becomes a comfortable parallax angle range width as in the case where the average value of the parallax angles of the entire screen is used as the parallax adjustment base angle. The viewer can view stereoscopic image data more comfortably.

  In particular, when the parallax angle range width of the stereoscopic image data is smaller than the comfortable parallax angle range width, the parallax angles of many pixels are closer to T2 degrees, which is the most comfortable parallax angle. ,effective.

<3-3-6. Sixth Example of Method for Determining Parallax Adjustment Base Angle: Parallax Angle of Farthest View>
A sixth example of the parallax adjustment base angle determination method in step S101 will be described. Here, the value of the parallax angle of the farthest view among the parallax angles of the entire screen is used as the parallax adjustment base angle.

  For viewers gazing at the subject in the farthest view, it is possible to view the stereoscopic image data more comfortably by adjusting the parallax so that the parallax angle of the farthest view is moved to T2 degrees, which is the most comfortable parallax angle. it can.

<3-3-7. Seventh Example of Method for Determining Parallax Adjustment Base Angle: Parallax Angle of Recent Scene>
A seventh example of the parallax adjustment base angle determination method in step S101 will be described. Here, the value of the parallax angle of the most recent scene among the parallax angles of the entire screen is used as the parallax adjustment base angle.

  For viewers gazing at the subject of the recent scene, stereoscopic image data can be viewed more comfortably by adjusting the parallax so that the parallax angle of the recent scene is moved to T2 degrees, which is the most comfortable parallax angle. it can.

<3-3-8. Summary of Normal Mode Processing>
In this way, the normal mode process comfortably adjusts the gaze area by adjusting the parallax so that the parallax angle of the main subject on the screen, that is, the area where the viewer is likely to gaze, moves to T2 degrees. To be able to see. For this reason, the parallax angle that is considered to be the parallax angle of the gaze area is set as the parallax adjustment base angle, and the parallax adjustment base angle is moved to T2 degrees. Thereby, it becomes possible to present stereoscopic image data more easily.

  In addition, since several ways of selecting the parallax angle of the gaze area can be considered, a plurality of examples have been described as described above.

<3-4. Step S65: Calculation of the parallax adjustment amount P in the enhanced parallax adjustment mode>
FIG. 19 is a flowchart of the operation when the parallax adjustment amount calculation unit 6 calculates the parallax adjustment amount P in the enhanced parallax adjustment mode in step S65.

  In determination step S103, the parallax adjustment amount calculation unit 6 determines whether or not the comfortable parallax adjustment mode M is the pop-up mode. If the parallax adjustment mode M is the pop-out mode, the process proceeds to step S104. Proceed to step S105.

  In the determination step S64, the comfortable parallax adjustment mode M is either the extended pop-up mode or the extended retract mode. However, when the process proceeds to step S65 because the parallax angle range width exceeds 1 degree, the extended pop-up mode is selected. In the case of the mode, the process proceeds to step S104, and in the case of the extended retraction mode, the process proceeds to step 105.

  When the process proceeds to step S104, that is, when the comfortable parallax adjustment mode M is the pop-up mode, the parallax adjustment amount calculation unit 6 refers to the parallax map and moves the parallax angle of the latest scene to T3 degrees. A correct parallax adjustment amount P is calculated. Here, the value of T3 is desirably a value between the parallax angle +0.3 degrees and 0.0 degrees, and more desirably in the range of +0.3 degrees to +0.2 degrees. This is the value of the parallax angle closest to the foreground within the comfortable parallax angle range obtained by the inventors' experiments. In the following description, the value of T3 is assumed to be +0.25 degrees, for example. The basis for the value of T3 will be described later.

  On the other hand, when the process proceeds to step S105 in the determination process of the determination step S103 described above, that is, when the comfortable parallax adjustment mode M is the retract mode, the parallax adjustment amount calculation unit 6 refers to the parallax map and The parallax adjustment amount P is calculated such that the parallax angle is moved to T4 degrees. Here, T4 is preferably a value in the range of the parallax angle of −0.7 degrees to −1.0 degrees, and more preferably a value in the range of −0.7 degrees to −0.8 degrees. This is a parallax angle value on the farthest distant side within a comfortable parallax angle range obtained by the inventors' experiment. In the following description, the value of T4 is assumed to be −0.75 degrees, for example. The basis for the value of T4 will be described later.

  FIG. 20 shows the parallax angle of the subject in the stereoscopic image data after performing the parallax adjustment so that the parallax angle of the most recent scene of the stereoscopic image data shown in FIG. 17 is moved to T3 degrees in the process of step S104. It is the figure which showed typically the position of the subject. In FIG. 20, in order to present a subject in the latest scene that was at the position of the dotted line 42 with the parallax angle +0.5 degrees before the parallax angle at the position of the dotted line 48 at the parallax angle +0.25 degrees, It is necessary to adjust the parallax so that the parallax angle of the subject in the scene is minus 0.25 degrees. The parallax adjustment amount calculation unit 6 outputs the parallax adjustment amount necessary at this time as the parallax adjustment amount P. As shown in FIG. 20, the farthest view subject at this time is presented at the position of the dotted line 49 where the parallax angle is −0.35 degrees.

  In this way, the parallax angle of the subject in the most recent scene in the stereoscopic image data is adjusted so that the parallax angle is moved to the closest parallax angle T3 degrees within the comfortable parallax angle range, thereby emphasizing the pop-out. It is possible to view safely and comfortably while performing stereoscopic display.

  FIG. 21 illustrates the parallax angle of the subject of the stereoscopic image data after performing the parallax adjustment so that the parallax angle of the farthest view of the stereoscopic image data illustrated in FIG. 17 is moved to T4 degrees in the process of step S105. It is the figure which showed typically the position of the subject. In FIG. 21, before the parallax adjustment, the farthest object at the position of the dotted line 43 where the parallax angle is −0.1 degrees is presented at the position of the dotted line 51 where the parallax angle is −0.75 degrees. The parallax adjustment amount calculation unit 6 outputs the parallax adjustment amount necessary at this time as the parallax adjustment amount P so that the parallax angle of the subject in the farthest view is minus 0.65 degrees. . Note that, as shown in FIG. 21, the subject of the latest scene at this time is presented at the position of the dotted line 50 where the parallax angle is −0.15 degrees.

  In this manner, the parallax adjustment is performed so that the parallax angle of the subject in the farthest view of the stereoscopic image data is moved to the parallax angle T4 degree on the farthest side within the comfortable parallax angle range, thereby enhancing the retraction. It is possible to view safely and comfortably while performing stereoscopic display.

<3-5. Step S66: Calculation of the parallax adjustment amount P in the extended parallax adjustment mode>
FIG. 22 is a flowchart of the operation when the parallax adjustment amount calculation unit 6 calculates the parallax adjustment amount P in the extended parallax adjustment mode in step S66.

  In the determination step S106, the parallax adjustment amount calculation unit 6 determines whether or not the comfortable parallax adjustment mode M is the extended pop-up mode. If the extended parallax adjustment mode M is Yes, the process proceeds to step S107, and the extended retraction mode, that is, No. In this case, the process proceeds to step S108.

  When the process proceeds to step S107, that is, when the comfortable parallax adjustment mode M is the extended pop-up mode, the parallax adjustment amount calculation unit 6 refers to the parallax map and moves the parallax angle of the latest scene to T5 degrees. Such a parallax adjustment amount P is calculated.

  On the other hand, when the process proceeds to step S108 in the above-described determination step S106, that is, when the comfortable parallax adjustment mode M is the extended retraction mode, the parallax adjustment amount calculation unit 6 refers to the parallax map and parallax of the farthest view. A parallax adjustment amount P that moves the angle to T6 degrees is calculated.

  As described in Non-Patent Document 1, the allowable value of the fusion limit of the parallax angle of the human eye is about ± 2.0 degrees. This means that if the parallax is less than the fusion limit of stereoscopic vision, there is a difference between individuals, but there is a high possibility that stereoscopic vision is possible. However, even if the parallax angle is within ± 2.0 degrees, the burden is large when the parallax angle range width is 1.0 degree or more. Therefore, the extended parallax adjustment mode is applied only to a stereoscopic image having a parallax angle range width within 1.0 degree. If the parallax angle range width of the stereoscopic image data is within 1.0 degree, even if the parallax adjustment is performed up to the allowable limit of the parallax angle fusion limit, the display time of the stereoscopic image data is reduced, This is because the possibility of viewing without fatigue is high if consideration is given to the eyes such as not making the change sudden.

  In the case of the processing in the extended pop-up mode (the processing in step S107), in order to make the nearest scene pop out to the limit of the fusion limit, the value of T5 that is the parallax angle of the nearest view after parallax adjustment is set to +2.0 degrees. It is desirable to set.

  However, the effect of popping out and safety are in a trade-off relationship, and when the emphasis is on safety, the value of T5 is larger than the parallax angle +1.0 degree and within the range of +2.0 degrees or less. It is desirable to set the value. In the following description, the value of T5 is assumed to be +2.0 degrees, for example.

  FIG. 23 shows the parallax angle of the subject in the stereoscopic image data after performing the parallax adjustment so that the parallax angle of the most recent scene of the stereoscopic image data shown in FIG. 7 is moved to T5 degrees in the process of step S107. It is the figure which showed typically the position of the subject. As shown in FIG. 23, after the parallax adjustment, the object in the closest view is at the position of the dotted line 52 where the parallax angle is +2.0 degrees, and the object of the farthest view is at the position of the dotted line 53 where the parallax angle is +1.0 degrees. Presented.

  On the other hand, in the case of the processing in step S108, that is, in the case of the processing in the extended retraction mode, similarly, in order to display the farthest view in the retraction direction to the limit of the fusion limit, the disparity angle T6 which is the disparity angle of the nearest scene after the parallax adjustment is displayed. It is desirable to set the value to -2.0 degrees.

  However, the effect of retraction and safety are in a trade-off relationship, and when the emphasis is on safety, the value of T6 is smaller than the parallax angle −1.0 degrees and −2.0 degrees or less. It is desirable to set with a range value. In the following description, the value of T6 is set to −2.0 degrees, for example.

  FIG. 24 shows the parallax angle of the subject of the stereoscopic image data after performing the parallax adjustment so that the parallax angle of the farthest view of the stereoscopic image data shown in FIG. 7 is moved to T6 degrees in the process of step S108. It is the figure which showed typically the position of the subject. As shown in FIG. 24, after the parallax adjustment, the object in the closest view is at the position of the dotted line 54 where the parallax angle is −1.0 degrees, and the object of the farthest view is at the position of the dotted line 55 where the parallax angle is −2.0 degrees. , Each presented.

  As described above, the extended pop-up mode can perform a stereoscopic display in which the pop-out is further emphasized by adjusting the parallax to the parallax angle that becomes the fusion limit of stereoscopic vision in the pop-out direction. Further, in the extended retraction mode, a stereoscopic display in which the retraction is further emphasized can be performed by performing the parallax adjustment up to the parallax angle at which the stereoscopic view is merged in the retraction direction.

  Note that the value of T5 or T6 may be changed in accordance with the parallax display time. For example, if the display time is short for T5, the value of T5 is set to +2.0 degrees, and the value of T5 is set to approach +1.0 degrees as the display time becomes long. Then, a safe display can be performed at the same time while effectively emphasizing the stereoscopic effect. The same applies to T6.

<3-6. Step S67: Calculation of the parallax adjustment amount P so that the parallax angle of the farthest view is T4 degrees>
Next, the processing when the parallax angle values of the stereoscopic image data are all negative values and the parallax angle range width is T1 degrees or less in the determination step S61, that is, the processing of step S67 will be described. To do. In step S67, the parallax adjustment amount calculation unit 6 refers to the input parallax map and calculates the parallax adjustment amount such that the parallax angle of the farthest view is T4 degrees. This T4 degree is the same value as the value used for the processing in the retraction enhancement mode in the above-described enhancement parallax adjustment mode.

  As an example in which the parallax angle values of the parallax map input to the parallax adjustment amount calculation unit 6 are all negative values and the parallax angle range width is T1 degrees or less, the parallax angle value is −0. A case where 5 degrees and the parallax angle range width is 0.0 degrees will be described. In this case, the stereoscopic image data is only for a distant subject and all the parallaxes in the parallax map have the same value.

  FIG. 25 is a diagram illustrating the parallax of the stereoscopic image data at this time. FIG. 25 (a) shows left-eye image data 58L. Of these, the most distant area when stereoscopic display is set as the farthest point 59L. FIG. 25B shows the right-eye image data 58R, and the corresponding point of the farthest point 59L of the left-eye image data is the farthest point 59R. Further, the distance from the left end of the left-eye image data 58L to the farthest point 59L is dfL4, and the distance from the left end of the right-eye image data 58R to the nearest point 58R is dfR4.

  Here, the farthest view parallax when stereoscopic display is performed using the left-eye image data 58L and the right-eye image data 58R is dfL4-dfR4.

  FIG. 26 is a diagram schematically showing the parallax angle of the subject of the stereoscopic image data and the apparent position of the subject at this time. 26, when the left-eye image data 58L and the right-eye image data 58R described in FIG. 25 are displayed on the screen 24 of the image display unit 8 and the viewer 25 views the screen 24 and performs a stereoscopic view, the stereoscopic image is displayed. The entire data is presented as stereoscopic image data at the position of the dotted line 60.

  FIG. 27 is a diagram for describing a method for performing parallax adjustment by shifting the left-eye image data described in FIG. 25 uniformly to the left and the right-eye image data uniformly to the right.

  FIG. 27A shows left-eye image data 61L generated by shifting the left-eye image data 58L described in FIG. 25A to the left. The area that is farthest when stereoscopic display is performed is the farthest point 62L. FIG. 27B shows right-eye image data 61R generated by shifting the right-eye image data 58R described in FIG. 25B to the right. The corresponding point of the farthest point 62L of the left-eye image data is set as the farthest point 62R.

  At this time, assuming that the distance from the left end of the left-eye image data 61L to the farthest point 62L is dfL5 and the distance from the left end of the right-eye image data 61R to the farthest point 62R is dfR5, the left-eye image data 61L The farthest view parallax when stereoscopic display is performed using the right-eye image data 61R is dfL5-dfR5. Note that the shift width of the left-eye image and the shift width of the right-eye image at this time are set to be equal or nearly equal to each other.

  In addition, here, the shift width of the image in FIG. 27 is such that, for example, the parallax angle of the farthest view parallax of the parallax-adjusted stereoscopic image data is moved to T4 degrees. As described above, T4 is preferably a value in the range of the parallax angle of −0.7 degrees to −1.0 degrees, and more preferably a value in the range of −0.7 degrees to −0.8 degrees. This is a parallax angle value on the farthest distant side within a comfortable parallax angle range obtained by the inventors' experiment. Hereinafter, similarly to the above description, the value of T4 is described as being −0.75 degrees, for example.

  FIG. 28 is a diagram schematically showing the parallax angle of the subject and the apparent position of the subject in the stereoscopic image data after the parallax adjustment shown in FIG. As shown in FIG. 28, after the parallax adjustment, the subject in the farthest view is presented at the position of the dotted line 63 where the parallax angle is −0.75 degrees. In this way, the parallax adjustment amount calculation unit 6 determines that the parallax angle of the farthest view is smaller when the parallax angle values of the stereoscopic image data are all negative values and the parallax angle range width is T1 degrees or less. A parallax adjustment amount P that is −0.75 degrees is calculated.

  As described above, when the subject in the stereoscopic image data is only a distant view, the disparity adjustment is performed so that the disparity angle of the farthest view subject moves to the distant disparity angle T4 degrees within the comfortable disparity angle range. By performing the above, it is possible to comfortably view a distant view while performing a stereoscopic display with emphasis on retraction.

<3-7. Step S68: Final confirmation of parallax adjustment amount>
29, in step S68, the parallax adjustment amount calculation unit 6 corrects the parallax adjustment amount P so that the parallax of the farthest view does not exceed 5 cm in the spreading direction on the screen of the image display unit 8. It is a flowchart about the operation | movement at the time of determining a suitable parallax adjustment amount.

  In step S109, the parallax adjustment amount calculation unit 6 performs the parallax adjustment on the input parallax angle map using the parallax adjustment amount P calculated in any of steps S63, S65, S66, and S67. Whether the parallax of the subject displayed in the farthest view, for example, the parallax of the farthest point 14 which is the portion of the sun in the background in FIG. 3, exceeds 5 cm in the spreading direction on the screen of the image display unit 8. If it is determined to exceed 5 cm, that is, Yes, the process proceeds to step S110. If No, nothing is performed and step S68 is terminated.

  In step S110, the parallax adjustment amount P is set so that the parallax of the subject displayed in the farthest view does not exceed 5 cm in the spreading direction on the screen of the image display unit 8 (in other words, not less than −5 cm). To correct.

  In general, viewing a video with a parallax larger than the width of the eyes may cause symptoms such as perspective and oblique positions, and when the parallax of the subject displayed in the farthest view exceeds 5 cm in the spreading direction However, there is a possibility of displaying a parallax stereoscopic image that is wider than the width of the child's eyes, which is problematic in terms of safety. However, comfortable and safe stereoscopic viewing can be achieved by calculating the parallax adjustment amount so that the parallax of the farthest view is smaller than the eye width as described above.

  For example, even when the parallax adjustment amount P is calculated such that the parallax angle of the farthest view is −2.0 degrees in the extended retraction emphasis mode in step S108 described above, the parallax adjustment is performed by the process in step S68 thereafter. The parallax adjustment amount P finally calculated by the adjustment amount calculation unit 6 is such that the parallax of the farthest view of the stereoscopic image data after parallax adjustment does not exceed 5 cm in the spreading direction on the screen of the image display unit 8. can do.

  For example, it will be described that the parallax adjustment amount P set by the extended retraction mode process shown in FIG. 24 is further adjusted by the process of step S68.

  FIG. 30 is a diagram schematically illustrating the parallax angle of the subject of the stereoscopic image data and the apparent position of the subject after further parallax adjustment is performed in the process of step S68. Here, T7 indicates the value of the parallax angle when the parallax of the farthest view does not exceed 5 cm in the spreading direction on the screen of the image display unit 8, and depends on the viewing distance of the image display unit 8 and the screen size. , Its value changes. For example, in the following description, the value of T7 is about −1.49 degrees. This value is the parallax angle when the parallax of the farthest view is 5 cm in the spreading direction when the image display unit 8 having a screen size of 52 inches is viewed at a viewing distance of 3H (3 times the screen height). Is the value of

  As shown in FIG. 30, after the parallax adjustment, the object of the closest view is at the position of the dotted line 56 where the parallax angle is −0.49 degrees, and the object of the farthest view is the dotted line 57 where the parallax angle is −1.49 degrees. Each is presented in a position.

  As described above, the parallax adjustment amount calculation unit 6 adjusts the parallax adjustment amount so that the parallax of the farthest view of the stereoscopic image data after parallax adjustment does not exceed 5 cm in the spreading direction on the screen of the image display unit 8. P can be calculated.

<4. Subjective evaluation experiment>
Next, the grounds for the values of T2, T3, and T4 described above will be described.

  The values of T2, T3, and T4 are values set based on the subjective evaluation experiment performed on the parallax angle that the viewer feels most comfortable and based on the experiment result.

<4-1. Experiment contents>
For the purpose of verifying comfortable parallax of stereoscopic image data, a verification experiment of comfortable parallax was performed using a subjective evaluation scale.

  The subjects of this experiment were 30 healthy adolescents with no ophthalmic diseases other than mild refractive errors. If the subjects had refractive errors, they were refracted with soft contact lenses. The experiment was conducted after confirming that the binocular vision function was normal with good visual acuity of 1.2 or higher and near vision 1.0 or higher.

  FIG. 31 is a view of a comfortable parallax verification experiment as seen from above. In this experiment, the subject 65 wearing the active shutter glasses 64 views the 52-inch size 3D television 67 connected to the video reproduction device 66 at a viewing distance three times the height of the screen of the 3D television 67. Then, the subject 64 has the parallax so that the subject 64 feels comfortable with respect to the stereoscopic image data I1 and the retracted stereoscopic image data I2 presented on the screen of the 3D television 67. Adjustments were made.

  Table 1 shows the parallax settings of the stereoscopic image data I1 and I2 used in the experiment.

  As shown in Table 1, the stereoscopic image data I1 is a stereoscopic image in which the parallax angle of the subject in the latest scene is +1.5 degrees and the parallax angle of the subject in the farthest scene is +0.5 degrees, and the whole image appears to pop out. It is data. The stereoscopic image data I2 is stereoscopic image data that appears to be retracted as a whole, with the parallax angle of the subject in the most recent scene being −0.48 degrees and the parallax angle of the subject in the farthest view being −1.48 degrees. In these two stereoscopic image data, the subject is the same, and only the parallax angle range is different. In addition, any stereoscopic image data has a parallax angle range width of 1.0 degree.

  The amount of parallax adjustment is 0 to ± 28 steps, and the parallax is set to change by 7070755 degrees in the parallax angle for each step. Accordingly, the maximum change amount of the parallax adjustment is ± 1.998114 degrees.

  In the experiment, the stereoscopic image data I1 and I2 were displayed three times in random order, six times in total, and a gray image was displayed between each display. In addition, two experiments were performed to confirm the reproducibility of the experimental results. Hereinafter, the first experiment is referred to as Experiment 1, and the second experiment is referred to as Experiment 2.

<4-2. Experimental results>
Table 2 shows the results of Experiment 1, and Table 3 shows the results of Experiment 2.

  Tables 2 and 3 describe the parallax adjustment amount by the subject in three displays and the average of the stereoscopic image data I1 and I2 for each subject. The average describes two values with different units. The parallax adjustment amounts for the first to third times and the average on the left side are values in units of “stage”. The average on the right side is a value obtained by converting the average value of the parallax adjustment amount into the parallax angle, and the unit is “degree”. Further, the lower part of the table shows a value obtained by averaging the average parallax adjustment amount of all subjects.

<4-2-1. Experimental Results on Stereoscopic Image Data I1>
In the result of Experiment 1 shown in Table 2, the average parallax adjustment amount when viewing the pop-out stereoscopic image data I1 is −17.078, and when this is converted into the parallax angle, it is −1.208 degrees. Become. Therefore, the parallax of the stereoscopic image data I1 after the parallax adjustment is performed so that the subject is comfortable is from the range of +1.5 degrees to +0.5 degrees before the parallax adjustment, and from +0.292 degrees to -0.708. The range of degrees has changed. This means that the parallax has been adjusted in the retracting direction.

  In the result of Experiment 2 shown in Table 3, the average of the parallax adjustment amount when viewing the pop-out stereoscopic image data I1 is −18.056, and when this is converted into the parallax angle, it is −1.278 degrees. Become. Therefore, the parallax of the stereoscopic image data I1 after the parallax adjustment is performed so that the subject is comfortable is from the range of +1.5 degrees to +0.5 degrees before the parallax adjustment, and from +0.222 degrees to −0.778. The range of degrees has changed. This also means that the parallax is adjusted in the retracting direction, similar to the result of Experiment 1.

  Further, regarding the average value of the parallax adjustment amount of each subject with respect to the stereoscopic image data I1, the correlation coefficient of the results of Experiment 1 and Experiment 2 was 0.695, and a moderate positive correlation was recognized. Therefore, this experimental result is considered to be a reproducible measurement value.

<4-2-2. Experimental Results on Stereoscopic Image Data I2>
In the result of Experiment 1 shown in Table 2, the average parallax adjustment amount when viewing the retracted stereoscopic image data I2 is 6.633 levels, which is +0.469 degrees when converted into a parallax angle. Accordingly, the parallax of the stereoscopic image data I2 after the parallax adjustment is performed so that the subject is comfortable is from −0.011 degrees to −0.011 degrees from the range of −0.48 degrees to −1.48 degrees before the parallax adjustment. It has changed to a range of 1.011 degrees. This means that the parallax has been adjusted in the protruding direction.

  In the result of Experiment 2 shown in Table 3, the average parallax adjustment amount when viewing the retracted stereoscopic image data I2 is 7.000 levels, which is +0.495 degrees when converted into the parallax angle. Therefore, the parallax of the stereoscopic image data I2 after the parallax adjustment is performed so that the subject is comfortable is +0.015 degrees to −0 from the range of −0.48 degrees to −1.48 degrees before the parallax adjustment. It changes to the range of 985 degrees. This also means that the parallax has been adjusted in the pop-out direction, similar to the result of Experiment 1.

  Further, regarding the average value of the parallax adjustment amount of each subject with respect to the stereoscopic image data I2, the correlation coefficient of the results of Experiment 1 and Experiment 2 was 0.817, and a strong positive correlation was recognized. Therefore, this experimental result is considered to be a reproducible measurement value.

<4-2-3. Summary of experimental results>
As a result of the above experiment, two parallax angle ranges (+0.292 degrees to −0.708 degrees, +0.222 degrees to −0.778 degrees) after parallax adjustment obtained in the experiment related to the stereoscopic image data I1; From the parallax angle ranges (−0.011 degrees to −1.011 degrees, +0.015 degrees to −0.985 degrees) after the parallax adjustment obtained in the experiment on the stereoscopic image data I1, these parallax angle ranges are obtained. Is a parallax angle range of +0.292 degrees to -1.011 degrees, and roughly a parallax angle range of +0.3 degrees to -1.0 degrees. This parallax angle range is considered to be a parallax angle range that makes the subject feel comfortable with respect to stereoscopic image data having a parallax angle range width of 1 degree. In the present invention, this is referred to as a “comfortable parallax angle range”.

  FIG. 32 is a diagram schematically illustrating a parallax angle range before parallax adjustment, a parallax angle range after parallax adjustment, and a comfortable parallax angle range of stereoscopic image data in a subjective evaluation experiment. The parallax angle range 68 before the parallax adjustment of the stereoscopic image data I1 is adjusted to the parallax angle range 69 after the parallax adjustment of the stereoscopic image data I1 by the parallax adjustment of Experiment 1, and the stereoscopic image data I1 of the stereoscopic image data I1 by the parallax adjustment of Experiment 2 is adjusted. The parallax angle range after the parallax adjustment was adjusted to 70. Further, the parallax angle range 71 before parallax adjustment of the stereoscopic image data I2 is adjusted to the parallax angle range 72 after parallax adjustment of the stereoscopic image data I2 by the parallax adjustment of Experiment 1, and the stereoscopic image data by the parallax adjustment of Experiment 2 The parallax angle range 73 after the parallax adjustment of I2 was adjusted. The comfortable parallax angle range 74 is a union of the parallax angle ranges 69 and 70 after the parallax adjustment of the stereoscopic image data I1 and the parallax angle ranges 72 and 73 after the parallax adjustment of the stereoscopic image data I2.

  According to this experimental result, it is considered that the stereoscopic image data in which the parallax angle range is slightly biased in the retracting direction is easier for the viewer to see. This is similar to the situation where the subject located behind the screen of the display device sees the subject behind the window, and the viewer does not feel uncomfortable, but the subject located in front of the screen of the display device is so called from the window. This is a situation where the subject is popping out to the front, and it seems that the viewer may feel uncomfortable. Furthermore, when a subject located in front of the screen of the display device touches the edge of the screen, the edge of the screen is perceived as being at the distance of the display device, while the subject in contact with it pops out to the front of the screen. Due to the visible parallax, it may be difficult to perceive as being in front due to the gap. For this reason, it is considered that the viewer has adjusted the parallax angle range of the stereoscopic image so that it is slightly biased in the retracting direction, as in the above experimental results.

<5. Setting each reference value used for parallax adjustment processing>
Based on the above experimental results, in the present invention, each reference value used for the parallax adjustment processing is set as follows.

  According to Non-Patent Document 1, the stereoscopic image data has a parallax angle range within one screen within a range of −1.0 degrees to +1.0 degrees and a parallax angle range width within 1.0 degrees. It is recommended. On the other hand, the comfortable parallax angle range obtained by the above experiment suggests that stereoscopic image data in which the parallax angle range is slightly biased in the retracting direction is easier to see. Therefore, within the recommendation of Non-Patent Document 1, it can be considered that more comfortable viewing can be achieved by performing parallax adjustment that slightly moves the parallax angle range in the retracting direction.

  As described above, the comfortable parallax angle range for stereoscopic image data having a parallax angle range width of 1 degree is approximately +0.3 degrees to -1.0 degrees in terms of the parallax angle. The parallax angle range width is 1.3 degrees.

  Here, as described in Non-Patent Document 1, it is easy to see the parallax angle range width within one screen within 1.0 degree. Therefore, within the comfortable parallax angle range, the parallax angle range width is 1. Consider setting a range of 0 degrees. In the present invention, this range is referred to as a “standard comfortable parallax angle range”. When the standard comfortable parallax angle range is set closest to the foreground side, it is + 0.3 ° to −0.7 °, and when set to the farthest side, it is 0.0 ° to −1.0 °. Thus, the standard comfortable parallax angle range setting has a width of 0.3 degrees.

Furthermore, if the parallax angle of the nearest scene with respect to the standard comfortable parallax angle range is defined as the standard parallax angle, the parallax angle of the farthest view is defined as the standard parallax angle, and the center parallax angle of the range is defined as It becomes the following values.
Standard parallax angle: +0.3 degrees to 0.0 degrees Standard farthest scene parallax angle: -0.7 degrees to -1.0 degrees Standard central parallax angle: -0.2 degrees to -0.5 degrees Every time

  Here, since the standard center parallax angle is the center of the standard comfortable parallax angle range, it can be considered to be the parallax angle at which stereoscopic viewing is most comfortable. Accordingly, the normal mode in step S63 described above is to perform the parallax adjustment so that the parallax angle of the main subject in the stereoscopic image, that is, the region that is likely to be watched by the viewer becomes the standard central parallax angle. This is the idea of parallax adjustment processing. Therefore, a value of −0.2 degrees to −0.5 degrees, which is the value of the standard central parallax angle, is suitable as the value of T2 in the normal mode.

  Further, as described above, the enhanced parallax adjustment mode (the pop-out mode or the retraction mode) in step S65 is a sensation in which the subject of the stereoscopic image data appears to pop out as much as possible within the parallax angle range in which the stereoscopic image data can be comfortably viewed ( In this mode, parallax adjustment is performed to emphasize the feeling of popping out or to enhance the feeling that the subject appears to expand toward the back of the screen (retraction feeling). Accordingly, the value of T3 in the pop-out mode is suitably +0.3 to 0.0 degrees, which is the value of the standard closest view parallax angle. Further, as the value of T4 in the retract mode, −0.7 degrees to −1.0 degrees that is the value of the standard farthest view parallax angle is suitable.

  Furthermore, as described above, it is suitable to adjust the parallax angle of the farthest view to be T4 degrees also in the process of step S67, that is, the process when the stereoscopic image data is only the distant view image. This is because T4 is a value of the standard farthest view parallax angle as described above, and it is considered that adjusting the far view image to T4 degrees is most easily seen.

  On the other hand, if the parallax angle range of the stereoscopic image is too biased in the retracting direction, there is no subject that appears to pop out, and there is a risk that the stereoscopic effect will be poor even if comfortable. Therefore, it can be considered that it is desirable that the parallax angle range also extends in the pop-out direction from the viewpoint of producing the power and fun of the stereoscopic image.

In consideration of the above, the comfortable parallax angle range obtained by the above experiment (parallax angle +0.292 degrees to −1.011 degrees) is a standard range that is close to the near view and has a parallax angle range width of about 1.0 degrees. It is more suitable to define a comfortable parallax angle range. Specifically, it is more suitable to set the standard closest-view parallax angle and the like and T2 to T4 to the following values.
Standard closest view parallax angle, T3: +0.3 degrees to +0.2 degrees Standard farthest view parallax angle, T4: −0.7 degrees to −0.8 degrees Standard central parallax angle, T2: −0.2 Degree -0.3 degree

<6. Other>
In the above description, the process in the parallax adjustment amount calculation unit 6 is performed using the parallax angle value, but may be performed using the parallax value corresponding thereto. In this case, for example, values such as the above-mentioned standard closest view parallax angle, standard farthest view parallax angle, standard center parallax angle, and the like may be determined by the corresponding parallax values. In this case, the parallax map may be input directly from the parallax calculation unit 4 to the parallax adjustment amount calculation unit 6, and the parallax angle calculation unit 5 is unnecessary.

  Further, the parallax adjustment amount P described above may not be directly used as the parallax adjustment amount in the parallax adjustment unit 7, but may be used as the maximum value of the parallax adjustment amount. For example, in automatic parallax adjustment, the maximum value of the parallax adjustment amount when parallax adjustment is performed by increasing / decreasing the parallax adjustment amount by a predetermined value as the predetermined time elapses instead of adjusting the parallax at once. As an alternative, the parallax adjustment amount may be used. When the parallax adjustment amount P is obtained for each frame of the stereoscopic image data, the parallax adjustment amount P may change abruptly due to the parallax status of the stereoscopic image data or due to an error in the process of calculating the parallax adjustment amount P. When the parallax adjustment is performed using the same as it is, the parallax changes abruptly, making it difficult to see. On the other hand, as described above, this can be prevented by increasing or decreasing the parallax adjustment amount by a predetermined value as the predetermined time elapses.

  Further, the parallax adjustment amount P may be used as the maximum value of the adjustment amount when the user performs parallax adjustment for each predetermined unit of parallax adjustment amount by manual input. Both parallax adjustment by the user himself and ensuring of comfort by setting the parallax adjustment amount P to the maximum value of the adjustment amount can be achieved.

  In step S7, the parallax adjustment in the parallax adjustment unit 7 is realized by uniformly shifting the entire screen. However, by dividing the subjects having the same parallax into groups and shifting the images for each group, the parallax can be adjusted. Adjustments may be made.

  At this time, the stereoscopic image data processing apparatus of the present invention performs the following operations in addition to the operations described in the above description.

  For example, when the parallax calculation unit 4 calculates the parallax map, subjects having the same parallax are divided into groups, and group information is output together with the parallax map.

  Further, the parallax adjustment amount calculation unit 6 sets the entire parallax angle range around the parallax adjustment base angle in addition to the parallax adjustment amount P that is the parallax adjustment amount of the parallax adjustment base angle itself. Either the latest view parallax angle, the adjusted farthest view parallax angle, or the adjusted parallax angle range may be output as an additional parameter of the parallax adjustment amount P. And the farthest view parallax angle after adjustment is output.

  It should be noted that the adjusted near-field parallax angle, the farthest-view parallax angle after adjustment, or the adjusted parallax angle range may be input by the user from the outside, or using values set in advance for each device. For example, when the parallax angle range width is set to be within 1.0 degree, the adjusted value of the closest view parallax angle may be +0.25 degrees and the adjusted farthest view parallax angle may be -0.75 degrees. Good.

  In consideration of the safety of stereoscopic viewing, it is basically recommended to set a value with a parallax angle range width within 1.0 degree. However, as an exception, a value with a parallax angle range width exceeding 1 degree is recommended. You can set it. However, in this case, the adjusted value of the closest view parallax angle is +0.292 degrees, and the adjusted farthest view parallax angle is -1.011 degrees.

  Furthermore, the parallax adjustment unit 7 receives the group information, the parallax adjustment amount P, the adjusted latest view parallax angle, and the adjusted farthest view parallax angle. First, the parallax adjustment base angle and the parallax adjustment amount P are input. On the basis of the parallax adjustment, and then the parallax adjustment is performed so that the overall parallax angle range after the parallax adjustment is equal to the adjusted closest disparity angle and the adjusted farthest disparity angle. The parallax adjustment for individual groups is performed by shifting the image again for each group so that the parallax size is enlarged or reduced and enlarged or reduced.

  The enlargement or reduction at this time may be linear, or may be weighted according to the magnitude of the value.

  Further, in the parallax adjustment for the individual group at this time, the individual group may be divided into a group having a parallax in the pop-out direction and a group having a parallax in the withdrawal direction, and the parallax adjustment may be performed.

  As described above, subjects having the same parallax are divided into groups, and parallax adjustment is performed for each group, so that the parallax angle T2 that can be viewed most comfortably is centered, and at the same time, a desired parallax angle range is obtained. The parallax adjustment can be performed, and further, safe and comfortable stereoscopic image data can be viewed with less burden on the eyes.

  In the above description, the case where stereoscopic image data is captured has been described. However, when a 2D image is captured, the processing is not performed in each device, and is output to the image display unit 8 as it is to display a planar image. good.

  Also, the stereoscopic image data input unit 2 newly generates image data of stereoscopic image data by performing 2D-3D conversion processing, which is processing for generating stereoscopic image data from the 2D image, on the captured 2D image. Also good.

  In the above description, a case where a camera is installed as the stereoscopic image data input unit 2 and stereoscopic image data is captured has been described. However, various recording media can be used for the stereoscopic image data input unit 2 instead of the captured image from the camera. Alternatively, the image data may be directly input from the outside, such as Internet distribution or broadcast wave reception. At this time, the stereoscopic image data input unit 2 converts the received image data into image data in a predetermined format and outputs the image data to the parallax calculation unit 4.

  In the above description, the case of an image with two left and right viewpoints has been described. For example, it may be multi-view stereoscopic image data captured by a multi-view imaging system. The parallax, the calculation of the parallax adjustment amount, the parallax adjustment, and the like may be performed in the same manner as when each image is processed.

  The present invention is not limited to the above-described stereoscopic image data display device 75 such as a stereoscopic television, but also a stereoscopic digital camera, a stereoscopic digital movie, a stereoscopic digital video recorder, a stereoscopic portable movie player, a stereoscopic mobile phone, The present invention can be widely applied to apparatuses that handle stereoscopic image data, such as a stereoscopic car navigation system, a stereoscopic portable DVD player, and a stereoscopic PC. In addition, these devices may be stereoscopic image data processing devices that do not include an image display unit and use an external stereoscopic image display device. Further, a stereoscopic image data display system including a stereoscopic image data processing device and an external stereoscopic image display device may be used.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like without departing from the gist of the present invention.

1 ... Stereoscopic image data processing device,
2 ... Stereoscopic image data input unit,
3 ... User instruction input part,
4 ... parallax calculation unit,
5 ... Parallax angle calculator,
6 ... parallax adjustment amount calculation unit,
7: Parallax adjusting unit,
8 ... Image display section,
9L ... Left camera,
9R ... Right camera,
10, 11 ... subject,
12L, 28L, 33L, 58L, 61L ... Left eye image data,
12R, 28R, 33R, 58R, 61R ... right eye image data,
13L, 13R, 29L, 29R, 34L, 34R ... Recent points,
14L, 14R, 30L, 30R, 35L, 35R, 59L, 59R, 62L, 62R ... Farthest point,
15L ... Left eye,
15R ... right eye,
16, 24 ... screen,
17, 18L, 18R, 19, 20L, 20R, 20, 22L, 22R, 23...
25… Viewers,
26, 27, 31, 32, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 60, 63 ... dotted line,
64 ... Active shutter glasses,
65… Subject,
66 ... video playback device,
67 ... 3D TV,
68 ... Parallax angle range before parallax adjustment of the stereoscopic image data I1,
69, 70 ... Parallax angle range after the parallax adjustment of the stereoscopic image data I1,
71 ... The parallax angle range before the parallax adjustment of the stereoscopic image data I2,
72, 73 ... The parallax angle range after the parallax adjustment of the stereoscopic image data I2,
74 ... Comfortable parallax angle range,
75… Stereoscopic image data display device

Claims (16)

In a stereoscopic image data processing apparatus that performs parallax adjustment on stereoscopic image data that is stereoscopic image data,
A stereoscopic image data input unit for inputting the stereoscopic image data;
A parallax adjustment amount calculation unit that calculates a parallax adjustment amount based on a parallax adjustment base angle that is a parallax angle used as a reference for parallax adjustment;
A parallax adjustment unit configured to perform parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generate parallax adjustment stereoscopic image data;
The parallax adjustment amount is for moving the parallax adjustment base angle to any value from -0.5 degrees to -0.2 degrees.   The stereoscopic image data processing apparatus according to claim 1, wherein the parallax adjustment base angle is 0 degree. A parallax calculation unit that calculates a parallax map from the stereoscopic image data;
The stereoscopic image data processing apparatus according to claim 1, wherein the parallax adjustment amount calculation unit calculates the parallax adjustment base angle using the parallax map.   The stereoscopic image data processing apparatus according to claim 3, wherein the parallax adjustment base angle is a median value of the parallax angle value of the farthest view and the parallax angle value of the closest view in the parallax map.   The stereoscopic image data processing apparatus according to claim 3, wherein the parallax adjustment base angle is a parallax angle having the highest appearance frequency in the parallax map.   The stereoscopic image data processing apparatus according to claim 3, wherein the parallax adjustment base angle is an average value of parallax angles in the parallax map.   4. The stereoscopic image according to claim 3, wherein the parallax adjustment base angle is a parallax angle determined so that a sum of squares of differences with each parallax angle in the parallax map is minimized with respect to the stereoscopic image data. Image data processing device. A stereoscopic image data processing method for performing parallax adjustment on stereoscopic image data, which is stereoscopic image data,
A stereoscopic image data input step for inputting the stereoscopic image data;
A parallax adjustment amount determination step for calculating a parallax adjustment amount based on a parallax adjustment base angle that is a parallax angle used as a reference for parallax adjustment;
A parallax adjustment step of performing parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generating parallax adjustment stereoscopic image data;
The parallax adjustment amount is obtained by moving the parallax adjustment base angle to any value from -0.5 degrees to -0.2 degrees.   The stereoscopic image data processing method according to claim 8, wherein the parallax adjustment base angle is 0 degree. A parallax calculation step of calculating a parallax map from the stereoscopic image data;
The stereoscopic image data processing method according to claim 8, wherein the parallax adjustment amount determining step calculates the parallax adjustment base angle using the parallax map. In a stereoscopic image data processing apparatus that performs parallax adjustment on stereoscopic image data that is stereoscopic image data,
A stereoscopic image data input unit for inputting the stereoscopic image data;
A parallax calculation unit that calculates a parallax map from the stereoscopic image data;
A parallax adjustment amount calculating unit that calculates a parallax angle of the nearest scene of the stereoscopic image data using the parallax map, and calculates a parallax adjustment amount based on the parallax angle of the nearest scene;
A parallax adjustment unit configured to perform parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generate parallax adjustment stereoscopic image data;
The parallax adjustment amount is for moving the parallax angle of the nearest scene to any value from +0.3 degrees to 0.0 degrees. A stereoscopic image data processing method for performing parallax adjustment on stereoscopic image data, which is stereoscopic image data,
A stereoscopic image data input step for inputting the stereoscopic image data;
A parallax calculation step of calculating a parallax map from the stereoscopic image data;
A parallax adjustment amount determining step of calculating a parallax angle of the nearest scene of the stereoscopic image data using the parallax map and calculating a parallax adjustment amount based on the parallax angle of the nearest scene;
A parallax adjustment step of performing parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generating parallax adjustment stereoscopic image data;
The stereoscopic image data processing method, wherein the parallax adjustment amount is a value for moving the parallax angle of the nearest scene to any value between +0.3 degrees and 0.0 degrees. In a stereoscopic image data processing apparatus that performs parallax adjustment on stereoscopic image data that is stereoscopic image data,
A stereoscopic image data input unit for inputting the stereoscopic image data;
A parallax calculation unit that calculates a parallax map from the stereoscopic image data;
A parallax adjustment amount calculation unit that calculates a parallax angle of the farthest view of the stereoscopic image data using the parallax map, and calculates a parallax adjustment amount based on the parallax angle of the farthest view;
A parallax adjustment unit configured to perform parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generate parallax adjustment stereoscopic image data;
The parallax adjustment amount is for moving the parallax angle of the farthest view to any value from -0.7 degrees to -1.0 degrees. A stereoscopic image data processing method for performing parallax adjustment on stereoscopic image data, which is stereoscopic image data,
A stereoscopic image data input step for inputting the stereoscopic image data;
A parallax calculation step of calculating a parallax map from the stereoscopic image data;
A parallax adjustment amount determining step of calculating a parallax angle of the farthest view of the stereoscopic image data using the parallax map, and calculating a parallax adjustment amount based on the parallax angle of the farthest view;
A parallax adjustment step of performing parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generating parallax adjustment stereoscopic image data;
The stereoscopic image data processing method, wherein the parallax adjustment amount moves the parallax angle of the farthest view from any one of -0.7 degrees to -1.0 degrees. In a stereoscopic image data processing apparatus that performs parallax adjustment on stereoscopic image data that is stereoscopic image data,
A stereoscopic image data input unit for inputting the stereoscopic image data;
A parallax calculation unit that calculates a parallax map from the stereoscopic image data;
A parallax adjustment amount calculation unit that calculates a parallax angle of the farthest view of the stereoscopic image data using the parallax map, and calculates a parallax adjustment amount based on the parallax angle of the farthest view;
A parallax adjustment unit configured to perform parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generate parallax adjustment stereoscopic image data;
The parallax adjustment amount calculation unit includes a parallax angle range included in the parallax map that does not exceed any value of −0.1 degrees to 0.0 degrees and is included in the parallax map. The parallax adjustment amount for moving the parallax angle of the farthest view from any one of -0.7 degrees to -1.0 degree is calculated when all the parallax angles are negative. A stereoscopic image data processing apparatus. A stereoscopic image data processing method for performing parallax adjustment on stereoscopic image data, which is stereoscopic image data,
A stereoscopic image data input step for inputting the stereoscopic image data;
A parallax calculation step of calculating a parallax map from the stereoscopic image data;
A parallax adjustment amount determination step of calculating a parallax angle of the farthest view of the stereoscopic image data using the parallax map, and calculating a parallax adjustment amount based on the parallax angle of the farthest view;
A parallax adjustment step of performing parallax adjustment on the stereoscopic image data based on the parallax adjustment amount and generating parallax adjustment stereoscopic image data;
In the parallax adjustment amount determining step, the width of the range of the parallax angle included in the parallax map does not exceed any value from −0.1 degrees to 0.0 degrees and is included in the parallax map. The parallax adjustment amount for moving the parallax angle of the farthest view from any one of -0.7 degrees to -1.0 degree is calculated when all the parallax angles are negative. 3D image data processing method.

Images