JP4150924B2 - Image processing system, projector, program, information storage medium, and image processing method - Google Patents

Description

  The present invention relates to an image processing system capable of correcting image distortion, a projector, a program, an information storage medium, and an image processing method.

  The image may be distorted depending on the relative angle between the optical axis of the projection light from the image projection apparatus such as a projector and the projection target such as a screen, and so-called trapezoidal distortion may occur in the vertical and horizontal directions. .

  Therefore, when displaying an image, the image display apparatus needs to display the image in a state in which the distortion of the image is eliminated.

  However, a general projector with an image distortion correction function has a built-in tilt sensor to correct only the vertical distortion of the image and cannot correct the horizontal distortion of the image.

  Also, when correcting the horizontal distortion of the image, the projector uses a mouse or the like to indicate the four corner points of the screen, so that the projector semi-automatically corrects the image distortion based on the instruction information. Yes. In addition, it is troublesome for the user to designate the four corner points of the screen using a mouse or the like.

  In order to solve such a problem, for example, in Patent Document 1, based on a video signal input from a monitor camera, a screen position in an image captured by the monitor camera is detected, and the detected screen position is determined. An automatic screen position adjustment device for a projector that adjusts trapezoidal distortion of a projected image based on data is disclosed.

Further, in the method of Patent Document 1, as described in paragraph No. 0029 of the specification of Patent Document 1, the trapezoidal distortion is adjusted by comparing the lengths of opposite sides of the screen in the captured monitor screen. ing.
JP 2000-241874 A JP 2002-247614 A

  However, when the aspect ratio (aspect ratio) of a projection object such as a screen is not known, the projector cannot properly grasp the distortion of the image only by comparing the lengths of the opposite sides of the screen.

  Japanese Patent Application Laid-Open No. 2004-228620 discloses a projector that identifies an external shape of a screen using an imaging unit and corrects an image so that the external shape is the same as the identified external shape.

  However, if the projector changes the shape of the image to the shape of the outer shape of the screen as in Patent Document 2, the image after the change will be different from the outer shape of the original image. In this case, since an impression different from that of the original image is given to the observer, it is not preferable to employ such a processing method.

  Further, even when the aspect ratio of the projection object is used as a reference, it is desirable that the projected image is displayed with a desired aspect ratio.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing system, a projector, and an image processing system capable of grasping the aspect ratio of the projection object and the distortion of the image and correcting the distortion of the projected image. To provide a program, an information storage medium, and an image processing method.

In order to solve the above problems, an image processing system and a projector project a single-color calibration image composed of an arbitrary single color toward a projection target, and include a first monochrome region smaller than the entire image region, And an image projecting means for projecting a multi-color calibration image in which a region other than the region is composed of a second single color other than the first single color toward the projection target;
The first imaging information is generated by capturing the single color calibration image projected on the projection object, and the second imaging information is generated by capturing the multi-color calibration image projected on the projection object. Imaging means for generating
Projection target area information generation means for generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging means based on the difference in image signal value for each pixel based on the first imaging information. When,
Based on the difference between the first imaging information and the second imaging information, the first monochrome area in the imaging area is detected as a partial projection area, and based on the partial projection area, in the imaging area Projection area information generating means for generating projection area information indicating the coordinates of the four corners of the projection area;
Correction information generating means for generating image distortion correction information based on the projection target area information and the projection area information;
Distortion correction means for correcting the image signal based on the image distortion correction information so that the distortion of the image is corrected;
Including
The image projecting unit projects an image based on the image signal corrected by the distortion correcting unit.

The program according to the present invention is a computer-readable program,
Computer
A single color calibration image composed of any single color is projected onto the projection target, includes a first monochrome region smaller than the entire image region, and a region other than the first region is other than the first monochrome color. Image projection means for projecting a multi-color calibration image composed of the second single color onto the projection object;
The first imaging information is generated by capturing the single color calibration image projected on the projection object, and the second imaging information is generated by capturing the multi-color calibration image projected on the projection object. Imaging means for generating
Projection target area information generation means for generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging means based on the difference in image signal value for each pixel based on the first imaging information. When,
Based on the difference between the first imaging information and the second imaging information, the first monochrome area in the imaging area is detected as a partial projection area, and based on the partial projection area, in the imaging area Projection area information generating means for generating projection area information indicating the coordinates of the four corners of the projection area;
Correction information generating means for generating image distortion correction information based on the projection target area information and the projection area information;
Based on the image distortion correction information, function as a distortion correction unit that corrects the image signal so that the distortion of the image is corrected,
The image projecting unit projects an image based on the image signal corrected by the distortion correcting unit.

An information storage medium according to the present invention is an information storage medium storing a computer-readable program,
The program is stored.

Further, the image processing method according to the present invention projects a single-color calibration image composed of any single color onto a projection object using an image projection unit ,
Imaging the monochromatic calibration image projected on the projection object using an imaging unit to generate first imaging information;
A multi-color calibration image including a first single color area smaller than the entire image area and a region other than the first region composed of a second single color other than the first single color is displayed on the image projection unit. Use to project toward the projection object,
The multi-color calibration image projected on the projection target is imaged using the imaging unit to generate second imaging information,
Based on the difference in image signal value for each pixel based on the first imaging information, generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging unit ,
Based on the difference between the first imaging information and the second imaging information, the first monochrome area in the imaging area is detected as a partial projection area, and based on the partial projection area, in the imaging area Generating projection area information indicating the coordinates of the four corners of the projection area;
Based on the projection area information and the projection area information, image distortion correction information is generated,
Based on the image distortion correction information, correct the image signal so that the image distortion is corrected,
An image is projected using the image projection unit based on the corrected image signal.

  According to the present invention, an image processing system or the like can grasp the outer shape of a projection target (for example, a screen or the like) based on a difference in image signal values (for example, a luminance value) based on imaging information.

  Further, according to the present invention, even when the calibration image is projected larger than the projection object, the image processing system or the like uses the multi-color calibration image to restore the shape of the original image. A region of the first single color (for example, white, gray close to white, etc.) that has been reduced can be formed on the projection target.

  Then, the image processing system or the like can grasp the distortion of the image by imaging the first monochrome region formed on the projection target.

  Also, the image processing system or the like provides image distortion correction information indicating how much the image should be adjusted based on the difference between the projection position of the image after distortion correction and the projection position of the image before distortion correction. And distortion of the image can be corrected appropriately.

In the image processing system, the projector, the program, and the information storage medium, the first monochrome area is an area obtained by reducing the entire image area to 1 / n,
The projection area information generation unit may detect an area obtained by multiplying the partial projection area by n as the projection area.

In the image processing method, the first monochrome region is a region obtained by reducing the entire image region to 1 / n,
An area obtained by multiplying the partial projection area by n may be detected as the projection area.

  According to this, the image processing system or the like can detect the projection area more efficiently.

In the image processing system, the projector, the program, and the information storage medium, the correction information generation unit includes:
Based on the projection target area information, on the optical axis of the origin in a three-dimensional space obtained by adding the optical axis of the imaging means as the third dimension to the two-dimensional plane of the imaging area, and the four corners of the projection target Calculate the relative distance,
Based on the projection target area information and the relative distance, the coordinates in the three-dimensional space of the four corners of the projection target are calculated,
Based on the coordinates, the first aspect ratio indicating the aspect ratio of the projection object is calculated by calculating the lengths of two opposite sides of the projection object,
Based on the first aspect ratio, the second aspect ratio indicating the aspect ratio of the projection panel included in the single color calibration image, the multiple color calibration image, or the image projection unit, and the projection target area information. Calculating the coordinates in the imaging region at the four corners of the image when the image projection means projects onto the projection object in the second aspect ratio without distortion,
The image distortion correction information may be generated based on the coordinates in the imaging area at the four corners of the image and the projection area information.

Further, in the image processing method, based on the projection target region information, an origin in a three-dimensional space obtained by adding the optical axis of the imaging unit as a third dimension to a two-dimensional plane of the imaging region, and 4 of the projection target Calculate the relative distance on the optical axis with the corner,
Based on the projection target area information and the relative distance, the coordinates in the three-dimensional space of the four corners of the projection target are calculated,
Based on the coordinates, the first aspect ratio indicating the aspect ratio of the projection object is calculated by calculating the lengths of two opposite sides of the projection object,
Based on the first aspect ratio, the second aspect ratio indicating the aspect ratio of the single color calibration image, the multiple color calibration image, or the projection panel of the image projection unit , and the projection target area information. Calculating coordinates in the imaging region at the four corners of the image when projected onto the projection object in the second aspect ratio without distortion,
The image distortion correction information may be generated based on the coordinates in the imaging area at the four corners of the image and the projection area information.

An image processing system and a projector according to the present invention include an image projecting unit that projects a calibration image onto a projection target,
Imaging means for capturing the calibration image and generating imaging information;
Projection target area information generating means for generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging means based on the imaging information;
Projection area information generating means for generating projection area information indicating the coordinates of the four corners of the projection area in the imaging area based on the imaging information;
Correction information generating means for generating image distortion correction information based on the projection target area information and the projection area information;
Distortion correction means for correcting the image signal based on the image distortion correction information so that the distortion of the image is corrected;
Including
The image projection unit projects an image based on the image signal corrected by the distortion correction unit,
The correction information generating means grasps a first aspect ratio indicating an aspect ratio of the projection target based on the projection target area information, and indicates a first aspect ratio indicating a desired aspect ratio according to the first aspect ratio. Generating projection coordinate system projection area information indicating the coordinates of the four corners of the projection area in the projection panel of the image projection means based on the projection area information so that an image is displayed with an aspect ratio of 2. The image distortion correction information is generated based on the projection coordinate system projection area information.

The program according to the present invention is a computer-readable program,
Computer
Image projection means for projecting the calibration image onto the projection object;
Imaging means for capturing the calibration image and generating imaging information;
Projection target area information generating means for generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging means based on the imaging information;
Projection area information generating means for generating projection area information indicating the coordinates of the four corners of the projection area in the imaging area based on the imaging information;
Correction information generating means for generating image distortion correction information based on the projection target area information and the projection area information so that an image is projected on the projection target area at a desired aspect ratio;
Based on the image distortion correction information, function as a distortion correction unit that corrects the image signal so that the distortion of the image is corrected,
The image projection unit projects an image based on the image signal corrected by the distortion correction unit,
The correction information generating means grasps a first aspect ratio indicating an aspect ratio of the projection target based on the projection target area information, and indicates a first aspect ratio indicating a desired aspect ratio according to the first aspect ratio. Generating projection coordinate system projection area information indicating the coordinates of the four corners of the projection area in the projection panel of the image projection means based on the projection area information so that an image is displayed with an aspect ratio of 2. The image distortion correction information is generated based on the projection coordinate system projection area information.

  An information storage medium according to the present invention is an information storage medium storing a computer-readable program, and stores the program.

The image processing method according to the present invention projects a calibration image onto a projection object,
Imaging the calibration image to generate imaging information,
Based on the imaging information, generate projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging means,
Based on the imaging information, generating projection area information indicating the coordinates of the four corners of the projection area in the imaging area,
Based on the projection area information, generating projection coordinate system projection area information indicating the coordinates of the four corners of the projection area in the projection panel of the image projection means,
Based on the projection coordinate system projection area information, grasp the first aspect ratio indicating the aspect ratio of the projection object,
Generating image distortion correction information so that an image is projected at a second aspect ratio indicating a desired aspect ratio in accordance with the first aspect ratio;
Based on the image distortion correction information, correct the image signal so that the image distortion is corrected,
An image is projected based on the corrected image signal.

  According to the present invention, the image processing system or the like does not match the shape of the projection target, but generates the image distortion correction information so that the image is projected at the original aspect ratio of the image. An image can be displayed at a ratio.

  Further, in the image processing system, the projector, the program, and the information storage medium, the correction information generation unit has a first aspect ratio indicating an aspect ratio of the projection target based on the projection target area information. Based on the projection area information, the projection on the projection panel of the image projection unit is grasped so that an image is displayed with a second aspect ratio indicating a desired aspect ratio according to the first aspect ratio. The projection coordinate system projection area information indicating the coordinates of the four corners of the area may be generated, and the image distortion correction information may be generated based on the projection coordinate system projection area information.

Further, in the image processing method, based on the projection target area information, a first aspect ratio indicating an aspect ratio of the projection target is grasped,
Projection indicating the coordinates of the four corners of the projection area on the projection panel based on the projection area information so that an image is displayed with a second aspect ratio indicating a desired aspect ratio according to the first aspect ratio. Generate coordinate system projection area information,
The image distortion correction information may be generated based on the projection coordinate system projection area information.

  According to this, the image processing system or the like does not match the shape of the projection target object, but generates the image distortion correction information so that the image is displayed with a desired aspect ratio. Regardless, the image can be displayed with a desired aspect ratio.

In the image processing system, the projector, the program, and the information storage medium, the correction information generation unit includes:
Based on the projection target area information, on the optical axis of the origin in a three-dimensional space obtained by adding the optical axis of the imaging means as the third dimension to the two-dimensional plane of the imaging area, and the four corners of the projection target Calculate the relative distance,
Based on the projection target area information and the relative distance, the coordinates in the three-dimensional space of the four corners of the projection target are calculated,
The first aspect ratio may be calculated by calculating the lengths of two opposite sides of the projection object based on the coordinates.

Further, in the image processing method, based on the projection target region information, an origin in a three-dimensional space obtained by adding the optical axis of the imaging unit as a third dimension to a two-dimensional plane of the imaging region, and 4 of the projection target Calculate the relative distance on the optical axis with the corner,
Based on the projection target area information and the relative distance, the coordinates in the three-dimensional space of the four corners of the projection target are calculated,
The first aspect ratio may be calculated by calculating the lengths of two opposite sides of the projection object based on the coordinates.

  According to this, the image processing system or the like can improve the degree of freedom of installation of the imaging means by executing image processing in a three-dimensional space with the imaging means as a reference.

  Further, in the image processing system, the projector, the program, and the information storage medium, the projection target area information generation unit searches the image of the imaging information so as to search each pixel radially from the vicinity of the center of the imaging area. The projection target area information may be generated by searching for a signal value, grasping the projection target area based on a difference between the image signal values being searched.

  Further, in the image processing method, when generating the projection target area information, the image signal value of the imaging information is searched so as to search each pixel radially from the vicinity of the center of the imaging area. The projection target area information may be generated by grasping the projection target area based on the difference between the image signal values.

  According to this, an image processing system or the like is less affected by noise and searches more accurately by searching for an image signal value of imaging information so that each pixel is searched radially from the vicinity of the center of the imaging area. The target area can be detected.

  Hereinafter, a case where the present invention is applied to a projector that performs image distortion correction will be described as an example with reference to the drawings. In addition, the embodiment shown below does not limit the content of the invention described in the claim at all. In addition, all of the configurations shown in the following embodiments are not necessarily essential as means for solving the problems described in the claims.

(First embodiment)
FIG. 1 is a schematic diagram showing a state during image projection.

  A projector 20 that is a kind of image processing system projects an image onto a screen 10 that is a kind of projection object. In the present embodiment, the projector 20 is not facing the screen 10. Further, the projected image 12 formed by the projection light from the projector 20 protrudes from the screen 10. In this embodiment, there is a wall behind the screen 10, and the projection image 12 is displayed by reflecting the projection light by the wall.

  In the present embodiment, the sensor 60 that is a part of the imaging unit captures an area including the projected image 12.

  Further, in the present embodiment, the projector 20 calculates the aspect ratio (aspect ratio) of the screen 10 based on the imaging information by the sensor 60. Based on the aspect ratio and the known aspect ratio of the projected image, the projector 20 corrects the distortion of the image so that the projected image 12 is within the screen 10 and has the aspect ratio of the projected image. .

  Next, functional blocks of the projector 20 for implementing such functions will be described.

  FIG. 2 is a functional block diagram of the projector 20 according to an example of the present embodiment.

  The projector 20 includes a signal input unit 110 that inputs an image signal, a distortion correction unit 130 that corrects the input image signal so that image distortion is corrected, and a signal output unit that outputs the corrected image signal. 160, an image projecting unit 190 which is a kind of image projecting means for projecting an image based on an image signal, and a calibration image information generating unit 170 for generating calibration image information.

  In addition, the projector 20 detects the area of the screen 10 in the imaging area of the sensor 60 based on the imaging unit 180 that captures the area 8 including the projection image 12 through the imaging surface and generates imaging information. A projection target area information generation unit 140 that is a kind of projection target area information generation means, a projection area information generation unit 150 that detects the area of the projection image 12 in the imaging area of the sensor 60, and a correction that generates distortion correction information. And a use information generation unit 120. Note that the imaging unit 180 includes a sensor 60.

  The image projection unit 190 includes a spatial light modulator 192, a drive unit 194 that drives the spatial light modulator 192, a light source 196, and a lens 198.

  The drive unit 194 drives the spatial light modulator 192 based on the image signal from the signal output unit 160. Then, the image projection unit 190 projects the light from the light source 196 via the spatial light modulator 192 and the lens 198.

  Further, as hardware for mounting each unit of the projector 20 described above, for example, the following can be applied.

  FIG. 3 is a hardware block diagram of the projector 20 according to an example of the present embodiment.

  For example, the signal input unit 110 may be an A / D converter 930, the distortion correction unit 130 may be an image processing circuit 970, a RAM 950, a CPU 910, or the like, and the signal output unit 160 may be a D / A converter 940, for example. As the correction information generation unit 120, the projection target region information generation unit 140, the projection region information generation unit 150, and the calibration image information generation unit 170, for example, an image processing circuit 970, a RAM 950, etc., and as the imaging unit 180, for example, a CCD sensor As the spatial light modulator 192 such as a CMOS sensor or an RGB sensor, for example, a liquid crystal panel 920, a ROM 960 that stores a liquid crystal light valve driving driver for driving the liquid crystal panel 920, or the like can be used.

  Note that these units can exchange information with each other via the system bus 980.

  In addition, some or all of these units may be implemented as hardware such as a circuit, or may be implemented as software such as a driver.

  Furthermore, the program may be read from an information storage medium 900 that stores a program for causing the computer to function as the projection target area information generation unit 140 or the like, and the functions of the projection target area information generation unit 140 or the like may be implemented in the computer.

  As such an information storage medium 900, for example, a CD-ROM, a DVD-ROM, a ROM, a RAM, an HDD, or the like can be applied, and the program reading method may be a contact method or a non-contact method. Good.

  Further, in place of the information storage medium 900, the functions described above can be implemented by downloading a program or the like for implementing the functions described above from a host device or the like via a transmission path.

(Description of the flow of image processing)
Next, the flow of image processing using these units will be described.

  FIG. 4 is a flowchart showing a flow of image processing according to an example of the present embodiment.

  First, the calibration image information generation unit 170 generates image information for an all-black single color calibration image, and the signal output unit 160 outputs a digital signal of the image information to the image projection unit 190. Output to.

  The image projection unit 190 projects an all-black calibration image onto the screen 10 based on the digital signal (step S1). As a result, an all-black calibration image is displayed on the screen 10.

  The imaging unit 180 captures an area including the projected image 12 via the imaging surface to generate first imaging information (step S2). Here, the imaging information is information indicating, for each pixel of the sensor 60, an image signal value that can generate a luminance value such as a luminance value or an XYZ value. Here, the XYZ value is an international standard defined by the International Lighting Commission (CIE), and is a kind of image signal value of a device independent color.

  FIG. 5 is a schematic diagram illustrating the imaging region 17 according to an example of the present embodiment.

  The imaging area 17 is an area corresponding to an imaging surface where the sensor 60 captures an image, and is also an area schematically showing imaging information of the sensor 60 in a rectangle. Specifically, the imaging area 17 corresponds to an area on the imaging surface of a CCD sensor, for example.

  In this embodiment, the optical axis of the sensor 60 and the optical axis of the lens 198 are oriented in the same direction, and the screen 10 and the projector 20 are not facing each other. For this reason, as shown in FIG. 5, when viewed from the sensor 60, the projection area 13 that is the area of the projected image 12 looks rectangular like the imaging area 17, and the projection target area 11 that is the area of the screen 10 is distorted. It looks in.

  Further, in the present embodiment, the projection target area information generation unit 140 is outside from the intersection P0 of the diagonal line of the projection area 13, that is, the center P0 of the projection area 13 (not necessarily the center but near the center). Search for luminance values radially toward. Then, the projection target area information generation unit 140 detects a line segment P0P1 in which substantially the same luminance value continues from the center P0 as a part of the projection target area 11. Then, the projection target area information generation unit 140 performs the same processing by changing the angle by the angle θ, for example, and detects the line segment P0P2 as a part of the projection target area 11. Let L be the distance of this line segment.

  FIG. 6 is a schematic diagram showing the relationship between the angle θ and the distance L.

  For example, as the angle θ (where θ is 0 to 2π) changes, the distance L changes as shown in FIG. 6 to form four vertices at each of the angles θ1, θ2, θ3, and θ4.

  The projection target area information generation unit 140 calculates the projection target area from the sensor coordinates of the center P0 of the imaging area 17 (coordinates in the imaging area 17) and the distances L1, L2, L3, and L4 in the case of angles θ1, θ2, θ3, and θ4. The sensor coordinates of each of the four four corner ABCDs are calculated.

  The projection target area information generation unit 140 generates projection target area information indicating the sensor coordinates of the four corners (step S3), and sends the projection target area information to the correction information generation unit 120.

  In addition, the calibration image information generation unit 170 is a white image obtained by reducing the original image at an arbitrary magnification such as a quarter (for example, not necessarily white, but a single color such as gray near white). Calibration information for a multi-color calibration image of a black image (not necessarily black, gray close to black, etc.) may be generated. The image projection unit 190 projects a multi-color calibration image (step S4).

  The imaging unit 180 captures the region 8 including the projection image 12 of the multi-color calibration image projected on the screen 10, and generates second imaging information (step S5).

  In the case of the second imaging information, as shown in FIG. 5, a white area 15 corresponding to a white image is formed in the projection target area 11.

  The projection area information generation unit 150 detects the white area 15 having a constant luminance ratio by comparing the luminance value of each pixel in the white area 15 with the luminance value of each pixel in the projection target area 11 (step S6). ).

  Then, the projection area information generation unit 150 obtains the sensor coordinates of the projection area 13 by multiplying the sensor coordinates of the four corners of the white area 15 by n, which is the reverse magnification when the reduction is performed.

  The projection area information generation unit 150 generates projection area information indicating the sensor coordinates of the projection area 13 (step S7), and sends the projection area information to the correction information generation unit 120.

  The correction information generation unit 120 derives the relative distance between the screen 10 and the sensor 60 on the optical axis of the sensor 60 (step S8).

  More specifically, for example, the sensor coordinates of the projection target area 11 are A (Ax, Ay), B (Bx, By), C (Cx, Cy), D (Dx, Dy), and the imaging area 17 of the sensor 60. In the XYZ three-dimensional space with the origin in the XYZ three-dimensional space obtained by adding the optical axis of the sensor 60 as the third dimension Z to the two-dimensional plane XY, the coordinates of the four corners of the screen 10 are represented by A ′ (s * Ax, s * Ay, s * Pz), B '(t * Bx, t * By, t * Pz), C' (u * Cx, u * Cy, u * Pz), D '(v * Dx, v * Dy, v * Pz). Here, Pz is the distance on the Z-axis from the origin of the sensor coordinate system to the virtual imaging plane assumed as the imaging area 17. In addition, the projection target area information generation unit 140 calculates Pz = (VR / 2) / tan (θc / 2) based on the relationship between the horizontal angle of view θc of the sensor 60 and the horizontal resolution VR, thereby obtaining Pz. be able to.

  FIG. 7 is a schematic diagram of an XYZ three-dimensional space according to an example of the present embodiment.

  Since it is known that the screen 10 is rectangular (may be square), the condition of line segment A′B ′ = line segment D′ C ′ is satisfied. Assuming that this condition is satisfied, the above t, u, and v are as follows.

t = s * ((Dy-Cy) * (Ax-Dx)-(Dx-Cx) * (Ay-Dy)) / ((Dy-Cy) * (Bx-Dx)-(Dx-Cx) * ( By-Dy))
u = s * ((By-Dy) * (Ax-Dx)-(Bx-Dx) * (Ay-Dy)) / ((By-Dy) * (Dx-Cx)-(Bx-Dx) * ( Dy-Cy))
v = s-t + u
Here, assuming that s is, for example, 1 (other than 1), a relative distance on the optical axis (Z-axis direction component) from the origin to the four corners ABCD of the screen 10 can be obtained.

  Then, the correction information generation unit 120 calculates the aspect ratio of the screen 10 by comparing | A′B ′ | and | B′C ′ | indicating the lengths of the two opposite sides of the screen 10 ( Step S9).

  Further, the correction information generation unit 120 derives a corrected projection area (step S10).

  FIG. 8 is a schematic diagram of the projection region 19 after correction according to an example of the present embodiment.

  For example, when the aspect ratio of the projected image 12 is 4: 3 and the aspect ratio of the screen 10 is 4: 2, in the imaging area 17, the inside of the projection target area 11 is one sixth from the left and right ends. The area becomes the corrected projection area 19. The aspect ratio of the projected image 12 is generally known as 4: 3, 16: 9.

  Then, the correction information generation unit 120 converts the corrected coordinates of the projection area 19 in the imaging area 17 into image processing coordinates of the spatial light modulator 192 (step S11).

  Furthermore, the correction information generation unit 120 calculates, for example, a difference value between the image processing coordinates of the spatial light modulator 192 after distortion correction and the image processing amount coordinates of the spatial light modulator 192 before distortion correction. Then, the difference value is generated as image distortion correction information.

  Then, the distortion correction unit 130 corrects the image signal so as to correct image distortion based on the image distortion correction information.

  Thereby, the image projection unit 190 can project an image in which the distortion of the image is corrected.

  As described above, according to the present embodiment, the outer shape of the projection target (for example, the screen 10) can be grasped based on the difference in the image signal value (for example, the luminance value) based on the imaging information.

  Further, according to the present embodiment, even when the calibration image 12 is projected larger than the screen 10, the projector 20 can maintain the shape of the original image by using the multi-color calibration image. A reduced monochromatic area (for example, white, gray close to white) can be formed on the screen 10.

  And the projector 20 can grasp | ascertain the distortion of an image by imaging the monochrome area | region formed on the projection target object.

  Furthermore, the image processing system or the like can determine the projection position of the image after distortion correction based on the aspect ratio of the projection target grasped by imaging and the aspect ratio of the calibration image. Also, the image processing system or the like provides image distortion correction information indicating how much the image should be adjusted based on the difference between the projection position of the image after distortion correction and the projection position of the image before distortion correction. And distortion of the image can be corrected appropriately.

  In addition, according to the present embodiment, the projector 20 does not match the shape of the screen 10, but generates image distortion correction information so that the image is projected with the original aspect ratio of the image, whereby the original aspect ratio is obtained. An image can be projected at a ratio.

  Therefore, according to the present embodiment, the projector 20 can automatically and accurately correct the distortion of the projected image in the vertical and horizontal directions that occurs when the projector 20 is used. In particular, the projector 20 can perform correction using the frame of the screen 10 without being restricted by the distortion detection angle in the vertical and horizontal directions of the image.

  Further, since the linear component of the projection area after correction is parallel to the frame of the screen 10, the projector 20 is visually easy to see even when the projector 20 is installed inclined with respect to a plane. Images can be displayed on the screen 10.

  Further, when the projector 20 detects the projection target area 11, the projector 20 compares the luminance value from the center to the outside, thereby being less affected by an obstacle on the screen 10 and more accurately detecting the projection target area 11. can do.

(Modification of the first embodiment)
In addition, application of this invention is not limited to the Example mentioned above.

  For example, in the above-described embodiment, the projector 20 handles the coordinates of the imaging region 17 of the sensor 60 in two dimensions and performs the processing assuming that the variable s is 1. However, the projector 60 coordinates the coordinates of the sensor 60 in three dimensions. Handling, the variable s may be set to a value suitable for the actual sensor 60.

  FIG. 9 is a schematic diagram of an imaging plane of the sensor 60 according to an example of the present embodiment.

  For example, when the horizontal resolution of the sensor 60 is W and the horizontal angle of view of the sensor 60 is θh, the above-described variable Pz = (W / 2) / tan (θh / 2).

  Then, the correction information generation unit 120 converts the three-dimensional coordinates A′B′C′D ′ of the screen 10 from the sensor three-dimensional coordinate system (xc, yc, zc) to the projector three-dimensional coordinate system (xp, yp, zp). ). More specifically, the correction information generation unit 120 may convert (xc, yc, zc, 1) to (xp, yp, zp, 1) using, for example, a 4 × 4 matrix. . The parameters in this matrix are determined by the positional relationship and angular relationship between the sensor 60 and the projector 20.

  Then, the correction information generation unit 120 uses three points among a (ax, ay, az) to d (dx, dy, dz) that are the coordinates of the four corners of the screen 10 of the projector three-dimensional coordinate system. Ten normal vectors N (Nx, Ny, Nz) are calculated.

  For example, when calculation is performed using three points a, c, and d, Nx = (dy−cy) * (az−dz) − (dz−cz) * (ay−dy), Ny = (dz−cz) ) * (Ax-dx)-(dx-cx) * (az-dz), Zn = (dx-cx) * (ay-dy)-(dy-cy) * (ax-dx).

  Since the angle (horizontal θx, vertical θy) formed by the normal vector N and the vector (0, 0, 1) corresponds to the angle formed by the optical axis of the projection light of the screen 10 and the projector 20, the correction information generation unit 120 retrieves the coordinates of the four corners of the projection area after correction from the two-dimensional lookup table using θx and θy.

  FIG. 10 is a schematic diagram illustrating the coordinates of the four corners of the corrected projection area according to an example of the present embodiment. FIG. 11 illustrates the coordinates of the four corners of the corrected projection area according to the example of the present embodiment. It is a two-dimensional lookup table shown.

  As shown in FIG. 11, in the two-dimensional lookup table, the values of θx and θy are associated with the xy coordinates of the four corner coordinates EFGH of the projection area after correction.

  The correction information generation unit 120 generates image distortion correction information using the xy coordinates of the coordinates EFGH using the two-dimensional lookup table illustrated in FIG. 11, and supplies the image distortion correction information to the distortion correction unit 130. send.

  Also by such a method, the projector 20 can perform image distortion correction based on the coordinates of the four corners of the screen 10.

(Second embodiment)
Next, the projector 20 for projecting an image with a predetermined aspect ratio will be described.

  For example, as shown in FIG. 1, even when a part of the projected image 12 is projected outside the screen 10, the projected image 12 is divided into nine equal parts, and the central area that is the central area is used as a reference. By doing so, the projector 20 can capture the central area of the projected image in the screen 10.

  For example, the projector 20 compares each pixel of the first calibration image that is all black with each pixel of the second calibration image that is white in the central area and black in the eight peripheral areas around it. Thus, the position of the sensor 60 in the central region in the imaging region 17 can be grasped.

  FIG. 12 is a schematic diagram illustrating the imaging region 17 in an example of the present embodiment. FIG. 13 is a schematic diagram when the screen 10 according to an example of the present embodiment is perspective-projected with the sensor 60 as an origin.

  In this case, the sensor 60 can capture the central region 15 in the projection target region 11 in the imaging region 17.

  For example, a sensor coordinate system with the center of the imaging region 17 of the sensor 60 as the origin is assumed. In the sensor coordinate system, it is assumed that the horizontal direction of the imaging region 17 is the x axis, the vertical direction of the imaging region 17 is the y axis, and the normal direction of the imaging region 17 is the z axis.

  The projection target area information generation unit 140 generates projection target area information indicating the coordinates of the projection target area 11 in the sensor coordinate system. In addition, the projection area information generation unit 150 generates projection area information indicating the coordinates of the projection area in the sensor coordinate system.

  Assume that the coordinates of the four corners of the projection target region 11 in the imaging region 17 are E (Ex, Ey), F (Fx, Fy), G (Gx, Gy), and H (Hx, Hy), respectively. These two-dimensional coordinates are obtained by perspectively projecting the four vertices E′F′G′H ′ of the screen 10 actually existing in the three-dimensional space onto the virtual imaging plane z = Pz in the normal direction of the origin O of the sensor 60. Equivalent to. Here, the origin O is the center point of the imaging region 17.

  Therefore, the sensor three-dimensional coordinates of the four vertices of the screen 10 projected on the virtual imaging surface are E (Ex, Ey, Pz), F (Fx, Fy, Pz), G (Gx, Gy) having the same z-axis component. , Pz) and H (Hx, Hy, Pz). Here, the projection target area information generation unit 140 calculates Pz = (VR / 2) / tan (θc / 2) by calculating Pz = (VR / 2) / tan (θc / 2) from the relationship between the horizontal angle of view θc of the sensor 60 and the horizontal resolution VR. Can do.

  In addition, since the four vertices of the actual screen 10 exist at the destination connecting the origin of the sensor 60 and the vertex EFGH, s * E (Ex, Ey, Pz), t * F (Fx, Fy, Pz), u * It can be expressed as G (Gx, Gy, Pz), v * H (Hx, Hy, Pz). Note that s, t, u, and v are real numbers.

  Here, since the actual screen 10 is rectangular, the sides E'F 'and G'H', the sides E'H 'and F'G' are parallel, and the lengths are equal. The condition is established. The projection target area information generation unit 140 can obtain the coordinates of the four vertices of the screen 10 in the three-dimensional coordinate system of the sensor 60 by solving for s, t, u, and v based on this condition. The generation unit 120 can obtain the aspect ratio of the screen 10 based on the coordinates.

  First, from the above assumption, E'F '= (t * Fx-s * Ex, t * Fy-s * Ey, t * Pz-s * Pz) and H'G' = (u * Gx-v * Hx, u * Gy-v * Hy, u * Pz-v * Pz).

  And t * Fx-s * Ex = u * Gx-v * Hx, t * Fy-s * Ey, u * Gy-v * Hy, t * Pz-s * Pz = u * Pz-v * Pz is there. From now on t * Fx-u * Gx + v * Hx = s * Ex (1), t * Fy-u * Gy + v * Hy = s * Ey (2), t * Pz-u * Pz + v * Pz = s * Pz (3) is established.

  Further, by dividing the equation (3) by Pz, t−u + v = s (4).

  (Fx−Hx) * t + (Hx−Gx) * u = (Ex−Hx) * s (5) by subtracting the value obtained by multiplying the equation (4) by Hx from the equation (1). .

  Further, by subtracting the value obtained by multiplying the expression (2) by the expression (4) by Hy, (Fy−Hy) * t + (Hy−Gy) * u = (Ey−Hy) * s (6) It is.

  In Equations (5) and (6), a = (Fx-Hx), b = (Hx-Gx), c = (Ex-Hx), d = (Fy-Hy), e = (Hy-Gy) , F = (Ey-Hy). In this case, Expression (5) can be expressed as a * t + b * u = c * s, and Expression (6) can be expressed as d * t + e * u = f * s.

  By solving this equation, t = s * (e * cb * f) / (e * ab * d) and u = s * (d * ca * f) / (d * ba * e) . That is, t and u can be expressed by s. Here, the position of the screen 10 when the variable s indicating the depth information is changed will be described.

  FIG. 14 is a schematic diagram when s is changed in an example of the present embodiment.

  As shown in FIG. 14, even when s is changed, the shape of the screen 10 is similar and the aspect ratio remains unchanged. Therefore, the aspect ratio can be calculated by assuming that s = 1. In this case, s = 1, t = (e * cb * f) / (e * ab * d), u = (d * ca * f) / (d * ba * e), v = 1-t * u Can be expressed as

  In this way, the correction information generation unit 120 calculates the aspect ratio using the three-dimensional coordinates of the four vertices of the actual screen 10 based on the projection target region information from the projection target region information generation unit 140. Here, the aspect ratio is assumed to be the length of the vertical side of the target area / the length of the horizontal side of the target area. For example, when the horizontal length: the vertical length = 4: 3, the aspect ratio is 3/4 = 0.75. Here, it is assumed that correction is performed so that the aspect ratio of the projected image 12 is 0.75.

  FIG. 15 is a diagram illustrating a relationship between the screen 10 according to an example of the present embodiment and the projection region after correction, and FIG. 15A is a diagram illustrating the screen 10 when the horizontal is longer than the vertical, FIG. 15B is a diagram showing a corrected projection area on the screen 10 of FIG. FIG. 16 is a diagram illustrating a relationship between the screen 10 according to an example of the present embodiment and the projection region after correction, and FIG. 16A is a diagram illustrating the screen 10 when the horizontal is shorter than the vertical. FIG. 16B is a diagram showing a corrected projection area on the screen 10 of FIG.

  When the aspect ratio of the screen 10 is substantially equal to 0.75 (it is acceptable if it does not match 0.75, the correction information generation unit 120 matches the shape of the projected image 12 with the shape of the screen 10). The correction information is generated as follows. In order to simplify the description, in this embodiment, it is assumed that the correction information generation unit 120 generates correction information so that the shape of the projected image 12 matches the shape of the screen 10 when the aspect ratio is 0.75. To do.

  Further, as shown in FIG. 15A, when the aspect ratio is <0.75, that is, when the screen 10 is horizontally long, the correction information generation unit 120 performs the right and left of the screen 10 as shown in FIG. The correction information is generated so that the corrected projection area A′B′C′D ′ is formed in the area having the margin.

  When the aspect ratio Sa <0.75, A ′ is a point that divides E′H ′ into m: n, and therefore A ′ = (n * E ′ + m * H ′) / (m + n). Here, it is assumed that m + n = 1.

  Further, since the horizontal of the screen 10: vertical = sa: 1, in order to maintain the corrected projection area aspect ratio at 0.75, m = (1−4 * Sa / 3) / 2, n = 1 -(1-4 * Sa / 3) / 2 is required. The projection area information generation unit 150 similarly calculates B′C′D ′ other than A ′.

  Also, as shown in FIG. 16A, when the aspect ratio is> 0.75, that is, when the screen 10 is vertically long, the correction information generating unit 120 moves the top and bottom of the screen 10 as shown in FIG. The correction information is generated so that the corrected projection area A′B′C′D ′ is formed in the area having the margin.

  When the aspect ratio Sa> 0.75, A ′ is a point that divides E′F ′ into m: n, and therefore A ′ = (n * E ′ + m * F ′) / (m + n). Here, it is assumed that m + n = 1.

  Since the aspect ratio of the projection area after correction is 0.75, m = (Sa−3 / 4) / 2, n = Sa− ( Sa-3 / 4) / 2 is required. The projection area information generation unit 150 similarly calculates B′C′D ′ other than A ′.

  In this manner, the projection area information generation unit 150 can generate projection area information indicating the corrected projection area coordinates A′B′C′D ′.

  Further, the correction information generation unit 120 may generate the correction information based on the projection area information from the projection area information generation unit 150 so as to slightly enlarge the corrected projection area. Thereby, it is possible to obtain a so-called borderless image and improve the appearance of the image. For example, assuming that the enlargement ratio is Ps and the intersection of the diagonal lines of the corrected projection area is P, the correction information generating unit 120 can obtain the vertex A ′ = Ps * PA ′ + OP of the enlarged projection area. . The correction information generation unit 120 calculates B′C′D ′ other than A ′ in the same manner.

  Next, A′B′C′D ′ in the sensor coordinate system obtained in this way is projected on the screen 10 in a perspective manner. This conversion process is executed by aligning z as depth information with Pz.

  For example, for the vertex A ′, the correction information generation unit 120 performs A′x = A′x / A′z * Pz, A′y = A′y / A′z * Pz, A′z = A ′. z / A'z * Pz. The correction information generation unit 120 calculates B′C′D ′ other than A ′ in the same manner.

  FIG. 17 is a schematic diagram illustrating a corrected projection area 19 in the imaging area 17 according to an example of the present embodiment. FIG. 18 is a schematic diagram when a projection region after correction according to an example of the present embodiment is perspective-projected with the sensor 60 as an origin.

  Finally, the correction information generation unit 120 converts the coordinates of the corrected projection area on the screen 10 into coordinates on the liquid crystal panel 920.

  First, the coordinates A (Ax, Ay), B (Bx, By), C (Cx, Cy), and D (Dx, Dy) of the four vertices of the central region in the known sensor coordinate system, and the center in the liquid crystal panel 920 A coordinate conversion parameter is generated based on the correspondence relationship with the coordinates I (Ix, Iy), J (Jx, Jy), K (Kx, Ky), and L (Lx, Ly) of the four vertices of the region.

  FIG. 19 is a diagram illustrating an arithmetic expression for obtaining a coordinate conversion parameter according to an example of the present embodiment.

  The correction information generation unit 120 converts the coordinates of the corrected projection area A′B′C′D ′ into coordinates on the liquid crystal panel 920 using the coordinate conversion parameters Pa to Ph shown in FIG.

  For example, in the case of the vertex A ′, A′x = (Pa * A′x + Pb * A′y + Pc) / (Pg * A′x + Ph * A′y + 1) and A′y = (Pd * A′x + Pe * A′y + Pf) / (Pg * A′x + Ph * A′y + 1). The same applies to the vertex B'C'D '.

  The correction information generation unit 120 generates correction information based on the difference between the original coordinates on the liquid crystal panel 920 and the corrected coordinates on the liquid crystal panel 920. Further, the distortion correction unit 130 corrects the image signal based on the correction information. In addition, the image projection unit 190 projects an image based on the corrected image signal.

  Thereby, the projector 20 can project an image without distortion in the center of the screen 10 by using the screen 10 as effectively as possible.

  As described above, according to the present embodiment, the projector 20 can project an image by effectively using the area of the screen 10 with a desired aspect ratio. Thereby, the projector 20 can project an image that is easy to see for the observer without being affected by the shape of the screen 10.

  Further, the projector 20 can appropriately correct the image without being affected by the difference in position between the projector 20 and the sensor 60 by performing calculation using the sensor coordinate system. Thereby, as the sensor 60, for example, various imaging means such as a sensor built in the projector 20, a sensor at a position away from the projector 20, and a digital camera can be employed.

(Modification)
The present invention is not limited to the above-described embodiments, and various modifications can be made.

  For example, the present invention is particularly effective when the projected image 12 includes the screen 10, but the projector 20 automatically determines the inclusion relationship between the projected image 12 and the screen 10 and responds to the determination result. The image correction method to be applied may be determined.

  Further, for example, when the projector 20 has a zoom function, the projector 20 may forcibly adjust the zoom so that the projected image 12 includes the screen 10 and project the image.

  In the above-described embodiment, the projection image 12 is divided into nine equal parts, and the center area, which is the center area, is used as a reference. However, the projector 20 can capture the center area of the projection image in the screen 10. If so, the size and position of the central region may be arbitrary.

  In the above-described embodiments, the screen 10 is used as the projection target. However, the projection target may be, for example, a whiteboard or a wall as long as the projection has a rectangular shape capable of detecting four corners. .

  In the above-described embodiment, the projector 20 is used as the image processing system. However, the present invention is not limited to the projector 20 and displays such as CRT (Cathode Ray Tube), LED (Light Emitting Diode), and EL (Electro Luminescence). It is also effective for an image processing system.

  Further, as the projector 20, for example, a liquid crystal projector, a projector using a DMD (Digital Micromirror Device), or the like may be used. DMD is a trademark of Texas Instruments Incorporated.

  Further, the functions of the projector 20 described above may be implemented, for example, by a single projector, or may be implemented by being distributed by a plurality of processing devices (for example, distributed processing by a projector and a PC).

It is a schematic diagram which shows the state at the time of image projection. It is a functional block diagram of the projector which concerns on an example of this embodiment. It is a hardware block diagram of the projector which concerns on an example of this embodiment. It is a flowchart which shows the flow of the image processing which concerns on an example of this embodiment. It is a schematic diagram which shows the imaging area which concerns on an example of this embodiment. It is a schematic diagram which shows the relationship between angle (theta) and distance L. FIG. It is a schematic diagram of XYZ three-dimensional space which concerns on an example of this embodiment. It is a schematic diagram of the projection area | region after the correction | amendment which concerns on an example of this embodiment. It is a schematic diagram of the imaging plane of the sensor which concerns on an example of this embodiment. It is a schematic diagram which shows the coordinate of the four corners of the projection area after correction | amendment which concerns on an example of this embodiment. 6 is a two-dimensional lookup table showing coordinates of four corners of a corrected projection area according to an example of the present embodiment. It is a schematic diagram which shows the imaging area in an example of this embodiment. It is a schematic diagram at the time of carrying out the perspective projection of the screen which concerns on an example of this embodiment by making a sensor into an origin. It is a schematic diagram at the time of changing s in an example of this embodiment. FIG. 15A is a diagram illustrating a relationship between a screen according to an example of the present embodiment and a projection region after correction, and FIG. 15A is a diagram illustrating the screen when the horizontal is longer than the vertical, and FIG. FIG. 16 is a diagram showing a corrected projection area on the screen of FIG. It is a figure which shows the relationship between the screen which concerns on an example of this embodiment, and the projection area | region after correction | amendment, FIG. 16 (A) is a figure which shows a screen when horizontal is shorter than vertical, FIG.16 (B) is FIG. FIG. 17 is a diagram showing a corrected projection area on the screen of FIG. It is a schematic diagram which shows the projection area | region after correction | amendment in the imaging area in an example of this embodiment. It is a schematic diagram at the time of carrying out the perspective projection of the corrected projection area which concerns on an example of this embodiment by making a sensor into an origin. It is a figure which shows the computing equation which calculates | requires the parameter for coordinate conversion based on an example of this embodiment.

Explanation of symbols

  10 screen (projection object), 12 projection image, 20 projector (image processing system), 60 sensor, 120 correction information generation unit, 130 distortion correction unit, 140 projection target region information generation unit, 150 projection region information generation unit, 170 calibration image information generation unit, 180 imaging unit, 190 image projection unit, 900 information storage medium

Claims (15)

A single color calibration image composed of any single color is projected onto the projection target, includes a first monochrome region smaller than the entire image region, and a region other than the first region is other than the first monochrome color. Image projection means for projecting a multi-color calibration image composed of the second single color onto the projection object;
The first imaging information is generated by capturing the single color calibration image projected on the projection object, and the second imaging information is generated by capturing the multi-color calibration image projected on the projection object. Imaging means for generating
Projection target area information generation means for generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging means based on the difference in image signal value for each pixel based on the first imaging information. When,
Based on the difference between the first imaging information and the second imaging information, the first monochrome area in the imaging area is detected as a partial projection area, and based on the partial projection area, in the imaging area Projection area information generating means for generating projection area information indicating the coordinates of the four corners of the projection area;
Correction information generating means for generating image distortion correction information based on the projection target area information and the projection area information;
Distortion correction means for correcting the image signal based on the image distortion correction information so that the distortion of the image is corrected;
Including
The image processing system, wherein the image projection unit projects an image based on the image signal corrected by the distortion correction unit. In claim 1,
The first monochromatic region is a region obtained by reducing the entire image region to 1 / n,
The projection area information generating means detects an area obtained by multiplying the partial projection area by n as the projection area. In any one of Claims 1, 2.
The correction information generating means includes
Based on the projection target area information, on the optical axis of the origin in a three-dimensional space obtained by adding the optical axis of the imaging means as the third dimension to the two-dimensional plane of the imaging area, and the four corners of the projection target Calculate the relative distance,
Based on the projection target area information and the relative distance, the coordinates in the three-dimensional space of the four corners of the projection target are calculated,
Based on the coordinates, the first aspect ratio indicating the aspect ratio of the projection object is calculated by calculating the lengths of two opposite sides of the projection object,
Based on the first aspect ratio, the second aspect ratio indicating the aspect ratio of the projection panel included in the single color calibration image, the multiple color calibration image, or the image projection unit, and the projection target area information. Calculating the coordinates in the imaging region at the four corners of the image when the image projection means projects onto the projection object in the second aspect ratio without distortion,
An image processing system, wherein the image distortion correction information is generated based on coordinates in the imaging area at the four corners of the image and the projection area information. In claim 1,
The correction information generating means grasps a first aspect ratio indicating an aspect ratio of the projection target based on the projection target area information, and indicates a first aspect ratio indicating a desired aspect ratio according to the first aspect ratio. Generating projection coordinate system projection area information indicating the coordinates of the four corners of the projection area in the projection panel of the image projection means based on the projection area information so that an image is displayed with an aspect ratio of 2. An image processing system that generates the image distortion correction information based on the projection coordinate system projection area information. Oite to claim 4,
The correction information generating means includes
Based on the projection target area information, on the optical axis of the origin in a three-dimensional space obtained by adding the optical axis of the imaging means as the third dimension to the two-dimensional plane of the imaging area, and the four corners of the projection target Calculate the relative distance,
Based on the projection target area information and the relative distance, the coordinates in the three-dimensional space of the four corners of the projection target are calculated,
An image processing system characterized in that the first aspect ratio is calculated by calculating the lengths of two opposite sides of the projection object based on the coordinates. In any one of Claims 1-5 ,
The projection target area information generation unit searches for an image signal value of the imaging information so as to search each pixel radially from the vicinity of the center of the imaging area, and based on the difference in the image signal value being searched An image processing system characterized by grasping a projection target area and generating the projection target area information. A single color calibration image composed of any single color is projected onto the projection target, includes a first monochrome region smaller than the entire image region, and a region other than the first region is other than the first monochrome color. Image projection means for projecting a multi-color calibration image composed of the second single color onto the projection object;
The first imaging information is generated by capturing the single color calibration image projected on the projection object, and the second imaging information is generated by capturing the multi-color calibration image projected on the projection object. Imaging means for generating
Projection target area information generation means for generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging means based on the difference in image signal value for each pixel based on the first imaging information. When,
Based on the difference between the first imaging information and the second imaging information, the first monochrome area in the imaging area is detected as a partial projection area, and based on the partial projection area, in the imaging area Projection area information generating means for generating projection area information indicating the coordinates of the four corners of the projection area;
Correction information generating means for generating image distortion correction information based on the projection target area information and the projection area information;
Distortion correction means for correcting the image signal based on the image distortion correction information so that the distortion of the image is corrected;
Including
The projector according to claim 1, wherein the image projecting unit projects an image based on the image signal corrected by the distortion correcting unit. A computer-readable program,
Computer
A single color calibration image composed of any single color is projected onto the projection target, includes a first monochrome region smaller than the entire image region, and a region other than the first region is other than the first monochrome color. Image projection means for projecting a multi-color calibration image composed of the second single color onto the projection object;
The first imaging information is generated by capturing the single color calibration image projected on the projection object, and the second imaging information is generated by capturing the multi-color calibration image projected on the projection object. Imaging means for generating
Projection target area information generation means for generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging means based on the difference in image signal value for each pixel based on the first imaging information. When,
Based on the difference between the first imaging information and the second imaging information, the first monochrome area in the imaging area is detected as a partial projection area, and based on the partial projection area, in the imaging area Projection area information generating means for generating projection area information indicating the coordinates of the four corners of the projection area;
Correction information generating means for generating image distortion correction information based on the projection target area information and the projection area information;
Based on the image distortion correction information, function as a distortion correction unit that corrects the image signal so that the distortion of the image is corrected,
The image projecting means projects an image based on the image signal corrected by the distortion correcting means. An information storage medium storing a computer-readable program,
An information storage medium storing the program according to claim 8 . Project a single color calibration image composed of any single color onto the projection object using the image projection unit ,
Imaging the monochromatic calibration image projected on the projection object using an imaging unit to generate first imaging information;
A multi-color calibration image including a first single color area smaller than the entire image area and a region other than the first region composed of a second single color other than the first single color is displayed on the image projection unit. Use to project toward the projection object,
The multi-color calibration image projected on the projection target is imaged using the imaging unit to generate second imaging information,
Based on the difference in image signal value for each pixel based on the first imaging information, generating projection target area information indicating the coordinates of the four corners of the projection target in the imaging area of the imaging unit ,
Based on the difference between the first imaging information and the second imaging information, the first monochrome area in the imaging area is detected as a partial projection area, and based on the partial projection area, in the imaging area Generating projection area information indicating the coordinates of the four corners of the projection area;
Based on the projection area information and the projection area information, image distortion correction information is generated,
Based on the image distortion correction information, correct the image signal so that the image distortion is corrected,
An image processing method, wherein an image is projected using the image projection unit based on a corrected image signal. In claim 10 ,
The first monochromatic region is a region obtained by reducing the entire image region to 1 / n,
An image processing method, wherein an area obtained by multiplying the partial projection area by n is detected as the projection area. In any one of Claims 10 and 11 ,
Based on the projection target area information, on the optical axis of the origin in a three-dimensional space obtained by adding the optical axis of the imaging unit as a third dimension to the two-dimensional plane of the imaging area , and the four corners of the projection target Calculate the relative distance,
Based on the projection target area information and the relative distance, the coordinates in the three-dimensional space of the four corners of the projection target are calculated,
Based on the coordinates, the first aspect ratio indicating the aspect ratio of the projection object is calculated by calculating the lengths of two opposite sides of the projection object,
Based on the first aspect ratio, the second aspect ratio indicating the aspect ratio of the projection panel included in the single color calibration image, the multiple color calibration image, or the image projection unit , and the projection target area information. Calculating coordinates in the imaging region at the four corners of the image when projected onto the projection object in the second aspect ratio without distortion,
An image processing method, wherein the image distortion correction information is generated based on coordinates in the imaging area at the four corners of the image and the projection area information. In claim 10 ,
Based on the projection target area information, grasp a first aspect ratio indicating the aspect ratio of the projection target,
Projection indicating the coordinates of the four corners of the projection area on the projection panel based on the projection area information so that an image is displayed with a second aspect ratio indicating a desired aspect ratio according to the first aspect ratio. Generate coordinate system projection area information,
An image processing method comprising: generating the image distortion correction information based on the projection coordinate system projection area information. In claim 13 ,
Based on the projection target area information, on the optical axis of the origin in a three-dimensional space obtained by adding the optical axis of the imaging unit as a third dimension to the two-dimensional plane of the imaging area , and the four corners of the projection target Calculate the relative distance,
Based on the projection target area information and the relative distance, the coordinates in the three-dimensional space of the four corners of the projection target are calculated,
An image processing method, wherein the first aspect ratio is calculated by calculating the lengths of two opposite sides of the projection object based on the coordinates. In any one of Claims 10-14 ,
When generating the projection target area information, the image signal value of the imaging information is searched so as to search each pixel radially from the vicinity of the center of the imaging area, and based on the difference between the image signal values being searched. And generating the projection target area information by grasping the projection target area.

Images