JP2008224323A - Stereoscopic photograph measuring instrument, stereoscopic photograph measuring method, and stereoscopic photograph measuring program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stereoscopic photograph measuring instrument capable of enhancing dimension measuring precision of a measuring object. <P>SOLUTION: A three-dimensional coordinate calculating means 15 calculates three-dimensional coordinates of the measuring object, corresponding to the first feature part constituted of feature points of one side image data, using a restriction condition determined based on geometrical information, a projection transformation matrix, a relative positional relation and a principle of a triangulation method, and the second feature part constituted of feature points of the other side image data, corresponding to the first feature part, and a dimension calculation means 16 calculates a dimension of a dimension measuring portion, based on the three-dimensional coordinates. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a stereo photo measurement device, a stereo photo measurement method, and a stereo photo measurement program for calculating a dimension of a measurement target from two pieces of image data photographed by using a camera from two locations apart from each other.

  Stereoscopic shape restoration and dimension measurement using the principle of stereo stereoscopic vision have the problem of how to associate feature points such as points, lines, and regions in a stereo image pair. As a measuring device, for example, in the three-dimensional restoration device disclosed in Patent Document 1, a method is proposed in which, when restoring a three-dimensional geometric shape, a plane constraint condition is obtained from segment connection relations, and corresponding points are corrected so as to satisfy the plane constraint condition. Has been.

  That is, the three-dimensional restoration apparatus disclosed in Patent Document 1 corresponds to a means for detecting an edge in a stereo image, dividing the segment into segments by a local shape, and obtaining a segment feature amount, and the segment based on the feature amount. A means for obtaining a correspondence between stereo images by a stereo correspondence search process as a unit, a means for converting a coordinate system to make it easy to obtain a correspondence in a stereo image, and a corresponding point from a corresponding part based on the correspondence A means for calculating a plane constraint condition based on the connection relation of the segments based on the correspondence relationship, a means for correcting the corresponding point so as to satisfy the plane constraint condition in the stereo image, and a corresponding point in the stereo image The means to restore the 3D shape from the relationship between the two and the 3D shape - fitting is provided with a means for performing the curve.

JP 2003-248814 A (paragraph 0009)

  The conventional stereo photo measurement apparatus is configured as described above, and the plane constraint condition obtained from the connectivity between the feature points is not guaranteed to match the constraint condition based on the actual coordinate axes established at the shooting target site. For this reason, there is a problem that an error from the actual measurement value occurs during the dimension measurement, and the dimension measurement accuracy of the measurement target is lowered.

  The present invention has been made to solve the above-described problems, and realizes a coordinate restoration using a constraint condition based on an actual coordinate axis and can improve a dimensional measurement accuracy of a measurement object. An object is to obtain an apparatus, a stereo photo measurement method, and a stereo photo measurement program.

  The stereo photo measurement apparatus according to the present invention includes an image data storage unit that stores two image data obtained by photographing a marker indicating a reference position and a measurement target using a camera from two locations apart from each other, and dimensions of the measurement target. Design information storage means for storing design information including geometric information of one or both of the measurement location and the linear direction component of the measurement target ridge line and the normal direction component of the plane, and the marker from the image data Recognizing by image processing, a camera position recognition means for obtaining a relative positional relationship between the camera and the marker using a projection transformation matrix of the lens of the camera, a point, a line captured in the image of the image data, Feature point extraction means for extracting feature points of a plane and a solid shape by image processing or by input information from an input device, and the geometric information Using the constraint conditions to be determined, the projection transformation matrix, the relative positional relationship, and the principle of triangulation, the first feature composed of feature points of one image data and the other image corresponding to the first feature Three-dimensional coordinate calculation means for calculating the three-dimensional coordinates of the measurement object corresponding to the second feature portion constituted by the feature points of the data, and dimension calculation means for calculating the dimension of the dimension measurement location from the three-dimensional coordinates; It is equipped with.

  By this invention, the effect that the dimension measurement precision of a measuring object can be improved is acquired.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of a stereo photo measurement apparatus according to Embodiment 1 of the present invention. The stereo photo measurement processing means 10 included in the stereo photo measurement device includes two pieces of image data 21, camera internal parameters 22, and design information 23 of the shooting target site 7 taken using the digital camera 6 from two places apart from each other. Is read and the dimension of the dimension measurement part to be measured is calculated. A camera external calibration marker 8 indicating a reference position is installed at the photographing target site 7, and the digital camera 6 performs imaging so that the dimension measurement location to be measured and the marker 8 are reflected. In this example, the site 7 to be imaged is an elevator hoistway.

  Two pieces of image data 21, camera internal parameters 22, and design information 23 of the shooting target site 7 taken by the digital camera 6 are stored in the storage unit 2. Here, as the camera internal parameter 22, for example, the screen resolution of the digital camera 6, the focal length of the lens, the central coordinates of the radial lens distortion aberration, and the lens distortion coefficient are stored, and the design information 23 is the measurement target. Design that includes geometric information of one or both of the dimension measurement location, the feature point that is the point on the edge where the shading changes on the image, and the linear direction component and the normal direction component of the plane to be measured Information is stored.

  Stereo photo measurement processing means 10 includes lens distortion correction means 11, camera position recognition means 12, feature point recognition means 13, feature point input means 14, three-dimensional coordinate calculation means 15, dimension calculation means 16, and dimension display means 17. ing.

  The lens distortion correction unit 11 inputs two pieces of image data 21 stored in the storage unit 2, and uses the camera internal parameters 22 stored in the storage unit 2, and the lens distortion of the two pieces of image data 21. Correct. The camera position recognition unit 12 recognizes the marker 8 from the two pieces of image data 21 whose lens distortion has been corrected by the lens distortion correction unit 11 by image processing, and the relative positional relationship between the digital camera 6 and the marker 8 and the marker 8. Is obtained using the projection transformation matrix of the lens of the digital camera 6. Note that the projection conversion matrix is a projection conversion matrix in which the lens of the digital camera 6 projects the imaging target onto the image.

  A feature point extraction unit 100 (not shown) includes a feature point recognition unit 13 that extracts, by image processing, feature points of points, lines, planes, and three-dimensional shapes photographed in each image of the two pieces of image data 21. The feature point input means 14 etc. which are extracted by the input information are included. In the present embodiment, the feature point recognizing unit 13 captures feature points such as points, lines, planes, and three-dimensional shapes captured in the two image data 21 images whose lens distortion has been corrected by the lens distortion correcting unit 11. Recognize by processing. Subsequently, in the feature point input unit 14, other feature points that are not recognized by the feature point recognition unit 13 from the two pieces of image data 21 are input by input information from the input device.

  The three-dimensional coordinate calculation means 15 is inputted with design information 23, and is composed of feature points of one image data using the constraint conditions, projection transformation matrix, relative positional relationship, and triangulation principles determined from geometric information. The three-dimensional coordinates of the measurement target corresponding to the first feature portion and the second feature portion constituted by the feature points of the other image data corresponding to the first feature portion are calculated. Note that the feature part is composed of feature points, refers to a portion to be measured in one image, and specifically, as a constraint condition determined from geometric information, the constraint is that it is on a straight line of a linear component. It is a constraint condition that is on a plane having a condition or a normal direction.

  The dimension calculation means 16 receives the design information 23 and calculates the dimensions of the dimension measurement location from the three-dimensional coordinates calculated by the three-dimensional coordinate calculation means 15. The dimension display means 17 displays the dimensions of the dimension measurement part calculated by the dimension calculation means 16.

  FIG. 2 is a block diagram showing the hardware configuration of the stereo photo measurement apparatus according to Embodiment 1 of the present invention. The stereo photo measurement apparatus includes an arithmetic processing means 1, a storage means 2, a temporary storage means 3, an input means 4 and Display means 5 is provided. The storage unit 2 stores image data 21, camera internal parameters 22, design information 23, and a program 24, and the input unit 4 captures image data 21, camera internal parameters 22, design information 23, and information from the user.

  The arithmetic processing means 1 reads the program 24 stored in the storage means 2 and uses the image data 21, the camera internal parameters 22, and the design information 23 to temporarily store the data, calculation results, etc. in the temporary storage means 3. The functions of the stereo photo measurement processing means 10 shown in FIG. 1 are realized by performing processing while storing the image data and displaying the image data 21, design information 23, processing results, and the like on the display means 5.

  FIG. 3 is a flowchart showing a process for measuring the dimension of the dimension measurement location according to the first embodiment of the present invention. Among the processes shown in FIG. 3, the process from step ST3 to step ST12 is a process performed by the stereo photograph measurement processing means 10.

  In step ST <b> 1, the user installs a camera external calibration marker 8 at an arbitrary position on the photographing target site 7. As the marker 8, it is preferable to use a marker that is portable, has a known size, and that can easily identify three orthogonal axes determined by a coordinate system. By photographing the marker 8 using the digital camera 6, the relative positional relationship between the digital camera 6 and the marker 8 can be accurately obtained. In addition, since the marker 8 uses the geometric information specified as the design information 23 in the three-dimensional coordinate calculation processing in the subsequent steps ST7 to ST10, the coordinate system established at the photographing target site 7 is used. The three axis directions in the design information 23 determined by the above and the three axis directions determined by the coordinate system with reference to the marker 8 are substantially matched (it is not necessary to match completely).

  When the marker 8 is installed at the shooting target site 7, the actual coordinate system established at the shooting target site 7 (that is, the three-axis direction in the design information 23) and the coordinate system based on the marker 8 are completely matched. It is desirable. However, in order to make the actual coordinate axis and the coordinate system based on the marker 8 completely coincide with each other, it is necessary to strictly adjust the direction and angle of the marker 8 using a level when installing the marker 8. is there. Therefore, in the first embodiment, in order to save this effort, the coordinate system based on the three-axis directions in the design information 23 determined by the actual coordinate system established at the photographing target site 7 and the marker 8 is used as a reference. The three-axis directions determined by the actual coordinate system established at the shooting target site 7 causing the dimension measurement error are used as a reference. Inconsistency with the three-axis directions defined by the coordinate system defined above is used to determine the three-dimensional characteristics of the feature points using constraint conditions determined by the geometric information defined as design information 23 by the three-dimensional coordinate calculation means 15 described later. By calculating the coordinates, the dimension measurement accuracy is improved.

  It should be noted that the dimension measurement location of the measurement target specified as the design information 23 includes at least the feature points serving as both end points for calculating the dimension and the dimension measurement direction in the coordinate system. The design information 23 is stored in the storage unit 2, but may be input from the input unit 4.

  FIG. 4 is a diagram showing an example of a camera external calibration marker 8 installed in the shooting target site 7. As the marker 8, for example, a black and white checkered pattern is drawn on a plane as shown in FIG. Use what you have. By performing image processing on the marker portion of the image data obtained by photographing the marker 8 with the digital camera 6, the relative positions of the digital camera 6 and the marker 8 with respect to the three axis directions determined by the coordinate system based on the marker 8. Can be calculated (see Intel OpenCV library). As the three-axis directions determined by the coordinate system with reference to the marker 8, the short axis direction of the paper is the X direction, the long axis direction of the paper is the Y direction, and the vertical direction of the paper is the Z direction.

  The marker 8 is embedded in the imaging target site 7 as long as it can restore the three-axis direction that substantially matches the three-axis direction in the design information 23 defined by the coordinate system established in the imaging target site 7. It is also possible to use a tile that is fixed to the shooting target site 7 and that can be easily recognized, such as a tile.

  In step ST <b> 2 of FIG. 3, the digital cameras 6 installed at two locations distant from each other image the measurement target and the marker 8 and output two pieces of image data 21. The two pieces of image data 21 are input from the input unit 4, stored in the storage unit 2, read out to the temporary storage unit 3, and then subjected to various calculations. Note that data corresponding to one pixel (image data such as RGB) of the image data 21 is stored in a predetermined storage area of the temporary storage unit 3, and a calculation result is also stored in the storage area of the temporary storage unit 3.

  In step ST3, the lens distortion correction unit 11 of the stereo photograph measurement processing unit 10 reads the two pieces of image data 21 from the storage unit 2 and the camera internal parameters 22 of the digital camera 6 obtained in advance with high accuracy into the temporary storage unit 3. Using the focal length of the lens, the central coordinates of the radial lens distortion aberration, and the lens distortion coefficient as the camera internal parameters 22, the image data 21 on the temporary storage means 3 is converted into the radial lens distortion aberration inherent to the wide-angle lens. The lens distortion is corrected by, for example, the cvUnDistort function of the Intel OpenCV library, and the corrected image data 21 is stored in the temporary storage unit 3.

  The camera internal parameters 22 of the digital camera 6 can be obtained with high accuracy using the Tsai method, the Zhang method, or the like widely used in the field of computer vision.

  In step ST4, the camera position recognition means 12 of the stereo photo measurement processing means 10 recognizes the marker 8 from the two corrected image data 21 stored in the temporary storage means 3 by image processing, and the digital camera 6 A relative positional relationship with the marker 8 and a coordinate system based on the marker 8 are obtained using the projection transformation matrix of the lens of the digital camera 6.

  Here, the camera position recognizing means 12 is, as a relative positional relationship between the digital camera 6 and the marker 8, specifically, the coordinates of the projection center of the digital camera 6 in the coordinate system based on the marker 8, and the line-of-sight direction thereof. And the direction of the pixel coordinate axis. For example, when the camera external calibration marker 8 shown in FIG. 4 is used, the relative positional relationship between the digital camera 6 and the marker 8 can be obtained using the Intel OpenCV library.

  For example, the camera position recognizing unit 12 recognizes the marker 8 from the corrected two pieces of image data 21 stored in the temporary storage unit 3 by image processing, and specifically performs the following processing to thereby execute the marker 8. 3 axis directions determined by a coordinate system based on That is, the camera position recognizing means 12 detects a lattice point that is a boundary point between two white cells and two black cells of the marker 8 by specifying a pixel having the highest degree of change in density gradient. First, two axes (for example, x-axis and y-axis) are specified from the degree of alignment of the lattice points of the entire marker 8, and the last one axis (for example, z-axis) is obtained so as to be perpendicular to these two axes. Thus, the three-axis directions determined by the coordinate system with reference to the marker 8 are obtained.

Further, since the distance between the lattice points is known, the camera position recognizing means 12 uses the matrix for calculating the camera external parameters and solves the simultaneous equations expressed by the following equation (1). Then, the relative positional relationship between the digital camera 6 and the marker 8, that is, the coordinates of the projection center of the digital camera 6 in the coordinate system based on the marker 8, which is a camera external parameter, and the direction of the line of sight and the pixel coordinate axis are obtained.
(2D pixel position of grid point) * (Matrix for calculating camera external parameters)
= (3D position of grid point) (1)
Here, the two-dimensional pixel position of the lattice point is specified when a coordinate system based on the marker 8 is obtained.

  Note that the matrix for calculating the camera external parameters is a projection conversion matrix of the lens of the digital camera 6, and is a projection conversion matrix m1 and m2 used by the three-dimensional coordinate calculation means 15 described later. The coordinate system and camera external parameters based on the marker 8 obtained by the camera position recognizing means 12 are used to calculate the two-dimensional pixel position on the image by the three-dimensional coordinate calculating means 15 using the projection transformation matrices m1 and m2. Used in processing to convert to 3D coordinates.

  In step ST5, the feature point recognition unit 13 of the stereo photo measurement processing unit 10 recognizes all recognizable feature points included in the image data 21 by image processing, displays them on the output unit 5, and displays the input unit 4. By using this, only the feature points described in the design information 23 are selected by the user. Regarding the feature point recognition method, for example, a point having a large density change gradient is extracted from color information and light / dark information in each image data, or a line is extracted by connecting points having a large density change gradient after extraction. There is a technique.

  In step ST 6, the feature point input unit 14 of the stereo photograph measurement processing unit 10 uses the feature point recognition unit 13 among the feature points displayed on the image described in the design information 23 and displayed on the output unit 5. For the feature points that are not recognized, the input unit 4 is used to cause the user to input feature points.

  After the process of step ST6 is completed, the three-dimensional coordinate calculation unit 15 of the stereo photograph measurement processing unit 10 proceeds to the process of step ST7 when the feature point to be calculated is represented by a point, and is the calculation target. When the feature point is expressed by a line, a plane, or a solid, the process proceeds to step ST8. At this time, if the feature point to be calculated is expressed in a plane or a solid, the three-dimensional coordinate calculation means 15 extracts one line in the plane or the solid, and proceeds to the process of step ST8.

  In steps ST7 to ST10, the three-dimensional coordinate calculation means 15 inputs the design information 23, the constraint condition determined from the geometric information, the relative positional relationship between the digital camera 6 and the marker 8 determined by the camera position recognition means 12, and A coordinate system based on the marker 8, a projection transformation matrix m1 for projecting and transforming a certain three-dimensional coordinate point onto the image a, a projection transformation matrix m2 for projecting and transforming a certain three-dimensional coordinate point onto the image b, and the principle of triangulation Using the feature point recognized by the feature point recognition unit 13 and the three-dimensional coordinates of the feature point input by the feature point input unit 14, that is, the first feature unit constituted by the feature point of one image data, Corresponding to the first feature portion, the three-dimensional coordinates of the measurement target corresponding to the second feature portion constituted by the feature points of the other image data are calculated.

  At this time, the three-dimensional coordinate calculation means 15 presents the feature points included in the two pieces of image data 21 to the user, and each of the first feature portions configured by the feature points of one of the image data and the first feature portion. The user is allowed to select / specify whether the feature portion pair is a combination with the second feature portion constituted by the feature points of the other image data corresponding to the one feature portion.

  When the feature points are represented by points and the constraint condition is given by a straight line, the three-dimensional coordinate calculation means 15 determines that the points constituting the feature pair in the three-dimensional coordinates are on the same straight line in step ST7. The three-dimensional coordinates of the feature points are calculated using the constraint condition that

  Also, the three-dimensional coordinate calculation means 15 is configured to extract a feature part pair in the three-dimensional coordinates in steps ST8 and ST9 when the feature points are represented by lines, planes, or solids and the constraint conditions are given by straight lines. The three-dimensional coordinates of the feature points are calculated using a constraint condition that both end points of the constituent line segments are on the same straight line.

  Furthermore, the three-dimensional coordinate calculation means 15 calculates the feature pair in the three-dimensional coordinates in steps ST8 and ST10 when the feature points are expressed by lines, planes, or solids and the constraint conditions are given by planes. The three-dimensional coordinates of the feature points are calculated using a constraint condition that both end points of each constituting line segment are on the same plane.

  In FIG. 5, the feature points are represented by points, and as a constraint condition, the feature pairs captured in the two pieces of image data 21 are not on the same point but on the same straight line in step ST7. It is a figure explaining the process of the coordinate calculation means. In FIG. 5, a point constituted by one feature point is a feature part which is a part to be measured in one image. Here, the feature part pair to be measured has a vertical direction component as design information 23. It is assumed that a constraint condition of the positional relationship based on the actual coordinate system established at the photographing target site 7 is given, that is, the direction vector is L = (0, 0, 1). In practice, this means can be applied not only to the vertical direction but also to an arbitrary direction.

  In FIG. 5, it is assumed that the feature part pair to be measured is designated as a point 51 on the image a and a point 52 on the image b. A point 61 obtained by projecting the point 51 on the image a onto the three-dimensional coordinate system using the projection transformation matrix m1 and a point 62 obtained by projecting the point 52 on the image b onto the three-dimensional coordinate system using the projection transformation matrix m2 are represented by the three-dimensional coordinates. It is assumed that the points are on the same straight line.

  In step ST7, the three-dimensional coordinate calculation means 15 inputs the design information 23, and the three-dimensional coordinate point 61 of the point 51 and the three-dimensional coordinate point 62 of the point 52 designated as the design information 23 are on the same straight line (here , On the same straight line having a vertical component), the relative positional relationship between the digital camera 6 and the marker 8 obtained by the camera position recognizing means 12, the coordinate system based on the marker 8, and the projection transformation matrix m1 Using the projection transformation matrix m2 and the principle of triangulation, the three-dimensional coordinates of the points 61 and 62 and a straight line (here, a vertical line) passing through the points 61 and 62 are calculated.

  In FIG. 5, a point 61 is always on a straight line passing through the position of the digital camera 6 that captured the image a and the point 51, and a point 62 is on a straight line passing through the position of the digital camera 6 that captured the image b and the point 52. First, the three-dimensional coordinate calculation means 15 obtains equations of the straight lines 51 and 61 and the straight lines 52 and 62. At this time, you may describe using projection transformation matrix m1, m2. Then, the three-dimensional coordinate calculation means 15 obtains an equation of the straight lines 61 and 62 by obtaining a vertical line that intersects both the straight lines 51 and 61 and the straight lines 52 and 62 which will be in the twisted position. By obtaining the intersection coordinates of the straight lines 61 and 62 and the straight lines 51 and 61, the three-dimensional coordinates of the point 61 can be obtained. The three-dimensional coordinate calculation means 15 calculates three-dimensional coordinates for the point 62 by the same method.

  In FIG. 6, the feature points are represented by lines, and as a constraint condition, steps ST8 and ST9 when the feature part pairs photographed in the two pieces of image data 21 are not on the same line segment but on the same straight line. It is a figure explaining the process of the three-dimensional coordinate calculation means 15 in. In FIG. 6, a line segment connecting two feature points in one image is a feature portion that is a measurement target portion in one image, and the feature portion pair to be measured has a vertical component, that is, a direction It is assumed that a constraint condition that the vector is L = (0, 0, 1) is given. In practice, this means can be applied to L in any direction.

  In FIG. 6, it is assumed that the feature part pair to be measured is designated as a line segment connecting the points 101 and 102 on the image a and as a line segment connecting the points 103 and 104 on the image b. The point 131 obtained by projecting the point 101 on the image a onto the three-dimensional coordinate system using the projection transformation matrix m1 and the point 133 obtained by projecting the point 103 on the image b onto the three-dimensional coordinate system using the projection transformation matrix m2 are not necessarily three-dimensional. It is not necessary to be the same place on the coordinate system, and a point 132 obtained by projecting the point 102 on the image a onto the three-dimensional coordinate system using the projection transformation matrix m1 and a point 104 on the image b are represented using the projection transformation matrix m2. The point 134 projected on the system does not necessarily have to be the same location on the three-dimensional coordinate system, and the points 131, 132, 133, and 134 are arbitrary points on the same straight line in the three-dimensional coordinate system. .

  If the points 131, 132, 133, and 134 are on the same straight line in the three-dimensional coordinate system, the midpoints of the points 131 and 132 and the midpoints of the points 133 and 134 are also on the same straight line. It should be. Therefore, in step ST8, the three-dimensional coordinate calculation means 15 inputs the design information 23, obtains the midpoint 111 of the line segments 101 and 102 connecting the point 101 and the point 102 in the image a, and similarly in the image b The midpoint 112 of the line segments 103 and 104 connecting the line 103 and the point 104 is obtained, and the three-dimensional coordinate point 121 of the point 111 and the three-dimensional coordinate point 122 of the point 112 are on the same straight line (here, the same having a vertical component) On the straight line), the relative positional relationship between the digital camera 6 and the marker 8 obtained by the camera position recognition means 12, the coordinate system based on the marker 8, the projection transformation matrix m1, the projection transformation matrix m2, the triangle Using the principle of surveying, the three-dimensional coordinates of the points 121 and 122 and a straight line (in this case, a vertical line) passing through the points 121 and 122 are calculated.

  Subsequently, in step ST9, the three-dimensional coordinate calculation unit 15 calculates the relative positional relationship between the digital camera 6 and the marker 8 calculated by the camera position recognition unit 12, the coordinate system based on the marker 8, the projection transformation matrix m1, and the projection. Using the transformation matrix m2 and the principle of triangulation, the three-dimensional coordinates of the points 101, 102, 103, 104 are calculated, and the calculated three-dimensional coordinates of the points 101, 102, 103, 104 are obtained in step ST8. It approximates to the closest point of a straight line (in this case, a vertical line) passing through the points 121 and 122. At this time, the three-dimensional coordinate calculation means 15 does not use a constraint condition when calculating the three-dimensional coordinates of the points 101, 102, 103, 104.

That is, the three-dimensional coordinate calculation means 15 calculates a straight line La obtained by projecting the line segments 101 and 102 onto the Z = 0 plane using the projection conversion matrix m1 of the image a, and similarly, the projection conversion matrix of the image b. A straight line Lb obtained by projecting the line segments 103 and 104 onto the Z = 0 plane is calculated using m2. Restoration of the straight lines 131 and 132 of the line segments 101 and 102 and the line segments 103 and 104 on the intersection line between the plane passing through the projection center of the straight line La and the image a and the plane passing through the projection center of the straight line Lb and the image b If there are leading straight lines 133 and 134, the three-dimensional coordinate calculation means 15 uses the principle of triangulation to calculate the intersection coordinates between the straight line passing through the point 101 and the projection center of the image a and the intersection line obtained earlier. By calculating, the three-dimensional coordinates of the point 131 can be obtained.
The three-dimensional coordinate calculation means 15 calculates three-dimensional coordinates for the points 132, 133, and 134 by the same method.

  At this time, a straight line passing through the points 131 and 132 intersects with the vertical line passing through the points 121 and 122 and the point 121, but a straight line passing through the points 131 and 132 is not necessarily vertical, and the points 121 and 122 are not necessarily vertical. It does not necessarily coincide with the vertical line passing through. Therefore, the distance between each of the points 131 and 132 is minimized, and the points 141 and 142 on the vertical line passing through the points 121 and 122 are calculated, and the points 141 and 142 are calculated as points 101 and 122 on the image a. 102 three-dimensional coordinates.

  Similarly, the three-dimensional coordinates of the three-dimensional coordinate points 133 and 134 projected on the point 103 and the point 104 in the image b are calculated using the projection transformation matrix m2 of the image b, and the points 133 and 134 are respectively calculated. The points 143 and 144 on the vertical line passing through the point 121 and the point 122 that minimize the distance between the points 103 and 144 are calculated and set as the three-dimensional coordinate points of the points 103 and 104 on the image b.

  The feature points are represented by lines, and as a constraint condition, the three-dimensional coordinate calculation means 15 in steps ST8 and ST10 when the feature point pairs photographed in the two pieces of image data 21 are on the same plane. The process will be described below. That is, instead of the constraint condition L = (0, 0, 1) in FIG. 6, the feature point pair is given a constraint condition that the normal vector is on the same plane with N = (0, 1, 0). In this case, the midpoints of the points 131 and 132 and the midpoints of the points 133 and 134 should be on the same straight line. Therefore, in step ST8, the three-dimensional coordinate calculation means 15 uses the plane normal vector as a constraint in the calculation of the three-dimensional coordinates of the points 111 and 112 in FIG. And a plane including points 121 and 122 is calculated.

  In step ST10 for calculating the three-dimensional coordinates of the points 101, 102, 103, and 104, which are the original coordinates to be restored, the three-dimensional coordinate calculation means 15 performs the point 121, The three-dimensional coordinates of the three-dimensional coordinate points 141 and 142 on the plane including 122 are obtained using the projection transformation matrix m1 of the image a. Similarly, the three-dimensional coordinate calculation means 15 projects and converts the three-dimensional coordinates of the three-dimensional coordinate points 143 and 144 on the plane including the points 121 and 122 into the image b when calculating the three-dimensional coordinates of the points 103 and 104. It calculates | requires using the matrix m2.

  In step ST11, the dimension calculation means 16 inputs the design information 23, and based on the three-dimensional coordinates of each feature point calculated by the three-dimensional coordinate calculation means 15, the dimension measurement location designated as the design information 23 is obtained. calculate. Each dimension of the calculated dimension measurement location is stored in the temporary storage means 3.

  In step ST <b> 12, the dimension display unit 17 displays the dimension of the dimension measurement location stored in the temporary storage unit 3 on the display unit 5.

  As described above, according to the first embodiment, when the external calibration of the camera is not accurately performed, that is, an actual coordinate system established at the photographing target site 7 (three-axis directions in the design information 23). ) And the coordinate system based on the marker 8 do not completely match, and the three-dimensional coordinate calculation means 15 calculates the three-dimensional coordinates of the feature points using the constraint conditions based on the actual coordinate system. The effect that the dimensional measurement accuracy of the object can be improved is obtained.

  Further, according to the first embodiment, the feature point recognition unit 13 recognizes feature points from the two pieces of image data 21 by image processing, so that even a user unfamiliar with the operation of the computer can easily perform the stereo photo measurement device. The effect that can be used is obtained.

Embodiment 2. FIG.
The block diagram showing the configuration of the stereo photo measurement device according to the second embodiment of the present invention is obtained by deleting the feature point recognition means 13 from the configuration shown in FIG. 1 of the first embodiment.
FIG. 7 is a flowchart showing a process for measuring the dimensions of the dimension measurement location according to the second embodiment of the present invention, and the process of step ST5 is deleted from the flowchart shown in FIG. 3 of the first embodiment. . Among the processes shown in FIG. 7, the processes from step ST <b> 3 to step ST <b> 12 are processes performed by the stereo photograph measurement processing means 10.

In the second embodiment, the feature point input unit 14 does not perform the feature point recognition process from the image data 21 by the feature point recognition unit 13 of the first embodiment. In the feature points described in the above, the corresponding points are present on the image displayed on the output unit 5, and the user inputs the feature points using the input unit 4.
Other processes in steps ST1 to ST4 and ST7 to ST12 are the same as those in the first embodiment.

  As described above, according to the second embodiment, even when the camera external calibration is not accurately performed, the three-dimensional coordinate calculation unit 15 uses the constraint condition based on the actual coordinate system to set the feature point 3. By calculating the dimensional coordinates, it is possible to improve the dimensional measurement accuracy of the measurement target.

  Further, according to the second embodiment, the feature point recognizing means 13 of the first embodiment cannot be reduced to reduce the load on the user who operates the stereo photo measurement device, but the performance is low. Even when the stereo photo measurement device is used, if the external calibration of the camera is not accurately performed, the effect of improving the dimensional measurement accuracy of the measurement target can be obtained.

  The stereo photo measurement apparatus according to the first embodiment and the second embodiment includes a computer and a stereo photo measurement program for causing the computer to function as each unit in the first embodiment and the second embodiment. Can also be realized.

It is a block diagram which shows the structure of the stereo photograph measuring device by Embodiment 1 of this invention. It is a block diagram which shows the hardware constitutions of the stereo photograph measuring device by Embodiment 1 of this invention. It is a flowchart which shows the process for measuring the dimension of the dimension measurement location by Embodiment 1 of this invention. In Embodiment 1 of this invention, it is a figure which shows the example of the marker for camera external calibration installed in the imaging | photography object site. In Embodiment 1 of the present invention, feature points are expressed by points, and as a constraint condition, feature point pairs photographed in two pieces of image data are not on the same point but on the same straight line. It is a figure explaining the process of a three-dimensional coordinate calculation means. In the first embodiment of the present invention, the feature points are represented by lines, and as a constraint condition, the feature point pairs photographed in two pieces of image data are not on the same line segment but on the same straight line It is a figure explaining the process of this three-dimensional coordinate calculation means. It is a flowchart which shows the process for measuring the dimension of the dimension measurement location by Embodiment 2 of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Computation processing means, 2 Storage means, 3 Temporary storage means, 4 Input means, 5 Display means, 6 Digital camera, 7 Shooting object site, 8 Marker, 10 Stereo photograph measurement processing means, 11 Lens distortion correction means, 12 Camera position Recognition means, 13 feature point recognition means, 14 feature point input means, 15 three-dimensional coordinate calculation means, 16 dimension calculation means, 17 dimension display means, 21 image data, 22 camera internal parameters, 23 design information, 24 program.

Claims (6)

Image data storage means for storing two sets of image data obtained by photographing a marker indicating a reference position and a measurement target using a camera from two locations separated from each other;
Design information storage means for storing design information including geometric information of one or both of the dimension measurement location of the measurement target and the linear direction component of the ridge line of the measurement target and the normal direction component of the plane;
Camera position recognition means for recognizing the marker from the image data by image processing, and obtaining a relative positional relationship between the camera and the marker using a projection transformation matrix of the lens of the camera;
Feature point extraction means for extracting feature points of points, lines, planes, and three-dimensional shapes photographed in the image of the image data by image processing or by input information from an input device;
Using the constraint conditions determined from the geometric information, the projection transformation matrix, the relative positional relationship, and the principle of triangulation, the first feature portion composed of feature points of one image data and the first feature portion Correspondingly, a three-dimensional coordinate calculation means for calculating the three-dimensional coordinates of the measurement object corresponding to the second feature portion constituted by the feature points of the other image data,
A stereo photo measurement apparatus comprising: a dimension calculation means for calculating a dimension of the dimension measurement location from the three-dimensional coordinates.   In the three-dimensional coordinate calculation means, when the feature point is expressed by a point and the constraint condition is given by a straight line, the combination of the first feature part and the second feature part in three-dimensional coordinates. 2. The stereo photograph measuring apparatus according to claim 1, wherein the three-dimensional coordinates of the feature points are calculated using a constraint condition that points constituting a certain feature pair are on the same straight line.   In the three-dimensional coordinate calculation means, when the feature point is expressed by a line, a plane, or a solid and the constraint condition is given by a straight line, the first feature portion and the second feature are expressed in three-dimensional coordinates. Using the constraint that the midpoints of the line segments that make up the feature pair that is a combination of parts are on the same straight line, and both end points of each feature point are on a straight line that passes through each midpoint, 2. The stereo photograph measuring apparatus according to claim 1, wherein three-dimensional coordinates of the feature points are calculated.   In the three-dimensional coordinate calculation means, when the feature point is expressed by a line, a plane, or a solid, and the constraint condition is given by a plane, the first feature portion and the second feature are expressed in three-dimensional coordinates. Using the constraint that the midpoints of the line segments that make up the feature point pair that is a combination of parts are on the same plane, and both end points of each line segment are on the plane that passes through each midpoint, the above 2. The stereo photograph measuring apparatus according to claim 1, wherein three-dimensional coordinates of the feature points are calculated. Image data storage means for storing two sets of image data obtained by photographing a marker indicating a reference position and a measurement target using a camera from two locations separated from each other;
Design information storage means for storing design information including geometric information of one or both of the dimension measurement location of the measurement target and the linear direction component and the plane normal direction component of the ridge line of the measurement target , A stereo photo measuring method for calculating the dimensions of the above dimension measuring points,
Camera position recognizing means for recognizing the marker from the image data by image processing, and determining a relative positional relationship between the camera and the marker using a projection transformation matrix of the lens of the camera;
A step of extracting feature points of points, lines, planes, and three-dimensional shapes taken by the feature point extraction means in the image of the image data by image processing or by input information from an input device;
A first feature portion in which the three-dimensional coordinate calculation means is composed of feature points of one image data using the constraint conditions determined from the geometric information, the projection transformation matrix, the relative positional relationship, and the principle of triangulation And calculating the three-dimensional coordinates of the measurement object corresponding to the second feature portion constituted by the feature points of the other image data corresponding to the first feature portion;
And a step of calculating a dimension of the dimension measurement portion from the three-dimensional coordinates. Computer
An image data storage means for storing a marker indicating a reference position using two cameras away from each other and two pieces of image data obtained by photographing a measurement target;
Design information storage means for storing design information including geometric information of one or both of the dimension measurement location of the measurement object and the linear direction component of the ridge line of the measurement object and the normal direction component of the plane;
Camera position recognizing means for recognizing the marker from the image data by image processing and obtaining a relative positional relationship between the camera and the marker using a projection transformation matrix of the lens of the camera;
Feature point extraction means for extracting feature points of points, lines, planes, and three-dimensional shapes photographed in the image of the image data by image processing or by input information from an input device;
Using the constraint conditions determined from the geometric information, the projection transformation matrix, the relative positional relationship, and the principle of triangulation, the first feature portion composed of feature points of one image data and the first feature portion Correspondingly, a three-dimensional coordinate calculation means for calculating the three-dimensional coordinates of the measurement object corresponding to the second feature portion constituted by the feature points of the other image data,
A dimension calculation means for calculating the dimension of the dimension measurement location from the three-dimensional coordinates;
Stereo photo measurement program to function as

Images