JPH0486957A - Method for inputting appearance data of stereoscopic object - Google Patents

Abstract

PURPOSE:To automatically process the three-dimensional coordinate measurement of a stereoscopic object and the input to a CAD by extracting the sequence of feature points from a picture group for which the image of the object is picked up from plural directions, and automatically obtaining the three-dimensional coordinates of the object by using a standard three-dimensional stereoscope model. CONSTITUTION:The image of the stereoscopic object is picked up from plural directions, and the picture data is stored in the frame memory of a picture processor 4. Since the coordinate value is made correspondent to a coordinate value in a reference coordinate system, plural reference feature points are calculated in advance and measured in the respective image pickup directions, and when the picture is analyzed, the positions of respective feature points of the picture in the respective plural directions are collated and corrected. After a series of image pickup processes are completed, a picture analysis process is started. First, the feature points are extracted concerning the first picture. The group of windows for extracting feature points is set to the picture of the object so as to instruct a range where feature points exist, and among the feature points, feature points having luminance higher than fixed one are extracted. The group of extracted feature points is collected in a feature point file. Then, the feature points are collected from the feature file to a two-dimensional feature point coordinate system and the feature point coordinates are made three-dimensional.

Description

[Detailed Description of the Invention] [Field of Industrial Application] This invention automatically extracts three-dimensional coordinate data of the external appearance of a three-dimensional object, and has applications such as graphic ink display of the object and measurement of three-dimensional dimensions. The present invention relates to a method for inputting external shape data of a three-dimensional object.

[Conventional technology]

Conventionally, when displaying or measuring the dimensions of a three-dimensional object, it is possible to use CAD data if it has been entered in advance using a CAD (computer-aided design device), etc.; If one is not available, the only methods currently available are to take images of the object from each direction and enter the coordinates manually, or to measure by shining a measuring tape directly onto the object. Note that such technology is required, for example, in the following fields.

i) In the case of apparel design In the case of clothing design, especially custom-made design, conventionally the dimensions of the target person are measured using a tape measure or the like. Furthermore, in the case of ready-made clothes, it is common to manufacture them without any particular measurements being taken.

In the future, when we enter an era where unique designs are required, it will be difficult to convert data measured with a measuring tape into CAD, so we will need a database input method for human body dimensions and clothing appearance, especially when CAD for apparel becomes popular. Therefore, it is predicted that a three-dimensional input device will be required.

ii) In the case of custom-made products whose performance is influenced by the shape of a special curve: The shape of golf club heads, the shape of shoes, metalwork, or handicrafts are currently made by hand, but in the future these fields will expand. As design using CAD becomes widespread, a three-dimensional shape input device becomes necessary as an input means for CAD.

111) When inputting the external shape of a three-dimensional object that cannot be directly measured It is virtually impossible to directly measure buildings and huge structures, and conventionally, they have been measured by, for example, triangulation. However, when very precise data is not required, there is a desire for a device that can more easily input the coordinates of an external shape.

Conventionally, methods known to address these various needs include, for example, using manual input operations, or irradiating laser slit light and measuring the slit pattern with a television camera, etc., to obtain a three-dimensional shape. There is.

[Problem to be solved by the invention]

However, manual work is time-consuming, and although the laser slit light pattern has the advantage of being able to measure with high precision, it is difficult to measure when the surface of the object reflects diffusely against the laser beam (such as a shiny metallic surface). Laser light irradiation is undesirable when dealing with humans, such as in the apparel design field, and cannot be put to practical use.Furthermore, laser light irradiation is not suitable for measurements in the outside world such as buildings. Irradiation of light is impossible.

Therefore, an object of the present invention is to enable easy input of external shape data of a three-dimensional object using a television camera.

[Means to solve the problem]

Markers are attached to the characteristic points (or characteristic lines) of the target three-dimensional object (or model), the object is imaged from multiple directions using a lifting device, and the obtained images are used to create a standard three-dimensional object. Extract the three-dimensional coordinates of the feature points (or lines) of the three-dimensional shape using the model, display the data graphically on a display device by superimposing it on the original image, make manual corrections if necessary, and input it to CAD. do.

[Effect]

A feature point sequence is extracted from a group of images of a target object taken from multiple directions, and the three-dimensional coordinates of the target object are automatically obtained using a standard three-dimensional solid model and can be input into CAD.

Specifically, the steps are as follows.

) Markers with different luminance levels from the surrounding background are attached to the feature points (or feature lines) of the three-dimensional object to be analyzed during the lifting and injury process, and images are taken from multiple directions using a camera or the like. Note that markers can be arbitrarily attached to feature points that are desired to be extracted.

ii) Feature point extraction and labeling process Feature points are extracted by a computer from a plurality of captured images. This feature point extraction is performed using the difference in luminance level from the surrounding background using the marker, and is performed for each image taken from a plurality of directions. Each image has a plurality of feature points, but in order to distinguish between each feature point, each point is labeled as follows.

That is, since the position of the assigned marker is within a range that is expected in advance, a search area (window for extracting feature points) for each feature point is set. In this way, by specifying the existence range of the corresponding feature points in each image using the window, it is possible to label the corresponding feature points in each image with the same number.

iii) Interpolation of feature points When images are captured from multiple directions with a camera, not all feature points appear in the image, and depending on the direction, they may be hidden from the camera's imaging field of view. Therefore, mutual interpolation is performed from feature points of a plurality of images captured in different directions.

iv) Concatenation and three-dimensional coordinate extraction of feature point group extraction coordinates.

The feature point groups extracted and labeled in ii) and 1ii) above are rearranged in the three-dimensional space axis direction for each feature point, and the feature point coordinates of the three-dimensional object are finally determined and input to CAD. . In order to process items 1ii) and iv), here we measure the coordinates of each grid point using a standard three-dimensional grid model, interpolate the hidden points, and calculate the three-dimensional coordinates from the two-dimensional coordinates. This makes it easy to convert etc.

〔Example〕

Figure 1 shows an overview of the external shape input device. Number 1 in the same figure
2 is a golf club, 2 is a television camera, 3 is an illuminator, 4 is an image processing device, and 5 is an image display, which is an example of a relatively small three-dimensional object. Note that although it is assumed here that one television camera 2 is used and images are taken from multiple directions by changing the position, a plurality of television cameras 2 may be installed.

The camera used here is determined in relation to the nature of the marker provided for feature point extraction, but here we will use a color marker as the feature point extraction marker. This color marker is a specific color that allows the feature point color to be clearly distinguished from the background color, such as a color in which the feature point color and other colors (background color) are opposite in hue, green for red, black for white, etc. Make sure to choose as follows.

FIG. 2 shows the relationship between hue frequency and intensity, and it can be seen that feature points can be distinguished from the background by appropriately selecting the hue frequency. Further, FIG. 3 shows an example of a color marker when the three-dimensional object is a golf club head. The size of each marker is determined appropriately based on the detection accuracy, the resolution of the imaging field of view, and the operating range of the feature points. We will use a camera.

Under these conditions, a three-dimensional object is imaged from multiple directions using one or more cameras, and the image data is stored in a frame memory within the image processing device 4. At this time, the coordinate values of the feature points in each image (hereinafter also referred to as values in the camera coordinate system) are associated with the coordinate values in the reference coordinate system (hereinafter also referred to as values in the reference coordinate system). For this purpose, a plurality of reference feature points are determined in advance, the reference feature points are measured in each imaging direction, and the feature point positions of the images in each plurality of directions are compared and corrected (calibrated) during image analysis. This situation is shown in Fig. 4, where the symbol M in Fig. 4 (a) indicates a reference grid model (calibrator), and markers are attached to each grid point of this model M, so that, for example, three cameras can be set up. 2A, 2B
From the coordinates of each grid point captured and read by 2C, a conversion table T between the camera coordinate system and the reference coordinate system as shown in the figure (b) is created.
Create. At this time, in order to transform and correct feature points extracted from images in each direction, a reference camera or camera position (camera coordinates) is determined in advance.

For example, the front camera or the camera position from the front direction is set as the reference camera position, and this coordinate system is set as the reference camera coordinate system.

As shown in Figure 5, a grid model is imaged with one camera,
An example of measuring each grid point is shown below. The process is the same as the case shown in FIG. 4 except that one camera is used to take images and measure at different positions, so the details will be omitted. In addition, FIG. 5(A) shows the relationship between the grid model and the camera, and FIG. 5(B) shows the conversion table T.
shows.

When a series of imaging processes are completed, the process moves to an image analysis process.

The object of image analysis is a group of images in a certain range taken from the plurality of directions.

First, feature points are extracted from the first image (frame). Since the position of the imaging camera and the object position are fixed, a group of feature point extraction windows are set to indicate the range of feature points in the object image, and those with brightness above a certain level are selected. Extracted. The extracted feature point group has the same number as the corresponding feature point extraction window group.
labeled with. Regarding the next image (2), feature points are extracted using the feature point extraction window group (2) prepared in advance in the same manner as above. The extracted feature points are labeled with the same number as the corresponding feature point extraction window group ■, and feature point groups after image ■ are extracted and labeled in the same manner. After processing up to the last frame, the process ends. The feature point group extracted in this way is compiled into a feature point file.

The feature point coordinate values extracted from each camera are values in the unique coordinate system of each camera, and in images captured from each direction, there may be feature points hidden from the imaging field of view, and camera distortion may occur. An error occurs due to Therefore, in the above feature point extraction and labeling, the coordinates of the same feature points are mutually verified and corrected from the feature point file created in the process, and each feature point coordinate is converted into a two-dimensional feature point in the standard camera coordinate system. Summarize into a coordinate system. Thereafter, feature point coordinates are converted into three-dimensional coordinates using a conversion table between the camera coordinate system and a reference coordinate system obtained in advance via a standard model. This is also done using a three-dimensional lattice model as shown in FIG. 4 or 5.

FIG. 6 is an explanatory diagram for explaining another embodiment of the invention.

This is an example applied to the head of a golf club 1 like in FIG. 1, but the difference is that a linear near-infrared marker is used instead of a dotted color marker. However, by thinning the linear object or processing it as a collection of points, this case can be handled in the same manner as in FIG. 1, so the details will be omitted. In addition, the same figure (a)
shows a near-infrared marker, and the same figure (b) shows a window and a special m line (binarized line segment pattern).

The above targets relatively small three-dimensional objects, but large objects such as structures can be treated in the same manner as above by using a marker, such as a colored lamp, and attaching it to the object as appropriate. It is possible to do so.

〔Effect of the invention〕

According to this invention, it is now possible to automatically process the three-dimensional coordinate measurement of three-dimensional objects and input to CAD using a television camera and computer, which was previously done manually, and can meet the demands of various industries. I can respond.

[Brief explanation of drawings]

FIG. 1 is a schematic diagram showing an embodiment of the present invention, FIG. 2 is an explanatory diagram for explaining a feature point extraction method using hue frequency,
Fig. 3 is an explanatory diagram for explaining the dot marker, Figs. 4 and 5 are explanatory diagrams for explaining the principle of interpolation and transformation using a three-dimensional grid model of images in each direction, and Fig. 6 is an explanatory diagram for explaining the principle of interpolation and conversion using a three-dimensional grid model of images in each direction. It is an explanatory view for explaining other examples of this invention. 1... Golf club head, 2.2A, 2B, 2C.
...TV camera, 3.. Illuminator, 4.. Image processing device, 5.. Image display. Figure 2 Figure 3

Claims (1)

[Claims] 1) Point-like or linear markers are attached to each feature point of a three-dimensional object, and images are taken from multiple directions using one or more imaging means, and each image is attached to the object. The coordinates of each feature point of the assigned marker are extracted, and these coordinates are compared with the results of measuring each grid point from each direction of the three-dimensional grid model that serves as a reference, and the coordinates of each feature point are determined by interpolation. It is expressed as a two-dimensional feature point coordinate group using the direction screen as a reference, and these coordinate groups are converted into three-dimensional coordinates based on the correspondence with the three-dimensional coordinates obtained when measuring the three-dimensional grid model, and then used on a computer. A method for inputting external shape data of a three-dimensional object, characterized in that the data can be input into a design device. 2) The method for inputting three-dimensional object external shape data according to claim 1, wherein the marker is of a color whose hue is opposite to that of the background. 3) The method for inputting three-dimensional object external shape data according to claim 1, wherein the marker is a near-infrared reflective marker including a near-infrared reflective tape.