JP5473683B2 - Feature detection system - Google Patents

Description

  The present invention relates to a feature detection system that detects, for example, a feature provided around a road using a laser point cloud and a camera image.

  2. Description of the Related Art In recent years, mobile mapping systems have been developed that continuously acquire surrounding camera images and laser survey data in three dimensions while driving a vehicle equipped with a GPS receiver, gyroscope, laser, camera, and the like. The mobile mapping system is mainly expected to be applied to technologies such as high-precision map creation.

In order to create a highly accurate map, it is necessary to detect features such as outdoor buildings and advertisements from video and laser survey data acquired by the mobile mapping system and extract necessary information.
Here, the laser survey mainly obtains the distance from the vehicle, and the position (point) is obtained therefrom, and the position depends on the position of the vehicle. The position surveying technique can perform highly accurate positioning using a GPS satellite or the like.

  In laser surveying, it is known that there is something due to the reflection of the laser, but it is not known what the object is, so a camera image or the like is required to identify the feature. That is, by determining to which position in the camera image the point cloud is mapped based on a set of laser survey data (a set of points, hereinafter referred to as “point cloud”), a camera image, and a shooting position of the camera image. The feature to which the point cloud belongs can be determined.

  On the other hand, how much the point cloud rides on the feature, that is, how much the laser hits the feature depends on the surveying situation. Moreover, since the point cloud exists discretely, there are cases where the entire feature cannot be covered. Therefore, there has been a problem that the feature cannot be sufficiently detected only by the point cloud, the camera image, and the shooting position of the camera image.

  In order to solve such a problem, a polygonal area corresponding to the range of the feature is designated on the camera image, a point cloud included in the designated polygonal area is extracted, and the extracted point cloud is used. A three-dimensional model construction system that calculates a real plane including a feature is known (see, for example, Patent Document 1).

JP 2004-348575 A

However, the prior art has the following problems.
In the conventional 3D model construction system, when the specified polygonal area is larger than the actual feature, for example, as shown in FIG. 7, it hits another feature behind the target feature. A point cloud may be added as a candidate for the real plane. As a result, there is a problem that the actual plane including the feature is bent or displaced.

  The present invention has been made in order to solve the above-described problems, and can calculate a real plane including a feature with high accuracy and accurately detect the position and size of the feature. The object is to obtain an object detection system.

  The feature detection system according to the present invention stores image storage means in which camera images around a target feature and laser survey data around the feature are associated with the camera image as point cloud data. Surveying data storage means, display means for displaying the camera image stored in the image storage means, designation means for designating two points on the camera image displayed on the display means, and two points designated by the designation means A point cloud included within a predetermined distance from the middle point is extracted from the survey data storage means, the plane containing the feature is calculated from the extracted point cloud, and the two points designated by the designation means are calculated. And a feature detection means for detecting the position and size of the feature by mapping on the plane.

According to the feature detection system of the present invention, the feature detection means extracts from the survey data storage means a point group included within a predetermined distance from the midpoint of the two points designated by the designation means, A plane including the target feature is calculated from the extracted point group, and two points designated by the designation unit are mapped on the calculated plane to detect the position and size of the feature.
Therefore, it is possible to calculate the actual plane including the feature with high accuracy and accurately detect the position and size of the feature.

It is a block block diagram which shows the terrestrial feature detection system which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the display part of the feature detection system which concerns on Embodiment 1 of this invention. It is a flowchart which shows in detail the process in the point cloud extraction part of the feature detection system which concerns on Embodiment 1 of this invention. (A), (b) is explanatory drawing for demonstrating operation | movement of the point group extraction part of the feature detection system which concerns on Embodiment 1 of this invention. (A), (b) is explanatory drawing for demonstrating operation | movement of the area | region calculation part of the terrestrial feature detection system which concerns on Embodiment 1 of this invention. It is another block block diagram which shows the terrestrial feature detection system which concerns on Embodiment 1 of this invention. It is explanatory drawing for demonstrating the problem of the conventional three-dimensional model construction system.

  Hereinafter, preferred embodiments of the feature detection system of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts will be described with the same reference numerals.

  First, when calculating an actual plane that includes outdoor features, if the end points of the feature are used, the calculated actual plane may be bent or deviated from the actual one as described above. . Therefore, it is desirable to use the middle point of the feature as much as possible. That is, if the real plane can be calculated using only the middle point, it is considered that the error is rather reduced if the outer point (end point) is not used.

  However, since the range of the feature cannot be calculated simply by specifying the point inside the feature, it is necessary to separately specify the range of the feature. If there are two pieces of information on the inside point of the feature and the range of the feature, the range occupied by the feature can be calculated.

  Therefore, in the following embodiment, a system for calculating (detecting) a range of a feature, that is, a position and a size of the feature by designating two points at both ends of the feature is proposed. Since the midpoints of the two points at both ends fall within the feature, unnecessary points can be eliminated by setting a certain range around the midpoint as the range of the point cloud. A real plane including the feature can be calculated with high accuracy. Here, the fixed range may be, for example, up to half of this distance based on the distance from the middle point to both end points.

  Further, after calculating the real plane, the two specified points are mapped on the real plane. Thereby, the actual position of the designated point can be calculated. At this time, even if the designated position is slightly deviated, the calculation of the actual plane is not affected. In addition, the effect of displacement appears only on the size of the feature, but this effect is minor.

  When the shape of the feature is rectangular, the position and size (width, height) of the feature can be uniquely obtained by designating two points on both ends of the diagonal line of the feature. Similarly, the range of a rectangular region that covers a feature can be calculated for other shapes.

  Also, if the feature is located on the premise that it can be seen from a vehicle or the like as in an advertisement, it is not obstructed by other objects during laser surveying, that is, the radar sent in laser surveying is You can expect to hit the whole thing. In this case, it can be expected that the point group used for calculation of the real plane is surely on the advertisement, and as a result, the real plane including the feature can be calculated with high accuracy.

Embodiment 1 FIG.
1 is a block diagram showing a feature detection system according to Embodiment 1 of the present invention. In FIG. 1, this feature detection system includes image data 11 (image storage means), point cloud data 12 (survey data storage means), vehicle parameters 13, an image designation section 21, a point designation section 22 (designation means), a display. A unit 23 (display means), a feature detection device 30 (feature detection means), and area output result data 41 are provided.

The feature detection apparatus 30 includes an image drawing unit 31, a point group extraction unit 32, a real plane calculation unit 33, and a region calculation unit 34.
The feature detection device 30 is configured by a microprocessor (not shown) having a CPU and a memory storing a program, and each block constituting the feature detection device 30 is stored as software in the memory. ing.
Hereinafter, the function of each part of the feature detection system configured as described above will be described.

  The image data 11 contains camera images taken by a camera mounted on a vehicle used in the mobile mapping system described above. Each camera image is stored together with the shooting time.

The point cloud data 12 stores laser survey data measured by a laser mounted on a vehicle used in the mobile mapping system described above. The laser survey data is stored as coordinate values and survey times of each point group.
Here, a plane rectangular coordinate system is used as the coordinate system, the x coordinate and the y coordinate indicate the distance from the origin in the latitude direction and the longitude direction, respectively, and the z coordinate indicates the altitude. The plane Cartesian coordinate system is shown in the Geospatial Information Authority of Japan, Plane Cartesian Coordinate System (Ministry of Land, Infrastructure, Transport and Tourism Notification No. 9 of 2004) (https://www.gsi.go.jp/LAW/heimencho.html). Therefore, detailed description is omitted.

  The vehicle parameter 13 contains parameters of the vehicle and camera from which the image data 11 and the point cloud data 12 are acquired. Specifically, data such as the position and orientation of the vehicle at the time of camera shooting, the arrangement of the camera on the vehicle, the focal length of the camera, and the like are stored.

  The image designation unit 21 designates a camera image to be displayed on the display unit 23 from each camera image in the image data 11 by an operator's operation. As a means for designating a camera image, for example, as shown in FIG. 2, each camera image in the image data 11 is displayed as a thumbnail on the display unit 23, and one to be displayed is selected and displayed at the same magnification. Such a method can be considered.

  The point designating unit 22 designates two points on the camera image designated by the image designating unit 21 by the operation of the operator. At this time, a range of about several percent of the upper, lower, left and right edges on the camera image displayed on the display unit 23 may be excluded from the designation target.

  The image drawing unit 31 displays the camera image specified by the image specifying unit 21, the two points on the camera image specified by the point specifying unit 22, the region calculated by the region calculating unit 34 described later, and the like on the display unit 23. indicate.

  The point cloud extraction unit 32 refers to the vehicle parameter 13 based on the camera image specified by the image specification unit 21 and the two points on the camera image specified by the point specification unit 22 from the point cloud data 12. A point group necessary for calculating the real plane in the real plane calculation unit 33 is extracted. The detailed operation of the point cloud extraction unit 32 will be described later.

  The real plane calculation unit 33 calculates a real plane including the feature from the point group extracted by the point group extraction unit 32. By the processing in the real plane calculation unit 33, the feature and the actual position (point cloud) are linked.

The region calculation unit 34 calculates the range of the target feature on the real plane calculated by the real plane calculation unit 33 and outputs the range to the region output result data 41. The position and size (width, height) of the feature can be calculated by processing in the area calculation unit 34.
The region output result data 41 stores the position and size of the feature calculated by the region calculation unit 34.

Next, the operation of the feature detection system configured as described above will be described.
When a camera image is designated by the image designation unit 21, the camera image and its shooting time are uniquely determined from the image data 11. This information and the area calculated by the area calculation unit 34 are displayed on the display unit 23 by the image drawing unit 31. When the calculated area extends outside the camera image, only the range that fits within the camera image is displayed.

  In addition, when the area is not calculated by the area calculation unit 34, the area is not displayed. Further, the image designating unit 21 may designate whether or not to display the calculated area. When it is designated not to display the region, only the camera image is displayed on the display unit 23 even after the region calculation unit 34 calculates the region.

After the camera image is displayed on the display unit 23, the point designating unit 22 designates two points on the camera image. The order in which the two points are specified does not matter.
After two points on the camera image are designated by the point designating unit 22, the coordinates of the two points (X coordinate and Y coordinate on the camera image) and the shooting time of the camera image displayed on the display unit 23 are used. Based on the point cloud data 12, the point cloud extraction unit 32 extracts a point cloud necessary for the real plane calculation by the real plane calculation unit 33.

Here, the processing in the point cloud extraction unit 32 will be described in detail with reference to the flowchart of FIG.
First, the point cloud extraction unit 32 extracts from the point cloud data 12 only the point cloud measured within a certain time range from the shooting time of the camera image (step S1). At this time, mainly used data is a point group near the photographing time of the camera image.

  Next, the point group extraction unit 32 executes position calculation when each extracted point group is mapped on the camera image (step S2). The reason why the point cloud is limited to the vicinity of the photographing time of the camera image in step S1 is to prevent the point cloud from being hit by an object before and after the target feature in step S2. It is. If the point cloud is near the shooting time of the camera image, there is a high possibility that the point cloud actually hits the feature when the mapped point cloud is in the feature on the camera image.

In the calculation of the position of each point cloud on the camera image, first, the camera photographing in which the relative position (x ₁ , y ₁ , z ₁ ) from the camera position of each point in the point cloud is stored in the vehicle parameter 13 is performed. It is calculated from the position and orientation of the vehicle at the time and the position and orientation of the camera on the vehicle. Thereafter, the focus of the camera length f, the distance per one pixel of the camera p _m and the camera optical axis center coordinates (xc, yc) on the basis of, the position of the point group of the camera image, FIGS. 4 (a), ( The position (x, y) of the calculated point cloud on the camera image is expressed by the following equation (1).

  Subsequently, the point group extraction unit 32 selects a plurality of points close to the midpoint of the two points designated by the point designation unit 22 from the positions on the camera image of each point group calculated in step S2 ( Step S3). At this time, the number of points to be selected is at least three. Note that when selecting a point, the coordinates on the camera image of the two specified points are used to set the selection range to, for example, “± 1/4 of the X (Y) coordinate width between the middle point and the two points. You may restrict | limit like the "contained range". Thereby, even when there is a slight deviation in the point designation in the point designation unit 22, the error can be absorbed.

  Here, when the number of points selected by the point group extraction unit 32 is less than 3, the subsequent processing is not executed as an error, and the process returns to the camera image designation by the image designation unit 21. On the other hand, when the number of points selected by the point group extraction unit 32 is three or more, the actual plane calculation unit 33 calculates the actual plane (the actual plane including the feature) including the obtained points. .

  Actually, since the obtained point has an error, there are few cases where a plane on which all the points are calculated is calculated. Therefore, the nearest plane is calculated using a least square method or the like. The plane thus obtained is an actual plane including the target feature.

  After the real plane is calculated by the real plane calculation unit 33, the range of the target feature is calculated by the region calculation unit 34. Specifically, first, the area calculation unit 34 performs position calculation when the two points specified by the point specification unit 22 are mapped on the actual plane calculated by the actual plane calculation unit 33. This position calculation is uniquely determined because the above formula (1) is satisfied and the position (x, y, z) of the calculated point satisfies the formula of the real plane calculated by the real plane calculation unit 33.

  Next, after calculating the positions of the two points designated by the point designating unit 22 on the real plane, the region calculating unit 34, as shown in FIG. 5A, (x, y) between the two points. The distance in the plane is the width, and the elevation difference between the two points is the height. In addition, the area calculation unit 34 calculates points that are at an intermediate point between the two points and are symmetrical with the two points specified by the point specification unit 22 with respect to the plane π parallel to the xy plane. That is, as shown in FIG. 5B, the area calculation unit 34 calculates points A ′ and B ′ that are symmetrical with the designated two points A and B, and points A, B ′, B, A A rectangular area surrounded by four points' is defined as the feature range.

  After the area calculation by the area calculation unit 34, the image drawing unit 31 displays the calculated area on the display unit 23 and shifts to a camera image designation waiting state by the image designation unit 21. At this time, the feature range (width, height, and four vertexes of the rectangle) calculated by the region calculation unit 34 is output to the region output result data 41.

  As described above, the position and size of the feature can be detected from the camera image and the point group. Further, the region calculated by the region calculation unit 34 is displayed on the camera image displayed on the display unit 23 so as to be confirmed by the operator's eyes at the same time.

  As described above, if there are two pieces of information on the inside point of the feature and the range of the feature, the range occupied by the feature can be calculated. Since there is an error, there is no guarantee that the accuracy is high. On the other hand, in the mobile mapping system, since information around the feature is continuously acquired, the possibility that the same feature appears in a plurality of camera images is high. Therefore, the accuracy of the calculation result can be determined by mapping the range calculation result in one camera image onto another camera image.

  Specifically, when another camera image is designated by the image designation unit 21, a new another camera image is displayed on the camera image in which the area is displayed as described above. It is possible to check the accuracy of the calculated area by superimposing the calculated area on another camera image that is the camera image other than the camera image in which the area is calculated. .

  As described above, regarding the features reflected in a plurality of camera images, the operator can best overlap each other by calculating a region in each camera image and superimposing the calculated region on another camera image. By determining what is, it is possible to select the exact position and size of the feature. As a result, errors that cannot be picked up only by calculation can be absorbed.

As described above, according to the first embodiment, the feature detection unit extracts from the survey data storage unit a point group included within a predetermined distance from the midpoint between the two points designated by the designation unit. The plane containing the target feature is calculated from the extracted point cloud, and the two points specified by the specifying means are mapped on the calculated plane, so that the position and size of the target feature are determined. To detect.
Therefore, it is possible to calculate the actual plane including the feature with high accuracy and accurately detect the position and size of the feature.

In the first embodiment, the area calculation unit 34 calculates the position of the feature and calculates the range of the feature in the camera image. At this time, the area calculation unit 34 may cut out the range of the feature from the camera image and save it separately. In this case, as shown in FIG. 6, in addition to the feature detection system shown in FIG. 1, region image data 42 for storing a camera image cut out according to the range of the feature is separately provided, and a region calculation unit 34 outputs the cut-out camera image to the area image data 42.
By separately displaying the position of the feature and the cut-out camera image, it is easy to check the feature detection status.

  In the first embodiment, the feature is detected by designating the two points on the camera image designated by the image designating unit 21 by the point designating unit 22. Here, two points are specified to calculate the width and height of the feature. However, when calculating only the position of the feature, it is not always necessary to specify two points. Only the coordinates (position) may be specified. At this time, instead of the midpoint of the two points specified by the point specifying unit 22 shown in the first embodiment, the specified one point is used, and the point group extracting unit 32 and the real plane calculating unit 33 By executing the processing, the position of the feature can be calculated.

  When only the center coordinates (position) of a feature are specified, as described above, the width and height of the feature cannot be calculated, but a camera image of a certain range is shown in FIG. 6, for example. It can be separately stored in the region image data 42 so that the operator can determine what the feature is. At this time, although the above-mentioned range varies depending on the symmetric feature, for example, if it is an outdoor advertisement (signboard, etc.), a camera image (size as a camera image) within a range of 1 to 2 meters from a designated point. It can be considered that the operator can make a judgment by storing the position).

  11 image data, 12 point group data, 13 vehicle parameters, 21 image designation unit, 22 point designation unit, 23 display unit, 30 feature detection device, 31 image drawing unit, 32 point group extraction unit, 33 real plane calculation unit, 34 area calculation unit, 41 area output result data, 42 area image data.

Claims (3)

Image storage means for storing camera images around the target feature;
Surveying data storage means for storing laser survey data around the feature in association with the camera image as point cloud data;
Display means for displaying a camera image stored in the image storage means;
Designation means for designating two points on the camera image displayed on the display means;
A point group included within a predetermined distance from the midpoint of the two points specified by the specifying unit is extracted from the survey data storage unit, and a plane including the feature is calculated from the extracted point group. , Feature detection means for mapping the two points designated by the designation means on the calculated plane and detecting the position and size of the feature;
A feature detection system comprising:   The feature detection system according to claim 1, wherein the feature detection unit displays a region including the feature detected for a certain camera image in a superimposed manner on another camera image.   3. The feature detection according to claim 1, wherein the feature detection unit cuts out and saves a portion corresponding to the position and size of the detected feature from the camera image. system.

Images