CN118424295A - Method and system for intelligent routing inspection route planning of unmanned aerial vehicle at construction site - Google Patents
Method and system for intelligent routing inspection route planning of unmanned aerial vehicle at construction site Download PDFInfo
- Publication number
- CN118424295A CN118424295A CN202410846012.XA CN202410846012A CN118424295A CN 118424295 A CN118424295 A CN 118424295A CN 202410846012 A CN202410846012 A CN 202410846012A CN 118424295 A CN118424295 A CN 118424295A
- Authority
- CN
- China
- Prior art keywords
- building
- inspection
- point cloud
- model
- patrol
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000010276 construction Methods 0.000 title claims abstract description 22
- 238000007689 inspection Methods 0.000 title claims description 181
- 239000000178 monomer Substances 0.000 claims abstract description 106
- 239000011159 matrix material Substances 0.000 claims abstract description 53
- 230000009466 transformation Effects 0.000 claims abstract description 42
- 230000007613 environmental effect Effects 0.000 claims abstract description 12
- 238000009499 grossing Methods 0.000 claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims description 42
- 239000002356 single layer Substances 0.000 claims description 33
- 238000005070 sampling Methods 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 10
- 238000002922 simulated annealing Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 9
- 230000009471 action Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000009435 building construction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Automation & Control Theory (AREA)
- Processing Or Creating Images (AREA)
Abstract
The disclosure relates to a construction site unmanned aerial vehicle intelligent routing planning method and system. The method comprises the following steps: acquiring image data which is acquired by an unmanned aerial vehicle and contains building to be detected and environmental information, and three-dimensionally reconstructing the image data to obtain a three-dimensional model; converting the outer surface of the BIM of the building to be detected into a first point cloud, and converting the three-dimensional model into a second point cloud; coarse registration and fine registration are carried out on the first point cloud and the second point cloud; determining a bounding box of the outer surface of the BIM, and rotationally importing the bounding box into a building monomer model of a building from the three-dimensional model through a preset transformation matrix; determining a patrol target based on the building monomer model, determining a patrol space area based on the patrol target, determining a group of waypoints in the patrol space area, determining the access sequence of the group of waypoints to form a patrol path, performing path smoothing on the patrol path to obtain patrol path parameters, and transmitting the patrol path parameters to the unmanned aerial vehicle so as to enable the unmanned aerial vehicle to execute patrol tasks.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of intelligent inspection of building construction, in particular to a method and a system for planning an intelligent inspection route of a unmanned aerial vehicle on a construction site.
Background
With the continuous development of the building industry, the inspection requirements in the construction and operation and maintenance processes are continuously improved, and the inspection safety, inspection efficiency and automation are more required. The traditional inspection scheme is mainly based on manual inspection or manual operation unmanned aerial vehicle, and the like, so that the problems of high labor consumption, low inspection efficiency, outstanding safety problem and the like are required to be solved.
In the related art, the automatic inspection of the building can be realized based on the unmanned aerial vehicle and the building information model BIM technology, and the inspection efficiency is high and the safety problem is less. However, when the path of the unmanned aerial vehicle is planned, part of route planning in the inspection area is rough and inaccurate, and even missing inspection exists, so that the accuracy of the inspection result of the unmanned aerial vehicle is reduced.
Disclosure of Invention
In order to solve the technical problems or at least partially solve the technical problems, an embodiment of the disclosure provides a method and a system for intelligent inspection route planning of a construction site unmanned aerial vehicle.
In a first aspect, an embodiment of the present disclosure provides a method for planning an intelligent inspection route of a construction site unmanned aerial vehicle, including:
Acquiring image data which is acquired by an unmanned aerial vehicle and contains building to be detected and environmental information, and three-dimensionally reconstructing the image data to obtain a three-dimensional model; acquiring a building information model BIM of the building to be detected, converting the outer surface of the BIM into first point cloud data, and converting the three-dimensional model into second point cloud data;
Performing point cloud rough registration on the first point cloud data and the second point cloud data based on a sampling consistency initial registration algorithm, and performing fine registration on a point cloud rough registration result by adopting a fine registration algorithm;
Determining a first bounding box of the outer surface represented by the first point cloud data, and rotationally importing the first bounding box into the three-dimensional model through a preset transformation matrix to realize segmentation from the three-dimensional model to obtain a first building monomer model corresponding to the building to be detected; the preset transformation matrix is a transformation matrix after the point cloud fine registration;
Determining a patrol target based on the first building monomer model, determining a patrol space area based on the patrol target, determining a group of waypoints in the patrol space area, determining the access sequence of the group of waypoints to form a patrol path, performing path smoothing on the patrol path to obtain patrol route parameters, and transmitting the patrol route parameters to the unmanned aerial vehicle so that the unmanned aerial vehicle executes a patrol task.
In one embodiment, the method further comprises:
Obtaining an orthographic image based on the two-dimensional reconstruction of the image data, and dividing the orthographic image to obtain a division result, wherein the division result at least comprises a two-dimensional image of the building to be detected;
Projecting the outer surface of the BIM to an XY plane to obtain first plane profile data; determining second plane contour data of the building to be detected based on the first plane contour data and the segmentation result;
Dividing the three-dimensional model based on the second plane contour data to obtain a second building monomer model corresponding to the building to be detected; merging the second building monomer model and the first building monomer model to obtain a final building monomer model;
And replacing the first building monomer model in the step of determining a patrol goal based on the first building monomer model with the final building monomer model.
In one embodiment, the method further comprises:
obtaining DSM data of a digital surface model corresponding to the building to be detected;
The determining the second plane contour data of the building to be detected based on the first plane contour data and the segmentation result includes:
Registering the DSM data with the segmentation result to obtain a first registration result; registering the first plane contour data with the first registration result to obtain registered second plane contour data;
the step of obtaining a second building monomer model corresponding to the building to be detected by segmentation from the three-dimensional model based on the second plane contour data comprises the following steps:
and projecting the plane contour represented by the second plane contour data into the three-dimensional model, and carrying out given threshold value growth on the projected plane contour on a Z axis to obtain a three-dimensional second bounding box, and dividing the three-dimensional model based on the second bounding box to obtain the second building monomer model.
In one embodiment, the method further comprises:
Registering and fusing the first bounding box and the second bounding box, taking a union of the first bounding box and the second bounding box, and merging the second building monomer model and the first building monomer model to obtain a final building monomer model when determining that the model range represented by the union meets the preset requirement;
Or alternatively
Acquiring single-layer outline data of the BIM, wherein the single-layer outline data represent the outline shape of any single floor of the building to be detected;
and projecting the plane outline represented by the single-layer outline data to an XY plane, and scaling according to a preset proportion to obtain a single-layer bounding box, and dividing the final building monomer model based on the single-layer bounding box to obtain a single-layer model of each floor.
In one embodiment, the coarse registration of the first and second point cloud data based on the sampling consistency initial registration algorithm includes:
calculating fast point characteristic histogram (FPFH) descriptors of the first point cloud data and the second point cloud data, and matching point clouds between the first point cloud data and the second point cloud data based on the FPFH descriptors;
randomly selecting at least three pairs of matched point clouds from all matched point clouds, calculating a transformation matrix based on the at least three pairs of matched point clouds, and repeating the steps until the error value meets a preset condition, and outputting a target transformation matrix;
And performing coarse registration on the first point cloud data and the second point cloud data based on the target transformation matrix.
In one embodiment, the determining a patrol goal based on the first building monomer model, determining a patrol spatial area based on the patrol goal, includes:
receiving input inspection parameters, and dividing and determining corresponding inspection targets from the first building monomer model based on the inspection parameters; the inspection parameters comprise identification information of inspection targets, different inspection parameters correspond to different inspection targets, and the inspection targets are at least partial areas in the first building monomer model;
And expanding the normal vector on the surface of the inspection target to obtain an initial inspection space region, discretizing the initial inspection space region based on a voxelized mode, and then downsampling the discretized initial inspection space region by adopting a farthest point sampling method to obtain the inspection space region.
In one embodiment, the determining a set of waypoints in the patrol space area, determining the access sequence of the set of waypoints forms a patrol path, includes:
Calculating a group of waypoints required by inspection through a greedy algorithm, and determining the visible range of each waypoint to be in the waypoint course;
based on the surrounding environment information of the inspection target, searching a flight path among the group of waypoints by adopting an A-type algorithm, and further obtaining a distance matrix of the group of waypoints; wherein the surrounding environment information includes at least obstacle position information;
And calculating the access sequence of the group of waypoints by adopting a simulated annealing algorithm according to the distance matrix so as to form a patrol path.
In a second aspect, an embodiment of the present disclosure provides a construction site unmanned aerial vehicle intelligent patrol route planning system, including:
The data processing module is used for acquiring image data which is acquired by the unmanned aerial vehicle and contains the building to be detected and the environmental information, and obtaining a three-dimensional model based on three-dimensional reconstruction of the image data; acquiring a building information model BIM of the building to be detected, converting the outer surface of the BIM into first point cloud data, and converting the three-dimensional model into second point cloud data;
The point cloud registration module is used for carrying out point cloud rough registration on the first point cloud data and the second point cloud data based on a sampling consistency initial registration algorithm, and then carrying out fine registration on a point cloud rough registration result by adopting a fine registration algorithm;
The model segmentation module is used for determining a first bounding box of the outer surface represented by the first point cloud data, and rotationally importing the first bounding box into the three-dimensional model through a preset transformation matrix so as to realize segmentation from the three-dimensional model to obtain a first building monomer model corresponding to the building to be detected; the preset transformation matrix is a transformation matrix after the point cloud fine registration;
And the inspection execution module is used for determining an inspection target based on the first building monomer model, determining an inspection space region based on the inspection target, determining a group of waypoints in the inspection space region, determining the access sequence of the group of waypoints to form an inspection path, performing path smoothing on the inspection path to obtain inspection route parameters, and transmitting the inspection route parameters to the unmanned aerial vehicle so as to enable the unmanned aerial vehicle to execute an inspection task.
In a third aspect, an embodiment of the present disclosure provides a computer readable storage medium having stored thereon a computer program that, when executed by a processor, implements the method for intelligent inspection route planning for a job site unmanned aerial vehicle according to any one of the above embodiments.
In a fourth aspect, an embodiment of the present disclosure provides an electronic device, including:
A processor; and
A memory for storing a computer program;
Wherein the processor is configured to execute the method for intelligent inspection route planning for a job site unmanned aerial vehicle according to any of the above embodiments via execution of the computer program.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages:
According to the intelligent inspection route planning method and system for the unmanned aerial vehicle at the construction site, which are provided by the embodiment of the disclosure, image data which are acquired by the unmanned aerial vehicle and contain building to be detected and environment information are acquired, and a three-dimensional model is obtained based on three-dimensional reconstruction of the image data; acquiring a building information model BIM of the building to be detected, converting the outer surface of the BIM into first point cloud data, and converting the three-dimensional model into second point cloud data; performing point cloud rough registration on the first point cloud data and the second point cloud data based on a sampling consistency initial registration algorithm, and performing fine registration on a point cloud rough registration result by adopting a fine registration algorithm; determining a first bounding box of the outer surface represented by the first point cloud data, and rotationally importing the first bounding box into the three-dimensional model through a preset transformation matrix to realize segmentation from the three-dimensional model to obtain a first building monomer model corresponding to the building to be detected; the preset transformation matrix is a transformation matrix after the point cloud fine registration; determining a patrol target based on the first building monomer model, determining a patrol space area based on the patrol target, determining a group of waypoints in the patrol space area, determining the access sequence of the group of waypoints to form a patrol path, performing path smoothing on the patrol path to obtain patrol route parameters, and transmitting the patrol route parameters to the unmanned aerial vehicle so that the unmanned aerial vehicle executes a patrol task. In this way, the scheme of the embodiment not only considers single BIM information during path planning, but also considers three-dimensional model information obtained by three-dimensional reconstruction of environmental images acquired based on unmanned aerial vehicle aerial photography, and adopts a sampling consistency initial registration algorithm for coarse registration and then combines fine registration during point cloud registration, so that a point cloud registration result is more accurate, a building monomer model segmented by a surrounding frame based on the outer surface of the BIM, namely an outer contour, is more accurate, a patrol target is segmented according to the building monomer model and a planning path is determined, and patrol regional route planning can be more accurate.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a flow chart of a method for intelligent inspection route planning for a unmanned aerial vehicle at a construction site in accordance with an embodiment of the present disclosure;
FIG. 2 is a flow chart of a method for intelligent inspection route planning for a job site unmanned aerial vehicle in accordance with another embodiment of the present disclosure;
FIG. 3 is a flow chart of a routing inspection path planning process in accordance with an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of an intelligent inspection route planning system for a unmanned aerial vehicle at a construction site according to an embodiment of the disclosure;
Fig. 5 is a schematic diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.
It should be understood that, hereinafter, "at least one (item)" means one or more, and "a plurality" means two or more. "and/or" is used to describe association relationships of associated objects, meaning that there may be three relationships, e.g., "a and/or B" may mean: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
Fig. 1 is a flowchart of a method for planning an intelligent inspection route of a construction site unmanned aerial vehicle according to an embodiment of the disclosure, where the method for planning an intelligent inspection route of a construction site unmanned aerial vehicle may be executed by a computing device, and may specifically include the following steps:
Step S101: acquiring image data which is acquired by an unmanned aerial vehicle and contains building to be detected and environmental information, and three-dimensionally reconstructing the image data to obtain a three-dimensional model; and acquiring a building information model BIM of the building to be detected, converting the outer surface of the BIM into first point cloud data, and converting the three-dimensional model into second point cloud data.
For example, a fixed route may be set at a fixed height, and the unmanned aerial vehicle flies along the fixed route to quickly acquire environmental information such as image data including a building to be detected and the environmental information, and completes three-dimensional reconstruction and two-dimensional reconstruction based on a plurality of images acquired by the unmanned aerial vehicle, so as to obtain a three-dimensional model and an orthographic image, which is described in detail below. Wherein the three-dimensional model may be a mesh grid model. In addition, a BIM model of a building or a construction object on a construction site is obtained, a BIM model shell is extracted, the BIM model shell is converted into first point cloud data, namely source point cloud data, and the reconstructed three-dimensional model is converted into second point cloud data, namely target point cloud data. The building to be detected may be a building such as a building which has been built for use, or a building which is being constructed and built, without limitation.
Step S102: and performing point cloud rough registration on the first point cloud data and the second point cloud data based on a sampling consistency initial registration algorithm (SAC_IA algorithm), and performing fine registration on a point cloud rough registration result by adopting a fine registration algorithm.
Illustratively, the sac_ia algorithm may improve the accuracy and robustness of registration. The fine registration algorithm may be, but is not limited to, ICP (Iterative Closest Point) algorithm, which uses ICP algorithm to fine register the point cloud data, which minimizes the distance between the closest points in the source and target point clouds by iteratively adjusting the transformation, until the average distance between the point pairs is satisfied to be less than a set threshold.
Step S103: determining a first bounding box of the outer surface represented by the first point cloud data, and rotationally importing the first bounding box into the three-dimensional model through a preset transformation matrix to realize segmentation from the three-dimensional model to obtain a first building monomer model corresponding to the building to be detected; the preset transformation matrix is a transformation matrix after the point cloud fine registration.
The first bounding box is an outline bounding box extracted based on a BIM model, the bounding box is rotated through a transformation matrix obtained through point cloud registration, namely a preset transformation matrix, the rotated bounding box is led into the three-dimensional model, whether more than 50% of triangular patches of the three-dimensional model are in the bounding box is judged, if yes, the triangular patches are patches of a single building or a building of a construction area, and the judging process is repeated until all triangular patches of the three-dimensional model are circularly finished. Thus, the part belonging to the three-dimensional model in the bounding box can be segmented to obtain a first building monomer model corresponding to the building to be detected.
Step S104: determining a patrol target based on the first building monomer model, determining a patrol space area based on the patrol target, determining a group of waypoints in the patrol space area, determining the access sequence of the group of waypoints to form a patrol path, performing path smoothing on the patrol path to obtain patrol route parameters, and transmitting the patrol route parameters to the unmanned aerial vehicle so that the unmanned aerial vehicle executes a patrol task.
By way of example, the inspection target may be a portion of the building to be inspected, such as the front, back, side, or one or more floors. The inspection target can be accurately determined by dividing the established first building monomer model, then the inspection space region, namely the unmanned aerial vehicle inspection flight region, is determined, a group of waypoints, namely the inspection points for hovering and collecting data during unmanned aerial vehicle inspection, are determined in the inspection space region, then an inspection path can be formed by determining the access sequence of the group of waypoints, the inspection path is subjected to path smoothing processing to obtain inspection route parameters, the inspection route parameters are led out to the unmanned aerial vehicle, and the unmanned aerial vehicle can execute inspection tasks based on the inspection route parameters, such as collecting building data such as image return so as to facilitate maintenance detection or quality supervision.
According to the scheme, single BIM information is considered during path planning, three-dimensional model information obtained through three-dimensional reconstruction of environmental images acquired based on unmanned aerial vehicle aerial photography is considered, and a sampling consistency initial registration algorithm is adopted during point cloud registration, and then fine registration is combined, so that a point cloud registration result is more accurate, a building monomer model segmented by a surrounding frame based on the outer surface of the BIM, namely an outer contour, is more accurate, a patrol target is segmented according to the building monomer model, a planning path is determined, patrol regional route planning can be more accurate, and therefore the scheme of the embodiment can achieve fine unmanned aerial vehicle patrol route planning based on a BIM-three-dimensional reconstruction segmentation mode, and accuracy of unmanned aerial vehicle patrol results is improved. In addition, the first point cloud, namely the BIM point cloud adopts an external point cloud instead of an integral point cloud, so that the processed data volume is reduced, and the efficiency of unmanned aerial vehicle inspection path planning can be improved as a whole.
Based on the above embodiments, and referring to fig. 2 in combination, in one embodiment, the method may further include the following steps:
Step S201: and obtaining an orthographic image based on the two-dimensional reconstruction of the image data, and dividing the orthographic image to obtain a division result, wherein the division result at least comprises a two-dimensional image of the building to be detected.
For example, the orthographic image generated by two-dimensional reconstruction can be accurately segmented by adopting a pre-trained UNet network, so that the accurate segmentation of a building or a construction scene is realized, and a two-dimensional image of the building to be detected is obtained.
Step S202: projecting the outer surface of the BIM to an XY plane to obtain first plane profile data; and determining second plane contour data of the building to be detected based on the first plane contour data and the segmentation result.
The outer surface, i.e. the outer contour, of the BIM model is projected to the XY plane to obtain first plane contour data, building boundary contour data can be determined based on the two-dimensional image of the building to be detected in the segmentation result, and more accurate second plane contour data can be obtained by fusing the first plane contour data and the building boundary contour data.
Step S203: dividing the three-dimensional model based on the second plane contour data to obtain a second building monomer model corresponding to the building to be detected; and merging the second building monomer model and the first building monomer model to obtain a final building monomer model.
After the second plane contour data is determined, a second building monomer model corresponding to the building to be detected can be obtained by segmentation from the reconstructed three-dimensional model. For example, the plane contour represented by the second plane contour data is moved to the bottom surface of the three-dimensional model, a prism is obtained by moving a certain height along the height direction, the prism and the three-dimensional model are subjected to Boolean intersection operation, and the intersection part is extracted to obtain the second building single model. The dividing method is of course not limited thereto.
Accordingly, the first building monomer model in the step of determining the inspection target based on the first building monomer model may be replaced with the final building monomer model in step S104. The remaining steps in step S104 may then be performed.
In this embodiment, the accurate building monomer model is obtained by dividing the orthographic image based on the two-dimensional reconstruction and the BIM model, and the accurate final building monomer model is obtained by combining the orthographic image based on the two-dimensional reconstruction and the BIM model, so that the inspection target is determined and the inspection path of the unmanned aerial vehicle is planned, and the inspection area route planning is further accurate.
In one embodiment, the method may further comprise: and acquiring data of a digital surface model DSM (Digital Surface Model) corresponding to the building to be detected. In particular, the DSM data may be a simulation of a surface of a vegetation, a surface of a building, or the like, with elevation information. The DSM data may be considered as a top-down depth map of the building.
The determining the second plane contour data of the building to be detected based on the first plane contour data and the segmentation result in step S202 may specifically include: registering the DSM data with the segmentation result to obtain a first registration result; and registering the first plane contour data with the first registration result to obtain registered second plane contour data.
For example, based on the building data identified by the DSM data, a SURF (Speeded Up Robust Features) algorithm may be used to register the two-dimensional segmentation result, that is, the two-dimensional image of the building to be detected, with the DSM data to obtain a first registration result, and output a registration matrix. And secondly, carrying out secondary registration on the first plane contour represented by the first plane contour data and a first registration result by taking the BIM outer contour projected to the XY plane by the BIM model as a reference, and outputting registered contour data, namely second plane contour data.
Correspondingly, in step S203, the second building monomer model corresponding to the building to be detected is obtained by dividing from the three-dimensional model based on the second plane contour data, which may specifically include: and projecting the plane contour represented by the second plane contour data into the three-dimensional model, and carrying out given threshold value growth on the projected plane contour on a Z axis to obtain a three-dimensional second bounding box, and dividing the three-dimensional model based on the second bounding box to obtain the second building monomer model.
The two-dimensional plane contour represented by the second plane contour data is projected into the three-dimensional model, the two-dimensional plane contour is increased by a given threshold value on the Z axis, a three-dimensional bounding box is obtained, all the patches of the three-dimensional model are judged based on the bounding box, whether more than 50% of the triangular patches of the three-dimensional model are in the bounding box or not is judged, if yes, the triangular patches are patches of a single building or a building of a construction area, and the judging process is repeated until all triangular patches of the three-dimensional model are circularly completed. Thus, the part belonging to the three-dimensional model in the bounding box can be segmented to obtain a second building monomer model corresponding to the building to be detected. The final building monomer model may then be obtained based on the combination of the second building monomer model and the first building monomer model.
In this embodiment, the building DSM data is further combined to register and segment the building and the BIM-two-dimensional reconstruction segmentation result again to obtain an accurate second building monomer model, and then the second building monomer model is combined with the BIM-three-dimensional reconstruction segmentation first building monomer model to obtain a final building monomer model, so that the determination of the inspection target and the planning of the inspection path of the unmanned aerial vehicle are realized, and the routing planning of the inspection area can be further accurate.
On the basis of the above embodiment, in one embodiment, the method may further include: registering and fusing the first bounding box and the second bounding box, taking a union of the first bounding box and the second bounding box, and merging the second building monomer model and the first building monomer model to obtain a final building monomer model when the model range represented by the union meets the preset requirement.
In the embodiment, fusion matching of two-dimensional and three-dimensional reconstruction segmentation results is realized. Specifically, the first bounding box and the second bounding box are fused, the two bounding boxes are registered and fused, the union of the two bounding boxes is taken, whether the union range of the two bounding boxes or the corresponding triangle patch number error does not meet the requirement is judged, the segmentation is invalid, two steps of two-dimensional segmentation and three-dimensional segmentation are carried out again, and under the condition that the error meets the requirement, the two models, namely the second building monomer model and the first building monomer model are combined, so that the final building monomer model is obtained. According to the scheme of the embodiment, the combined final building monomer model can be more accurate, the inspection target is determined and the inspection path of the unmanned aerial vehicle is planned according to the combined final building monomer model, so that the routing planning of the inspection area can be further accurate, the scheme of the embodiment can realize further refined unmanned aerial vehicle inspection routing planning, and the accuracy of the inspection result of the unmanned aerial vehicle is further improved.
In one embodiment, the method may further comprise: acquiring single-layer outline data of the BIM, wherein the single-layer outline data represent the outline shape of any single floor of the building to be detected; and projecting the plane outline represented by the single-layer outline data to an XY plane, and scaling according to a preset proportion to obtain a single-layer bounding box, and dividing the final building monomer model based on the single-layer bounding box to obtain a single-layer model of each floor.
For example, in this embodiment, a single-layer matching of the segmentation result, i.e., the final building monomer model, may be achieved. Specifically, a single-layer bounding box, which is a layered bounding box of the BIM point cloud, is extracted based on the BIM model, for example, the single-layer outline projection of the BIM model is scaled appropriately on an XY plane to obtain the layered bounding box, the triangular surface patch of the final building monomer model is circularly judged based on the layered bounding box, and if 50% of the triangular surface patch is in the layered bounding box, the triangular surface patch belongs to a floor corresponding to the layered bounding box. Therefore, floor singulation of the final building monomer model can be realized, and a single-layer model is obtained. The upper and lower bottom surfaces of the layered bounding box can be materialized, the cross detection is carried out with the layered model, namely the single-layer model, the surfaces in the layered model are stored, and the visualization of the single-layer model is optimized. After single-layer matching processing, when the inspection target is determined based on the inspection parameters, the inspection target can be determined rapidly, and the efficiency of inspection path planning is improved. Meanwhile, each layered single-layer model is used as a part of a final building single model, so that the final building single model is more refined, for example, the final building single model is refined to a floor, when the inspection target is determined and the inspection path of the unmanned aerial vehicle is planned, the routing planning of the inspection area is further accurate, and therefore, the scheme of the embodiment can realize the further refined routing planning of the unmanned aerial vehicle, and further improve the accuracy of the inspection result of the unmanned aerial vehicle.
Based on any one of the foregoing embodiments, in one embodiment, performing coarse registration on the first point cloud data and the second point cloud data in step S102 based on a sampling consistency initial registration algorithm may specifically include: calculating Fast Point Feature Histogram (FPFH) (FAST PERSISTENT Feature Histograms) descriptors of the first point cloud data and the second point cloud data, and matching point clouds between the first point cloud data and the second point cloud data based on the FPFH descriptors; randomly selecting at least three pairs of matched point clouds from all matched point clouds, calculating a transformation matrix based on the at least three pairs of matched point clouds, and repeating the steps until the error value meets a preset condition, and outputting a target transformation matrix; and performing coarse registration on the first point cloud data and the second point cloud data based on the target transformation matrix.
The method comprises the steps of realizing rough registration of point cloud data by adopting a SAC_IA method, calculating a Fast Point Feature Histogram (FPFH) feature descriptor (namely a descriptor) of a source point cloud (a first point cloud of a BIM) and a target point cloud (a second point cloud of a three-dimensional reconstruction model), capturing local geometric information by the FPFH descriptor, providing a steady representation of point cloud features, establishing a corresponding relation between the source point cloud and the target point cloud through the FPFH descriptor, matching points between the source point cloud and the target point cloud based on the FPFH, randomly selecting x (x > =3) pairs of matching points, calculating a transformation matrix between the source point cloud and the target point cloud based on the matching points, repeating the process until an error value such as an error between the point pairs meets a preset condition such as the error value is smaller than or equal to a preset threshold, outputting the transformation matrix (namely the target transformation matrix), and realizing rigid transformation from the source point cloud to the target point cloud according to the local geometric information, namely realizing rough registration. Therefore, the random sampling selects the point pairs consistent with other point pairs as much as possible, the error is reduced, the transformation matrix with the smallest error is selected to realize registration, the precision and the robustness of registration are improved, the route planning of the unmanned aerial vehicle is finer, and the accuracy of the unmanned aerial vehicle inspection result is improved.
On the basis of any one of the foregoing embodiments, in one embodiment, the determining a patrol target based on the first building monomer model, and determining a patrol space area based on the patrol target may specifically include:
receiving input inspection parameters, and dividing and determining corresponding inspection targets from the first building monomer model based on the inspection parameters; the inspection parameters comprise identification information of inspection targets, different inspection parameters correspond to different inspection targets, and the inspection targets are at least partial areas in the first building monomer model;
And expanding the normal vector on the surface of the inspection target to obtain an initial inspection space region, discretizing the initial inspection space region based on a voxelized mode, and then downsampling the discretized initial inspection space region by adopting a farthest point sampling method to obtain the inspection space region.
As an example, referring to fig. 3, according to input identification parameters such as foundation pit, building, 3-10 floors of a-1 building, etc., corresponding different inspection targets are automatically segmented, i.e. inspection ranges are extracted. In addition, the surrounding environment information of the inspection target can be enlarged and divided based on the bounding box of the inspection target. And carrying out discretization processing on the initial inspection space region based on a voxelized mode, and then downsampling the initial inspection space region again by adopting a farthest point sampling method to obtain the inspection space region. Therefore, the data volume can be reduced, the path planning efficiency of the unmanned aerial vehicle is improved, and meanwhile, the accuracy of the planned path is maintained. According to the scheme, the corresponding local inspection target can be determined by dividing from the whole building monomer model or the multiple layering models in a finer manner, the inspection range is automatically extracted according to the inspection parameters, and the inspection path planning is conducted aiming at the inspection target, so that the finer path planning can be realized, the route planning of the unmanned aerial vehicle is finer, and the accuracy of the unmanned aerial vehicle inspection result is improved.
On the basis of any one of the foregoing embodiments, in one embodiment, the determining a set of waypoints (also referred to as viewpoints) in the patrol space area, and determining the access sequence of the set of waypoints forms a patrol path may specifically include: calculating a group of waypoints required by inspection through a greedy algorithm, and determining the visible range and the waypoint heading of each waypoint; based on the surrounding environment information of the inspection target, searching a flight path among the group of waypoints by adopting an A-type algorithm, and further obtaining a distance matrix of the group of waypoints; wherein the surrounding environment information includes at least obstacle position information; and calculating the access sequence of the group of waypoints by adopting a simulated annealing algorithm according to the distance matrix so as to form a patrol path.
For example, the visible range of the waypoint may be calculated by means of light projection, and the heading of the waypoint of the unmanned aerial vehicle may be determined by means of an improved artificial potential field method (ARTIFICIAL POTENTIAL FIELD) using the triangular patch area of the model as the potential field strength, and the actual visibility of the waypoint is calculated after the heading is determined. And calculating the waypoints required by the inspection through a greedy algorithm. Based on the extracted environmental information, an A-algorithm is adopted to find a flight path between the waypoints, a distance matrix of the waypoints is calculated, and the distance matrix is a matrix containing distances between a group of the waypoints. And calculating the access sequence of the inspection points, namely the waypoints, by adopting a simulated annealing algorithm according to the distance matrix of the waypoints to obtain an inspection path. And (3) carrying out path smoothing by adopting a three-time Bezier curve based on the obtained routing inspection path to obtain final routing inspection route parameters. And finally, the routing parameters can be led out to the unmanned aerial vehicle to execute the routing task.
It should be noted that although the steps of the methods of the present disclosure are illustrated in a particular order in the figures, this does not require or imply that the steps must be performed in that particular order or that all of the illustrated steps must be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc. In addition, it is also readily understood that these steps may be performed synchronously or asynchronously, for example, in a plurality of modules/processes/threads.
As shown in fig. 4, an embodiment of the present disclosure provides an intelligent inspection route planning system for a construction site unmanned aerial vehicle, including:
The data processing module 401 is configured to obtain image data including building to be detected and environmental information acquired by an unmanned aerial vehicle, and obtain a three-dimensional model based on three-dimensional reconstruction of the image data; acquiring a building information model BIM of the building to be detected, converting the outer surface of the BIM into first point cloud data, and converting the three-dimensional model into second point cloud data;
The point cloud registration module 402 is configured to perform point cloud coarse registration on the first point cloud data and the second point cloud data based on a sampling consistency initial registration algorithm, and then perform fine registration on a point cloud coarse registration result by adopting a fine registration algorithm;
the model segmentation module 403 is configured to determine a first bounding box of the outer surface represented by the first point cloud data, and rotationally import the first bounding box into the three-dimensional model through a preset transformation matrix, so as to achieve segmentation from the three-dimensional model to obtain a first building monomer model corresponding to the building to be detected; the preset transformation matrix is a transformation matrix after the point cloud fine registration;
and the inspection execution module 404 is configured to determine an inspection target based on the first building monomer model, determine an inspection space region based on the inspection target, determine a set of waypoints in the inspection space region, determine an access sequence of the set of waypoints to form an inspection path, perform path smoothing processing on the inspection path to obtain an inspection route parameter, and send the inspection route parameter to the unmanned aerial vehicle so that the unmanned aerial vehicle executes an inspection task.
In one embodiment, the system may further comprise a two-dimensional segmentation module for:
Obtaining an orthographic image based on the two-dimensional reconstruction of the image data, and dividing the orthographic image to obtain a division result, wherein the division result at least comprises a two-dimensional image of the building to be detected;
Projecting the outer surface of the BIM to an XY plane to obtain first plane profile data; determining second plane contour data of the building to be detected based on the first plane contour data and the segmentation result;
Dividing the three-dimensional model based on the second plane contour data to obtain a second building monomer model corresponding to the building to be detected; merging the second building monomer model and the first building monomer model to obtain a final building monomer model;
And the inspection execution module is further used for replacing the first building monomer model with the final building monomer model when the step of determining the inspection target based on the first building monomer model is executed.
In one embodiment, the system may further comprise a data acquisition module for: obtaining DSM data of a digital surface model corresponding to the building to be detected;
the two-dimensional segmentation module determines second plane contour data of the building to be detected based on the first plane contour data and the segmentation result, and specifically comprises the following steps:
Registering the DSM data with the segmentation result to obtain a first registration result; registering the first plane contour data with the first registration result to obtain registered second plane contour data;
The two-dimensional segmentation module segments the three-dimensional model based on the second planar contour data to obtain a second building monomer model corresponding to the building to be detected, and the two-dimensional segmentation module specifically comprises:
and projecting the plane contour represented by the second plane contour data into the three-dimensional model, and carrying out given threshold value growth on the projected plane contour on a Z axis to obtain a three-dimensional second bounding box, and dividing the three-dimensional model based on the second bounding box to obtain the second building monomer model.
In one embodiment, the system may further comprise a segmentation fusion module for:
Registering and fusing the first bounding box and the second bounding box, taking a union of the first bounding box and the second bounding box, and merging the second building monomer model and the first building monomer model to obtain a final building monomer model when the model range represented by the union meets the preset requirement.
In one embodiment, the system may further comprise a single layer segmentation module for:
Acquiring single-layer outline data of the BIM, wherein the single-layer outline data represent the outline shape of any single floor of the building to be detected;
and projecting the plane outline represented by the single-layer outline data to an XY plane, and scaling according to a preset proportion to obtain a single-layer bounding box, and dividing the final building monomer model based on the single-layer bounding box to obtain a single-layer model of each floor.
In one embodiment, the point cloud registration module is specifically configured to:
calculating fast point characteristic histogram (FPFH) descriptors of the first point cloud data and the second point cloud data, and matching point clouds between the first point cloud data and the second point cloud data based on the FPFH descriptors;
randomly selecting at least three pairs of matched point clouds from all matched point clouds, calculating a transformation matrix based on the at least three pairs of matched point clouds, and repeating the steps until the error value meets a preset condition, and outputting a target transformation matrix;
And performing coarse registration on the first point cloud data and the second point cloud data based on the target transformation matrix.
In one embodiment, the inspection execution module determines an inspection target based on the first building monomer model, and determines an inspection space region based on the inspection target, and specifically includes:
receiving input inspection parameters, and dividing and determining corresponding inspection targets from the first building monomer model based on the inspection parameters; the inspection parameters comprise identification information of inspection targets, different inspection parameters correspond to different inspection targets, and the inspection targets are at least partial areas in the first building monomer model;
And expanding the normal vector on the surface of the inspection target to obtain an initial inspection space region, discretizing the initial inspection space region based on a voxelized mode, and then downsampling the discretized initial inspection space region by adopting a farthest point sampling method to obtain the inspection space region.
In one embodiment, the patrol execution module determines a set of waypoints in the patrol space area, and determines an access sequence of the set of waypoints to form a patrol path, which specifically includes:
Calculating a group of waypoints required by inspection through a greedy algorithm, and determining the visible range of each waypoint to be in the waypoint course;
based on the surrounding environment information of the inspection target, searching a flight path among the group of waypoints by adopting an A-type algorithm, and further obtaining a distance matrix of the group of waypoints; wherein the surrounding environment information includes at least obstacle position information;
And calculating the access sequence of the group of waypoints by adopting a simulated annealing algorithm according to the distance matrix so as to form a patrol path.
The specific manner in which the respective modules perform the operations and the corresponding technical effects thereof have been described in corresponding detail in relation to the embodiments of the method in the above embodiments, which will not be described in detail herein.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied. The components shown as modules or units may or may not be physical units, may be located in one place, or may be distributed across multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The embodiment of the disclosure also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the intelligent inspection route planning method for the unmanned aerial vehicle at the construction site according to any one of the above embodiments.
By way of example, the readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable storage medium may also be any readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
The embodiment of the disclosure also provides an electronic device comprising a processor and a memory, wherein the memory is used for storing a computer program. Wherein the processor is configured to perform the job site unmanned aerial vehicle intelligent patrol route planning method of any one of the embodiments above via execution of the computer program.
An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 5. The electronic device 600 shown in fig. 5 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 5, the electronic device 600 is embodied in the form of a general purpose computing device. Components of electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different system components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code that is executable by the processing unit 610 such that the processing unit 610 performs steps according to various exemplary embodiments of the present invention described in the above method examples section of the present specification. For example, the processing unit 610 may perform the steps of the method as shown in fig. 1.
The memory unit 620 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 6201 and/or cache memory unit 6202, and may further include Read Only Memory (ROM) 6203.
The storage unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
Bus 630 may be a local bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or using any of a variety of bus architectures.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 600, and/or any device (e.g., router, modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 650. Also, electronic device 600 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 over the bus 630. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 600, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a usb disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause a computing device (may be a personal computer, a server, or a network device, etc.) to perform the steps of the method for intelligent inspection route planning of the construction site unmanned aerial vehicle according to the embodiments of the present disclosure.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. The intelligent inspection route planning method for the unmanned aerial vehicle on the construction site is characterized by comprising the following steps of:
Acquiring image data which is acquired by an unmanned aerial vehicle and contains building to be detected and environmental information, and three-dimensionally reconstructing the image data to obtain a three-dimensional model; acquiring a building information model BIM of the building to be detected, converting the outer surface of the BIM into first point cloud data, and converting the three-dimensional model into second point cloud data;
Performing point cloud rough registration on the first point cloud data and the second point cloud data based on a sampling consistency initial registration algorithm, and performing fine registration on a point cloud rough registration result by adopting a fine registration algorithm;
Determining a first bounding box of the outer surface represented by the first point cloud data, and rotationally importing the first bounding box into the three-dimensional model through a preset transformation matrix to realize segmentation from the three-dimensional model to obtain a first building monomer model corresponding to the building to be detected; the preset transformation matrix is a transformation matrix after the point cloud fine registration;
Determining a patrol target based on the first building monomer model, determining a patrol space area based on the patrol target, determining a group of waypoints in the patrol space area, determining the access sequence of the group of waypoints to form a patrol path, performing path smoothing on the patrol path to obtain patrol route parameters, and transmitting the patrol route parameters to the unmanned aerial vehicle so that the unmanned aerial vehicle executes a patrol task.
2. The method according to claim 1, characterized in that the method further comprises:
Obtaining an orthographic image based on the two-dimensional reconstruction of the image data, and dividing the orthographic image to obtain a division result, wherein the division result at least comprises a two-dimensional image of the building to be detected;
Projecting the outer surface of the BIM to an XY plane to obtain first plane profile data; determining second plane contour data of the building to be detected based on the first plane contour data and the segmentation result;
Dividing the three-dimensional model based on the second plane contour data to obtain a second building monomer model corresponding to the building to be detected; merging the second building monomer model and the first building monomer model to obtain a final building monomer model;
And replacing the first building monomer model in the step of determining a patrol goal based on the first building monomer model with the final building monomer model.
3. The method according to claim 2, characterized in that the method further comprises:
obtaining DSM data of a digital surface model corresponding to the building to be detected;
The determining the second plane contour data of the building to be detected based on the first plane contour data and the segmentation result includes:
Registering the DSM data with the segmentation result to obtain a first registration result; registering the first plane contour data with the first registration result to obtain registered second plane contour data;
the step of obtaining a second building monomer model corresponding to the building to be detected by segmentation from the three-dimensional model based on the second plane contour data comprises the following steps:
and projecting the plane contour represented by the second plane contour data into the three-dimensional model, and carrying out given threshold value growth on the projected plane contour on a Z axis to obtain a three-dimensional second bounding box, and dividing the three-dimensional model based on the second bounding box to obtain the second building monomer model.
4. A method according to claim 3, characterized in that the method further comprises:
Registering and fusing the first bounding box and the second bounding box, taking a union of the first bounding box and the second bounding box, and merging the second building monomer model and the first building monomer model to obtain a final building monomer model when determining that the model range represented by the union meets the preset requirement;
Or alternatively
Acquiring single-layer outline data of the BIM, wherein the single-layer outline data represent the outline shape of any single floor of the building to be detected;
and projecting the plane outline represented by the single-layer outline data to an XY plane, and scaling according to a preset proportion to obtain a single-layer bounding box, and dividing the final building monomer model based on the single-layer bounding box to obtain a single-layer model of each floor.
5. The method of any one of claims 1-4, wherein the coarse registration of the first and second point cloud data based on the sampling consistency initial registration algorithm comprises:
calculating fast point characteristic histogram (FPFH) descriptors of the first point cloud data and the second point cloud data, and matching point clouds between the first point cloud data and the second point cloud data based on the FPFH descriptors;
randomly selecting at least three pairs of matched point clouds from all matched point clouds, calculating a transformation matrix based on the at least three pairs of matched point clouds, and repeating the steps until the error value meets a preset condition, and outputting a target transformation matrix;
And performing coarse registration on the first point cloud data and the second point cloud data based on the target transformation matrix.
6. The method according to any one of claims 1 to 4, wherein determining a patrol target based on the first building monomer model, and determining a patrol space area based on the patrol target, comprises:
receiving input inspection parameters, and dividing and determining corresponding inspection targets from the first building monomer model based on the inspection parameters; the inspection parameters comprise identification information of inspection targets, different inspection parameters correspond to different inspection targets, and the inspection targets are at least partial areas in the first building monomer model;
And expanding the normal vector on the surface of the inspection target to obtain an initial inspection space region, discretizing the initial inspection space region based on a voxelized mode, and then downsampling the discretized initial inspection space region by adopting a farthest point sampling method to obtain the inspection space region.
7. The method according to any one of claims 1 to 4, wherein determining a set of waypoints in the patrol space area, determining an access order of the set of waypoints to form a patrol path, comprises:
Calculating a group of waypoints required by inspection through a greedy algorithm, and determining the visible range of each waypoint to be in the waypoint course;
based on the surrounding environment information of the inspection target, searching a flight path among the group of waypoints by adopting an A-type algorithm, and further obtaining a distance matrix of the group of waypoints; wherein the surrounding environment information includes at least obstacle position information;
And calculating the access sequence of the group of waypoints by adopting a simulated annealing algorithm according to the distance matrix so as to form a patrol path.
8. The utility model provides a job site unmanned aerial vehicle intelligence inspection route planning system which characterized in that includes:
The data processing module is used for acquiring image data which is acquired by the unmanned aerial vehicle and contains the building to be detected and the environmental information, and obtaining a three-dimensional model based on three-dimensional reconstruction of the image data; acquiring a building information model BIM of the building to be detected, converting the outer surface of the BIM into first point cloud data, and converting the three-dimensional model into second point cloud data;
The point cloud registration module is used for carrying out point cloud rough registration on the first point cloud data and the second point cloud data based on a sampling consistency initial registration algorithm, and then carrying out fine registration on a point cloud rough registration result by adopting a fine registration algorithm;
The model segmentation module is used for determining a first bounding box of the outer surface represented by the first point cloud data, and rotationally importing the first bounding box into the three-dimensional model through a preset transformation matrix so as to realize segmentation from the three-dimensional model to obtain a first building monomer model corresponding to the building to be detected; the preset transformation matrix is a transformation matrix after the point cloud fine registration;
And the inspection execution module is used for determining an inspection target based on the first building monomer model, determining an inspection space region based on the inspection target, determining a group of waypoints in the inspection space region, determining the access sequence of the group of waypoints to form an inspection path, performing path smoothing on the inspection path to obtain inspection route parameters, and transmitting the inspection route parameters to the unmanned aerial vehicle so as to enable the unmanned aerial vehicle to execute an inspection task.
9. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the method for intelligent inspection route planning for a job site unmanned aerial vehicle according to any one of claims 1 to 7.
10. An electronic device, comprising:
A processor; and
A memory for storing a computer program;
Wherein the processor is configured to perform the job site unmanned aerial vehicle intelligent patrol route planning method of any one of claims 1 to 7 via execution of the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410846012.XA CN118424295B (en) | 2024-06-27 | 2024-06-27 | Method and system for intelligent routing inspection route planning of unmanned aerial vehicle at construction site |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410846012.XA CN118424295B (en) | 2024-06-27 | 2024-06-27 | Method and system for intelligent routing inspection route planning of unmanned aerial vehicle at construction site |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118424295A true CN118424295A (en) | 2024-08-02 |
| CN118424295B CN118424295B (en) | 2024-10-11 |
Family
ID=92310489
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410846012.XA Active CN118424295B (en) | 2024-06-27 | 2024-06-27 | Method and system for intelligent routing inspection route planning of unmanned aerial vehicle at construction site |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118424295B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112365369A (en) * | 2020-11-09 | 2021-02-12 | 南京工程学院 | Method for automatically monitoring construction progress based on machine vision |
| CN113433971A (en) * | 2021-07-09 | 2021-09-24 | 深圳大学 | Method, device, equipment and storage medium for acquiring data of high-rise building exterior wall |
| CN116697960A (en) * | 2023-05-31 | 2023-09-05 | 中国矿业大学 | Building exterior wall hollowing information estimation method and system based on infrared thermal imaging |
| CN117994484A (en) * | 2024-03-13 | 2024-05-07 | 沈阳建筑大学 | Device and method for splicing three-dimensional images of prefabricated parts of large building |
-
2024
- 2024-06-27 CN CN202410846012.XA patent/CN118424295B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112365369A (en) * | 2020-11-09 | 2021-02-12 | 南京工程学院 | Method for automatically monitoring construction progress based on machine vision |
| CN113433971A (en) * | 2021-07-09 | 2021-09-24 | 深圳大学 | Method, device, equipment and storage medium for acquiring data of high-rise building exterior wall |
| CN116697960A (en) * | 2023-05-31 | 2023-09-05 | 中国矿业大学 | Building exterior wall hollowing information estimation method and system based on infrared thermal imaging |
| CN117994484A (en) * | 2024-03-13 | 2024-05-07 | 沈阳建筑大学 | Device and method for splicing three-dimensional images of prefabricated parts of large building |
Non-Patent Citations (1)
| Title |
|---|
| 杜甘霖: "无人机航测技术在城市违法建筑变化监测中的应用", 工程勘察, no. 05, 26 April 2018 (2018-04-26), pages 64 - 67 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118424295B (en) | 2024-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7179186B2 (en) | OBJECT DETECTION METHOD, APPARATUS, ELECTRONIC DEVICE, AND COMPUTER PROGRAM | |
| US20220028163A1 (en) | Computer Vision Systems and Methods for Detecting and Modeling Features of Structures in Images | |
| AU2018212700B2 (en) | Apparatus, method, and system for alignment of 3D datasets | |
| US11222204B2 (en) | Creation of a 3D city model from oblique imaging and lidar data | |
| CN108509820B (en) | Obstacle segmentation method and device, computer equipment and readable medium | |
| CN108470174B (en) | Obstacle segmentation method and device, computer equipment and readable medium | |
| CN113724388B (en) | High-precision map generation method, device, equipment and storage medium | |
| CN115147798A (en) | Method, model and device for predicting travelable area and vehicle | |
| CN117974899B (en) | Three-dimensional scene display method and system based on digital twinning | |
| CN114092660A (en) | High-precision map generation method and device and vehicle for generating map | |
| CN118424295B (en) | Method and system for intelligent routing inspection route planning of unmanned aerial vehicle at construction site | |
| Singh et al. | Multi-sensor data fusion for accurate surface modeling | |
| CN117036634A (en) | Automatic construction method for three-dimensional scene of smart city | |
| CN115909277A (en) | Obstacle detection method, obstacle detection device, electronic device and computer-readable storage medium | |
| Roman-Rivera et al. | 3D Point Cloud Outliers and Noise Reduction Using Neural Networks | |
| CN118279091B (en) | Intelligent monitoring method and system for construction progress of assembled bridge | |
| Han | Detail-preserving Urban-scale Mesh Simplification | |
| Leroux et al. | Improving 3D Building Segmentation on 3D City Models Through Simulated Data and Contextual Analysis for Building Extraction | |
| CN118657932A (en) | Target detection method and device based on data synthesis, electronic equipment and medium | |
| CN118279486A (en) | Three-dimensional model construction method based on deep learning | |
| CN118247424A (en) | Method, device and equipment for automatically generating urban road and street lamp scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |