CN112484663A - Intelligent visual 3D information acquisition equipment of many angles of rolling - Google Patents

Intelligent visual 3D information acquisition equipment of many angles of rolling Download PDF

Info

Publication number
CN112484663A
CN112484663A CN202011341023.0A CN202011341023A CN112484663A CN 112484663 A CN112484663 A CN 112484663A CN 202011341023 A CN202011341023 A CN 202011341023A CN 112484663 A CN112484663 A CN 112484663A
Authority
CN
China
Prior art keywords
image acquisition
image
angle setting
acquisition device
acquisition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011341023.0A
Other languages
Chinese (zh)
Other versions
CN112484663B (en
Inventor
左忠斌
左达宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianmu Aishi Beijing Technology Co Ltd
Original Assignee
Tianmu Aishi Beijing Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianmu Aishi Beijing Technology Co Ltd filed Critical Tianmu Aishi Beijing Technology Co Ltd
Priority to CN202011341023.0A priority Critical patent/CN112484663B/en
Publication of CN112484663A publication Critical patent/CN112484663A/en
Priority to PCT/CN2021/123707 priority patent/WO2022111104A1/en
Application granted granted Critical
Publication of CN112484663B publication Critical patent/CN112484663B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/24Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • FMECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F16ENGINEERING ELEMENTS AND UNITS; GENERAL MEASURES FOR PRODUCING AND MAINTAINING EFFECTIVE FUNCTIONING OF MACHINES OR INSTALLATIONS; THERMAL INSULATION IN GENERAL
    • F16MFRAMES, CASINGS OR BEDS OF ENGINES, MACHINES OR APPARATUS, NOT SPECIFIC TO ENGINES, MACHINES OR APPARATUS PROVIDED FOR ELSEWHERE; STANDS; SUPPORTS
    • F16M11/00Stands or trestles as supports for apparatus or articles placed thereon ; Stands for scientific apparatus such as gravitational force meters
    • F16M11/02Heads
    • F16M11/04Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand
    • F16M11/043Allowing translations
    • F16M11/046Allowing translations adapted to upward-downward translation movement
    • FMECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F16ENGINEERING ELEMENTS AND UNITS; GENERAL MEASURES FOR PRODUCING AND MAINTAINING EFFECTIVE FUNCTIONING OF MACHINES OR INSTALLATIONS; THERMAL INSULATION IN GENERAL
    • F16MFRAMES, CASINGS OR BEDS OF ENGINES, MACHINES OR APPARATUS, NOT SPECIFIC TO ENGINES, MACHINES OR APPARATUS PROVIDED FOR ELSEWHERE; STANDS; SUPPORTS
    • F16M11/00Stands or trestles as supports for apparatus or articles placed thereon ; Stands for scientific apparatus such as gravitational force meters
    • F16M11/02Heads
    • F16M11/04Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand
    • F16M11/06Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand allowing pivoting
    • F16M11/12Means for attachment of apparatus; Means allowing adjustment of the apparatus relatively to the stand allowing pivoting in more than one direction
    • FMECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F16ENGINEERING ELEMENTS AND UNITS; GENERAL MEASURES FOR PRODUCING AND MAINTAINING EFFECTIVE FUNCTIONING OF MACHINES OR INSTALLATIONS; THERMAL INSULATION IN GENERAL
    • F16MFRAMES, CASINGS OR BEDS OF ENGINES, MACHINES OR APPARATUS, NOT SPECIFIC TO ENGINES, MACHINES OR APPARATUS PROVIDED FOR ELSEWHERE; STANDS; SUPPORTS
    • F16M11/00Stands or trestles as supports for apparatus or articles placed thereon ; Stands for scientific apparatus such as gravitational force meters
    • F16M11/02Heads
    • F16M11/18Heads with mechanism for moving the apparatus relatively to the stand
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The embodiment of the invention provides visual 3D information acquisition equipment and an acquisition method, wherein the visual 3D information acquisition equipment comprises an image acquisition device, a rotating device, a supporting device and an angle setting device; wherein the rotating device drives the supporting device to rotate; the supporting device is provided with an angle setting device and an image acquisition device; the angle setting device is used for setting a rolling included angle between the image acquisition device and the rotating plane; the method and the equipment which can be suitable for acquiring the 3D information of the outer surface and the inner space of the object are firstly provided. The acquisition position of the camera is optimized by measuring the distance between the rotation center and the target object and the distance between the image sensing element and the target object, so that the speed and the effect of 3D construction are considered.

Description

Intelligent visual 3D information acquisition equipment of many angles of rolling
Technical Field
The invention relates to the technical field of topography measurement, in particular to the technical field of 3D topography measurement.
Background
When performing 3D measurements, it is necessary to first acquire 3D information. Currently common methods include the use of machine vision and structured light, laser ranging, lidar.
Structured light, laser ranging and laser radar all need an active light source to emit to a target object, and can affect the target object under certain conditions, and the light source cost is high. And the light source structure is more accurate, easily damages.
The machine vision mode is to collect the pictures of the object at different angles and match and splice the pictures to form a 3D model, so that the cost is low and the use is easy. When the device collects pictures at different angles, a plurality of cameras can be arranged at different angles of an object to be detected, and the pictures can be collected from different angles through rotation of a single camera or a plurality of cameras. At present, the collected object comprises an outer surface of an object and an inner part of the object, and how to combine the outer surface of the object and the inner part of the object is never mentioned in the prior art and a unified solution is provided. That is, there is no acquisition method or apparatus that can be applied to the outer surface of an object and the inner space of an object.
In addition, in the prior art, it has also been proposed to use an empirical formula including a rotation angle, a target size, and an object distance to define a camera position so as to achieve both a synthesis speed and an effect. However, in practice this has been found to be feasible in wrap-around 3D acquisition, where the target size can be measured in advance. However, it is difficult to measure the target object in advance in an open space, and it is necessary to acquire 3D information of streets, traffic intersections, building groups, tunnels, traffic flows, and the like (not limited thereto). Which makes this approach difficult to work. Even if the dimensions of fixed, small objects, such as furniture, human body parts, etc., can be measured beforehand, this method is still subject to major limitations: the size of the target is difficult to accurately determine, and particularly, the target needs to be frequently replaced in certain application occasions, each measurement brings a large amount of extra workload, and professional equipment is needed to accurately measure irregular targets. The measured error causes the camera position setting error, thereby influencing the acquisition and synthesis speed and effect; accuracy and speed need to be further improved.
Therefore, there is an urgent need for a method and apparatus for acquiring 3D information of an object, which is accurate, efficient, and convenient, and can be applied to the external surface and the internal surface of the object.
Disclosure of Invention
In view of the above, the present invention has been made to provide a visual 3D information collecting apparatus that overcomes or at least partially solves the above-mentioned problems.
The embodiment of the invention provides visual 3D information acquisition equipment and an acquisition method, wherein the visual 3D information acquisition equipment comprises an image acquisition device, a rotating device, a supporting device and an angle setting device;
wherein the rotating device drives the supporting device to rotate;
the supporting device is provided with an angle setting device and an image acquisition device;
the angle setting device is used for setting a rolling included angle between the image acquisition device and the rotating plane;
the included angle alpha of the optical axes of two adjacent acquisition positions of the image acquisition device in the rotation process meets the following condition:
Figure BDA0002798606590000021
wherein, R is the distance from the rotation center to the surface of the target object, T is the sum of the object distance and the image distance during acquisition, d is the length or the width of a photosensitive element of the image acquisition device, F is the focal length of a lens of the image acquisition device, and u is an empirical coefficient.
In alternative embodiments u <0.498, preferably u <0.411, particularly preferably u <0.359, u <0.250, or u <0.216, or u <0.197, or u <0.055, or u < 0.028.
In an alternative embodiment, the set included angle may be in a range of 0 ° to 180 °, 90 ° to-90 °, and 0 ° to-180 °.
In an alternative embodiment, the image capture device is translated in relation to the plane of rotation.
In alternative embodiments, the set angle is 20 °, 40 °, 60 °, 80 °, 90 °, 100 °, 120 °, 160 °, 180 °. .
In an alternative embodiment, the angle setting device is an adjustable angle setting device.
In an alternative embodiment, the angle setting device is a fixed angle setting device.
In an alternative embodiment, the support means comprise a translation unit such that the image acquisition means are offset with respect to the centre of rotation of the device. .
In an alternative embodiment, the support means is such that the image capture means is located at any point in space and is offset from the centre of rotation of the apparatus.
The embodiment of the invention also provides a 3D synthesis/identification device and a method, which comprise any one of the equipment and the method.
The embodiment of the invention also provides an object manufacturing/displaying device and method, which comprise any one of the equipment and the method.
Invention and technical effects
1. The method and the equipment which can be suitable for acquiring the 3D information of the outer surface and the inner space of the object are firstly provided.
2. The acquisition position of the camera is optimized by measuring the distance between the rotation center and the target object and the distance between the image sensing element and the target object, so that the speed and the effect of 3D construction are considered.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 shows a schematic structural diagram of a 3D information acquisition device provided by an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating an angle setting device of a 3D information acquisition apparatus according to an embodiment of the present invention is set to 20 °;
fig. 3 is a schematic diagram illustrating an angle setting device of a 3D information acquisition apparatus according to an embodiment of the present invention is set to 80 °;
fig. 4 is a schematic structural diagram of a 3D information acquisition device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a 3D information acquisition device according to an embodiment of the present invention;
the correspondence of reference numerals to the various components in the drawings is as follows:
1, an image acquisition device;
2, a rotating device;
3, carrying device;
4 supporting the device;
5 an angle setting device;
6 rods.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
3D information acquisition equipment structure
To solve the above technical problem, an embodiment of the present invention provides an intelligent visual 3D information acquisition device, please refer to fig. 1, which includes an image acquisition device 1, a rotation device 2, a support device 4, an angle setting device 5, and a carrying device 3.
One end of the supporting device 4 is connected with the rotating device 2, and the other end of the supporting device is provided with the image acquisition device 1 through the angle setting device 5, so that the supporting device 4 drives the image acquisition device 1 to rotate around the rotating center under the driving of the rotating device 2. The supporting device 4 may preferably be a telescopic structure, that is, the length of the supporting device 4 is adjustable, so that supporting devices with different lengths can be selected according to the size of the target object or the target area to adapt to the acquisition requirement. Meanwhile, the shape of the supporting device 4 is variable, and the supporting device can extend in three directions of XYZ.
The XY plane is a plane in which the rotating device drives the supporting device and the image acquisition device to rotate, and the direction of the lens of the image acquisition device faces to the Y direction; the long side direction of a CCD or CMOS chip of the image acquisition device is taken as the X direction, and when the CCD or CMOS chip is seen towards the lens, the right side is taken as the X positive direction; the vertical XY plane is upward Z-direction positive direction. The angle setting device 5 is capable of adjusting and setting the direction of the optical pickup port (optical axis p) of the image pickup device 1. The plane on which the rotating device drives the supporting device and the image acquisition device to rotate is called an XY plane, the direction of the lens of the image acquisition device is the Y direction, the long side direction of a CCD or CMOS chip of the image acquisition device is the X direction, and when the lens is seen, the right side of the lens is the X positive direction. The angle setting device can set the rolling angle of the image acquisition device, namely, the image acquisition device can rotate in the XZ direction through the angle setting device. The direction of the image capturing device is 0 ° in the vertical upward Z direction, as shown in fig. 1, and the counterclockwise rolling angle of the image capturing device when viewed from the Y direction is defined as the rolling angle increasing direction.
Referring to fig. 2-3, the angle setting device 5 can make the optical axis p of the optical collection port of the image collection device 1 rotate a certain angle on the rotation plane, for example, 20 °, 40 °, 60 °, 80 °, 90 °, 100 °, 120 °, and 160 °. That is, an included angle, i.e., a set angle, exists between the optical axis of the image capturing device and the rotation plane along the capturing direction of the optical capturing port.
Of course, it is also possible to continue to increase the set angle, for example 180 °, 270 °, etc. However, when the image capturing device is rotated over 180 °, it is actually equivalent to a rotation of less than 180 ° in the opposite direction setting, i.e. a reverse rotation setting.
A rod 6 can be added between the angle setting device and the image acquisition device, as shown in fig. 4, that is, the image acquisition device, the supporting device and the rotating device are not in the same plane, and the distance is large, so that the acquisition range is not blocked. It will be appreciated that the rod 6 may be a straight rod extending in yet another direction, or may be a curved rod extending in any three dimensions in space. In particular, the rod 6 may also be a length-adjustable rod. In addition, please refer to fig. 5, it can also be realized by changing the shape structure of the supporting device. For example, the support means may be provided like an L-shape, i.e. comprising a vertical portion connected to the rotation means and a lateral portion connected to the angle setting means (or a lateral portion connected to the rotation means and a vertical portion connected to the angle setting means). Of course, the support device may have other complicated shapes (e.g., the support device extends along the Z-axis, the Y-axis, and the Z-axis), so that the image capturing device can be located at any position in space and offset from the rotation axis of the apparatus. Similarly, the supporting device may also be a displacement device capable of freely adjusting the position in XYZ axes, so that the image capturing device can translate thereon in XYZ directions.
It can be understood that the upright can be a telescopic device, that is, the extending length of the upright can be adjusted according to actual requirements, so as to adapt to the requirements of different target sizes or collecting spaces. Furthermore, although the carrier 3 is shown in the figures as being used to carry the entire apparatus at its lowermost level, it will be appreciated that the entire apparatus may be used upside down. Of course, the whole device is not necessarily used vertically, and the whole device is rotated by 90 degrees so as to be used horizontally, or the device can be used at any inclination angle and can be selected according to actual needs.
The angle setting device is an adjustable rotating device and is locked after rotating to a set angle. The adjustable rotating device can be manually adjusted or can be automatically adjusted electrically. Meanwhile, the device can also be a fixed angle device, namely the direction of the optical axis naturally meets the set angle after the image acquisition device is installed on the device, and the device is fixed and cannot be adjusted. Therefore, the set angle does not need to be adjusted every time in some known fixed use occasions, and inaccuracy caused by repeated adjustment is avoided.
Of course, the rotation shaft of the rotation device may also be connected to the image capturing device through a reduction device, such as a gear set. When the image capturing device makes a 360 ° rotation in the horizontal plane, it captures an image of the corresponding object at a specific position (the specific capturing position will be described later in detail). The shooting can be performed synchronously with the rotation action, or shooting can be performed after the rotation of the shooting position is stopped, and the rotation is continued after the shooting is finished, and the like. The rotating device can be a motor, a stepping motor, a servo motor, a micro motor and the like. The rotating device (e.g., various motors) can rotate at a prescribed speed under the control of the controller and can rotate at a prescribed angle, thereby achieving optimization of the acquisition position, which will be described in detail below. Of course, the image acquisition device can be mounted on the rotating device in the existing equipment.
In addition, the supporting device can also realize translation relative to the rotation center in the XY plane, thereby realizing more flexible acquisition.
The bearing device is used for bearing the weight of the whole equipment, and the rotating device 2 is connected with the bearing device 3. The carrying device may be a tripod, a base with a support device, etc. Typically, the rotating means is located in the central part of the carrying means to ensure balance. But in some special cases it can be located anywhere on the carrier. And the carrier is not necessary. The rotating device may be mounted directly in the application, for example, may be mounted on the roof of a vehicle. The carrying device can also be a handheld part, so that the 3D acquisition equipment can be used in a handheld mode.
The 3D information acquisition equipment can also comprise a distance measuring device, the distance measuring device is fixedly connected with the image acquisition device, and the pointing direction of the distance measuring device is the same as the direction of the optical axis of the image acquisition device. Of course, the distance measuring device can also be fixedly connected to the rotating device, as long as the distance measuring device can synchronously rotate along with the image acquisition device. Preferably, an installation platform can be arranged, the image acquisition device and the distance measurement device are both positioned on the platform, and the platform is installed on a rotating shaft of the rotating device and driven to rotate by the rotating device. The distance measuring device can use various modes such as a laser distance measuring instrument, an ultrasonic distance measuring instrument, an electromagnetic wave distance measuring instrument and the like, and can also use a traditional mechanical measuring tool distance measuring device. Of course, in some applications, the 3D acquisition device is located at a specific location, and its distance from the target object is calibrated, without additional measurements.
The 3D information acquisition equipment can also comprise a light source, and the light source can be arranged on the periphery of the image acquisition device, the rotating device and the mounting platform. Of course, the light source may be separately provided, for example, a separate light source may be used to illuminate the target. Even when the lighting conditions are good, no light source is used. The light source can be an LED light source or an intelligent light source, namely, the light source parameters are automatically adjusted according to the conditions of the target object and the ambient light. Usually, the light sources are distributed around the lens of the image capturing device 1, for example, the light sources are ring-shaped LED lamps around the lens. Since in some applications it is desirable to control the intensity of the light source. In particular, a light softening means, for example a light softening envelope, may be arranged in the light path of the light source. Or the LED surface light source is directly adopted, so that the light is soft, and the light is more uniform. Preferably, an OLED light source can be adopted, the size is smaller, the light is softer, and the flexible OLED light source has the flexible characteristic and can be attached to a curved surface.
In order to facilitate the actual size measurement of the target object, a plurality of marking points can be arranged at the position of the target object. And the coordinates of these marked points are known. The absolute size of the 3D synthetic model is obtained by collecting the mark points and combining the coordinates thereof. These marking points may be previously set points or may be laser light spots. The method of determining the coordinates of the points may comprise: using laser to measure distance: and emitting laser towards the target object by using the calibration device to form a plurality of calibration point light spots, and obtaining the coordinates of the calibration points through the known position relation of the laser ranging units in the calibration device. And emitting laser towards the target by using the calibration device, so that the light beam emitted by the laser ranging unit in the calibration device falls on the target to form a light spot. Since the laser beams emitted from the laser ranging units are parallel to each other, the positional relationship between the respective units is known. The two-dimensional coordinates in the emission plane of the plurality of light spots formed on the target object can be obtained. The distance between each laser ranging unit and the corresponding light spot can be obtained by measuring the laser beam emitted by the laser ranging unit, namely the depth information equivalent to a plurality of light spots formed on the target object can be obtained. I.e. the depth coordinate perpendicular to the emission plane, can be obtained. Thereby, three-dimensional coordinates of each spot can be obtained. Secondly, distance measurement and angle measurement are combined: and respectively measuring the distances of the plurality of mark points and the included angles between the mark points, thereby calculating respective coordinates. Using other coordinate measuring tools: such as RTK, global coordinate positioning systems, satellite-sensitive positioning systems, position and pose sensors, etc.
3D information acquisition process
The rotating device drives the image acquisition device to rotate at a certain speed, and the image acquisition device acquires images at a set position in the rotating process. At the moment, the rotation can not be stopped, namely, the image acquisition and the rotation are synchronously carried out; or stopping rotation at the position to be acquired, acquiring images, and continuing to rotate to the next position to be acquired after acquisition is finished. The rotating means may be driven by a program in a control unit set in advance. The device can also communicate with an upper computer through a communication interface, and the rotation is controlled through the upper computer. Particularly, the rotating device can be connected with a mobile terminal in a wired or wireless mode, and the rotating device is controlled to rotate through the mobile terminal (such as a mobile phone). The rotating device can set rotating parameters through the remote platform, the cloud platform, the server, the upper computer and the mobile terminal, and the rotating start and stop of the rotating device are controlled.
The image acquisition device acquires a plurality of images of the target object, sends the images to a remote platform, a cloud platform, a server, an upper computer and/or a mobile terminal through the communication device, and carries out 3D synthesis on the target object by using a 3D model synthesis method.
In particular, the distance measuring device may be used to measure the corresponding distance parameters in the relevant formula conditions, i.e. the distance from the center of rotation to the target object and the distance from the sensor element to the target object, before or simultaneously with the acquisition. And calculating the acquisition position according to a corresponding condition formula, and prompting a user to set rotation parameters or automatically setting the rotation parameters.
When the distance measurement is carried out before the collection, the rotating device can drive the distance measurement device to rotate, so that the two distances at different positions can be measured. And respectively averaging two distances measured by a plurality of measuring points, and taking the average value as a uniform distance value acquired at this time to be introduced into a formula. The average value can be obtained by using a sum average, a weighted average, other averaging methods, or a method of discarding outliers and then averaging.
When distance measurement is carried out in the acquisition process, the rotating device rotates to the first position to carry out image acquisition, the two distance values are measured at the same time, the two distance values are brought into a condition formula to calculate the interval angle, and the next acquisition position is determined according to the angle.
Optimization of camera position
In order to ensure that the device can give consideration to the effect and efficiency of 3D synthesis, the method can be used for optimizing the acquisition position of the camera besides the conventional method for optimizing the synthesis algorithm. Especially in the case of 3D acquisition synthesis devices in which the acquisition direction of the camera and the direction of its axis of rotation deviate from each other, the prior art does not mention how to perform a better optimization of the camera position for such devices. Even if some optimization methods exist, they are different empirical conditions obtained under different experiments. In particular, some existing position optimization methods require obtaining the size of the target, which is feasible in the wrap-around 3D acquisition, and can be measured in advance. However, it is difficult to measure in advance in an open space. It is therefore desirable to propose a method that can be adapted to camera position optimization when the acquisition direction of the camera of the 3D acquisition composition device and its rotation axis direction deviate from each other. This is the problem to be solved by the present invention, and a technical contribution is made.
For this reason, the present invention has performed a large number of experiments, and it is concluded that an empirical condition that the interval of camera acquisition is preferably satisfied when acquisition is performed is as follows.
When 3D acquisition is carried out, the included angle alpha of the optical axis of the image acquisition device at two adjacent positions meets the following condition:
Figure BDA0002798606590000081
wherein,
r is the distance from the center of rotation to the surface of the target,
t is the sum of the object distance and the image distance during acquisition, namely the distance between the photosensitive unit of the image acquisition device and the target object.
d is the length or width of a photosensitive element (CCD) of the image acquisition device, and when the two positions are along the length direction of the photosensitive element, the length of the rectangle is taken as d; when the two positions are along the width direction of the photosensitive element, d takes a rectangular width.
And F is the focal length of the lens of the image acquisition device.
u is an empirical coefficient.
Usually, a distance measuring device, for example a laser distance meter, is arranged on the acquisition device. The optical axis of the distance measuring device is parallel to the optical axis of the image acquisition device, so that the distance from the acquisition device to the surface of the target object can be measured, and R and T can be obtained according to the known position relation between the distance measuring device and each part of the acquisition device by using the measured distance.
When the image acquisition device is at any one of the two positions, the distance from the photosensitive element to the surface of the target object along the optical axis is taken as T. In addition to this method, multiple averaging or other methods can be used, the principle being that the value of T should not deviate from the sum of the image distances from the object at the time of acquisition.
Similarly, when the image pickup device is in any one of the two positions, the distance from the rotation center to the surface of the object along the optical axis is defined as R. In addition to this method, multiple averaging or other methods can be used, with the principle that the value of R should not deviate from the radius of rotation at the time of acquisition.
In general, the size of an object is adopted as a method for estimating the position of a camera in the prior art. Since the object size will vary with the measurement object. For example, when a large object is acquired 3D information and then a small object is acquired, the size needs to be measured again and reckoning needs to be performed again. The inconvenient measurement and the repeated measurement bring errors in measurement, thereby causing errors in camera position estimation. According to the scheme, the experience conditions required to be met by the position of the camera are given according to a large amount of experimental data, and the size of an object does not need to be directly measured. In the empirical condition, d and F are both fixed parameters of the camera, and corresponding parameters can be given by a manufacturer when the camera and the lens are purchased without measurement. R, T is only a straight line distance that can be easily measured by conventional measuring methods such as a ruler and a laser rangefinder. Meanwhile, in the apparatus of the present invention, the capturing direction of the image capturing device (e.g., camera) and the direction of the rotation axis thereof are away from each other, that is, the lens is oriented substantially opposite to the rotation center. At the moment, the included angle alpha of the optical axis for controlling the image acquisition device to perform twice positions is easier, and only the rotation angle of the rotary driving motor needs to be controlled. Therefore, it is more reasonable to use α to define the optimal position. Therefore, the empirical formula of the invention enables the preparation process to be convenient and fast, and simultaneously improves the arrangement accuracy of the camera position, so that the camera can be arranged in an optimized position, thereby simultaneously considering the 3D synthesis precision and speed.
According to a number of experiments, to ensure the speed and effect of the synthesis, u should be less than 0.498, for better synthesis effect, u is preferably <0.411, especially preferably u <0.359, in some applications u <0.250, or u <0.216, or u <0.197, or u <0.055, or u < 0.028.
Experiments were carried out using the apparatus of the invention, and some experimental data are shown below, in mm. (the following data are given by way of example only)
When the camera is set to have an angle of 0,
Figure BDA0002798606590000091
when the camera setting angle is not 0
Figure BDA0002798606590000101
Figure BDA0002798606590000111
Figure BDA0002798606590000121
The above data are obtained by experiments for verifying the conditions of the formula, and do not limit the invention. Without these data, the objectivity of the formula is not affected. Those skilled in the art can adjust the equipment parameters and the step details as required to perform experiments, and obtain other data which also meet the formula conditions.
3D model synthesis method
A plurality of images acquired by the image acquisition device are sent to the processing unit, and a 3D model is constructed by using the following algorithm. The processing unit can be located in the acquisition equipment or remotely, such as a cloud platform, a server, an upper computer and the like.
The specific algorithm mainly comprises the following steps:
step 1: and performing image enhancement processing on all input photos. The contrast of the original picture is enhanced and simultaneously the noise suppressed using the following filters.
Figure BDA0002798606590000122
In the formula: g (x, y) is the gray value of the original image at (x, y), f (x, y) is the gray value of the original image at the position after being enhanced by the Wallis filter, and mgIs the local gray average value, s, of the original imagegIs the local standard deviation of gray scale of the original image, mfFor the transformed image local gray scale target value, sfThe target value of the standard deviation of the local gray scale of the image after transformation. c belongs to (0, 1) as the expansion constant of the image variance, and b belongs to (0, 1) as the image brightness coefficient constant.
The filter can greatly enhance image texture modes of different scales in an image, so that the quantity and the precision of feature points can be improved when the point features of the image are extracted, and the reliability and the precision of a matching result are improved in photo feature matching.
Step 2: and extracting feature points of all input photos, and matching the feature points to obtain sparse feature points. And extracting and matching feature points of the photos by adopting a SURF operator. The SURF feature matching method mainly comprises three processes of feature point detection, feature point description and feature point matching. The method uses a Hessian matrix to detect characteristic points, a Box filter (Box Filters) is used for replacing second-order Gaussian filtering, an integral image is used for accelerating convolution to improve the calculation speed, and the dimension of a local image characteristic descriptor is reduced to accelerate the matching speed. The method mainly comprises the steps of firstly, constructing a Hessian matrix, generating all interest points for feature extraction, and constructing the Hessian matrix for generating stable edge points (catastrophe points) of an image; secondly, establishing scale space characteristic point positioning, comparing each pixel point processed by the Hessian matrix with 26 points in a two-dimensional image space and a scale space neighborhood, preliminarily positioning a key point, filtering the key point with weak energy and the key point with wrong positioning, and screening out a final stable characteristic point; and thirdly, determining the main direction of the characteristic points by adopting the harr wavelet characteristics in the circular neighborhood of the statistical characteristic points. In a circular neighborhood of the feature points, counting the sum of horizontal and vertical harr wavelet features of all points in a sector of 60 degrees, rotating the sector at intervals of 0.2 radian, counting the harr wavelet feature values in the region again, and taking the direction of the sector with the largest value as the main direction of the feature points; and fourthly, generating a 64-dimensional feature point description vector, and taking a 4-by-4 rectangular region block around the feature point, wherein the direction of the obtained rectangular region is along the main direction of the feature point. Each subregion counts haar wavelet features of 25 pixels in both the horizontal and vertical directions, where both the horizontal and vertical directions are relative to the principal direction. The haar wavelet features are in 4 directions of the sum of the horizontal direction value, the vertical direction value, the horizontal direction absolute value and the vertical direction absolute value, and the 4 values are used as feature vectors of each sub-block region, so that a total 4 x 4-64-dimensional vector is used as a descriptor of the Surf feature; and fifthly, matching the characteristic points, wherein the matching degree is determined by calculating the Euclidean distance between the two characteristic points, and the shorter the Euclidean distance is, the better the matching degree of the two characteristic points is.
And step 3: inputting matched feature point coordinates, resolving the sparse three-dimensional point cloud of the target object and the position and posture data of the photographing camera by using a light beam method adjustment, namely obtaining model coordinate values of the sparse three-dimensional point cloud of the target object model and the position; and performing multi-view photo dense matching by taking the sparse feature points as initial values to obtain dense point cloud data. The process mainly comprises four steps: stereo pair selection, depth map calculation, depth map optimization and depth map fusion. For each image in the input data set, we select a reference image to form a stereo pair for use in computing the depth map. Therefore, we can get rough depth maps of all images, which may contain noise and errors, and we use its neighborhood depth map to perform consistency check to optimize the depth map of each image. And finally, carrying out depth map fusion to obtain the three-dimensional point cloud of the whole scene.
And 4, step 4: and reconstructing the curved surface of the target object by using the dense point cloud. The method comprises the steps of defining an octree, setting a function space, creating a vector field, solving a Poisson equation and extracting an isosurface. And obtaining an integral relation between the sampling point and the indicating function according to the gradient relation, obtaining a vector field of the point cloud according to the integral relation, and calculating the approximation of the gradient field of the indicating function to form a Poisson equation. And (3) solving an approximate solution by using matrix iteration according to a Poisson equation, extracting an isosurface by adopting a moving cube algorithm, and reconstructing a model of the measured point cloud.
And 5: full-automatic texture mapping of object models. And after the surface model is constructed, texture mapping is carried out. The main process comprises the following steps: texture data is obtained to reconstruct a surface triangular surface grid of a target through an image; and secondly, reconstructing the visibility analysis of the triangular surface of the model. Calculating a visible image set and an optimal reference image of each triangular surface by using the calibration information of the image; and thirdly, clustering the triangular surface to generate a texture patch. Clustering the triangular surfaces into a plurality of reference image texture patches according to the visible image set of the triangular surfaces, the optimal reference image and the neighborhood topological relation of the triangular surfaces; and fourthly, automatically sequencing the texture patches to generate texture images. And sequencing the generated texture patches according to the size relationship of the texture patches to generate a texture image with the minimum surrounding area, and obtaining the texture mapping coordinate of each triangular surface.
It should be noted that the above algorithm is an algorithm used by the present invention, and the algorithm is matched with the image acquisition condition, and the time and quality of the synthesis are considered by using the algorithm. It will be appreciated that conventional 3D synthesis algorithms known in the art may be used with the solution of the invention.
Examples of the applications
In order to construct a 3D model of the engine as a whole, 3D modeling of its outer surface and inside of its internal cavity is required. When modeling the outer surface, the set angle is 180 degrees, and a plurality of images are acquired by rotating and collecting. When the inner cavity surface modeling is carried out, the set angle is 10 degrees, the equipment with the miniature camera arranged on the rod part is stretched into the inner cavity to take a picture through rotation, and a plurality of images are obtained. And synthesizing the 3D model of the outer surface and the inner cavity of the engine through the two parts of images, thereby realizing the quality inspection of the inner cavity of the engine.
The target object, and the object all represent objects for which three-dimensional information is to be acquired. The object may be a solid object or a plurality of object components. For example, buildings, parts, etc. The 3D information of the target object comprises a three-dimensional image, a three-dimensional point cloud, a three-dimensional grid, a local three-dimensional feature, a three-dimensional size and all parameters with the three-dimensional feature of the target object. Three-dimensional in the present invention means having XYZ three-direction information, particularly depth information, and is essentially different from only two-dimensional plane information. It is also fundamentally different from some definitions, which are called three-dimensional, panoramic, holographic, three-dimensional, but actually comprise only two-dimensional information, in particular not depth information.
The capture area in the present invention refers to a range in which an image capture device (e.g., a camera) can capture an image. The image acquisition device can be a CCD, a CMOS, a camera, a video camera, an industrial camera, a monitor, a camera, a mobile phone, a tablet, a notebook, a mobile terminal, a wearable device, intelligent glasses, an intelligent watch, an intelligent bracelet and all devices with image acquisition functions.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It will be appreciated by those skilled in the art that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components in an apparatus in accordance with embodiments of the present invention. The present invention may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.

Claims (11)

1. A visual 3D information acquisition device and an acquisition method are characterized in that: comprises an image acquisition device, a rotating device, a supporting device and an angle setting device;
wherein the rotating device drives the supporting device to rotate;
the supporting device is provided with an angle setting device and an image acquisition device;
the angle setting device is used for setting a rolling included angle between the image acquisition device and the rotating plane;
the included angle alpha of the optical axes of two adjacent acquisition positions of the image acquisition device in the rotation process meets the following condition:
Figure FDA0002798606580000011
wherein, R is the distance from the rotation center to the surface of the target object, T is the sum of the object distance and the image distance during acquisition, d is the length or the width of a photosensitive element of the image acquisition device, F is the focal length of a lens of the image acquisition device, and u is an empirical coefficient.
2. The apparatus of claim 1, wherein: u <0.498, preferably u <0.411, particularly preferably u <0.359, u <0.250, or u <0.216, or u <0.197, or u <0.055, or u < 0.028.
3. The apparatus of claim 1, wherein: the set included angle can be in the range of 0-180 degrees, 90-90 degrees and 0-180 degrees.
4. The apparatus of claim 1, wherein: the image acquisition device translates relative to the rotation plane.
5. The apparatus of claim 1, wherein: the set angles are 20 °, 40 °, 60 °, 80 °, 90 °, 100 °, 120 °, 160 °, 180 °.
6. The apparatus of claim 1, wherein: the angle setting device is an adjustable angle setting device.
7. The apparatus of claim 1, wherein: the angle setting device is a fixed angle setting device.
8. The apparatus of claim 1, wherein: the support means comprise a translation unit such that the image acquisition means are offset with respect to the centre of rotation of the device. .
9. The apparatus of claim 1, wherein: the supporting device enables the image acquisition device to be located at any point in space and is deviated from the rotation center of the equipment.
10. A 3D synthesis/identification apparatus and method comprising the apparatus and method of any preceding claim.
11. An object manufacturing/display apparatus and method comprising the apparatus and method of any preceding claim.
CN202011341023.0A 2020-11-25 2020-11-25 Intelligent visual 3D information acquisition equipment of many angles of rolling Active CN112484663B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202011341023.0A CN112484663B (en) 2020-11-25 2020-11-25 Intelligent visual 3D information acquisition equipment of many angles of rolling
PCT/CN2021/123707 WO2022111104A1 (en) 2020-11-25 2021-10-14 Smart visual apparatus for 3d information acquisition from multiple roll angles

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011341023.0A CN112484663B (en) 2020-11-25 2020-11-25 Intelligent visual 3D information acquisition equipment of many angles of rolling

Publications (2)

Publication Number Publication Date
CN112484663A true CN112484663A (en) 2021-03-12
CN112484663B CN112484663B (en) 2022-05-03

Family

ID=74934487

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011341023.0A Active CN112484663B (en) 2020-11-25 2020-11-25 Intelligent visual 3D information acquisition equipment of many angles of rolling

Country Status (2)

Country Link
CN (1) CN112484663B (en)
WO (1) WO2022111104A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114217602A (en) * 2021-10-28 2022-03-22 北京星航机电装备有限公司 Positioning installation device and positioning method
WO2022111104A1 (en) * 2020-11-25 2022-06-02 左忠斌 Smart visual apparatus for 3d information acquisition from multiple roll angles

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014160020A (en) * 2013-02-20 2014-09-04 Nsk Ltd Article shape measurement method and measurement device
CN204963811U (en) * 2015-09-18 2016-01-13 中国科学院紫金山天文台 Measuring device is shot to colourful attitude of low temperature
CN109458958A (en) * 2018-12-21 2019-03-12 中国航空工业集团公司北京航空精密机械研究所 A kind of scaling method of turntable center position in four axis vision measurement device
US20190186907A1 (en) * 2017-12-17 2019-06-20 AP Robotics, LLC. Multi-dimensional measurement system for precise calculation of position and orientation of a dynamic object
CN111292364A (en) * 2020-01-21 2020-06-16 天目爱视(北京)科技有限公司 Method for rapidly matching images in three-dimensional model construction process

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1314000B8 (en) * 2000-08-25 2009-08-12 3Shape A/S Method and apparatus for three-dimensional optical scanning of interior surfaces
CN106989690A (en) * 2017-02-20 2017-07-28 上海大学 Portable non-contact object inner chamber pattern spy testing digitizer
CN208579103U (en) * 2018-07-16 2019-03-05 天目爱视(北京)科技有限公司 A kind of acquisition device of 3D data
CN111076674B (en) * 2019-12-12 2020-11-17 天目爱视(北京)科技有限公司 Closely target object 3D collection equipment
CN211373522U (en) * 2019-12-12 2020-08-28 天目爱视(北京)科技有限公司 Short-distance 3D information acquisition equipment and 3D synthesis, microscopy and attachment manufacturing equipment
CN112484663B (en) * 2020-11-25 2022-05-03 天目爱视(北京)科技有限公司 Intelligent visual 3D information acquisition equipment of many angles of rolling

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014160020A (en) * 2013-02-20 2014-09-04 Nsk Ltd Article shape measurement method and measurement device
CN204963811U (en) * 2015-09-18 2016-01-13 中国科学院紫金山天文台 Measuring device is shot to colourful attitude of low temperature
US20190186907A1 (en) * 2017-12-17 2019-06-20 AP Robotics, LLC. Multi-dimensional measurement system for precise calculation of position and orientation of a dynamic object
CN109458958A (en) * 2018-12-21 2019-03-12 中国航空工业集团公司北京航空精密机械研究所 A kind of scaling method of turntable center position in four axis vision measurement device
CN111292364A (en) * 2020-01-21 2020-06-16 天目爱视(北京)科技有限公司 Method for rapidly matching images in three-dimensional model construction process

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
贾传宝等: "倾斜位置成像的轮对几何参数计算方法", 《传感器与微系统》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022111104A1 (en) * 2020-11-25 2022-06-02 左忠斌 Smart visual apparatus for 3d information acquisition from multiple roll angles
CN114217602A (en) * 2021-10-28 2022-03-22 北京星航机电装备有限公司 Positioning installation device and positioning method

Also Published As

Publication number Publication date
CN112484663B (en) 2022-05-03
WO2022111104A1 (en) 2022-06-02

Similar Documents

Publication Publication Date Title
CN112492292B (en) Intelligent visual 3D information acquisition equipment of free gesture
CN112254670B (en) 3D information acquisition equipment based on optical scanning and intelligent vision integration
CN112361962B (en) Intelligent visual 3D information acquisition equipment of many every single move angles
WO2022078418A1 (en) Intelligent three-dimensional information acquisition appratus capable of stably rotating
CN112254680B (en) Multi freedom&#39;s intelligent vision 3D information acquisition equipment
CN112257537B (en) Intelligent multi-point three-dimensional information acquisition equipment
CN112254675B (en) Space occupancy rate acquisition and judgment equipment and method containing moving object
CN112253913B (en) Intelligent visual 3D information acquisition equipment deviating from rotation center
CN112254638B (en) Intelligent visual 3D information acquisition equipment that every single move was adjusted
CN112082486B (en) Handheld intelligent 3D information acquisition equipment
CN112254676B (en) Portable intelligent 3D information acquisition equipment
CN112484663B (en) Intelligent visual 3D information acquisition equipment of many angles of rolling
CN112254669B (en) Intelligent visual 3D information acquisition equipment of many bias angles
CN112254673B (en) Self-rotation type intelligent vision 3D information acquisition equipment
CN112254678B (en) Indoor 3D information acquisition equipment and method
CN112254677B (en) Multi-position combined 3D acquisition system and method based on handheld device
CN112254671B (en) Multi-time combined 3D acquisition system and method
CN112254653B (en) Program control method for 3D information acquisition
CN112672134B (en) Three-dimensional information acquisition control equipment and method based on mobile terminal
CN112254679B (en) Multi-position combined type 3D acquisition system and method
CN112254674B (en) Close-range intelligent visual 3D information acquisition equipment
CN112254672B (en) Height-adjustable&#39;s intelligent 3D information acquisition equipment
WO2022078417A1 (en) Rotatory intelligent visual 3d information collection device
WO2022078421A1 (en) Multi-pitch-angle intelligent visual 3d information collection device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant