WO2024085836A1

WO2024085836A1 - High-resolution dispersion-based lidar imaging system

Info

Publication number: WO2024085836A1
Application number: PCT/TR2023/051006
Authority: WO
Inventors: Emre Yuce; Koray URKMEN
Original assignee: Orta Dogu Teknik Universitesi
Priority date: 2022-10-21
Filing date: 2023-09-21
Publication date: 2024-04-25

Abstract

The invention relates to a high-resolution dispersion-based LIDAR imaging system.

Description

HIGH-RESOLUTION DISPERSION-BASED LIDAR IMAGING SYSTEM

Technical Field

The present invention relates to a high-resolution dispersion-based LIDAR imaging system.

State of the Art

When we examine the imaging systems in the state of the art, it is seen that there are many different LIDAR types. LIDARs are of great importance for today's technologies where automation has gained great importance, measurement technologies where three-dimensional information is needed and security/defense technologies where depth information can be used as classification and identification feature and different versions are currently used. These can be listed as flash LIDAR, solid state LIDARs, optical phase sequence LIDARs and MEMS based LIDAR etc. In all of them, the principle of illuminating the target with the signal generated by the light source and calculating the time difference (time of flight) between the signal returning from the illuminated target and the first signal, by using the time-sensing electronic circuit. The Time of Flight (TOF) of the time-sensing electronic circuits are detected at the signal level where the returned signal exceeds the signal/noise ratio (SNR) level determined at the level to prevent miscalculations. The time difference between the signal sent and the signal returning from the target defines the distance of the target to the LIDAR system. Basically, as a result of the measurements made at different spatial points, the distances of the different points on the target to the LIDAR system and therefore their relative distances to each other (depth information) can be calculated. In this method, the depth information depends on the rise time of the signal and the time resolution that can be measured with the time-sensing electronic circuit. The depth resolutions of the current LIDAR systems are limited by the rise time of the illumination systems that used, and the use of high-cost illumination sources are required for measurements with high depth resolution. Another problem that can be encountered in current systems is that if there are target points with different depths within the spatially illuminated region, a signal sequence instead of a single signal return from the target and different depths within this spatial resolution cannot be calculated.

Another method that can be used for high depth measurement is the interferometric measurement method. In this method, a phase shift occurs as much as the time of flight of the signal that reaches the target and returns, and this phase difference can be measured by comparing the signal that reaches the target and the output signal with an interferometer system. This method, which can provide high depth measurement at the nm (nanometer) level in the optical wavelength, is limited to the wavelength of the lighting source using the maximum depth information that can be measured. Since the information obtained in depth is given only as multiples of the wavelength, it provides only a relative measurement opportunity. In addition, the use of these systems in the field is limited by their sensitivity.

Another parameter in the examination of current LIDAR systems is the creation of spatial resolution. In the creation of the 3D image, the depth information for each point on the ascent and orientation axes is measured and combined with the position information on the ascent and orientation axes to create the 3D image. If the pulse method is used to measure the depth information for each spot, the depth resolution is limited by the signal rise time of the illumination system and the time resolution that can be measured with sensor electronics.

According to the gathering the location information on the azimuth and elevation axes, LIDARs can be classified under two main systems. These systems are referred to as mechanical and electromagnetic beam guided systems. As mentioned above, sensors are stimulated by the source and time of flight calculations are made. If the light source is directed, these systems operate without mirrors or with the help of moving mirrors. These systems are examples of mechanical systems.

In mechanical guidance systems, lasers or detectors guide with the help of mirrors. However, these systems make space scanning very slow. Solid-state systems, on the other hand, are the beam orientation form obtained as a result of scanning a certain solid angle by one or more micro mirrors. However, due to the high technology product of these systems and therefore the high costs they bring, their spread in civilian applications has been limited. Another method used to derive a 3D image is multi-focal imaging methods. These methods use the focus of points with different depths on the target to focus on different points on the focal plane. The amount of shift that different target depth creates on the focal point depends on the optical magnification feature of the optic and it is possible to provide high resolution depth information with different detectors placed consecutively on the focal plane only in microscopy applications.

As mechanical systems for the end user in the state of the art;

An example of the system that is the subject of the "high-definition LIDAR system" article in the document numbered US7969558 and is also sold on the website. The DMD (digital micromirror device), patented by Texas Instrument company with the number US2017/000339, has taken its place in the literature. DMD's from Texas Instrument, a hybrid system consisting of solid-state and mechanical systems, are used in many projector systems. The production of these systems is very difficult, and the costs are very high, and although it offers high resolution in the uptrend and orientation axis, the depth measurement resolution is limited by the time resolution that can be measured with the pulse rise time and sensor electronics.

Systems that achieve high resolution and high scan rates generally consist of expensive mechanical systems. In systems that solve this problem with optical phase sequences, scanning distance is a problem based on the decrease in optical power.

LIDAR systems, including the systems described in the inventions in US10585173 and CN113795773, basically measure the distance of the target and the different points on the target from the LIDAR system by measuring the return time (time of flight) of a modular or pulsed light source directed to the light. The distance information taken may also include different angular locations within the field of view (FOV) depending on the system design used, or angular location information for point measurement systems can be obtained with scanning systems (including control electronics) such as electronic, mechanical, electromechanical, etc. It is understood that the optical element of the invention US10585173, characterized in that the diffusion feature may be present, is used for modulating the light and not for separating the light into wavelengths. When the above-mentioned Chinese and American patent documents and the distance measurement methods used in the invention are examined; Time of Flight (TOF) measurement is performed for all points on the target and for each different frame. In the patent document numbered US 10585173, it is made in the form of continuous TOF measurement for all points on the target and for each different image frame. In inventions US10585173 and CN113795773, the distance (depth) information of the points on the target is directly or indirectly measured by "time of flight measurement".

In all the above-mentioned systems, one or several laser pulses in series are sent to the target point and the distance from the time of flight of the returned signal is determined. The depth resolution to be obtained from the target area depends on the duration of the pulse sent at the time. If more precise measurements are desired, the impact times should be shortened. Since short pulse lasers have high costs and are large in size, their use in LIDAR applications is limited.

Some of the high-speed versions of LIDAR technologies available on the market use MEMS or optical scanning systems with fast rise time illumination sources. In order to produce these products, clean rooms and high-tech equipment requiring high cost and electronic circuit designs with high measurement accuracy are required. The cost of MEMS based mirrors, for example, is high, making it possible to scan only 20,000 points per second. Similarly, the costs of lighting sources with fast rise times are even higher, and the costs of electronic circuits to calculate time of flight with this source also increase due to the high-speed requirement. The dimensions and weights of these high-capacity systems reflect their costs and constitute a size and weight constraint for the LIDAR system to be created.

As can be understood from the above, the depth measurement resolutions are low with the existing LIDAR systems, and the depth resolutions are limited despite the high-cost fast lighting sources and high-resolution time of flight measurement electronics. In addition, although the existing systems have limited depth resolutions, these systems, which are called high depth resolutions in the current situation, are high cost and limit the areas of use of size and weight limits. Although there are methods to increase the resolution and speed on the horizontal and vertical axis presented by the inventors in other existing documents, it has been noticed that the resolution given by these systems in depth is low and there is a need to develop a new system.

Objects and Brief Description of the Invention

Said invention relates to the high-resolution dispersion-based LIDAR imaging system in order to bring new technical advantages to the above-mentioned field.

The light source used by the invention is not included in the light modulation, time of flight measurement and/or optical scanning systems section, and the innovation of the invention is that a new depth measurement method with a high depth resolution beyond the limitations of time-of-flight measurement is intended for use within the scope of the LIDAR request. The invention offers resolution beyond the resolution limits determined by the laser pulse duration. In addition, the invention differs from the state of the art in that it can be integrated into many systems and offers superior performance at low cost.

The system described in the invention is a system designed to achieve high resolution depth information without measuring multiple times of flight.

The optical system is specifically designed to be sensitive to different wavelengths and is intended to focus light of different wavelengths at focal points as far apart as possible on the focal plane. In this way, the depth information can be obtained at higher resolution.

When the significant differences of said invention are examined:

• In the invention, the time of flight (TOF) measurement method is used only for the first distance measurement. In this way, it is possible to obtain much better resolution than the resolution limited by the impact time.

• In the invention, the depth information of the different points on the target is determined optically using detectors positioned in the image plane converted to the horizontal position by the dispersion system or a linear detector array in addition to the pulse duration. • It is that depth measurement is used directly, not to minimize the errors that may arise from the target reflection of different wavelengths.

• A diffusion based optical element is used to move the positions in the optical plane to the horizontal plane by separating the light into wavelengths.

Cost advantage of the invention:

The design in the invention basically consists of an illuminating, optical system, and sensor block system. The costs of these products are much lower than the high-speed LIDARs available on the market. With the invention, which has the ability to take a full-screen image (full frame) with high depth resolution between 10-60 hz (hertz) per second, the great cost advantage of the invention is clearly seen when the high costs of other products in the market that have a more think depth resolution and can take a slower full-screen image are compared. The advantage of the invention in terms of time, ease of production and size and weight:

With the developed design, it is possible to produce a more compact and lightweight system without the need for a clean room, high-cost equipment, low tolerance and therefore high cost optical system production, high cost illuminator, high cost electronic circuits, high cost scanning systems. Therefore, the design has faster and easier manufacturability at lower cost and can find more application areas with the advantage of size and weight.

When the application areas of the invention are examined, autonomous land and aircraft, robotics, navigation, scanning and alarm systems, security, military imaging systems, military target detection, classification and identification systems, and geodesy and photogrammetry can be listed.

The invention can be used directly in automation technology and can also be used in robotic, security field, medical field and autonomous driving vehicles. In this sense, the application and impact area is fairly wide. LIDAR application areas will increase as smart vehicles and application areas expand.

Another feature of the invention is that it can be integrated into each of the technologies used by the companies using the LIDAR feature. In this sense, it is not a competitor to these products but a method that increases their performance. Therefore, the impact area is much wider than many methods.

Definitions of Figures Describing the Invention

The figures and related descriptions used to better explain the system developed by this invention are as follows.

Figure-la: A high resolution spectral LIDAR system flow diagram for the target at close/medium distance.

Figure-lb: A high resolution spectral LIDAR system for near/mid-range targets. The parts indicated by A, B, C, D, are the protruding parts on the object. It denotes the separation of the beam reflected by > into different wavelengths (D i_; IZI2, □ S . DN)-

Definitions of Components/Pieces/Parts of the Invention

In order to better explain the device developed by this invention, the parts and pieces in the figures are numbered and the corresponding numbers are given below.

1 Optical Axis

100 Light Source

110 Sensor Block

120 Optical Element

130 Optical System

140 Target

150 Analog and/or Digital Control Systems

Detailed Description of the Invention

The invention relates to a high-resolution dispersion-based LIDAR imaging system. The system is described in detail below. It is possible to obtain a high-resolution image by providing the opportunity to measure by separating the colors that are very difficult to be distinguished from each other spatially with dispersion-based optics in the invention in the horizontal or vertical plane.

With the optical determination of the relative depth information in the invention, a system that is free from the depth resolution limitation resulting from the rise time and/or modulation of the pulse (or signal) of the light sources that can be used and the speed experienced by the systems that continuously measure the time of flight on the target is obtained.

The basic elements used in the system are; optical element (120), sensor block (110), optical elements (120), target (140), light source (100). In addition, time of flight measurement is performed with the analog and/or digital control system (150).

As in the current systems, the invention makes distance measurement with the time of flight of time-sensing electronic circuit in determining the first distance of the target to the LIDAR system but uses the focus change in the sensor plane of the different wavelengths in the light returning from the target for the measurement of the relative distances of the points on the target (depth information on the target) to each other.

In the invention, the optical system (130) is specifically designed to be sensitive to different wavelengths, focusing at focal points as far apart as possible on the focal plane of light of different wavelengths. A refractive optical element (120) used in the focal plane separates the light into spectra, moving these different focal points on the focal plane to the horizontal plane and allowing it to fall on different detectors on a sensor block consisting of side-by-side detectors or using the plane matrix detector structure. Therefore, the signals to be obtained from the adjacent detectors and/or pixels belong to different wavelengths and therefore to different depths on the target. With this method, there is no need for a separate signal lighting and time of flight calculation for each spatial point, and there is no need for scanning systems that may be needed for the illumination system, and there is no need to wait for at least the time of flight for each spatial point on the target. In this way, the frame rate of the depth resolution of the LIDAR system is also significantly increased. Since the relative distances of the different spatial points on the target will be measured depending on the wavelength, there is no limitation on the time-of-flight measurement resolution limit (the fastest rise time that can be obtained with the illumination source or the smallest time difference that can be measured with the time-sensing electronic circuit) in determining the relative distances. It is possible to achieve a high depth resolution of the focal point changes transferred to the horizontal plane in line with the measurement resolution.

With the dispersion-based optics used in the invention, it is possible to measure by separating the colors, which are very difficult to distinguish from each other, spatially from each other in the horizontal or vertical plane. It is possible to obtain a high-resolution image by this way. The use of lasers with rapid rise times greatly increases the cost and size of LIDARs. Since a laser with a femtosecond speed is costly, the application of these lasers for LIDARs is limited. By increasing the number of detectors used in the invention, it is possible to obtain high resolution using lasers with much slower pulse time and rise rate. Therefore, the system defined by the invention enables faster and higher depth resolution three-dimensional images to be obtained with a simpler and more cost-effective method compared to existing LIDAR systems.

Based on the detailed information above, the invention is a high-resolution dispersion based LIDAR imaging system used to measure colors that are very difficult to distinguish spatially from dispersion based optics by separating them from each other in a horizontal or vertical plane, characterized in that it comprises the following features:

• At least one light source (100) that illuminates the target (140),

• At least one sensor block (110), which provides the acquisition of the image of the target (140) at different wavelengths by focusing the light, which is divided into wavelengths with its detectors, so that different wavelengths fall on different sensors,

• At least one optical system (130) enabling the calculation of the depth information on the target (140) where the light reflected from the target (140) passes, where the image in each wavelength corresponds, and the collection of the light,

• At least one refractive optical element (120), which ensures that the different focal points on the focal plane are transported to the horizontal plane by separating the light into spectra and that the light falls on different detectors on the sensor block (110), which is used in the focal plane, provides dispersion. The information about the systems specified in Figure la and Figure lb is listed below.

The light source (100) may be one and/or more pulsed laser(s), continuous laser(s), LED(s), or a light source including different wavelengths, etc. This may include optical, electronic systems such as focuser, distributor, scanner, etc. It is also possible to include signal shaping optical/electronic systems such as modulators. In this way, temporal and spatial control over light is ensured.

Said light source (100) comprises one and/or more pulsed laser(s), continuous laser(s), LED(s), or a wide wavelength range.

The sensor block (110) element can be one or more photodiodes, phototransistors, thermal sensors, single photon detectors, avalanche photo detectors, photon counters, sensors with matrix structures, etc. The color filter may include matrix filters, reflective/refractive/permeable optical elements.

The optical element (120) that provides dispersion may be prism, filter, refractive grid, metamaterial, metalens, diffraction optical element, etc. It may vary according to the application.

The optical system (130) may comprise one and/or more refractive and/or reflecting optical elements and may be used for spatial scanning.

The target (140) for which the image is to be taken is shown in the figures.

Analog and/or digital control systems 150 may perform 3D image creation and similar operations with spatial location information such as lighting time, shape, modulation, scanning system control and similar lighting functions, optical system scanning and similar functions, sensor settings and similar functions, control of refractive grid functions, processing of the detected signal, time of flight calculation, calculation of depth information according to the values read on the sensor block 110, detected signals, scanning system location, if any. The computer may include components such as the power supply, drive circuit, reader circuit, special software, etc. Under the control of the analog and/or digital control system (150), the light source (100) illuminates the desired target (140) in a predetermined or undetermined time period. Within the light source (100), there may be other optical components for directing or recovering the beam. The light reflected from the target passes through the optical element(s) (130). The time-of-flight measurement is performed with the analog and/or digital control system (150) using the pulse time of the light source (100) and the time of the returned signal from target to the sensor block (110). The sensor block (110) may consist of single pixel, index, or focal plane matrix type sensors. The distance of the target's reference surface from the time-of- flight measurement to the LIDAR system is determined with the help of an analog and/or digital control system (150). Subsequently, the light collected by the optical system (130) is divided into wavelengths by the optical element (120) by turning from the target (140) as a result of the illumination made by the light source (100) or the natural illumination from the environment. By focusing on the light sensor block (110), which is divided into wavelengths, so that different wavelengths fall on different sensors, the image of the target at different wavelengths is obtained. The illumination and image acquisition process can be repeated for different spatial coordinate positions by a predetermined or undetermined number. The images taken at each wavelength will form an image of a different depth. Using the optical properties of the optical system (130), the depth information on the target at which the image at each wavelength corresponds is calculated. The images obtained for N different wavelengths are processed by the analog and/or digital control system (150) to obtain different depth information on the target. The whole system, light source (100) or optical system (130), sensor block (110), optical elements (120) shown in Figure lb can be rotated with or independently of the whole system. In this way, it can be scanned horizontally or vertically. The whole system described in Figure lb can be produced as a series. Further, the optical system (130), the sensor block (110), the optical elements (120) may be produced as a layered array (layered in succession). Or, for the sensor block (110), spatial scanning may be performed optically and/or electronically using the sensor or sensors of the array sensor or focal plane matrix structure. The optical and/or electronic scan location information controlled by the analog and/or digital control system (150) may be used to create a 3D image together with the depth information calculated by the analog and/or digital control system (150). In said invention, the time of flight (TOF) of the first distance is measured, and the relative depth information of the other points (closer or farther) on the target is obtained much better resolution than the existing systems with the detectors or linear detector array in the image plane converted into horizontal axis with the dispersion system.

In the invention, the relative distance (depth) information of the different points on the target is measured by optical methods. The optical method used in the invention provides a measurement opportunity beyond the limits of the depth measurement resolution of other inventions examined with a simpler and cost-effective system structure.

Since the invention transports the focal shifts in the focal plane to the horizontal plane and allows the image at different focal points to be detected by detector arrays, it also allows the use of multi-focal imaging methods with systems with a smaller optical magnification ratio.

Claims

CLAIMS . A high-resolution dispersion-based LIDAR imaging system used to measure colors, which are very difficult to distinguish, spatially from dispersion-based optics by separating them from each other in a horizontal or vertical plane, characterized in that it comprises the following features:

• At least one light source (100) that illuminates the target (140),

• At least one refractive optical element (120), which ensures that the different focal points on the focal plane are transported to the horizontal plane by separating the light into spectra and that the light falls on different detectors on the sensor block (110), which is used in the focal plane, provides dispersion.

2. The system according to Claim 1, characterized in that said optical system (130) comprises an optical system (130) for providing light of different wavelengths at focal points distant from each other on a focal plane for obtaining depth information at high resolution.

3. The system according to Claim 1, characterized in that it comprises an analog and/or digital control system (150) that performs the processes of creating a 3D image with spatial location information such as lighting time, shape, modulation, scanning system control, lighting functions, optical system scanning functions, sensor settings functions, refractive grid functions, processing the detected signal, detected signals, scanning system location, if any, and determining the distance of the reference surface of the target from the time of flight measurement to the LIDAR system, processing the images obtained for different wavelength and obtaining different depth information on the target.

4. The system according to Claim 1, characterized in that the element of said sensor block (110) is one or more photodiodes, phototransistors, thermal sensors, a single photon detector, a photon counter, a sensor with a matrix structure.

5. The system according to Claim 1 or Claim 4, characterized in that the sensor block (110) comprises a color filter, matrix filters, reflective/refractive/permeable optical elements of the element.

6. The system according to Claim 1, characterized in that said light source (100) comprises one and/or more pulsed laser(s), continuous laser(s), led(s) or a wide wavelength range.

7. The system according to Claim 1 or Claim 6, characterized in that said light source (100) comprises focusing, distributing, scanning optical, electronic systems or modulator, signal shaping optical/electronic systems for providing temporal and spatial control over light.

8. The system according to Claim 1, characterized in that said optical element (120) is a prism, a filter, a refractive grid, a meta-material, a metalens or a diffraction optical element.

9. The system according to Claim 1 or Claim 2, characterized in that said optical system (130) comprises one and/or more refractive and/or reflective optical elements.

10. The system according to Claim 3, characterized in that the analog and/or digital control system (150) comprises computer, power supply, driver circuit, reader circuit, software components.

11. The system according to Claim 1, characterized in that the entire system can be rotated together with or independently of the light source (100) or the optical system (130), the sensor block (110), the optical elements (120) for horizontal and vertical scanning. The system according to Claim 1, characterized in that the optical system (130), the sensor block (110), and the optical elements (120) are a layered series. The system according to Claim 1, characterized in that an array sensor or focal plane matrix sensor or sensors are used in said sensor block (110) for performing spatial scanning optically and/or electronically. The system according to Claim 1 or Claim 4 or Claim 13, characterized in that it has a sensor block (110) consisting of side-by-side detectors or using the plane matrix detector structure.