JP7557753B1 - Distance Detection Device - Google Patents

Abstract

[Problem] To provide a distance detection device that has high distance detection accuracy and azimuth resolution regardless of the distance to an object, is resistant to interference and disturbance between distance detection devices, is small and low-cost, and is integrated with an imaging device. [Solution] The distance detection device has a light source that generates illumination light obtained by subjecting the amplitude modulation of a carrier light to chirp modulation, a light source that generates reference light obtained by subjecting the amplitude modulation of the carrier light to the same chirp modulation as that of the illumination light, the chirp modulation is modulated by a chirp modulation signal, a coherence reduction mechanism that reduces the temporal coherence and spatial coherence of both or one of the carrier lights while maintaining the temporal coherence and spatial coherence of the chirp modulation signal, an imaging optical system that images the reflected light from an object illuminated with the illumination light on a light receiving element, and a multiplexer that multiplexes the reflected light and the reference light, and the light receiving element receives the multiplexed reflected light and reference light, and detects an interference fringe signal generated by heterodyning between the chirp modulation signals of the illumination light and the reference light. [Selected Figure] Figure 3

Description

The present invention relates to a distance detection device for use in autonomous driving, robots, drones, etc.

It is desirable for a distance detection device for autonomous driving to satisfy the following requirements:
- High distance detection accuracy and azimuth resolution ranging from a few centimeters (cm) to several hundred meters (m).
- Resistant to interference and disturbance between distance detection devices.
- It must be small, low-cost, and integrated with an imaging device.

Conventional distance detection devices include stereo cameras, ToF cameras, LiDAR, millimeter wave radar, and ultrasonic sensors (see, for example, Non-Patent Document 1). The advantages and disadvantages of each distance detection device will be described later.

Nobuyoshi Fugono et al., "Millimeter Wave Propagation and Radar Rain Gauges", Quarterly Report of the Radio Research Laboratory, December 1976, pp. 407-426, Vol. 22, No. 121

It is difficult for conventional distance detection devices to simultaneously satisfy all of the above requirements. For this reason, currently, three or four types of distance detection devices are used in addition to an imaging device, compensating for the disadvantages of each.

The present invention has been made in consideration of these circumstances. That is, the object of the present invention is to provide a distance detection device that has high distance detection accuracy and azimuth resolution regardless of the distance to the object, is resistant to interference and disturbance between distance detection devices, is small and low-cost, and can be integrated with an imaging device.

In order to solve the above-mentioned problems and achieve the object, a first aspect of the distance detection device of the present invention is to
a light source for generating illumination light in which chirp modulation is applied to amplitude modulation of carrier light;
a light source for generating a reference light obtained by performing the same chirp modulation as the illumination light on the amplitude modulation of a carrier light;
Chirp modulation is modulated by a chirp modulation signal,
a coherence reduction mechanism that reduces the temporal coherence and the spatial coherence of both or one of the carrier lights while maintaining the temporal coherence and the spatial coherence of the chirp modulation signal;
an imaging optical system that forms an image of reflected light from an object illuminated with illumination light on a light receiving element;
a combiner for combining the reflected light and the reference light,
The light receiving element receives the combined reflected light and reference light, and detects an interference fringe signal generated by heterodyning between the linear chirp modulated signals of the illumination light and the reference light.

Furthermore, in order to solve the above-mentioned problems and achieve the object, a second aspect of the distance detection device of the present invention is
a light source for generating illumination light in which chirp modulation is applied to amplitude modulation of carrier light;
an imaging optical system that forms an image of reflected light from an object illuminated with illumination light on a light receiving element;
Further, the optical fiber has a mixer for detecting an interference fringe signal by multiplying the chirp-modulated electrical signal by the same chirp-modulated signal as the illumination light, or a light-receiving element that also functions as a mixer by direct switching using the same chirp-modulated signal as the illumination light,
The light receiving element detects the reflected light and converts the chirp modulation of the reflected light into an electrical signal.

A third aspect of the distance detection device of the present invention is the distance detection device of the first or second aspect, wherein the light receiving elements are two-dimensionally arranged,
Alternatively, the light receiving elements arranged one-dimensionally are scanned in a direction intersecting the one-dimensional arrangement direction across the image forming surface of the image forming optical system,
Alternatively, a single light receiving element is two-dimensionally scanned across the image forming surface,
Alternatively, the light receiving elements are arranged in a one-dimensional array on the imaging surface, and the directivity of the illumination light in a direction intersecting the one-dimensional array is increased to one-dimensionally scan the object in the direction.
Alternatively, a single light receiving element is disposed on the imaging plane, and the illumination light is increased in directivity to a beam to two-dimensionally scan the object,
A two-dimensional detection mechanism is provided for capturing an image of the object and simultaneously detecting the interference fringe signal in two-dimensional positions.

A fourth aspect of the distance detection device of the present invention is the distance detection device of the third aspect, further comprising a determination mechanism that detects a position of interference light that interferes with detection of the distance to the object from an image captured by the two-dimensional detection mechanism and determines a local position where the distance to the object is detected,
The distance to the local position of the object is detected while avoiding the interfering light.

A fifth aspect of the distance detection device of the present invention is the distance detection device of the third or fourth aspect, further comprising a determination mechanism that determines a position of interference light that interferes with reception of optical wireless communication from an image captured by the two-dimensional detection mechanism, and determines a position of a transmitter of an optical wireless communication device that receives the optical wireless communication,
The optical wireless communication is received while avoiding the interfering light.

By using a method of chirp modulation of the light amplitude, it is possible to obtain high lateral resolution and distance detection accuracy over a wide range of several centimeters to several hundred meters. In addition, by using an imaging element, it is possible to simultaneously detect color images and distance images, making it possible to realize a small, low-cost visual sensor.

The imaging device and distance detection device are both visual sensors essential for autonomous driving. Using the information from these two sensors, artificial intelligence (hereinafter referred to as "AI" or "determination mechanism" as appropriate) performs object detection in the external world and makes autonomous driving decisions. Object detection for autonomous driving identifies objects around the vehicle, detects the size and position or area of objects related to autonomous driving, and detects the relative position, speed, and direction of those objects and the vehicle. The higher the level of autonomous driving, the more important these two sensors become.

This shows the change in the amplitude of the carrier light over time. Indicates the frequency of the chirp modulation. 1 shows a distance detection device according to a first embodiment. 1 shows a configuration of a distance detection device according to a second embodiment. 3 shows a modification of the first embodiment. 4 shows the flow of processing of an electrical signal 43. Shows harmonics. Indicates the frequency of the interference fringe signal. 1 shows the relationship between the coherence of the carrier light and the size of the light source. 1 shows the front configuration of the image sensor. Indicates a situation where multiple objects exist. 1 is a timing chart showing the reading of pixels of a horizontal line of an image sensor; The front configuration of the filter is shown. 13 shows a schematic configuration of a distance detection device according to a modified example of the second embodiment. 13 shows another schematic configuration of the distance detection device according to the modified example of the second embodiment.

The effects of the embodiments and modifications of the present invention are described below. Specific examples will be given to specifically explain the effects of the embodiments and modifications. However, these exemplified embodiments and modifications are merely a part of the examples included in the present invention, and there are many variations in its aspects. Therefore, the present invention is not limited to the exemplified embodiments and modifications.

Before describing the embodiments and variations, the advantages and disadvantages of each distance detection device, including a stereo camera, a ToF camera, LiDAR, a millimeter wave radar, and an ultrasonic sensor, will be explained below.

(Stereo camera)
Distance detection using a stereo camera is the same as human distance perception using binocular vision, and is a triangulation method. The amount of misalignment between images captured by the two cameras is detected locally to obtain a distance image.

The advantage of a stereo camera is that it can detect color images and distance images at the same time. Also, because it is a passive system, there is no interference between the stereo cameras.

The disadvantages of stereo cameras are explained below. With stereo cameras, the amount of deviation in each local area of the images captured by the two cameras is detected by correlation calculations. A distance image is then obtained. As a result, the distance detection accuracy and azimuth resolution vary based on the frequency components in each local area of the image.

For example, if there is little change in brightness in a local area, distance detection accuracy and azimuth resolution will decrease. If there is periodic change in brightness in a local area, false detection will occur. Also, because stereo cameras are passive, their accuracy is low in dark places such as at night.

In principle, stereo cameras require high precision regarding the six-axis spatial position of the two cameras. Therefore, the accuracy of distance detection is prone to degradation due to changes over time and temperature.

In addition, the stereo angle of a stereo camera becomes smaller as the object becomes farther away from the camera. When the stereo angle becomes smaller, the amount of deviation between the two images becomes smaller as a fractional function. This causes the accuracy of distance detection to rapidly decrease.

Increasing the stereo angle to improve distance detection accuracy for distant objects results in a larger camera as a whole. This makes the positional accuracy of the two cameras even more difficult. For this reason, the distance detection accuracy and azimuth resolution of stereo cameras are generally lower than those of other distance detection devices.

To mitigate the above disadvantages, there are active stereo cameras that project highly autocorrelated patterns. However, the accuracy of detecting the distance to distant objects remains low, and interference between the stereo cameras occurs.

A method has also been reported that uses AI to detect distance from footage captured by a single camera. The AI learns large amounts of data on various image information about an object that changes depending on the relative position of the object and the vehicle, such as changes in the object's size, shape, texture, out-of-focus, and overlap due to motion parallax. This is how the method detects the distance to an object.

This method is similar to the spatial perception processing that takes place in the dorsal pathway of the visual cortex to compensate for the fact that distance perception in human binocular vision rapidly declines for distant objects. However, most spatial perception processing is based on triangulation of monocular disparity. For the same reasons as mentioned above, therefore, highly accurate distance detection is difficult. Even with one eye closed, humans can roughly grasp the distance to an object based on past experience. However, driving a car with one eye (monocular) is extremely dangerous, even with a lot of experience.

If it were possible to obtain the necessary distance information from monocular images with experience, humans would not have chosen binoculars during the evolutionary process. Similarly, no matter how much AI learns, it will be difficult to detect highly accurate distance images from information from a single camera.

To operate a vehicle that moves much faster than humans, faster, more accurately, and safely than humans, in place of humans, highly accurate distance detection that far exceeds human vision is required. Due to the disadvantages mentioned above, it is said that there are limitations to the application of stereo cameras to autonomous driving. Furthermore, it can be said that a method of "detecting distance images using AI from a single camera image" is even more difficult.

The ToF camera uses the ToF (time of flight) method for distance detection. In the ToF method, the light intensity of the illumination light is modulated at a constant frequency to illuminate the measurement space. The reflected light from the illuminated object is imaged by the optical system. When the light is received by the image sensor, switching detection is performed at the same frequency for each pixel of the image sensor. From the obtained charge amount, the phase of the signal modulating the reflected light (reflection delay time) is detected for each pixel. A distance image is then obtained.

The advantages of a ToF camera are that it can form an image using an optical system, so it can obtain distance images with high lateral resolution, its distance detection accuracy is consistent regardless of distance, and it is small and low-cost.

The disadvantage of a ToF camera is that the detection speed of the light receiving section of the image sensor is limited, making it difficult to measure close and long distances in terms of the signal-to-noise ratio. Also, in principle, the ToF method is strongly affected by background light (sunlight, etc.), which causes significant interference between ToF cameras.

LiDAR primarily uses the ToF method for distance detection. With LiDAR, distance is measured using the ToF method, while an infrared laser beam with a wavelength of 1550 nm is scanned two-dimensionally to obtain a distance image.

The advantages of LiDAR are that it has high azimuth resolution, and some single light receiving sensors have fast detection speeds, high sensitivity and high signal-to-noise ratios, and because infrared light with a wavelength of 1550 nm is rare in external light, it outperforms ToF cameras in measuring both short and long distances.

In addition, because the infrared laser beam is highly directional, there is a low probability of beams overlapping between LiDARs, making it less susceptible to interference between LiDARs.

The disadvantages of LiDAR are that it requires a two-dimensional scanning mechanism, and the light receiving sensor is expensive, which raises issues in terms of durability and cost. Furthermore, for reasons that will be described later regarding millimeter wave radar, ToF LiDAR is less accurate than chirp modulation millimeter wave radar when it comes to long-distance detection.

Millimeter-wave radar uses a linear chirp modulation method for distance detection. Linear chirp modulation is applied to the millimeter-wave frequency, which is then transmitted and received via an antenna. The received wave is multiplied by a reference signal that has the same chirp modulation as the transmitted wave, and heterodyne detection is performed. The signal is converted into an interference fringe signal (IF signal) with a frequency proportional to the distance to the object. The interference fringe signal is then converted into frequency components using an FFT (fast Fourier transform). The distance to the object is measured by detecting the peak value of the frequency component.

Although millimeter wave radar has low distance resolution for distance detection, its high signal-to-noise ratio allows it to achieve the same level of distance detection accuracy as LiDAR by detecting peak values.

The advantage of millimeter wave radar is its highly autocorrelated coding technique known as chirp modulation. The chirp modulation method improves the power signal-to-noise ratio for distance detection in proportion to the modulation bandwidth and chirp time, and spreads the spectrum of external light when detecting distance. This makes it resistant to jamming.

In addition, the difference in reflectivity and reflection distance makes it possible to detect the distance to an object while excluding unwanted reflections from rain, fog, etc. For this reason, even though the attenuation of millimeter waves and infrared rays with a wavelength of 1550 nm in rain is about the same (Figure 1 of Non-Patent Document 1), millimeter wave radar far surpasses LiDAR in long-distance detection. Other major advantages include the ability to detect the relative speed of an object using simple means and the relatively low cost of the device.

The greatest advantage of millimeter wave radar is that it can apply various jamming suppression technologies developed in radar and communications. Chirp modulation, a highly autocorrelated coding technique, is used, which spreads and suppresses the frequency components of the jamming signal. In addition, by applying a special double coding technique, jamming can be further suppressed, making it possible to avoid intentional jamming (deceptive jamming) in addition to denial jamming.

The disadvantage of millimeter wave radar is that, due to the practical size of the antenna and the wavelength of the radio waves, the azimuth resolution (directivity) is more than two orders of magnitude lower than that of LiDAR, making it difficult to identify small objects or objects that are close by. In particular, at close ranges, it is difficult to avoid clutter from the ground or guardrails due to side lobes.

In addition, there are objects whose reflectivity differs from that of optical images. Also, obtaining a distance image requires mechanical two-dimensional scanning or electronic scanning using an array antenna. Furthermore, when applying millimeter wave radar to various applications, there is a major disadvantage in that design freedom is limited by radio wave protection regulations.

To solve the above disadvantages (issues), it is known to perform distance detection by chirp-modulating the frequency of LiDAR's laser light. This provides azimuth resolution equivalent to that of optical images, and similar to millimeter-wave radar, the chirp modulation method offers various benefits. On the other hand, laser light sources that have a coherence length corresponding to distance detection of several hundred meters and enable highly linear chirp modulation are very large and expensive. And the disadvantage of requiring an expensive highly sensitive sensor and a two-dimensional scanning mechanism remains.

Ultrasonic sensors use a pulse radar method for distance detection. The distance to an object is detected from the time it takes for an ultrasonic pulse to be transmitted and received. The advantages are that it is small and low-cost. However, ultrasonic waves are highly attenuated in the air and have low lateral resolution, so multiple sensors are used to detect objects at very close ranges.

First Embodiment
Next, an embodiment of the present application will be described.

The distance detection device (imaging device) according to the first embodiment includes a light source that generates illumination light in which chirp modulation is performed on the amplitude modulation of carrier light;
another light source that generates a reference light obtained by performing the same chirp modulation as the illumination light on the amplitude modulation of the carrier light;
The chirp modulation is modulated by a linear chirp modulation signal,
a coherence reduction mechanism that reduces the temporal coherence and the spatial coherence of both or one of the carrier lights while maintaining the temporal coherence and the spatial coherence of the chirp modulation signal;
an imaging optical system that forms an image of reflected light from an object illuminated with illumination light on a light receiving element;
a combiner for combining the reflected light and the reference light,
The light receiving element receives the combined reflected light and reference light, and detects an interference fringe signal generated by heterodyning between the chirp modulation signals of the illumination light and the reference light.

In this embodiment, distance detection is performed using a linear chirp modulation method that has excellent sensitivity and interference suppression. However, chirp-modulated signals are transmitted and received using light as a carrier, rather than radio waves as in millimeter-wave radar. And, instead of chirp-modulating the frequency of the carrier light, chirp modulation is performed on the frequency that modulates the amplitude of the carrier light.

Figure 1 shows the change in the amplitude of the carrier light over time. Specifically, as shown in Figure 1, a small, low-cost light-emitting diode capable of high-speed switching is used as the light source of the carrier light, and the light from the light-emitting diode is turned on and off, with linear chirp modulation being performed on the on-off frequency.

Figure 2 shows the frequency of chirp modulation. Using Figure 2, we will explain the distance detection method using the chirp modulation method.

Line 1 indicates a reference (standard) chirp signal whose frequency changes linearly with time. Line 2 indicates the chirp signal reflected from the object whose distance is to be measured. The reflected chirp signal 2 generates a time delay 3 relative to line 1, the reference chirp signal, that corresponds to the round-trip distance to the object. As a result, heterodyning between the chirp signals generates a low-frequency interference fringe signal that corresponds to the frequency difference 4 between the chirp signals.

When the chirp modulation has high linearity, the frequency of the interference fringe signal remains constant over the chirp time. As can be seen from Figure 2, as the optical path difference increases, the time delay 3 increases. The frequency difference 4 between the chirp signals increases, and the frequency of the interference fringe signal increases. In this way, the frequency of the interference fringe signal is proportional to the distance to the object. Figure 2 also shows that the frequency of the interference fringe signal increases as the chirp modulation bandwidth 5 becomes wider, and decreases as the chirp time 6 becomes longer.

Figure 3 shows the configuration of a distance detection device 10A according to the first embodiment. The amplitude of the carrier light 7 is linearly chirp modulated by a chirp modulation signal 9 using an amplitude modulator 8. Then, the illumination light 10 is used to illuminate an object 11, the distance of which is to be detected. The optical system 13 forms an image of the reflected light 12 from the object 11 on the light receiving element 14. The reference light 15 is combined with the reflected light 12 by the combiner 16. The light receiving element 14 receives the combined light.

When light is received by the light receiving element 14, heterodynes between the carrier lights and between the chirp signals occur. The reference light 15 is generated by applying linear chirp modulation to the amplitude of the carrier light 17 using the chirp modulation signal 9, using an amplitude modulator 21.

When the coherence of carrier light 7 and carrier light 17 is low, the heterodyne amplitude between the carrier lights approaches a constant value of the product of their respective power values due to the time average of the light receiving element. This constant value becomes the coefficient of the interference fringe signal generated by the heterodyne between the chirp signals, and the interference fringe signal 18 can be stably detected.

The frequency of the interference fringe signal 18 is proportional to the distance to the object 11. If the coherence of the carrier light 7 and the carrier light 17 is high, the heterodyne amplitude between the carrier lights will produce a specific frequency, making it impossible to detect the interference fringe signal, or a frequency component equivalent to the interference fringe signal will be produced, causing noise in the interference fringe signal.

When a high-coherence light source such as an LD (laser diode) that excels at high-speed amplitude modulation is used for generating and modulating the carrier light, the temporal coherence and spatial coherence of both or one of the carrier lights 7 and 17 are reduced by the coherence reduction mechanism 19 and the coherence reduction mechanism 20. The heterodyne amplitude between the carrier lights is then brought close to a constant value that is the product of the respective power values, thereby ensuring the signal-to-noise ratio of the interference fringe signal 18. The specific configuration of the coherence reduction mechanisms 19 and 20 will be described later.

The interference fringe signal 18 generated by heterodyning between chirp signals is converted into frequency components by a frequency converter such as an FFT (not shown), and the peak value of the frequency component is detected by a correlator and an interpolation circuit (not shown) and converted into the distance to the object. The processing of the correlator will be described later.

In this embodiment, the image sensor can be used to simultaneously detect color images and distance images, making it possible to realize a small, low-cost visual sensor integrated with an image capture device.

Second Embodiment
The distance detection device 20A according to the second embodiment is
a light source for generating illumination light in which chirp modulation is applied to amplitude modulation of carrier light;
an imaging optical system that forms an image of reflected light from an object illuminated with the illumination light on a light receiving element;
Further, the optical fiber includes a mixer that detects an interference fringe signal by multiplying the chirp-modulated electrical signal by a chirp-modulated signal that is the same as that of the illumination light, or the light receiving element that also functions as the mixer by direct switching using the chirp-modulated signal that is the same as that of the illumination light,
The light receiving element detects the reflected light and converts the chirp modulation of the reflected light into an electrical signal.

FIG. 4 shows the configuration of a distance detection device 20A according to the second embodiment.
A modulator 23 is used to linearly chirp modulate the amplitude of the carrier light 22 with a chirp modulation signal 24. Then, the light is used as illumination light to illuminate an object 25 whose distance is to be detected. Reflected light 26 from the object 25 is imaged on a light receiving element 28 by an optical system 27. Direct detection is performed by the light receiving element 28, which is capable of high-speed detection, and the chirp modulated signal of the reflected light 26 is converted into an electrical signal.

Then, a mixer 29 multiplies the chirp modulation signal 24 as a reference signal and converts it into an interference fringe signal 30. Then, as in the first embodiment, it is converted into the distance to the object through processing by a frequency converter, a correlator, and an interpolation circuit (not shown).

In this case, instead of the light receiving element 28 and the mixer 29, a light receiving element capable of direct switching may be used, and heterodyne detection may be performed by switching using the chirp modulation signal 24 to detect the interference fringe signal 30.

The higher the coherence of a light-emitting diode light source, the more possible high-speed amplitude modulation by direct switching or by an external element such as a Mach-Zehnder element is. In the second embodiment, the carrier light may be high-coherence light. Therefore, by using such a high-coherence light-emitting diode or light source, the bandwidth of the chirp modulation can be widened. This makes it possible to perform highly accurate distance detection.

The effects of the distance detection device according to the first embodiment and the distance detection device according to the second embodiment will be described.
The distance detection device of the first embodiment and the distance detection device of the second embodiment can satisfy all the requirements for a distance detection device and can solve the problems (disadvantages) encountered in conventional distance detection devices.

For example, when infrared light with a wavelength of 1550 nm is used as the carrier light, long-distance detection is possible at the same level as millimeter-wave radar. In addition, because an imaging optical system can be used, a much higher azimuth resolution can be obtained than with millimeter-wave radar.

High azimuth resolution can be achieved from close to long distances using autofocus and pan focus. This means that there is no ghost reflection (clutter) at close range caused by side lobes, as with millimeter wave radar.

In addition, the chirp modulation method transmits the power for irradiating the illumination light by diffusing (dispersing) it within the chirp modulation time according to the chirp modulation. For this reason, even if the intensity of the illumination light is the same, the total power of the illumination, including the time axis, is two to three orders of magnitude larger than the ToF method, and the power S/N ratio of distance detection is accordingly higher. If the intensity (power) of the illumination light of a ToF LiDAR or camera is increased to achieve the same S/N ratio, the reflected light from nearby objects becomes too large, and the light receiving circuit quickly becomes saturated. Thus, the ToF method has little room for dynamic range (S/N ratio), making it impossible to measure distances over a wide range, such as from a few centimeters to several hundred meters.

In this way, the chirp modulation method spreads the power of the illumination light over the chirp modulation time, which allows for a generous dynamic range in the light-receiving circuit system and avoids saturation of the circuit system caused by strong reflections at close range, as occurs with the ToF method, making it possible to detect distances with high accuracy over a wide range from a few centimeters to several hundred meters.

In addition, because the method involves chirp modulation of the carrier light amplitude, it is possible to apply not only radar jamming suppression technology but also optical communication interference avoidance technology in a dual manner. In addition, by taking advantage of the high azimuth resolution and linking it with a determination mechanism such as AI, it is possible to detect distances while avoiding interference from the illumination light of distance detection sensors on other vehicles or strong external light.

Furthermore, according to the first embodiment, it is possible to simultaneously detect a color image and a distance image using an imaging element, and it is possible to realize a small, low-cost sensor integrated with an imaging device.

In addition, since there are no regulations such as radio wave protection regulations, there is a high degree of freedom in design, making it easy to apply to a variety of uses, such as robots and drones.

In addition, by taking advantage of its high azimuth resolution and linking it with a determination mechanism such as AI, it is possible to receive optical wireless communications.

FIG. 5 shows a modification of the first embodiment.
The output light of an illumination light source 33 is turned on and off by a linear chirp modulation signal 32 of, for example, 0.1 to 1.1 GHz generated by a synthesizer oscillator 31 or the like, and is used as illumination light 34 to illuminate an object 35 for which distance is to be detected.

The illumination light source 33 uses a light-emitting diode capable of high-speed switching, such as an infrared LD with a central wavelength of 1550 nm or an SLD (super luminescent diode).

Infrared LDs are often low-priced due to their widespread use in optical communications and other fields. In addition, infrared light with a wavelength of 1550 nm is safe for human eyes, has little external light, is attenuated little in the atmosphere, and has almost the same attenuation in rain as 76-77 GHz millimeter wave radar.

The output light of the illumination light source 33 is turned on and off by direct switching of the light-emitting diode described above, or by using an external junction element such as an EA (Electro-Absorption) modulator or a Mach-Zehnder (MZ) modulator.

Reflected light 36 from an object 35 is imaged on an image sensor 38 by an imaging optical system 37. At that time, reference light 40, which is the output light of a reference light source 39 that is turned on and off by a linear chirp modulation signal 32, is combined with the reflected light 36 by a combiner 41.

When the light is received by the image sensor 38, an interference fringe signal with a frequency corresponding to the distance image of the object 35 is generated at each light receiving pixel of the image sensor 38 by heterodyning between the chirp signals. An infrared LD or SLD with a central wavelength of 1550 nm is used as the reference light source 39.

At this time, the position of the reference light source 39 is set by the optical system 42 so that it is optically conjugate with the position of the exit pupil of the imaging optical system 37. This allows the direction of travel of the wavefront of the reference light and the chief ray imaged on each pixel of the imaging element 38 to coincide, improving the efficiency of the heterodyne and suppressing unevenness in intensity for each pixel.

Then, by repeatedly capturing images using the image sensor 38 at high speed, the interference fringe signal is sampled in parallel for all pixels and converted into an electrical signal (interference fringe signal) 43.

Figure 6 shows the flow of processing the electrical signal 43. A frequency converter 44 such as an FFT converts the signal into frequency components for each pixel. A correlator 48 and a distance converter 49 detect the peak value of the frequency components and convert the signal into a distance image. The processing of the correlator 48 will be described later.

When the time coherence and spatial coherence of the carrier light of the reflected light 36 and the carrier light of the reference light 40 are sufficiently low, the heterodyne amplitude between the carrier lights approaches the product of their respective powers due to the average exposure time of the sampling image of the image sensor 38 and the surface integral on the light-receiving pixel. Then, an interference fringe signal with a high signal-to-noise ratio can be detected.

The heterodyne output I of the reflected light 36 and the reference light 40 at that time is expressed by the following equation.
I = K·Ps PL・{1/2・cos[φ(t+d)−φ(t)]+1} (A)
Becomes.

The reason why the heterodyne output I is expressed by equation (A) will be described later.
In the above formula,
Ps is the power of the carrier light of the reflected light 36,
PL is the power of the carrier light of the reference light 40,
φ(t) is the phase of the linear chirp (modulation) signal 32;
t is time,
d is the time delay due to the optical path difference between the reflected light 36 and the reference light 40,
K denotes a constant determined by the characteristics of the light-receiving pixel.

According to equation (A), heterodyning between chirp signals produces interference fringe signals at each pixel that are proportional to the distance to the object (proportional to d). The amplitude of these signals is proportional to the power Ps of the reflected light 36 at each pixel, and is amplified by the power PL of the reference light.

Here is a concrete example that follows formula (A). When a rapidly rotating object is illuminated with a stroboscope, if the blinking rate of the strobe light matches the rotational period, the object will appear to be stationary. If the blinking rate of the strobe light is slightly slower than the rotational period, the object will rotate more slowly. If the blinking rate of the strobe light is slightly faster than the rotational period, the object will appear to rotate in the opposite direction.

At this time, the light that is imaged at a point on the observer's retina is a mixture of the light reflected from the object by the room lighting, which is brightness-modulated by the object's rotation, and the light reflected from the object that is brightness-modulated by the flashing strobe.

When the retina receives the reflected light, a heterodyne between the two modulated signals occurs, which is time-averaged by the persistence of vision effect in the eye. This leaves a low-frequency intensity modulation that is the difference between the rotation of the object and the flashing of the strobe, resulting in the phenomenon observed above.

Indoor lighting and strobe light are incoherent lights and therefore have no coherence. The brightness of the object image changes in proportion to the product of the brightness (light intensity) of the indoor lighting and strobe light. The object image becomes brighter whether the indoor light or strobe light is made brighter. In that case, the object image and rotational movement do not change. Now, if the indoor light and strobe light are intentionally made coherent, the brightness of the object image will fluctuate, interference fringes will appear, and the rotational movement will appear to fluctuate.

A one-dimensional array of light receiving sensors may be used instead of the imaging element 38. A distance image can also be obtained by sampling interference fringe signals while scanning the imaging surface of the imaging optical system 37 one-dimensionally in a direction intersecting the array direction with the one-dimensional array of light receiving sensors, and performing the above-mentioned processing.

Furthermore, any one of the following configurations (a), (b), and (c) may be used.
(a) It is also possible to obtain a distance image by using a single light receiving element and two-dimensionally scanning the imaging surface of the imaging optical system 37.
(b) A distance image can also be obtained by placing a one-dimensional array sensor on the imaging plane, increasing the directivity of the illumination light in a direction intersecting the sensor array direction to illuminate object 35, and scanning one-dimensionally in the direction intersecting the array direction.
(c) A distance image can also be obtained by placing a single light receiving element on the imaging plane of the imaging optical system 37 and scanning the object 35 two-dimensionally with illumination light converged into a beam.

In this way, the two-dimensional array of light-receiving elements allows
Alternatively, the one-dimensionally arranged light receiving elements are scanned one-dimensionally in a direction intersecting the one-dimensional arrangement direction across the imaging surface of the imaging optical system,
Alternatively, a single light receiving element is scanned two-dimensionally across the image plane,
Alternatively, a one-dimensional array of light receiving elements is placed on the imaging plane, and the directivity of the illumination light in the direction intersecting the one-dimensional array is increased to perform one-dimensional scanning of the object in the direction.
Alternatively, a single light receiving element is placed on the imaging plane, and the illumination light is increased in directivity to a beam to scan the object in two dimensions.
It is desirable to provide a two-dimensional detection mechanism that detects interference fringe signals in two-dimensional positions while simultaneously capturing an image of the object.

The processing of the correlator 48 shown in FIG. 6 will now be described.
The interference fringe signal 43 is converted into a frequency component 45 by a frequency converter 44. On the frequency axis, a correlator 48 performs a correlation calculation between the frequency component 45 of the interference fringe signal and a reference signal (standard signal) 47 which is a complex conjugate.

The output of the correlator 48 is then subjected to data interpolation in the distance conversion unit 49. The peak value is then detected and converted into the distance L to the object 35 according to equation (8) described below.

Figure 7 shows harmonics. We will now describe the reference signal generating unit 46 that generates the reference signal 47. The frequency components of the square wave that performs chirp modulation by switching the carrier light include a fundamental wave 53 as well as a harmonic that is 2n+1 times the fundamental wave (n: natural number including 0), as shown in Figure 7.

Symbol 54 (first harmonic) in Figure 7 is the first harmonic when n = 1. The amplitude of harmonics for n = 2 and above is small, and can be ignored because it is suppressed by the response time of the light source switching. In addition, if the rise and fall times of the light source switching are different, a harmonic also appears at position 2n, but this also has a small amplitude and can be ignored.

Figure 8 shows the frequency of the interference fringe signal. As described above, the frequency component 45 of the interference fringe signal has a frequency 55 of the interference fringe signal generated by heterodyning the chirp signal of the fundamental wave 53, and a frequency (component) 56 of the interference fringe signal generated by heterodyning the chirp signal of the first harmonic 54.

The frequencies (components) 55, 56 of these two interference fringe signals change according to the distance to the object 35, as shown in FIG. 8. Then, when the distance of the object 35 increases and the frequency 56 of the interference fringe signal generated by the chirp signal of the first harmonic 54 exceeds the Nyquist frequency 57 of sampling, a reflection component 58 is generated. When the reflection component 58 falls below 0 frequency, a further reflection component 59 is generated.

As described above, the frequency component 56 of the interference fringe signal changes depending on the distance of the object 35. Therefore, the reference signal 47 used to perform the correlation calculation in the correlator 48 is changed in accordance with the change in the frequency component 56 of the interference fringe signal.

In this way, a correlation calculation is performed between the frequency components 45 of the interference fringe signal, which change according to the distance to the object 35, and the reference signal 47 that takes this into account. Then, by performing data interpolation in the distance conversion unit 49 to detect the peak value, the harmonic components can be concentrated at the peak value. As a result, the signal-to-noise ratio of the interference fringe signal can be increased. As will be described later, the distance detection accuracy is proportional to the distance resolution and the signal-to-noise ratio; for example, when the signal-to-noise ratio is 40 dB, the distance detection accuracy is about 1/40 of the distance resolution.

Furthermore, the frequency 55 and initial phase of the interference fringe signal can be calculated by equation (6) described later. The frequency 56 of the interference fringe signal can also be calculated by equation (7) because a and f0 in equation (6) can be tripled.

The aliasing components can be calculated from these calculation results. By normalizing the amplitudes and taking the phase conjugate, the reference signal 47 can be calculated.

Alternatively, the reflector can be placed at an appropriate distance, the frequency components of the interference fringe signal can be measured in advance, and reference signals at other distances can be calculated by interpolation. If the rise and fall times of the LD switching waveform change due to temperature changes, the amplitude of the harmonics can change slightly, or harmonics with twice the frequency of the fundamental wave can be generated. Here, these amplitudes are small and can be ignored.

Explains why heterodyning between chirp signals produces an interference fringe signal, and why the heterodyne amplitude between carrier lights approaches a constant value when the coherence of the carrier lights is low.

When the carrier light has low coherence and a spatially random phase, the heterodyne outputs of the reflected light 36 and the reference light 40 differ due to the spatial coherence on those light-receiving pixels.

Therefore, the electric field component of the carrier light of the reflected light 36 and the electric field component of the carrier light of the reference light 40 (see FIG. 5) are assumed to be functions of time t and position r on the light receiving surface of the light receiving element. In the case of a chirp signal, the wavelength of the fundamental wave is about ¹⁰⁴ times the size of the light receiving element. For this reason, the phase change with respect to the change in position r is extremely small, and will be explained as a function of time t only.

Where:
The carrier light of the reflected light 36 is represented as As(r,t),
The carrier light of the reference light 40 is denoted as AL(r,t).
If the square wave functions with an amplitude of 1 that chirp modulate the amplitudes of the carrier lights As(r,t) and AL(r,t) are S(t+d) and S(t), respectively, then
The electric field components of the reflected light 36 and the reference light 40 imaged at position r on the light-receiving pixel are As(r,t)S(t+d) and AL(r,t)S(t), respectively.
d indicates a delay time according to the optical path difference between the reflected light 36 and the reference light 40 .

At this time, the heterodyne amplitude H(r, t) when the reflected light 36 and the reference light 40 are combined and received is given by
H(r,t)=As(r,t) S(t+d)・AL(r,t) S(t) (B)
It becomes.

(B) By replacing S(t+d) and AL(r,t)
H(r,t)=As(r,t) AL(r,t)・S(t+d) S(t) (C)
Let us assume that.

As(r,t) AL(r,t) indicates the heterodyne amplitude between the carrier light of the reflected light 36 and the reference light 40, and S(t+d) S(t) indicates the heterodyne amplitude between the chirp signals.

In this case, when the correlation distance on the light receiving pixel of both or either of the carrier lights As(r,t) AL(r,t) is extremely short in space and time, as described below, the heterodyne amplitude As(r,t) AL(r,t) between the carrier lights approaches a constant value regardless of the position on the light receiving pixel due to the exposure time average of the sampling imaging. Then, due to the surface integral on the light receiving pixel, it converges to the product of the respective power values, and the interference fringe signal generated by the heterodyne of S(t+d) S(t) can be detected with a good S/N ratio.

Where:
The two positions on the light-receiving pixel are r1 and r2.
When the exposure time of the sampling image is τ,
The correlation function C(r1,r2) between H(r1,t) and H(r2,t) is

となる。

It becomes.

The heterodyne output I of the reflected light 36 and the reference light 40 is obtained by integrating the correlation function C(r1, r2) with respect to r1 and r2 on the light-receiving pixel as follows. K is a constant determined by the characteristics of the light-receiving pixel of the image sensor 38.

Square wave functions S(t+d) and S(t) with an amplitude of 1 become the same square wave function even when squared. Therefore, H(r1,t) and H(r2,t) in equation (1) can be written as follows:
H(r1,t) H(r2,t)=As(r1,t) AL(r1,t)・As(r2,t) AL(r2,t)・S(t+d) S(t) (D)
Let us assume that.

Furthermore, we will explain by replacing S(t+d) and S(t) with the equations for the fundamental wave and DC component of those frequency components, cosφ(t+d)＋1 and cosφ(t)＋1.

The same idea can be used to explain the interference fringe signals that arise from the harmonic components of S(t+d) and S(t). Here, the amplitude coefficients are omitted because they are meaningless for the purposes of explanation.

(D) Replace S(t+d) and S(t) with cosφ(t+d)＋１ and cosφ(t)＋１, and swap the positions of AL(r1,t) and As(r2,t),
H(r1,t) H(r2,t)=As(r1,t) As(r2,t)・AL(r1,t) AL(r2,t)・[cosφ(t+d)+1] [cosφ(t)+1] (E)
Let us assume that.

After expanding [cosφ(t+d)＋1] [cosφ(t)＋1] in equation (E) using a product-sum formula for trigonometric functions, substitute it into equation (2) via equation (1). When the time coherence and spatial coherence of the carrier light, As(r,t) and AL(r,t), are sufficiently low, the spatial correlation distance of the carrier light, As(r1,t) and As(r2,t), and AL(r1,t) and AL(r2,t) on the photodetector is extremely short, and the heterodyne amplitudes of As(r1,t) As(r2,t) and AL(r1,t) AL(r2,t) in the above equations are squared on the time axis regardless of the position of r. Therefore, they approach a constant value due to the time averaging of the sampling imaging.

We are then left with the following terms, which represent the low frequency interference fringe signal:
As(r1,t) As(r2,t)・AL(r1,t) AL(r2,t)・{1/2・cos[φ(t+d)−φ(t)]+1}
The other terms converge to zero by time averaging of the sampling images.

And 1/2 cos[φ(t+d)－φ(t)]+1 does not have the variable r. Therefore, it is excluded from the integral in equation (2) and becomes a coefficient term. In the above equation, As(r1,t) As(r2,t) AL(r1,t) AL(r2,t) is calculated by the time average in equation (1) and the integration on the photosensitive pixel in equation (2), and is the product of the total power of the carrier light on the photosensitive pixel, K Ps Converges to PL.

As a result, the heterodyne output I of the reflected light 36 and the reference light 40 is the same as the above formula,
I = K Ps PL {1/2・cos[φ(t+d)－φ(t)]+1} (3)
Becomes.

From equation (3), we can see that heterodyning between chirp signals generates an interference fringe signal of low frequency (2 in Figure 2) corresponding to the delay time d (3 in Figure 2), the amplitude of which is proportional to the power Ps of the reflected light and is amplified by the power PL of the reference light.

In this example, a method for ensuring the S/N ratio of the interference fringe signal will be described.
When the reflected light 36 and the reference light 40 are combined and received to detect the interference fringe signal, if the coherence of the carrier light is not sufficiently low, low frequency components equivalent to the interference fringe signal will be generated due to heterodyning between the carrier lights. These will not be averaged over the exposure time of the sampling image capture, and will remain as noise (beat noise, speckle noise) in the interference fringe signal.

When the bandwidth of the spatial frequency components of the carrier light of the reflected light 36 and the reference light 40 on the light receiving pixel is widened, the spatial frequency components of the heterodyne amplitude between the carrier lights become even wider due to their convolution integral. As a result, the coefficient K Ps PL of the interference fringe signal in equation (3) becomes larger. On the other hand, the bandwidth of the noise power does not change because the sampling exposure time is constant, and the power S/N ratio of the interference fringe signal increases.

Here, the temporal coherence of light is proportional to the inverse of the bandwidth of the spatial frequency components in the direction of light propagation, and the spatial coherence is proportional to the inverse of the bandwidth of the spatial frequency components of a plane normal to the direction of light propagation.

Therefore, the above statement that "widening the bandwidth of the spatial frequency components of the carrier light of the reflected light 36 and the reference light 40 on the light-receiving pixel increases the power S/N ratio of the interference fringe signal" can be rephrased as "lowering the temporal coherence and spatial coherence of the carrier light on the light-receiving pixel increases the S/N ratio of the interference fringe signal."

Infrared LDs with a wavelength of 1550 nm are capable of direct switching in the GHz range, and are small and low-cost, making them suitable as light sources for chirp modulation using on-off switching.

The time coherence and spatial coherence of the output light are relatively high, so the frequency bandwidth is narrow at 15 GHz. If the maximum detection distance is 150 m, the chirp time is 15 ms, and the bandwidth is 1 GHz, the frequency of the interference fringe signal is 67 kHz according to equation (7) described below.

The sampling frequency must be more than twice as high, at about 150 kHz. At that time, the ratio of the frequency bandwidth of the carrier light to the bandwidth of the low-pass cutoff frequency due to sampling exposure, i.e., the power S/N ratio, is 94 dB.

The amplitude SNR of the interference fringe signal is half that, at 47 dB. When considering the attenuation of the interference fringe signal over long distances, if an SNR of 60 dB is required, this is about 12 dB short. Therefore, the following method is used to broaden the bandwidth of the carrier light and ensure the SNR.

Next, a method for reducing the temporal and spatial coherence of the carrier light will be described.
In order to ensure the light intensity of the light source, multiple LD light sources are arranged two-dimensionally on a plane whose normal line is the principal axis of the light from the light source, and the output light is synthesized, thereby reducing coherence.

The coherence of the composite light can be reduced in linear proportion to the diameter of the light emitted from the two-dimensionally arranged LD light sources. In this case, the reduction in spatial coherence is much more efficient than the reduction in temporal coherence.

The spacing between the two-dimensionally arranged LD light sources is appropriately selected in consideration of the frequency bandwidth (wavelength bandwidth) of the LD light source and the waveform of its envelope so that the spatial frequency bandwidth of the multiple LD light beams combined on the light-receiving pixel surface is wide and uniform.

In this case, similar to staggered tuning, by shifting the central wavelengths of multiple LDs and overlapping parts of the frequency bands to create a wide, uniform combination, it is possible to reduce spatial coherence, including temporal coherence, even more efficiently.

When the structure of the light source assembly requires a wide spacing between light sources, the light sources are divided into a fine arrangement using collimator optics, fly's eye optics, and Kohler illumination optics. Furthermore, by combining this with the above method, it is possible to efficiently reduce coherence. In that case, the length of the fly's eye lens can be made random, allowing reduction including time coherence.

In addition, if the switching speed allows, low-coherence light-emitting diodes such as surface-emitting SLDs and LEDs may be used instead of LDs. In recent years, efforts have been made to further increase the output and switching speed of these low-coherence light-emitting diodes in anticipation of visible light communication, optical wireless, Li-Fi (Light Fidelity), and other applications.

Instead of direct switching, high-speed switching can also be achieved by using an external switching element such as an EA element. However, since these low-coherence light sources often have low temporal coherence but high spatial coherence (point source properties), they must be combined with the above-mentioned means for reducing spatial coherence.

In addition, by using separate light sources for the illumination light and the carrier light for the reference light, the phase and polarization plane between the carrier lights can be made random, allowing the beat noise to be spread over a wide bandwidth.

Also, the illumination light can be split by a splitter and guided through a fiber bundle to be used as reference light. The coherence length of an LD is about 10 cm. Therefore, when the round-trip distance to the object is 1 m or more, the temporal coherence of the reflected light is already sufficiently broken. In addition, by using a multimode fiber bundle, etc., and adjusting the length and diameter of the fiber bundle, the temporal and spatial coherence can be further reduced.

In addition to the methods mentioned above, there are many other ways to reduce the coherence of the carrier light, but no single method can produce light that is close to incoherent. Depending on the application, several methods can be combined to achieve a sufficient reduction in the coherence of the carrier light and ensure the signal-to-noise ratio of the interference fringe signal to the beat noise mentioned above.

Figure 9 shows the relationship between the coherence of the carrier light and the size of the light source. In either case, the size of the light emitting portion of the illumination light and the reference light is set to the range 61 shown in Figure 9. Then, by applying the same chirp modulation to the illumination light and the reference light, it is necessary to sufficiently reduce the temporal coherence and spatial coherence of the carrier light while maintaining the temporal coherence and spatial coherence of the chirp signal.

In Figure 9, graph 62 shows the decrease in spatial coherence of the carrier light relative to the size of the light-emitting section, and graph 63 shows the decrease in spatial coherence of the chirp signal. In Figure 9, range 61 is depicted narrowly due to space limitations, but since the ratio of the wavelengths of the carrier light and the chirp signal is about five orders of magnitude, the actual range 61 is quite wide, and it is easy to make the settings described above.

Graph 62 of reflected light 36 changes depending on the distance to object 35, magnification (angle of view), and size of the light-receiving pixel. For this reason, graph 62 uses a graph of coherence reduction under the worst conditions, and graph 63 of the chirp signal uses a graph under the best conditions for coherence reduction.

In the case of reference light 40, the distance from the light source to the light receiving element is short and constant, so the settable range 61 is wide and it is easy to reduce coherence.

Next, the image sensor 38 that simultaneously detects an RGB image and a distance image will be described.
An image sensor with 1.34 million pixels and wavelength characteristics of 400 nm to 1700 nm is commercially available. The light receiving section is made of InGaAs semiconductor, and the circuit system such as the readout mechanism is made of Si, making it a two-layer structure image sensor.

Figure 10 shows the front configuration of the imaging element. Using such an imaging element, RGB color filters and IR wavelength filters are arranged in a mosaic pattern as shown in Figure 10. The wavelength characteristics of the IR filter have a central wavelength of 1550 nm, and are adapted to the wavelength bands of the illumination light 34 and reference light 40 to prevent noise caused by unnecessary external light.

By driving such an image sensor 38 at high speed, the interference fringe signals generated by the IR pixels are sampled in parallel for all pixels, and a distance image is detected after processing by the frequency converter 44, correlator 48, and distance conversion unit 49 shown in Figure 6.

In parallel, multiple RGB images and IR images obtained by high-speed imaging are cumulatively added by a cumulative adder (not shown) to obtain RGB images and IR images with a wide dynamic range.

From these RGB and IR images, AI51 can detect objects in the outside world, as well as determine the position of interfering illumination light from distance detection devices on other vehicles and external light, and can eliminate pixels of interference fringe signals that correspond to those positions. AI51 is an essential judgment mechanism for autonomous driving and object detection by robots. Object detection by AI51 will be described later.

We will now describe how to apply the above-mentioned method of obtaining a distance image to a robot's visual sensor. The RGB image, IR image, and distance image are merged, and the AI51 is trained to learn the image as one with five channels of information per pixel.

Many of the objects handled by robots cannot be identified by their shape, texture, or color alone in RGB images. For this reason, the accuracy of classification of work objects can be improved by using three-dimensional shape information (distance image) obtained by distance detection in addition to the image. This makes it possible to detect the three-dimensional position of the object being grasped by the robot's handpiece.

For example, the range for detecting the distance to the work object is divided into two stages: 0 to 0.3 m and 0.3 to 3 m. When the distance is 3 m, the chirp modulation width is set to 3 GHz to achieve a distance detection accuracy of 1.25 mm. Then, the frequency of the interference fringe signal becomes 4 kHz. The frame rate required for the image sensor is 8000 frames/second.

When the robot handpiece approaches within 0.3 m, the chirp modulation bandwidth is switched to 30 GHz to set the distance detection accuracy to 125 μm, and the frequency of the interference fringe signal becomes 4 kHz.

The frame rate required for the image sensor will also be 8,000 frames per second, which is the level that can be achieved with current image sensors with high-speed imaging. Switching above 30 GHz can be achieved by switching the output light of the LD using a Mach-Zehnder element or similar.

And as mentioned above, from the five-channel images, AI51 can determine interference from multiple distance sensors used in the robot's workspace and external light, and avoid their effects.

Next, we will explain how the image sensor 38 is driven when detecting RGB images and long-distance distances simultaneously, as in a vehicle.

In the case of a vehicle, the detection distance is as long as 150 m, so even if the chirp modulation bandwidth is 1 GHz, the frequency of the interference fringe signal is high at 67 kHz. If this interference fringe signal is sampled in parallel for all pixels by the high-speed imaging of the image sensor 38, high-speed imaging of 134,000 frames per second or more is required according to the sampling theorem, which exceeds the limit of high-speed imaging of current image sensors.

However, object detection in a vehicle only requires detecting object 35 related to autonomous driving and detecting its distance. For this purpose, AI object detection techniques such as bounding boxes, which detect the size and position of objects related to autonomous driving, and semantic segmentation, which detects the area and position, are used. Then, only the interference fringe signals of IR pixels corresponding to the localized parts of object 35 where distance detection is required are read out. This reduces the number of pixels to be read out, and allows the rate at which interference fringe signals are sampled to be sufficiently high.

Figure 11 shows a situation where multiple objects exist. As shown in Figure 11, when there are multiple automobiles 70A and 70B, which are objects related to safe driving, AI 51 determines multiple local parts 71, 72, 73, and 74 from the RGB image and the IR image to detect the distances of the objects 70A and 70B, avoiding interfering lights 75 and 76.

Reference numeral 52 in FIG. 6 indicates a signal that AI51 transmits to the readout mechanism of the image sensor 38 in FIG. 5 the pixel address of the local position of the object where the interference fringe signal is detected, determined from the RGB image and the IR1 image, and the pixel address of the position where the optical wireless communication is received, determined from the IR2 image.

Based on the result (reference number 52 in Figure 6), the readout mechanism of the image sensor 38 repeatedly reads out the interference fringe signals of the multiple IR pixels determined by AI 51 in sequence by random access so that the same sampling rate is obtained.

As a result, the required sampling rate can be sufficiently ensured by simply increasing the frame rate of the image sensor 38 by about one digit. The interference light 75 and interference light 76 indicate the positions of the illumination light sources of the distance detection devices of other vehicles that cause interference.

Figure 12 is a timing chart showing the reading of pixels on a horizontal line of the image sensor. We will describe a method for using the image sensor 38 to rapidly sample the interference fringe signal of the IR pixel determined by AI 51 while simultaneously reading out the RGB image and the IR image.

By sequentially repeating the reading of the pixels of the horizontal line of the image sensor 38 and the reading of the horizontal line containing the IR pixel determined by AI51, it is possible to rapidly sample the interference fringe signals of multiple IR pixels determined by AI51 while simultaneously reading the RGB image and the IR image.

When a CMOS image sensor is used for the image sensor 38, if the horizontal lines including the IR pixels (corresponding to reference numbers 71 to 74 in FIG. 11) for which distance detection is required as determined by AI 51 are designated m, n, o, and p, then, as shown in FIG. 12, first, the signals of the R and G pixels on horizontal line 1 are transferred to the horizontal shift register of the image sensor 38 at timing 81. Then, at timing 82, the accumulated charge is reset, and horizontal readout of the R and G pixels and exposure for a frame period 83 begins.

Next, horizontal lines m, n, o, and p are selected in sequence. At the timings indicated by symbols 84 to 87, the signals of the B and IR pixels are transferred in sequence to the horizontal shift register, while at the timings indicated by symbols 88 to 91, they are reset in sequence. Horizontal readout of the B and IR pixels and exposure of the sampling periods 92 to 95 are started in sequence.

Next, moving to horizontal line 2, the B and IR pixels are shifted to the horizontal shift register at timing 96 and reset at timing 97, starting horizontal readout of the B and IR pixels and exposure for frame period 98.

Next, horizontal lines m, n, o, and p are selected in sequence again, and horizontal readout of the B and IR pixels and exposure for the sampling period are started. By repeating the above process for the number of horizontal lines, the RGB and IR images are detected and at the same time, the interference fringe signal of the IR pixel determined by AI51 is sampled.

At this time, the exposure time of the B pixels on horizontal lines m, n, o, and p is the sampling period 92 to 95 of the interference fringe signal. Therefore, cumulative addition is performed within the frame period to obtain the B image on horizontal lines m, n, o, and p.

However, even when cumulative addition is performed, there is a slight shortage in the exposure time due to the time it takes to transfer the signal to the horizontal shift register and to reset, so the gain is adjusted to compensate. This allows the RGB image and IR image to be obtained, and at the same time, the interference fringe signals of the IR pixels corresponding to the four positions 71 to 74 where AI51 has determined that distance detection is necessary to be performed, to be sampled at high speed. The information on the detected RGB image and IR image is used for AI51's next determination.

Now, let's say the chirp modulation bandwidth is 1 GHz.
Chirp time is 15 ms,
When the maximum detection distance to the object 35 is 150 m,
The maximum frequency of the interference fringe signal is 67 kHz.

If the AI51 determines that there are four pixels, a horizontal scanning frequency of about 750 kHz, five times the frequency, is required. Converting this to a frame rate of an HD image sensor, it is about 600 frames per second. This is a level that can be easily achieved with the high-speed imaging technology of existing HD image sensors.

Thus, in a preferred aspect of this embodiment,
a determination mechanism that detects a position of interference light that interferes with detection of the distance to the object from an image captured by the two-dimensional detection mechanism and determines a local position where the distance to the object is detected;
It is desirable to detect the distance to the local location of an object while avoiding interfering light.
According to a preferred aspect of this embodiment,
a determination mechanism that determines a position of interference light that interferes with reception of the optical wireless communication from an image captured by the two-dimensional detection mechanism, and determines a position of a transmitter of an optical wireless communication device that receives the optical wireless communication,
It is desirable to receive wireless optical communications while avoiding interfering light.

A method of receiving optical wireless communications using the imager 38 will now be described.
Fig. 13 shows the front configuration of the filters. When the image sensor 38 also has a receiving function for optical wireless communication, as shown in Fig. 13, RGB color filters for color image detection, an IR1 filter for distance detection, and an IR2 filter for optical wireless communication are arranged in a mosaic pattern on the pixels of the image sensor 38.

The timing for reading out these pixels is the timing chart in Figure 12 with the addition of the timing for reading out and resetting the IR2 pixel for optical wireless communication. If there are four IR1 pixels and four IR2 pixels determined by AI51, the horizontal scanning repetition frequency is 1200 kHz, nine times that of the timing chart in Figure 12. Converted into the frame rate of the image sensor, this is 1150 frames per second, a level that can be achieved with existing high-speed imaging technology for HD image sensors. In this case, the communication capacity of the optical wireless communication is 67 kbps.

Alternatively, a multi-plate prism can be used, and a dichroic mirror can be used to separate the wavelength bands for RGB images, the IR1 wavelength band for distance measurement, and the IR2 wavelength band for optical wireless communication, with dedicated imaging elements provided for each detection.

Above, we have described a method that allows random selection of horizontal lines without changing the basic configuration of the light receiving section of the CMOS image sensor, thereby enabling high-speed sampling of the interference fringe signal of the IR pixel determined by AI51 and reception of optical wireless communication at the same time as detecting the RGB image and IR image.

By modifying the semiconductor configuration and readout mechanism of the pixels of the CMOS image sensor, it is possible to sample the interference fringe signals of the IR pixels at even higher speeds and receive optical wireless communication.

For example, first, the horizontal and vertical scanning circuits of the CMOS image sensor are changed to a circuit configuration (decoder circuit, etc.) that can randomly select pixels. In addition, a transistor that resets the vertical line is placed in series (cascode connection) with the transistor of each pixel that resets the charge of the pixels on the horizontal line. This creates a configuration in which resetting each pixel is performed by an XY matrix switch.

Using a procedure similar to that described above, it becomes possible to read out the light-receiving pixels of the image sensor 38 and the IR pixels determined by the AI 51, repeatedly and sequentially on a pixel-by-pixel basis. In parallel with the detection of the RGB and IR images, the sampling rate of the IR pixel interference fringe signal and the receiving capacity of the optical wireless communication can be increased by about three orders of magnitude. However, this means that one more transistor that resets the vertical line is required for each pixel, and one more wiring for the signal that drives it is required for each vertical line.

In addition, if a semiconductor configuration such as an InGaAs PIN semiconductor is used for the light receiving section, or a single photon avalanche diode (SPAD) is used for the IR light receiving pixel, high frequency interference fringe signals can be detected with high sensitivity. Also, high bit rate optical wireless communication can be received with high sensitivity.

Figures 14 and 15 each show a schematic configuration of a distance measuring device according to a modified example of the second embodiment.

The light from the illumination light source 102 is turned on and off by a linear chirp signal 119A of, for example, 0.1-1.1 GHz, generated by a synthesizer oscillator 101 or the like. After passing through a circulator 103, the light is narrowed down into a beam by an optical system 104.

The illumination light 105 is scanned two-dimensionally over the object 107 by a two-dimensional scanning mechanism 106. The illumination light source 102 is a light-emitting diode capable of high-speed switching, such as an LD or SLD with a central wavelength of 1550 nm.

The reflected light 108 from the object 107 passes through the optical system 104 and the circulator 103, and is then split into the wavelength bands IR1, IR2, R, G, and B by the splitter 109. The light is then input to the light receiving elements 110, 111, 112, 113, and 114 via the optical switches 115 and 116, and converted into electrical signals.

The symbol IR1 indicates infrared light with a central wavelength of 1550 nm generated by the illumination light source 102. The symbol IR2 indicates light for external infrared communication. Here, the circulator 103, the splitter 109, the optical switches 115 and 116, and the light receiving elements 110, 111, 112, 113, and 114 are optical devices connected by optical fibers. Similar optical devices are often used in optical fiber communication.

The AI 120 shown in FIG. 15 determines the position (71-74 in FIG. 11) where distance detection is required while avoiding interference. Then, according to the instruction 121, when the scanning of the illumination light 105 beam overlaps that position (71-74 in FIG. 11), the optical switch 115 switches the connection to the light receiving element 117.

The other positions (71-74 in Figure 11) are connected to the light receiving element 110 to detect the image signal with the code IRI. The light receiving element 117 is composed of an InGaAs PIN type semiconductor or the like, has a current-voltage conversion type amplifier, and is capable of high-speed direct detection with a high signal-to-noise ratio.

The chirp signal of the reflected light 108 is directly detected and converted into an electrical signal, and the detected chirp signal is multiplied by the chirp signal 119A as a reference signal by the mixer 118 to detect the interference fringe signal.

This interference fringe signal is processed in the same way as shown in Figure 6 to detect the distance to the object 107. If the distances of all pixels are detected at that time, a distance image can be obtained.

Alternatively, instead of the light receiving element 117 and the mixer 118, a light receiving element that can receive single photons (Geiger mode) by the avalanche effect and can perform heterodyne detection by high-speed direct switching, such as a single-photon avalanche diode (SPAD), can be used. At the same time as amplifying the received light with a high noise figure, heterodyne detection can be performed directly by switching with the chirp signal 119A to detect the interference fringe signal.

Similarly, the AI 120 determines the position (reference numbers 77 and 78 in FIG. 11) where the optical wireless communication is received while avoiding interference light (not shown) such as strong reflections of sunlight. When the scanning of the illumination light 105 beam overlaps with the position (reference numbers 77 and 78 in FIG. 11) according to the instruction 122, the optical switch 116 switches to the signal line of the optical fiber connected to the input terminal of the optical wireless receiver (not shown).

Reference numerals 77 and 78 in FIG. 11 indicate the positions of optical wireless transmitters installed on other vehicles. Alternatively, they may be optical wireless transmitters installed on side roads that provide position information. In this case, the current position can be detected with centimeter-level accuracy instead of GPS from the position information provided by such transmitters and the relative positions of the transmitter and vehicle detected by the distance detection device of the present invention, and can be electronically compared with a map.

Other than the positions (reference numbers 77 and 78 in Figure 11), it is connected to the light receiving element 111 and detects the image signal of the IR2 pixel.

The separated R, G, and B light are also converted into electrical signals as a color image by light receiving elements 112-114. The IR1, IR2, R, G, and B signals are then input to AI 120 via buffer memory 119 as a 5-channel/pixel image.

The next object detection is performed, and the position where interference light is avoided and where distance detection and reception of optical wireless communication are required are determined.

As described above, variant example 21B of the second embodiment in FIG. 14 is an example in which a distance image is detected using a single light receiving element 117 by scanning an object two-dimensionally with illumination light 105 narrowed into a beam shape.

Furthermore, as a new modified example of the second embodiment, a semiconductor capable of high sensitivity and high-speed switching, such as an InGaAs PIN-type semiconductor or a SPAD, may be used for the light receiving portion of the pixel of the image sensor 38 of modified example 11A of the first embodiment in FIG. 5.

By receiving the chirp signal of the reflected light 36 with high sensitivity and switching it with the chirp-modulated reference signal 32 shown in FIG. 5, heterodyne detection can be performed directly for each light-receiving pixel to detect the interference fringe signal and obtain a distance image. In this case, a high-coherence light source can be used for the illumination light source 33 in FIG. 5, and the reference light source 39 and multiplexer 41 in FIG. 5 are no longer necessary.

We will now describe AI51 and AI120 shown in Figures 6 and 15. AI51 and AI120, which perform object detection, use a convolutional neural network (CNN). CNN excels at pattern recognition such as image recognition, and is suitable for object detection based on pattern recognition. CNN learning is mainly performed by supervised learning.

In specific areas where the number of decision-making situations is small, it is easier to collect learning data, and there are an increasing number of cases where AI outperforms humans in competitions due to the difference in the amount of learning. In general areas where the number of decision-making situations is large, AI cannot possibly compete with humans.

When AI divides up the feature space to make judgments, if the distances between the feature vectors of the images used for training are not uniform but biased, over-learning or under-learning will occur, resulting in reduced judgment accuracy. For this reason, know-how for setting up training data is important.

By randomly increasing the amount of data, bias can be mitigated probabilistically, and object detection accuracy can be improved in proportion to the logarithm of the amount of training data. However, this requires setting up a huge amount of data. Several proposals have been made to make the setup of training data more efficient based on feature space, but there is currently no conclusive solution.

AI never tires or forgets, and can complete huge amounts of learning at the speed of light, day or night. In addition, there are an increasing number of cases where AI outperforms humans in terms of the amount of learning it can do. As a result, the world is now moving toward acquiring infrastructure that can set up huge amounts of learning data.

As methods for collecting training data evolve in the future, the way we think about and the methods of setup will change, and there will undoubtedly be a sharp increase in cases where AI outperforms humans in certain areas.

The basic configuration of a CNN processing circuit is a feedforward processing circuit consisting of an input layer, a feature extraction layer, a decision layer, and an output layer.

The input layer has a number of input terminals equal to the number of information channels per pixel multiplied by the number of pixels. For RGB images, the number of information channels is 3 channels/pixel, and for images that combine RGB images, distance images, and velocity images, the number is 5 channels/pixel.

The feature extraction layer is composed of a set of convolutional layers, activation functions, and pooling layers stacked in multiple stages. When there are a large number of information channels and tasks, object detection accuracy can be ensured by increasing the number of feature extraction layers.

The convolution layer is a multidimensional filter whose dimension is the number of information channels, and performs a convolution operation on the kernels (weighting coefficients of the neural network) acquired through learning. Then, a feature map is extracted.

In shallow layers where the correlation between neighboring pixels is high, feature maps close to principal component analysis are extracted, while deeper layers extract feature maps closer to independent component analysis. In order to strike a balance between the number of layers and the number of calculations, a kernel size of 3x3x3 is often used for three-channel images such as RGB images.

If the number of input channels to the convolutional layer is n, the kernel size is ³ⁿ , and if the number of kernel types acquired by learning is m, then m types of ³ⁿ convolution operations are performed. The number of input channels n of the next convolutional layer is the number of kernel types m of the previous stage.

The activation function has a simple structure in which the output of m types of convolution operations is multiplied by weighting coefficients obtained through learning, a bias is added, and a nonlinear function transformation is performed. By performing processing similar to fuzzy logic operations, it plays an important role in linking to higher-level feature extraction from the next layer onwards. There are several types of activation functions, and for object detection, rectified linear functions are often used to balance the scale of operations.

The pooling layer performs a process similar to data compression, which subsamples a certain range of the output (feature map) of the convolution layer, for example a 2x2 range, into one. The convergence is performed by selecting the maximum value (max pooling) that is the first principal pole of independent component analysis (similar to principal component analysis in shallow layers) from the 2x2 range of signals.

The decision layer and output layer are configured according to the task. The decision layer for category classification is configured by stacking one or two fully connected layers and sets of activation functions depending on the application. When there are multiple categories, the output layer is provided with the same number of output terminals as the number of classes. The activation function placed between the decision layer and the output layer uses a softmax function to express the decision result as a probability.

In object detection for autonomous driving, in parallel with the decision layer and output layer for object category classification, there are decision layers and output layers for detecting the size and position of objects (bounding boxes) and classifying object regions (semantic segmentation).

If the input layer and feature extraction layer are encoding functions that extract feature vectors, then the decision layer and output layer are decoding functions that generate output according to the purpose from the feature vectors, and are flexibly configured for each task. For example, in the configuration of a U-Net that detects object regions, the output of each stage of the feature extraction layer is extracted using a shortcut, and unpooling and deconvolution are performed to decode the object region.

In AI51 shown in FIG. 6 and AI120 shown in FIG. 15, in addition to the judgment layer and output layer for object detection necessary for autonomous driving, judgment layers and output layers for judging the local position of an object that requires distance measurement and the position of the transmitter of the other party receiving the optical wireless communication are provided in parallel.

Then, the 5-channel/pixel image obtained by merging the IR1 image and the IR2 image with the RGB image is learned by AI51 and AI120 as learning data, and the 5-channel/pixel image detected by the distance detection device of this embodiment is input to AI51 and AI120, whereby AI51 and AI120 determine the IR1 pixels and IR2 pixels that need to be detected and transmit the addresses of those pixels to the image sensor 38 and the optical switches 115 and 116 in FIG. 14.

We will discuss the frequency and initial phase of the interference fringe signal, the distance to the object, the distance resolution, and the distance measurement accuracy.

Where:
f ₀ is the starting frequency of the linear chirp modulation;
fw is the initial phase φ ₀ , the bandwidth of the chirp modulation,
T is the chirp time.
a = fw/T is the rate of change of frequency of the chirp modulation.
If t is time, the instantaneous frequency f(t) of the chirp signal is
f(t) = at + f ₀
It becomes.

If the phase of the chirp signal modulating the reference light is φ(t) and the initial phase is _φ0 , then φ(t) is obtained by integrating the above equation with respect to t as follows:
φ(t)＝2πf(t)
=2π(1/2・a t+f ₀ ) t+φ ₀ (4)
It becomes.

If the time delay corresponding to the optical path difference between the reflected light and the reference light is d, the phase φ(t+d) of the chirp signal of the reflected light is
φ(t+d)=2π[1/2・a (t+d)+f ₀ ](t+d)+φ ₀ (5)
This can be expressed as:

Substituting equations (4) and (5) into equation (3), we get
I＝K Ps PL cos [2π(a・d)t +πa・d ² +2πf ₀・d] (6)
It becomes.

If the speed of light is C and the distance to the object is L, then L = C d/2, so
From equation (6), the frequency fi of the interference fringe signal is given by
fi=a・d=(fw/T) d=2 fw L/CT (7)
It becomes.

It can be seen that the frequency fi of the interference fringe signal increases in proportion to the distance to the object L (delay time d) and the chirp modulation bandwidth fw, and decreases in proportion to the chirp time T.

And the initial phase φ ₀ of the interference fringe signal is
φ ₀ =πa・d ² +2πf ₀・d=π/a・fi ² +2πf ₀ /a・fi
It becomes.
The initial phase of the interference fringes varies parabolically in proportion to the frequency fi of the interference fringes (the object distance).

In addition, the distance L to the object is L = C·d/2. Therefore, from equation (7),
L=C・d/2=C・fi/2a=C・T fi/2fw (8)
It becomes.
The distance L can be calculated using the frequency fi of the interference fringes, the chirp modulation bandwidth fw, and the chirp time T.

The distance resolution ρ is the ability to resolve two objects lined up in the distance direction. If the envelope of the chirp signal is a square wave, the envelope of the interference fringe signal will also be a square wave, and when such an interference fringe signal is converted into frequency components, the waveform of the envelope will be a sinc function.

The full width at half maximum of the sinc function, 1/T, is the spectral resolution on the frequency axis. The spectral resolution, 1/T, is the change in fi when light travels a distance equivalent to the distance resolution ρ.

Therefore, by replacing fi and L in equation (7) with 1/T and ρ, the distance resolution ρ is given by
ρ＝C/2fw (9)
It becomes.

The waveform of the envelope is a sinc function with a full width at half maximum of 1/T being 1/2fw. This shows that the distance resolution ρ is uniquely determined by the bandwidth fw of the chirp modulation.

Distance detection accuracy is the accuracy when detecting the distance L to an object, and is the width by which the position of the peak of the distance resolution waveform fluctuates due to noise. Therefore, distance detection accuracy increases in proportion to both the distance resolution and the S/N ratio.

The higher the signal-to-noise ratio, the smaller the distance detection accuracy becomes. As mentioned above, when the correlator 48 performs a correlation calculation with the reference signal 47 and detects the peak value, the waveform of the distance resolution (sinc function) becomes close to a Mexican hat function.

For example, if the S/N ratio is 40 dB, the distance detection accuracy is about 1/40 to 1/50 of the distance resolution. Increasing the chirp time T narrows the spectral width 1/T of the interference fringe signal, and the S/N ratio increases in proportion to the square root of the ratio of the spectral width 1/T to the chirp modulation bandwidth fw.

Increasing the chirp modulation bandwidth fw increases the distance resolution ρ, and so the distance detection accuracy increases in proportion to both the chirp time T and the chirp modulation bandwidth fw. This is why the chirp modulation method is superior to other methods in terms of sensitivity and distance detection accuracy.

As described above, the present invention has high distance detection accuracy and azimuth resolution regardless of the distance to the object, is resistant to interference and disturbance between distance detection devices, is small and low-cost, and is suitable for distance detection devices that are integrated with an imaging device.

10A, 20A Distance detection device; 10 Illumination light; 11 Object; 12 Reflected light; 13 Optical system; 14 Light receiving element; 15 Reference light; 16 Wave combiner; 17 Carrier light; 18 Interference fringe signal; 19, 20 Coherence reduction mechanism; 21 Amplitude modulator; 22 Carrier light; 23 Modulator; 24 Chirp modulation signal; 25 Object; 26 Reflected light; 27 Optical system; 28 Light receiving element; 29 Mixer; 30 Interference fringe signal; 31 Synthesizer oscillator; 32 Linear chirp modulation signal; 33 Illumination light source; 34 Illumination light; 35 Object; 36 Reflected light; 37 Imaging optical system; 38 Imaging element; 39 Reference light source; 40 Reference light 41 Wave combiner; Reference Signs List 42 Optical system; 43 Interference fringe signal; 44 Frequency converter; 45 Frequency component; 46 Reference signal generator; 47 Reference signal (reference signal); 48 Correlator; 49 Distance converter; 51 AI; 52 Reference symbol; 53 Fundamental wave; 54 First harmonic wave; 70A, 70B Object (automobile); 71, 72, 73, 74 Local part; 75, 76 Interference light; 101 Synthesizer oscillator; 102 Illumination light source; 103 Circulator; 104 Optical system 105 Illumination light 106 Two-dimensional scanning mechanism; 107 Object; 108 Reflected light; 109 Splitter; 110 to 114, 117 Light receiving element; 115, 116 Optical switch; 118 Mixer; 119 Buffer memory; 119A Linear chirp signal

Claims (6)

a light source that generates illumination light for illuminating an object whose distance is to be detected by modulating the amplitude of carrier light with a chirp signal;
a light source that generates a reference light by modulating the amplitude of another carrier light with the same chirp signal as the illumination light;
a coherence reduction mechanism for reducing temporal coherence and spatial coherence of both or one of the carrier lights;
a light receiving element having a light receiving surface for receiving reflected light from the object;
an imaging optical system that forms an image of the reflected light on the light receiving element;
a combiner that combines the reflected light and the reference light,
The coherence reduction mechanism expands the spatial frequency components of the interference noise generated by heterodyning between the carrier lights to a predetermined bandwidth, reduces the power of the interference noise by the time average of the receiving time when the photodetector receives the combined reflected light and reference light and the area integral of the light receiving surface, and brings the heterodyne amplitude of the carrier lights of the illumination light and the reference light closer to a constant value of the product of the total power values between the carrier lights, thereby enabling the interference fringe signal generated by the heterodyne between the chirp signals of the reflected light and the reference light to be detected with a predetermined signal-to-noise ratio. a two-dimensional detection mechanism for detecting an image of external light having the same wavelength as the carrier light and a color image of the external light at the same time as detecting the interference fringe signal;
the wavelength of the carrier light is different from the wavelength of the color image;
The two-dimensional detection mechanism includes:
a first light receiving element constituting the light receiving element to which a wavelength filter that selectively transmits the wavelength of the carrier light is added, and a second light receiving element to which a primary color filter of the color image is added, the first and second light receiving elements being two-dimensionally arranged in a mosaic pattern on an imaging plane of the imaging optical system, or
a third light receiving element constituting the light receiving element arranged two-dimensionally to detect the interference fringe signal, on an imaging plane that can selectively transmit the carrier light by a multi-plate prism of a dichroic mirror; and a fourth light receiving element arranged two-dimensionally to detect a color image, on another imaging plane;
The distance detection device according to claim 1, characterized in that the two-dimensional detection mechanism separates the outputs from the first and second light receiving elements that have passed through the wavelength filter and the primary color filter, or separates the outputs from the third and fourth light receiving elements. a light source that generates illumination light for illuminating an object whose distance is to be detected by modulating the amplitude of carrier light with a chirp signal;
a light receiving element that receives the reflected light from the object and switches the photoelectric conversion operation by the same chirp signal as the illumination light, thereby also functioning as a mixer and converting the reflected light into a low-frequency interference fringe signal generated by heterodyning the chirp signal;
an imaging optical system that forms an image of the reflected light on the light receiving element;
a two-dimensional detection mechanism for detecting an image of external light having the same wavelength as the carrier light and a color image of the external light at the same time as detecting the interference fringe signal;
the wavelength of the carrier light is different from the wavelength of the color image;
The two-dimensional detection mechanism includes:
a first light receiving element constituting the light receiving element to which a wavelength filter that selectively transmits the wavelength of the carrier light is added, and a second light receiving element to which a primary color filter of the color image is added, the first and second light receiving elements being two-dimensionally arranged in a mosaic pattern on an imaging plane of the imaging optical system, or
a third light receiving element constituting the light receiving element arranged two-dimensionally to detect the interference fringe signal, on an imaging plane that can selectively transmit the carrier light by a multi-plate prism of a dichroic mirror, and a fourth light receiving element arranged two-dimensionally to detect a color image, on another imaging plane;
A distance detection device characterized in that the two-dimensional detection mechanism separates the outputs from the first and second light receiving elements that have passed through the wavelength filter and the primary color filter, or separates the outputs from the third and fourth light receiving elements. a first determination mechanism that determines, from an image of external light having the same wavelength as the carrier light detected by the two-dimensional detection mechanism and a color image of the external light, a position on the image of interference light that interferes with detection of the distance to the object, and a local position of the object;
4. The distance detection device according to claim 2, wherein the distance to the local position of the object is detected while avoiding the position of the interfering light. The distance detection device according to claim 4, characterized in that the two-dimensional detection mechanism has a fifth light receiving element that detects light in the optical wireless communication having a wavelength different from the wavelength of the carrier light and is arranged two-dimensionally, and the two-dimensional detection mechanism detects an image of external light having the same wavelength as the light in the optical wireless communication at the same time as receiving the light in the optical wireless communication by the fifth light receiving element. A distance detection device according to claim 5,
The present invention further includes a second determination mechanism that determines, from a color image of the outside light detected by the two-dimensional detection mechanism and an image of the outside light having the same wavelength as the light of the optical wireless communication, a position on the image of interference light that interferes with reception of the optical wireless communication and a position of a transmitter of the optical wireless communication that needs to be received, and the two-dimensional detection mechanism receives the light of the optical wireless communication while avoiding the position of the interference light, and the second determination mechanism:
acquiring location information of the transmitter through the optical wireless communication;
A distance detection device characterized by calculating the current positions of the object and the distance detection device from relative position information of the transmitter and the distance detection device detected by the second determination mechanism and relative position information of the object and the distance detection device.

Images