JP2018185179A - Distance measuring device, monitoring device, three-dimensional measuring device, mobile body, robot, and method for measuring distance - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distance measuring device which can increase the distance resolution in a desired distance range while suppressing reduction in the frame rate.SOLUTION: A distance measuring sensor 20 (distance measuring device) includes: a light source 21; an imaging sensor 29 (imaging element) for receiving light emitted from the light source 21 and reflected from an object and photoelectrically converting the light; a synchronization control unit 204 for controlling the exposure period for the image sensor 29 to the emission initiation timing of the light source 21 and causing the image sensor 29 to acquire electric signals (light receiving signals) generated by the photoelectric conversion during the exposure period divided into a plurality of phase signals; and an imaging processing unit 203 for calculating the distance to an object in the distance measuring range corresponding to the exposure period on the basis of the plurality of phase signals.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a distance measuring device, a monitoring device, a three-dimensional measuring device, a moving body, a robot, and a distance measuring method.

  2. Description of the Related Art In recent years, a distance measuring technique for measuring a distance to an object has been actively developed.

  For example, in Patent Document 1 and Non-Patent Document 1, light emitted from a light source and reflected by an object is received by an image sensor, and the distance to the object is based on a time difference between the light emission timing of the light source and the light reception timing of the image sensor. Ranging technology using a so-called TOF (Time of Flight) calculation method is calculated.

  However, the distance measurement techniques disclosed in Patent Document 1 and Non-Patent Document 1 have room for improvement in terms of improving distance resolution in a desired distance range while suppressing a decrease in frame rate.

  The present invention relates to a charge storage comprising a light source, a light receiving region for receiving and photoelectrically converting light emitted from the light source and reflected by an object, and a plurality of charge storage regions for storing charges generated by the photoelectric conversion. An image pickup device having a region group and a charge discharge region for discharging the charge, and a light emission start timing of the light source by switching the charge destination between the charge discharge region and the charge storage region group Controlling a period for sending the charge to the charge accumulation region group, and sending the charge generated in each of a plurality of different time zones in the period to any of the plurality of charge accumulation regions, And a calculation unit that calculates a distance to the object existing in a distance range corresponding to the period based on charge accumulation amounts of a plurality of charge accumulation regions.

  According to the present invention, it is possible to improve distance resolution in a desired distance range while suppressing a decrease in frame rate.

It is an external view of the traveling body carrying the distance sensor which concerns on one Embodiment of this invention. It is a block diagram for demonstrating the structure of a driving | running | working management apparatus. It is a figure for demonstrating the structure of a distance sensor. It is a figure for demonstrating a light projection system. It is a figure for demonstrating the light emission control signal. It is a figure for demonstrating a light source drive signal. It is a figure for demonstrating a light-receiving system. It is a figure which shows the flow of the signal in a distance sensor. It is a block diagram for demonstrating the structure of an audio | voice / alarm generator. It is a figure which shows an example of the pixel structure of an image sensor. FIG. 6 is a timing diagram illustrating a light emission period and an exposure period (charge accumulation period) of an image sensor. It is an image figure of the distance range corresponding to an exposure period. It is a figure for demonstrating the calculation method of Td. It is a figure which shows the example of a flame | frame structure which acquires several slice images corresponding to several distance ranges. FIG. 10 is a timing diagram (part 1) illustrating a light emission control signal and a TXD signal when acquiring a plurality of slice images. It is a figure which shows the correspondence of several slice images and several exposure periods. FIG. 10 is a timing diagram (part 2) illustrating a light emission control signal and a TXD signal when acquiring a plurality of slice images. FIG. 12 is a timing diagram (part 3) illustrating a light emission control signal and a TXD signal when acquiring a plurality of slice images.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows the appearance of a traveling body 1 equipped with a distance sensor 20 as a distance measuring device according to an embodiment. The traveling body 1 is for unattended transport of luggage to a destination. In the present specification, in the XYZ three-dimensional orthogonal coordinate system, the direction orthogonal to the road surface is described as the Z-axis direction, and the forward direction of the traveling body 1 is described as the + X direction.

  Here, as an example, the distance sensor 20 is attached to the front portion of the traveling body 1 and obtains three-dimensional information on the + X side (front) of the traveling body 1. Note that an area measurable by the distance sensor 20 is also referred to as a measurement area.

  As shown in FIG. 2 as an example, the traveling body 1 includes a display device 30, a position control device 40, a memory 50, a voice / alarm generating device 60, and the like. These are electrically connected via a bus 70 capable of transmitting data.

  Here, the travel management device 10 is configured by the distance sensor 20, the display device 30, the position control device 40, the memory 50, and the voice / alarm generation device 60. That is, the traveling management device 10 is mounted on the traveling body 1. The travel management device 10 is electrically connected to the main controller 80 of the traveling body 1.

  As shown in FIG. 3 as an example, the distance sensor 20 includes a light projecting system 201, a light receiving system 202, an image processing unit 203, a synchronization control unit 204, and the like. And these are accommodated in the housing | casing. This housing has a window through which light projected from the light projecting system 201 and light reflected by an object and directed toward the light receiving system 202 pass, and glass is attached to the window.

  The light projecting system 201 is disposed on the −Z side of the light receiving system 202. As shown in FIG. 4 as an example, the light projecting system 201 includes a light source 21 and a light source driving unit 25.

  The light source 21 is turned on and off by the light source driving unit 25. Here, an LED (light emitting diode) is used as the light source 21, but the present invention is not limited to this, and other light sources such as a semiconductor laser (an edge emitting laser or a surface emitting laser) may be used. The light source 21 is disposed so as to emit light in the + X direction. Hereinafter, a signal generated by the light source driving unit 25 and driving the light source 21 is referred to as a “light source driving signal”.

The light source drive unit 25 generates a light source drive signal (see FIG. 6) based on the light emission control signal from the synchronization control unit 204 (for example, a pulse signal having a pulse width T ₀ and a pulse period T ₁ , see FIG. 5). This light source drive signal is sent to the light source 21.

  As a result, the light source 21 emits pulsed light having a pulse width and a pulse period instructed from the synchronization control unit 204. The pulse light emitted from the light source 21 is set in the synchronization control unit 204 so that the duty is 50% or less. Hereinafter, the pulsed light emitted from the light source 21 is also referred to as a “projection wave” or a “projection pulse”.

  When the traveling body 1 travels, the main controller 80 of the traveling body 1 sends a position control start request to the position control device 40. Then, when the traveling body 1 reaches the target position, the main controller 80 of the traveling body 1 sends a position control end request to the position control device 40.

  When the position control device 40 receives a position control start request and a position control end request, the position control device 40 sends them to the image processing unit 203.

  Part of the light emitted from the distance sensor 20 and reflected by the object returns to the distance sensor 20. Hereinafter, for convenience, the light reflected by the object and returning to the distance sensor 20 is also referred to as “reflected light from the object” or “reflected light”.

  The light receiving system 202 detects reflected light from the object. As shown in FIG. 7 as an example, the light receiving system 202 includes an imaging optical system 28, an image sensor 29, and the like.

  The imaging optical system 28 is disposed on the optical path of the reflected light from the object and collects the light. Here, the imaging optical system 28 is composed of one lens, but it may be composed of two lenses, may be composed of three or more lenses, or uses a mirror optical system. May be.

  The image sensor 29 receives reflected light from the object via the imaging optical system 28. Here, as the image sensor 29, an area image sensor in which a plurality of light receiving portions (for example, PD: photodiode) is two-dimensionally arranged is used. Hereinafter, the reflected light from the object received by the image sensor 29 is also referred to as “received wave” or “received pulse”. The “light receiving part” of the image sensor 29 is also called “pixel”.

  The image sensor 29 photoelectrically converts light received by a plurality of pixels for each pixel, temporally divides the electric signal (light reception signal), and distributes the signal to a plurality of signals (phase signals) for each time.

More specifically, the image sensor 29 has FD1, FD2, and FD3 (see FIG. 10) as charge storage regions for each light receiving unit, and when the TXD signal shown in FIG. 8 is high, the charge is charged. Charge accumulation is performed when the TXD signal is low. “FD” is an abbreviation for floating diffusion.
More specifically, when the TXD signal goes from high to low, the TX1 signal goes from low to high, when the TX1 signal goes from high to low, the TX2 signal goes from low to high, and the TX2 signal goes from high. When it goes low, the TX3 signal goes from low to high, and when the TX3 signal goes from high to low, the TXD signal goes from low to high (see FIG. 11).
During the period when the TX1 signal is high, signal charges generated by photoelectric conversion in the light receiving unit are accumulated in the FD1.
During the period in which the TX2 signal is high, the signal charge generated by the photoelectric conversion in the light receiving unit is accumulated in the FD2.
During the period when the TX3 signal is high, signal charges generated by photoelectric conversion in the light receiving unit are accumulated in the FD3.
When the TXD signal changes from low to high, signal charges stored in FD1, FD2, and FD3 are read and reset.
As can be seen from the above description, the TX1 signal, the TX2 signal, and the TX3 signal are signals that are obtained by dividing the received light signal into three in terms of time.

  As shown in FIG. 8, when receiving the measurement start signal from the image processing unit 203, the synchronization control unit 204 outputs a light emission control signal to the light source driving unit 25 for each measurement and transmits the TXD signal, the TX1 signal, The TX2 signal and the TX3 signal are sequentially output to the image sensor 29. When the synchronization control unit 204 receives a measurement end signal from the image processing unit 203, the synchronization control unit 204 stops outputting the signals.

  Based on a plurality of phase signals for each pixel read by the image sensor 29 (using a relational expression described later), the image processing unit 203 determines the light emission timing (projected wave output timing) of the light source 21 and the image sensor. A time difference from the light reception timing (light reception wave input timing) of the corresponding 29 pixels is obtained, a distance to the object is calculated from the time difference, and a distance image (depth map) indicating the three-dimensional information of the object is generated to control the position. Output to the device 40.

  More specifically, the image processing unit 203 integrates a plurality of distance images corresponding to a plurality of distance ranges obtained by dividing the distance measurement range, as will be described later, and generates a distance image of the entire distance measurement range.

  Here, the image processing unit 203 is a component of the distance sensor 20, but instead of this, for example, an external device such as a PC may have the function of the image processing unit 203.

  Returning to FIG. 2, when the position control device 40 receives a distance image from the image processing unit 203, the position control device 40 displays the distance image on the display device 30. Further, the position control device 40 performs position control based on the distance image so that the position of the traveling body 1 becomes a predetermined position.

  As shown in FIG. 9 as an example, the voice / alarm generator 60 includes a voice synthesizer 61, an alarm signal generator 62, a speaker 63, and the like.

  The voice synthesizer 61 has a plurality of pieces of voice data. Upon receiving information that there is danger from the position controller 40, the voice synthesizer 61 selects corresponding voice data and outputs it to the speaker 63.

  When the warning signal generation device 62 receives information indicating that there is danger from the position control device 40, the warning signal generation device 62 generates a corresponding warning signal and outputs it to the speaker 63.

Next, an example of the pixel structure of the image sensor 29 will be described. FIG. 10 is a cross-sectional view of one pixel of the image sensor 29 when viewed in plan. As shown in FIG. 10, the image sensor 29 includes three gate structures for temporally distributing the received light signal and one gate structure for controlling the distributed period (charge accumulation period) of the received light signal for each pixel. have.
Note that although a pixel structure using an FD is used as the charge storage region here, a pixel structure using a capacitor as the charge storage region may be used.

  More specifically, in each pixel of the image sensor 29, a PD (photodiode) that forms a light receiving region and performs photoelectric conversion is transferred to FD1, FD2, and FD3 as charge storage regions via transfer gates TG1, TG2, and TG3, respectively. In addition to being connected, it is connected to the drain region DR (charge discharging region) via the drain gate DG.

  FD1 is connected to the drain region DR1 via the drain gate DG1. FD2 is connected to the drain region DR2 through the drain gate DG2. FD3 is connected to the drain region DR3 via the drain gate DG3.

Here, the TXD signal is a signal for turning on / off (opening / closing) the drain gates DG, DG1, DG2, and DG3. The TX1 signal is a signal for turning on / off (opening / closing) the transfer gate TG1. The TX2 signal is a signal for turning on / off (opening / closing) the transfer gate TG2. The TX3 signal is a signal for turning on / off (opening / closing) the transfer gate TG3. In each gate, “on” means “open” and “off” means “closed”.
During the period when the TXD signal is high (each drain gate is on) and the TX1, TX2, and TX3 signals are low, the charge generated by the photoelectric conversion in the PD is transferred to the drain region DR via the drain gate DG. Discharged.
During the period when the TXD signal is low (period in which each drain gate is off), the charge generated by photoelectric conversion at the PD during the period when the TX1 signal is high (period when the transfer gate TG1 is on) is transferred to the FD1 via the transfer gate TG1. The charge generated by the light source conversion in the PD during the period when the TX2 signal is high (the period when the transfer gate TG2 is on) is transferred and accumulated to the FD2 via the transfer gate TG2, and the TX3 signal is high. Charges generated by photoelectric conversion at the PD in the period (period in which the transfer gate TG3 is on) are transferred and accumulated in the FD 3 via the transfer gate TG3.

  When the TXD signal changes from low to high (when each drain gate is turned on from off), the charge accumulated in FD1 after the voltage of FD1 is read out via the drain gate DG1 The charge accumulated in FD2 after being discharged to DR1 and the voltage of FD2 being read out is discharged to the drain region DR2 via the drain gate DG2, and the charge accumulated in FD3 after the voltage of FD3 is read out It is discharged to the drain region DR3 through the drain gate DG3. That is, after the voltage of each FD is read, the accumulated charge amount of the FD is reset.

  FIG. 11 is a timing chart showing the relationship between the light emission period of the light source and the exposure period (charge accumulation period) of the image sensor 29. Note that since there is actually a circuit delay, it is necessary to adjust the circuit delay for the TXD signal. Here, it is assumed that there is no circuit delay.

  The image sensor 29 is formed with a light shielding layer other than on the light receiving region of the light receiving unit (pixel), and light is incident only on the light receiving region. As described above, the light incident on the light receiving unit is converted into electric charges (signal charges) by photoelectric conversion, and the electric charges are discharged to the drain region DR by bias control in the drain gate DG and the transfer gates TG1 to TG3, Sorted into FD1 to FD3.

  Here, the timing chart shown in FIG. 11 shows the timing of the bias (TXD signal, TX1 signal, TX2 signal, TX3 signal) applied simultaneously to all the pixels of the image sensor 29.

  The TXD signal has a so-called global shutter function and is a control signal for controlling the exposure period (charge accumulation period) with respect to the light emission start timing (rise timing of the light emission control signal) of the light source 21. Hereinafter, the time from the rising timing of the light emission control signal to the falling timing of the TXD signal is referred to as “delay time Tdelay” (see FIG. 11).

  During the period when the TXD signal is high, the charge of the light reception signal is constantly being swept out to the drain region DR, and the reset is continued. That is, the period when the TXD signal is high is a non-exposure period in the image sensor 29.

  Then, from the moment when the TXD signal changes from high to low, charge accumulation is started in the light receiving unit (or in FD1 to FD3 depending on the conditions of the TX1 signal to TX3 signal). That is, the period during which the TXD signal is low is the exposure period (charge accumulation period) in the image sensor 29.

As can be seen from the above description, the exposure periods of all the pixels of the image sensor 29 coincide with each other by the global shutter (exposure is started simultaneously and ended simultaneously).
In other words, all the pixels of the image sensor 29 have the same charge accumulation period due to the global shutter (charge accumulation starts simultaneously and ends simultaneously).

  Next, a period in which the TXD signal is low and the TX1, TX2, and TX3 signals are high will be described.

  The lengths of the high periods of the TX1 signal, the TX2 signal, and the TX3 signal are set to be equal to the pulse width Tw of the light emission control signal. At this time, the exposure period of each pixel (each light receiving unit) of the image sensor 29 is 3 Tw. Charges generated in each light receiving unit during the exposure period are transferred to the corresponding FD1, FD2, and FD3 during periods when the TX1, TX2, and TX3 signals are high.

  Here, the accumulated charge amounts of FD1, FD2, and FD3 after the above operation (light emission / light reception) is repeatedly performed a predetermined number of times are defined as A1, A2, and A3, respectively.

In the following description, it is assumed that there is no background light component. In this assumption, the luminance information obtained by mixing A1, A2, and A3 reflects only the reflected light in the range expressed by the following equation (1), where t is the delay time from the light emission timing. The
Tdelay-Tw <t <Tdelay + 3Tw (1)

The delay time t is an amount determined by TOF (Time of Flight) depending on the distance from the distance sensor 20 to the object. When converted to the distance d using the speed of light c, the following equation (2) is obtained. Expressed (considering the round trip of light).
(Tdelay−Tw) × c / 2 <d <(Tdelay + 3Tw) × c / 2 (2)
Accordingly, the image sensor 29 captures an object in the distance range closer to the distance d1 (= (Tdelay−Tw) × c / 2) and an object in the distance range farther than the distance d2 (= (Tdelay + 3Tw) × c / 2). Since the image does not appear in the image, for example, in a scene where fog is generated, it is possible to clearly (clearly) observe the state on the back side that is not normally visible due to the fog in front.
This principle is the basic principle of Gated Imaging described in Non-Patent Document 1.

  A distance image obtained by photographing only an image that exists only in a distance range corresponding to an exposure period that can be controlled by Tdelay and Tw as in the above equation (2) is referred to as a “slice image”.

  Here, according to the conventional gated imaging, the difference between the distance d1 and the distance d2 cannot be determined. However, in the present embodiment, the accumulated charge amounts A1, A2, and A3 are used as follows. It is possible to obtain distance information with finer resolution.

When Case1: A1> A3, ta is obtained by the following equation (3).
ta = Tdelay + A2 / (A1 + A2) × Tw (3)
At this time, Tdelay becomes the lower limit value.
Case2: When A1 <A3, ta is obtained by the following equation (4).
ta = Tdelay + {1 + A3 / (A2 + A3)} × Tw (4)
Here, ta is a TOF measured using the distance sensor 20, and the distance to the subject can be obtained using ta and the speed of light c.

However, the lower limit value in Case 1 is Tdelay, and all objects existing in a distance range corresponding to Tdelay-Tw <t <Tdelay, which is a range of Gated Imaging, correspond to Tdelay when considered under ideal conditions without measurement errors. Or a sufficient S / N cannot be obtained and accurate measurement cannot be performed.
Similarly, the upper limit value in Case 2 is Tdelay + 2Tw, and all objects existing in the distance corresponding to the range of Tdelay + 2Tw <t <Tdelay + 3Tw are Tdelay + 2Tw or sufficient when considered under ideal conditions without measurement error. S / N cannot be obtained and accurate measurement is not possible.

  For such a distance range, it is meaningful to make a determination of Tdelay or less or Tdelay + 2Tw or more, and as a workaround, it can be avoided by combining information captured in a plurality of gated imaging ranges. .

  Further, the case where the background light component is ideally absent at all has been described above. In Cases 1 and 2, the accumulated charge amount A3 in Case 1 and the accumulated charge amount A1 in Case 2 coincide with the amount of background light. Therefore, by subtracting these, it is possible to perform shooting and calculation without the influence even in an environment where background light exists.

Specific examples are shown below.
In the timing chart of FIG. 11, when Tdelay = 150 ns and Tw = 50 ns, the shortest distance that appears in the slice image to be taken is (Tdelay−Tw) / 2 × c = 15 m, and the longest distance is (Tdelay + 3Tw) / 2 ×. c = 45 m.
In the case of conventional Gated Imaging, it is only known that an object in a distance range of 15 m to 45 m is being photographed.
On the other hand, according to the present embodiment, it is possible to further measure the distance to an object in the distance range of Tdelay / 2 × c to (Tdelay + 2Tw) / 2 × c. In this example, it is possible to measure a distance to an object in a distance range of 22.5 m to 37.5 m. In addition, an object in a distance range of 15 m to 22.5 m is an object in a distance closer than 22.5 m, and an object in a distance range of 37.5 m to 45 mm is in a distance farther than 37.5 m. It is detected as an object (see FIG. 12).
FIG. 12 is an image diagram of the distance range included in the slice image of this example (Tdelay = 150 ns, Tw = 50 ns).

The reason why the distance of the object in the slice image can be measured by the distance sensor 20 will be described below with reference to FIG. FIG. 13 shows a case corresponding to the above Case1.
In FIG. 13, ta = Tdelay + Td [sec].
Tdelay is a value determined by the setting of Gated Imaging.
Td is the difference between Tde and the time ta from when the light is projected from the light projecting system 201 until the light reflected by the object in the distance range of Gated Imaging returns to the light receiving system 202.

  The conventional Gated Imaging technique cannot detect this Td.

On the other hand, in the present embodiment, it is possible to derive Td by detecting the reflected light signal divided in time by the TX1 signal, the TX2 signal, and the TX3 signal. Here, the case 1 described above (in the case of A1> A3) will be described as an example.
As shown in FIG. 13, Td is equal to the time when the pulse of the reflected light signal (pulse width is Tw) and the pulse of the TX2 signal overlap. Therefore, Td can be derived from the ratio of the accumulated charge amounts A1 and A2 of the TX1 signal and the TX2 signal, and can be calculated by the following equation (5).
Td = A2 / (A1 + A2) × Tw (= Q2 / (Q1 + Q2) × Tw) (5)
In this way, it is possible to measure the distance for each object that appears in one slice image, which cannot be measured by the conventional gated imaging, and far away (for example, up to several hundred meters) under poor visibility conditions. It is possible to obtain a distance image with a clear distance resolution with clear (clear) and high-speed imaging (for example, 30 fps or more).

  On the other hand, in the case of the conventional gated imaging method, in order to acquire a distance image with a fine distance resolution, it is necessary to synthesize a distance image obtained by many shootings, and high-speed imaging cannot be performed.

  By the way, when a distance measurement range is divided into a plurality of distance ranges and a plurality of slice images corresponding to the plurality of distance ranges are acquired, one slice image is acquired to cover the same distance measurement range. The distance range of each slice image can be narrowed. As the distance range of each slice image is narrower, it is possible to take an image under conditions more suitable for the distance range.

  FIG. 14 is a conceptual diagram when acquiring a plurality of slice images corresponding to a plurality of distance ranges. FIG. 15 shows a timing diagram of the light emission control signal and the TXD signal when a plurality of slice images are acquired (when X = Y = Z = 1 in FIG. 14). FIG. 16 shows the correspondence between a plurality of slice images and a plurality of exposure periods.

  Here, as a plurality of slice images, a short-distance side slice image that is a short-distance range slice image, a middle-distance side slice image that is a medium-distance range slice image, and a long-distance side slice image that is a long-distance range slice image An example of performing three-stage Gated Imaging is given as an example.

  As shown in FIG. 14 to FIG. 16, a short-distance side slice image acquisition frame, a medium-distance side slice image acquisition frame, and a long-distance side slice image acquisition frame are set continuously, and the frame is composed of these three frames. The group is executed repeatedly. The order of the three frames in this frame group can be changed as appropriate.

In the near-distance slice image acquisition frame, the exposure period Ta1 is set after the delay time Tdelay1 from the start of the light emission period Tw1, and the blank period Tb1 is set after the exposure period Ta1.
In the middle-distance slice image acquisition frame, the exposure period Ta2 is set after the delay time Tdelay2 from the start of the light emission period Tw2, and the blank period Tb2 is set after the exposure period Ta2.
In the long-distance slice image acquisition frame, the exposure period Ta3 is set after the delay time Tdelay3 from the start of the light emission period Tw3, and the blank period Tb3 is set after the exposure period Ta3.
Here, as an example, Tw1 = Tw2 = Tw3 and Ta1 = Ta2 = Ta3, and the pulse period of the light emission control signal is also constant.
In addition, the length of the distance range corresponding to an exposure period can also be varied between frames by varying the exposure period between frames.

  Here, as shown in FIGS. 15 and 16, it is preferable to satisfy Tdelay1 <Tdelay2 <Tdelay3. In this case, the distance measuring range of the distance sensor 20 can be increased.

When measuring with the distance measurement range divided into multiple distance ranges (when acquiring multiple slice images), the delay range and exposure period are set to the desired values because the distance range is determined by the delay time Tdelay and the exposure period. After that, it is preferable to set a blank period according to these set values.
The period for reading the voltage of each FD and the period for resetting the FD are after the last frame (X-th frame, Y-th frame, Z-th frame) in each distance range as shown in FIG. It may be provided, or may be provided within a blank period of each distance range as shown in FIG.

  Furthermore, by synthesizing a plurality of acquired slice images, a distance image of the entire distance measurement range can be obtained by integrating distance images photographed under conditions suitable for each distance range, and the signal amount (charge amount) on the short distance side Thus, it is possible to obtain a distance image having sufficient brightness even on the long distance side.

The distance range of each slice image is a distance range corresponding to the TOF from time Tdelay to time (Tdelay + exposure period).
In particular, during medium-range shooting or long-distance shooting, if the signal amount (charge amount) is reduced by receiving only one pulse of the projected wave, the light is emitted and received multiple times and the phase signal is accumulated. It is preferable to read out and reset the phase signal.
That is, since the signal light intensity decreases in inverse proportion to the square of the distance from the distance sensor 20, for example, as shown in FIG. 17, the number of times of light emission / light reception is increased as the far side is photographed (see FIG. 17). 14, the relationship X ≦ Y ≦ Z) is desirable.
Note that light emission and light reception may be performed a plurality of times during close-up shooting.

In FIG. 17, as an example, light emission and light reception are performed once in the short-distance side slice image acquisition frame, and light emission and light reception are repeated twice in the medium-distance side slice image acquisition frame. The number of repetitions of light emission and light reception is set to 3 in the acquisition frame.
In FIG. 17, in the short-distance slice image acquisition frame, a plurality of phase signals are read out and reset in a blank period Tb1 after one light emission / light reception.
In the middle-distance slice image acquisition frame, a plurality of phase signals are read out and reset in a blank period Tb2 after two light emission / light receptions.
In the long-distance slice image acquisition frame, a plurality of phase signals are read out and reset in a blank period Tb3 after three times of light emission / light reception.
The example of FIG. 17 is an example when X = 1, Y = 2, and Z = 3 in FIG. In FIG. 17, Tw1 <Tw2 <Tw3, Tdelay1 <Tdelay2 <Tdelay3, and Ta1 = Ta2 = Ta3.
In addition, the length of the distance range corresponding to an exposure period can also be varied between frames by varying the exposure period between frames.
Again, a frame group consisting of a short-distance slice image acquisition frame, a medium-distance slice image acquisition frame, and a long-distance slice image acquisition frame is repeatedly executed, but the order of the three frames in the frame group can be changed as appropriate. It is. The pulse cycle of the light emission control signal may be the same or different between frames.

In addition, here, when performing the distance calculation using the phase signals obtained by dividing the signals by the TX1 signal to the TXN signal into N, the light emission time of the light source (the pulse width of the light emission control signal) is theoretically determined. It is known that the shorter the distance, the higher the distance resolution.
However, if the light emission time of the light source is shortened, the rise / fall of the light waveform cannot catch up, and the output value of the light projection pulse is lowered, or the contrast of the image sensor distribution is lowered. In particular, it is known that S / N decreases.
In addition, as described above, the S / N is more severe as long distance imaging is performed. Therefore, it is preferable to shorten the pulse width during short distance imaging and increase the pulse width during long distance imaging. Therefore, for example, as shown in FIG. 18, it is preferable to satisfy Tw1 <Tw2 <Tw3.

The example of FIG. 18 is an example in the case of X = Y = Z = 1 in FIG. In FIG. 18, Tdelay1 <Tdelay2 <Tdelay3, Ta1 = Ta2 = Ta3, and the pulse period of the light emission control signal is constant.
In addition, the length of the distance range corresponding to an exposure period can also be varied between frames by varying the exposure period between frames.

Here, FIG. 16 shows an example in which the distance range of the short distance side slice image and the middle distance side slice image partially overlap, and the distance range of the middle distance side slice image and the far distance side slice image partially overlap. However, it is not always necessary to overlap.
However, by shooting so that an overlapping range occurs, for example, an object that appears only in the short-distance slice image, an object that appears in both the short-distance slice image and the medium-distance slice image, and the medium-distance slice image It is possible to divide the distance information to the object into three distance levels by using only the information of the gated imaging for the short-distance side slice image and the middle-distance side slice image. If the overlapping range is not set, if there are two slice images, it can be divided only into two distance levels.
Note that the end on the short distance side of the distance range is determined by the delay time Tdelay, and the length of the distance range (the distance from the short distance side end to the long distance end of the distance range) is determined by the exposure period. Therefore, the overlapping range (including 0) of two adjacent distance ranges can be adjusted by the delay time and the exposure period corresponding to each distance range.

By the way, “Active Gated Imaging technology” described in Non-Patent Document 1 makes it possible to take clear images far away even in the case of poor visibility such as rain and snow or fog.
That is, distance information of a plurality of distance ranges can be obtained by slicing (dividing) the distance measurement range into a plurality of distance ranges and acquiring a plurality of slice images by conventional Gated Imaging.
However, the distance information of each distance range is only distance information for each distance range, and is not distance information of an arbitrary position within the distance range.
Therefore, if the number of slices (number of slice images) in the distance measurement range is increased, the distance resolution can be increased.
However, since increasing the number of slices leads to an increase in the number of times of shooting, the shooting time required to acquire the entire image (all slice images) becomes longer.
That is, in the conventional gated imaging, the distance resolution and the frame rate are in a trade-off relationship.

  From the first viewpoint, the distance sensor 20 (ranging device) according to the present embodiment described above has a light source 21 and an image sensor 29 that receives and photoelectrically converts light emitted from the light source 21 and reflected by an object. (Image sensor) and the exposure period of the image sensor 29 with respect to the light emission start timing of the light source 21 are controlled, and the electric signal (light reception signal) generated by the photoelectric conversion in the exposure period is distributed to the plurality of phase signals. Ranging control unit 204 (control unit), and an image processing unit 203 (calculation unit) that calculates a distance to an object existing in a distance range corresponding to the exposure period based on a plurality of phase signals Device.

  From the second viewpoint, the distance sensor 20 (ranging device) according to the present embodiment includes a light source 21, a light receiving region that receives light emitted from the light source 21 and reflected by an object, and performs photoelectric conversion, and a photoelectric sensor. An image sensor 29 (imaging device) having a charge accumulation region group composed of a plurality of charge accumulation regions (for example, FD) for accumulating charges generated by the conversion and a charge discharge region (for example, a drain region) for discharging charges. ) And the charge destination are switched between the charge discharge region and the charge accumulation region group, thereby controlling the exposure period (period) in which the charge is transmitted to the charge accumulation region group with respect to the light emission start timing of the light source 21. Based on the synchronization control unit 204 (control unit) that sends charges generated in a plurality of different time zones within a period to any of the plurality of charge storage regions, and the charge storage amount of the plurality of charge storage regions A distance calculator 203 for calculating the distance to an object present in a distance range corresponding to the exposure period, a distance measuring device comprising a.

In the distance sensor 20 of the present embodiment, since the charge of the light reception signal obtained during the exposure period is detected by dividing it into a plurality of time components, the distance to the object existing in the distance range corresponding to the exposure period is measured. Can do.
That is, it is possible to measure the distance for each object existing in the desired distance range (shown in the slice image).

  As a result, according to the distance sensor 20 of the present embodiment, it is possible to improve the distance resolution in a desired distance range while suppressing a decrease in the frame rate.

  On the other hand, Patent Document 1 relates to a pixel structure provided with a drain structure for removing background light and a driving timing thereof in a general indirect TOF method, and obtains slice images for each distance as in Gated Imaging. However, it is possible to obtain only an image with high brightness at a short distance and low brightness at a long distance.

  Further, in Non-Patent Document 1, a sliced image captured at an exposure timing suitable for each distance range can be obtained by gated imaging, so that a clear image from the short distance side to the long distance side can be acquired. As for, only distance information indicating that it exists in a predetermined distance range can be obtained, and distance information for each object shown in one slice image cannot be obtained. For example, when a sliced image having a gated imaging condition of 5 to 15 m is acquired, it is not possible to distinguish the distance between an object having a distance of 5 m and an object having a distance of 15 m that are reflected in the image. Therefore, when a plurality of slice images are obtained by slicing (dividing) the distance range in order to improve the distance resolution in the desired distance range, the number of times of photographing increases and the reduction in the frame rate cannot be suppressed.

  That is, in Patent Document 1 and Non-Patent Document 1, there is room for improvement in improving distance resolution in a desired distance range while suppressing a decrease in frame rate.

  In addition, the synchronization control unit 204 controls a delay time Tdelay that is a time from the light emission start timing to the exposure period start timing using the control signal (TXD signal), and the image processing unit 203 uses the delay time and each phase signal. It is preferable to calculate the distance using the length Tw of the acquisition period (each time period) and the signal amount of each phase signal (charge accumulation amount of each charge accumulation region). In this case, highly accurate distance measurement can be reliably performed in a desired distance range.

  In addition, the synchronization control unit 204 controls a plurality of (n) phase signal acquisition periods (the above-described time period) using the control signal, and the exposure period has a length of the plurality of phase signal acquisition periods (the above-described time periods). It is preferable that the length is a sum of the lengths Tw of the band). In this case, a plurality of phase signals can be efficiently acquired in a limited exposure period.

The time from the light emission start timing to the exposure period start timing between two temporally adjacent frames each including at least one set of the light emission period Tw and the exposure period corresponding to the light emission period. (Delay time) may be varied. In this case, the distance to an object existing in two different distance ranges in two temporally adjacent frames can be measured. Furthermore, in this case, since the measurement can be performed under conditions suitable for each distance range, the distance to the object existing in each distance range can be calculated with high accuracy.
Specifically, a distance image with a fine distance resolution can be obtained not only under good visibility conditions but also under poor visibility conditions. It is possible.

  Further, the distance ranges corresponding to the exposure periods of two temporally adjacent frames may partially overlap.

  Further, the number of the sets may be different between two temporally adjacent frames. In this case, measurement can be performed under suitable conditions for each distance range.

In addition, it is preferable that the frame having the shorter delay time of the two frames adjacent in time has a smaller number of the above-mentioned groups than the frame having the longer delay time.
In other words, when measuring the distance to the object for each distance range, the synchronization control unit 204 decreases the number of repetitions of light emission of the light source 21 and light reception of the image sensor 29 in the distance range closer to the distance range. It is preferable to do.
In this case, it is possible to suppress the charge saturation on the short distance side and improve the ranging accuracy on the long distance side.

In addition, it is preferable that the frame with the shorter delay time of two temporally adjacent frames has a shorter light emission period (light emission time) of the light source 21 than the frame with the longer delay time.
In other words, when measuring the distance to the object for each distance range, the synchronization control unit 204 shortens the light emission time (pulse width of the light emission control signal) of the light source 21 in the distance range closer to the short side among the plurality of distance ranges. It is preferable to do. In this case, it is possible to suppress the charge saturation on the short distance side and improve the ranging accuracy on the long distance side.

  The image sensor 29 preferably includes a plurality of light receiving units arranged in a two-dimensional array. In this case, a two-dimensional distance image can be generated.

  The image sensor 29 may include a single light receiving unit, or may include a plurality of light receiving units arranged in a one-dimensional array.

  Moreover, according to the traveling body 1 (moving body) having the distance sensor 20, the traveling body 1 can travel safely with respect to an object existing at an arbitrary position within a desired distance range.

  Further, from the first viewpoint, the distance measuring method of the present embodiment receives light emitted from the light source 21 and reflected by the object by the image sensor 29 (imaging device) and photoelectrically converts the distance to the object. A distance measuring method for measuring, which controls the exposure period of the image sensor 29 with respect to the light emission start timing of the light source 21, and causes the image sensor 29 to acquire an electric signal generated by photoelectric conversion during the exposure period by dividing it into a plurality of phase signals. And a step of calculating a distance to an object existing in a distance range corresponding to an exposure period based on a plurality of phase signals.

  From the second viewpoint, the distance measuring method according to the present embodiment includes an image sensor having a light source 21, a charge accumulation region group including a plurality of charge accumulation regions (for example, FD), and a charge discharge region (for example, a drain region). 29 (imaging device), a step of receiving the light emitted from the light source 21 and reflected by the object by the image sensor 29 and sending the charge generated by the photoelectric conversion to the charge discharge region; And sending the charge generated in each of a plurality of different time zones within the exposure period (predetermined period) to any one of the plurality of charge accumulation regions. The distance measurement method further includes a step of calculating a distance to an object existing in a distance range corresponding to an exposure period based on the charge accumulation amounts of the plurality of charge accumulation regions.

  In this case, it is possible to improve the distance resolution in a desired distance range while suppressing a decrease in the frame rate.

  In the above embodiment, the light projecting system is a non-scanning type, but may be a scanning type including an optical deflector (for example, a polygon mirror, a galvano mirror, a MEMS mirror, etc.). In this case, for example, scanning a plurality of lights respectively emitted from a plurality of light emitting units (line light sources) arranged in one direction in a direction (for example, a vertical direction) non-parallel to the arrangement direction of the light emitting units, A distance image may be generated by receiving light with a plurality of light receiving units (line image sensors) arranged in parallel to the arrangement direction corresponding to the plurality of light emitting units. Alternatively, the distance image may be generated by two-dimensionally scanning the light from the single light emitting unit with the light deflecting unit and receiving the reflected light from the object with the area image sensor.

  Moreover, although the said embodiment demonstrated the case where the distance sensor 20 which is an example of the distance measuring device of this invention was used for the traveling body 1, it is not limited to this. For example, the distance sensor 20 is a moving body other than the traveling body 1 (for example, a passenger car, a ship, an aircraft, etc.), a monitoring device, a three-dimensional measuring device that measures the three-dimensional shape of an object, and the distance sensor 20 confirms its position. However, you may use for the robot etc. which move autonomously.

  Therefore, according to the monitoring device having the distance sensor 20, it is possible to obtain a high-quality monitor image of an object existing at an arbitrary position within a desired distance range.

  In addition, according to the three-dimensional measurement apparatus having the distance sensor 20, it is possible to accurately measure the three-dimensional information of an object existing at an arbitrary position within a desired distance range.

  Moreover, according to the robot having the distance sensor 20, appropriate autonomous movement (approach operation, separation operation, and parallel movement) with respect to an object existing at an arbitrary position within a desired distance range can be performed.

  Further, for example, an APD (avalanche photodiode), a phototransistor, or the like may be used as the light receiving unit instead of the PD.

  In addition, the number of temporal divisions of the received light signal is not limited to the number described in the above-described embodiment, and may be any plural number.

  In addition, the image sensor has a gate structure for controlling the number of gate structures (number of charge storage regions) for temporally distributing the received light signals and the distribution period (charge storage period) of the received light signals, for each pixel. The number (number of charge discharge regions) is not limited to the number described in the above embodiment, and can be changed as appropriate.

  Moreover, although the said embodiment demonstrated the case where a single LED (light emission part) light-emitted a pulse and received the reflected light from an object with an area image sensor, it is not limited to this.

  For example, a plurality of light emitting units arranged two-dimensionally may be sequentially pulsed, and light emitted from each light emitting unit and reflected by an object may be sequentially received by a single light receiving unit.

  For example, a plurality of light emitting units arranged two-dimensionally may simultaneously emit pulses, and a plurality of lights emitted from the plurality of light emitting units and reflected by an object may be simultaneously received by the plurality of light receiving units arranged two-dimensionally. .

  For example, when measuring the distance to a certain object instead of the three-dimensional information (distance image) of the object, both the light emitting unit of the light projecting system and the light receiving unit of the light receiving system may be singular. good.

  In the above embodiment, the position control device 40 may perform part of the processing in the image processing unit 203, or the image processing unit 203 may perform part of the processing in the position control device 40.

  Moreover, although the said embodiment demonstrated the case where the driving | running | working management apparatus 10 was provided with the one distance sensor 20, it is not limited to this. A plurality of distance sensors 20 may be provided according to the size of the traveling body, the measurement area, and the like.

  Moreover, although the said embodiment demonstrated the case where the distance sensor 20 was used for the traveling management apparatus 10 which monitors the advancing direction of a traveling body, it is not limited to this. For example, you may use for the apparatus which monitors the back and side surface of a traveling body.

  As can be seen from the above description, the distance measuring device and the distance measuring method of the present invention can be widely applied to all distance measuring techniques using TOF (Time of Flight).

  That is, the distance measuring apparatus and the distance measuring method of the present invention can also be used for acquiring two-dimensional information of an object and detecting the presence or absence of the object.

  The numerical values, shapes, and the like used in the description of the above embodiments can be changed as appropriate without departing from the spirit of the present invention.

Below, the thought process which the inventor came up with the said embodiment is demonstrated.
In TOF sensing, if the amount of accumulated background light is large, the S / N deterioration due to the shot noise and the dynamic range are reduced (the charge storage unit has an upper limit, so if the background light component accumulates there, the signal component Adversely occurs). Therefore, it is necessary to provide a TOF sensor structure and driving timing capable of removing background light by a drain structure for each pixel.

  Non-Patent Document 1 is a paper in which an outline of Gated Imaging and an effect confirmation result are described. By using the gated imaging, it is possible to take an image under an appropriate exposure condition for a desired distance range in the distance measurement range, and clear image sensing from the short distance side to the long distance side is possible. In addition, even when fog is generated, it is possible to photograph only objects within a desired distance range, so that the image of the near mist does not appear in the image, and clear image sensing up to a distance is possible. Is possible.

  However, when using Gated Imaging, there is a concern that if the number of divisions of the distance measurement range (number of distance ranges) is increased in order to improve the distance resolution, the number of shootings will increase (the frame rate will decrease). The

  In addition, since the conventional active sensor has higher brightness as the object at the short distance side, when attempting to photograph the object at the long distance side, the object at the short distance side is overexposed, and saturation and blooming are likely to occur. There are cases where clear image sensing is not possible.

  Therefore, the inventor has devised the above-described embodiment in order to realize image sensing that enables clear (clear) high-speed imaging from the short-distance side to the long-distance side and high-resolution resolution.

  DESCRIPTION OF SYMBOLS 1 ... Running body (moving body), 20 ... Distance sensor (ranging device), 21 ... Light source, 29 ... Image sensor (imaging element), 203 ... Image processing part (calculation part), 204 ... Synchronization control part (control part) ).

Japanese Patent No. 5110520 Paper "Active Gated Imaging for Automotive Safety Applications" BrightWay Vision (SPIE IS & T /Vol.9407 94070F-18)

Claims (15)

A light source;
A light receiving region for receiving and photoelectrically converting light emitted from the light source and reflected by an object, a charge storage region group including a plurality of charge storage regions for storing charges generated by the photoelectric conversion, and the charge An image pickup device having a charge discharge region for discharging;
By switching the charge destination between the charge discharge region and the charge accumulation region group, a period during which the charge is sent to the charge accumulation region group with respect to the light emission start timing of the light source is controlled. A controller that sends the charge generated in each of a plurality of different time zones to any one of the plurality of charge accumulation regions;
A distance measuring apparatus comprising: an arithmetic unit that calculates a distance to the object existing in a distance range corresponding to the period based on a charge accumulation amount of the plurality of charge accumulation regions. The control unit controls a delay time that is a time from the light emission start timing to the start timing of the period using a control signal,
The distance measuring apparatus according to claim 1, wherein the calculation unit calculates the distance using the delay time, the length of the time zone, and the charge accumulation amount. The control unit controls the plurality of time zones using a plurality of control signals,
The distance measuring apparatus according to claim 1, wherein the length of the period is a length obtained by adding the lengths of the plurality of time zones.   The time from the light emission start timing to the period start timing differs between two temporally adjacent frames each including at least one set of the light emission period and the period corresponding to the light emission period. The distance measuring device according to any one of claims 1 to 3, wherein   The distance measuring apparatus according to claim 4, wherein the distance ranges corresponding to the periods of the two frames partially overlap.   6. The distance measuring device according to claim 4, wherein the number of the sets is different between the two frames.   The distance measurement according to any one of claims 4 to 6, wherein the frame having the shorter time among the two frames has a smaller number of the sets than the frame having the longer time. apparatus.   8. The frame according to claim 4, wherein the shorter one of the two frames has a shorter light emission period than the longer frame. 9. Distance measuring device.   The distance measuring device according to claim 1, wherein the image pickup device includes a plurality of light receiving portions arranged in a two-dimensional array.   The distance measuring device according to claim 1, wherein the image sensor includes a single light receiving unit.   A monitoring device comprising the distance measuring device according to claim 1.   A three-dimensional measuring device comprising the distance measuring device according to claim 1.   A moving body comprising the distance measuring device according to claim 1.   A robot having the distance measuring device according to claim 1. A distance measuring method using a light source and an image sensor having a charge accumulation region group and a charge discharge region composed of a plurality of charge accumulation regions,
Receiving the light emitted from the light source and reflected by the object with the imaging device and sending the charge generated by photoelectric conversion to the charge discharge region;
Sending the charge to the charge storage region group;
In the step of sending to the charge accumulation region group, the charge generated in each of a plurality of different time zones within a predetermined period is sent to any of the plurality of charge accumulation regions,
A distance measuring method further comprising a step of calculating a distance to the object existing in a distance range corresponding to the predetermined period based on a charge accumulation amount of the plurality of charge accumulation regions.

Images