JP2020187762A - Vehicle monitoring system - Google Patents

Abstract

To provide means for providing situation recognition and collision avoidance.SOLUTION: Vehicle monitoring uses three-dimensional (3D) information of an area adjacent to a vehicle to visually highlight an object closer to the vehicle than a threshold distance. A vehicle monitoring system 100 includes a 3D scanner 110 for scanning the area adjacent to the vehicle and providing a 3D model including space constitution of an object located in a scanning area 102. The vehicle monitoring system further includes an electronic display 120 for displaying a part of the scanning area and visually highlighting the object in a display area located closer than the threshold distance from the vehicle by using the 3D model.SELECTED DRAWING: Figure 1

Description

Cross-reference of related applications Not applicable

Federally funded R & D description Not applicable

Vehicle monitoring systems are becoming more and more popular as a means of providing situational awareness and collision avoidance. Such vehicle monitoring systems often include electronic displays for transmitting information to users (eg, vehicle drivers). In particular, the electronic display can provide a visual image of the area adjacent to the vehicle to make the user aware of the object in the adjacent area and facilitate the user to avoid it.

Cathode ray tube (CRT), plasma display panel (PDP), liquid crystal display (L)
CDs), electronic emission (EL) displays, organic light emitting diode (OLED) and active matrix OLED (AMOLED) displays, electrophoresis (EP) displays, and electromechanical or electrofluid light modulation (eg, digital micromirror devices, etc.)
A wide variety of electronic displays, including, but not limited to, displays based on various displays using (such as electrowetting displays) can be used in vehicle surveillance systems. In general, electronic displays can be categorized as active displays (ie, displays that emit light) or passive displays (ie, displays that modulate the light produced by another source). The most obvious examples of active displays are CRTs, PDPs, and OLEDs / AMOLEDs.
The displays that are usually classified as passive when considering the emitted light are LCD and EP displays. Passive displays often offer attractive performance characteristics, including their inherently low power consumption, but given their inability to emit light, they are often practical. Use may be somewhat restricted in the application.

To overcome the limitations of passive displays associated with emitted light, many passive displays are coupled to an external light source. A coupled light source may allow these normally passive displays to radiate light and effectively function as active displays. An example of such a connected light source is a backlight. The backlight is a light source (often a panel light source) that is usually arranged behind the passive display to illuminate the passive display. For example, the backlight is L
It may be connected to a CD or EP display. The backlight emits light that passes through the LCD or EP display. The emitted light is modulated by the LCD or EP display, and the modulated light is then emitted from the LCD or EP display. Backlights are often configured to emit white light. In that case, color filters are used to convert the white light into the various colors used in the display. Color filters may be located, for example, on the output of an LCD or EP display (less common), or between the backlight and the LCD or EP display.

Various features of examples and embodiments according to the principles described herein are by reference to embodiments for carrying out the invention below, in conjunction with accompanying drawings in which similar reference numbers represent similar structural elements. It can be understood more easily.

It is a block diagram of the vehicle monitoring system in one example by one Embodiment which is consistent with the principle described in this specification. FIG. 5 is a perspective view of a region monitored by using the vehicle monitoring system of FIG. 1 in an example according to an embodiment consistent with the principles described herein. It is a side view of the monitoring area of FIG. 2A in one example according to one embodiment consistent with the principle described herein. It is a figure which shows the display part of the scanned area in one example by one Embodiment which was consistent with the principle described in this specification. (B) It is a figure which shows the visually emphasized object in the display part 108 of FIG. 3A in one example by one Embodiment which is consistent with the principle described herein. (C) FIG. 6 shows a visually emphasized object within the display portion 108 of FIG. 3A in an example, according to another embodiment consistent with the principles described herein. FIG. 3 is a perspective view of a 3D electronic display depicting a visually emphasized object in an example, according to another embodiment consistent with the principles described herein. FIG. 5 is a cross-sectional view of a three-dimensional (3D) electronic display with a multi-beam grid-based backlight in one example, according to an embodiment consistent with the principles described herein. (B) FIG. 6 is a cross-sectional view of a three-dimensional (3D) electronic display with a multi-beam grid-based backlight in one example, according to another embodiment consistent with the principles described herein. (C) FIG. 3 is a perspective view of a portion of a 3D electronic display with a multi-beam grid-based backlight in an example, according to an embodiment consistent with the principles described herein. FIG. 3 is a block diagram of a three-dimensional (3D) vehicle monitoring system in an example according to an embodiment of the principles described herein. It is a flowchart of the vehicle monitoring method in one example by one Embodiment which is consistent with the principle described in this specification.

Some examples and embodiments have other features that are one of the additions and substitutions of the features shown in the figures referenced above. These and other features are detailed below with reference to the figures referenced above.

One embodiment consistent with the principles described herein provides a vehicle monitoring system that uses three-dimensional (3D) information. In particular, according to some embodiments of the principles described herein.
3D information with 3D scanning or images is collected for a surveillance area adjacent to the vehicle.
For example, the monitoring area can be one or more of the front, side, and rear of the vehicle. The 3D information from the monitoring area is then used to build a 3D model that includes the spatial composition of the objects in the monitoring area. Further, a part of the monitoring area based on the 3D model is displayed, and an object in the display part closer than a predetermined threshold distance from the vehicle is a visually emphasized object. According to various embodiments, the perception of the visually emphasized object by the user can be improved. Further, for example, improved perception of visually emphasized objects can facilitate collision avoidance for visually emphasized objects that are too close to the vehicle.

According to various embodiments, the 3D scanner or 3D camera is combined with an electronic display as a vehicle monitoring system for monitoring an area adjacent to the vehicle. The 3D scanner collects 3D information about objects in the surveillance area to facilitate the construction of 3D models. The electronic display provides an image of the surveillance area based on the 3D model. In addition, the electronic display provides visual enhancement of objects that are considered too close to the vehicle (ie, objects that are closer than a predetermined threshold distance from the vehicle).

According to some embodiments described herein, visual enhancement of an object is provided by a 3D electronic display (single color and / or color). In yet various embodiments, the 3D electronic display may be configured to present images and related information in a so-called "naked eye" 3D or autostereoscopic 3D embodiment.
In particular, in some embodiments, the 3D electronic display may use a grating-based or "grating-based" backlight to generate 3D images or different visual images of information. In a 3D electronic display that uses a grating-based backlight, light is coupled out of the light guide using multiple gratings. The light that is coupled and emitted to the outside forms a plurality of light beams directed in a predefined direction (for example, a viewing direction). Further, the light beams among the plurality of light beams may have different main maximum angular directions to form or provide a light irradiation field in the viewing direction of the electronic display, and in some embodiments, the plurality of light beams may be formed or provided. It may represent the primary color. Light beams with different principal maximal directions (also referred to as "light beams directed in different directions"), and light beams representing different colors in some embodiments, autoautomate information, including stereoscopic (3D) information. Can be used to display stereoscopically. For example, light beams of different colors directed in different directions are modulated and "
It may function as a color pixel of a different visual image of a "naked eye" 3D electronic display, or may represent the different visual image.

As used herein, a "light guide" is defined as a structure that guides light within a structure using total internal reflection. In particular, the light guide may include a core that is substantially transparent at the operating wavelength of the light guide. In various embodiments, the term "light guide" generally refers to internal total internal reflection to guide light at the interface between the dielectric material of the light guide and the material or medium surrounding the light guide. Refers to a dielectric optical waveguide that uses. By definition
The condition for total internal reflection is that the index of refraction of the light guide is greater than the index of refraction of the surrounding medium adjacent to the surface of the light guide material. In some embodiments, the light guide may include a coating in addition to or instead of the refractive index differences described above to further facilitate total internal reflection. The coating may be, for example, a reflective coating.
The light guide can be any of several light guides, including but not limited to flat or slab guides and / or strip guides.

Further herein, the term "flat plate" is defined as a piecewise or individually planar layer or sheet when applied to a light guide, such as "flat plate light guide", which is sometimes referred to as "flat plate". Called a "slab" guide. In particular, a flat plate light guide is defined as a light guide configured to guide light in two substantially orthogonal directions demarcated by the top and bottom surfaces (ie, opposite surfaces) of the light guide. Orthogonal. Moreover, by definition herein, both the top and bottom surfaces are separated from each other and can be substantially parallel to each other, at least in their individual sense. That is, the top and bottom surfaces are substantially parallel or coplanar within any of the individually small compartments of the flat plate light guide.

In some embodiments, the flat plate light guide may be substantially flat (ie, limited to a flat surface), thus the flat plate light guide is a flat surface light guide. In other embodiments, the flat plate light guide may be curved in one or two orthogonal dimensions. For example, the flat plate light guide may be curved in a single dimension so as to form a cylindrical flat plate light guide. However, both curvatures ensure that total internal reflection is maintained within the flat plate light guide to guide the light.
It has a sufficiently large radius of curvature.

According to the various embodiments described herein, a diffraction grating (eg, a multi-beam diffraction grating) scatters or couples light out of a light guide (eg, a flat plate light guide) as a light beam. It may be used to put out. In the present specification, a "diffraction grating" is generally defined as a plurality of feature portions (that is, diffraction feature portions) arranged so as to realize diffraction of light incident on the diffraction grating. In some embodiments, the plurality of features
It may be arranged periodically or quasi-periodically. For example, the grating may include a plurality of features (eg, a plurality of grooves on the surface of the material) arranged in a one-dimensional (1D) array. In another example, the grating may be a two-dimensional (2D) array of features. For example, the grating may be a 2D array of protrusions or holes on the surface of the material.

As described above, and by definition in the present specification, the "diffraction grating" is a structure that realizes diffraction of light incident on the diffraction grating. When light enters the grating from the light guide, the diffraction or diffraction scattering realized there can result in diffraction coupling, which is therefore called "diffraction coupling", where the grating cups the light from the light guide by diffraction. You can ring and go outside. The grating also redirects or changes the angle of light (ie, the diffraction angle) by diffraction. In particular, as a result of diffraction, light exiting the diffraction grating (ie, diffracted light) generally has a propagation direction different from the propagation direction of light incident on the grating (ie, incident light). In the present specification, the change in the direction of light propagation due to diffraction is referred to as "diffractive redirection". Therefore, it can be understood that the diffraction grating is a structure including a diffraction feature portion that changes the direction of the light incident on the diffraction grating by diffraction, and when the light is incident from the light guide, the diffraction grating is a light guide. It is also possible to couple the light from the grating by diffraction and let it go out.

Further, by definition herein, a feature of a diffraction grating is called a "diffraction feature" and is a surface (ie, where "surface" refers to the boundary between two materials) within the surface. , And one or more on the surface (at, in and on). This surface can be the surface of a flat plate light guide. The diffraction feature may include any of a variety of light diffracting structures, including but not limited to, one or more of grooves, ridges, holes, and protrusions. The structure may be one or more of the surface, the surface, and the surface. For example, the grating may include a plurality of parallel grooves in the surface of the material. In another example, the grating can include multiple parallel ridges rising from the surface of the material. Diffraction features (even grooves, ridges, holes, protrusions, etc.) are sinusoidal contours,
Various cross-sectional shapes or contours that provide diffraction, including, but not limited to, one or more of rectangular contours (eg, binary gratings), triangular contours, and serrated contours (eg, blazing gratings). You can have any of them.

As defined herein, a "multi-beam diffraction grating" is a diffraction grating that includes a plurality of light beams to generate light that is coupled and emitted to the outside. Further, the plurality of light beams generated by the multi-beam diffraction grating have different principal angular directions by definition in the present specification. In particular, by definition, as a result of diffractive coupling and diffraction redirection of incident light by a multi-beam grating, one light beam of multiple light beams is with another light beam of multiple light beams. Has different predetermined principal maximal angular directions. The plurality of light beams may form a light irradiation field. For example, the plurality of light beams may include eight light beams having eight different principal maximal angular directions. For example, eight combined light beams (ie, multiple light beams) are combined into a light field.
) May be expressed. According to various embodiments, the different principal maximal angular directions of the various light beams are by combining the lattice pitch or spacing of the diffraction gratings of the multi-beam grating at the origin of each light beam with orientation or rotation. , Determined with respect to the propagation direction of the light incident on the multi-beam diffraction grating.

According to the various embodiments described herein, the light that is coupled by a diffraction grating (eg, a multi-beam diffraction grating) and emitted out of the light guide represents a pixel in an electronic display. In particular, light guides with multi-beam diffraction gratings for generating multiple light beams with different principal maximal orientations are "naked-eye" three-dimensional (3D) electronic displays (multi-view or "holographic" electronic displays, or It can be part of an electronic display backlight, such as (also called an autostereoscopic display), but not limited to these, or a backlight used in conjunction with those electronic displays. Therefore, the light beams directed in different directions generated by coupling the waveguide light from the light guide to the outside using a multi-beam diffraction grating can be the "pixels" of the 3D electronic display. Alternatively, it can represent a "pixel".

As used herein, a "collimation" mirror is defined as a mirror having a curved shape configured to collimate the light reflected by the collimation mirror.
For example, the collimation mirror may have a reflective surface characterized by a paraboloidal curve or parabolic shape. In another example, the collimation mirror may include a formed parabolic mirror. A "formed paraboloid" is an embodiment in which the curved reflective surface of the formed parabolic mirror is determined to achieve a predetermined reflection characteristic (eg, degree of collimation) and is "true". It means that it deviates from the parabolic curve. In some embodiments, the collimation mirror may be a continuous mirror (ie, having a substantially smooth continuous reflective surface), in other embodiments the mirror is a Fresnel reflection that achieves light collimation. It may be equipped with a body or Fresnel mirror. According to various embodiments, the amount of collimation achieved by the collimation mirror may vary by a predetermined degree or amount depending on the embodiment. Further, the collimation mirror may be configured to achieve collimation in one or both of two orthogonal directions (eg, vertical and horizontal).
That is, according to various examples, the collimation mirror may include a paraboloid or a paraboloid formed in one or both of the two orthogonal directions.

As used herein, the term "zero disparity plane", when used with respect to a 3D electronic display, is used in all visions of the 3D electronic display within the displayed or rendered 3D scene or region. It is defined as a flat or flat section that looks identical (ie, has no visual difference). Moreover, by definition herein, the zero parallax plane appears to be, correspond to, or coincide with, as it is on the physical surface of a 3D electronic display. That is, objects in a display scene or region located in the zero parallax plane within the 3D region are rendered by the 3D electronic display so that they are juxtaposed on the physical surface of the 3D electronic display when viewed on it. appear. Objects farther than the zero parallax plane appear to be behind the physical surface, and objects closer to the zero parallax surface appear to be in front of the physical surface.

As used herein, a "projective transformation" or, similarly, a "projective transform" is a (possibly non-linear) mapping of lines to lines (or rays to rays) in 3D space. ) Defined as a transformation. It should be noted that the projective transformation can be generally represented in terms of a linear transformation with a 4x4 matrix in four-dimensional (4D) space (ie, "projection space"). In some embodiments, the projective transformation may include an optical transformation configured to compress depth content in a manner substantially similar to viewing a scene through a divergent lens (eg, a fisheye lens). Especially for projective transformation
The infinite far plane is mapped to the desired distance 1 / h from the zero parallax plane. The projective transformation that provides the infinity plane for 1 / h distance mapping is according to equation (1).

Can be given as, where (x', y', z', w') is the image coordinate corresponding to the projective transformation of the coordinates (x, y, z, w). Moreover, by definition herein, the projective transformation of equation (1) generally does not compress the depth or parallax near the zero parallax plane itself. In other embodiments, another optical transformation (eg, represented by a 4x4 matrix) may be used as the projective transformation herein. According to some embodiments of the principles described herein, for example, a projective transformation can be a substantially arbitrary projective transformation that emphasizes an object or a portion thereof, as described below. Further according to various embodiments, the projective transformations herein may include either linear or non-linear transformations.

Further, as used herein, the article "a" is intended to have the usual meaning in patented technology, i.e., "one or more." As used herein, for example, "a grating" means one or more grids, and thus "the grating" means "one or more grids". (The grating (s)) "means. In addition, "top", "bottom", "upper", "lower", "up", "down", "down" in the present specification. "Front", "back", "first", "second", "left"
No reference to, or "right" is intended to be limiting herein. As used herein, the term "about" means, when applied to a value, generally within the tolerances of the equipment used to generate that value, or unless explicitly specified otherwise. , ± 1
It may mean 0%, or ± 5%, or ± 1%. Moreover, as used herein, the term "substantially" means the majority, or almost all, or all, or an amount in the range of about 51% to about 100%. Moreover, the examples herein are provided by way of illustration only for purposes of consideration and are intended not for limitation.

According to some embodiments of the principles described herein, a vehicle monitoring system is provided. FIG. 1 shows a block diagram of a vehicle monitoring system 100 in an example according to an embodiment consistent with the principles described herein. FIG. 2A shows a region 1 monitored using the vehicle monitoring system 100 of FIG. 1 in an example according to an embodiment consistent with the principles described herein.
The perspective view of 02 is shown. FIG. 2B is based on an embodiment consistent with the principles described herein.
A side view of the monitoring area 102 of FIG. 2A in one example is shown. As defined herein, an "area" monitored by the vehicle monitoring system 100 is an area near, adjacent to, or around the vehicle (hereinafter, "area" is collectively referred to as an "adjacent area". Is sometimes called). According to various embodiments, the monitored area 102 may include one or more of the front of the vehicle, the rear of the vehicle, and the sides of the vehicle.

For example, the vehicle monitoring system 100 may be used to monitor the area 102 behind the vehicle and thus functions as a reverse or rear view monitoring system. In particular, the vehicle monitoring system 100 may be configured as a rear view or reverse assisted vehicle monitoring system to assist in avoiding a collision when the vehicle is moving backwards. In another example, the monitored area 102 may be in front of the vehicle. Therefore, for example, the vehicle monitoring system 1
00 may function as a front end collision avoidance system for when the vehicle is moving forward.

As shown in FIG. 1, the vehicle monitoring system 100 includes a three-dimensional (3D) scanner 110. The 3D scanner 110 is configured to scan the area 102 adjacent to the vehicle. In this case, scanning by a 3D scanner is used to generate or provide a 3D model of the area 102. In particular, the 3D model includes the spatial configuration of the object 104 located within the scanned area 102. The scanned area 102 adjacent to the vehicle may also be referred to herein as the "scanning" area 102, or, likewise, the "imaging" area 102.

In general, the 3D scanner 110 may include any of a variety of different 3D scanning systems or imaging systems capable of determining distances to various objects 104 within the scanning area 102. According to some embodiments, the 3D scanner 110 comprises a plurality of cameras offset from each other. The distance from the 3D scanner 110 to the object 104 in the scanning area 102 may be determined by parallax estimation using, for example, different images captured by different cameras among the plurality of cameras. For example, the plurality of cameras may include a binocular pair of cameras, and the distance to the object 104 may be determined using binocular parallax estimation within the scanned area 102. According to some embodiments, the image processor of the vehicle surveillance system 100 (FIG. 1).
, 2A, 2B) may perform parallax estimation, and a 3D model may be generated from the distance to the object 104 determined by the parallax estimation or provided in other embodiments. For example, the image processor may be part of the 3D scanner 110.

In another embodiment, the 3D scanner 110 is a time-of-flight c whose distance is determined from the propagation time of a pulse of light reflected by an object in the scanned area 102.
Light detection and ranging (LIDAR) without a scanner, such as amera)
) Equipped with a system. In particular, time-of-flight cameras use lasers or other similar light sources to generate pulses of light. The light pulse is then used to illuminate the scanning region 102.
Any object 104 in the scanning area 102 reflects the irradiation light pulse back to the flight time camera.
The distance to the object 104 is the length of time it takes for the irradiation light pulse to propagate to the object, reflect off the object, and then return to the optical sensor of the flight time camera (eg, focal plane array).
That is, it is determined from the flight time. For example, the flight time distance may be determined pixel by pixel using a flight time camera to provide a 3D model of the scanning area 102.

In another embodiment, the 3D scanner 110 comprises a distance sensor configured to measure distances to a plurality of points within the scanned area 102. For example, the plurality of points may include the object 104. In some embodiments, including a distance sensor, the 3D scanner 110 may further include a camera configured to capture the corresponding two-dimensional (2D) image of the scanning area 102. The measurement distance provided by the distance sensor is a point cloud in scan area 102.
d) It may be used to generate one or both of an object mesh. The point cloud and object mesh may then be either used directly as a 3D model (eg, in the image processor of vehicle surveillance system 100) or used to generate a 3D model. The 2D image captured by the camera may be used to paint the 3D model. "Painting" means overlaying a 2D model on a 3D model or combining it with a 3D model (eg, when the 3D model is rendered on a display). For example, various distance sensors including, but not limited to, acoustic distance sensors and optical (eg, scanning laser-based) distance sensors include 3
It may be used in this embodiment of the D-scanner 110.

In particular, the distance sensor of the 3D scanner 110 may include a laser configured to scan the region 102. Further, the distance sensor may include an optical sensor configured to measure distances to a plurality of points using laser light reflected from one or more objects 104 in the scanned area 102. .. For example, the 3D scanner 110 is an Intel RealSense that combines a 2D camera, a second infrared camera, and an infrared laser projector.
A 3D camera (registered trademark) may be provided. The 2D camera is configured to capture a 2D image of the scanning area 102, and the infrared laser projector and the second infrared camera work together as a distance sensor to collect distance information within the scanning area 102. Intel Rea
lSense (registered trademark) and Intel (registered trademark) are registered trademarks of Intel Corporation in Santa Clara, California, USA.

The vehicle monitoring system 100 shown in FIG. 1 further includes an electronic display 120.
The electronic display 120 is configured to display a portion of the region 102 using a 3D model. Further, the electronic display 120 is an object 104'in the display portion located closer than the threshold distance from the vehicle (or, likewise, from the vehicle monitoring system 100).
Is configured to visually emphasize. According to various embodiments, the visual enhancement is configured to improve the user's perception of the object 104', which is closer to the vehicle than the threshold distance.
For example, improving the perception of the visually emphasized object 104'may facilitate avoiding a collision between the object 104' and the vehicle. In FIG. 2A and FIG. 2B, d _T and signed threshold distance is the distance from the vehicle monitoring system 100, to the plane 106 shown as the monitoring region intersects the object 104 'in the 102 (e.g. bisecting) dashed boundary Shown as.

FIG. 3A shows a region 1 in an example according to an embodiment consistent with the principles described herein.
The display portion 108 of 02 is shown. In particular, FIG. 3A shows various objects 104 in the scanned area 102 when the objects 104 can be seen on the display screen of the electronic display 120. For example
The electronic display 120 may be a two-dimensional (2D) electronic display 120 (eg, an LDC display), and the display portion 108 of the scanning area 102 may be displayed or rendered as a 2D image on the 2D electronic display. Further, as shown in FIG. 3A, none of the objects 104 are visually emphasized.

According to some embodiments, the object 104'closer than the threshold distance may be visually enhanced on or by the electronic display 120 with the mask applied to the object 104'. An object 104 where, for example, a mask with shading or color shading is emphasized.
'Or part of it. Further, for example, the mask is a masked object 1
It may have a color (eg, yellow, red, etc.) configured to draw attention to 04'. In some examples, the mask may be applied to any portion of the scanning area 102 (eg, any pixel) where the distance from the vehicle is determined to be less than the threshold distance.

FIG. 3B is an example of FIG. 3A according to an embodiment consistent with the principles described herein.
Indicates a visually emphasized object 104'in the display portion 108 of. In particular, FIG. 3B shows the display portion of the scanning region 102 including the object 104'visually emphasized using the mask 210. As shown in FIG. 3B, the mask 210 comprises shading on or over an object 104'closer than the threshold distance. In this example, the entire object 104'is a shaded mask 21
It is substantially covered by 0. In another example (not shown), only the portion of object 104'that is actually closer than the threshold distance is covered by mask 210 to provide visual enhancement and object 1
The rest of 04'may be substantially free of mask 210 (ie, not covered by mask 210).

In another example, the visual enhancement displayed on the electronic display 120 may include contour lines that surround the edges or perimeter of the object closer than the threshold distance. For example, the contour line may have a color such as, but not limited to, yellow, orange, or red to draw special attention to the object. In some further examples, contour lines may be provided around any portion of region 102 that is determined to be closer than the threshold distance from the vehicle. In some examples, contours may be used in conjunction with masks (eg, masked objects may be further contoured).

In yet another example, a warning icon may be displayed on the electronic display 120 to emphasize an object whose distance from the vehicle is determined to be shorter than the threshold distance. For example, the warning icon may be displayed as an object within a threshold distance, or, likewise, superimposed on a portion of the scanning area 102. For example, the warning icon may be rendered in a color that attracts attention and may include, but is not limited to, a triangle, a circle, or a rectangle. In some examples, a triangle, circle, or rectangle may enclose an exclamation point or another alphanumeric character to draw more attention to the warning icon. In some embodiments, the warning icon may be used in combination with one or both of the mask and contour (eg, the warning icon may be located within the mask or contour).

FIG. 3C is an example of FIG. 3 according to another embodiment consistent with the principles described herein.
Shows a visually emphasized object 104'in display portion 108 of A. In particular, FIG. 3C shows the display portion of the scanning region 102 including the object 104'visually emphasized using the contour line 220. Further, FIG. 3C shows a warning icon 230. The warning icon 230 may be generally located anywhere on the display portion, but as an example, in FIG. 3C, the warning icon 23 may be located.
0 is located within the contour line 220 and overlaps the object 104'.

With reference to FIG. 1 again, according to some embodiments, the electronic display 120 of the vehicle monitoring system 100 may include a 3D electronic display 120. In some of these embodiments, the threshold distance is the area 10 displayed on the 3D electronic display 120.
It may correspond to the zero parallax plane associated with a part of 2. According to still some embodiments, the visually emphasized object 104'may be an object that is perceived to be in front of the zero parallax plane of the 3D electronic display 120. The object 104'is rendered in front of the zero parallax plane on the 3D electronic display 120, especially when the object 104'located at a distance shorter than the threshold distance is displayed or rendered. Thus, according to various embodiments, the object is perceived as protruding, projecting, or in front of the physical display surface of the 3D electronic display 120 (eg, by the visual eye), the object 104. 'Is visually emphasized.

FIG. 4 shows a perspective view of a 3D electronic display 120 depicting a visually emphasized object 104'in one example, according to another embodiment consistent with the principles described herein. Especially in Figure 4
Is visually enhanced by being rendered in front of the zero parallax plane 10.
The display portion 108 of the scanning area 102 including 4'is shown. The object 104'(eg, a children's toy bicycle) appears on the 3D electronic display as projecting from or in front of the physical surface 120' of the 3D electronic display. As shown in FIG. 4, the other object 104 at a distance farther than the threshold distance is the physical surface 12 of the 3D electronic display.
It can be seen behind 0'.

In some embodiments, the projective transformation may be applied to the 3D model before the display portion is rendered on the 3D electronic display 120. In particular, the vehicle monitoring system 100
An image processor may be further provided, for example, as described below with respect to FIG. For example, the image processor may be part of the 3D electronic display 120. In another example, the image processor may be part of the 3D scanner 110 or another (eg, separate) element of the vehicle surveillance system 100.

The image processor may be configured to apply a projective transformation to the 3D model before the display portion is rendered on the 3D electronic display 120. According to various embodiments
The projective transformation is the other object 1 in the image at a distance farther than the threshold distance corresponding to the zero parallax plane.
It is configured to increase the relative size of the visually emphasized object 104'as compared to 04. As a result, not only is the visually emphasized object 104'visible in front of the physical display surface 120' of the 3D electronic display 120, but the visually emphasized object 104'is 3
When rendered on the D electronic display 120, it is spatially distorted or increased in size. The effect is as if the visually emphasized object 104'is magnified, enlarged, visually inflated, or magnified compared to other objects displayed by the 3D electronic display 120. It's like. For example, the object 104'shown in FIG. 4 is spatially distorted by applying a projective transformation. Therefore, according to various embodiments, the perception of the visually emphasized object 104'is further enhanced by the size-distorted rendering obtained by applying the projective transformation.

According to various embodiments, the 3D electronic display 120 can be a substantially arbitrary 3D electronic display. Especially in some embodiments, the 3D electronic display 1
Reference numeral 20 denotes a multi-beam grid-based 3D electronic display 120 having a multi-beam grid-based backlight and an optical modulation layer. FIG. 5A shows a cross-sectional view of an example 3D electronic display 120 having a multi-beam grid-based backlight according to an embodiment consistent with the principles described herein. FIG. 5B shows a cross-sectional view of an example 3D electronic display 120 having a multi-beam grid-based backlight, according to another embodiment consistent with the principles described herein. FIG. 5C has a multi-beam grid-based backlight in an example according to an embodiment consistent with the principles described herein.
A perspective view of a part of the D electronic display 120 is shown. For example, a portion of the 3D electronic display 120 shown in FIG. 5C may represent the 3D electronic display 120 shown in either FIG. 5A or FIG. 5B. According to various embodiments, the 3D electronic display 120 with a multi-beam grid-based backlight can provide an improved perception of the visually enhanced object described above.

According to various embodiments, the 3D electronic display 120 shown in FIGS. 5A-5C
It is configured to provide modulated "directional" light, i.e. light with light beams having different principal maximal angles. For example, as shown in FIGS. 5A-5C, the 3D electronic display 120 is oriented to exit the 3D electronic display 120 in different predetermined principal maximal angles (eg, as a light field) indicated by arrows. Can result in or generate multiple light beams. The plurality of light beams may then be modulated to facilitate the display of information having 3D content including, but not limited to, visually emphasized objects. In some embodiments, the modulated light beam with different predetermined principal maximal directions forms a plurality of pixels of the 3D electronic display 120. further,
The 3D electronic display 120 is a so-called "naked eye" 3D electronic display (eg, multi-view, "holographic", or autostereoscopic, in which the light beam corresponds to pixels associated with different "visual images" of the 3D electronic display 120. Display).

As shown in FIGS. 5A, 5B, and 5C, the multi-beam grid-based backlight of the 3D electronic display 120 includes a light guide 122. In particular, according to some embodiments, the light guide 122 may be a flat plate light guide 122. The flat plate light guide 122 guides light from a light source (not shown in FIGS. 5A-5C) as waveguide light (shown as a long arrow propagating within the flat plate light guide 122, further described below). It is configured to do. For example, the flat plate light guide 122 may include a dielectric material configured as an optical waveguide. The dielectric material may have a first index of refraction that is greater than the second index of refraction of the medium surrounding the dielectric optical waveguide. For example, the difference in refractive index is the flat plate light guide 122.
It is configured to facilitate total internal reflection of the waveguide light according to one or more waveguide modes of.

In various embodiments, the light from the light source is guided as a beam of light along the length of the flat plate light guide 122. Further, the flat plate light guide 122 may be configured to guide light (ie, a waveguide light beam) at a non-zero propagation angle. For example, the waveguide light beam may be guided at a non-zero propagation angle within the flat plate light guide 122 using internal total internal reflection. In particular, the waveguide light beam propagates at a non-zero propagation angle by reflecting or "bounce" between the top and bottom surfaces of the flat plate light guide 122 (eg, by a long sloping arrow representing the light beam of the waveguide light beam. Shown).

The "non-zero propagation angle" as defined herein is the angle of the flat plate light guide 122 with respect to the surface (eg, top or bottom). In some examples, the non-zero propagation angle of the waveguide light beam is from about 10 degrees to about 50 degrees, or in some examples from about 20 degrees to about 40 degrees, or from about 25 degrees to about 35 degrees. be able to. For example, the non-zero propagation angle can be about 30 degrees. In another example, the non-zero propagation angle can be about 20 degrees, or about 25 degrees, or about 35 degrees.

In some examples, the light from the light source is introduced or coupled into the flat plate light guide 122 at a non-zero propagation angle (eg, about 30-35 degrees). For example, a lens, mirror, or similar reflector (eg, collimating).
(Reflector), and one or more of the prisms (not shown)) make it easy to couple light into the input end of the flat plate light guide 122 as a beam of light at a non-zero propagation angle. Can be. When coupled into the flat plate light guide 122, the waveguide light beam propagates along the flat plate light guide 122 in a direction generally away from the input end (eg, along the x-axis of FIGS. 5A-5B). (Indicated by a thick arrow pointing in the direction).

Further, according to some examples, the waveguide light beam generated by coupling light into the flat plate light guide 122 can be a collimated waveguide light beam. In particular, "collaborated light beam" means that a plurality of light rays in the waveguide light beam are substantially parallel to each other in the waveguide light beam. By definition herein, the light emitted or scattered from a collimated light beam of a waveguide light beam is not considered part of the collimated light beam. For example, light collimation to generate a collimated waveguide light beam is achieved by a lens or mirror (eg, a tilted collimated reflector) used to couple the light into the flat plate light guide 122. be able to.

In some examples, the flat plate light guide 122 may be a slab or flat plate optical waveguide with a long, substantially planar sheet of optically clear dielectric material. A substantially planar sheet of dielectric material is configured to guide a waveguide light beam using internal total internal reflection. According to various examples, the optically transparent material of the flat plate light guide 122 is various types of glass (eg quartz glass, alkaline aluminosilicate glass, borosilicate glass, etc.), and substantially optically. It may include any of a variety of dielectric materials including, but not limited to, one or more of clear plastics or polymers (eg, poly (methylmethacrylate) or "acrylic glass", polycarbonate, etc.). , Or any of these. In some examples, the flat plate light guide 122 may further include a clad layer on at least a portion of the surface of the flat plate light guide 122 (eg, one or both of the top and bottom surfaces) (not shown). According to some examples, the clad layer may be used to further facilitate total internal reflection.

In FIGS. 5A, 5B, and 5C, the multi-beam grating-based backlight of the 3D electronic display 120 shown further comprises an array of multi-beam gratings 124. As shown in FIGS. 5A-5B, the multi-beam diffraction grating 124 is positioned on the surface (eg, top or front) of the flat plate light guide 122. In another example (not shown), one or more of the multi-beam diffraction gratings 124 may be positioned within the flat plate light guide 122. In yet another example (not shown), one or more of the multi-beam diffraction gratings 122 may be on or above the bottom surface or back surface of the flat plate light guide 120 (ie, opposite the plane indicated by the multi-beam diffraction grating 124). It may be positioned (on the surface of). By combining the flat plate light guide 122 with an array of multi-beam gratings 124, they realize or act as a backlight for the multi-beam grating-based backlight of the 3D electronic display 120.

According to various embodiments, the array's multi-beam diffraction grating 124 scatters or diffracts a portion of the waveguide light beam in different principal maximal angles corresponding to different visual images of the 3D electronic display 120. It is configured to go out as a plurality of light beams having. For example, a portion of the waveguide light beam may be diffractively coupled out by a multi-beam diffraction grating 124 through the flat plate light guide surface (eg, through the top surface of the flat plate light guide 122). Further, the multi-beam diffraction grating 124 couples a part of the waveguide light beam by diffraction, couples the light beam to the outside as a light beam to be emitted to the outside, and the light beam to be coupled to the outside is emitted. It is configured to change the direction by diffraction so as to be separated from the flat plate light guide surface as a plurality of light beams.
As described above, each of the plurality of light beams has a different predetermined principal maximal angular direction determined by the characteristics of the diffraction feature of the multi-beam diffraction grating 124.

According to various embodiments, the array's multi-beam diffraction grating 124 includes a plurality of diffraction features that provide diffraction. The diffraction achieved results in a diffractive coupling of a portion of the waveguide light beam from the flat plate light guide 122. For example, the multi-beam diffraction grating 124
May include one or both grooves in the surface of the flat plate light guide 122 that function as diffraction features and ridges protruding from the flat plate light guide surface. The grooves and ridges may be arranged parallel to each other, and the grooves and ridges may be located at least at some point along the diffraction feature.
It is perpendicular to the propagation direction of the waveguide light beam that will be coupled to the outside by the multi-beam diffraction grating 124.

In some examples, grooves and ridges may be etched, milled, or molded into the flat plate light guide surface. Therefore, the material of the multi-beam diffraction grating 124 may include the material of the flat plate light guide 122. As shown in FIG. 5A, for example, the multi-beam diffraction grating 124 includes a plurality of substantially parallel grooves penetrating the surface of the flat plate light guide 122. In FIG. 5B, the multi-beam diffraction grating 124 includes a plurality of substantially parallel ridges protruding from the surface of the flat plate light guide 122. In another example (not shown), the multi-beam diffraction grating 124 can be a film or layer coated or affixed to a flat plate light guide surface.

According to some embodiments, the multi-beam diffraction grating 124 may be a chirp diffraction grating or may include it. By definition, a "charp" grating is a grating that exhibits or has diffraction intervals (ie, diffraction pitches) of diffraction features that vary over the range or length of the charp grating, as shown in FIGS. 5A-5C. Is. In the present specification, the changing diffraction interval is defined and referred to as a "chirp". Due to the presence of the chap, the portion of the waveguide light beam that is diffracted and emitted out from the flat plate light guide 122 is coupled at a different diffraction angle corresponding to a different origin across the chap diffraction grating of the multi-beam diffraction grating 124. As a light beam emitted to the outside, it is emitted or emitted from a charp diffraction grating. Thanks to the predefined chirps, the chirp diffraction grating provides predetermined different principal maximal angular directions of multiple light beams that are coupled out.

In some examples, the chirp grating of the multi-beam grating 124 can have or exhibit a chirp of diffraction intervals that changes linearly with distance. Therefore, by definition, a chirp diffraction grating is called a "linear chirp" grating. For example, FIGS. 5A-5C show the multi-beam diffraction grating 124 as a linear chirp diffraction grating. Especially,
As shown, the diffraction feature is located at the second end of the multi-beam grating 124.
Closer to each other than at the first end. Furthermore, the diffraction interval of the diffraction feature shown is
It varies linearly from the first end to the second end, shown as an example and not as a limitation.

In another example (not shown), the chirp diffraction grating of the multi-beam diffraction grating 124 may exhibit a non-linear chirp of diffraction intervals. The various nonlinear chirps that can be used to implement the multi-beam grating 124 include exponential chirps, logarithmic chirps, or other substantially heterogeneous or random but still monotonously varying chirps, although
Not limited to these. Non-monotonic chirps, such as, but not limited to, sinusoidal chirps, or triangular or serrated chirps, are also available. Any combination of these types of chirps can be used.

According to some embodiments, the multi-beam diffraction grating 124 may include diffraction features that are curvilinear and / or chirped. Figure 5C
Shows a perspective view of a curved and chirped multi-beam diffraction grating 124 that is on or on the surface of the flat plate light guide 122 (ie, the multi-beam diffraction grating 124 is a curved chirp diffraction. It is a grating). In FIG. 5C, the waveguide light beam has an incident direction with respect to the multi-beam diffraction grating 124, which is indicated by a thick arrow at the first end of the multi-beam diffraction grating 124. Also, a plurality of coupled light beams emitted or emitted on the surface of the flat plate light guide 122 indicated by arrows pointing in a direction away from the multi-beam diffraction grating 124 are shown. As shown, the light beam is radiated in multiple predetermined different principal maximal angles. In particular, the predetermined different principal maximal directions of the emitted light beam are different from each other in both azimuth and elevation as shown. According to various examples, both the pre-defined chirp of the diffractive features and the curves of the diffractive features can result in predetermined different principal maximal angular directions of the emitted light beam.

In particular, the "underneath diffraction grating" of the multi-beam diffraction grating 124 associated with the curvilinear diffraction grating has different azimuth orientation angles φ _f at different points along the curve of the diffraction feature.
The "lower diffraction grating" means one of a plurality of non-curved diffraction gratings that, when overlapped, creates a curved diffraction feature of the multi-beam diffraction grating 124. At a given point along the curved diffraction feature, the curve has a particular orientation angle φ _f that is generally different from the orientation angle φ _f at another point along the curved diffraction feature. Further, a particular azimuth orientation angle φ _f gives the corresponding azimuth component φ in the principal maximal angular direction {θ, φ} of the light beam emitted from a given point. In some examples, the curve of the diffraction feature (eg, groove, ridge, etc.) may represent a section of the circle. The circle may be coplanar with the surface of the light guide. In another example, the curve may represent, for example, an ellipse or another section of curved shape that is coplanar with the light guide surface.

Referring again to FIGS. 5A-5B, the modulation layer of the 3D electronic display 120 includes a light valve array 126. According to various embodiments, the light bulb array 126 is 3
The D electronic display 120 is configured to modulate light beams directed in different directions corresponding to different visual images (ie, a plurality of light beams having different principal maximal angular directions).
In particular, the light beam of the plurality of light beams passes through and is modulated by the individual light bulbs of the light bulb array 126. According to various embodiments, the modulated light beams directed in different directions may represent pixels of the 3D electronic display 120. In various examples, different types of light bulbs are used within the light bulb array 126, including, but not limited to, liquid crystal light bulbs, electrophoretic light bulbs, and one or more of electrowetting-based light bulbs. You may.

According to some examples of the principles described herein, a three-dimensional (3D) vehicle monitoring system is provided. FIG. 6 shows a block diagram of a three-dimensional (3D) vehicle monitoring system 300 in an example according to an embodiment of the principles described herein. Especially 3D vehicle monitoring system 300
Is configured to provide collision avoidance when the vehicle is moving in the surveillance direction. For example 3
The D vehicle monitoring system 300 may be configured to monitor the area behind the vehicle, thus providing the vehicle operator with reverse movement assistance.

As shown in FIG. 6, the 3D vehicle monitoring system 300 includes a 3D camera 310.
In some embodiments, the 3D camera 310 is a rear facing 3D camera 310.
The 3D camera 310 is configured to capture a 3D image of an area adjacent to the vehicle (eg, behind the vehicle). According to various embodiments, the 3D camera 310 is a vehicle monitoring system 10.
With respect to 0, it may be substantially the same as the 3D scanner 110 described above. In particular, the 3D camera 310 is one of a plurality of staggered cameras, flight time cameras, and a composite of a laser-based distance sensor and a two-dimensional (2D) camera configured to monitor an area adjacent to the vehicle. It may have one or more.

The 3D vehicle monitoring system 300 shown in FIG. 6 further includes an image processor 320. The image processor 320 uses the 3D image captured by the 3D camera 310 to perform 3D.
It is configured to provide a 3D model of the D imaging region. According to various embodiments, 3D
The model includes the spatial composition of the object within the 3D imaging region. The 3D imaging region and the objects in the 3D imaging region may be substantially similar to the regions 102 and objects 104, 104'described above with respect to the vehicle monitoring system 100.

According to various embodiments, the 3D vehicle monitoring system 300 is a 3D electronic display 33.
It further includes 0. The 3D electronic display 330 is configured to display a portion of the 3D imaging region using a 3D model. According to some embodiments, any object in the display portion located at a distance from the vehicle (eg, distance from the rear of the vehicle) shorter than the threshold distance is visually highlighted on the 3D electronic display 330. You may. In particular, according to some embodiments, the 3D electronic display 330 is further configured to visually emphasize objects in the display portion located closer than the threshold distance from the vehicle.

In some embodiments, the 3D electronic display 330 is substantially similar to the 3D electronic display 120 of the vehicle monitoring system 100 described above. For example, the threshold distance is
It may correspond to a zero parallax plane associated with a portion of the 3D imaging region displayed by the 3D electronic display 330. In some embodiments, the visually emphasized object can be perceived as being in front of the zero parallax plane (ie, in front of the physical surface of the 3D electronic display 330). In some further embodiments, the image processor 3
20 is further configured to apply a projective transformation to the 3D model before the display portion is rendered on the 3D electronic display 330. The applied projective transformation is configured to increase the relative size of the visually emphasized object compared to other objects in the display area located at a distance longer than the threshold distance corresponding to the zero parallax plane. You may.

In some embodiments, the 3D electronic display 330 may include a multi-beam grid-based 3D electronic display. For example, in some embodiments, the 3D electronic display 330 may be substantially similar to the 3D electronic display 120 having the multi-beam grid-based backlight and modulation layer described above for the vehicle surveillance system 100. In particular, the 3D electronic display 330 may include a multi-beam grid-based backlight and modulation layer. A multi-beam grating-based backlight is oriented away from a light guide by diffracting a portion of the waveguide light with a light guide for guiding a beam of light (eg, a collimated light beam). It may include an array of multi-beam diffraction gratings configured to exit as light beams directed in a plurality of different directions. The light guide and the multi-beam diffraction grating are the flat plate light guide 122 and the multi-beam diffraction grating 1 described above for the vehicle monitoring system 100.
It may be substantially the same as 24. In addition, the modulation layer may include a light bulb array for modulating light beams directed in different directions. According to various embodiments, the modulated and differently directed light beams form a plurality of different visual images of the 3D electronic display 330. The light bulb array may be substantially similar to the light bulb array 126 described above with respect to the vehicle monitoring system 100. In particular, the modulated and differently directed light beams have different predetermined principal maximal angular directions that form multiple pixels associated with different "visual images" of the 3D electronic display 330.

According to some embodiments (not shown), the 3D electronic display 300 may further include a light source. The light source is configured to provide light propagating within the light guide as a waveguide light beam. In particular, according to some embodiments, the waveguide light is light from a light source that is coupled and placed at the edge (or input end) of the light guide. For example, a lens, collimating reflector, or similar device (not shown) can facilitate the coupling of light into a light guide at its input end or edge. In various examples, the light source may include substantially any source of light, including but not limited to light emitting diodes (LEDs). In some examples, the light source may include an optical emitter configured to produce substantially monochromatic light with a narrow band spectrum represented by a particular color. In particular, the color of the monochromatic light can be the primary color of a particular color space or color model (eg, red-green-blue (RGB) color model).

According to some examples of the principles described herein, vehicle monitoring methods are provided. Vehicle monitoring methods can be used to monitor an area or area adjacent to a vehicle. For example, the area or area may include, but is not limited to, the front, side, and rear or rear areas of the vehicle.

FIG. 7 shows a flowchart of the vehicle monitoring method 400 in one example according to one embodiment consistent with the principles described herein. As shown in FIG. 7, the vehicle monitoring method 400 is 3.
A step 410 is provided for capturing a 3D scan of an area adjacent to the vehicle using a D-scanner. According to various embodiments, the 3D scanner used in step 410 to capture may be substantially similar to the 3D scanner 110 described above with respect to the vehicle monitoring system 100.
For example, a 3D scanner comprises one or more cameras that are offset from each other (eg, binocular pairs of cameras), a flight time camera, and a composite of a laser-based distance sensor and a two-dimensional (2D) camera. You may. In some examples, the 3D scanner is the 3D vehicle monitoring system 30.
With respect to 0, it may be substantially the same as the 3D camera 310 described above.

The vehicle monitoring method 400 shown in FIG. 7 further comprises a step of generating a 3D model from the captured 3D scan. According to various embodiments, the 3D model includes a spatial configuration of an object located within the scanning area. For example, a 3D model can be generated from a 3D scan using one or both of the point cloud and object mesh generated by the 3D scanner.

According to various embodiments, the vehicle monitoring method 400 further comprises step 430 displaying a portion of the scanning area using a 3D model. According to various embodiments, step 430 displaying a portion of the scanning area comprises a step of visually emphasizing an object in the display portion whose distance from the vehicle is closer than the threshold distance. In some embodiments, step 430 displaying a portion of the scanning area is the electronic display 1 of the vehicle monitoring system 100 described above.
An electronic display substantially similar to 20 may be used. Further, the steps of visually enhancing an object may be substantially similar to any form of visual enhancement described above. For example, an object may be visually highlighted with one or more of masks, contours, and warning icons. In another example, step 430 displaying a portion of the scanning area.
Provide, for example, a step using the 3D electronic display described above with respect to the 3D electronic display 330 of the 3D vehicle monitoring system 300. Using a 3D electronic display, the visually emphasized object appears to be in front of the zero parallax plane of the 3D electronic display corresponding to the threshold distance.

According to some embodiments, step 430 displaying a portion of the scanning area using a 3D electronic display (eg, a 3D electronic display with a multi-beam grid-based backlight) emits light in a flat plate light guide. It may further include a step of waveguideing at a non-zero propagation angle. Step 430, which displays a portion of the scanning area using a 3D electronic display, uses an array of multi-beam diffraction gratings on a flat plate light guide.
Further, a step of coupling a part of the waveguide light beam by diffraction to the outside may be provided. For example, the step of coupling a part of the waveguide light beam by diffraction and putting it out is
Even with multiple couplings oriented away from the flat plate light guide to generate an outgoing light beam in multiple different principal maximal angles corresponding to different visions of the 3D electronic display. Good. Further, in step 430, which displays a part of the scanning area using a 3D electronic display, a plurality of coupling light beams to be emitted to the outside are generated.
It may further include a step of modulating with a plurality of light bulbs, where the modulated light beam represents a pixel in a 3D electronic display.

Further, the vehicle monitoring method 400 may include a step of applying a projective transformation to the 3D model before the step 430 of displaying a part of the scanning area on the 3D display.
In some embodiments, the projective transformation is the depth of the display area to increase the relative size of the highlighted object compared to the object in the image at a distance farther than the threshold distance corresponding to the zero parallax plane. It may include compression.

In this way, a vehicle monitoring system that visually emphasizes objects closer to the vehicle than the threshold distance, 3.
An example of the D vehicle monitoring system and the vehicle monitoring method was explained. It should be understood that the above examples merely exemplify some of the many specific examples that represent the principles described herein. Obviously, one of ordinary skill in the art can easily devise a number of other configurations without departing from the scope defined by the appended claims.

100 Vehicle monitoring system 102 Scanning area 104 Object 106 Plane 108 Display part 110 3D (3D) scanner 120 Electronic display 122 Flat plate light guide 124 Multi-beam diffracting grid 126 Light valve array 210 Mask 220 Contour line 230 Warning icon 300 3D (3D) Vehicle monitoring system 310 3D camera 320 Image processor 330 3D electronic display 400 Vehicle monitoring method

The present disclosure includes the following [1] to [20].
[1] A three-dimensional (3D) scanner configured to scan a region adjacent to a vehicle, the scan providing a 3D model including a spatial configuration of an object located within the scanned region. And the scanner used to
An electronic display configured to display a portion of the scanning area using the 3D model and visually emphasize an object in the displayed portion located closer than the threshold distance from the vehicle. With
A vehicle monitoring system in which the visual enhancement is configured to improve the user's perception of the object closer to the vehicle than the threshold distance.
[2] The 3D scanner includes a plurality of cameras displaced from each other, and the distance from the 3D scanner is determined by parallax estimation using different images captured by different cameras among the plurality of cameras. , The vehicle monitoring system according to the above [1].
[3] The vehicle monitoring system according to the above [1], wherein the 3D scanner includes a flight time camera.
[4] The above 3D scanner
The distance sensor configured to measure the distance from the distance sensor to a plurality of points in the scanning area, and the distance sensor.
With a camera configured to capture a two-dimensional (2D) image of the scanning area, the measured distance may provide one or both of a point cloud and an object mesh to provide the 3D model. The vehicle monitoring system according to the above [1], which is used for generating and the above 2D image is used for painting the above 3D model.
[5] The distance sensor
A laser configured to scan the scanning area and
The vehicle monitoring system according to [4] above, comprising an optical sensor configured to measure distances to the plurality of points using laser light reflected from an object in the scanning region.
[6] The visual enhancement of the object located closer than the threshold distance from the vehicle in the display portion is applied to the displayed object by the electronic display, the mask applied to the displayed object, and the displayed object. The vehicle monitoring system according to [1] above, comprising a contour line around the object and one or more of the warning icons indicating the displayed object.
[7] The electronic display includes a 3D electronic display, and the threshold distance corresponds to a zero parallax plane associated with the portion of the scanning region displayed by the 3D electronic display, and the threshold distance corresponds to the vehicle. The vehicle according to [1] above, wherein the visual enhancement of the object located closer than the threshold distance is the visual perception that the object is in front of the zero dilatation plane of the 3D electronic display. Monitoring system.
[8] An image processor configured to apply a projective transformation to the 3D model before the display portion is rendered on the 3D display is further provided, and the projective transformation is performed on the zero parallax plane. The vehicle monitoring system according to [7], wherein the relative size of the emphasized object is made larger than that of an object in a display portion at a distance longer than the corresponding threshold distance.
[9] The above 3D electronic display
A flat plate light guide configured to guide the light beam at a non-zero propagation angle,
An array of multi-beam diffraction gratings, wherein the multi-beam diffraction grating of the array is used as a plurality of coupled light beams having different principal angular directions corresponding to different visual images of the 3D electronic display. An array of multi-beam diffraction gratings configured to couple a part of the waveguide light beam by diffraction and put it out.
A light bulb array configured to modulate a plurality of coupled and outgoing light beams corresponding to the different visual images of the 3D electronic display, wherein the modulated light beam is the 3D. The vehicle monitoring system according to [7] above, comprising a light bulb array representing pixels of an electronic display.
[10] The multi-beam diffraction grating includes a linear charp diffraction grating, and the diffraction feature portion of the multi-beam diffraction grating is a curved groove in the flat plate light guide surface and a curved shape on the flat plate light guide surface. The vehicle monitoring system according to [9] above, comprising one or both of the ridges of the above.
[11] The 3D scanner is configured to scan the area behind the vehicle, and the improved user perception is configured to assist collision avoidance when the vehicle is moving backwards. The vehicle monitoring system according to the above [1], wherein the vehicle monitoring system is a rear view auxiliary vehicle monitoring system.
[12] A 3D camera configured to capture a 3D image of an area adjacent to the vehicle.
An image processor configured to use the 3D image to provide a 3D model of the 3D imaged region, wherein the 3D model includes a spatial configuration of an object in the 3D imaged region. With an image processor
A 3D electronic display configured to display a part of the 3D imaging region using the 3D model and visually emphasize an object in the display portion located closer than the threshold distance from the vehicle. With
A three-dimensional (3D) vehicle surveillance system configured such that the visually emphasized object provides collision avoidance when the vehicle is moving in the direction of the adjacent region.
[13] In the above [12], the above 3D camera includes a plurality of cameras displaced from each other, a flight time camera, and one or a plurality of a composite of a laser-based distance sensor and a two-dimensional (2D) camera. The described 3D vehicle monitoring system.
[14] The threshold distance corresponds to the zero parallax plane associated with the portion of the 3D imaging region configured to be displayed by the 3D electronic display, and the visual within the display portion. The 3D vehicle surveillance system according to [12] above, wherein the object emphasized in is perceived as being in front of the zero parallax plane.
[15] The image processor is further configured to apply a projective transformation to the 3D model before a portion of the 3D imaging region is displayed on the 3D electronic display, which displays the display. The relative size of the visually emphasized object is configured to be larger than the other objects in the portion so that the other objects are at a distance longer than the threshold distance corresponding to the zero parallax plane. The 3D vehicle monitoring system according to [14] above, which is located.
[16] The 3D vehicle monitoring system according to [12] above, wherein the 3D electronic display includes a multi-beam grid-based 3D electronic display.
[17] A step of capturing a 3D scan of an area adjacent to the vehicle using a 3D scanner.
A step of generating a 3D model from the captured 3D scan, wherein the 3D model includes a spatial configuration of an object located within the scanned area.
Including a step of displaying a part of the scanning area using the 3D model.
The step of displaying a part of the scanning area includes a step of visually emphasizing an object in the display part where the distance from the vehicle is shorter than the threshold distance.
A vehicle monitoring method in which the visually emphasized object is configured to improve the user's perception of the object closer to the vehicle than the threshold distance.
[18] The step of displaying a part of the scanning area further includes the step of using the 3D electronic display, and the visually emphasized object is in front of the zero parallax plane of the 3D electronic display corresponding to the threshold distance. The step of generating the 3D model includes applying a projection transformation to the 3D model before the step of displaying the portion of the scanning area on the 3D electronic display. The vehicle monitoring method according to the above [17].
[19] In order for the projective transformation to increase the relative size of the visually emphasized object as compared to an object in the scanning region at a distance longer than the threshold distance corresponding to the zero parallax plane. The vehicle monitoring method according to the above [18], which includes the depth compression of the display portion.
[20] The step of displaying a part of the scanning area using the 3D electronic display is
Steps to guide light as a light beam at a non-zero propagation angle in a flat plate light guide,
A plurality of steps corresponding to different visual images of the 3D electronic display, which is a step of coupling a part of the waveguide light beam by diffraction and taking it out by using an array of multi-beam diffraction gratings on the flat plate light guide. A portion of the waveguide light beam is diffracted and coupled, comprising a plurality of couplings directed away from the flat plate light guide to generate an outgoing light beam in different principal angular directions. Steps to get out and
A step of modulating a plurality of coupling light beams emitted to the outside by using a plurality of light bulbs, wherein the modulated light beam displays a part of the light beam as a 3D image. The vehicle monitoring method according to [18] above, comprising a step of modulating, representing pixels of a display.
To overcome the limitations of passive displays associated with emitted light, many passive displays are coupled to an external light source. A coupled light source may allow these normally passive displays to radiate light and effectively function as active displays. An example of such a connected light source is a backlight. The backlight is a light source (often a panel light source) that is usually arranged behind the passive display to illuminate the passive display. For example, the backlight may be connected to an LCD or EP display. The backlight emits light that passes through the LCD or EP display. The emitted light is modulated by the LCD or EP display, and the modulated light is then emitted from the LCD or EP display. Backlights are often configured to emit white light. In that case, color filters are used to convert the white light into the various colors used in the display. Color filters may be located, for example, on the output of an LCD or EP display (less common), or between the backlight and the LCD or EP display.

It is a block diagram of the vehicle monitoring system in one example by one Embodiment which is consistent with the principle described in this specification. FIG. 5 is a perspective view of a region monitored by using the vehicle monitoring system of FIG. 1 in an example according to an embodiment consistent with the principles described herein. It is a side view of the monitoring area of FIG. 2A in one example according to one embodiment consistent with the principle described herein. It is a figure which shows the display part of the scanned area in one example by one Embodiment which was consistent with the principle described in this specification. (B) according to an embodiment consistent with the principles described herein, it is a diagram illustrating a visually highlighted objects in the display unit of FIG. 3A min in an example. According to another embodiment consistent with the principles described (C) herein illustrates the visually highlighted objects in the display unit content of Figure 3A in an example. FIG. 3 is a perspective view of a 3D electronic display depicting a visually emphasized object in an example, according to another embodiment consistent with the principles described herein. FIG. 5 is a cross-sectional view of a three-dimensional (3D) electronic display with a multi-beam grid-based backlight in one example, according to an embodiment consistent with the principles described herein. (B) FIG. 6 is a cross-sectional view of a three-dimensional (3D) electronic display with a multi-beam grid-based backlight in one example, according to another embodiment consistent with the principles described herein. (C) FIG. 3 is a perspective view of a portion of a 3D electronic display with a multi-beam grid-based backlight in an example, according to an embodiment consistent with the principles described herein. FIG. 3 is a block diagram of a three-dimensional (3D) vehicle monitoring system in an example according to an embodiment of the principles described herein. It is a flowchart of the vehicle monitoring method in one example by one Embodiment which is consistent with the principle described in this specification.

According to some embodiments described herein, visual enhancement of an object is provided by a 3D electronic display (single color and / or color). In yet various embodiments, the 3D electronic display may be configured to present images and related information in a so-called "naked eye" 3D or autostereoscopic 3D embodiment. In particular, in some embodiments, the 3D electronic display may use a grating-based or "grating-based" backlight to generate 3D images or different visual images of information. In a 3D electronic display that uses a grating-based backlight, light is coupled out of the light guide using multiple gratings. The light that is coupled and emitted to the outside forms a plurality of light beams directed in a predefined direction (for example, a viewing direction). Further the light beam of the plurality of light beams have different principal angles of directions may form or provide light field in the viewing direction of the electronic display, in some embodiments, a plurality of primary colors May be represented. Light beam, auto stereo information including 3-dimensional (3D) information representative of the different colors in different principal angles of the light beam having a direction (also referred to as "light beam directed to a different direction"), and some embodiments Can be used to display in a scopic. For example, light beams of different colors directed in different directions may be modulated to function as color pixels of different visual images of a "naked eye" 3D electronic display, or to represent different visual images thereof.

As defined herein, a "multi-beam diffraction grating" is a diffraction grating that includes a plurality of light beams to generate light that is coupled and emitted to the outside. Further, a plurality of light beams generated by the multi-beam diffraction grating, by definition herein, have different principal angles of direction (principal angular direction) to each other. In particular, by definition, as a result of diffractive coupling and diffractive redirection of incident light by a multi-beam grating, one light beam of multiple light beams is with another light beam of multiple light beams. having a predetermined main angle direction different from. The plurality of light beams may form a light irradiation field. For example, a plurality of light beams may include eight light beams having eight different principal angles of direction. For example, eight combined light beams (ie, a plurality of light beams) may represent a light field. According to various embodiments, different principal angles of direction of the various light beams, the grating pitch or spacing of the diffractive features of the multi-beam diffraction grating at the origin of each light beam, by combining the orientation or rotation, It is determined with respect to the propagation direction of the light incident on the multi-beam diffraction grating.

According to the various embodiments described herein, the light that is coupled by a diffraction grating (eg, a multi-beam diffraction grating) and emitted out of the light guide represents a pixel in an electronic display. In particular, the light guide having a multi-beam diffraction grating for generating a plurality of light beams having a main angle of direction different from "naked eye" 3-dimensional (3D) electronic display (multiview or "Holographic" electronic display or auto, It can be part of an electronic display backlight, such as (also called a stereoscopic display), but not limited to these, or a backlight used in conjunction with those electronic displays. Therefore, the light beams directed in different directions generated by coupling the waveguide light from the light guide to the outside using a multi-beam diffraction grating can be the "pixels" of the 3D electronic display. Alternatively, it can represent a "pixel".

According to various embodiments, 3D electronic display 120 shown in FIG 5A~ 5C are to provide light, comprising a light beam having a modulated "directional" light, i.e. different principal angles of direction It is composed. For example, as shown in FIG 5A~ Figure 5C, 3D electronic display 120 is oriented so leave the different predetermined main Angle direction (for example, as light-field) 3D electronic display 120 indicated by the arrow Multiple light beams can be brought about or generated. The plurality of light beams may then be modulated to facilitate the display of information having 3D content including, but not limited to, visually emphasized objects. In some embodiments, the modulated light beams having different predetermined main angle of direction to form a plurality of pixels of the 3D electronic display 120. Further, the 3D electronic display 120 is a so-called "naked eye" 3D electronic display (eg, multi-view, "holographic", or autostereo) in which the light beam corresponds to pixels associated with different "visual images" of the 3D electronic display 120. It is a scopic display).

In FIGS. 5A, 5B, and 5C, the multi-beam grating-based backlight of the 3D electronic display 120 shown further comprises an array of multi-beam gratings 124. As shown in FIGS. 5A-5B, the multi-beam diffraction grating 124 is positioned on the surface (eg, top or front) of the flat plate light guide 122. In another example (not shown), one or more of the multi-beam diffraction gratings 124 may be positioned within the flat plate light guide 122. In yet another example (not shown), multi one or more beams diffraction grating 124 is opposite to the surface shown in the lower surface or back surface of the flat light guide 122 or by on (i.e. multi-beam diffraction grating 124 that It may be positioned (on the surface of). By combining the flat plate light guide 122 with an array of multi-beam gratings 124, they realize or act as a backlight for the multi-beam grating-based backlight of the 3D electronic display 120.

According to various embodiments, the multi-beam diffraction grating 124 of the array is coupled by or diffraction by scattering a portion of the guided light beam, the main angle of different directions corresponding to different viewing image with 3D electronic display 120 It is configured to go out as a plurality of light beams having. For example, a portion of the waveguide light beam may be diffractively coupled out by a multi-beam diffraction grating 124 through the flat plate light guide surface (eg, through the top surface of the flat plate light guide 122). Further, the multi-beam diffraction grating 124 couples a part of the waveguide light beam by diffraction, couples the light beam to the outside as a light beam to be emitted to the outside, and the light beam to be coupled to the outside is emitted. It is configured to change the direction by diffraction so as to be separated from the flat plate light guide surface as a plurality of light beams. As described above, each of the plurality of light beams, having the features given in the main angle of direction different as determined by the diffraction characteristics of the multi-beam diffraction grating 124.

According to some embodiments, the multi-beam diffraction grating 124 may be a chirp diffraction grating or may include it. By definition, a "charp" grating is a grating that exhibits or has diffraction intervals (ie, diffraction pitches) of diffraction features that vary over the range or length of the charp grating, as shown in FIGS. 5A-5C. Is. In the present specification, the changing diffraction interval is defined and referred to as a "chirp". Due to the presence of the chap, the portion of the waveguide light beam that is diffracted and emitted out from the flat plate light guide 122 is coupled at a different diffraction angle corresponding to a different origin across the chap diffraction grating of the multi-beam diffraction grating 124. As a light beam emitted to the outside, it is emitted or emitted from a charp diffraction grating. Thanks to the predefined chirp, the chirped grating, the predetermined different principal angles of direction of the plurality of light beams issued out by coupling results.

According to some embodiments, the multi-beam diffraction grating 124 may include diffraction features that are curvilinear and / or chirped. FIG. 5C shows a perspective view of a curved and chirped multi-beam diffraction grating 124 that is on or on the surface of the flat plate light guide 122 (ie, the multi-beam diffraction grating 124 is curved. It is a chirp diffraction grating). In FIG. 5C, the waveguide light beam has an incident direction with respect to the multi-beam diffraction grating 124, which is indicated by a thick arrow at the first end of the multi-beam diffraction grating 124. Also, a plurality of coupled light beams emitted or emitted on the surface of the flat plate light guide 122 indicated by arrows pointing in a direction away from the multi-beam diffraction grating 124 are shown. As shown, the light beam is emitted in a plurality of predetermined different principal angles of direction. In particular, predetermined different principal angles of the direction of the light beam emitted are different from each other in azimuth and elevation both as shown. According to various examples, by both the curve of the predefined chirped the diffraction feature of the diffractive features, predetermined different principal angles of the direction of the light beam emitted may result.

In particular, the "underlying diffraction grating" of the multi-beam diffraction grating 124 associated with the curvilinear diffraction grating has different azimuth orientation angles φf at different points along the curve of the diffraction feature. The "lower diffraction grating" means one of a plurality of non-curved diffraction gratings that, when overlapped, creates a curved diffraction feature of the multi-beam diffraction grating 124. At a given point along the curved diffraction feature, the curve has a specific orientation angle φf that is generally different from the orientation angle φf at another point along the curved diffraction feature. Furthermore, the particular azimuthal orientation angle .phi.f, the main angle of direction {theta, phi} of the light beam emitted from a given point azimuthal component phi is given to the corresponding. In some examples, the curve of the diffraction feature (eg, groove, ridge, etc.) may represent a section of the circle. The circle may be coplanar with the surface of the light guide. In another example, the curve may represent, for example, an ellipse or another section of curved shape that is coplanar with the light guide surface.

Referring again to FIGS. 5A-5B, the modulation layer of the 3D electronic display 120 includes a light valve array 126. According to various embodiments, the light valve array 126 modulates the 3D electronic display 120 different visual image light directed in different directions corresponding to the beam (i.e., a plurality of light beams having a main angle of direction different) It is configured as follows. In particular, the light beam of the plurality of light beams passes through and is modulated by the individual light bulbs of the light bulb array 126. According to various embodiments, the modulated light beams directed in different directions may represent pixels of the 3D electronic display 120. In various examples, different types of light bulbs are used within the light bulb array 126, including, but not limited to, liquid crystal light bulbs, electrophoretic light bulbs, and one or more of electrowetting-based light bulbs. You may.

In some embodiments, the 3D electronic display 330 may include a multi-beam grid-based 3D electronic display. For example, in some embodiments, the 3D electronic display 330 may be substantially similar to the 3D electronic display 120 having the multi-beam grid-based backlight and modulation layer described above for the vehicle surveillance system 100. In particular, the 3D electronic display 330 may include a multi-beam grid-based backlight and modulation layer. A multi-beam grating-based backlight is oriented away from a light guide by diffracting a portion of the waveguide light with a light guide for guiding a beam of light (eg, a collimated light beam). It may include an array of multi-beam diffraction gratings configured to exit as light beams directed in a plurality of different directions. The light guide and the multi-beam diffraction grating may be substantially similar to the flat plate light guide 122 and the multi-beam diffraction grating 124 described above for the vehicle monitoring system 100. In addition, the modulation layer may include a light bulb array for modulating light beams directed in different directions. According to various embodiments, the modulated and differently directed light beams form a plurality of different visual images of the 3D electronic display 330. The light bulb array may be substantially similar to the light bulb array 126 described above with respect to the vehicle monitoring system 100. In particular, modulated, the light beam directed in different directions, with a predetermined main Angle direction different to form a plurality of pixels associated with different "visual image" of 3D electronic display 330.

According to some embodiments (not shown), the 3D electronic display 330 may further include a light source. The light source is configured to provide light propagating within the light guide as a waveguide light beam. In particular, according to some embodiments, the waveguide light is light from a light source that is coupled and placed at the edge (or input end) of the light guide. For example, a lens, collimating reflector, or similar device (not shown) can facilitate the coupling of light into a light guide at its input end or edge. In various examples, the light source may include substantially any source of light, including but not limited to light emitting diodes (LEDs). In some examples, the light source may include an optical emitter configured to produce substantially monochromatic light with a narrow band spectrum represented by a particular color. In particular, the color of the monochromatic light can be the primary color of a particular color space or color model (eg, red-green-blue (RGB) color model).

According to some embodiments, step 430 displaying a portion of the scanning area using a 3D electronic display (eg, a 3D electronic display with a multi-beam grid-based backlight) emits light in a flat plate light guide. It may further include a step of waveguideing at a non-zero propagation angle. Step 430, which uses a 3D electronic display to display a portion of the scanning area, further steps to couple a portion of the waveguide light beam by diffraction using an array of multi-beam diffraction gratings on a flat plate light guide to bring it out. You may prepare. For example the step of issuing a portion of the guided light beam to the outside by coupling with diffraction to different principal angles of direction corresponding to the different viewing image with 3D electronic display, a plurality of which are directed away from the flat light guide It may be provided with a step of coupling to generate an outgoing light beam. Further, step 430 displaying a part of the scanning area using a 3D electronic display may further include a step of modulating a plurality of coupled light beams emitted to the outside using a plurality of light valves. The modulated light beam represents a pixel in a 3D electronic display.

Claims (20)

A three-dimensional (3D) scanner configured to scan a region adjacent to a vehicle, the scan to provide a 3D model that includes a spatial configuration of an object located within the scanned region. The scanner used and
An electronic display configured to display a portion of the scanning area using the 3D model and visually emphasize objects within the displayed portion located closer than a threshold distance from the vehicle.
With
A vehicle monitoring system in which the visual enhancement is configured to improve the user's perception of the object closer to the vehicle than the threshold distance. The 3D scanner includes a plurality of cameras that are offset from each other, and the distance from the 3D scanner is determined.
The vehicle monitoring system according to claim 1, wherein the vehicle monitoring system is determined by parallax estimation using different images captured by different cameras among the plurality. The vehicle monitoring system according to claim 1, wherein the 3D scanner includes a flight time camera. The 3D scanner
The distance sensor configured to measure the distance from the distance sensor to a plurality of points in the scanning area, and the distance sensor.
With a camera configured to capture a two-dimensional (2D) image of the scanning area, the measured distance provides one or both of a point cloud and an object mesh to provide the 3D model. The vehicle monitoring system according to claim 1, wherein the 2D image is used to generate and paint the 3D model. The distance sensor
A laser configured to scan the scanning area and
The vehicle monitoring system according to claim 4, further comprising an optical sensor configured to measure distances to the plurality of points using laser light reflected from an object in the scanning region. The visual enhancement of the object located within the display portion closer than the threshold distance from the vehicle is applied by the electronic display to the mask applied to the displayed object, around the displayed object. The vehicle monitoring system according to claim 1, further comprising a contour line and one or more of the warning icons indicating the displayed object. The electronic display comprises a 3D electronic display, and the threshold distance corresponds to a zero parallax plane associated with the portion of the scanning region displayed by the 3D electronic display, from the threshold distance from the vehicle. The vehicle monitoring system according to claim 1, wherein the visual enhancement of the object located also nearby is a visual perception that the object is in front of the zero parallax plane of the 3D electronic display. The image processor is further configured to apply a projective transformation to the 3D model before the display portion is rendered on the 3D display, the projectile transformation corresponding to the zero parallax plane. The vehicle monitoring system according to claim 7, wherein the relative size of the emphasized object is made larger than that of an object in the display portion at a distance longer than the threshold distance. The 3D electronic display
A flat plate light guide configured to guide the light beam at a non-zero propagation angle,
An array of multi-beam diffraction gratings, wherein the multi-beam diffraction grating of the array is used as a plurality of coupled light beams having different principal maximum angular directions corresponding to different visual images of the 3D electronic display. An array of multi-beam diffraction gratings configured to couple a portion of the waveguide light beam to the outside by diffraction.
A light valve array configured to modulate the plurality of coupled and outgoing light beams corresponding to the different visual images of the 3D electronic display, wherein the modulated light beams are the 3D. The vehicle monitoring system according to claim 7, further comprising a light valve array representing pixels of an electronic display. The multi-beam diffraction grating includes a linear charp diffraction grating, and the diffraction feature of the multi-beam diffraction grating is a curved groove in the flat plate light guide surface and a curved ridge on the flat plate light guide surface. The vehicle monitoring system according to claim 9, further comprising one or both of the above. The 3D scanner is configured to scan the area behind the vehicle, and the improved user perception is configured to assist collision avoidance when the vehicle is moving backwards, said vehicle. The vehicle monitoring system according to claim 1, wherein the monitoring system is a rear-view auxiliary vehicle monitoring system. A 3D camera configured to capture a 3D image of the area adjacent to the vehicle,
An image processor configured to use the 3D image to provide a 3D model of the 3D imaged region, wherein the 3D model includes a spatial configuration of an object within the 3D imaged region. With an image processor
A 3D configured to display a portion of the 3D imaging region using the 3D model and visually emphasize objects in the display portion located closer than the threshold distance from the vehicle.
Equipped with an electronic display
A three-dimensional (3D) vehicle monitoring system configured such that the visually emphasized object provides collision avoidance when the vehicle is moving in the direction of the adjacent region. The 3D vehicle according to claim 12, wherein the 3D camera comprises one or more of a plurality of cameras displaced from each other, a flight time camera, and a composite of a laser-based distance sensor and a two-dimensional (2D) camera. Monitoring system. The threshold distance corresponds to the zero parallax plane associated with the portion of the 3D imaging region configured to be displayed by the 3D electronic display and is visually enhanced within the display portion. The 3D vehicle monitoring system according to claim 12, wherein the object is perceived as being in front of the zero parallax plane. The image processor is further configured to apply a projection transformation to the 3D model before a portion of the 3D imaging region is displayed on the 3D electronic display, and the projection transformation is performed within the display portion. The relative size of the visually emphasized object is configured to be larger than that of the other object, and the other object is located at a distance longer than the threshold distance corresponding to the zero parallax plane. The 3D vehicle monitoring system according to claim 12. The 3D vehicle monitoring system according to claim 12, wherein the 3D electronic display includes a multi-beam grid-based 3D electronic display. A step of capturing a 3D scan of an area adjacent to the vehicle using a 3D scanner,
A step of generating a 3D model from the captured 3D scan, wherein the 3D model includes a spatial configuration of an object located within the scanned area.
A step of displaying a part of the scanning area using the 3D model is provided.
The step of displaying a part of the scanning area includes a step of visually emphasizing an object in the display portion whose distance from the vehicle is shorter than the threshold distance.
A vehicle monitoring method in which the visually emphasized object is configured to improve user perception of the object closer to the vehicle than the threshold distance. The step of displaying a portion of the scanning area further comprises the step of using a 3D electronic display so that the visually emphasized object is in front of the zero parallax plane of the 3D electronic display corresponding to the threshold distance. 17. The step of generating the 3D model comprises applying a projection transformation to the 3D model prior to the step of displaying the portion of the scanning area on the 3D electronic display. The vehicle monitoring method described in. The display is such that the projective transformation increases the relative size of the visually emphasized object as compared to an object in the scanning region at a distance greater than the threshold distance corresponding to the zero parallax plane. The vehicle monitoring method according to claim 18, wherein the depth compression of the portion is included. The step of displaying a part of the scanning area using the 3D electronic display is
Steps to guide light as a light beam at a non-zero propagation angle in a flat plate light guide,
A plurality of steps corresponding to different visual images of the 3D electronic display, which is a step of coupling a part of the waveguide light beam by diffraction and taking it out by using an array of multi-beam diffraction gratings on the flat plate light guide. A portion of the waveguide light beam is diffraction-coupled, comprising a plurality of couplings oriented away from the flat plate light guide to generate an outward light beam in different principal maximal angles. And the steps to go outside
A step of modulating a plurality of coupled light beams emitted to the outside by using a plurality of light valves, wherein the modulated light beam displays a part of the light beam as a 3D image. 18. The vehicle monitoring method of claim 18, comprising a step of modulating, representing pixels of a display.