JPH10117340A - Vehicle periphery monitoring device, alarm display method to be used for device and medium-storing display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a device enabling caution by displaying information showing which object can become an obstacle dangerous for the travel of a present vehicle to a driver by displaying the image of the arbitrary designated obstacle, while intensifying it when the existence of plural obstacle is estimated based on a picked-up image. SOLUTION: This device has an image-reading means 12, feature point extracting means 14, marking operating means 22 and superimpose operating means 24 or the like. The image-reading means 12 is installed on the vehicle and by picking up the stereoscopic image of a monitoring area around the vehicle, raw image information 12a having three-dimensional coordinate information is generated. Then, this device has a function for performing intensified image display through superimposed display using superimposed image information 24a, concerning the image of the obstacle estimated existent by a position close to a traveling direction among obstacles inside an estimated present vehicle traveling track range, when the existence of plural obstacles inside the estimated present vehicle traveling track range is estimated based on the picked-up image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle periphery monitoring device, an alarm display method used in the vehicle periphery monitoring device,
And a medium storing the alarm display method, in particular, a vehicle periphery monitor that is installed in a vehicle such as a bus or a truck, captures an image of a monitoring area around the vehicle, and monitors the periphery of the vehicle based on the captured image. Device, a method for displaying an obstacle to be alerted at the time of surrounding monitoring to a driver or a worker by using the vehicle surroundings monitoring device, and a computer for controlling the vehicle surroundings monitoring device which stores the warning displaying method. Regarding possible media.

[0002]

2. Description of the Related Art FIG. 5 is a functional block diagram for explaining a conventional vehicle periphery monitoring device and an alarm display method used in the device.

Conventionally, this type of vehicle periphery monitoring device includes:
For example, as shown in FIG. 5, Japanese Unexamined Patent Publication No. 7-250319 (Title of Invention: Vehicle Perimeter Monitoring Device, Filing Date: 199)
(March 14, 4).

That is, as shown in FIG. 5, a vehicle periphery monitoring device A monitors the periphery of a vehicle based on image data obtained by two photographing means. A memory (right) 1 and a memory (left) 2 for recording signals, a distortion correction unit 7 for correcting distortion of a recorded image in the memory 1 (2), and a recorded image in one of the memories 1 (2) Is assumed to have a height of 0, a projected image is created by projecting this image to the other memory 2 (1), and the image on the road surface is removed from the difference between the projected image and the image data of the other memory 2 (1). Road surface image removing means 3, an object edge detecting means 4 for detecting an edge of an object from a horizontal differential value of the other memory 2 (1) and image data output from the road surface image removing means 3, and a memory 1 ( 2) Image data and object edge There was an object position calculating means 5 for calculating the position of the obstacle from the image data output from the outputting means 4, and an alarm means 6 for outputting an alarm based on the position data calculated by the object position calculating means. .

[0005] The vehicle periphery monitoring device A having such a configuration removes a pattern drawn on a road surface by image signals photographed by two cameras, extracts only a tall object, and reduces the processing time. The effect that the length can be shortened is disclosed.

[0006]

However, such a conventional vehicle periphery monitoring apparatus A removes a pattern drawn on a road surface by image signals photographed by two cameras and removes only a tall object. And display or alarm was executed.

For this reason, all objects having a height can be extracted and displayed indiscriminately, but information on which objects can be dangerous obstacles to the running of the host vehicle (that is, information on the objects). , Information about the degree of danger) is not warned to the driver or operator.

That is, the driver or the worker himself / herself monitors an extraction display or a warning of a tall object while the own vehicle is running,
Concentrate on safe operation of your vehicle as a result of having to determine which of the objects for which the indication or warning has been issued can be a dangerous obstacle to your vehicle's traveling while operating your vehicle. There was a technical problem that was difficult.

An object of the present invention is to solve such a conventional problem. In particular, the present invention relates to a method for estimating a trajectory of a vehicle in accordance with an operation of the vehicle. When it is estimated that at least one or more obstacles are present based on the captured image, from among the obstacles,
Display of an emphasized image of an obstacle to be emphasized which is presumed to be located closest to the vehicle with respect to the traveling direction, and movement of the obstacle while maintaining a predetermined positional relationship with display coordinate points of the obstacle to be emphasized A vehicle surroundings monitoring device that is displayed as a superimposed image synthesized by superimposing a marking display that displays a mark of a predetermined shape in synchronization with a stereoscopic image of a monitoring area around the vehicle installed on the vehicle Image reading means for generating raw image information having three-dimensional coordinate information by capturing images of the vehicle, travel path estimating means for generating own vehicle travel expected locus information over the own vehicle travel expected locus range, and own vehicle travel expected locus information When it is estimated that at least one or more obstacles are present within the range of the expected trajectory of the vehicle based on the raw image information, a scanning line having a predetermined resolution is given to the raw image information. Using the three-dimensional coordinate information and the feature point information included in the raw image information while detecting the intersection between the scanning line and the obstacle image and generating feature point information on the obstacle based on the intersection. From among the obstacles, an obstacle existing at a position closest to the vehicle with respect to the traveling direction of the vehicle is estimated, and a nearby feature point of the obstacle estimated to be located at the closest position is referred to as nearby feature point information. Closest feature point extraction means to select from feature point information as
A three-dimensional space range having a predetermined distance from the closest feature point is provided using the three-dimensional coordinate information, feature point information, and closest feature point information of the raw image information, and three-dimensional space included in the three-dimensional space range is provided. A feature point group extracting means for selecting a feature point group, which is a set of feature points having coordinates, from feature point information as feature point group information; and obtaining a rectangle surrounding the entire feature point group, and calculating coordinates of the rectangle. Coordinate extracting means for generating as coordinate information, marking calculating means for generating coordinates for marking and displaying a rectangle as a mark on an obstacle image using the coordinate information and feature point information as marking information, and marking information for the raw image Superimposition calculating means for generating superimposed image information, which is a superimposed image superimposed on the information, and a predicted trajectory of the vehicle Display means for performing image emphasis display by superimposing and displaying an image of an obstacle presumed to be present at a position closest to the traveling direction from the obstacles in the surroundings using superimposed image information; The information (information about the degree of danger) as to which object can be a dangerous obstacle to the traveling of the own vehicle is provided at a position closest to the vehicle in the traveling direction by the vehicle periphery monitoring device having Then, it is possible to warn the driver or worker by displaying the superimposed image obtained by superimposing the emphasized image display and the marking display of the estimated obstacle to be emphasized and combining them. It is an issue.

Further, as a result of this superimposed display, the driver or the operator himself / herself can identify obstacles that have appeared in the range of the expected trajectory of the own vehicle by any object that has appeared in the range of the expected trajectory of the own vehicle. The device automatically estimates whether the vehicle may become a dangerous obstacle to the running of the host vehicle and displays the marking.Therefore, while operating the host vehicle, monitor and determine while the host vehicle is running. It is an object of the present invention to eliminate the necessity and easily concentrate on the safe operation of the vehicle.

Further, there is provided an alarm display method used in a vehicle periphery monitoring device having such a configuration, and a computer-readable medium storing such an alarm display method, wherein the medium is installed in a vehicle and is located around the vehicle. An image reading step having a step of capturing a stereoscopic image of the monitoring area and a step of generating raw image information having three-dimensional coordinate information;
A traveling locus estimating step of generating the own vehicle traveling predicted locus information; and a step of estimating the presence of an obstacle within the own vehicle traveling predicted locus range based on the own vehicle traveling predicted locus information and the raw image information. When it is estimated that there is one or more obstacles, a process of giving a scanning line of a predetermined resolution to the raw image information to detect an intersection of the scanning line and the obstacle image, and generating feature point information based on the intersection And a step of estimating an obstacle existing at a position closest to the vehicle with respect to the traveling direction of the vehicle from the obstacles using the three-dimensional coordinate information and the feature point information. A step of selecting the closest feature point from the feature point information as the closest feature point information, and using the three-dimensional coordinate information, the feature point information, and the closest feature point information included in the raw image information. The closest feature point A feature point group extracting step including a step of providing a three-dimensional space range in the heart, a step of selecting a feature point group included in the three-dimensional space range from the feature point information as feature point group information, and a feature point group A step of obtaining a rectangle that encloses all of them, a coordinate extraction step having a step of generating the coordinates of the rectangle as coordinate information, and using the coordinate information and feature point information, marking information on the obstacle image with the rectangle as a mark A marking calculation step having a step of generating, and a superimposition calculation step having a step of generating superimposed image information by superimposing the marking information on the raw image information; and From inside, an image of an obstacle presumed to be present at a position closest to the traveling direction is superimposed and displayed using superimposed image information. And a computer-readable medium storing such an alarm display method, which objects are dangerous obstacles to the traveling of the vehicle. (Information relating to the degree of danger) is superimposed on the highlighted image display and the marking display of the obstacle to be emphasized, which is estimated to be located closest to the vehicle with respect to the traveling direction, and synthesized. It is an object to be able to display and warn a driver or worker as a superimposed image.

Furthermore, as a result of this superimposed display, the driver or the operator himself / herself can identify obstacles that have appeared in the expected travel range of the own vehicle by any object that has appeared in the expected travel range of the own vehicle. The device automatically estimates whether the vehicle may become a dangerous obstacle to the running of the host vehicle and displays the marking.Therefore, while operating the host vehicle, monitor and determine while the host vehicle is running. It is an object of the present invention to eliminate the necessity and easily concentrate on the safe operation of the vehicle.

[0013]

According to a first aspect of the present invention, there is provided a vehicle periphery monitoring system which is installed in a vehicle, captures an image of a monitoring area around the vehicle, and monitors the periphery of the vehicle based on the captured image. In the apparatus, at least one vehicle based on the captured image is included in a range of the estimated travel trajectory 30A of the vehicle, which is an estimated range of the trajectory in which the vehicle travels in response to the operation of the vehicle.
When there is more than one obstacle, it is configured to highlight and display an image of any specified obstacle from the obstacles. The device 10.

In the present invention, the vehicle means a vehicle operated by a driver or a worker himself. In addition, the monitoring area around the vehicle means a blind spot area that is difficult for the driver to visually recognize, such as an area near the tail of the vehicle or an area near the side of the vehicle. Means the area near the tail and the area near the side that are difficult to see with a room mirror or side mirror. In addition, the obstacle means an object that may cause a problem if the vehicle comes into contact with the vehicle when traveling, for example, a pedestrian or a building existing within the range of the predicted traveling trajectory of the vehicle near the rear. It means things and the like.

According to the first aspect of the present invention, information (risk degree information) on which object can be a dangerous obstacle to the traveling of the vehicle is provided to the driver or the worker. This makes it possible to display and warn the user.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined as to which objects appearing within the expected travel trajectory 30A can be dangerous obstacles for the travel of the own vehicle. Since the device automatically estimates and highlights the data, it is not necessary for the driver or operator to monitor and determine while operating the vehicle while operating the vehicle. The burden of monitoring work on
Another effect is that the user can easily concentrate on the safe operation of the vehicle.

According to a second aspect of the present invention, in the vehicle surroundings monitoring device according to the first aspect, at least one or more obstacles based on the captured image are present within the range of the expected trajectory of the own vehicle traveling 30A. When it is estimated that the obstacle exists, the image of the obstacle that is estimated to be present within a predetermined distance range with respect to the traveling vehicle from the obstacle is emphasized and displayed. The vehicle periphery monitoring device 10 described above.

According to the invention described in claim 2, according to claim 1,
In addition to the effects described in (1), it is estimated that information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the own vehicle exists within a predetermined distance range with respect to the traveling direction. It is possible to display an emphasized image of an obstacle to be emphasized and warn a driver or an operator.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined as to which objects appearing within the expected travel trajectory 30A can be dangerous obstacles for the travel of the own vehicle. Since the device automatically estimates and highlights the data, it is not necessary for the driver or operator to monitor and determine while operating the vehicle while operating the vehicle. The burden of monitoring work on
Another effect is that the user can easily concentrate on the safe operation of the vehicle.

The invention described in claim 3 is the first or second invention.
In the vehicle surroundings monitoring device 10 described in the above, if it is estimated that at least one or more obstacles are present in the range of the expected traveling trajectory 30A of the own vehicle based on the captured image, the vehicle travels from the obstacles. A vehicle surroundings monitoring device configured to emphasize and display an image of an obstacle estimated to be present at a position closest to a vehicle with respect to a direction.

According to the invention described in claim 3, according to claim 1 of the present invention,
Or, in addition to the effect described in 2, the information (risk degree information) on which object can be a dangerous obstacle to the traveling of the own vehicle exists at a position closest to the vehicle in the traveling direction. Then, it is possible to display an emphasized image of an estimated obstacle to be emphasized and warn a driver or an operator.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined as to which object appearing within the expected travel trajectory 30A can be a dangerous obstacle for the travel of the own vehicle. Since the device automatically estimates and highlights the data, it is not necessary for the driver or operator to monitor and determine while operating the vehicle while operating the vehicle. The burden of monitoring work on
Another effect is that the user can easily concentrate on the safe operation of the vehicle.

The invention described in claim 4 is the first to third aspects of the present invention.
In the vehicle surroundings monitoring device 10 according to any one of the above, the image highlighting displays a mark of a predetermined shape while maintaining a predetermined positional relationship with a display coordinate point of the obstacle to be emphasized. The vehicle periphery monitoring device 10 is configured to be executed together with
It is.

The image highlighting in the present invention is
This means that the information is displayed on the display screen in a display form that is easy for the driver or operator to visually recognize.

According to the invention described in claim 4, according to claim 1 of the present invention,
In addition to the effects described in any one of (1) to (3), information (risk degree information) on which object can be a dangerous obstacle to the traveling of the own vehicle is transmitted to the vehicle in the traveling direction. The effect that it becomes possible to warn a driver or an operator by using an enhanced image display and a marking display executed on an obstacle to be emphasized which is presumed to be located near Play.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined as to which objects appearing within the expected travel trajectory 30A can be dangerous obstacles to the travel of the own vehicle. Since the device automatically estimates and displays the markings, there is no need for the driver or operator to monitor and determine while operating the own vehicle while operating the own vehicle. This also has the effect of reducing the burden of monitoring work by the user and making it easier to concentrate on the safe operation of the vehicle.

According to a fifth aspect of the present invention, in the vehicle periphery monitoring device according to the fourth aspect, the marking display has a predetermined positional relationship with a display coordinate point of the obstacle to be emphasized. The vehicle periphery monitoring device 10 is configured to move the mark in synchronization with the movement of the obstacle while maintaining the predetermined positional relationship.

In the claims, synchronizing with the movement of an obstacle means that the timing and amount of movement of the obstacle, the movement amount, the movement direction, the movement speed, and the like are made to match the movement state of the obstacle. .

According to the invention described in claim 5, according to claim 4,
In addition to the effects described in (1), it is estimated that information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the own vehicle exists at a position closest to the vehicle in the traveling direction. There is an effect that the marking can be displayed in synchronization with the movement of the emphasized image display of the obstacle to be emphasized to warn the driver or the operator.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined as to which objects appearing within the expected travel trajectory 30A can be dangerous obstacles for the travel of the own vehicle. The device automatically estimates and displays markings in synchronization with the movement of obstacles, so that the driver or operator can monitor and determine while operating his own vehicle while operating his own vehicle. This eliminates the necessity, reduces the burden on the driver and the operator for monitoring work, and also makes it easier to concentrate on the safe operation of the vehicle.

The invention according to claim 6 is the invention according to claim 4 or 5
In the vehicle surroundings monitoring device 10, the image highlighting is configured to be displayed as a superimposed image in which the marking display and the image display of the obstacle to be emphasized are superimposed and synthesized. The vehicle periphery monitoring device 10 is characterized in that:

According to the invention of claim 6, according to claim 4,
Or in addition to the effect described in 5, the information (risk degree information) on which object can be a dangerous obstacle to the traveling of the own vehicle exists at a position closest to the vehicle in the traveling direction. Then, the enhanced image display and the marking display of the estimated obstacle to be emphasized are superimposed and displayed as a superimposed image combined with the driver or the operator to be able to warn. To play.

Further, as a result of this superimposed display, obstacles appearing within the predicted travel trajectory 30A of the own vehicle can be compared with obstacles appearing within the predicted travel trajectory 30A of the own vehicle. The device will automatically estimate whether it may become a dangerous obstacle and display the markings, so that the driver or worker can monitor the vehicle while operating while operating the vehicle. This eliminates the need to make a determination, and reduces the burden on the driver and the worker in monitoring work, thereby making it easier to concentrate on the safe operation of the vehicle.

According to a seventh aspect of the present invention, there is provided the vehicle periphery monitoring apparatus according to the sixth aspect, wherein a three-dimensional image of the monitoring area in the vicinity of the vehicle, which is installed in the vehicle, is taken and has three-dimensional coordinate information. Image reading means 12 for generating image information 12a, travel trajectory estimating means 30 for generating own vehicle travel expected trajectory information 30a over the range of the own vehicle travel expected trajectory 30A, the own vehicle travel expected trajectory information 30a and the raw Based on the image information 12a, when it is estimated that at least one or more of the obstacles is present within the range of the expected traveling trajectory 30A of the host vehicle, the raw image information 12a
, Lm of a predetermined resolution are provided to detect the intersections of the scanning lines L1,..., Lm with the obstacle image, and based on the intersections, feature point information P1,. , Pn, and the three-dimensional coordinate information of the raw image information 12a and the feature point information P1,..., Pn from the obstacle in the traveling direction of the vehicle. .., Pn, the closest feature point of the obstacle estimated to be at the closest position is determined as the closest feature point information Pmin. The closest feature point extracting means 16 for selecting from among the raw image information 12a
Using the three-dimensional coordinate information, the feature point information P1,..., Pn and the close feature point information Pmin, a three-dimensional space range having a predetermined distance around the closest feature point is provided and included in the three-dimensional space range. A feature point group, which is a set of feature points having three-dimensional coordinates, is referred to as feature point group information {P}.
A feature point group extracting means 18 for selecting from among 1,..., Pn;
Along with finding a rectangle that surrounds all the feature points,
Using the coordinate extracting means 20 for generating the coordinates of the rectangle as coordinate information {a1, a2} and the coordinate information {a1, a2} and the feature point information P1,. As the mark, a marking calculating means 22 for generating coordinates for displaying the marking as marking information 22a, and a superimposed image which is the superimposed image obtained by superimposing and combining the marking information 22a with the raw image information 12a. Superimpose calculating means 24 for generating image information 24a;
From the obstacles within the range of the expected traveling trajectory 30A of the host vehicle, an image of an obstacle estimated to be located at a position closest to the traveling direction is displayed as the superimposed image information 24a.
And a display unit 26 for performing the image emphasis display by performing the superimposed display by using the image display device.

In the present invention, the scanning lines L1,
, Lm may be horizontal scanning lines or vertical scanning lines with respect to the display.

According to the invention described in claim 7, according to claim 6,
In addition to the effects described in the above, the risk information estimated around the traveling trajectory estimating means 30, the feature point extracting means 14, and the close feature point extracting means 16 is set to a position closest to the vehicle in the traveling direction. The display of the emphasized image of the obstacle to be emphasized, which is presumed to be present, and the marking display performed mainly by the marking calculation means 22 are performed by the driver as a superimposed display performed mainly by the superimposition calculation means 24. And an operator can be warned.

Furthermore, as a result of the superimposition display being enabled by the superimposition calculating means 24, any obstacles appearing within the expected travel trajectory 30A of the own vehicle will be replaced by obstacles that appear within the expected travel trajectory 30A. The apparatus side (specifically, the traveling locus estimating means 30, the feature point extracting means 14, and the closest feature point extracting means 16) automatically determines whether the obstacle can be a dangerous obstacle to the traveling of the vehicle. Since the marking calculation means 22 displays the markings, the driver and the worker do not need to monitor and determine while operating the own vehicle while operating the own vehicle. This also has the effect of reducing the work load and making it easier to concentrate on the safe operation of the vehicle.

The invention described in claim 8 is the invention according to claims 1 to 7
In the alarm display method used in the vehicle periphery monitoring device according to any one of the above, it is estimated whether or not the obstacle is present within the range of the expected traveling trajectory 30A of the vehicle according to the operation of the vehicle. And, if it is estimated that at least one or more obstacles are present based on the captured image, a step of executing the image highlighting of any designated obstacle from the obstacles And a warning display method.

According to the eighth aspect of the present invention, information (risk degree information) on which object can be a dangerous obstacle to the traveling of the host vehicle is provided to the driver or the worker. This makes it possible to display and warn the user.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined as to which object appearing within the expected travel trajectory 30A can be a dangerous obstacle for the travel of the own vehicle. Since the device automatically estimates and highlights the data, it is not necessary for the driver or operator to monitor and determine while operating the vehicle while operating the vehicle. The burden of monitoring work on
Another effect is that the user can easily concentrate on the safe operation of the vehicle.

According to a ninth aspect of the present invention, in the alarm display method according to the eighth aspect, it is determined whether or not the obstacle is present within the range of the expected traveling trajectory 30A of the vehicle according to the operation of the vehicle. Estimating whether there is at least one or more obstacles based on the captured image, from among the obstacles, it is estimated that the obstacles are present within a predetermined distance range with respect to the traveling vehicle. Performing the image highlighting of the obstacle.

According to the ninth aspect of the present invention, information (risk degree information) on which object can be a dangerous obstacle to the traveling of the host vehicle is determined with respect to the traveling direction. There is an effect that an emphasis image is displayed for an obstacle to be emphasized which is presumed to be present within the distance range to warn a driver or an operator.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined by determining which objects appearing within the expected travel trajectory 30A of the own vehicle can be dangerous obstacles for the travel of the own vehicle. Since the device automatically estimates and highlights the data, it is not necessary for the driver or operator to monitor and determine while operating the vehicle while operating the vehicle. The burden of monitoring work on
Another effect is that the user can easily concentrate on the safe operation of the vehicle.

According to a tenth aspect of the present invention, in the warning display method according to the eighth or ninth aspect, the obstacle is present within the range of the expected traveling trajectory 30A of the vehicle in accordance with the operation of the vehicle. Estimating whether or not, and if it is estimated that at least one or more obstacles are present based on the captured image, from among the obstacles, it is estimated that they are present at the closest position to the traveling direction Performing the image highlighting of the obstacle.

According to the tenth aspect of the present invention, information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the own vehicle is determined based on the traveling direction. This makes it possible to display an emphasized image on an obstacle to be emphasized, which is presumed to be present at a position close to the image, and warn a driver or an operator.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined as to which objects appearing within the expected travel trajectory 30A can be dangerous obstacles for the travel of the own vehicle. Since the device automatically estimates and highlights the data, it is not necessary for the driver or operator to monitor and determine while operating the vehicle while operating the vehicle. The burden of monitoring work on
Another effect is that the user can easily concentrate on the safe operation of the vehicle.

According to an eleventh aspect of the present invention, in the alarm display method according to any one of the eighth to tenth aspects, the image highlighting is performed by using a display coordinate point of the obstacle to be emphasized and a predetermined coordinate point. An alarm display method, comprising: performing a step of executing the display together with the marking display for displaying the mark while maintaining a positional relationship.

According to the eleventh aspect of the present invention, in addition to the effects of any one of the eighth to tenth aspects, any object can be a dangerous obstacle to the running of the host vehicle. Information (risk degree information) on the vehicle is driven by using an emphasized image display and a marking display executed on an obstacle to be emphasized, which is estimated to be located closest to the vehicle with respect to the traveling direction. There is an effect that a warning can be given to a worker or an operator.

As a result, the obstacle appearing in the predicted traveling trajectory 30A of the own vehicle can be determined as to which objects appearing in the predicted traveling trajectory 30A can be dangerous obstacles for the traveling of the own vehicle. Since the device automatically estimates and displays the markings, there is no need for the driver or operator to monitor and determine while operating the own vehicle while operating the own vehicle. This also has the effect of reducing the burden of monitoring work by the user and making it easier to concentrate on the safe operation of the vehicle.

According to a twelfth aspect of the present invention, in the alarm display method according to the eleventh aspect, the marking display includes:
A step of holding the display coordinate point of the obstacle to be emphasized and a predetermined positional relationship, and moving the mark in synchronization with the movement of the obstacle while maintaining the predetermined positional relationship. This is a feature of the alarm display method.

According to the twelfth aspect of the present invention, in addition to the effect of the eleventh aspect, information on which object can be a dangerous obstacle to the running of the host vehicle (degree of risk) Information) in synchronization with the movement of the highlighted image display of the obstacle to be emphasized, which is presumed to be located closest to the vehicle with respect to the traveling direction, to warn the driver or operator. This has the effect of being able to perform.

As a result, the obstacle appearing within the expected travel trajectory 30A of the own vehicle can be determined by determining which objects appearing within the expected travel trajectory 30A can be dangerous obstacles for the travel of the own vehicle. The device automatically estimates and displays markings in synchronization with the movement of obstacles, so that the driver or operator can monitor and determine while operating his own vehicle while operating his own vehicle. This eliminates the necessity, reduces the burden on the driver and the operator for monitoring work, and also makes it easier to concentrate on the safe operation of the vehicle.

According to a thirteenth aspect of the present invention, in the alarm display method according to the eleventh or twelfth aspect, the image highlighting is performed by superimposing the marking display and the image display of the obstacle to be emphasized. And displaying the displayed superimposed image.

According to the thirteenth aspect of the present invention, in addition to the effect of the eleventh or twelfth aspect, information on which object can be a dangerous obstacle to the running of the host vehicle ( Risk information) is superimposed as a superimposed image obtained by superimposing and marking the highlighted image display and the marking display of the obstacle to be emphasized, which is estimated to be located closest to the vehicle with respect to the traveling direction. It is possible to display and warn an operator.

Further, as a result of this superimposed display, obstacles appearing within the expected travel trajectory 30A of the own vehicle can be compared with any objects appearing within the expected travel trajectory 30A of the own vehicle. The device will automatically estimate whether it may become a dangerous obstacle and display the markings, so that the driver or worker can monitor the vehicle while operating while operating the vehicle. This eliminates the need to make a determination, and reduces the burden on the driver and the worker in monitoring work, thereby making it easier to concentrate on the safe operation of the vehicle.

According to a fourteenth aspect of the present invention, in the alarm display method according to the thirteenth aspect, a step of capturing a three-dimensional image of the monitoring area in the vicinity of the vehicle installed on the vehicle,
An image reading step having a step of generating the raw image information 12a having the three-dimensional coordinate information; a travel path estimating step of generating the own vehicle travel expected track information 30a; The raw image information 1
Estimating the presence of the obstacle within the range of the predicted travel trajectory 30A based on 2a;
When it is estimated that there are two or more obstacles, the raw image information 12a includes scanning lines L1,.
., Lm and the obstacle image, and generating the feature point information P1,..., Pn based on the intersection. And the three-dimensional coordinate information and the feature point information P
Estimating an obstacle located closest to the vehicle with respect to the traveling direction of the vehicle from the obstacles using 1,..., Pn;
A step of selecting from the feature point information P1,..., Pn;
2a has the three-dimensional coordinate information and the feature point information P
Providing the three-dimensional space range around the closest feature point using 1,..., Pn and the closest feature point information Pmin;
Selecting the feature point group included in the three-dimensional space range from the feature point information P1,..., Pn as the feature point group information {P}; A coordinate extraction step having a step of obtaining the rectangle so as to surround all the point groups, a step of generating coordinates of the rectangle as the coordinate information {a1, a2}, the coordinate information {a1, a2} and the feature Using the point information P1,..., Pn to generate the marking information 22a using the rectangle as the mark on the obstacle image; and superimposing the marking information 22a on the raw image information 12a. A superimposition calculating step including a step of generating the superimposed image information 24a together; A display step of performing the image highlighting by displaying the image of the obstacle estimated to be present at the close position with respect to the direction using the superimposed image information 24a. An alarm display method characterized by the following.

According to the fourteenth aspect of the present invention, in addition to the effects of the thirteenth aspect, there is a danger that is estimated centering on the traveling trajectory estimation step, the feature point extraction step, and the closest feature point extraction step. The degree information, the emphasized image display of the obstacle to be emphasized, which is estimated to be located closest to the vehicle with respect to the traveling direction, and the marking display executed mainly in the marking calculation step are superimposed calculation steps. There is an effect that it is possible to warn a driver or an operator as a superimposed display executed mainly.

Further, as a result of the superimposed display being performed by the superimposed calculation step, the driver or the operator himself / herself can move the obstacle appearing within the predicted traveling trajectory A of the own vehicle into the predicted traveling trajectory A of the own vehicle. The device (specifically, the trajectory estimation process, the feature point extraction process, and the closest feature point extraction process) automatically determines which objects appearing in the vehicle can be dangerous obstacles to the running of the vehicle. And the marking calculation process displays the markings, eliminating the need to monitor and judge while the own vehicle is running while operating the own vehicle, reducing the burden on the driver and workers in monitoring work It is also possible to easily concentrate on the safe operation of the vehicle.

According to a fifteenth aspect of the present invention, there is provided a computer-readable medium storing the alarm display method according to the tenth aspect, wherein the expected traveling trajectory 30A of the vehicle according to the operation of the vehicle is provided. Estimating whether or not the obstacle exists in the range, and when it is estimated that at least one or more obstacles are present based on the captured image, from among the obstacles, A step of executing the image highlighting of the obstacle estimated to be present at the close position.

According to the fifteenth aspect, the same effect as the tenth aspect can be obtained.

According to a sixteenth aspect of the present invention, there is provided a computer-readable medium storing the alarm display method according to the eleventh aspect, wherein the image highlighting is a display coordinate of the obstacle to be emphasized. A medium which has a step executed together with the marking display for displaying the mark while maintaining a predetermined positional relationship with a point.

According to the sixteenth aspect, the same effect as the eleventh aspect can be obtained.

According to a seventeenth aspect of the present invention, there is provided a computer-readable medium storing the alarm display method according to the twelfth aspect, wherein the marking display includes a display coordinate point of the obstacle to be emphasized. And a step of moving the mark in synchronization with the movement of the obstacle while maintaining the predetermined positional relationship.

According to the seventeenth aspect, the same effect as the twelfth aspect is obtained.

The invention according to claim 18 is a computer-readable medium storing the alarm display method according to claim 13, wherein the image highlighting includes the marking display and the obstacle to be emphasized. And displaying a superimposed image that is superimposed and synthesized with the image display.

According to the eighteenth aspect, the same effect as that of the thirteenth aspect can be obtained.

According to a nineteenth aspect of the present invention, there is provided a computer-readable medium storing the alarm display method according to the fourteenth aspect, wherein the three-dimensional image of the monitoring area installed in a vehicle and around the vehicle is captured. An image reading step having a step of generating the raw image information 12a having the three-dimensional coordinate information; a traveling path estimating step of generating the own vehicle traveling expected path information 30a; A step of estimating the existence of the obstacle within the range of the expected traveling trajectory 30A of the vehicle based on the trajectory information 30a and the raw image information 12a; and a case where it is estimated that at least one or more of the obstacles are present. .., Lm having a predetermined resolution to the raw image information 12a to detect an intersection between the scanning lines L1,..., Lm and the obstacle image. The feature point information P based on the intersection
, Pn generating step, the three-dimensional coordinate information and the characteristic point information P1,.
Estimating an obstacle present at a position closest to the vehicle with respect to the traveling direction of the vehicle from the obstacles using Pn, and using the close feature point as close feature point information Pmin; , Pn, and a step of selecting from the closest feature points, and the three-dimensional coordinate information and the feature point information P1,..., Of the raw image information 12a.
Providing the three-dimensional space range around the closest feature point using Pn and the closest feature point information Pmin; and converting the feature point group included in the three-dimensional space range into the feature point group information ｛P .., Pn as｝, a step of obtaining the quadrangle surrounding the entirety of the characteristic point group, and a step of obtaining coordinates of the quadrangle by the coordinates A coordinate extraction step having a step of generating as information {a1, a2};
a2} and the feature point information P1,..., Pn to generate the marking information 22a using the rectangle as the mark in the obstacle image; and generating the marking information 22a. A superimposition calculation step including a step of generating the superimposed image information 24a by superimposing the superimposed image information 24a on the image information 12a; A display step of performing the image emphasis display by superimposing and displaying the image of the obstacle estimated to be present at the position using the superimposed image information 24a. is there.

According to the nineteenth aspect, the same effect as that of the fourteenth aspect can be obtained.

[0069]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The basic functions of the present invention will be described below with reference to the drawings.

The present invention relates to a vehicle periphery monitoring device which is installed in a vehicle such as a bus or a truck, captures an image of a monitoring area around the vehicle, and monitors the periphery of the vehicle based on the captured image. Using the device, a method of displaying obstacles to be alerted during driver monitoring to drivers and workers,
And a computer-readable medium that stores the alarm display method and controls the vehicle periphery monitoring device.

The vehicle surroundings monitoring device according to the present invention includes at least one vehicle based on a captured image within an estimated travel range of the own vehicle, which is an estimated range of a travel route of the vehicle in response to the operation of the vehicle.
When it is estimated that there is one or more obstacles, an image of a specified arbitrary obstacle is emphasized and displayed (hereinafter, referred to as image highlighting) from the obstacles.

In the present invention, the vehicle means a vehicle operated by a driver or a worker himself. In addition, the monitoring area around the vehicle means a blind spot area that is difficult for the driver to visually recognize, such as an area near the tail of the vehicle or an area near the side of the vehicle. Means the area near the tail and the area near the side that are difficult to see with a room mirror or side mirror.

The obstacle means an object which becomes a problem if it comes into contact when the vehicle is running.
For example, it means a pedestrian, a building, or the like existing within the range of the predicted traveling trajectory of the vehicle near the rear.

Such a vehicle periphery monitoring device displays information (risk degree information) on which object can become a dangerous obstacle to the traveling of the host vehicle to the driver or the operator. Warning function. In other words, the driver or the worker himself / herself converts an obstacle that appears in the expected travel range of the own vehicle into an obstacle that is dangerous to the travel of the own vehicle. The device has a function of automatically estimating whether to obtain it and highlighting it, eliminating the need to monitor and judge while the own vehicle is running while operating the own vehicle. Can be reduced, and it is easy to concentrate on the safe operation of the vehicle.

Further, the vehicle periphery monitoring device is configured to emphasize and display an image of an obstacle which is presumed to be within a predetermined distance range with respect to the traveling vehicle from among the obstacles. Is also possible.

In the present invention, the predetermined distance range refers to a monitoring area in the vicinity of the vehicle, particularly in a blind spot area where the driver or worker is hard to see, such as an area near the tail of the vehicle or an area near the side of the vehicle. And a monitoring area that the operator particularly wants to pay attention to. It should be noted that the predetermined distance range can be designated as desired by the driver or operator at any time via the above-described keyboard.

Such a vehicle periphery monitoring device can provide information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the own vehicle within a predetermined distance range with respect to the traveling direction. Has a function of displaying an emphasized image of an obstacle to be emphasized, which is presumed to be present, to warn a driver or an operator.

Further, the present vehicle periphery monitoring apparatus emphasizes and displays an image of an obstacle that is presumed to be present at a position closest to the vehicle (that is, a closest position) with respect to the traveling direction among the obstacles. It is also possible.

Such a vehicle periphery monitoring device can provide information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the own vehicle, at a position closest to the vehicle in the traveling direction. Has a function of displaying an emphasized image of an obstacle to be emphasized, which is presumed to be present, to warn a driver or an operator.

When it is estimated that at least one or more obstacles are present on the basis of the captured image, the present vehicle periphery monitoring device designates an obstacle by a driver or a worker at any time. It has a function of specifying a plurality of items via the above-mentioned keyboard.

Further, the present vehicle periphery monitoring apparatus executes a marking display for displaying a mark of a predetermined shape while maintaining a predetermined positional relationship with a display coordinate point of an obstacle to be image-enhanced display, together with the above-described image emphasis display. It is also possible to add functions.

Here, the image highlighting in the present invention is
This means that the information is displayed on the display screen of the display in a display form that is easy for the driver or operator to visually recognize.

By providing such a simultaneous execution function of the marking display and the image emphasis display, a vehicle periphery monitor that displays and warns an obstacle in the range of the expected trajectory of the vehicle in a form that is easy for the driver or operator to see. Becomes possible.

Further, in the present vehicle periphery monitoring apparatus, the above-mentioned marking display maintains a predetermined positional relationship with the display coordinate point of the obstacle to be image-enhanced and displays the obstacle in a state where the predetermined positional relationship is maintained. It is also possible to add a function of moving the mark in synchronization with the movement.

Here, synchronizing with the movement of the obstacle means that the movement of the obstacle and the movement timing, movement amount, movement direction, movement speed, etc. are displayed on the display in accordance with the movement state of the obstacle. means.

By providing such a simultaneous execution function of the marking display and the image emphasis display, an obstacle that suddenly intrudes into the range of the expected traveling trajectory of the host vehicle is immediately captured to warn the driver or the operator. Dynamic monitoring around the vehicle.

Further, in the vehicle periphery monitoring device, a function of displaying a superimposed image obtained by superimposing the above-described marking display and the above-described image emphasis display (hereinafter, referred to as a superimposed display) is added. It is also possible.

By providing such a superimpose display function, information (risk degree information) on which object can be a dangerous obstacle to the running of the host vehicle.
To the driver or operator as a superimposed image synthesized by superimposing and marking the highlighted image display and the marking display of the obstacle to be emphasized, which is estimated to be located closest to the vehicle with respect to the traveling direction. This makes it possible to display and warn the user.

Further, as a result of the superimposed display function being enabled, the driver or the worker himself / herself replaces an obstacle that has appeared in the expected travel range of the own vehicle with any object that has appeared in the expected travel range of the own vehicle. The device automatically estimates whether or not it may be a dangerous obstacle to the running of the own vehicle and displays the marking, so that while operating the own vehicle, monitor and judge while running the own vehicle This eliminates the necessity to perform the operation, and reduces the burden on the driver and the operator for monitoring work, thereby making it easier to concentrate on the safe operation of the vehicle.

Such a vehicle periphery monitoring device executes a CPU for executing calculations and controls, a RAM for holding results of calculations, a ROM for recording programs and the like, and controls the transmission and reception of information to and from the outside. Peripheral interface, a floppy disk drive or hard disk drive for reading and writing programs and various information stored in the medium described later, a keyboard for inputting commands, etc., and a keyboard for displaying various messages It is realized by a microcomputer mainly composed of a display and the like.

Next, an embodiment of the above-described vehicle periphery monitoring device will be described with reference to the drawings.

FIG. 1 shows a vehicle periphery monitoring device 1 according to this embodiment.
FIG. 4 is a functional block diagram for explaining the function No. 0;

As shown in FIG. 1, the vehicle surroundings monitoring device 10 of the present embodiment has
When it is estimated that at least one or more obstacles are present based on the captured image, an obstacle that is estimated to be present at a position closest to the traveling direction from obstacles within the range of the expected traveling trajectory 30A of the vehicle. Has a function of performing image emphasis display by superimposing and displaying the image of the image using the superimposed image information 24a. The image reading means 12, the feature point extracting means, the closest feature point extracting means, and the feature point group extraction Means 18, coordinate extracting means 20, marking calculating means 22, superimposing calculating means 24, and displaying means 2
6 and a running locus estimating means 30.

The image reading means 12 has a function of picking up a three-dimensional image of a monitoring area in the vicinity of the vehicle installed on the vehicle and generating raw image information 12a having three-dimensional coordinate information.

The image reading means 12 comprises at least one or more CCD cameras and an image memory. Also, a CP for executing calculations and controls
U, RAM for holding calculation results and the like, ROM for recording programs and the like, peripheral interface for controlling transmission and reception of information with the outside, reading and reading of programs and various information held in a medium described later. A microcomputer mainly composed of a floppy disk device, a hard disk device and the like for executing writing is built in as an image processing processor.

The traveling trajectory estimating means 30 calculates the own vehicle traveling expected trajectory information 30a in the range of the own vehicle traveling expected trajectory 30A.
Has the function of generating

The traveling trajectory estimating means 30 controls the CPU for executing calculations and controls, the RAM for holding the results of calculations, the ROM for recording programs and the like, and controls the transmission and reception of information with the outside. It is realized by a microcomputer mainly composed of a peripheral interface to be executed, a floppy disk device and a hard disk device for reading and writing programs and various information stored in a medium described later.

The feature point extracting means 14 determines that at least one or more obstacles are present in the range of the expected travel trajectory 30A of the own vehicle based on the expected travel trajectory information 30a and the raw image information 12a. , Lm of the predetermined resolution to the raw image information 12a to provide the scanning lines L1,.
.., Lm (horizontal scanning line) and the intersection of the obstacle image are detected, and feature point information P on the obstacle is determined based on the intersection.
1,..., Pn.

In the present embodiment, the scanning lines L1,
, Lm may be horizontal scanning lines or vertical scanning lines with respect to the display.

The feature point extracting means 14 controls the CPU for executing calculations and controls, the RAM for holding the calculation results and the like, the ROM for recording programs and the like, and the control of transmission and reception of information with the outside. It is realized by a microcomputer mainly composed of a peripheral interface to be executed, a floppy disk device and a hard disk device for reading and writing programs and various information stored in a medium described later.

The closest feature point extracting means 16 outputs the raw image information 1
2a has three-dimensional coordinate information and feature point information P1,.
Using Pn, an obstacle existing at a position closest to the vehicle with respect to the running direction of the vehicle is estimated from obstacles, and a close feature point of the obstacle estimated to be located at the closest position is located close to the vehicle. As feature point information Pmin, feature point information P1,.
It has a function of selecting from Pn.

The close feature point extracting means 16 includes a CPU for executing calculations and control, a RAM for holding calculation results and the like, a ROM for recording programs and the like, and a control for transmitting and receiving information to and from the outside. And a microcomputer mainly configured with a floppy disk device, a hard disk device, and the like for reading and writing programs and various information stored in a medium described later.

The feature point group extracting means 18 outputs the raw image information 12
A three-dimensional space range having a predetermined distance from the closest feature point is provided using the three-dimensional coordinate information, feature point information P1,..., Pn, and the closest feature point information Pmin. A feature point group, which is a set of feature points having three-dimensional coordinates included therein, is defined as feature point group information {P} (point group surrounded by a broken line in FIG.
.., Pn.

The feature point group extracting means 18 includes a CPU for executing calculations and controls, a RAM for holding calculation results and the like, a ROM for recording programs and the like, and a control for transmitting and receiving information to and from the outside. And a microcomputer mainly configured with a floppy disk device, a hard disk device, and the like for reading and writing programs and various information stored in a medium described later.

The coordinate extracting means 20 obtains a square (specifically, a rectangle) surrounding the entire feature point group,
The coordinates of the rectangle are represented by coordinate information ｛a1 (x1, x2), a2 (x
1, x2)}.

The coordinate extracting means 20 executes a CPU for executing calculations and controls, a RAM for holding calculation results and the like, a ROM for recording programs and the like, and controls the transmission and reception of information to and from the outside. This is realized by a microcomputer mainly composed of a floppy disk device, a hard disk device, and the like for reading and writing a program and various kinds of information stored in a medium described later.

The marking calculating means 22 calculates the coordinate information {a
1, a2} and the feature point information P1,..., Pn are used as marking information 22a to indicate the coordinates for marking and displaying a rectangle on the obstacle image as a mark (specifically, a rectangle formed in a mesh shape). Has the function of generating.

The marking calculation means 22 executes a CPU for executing calculations and controls, a RAM for holding calculation results and the like, a ROM for recording programs and the like, and controls the transmission and reception of information with the outside. This is realized by a microcomputer mainly composed of a floppy disk device, a hard disk device, and the like for reading and writing a program and various kinds of information stored in a medium described later.

The superimposition calculating means 24 has a function of generating superimposed image information 24a which is a superimposed image obtained by superimposing the marking information 22a on the raw image information 12a.

Such a superimpose calculation means 2
Reference numeral 4 denotes a CPU for executing calculations and controls, a RAM for holding calculation results and the like, and a RO for recording programs and the like.
M, a peripheral interface that controls transmission and reception of information to and from the outside, a microcontroller mainly configured with a floppy disk device and the like for reading and writing programs and various information stored in a medium described below. It is realized by a computer.

The display means 26 uses a display mode (for example, a user-friendly display mode such as a GUI) in which the driver and the operator can easily understand the superimposed image information 24a that has undergone such processing. Display screen 26
2 can be displayed.

The display means 26 is realized by a microcomputer mainly composed of a display for displaying various messages.

Next, the operation of the vehicle periphery monitoring device 10 will be described.

FIG. 2 is a view for explaining various display forms for explaining the alarm display operation of the vehicle periphery monitoring device 10 of FIG. 1. FIG. 2 (a) is shown on the display screen 262 of the display means 26. FIG. 2 is a display form of the raw image information 12a obtained by
FIG. 2B is a diagram for explaining a display mode in which the range of the expected travel trajectory 30A of the own vehicle is superimposed and displayed on the raw image information 12a within the range of the expected travel trajectory 30A of the own vehicle. 2C is a diagram for explaining a display form in which the marking information 22a is further superimposed and displayed on the superimposed display of FIG. 2B, and FIG. For the raw image information 12a outside the 30A range, in the superimposed display mode,
FIG. 2E is a diagram for explaining a display mode in which the marking information 22a is displayed in a superimposed manner. FIG.
A display mode in which the range of the own vehicle travel expected trajectory 30A is superimposed with respect to 2a and the marking information 22a for the raw image information 12a from which the closest feature point is extracted is further superimposed and displayed will be described. FIG. 2 (f) is a display form in which the range of the expected travel trajectory 30A for the plurality of raw image information 12a is superimposed and is outside the range of the expected travel trajectory 30A. Marking information 22a is applied to the raw image information 12a from which the closest feature point is extracted.
Is not superimposed on the display, and the expected track 3
FIG. 11 is a diagram for explaining a display mode in which marking information 22a is superimposed and displayed on raw image information 12a in which a closest feature point is extracted within a 0A range.

In the vehicle periphery monitoring device 10, the raw image information 12a is displayed on the display screen 262 of the display means 26 as shown in FIG.

As shown in FIG. 2B, the vehicle periphery monitoring device 10 includes the feature point extracting means 14 and the closest feature point extracting means 1.
6. Feature point group extracting means 18 and running trajectory estimating means 30
Is controlled to superimpose and display the range of the expected travel trajectory 30A of the own vehicle with respect to the raw image information 12a within the range of the expected travel trajectory 30A of the own vehicle.

Subsequently, as shown in FIG. 2C, the marking calculation means 22 and the superimposition calculation means 24
, The marking information 22a is further superimposed on the superimposed display shown in FIG. 2B.

Subsequently, as shown in FIG. 2D, the marking calculation means 22 and the superimposition calculation means 24
Is controlled, the marking information 22a is superimposed and displayed in the superimposed display mode in the range of the own vehicle running expected locus 30A in the superimposed display mode for the raw image information 12a outside the range of the own vehicle running expected locus 30A. .

Further, as shown in FIG. 2E, a plurality of pieces of raw image information 12a within the
On the other hand, in the display form in which the range of the predicted traveling trajectory 30A of the host vehicle is superimposed, the marking information 22a for the raw image information 12a from which the closest feature point is extracted can be superimposed.

Further, as shown in FIG. 2 (f), the own vehicle traveling expected trajectory 30A for a plurality of pieces of raw image information 12a.
In the display mode where the range is superimposed,
The superimposed display of the marking information 22a is not performed on the raw image information 12a from which the closest feature points outside the predicted range of the own vehicle traveling trajectory 30A are extracted. The marking information 22a can be superimposed and displayed on the extracted raw image information 12a.

Next, the vehicle periphery monitoring device 10 of the present embodiment
An alarm display method used for the following will be described.

FIG. 3 is a flowchart for explaining an alarm display method used in the vehicle periphery monitoring device 10 of FIG.

FIG. 4 is an operation explanatory diagram for explaining an alarm display method used in the vehicle periphery monitoring device 10 of FIG. 1. FIG. 4 (a) is shown on a display screen 262 of the display means 26. FIG. 4 shows a display format of the raw image information 12a.
4B is a diagram for explaining a feature point extraction process, and FIG. 4C is a diagram for explaining a nearby feature point extraction process and a feature point group extraction process. (D)
FIG. 4E is a diagram for explaining a process of extracting coordinate points for a feature point group, and FIG. 4E is a diagram for explaining a process of generating marking information 22a for the extracted coordinate points.

Step S1 has an image reading step.

As shown in FIG. 4A, the image reading step includes a step in which the traveling trajectory estimating means 30 captures a stereoscopic image of a monitoring area provided around the vehicle and installed in the vehicle. This is a step of generating and storing the raw image information 12a having the following (see step S1 in FIG. 3).

Step S2 comprises a stop of the running locus estimating step, a running locus estimating step, and a closest feature point extracting step.

The traveling locus estimating step is a step in which the traveling locus estimating means 30 generates the own vehicle traveling predicted locus information 30a as shown in FIG. 4B (see step S2 in FIG. 3).

In the feature point extracting step, as shown in FIG. 4B, the aforementioned feature point extracting means (microcomputer) 1
4 is the own vehicle travel expected trajectory information 30a and the raw image information 1
2a, based on the own vehicle travel expected locus 30A range,
Estimating the presence of an obstacle; and, if it is estimated that at least one or more obstacles are present, the raw image information 12a
Are given scanning lines L1,..., Lm having a predetermined resolution.
Detecting a point of intersection between the Lm and the obstacle image;
And generating a feature point information P1,..., Pn based on the intersection (see step S2 in FIG. 3).

In the closest feature point extracting step, as shown in FIG. 4B, the aforementioned closest feature point extracting means (microcomputer) 16 uses the three-dimensional coordinate information and the feature point information P 1,
.., Pn to estimate an obstacle present at a position closest to the vehicle with respect to the traveling direction of the vehicle from the obstacles, and feature point information using close feature points as close feature point information Pmin. , Pn and a step of selecting from Pn (see step S2 in FIG. 3).

Step S3 has a feature point group extraction step. The feature point group extraction step in step S3 is the same as that in FIG.
As shown in (c), the above-mentioned feature point group extracting means (microcomputer) 18
Using the three-dimensional coordinate information, the feature point information P1,..., Pn and the closest feature point information Pmin to provide a three-dimensional space range around the closest feature point; .., Pn as feature point group information {P}
(See step S3 in FIG. 3).

Step S4 has a coordinate extracting step.
The coordinate extraction step in step S4 includes a step in which the coordinate extraction means (microcomputer) 20 determines a rectangle surrounding the entire feature point group, as shown in FIG. And a step of generating as {a1, a2}.

Step S4 has a marking calculation step. In the marking calculation step, as shown in FIG. 4D, the above-described marking calculation means (microcomputer) 22 performs the coordinate information {a1, a2} and the feature point information P
This is a step of generating marking information 22a by using a square as a mark on the obstacle image using 1,..., Pn (see step S4 in FIG. 3).

Step S5 includes a superimpose calculation step and a display step.

The superimpose calculation step is performed as shown in FIG.
As shown in (e), the superimpose calculating means (microcomputer) 24 described above outputs the marking information 2
In this step, superimposed image information 24a is generated by superimposing 2a on raw image information 12a (see step S5 in FIG. 3).

In the display step, as shown in FIG. 4 (e), the above-mentioned display means (display) 26 moves from the obstacle within the range of the expected traveling trajectory 30A of the own vehicle to a position closest to the traveling direction. This is a step of performing image emphasis display by superimposing and displaying an image of an obstacle presumed to be present using the superimposed image information 24a (see step S5 in FIG. 3).

As described above, according to such an embodiment of the alarm display method, the risk information estimated mainly on the traveling locus estimation step, the feature point extraction step, and the closest feature point extraction step is obtained. The emphasis image display of the obstacle to be emphasized presumed to be present at the position closest to the vehicle with respect to the traveling direction and the marking display executed around the marking operation step are mainly performed in the superimposition operation step. There is an effect that the driver or the worker can be warned as the superimposed display to be executed.

Further, as a result of the superimposed display being possible in the superimposed calculation step, the driver or the worker himself / herself can move the obstacle appearing within the predicted travel trajectory A of the own vehicle to the expected travel trajectory A of the own vehicle. The device (specifically, the trajectory estimation process, the feature point extraction process, and the closest feature point extraction process) automatically determines which objects appearing in the vehicle can be dangerous obstacles to the running of the vehicle. And the marking calculation process displays the markings, eliminating the need to monitor and judge while the own vehicle is running while operating the own vehicle, reducing the burden on the driver and workers in monitoring work It is also possible to easily concentrate on the safe operation of the vehicle.

[0138]

According to the first aspect of the present invention, information (risk degree information) on which object can be a dangerous obstacle to the traveling of the host vehicle is provided to the driver or the worker. And the warning can be displayed.

[0139] As a result, the obstacles appearing within the predicted travel trajectory range of the own vehicle can be determined on the device side as to which objects appearing within the predicted travel trajectory of the own vehicle can be dangerous obstacles to the travel of the own vehicle. Automatically estimates and highlights the data, so there is no need for the driver or worker to monitor and determine while operating the vehicle while operating the vehicle. This also has the effect of reducing the burden of monitoring work and making it easier to concentrate on safe operation of the vehicle.

According to the invention described in claim 2, according to claim 1
In addition to the effects described in (1), it is estimated that information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the own vehicle exists within a predetermined distance range with respect to the traveling direction. It is possible to display an emphasized image of an obstacle to be emphasized and warn a driver or an operator.

As a result, the obstacles appearing in the predicted travel trajectory range of the own vehicle can be determined on the device side as to which objects appearing in the predicted travel trajectory range of the own vehicle can be dangerous obstacles for the travel of the own vehicle. Automatically estimates and highlights the data, so there is no need for the driver or worker to monitor and determine while operating the vehicle while operating the vehicle. This also has the effect of reducing the burden of monitoring work and making it easier to concentrate on safe operation of the vehicle.

According to the third aspect of the present invention, a first aspect is provided.
Or, in addition to the effect described in 2, the information (risk degree information) on which object can be a dangerous obstacle to the traveling of the own vehicle exists at a position closest to the vehicle in the traveling direction. Then, it is possible to display an emphasized image of an estimated obstacle to be emphasized and warn a driver or an operator.

As a result, the obstacles appearing in the expected travel range of the own vehicle can be determined by the device as to which objects appearing in the expected travel range of the own vehicle can be dangerous obstacles for the travel of the own vehicle. Automatically estimates and highlights the data, so there is no need for the driver or worker to monitor and determine while operating the vehicle while operating the vehicle. This also has the effect of reducing the burden of monitoring work and making it easier to concentrate on safe operation of the vehicle.

According to the invention set forth in claim 4, according to claim 1
In addition to the effects described in any one of (1) to (3), information (risk degree information) on which object can be a dangerous obstacle to the traveling of the own vehicle is transmitted to the vehicle in the traveling direction. The effect that it becomes possible to warn a driver or an operator by using an enhanced image display and a marking display executed on an obstacle to be emphasized which is presumed to be located near Play.

As a result, the obstacle which appears in the expected travel range of the own vehicle can be determined by the device as to which objects appearing in the expected travel range of the own vehicle can be dangerous obstacles for the travel of the own vehicle. It automatically estimates and displays markings, so there is no need for the driver or worker to monitor and determine while the vehicle is running while operating the vehicle. This also has the effect of reducing the burden of monitoring work and making it easier to concentrate on safe operation of the vehicle.

According to the invention set forth in claim 5, according to claim 4,
In addition to the effects described in (1), it is estimated that information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the own vehicle exists at a position closest to the vehicle in the traveling direction. There is an effect that the marking can be displayed in synchronization with the movement of the emphasized image display of the obstacle to be emphasized to warn the driver or the operator.

[0147] As a result, the apparatus determines whether an object that has appeared within the predicted range of the own-vehicle traveling trajectory can be a dangerous obstacle for the traveling of the own vehicle. It automatically estimates and displays markings in synchronization with the movement of obstacles, so it is necessary for the driver or worker to monitor and determine while operating the own vehicle while operating the own vehicle. As a result, it is possible to reduce the burden of the monitoring work of the driver and the operator, and to easily concentrate on the safe operation of the own vehicle.

According to the invention described in claim 6, according to claim 4,
Or in addition to the effect described in 5, the information (risk degree information) on which object can be a dangerous obstacle to the traveling of the own vehicle exists at a position closest to the vehicle in the traveling direction. Then, the enhanced image display and the marking display of the estimated obstacle to be emphasized are superimposed and displayed as a superimposed image combined with the driver or the operator to be able to warn. To play.

Furthermore, as a result of this superimposed display, any obstacles appearing in the expected range of the own vehicle travel trajectory may cause an obstacle appearing in the expected range of the own vehicle travel to be dangerous to the running of the own vehicle. The device automatically estimates whether or not it can become an obstacle, and displays the markings, so that the driver or the worker himself / herself can monitor and determine while operating the own vehicle while operating the own vehicle. This eliminates the necessity, reduces the burden on the driver and the operator for monitoring work, and also makes it easier to concentrate on the safe operation of the vehicle.

According to the invention of claim 7, according to claim 6,
In addition to the effects described in the above, it is estimated that the risk information estimated around the traveling locus estimating means, the feature point extracting means, and the closest feature point extracting means is present at a position closest to the vehicle with respect to the traveling direction. The emphasis image display of the obstacle to be emphasized and the marking display executed mainly by the marking calculation means are provided to the driver and the operator as a superimposed display executed mainly by the superimposition operation means. This gives an effect that a warning can be issued.

Further, as a result of the superimposed display being enabled by the superimposed calculation means, an obstacle appearing within the predicted travel trajectory range of the own vehicle is replaced with an object appearing within the predicted travel trajectory range of the own vehicle. The device side (specifically, the running locus estimating means, the feature point extracting means, and the closest feature point extracting means) automatically estimates whether or not the obstacle may be a dangerous obstacle to traveling, and the marking calculating means performs the marking. Because it displays, it is not necessary for the driver or operator to monitor and judge while the own vehicle is running while operating the own vehicle, and the burden of monitoring work for the driver and operator can be reduced, Another effect is that the user can easily concentrate on the safe operation of the vehicle.

According to the eighth aspect of the present invention, information (risk degree information) on which object can be a dangerous obstacle to the traveling of the vehicle is provided to the driver or the worker. This makes it possible to display and warn the user.

As a result, the obstacle which appears in the range of the expected running trajectory of the own vehicle can be determined on the device side as to which objects appearing in the range of the expected running trajectory of the own vehicle can be dangerous obstacles for the running of the own vehicle. Automatically estimates and highlights the data, so there is no need for the driver or worker to monitor and determine while operating the vehicle while operating the vehicle. This also has the effect of reducing the burden of monitoring work and making it easier to concentrate on safe operation of the vehicle.

According to the ninth aspect of the present invention, information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the host vehicle is provided in a predetermined manner with respect to the traveling direction. There is an effect that an emphasis image is displayed for an obstacle to be emphasized which is presumed to be present within the distance range to warn a driver or an operator.

As a result, an obstacle which appears in the expected travel range of the own vehicle can be determined on the device side as to which objects appearing in the expected travel range of the own vehicle can be dangerous obstacles for the travel of the own vehicle. Automatically estimates and highlights the data, so there is no need for the driver or worker to monitor and determine while operating the vehicle while operating the vehicle. This also has the effect of reducing the burden of monitoring work and making it easier to concentrate on safe operation of the vehicle.

According to the tenth aspect of the present invention, information (risk degree information) as to which object can be a dangerous obstacle to the traveling of the own vehicle is provided in the vehicle in the traveling direction. This makes it possible to display an emphasized image on an obstacle to be emphasized, which is presumed to be present at a position close to the image, and warn a driver or an operator.

As a result, the obstacle which appears in the range of the expected running trajectory of the own vehicle can be determined by the device as to which objects appearing in the range of the expected running trajectory can be dangerous obstacles for the running of the own vehicle. Automatically estimates and highlights the data, so there is no need for the driver or worker to monitor and determine while operating the vehicle while operating the vehicle. This also has the effect of reducing the burden of monitoring work and making it easier to concentrate on safe operation of the vehicle.

According to the eleventh aspect of the invention, in addition to the effect of any one of the eighth to tenth aspects, any object can be a dangerous obstacle to the running of the host vehicle. Information (risk degree information) on the vehicle is driven by using an emphasized image display and a marking display executed on an obstacle to be emphasized, which is estimated to be located closest to the vehicle with respect to the traveling direction. There is an effect that a warning can be given to a worker or an operator.

As a result, the apparatus determines whether an object that has appeared in the range of the expected travel of the own vehicle can be a dangerous obstacle for the travel of the own vehicle. It automatically estimates and displays markings, so there is no need for the driver or worker to monitor and determine while the vehicle is running while operating the vehicle. This also has the effect of reducing the burden of monitoring work and making it easier to concentrate on safe operation of the vehicle.

According to the twelfth aspect of the present invention, in addition to the effects of the eleventh aspect, information on which object can be a dangerous obstacle to the traveling of the host vehicle (risk level) Information) in synchronization with the movement of the highlighted image display of the obstacle to be emphasized, which is presumed to be located closest to the vehicle with respect to the traveling direction, to warn the driver or operator. This has the effect of being able to perform.

As a result, the obstacles appearing in the predicted travel trajectory range of the own vehicle can be determined on the device side as to which objects appearing in the predicted travel trajectory of the own vehicle can be dangerous obstacles for the travel of the own vehicle. It automatically estimates and displays markings in synchronization with the movement of obstacles, so it is necessary for the driver or worker to monitor and determine while operating the own vehicle while operating the own vehicle. As a result, it is possible to reduce the burden of the monitoring work of the driver and the operator, and to easily concentrate on the safe operation of the own vehicle.

According to the thirteenth aspect of the present invention, in addition to the effect of the eleventh or twelfth aspect, information on which object can be a dangerous obstacle to the traveling of the host vehicle ( Risk information) is superimposed as a superimposed image obtained by superimposing and marking the highlighted image display and the marking display of the obstacle to be emphasized, which is estimated to be located closest to the vehicle with respect to the traveling direction. It is possible to display and warn an operator.

Further, as a result of this superimposed display, any obstacles appearing in the range of the expected trajectory of the own vehicle may cause an obstacle that appears in the range of the expected trajectory of the own vehicle. The device automatically estimates whether or not it can become an obstacle, and displays the markings, so that the driver or the worker himself / herself can monitor and determine while operating the own vehicle while operating the own vehicle. This eliminates the necessity, reduces the burden on the driver and the operator for monitoring work, and also makes it easier to concentrate on the safe operation of the vehicle.

According to the fourteenth aspect of the invention, in addition to the effect of the thirteenth aspect, there is a danger that is estimated centering on the traveling trajectory estimation step, the feature point extraction step, and the closest feature point extraction step. The degree information, the emphasized image display of the obstacle to be emphasized, which is estimated to be located closest to the vehicle with respect to the traveling direction, and the marking display executed mainly in the marking calculation step are superimposed calculation steps. There is an effect that it is possible to warn a driver or an operator as a superimposed display executed mainly.

Further, as a result of the superimposed display being possible in the superimposed calculation step, an obstacle appearing in the predicted traveling trajectory A of the vehicle can be replaced by an obstacle appearing in the predicted traveling trajectory A of the own vehicle. The device determines whether it can be a dangerous obstacle to the running of the vehicle (specifically, the travel locus estimation step, the feature point extraction step, and the closest feature point extraction step).
Automatically, and the marking calculation process displays the markings, so there is no need for the driver or operator to monitor and determine while operating the own vehicle while operating the own vehicle. This also has the effect of reducing the burden on the monitoring work of workers and workers, and making it easier to concentrate on the safe operation of the vehicle.

According to the fifteenth aspect, the same effect as the tenth aspect can be obtained.

According to the sixteenth aspect, the same effect as that of the eleventh aspect can be obtained.

According to the seventeenth aspect, the same effect as the twelfth aspect can be obtained.

According to the eighteenth aspect, the same effect as that of the thirteenth aspect can be obtained.

According to the nineteenth aspect, the same effect as that of the fourteenth aspect can be obtained.

[Brief description of the drawings]

FIG. 1 is a functional block diagram for explaining a vehicle periphery monitoring device according to the present invention.

FIG. 2 is an explanatory diagram of various display forms for explaining an alarm display operation used in the vehicle periphery monitoring device of FIG. 1;
FIG. 2A shows a display form of the raw image information displayed on the display screen of the display means, and FIG. 2B shows the display mode of the own vehicle with respect to the raw image information within the expected travel range of the own vehicle. FIG. 2C is a diagram for explaining a display form in which a predicted travel trajectory range is displayed in a superimposed manner, and FIG. 2C shows a superimposed display of marking information in addition to the superimposed display in FIG. FIG. 2D is a view for explaining a display form. FIG. 2D shows the marking information in the superimposed display form in which the own vehicle travel expected trajectory range is compared with the raw image information outside the own vehicle travel expected trajectory range. FIG. 2 (e) is a diagram for explaining a display mode in which is displayed in a superimposed manner. FIG. 2F is a diagram for explaining a display mode in which marking information for raw image information from which the closest feature points are extracted is superimposed and displayed in the display mode in which the raw image is extracted. In the display mode in which the own vehicle travel expected trajectory range is superimposed with respect to the information, the marking information is superimposed on the raw image information from which the closest feature points outside the own vehicle travel expected trajectory range are extracted. FIG. 13 is a diagram for explaining a display mode in which marking information is superimposed and displayed on the raw image information from which the closest feature point is extracted within the range of the estimated travel trajectory of the host vehicle without being displayed.

FIG. 3 is a flowchart illustrating a warning display method used in the vehicle periphery monitoring device of FIG. 1;

FIG. 4 is an operation explanatory diagram for explaining an alarm display method used in the vehicle periphery monitoring device of FIG. 1, and FIG.
FIG. 4 (b) is a view for explaining a process of extracting feature points, and FIG. 4 (c) is a view for explaining a process of extracting feature points. FIG. 4D is a diagram for explaining a point extraction process and a feature point group extraction process, and FIG. 4D is a diagram for explaining a coordinate point extraction process for the feature point group, and FIG. FIG. 8 is a diagram for explaining a process of generating marking information for an extracted coordinate point.

FIG. 5 is a functional block diagram for explaining a conventional vehicle periphery monitoring device and an alarm display method used in the device.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 10 Vehicle periphery monitoring apparatus 12 Image reading means (CCD camera) 12a Raw image information 14 Feature point extracting means (microcomputer) 16 Closest feature point extracting means (microcomputer) 18 Feature point group extracting means (microcomputer) 20 Coordinate extracting means (Microcomputer) 22 Marking calculation means (microcomputer) 22a Marking information 24 Superimposition calculation means (microcomputer) 24a Superimposed image information 26 Display means (display) 262 Display screen 30 Traveling path estimating means 30a Own vehicle running prediction Trajectory information P1, ..., Pn feature point information Pmin nearest feature point information {P} feature point group information {a1, a2} coordinate information L1, ..., Lm scanning line

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Takeyuki Amari 1500 Onjuku 1500, Susono City, Shizuoka Prefecture Inside Yazaki Corporation

Claims (19)

[Claims] 1. A vehicle periphery monitoring device that is installed in a vehicle and captures an image of a monitoring area around the vehicle and monitors the periphery of the vehicle based on the captured image, wherein the vehicle travels in response to an operation of the vehicle. When it is estimated that at least one or more obstacles are present on the basis of the captured image within the estimated range of the own vehicle traveling, which is the estimated range of the trajectory to perform, any of the specified obstacles is selected from the obstacles. A vehicle periphery monitoring device configured to highlight and display the image of the vehicle. 2. When it is estimated that at least one or more obstacles are present within the range of the predicted travel trajectory of the host vehicle based on the captured image, a predetermined value for the traveling vehicle is selected from the obstacles. The vehicle surroundings monitoring device according to claim 1, wherein an image of an obstacle estimated to be present within the distance range is configured to be emphasized and displayed. 3. When it is estimated that at least one or more obstacles are present in the range of the estimated trajectory of the host vehicle based on the captured image, the vehicle in the traveling direction is selected from the obstacles. 3. The vehicle periphery monitoring device according to claim 1, wherein an image of an obstacle estimated to be present at a position close to the vehicle is displayed in an emphasized manner. 4. 4. The image emphasis display is executed together with a marking display for displaying a mark of a predetermined shape while maintaining a predetermined positional relationship with a display coordinate point of the obstacle to be emphasized. The vehicle periphery monitoring device according to any one of claims 1 to 3, wherein: 5. The marking display retains a predetermined positional relationship with a display coordinate point of the obstacle to be emphasized and synchronizes with the movement of the obstacle while maintaining the predetermined positional relationship. The vehicle periphery monitoring device according to claim 4, wherein the device is configured to move a mark. 6. The image emphasis display is configured to be displayed as a superimposed image in which the marking display and the image display of the obstacle to be emphasized are superimposed and synthesized. The vehicle periphery monitoring device according to claim 4 or 5, wherein 7. An image reading means which is installed in a vehicle and captures a stereoscopic image of the monitoring area around the vehicle to generate raw image information having three-dimensional coordinate information. Traveling trajectory estimating means for generating vehicle traveling predicted trajectory information; and at least one vehicle traveling expected trajectory range based on the own vehicle traveling predicted trajectory information and the raw image information.
When it is estimated that there are two or more obstacles, a scan line of a predetermined resolution is given to the raw image information to detect an intersection between the scan line and the obstacle image, and an obstacle is determined based on the intersection. Feature point extraction means for generating feature point information according to the three-dimensional coordinate information and the feature point information included in the raw image information, from among the obstacles, the vehicle in the traveling direction of the vehicle most Estimating an obstacle present at a close position, and selecting a close feature point of the obstacle estimated to be at the close position from the feature point information as the close feature point information, Using the three-dimensional coordinate information, the feature point information, and the closest feature point information of the raw image information, a three-dimensional space range having a predetermined distance around the closest feature point is provided. Included A feature point group extraction means for selecting from among the feature point information as the feature point group information feature point group is a set of feature points having a three-dimensional coordinate, together with obtaining a rectangle that encloses all of the feature points,
Coordinate extraction means for generating the coordinates of the quadrangle as coordinate information, and generating, as the marking information, the coordinates for displaying the quadrangle as the mark on the obstacle image using the coordinate information and the feature point information as the mark. Marking operation means for performing superimposition operation means for generating superimposed image information which is the superimposed image obtained by superimposing the marking information on the raw image information; and Display for executing the image emphasis display by superimposing and displaying the image of the obstacle presumed to be present at the closest position with respect to the traveling direction from among the obstacles using the superimposed image information. The vehicle surroundings monitoring device according to claim 6, comprising: 8. An alarm display method used in the vehicle periphery monitoring device, comprising: a step of estimating whether or not the obstacle is present within the range of the expected vehicle travel trajectory of the vehicle in accordance with an operation of the vehicle. When it is estimated that at least one or more obstacles are present based on the captured image, from among the obstacles,
The warning display method used in the vehicle periphery monitoring device according to any one of claims 1 to 7, further comprising a step of performing the image highlighting of an arbitrary designated obstacle. 9. A step of estimating whether or not the obstacle is present within the range of the expected traveling trajectory of the vehicle according to the operation of the vehicle, and at least one of the steps of:
When it is estimated that one or more obstacles are present, among the obstacles, comprises a step of performing the image highlighting of the obstacles that are estimated to exist within a predetermined distance range for the traveling vehicle, The alarm display method according to claim 8, wherein: 10. A step of estimating whether or not the obstacle is present within the range of the expected traveling trajectory of the vehicle according to the operation of the vehicle, and at least one or more obstacles based on the captured image. When it is presumed that an obstacle is present, the method further includes a step of executing the image highlighting of the obstacle that is presumed to be present at the closest position with respect to the traveling direction from the obstacles. Item 10. The alarm display method according to item 8 or 9. 11. The method according to claim 1, further comprising the step of performing the image highlighting together with the marking display for displaying the mark while maintaining a predetermined positional relationship with a display coordinate point of the obstacle to be emphasized. The alarm display method according to any one of claims 8 to 10, wherein 12. The marking display holds a predetermined positional relationship with a display coordinate point of the obstacle to be emphasized, and holds the predetermined positional relationship in synchronization with the movement of the obstacle. The alarm display method according to claim 11, further comprising a step of moving a mark. 13. The image emphasis display having a step of superimposing the marking display and the image display of the obstacle to be emphasized and displaying the superimposed image as a superimposed image. Item 13. The alarm display method according to item 11 or 12. 14. An image reading step comprising: a step of capturing a stereoscopic image of the monitoring area in the vicinity of the vehicle installed on the vehicle; and a step of generating the raw image information having the three-dimensional coordinate information. A traveling locus estimating step of generating vehicle traveling expected locus information; and a step of estimating the presence of the obstacle in the own vehicle traveling expected locus range based on the own vehicle traveling expected locus information and the raw image information. When it is estimated that at least one or more obstacles are present, a step of giving a scan line of a predetermined resolution to the raw image information to detect an intersection of the scan line and the obstacle image, A feature point extraction step having a step of generating the feature point information based on the three-dimensional coordinate information and the feature point information. A step of estimating an obstacle present at a position close to both, and a step of selecting the close feature point from the feature point information as the close feature point information; and a step of extracting the raw image information. Using the three-dimensional coordinate information, the feature point information, and the close feature point information to provide the three-dimensional space range around the closest feature point; and the feature included in the three-dimensional space range Selecting a point group from among the feature point information as the feature point group information; a step of obtaining the rectangle that entirely surrounds the feature point group; and A coordinate extracting step having a step of generating the coordinate information as the coordinate information; and a step of generating the marking information using the coordinate information and the feature point information, using the rectangle as the mark in the obstacle image. A superposition calculation step having a step of generating the superimposition image information by superimposing the marking information on the raw image information; and an obstacle within the expected travel range of the host vehicle. A display step of executing the image highlighting by displaying the image of the obstacle estimated to be present at the close position with respect to the traveling direction using the superimposed image information. The alarm display method according to claim 13, comprising: 15. A computer-readable medium storing the warning display method, the method comprising: determining whether the obstacle is present in the range of the expected traveling trajectory of the vehicle according to an operation of the vehicle. Estimating, and when it is estimated that at least one or more obstacles are present based on the captured image, from among the obstacles,
The medium storing the alarm display method according to claim 10, further comprising a step of executing the image highlight display of an obstacle estimated to be present at the close position with respect to a traveling direction. 16. A computer-readable medium storing the alarm display method, wherein the image highlighting is performed by holding a predetermined positional relationship with a display coordinate point of the obstacle to be emphasized. 12. A medium storing the alarm display method according to claim 11, further comprising a step executed together with the marking display for displaying the warning. 17. A computer-readable medium storing the alarm display method, wherein the marking display has a predetermined positional relationship with a display coordinate point of the obstacle to be emphasized, and The medium storing the alarm display method according to claim 12, further comprising a step of moving the mark in synchronization with the movement of the obstacle while maintaining the positional relationship. 18. A computer-readable medium storing the alarm display method, wherein the image emphasis display is superimposed by superimposing the marking display and the image display of the obstacle to be emphasized. Including a step of being displayed as an imposed image,
14. A medium storing the alarm display method according to claim 13. 19. A computer-readable medium storing the alarm display method, comprising: a step of capturing a three-dimensional image of the monitoring area around a vehicle, which is installed in a vehicle, and comprising the three-dimensional coordinate information. An image reading step having a step of generating raw image information; a traveling path estimating step of generating the own vehicle traveling predicted path information; and the own vehicle based on the own vehicle traveling predicted path information and the raw image information. Estimating the presence of the obstacle within the range of the expected traveling trajectory; Detecting a point of intersection with the obstacle image, generating the characteristic point information based on the point of intersection, and extracting the characteristic point information. Estimating an obstacle located closest to the vehicle with respect to the traveling direction of the vehicle from the obstacles using the feature point information; and A closest feature point extracting step having a step of selecting from among information; and using the three-dimensional coordinate information, the feature point information, and the closest feature point information, which are included in the raw image information, with a focus on the closest feature point. Providing the three-dimensional space range; and selecting the feature point group included in the three-dimensional space range from the feature point information as the feature point group information, A step of obtaining the rectangle so as to surround the whole feature point group; a step of generating coordinates of the rectangle as the coordinate information; a coordinate extraction step; using the coordinate information and the feature point information, Stuff Superimposing, comprising: a marking calculation step having a step of generating the marking information using the rectangle as the mark in an image; and a step of generating the superimposed image information by superimposing the marking information on the raw image information. Calculating the obstacle image estimated to be present at the closest position with respect to the traveling direction from the obstacles within the range of the host vehicle traveling expected trajectory by using the superimposed image information to obtain the superimposed image. 15. A medium storing the alarm display method according to claim 14, further comprising a display step of executing the image highlighting by performing a pause display.