DE102019213403A1 - Method for the sensor-based localization of a host vehicle, host vehicle and a computer program - Google Patents

Abstract

The invention relates to a method for the sensor-based localization of a host vehicle (1) with the following steps: - generation of successive frames of environment sensor data of an environment of the host vehicle, - extraction of features in at least one frame, - combining the features into individual dynamic objects of interest, - Classification of the individual dynamic objects of interest, - Determination of kinematic data of the individual dynamic objects of interest and assignment of the kinematic data to the individual dynamic objects of interest, - Transformation of the recorded environment sensor data of the dynamic objects of interest of the individual frames into a local map ( 11), - clustering of the transformed environment sensor data of the dynamic objects of interest in the local map (11) into trajectories and assigning the trajectories to the individual dynamic objects of interest, - determination of static environment data of static environment objects from at least one of the frames and complete the local map (11) using the static environment data to form a completed local environment map (12) and assign the individual dynamic objects of interest to the static environment data, - provide a reference road map and map the completed local environment map (12) The invention also relates to a host vehicle, a computer program and a data carrier signal.

Description

The invention relates to a method for the sensor-based localization of a host vehicle. The invention also relates to a host vehicle with such a method, a computer program and a data carrier signal.

In vehicle navigation systems, the position of the vehicle is usually measured with the support of the GPS (Global Positioning System) satellite positioning system. The accuracy of the measured and the actual position is specified with GPS at 15 meters. However, due to the imprecise nature of GPS coordinates, it can be difficult to determine an accurate vehicle location on a digital map on a high-density, multi-lane road.

For the navigation device, for example, the position of the vehicle must now be determined on the digital map so that, for example, a meaningful route calculation can be made from the current location to the destination. Without a comparison of the measured position with the map information, the vehicle could be outside the digitized streets or on the wrong street on the map. Such a comparison is made more difficult by the high traffic density.

The DE 10 2016 205 871 A1 discloses in a method for localizing a vehicle with a localization module for carrying out at least one localization method for localizing the vehicle. In this case, a landmark density value, which represents a density of landmarks in the area of a route section assigned to the vehicle, is first read in. In a further step, using the landmark density value, a first part of a computing capacity of the localization module is allocated to a first localization method. In addition or as an alternative, a second part of the computing capacity is allocated to a second localization method that differs from the first localization method. Finally, the first localization method using the first part or the second localization method using the second part is carried out in order to localize the vehicle.

The invention is now based on the object of specifying means for localizing an ego vehicle when there is a very high volume of traffic, for example when there is a traffic jam / rush hour in a busy city.

This object is achieved by a method for the sensor-based localization of an object with the features of claim 1 and an ego vehicle with the features of claim 13. Furthermore, the object is achieved by a computer program with the features of claim 14 and a data carrier signal with the features of claim 15.

The subclaims list further advantageous measures which can be combined with one another in any suitable manner in order to achieve further advantages.

• - Generierung von aufeinanderfolgenden Frames von Umfeldsensordaten eines Umfelds des Egofahrzeugs durch ein Sensorsystem,
• - Extraktion von Merkmalen in zumindest einem Frame durch ein Merkmalsextraktionsverfahren,
• - Zusammenfassen der Merkmale zu einzelnen dynamischen Objekten von Interesse,
• - Klassifikation der einzelnen dynamischen Objekte von Interesse,
• - Bestimmung von kinematischen Daten der einzelnen dynamischen Objekte von Interesse aus den aufeinanderfolgenden Frames unter Berücksichtigung der Egodaten des Egofahrzeugs und zuordnen der kinematischen Daten zu den einzelnen dynamischen Objekten von Interesse,
• - Transformieren der erfassten Umfeldsensordaten der dynamischen Objekte von Interesse der einzelnen Frames unter Verwendung der kinematischen Daten in eine lokale Karte,
• - Clustern der transformierten Umfeldsensordaten der dynamischen Objekte von Interesse in der lokalen Karte zu Trajektorien und zuweisen der Trajektorien zu den einzelnen dynamischen Objekten von Interesse,
• - Bestimmen von statischen Umgebungsdaten von statischen Umgebungsobjekten aus zumindest einem der Frames und vervollständigen der lokalen Karte anhand der statischen Umgebungsdaten von statischen Umgebungsobjekten zu einer vervollständigten lokalen Umgebungskarte und zuweisen der einzelnen dynamischen Objekte von Interesse zu den statischen Umgebungsdaten von statischen Umgebungsobjekten,
• - Bereitstellen einer Referenzstraßenkarte und abbilden der vervollständigten lokalen Umgebungskarte auf die Referenzstraßenkarte.

The task is solved by a method for the sensor-based localization of a host vehicle with the following steps:

• - Generation of successive frames of environment sensor data of an environment of the host vehicle by a sensor system,
• - Extraction of features in at least one frame by a feature extraction method,
• - Combining the characteristics into individual dynamic objects of interest,
• - classification of the individual dynamic objects of interest,
• - Determination of kinematic data of the individual dynamic objects of interest from the successive frames, taking into account the ego data of the host vehicle and assigning the kinematic data to the individual dynamic objects of interest,
• - Transforming the captured environment sensor data of the dynamic objects of interest of the individual frames using the kinematic data into a local map,
• - Cluster the transformed environment sensor data of the dynamic objects of interest in the local map to form trajectories and assign the trajectories to the individual dynamic objects of interest,
• - Determination of static environment data of static environment objects from at least one of the frames and complete the local map on the basis of the static environment data of static environment objects to a completed local environment map and assign the individual dynamic objects of interest to the static environment data of static environment objects,
• - Providing a reference road map and mapping the completed local area map onto the reference road map.

A frame is to be understood as meaning summarized environment sensor data which are recorded at a specific point in time.

Features are mainly L- or I-shaped, bounding boxes, clusters with reference points, sensor data segments. The features to be extracted can be specified.

The classification method is used to divide the dynamic objects into classes such as cars, pedestrians, etc., and also advantageously to divide the statistical objects recorded on the basis of the static environment data into classes such as curb stones, street signs, etc.

First and foremost, ego data can be the direction of movement (orientation and position) and / or the speed of the ego vehicle. With the help of the ego data, the environment sensor data can be better interpreted, since the environment sensors installed in the ego vehicle move with the vehicle so that their location and position change constantly. For the temporal correlation of the environment sensor data, knowledge of the path covered by the host vehicle and the position is advantageous. The ego data can also be determined by an ego motion estimation, which estimates the accelerations and the rate of rotation of the ego vehicle, for example by means of an inertial navigation system.

The cluster method is a method for determining similarity structures in databases. The groups of data found in this way are known as clusters.

A local map can, for example, be created by the host vehicle itself. A local map is preferably based on the trajectories (temporally referenced object with all its properties such as its position, class, geometry). In addition, the static environment is also merged into a local map. These include the road boundaries by road markings, the curb, traffic signs, etc.

Static environment data are preferably in the form of semantic information or geometric information, which complete the local map with, for example, static information such as the lane (number, position, width), curb, lane boundary, etc. to form an environment map.

If one or more sensors, in particular a front sensor, are blocked by other road users and, for example, lanes cannot be recognized, it is possible to use the determined trajectories of the dynamic objects of interest for precise localization of the host vehicle. With the method according to the invention, if the sensor (s) are blocked by other road users and, for example, the lanes / road boundaries cannot be recognized, the trajectories of the dynamic objects of interest can be used for precise localization of the host vehicle (in a reference road map) .

The host vehicle can thus be localized in a given reference road map using the method according to the invention. The localization can be based on the trajectories of the identified dynamic objects of interest. The localization on the basis of dynamically tracked objects of interest is necessary if the field of view of the sensor or sensors, in particular the front field of view, is obscured by many road users in busy urban areas or traffic jams on the motorway.

The mapping is preferably carried out using a map matching algorithm. Map matching is a method that compares the position of an object measured by localization with the location information on a digital map. The alignment of the completed local environment map with the reference map can be based on a transformation using key data points, i.e. features or objects that can be uniquely matched between two given sets and to a minimization of an error function for an association of the two given sets of features / Objects. In addition, a rough estimate of the pose based on GPS data (Global Positioning System) or an improved estimation of the movement of the host vehicle can be used as a starting point for the optimization problem. The matching can take place by means of an optimization method, for example the nearest neighbor algorithm, using a distance function such as the square Euclidean distance. An ICP (Iterative Closest Point) algorithm or another optimization method can also be used.

The reference road map is preferably designed as an HDMap (high definition map). These maps are extremely precise and show the surroundings with an accuracy of up to ten to 20 centimeters. HDMap can also show an image of the road surface, the number of lanes and their width. In addition, these can show the inclination and curvature of the lanes as well as lane markings and static objects on the roadside, such as street signs, trees, curbs, etc.

The HDMap preferably comprises at least static infrastructure data. These can, for example, be a road model, traffic signs, Traffic lights, road markings, lane model (left, right lane) as well as geometric information such as road courses, height profile, curvature, width as well as right of way, zebra crossings, speed limits and structures such as traffic lights, traffic signs, tunnels, houses, etc.

The reference map is preferably designed as a self-learning road map. This is updated continuously, so that the current traffic situation and / or the environment can be mapped in real time.

The static environmental data of static environmental objects are preferably designed as lanes and / or lane boundaries. Furthermore, the static environmental data of static environmental objects are preferably designed as active traffic display data such as traffic lights and / or passive traffic display data such as signs. Furthermore, the static environment data of static environment objects are preferably designed as environment features. Such distinctive objects / points / features can be used to achieve a transformation more quickly.

In a preferred embodiment, a density-based method is used as the cluster method. DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used particularly advantageously as the cluster method. The DBSCAN works based on density and is able to recognize several clusters. Furthermore, it does not have to be known in advance how many clusters exist. The algorithm can also recognize any cluster of any shape. Another advantage of using the DBSCAN is that the algorithm works largely deterministically and independently of the sequence. Other algorithms based on DBSAN can also be used. This is, for example, the HDBSCAN - Alogithm (Hierarchical Density Based Clustering), which extends the algorithm by converting it into a hierarchical clustering algorithm.

A reference point is preferably determined for each dynamic object of interest.

To determine kinematic data of the individual dynamic objects of interest from the successive frames, it is useful to determine the position, size and orientation of the dynamic object of interest. The determination of the position is preferably calculated from the individual distance measurements of the object using a suitable reference point. This is especially important for objects that change their orientation to the sensor system.

Furthermore, kinematic data of the individual dynamic objects of interest are preferably determined from the successive frames on the basis of the reference point. This enables the process to be accelerated.

Furthermore, the object is achieved by a host vehicle which is designed to carry out the method as described above, comprising at least one lidar sensor, one radar sensor and one image sensor. The lidar / radar sensors capture the environment more and more dimensionally, i.e. not only are detections delivered in 3D space, but also the radar cross section (RCS), the Doppler speed and other signals (SNR, power, etc.) for the radar sensor. A lidar sensor typically also supplies an echo pulse width and an intensity.

Furthermore, the object is achieved by a computer program comprising commands which cause the host vehicle as described above to carry out the method as described above.

The object is also achieved by a data carrier signal that is transmitted by a computer program as described above. This also enables the method to be easily subsequently installed in suitable host vehicles.

• 1: ein Egofahrzeug mit einem erfindungsgemäßen Fahrerassistenzsystem,
• 2: einen Grobüberblick über das erfindungsgemäße Verfahren,
• 3: den ersten Verfahrensschritt S1 im Detail,
• 4: die zu verfolgenden Objekte, in Klassen eingeteilt,
• 5: den zweiten Verfahrensschritt S2 im Detail,
• 6: die dynamischen klassifizierten Objekte von Interesse und deren Trajektorien,
• 7: den dritten Verfahrensschritt S3 im Detail,
• 8: eine lokale Umgebungskarte,
• 9: die lokale Umgebungskarte in Draufsicht zu einem späteren Zeitpunkt,
• 10: den vierten Verfahrensschritt S4 im Detail ,
• 11: teilweise das erfindungsgemäße Verfahren grafisch.

Further features, properties and advantages of the present invention emerge from the following description with reference to the accompanying figures. They show schematically:

• 1 : an ego vehicle with a driver assistance system according to the invention,
• 2 : a rough overview of the method according to the invention,
• 3rd : the first process step S1 in detail,
• 4th : the objects to be tracked, divided into classes,
• 5 : the second process step S2 in detail,
• 6th : the dynamic classified objects of interest and their trajectories,
• 7th : the third process step S3 in detail,
• 8th : a local map of the area,
• 9 : the local area map in plan view at a later point in time,
• 10 : the fourth method step S4 in detail,
• 11 : partially the method according to the invention graphically.

Although the invention has been illustrated and described in more detail by the preferred exemplary embodiments, the invention is not restricted by the examples disclosed.

1 shows a host vehicle 1 with a driver assistance system 2 , which has a method according to the invention. The ego vehicle 1 exhibits a sensor system 3rd on which, for example, radar sensors 4th , Lidar sensors 5 and image sensors 6th includes. The environment of the host vehicle 1 is based on the sensor system 3rd recorded as environment sensor data at a point in time and then merged into a common frame. There are different options for the sensor data fusion. For example, the recorded environment sensor data can be merged at signal level. The raw data or minimally preprocessed data are linked to one another. Another fusion can be, for example, a fusion at the feature level or at the object level. The merged environment sensor data are preferably used to form the driver assistance system 2 forwarded.

2 shows a rough overview of the method according to the invention. First of all, by means of the sensor system 3rd several frames at successive points in time from the environment of the host vehicle 1 created. Various features are then extracted from at least one of the frames in a first method step S1 by means of a feature extraction method. Known feature extraction methods can be used for this. These features can be, for example, an L-shape or an I-shape, or a bounding box, etc. of related environment sensor data. The extracted features can accordingly be grouped together as coherent dynamic objects of interest. The aim of the feature extraction is to provide an object description of objects of interest that is abstracted from the environment sensor data. In order to be able to provide position information for extended objects, a reference point that represents the object is also determined. This is determined in a respective sensor data segment, which represents an object of interest.

Then the dynamic objects of interest are classified. Vehicle, bicycle, motorcycle, for example, can be defined as a class. The main purpose of the classification at the feature level is to filter out unwanted, incorrectly identified objects at an early stage and to delete them. On the one hand, this saves computing time and, on the other hand, increases the robustness of the method, since fewer irrelevant objects are created. The Bayes classifier, Gaussian mixture models, neural networks, support vector machines or decision trees, for example, can be used as classification methods. Decision trees are ordered, directed trees that are used to represent decision rules. Other classification methods are also possible.

Furthermore, the objects of interest are assigned certain movement assumptions related to the kinematic state of the object of interest, for example that it moves at an approximately constant speed in the direction of its longitudinal axis and at what speed it continues to move. This assumption or estimate is preferably made using at least the ego speed of the host vehicle 1 and the ego position of the ego vehicle 1 certainly. The kinematic state, that is, for example, movement / position and direction of movement, is now assigned to the individual dynamic objects of interest.

In a second method step S2, the trajectory of the individual dynamic objects of interest is determined and assigned to each of them. For this purpose, the environment sensor data of the dynamic objects of interest are shown over various successive frames in a local map 11 ( 6th ) accumulated. There is also a local map 11 ( 6th ) a map of the surroundings of the host vehicle 1 , which are preferably from the host vehicle 1 ( 6th ) is created by yourself. For the sake of simplicity, the local map is 11 ( 6th ) for example the first frame at the beginning of the procedure. As the method progresses, a subsequent frame or a reference map, as will be described below, can be used. The accumulation of the dynamic objects is determined by the host vehicle with the aid of a first prediction / estimate of the pose, that is to say the position and orientation of the dynamic objects of interest 1 , performed. The final estimate can be made, for example, by a Kalman filter. Are the environment sensor data in the local map 11 ( 6th ) are accumulated, they are clustered using a clustering process. A clustering method searches for groups of tracks in the context of the accumulated surroundings sensor data. From this, the trajectories can be determined which can be assigned to the individual dynamic objects of interest.

This makes a local map 11 ( 6th ) which contain the classified, dynamic objects and their trajectories. A verification of the dynamic objects of interest can also be achieved by the cluster method. If an identified object appears implausible due to the clustering process, it can be deleted. Other plausibility procedures can also be used for deletion. By deleting of objects, the robustness and the speed of the process are increased. There may also be other reasons for deletion, which are described below.

Another reason for deletion is, for example, the downgrading of the dynamic object of interest as uninteresting. Due to the limited field of view and the movement of the objects, they regularly leave the relevant area and thus become uninteresting.

Furthermore, objects can be deleted when the lifetime of the dynamic object of interest exceeds a predetermined threshold time value.

In addition, incorrect objects can preferably also be identified and deleted. Due to random shapes in the environment sensor data or due to misinterpretations, it can happen again and again that objects are created that do not exist in reality. This is recognized, for example, as described above, by the cluster method or the classification. For example, a plausibility check can be used to check whether the measured dimensions of the individual objects of interest correspond to the expected dimensions of road users. If this is not the case, the object is preferably deleted.

The deletion of objects that are no longer of interest contributes to the speed of the method when tracking several, changing objects.

The local map is then used in a further method step S3 11 ( 6th ) completed with background information. The background information is designed as preferably classified, static environmental data from static environmental objects such as road boundaries, for example curb stones, road markings, individual road lanes. Furthermore, static environmental data from static environmental objects such as active traffic display data such as traffic lights and / or passive traffic display data such as signs can be detected and the local map 11 ( 6th ) to be added.

This enables a completed local area map 12th ( 8th ) can be achieved by completing the traffic scene and thus a complete description of the scene. If the streets to be recognized have no marking, the texture is evaluated in addition to edges and the street color, for example. Such a complete scene description and scene detection can be achieved, for example, by means of a bottom-up method.

This results in a consistent, completed local area map 12th ( 8th ) that shows the current traffic scene with lanes 13th ( 8th ), Road boundaries etc. and the dynamic objects and their trajectories.

In a subsequent method step S4, the completed local area map is created 12th ( 8th ) on a given reference road map, preferably an HDMap (High Definition Map) 14th ( 10 ) using a map matching algorithm. Using a map matching algorithm, the position of an object measured by a location is compared with the location information on a digital map. The illustration of the completed local area map 12th ( 8th ) on the HDMap 14th ( 10 ) can be based on a transformation using key data points, i.e. sensor data points that can be uniquely compared between two given quantities, or on the minimization of an error function. In addition, a rough estimate of the pose based on GPS data (Global Positioning System) or based on an improved estimation of the movement of the host vehicle can be made 1 ( 1 ) can be used as the starting point for the optimization problem.

This enables the local area map to be mapped more quickly 12th ( 8th ) on the reference road map.

The optimization method can be based on the nearest neighbor algorithm with, for example, an optimizing distance function such as the square Euclidean distance between recognized key data points on the local map and the corresponding key data points on the reference road map. An ICP (Itertiv Closest Point) algorithm or another optimization method can also be used.

The method according to the invention can be used to localize the host vehicle 1 take place in a given reference road map, the localization being based on the trajectories of the detected dynamic objects and the static environment data.

The localization on the basis of dynamically tracked objects is necessary if the field of view of the sensor or sensors, in particular the front field of view, is obscured by many road users in busy urban areas or traffic jams on the motorway.

Therefore, one or more sensors, especially a front sensor, are different from others Road users are blocked and, for example, lanes cannot be recognized, the method makes it possible to use the method to determine the trajectories of the dynamic objects of interest for localizing the host vehicle 1 to use.

With the method according to the invention, if the sensor or sensors are blocked by other road users and, for example, the lanes / road boundaries cannot be recognized, the trajectories of the dynamic objects can be used to localize the host vehicle 1 ( 1 ) be used. These trajectories are preferably assigned to a specific lane.

The optimal transformation between the local map 11 ( 8th ) and the reference map is achieved by the minimum of this optimization problem.

Such a transformation can also be achieved by means of a pose graph optimization with so-called nodes. This is the mapping of the local map 11 ( 8th ) is used as a node on the reference road map, which is used to create a new local map. With the pose graph optimization, restrictions that arise from the connection of two consecutive local maps can be used. This allows a better match between a local map 11 ( 8th ) and a reference road map.

3rd shows the first method step S1 in detail. In a first method step, environment sensor data from a sensor system which is in and / or on a host vehicle is used as a frame 1 ( 1 ) is arranged, recorded at a first point in time. Preferably multiple frames 1 , .., N at successive times 1 , .., N added. The environment sensor data are raw data, that is, unprocessed data supplied by the sensor. The sensor system 3rd can image sensors 6th , Lidar sensors 5 , and radar sensors 4th exhibit. The environment sensor data are recorded at a point in time and then merged into a common frame.

A first frame thus has a recording of the surroundings of the host vehicle that is merged by sensor data 1 ( 1 ) at a first point in time.

Characteristics, so-called features, are extracted in the frame. These features are extracted from the merged environment sensor data of the frame using a feature extraction process.

The extraction of features tries to identify certain objects or shapes in the frame and to determine their features such as position, orientation and extent. Most of the time, the sensor data can already be divided into related sensor data segments beforehand. Features for dynamic object tracking can be, for example, geometric properties such as L-, I -shapes, or bounding boxes or ellipses, clusters with reference points, sensor data segments, etc.

There are differently complex approaches to feature extraction. The aim of feature extraction is to provide an object description of objects of interest.

This is used as input for subsequent processing steps and can thus significantly reduce the runtime of the process.

In order to be able to provide position information for extended objects, a reference point that represents the object is preferably also determined. Based on their reference point, sensor data are assigned to objects belonging to them in each frame, which can enable the objects to be tracked in terms of space and time. The reference point can be a single sensor data point or a virtual reference point that can reproduce the movement of the underlying object.

Then the individual dynamic objects of interest to be tracked are classified into object classes. The primary purpose of the classification is to filter out unwanted objects at an early stage and thus remove them from the tracking level. These classes can be designed as cars (motor vehicles), trucks, bicycles, pedestrians, etc., for example. Furthermore, static elements such as traffic lights etc. can also be extracted in the first method step S1.

Bayes classifiers, Gaussian mixture models, neural networks, support vector machines algorithms or decision trees can be used as classification methods, since these can be implemented efficiently. A decision tree is a static classification method that is easy to implement. Recognized objects can be checked for plausibility through the classification. For example, a plausibility check can be used to check whether the measured dimensions of the individual objects of interest explicitly match the dimensions expected by road users. So that an object comes into question as a dynamic object of interest, for example a vehicle of interest, does not have to be only the recognized outline corresponds to a predetermined outline, but also, for example, the dimensions correspond to the expected values. For example, if an L-shape that is 8 meters wide and 8 meters long is detected, it is likely a house. It is preferably decided on the basis of the dimensions whether an object of interest is to be generated from the shape.

If this is not the case, the object is preferably deleted. Thus, the classification method can be used to delete incorrectly recognized objects for the first time. This increases the speed as well as the robustness of the process.

The objects of interest to be tracked dynamically are also assigned the object speed and a position (position data), i.e. the kinematic state, which is derived from the frames with the aid of the vehicle's own speed and the position (rotational movement) of the host vehicle 1 can be determined / estimated.

4th shows the classified objects to be tracked, each of the objects being assigned an object frame for visualization. Each of these object frames thus represents a dynamic object of interest to be tracked.

The first method step S1 thus generates a list of dynamic objects of interest for object tracking, with each object, the geometry (for example length, width, ellipse, L-shape, I-shape, etc.) with the associated reference point, the kinematic State (speed and position) and the class includes.

5 shows the subsequent second method step S2 in detail. In the second method step S2, the various recorded objects of interest of the 1,..., N frames are accumulated and the respective trajectory of each of the dynamic objects to be tracked of interest is determined.

For this purpose, all of the environment sensor data of the objects of interest are first entered in a local map 11 ( 6th ) transformed. A local map 11 ( 6th ) is preferred by the host vehicle 1 ( 1 ) by, for example, the sensor system 3rd ( 1 ) self-generated. One of the frames can also be used as a local map 11 ( 6th ) be used.

There is also a local map 11 a map of the surroundings of the host vehicle 1 ( 1 ), preferably from the host vehicle 1 ( 1 ) is created by yourself. For the sake of simplicity, the local map is 11 ( 6th ) for example the first frame at the beginning of the procedure. As the process progresses, a subsequent frame can be used.

The accumulation of the dynamic objects is determined by the host vehicle with the aid of a first prediction / estimate of the pose, that is to say the position and orientation of the dynamic objects of interest 1 ( 1 ), performed. The final estimate can be made, for example, by a Kalman filter. Are the corresponding environment sensor data in the local map 11 ( 6th ) are accumulated, they are clustered using a clustering process.

The clustering can for example be carried out using a DBSCAN algorithm.

A DBSCAN is a density-based process. DBSCAN stands for Density-Based Spatial Clustering of Applications with Noise. A multidimensional sensor data image can have several cluster focal points. The DBSCAN works based on density and is able to recognize several clusters. Furthermore, it does not have to be known in advance how many clusters exist. The algorithm can also recognize any cluster of any shape. Another advantage of using the DBSCAN is that the algorithm works largely deterministically and independently of the sequence. Other algorithms based on DBSAN can also be used. This is, for example, the HDBSCAN - Alogithm (Hierarchical Density Based Clustering), which extends the algorithm by converting it into a hierarchical clustering algorithm. The HDBSCAN algorithm is a very data-dependent method of clustering and requires very little user input. Another algorithm is, for example, the OPTICS (Ordering Points To Identify the Clustering Structure) algorithm, which can recognize clusters of different densities. Other algorithms based on DBSCAN, such as the shared nearest neighbor clustering algorithm, are also possible.

By means of the cluster analysis, on the one hand, the trajectories of the dynamic objects of interest can be determined and assigned to the objects of interest. Furthermore, the clustering uses a plausibility check to verify whether the object of interest determined from the frame is also one. For example, an object of interest identified by the first method step can, for example, assume the following cases: in the first case, the object is a verified object of interest, for example a car, pedestrian, etc .; the object was in the second case incorrectly identified as an object of interest; and in the third case it is still not possible to make a clear statement. In the second case, in which the object is incorrectly identified as a dynamic object of interest, the object is deleted. In the latter case, the object can be classified as an object of interest or not classified as an object of interest on the basis of a probability function or the like. This increases the robustness and speed of the process.

6th shows the dynamic classified objects of interest and their trajectories in a local map 11 . So are in 6th a first passenger car 7th with associated trajectory and a second passenger car 8th with the associated trajectory and a bus 9 and also a bike 10 to be seen with the associated trajectory.

The second method step S2 thus creates a local map 11 ( 6th ) with the classified dynamic objects of interest, the trajectories associated with the dynamic objects, as well as with the orientation and position of the dynamic objects.

7th shows the fourth method step S4 graphically. In the fourth method step S4, the local map created in the second method step S2 11 ( 4th ) through traffic-relevant static environmental data from static surrounding objects, such as roadways 13th ( 8th ) added. In this case, road boundaries, for example curb stones, road markings, individual, multi-lane roads, etc., can also be designed as static environmental data of static surrounding objects. Furthermore, static environmental data from static environmental objects such as active traffic display data such as traffic lights and / or passive traffic display data such as signs can be detected and the local map 11 ( 6th ) to be added. Static environmental data of static environmental objects can also be designed as environmental features such as a structure, in particular a tunnel, a bridge, a building, a gas station, a lake, a river, a stream, a forest, a parking bay, a mountain range, etc.

As a result, a complete local area map can be obtained by completing the traffic scene 12th can be achieved with a complete description of the scene.

The detection of the roadway 13th ( 8th ) and the lane boundary can also be implemented using methods of digital image processing, for example using a lane departure warning system.

8th shows such a completed local area map 12th showing a complete traffic scenario with lane 13th and the trajectories of the dynamic objects and the dynamic objects themselves. Here the dynamic objects are the first passenger cars 7th with the associated trajectory, the second passenger car 8th with the associated trajectory and the bus 9 and the bike 10 each with the associated trajectory. The additions result in a consistent, complete local area map 12th generated by the current traffic scene. The dynamic objects of interest can preferably be one of the recognized lanes 13th etc. can be assigned.

9 shows the completed local area map 12th in plan view and at a later point in time with a roadway 13th and with the first passenger car 7th with the associated trajectory, the second passenger car 8th with the associated trajectory and the bus 9 and the bike 10 each with the associated trajectory. Furthermore, as static environmental data from static surrounding objects, there are still parking spaces 15th recognized and the completed local area map 12th added. Image processing methods can be used for the detection of static environmental data from static environmental objects. Typical methods for extracting lanes and road boundaries are machine learning methods based on camera images or the semantic segmentation of a high-resolution point cloud-based sensor, such as the lidar sensor 5 ( 1 ) and the radar sensor 4th ( 1 ).

10 shows the fourth method step S4 in detail. The completed local area map is created in the subsequent method step S4 12th on a given reference road map, preferably an HDMap (High Definition Map) 14th pictured. HDMaps 14th have extremely high precision at the centimeter level and are particularly useful for autonomous vehicles. The mapping can be carried out using a map matching algorithm, preferably an optimization method. With such a map matching algorithm in digital image processing, approximately correlating patterns are created in the completed local environment map 12th ( 8th ) and the given HDMap 14th searched. On the basis of this correlation, a projective transformation is calculated which enables the completed local environment map 12th and the HDMap 14th map to each other in the best possible way.

11 shows the method according to the invention in sections, with only one radar sensor here 4th ( 1 ) was used to record environmental sensor data. Initially, a first frame of environment sensor data for the environment of the host vehicle is generated 1 ( 1 ) added (I). Subsequently features are extracted from the frame by a feature extraction process and these are combined into dynamic objects of interest. The dynamic objects of interest are then classified, and kinematic data are assigned to the individual dynamic objects of interest. Figure II shows the grouped objects of interest. Then the trajectories of the individual objects of interest are determined by clusters and a local map 11 created with objects and their trajectories (III).

Subsequently, lanes are determined as static environment data of static environment objects (IV), which of the local map 11 can be added to complete.

List of reference symbols

1

Ego vehicle

2

Driver assistance system

3

Sensor system

4th

Radar sensor

5

Lidar sensor

6th

Image sensor

7th

first passenger car with associated trajectory

8th

second passenger car with associated trajectory

9

Bus with associated trajectory

10

Bicycle with associated trajectory

11

local map

12th

completed local area map

13th

roadway

14th

HDMap

15th

Parking spaces

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102016205871 A1 [0004]

Claims (15)

Method for the sensor-based localization of a host vehicle (1) characterized by the steps: - generation of successive frames of environment sensor data of an environment of the host vehicle (1) by a sensor system (3), - extraction of features in at least one frame by a feature extraction method, - summarization of the Characteristics of individual dynamic objects of interest, - classification of the individual dynamic objects of interest, - determination of kinematic data of the individual dynamic objects of interest from the successive frames, taking into account the ego data of the host vehicle (1) and assigning the kinematic data to the individual dynamic Objects of interest, - transforming the captured environment sensor data of the dynamic objects of interest of the individual frames using the kinematic data into a local map (11), - clusters of the transformed environment sensor data of the dynamic objects of interest in the local map (11) to trajectories and assign the trajectories to the individual dynamic objects of interest, - determination of static environment data of static environment objects from at least one of the frames and complete the local map (11) based on the static environment data of static environment objects a completed local environment map (12) and assigning the individual dynamic objects of interest to the static environment data of static environment objects, - providing a reference road map and mapping the completed local environment map (12) onto the reference road map. Procedure according to Claim 1 , characterized in that the mapping is performed using a map matching algorithm. Procedure according to Claim 1 or 2 , characterized in that the reference road map is designed as an HDMap (high definition map). Procedure according to Claim 3 , characterized in that the HDMap comprises at least static infrastructure data. Method according to one of the preceding claims, characterized in that the reference map is designed as a self-learning road map. Method according to one of the preceding claims, characterized in that the static environmental data of static surrounding objects are designed as roadways and / or roadway boundaries. Method according to one of the preceding claims, characterized in that the static environmental data of static surrounding objects are designed as active traffic display data such as traffic lights and / or passive traffic display data such as signs. Method according to one of the preceding claims, characterized in that the static environment data of static environment objects are designed as environment features. Method according to one of the preceding claims, characterized in that a density-based method is used as the cluster method. Procedure according to Claim 9 , characterized in that the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is used as the clustering method. Method according to one of the preceding claims, characterized in that a reference point is determined for each dynamic object of interest. Procedure according to Claim 11 , characterized in that the determination of kinematic data of the individual dynamic objects of interest is carried out from the successive frames on the basis of the reference point. The host vehicle (1), which is designed to carry out a method according to one of the preceding claims, comprising at least one lidar sensor (4), a radar sensor (5) and an image sensor (6). Computer program, comprising instructions that cause the host vehicle (1) of the Claim 13 the method according to any one of the preceding Claims 1 to 12th executes. Data carrier signal that a computer program according to Claim 14 transmits.

Images