WO2024159351A1

WO2024159351A1 - Point cloud test method and apparatus

Info

Publication number: WO2024159351A1
Application number: PCT/CN2023/073812
Authority: WO
Inventors: 吴佳玉
Original assignee: 华为技术有限公司
Priority date: 2023-01-30
Filing date: 2023-01-30
Publication date: 2024-08-08

Abstract

A point cloud test method and apparatus, which are applied to the technical field of detection and can test a point cloud of a body target that is output by a detection apparatus to be tested, wherein the detection apparatus may specifically be a millimeter-wave radar. During a test, the point cloud test apparatus matches a true value of a body target with a point cloud of the body target, so as to obtain a matching situation between the point cloud of the body target and the true value of the body target. Since the body target is closer to a detection target in the actual installation environment of a detection apparatus to be tested, the quality of a point cloud which is output by the detection apparatus can be tested more accurately, thereby facilitating the evaluation of the detection capability of the detection apparatus. In addition, by means of automated matching between a true value of a body target and a point cloud of the body target, an evaluation error can be significantly reduced, and the test precision and the test efficiency are improved.

Description

Point cloud testing method and device

Technical Field

The present application relates to the field of detection technology, and in particular to a point cloud testing method and device.

Background Art

With the development of information technology, detection technology has achieved rapid development, and various detection devices have brought great convenience to people's lives and travel. For example, the advanced driving assistance system (ADAS) plays a very important role in smart cars. It uses the detection device installed on the car to detect the surrounding environment during the driving process of the vehicle, collect data, identify static and moving objects, etc., and combine with the map to perform systematic calculations and analysis, so that the driver can be aware of possible dangers in advance, effectively increasing the comfort and safety of car driving. The detection device can be regarded as the "eyes" of the device to perceive the environment. It can detect the surrounding environment and output point clouds. The quality of the point cloud represents the detection capability of the detection device. Therefore, the test of the point cloud output by the detection device (hereinafter referred to as point cloud test) has always been the key test item of the detection device in the industry.

The technology of using point targets to perform point cloud testing on detection devices is already very mature. Among them, point targets are targets that exist in the form of "points". Exemplarily, a point cloud testing method for point targets is as follows: the device under test (DUT) is placed in a darkroom (the darkroom is surrounded by absorbing materials), and point targets are set in the darkroom to test the point cloud output by the DUT when detecting the point targets in the darkroom.

However, point targets are quite different from detection targets in actual installation environments (hereinafter referred to as actual targets). For example, point targets usually only have characteristics such as position, distance, and orientation, but actual targets also have characteristics such as posture or size, and scattering characteristics also have various types. Currently, some suppliers only use human eyes to estimate the quality of the point cloud output by the detection device when detecting actual targets. In short, the current point cloud testing methods are difficult to accurately evaluate the quality of the point cloud output by the detection device.

Summary of the invention

The embodiments of the present application provide a point cloud testing method and device, which can test the point cloud of a body target and can more accurately test the quality of the point cloud output by a detection device.

In a first aspect, an embodiment of the present application provides a point cloud testing method, comprising:

Obtain true value data and point cloud. The true value data is the true value of the volume target, and the point cloud is the detection result obtained by the DUT detecting the volume target.

The point cloud and the true value data are matched to obtain a matching result set, where the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

Among them, the body target can be regarded as an object with at least two of the length, height and width, used as a target for testing the DUT. The body target includes but is not limited to one or more of a sphere, a cuboid, a plate, a dihedral, a trihedral, a cylinder, or a round top hat. When the detection device detects the body target, the detection result of a body target includes multiple sampling points. Optionally, when detecting the body target, the detection device can detect different surfaces of the body target at different viewing angles. In some scenarios, during the process of the detection device detecting the body target, the body target has characteristics such as posture and size.

The embodiment of the present application is based on matching the true value of the volume target and the point cloud of the volume target to obtain the matching between the point cloud of the volume target and the true value of the volume target. Since the volume target is closer to the actual target, the quality of the point cloud output by the detection device can be more accurately tested through the embodiment of the present application, which is conducive to evaluating the detection capability of the detection device.

In addition, the embodiments of the present application can significantly reduce the evaluation error and improve the test accuracy and test efficiency by automatically obtaining the matching result based on the true value of the volume target and the point cloud of the volume target.

Optionally, the number of volume targets may be one or more. In order to facilitate the description of the solution of the present application, the number of volume targets is described as at least one below.

Optionally, the above method can be implemented by a point cloud testing device. The following description is made by taking the point cloud testing device as an example, and the present application is also applicable to other forms of execution entities.

In another possible implementation of the first aspect, matching the point cloud with the true value data to obtain a matching result set includes:

Establish a three-dimensional matching box based on the true value data;

The point cloud is matched with the three-dimensional matching box to obtain a matching result set, where the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

In the above implementation, the point cloud testing device matches the point cloud with the three-dimensional matching box, and can accurately calculate the positional relationship between the point cloud and the volume target, obtain the matching result, and improve the test accuracy.

In a possible implementation of the first aspect, matching the point cloud with the true value data to obtain a matching result set includes:

Project the true value data to obtain two-dimensional true value data;

Project the point cloud to obtain a two-dimensional point cloud;

The two-dimensional point cloud is matched with the two-dimensional true value data to obtain a matching result set, where the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

In the above implementation, both the true value data and the point cloud are processed into two-dimensional data for matching. First, matching in two dimensions can save the amount of calculation during matching and further improve the test efficiency. Secondly, in some scenarios, the evaluation of the detection device mainly focuses on its ranging ability, speed measurement ability and angle resolution ability. These capabilities are more related to the longitudinal and lateral data in the detection results. Therefore, two-dimensional projection of the data can test the point cloud quality output by the detection device without significantly losing accuracy.

In a possible implementation of the first aspect, projecting the true value data to obtain two-dimensional true value data includes:

Project the true value data onto the horizontal plane to obtain two-dimensional true value data;

Project the point cloud to obtain a two-dimensional point cloud, including:

Project the point cloud onto a horizontal plane to obtain a two-dimensional point cloud.

In the above implementation, the point cloud and the true value data can be projected onto a horizontal plane during projection. The projection onto the horizontal plane can largely retain the horizontal and vertical data of the volume target, which is beneficial to improving the test efficiency and saving the amount of calculation.

The horizontal plane refers to a relatively horizontal plane, such as the XY plane.

Take the projection of a point cloud as an example. For example, the point cloud contains multiple sampling points, each sampling point corresponds to a three-dimensional coordinate (taking the Cartesian coordinate system as an example). For any of the sampling points, the vertical data can be discarded during projection (or the Z-axis value is set to 0, or the vertical data is ignored), thereby obtaining a two-dimensional sampling point.

In another possible implementation of the first aspect, the time of the true value data and the point cloud is aligned. Further, when the true value data and the point cloud are projected as two-dimensional data, the time of the two-dimensional true value data and the two-dimensional point cloud is aligned.

For example, the true value data contains A frames from the first moment to the second moment, A is an integer and A>0; the point cloud contains B frames from the third moment to the fourth moment, B is an integer and B>0. When the two are aligned in time, for any frame in the B frames, the true value of the frame closest to it in timestamp can be found.

In the above implementation, the true value data and the point cloud are aligned in time, so that the point cloud at a certain moment can find a frame of true value that is closest in time, which can improve the accuracy of matching and further improve the accuracy of the point cloud test.

In another possible implementation of the first aspect, the coordinates of the true value data and the point cloud are aligned. Exemplarily, the true value data and the point cloud are obtained by a true value system and DUT detection installed on the vehicle, respectively, and the origin of the true value data and the point cloud can be converted to the rear axle center of the vehicle. The above implementation can improve the accuracy of matching, thereby improving the accuracy of point cloud testing.

In yet another possible implementation of the first aspect, the two-dimensional truth data includes a plurality of truth frames, and the two-dimensional point cloud includes a plurality of point cloud frames;

Match the 2D point cloud with the 2D true value data to obtain a matching result set, including:

Determine at least one truth frame in a first truth frame, wherein one truth frame corresponds to one volume target, and the first truth frame belongs to multiple truth frames;

A matching result subset is obtained according to the range of at least one truth frame and the positions of multiple sampling points in the first point cloud frame.

The first point cloud frame belongs to the plurality of point cloud frames, and the timestamps of the first point cloud frame and the first true value frame are the same. The matching result subset belongs to the matching result set.

In the above implementation, a matching method is introduced by taking the matching of the first point cloud frame as an example. During matching, the true value data is projected to obtain two-dimensional true value data, and a true value frame (or two-dimensional true value frame) is established. The two-dimensional true value data contains data at multiple moments. For a frame of two-dimensional true value data at a certain moment, matching is performed based on the position of the truth value frame and the position of the sampling point in the two-dimensional point cloud at the same moment. In this way, the true value and point cloud at the same moment can be matched, thereby improving the test accuracy.

Understandably, the following situations may occur during matching: for a certain sampling point, it may fall into one or more true value boxes, or may not fall into any true value box; and for a certain true value box, its range may contain one or more sampling points, or may not contain the sampling point.

In yet another possible implementation of the first aspect, at least one truth box includes a first truth box corresponding to a first volume target, and the first volume target belongs to at least one volume target.

Exemplarily, when the first point cloud frame includes the first sampling point and the first sampling point falls into the first true value frame, the first sampling point belongs to the matching sampling point. Further, the first sampling point matches the true value of the first object.

Exemplarily, when the first point cloud frame includes the second sampling point and the second sampling point does not fall into any truth frame of at least one truth frame, the second sampling point belongs to an unmatched sampling point;

Exemplarily, when any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.

In the above implementation, the classification of the matching results is explained by taking the first truth box, the first sampling point, and the second sampling point as examples. Among them, the matched sampling point is the sampling point that successfully matches the true value of the volume target. The unmatched sampling point is the point cloud that does not successfully match the true value of the volume target. The unmatched true value is the true value that does not successfully match any sampling point. It is not difficult to see that the number of matched point clouds is usually positively correlated with the quality of the point cloud, and the number of unmatched point clouds and the number of unmatched true values are negatively correlated with the quality of the point cloud. Therefore, the classification of the matching results preliminarily reflects the accuracy of the point cloud, which is conducive to the subsequent classification test of the matching results of different categories, and improves the richness and accuracy of the point cloud test.

In another possible implementation of the first aspect, the number of truth boxes is greater than or equal to 2. Matching the two-dimensional point cloud with the two-dimensional truth data to obtain a matching result set also includes:

When the first point cloud frame includes the third sampling point and the third sampling point falls into at least two true value frames, the true value of the volume target matching the third sampling point is determined according to the position between the third sampling point and the true values of the volume targets corresponding to the at least two true value frames.

The above implementations illustrate how to determine the true value of the volume target matched by the sampling point when the sampling point falls into multiple true value boxes. In this way, the matching relationship between the point cloud and the true value can be clarified, and the accuracy of the point cloud test can be improved.

In another possible implementation manner of the first aspect, the position between the third sampling point and at least two truth value frames is determined Determine the true value of the volume target that matches the third sampling point, including:

A point pair is established between the true value of the volume target corresponding to at least two truth value frames and the third sampling point, a distance matrix is constructed according to the point pair, the distance between the true value and the third sampling point is obtained, and the true value with the closest distance is taken as the true value matching the third sampling point.

By constructing a distance matrix, the distance relationship between the sampling points and the true value can be determined more accurately, the true value of the volume target matching the sampling points can be determined, and the accuracy of the point cloud test can be improved.

In yet another possible implementation of the first aspect, obtaining true value data and a point cloud includes:

The initial truth value and the initial point cloud are preprocessed to obtain the truth value data and the point cloud. The preprocessing may include one or more of the following processes: time alignment (or timestamp alignment), coordinate conversion, and format conversion. Preprocessing can improve the correspondence between the truth value data and the point cloud, reduce the complexity of matching, and improve the efficiency of point cloud testing.

In another possible implementation of the first aspect, test items such as accuracy, false alarms, and missed detections of the point cloud are evaluated through a set of matching results.

In another possible implementation manner of the first aspect, the method further includes:

According to the matching sampling points in the matching result set, the accuracy evaluation data about the DUT is obtained, wherein the accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy and height accuracy.

In another possible implementation of the first aspect, obtaining accuracy evaluation data about the DUT according to the matching sampling points in the matching result set includes:

According to the number of sampling points in the matching sampling points that match the true value of the fourth body target, the number of matching sampling points about the fourth body target is obtained.

In another possible implementation of the first aspect, the matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M＞0, and N is an integer and N＞0.

The distance measurement accuracy of the DUT when detecting the second target can be evaluated through N sampling points and M corner points.

In another possible implementation of the first aspect, the N sampling points include the nearest sampling point, and the M corner points include the nearest corner point in the horizontal direction. Here, "nearest" refers to the distance between the point and a reference point or a reference device. For example, the reference point may be the point where the DUT is located. In this case, the nearest sampling point is the sampling point that is closest to the DUT among the N sampling points, and the nearest corner point in the horizontal direction is the corner point that is closest to the DUT in the horizontal direction among the M corner points.

Furthermore, the ranging accuracy includes a lateral ranging accuracy with respect to the second object, and the lateral ranging accuracy with respect to the second object is related to a lateral distance between the nearest sampling point and the DUT, and a lateral distance between the nearest lateral corner point and the DUT.

It is understandable that the above description is made by taking DUT as a reference point as an example. In the specific implementation process, DUT can also be replaced by a vehicle, a rear axle center of a vehicle, or a true value system.

In yet another possible implementation of the first aspect, the lateral ranging accuracy σ _x of the second body target satisfies the following formula:

σ _x = |X _pi -X _cj |

Where _Xpi is the lateral distance between the nearest sampling point and the DUT, and _Xcj is the lateral distance between the nearest lateral corner point and the DUT.

In another possible implementation of the first aspect, the N sampling points include the nearest sampling point, and the M corner points include the radially nearest corner point. Here, "nearest" refers to the distance between the point and a preset point. The preset vertex may be, for example, a DUT. In this case, the nearest sampling point is the sampling point that is closest to the DUT among the N sampling points, and the radially nearest corner point is the corner point that is closest to the DUT in radial distance among the M corner points.

Furthermore, the ranging accuracy includes a longitudinal ranging accuracy with respect to the second object, and the longitudinal ranging accuracy with respect to the second object is related to a radial distance between a nearest sampling point and the DUT and a radial distance between a radial nearest corner point and the DUT.

In another possible implementation of the first aspect, the longitudinal ranging accuracy _σd of the second target satisfies the following formula:

σ _d ＝|D _pi −D _ck |

Where _Dpi is the radial distance between the nearest sampling point and the DUT, and _Dck is the radial distance between the radial nearest corner point and the DUT.

In yet another possible implementation of the first aspect, the speed measurement accuracy includes speed measurement accuracy with respect to a third-body target;

The matching sampling points include K sampling points that match the true value of the third-body target, where K is an integer and K>0;

The K sampling points include the strongest sampling point. The velocity measurement accuracy of the third-body target is related to the radial velocity of the strongest sampling point and the radial velocity of the true value of the third-body target.

The above embodiment describes a method for determining the speed measurement accuracy. The speed accuracy σ _v can be indicated by the absolute value of the radial speed error between the strongest point in the matching point and the reference true value. For example, the speed accuracy σ _v satisfies the following formula:

σ _v ＝|V _pi −V _t |

Among them, _Vpi is the radial velocity of the strongest sampling point, and _Vt is the radial velocity of the true value of the third body target.

As a possible implementation, the strongest sampling point is the sampling point with the strongest radar cross section (RCS) among the K sampling points, the number of sampling points matching the true value is K, and the RCS of the K sampling points are respectively expressed as R _p1 , R _p2 , R _p3 ,…, R _pK . The RCS of the strongest sampling point can be expressed as R _pi , which can satisfy the following formula:

R _pi =max(R _p1 ,R _p2 ,R _p3 ,…,R _pK )

Optionally, the true radial velocity of the third body target may be replaced by the radial velocity of the third body target.

In another possible implementation of the first aspect, unmatched sampling points can be used for false alarm judgment. A false alarm is a time when a target does not exist but the detection device judges that there is a target and outputs a point cloud under certain circumstances. A false alarm may correspond to a point cloud that does not match the true value. When judging a false alarm, the unmatched sampling points in the point cloud frame can be tracked, and a multi-frame association method is used to determine whether there is a false alarm in the point cloud. In this way, the unmatched sampling points can be tracked in subsequent point cloud frames, which can reduce the false alarm judgment error caused by point cloud flickering, improve the accuracy of false alarm judgment, and improve the accuracy of point cloud testing.

In another possible implementation of the first aspect, the two-dimensional point cloud includes a plurality of continuous point cloud frames, the matching set includes unmatched sampling points, and the method further includes:

According to sampling points in the multiple continuous point cloud frames that are among the unmatched sampling points, false alarm targets in the multiple continuous point cloud frames are determined.

In another possible implementation manner of the first aspect, the plurality of continuous point cloud frames include the second point cloud frame and Q point cloud frames after the second point cloud frame, where Q is an integer and Q>0;

According to the sampling points in the unmatched sampling points located in multiple continuous point cloud frames, the false alarm targets in the point cloud are determined, including:

Clustering the sampling points in the second point cloud frame among the unmatched sampling points to obtain at least one point cloud cluster;

assigning an initial life value to a first point cloud cluster in the at least one point cloud cluster;

According to the sampling points in the Q point cloud frames among the unmatched sampling points, point cloud clusters in the Q point cloud frames are determined;

False alarm targets in a plurality of consecutive point cloud frames are determined according to the position of the first point cloud cluster and the positions of the point cloud clusters in the Q point cloud frames.

Here, Q can be a fixed number or a non-fixed number.

In the above embodiment, the testing device clusters the unmatched point cloud to obtain multiple point cloud clusters, and each point cloud cluster is assigned an initial life value. For a point cloud cluster existing in a certain point cloud frame, the point cloud frame is matched with multiple subsequent point cloud frames. If there is a point cloud cluster matching it in the subsequent point cloud frame, the life value of the point cloud cluster is increased, otherwise the life value of the point cloud cluster is reduced; repeat the matching of multiple point cloud frames. If the life value of the point cloud cluster reaches the first threshold, the point cloud cluster forms a false alarm target. If the life value of the point cloud cluster reaches the second threshold or is lower than the third threshold, the point cloud cluster does not form a false alarm target, and the point cloud cluster can be discarded. Because a single sampling point is prone to flickering, the matching complexity is high and the results are unreliable The above implementation method clusters the unmatched sampling points and tracks the unmatched point clouds in the form of point cloud clusters, which not only reduces the complexity of matching, but also greatly improves the reliability and availability of false alarm judgment and improves the accuracy of point cloud testing.

It should be understood that reaching the first threshold may be higher than or equal to the first threshold, which is subject to specific design. The same is true for the second threshold and the third threshold.

In some scenarios, when matching point cloud clusters, the point cloud cluster frame is determined according to the size of the point cloud cluster in the point cloud frame, and the matching frame can enclose the point cloud in the cluster. For the current point cloud frame, if there is a point cloud cluster frame in the next point cloud frame that overlaps with the point cloud cluster frame of the current frame, the overlap matrix between the point cloud cluster frame and the point cloud cluster frame is calculated to establish the association between the frames. If there is a point cloud cluster frame in the next point cloud frame that successfully matches the point cloud cluster frame of the current frame, the life value of this point cloud cluster increases; otherwise, the life value of the point cloud cluster decreases.

By establishing associations between frames through matching frames, the computational complexity can be further reduced and the efficiency of point cloud testing can be improved.

In yet another possible implementation of the first aspect, the unmatched point cloud may also be used to determine a false alarm rate.

Exemplarily, the point cloud testing device determines the false alarm rate according to the number of point cloud frames involved in the early warning target calculation and the number of false alarm point cloud frames, wherein the false alarm point cloud frame is a point cloud frame with a false alarm target, or the false alarm point cloud frame is a point cloud frame containing at least one point cloud cluster whose life value reaches the first threshold.

Exemplarily, the false alarm rate ρ satisfies the following formula:

Among them, n is the number of point cloud frames involved in the calculation of false alarm targets, and n _false is the number of frames with false alarm targets.

In another possible implementation of the first aspect, the unmatched true value can be used for missed detection judgment. Missed detection refers to the event that in some cases, a target exists but the radar judges that there is no target and does not output a point cloud. The missed detection of the point cloud can correspond to the true value of the unmatched point cloud.

Optionally, when judging missed detection, it can be determined whether the volume target is blocked. If the volume target is blocked, the missed detection of the volume target does not constitute a valid missed detection. In this way, the missed detection judgment error caused by the occlusion of the volume target can be reduced, the accuracy of missed detection judgment can be improved, and the accuracy of point cloud testing can be improved.

In another possible implementation of the first aspect, the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame;

The method also includes:

Determine the suspected missed detection target in the third point cloud frame according to the unmatched true value corresponding to the third point cloud frame;

Determining the obscured volume target according to a field of view relationship between at least one volume target and the radar;

The occluded volume targets in the suspected missed targets in the third point cloud frame are filtered to determine the missed targets contained in the third point cloud frame.

In the above embodiment, the testing device determines the suspected missed detection targets according to the unmatched true value, removes the obscured body targets from the suspected missed detection targets according to the occlusion relationship, and reduces the missed detection judgment error caused by the occlusion of the body targets.

In another possible implementation of the first aspect, determining the obscured volume target according to a field of view relationship between at least one volume target and a radar includes:

Whether the fifth volume object is blocked is determined according to the intersection of a line between a fifth volume object in the at least one volume object and the DUT and edges of other volume objects.

For example, if there are V occluded corner points among the multiple corner points of the fifth object, the fifth object is occluded, V is an integer and V>0, wherein the occluded corner points are corner points where the lines connecting the corner points intersect with the edges of other objects.

For example, if the fifth object is an occluded object, it satisfies the following two conditions: ① V corner points of the fifth object intersect with the DUT, V is an integer and V>0; ② The number of valid edges in the fifth object is greater than or equal to the fourth threshold. The four thresholds may be predefined or pre-set. For example, the fourth threshold may be 4, or the fourth threshold may be 1. The valid edge may be determined in the following manner: for any corner point or any edge corner point (edge corner point refers to a corner point located on an edge) in the fifth body target, if the line connecting the corner point (or the edge corner point) and the DUT intersects on any edge of the fifth body target, the edge where the corner point (or the edge corner point) is located is invalid. If the lines connecting the corner point on the first edge of the fifth body target and the DUT do not intersect with other edges in the fifth body target, the first edge is a valid edge.

In another possible implementation of the first aspect, when judging missed detection, the area corresponding to the unmatched true value can be tracked in the point cloud frame, and a multi-frame association method can be used to determine whether the volume target is missed. In this way, the unmatched true value can be tracked in subsequent point cloud frames, which can reduce the missed detection judgment error caused by point cloud flickering, improve the accuracy of false alarm judgment, and enhance the accuracy of point cloud testing.

Exemplarily, when there is a missed detection of a certain volume target in three consecutive point cloud frames, the volume target is determined to be a missed detection target.

In yet another possible implementation of the first aspect, the unmatched true value may also be used to determine the missed detection rate.

Exemplarily, the point cloud testing device determines the missed detection rate according to the number of point cloud frames involved in the early warning target calculation and the number of missed detection point cloud frames, wherein the missed detection point cloud frames are point cloud frames with missed detection targets (or determined missed detection targets).

Exemplarily, the missed detection rate γ satisfies the following formula:

Among them, n is the number of point cloud frames involved in the calculation of missed targets, and n_lose is the number of missed point cloud frames.

In a second aspect, an embodiment of the present application provides a point cloud testing device, the point cloud testing device comprising a data acquisition module and a data matching module, wherein:

The data acquisition module is used to obtain true value data and point cloud. The true value data is the true value of the volume target, and the point cloud is the detection result obtained by the DUT on the volume target. The detection result includes sampling points.

The data matching module is used to match the point cloud and the true value data to obtain a matching result set, which includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

In yet another possible implementation manner of the second aspect, the data matching module is used to:

Establish a three-dimensional matching box based on the true value data;

In a possible implementation manner of the second aspect, the data matching module is used to:

Project the true value data to obtain two-dimensional true value data;

Project the point cloud to obtain a two-dimensional point cloud;

Project the point cloud to obtain a two-dimensional point cloud, including:

In another possible implementation of the second aspect, the time of the true value data and the point cloud is aligned. Further, when the true value data and the point cloud are projected as two-dimensional data, the time of the two-dimensional true value data and the two-dimensional point cloud is aligned.

In yet another possible implementation of the second aspect, the coordinates of the true value data and the point cloud are aligned.

In yet another possible implementation of the second aspect, the two-dimensional truth data includes a plurality of truth frames, and the two-dimensional point cloud includes a plurality of point cloud frames;

The data matching module is also used to:

A matching result subset is obtained according to a range of at least one true value frame and positions of multiple sampling points in a first point cloud frame, wherein the first point cloud frame belongs to multiple point cloud frames, the first point cloud frame and the first true value frame have the same timestamp, and the matching result subset belongs to a matching result set.

In yet another possible implementation of the second aspect, the at least one truth box includes a first truth box corresponding to a first volume target, and the first volume target belongs to the at least one volume target;

When the first point cloud frame includes the first sampling point and the first sampling point falls into the first true value frame, the first sampling point belongs to the matching sampling point, and the first sampling point matches the true value of the first volume target;

In the case where the first point cloud frame includes the second sampling point and the second sampling point does not fall into any truth frame of the at least one truth frame, the second sampling point belongs to an unmatched sampling point;

When any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.

In yet another possible implementation of the second aspect, the number of at least one truth box is greater than or equal to 2,

The data matching module is also used to:

In yet another possible implementation manner of the second aspect, the data matching module is further configured to:

In yet another possible implementation manner of the second aspect, the data acquisition module is further configured to:

The initial truth value and the initial point cloud are preprocessed to obtain the truth value data and the point cloud. The preprocessing may include one or more of the following processes: time alignment, coordinate conversion, and format conversion. Preprocessing can improve the correspondence between the truth value data and the point cloud, reduce the complexity of matching, and improve the efficiency of point cloud testing.

In another possible implementation of the second aspect, the point cloud testing device further includes a data calculation module, which is used to evaluate the accuracy, false alarms, and missed detections of the point cloud through a matching result set.

In another possible implementation of the second aspect, the point cloud testing device further includes a data calculation module, which is used to obtain accuracy evaluation data about the DUT based on the matching sampling points in the matching result set. The accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy, and height accuracy.

In yet another possible implementation manner of the second aspect, the data calculation module is further used to:

In another possible implementation of the second aspect, the matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M＞0, and N is an integer and N＞0.

The data calculation module is also used to evaluate the ranging accuracy of the DUT when detecting the second body target based on the N sampling points and the M corner points.

In another possible implementation manner of the second aspect, the N sampling points include the nearest sampling point, and the M corner points include Furthermore, the ranging accuracy includes the lateral ranging accuracy with respect to the second object, and the lateral ranging accuracy with respect to the second object is related to the lateral distance between the nearest sampling point and the DUT and the radial distance between the radial nearest corner point and the DUT.

In yet another possible implementation of the second aspect, the lateral ranging accuracy σ _x of the second body target satisfies the following formula:

σ _x = |X _pi -X _cj |

In another possible implementation manner of the second aspect, the N sampling points include the nearest sampling point, and the M corner points include the radial nearest corner point. Further, the ranging accuracy includes the longitudinal ranging accuracy with respect to the second body target, and the longitudinal ranging accuracy with respect to the second body target is related to the radial distance between the nearest sampling point and the DUT and the radial distance between the radial nearest corner point and the DUT.

In yet another possible implementation of the second aspect, the longitudinal ranging accuracy _σd of the fourth target satisfies the following formula:

σ _d ＝|D _pi −D _ck |

In yet another possible implementation of the second aspect, the speed measurement accuracy includes speed measurement accuracy with respect to a third-body target;

The matching sampling points include K sampling points that match the true value of the third body target, where K is an integer and K>0;

σ _v ＝|V _pi −V _t |

R _pi =max(R _p1 ,R _p2 ,R _p3 ,…,R _pK )

In yet another possible implementation manner of the second aspect, unmatched sampling points may be used for false alarm judgment.

In yet another possible implementation manner of the second aspect, the point cloud testing device further includes a data calculation module, and the data calculation module is further used to:

In another possible implementation of the second aspect, the plurality of continuous point cloud frames include the second point cloud frame and Q point cloud frames after the second point cloud frame, where Q is an integer and Q>0;

The data calculation module is also used for:

In the above embodiment, the testing device clusters the unmatched point cloud to obtain a plurality of point cloud clusters, and each point cloud cluster is assigned an initial life value. For a point cloud cluster existing in a certain point cloud frame, the point cloud frame is matched with a plurality of subsequent point cloud frames. If there is a point cloud cluster matching it in the subsequent point cloud frame, the life value of the point cloud cluster is increased, otherwise the life value of the point cloud cluster is reduced; and the matching of multiple point cloud frames is repeated in this way. If the life value of the point cloud cluster reaches the first threshold, the point cloud cluster forms a false alarm target. If the life value of the point cloud cluster reaches the second threshold or is lower than the third threshold, the point cloud cluster does not form a false alarm target, and the point cloud cluster can be optionally discarded.

In yet another possible implementation of the second aspect, the unmatched point cloud may also be used to determine a false alarm rate.

Exemplarily, the false alarm rate ρ satisfies the following formula:

In another possible implementation of the second aspect, the unmatched true value can be used for missed detection judgment. Missed detection refers to the event that in some cases, a target exists but the radar judges that there is no target and does not output a point cloud. The missed detection of the point cloud can correspond to the true value of the unmatched point cloud.

In yet another possible implementation of the second aspect, the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame;

The data calculation module is also used for:

In another possible implementation of the second aspect, the data calculation module is further used to

For another example, if the fifth body target is an occluding target, it satisfies the following two conditions: ① The lines connecting the V corner points of the fifth body target and the DUT intersect with the edges of other body targets, V is an integer and V>0; ② The number of valid edges in the fifth body target is greater than or equal to the fourth threshold. The fourth threshold may be predefined or pre-set. For example, the fourth threshold may be 4, or the fourth threshold may be 1. The valid edges may be determined as follows: for any corner point or any edge corner point (edge corner point refers to a corner point located on an edge) in the fifth body target, if the line connecting the corner point (or the edge corner point) and the DUT intersects with any edge of the fifth body target, the edge where the corner point (or the edge corner point) is located is invalid. If the lines connecting the corner points on the first edge of the fifth body target and the DUT do not intersect with other edges in the fifth body target, the first edge is a valid edge.

In another possible implementation of the second aspect, when judging missed detection, the area corresponding to the unmatched true value can be tracked in the point cloud frame, and a multi-frame association method can be used to determine whether the volume target is missed.

In yet another possible implementation of the second aspect, the unmatched true value may also be used to determine the missed detection rate.

Exemplarily, the missed detection rate γ satisfies the following formula:

In a third aspect, an embodiment of the present application provides a chip, the chip comprising a processor. When the processor calls a computer program or instruction, the method described in any one of the first aspects is executed. That is, the processor is used to implement the method described in any one of the first aspects.

Optionally, the chip further includes a communication interface, where the communication interface is used to receive and/or send data, and/or the communication interface is used to provide input and/or output for the processor.

Optionally, the chip may also include a memory, which may be used to store computer programs or instructions. Furthermore, the memory may be located outside the processor, or may be integrated with the memory.

In a fourth aspect, an embodiment of the present application provides a computing device, which includes a processor; when the processor calls a computer program or instruction in a memory, the method described in any one of the first aspects above is executed.

Optionally, the computing device further includes a communication interface, where the communication interface is used to receive and/or send data, and/or the communication interface is used to provide input and/or output for the processor.

It should be noted that the above embodiment is described by taking a processor (or general-purpose processor) that executes the method by calling a computer specification as an example. In the specific implementation process, the processor can also be a dedicated processor, in which case the computer instructions have been pre-loaded in the processor. Optionally, the processor can also include both a dedicated processor and a general-purpose processor.

Optionally, the computing device may further include a memory, which may be used to store computer programs or instructions. Furthermore, the memory may be located outside the processor, or may be integrated with the memory.

In a fifth aspect, an embodiment of the present application provides a point cloud testing system, which includes a data preprocessing module, a data matching module and a data calculation module. The point cloud test is used to implement any method described in the first aspect.

Furthermore, the point cloud testing system also includes a data statistics module, which is used to count the matching results.

Furthermore, the point cloud test system further includes a data storage module, which is used to store the initial true value data and the initial point cloud. Furthermore, it is also used to store the true value data, the point cloud, the matching result set, the test item results, etc.

Furthermore, the point cloud test system also includes a test vehicle, on which a truth system and a DUT are installed, wherein the truth system is used to collect initial truth data, and the DUT is used to collect initial point clouds. Alternatively, the test vehicle can be replaced by a movable terminal, such as a drone, a robot or other transportation tool or an intelligent terminal.

In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store instructions or computer programs. When the instructions or computer programs are executed, the method described in any one of the first aspects above is implemented.

In a seventh aspect, the present application provides a computer program product, the computer program product comprising computer instructions or a computer program,

When the instruction or computer program is executed, the method described in any one of the first aspects above is implemented.

Optionally, the computer program product may be a software installation package or an image package. When the aforementioned method is required, the computer program product may be downloaded and executed on a computing device.

The beneficial effects of the technical solutions provided in the second to seventh aspects of the present application can refer to the beneficial effects of the technical solution of the first aspect, and will not be repeated here.

BRIEF DESCRIPTION OF THE DRAWINGS

The following is a brief introduction to the drawings required for describing the embodiments.

FIG1 is a schematic diagram of a point cloud testing technology based on a point target;

FIG2 is a schematic diagram of a scene for collecting the true value of a volume target and a point cloud of a volume target provided by an embodiment of the present application;

FIG3 is a schematic diagram of the architecture of a point cloud testing system provided in an embodiment of the present application;

FIG4 is a schematic diagram of a flow chart of a point cloud testing method provided in an embodiment of the present application;

FIG5 is a schematic diagram of true value data and point cloud provided in an embodiment of the present application;

FIG6 is a schematic diagram of two-dimensional true value data provided by an embodiment of the present application;

FIG7 is a schematic diagram of a two-dimensional point cloud provided in an embodiment of the present application;

FIG8 is a schematic diagram of a point cloud frame and a true value frame provided in an embodiment of the present application;

FIG9 is a schematic diagram of a truth value box provided in an embodiment of the present application;

FIG10 is a schematic diagram of a matching result provided in an embodiment of the present application;

FIG11 is a flow chart of another point cloud testing method provided in an embodiment of the present application;

FIG12 is a schematic diagram of a three-dimensional truth value frame provided in an embodiment of the present application;

FIG13 is a schematic diagram of a matching result provided in an embodiment of the present application;

FIG14 is a schematic diagram of a possible number of matching sampling points provided in an embodiment of the present application;

FIG15 is a schematic diagram of the distance between a sampling point and a DUT provided in an embodiment of the present application;

FIG16 is a schematic diagram of the distance between another sampling point and the DUT provided in an embodiment of the present application;

FIG17 is a schematic diagram of a possible unmatched point cloud provided by an embodiment of the present application;

FIG18 is a flow chart of another false alarm determination method provided in an embodiment of the present application;

FIG19 is a schematic diagram of the position of a body target provided in an embodiment of the present application;

FIG20 is a schematic diagram of the structure of a point cloud testing device provided in an embodiment of the present application;

FIG. 21 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application.

DETAILED DESCRIPTION

The embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings.

For ease of understanding, the following examples provide some explanations of concepts related to the embodiments of the present application for reference.

1. Detection device: The detection device can output point clouds, including but not limited to radar or laser radar, etc. Among them, the radar can be a millimeter wave radar, a centimeter wave radar, etc. In some scenarios, a device that integrates radar and camera at the same time (fusion detection device) can also output point clouds, and this fusion detection device also falls within the scope of the detection device of this application.

2. Field of view: There needs to be a line of sight (LOS) between the transmitter of the detection device and the target, and/or between the receiver of the detection device and the target, where the signal (e.g., radio wave, laser) transmission is uninterrupted. This line of sight can be understood as the field of view.

3. Device under test (DUT): The detection device being tested.

4. True value: The true value refers to the true value of the measured target under certain time and space (or position, state) conditions. The true value is the true value of a variable itself, usually an ideal concept. In the embodiment of the present application, the true value can be a reference true value.

5. Volume Target: Volume target can be regarded as an object with at least two of length, height and width, as the test DUT When the DUT detects a solid target, multiple sampling points can usually be obtained.

Exemplarily, the body target includes but is not limited to one or more of a sphere, a cuboid (including a cube), a plate, a cylinder, or a round top hat. It is not difficult to see that when the detection device detects the body target, it can detect different surfaces of the body target at different viewing angles. In some scenarios, when the detection device detects the body target, the body target has characteristics such as posture and size, so that it is closer to objects in production and living environments (including living organisms).

The above description of technical terms may be optionally used in the following embodiments.

The quality of the point cloud represents the detection capability of the detection device. At present, the test of the point cloud output by the detection device (hereinafter referred to as point cloud test) has always been the key test item of the detection device in the industry.

As shown in Figure 1, it is a schematic diagram of a point cloud test technology based on point targets. The radar under test (exemplary DUT) is placed on a rotatable turntable and placed in a microwave darkroom (the ground, walls, and ceiling are all equipped with absorbing materials). There is also a radar target simulator in the microwave darkroom, which can simulate point targets of different distances and speeds. The radar under test detects the point target (the double-headed arrows shown in Figure 1 represent the radar signal emission and its echo) to evaluate the long-range measurement and accuracy of the radar under test.

As shown in Figure 1, point cloud testing technology based on point targets has become increasingly mature. However, point targets are quite different from actual targets. Actual targets can be represented by volume targets. When the detection device detects a volume target, it will output multiple sampling points. Currently, there is basically no automated testing solution for volume target point clouds. The industry urgently needs to test the point cloud output by the DUT for volume targets.

In view of this, an embodiment of the present application provides a point cloud testing method and device. The embodiment of the present application matches the true value of the volume target with the point cloud of the volume target to obtain the matching situation between the point cloud of the volume target output by the DUT and the true value of the volume target. Since the volume target is closer to the actual target, the quality of the point cloud output by the detection device can be more accurately tested through the embodiment of the present application, which is conducive to evaluating the detection capability of the detection device.

A method of obtaining the true value of a volume target and a point cloud of the volume target is first introduced as an example below.

Please refer to Figure 2, which is a schematic diagram of a scene for collecting the true value of a volume target and a point cloud of a volume target provided in an embodiment of the present application, and a device to be tested is loaded on a vehicle. The vehicle is placed in an environmental test field, and further, the vehicle can travel in the environmental test field. One or more volume targets are also set in the environmental test field, such as volume target T1, volume target T2, volume target T3 and volume target T4 shown in Figure 2. The device to be tested can collect an initial point cloud (or original point cloud), which includes a point cloud obtained by detecting the aforementioned volume target.

In one embodiment, the vehicle further includes a truth system, which can collect initial truth data, and the truth accuracy of the initial truth data meets the preset accuracy requirements. Exemplarily, the truth system includes a laser radar, or a camera that can obtain depth data.

Of course, the aforementioned vehicle is an exemplary device for carrying the detection device, which can be replaced by other mounting platforms, or moving devices, such as logistics robots, drones and other means of transportation.

The following is an architectural diagram of the point cloud testing system of the present application. It should be noted that the system architecture and business scenarios described in this application are intended to more clearly illustrate the technical solution of the present application and do not constitute a limitation on the technical solution provided by the present application. With the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided by this application is also applicable to similar technical problems.

Please refer to FIG. 3, which is a schematic diagram of the architecture of a point cloud testing system provided by an embodiment of the present application. The point cloud testing system 30 includes a point cloud testing device 301. The point cloud testing device 301 has computing capabilities and can calculate the true value and volume target of the point cloud testing device 301. The target point cloud is matched to obtain the matching situation between the point cloud of the volume target output by the DUT and the true value of the volume target.

As shown in FIG3 , the point cloud test device 301 includes a data matching module, and the above matching operation can be completed by the data matching module. Optionally, the test device also includes one or more of a data calculation module, a data statistics module, and a visualization module. The data calculation module and the data statistics module are used to evaluate the detection capability of the device under test based on the matching results. The visualization module is used to output a test report, and the test report can indicate the detection capability of the device under test.

Optionally, the point cloud testing system 30 further comprises a data preprocessing module. The data preprocessing module can preprocess the data collected by the test vehicle. The data collected by the test vehicle includes an initial point cloud and optionally also includes true value data (as shown in FIG. 2 ).

In conjunction with FIG3 and FIG2 , the test vehicle is loaded with a DUT, and the DUT can detect the target to be tested set in the test field, and the target to be tested is a body target. Optionally, a data storage module is provided on the test vehicle, and the data collected by the test vehicle can be stored in the data storage module. The data preprocessing module can obtain the data collected by the test vehicle from the data storage module, for example, by communication, or by copying.

In some scenarios, the data preprocessing module may be included in the point cloud testing device 301. Alternatively, the data preprocessing module may also be located outside the point cloud testing device 301.

In one possible implementation, the data preprocessing module can be located in a data center (DC), and the point cloud testing device 301 can obtain the preprocessed point cloud or the true value data from the DC.

In addition, the names of the devices and modules in the embodiments of the present application are only examples. During the specific implementation process, the names of the devices, modules, etc. can be replaced arbitrarily.

The method of the embodiment of the present application is introduced below. Please refer to Figure 4, which is a flow chart of a point cloud testing method provided by the embodiment of the present application. Optionally, the method can be implemented based on the system shown in Figure 3.

The point cloud testing method shown in FIG4 may include one or more steps from step S401 to step S404. It should be understood that for the convenience of description, the description is given in the order of S401 to S404, and it is not intended to limit the execution to the above order. The embodiment of the present application does not limit the execution order, execution time, execution number, etc. of the above one or more steps. S401 to step S404 are as follows:

Step S401: The point cloud testing device obtains true value data and point cloud.

Among them, the point cloud testing device is a device with computing capabilities, such as a server, a personal computer (PC), or an intelligent terminal. When the point cloud testing device is implemented by a server, the number of servers used to implement its functions may be one or more (such as a server cluster). In some possible schemes, the point cloud testing device may be implemented by a software functional unit. Exemplarily, the point cloud testing device may be implemented by a virtual machine, a container, a cloud, etc. Among them, a virtual machine is a computer system with complete hardware system functions and running in an isolated environment simulated by software. A container is an isolated environment obtained by packaging applications and application dependency packages. The cloud is a software platform that uses application virtualization technology, which enables one or more software and applications to be developed and run in an independent virtualized environment.

The true value data is the true value (or reference true value) of the volume target. For example, the true value data can be obtained by detecting the volume target by a true value system. For another example, the true value data can also be annotated by a user or corrected by an artificial intelligence program to reduce the error between the true value data and the actual state of the volume target.

A point cloud is a collection of points (i.e., sampling points), which contains one or more sampling points. A sampling point in the collection usually represents a set of data, which can indicate features such as coordinates, distance, intensity, speed, reflectivity, or color.

In the scenario of performing point cloud testing on the DUT, the point cloud is the detection result obtained by the DUT detecting the volume target. Exemplarily, the DUT can transmit a detection signal and receive an echo of the detection signal, which can be processed to obtain a point cloud.

The aforementioned true value data and point cloud are both data about volume targets. Optionally, the number of volume targets can be one or more. In some embodiments, the number of volume targets is described as at least one. It should be understood that when the detection device detects the field of view, During the process, at certain moments, the field of view of the detection device may not cover or not completely cover the volume target due to the angle, movement route, etc. However, the above special circumstances do not affect the detection of the volume target by the detection device when its field of view covers the volume target.

In a possible implementation, the coordinates of the truth data and the point cloud are aligned. Exemplarily, the truth data and the point cloud are obtained by a truth system and a DUT detection installed on the vehicle, respectively, and the origin of the truth data and the point cloud can be converted to the rear axle center of the vehicle.

Please refer to Figure 5, which is a schematic diagram of a true value data and a point cloud provided by an embodiment of the present application, wherein the true value data is shown in part (a) of Figure 5, and the point cloud is shown in part (b) of Figure 5, and the coordinate axes of the two are aligned, that is, they have the same origin. Furthermore, the directions of their coordinate axes are also aligned. Aligning the coordinate axes can reduce the complexity of calculation, improve the accuracy of matching, and thus improve the accuracy of point cloud testing.

As shown in Figure 5, the true value data is the reference true value of the volume target, and the true value data can also contain multiple points. For the sake of distinction, Figure 5 represents the points in the true value as solid black dots, and represents the points in the point cloud obtained by the DUT as hollow dots. Of course, this is only to facilitate the distinction between the true value and the point cloud to be tested, and does not indicate any difference in presentation method or data content between the two. In addition, in order to facilitate the identification of the contour of the volume target, the contour line of the volume target is represented by a dotted line in Figure 5. In actual implementation, the contour line of the volume target may not necessarily exist in the true value and/or point cloud.

In a possible implementation, the time of the true value data and the point cloud is aligned. For example, the true value data includes A frames from the first moment to the second moment, A is an integer and A>0; the point cloud includes B frames from the third moment to the fourth moment, B is an integer and B>0. When the two are aligned in time, for any frame in the B frames, the true value frame closest to it in terms of timestamp can be found.

Optionally, the frame rates of the true value data and the point cloud may be the same or different. The frame rate is usually used to describe the number of frames per unit time, and each frame may be data obtained when the detection device completes a detection of the field of view. For example, the frame rate of the point cloud may be 120 frames per second, and similarly, the frame rate of the true value data may also be 120 frames per second. For another example, the frame rate of the point cloud may be not less than 100 frames per second.

In a possible implementation, the true value data and the point cloud are preprocessed. Exemplarily, the data preprocessing includes time alignment, coordinate conversion, or format conversion of the point cloud and the true value data, and the preprocessed data is provided to the point cloud testing device for point cloud testing.

Optionally, the preprocessing may be performed by a point cloud testing device. For example, the point cloud testing device preprocesses the initial true value data and the initial point cloud to obtain the aforementioned true value data and point cloud.

Alternatively, the preprocessing can be performed by other modules or devices. In the system shown in FIG3 , the preprocessing module provides the initial true value data and the initial point cloud to the point cloud testing device after preprocessing, and accordingly, the point cloud testing device can obtain the true value data and the point cloud.

Step S402: The point cloud testing device projects the true value data to obtain two-dimensional true value data.

Since the true value data represents the true value of the volume target, and the true value of the volume target is three-dimensional, projection refers to projecting the true value data onto a plane.

As an example of projection, as shown in part (a) of Figure 5, for one of the points F1 in the true value data, its position in the Cartesian coordinate system can be expressed as F1 ( _x1 , _y1 , _z1 ). Please refer to Figure 6, which is a schematic diagram of two-dimensional true value data provided in an embodiment of the present application. The two-dimensional true value data shown in Figure 6 is obtained by projecting the true value data shown in part (a) of Figure 5. Point F1 is projected onto the XY plane to obtain point F1' ( _x1 , _y1 ). Optionally, during the projection of F1, the data on its Z-axis dimension is discarded or set to a preset value (such as 0).

It is understandable that the points after projection correspond to the points before projection one by one. That is, the projection process does not generate new points. For F1' in a two-dimensional true value data, the corresponding point F1 can be found in the true value data.

In a possible implementation, the point cloud testing device projects the true value data onto a horizontal plane. A horizontal plane refers to a relatively horizontal plane. For example, a three-dimensional Cartesian coordinate system is established based on the origin, and three mutually perpendicular axes are drawn through the origin, namely: the x-axis (horizontal axis), the y-axis (longitudinal axis), and the z-axis (vertical axis). The three-dimensional Cartesian coordinate system includes three planes, namely, the X-Y plane, the Y-Z plane, and the X-Z plane. The horizontal plane can be one of the planes, such as the X-Y plane.

Optionally, the origin, X-axis, Y-axis and Z-axis can be defined by the user or manufacturer. As a possible example, for example, the truth value and the point cloud are obtained by the truth value system and DUT detection installed on the vehicle, respectively. The origin can be the center of the rear axle of the vehicle, the Y-axis can be the front direction of the vehicle, and the X-axis can be the side direction of the vehicle. Of course, the above parameters can be defined in other ways during the specific implementation process.

In addition, the Cartesian coordinate system is used to list the dimensions for ease of understanding. In the specific implementation, the coordinate system may also be a spherical coordinate system, a polar coordinate system, etc. This application does not strictly limit the coordinate system, projection plane, etc. used for projection.

In a possible implementation, the two-dimensional true value data obtained by projection may include multiple frames. For ease of description, the frames included in the two-dimensional true value data are referred to as true value frames in each embodiment of the present application. Furthermore, since the two-dimensional true value data is obtained by projecting the true value data, it also includes multiple frames at multiple moments, which are referred to as original true value frames for ease of distinction.

Step S403: The point cloud testing device projects the point cloud to obtain a two-dimensional point cloud.

As an example of projection, as shown in part (b) of FIG. 5 , there is a point L1 (x ₂ , y ₂ , z ₂ ) in the point cloud. Please refer to FIG. 7 , which is a schematic diagram of a two-dimensional point cloud provided in an embodiment of the present application. The two-dimensional point cloud shown in FIG. 7 is obtained by projecting the point cloud shown in part (b) of FIG. 5 . L1 is projected onto the XY plane to obtain point L1' (x ₂ , y ₂ ). Optionally, during the projection of L1, the data on its Z-axis dimension is discarded or set to a preset value (such as 0).

In a possible implementation, the point cloud testing device projects the point cloud onto a horizontal plane to obtain a two-dimensional point cloud. The horizontal plane is, for example, an X-Y plane. For related descriptions, reference may be made to step S402, which will not be repeated here.

Optionally, the projected two-dimensional point cloud may include multiple frames. For ease of description, the embodiments of the present application refer to the frames included in the two-dimensional point cloud as point cloud frames. Furthermore, since the two-dimensional point cloud is obtained by projecting the point cloud, it also includes multiple frames at multiple times, which are referred to as original point cloud frames for ease of distinction.

In a possible implementation, the two-dimensional point cloud and the two-dimensional truth are time-aligned. Figure 8 is a schematic diagram of a possible point cloud frame and a truth frame provided in an embodiment of the present application, the two-dimensional truth data includes truth frames such as truth frame #0, truth frame #1, truth frame #2, and truth frame #3 (the number is only an example), and the two-dimensional point cloud includes point cloud frame #0, point cloud frame #1, point cloud frame #2, point cloud frame #3 and other point cloud frames (the number is only an example). Exemplarily, the truth frame aligned with point cloud frame #0 in timestamp is truth frame #0, and similarly, the truth frame aligned with point cloud frame #1 in timestamp is truth frame #1, and so on for other cases. Of course, the above numbering is only an example and is not intended to be a limitation of the embodiments of the present application.

Optionally, when the frame rate of the 2D true value data is different from the frame rate of the 2D point cloud, the point cloud frame and the true value frame may not correspond one to one. In this case, the subsequent matching process can find the true value frame with the closest timestamp to the point cloud frame for matching.

Step S404: The point cloud testing device matches the two-dimensional point cloud with the two-dimensional true value data to obtain a matching result set.

Matching refers to the process of verifying whether the sampling point and the true value of the volume target can correspond (or be associated). For the sampling points in the two-dimensional point cloud, they are matched with the two-dimensional true value data to determine whether the sampling points can correspond (or be associated) to the true value of a certain volume target.

The matching result set may include one or more of the following matching results: matched sampling points, unmatched sampling points, and unmatched true values, etc. Among them, the matched sampling points are sampling points that successfully match the true value of the volume target, the unmatched sampling points are point clouds that do not successfully match the true value of the volume target, and the unmatched true value is the true value that does not successfully match any sampling point.

As a possible implementation, the number of matched point clouds is usually positively correlated with the quality of the point clouds, and the number of unmatched point clouds is positively correlated with the quality of the point clouds. The number of unmatched true values is negatively correlated with the quality of the point cloud. It can be seen that the matching results preliminarily reflect the accuracy of the point cloud, which is conducive to the subsequent classification test of different categories of matching results and improves the richness and accuracy of the point cloud test.

In a possible implementation, when matching the two-dimensional point cloud and the two-dimensional true value, the matching is performed sequentially by frame. Optionally, the matching can be performed sequentially by true value frame, or sequentially by point cloud frame.

As an example of sequential matching according to point cloud frames, taking the point cloud frame shown in Figure 8 as an example, the point cloud test device matches point cloud frame #0 with the true value frame at the same time (i.e., true value frame #0), and matches point cloud frame #1 with the true value frame at the same time (i.e., true value frame #1), and the rest of the point cloud frames are matched in the same way. It is understandable that if the frame rate of the two-dimensional point cloud is lower than the frame rate of the two-dimensional true value data, some true value frames may not be matched; if the frame rate of the two-dimensional point cloud is lower than the frame rate of the two-dimensional true value data, multiple point cloud frames may be matched to the same true value frame. The method of matching according to point cloud frames is mainly based on point cloud frames, which can avoid the occurrence of missing point cloud frames and facilitate the testing of the number of points and detection accuracy of the point cloud.

As an example of matching true value frames in sequence, taking the frame shown in Figure 8 as an example, the point cloud testing device matches the true value frame #0 with the point cloud frame at the same time (i.e., point cloud frame #0), and matches the true value frame #1 with the point cloud frame at the same time (i.e., point cloud frame #1), and the same goes for the remaining true value frames.

The following are two possible ways to match 2D point clouds with 2D ground truth data:

Implementation method 1, when matching, use the two-dimensional true value of the volume target (that is, the true value of the volume target after projection) for matching. Taking point cloud frame #0 as an example, if the sampling point in point cloud frame #0 (for convenience of description, it is called sampling point P1) coincides with the true value (or a point in the true value) in the volume target or the distance is less than the preset matching distance threshold, then the sampling point P1 belongs to the matching sampling point. If the sampling point in point cloud frame #0 (for convenience of description, it is called sampling point P2) does not coincide with the true value (or a point in the true value) of any volume target, or the distance with the true value (or a point in the true value) of any volume target is greater than the preset matching distance threshold, then the sampling point P2 belongs to the unmatched sampling point. If any of the sampling points in point cloud frame #0 does not successfully match a certain volume target (for convenience of distinction, it is called Target1), then Target1 belongs to the unmatched true value (or the unmatched true value under point cloud frame #0).

Implementation method 2: The point cloud testing device establishes a truth value frame according to the two-dimensional truth value data, and obtains a matching result according to the truth value frame and the sampling point. For example, the matching result is obtained according to the position of the sampling point within the range of the truth value frame. For another example, the matching result is obtained according to the distance between the sampling point and the truth value frame. Optionally, the size of the truth value frame can be designed according to requirements, for example, related to the size of the truth value of the volume target.

As a possible example, taking the matching of true value frame #0 as an example, the point cloud testing device determines the truth frame in the truth frame #0. According to the range of the truth frame and the position of the sampling point in the point cloud frame #0, a subset of matching results is obtained. Among them, the point cloud frame #0 and the truth frame #0 are timestamp aligned (or located at the same time), or the truth frame #0 is the closest truth frame to the point cloud frame #0, or the point cloud frame #0 is the closest point cloud frame to the truth frame #0. Optionally, the number of truth frames is usually the same as the number of volume targets, and one truth frame corresponds to one volume target.

Fig. 9 is a schematic diagram of a truth value frame provided by an embodiment of the present application, which is exemplified as a truth value frame in truth value frame #0. There are multiple frames, which are conveniently distinguished and represented as truth value frame C1, truth value frame C2, truth value frame C3 and truth value frame C4. It is not difficult to see from Fig. 1 that truth value frame C1 corresponds to body target T1, truth value frame C2 corresponds to body target T2, truth value frame C3 corresponds to body target T3, and truth value frame C4 corresponds to body target T4.

As a possible example, during matching, the point cloud test device searches for sampling points in the corresponding point cloud frame that are in the true value frame, and the match is successful if the sampling points fall into the true value frame. Understandably, the following matching results may appear during matching: for a certain sampling point, it may fall into one or more true value frames, or it may not fall into any true value frame; and for a certain true value frame, its range may contain one or more sampling points, or it may not contain the sampling point.

Please refer to FIG. 10, which is a schematic diagram of a matching result provided in an embodiment of the present application, which shows the matching result of point cloud frame #0, including the following three categories:

Category 1, matching sampling points. For the first sampling point in point cloud frame #0, if the first sampling point falls into the first truth box (the first truth box corresponds to the first volume target), the first sampling point belongs to the matching sampling point, and the first sampling point matches the truth value of the first volume target (or matches the first volume target). As shown in FIG10 , the first sampling point is, for example, point P1, which falls into the truth box C1, that is, it matches the truth value of volume target T1 (or matches volume target T1); the first sampling point can also be point P3, which falls into the truth box C2, that is, it matches the truth value of volume target T2 (or matches volume target T2); the first sampling point can also be point P4, which falls into the truth box C3, that is, it matches the truth value of volume target T3 (or matches volume target T3).

Category 2, unmatched sampling point. For the second sampling point in point cloud frame #0, if the second sampling point does not fall into any of the truth value frames, the second sampling point belongs to an unmatched sampling point. As shown in FIG10 , the second sampling point, such as point P2, does not fall into any of the four truth value frames and belongs to an unmatched sampling point.

Category 3, unmatched truth value. If any sampling point in point cloud frame #3 does not fall into the first truth value box, the truth value corresponding to the first truth value box belongs to the unmatched truth value. As shown in Figure 10, none of the sampling points in point cloud frame #0 falls into the truth value box C4, so the truth value of volume target T4 is the unmatched truth value.

The results shown in FIG10 above are only examples. In some scenarios, more or fewer categories of results may be obtained during matching, which will not be listed here one by one.

Since there may be multiple truth value frames, there may be a situation where a sampling point can match multiple truth value frames during matching. In a possible implementation, when a sampling point (referred to as a third sampling point for easy distinction) falls into multiple truth value frames, the true value (or volume target) of the volume target matched by the third sampling point can be determined by the positional relationship between the third sampling point and the true value of the volume target. The positional relationship may be distance, inclusion, or overlap, etc.

It should be noted that the true values of the sampling points and volume targets used in determining the positional relationship may be projected (i.e., belonging to a two-dimensional point cloud and two-dimensional true value data, respectively), or may be unprojected (i.e., belonging to a point cloud and true value data).

In a possible implementation, taking the first point cloud frame as an example, when the first point cloud frame includes the third sampling point and the third sampling point falls into at least two truth value frames, the point cloud testing device determines the true value of the volume target matching the third sampling point according to the position between the third sampling point and the true value of the volume target corresponding to the at least two truth value frames. The first point cloud frame and the third sampling point here are both for distinguishing and representing a certain quantity, and are not used as a limitation of order, importance, etc.

In a possible implementation, the test device can determine the true value of the volume target matched by the sampling point in the following manner: establish a point pair (or corresponding point pair) with the true value of the volume target corresponding to at least two truth value frames and the third sampling point, construct a distance matrix based on the point pair, obtain the distance between the true value and the third sampling point, and use the true value with the closest distance as the true value matched with the third sampling point. By constructing the distance matrix, the distance relationship between the sampling point and the true value can be determined more accurately, the true value of the volume target matched with the sampling point can be determined, and the accuracy of the point cloud test can be improved.

In a possible implementation, the point cloud test device can also evaluate the accuracy, false alarm, missed detection, etc. of the point cloud obtained by the DUT through the matching result set. The accuracy can include one or more of the number of point clouds, distance measurement accuracy, speed measurement accuracy, or height accuracy.

In the embodiment shown in FIG5 , the point cloud test device matches the true value of the volume target and the point cloud of the volume target to obtain a set of matching results between the point cloud of the volume target output by the DUT and the true value of the volume target. Since the volume target is closer to the actual target, the quality of the point cloud output by the detection device can be more accurately tested through the embodiment of the present application, which is conducive to evaluating the detection capability of the detection device. Moreover, by automatically comparing the true value of the volume target and the point cloud of the volume target to obtain the matching result, the evaluation error can be significantly reduced, and the test accuracy and test efficiency can be improved.

In addition, in the embodiment shown in FIG5 , both the true value data and the point cloud are processed into two-dimensional data, and the matching is performed based on the two-dimensional data. On the one hand, the amount of calculation during matching is saved, and the test efficiency is improved. On the other hand, the evaluation of the detection device mainly focuses on its distance measurement capability, speed measurement capability and angle resolution capability, which are related to the detection results. The data in the longitudinal and transverse directions are more relevant, so a 2D projection of the data can be used to test the quality of the point cloud output by the detection device without significant loss of accuracy.

Moreover, considering that the height accuracy of the detection device in some scenarios is low, projecting the data onto the X-Y plane for matching can reduce the matching error caused by the low height accuracy and improve the accuracy of test items such as ranging and speed measurement.

The above describes the point cloud test method for matching after projection. In some possible implementations, when matching the point cloud and the true value data, the matching can also be performed directly in three dimensions without projection.

Please refer to Figure 11, which is a flow chart of another point cloud testing method provided in an embodiment of the present application. Optionally, the method can be implemented based on the system shown in Figure 3.

The point cloud testing method shown in FIG11 may include one or more steps from step S1101 to step S1103. It should be understood that for the convenience of description, the order from S1101 to S1103 is described here, and it is not intended to limit the execution to the above order. The embodiment of the present application does not limit the execution order, execution time, execution number, etc. of the above one or more steps. S1101 to step S1103 are as follows:

Step S1101: The point cloud testing device obtains true value data and point cloud. See step S401 for details.

Optionally, the true value data may include multiple true value frames (or original true value frames), and the point cloud may include multiple point cloud frames (or original point cloud frames). For related descriptions, please refer to the above descriptions of the true value frame and the point cloud frame, but the point cloud frame and the true value frame in this embodiment are not projected.

The time alignment and/or coordinate alignment of the true value data and the point cloud. For example, the true value frame #0 corresponds to the point cloud frame #0. For details, please refer to the relevant description of FIG. 8.

Step S1102: The point cloud testing device establishes a three-dimensional matching box based on the true value data.

Optionally, the number of 3D truth boxes is usually the same as the number of volume targets, and one 3D truth box corresponds to one volume target.

Please refer to Figure 12, which is a schematic diagram of a three-dimensional truth box provided in an embodiment of the present application. The three-dimensional truth box is exemplified as a three-dimensional truth box in truth frame #0. There are multiple boxes, which are easily distinguished in Figure 12 as truth box D1, truth box D2, truth box D3 and truth box D4, corresponding to body targets T1-T4 respectively.

Step S1103: The point cloud testing device matches the point cloud with the three-dimensional matching box to obtain a matching result set.

The matching result set includes at least one of the following three types of matching results: matching sampling points, unmatched sampling points, and unmatched true values.

As a possible example, taking the matching of the true value frame #0 as an example, the point cloud testing device determines the three-dimensional true value frame in the true value frame #0. According to the range of the three-dimensional true value frame and the position of the sampling point in the point cloud frame #0, a matching result subset is obtained. Among them, the point cloud frame #0 and the true value frame #0 are time stamp aligned (or located at the same time), or the true value frame #0 is the closest true value frame to the point cloud frame #0, or the point cloud frame #0 is the closest point cloud frame to the true value frame #0.

During matching, the point cloud test device searches for sampling points in the corresponding point cloud frame that are in the true value frame. If the sampling points fall into the true value frame, the match is successful. Understandably, the following matching results may appear during matching: for a certain sampling point, it may be included in one or more true value frames, or it may be included in any true value frame; and for a certain true value frame, its range may contain one or more sampling points, or it may not contain the sampling points.

Please refer to FIG. 13, which is a schematic diagram of a matching result provided in an embodiment of the present application, which shows the matching result of point cloud frame #0, including the following three categories:

Category 1, matching sampling points. For the first sampling point in point cloud frame #0, if the first sampling point is included in the first truth box (the first truth box corresponds to the first volume target), then the first sampling point belongs to the matching sampling point and the first sampling point matches the truth value of the first volume target (or matches the first volume target). As shown in FIG. 13 , the first sampling point, such as point P5, is included in the truth box D1, that is, it matches the truth value of volume target T1 (or matches volume target T1).

Category 2, unmatched sampling point. For the second sampling point in point cloud frame #0, if the second sampling point does not fall into any of the truth value frames, the second sampling point belongs to an unmatched sampling point. As shown in FIG13 , the second sampling point, such as point P6, does not fall into any of the four truth value frames and belongs to an unmatched sampling point.

Category 3, unmatched truth value. If any sampling point in point cloud frame #3 does not fall into the first truth value box, the truth value corresponding to the first truth value box belongs to the unmatched truth value. As shown in Figure 13, the truth value box D4 does not contain any sampling point in point cloud frame #0, so the truth value corresponding to volume target T4 is the unmatched truth value.

In a possible implementation, the point cloud testing device may also evaluate the accuracy, false alarms, missed detections, etc. of the point cloud obtained by the DUT through a matching result set.

For related descriptions, reference may also be made to the description in step S404 , but the point cloud frame and the true value frame in this embodiment are not projected.

In the embodiment shown in FIG11 , the point cloud test device matches the true value of the volume target and the point cloud of the volume target to obtain a set of matching results between the point cloud of the volume target output by the DUT and the true value of the volume target. Since the volume target is closer to the actual target, the quality of the point cloud output by the detection device can be more accurately tested through the embodiment of the present application, which is conducive to evaluating the detection capability of the detection device. Moreover, by automatically comparing the true value of the volume target and the point cloud of the volume target to obtain the matching result, the evaluation error can be significantly reduced, and the test accuracy and test efficiency can be improved.

In addition, in the embodiment of the present application, the point cloud is matched with the three-dimensional matching box, and the positional relationship between the point cloud and the volume target can be accurately calculated and the matching result can be obtained, and the accuracy of the point cloud test is high.

In the above embodiment, the point cloud testing device can obtain a matching result set. The following describes a possible design for evaluating the point cloud of the DUT based on part or all of the matching results in the matching result set.

As a possible design, matching sampling points can be used to perform accuracy testing on the DUT output point cloud. Specifically, the point cloud test device obtains accuracy evaluation data about the DUT based on the matching sampling points in the matching result set. The accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy, and height accuracy. The following describes several test items separately:

Test item 1, number of matching sampling points. The number of matching sampling points is the number of sampling points that match the true value of the volume target. The number of point clouds of the DUT can reflect the algorithm processing capability.

As a possible implementation, the number of matching sampling points may be calculated in units of a single target. For example, taking the fourth target as an example, the point cloud testing device obtains the number of matching sampling points about the fourth target based on the number of sampling points in the matching sampling points that match the true value of the fourth target.

Optionally, there may be multiple possible calculation methods for the number of matching sampling points of a single target. For example, calculation method 1 is: accumulating the number of sampling points that match the true value of the fourth body target in multiple point cloud frames. For another example, calculation method 2 is: listing the number of sampling points that match the true value of the fourth body target in each point cloud frame. For another example, calculation method 3 is: calculating the average value of the number of sampling points that match the true value of the fourth body target in multiple point cloud frames.

Please refer to Figure 14, which is a schematic diagram of a possible number of matching sampling points provided by an embodiment of the present application. As shown in Figure 14, the truth box C1 (the truth box corresponding to the volume target T1) can match the sampling points in point cloud frame #0, point cloud frame #1, point cloud frame #2 and point cloud frame #3. In point cloud frame #0, the number of sampling points in the truth box C1 is 38; in point cloud frame #1, the number of sampling points in the truth box C1 is 39; in point cloud frame #2, the number of sampling points in the truth box C1 is 20; in point cloud frame #3, the number of sampling points in the truth box C1 is 15. When calculation method 1 is used, the number of matching sampling points corresponding to the volume target T1 is 112.

The above is an example of the number of matching sampling points for a body target. If the number is multiple, the number of matching sampling points can also be multiple. As shown in Table 1, the number of matching sampling points corresponding to a volume target provided in an embodiment of the present application includes a sequence number, the ID of the volume target and the number of matching sampling points corresponding to it. For example, the number of matching sampling points corresponding to volume target T1 is 112, and the number of matching sampling points corresponding to volume target T2 is 287. For other situations, see Table 1. Of course, the format, attributes, numbers, etc. of Table 1 are only examples.

Table 1 Number of matching sampling points corresponding to volume targets

In some scenarios, the number of matching sampling points can also be calculated based on multiple targets. For example, the number of matching sampling points of each volume target is accumulated to obtain the number of matching sampling points.

It should be understood that the above is only for the sake of ease of understanding, so the number of matching sampling points is described in a table form, and is not intended to limit the storage form, output form, and transmission form of the number of matching sampling points. In the specific implementation process, the number of matching sampling points can also be indicated by other data formats, such as linked lists, heaps, stacks, database tables, objects, etc., which are not listed one by one here. Similarly, other tables in this application are also exemplary data formats and are not used as strict limitations on the solution of this application.

Test item 2, ranging accuracy. The ranging accuracy can include one or more of the lateral ranging accuracy, radial ranging accuracy, longitudinal ranging accuracy, etc. The ranging accuracy can reflect the accuracy of the DUT's position detection of the body target. The higher the accuracy, the more conducive it is to back-end function processing. For example, the higher the accuracy, the more conducive it is to improving the accuracy of the following functions: collision warning, identification of passable areas, etc.

As a possible implementation, the ranging accuracy may be calculated on a single target basis. For example, taking the second target as an example, the point cloud testing device determines the ranging accuracy of the second target based on the sampling points that match the true value of the second target and the true value of the second target.

In one possible implementation, the matching sampling points include N sampling points that match the true value of the second body target, where N is an integer and N>0. The true value of the second body target includes M corner points, where M is an integer and M>0. Among them, the corner point is a point with particularly prominent attributes in some aspect. Exemplarily, the corner point can be a point located at the intersection of two edges of the body target (or a point close to the intersection), and/or the midpoint of the edge of the body target (or a point close to the midpoint). In some scenarios, the conditions of the corner point can be defined by the user or manufacturer (for example, setting specific conditions for corner point detection). For example, the corner point can be a point obtained by the Harris corner point detection algorithm.

The point cloud test device can evaluate the ranging accuracy of the DUT when detecting a second body target through N sampling points and M corner points.

The following description is made by taking the lateral ranging accuracy of a single target as an example. Please refer to Figure 15, which is a schematic diagram of the distance between a sampling point and a DUT provided in an embodiment of the present application. The N sampling points include the nearest sampling point, and the M corner points include the lateral nearest corner point. The lateral ranging accuracy of the second body target is related to the lateral distance between the nearest sampling point and the DUT (i.e., Xpi shown in Figure 15) and the lateral distance between the lateral nearest corner point and the DUT (i.e., Xcj shown in Figure 15).

For example, the lateral ranging accuracy σ _x of the second target satisfies the following formula:

σ _x = |X _pi -X _cj |

It should be understood that "nearest" refers to the distance between the point and a preset point. The above example takes the preset point as the DUT. For example, it can be replaced by other points or other devices during the specific implementation process.

Referring to FIG15 , the nearest sampling point is the sampling point that is closest to the DUT (shortest in radial distance) among the N sampling points. For example, the distances between the N sampling points and the DUT can be expressed as D _p1 , D _p2 , D _p3 , …, D _pN , respectively, where the distance D _pi between the nearest sampling point and the DUT satisfies the following formula:

D _pi =min(D _p1 ,D _p2 ,D _p3 ,…,D _pn )

Similarly, the lateral closest corner point is the corner point with the shortest lateral distance to the DUT among the M corner points. For example, the lateral distances between the M corner points and the DUT can be expressed as X _c1 , X _c2 , X _c3 , …, X _cm , respectively, where the lateral distance X _cj between the lateral closest corner point and the DUT satisfies the following formula:

X _cj =min(X _c1 ,X _c2 ,X _c3 ,…,X _cm )

The following is an example of the mirror ranging accuracy of a single target. Please refer to Figure 16, which is a schematic diagram of the distance between another sampling point and the DUT provided in an embodiment of the present application. Among them, the N sampling points include the nearest sampling point, and the M corner points include the radial nearest corner point. The radial ranging accuracy of the second body target is related to the radial distance between the nearest sampling point and the DUT (i.e., Dpi shown in Figure 16) and the lateral distance between the lateral nearest corner point and the DUT (i.e., Dck shown in Figure 16).

For example, the longitudinal ranging accuracy _σd of the second target satisfies the following formula:

σ _d ＝|D _pi −D _ck |

Wherein, _Dpi is the radial distance between the nearest sampling point and the DUT, and _Dck is the radial distance between the nearest radial corner point and the DUT. It should be understood that "nearest" refers to the distance between the point and a preset point. The above example uses the preset point as the DUT as an example, which can be replaced by other points or other devices during the specific implementation process.

Optionally, the calculation method of the nearest sampling point can refer to the above.

Optionally, the lateral closest corner point is the corner point with the shortest radial distance to the DUT among the M corner points. For example, the radial distances between the M corner points and the DUT can be expressed as D _c1 , D _c2 , D,…, D _cm , respectively, where the radial distance D _ck between the radially closest corner point and the DUT satisfies the following formula:

D _ck =min(D _c1 ,D _c2 ,D,...,D _cm )

The above is explained by taking the ranging accuracy of a body target as an example. In the specific implementation process, if the number of body targets is multiple, the ranging accuracy can also be multiple. As shown in Table 2, the ranging accuracy corresponding to a body target provided in an embodiment of the present application includes a sequence number, an ID of the body target, a lateral ranging accuracy, or a longitudinal ranging accuracy, etc. For example, the lateral ranging accuracy of the DUT with respect to the body target T1 is σ _x1 , and the longitudinal ranging accuracy of the DUT with respect to the body target T1 is σ _d1 . The lateral ranging accuracy of the DUT with respect to the body target T2 is σ _x2 , and the longitudinal ranging accuracy of the DUT with respect to the body target T2 is σ _d2 . For other situations, please refer to Table 2, which will not be described one by one here. Of course, the format, attributes, numbers, etc. of Table 2 are only examples.

Table 2 Ranging accuracy of volume targets

In some scenarios, the ranging accuracy can also be calculated based on multiple targets. For example, the ranging accuracy corresponding to each volume target is averaged, or weighted averaged, to obtain the average ranging accuracy.

Test item 3, speed measurement accuracy. Speed measurement accuracy reflects the accuracy of DUT speed detection, and speed measurement accuracy may affect the decision-making accuracy of intelligent driving functions. Intelligent driving functions include but are not limited to autonomous emergency braking (AEB), lane keeping assist (LKA), adaptive cruise control (ACC), parking assistance (PA), or lane change assist (LCA). Taking ACC as an example, ACC can determine the driving state of the vehicle (such as acceleration or deceleration) based on the speed of the vehicle in front. If the speed measurement of the vehicle in front is inaccurate, it may lead to driving safety.

As a possible implementation, the ranging accuracy may be calculated on a single target basis. Taking the volume target as an example, the point cloud testing device determines the ranging accuracy of the second volume target based on the sampling points matching the true value of the third volume target and the true value of the third volume target.

In a possible implementation, the matching sampling points include K sampling points that match the true value of the third body target, K is an integer and K>0. The K sampling points include the strongest sampling point. Optionally, the strongest sampling point can be a sampling point with the strongest echo energy. The speed measurement accuracy of the third body target is related to the radial velocity of the strongest sampling point and the radial velocity of the true value of the third body target. Since a body target may correspond to multiple points, this implementation uses the strongest sampling point to evaluate the speed measurement error, which can improve the test accuracy of the speed measurement accuracy.

Exemplarily, the velocity accuracy σ _v can be indicated by the absolute value of the radial velocity error between the strongest point radial velocity in the matching point and the reference true value. For example, the velocity accuracy σ _v satisfies the following formula:

σ _v ＝|V _pi −V _t |

R _pi =max(R _p1 ,R _p2 ,R _p3 ,…,R _pK )

Optionally, the true radial velocity of the aforementioned third body target may be replaced by the radial velocity of the third body target.

Test item 4, height accuracy. Height accuracy can reflect the accuracy of the height measurement of the point cloud.

In a possible implementation, the height accuracy may be indicated by the height distribution being a data distribution range and/or a normal value distribution range located in the middle 50% of the height values of the matching sampling points.

The middle 50% of the data is obtained by arranging the height values corresponding to the matching sampling points from small to large and dividing them into four equal parts to obtain three quartiles (values at three split points), and the data between the first quartile and the third quartile is the middle 50% of the data. By analyzing the distribution of the middle 50% of the data, the distribution range of the middle 50% of the data is obtained.

Furthermore, two normal values are obtained by expanding the first quartile and the third quartile by 1.5 times. The height value between these two normal values is the normal value data, and the distribution range of the normal value data is the normal value distribution range.

In some scenarios, the determination of the above data distribution range can be achieved through box plot statistics. Box plot statistics are suitable for data that do not strictly follow the normal distribution, and the quartiles are highly resistant to outliers and can identify outliers relatively objectively.

As another possible design, the unmatched sampling points in the matching result set can be used for false alarm judgment. A false alarm is an event in which the target does not exist but the detection device judges that there is a target and outputs sampling points. A false alarm can correspond to a point cloud that does not match the true value.

In a possible implementation, when judging a false alarm, the unmatched sampling points in the point cloud frame can be tracked (or traced), and a multi-frame association method can be used to determine whether there is a false alarm in the point cloud. Exemplarily, the two-dimensional point cloud includes multiple continuous point cloud frames, and the matching set includes unmatched sampling points. The point cloud testing device determines the false alarm targets in the multiple continuous point cloud frames based on the sampling points in the unmatched sampling points that are located in the multiple continuous point cloud frames.

A possible method for tracking unmatched point clouds is provided below.

If a certain point cloud cluster appears multiple times in multiple consecutive point cloud frames, a false alarm may be generated. Specifically, the point cloud testing device clusters the sampling points in the second point cloud frame among the unmatched sampling points to obtain point cloud clusters, where the number of point cloud clusters can be one or more. The point cloud testing device assigns an initial life value to the first point cloud cluster, and determines the point cloud clusters in Q point cloud frames based on the sampling points in Q (Q is an integer and Q>0) point cloud frames after the second point cloud frame. According to the position of the first point cloud cluster and the position of the point cloud clusters in the Q point cloud frames, the false alarm targets in multiple consecutive point cloud frames are determined. Among them, Q can be a fixed number or a non-fixed number.

In some possible implementations, for a point cloud cluster existing in a certain point cloud frame, the point cloud frame is matched with one or more subsequent point cloud frames. If there is a matching point cloud cluster in the subsequent point cloud frame, the life value of the point cloud cluster is increased, otherwise the life value of the point cloud cluster is reduced; repeat the matching of multiple point cloud frames. If the life value of the point cloud cluster meets the false alarm condition (for example, when it reaches the first threshold), the point cloud cluster forms a false alarm target. If the life value of the point cloud cluster meets the condition for canceling the false alarm (for example, reaching the second threshold or being lower than the third threshold), the point cloud cluster does not form a false alarm target, and the point cloud cluster can be optionally discarded. It should be understood that reaching the first threshold may be higher than or higher than or equal to the first threshold, subject to the specific design. The same applies to the multiple thresholds and conditions below.

Please refer to Figure 17, which is a schematic diagram of a possible unmatched point cloud provided by an embodiment of the present application. The unmatched sampling points in point cloud frame #0 (which can be regarded as the second point cloud frame) can be clustered to obtain point cloud cluster G1 and point cloud cluster G2, wherein the life value of point cloud cluster G1 and the life value of point cloud cluster G2 are both 60 (initial life value). When the life value of the point cloud cluster is greater than or equal to 100, the false alarm condition is met, and when the life value of the point cloud cluster is less than 60, the false alarm elimination condition is met.

For point cloud frame #1, which is the next point cloud frame of point cloud frame #0, the unmatched sampling points in the point cloud frame can be clustered to obtain point cloud cluster G3 and point cloud cluster G4. The point cloud testing device matches the point cloud clusters in point cloud frame #0 according to point cloud clusters G3 and G4. Exemplarily, if point cloud cluster G4 matches point cloud cluster G1, the life value of point cloud cluster G4 (i.e., G1) increases by 20, and the current life value is 80; while point cloud cluster G2 has no matching point cloud cluster in point cloud frame #1, the life value of point cloud cluster G2 decreases by 20, and the life value of point cloud cluster G2 is 40. When the third threshold is 60, since point cloud cluster G2 is already below 60, it does not form a false alarm target and is discarded. Point cloud cluster G3 is a newly discovered point cloud cluster in point cloud frame #0 and is assigned a life value of 60 (initial life value).

For point cloud frame #2, which is the next point cloud frame of point cloud frame #1, the unmatched sampling points in the point cloud frame can be clustered to obtain point cloud cluster G5. The point cloud testing device matches the point cloud cluster G5 with the point cloud cluster in point cloud frame #1. Exemplarily, if point cloud cluster G5 matches point cloud cluster G4 (i.e., G1), the life value of point cloud cluster G5 (i.e., G1) increases by 20, and the current life value is 100. When the first threshold is 100, since the life value of point cloud cluster G5 (i.e., G1) reaches the first threshold, point cloud cluster G5 forms a false alarm target. Further, the point cloud testing device matches the point cloud cluster 5 with the point cloud cluster in point cloud frame #1. If point cloud cluster G3 has no matching point cloud cluster in point cloud frame #2, the life value of point cloud cluster G2 is reduced by 20, and the life value of point cloud cluster G3 is 40, which is already lower than 60, and does not form a false alarm target and is discarded.

For point cloud frame #3, it is the next point cloud frame after point cloud frame #2. The unmatched sampling points in the point cloud frame can be clustered to obtain point cloud cluster G6. The point cloud testing device matches the point cloud cluster G6 with the point cloud cluster in point cloud frame #2. Exemplarily, if point cloud cluster G5 matches point cloud cluster G5 (i.e., G1), the life value of point cloud cluster G6 (i.e., G1) increases by 20, and the current life value is 120, reaching the first threshold, then point cloud cluster G6 forms a false alarm target.

As a possible implementation method, when matching point cloud clusters, the point cloud cluster frame is determined according to the size of the point cloud cluster in the point cloud frame, and the point cloud cluster frame can enclose the point cloud in the cluster. For the current point cloud frame, if there is a point cloud cluster frame in the next point cloud frame that overlaps with the point cloud cluster frame of the current frame, the overlap matrix between the point cloud cluster frame and the point cloud cluster frame is calculated to establish an inter-frame association. If the point cloud cluster frame in the next point cloud frame successfully matches the point cloud cluster frame of the current frame, the life value of this point cloud cluster increases; otherwise, the life value of the point cloud cluster decreases. By establishing inter-frame associations through matching frames, the computational complexity can be further reduced and the efficiency of point cloud testing can be improved.

It should be noted that the above embodiments are described by taking backward matching as an example, that is, the point cloud clusters in the current point cloud frame are matched with the point cloud clusters in the subsequent point cloud frame. In some possible implementations, the point cloud clusters in the current point cloud frame can also be matched forward, that is, the point cloud clusters in the current point cloud frame are matched with the point cloud clusters in the previous point cloud frame.

The above is an introduction to false alarm judgment in a graphical manner. For ease of understanding, a flow chart of false alarm judgment is provided below. Please refer to Figure 18, which is a flow chart of another false alarm judgment method provided by an embodiment of the present application. The false alarm judgment method can be executed by a point cloud testing device. Specifically, it can include steps S1801 to S1804, as shown in FIG. Down:

Step S1801: point cloud clustering.

The point cloud testing device clusters the unmatched point clouds in the current point cloud frame to obtain point cloud clusters.

Step S1802: Allocate ID and initial life value.

The point cloud testing device assigns an ID (optional) and an initial life value to the point cloud cluster. The ID is, for example, G1 or G2 as shown in FIG17 . Optionally, when point cloud clusters in different point clouds can be matched, they can share the same ID to facilitate false alarm judgment.

The initial health value is 60, for example.

Step S1803: Matching with the point cloud cluster in the next point cloud frame.

As shown in FIG. 17 , when the next frame of point cloud frame #0 is point cloud frame #1, the point cloud clusters in point cloud frame #0 are matched with the point cloud clusters in point cloud frame #1.

Step S1804: If the match is successful, the health value is +20; if the match is unsuccessful, the health value is -20.

As shown in FIG17 , point cloud cluster G4 successfully matches point cloud cluster G1, and the life value of point cloud cluster G4 (ie, G1) increases by 20. Point cloud cluster G2 fails to match in point cloud frame #1, and its life value decreases by 20.

If the life value of the point cloud cluster in the next point cloud frame is ≤60, the point cloud cluster is discarded. For example, if the life value of point cloud cluster G2 is 40, the point cloud cluster G2 is discarded if the condition is met.

If the life value of the point cloud cluster in the next point cloud frame is 100, a false alarm is generated. This cycle is repeated to determine the false alarm target in the point cloud frame.

In the above implementation, since a single sampling point is prone to flickering, the matching complexity is high and the result reliability is low, the above implementation clusters the unmatched sampling points and tracks the unmatched point cloud in the form of point cloud clusters, which not only reduces the complexity of matching, but also greatly improves the reliability and availability of false alarm judgment, and improves the accuracy of point cloud testing.

In a possible implementation manner, the unmatched point cloud may also be used to determine the false alarm rate.

Exemplarily, the false alarm rate ρ satisfies the following formula:

For example, taking Figure 17 as an example, the number of point cloud frames involved in the calculation of false alarm targets is 4, among which the point cloud frames with false alarm targets are point cloud frames #3 and point cloud frames #4, so the false alarm rate is 50%. In this case, although point cloud clusters that form false alarm targets also exist in point cloud frames #0 and point cloud frames #1, the false alarm targets have not been diagnosed at that time, so they are not involved in the calculation of false alarm rate.

Of course, in some scenarios, once the false alarm target is determined, the point cloud frame where the point cloud cluster in the undetermined stage is located is also regarded as the point cloud frame with false alarm. At this time, point cloud frame #0 and point cloud frame #1 can also be regarded as false alarm point cloud frames, that is, the false alarm rate is 100%.

As another possible design, the unmatched true value can be used for missed detection judgment. Missed detection refers to the event that in some cases, the target exists but the radar judges that there is no target and does not output the point cloud. The missed detection of the point cloud can correspond to the true value of the unmatched point cloud.

Optionally, when judging missed detection, it can be determined whether the volume object is blocked. If the volume object is blocked, the missed detection of the volume object does not belong to a valid missed detection.

In a possible implementation, the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame. When judging missed detection, the point cloud testing device determines the suspected missed detection target in the third point cloud frame according to the unmatched true value corresponding to the third point cloud frame, and determines the obscured volume target according to the field of view relationship between at least one volume target and the radar. The point cloud testing device filters the obscured volume targets in the suspected missed detection targets in the third point cloud frame, and determines the third point cloud frame. In this way, the occluded volume targets in the suspected missed detection targets are removed according to the occlusion relationship, thus reducing the missed detection judgment error caused by the occlusion of the volume targets.

In a possible implementation, determining the obscured volume target according to the field of view relationship between at least one volume target and the radar includes:

Please refer to Figure 19, which is a schematic diagram of the position of a body target provided in an embodiment of the present application. The body targets shown in Figure 19 include body target T5, body target T6, body target T7 and body target T8, where the number and ID are only examples. If in a certain point cloud frame, the true values of body target T5, body target T7 and body target T7 are all unmatched true values, the occlusion relationship between the body targets needs to be determined. Among them, the occlusion relationship can be determined by the corner points in the true value of the body target.

As an example of a method for confirming an occlusion relationship, if there are V occluded corner points among the multiple corner points of the fifth body target, the fifth body target is occluded, V is an integer and V>0, wherein the occluded corner point is a corner point where the line connecting the corner points intersects with the edges of other body targets. As shown in FIG19 , taking V=4 as an example, the eight corner points of body target T5 are respectively represented as _A1 to _A8 , wherein the lines connecting _A3 , _A5 , _A7 and _A8 with the DUT all intersect with the edges of body target T8, so the above four corner points are all occluded corner points, and therefore body target T5 belongs to the occluded target. In body target T6, the eight corner points are respectively represented as _B1 to _B8 , and only _B6 is an occluded corner point, so body target _T6 does not belong to the occluded target.

As another example of a method for confirming an occlusion relationship, if the fifth object is an occlusion object, it satisfies the following two conditions: ① The lines connecting the V corner points of the fifth object and the DUT intersect with the edges of other objects, V is an integer and V>0; ② The number of valid edges (see explanation below) in the fifth object is greater than or equal to a fourth threshold. The fourth threshold may be predefined or pre-set. For example, the fourth threshold may be 4, or the fourth threshold may be 1.

For condition ①, V can be equal to the total number of corner points, for example, V = 8. For condition ②, the valid edge can be determined as follows: for any corner point or any edge corner point in the fifth body target (edge corner points refer to corner points located on the edge, such as A2, A4, A5, A7 shown in Figure 19), if the line connecting the corner point (or the edge corner point) and the DUT intersects on any edge of the fifth body target, then the edge where the corner point (or the edge corner point) is located is invalid. If the lines connecting the corner point on the first edge of the fifth body target and the DUT do not intersect with other edges in the fifth body target, then the first edge is a valid edge. For example, as shown in Figure 19, the corner point _A2 intersects with the DUT line at the edge where _A5 is located, so the edge where the corner point _A2 is located is an invalid edge of body target T5. Similarly, the edge where the corner point _A4 is located is an invalid edge. The line connecting corner point _A5 and DUT does not intersect with other edges in volume target T5, so the edge where corner point _A5 is located is a valid edge of volume target T5. Similarly, the edge where corner point _A7 is located is a valid edge.

Taking V=8 and the fourth threshold value of 1 as an example, as shown in FIG19 , the lines connecting the eight corner points in volume target T5 all intersect with the true value of volume target T8, so condition ① is satisfied; the edges where _A5 and _A7 in volume target T5 are located are valid edges, satisfying condition ②. Therefore, volume target T5 is an occluded target.

In another possible implementation, when judging missed detection, the area corresponding to the unmatched true value can be tracked in the point cloud frame, and a multi-frame association method can be used to determine whether the volume target is missed. In this way, the unmatched true value can be tracked in subsequent point cloud frames, which can reduce the missed detection judgment error caused by point cloud flickering, improve the accuracy of false alarm judgment, and improve the accuracy of point cloud testing.

Exemplarily, when there is a missed detection of a sixth object in three consecutive point cloud frames, the sixth object is determined to be a missed detection object.

In yet another possible implementation, the unmatched true value may also be used to determine the missed detection rate.

Exemplarily, the missed detection rate γ satisfies the following formula:

Where n is the number of point cloud frames involved in the missed target calculation, and n_lose is the number of missed point cloud frames. For related descriptions, please refer to the above example of calculating the false alarm rate.

The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

It should be understood that the division of units in the device provided in the embodiments of the present application is only a division of logical functions, and in actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. In addition, the units in the device can be implemented in the form of a processor calling software; for example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit of the device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory inside the device or a memory outside the device. Alternatively, the units in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units may be implemented by designing the hardware circuits, and the hardware circuits may be understood as one or more processors; for example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all of the above units may be implemented by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit may be implemented by a programmable logic device (PLD), and a field programmable gate array (FPGA) may be used as an example, which may include a large number of logic gate circuits, and the connection relationship between the logic gate circuits may be configured by a configuration file, so as to implement the functions of some or all of the above units. All units of the above devices may be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the rest by hardware circuits.

The present application is reasonable, and each unit in the device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

In addition, the units in the above device can be fully or partially integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

Several possible arrangements are listed below.

Please refer to Figure 20, which is a schematic diagram of the structure of a point cloud testing device provided in an embodiment of the present application. Optionally, the point cloud testing device 200 can be an independent device, such as a server. Alternatively, the point cloud testing device 200 can also be a device in an independent device (such as a node), such as a chip or an integrated circuit. The point cloud testing device 200 is used to implement the aforementioned point cloud testing method, such as the point cloud testing method shown in Figure 5, Figure 11, or Figure 18. For example, the point cloud testing device 200 can replace the point cloud testing device 301 in the system shown in Figure 3.

The point cloud testing device 200 shown in FIG. 20 includes a data acquisition module 2001 and a data matching module 2002, wherein:

The data acquisition module 2001 is used to obtain true value data and point cloud, the true value data is the true value of the volume target, and the point cloud is the detection result obtained by the DUT detecting the volume target, and the detection result includes sampling points;

The data matching module 2002 is used to match the point cloud with the true value data to obtain a matching result set, which includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.

In yet another possible implementation, the data matching module 2002 is used to:

Establish a three-dimensional matching box based on the true value data;

In a possible implementation, the data matching module 2002 is used to:

Project the true value data to obtain two-dimensional true value data;

Project the point cloud to obtain a two-dimensional point cloud;

In a possible implementation, the data matching module 2002 is used to:

Project the point cloud to obtain a two-dimensional point cloud, including:

In yet another possible implementation, the time of the true value data and the point cloud is aligned. Further, when the true value data and the point cloud are projected as two-dimensional data, the time of the two-dimensional true value data and the two-dimensional point cloud is aligned.

In yet another possible implementation, the coordinates of the true value data and the point cloud are aligned.

In yet another possible implementation, the two-dimensional truth data includes a plurality of truth frames, and the two-dimensional point cloud includes a plurality of point cloud frames;

The data matching module 2002 is further used for:

In yet another possible implementation, the at least one truth box includes a first truth box corresponding to a first volume target, and the first volume target belongs to the at least one volume target;

In yet another possible implementation, the number of at least one truth box is greater than or equal to 2.

The data matching module 2002 is further used for:

In yet another possible implementation, the data matching module 2002 is further configured to:

In yet another possible implementation, the data acquisition module 2001 is further configured to:

Preprocess the initial truth value and initial point cloud to obtain the truth value data and point cloud. The preprocessing may include the following: One or more of the processing: time alignment, coordinate conversion, format conversion, etc. Preprocessing can improve the correspondence between the true value data and the point cloud, reduce the complexity of matching, and improve the efficiency of point cloud testing.

In another possible implementation, the point cloud testing device 200 further includes a data calculation module 2003, which is used to evaluate the accuracy, false alarms, and missed detections of the point cloud through a set of matching results.

In another possible implementation, the point cloud testing device further includes a data calculation module, which is used to obtain accuracy evaluation data about the DUT based on the matching sampling points in the matching result set. The accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy, and height accuracy.

In yet another possible implementation, the data calculation module 2003 is further configured to:

In another possible implementation, the matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M＞0, and N is an integer and N＞0.

In another possible implementation, the N sampling points include the nearest sampling point, and the M corner points include the lateral nearest corner point. Further, the ranging accuracy includes the lateral ranging accuracy with respect to the second object, and the lateral ranging accuracy with respect to the second object is related to the lateral distance between the nearest sampling point and the DUT and the radial distance between the radial nearest corner point and the DUT.

In yet another possible implementation, the lateral ranging accuracy σ _x of the second body target satisfies the following formula:

σ _x = |X _pi -X _cj |

In another possible implementation, the N sampling points include the nearest sampling point, and the M corner points include the radial nearest corner point. Further, the ranging accuracy includes the longitudinal ranging accuracy with respect to the second object, and the longitudinal ranging accuracy with respect to the second object is related to the radial distance between the nearest sampling point and the DUT and the radial distance between the radial nearest corner point and the DUT.

In another possible implementation, the longitudinal distance measurement accuracy _σd of the fourth target satisfies the following formula:

σ _d ＝|D _pi −D _ck |

In yet another possible implementation, the velocity measurement accuracy includes velocity measurement accuracy with respect to a third-body target;

σ _v ＝|V _pi −V _t |

R _pi =max(R _p1 ,R _p2 ,R _p3 ,…,R _pK )

In yet another possible implementation, unmatched sampling points may be used for false alarm determination.

In another possible implementation, the point cloud testing device further includes a data calculation module, and the data calculation module is further used to:

In yet another possible implementation, the plurality of continuous point cloud frames include a second point cloud frame and Q point cloud frames after the second point cloud frame, where Q is an integer and Q>0;

The data calculation module 2003 is also used for:

In yet another possible implementation, the unmatched point cloud may also be used to determine the false alarm rate.

Exemplarily, the false alarm rate ρ satisfies the following formula:

In another possible implementation, the unmatched true value can be used for missed detection judgment. Missed detection refers to the event that in some cases, a target exists but the radar judges that there is no target and does not output a point cloud. The missed detection of the point cloud can correspond to the true value of the unmatched point cloud.

In yet another possible implementation, the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame;

The data calculation module 2003 is also used for:

In another possible implementation, when making missed detection judgments, the point cloud testing device 210 may track the area corresponding to the unmatched true value in the point cloud frame, and use a multi-frame association method to determine whether the volume target is missed.

Exemplarily, the missed detection rate γ satisfies the following formula:

Please refer to Figure 21, which is a structural diagram of a computing device provided in an embodiment of the present application.

The computing device 210 may be an independent device, such as a node, or a device included in an independent device, such as a chip, a software module, or an integrated circuit. The computing device 210 may include at least one processor 2101 and a communication interface 2102. Optionally, it may also include at least one memory 2103. Further optionally, it may also include a connection line 2104, wherein the processor 2101, the communication interface 2102 and/or the memory 2103 are connected via the connection line 2104, and/or communicate with each other via the connection line 2104 to transmit control signals and/or data signals.

Among them, the processor 2101 is a module for performing arithmetic operations and/or logical operations. In one implementation, the processor can be a circuit with the ability to read and run instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP); in another implementation, the processor can realize certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a processor that is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

The communication interface 2102 may be used to provide information input or output for at least one processor, or to receive externally transmitted signals and/or to transmit externally transmitted signals.

For example, the communication interface 2102 may include an interface circuit. For example, the communication interface 2102 may include a wired link interface such as an Ethernet cable, or a wireless link (Wi-Fi, Bluetooth, general wireless transmission, vehicle-mounted short-range communication technology, and other short-range wireless communication technologies, etc.) interface.

Optionally, the communication interface 2102 may further include a radio frequency transmitter, an antenna, etc. When the communication interface 2102 includes an antenna, the number of antennas may be one or more.

As a possible design, if the computing device 210 is an independent device, the communication interface 2102 may include a receiver and a transmitter. The receiver and the transmitter may be the same component or different components. When the receiver and the transmitter are the same component, the component may be referred to as a transceiver.

As another possible design, if the computing device 210 is a chip or a circuit, the communication interface 2102 may include an input interface and an output interface, and the input interface and the output interface may be the same interface, or may be different interfaces.

Optionally, the functions of the communication interface 2102 may be implemented by a transceiver circuit or a dedicated transceiver chip.

The memory 2103 is used to provide a storage space in which data such as an operating system and a computer program can be stored. The memory 2103 can be a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM), etc., or a combination of multiple thereof.

The functions and actions of the modules or units in the computing device 210 listed above are only for illustrative purposes.

Each functional unit in the computing device 210 may be used to implement the aforementioned point cloud testing method, such as the point cloud testing method shown in FIG. 5 , FIG. 11 , or FIG. 18 .

Optionally, the processor 2101 may be a processor specifically used to execute the aforementioned method (for convenience of distinction, referred to as a dedicated processor), or may be a processor that executes the aforementioned method by calling a computer program (for convenience of distinction, referred to as a dedicated processor). Optionally, the at least one processor may include both a dedicated processor and a general-purpose processor.

Optionally, in the case where the computing device 210 includes at least one memory 2103 , if the processor 2101 implements the aforementioned point cloud testing method by calling a computer program, the computer program may be stored in the memory 2103 .

The embodiment of the present application further provides a chip, which includes a logic circuit and a communication interface. The communication interface is used to receive a signal or send a signal; the logic circuit is used to receive a signal or send a signal through the communication interface. The chip is used to implement the aforementioned point cloud testing method, such as the point cloud testing method shown in FIG. 5, FIG. 11, or FIG. 18.

An embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on at least one processor (or communication device), the aforementioned point cloud testing method is implemented, such as the point cloud testing method shown in Figure 5, Figure 11, or Figure 18.

An embodiment of the present application also provides a computer program product, which includes computer instructions, and the computing instructions are used to implement the aforementioned point cloud testing method, such as the point cloud testing method shown in Figure 5, Figure 11, or Figure 18.

It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items.

For example, at least one of a, b, or c can be represented by: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, and c can be single or multiple. "And/or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and/or B can be represented by: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character "/" generally indicates the previous and next associated objects. It is an "or" relationship.

Furthermore, unless otherwise specified, the ordinal numbers such as "first" and "second" used in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. For example, the first point cloud frame, the second point cloud frame, and the third point cloud frame are only for the convenience of describing the point cloud frames in different implementations, and do not indicate the difference in order, importance, data content, etc. between the two. In some scenarios, the first point cloud frame and the second point cloud frame can be the same point cloud frame.

In the above embodiments, the terms "when..." and "if..." can be interpreted as meaning "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the concept and principle of the present application shall be included in the protection scope of the present application.

A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

Claims

A point cloud testing method, characterized in that the method comprises:

Acquire true value data and a point cloud, wherein the true value data is the true value of at least one volume target, and the point cloud is a detection result obtained by the device under test (DUT) detecting the at least one volume target;

Projecting the true value data to obtain two-dimensional true value data;

Projecting the point cloud to obtain a two-dimensional point cloud;

The two-dimensional point cloud is matched with the two-dimensional true value data to obtain a matching result set, wherein the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.
The method according to claim 1, characterized in that projecting the true value data to obtain two-dimensional true value data comprises:

Projecting the true value data onto a horizontal plane to obtain the two-dimensional true value data;

Projecting the point cloud to obtain a two-dimensional point cloud includes:

The point cloud is projected onto the horizontal plane to obtain the two-dimensional point cloud.
The method according to claim 1 or 2, characterized in that the two-dimensional true value data includes a plurality of true value frames, and the two-dimensional point cloud includes a plurality of point cloud frames;

The matching of the two-dimensional point cloud with the two-dimensional true value data to obtain a matching result set includes:

Determine at least one truth frame in a first truth frame, wherein one truth frame corresponds to one volume target, and the first truth frame belongs to the plurality of truth frames;

A matching result subset is obtained based on the range of the at least one true value frame and the positions of multiple sampling points in the first point cloud frame, wherein the first point cloud frame belongs to the multiple point cloud frames, the first point cloud frame and the first true value frame have the same timestamp, and the matching result subset belongs to the matching result set.
The method according to claim 3, characterized in that the at least one truth box includes a first truth box corresponding to a first volume target, and the first volume target belongs to the at least one volume target;

In a case where the first point cloud frame includes a first sampling point and the first sampling point falls within the first true value frame, the first sampling point belongs to a matching sampling point, and the first sampling point matches the true value of the first volume target;

In the case where the first point cloud frame includes a second sampling point and the second sampling point does not fall into any truth frame of the at least one truth frame, the second sampling point belongs to an unmatched sampling point;

When any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.
The method according to any one of claims 1 to 4, characterized in that the method further comprises:

According to the matching sampling points in the matching result set, accuracy evaluation data about the DUT is obtained, and the accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy and height accuracy.
The method according to claim 5, characterized in that the matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M>0, and N is an integer and N>0;

The N sampling points include the nearest sampling point, and the nearest sampling point is one of the N sampling points that is closest to the DUT. The M corner points include the closest lateral corner point and the closest radial corner point. The closest lateral corner point is the corner point with the closest lateral distance to the DUT among the M corner points, and the closest radial corner point is the corner point with the closest radial distance to the DUT among the M corner points.

The ranging accuracy includes a lateral ranging accuracy with respect to a second object, and the lateral ranging accuracy with respect to the second object is related to a lateral distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT;

And/or, the ranging accuracy includes a longitudinal ranging accuracy with respect to a second object, and the longitudinal ranging accuracy with respect to the second object is related to a radial distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT.
The method according to claim 5, characterized in that the speed measurement accuracy includes the speed measurement accuracy with respect to a third body target;

The matching sampling points include K sampling points that match the true value of the third body target, where K is an integer and K>0;

The K sampling points include the strongest sampling point, and the strongest sampling point is the sampling point with the strongest radar cross section RCS among the K sampling points.

The velocity measurement accuracy of the third object is related to the radial velocity of the strongest sampling point and the radial velocity of the third object.
The method according to any one of claims 1 to 7, characterized in that the two-dimensional point cloud comprises a plurality of continuous point cloud frames, the matching set comprises unmatched sampling points, and the method further comprises:

According to sampling points in the unmatched sampling points that are located in the multiple continuous point cloud frames, false alarm targets in the multiple continuous point cloud frames are determined.
The method according to claim 8, characterized in that the plurality of continuous point cloud frames include the second point cloud frame and Q point cloud frames after the second point cloud frame, where Q is an integer and Q>0;

The determining of the false alarm target in the point cloud according to the sampling points in the unmatched sampling points that are located in the plurality of continuous point cloud frames comprises:

Clustering the sampling points in the second point cloud frame among the unmatched sampling points to obtain at least one point cloud cluster;

assigning an initial life value to a first point cloud cluster among the at least one point cloud cluster;

Determine a point cloud cluster in the Q point cloud frames according to the sampling points in the unmatched sampling points that are located in the Q point cloud frames;

The false alarm targets in the plurality of consecutive point cloud frames are determined according to the position of the first point cloud cluster and the positions of the point cloud clusters in the Q point cloud frames.
The method according to any one of claims 1 to 6, characterized in that the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame;

The method further comprises:

Determining a suspected missed target in the third point cloud frame according to an unmatched true value corresponding to the third point cloud frame;

Determining the obscured volume target according to the field of view relationship between the at least one volume target and the radar;

The obscured volume targets in the suspected missed detection targets in the third point cloud frame are filtered to determine the missed detection targets included in the third point cloud frame.
A point cloud testing device, characterized in that the point cloud testing device comprises a data acquisition module and a data matching module, wherein:

The data acquisition module is used to acquire true value data and point cloud, wherein the true value data is the true value of at least one volume target, and the point cloud is the detection result obtained by the device under test DUT detecting the at least one volume target;

The data matching module is used to:

Projecting the true value data to obtain two-dimensional true value data;

Projecting the point cloud to obtain a two-dimensional point cloud;

The two-dimensional point cloud is matched with the two-dimensional true value data to obtain a matching result set, wherein the matching result set includes at least one of the following three types of matching results: matched sampling points, unmatched sampling points, and unmatched true values.
The point cloud testing device according to claim 11, characterized in that the data matching module is used to:

Projecting the true value data onto a horizontal plane to obtain the two-dimensional true value data;

The point cloud is projected onto the horizontal plane to obtain the two-dimensional point cloud.
The point cloud testing device according to claim 11 or 12, characterized in that the two-dimensional true value data includes a plurality of true value frames, and the two-dimensional point cloud includes a plurality of point cloud frames;

The data matching module is also used for:

Determine at least one truth frame in a first truth frame, wherein one truth frame corresponds to one volume target, and the first truth frame belongs to the plurality of truth frames;

A matching result subset is obtained based on the range of the at least one true value frame and the positions of multiple sampling points in the first point cloud frame, wherein the first point cloud frame belongs to the multiple point cloud frames, the first point cloud frame and the first true value frame have the same timestamp, and the matching result subset belongs to the matching result set.
The point cloud testing device according to claim 13, characterized in that the at least one truth frame includes a first truth frame corresponding to a first volume target, and the first volume target belongs to the at least one volume target;

In a case where the first point cloud frame includes a first sampling point and the first sampling point falls within the first true value frame, the first sampling point belongs to a matching sampling point, and the first sampling point matches the true value of the first volume target;

In the case where the first point cloud frame includes a second sampling point and the second sampling point does not fall into any truth frame of the at least one truth frame, the second sampling point belongs to an unmatched sampling point;

When any sampling point in the first point cloud frame does not fall into the first true value frame, the true value corresponding to the first true value frame is an unmatched true value.
The point cloud testing device according to any one of claims 1 to 14, characterized in that the point cloud testing device further comprises a data calculation module, wherein the data calculation module is used to:

According to the matching sampling points in the matching result set, accuracy evaluation data about the DUT is obtained, and the accuracy evaluation data includes one or more of the number of matching sampling points, ranging accuracy, speed accuracy and height accuracy.
The point cloud testing device according to claim 15, characterized in that the matching sampling points include N sampling points that match the true value of the second body target, the true value of the second body target includes M corner points, M is an integer and M>0, and N is an integer and N>0;

The N sampling points include a nearest sampling point, which is a sampling point that is closest to the DUT among the N sampling points; the M corner points include a lateral nearest corner point and a radial nearest corner point, which is a corner point that is closest to the DUT in lateral distance among the M corner points; and the radial nearest corner point is a corner point that is closest to the DUT in radial distance among the M corner points;

The ranging accuracy includes a lateral ranging accuracy with respect to a second object, and the lateral ranging accuracy with respect to the second object is related to a lateral distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT;

And/or, the ranging accuracy includes a longitudinal ranging accuracy with respect to a second object, and the longitudinal ranging accuracy with respect to the second object is related to a radial distance between the nearest sampling point and the DUT and a radial distance between the radial nearest corner point and the DUT.
The point cloud testing device according to claim 16, characterized in that the speed measurement accuracy includes the speed measurement accuracy with respect to a third body target;

The matching sampling points include K sampling points that match the true value of the third body target, where K is an integer and K>0;

The K sampling points include the strongest sampling point, and the strongest sampling point is the sampling point with the strongest radar cross section RCS among the K sampling points.

The velocity measurement accuracy of the third object is related to the radial velocity of the strongest sampling point and the radial velocity of the third object.
The point cloud testing device according to any one of claims 11 to 17, characterized in that the two-dimensional point cloud comprises a plurality of continuous point cloud frames, and the matching set comprises unmatched sampling points;

The point cloud testing device also includes a data calculation module, which is used to determine false alarm targets in the multiple continuous point cloud frames based on sampling points in the unmatched sampling points that are located in the multiple continuous point cloud frames.
The point cloud testing device according to claim 18, characterized in that the plurality of continuous point cloud frames include the second point cloud frame and Q point cloud frames after the second point cloud frame, where Q is an integer and Q>0;

The point cloud testing method further comprises a data calculation module, which is used to:

Clustering the sampling points in the second point cloud frame among the unmatched sampling points to obtain at least one point cloud cluster;

assigning an initial life value to a first point cloud cluster among the at least one point cloud cluster;

Determine a point cloud cluster in the Q point cloud frames according to the sampling points in the unmatched sampling points that are located in the Q point cloud frames;

The false alarm targets in the plurality of consecutive point cloud frames are determined according to the position of the first point cloud cluster and the positions of the point cloud clusters in the Q point cloud frames.
The point cloud testing device according to any one of claims 11 to 17, characterized in that the two-dimensional point cloud includes a third point cloud frame, and the matching result set includes an unmatched true value corresponding to the third point cloud frame;

The point cloud testing device further comprises a data calculation module, which is used for:

Determining a suspected missed target in the third point cloud frame according to an unmatched true value corresponding to the third point cloud frame;

Determining the obscured volume target according to the field of view relationship between the at least one volume target and the radar;

The obscured volume targets in the suspected missed detection targets in the third point cloud frame are filtered to determine the missed detection targets included in the third point cloud frame.
A chip, characterized in that the chip comprises a processor and a communication interface;

The communication interface is used to receive and/or send data, and/or, the communication interface is used to provide input and/or output for the processor;

The processor is configured to implement the method according to any one of claims 1 to 10.
A computing device, characterized in that the computing device comprises a memory and a processor, the memory stores computer instructions, and the processor is used to call the computer instructions to implement the method according to any one of claims 1 to 10.
A computer-readable storage medium, characterized in that instructions are stored in the computer-readable storage medium, and when the instructions are executed on at least one processor, the method according to any one of claims 1 to 10 is implemented.