DE102009057996A1 - Method for determining position and/or alignment of camera relative to position of vehicle and semi-trailer, involves determining rotation parameter and/or translation parameter of camera-coordinate system to vehicle-coordinate system - Google Patents

Abstract

The method involves recording a sequence of images by a camera. A movement vector is determined in a vehicle-coordinate system (1), and an image point of one of the images is projected in the system to provide a projected point. The projected point is shifted based on the movement vector so that a point is generated and is projected in a camera-coordinate system (2). A rotation parameter and/or a translation parameter of the camera-coordinate system to the vehicle-coordinate system are determined from a difference between a transformed point and another image point of the other image.

Description

The invention relates to a method for determining a position and / or orientation of a camera with an associated camera coordinate system relative to a position of a vehicle or vehicle combination with an associated vehicle coordinate system, wherein the camera is arranged on the vehicle or vehicle combination.

From the DE 10 2007 040 250 A1 is a method and a device for driver assistance in maneuvering a vehicle or a vehicle combination with several mutually kinkable vehicle elements known. The method provides that on a display unit, the vehicle or vehicle combination in a straight position with a static display element corresponding to the straight position of the vehicle elements and additionally each with a respective the vehicle element associated dynamic display element, the function of a current or future position of the mutually movable vehicle elements is determined to be displayed. The dynamic display elements are amplified many times over and highlighted in color.

In addition, from the DE 10 2008 006 309 A1 a method and a device for driver assistance when maneuvering a vehicle or a vehicle combination, which has mutually kinkable vehicle elements known. In this case, a future driving course is predicted, which indicates at which position the vehicle or vehicle combination will reach a rectilinear position.

The invention has for its object to provide a method for determining a position of a camera relative to a position of a vehicle or a vehicle combination, on which the camera is arranged.

The object is achieved by the features specified in claim 1. Advantageous developments of the invention are the subject of the dependent claims.

In a method according to the invention for determining a position and / or orientation of a camera with an associated camera coordinate system relative to a position of a vehicle or vehicle combination with an associated vehicle coordinate system, wherein the camera is arranged on the vehicle or vehicle combination, it is provided that a sequence of images is taken by the camera and a first motion vector in the camera coordinate system is determined on the basis of a first image point of a first image and a corresponding second image point of a subsequent image, wherein a second motion vector is determined in the vehicle coordinate system and the first image point in the camera coordinate system is projected into the vehicle coordinate system and results in a projected point, wherein the projected point is shifted according to the second motion vector in the first coordinate system, so that a point is generated and projected into the camera. Coordinate system is projected, whereby a transformed point is formed and determined in the camera coordination system from a difference between the transformed point and the second pixel rotation parameters and / or translation parameters of the camera coordinate system to the vehicle coordinate system.

In this case, under the motion vector in connection with the method, a combination of translation and rotation, which describe the movement of the vehicle or vehicle combination, understood.

Using the method according to the invention, it is possible in a particularly advantageous manner that a camera and / or a retrofit camera can be arranged at any position on the vehicle or vehicle combination and a superposition of driving corridors can be displayed as a maneuvering assistance for a driver.

In addition, captured images of the camera can be integrated into already existing in the vehicle Bird's Eye View implementations.

Further, when using the method, it is not necessary to manually calibrate the camera and / or perform a survey of the position of the camera.

Embodiments of the invention will be explained in more detail with reference to a drawing.

Showing:

1 schematically a vehicle coordinate system and a camera coordinate system, wherein a camera is arranged at an arbitrary position on a vehicle or a vehicle combination, and

2 schematically a vehicle coordinate system and a camera coordinate system, wherein on the basis of the coordinate systems, a sequence of the method is shown.

In the 1 and 2 are a vehicle coordinate system 1 and a camera coordinate system 2 shown. In particular, the vehicle coordinate system 1 the reference coordinate system used to perform the procedure. The camera is arranged at any position on the vehicle. The vehicle may, for example, also be a vehicle combination, which has mutually bendable vehicle elements.

For example, the camera is a retrofit camera, the z. B. has been arranged by means of a magnetic base on the vehicle or vehicle combination, the position and / or orientation, in particular for a vehicle-implemented system for maneuvering, in which usually a driving corridor is displayed on a display unit is not known or are. In particular, a processor module is part of the system.

The vehicle coordinate system 1 has three axes x _F , y _F , z _F , wherein a coordinate origin is for example on a wheel axle of the vehicle and a first axis x _{F is arranged} in the direction of longitudinal extent of the vehicle. A second axis y _F extends in the direction of the wheel axis or parallel to the wheel axis. A third axis z _F describes the vertical axis of the vehicle.

The camera coordinate system 2 Also includes three axes x _K , y _K , z _K , wherein the axes x _K , y _K , z _K relate in particular to the orientation of the optical axis of the camera.

In particular, in the 1 the vehicle coordinate system 1 separately and the camera coordinate system 2 overlaid.

An example of this is a deviation from the translation parameter x, y, z of the camera coordinate system 2 to the first vehicle coordinate system 1 shown, wherein a deviation of the optical axis of the camera with respect to rotation parameters ψ, Θ, φ is also shown.

The method provides, in particular, based on the orientation of the coordinate systems 1 . 2 to determine the position and / or orientation of the camera relative to a position of the vehicle or vehicle combination.

In a first method step, a first image is taken by means of the camera at a first time t ₀ , wherein a first pixel P _B1 , in particular a stationary object in a vehicle environment, for example, based on the processor module is determined. Within a second image, which was recorded temporally after the first image at a time t ₁ , a second pixel P _B2 corresponding to the first pixel P _{B1 is} determined and with respect to the camera coordinate system 2 pictured in this. In this case, the same pixels P _B1 , P _B2 are associated with one another in different images by means of known correspondence algorithms which are stored in the processor module.

In particular, it is provided that the images are supplied to the processor module by means of a wireless connection.

Based on the determined pixels P _B1 , P _B2 becomes a first motion vector B _K in the camera coordinate system 2 determined, wherein the first motion vector B _K may be faulty. The detected pixels P _B1 , P _B2 are in the vehicle coordinate system 1 transfer.

Subsequently, in a second method step, for example based on a steering angle, a yaw rate, a movement distance, a lateral acceleration and / or possibly a movement model of the vehicle, a second motion vector B _F in the vehicle coordinate system 1 determined. In this case, the second motion vector B _{F represents} a vehicle movement between the first and the second time t ₀ , t ₁ . In particular, the steering angle, the yaw rate, the movement distance and / or the lateral acceleration are or are detected on the basis of detection units present in the vehicle. For this purpose, an instantaneous steering angle, a momentary yaw rate, a momentary movement distance and / or an instantaneous lateral acceleration, in particular at the first time t ₀ and at the second time t ₁ , are determined. For example, the instantaneous steering angle is made available via a CAN bus structure arranged in the vehicle. At this time, the second motion vector B _F becomes the vehicle coordinate system 1 displayed.

In particular, a relative vehicle movement in the form of the second motion vector B _F is or are determined on the basis of the steering angle, the yaw rate, the movement distance and / or the lateral acceleration at the first time t ₀ and the second time t _{1, the} relative vehicle movement being based, for example, on at least of a known dynamics model is estimated.

Depending on a processing speed of the processor module, a frequency of processed images may be higher than a frequency of processed information, in particular that of the current vehicle motion. For this purpose, it is provided that the relative movement of the vehicle is interpolated between information of the respectively supplied current vehicle movement. Thereby time stamps of the processor module become supplied images and the information used, for example, the current steering angle. To interpolate the relative vehicle motion, an application of various approaches, such. B: linear, square and / or cubic, possible.

By means of the method it is possible to use three rotation parameters ψ, Θ, φ and / or three translation parameters x, y, z of the camera coordinate system 2 relative to the vehicle coordinate system 1 to determine. It is not necessary that initial values for the search of the rotation parameters ψ, Θ, φ and / or translation parameters x, y, z exist, since the position and / or the orientation of the camera is unknown or is. Exist for example B. roughly estimated initial values, a computing time for determining the rotation parameters ψ, Θ, φ and / or translation parameters x, y, z can be shortened.

In order to be able to evaluate the rotation parameters ψ, Θ, φ and / or translation parameters x, y, z, an error function designated in this way is defined, which is described in more detail below:
The determined correspondence of the pixels P _B1 , P _B2 , which was determined in the first method step, connects an object in the two temporally successive camera images. The location of the object in the form of the first pixel P _B1 and the location of the object as a corresponding second pixel P _{B2 are} given. For this purpose, the first pixel P _B1 is in the vehicle coordinate system by means of the rotation parameters ψ, Θ, φ and / or translation parameters x, y, z of the camera to be evaluated 1 projected back. It is assumed that the first pixel P _{B1 is located} on the ground / the road, the backprojection resulting in a projected point P ' _B1 .

Subsequently, the projected point P ' _B1 becomes corresponding to the second motion vector in the vehicle coordinate system 1 shifted and results in a shifted point P _F.

Subsequently, the shifted point P _{F is} projected back into the first image and results in the transformed point P ' _F. On the basis of this process, a movement of the object, in particular the first pixel P _B1 , on the basis of the evaluated rotation parameters ψ, Θ, φ and / or translation parameters x, y, z and a measured vehicle movement from the first pixel P _B1 to the transformed point P ' _F appreciated.

From this estimate results a vector B ' _K , which is subsequently compared to the actual first motion vector B _K and the actual second motion vector B _F in the first image. A resulting two-dimensional error F, for example, summed over all correspondences, z. B. over a longer period of time represents an evaluation function of the rotation parameters ψ, Θ, φ and / or translation parameters x, y, z. The evaluation function is now using known optimization methods, eg. B. the RANSAC algorithm minimized, using the optimization method, the sought rotation parameters ψ, Θ, φ and / or translation parameters x, y, z for a transformation between the vehicle coordinate system 1 and the second camera coordinate system 2 be determined.

The movement information of the vehicle is stored for each image and thus for each correspondence. This allows the correspondence to be collected for a longer period of time and processed together. Therefore, the method can be used even in relatively poor visibility.

The method described here can be applied simultaneously by searching for optimum rotational parameters ψ, Θ, φ and / or translation parameters x, y, z on the basis of correspondences of many images taken by the camera with the respective relative vehicle movement.

In a particularly advantageous manner, the method for determining the position and / or orientation of the camera can be used for any type of camera.

Compared to conventional methods for determining the position and / or orientation of the camera, the three translation parameters x, y, z can be estimated. Although these fluctuate only slightly when the camera is installed in the factory, they are unknown when the camera is positioned freely, for example on a trailer.

The method described is used for machine-based, for example computer-dependent, calibration, wherein a possible decalibration of the camera can also be determined by means of the method. By continuously performing the method, automatic calibration, it can be seen that the orientation and / or position of the camera, e.g. B. has changed by mechanical influences and thus the system of maneuvering no longer works reliably, so the camera must be calibrated manually and / or automatically. It is also possible by the method, a comparatively improper functioning of the camera, z. As disturbances in the image can be detected.

In particular, the method can be used, for example, based on the information obtained with regard to the position and / or orientation of the camera on an image on a Display unit driving corridors as a maneuvering aid to superimpose the displayed image.

It is also possible to integrate the captured images of the camera into Bird's Eye-View systems.

In this case, a use of the method is not limited to the specified application examples.

LIST OF REFERENCE NUMBERS

1

Vehicle coordinate system

2

Camera coordinate system

F

error

PB ₁

first pixel

P ' _B1

projected point

P _B2

corresponding second pixel

P _F

generated point

P ' _F

transformed point

B _K

first motion vector

B _F

second motion vector

t ₀

first time

t ₁

second time

x _F , y _F , z _F

Axes vehicle coordinate system

x _K , y _K , z _K

Axes Camera coordinate system

x, y, z

translation parameters

ψ, Θ, φ

rotation parameters

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102007040250 A1 [0002]
• DE 102008006309 A1 [0003]

Claims (4)

Method for determining a position and / or orientation of a camera with an associated camera coordinate system ( 2 ) relative to a position of a vehicle or vehicle combination with an associated vehicle coordinate system ( 1 ), wherein the camera is arranged on the vehicle or vehicle combination, characterized in that a sequence of images is taken by the camera and based on a first pixel (P _B1 ) of a first image and a corresponding second pixel (P _B2 ) of a subsequent Image a first motion vector (B _K ) in the camera coordinate system ( 2 ), wherein in the vehicle coordinate system ( 1 ) a second motion vector (B _F ) is determined and the first pixel (P _B1 ) in the camera coordinate system ( 2 ) in the vehicle coordinate system ( 1 ) and results in a projected point (P ' _B1 ), wherein the projected point (P' _B1 ) in accordance with the second motion vector (B _F ) in the first coordinate system ( 1 ) so that a point (P _F ) is generated and fed into the camera coordinate system ( 2 ), resulting in a transformed point (P ' _F ) and in the camera coordination system (FIG. 2 ) from a difference (F) between the transformed point (P ' _F ) and the second pixel (P _B2 ) rotational parameters (ψ, Θ, φ) and / or translation parameters (x, y, z) of the camera coordinate system ( 2 ) to the vehicle coordinate system ( 1 ) be determined. A method according to claim 1, characterized in that the second motion vector (B _F ) is determined at least on the basis of a movement distance and / or a steering angle of the vehicle or vehicle combination. A method according to claim 1 or 2, characterized in that a resulting vehicle movement is estimated. Method according to one of claims 1 to 3, characterized in that a movement information of the vehicle or vehicle combination is stored for each captured by the camera image.

Images