DE102022100579A1 - Driver support system, method for supporting an operating process of a device by an operator, and vehicle and electronic processing unit - Google Patents

Abstract

The invention relates to a driver support system (100) and a corresponding method for supporting an operator's operation of a device. The device is a vehicle (20). The vehicle (20) is equipped with the driver support system (100). This includes at least one surroundings camera (28) for recording moving images and a processing unit (36) for evaluating the moving images. Furthermore, a display unit (32) for displaying the moving images recorded by the surroundings camera (28) is part of the driver assistance system. The driver carries out an operating procedure manually, e.g. he steers the vehicle (20). The driver support system (100) is equipped with a detection device (31) that detects the operating process. The recorded data are evaluated in the processing unit (36). The driver support system (100) has an additional output unit (33) to support the display unit (32), the processing unit (36) controlling the further output unit (33) to output additional information if the processing unit (36) determines that the operator's view of the display unit (32) is prevented by the operating process detected by the detection device (31).

Description

The invention relates to the technical field of vehicles that are equipped with a driver assistance system. Various forms of assistance systems are available for vehicles. A few examples of assistance systems are given as examples, but the list does not claim to be complete. These include a turning assistant, distance control systems which are often referred to internationally as ACC systems, corresponding to "Adaptive Cruise Control" systems, a start-up assistant, a trailer assistant, an avoidance assistant, a curb assistant, a brake assistant, a high-beam assistant, a cruise control system, an intersection assistant, a maneuvering assistant, a night vision assistant, an emergency assistant, an emergency brake assistant, a parking assistant, a cross traffic assistant, a lane departure warning system, and a traffic jam assistant. This also includes the surroundings cameras, the images of which are shown to the driver on a display. The reversing camera, which helps the driver when reversing, is also mentioned here in particular.

Such assistance systems increase the comfort of the driver. They can take the strain off the driver, especially on long motorway journeys, but they can also be helpful for difficult driving tasks, such as maneuvering in parking lots or in underground car parks, by relieving them of the task of frequent braking and accelerating or steering the vehicle. Some assistance systems can increase safety directly or even save lives, such as the emergency brake assistant. Secondarily, there is also a safety gain with other assistance systems because the driver tires less quickly because he does not have to concentrate so much for the driving task.

In the following, the parking assistant and the trailer assistant, which also use the surroundings cameras, are described in more detail, as well as manual reversing with the support of the surroundings cameras. However, it is noted that the application of the invention is not only limited to these three assistance systems, but can also be used in other assistance systems. Such assistance systems are used in various vehicle categories. These include passenger cars, including motorcycles, camping vehicles and buses and commercial vehicles such as trucks, agricultural machinery such as tractors, combine harvesters, choppers, forestry equipment, construction machinery, etc.

The invention described below is described using the example of a commercial vehicle (truck). With these, reversing is one of the most difficult tasks for the driver. This is already evident from the fact that the assistance of a guide is required when reversing the commercial vehicle. This is a requirement because of the size and the obscuration of the rear view by the superstructure. This is especially true when a trailer is towed by the commercial vehicle. It was only when video cameras became miniaturized that they could be installed at various points in the vehicle. The reversing cameras deserve a special mention. Because with the installation of these cameras, it was allowed for the drivers to reverse the commercial vehicles without the support of an attendant. But there are regulations for these reversing information systems. The UN directive ECE/TRANS/WP.29/GRSG/2020/4 is given as an example.

The reversing information system (ReIS) according to the General Safety Regulation (GSR) requires either a camera image from a rear-facing camera or a detecting and warning system. The camera image can be displayed on a monitor that is either part of the infotainment system, e.g. above the center console or in the speedometer area of the instrument cluster. The requirement that the image must always be visible to the driver is violated with the installation of the display unit in the speedometer area at certain angles of rotation of the steering wheel, so that this installation position may not be used in practice.

From the document EP 1 955 904 A1 there is known a parking assist device for easily and quickly moving or rotating a target parking frame to an appropriate position. At this time, a target parking frame is superimposed on an actual image showing an area around the vehicle.

From the document EP 2 623 376 A1 There is known a parking assist device for clearly displaying a display overhead image showing an area to be paid attention to when parking on a monitor. The monitor is mounted above the center console. In addition, a voice output can be given, through which instructions are given that support the driver in adhering to the planned route.

From the document U.S. 2016/0152182 A1 a collision avoidance system is known that warns the driver of a vehicle when an imminent collision is detected. This is done as part of a driver support system, by displaying it on one or more monitors in the cockpit are arranged. The driver support system also includes an interior camera that monitors the driver. It even determines the direction in which the driver is looking. In this way it can be determined whether the driver has noticed a warning displayed on the monitor in critical driving situations. The criticality of the warning is adjusted depending on whether the driver has noticed the warning or not.

This can in the similar working system WO 2015/048741 A1 be done in such a way that the warning is output acoustically or haptically.

In all of the support systems of the prior art mentioned, the view of the monitor of the driver support system is not blocked. However, if the driver reverses using the reversing camera, with the image from the reversing camera being displayed on a display unit in the instrument cluster, there is a risk of it being covered. The driver then has to look away and work with the rear-view mirrors. This affects the acceptance of this driver support system and, moreover, the driver support system does not meet the criteria in the guideline. On the other hand, it is not possible in all vehicles to place the monitor somewhere else, e.g. above the center console.

Thus, there is a need for an improved driver support system that avoids these problems.

It is therefore the object of the invention to improve a driver assistance system in such a way that the driver does not have to take his eyes off the display unit of the driver assistance system, despite the fact that it may be covered.

This object is achieved by a driver support system according to claim 1, a method for supporting an operating process of a device by an operator according to claim 8, a correspondingly equipped vehicle according to claim 13 and an electronic processing unit according to claim 14.

The dependent claims contain advantageous developments and improvements of the invention according to the following description of these measures.

The solution according to the invention consists in a driver assistance system with at least one surroundings camera for recording moving images, and a processing unit for evaluating the moving images. The processed moving images are displayed on a display unit. The driver support system is equipped with a detection device that detects the operating procedure and forwards it to the processing unit. For this purpose, an operating device is present, which is used to control a device as part of the operating process. The driver assistance system according to the invention is characterized in that an output unit is provided to support the display unit, with the processing unit controlling the further output unit to output additional information if the processing unit determines that the operator's view is blocked by the operating operation detected by the detection device the display unit is prevented.

The processing unit processes the camera images and detects objects and obstacles. With certain installation positions for the display unit, a clear view of the display unit, where the camera images are displayed, is not always guaranteed. This is particularly the case when the display unit is built into the instrument cluster. The display unit can then be covered by the spokes of the steering wheel if the steering wheel is turned. Such situations are automatically recognized by the invention and auditory information is generated, with which the driver is additionally supported. This allows the driver to continue operating the vehicle without having to take their eyes off the display unit. Such a situation is recognized by detecting and evaluating the operation.

In a driver support system, the device to be controlled is typically the vehicle and the operating device is a steering wheel, for example. The angle of rotation of the steering wheel is recorded. If it deviates from the zero position by a certain amount, the output of the acoustic information is activated. Distance information can be output as an indication, informing the driver how far away the vehicle is from an object recognized in the image of the reversing camera. If the vehicle approaches the object and there is a risk of an imminent collision, a clear warning can advantageously also be output as an indication.

A steering angle sensor is preferably used to detect the angle of rotation of the steering wheel, with which it is detected whether the operator's view of the display unit is blocked by the position of the steering wheel. According to the number and width of the spokes of the steering wheel, those angular ranges are programmed in which the display unit is covered. This can be done experimentally for each driving vehicle model can be determined. For this purpose, a configuration mode provided only for test operation can be called up, in which the angles of rotation measured at the beginning and end of the respective coverage area are recorded and programmed by pressing a button. This can be done separately for both directions of rotation right/left.

It is also very advantageous for the driver support system if the processing unit for recording the types of journey is connected to a journey selector switch in order to record the type of journey and the output of the additional information for the operating procedure is then switched on/off depending on the type of journey activated. It is not necessary for every type of journey that the driver orients himself to the images shown on the display unit.

In practice, however, it has been found that it is very advantageous if the processing unit can output the additional information, in particular when the type of travel “reverse travel” is detected.

In a further expansion stage, the driver support system is combined with a driver monitoring system, in which the direction of view of the driver is evaluated in addition to detecting the angle of rotation of the steering wheel or instead of detecting the angle of rotation and the auditory information is output depending on the direction of view of the driver. If the driver does not look at the display unit while reversing (but, for example, at the exterior mirror(s), the acoustic message can be output in this situation.

In a particularly preferred case, a vehicle interior camera is provided in addition to the rotation angle detection device, with which the viewing direction of the driver is detected, the processing unit controlling the further output unit to output additional information if the processing unit determines that the view the operator is turned away from the display unit.

It is also advantageous if the further output unit is an output unit for acoustic signals, in particular a loudspeaker arrangement or buzzer arrangement, and audio signals and/or voice signals are output as acoustic signals, which are changed in such a way that the degree of approach to an obstacle or a target becomes perceptible. Specific distance information is preferably output in spoken form, so that the driver is specifically informed whether a collision is already imminent.

• • Aufnehmen der Auswirkungen des Bedienvorgangs mit wenigstens einer Umgebungskamera
• • Anzeigen der aufgenommen Auswirkungen auf einer Anzeigeeinheit,
• • Erfassen des Bedienungsvorgangs seitens der Bedienperson mit einer Erfassungseinrichtung und
• • Ausgeben von Zusatzinformationen durch eine zusätzliche Ausgabeeinheit, wenn eine Auswertung des Erfassungsvorgangs ergibt, dass die Sicht der Bedienperson auf die Anzeigeeinheit verdeckt ist.

In another embodiment, the invention relates to a method for supporting an operating process of a device by an operator. The following steps are processed in the process.

• • Recording the effects of the operation with at least one surrounding camera
• • Displaying the recorded effects on a display unit,
• • Detecting the operating process on the part of the operator with a detection device and
• • Output of additional information by an additional output unit if an evaluation of the detection process shows that the operator's view of the display unit is blocked.

The device is preferably a vehicle and the operating device is a steering wheel.

It is advantageous for the method if the detection device is a steering wheel angle sensor, which is used to detect whether the operator's view of the display unit is blocked by the position of the steering wheel.

It is also advantageous if a vehicle interior camera is provided as an additional detection device and a step is carried out with which the driver's viewing direction is detected, with a processing unit also being provided which carries out a step in which the further output unit for Output of additional information is driven when it is determined by the processing unit that the operator's view is averted from the display unit.

A further advantageous addition to the method according to the invention is that the additional output unit is an output unit for outputting acoustic signals, in particular a loudspeaker arrangement or a buzzer arrangement, and audio signals and/or voice signals are output as acoustic signals, which are changed in such a way that the operator the degree of approach to an obstacle or a target object becomes perceptible. In this way, the driver is always kept up to date and is informed whether the vehicle has already approached so critically that a collision could occur. If a critical approach is already given, a dedicated acoustic warning can also be issued. In addition, it can be continuously monitored whether a person who is endangered by the reversing process has entered the danger area behind the vehicle. If so, an acoustic warning is then also issued for the driver. The detection of whether a person is in the danger area is also carried out by evaluating the images recorded by the rear camera.

A further embodiment of the invention consists in a vehicle, the vehicle being equipped with a driver support system according to the invention.

Finally, the invention also consists in an electronic processing unit for a driver support system, in which the electronic processing unit is set up to carry out a method according to the invention. This electronic processing unit has the same advantages as explained in connection with the method according to the invention and the device according to the invention.

Embodiments of the invention are shown in the drawings and are explained in more detail below with reference to the figures.

• 1 ein Nutzfahrzeug beim Ankoppeln eines Anhängerfahrzeuges auf einem Betriebshof;
• 2 ein Blockschaltbild einer ersten Ausführungsform des erfindungsgemäßen Fahrerunterstützungssystems;
• 3 eine erste Darstellung der Sicht des Fahrers auf den Monitor des Fahrerunterstützungssystems beim Rücksetzen des Fahrzeuges;
• 4 eine zweite Darstellung der Sicht des Fahrers auf den Monitor des Fahrerunterstützungssystems beim Rücksetzen des Fahrzeuges bei Vorliegen einer Verdeckung des Monitors des Fahrerunterstützungssystems mit einer Ausgabe einer akustischen Warnung;
• 5 ein Flussdiagramm für ein Computerprogramm zur Umsetzung des Verfahrens gemäß der ersten Ausführungsform der Erfindung;
• 6 ein Blockschaltbild einer zweiten Ausführungsform des erfindungsgemäßen Fahrerunterstützungssystems; und
• 7 ein Flussdiagramm für ein Computerprogramm zur Umsetzung des Verfahrens gemäß der zweiten Ausführungsform der Erfindung.

Show it:

• 1 a commercial vehicle when coupling a trailer vehicle at a depot;
• 2 a block diagram of a first embodiment of the driver support system according to the invention;
• 3 a first representation of the driver's view of the monitor of the driver support system when reversing the vehicle;
• 4 a second representation of the driver's view of the monitor of the driver support system when reversing the vehicle when the monitor of the driver support system is covered, with an output of an acoustic warning;
• 5 a flowchart for a computer program for implementing the method according to the first embodiment of the invention;
• 6 a block diagram of a second embodiment of the driver support system according to the invention; and
• 7 a flowchart for a computer program for implementing the method according to the second embodiment of the invention.

This description illustrates the principles of the inventive disclosure. It is thus understood that those skilled in the art will be able to devise various arrangements which, while not explicitly described herein, embody principles of the inventive disclosure and are intended to be protected within their scope.

The invention is explained using the example of a towing vehicle that performs a coupling process to pick up a trailer that is ready to be coupled. Namely, the trailer vehicle ready for collection is a semi-trailer trailer vehicle. With them, the coupling process should take place particularly carefully so that there is no damage to the towing vehicle or to the trailer vehicle standing on supports. The particular problem is that the towing vehicle is equipped with a coupling element that has a saddle plate. This fifth wheel plate is slightly raised and rotatably mounted. The fifth wheel plate has a V-shaped opening on the rear side, into which a so-called kingpin of the trailer vehicle can engage. When coupling, the towing vehicle must be reversed so that the kingpin enters the opening in the fifth wheel plate. The lateral guides of the V-shaped opening ensure that the kingpin is guided to the central point of the fifth wheel plate and then snaps into place at the central point. Locking elements are attached in front of the central point, which are held in place by spring force. In order for the king pin to engage in the central point, the locking elements must be pushed to the side against the spring force.

This type of docking operation is a difficult task for the driver. He must carry out the longitudinal and lateral control of the towing vehicle at the same time.

For this purpose, a driver support system is installed in the towing vehicle, which supports the driver in the driving task.

1 shows the towing vehicle 20 aligned with the trailer vehicle 10 that is ready for collection. As shown, it is a semi-trailer trailer vehicle that only has rear wheel axles and stands on extendable or folding supports 12 at the front. During the coupling process, the driver must drive the towing vehicle 20 in front of the trailer vehicle 10 and then reverse for coupling. First, the towing vehicle 20 is aligned in the longitudinal direction relative to the trailer vehicle 10 such that the kingpin enters the catch area of the V-shaped opening in the fifth wheel plate 22 as the towing vehicle 20 continues to reverse. The fifth wheel plate 22 is typically mounted at an angle so that it is lower with the open side than with the closed side facing the driver's cab. The driver must visually inspect the position of the saddle plate and, if necessary, readjust the height of the saddle make a plate. After this has been done, he can continue to back up the towing vehicle 20 . This should be done as parallel to the longitudinal alignment of the towing vehicle 20 as possible. The driver will typically perform multiple steering movements in both directions so that this goal is achieved. He will also perform the control of the tow vehicle 20 in the longitudinal direction, ie he applies the throttle of the engine 24 to accelerate the vehicle and he applies the brake to decelerate the tow vehicle 20 . The calipers on the wheels of the towing vehicle 20 are in the 1 denoted by reference numeral 26. The reversing camera 28 as part of the driver support system is in the 1 also shown.

The 2 FIG. 12 shows a block diagram of driver assistance system 100. Reference number 28 designates the reversing camera of driver assistance system 100. Only one surroundings camera is shown. However, it goes without saying for the person skilled in the art that further surroundings cameras can belong to the driver support system. Cameras are often installed in the side mirrors of the towing vehicle 20 and a front camera. A special camera could also be provided, particularly in the case of towing vehicles in the utility vehicle sector, which supplies the exact view of the king pin of the trailer vehicle 10 . Typically, cameras with a wide-angle lens are installed as surroundings cameras. In the case of the special camera for viewing the kingpin, a camera with a telephoto lens could also be used instead. Likewise, the use of a stereo camera would also be possible, which enables a spatial representation of the recorded environment. The rear view camera 28 is connected to an ADAS-ECU 36 corresponding to Advanced Driver Assistance System - Electronic Control Unit. The ADAS-ECU 36 is an image processing unit, also referred to as a camera ECU. The processing of the images supplied by the reversing camera 28 takes place there. If several surroundings cameras are provided, their images are also processed in the ADAS-ECU 36 . Various processing operations can be performed in the ADAS-ECU 36 . Zoom processes and image format conversions are given as examples so that the images can be displayed on the associated monitor to fill the entire display. If there are multiple surround cameras, the images can be combined to provide an improved overall view. Auxiliary lines and instructions and warnings can also be calculated in the correct position and displayed in the images. Such auxiliary lines can also be scaled, so that the driver can use them to estimate, for example, the distance from the vehicle 20 to the approaching object. In 1 this would be, for example, the distance to the trailer vehicle 10 in the coupling process shown therein. The communication connection 45 via which the camera data from the rear-view camera 28 is transmitted to the ADAS-ECU 36 can preferably be an Ethernet data line. An important part of the driver support system is a display unit 32. This is used to display the image data processed by the ADAS-ECU 28. Typically, an LCD panel is used as the display unit 32, which can also be an LCD touch panel, that is, designed as a touch-sensitive display unit. The driver information that can be displayed graphically and graphically is displayed on this display unit 32 . The display unit 32 should be positioned in a central location in the cockpit of the vehicle 20 so that the driver has a good view of it. In the exemplary embodiment described here, the display unit 32 is arranged in the instrument cluster, preferably in the middle position. Alternatively, the display unit 23 can also be positioned in the center console or above the center console. However, in the commercial vehicles considered in this example, the driver's seating position is significantly higher and the steering wheel is significantly larger, so that the installation position in the instrument cluster is suitable, especially if no touch panel is required. The connection 42 to the LCD panel is typically implemented in the form of an LVDS communication connection, corresponding to Low Voltage Differential Signal.

The ADAS-ECU 36 also includes a voice output unit 38. This is used to generate acoustic information for the driver, which are then output via the connected loudspeaker 33. The connection 43 to the loudspeaker 33 can accordingly be designed as a conventional AF line, e.g. in the form of loudspeaker cables. An alternative embodiment may consist, for example, in the ADAS-ECU 36 being connected via a communication link to the vehicle's infotainment system 20, which also includes the instrument cluster with the LCD panel 32 and the loudspeaker 33. The audio and video data would be transmitted to the infotainment system via the communication link. The infotainment system also contains a central processing unit with which the data can be processed or converted if necessary. The data ready for output are forwarded from this processing unit to the display unit 32 or the LF signals are forwarded to the amplifier unit of the audio system, from where they then reach the loudspeaker 33 .

The ADAS-ECU 36 is also connected to the travel selector switch 35 of the automatic transmission of the towing vehicle 20 in order to detect the driving mode. For this, as a communication link 41 a LIN bus connection can be used, according to Local Interconnect Network.

The drive selector switch 35 is used at least to select the drive modes forward drive, reverse drive, neutral, and parking. Additional driving modes can also be provided. In addition, the travel selector switch 35 often also contains a plus/minus shift lever, with which the gears of the automatic transmission can also be shifted manually. Furthermore, a steering wheel angle sensor 31 is also connected to the ADAS-ECU 36, with which the position of the steering wheel 30 is detected. The communication connection 40 can be designed in the form of a CAN bus connection, corresponding to the Controller Area Network.

The 3 12 shows the driver's view of the instrument cluster LCD panel 32 when the steering wheel 30 is rotated to the home straight-ahead position. The driver has a clear view of the LCD panel 32 because the spokes 39 of the steering wheel 30 do not cover it. An image captured by the rear camera 28 is displayed on the LCD panel 32 . By looking at the LCD panel 32, the driver can check how far away the towing vehicle 20 is from the displayed object and can also use the displayed image to guide the towing vehicle 20 longitudinally and laterally. In addition, auxiliary lines can also be superimposed in the displayed image, which, for example, can also be curved in order to illustrate the probable route when the object displayed is approached further.

The 4 FIG. 12 now shows a further representation of the driver's view of the LCD panel 32 as the object represented is approached further. The driver has performed a steering movement to the right as part of the lateral guidance of the towing vehicle 20, so that a spoke 39 of the steering wheel 30 in the position shown partially covers the view of the LCD panel 32 of the instrument cluster. There is therefore a risk that the driver will no longer have an object or a part of an object in view when reversing further and approaching the object, and that the object may then be bumped into as a result. It would even be possible for people to be injured if they were covered. Therefore, according to the invention, it is provided that when the LCD panel 32 is completely and partially covered, as in 4 shown, in addition, there is an acoustic output of instructions to the driver. In a first example, the information can be in the form of an indication of the estimated distance from the object being displayed. This notice will be updated regularly as long as the cover is present. The embodiment can be designed in such a way that the notice is also accompanied by warning tones, which also provide information about the approach. Such warning tones are known from distance warning devices that work with ultrasonic sensors for distance measurement. In addition, hazard warnings can be issued. If, for example, a person is detected in the danger area during the image evaluation in the ADAS-ECU 36, a particularly urgent warning can be issued, for example. The warning "Caution, person in the path!" is given as an example. In the same way, an all-clear can be given when it is recognized that the person has left the danger area. The message "Person has left the route!" is given as an example. If the concealment is no longer given by further steering movements, no further acoustic information is given as long as there is no concealment. If the concealment is restored, the acoustic instructions are issued again.

The operation of the ADAS-ECU 36 of the driver support system 100 will now be explained with reference to the flowchart in FIG 5 explained. The program starts in step S1 when the driver actuates the travel selector switch 35 and switches on the driving position R for reverse travel.

In program step S2, the reversing camera 28 is switched on and the recorded images are transmitted to the ADAS-ECU 36 via the Ethernet connection. The processing of the video data stream takes place there and the image data are output to the display unit 32, with which the finished images are displayed to the driver. Typically, the captured images are compressed in camera 28 and encoded for transmission. The video data are decoded accordingly in the ADAS-ECU and they are transmitted to the display unit 32 uncompressed. In the next program step S3, the position of the steering wheel 30 is detected. For this purpose, the next measured value supplied via the CAN bus 40 from the angle of rotation sensor 31 attached to the steering column is read. The measured value taken over for the current steering wheel rotation angle LW4 is evaluated in program step S4. In program step S4, a check is made as to whether the measured value is in the programmed steering wheel rotation angle range VB, in which the LCD panel 28 is completely or partially covered. Depending on the number and width of the spokes 39 of the steering wheel 30, several coverage areas VB for a complete rotation of the steering wheel 30 can be programmed. These can also be programmed separately for the different directions of rotation of the steering wheel 30 . If the result of this evaluation is that the view of the LCD panel 28 is not obscured, the next program steps S5 and S6 are skipped and that Program is continued with program step S7. It is also checked whether the driving position R for reversing is still available. It is possible to do this by querying a register in the ADAS-ECU 36, in which it is entered which gear is switched on. If the driving position R is still entered, the program branches back to the beginning and the program steps of the program are processed again. If it is determined in program step S4 that rotating the steering wheel 30 obscures the view of the LCD panel 28 in the instrument cluster, an acoustic message is generated in program step S5 to support the driver. A voice output unit 38 is used for this, which converts the instruction stored in text form into digital audio signals. These audio signals are then played back by an audio player in the next program step S6. The DA conversion of the audio signals takes place, resulting in the LF signals, which are then amplified in the amplifier unit of the infotainment system and fed to the sound converter, which finally generates the acoustic signals. In the example, the transducer is used as the transducer 2 the loudspeaker 33. In step S5, the notification that is provided in the memory for the respective priority level is selected. This can depend very much on how far away the approaching object is. The distance is determined by image evaluation in this step. Various image evaluation algorithms that can be used are known for this. If other dangers are recognized during the image evaluation, such as a person entering the danger area, the warning is selected instead of the regular distance information. As previously described, the level of urgency can be signaled by adding warning tones. The urgency can be represented by the pitch of the warning tone. After the acoustic signal has been output, the program returns to the position of program step S7, which has already been described above. The program finally ends in program step S8.

The 6 shows a second embodiment of the invention. The same reference numbers denote the same components as in FIG 2 . The second embodiment relates to an extension of the first embodiment. An interior camera 34 in the cockpit of the vehicle 20 is additionally provided therein. The driver's viewing direction is thus recorded. Various techniques for detecting the driver's line of sight are known in the prior art. Such systems are also known under the term "eye tracking". While the line of sight can be determined very precisely with "Head Mounted Eye Trackers", there are also systems that work with webcams. These cannot capture the line of sight as precisely, but the accuracy is sufficient for simple applications such as the application described here. At the same time, the interior camera 34 can also be used for driver monitoring. The document is referred to again as an example of an "eye-tracking" solution that was implemented with an interior camera US2016/0152182 Claim 1 referenced. The algorithm with which the viewing direction is calculated from the recorded camera images is processed in a "driver monitoring" camera ECU 37, which also carries out the image evaluation for driver monitoring. The DM camera ECU 37 is connected to the interior camera 34 via a communication link 46, which can be implemented in the form of an Ethernet data link. The DM camera ECU 37 is connected to the ADAS-ECU 36 via another communication link 44 . This communication connection 44 can in turn be implemented as a CAN bus connection. The ascertained line of sight is transmitted to the ADAS-ECU 36 via this, for example.

With the addition of the interior camera 34 and the DM camera ECU 36 to the driver support system 100, the mode of operation of the ADAS-ECU 36 also changes 7 explained. Therein, the same reference numerals denote the same steps as in the example of FIG 5 . The difference is the addition of another query criterion after query S4. Query step S9 is added. A check is made therein as to whether the determination of the driver's viewing direction with the aid of the interior camera 28 has shown that the driver is looking at the display unit 32 . If so, the two steps S5 and S6, with which the acoustic message is generated and output, are skipped and the program continues with step S7. If no, the acoustic indication is generated in step S5 and output in S6, as previously explained. The acoustic message is therefore output in both cases when the view of the LCD panel 32 is blocked by the steering wheel 30 and also when there is no blocking, but the driver is distracted or prefers to do this by looking in the side mirror Vehicle 20 resets. The driver support is thus even better thanks to the extended driver support system.

All examples mentioned herein, as well as conditional language, are intended to be understood as not being limited to such specifically cited examples. For example, it will be appreciated by those skilled in the art that the block diagram presented herein represents a conceptual view of exemplary circuitry. Similarly, it can be seen that an illustrated flowchart, state transition diagram, pseudocode and the like represent different variants for the representation of processes, which are essentially stored in computer-readable media and can thus be executed by a computer or processor.

It should be understood that the proposed method and associated devices can be implemented in various forms of hardware, software, firmware, special purpose processors or a combination thereof. Specialty processors can include Application Specific Integrated Circuits (ASICs), Reduced Instruction Set Computers (RISC), and/or Field Programmable Gate Arrays (FPGAs). Preferably, the proposed method and device is implemented as a combination of hardware and software. The software is preferably installed as an application program on a program storage device. Typically, it is a computer platform based machine that includes hardware such as one or more central processing units (CPU), random access memory (RAM), and one or more input/output (I/O) interfaces. An operating system is typically also installed on the computer platform. The various processes and functions described here can be part of the application program or a part that runs through the operating system.

The disclosure is not limited to the exemplary embodiments described here. There is room for various adaptations and modifications that those skilled in the art would contemplate based on their skill in the art as well as belonging to the disclosure.

Reference List

10

trailer vehicle

12

Support

20

towing vehicle

22

saddle plate

24

internal combustion engine

26

service brake

28

backup camera

30

steering wheel

31

steering wheel angle sensor

32

display unit

33

speaker

34

interior camera

35

trip selector switch

36

ADAS ECU

37

DM ECU

38

speech output unit

40

1. Communication link

41

2. Communication link

42

3. Communication link

43

4. Communication link

44

5. Communication link

45

6. Communication link

46

7. Communication link

50

instrument cluster

100

driver support system

S1-S9

various program steps of computer programs

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• EP 1955904 A1 [0006]
• EP 2623376 A1 [0007]
• US 2016/0152182 A1 [0008]
• WO 2015/048741 A1 [0009]
• US2016/0152182 [0046]

Claims (14)

Driver support system with at least one surroundings camera (28) for recording moving images, a processing unit (36) for evaluating the moving images, with a display unit (32) for displaying the moving images recorded by the surroundings camera (28), with an operating device (30) for controlling a Device (20), characterized in that the driver support system (100) is equipped with a detection device (31), which detects the operating procedure and transmits it to a processing unit (36), that the driver support system (100) has an output unit (33) to support the Display unit (32), wherein the processing unit (36) controls the further output unit (33) to output additional information if the processing unit (36) determines that the operator's view of the display unit (32) is prevented. driver support system claim 1 , wherein the device is a vehicle (20) and the operating device is a steering wheel (30). driver support system claim 2 , wherein the detection device (31) is a steering angle sensor, with which it is detected whether the operator's view of the display unit (32) is covered by the position of the steering wheel (30). Driver support system according to any of the preceding claims 2 , wherein the processing unit (36) for detecting the trip types is connected to a trip selector switch (35) in order to detect the trip type and switches on the output of the additional information of the operating procedure depending on the trip type switched on. driver support system claim 4 , the output of the additional information being able to be carried out by the processing unit (36), in particular when the type of travel in reverse is detected. Driver support system according to one of the preceding claims, in which, in addition to the detection device (31), a vehicle interior camera (34) is provided, with which the viewing direction of the driver is detected, the processing unit (36) using the further output unit (33) for outputting Additional information controls when it is determined by the processing unit (36) that the operator's view of the display unit (32) is averted. Driver support system according to one of the preceding claims, wherein the further output unit (33) is an output unit for acoustic signals, in particular a loudspeaker arrangement or buzzer arrangement, and sound signals or voice signals are output as acoustic signals, which are changed in such a way that the degree of approach for the operator to an obstacle (5) or a target object (10) becomes perceptible. Method for supporting an operating process of a device by an operator, with the steps • recording the effects of the operating process with at least one camera (28) • displaying the recorded effects on a display unit (32), • characterized by the further steps • detecting the operating process on the part the operator with a detection device (31) • outputting additional information by an additional output unit (33) if an evaluation of the detection process shows that the operator's view of the display unit (32) is blocked. procedure after claim 8 , wherein the device is a vehicle (20) and the operating device is a steering wheel (30). procedure after claim 9 , wherein the detection device is a steering angle sensor (31) with which it is detected whether the operator's view of the display unit (32) is covered by the position of the steering wheel (30). Procedure according to one of Claims 8 until 10 , wherein a vehicle interior camera (34) is provided as an additional detection device, with which the viewing direction of the driver is detected, wherein a processing unit (36) is provided, which controls the further output unit (33) to output additional information if from the processing unit (36) determines that the operator's view is averted from the display unit (32). procedure after claim 10 or 11 , wherein the additional output unit (33) is an output unit for outputting acoustic signals, in particular a loudspeaker arrangement or a buzzer arrangement, and sound signals and/or voice signals are output as acoustic signals, which are changed in such a way that the degree of approach to a Obstacle (5) or a target object (10) becomes perceptible. Vehicle, characterized in that the vehicle (20) with a driver support system (100) after one of the Claims 1 until 7 Is provided. Electronic processing unit for a driver support system (100) according to one of Claims 1 until 7 , characterized in that the electronic processing unit (36) is set up a method according to one of Claims 8 until 12 to perform.

Images