JP2017146744A - Driver state determination device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress a deterioration in determination accuracy of a driver state even while a vehicle is executing an automatic operation in a driver state determination device capable of using a vehicle signal for determining the driver state.SOLUTION: A driver state determination device comprises: a biological signal acquisition part 402 for acquiring a biological signal received from a wearable device; an image acquisition part 403 for acquiring a driver image; a vehicle signal acquisition part 404 for acquiring a vehicle signal; and a driver state determination part 406 for determining a driver state by using at least any of the biological signal acquired by the biological signal acquisition part 402, the driver image acquired by the image acquisition part 403 and the vehicle signal acquired by the vehicle signal acquisition part. The driver state determination part 406 determines the driver state by making the weights of the driver image and the biological signal other than the vehicle signal larger than the vehicle signal when an automatic operation is executed in comparison with the time when the automatic operation is not executed.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a driver state determination device that determines the state of a driver.

  Conventionally, a technique for determining the state of a driver of a vehicle is known. For example, Patent Document 1 discloses a technique for determining a driver's casual state from vehicle signals such as a vehicle speed and a steering angle that change due to the driving operation of the driver.

Japanese Patent Laying-Open No. 2015-11536

  However, the technique disclosed in Patent Document 1 does not consider automatic driving that automatically controls part or all of acceleration, braking, and steering of a vehicle. Therefore, the following problems occur in a vehicle that can perform automatic driving. For example, the steering angle when the automatic driving for automatically controlling the steering is performed does not change due to the driving operation of the driver. Therefore, there is a high possibility that the driver state determined from the steering angle at the time of performing the automatic driving will deviate from the actual driver state. In addition, the vehicle speed when the automatic driving for automatically controlling acceleration and / or braking is performed does not change due to the driving operation of the driver. Therefore, it is highly possible that the driver state determined from the vehicle speed at the time of the automatic driving is also different from the actual driver state. As described above, the technique disclosed in Patent Document 1 has a problem in that the accuracy of determining the driver state is reduced when the automatic operation is performed.

  The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to provide a driver state determination apparatus that can use a vehicle signal for determination of a driver state. This is also to suppress a decrease in the accuracy of determining the driver state.

  The above object is achieved by a combination of the features described in the independent claims, and the subclaims define further advantageous embodiments of the invention. Reference numerals in parentheses described in the claims indicate a correspondence relationship with specific means described in the embodiments described later as one aspect, and do not limit the technical scope of the present invention. .

  In order to achieve the above object, the driver state determination device of the present invention is used in a vehicle that can perform automatic driving that automatically controls at least one of acceleration, braking, and steering, and does not implement automatic driving. Is a vehicle signal acquisition unit (404) that acquires the vehicle signal of the vehicle that can be determined and can determine the driver state that is the state of the driver of the vehicle, and information other than the vehicle signal as information that can determine the driver state. The driver state is determined using at least one of the determination information acquisition unit (402, 403) that acquires the determination information, the vehicle signal acquired by the vehicle signal acquisition unit, and the determination information acquired by the determination information acquisition unit. A state determination unit (406, 406a) for determining, and the state determination unit is more discriminating from the vehicle signal when automatic driving is performed than when automatic driving is not performed. Determines driver condition the weight of the use information is increased to.

  According to this, when the automatic driving is performed, the driver state is determined by making the weight of the determination information larger than the vehicle signal compared to when the automatic driving is not performed. It is possible to suppress the deviation between the actual driver state and the determination result due to a vehicle signal that is not a thing. Therefore, it is possible to suppress a decrease in the accuracy of determining the driver state even during the automatic operation.

1 is a diagram illustrating an example of a schematic configuration of a driver state determination system 1. FIG. 2 is a diagram illustrating an example of a schematic configuration of a wearable device 2. FIG. FIG. 3 is a diagram illustrating an example of a schematic configuration of a vehicle side unit 3. It is a figure which shows an example of a schematic structure of HCU40 in Embodiment 1. FIG. It is a figure for demonstrating an example of the weight of each parameter according to an automation level. 7 is a flowchart illustrating an example of a flow of a driver state determination related process in the HCU 40. It is a figure for demonstrating an example of the aspect of the stimulus according to the automation level and the arousal state. It is a figure which shows an example of the schematic structure of HCU40a in Embodiment 2. FIG.

  A plurality of embodiments and modifications for disclosure will be described with reference to the drawings. For convenience of explanation, between the plurality of embodiments and the modified examples, parts having the same functions as those shown in the drawings used for the explanation so far are denoted by the same reference numerals and description thereof is omitted. There is a case. For the portions denoted by the same reference numerals, the description in other embodiments and / or modifications can be referred to.

(Embodiment 1)
<Schematic configuration of driver state determination system 1>
Embodiment 1 of the present invention will be described below with reference to the drawings. As shown in FIG. 1, the driver state determination system 1 includes a wearable device 2 and a vehicle side unit 3.

  The wearable device 2 is worn by a user and measures the user's biological signal. The wearable device 2 includes a wristwatch type that is worn on the user's arm and a glasses type that is worn on the user's head in the same manner as the eyeglasses. However, the wristwatch type will be described below as an example.

  The vehicle side unit 3 is used in the vehicle HV and performs wireless communication with the wearable device 2. In addition, the vehicle-side unit 3 estimates the state of the driver of the vehicle HV (hereinafter referred to as the driver state), presents the driver state, and gives the driver a stimulus for improving the driver state.

<Schematic configuration of wearable device 2>
Next, a schematic configuration of the wearable device 2 will be described with reference to FIG. As shown in FIG. 2, the wearable device 2 includes a main control unit 20, a measurement related unit 21, an operation input unit 22, a communication unit 23, a presentation unit 24, and a stimulation unit 25.

  As shown in FIG. 2, the measurement related unit 21 includes an acceleration sensor 211, a body motion determination unit 212, a pulse wave sensor 213, a heart rate measurement unit 214, and a blood pressure measurement unit 215.

  The acceleration sensor 211 is a sensor that sequentially measures acceleration generated in the wearable device 2 as a voltage value when activated. That is, the acceleration sensor 211 measures an acceleration generated at the user's part wearing the wearable device 2 as a user's biological signal. For example, the acceleration sensor 211 may be configured to start in accordance with an instruction from the main control unit 20 when the wearable device 2 is powered on. As the acceleration sensor 211, a triaxial acceleration sensor that can measure acceleration along the axial directions of the three axes orthogonal to each other may be used. The acceleration sensor 211 may be configured to use a uniaxial acceleration sensor or a biaxial acceleration sensor. The acceleration sensor 211 outputs the measured acceleration to the body movement determination unit 212.

  The body motion determination unit 212 determines the presence or absence of a user's body motion based on the acceleration measured by the acceleration sensor 211. As an example, if any of the three-axis accelerations is equal to or greater than the threshold, it is determined that there is a user's body movement, and if any of the three-axis accelerations is less than the threshold, it is determined that there is no user's body movement. To do. The threshold value referred to here may be an arbitrary value set to such an extent that, for example, noise in the absence of body movement of the user is not determined to be body movement. Note that the three axes may have different threshold values.

  When activated, the pulse wave sensor 213 detects a user's pulse wave at a site where the wearable device 2 is worn. For example, the pulse wave sensor 213 may be configured to operate when the body motion determination unit 212 determines that there is no body motion. The pulse wave sensor 213 is a pulse wave sensor that can change the measurement cycle (in other words, the sampling rate), and can switch the sampling rate to, for example, a low cycle of 20 Hz and a high cycle of 200 Hz. As such a pulse wave sensor 213, for example, a photoelectric pulse wave sensor, an impedance pulse wave sensor, or the like can be used.

  The heartbeat measuring unit 214 performs known signal processing on the volume pulse wave signal sequentially detected by the pulse wave sensor 213, and measures the user's heart rate from the volume pulse wave. The measured heart rate is output to the main controller 20. The blood pressure measurement unit 215 performs well-known signal processing on the volume pulse wave signal sequentially detected by the pulse wave sensor 213, and measures the user's blood pressure from the volume pulse wave. The measured blood pressure is output to the main control unit 20.

  In order to measure the blood pressure based on the volume pulse signal, it is necessary to make the waveform of the volume pulse wave clean by increasing the sampling rate higher than the sampling rate necessary for measuring the heart rate. Therefore, when the blood pressure is measured using the volume pulse wave detected by the pulse wave sensor 213, the sampling rate is set to be larger than that when only the heart rate of the heart rate and the blood pressure is measured. Good. In the example of the present embodiment, only the heart rate of the heart rate and the blood pressure may be measured at a sampling rate of 20 Hz, and the blood pressure may be measured at a sampling rate of 200 Hz.

  The operation input unit 22 is a switch group operated by a user. The operation input unit 22 may be a mechanical switch or a touch panel switch. As an example, it has a power switch for switching the power supply of the wearable device 2 between an on state and an off state, a measurement start switch for instructing measurement start of a biological signal, a presentation start switch for instructing presentation of a measurement result, and the like. . The measurement start switch and the presentation start switch may be configured so that each of a plurality of biological signals can be designated as a target.

  Note that power on / off of the wearable device 2 may not be switched by an operation input from the user to the operation input unit 22. For example, it may be configured to automatically switch according to the attachment / detachment of the wearable device 2.

  The communication unit 23 performs short-range wireless communication with a short-range communication device 5 described later of the vehicle-side unit 3. The communication unit 23 may be configured to transmit and receive signals according to short-range wireless communication standards such as Bluetooth (registered trademark) and ZigBee (registered trademark), but in the following, the signals are transmitted and received according to Bluetooth standards. An example will be described. From the viewpoint of reducing power consumption, it is more preferable that a signal is transmitted and received in accordance with the Bluetooth Low Energy standard.

  The presentation unit 24 presents information in accordance with instructions from the main control unit 20. Information presentation may be performed by a display device or an audio output device. As an example, measurement values of biological signals such as a user's heart rate and blood pressure measured by the measurement related unit 21 may be displayed.

  The stimulation unit 25 stimulates the user wearing the wearable device 2 in accordance with an instruction from the main control unit 20. As an example, a configuration may be used in which stimulation is performed by vibrating a vibrator, or a configuration in which stimulation is performed by outputting an alarm such as an alarm.

The main control unit 20 includes a volatile memory, a non-volatile memory, an I / O, and a bus connecting them, and executes various processes by executing a control program stored in the non-volatile memory. For example, the acceleration sensor 211 is activated when the power of the wearable device 2 is turned on. Further, the pulse wave sensor 213 is activated when a measurement start switch in the operation input unit 22 is operated. In addition, when the presentation start switch in the operation input unit 22 is operated, the measurement value of the biological signal measured by the measurement related unit 21 is presented to the presentation unit 24. In addition, biological signals such as heart rate and blood pressure measured by the measurement related unit 21 are transmitted from the communication unit 23 to the vehicle side unit 3. Further, in accordance with the request received from the vehicle side unit 3 via the communication unit 23, the stimulation unit 25 is caused to perform stimulation. Note that some or all of the functions executed by the main control unit 20 may be configured in hardware by one or a plurality of ICs.

<Schematic configuration of vehicle side unit 3>
Next, a schematic configuration of the vehicle side unit 3 will be described with reference to FIG. The vehicle-side unit 3 is mounted on a vehicle HV. As shown in FIG. 3, an HMI (Human Machine Interface) system 4, a short-range communication device 5, an ITS (Intelligent Transport Systems) communication device 6, ADAS ( Advanced Driver Assistance Systems) locator 7, peripheral monitoring system 8, and vehicle control ECU 9. The HMI system 4, the near field communication device 5, the ITS communication device 6, the ADAS locator 7, the periphery monitoring system 8, and the vehicle control ECU 9 are connected to, for example, an in-vehicle LAN, and can exchange information with each other by communication.

  The short-range communication device 5 transmits and receives signals in accordance with the short-range wireless communication standard. The short-range communication device 5 preferably has a communication range that stops in the passenger compartment of the vehicle HV. For example, the maximum communication distance may be less than about 1 m. The short-range communication device 5 performs wireless communication with the wearable device 2 worn by the driver of the vehicle HV. In the example of this embodiment, the short-range communication device 5 transmits and receives signals in accordance with the Bluetooth Low Energy standard.

  Hereinafter, the following description will be given on the assumption that pairing between the wearable device 2 worn by the user and the short-range communication device 5 has been executed. Further, even when the paired wearable device 2 other than the wearable device 2 worn by the user who is the driver of the vehicle HV is in the vehicle HV, the driver wearable device 2 and the short-range communication device 5 The following description will be made assuming that the connection is selected by, for example, a driver operation input.

  The ITS communication device 6 performs wireless communication with an in-vehicle communication device mounted on a vehicle around the vehicle HV and / or a roadside device installed on the roadside. For example, the ITS communicator 6 acquires position information, travel speed information, and the like of surrounding vehicles of the vehicle HV through vehicle-to-vehicle communication with the in-vehicle communication device and road-to-vehicle communication with the roadside device. The ITS communication device 6 outputs the acquired information to the in-vehicle LAN.

  The ADAS locator 7 includes a GNSS receiver, an inertial sensor such as a 3D gyro sensor, and a memory that stores map data. A GNSS (Global Navigation Satellite System) receiver receives positioning signals from a plurality of artificial satellites. The 3D gyro sensor includes, for example, a triaxial gyro sensor and a triaxial acceleration sensor.

  The ADAS locator 7 measures the position of the vehicle HV by combining the positioning signal received by the GNSS receiver and the measurement result of the inertial sensor. The ADAS locator 7 reads map data such as link data, node data, road shape, and structures ahead of the vehicle HV from the memory. The ADAS locator 7 sequentially outputs the position of the vehicle HV and the read map data ahead of the route to the in-vehicle LAN. The map data may be acquired using an in-vehicle communication module used for telematics communication such as DCM (Data Communication Module) mounted on the vehicle HV.

  The periphery monitoring system 8 includes a periphery monitoring ECU 80 and a periphery monitoring sensor 81. The surrounding monitoring system 8 detects obstacles such as pedestrians, moving animals such as animals other than humans, bicycles, motorcycles, and other vehicles, and falling objects on the road, guardrails, curbs, and stationary objects such as trees. In addition, road markings such as travel lane markings and stop lines are also detected.

  The peripheral monitoring sensor 81 is a peripheral monitoring camera that images a predetermined range around the vehicle HV, a millimeter wave radar that transmits a search wave to the predetermined range around the vehicle HV, sonar, and LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging). ) Etc. As a periphery monitoring camera, a configuration using a stereo camera or a configuration using a monocular camera may be used. The periphery monitoring camera sequentially outputs captured images to be sequentially captured to the periphery monitoring ECU 80. A sensor that transmits an exploration wave such as sonar, millimeter wave radar, or LIDAR sequentially outputs a scanning result based on a reception signal obtained when a reflected wave reflected by an obstacle is received to the periphery monitoring ECU 80.

  The periphery monitoring sensor 81 may be configured to use only a single type of sensor, or may be configured to use a plurality of types of sensors. Moreover, it is good also as a structure which has a sensing range with which the multiple types of periphery monitoring sensor 81 overlapped, such as performing sensing in front of vehicle HV together using a periphery monitoring camera and a millimeter wave radar.

  The peripheral monitoring ECU 80 includes a CPU, a volatile memory, a nonvolatile memory, an I / O, and a bus for connecting them, and executes various processes by executing a control program stored in the nonvolatile memory. Note that some or all of the functions executed by the periphery monitoring ECU 80 may be configured in hardware by one or a plurality of ICs.

  The surrounding monitoring ECU 80 recognizes the traveling environment of the vehicle HV from the sensing result of the surrounding monitoring sensor 81. For example, a distance from the vehicle HV, a relative position with respect to the vehicle HV, a relative speed with respect to the vehicle HV, and the like are detected from the sensing result of the surrounding monitoring sensor 81. Further, the periphery monitoring ECU 80 may be configured to detect a road surface marking such as a running lane line, a position of a road surface marking with respect to the vehicle HV, and the like from a captured image captured by the periphery monitoring camera and a sensing result of LIDAR.

  Further, the periphery monitoring ECU 80 may recognize the traveling environment of the vehicle HV using a result other than the sensing result of the periphery monitoring sensor 81. For example, from the position, speed, etc. of other vehicles obtained by inter-vehicle communication and / or road-to-vehicle communication via the ITS communication device 6, for other vehicles existing around the vehicle HV, the distance from the vehicle HV, the vehicle HV A relative position with respect to the vehicle, a relative speed with respect to the vehicle HV, and the like may be detected. The periphery monitoring ECU 80 outputs various types of detected information to the in-vehicle LAN as monitoring information.

  The vehicle control ECU 9 is an electronic control device that performs acceleration / deceleration control and / or steering control of the vehicle HV. The vehicle control ECU 9 includes a steering ECU that performs steering control, a power unit control ECU that performs acceleration / deceleration control, a brake ECU, and the like. The vehicle control ECU 9 acquires detection signals output from sensors such as an accelerator position sensor, a brake pedal force sensor, a rudder angle sensor, a vehicle speed sensor, and an acceleration sensor mounted on the vehicle HV, and performs electronic control throttle, brake actuator, EPS (Electric Power Steering) Outputs a control signal to each travel control device such as a motor. Further, the vehicle control ECU 9 can output detection signals (hereinafter referred to as vehicle signals) from the above-described sensors to the in-vehicle LAN.

  In addition, the vehicle control ECU 9 automatically controls the acceleration, braking, and / or steering of the vehicle HV based on the monitoring information detected by the surrounding monitoring ECU 80, thereby performing a driving operation by the driver. It has. As an example of the automatic driving function, the vehicle HV travels so as to maintain the target inter-vehicle distance from the preceding vehicle by adjusting the driving force and the braking force based on the monitoring information of the preceding vehicle acquired from the surrounding monitoring ECU 80. There is an ACC (Adaptive Cruise Control) function to control the speed. Further, based on the monitoring information of the traveling section line in the traveling direction acquired from the peripheral monitoring ECU 80, the steering lane is generated in a direction that obstructs the approach to the traveling lane line, so that the vehicle is maintained in the traveling lane. There is an LKA (Lane Keeping Assist) function for running HV.

  Besides, the LCA (Lane Change Assist) that automatically moves the vehicle HV to the adjacent lane based on the monitoring information of the traveling lane line in the traveling direction acquired from the surrounding monitoring ECU 80 and the monitoring information of other vehicles in the adjacent lane. ) There is a function. Further, there is an AEB (Autonomous Emergency Braking) function for forcibly automatically decelerating the vehicle HV by generating a braking force based on the monitoring information ahead of the vehicle HV acquired from the surrounding monitoring ECU 80. As an example, the AEB function forcibly decelerates the vehicle HV automatically when the TTC (time to collision) with respect to an object ahead of the vehicle HV falls below a set value such as 5 sec, for example, and the emergency control condition is satisfied. Further, as an example of the automatic driving function, there are an automatic merging function for automatically joining the vehicles HV, an automatic retreating function for automatically stopping on the road shoulder in an emergency, and the like. In addition, what was described here is an example to the last, and it is good also as a structure provided with another function as an automatic driving | operation function.

  It is assumed that the vehicle control ECU 9 can switch the level of automation of driving operation (hereinafter referred to as automation level) in a plurality of stages. The first embodiment will be described by taking as an example a case where execution of automatic driving can be switched and automatic level in automatic driving can also be switched. In the first embodiment, the automation level 0 (No-Automation), the automation level 1 (Function-specific Automation), and the automation level 2 (Combined) in accordance with the classification of automation levels defined by NHTSA (National Highway Traffic Safety Administration). Function Automation), automation level 3 (Limited Self-Driving Automation), and automation level 4 (Full Self-Driving Automation).

  The automation level 0 is a stage where the driver operates all the main control systems such as brake, steering, throttle, and driving force of the vehicle HV without performing automation. In other words, it is a stage where automatic driving is not carried out in which any of acceleration, braking, and steering is not automatically controlled.

  Automation level 1 is a stage in which a function that automates one of the main control systems of the vehicle HV is executed alone. In other words, it is a stage of automating a specific function that automatically controls any one of acceleration, braking, and steering. An example of the stage of automating the specific function is a stage of executing the ACC function, the LKA function, and the ABE function independently.

  The automation level 2 is a stage in which one of the main control systems of the vehicle HV is combined and executed. In other words, it is a stage of automating a composite function that automatically controls a plurality of acceleration, braking, and steering. As an example of the stage of automation of the composite function, there is a stage in which the ACC function and the LKA function are executed in combination, or the ACC function and the LCA function are executed in combination.

  The automation level 3 is a stage in which all of the main control system of the vehicle HV is automated, and the driver performs a driving operation only when the driver changes to a traffic situation to be driven. In other words, it is a semi-automatic operation stage in which acceleration, braking, and steering are all automatically controlled except in an emergency.

  The automation level 4 is a stage in which all of the main control system of the vehicle HV is automated and the driver does not need to perform a driving operation at any time during traveling. In other words, even in an emergency, it is a fully automatic operation stage in which acceleration, braking, and steering are all automatically controlled. The automation level 1 to the automation level 4 can be said to be an automatic driving stage in which at least one of acceleration, braking, and steering is automatically controlled.

  The switching of the automation level in the vehicle control ECU 9 may be performed according to an input operation by the driver to the operation device 42. In addition, the vehicle control ECU 9 may be configured to perform autonomously according to the traffic situation determined based on the monitoring information obtained from the surrounding monitoring ECU 80, the sensing failure in the surrounding monitoring sensor 81, and the like. Further, the stage division when switching the automation level is not limited to the example described above. For example, it may be divided into stages such as a stage where automatic driving is not performed, a stage where some driving operations are performed automatically, and a stage where all driving operations are performed automatically, or may be other stages. .

  The HMI system 4 receives an input operation from the driver of the own vehicle, presents information to the driver of the own vehicle, monitors the state of the driver of the own vehicle, or gives a stimulus to the driver of the own vehicle. To do. The HMI system 4 will be described in detail below.

<Schematic configuration of HMI system 4>
Next, a schematic configuration of the HMI system 4 will be described with reference to FIG. As shown in FIG. 3, the HMI system 4 includes an HCU (Human Machine Interface Control Unit) 40, a driver monitor camera 41, an operation device 42, a display device 43, an audio output device 44, and an actuation device 45.

  The driver monitor camera 41 includes a near infrared light source and a near infrared camera. The driver monitor camera 41 is arranged, for example, on the upper surface of the instrument panel in a posture in which the near-infrared camera faces the driver seat side of the vehicle HV. In the driver monitor camera 41, a range including the face of the driver irradiated with near-infrared light from a near-infrared light source is photographed by the near-infrared camera. A captured image (hereinafter referred to as a driver image) in a range including the driver's face by the near-infrared camera is output to the HCU 40.

  The operation device 42 is a switch group operated by a driver of the vehicle HV. The operation device 42 is used for performing various settings. For example, as the operation device 42, there is a steering switch or the like provided in a spoke spoke portion of the vehicle HV.

  Examples of the display device 43 include a combination meter, a CID (Center Information Display), and a HUD (Head-Up Display). The combination meter is disposed in front of the driver's seat of the vehicle HV. The CID is arranged above the center cluster in the passenger compartment of the vehicle HV. The combination meter and the CID display various images for information presentation on the display screen based on the image data acquired from the HCU 40. The HUD projects light of an image based on the image data acquired from the HCU 40 onto a projection area defined on the windshield of the vehicle HV. The light of the image reflected on the vehicle interior side by the windshield is perceived by the driver sitting in the driver's seat. The driver can visually recognize the virtual image of the image projected by the HUD by superimposing it on the outside scene in front of the vehicle HV.

  Examples of the audio output device 44 include an audio speaker. The audio speaker is disposed in the lining of the door of the vehicle HV. The audio speaker presents information to the occupant by the reproduced voice.

  The actuation device 45 stimulates the driver. As an example, a configuration may be adopted in which a vibrator provided on the steering wheel of the vehicle HV is vibrated to stimulate the driver's hand holding the steering wheel.

  The HCU 40 includes a CPU, a memory such as a ROM and a RAM, an I / O, and a bus for connecting them, and executes various processes by executing a control program stored in the memory. For example, the driver state is determined based on parameters such as a driver image acquired from the driver monitor camera 41, a vehicle signal detected by a sensor mounted on the vehicle HV, and a biological signal received from the wearable device 2 by the short-range communication device 5. judge. As the driver state, there are an inoperable state, an awake state, a side look state, and the like. The HCU 40 corresponds to the driver state determination device in the claims.

  Also, the HCU 40 presents information from the display device 43 and / or the audio output device 44, or causes the driver to stimulate the actuation device 45 according to the determined driver state. Note that part or all of the functions executed by the HCU 40 may be configured by hardware using one or a plurality of ICs.

<Schematic configuration of HCU 40>
Here, a schematic configuration of the HCU 40 will be described with reference to FIG. As shown in FIG. 4, the HCU 40 includes a cooperation status detection unit 401, a biological signal acquisition unit 402, an image acquisition unit 403, a vehicle signal acquisition unit 404, an automation level determination unit 405, a driver state determination unit 406, a presentation control unit 407, A stimulus control unit 408 and a request transmission unit 409 are provided.

  The cooperation status detection unit 401 detects a status where the wearable device 2 of the driver of the vehicle HV is located in the vehicle interior. As an example, the state from when the communication is established between the short-range communication device 5 and the driver's wearable device 2 until the establishment of the communication is canceled is detected as a situation where the driver's wearable device 2 is located in the vehicle interior. . The state from the establishment of communication until the establishment of communication is canceled includes the state before communication is established and before connection, the state where communication is established and connected, and the state where connection has been established but is quickly connected. Includes power-saving states that can be restored.

  A case where the situation where the wearable device 2 of the driver is located in the vehicle interior is detected in the cooperation status detection unit 401 will be described below. When the driver wearing the wearable device 2 gets on the vehicle HV and turns on the ACC power, the short-range communication device 5 is activated. Subsequently, when the wearable device 2 of the driver and the short-range communication device 5 have been paired and the wearable device 2 is selected as a connection target with the short-range communication device 5, the short-range communication is performed. Communication is established between the device 5 and the wearable device 2 and communication is started. At this time, the cooperation status detection unit 401 detects the situation where the wearable device 2 of the driver is located in the vehicle interior. On the other hand, when the driver turns off the ACC power source to get off the vehicle HV, the operation of the short-range communication device 5 is terminated, and the connection between the wearable device 2 of the driver and the short-range communication device 5 is canceled. At this time, the cooperation status detection unit 401 does not detect the situation where the wearable device 2 of the driver is located in the vehicle interior.

  In addition, the detection of the situation where the wearable device 2 of the driver is located in the vehicle interior may be configured on the condition that a predetermined input operation by the driver to the operation device 42 is a condition in addition to the communication state as described above. Good. The predetermined input operation referred to here is an input operation for requesting a connection between the wearable device 2 of the driver and the HCU 40 (specifically, a connection via the short-range communication device 5).

  The biological signal acquisition unit 402 sequentially acquires biological signals received by the short-range communication device 5 from the wearable device 2 and outputs them to the driver state determination unit 406. In the example of the first embodiment, the biological signal is a heart rate and a blood pressure. The image acquisition unit 403 sequentially acquires driver images output from the driver monitor camera 41 and outputs them to the driver state determination unit 406. The biological signal acquisition unit 402 and the image acquisition unit 403 correspond to the determination information acquisition unit in the claims. The vehicle signal acquisition unit 404 sequentially acquires vehicle signals detected by sensors mounted on the vehicle HV and outputs the vehicle signals to the driver state determination unit 406. As an example, the vehicle signal is a vehicle speed detected by a vehicle speed sensor, a steering angle detected by a steering angle sensor, or the like.

  The automation level determination unit 405 determines the automation level of the driving operation in the vehicle HV based on the execution status of the automatic driving function in the vehicle control ECU 9. As an example, the determination may be made as follows. When no automatic driving function is executed, it is determined that the automation level is zero. When the automatic driving function is executed alone, it is determined as the automation level 1. When the automatic driving function is executed in combination so as to automate two of acceleration, braking, and steering, it is determined as the automation level 2. When the automatic driving function is executed in combination so as to automate all of acceleration, braking, and steering, and when the automatic evacuation function is not executable, it is determined as the automation level 3. When the automatic driving function is executed in combination so as to automate all of acceleration, braking, and steering, and when the automatic evacuation function can be executed, it is determined as the automation level 4. .

  In addition, when the stage division | segmentation in the case of switching an automation level is set as the structure different from the example of Embodiment 1, the automation level determination part 405 should just determine an automation level according to the stage division | segmentation. For example, in a configuration in which the steps are classified into the automation level 0, the automation level 1, the automation level 2, the automation level 3 or more, the following may be performed. For example, the determination is made in the same manner as described above from automation level 0 to automation level 2, and automation is performed when the automatic driving function is executed in combination so as to automate all of acceleration, braking, and steering. What is necessary is just to set it as the structure determined to be 3 or more levels.

  Further, in a configuration that performs stage division such as a stage where automatic driving is not performed, or a stage where a part of the driving operation is performed automatically, the automation level may be determined based on whether or not one automatic driving function is being executed. Moreover, what is necessary is just to do as follows in the structure which performs stage division | segmentation, such as the stage which does not carry out automatic driving | operation, the stage which performs some driving operation automatically, and the stage which performs all driving operation automatically. For example, when no automatic driving function is executed, it is determined that automatic driving is not performed. When the automatic driving function is executed so as to automate one or two of acceleration, braking, and steering, it is determined that a part of the driving operation is performed automatically. When the automatic driving function is executed in combination so as to automate all of acceleration, braking, and steering, it is determined that all driving operations are performed automatically.

  The driver state determination unit 406 is based on parameters such as a driver image acquired from the driver monitor camera 41, a vehicle signal detected by a sensor mounted on the vehicle HV, and a biological signal received by the short-range communication device 5 from the wearable device 2. Next, the driver state is determined. The driver state determination unit 406 corresponds to a state determination unit in claims. When the cooperation state detection unit 401 detects the situation where the wearable device 2 of the driver is located in the vehicle interior, the driver state determination unit 406 uses a biological signal in addition to the driver image and the vehicle signal. Switch to a mode where the status can be determined. On the other hand, the driver state determination unit 406 is a mode for determining the driver state without using a biological signal when the cooperation state detection unit 401 does not detect the state where the wearable device 2 of the driver is located in the vehicle interior. Switch to

  Further, the driver state determination unit 406 determines the driver state by changing the weight of each parameter according to the automation level determined by the automation level determination unit 405. As an example, after estimating the driver state for each parameter, the configuration may be such that the driver state is determined so that the driver state estimated from the parameter having a large weight is given higher priority. Below, the case where a driving impossible abnormal state and an arousal state are determined as a driver state is mentioned as an example, and it demonstrates.

  First, the inoperable state and the arousal state from the driver image acquired from the driver monitor camera 41 may be estimated as follows as an example. From the captured image acquired from the driver monitor camera 41, the face direction of the driver, the posture of the driver, the open / close state of the eyelid, and the like are detected by image recognition processing. Based on the detected face orientation and driver posture, the inoperable state is estimated. For example, the driver's inability to drive such as fainting is estimated based on the fact that the situation where the face direction is downward continues and the posture of the driver is broken. In the following, description will be made by taking as an example a case where the operation impossible state is two stages of “operation possible” and “operation impossible”.

  The awake state is estimated based on the detected opening / closing state of the heel and the posture of the driver. For example, the driver's obscure state can be estimated from the frequency of opening and closing the heel, the start of sleepiness can be estimated from the amplitude versus speed of the heel opening and closing, the sleep can be estimated from the duration of the closed state of the heel, Estimate the depth of sleep from the collapse of posture. In the following, the case of the awakening state will be described by taking as an example a case where there are five levels of “awakening”, “manage”, “sleepiness”, “shallow sleep”, and “deep sleep” in descending order of arousal level.

  The estimation of the inoperable state and the arousal state from the vehicle signal detected by a sensor mounted on the vehicle HV may be performed as follows as an example. For example, the inability to drive may be estimated based on whether or not the vehicle speed that is sequentially detected by the vehicle speed sensor and the standard deviation of the steering angle that is sequentially detected by the steering angle sensor exceed a threshold value. The awakening state may be estimated from the vehicle speed that is sequentially detected by the vehicle speed sensor and the change pattern of the steering angle that is sequentially detected by the steering angle sensor. As an example, frequency analysis is performed on time-series data of vehicle speed and steering angle, and based on the integrated value of a specific frequency among the frequency components of the time-series data, it is determined whether the vehicle HV fluctuates and / or the driver's driving operation is smooth. By doing so, the degree of arousal may be detected.

  The estimation of the inoperable state and the awake state from the biological signal received by the short-range communication device 5 from the wearable device 2 may be performed as follows as an example. For example, the configuration may be such that the inoperable state is estimated based on a matrix in which the heart rate and blood pressure are associated with the inoperable state in advance. Moreover, what is necessary is just to set it as the structure which estimates an arousal state based on the matrix which matched the heart rate, blood pressure, and the arousal state beforehand. It is good also as a structure which estimates an arousal state by fluctuation analysis of heart rate variability.

  In the estimation of the arousal state by each parameter, the number of stages of the arousal level for each parameter may be different as long as the correspondence of the level of the arousal level is taken for each parameter. As an example, the estimation of the arousal state from the driver image has five stages of “awakening”, “manage”, “sleepiness”, “shallow sleep”, and “deep sleep”, while the estimation of the arousal state from the vehicle signal is , “Awakening”, “manage”, “shallow sleep”, etc., may be used.

  Subsequently, the weight of each parameter according to the automation level determined by the automation level determination unit 405 in the driver state determination unit 406 will be described. As for the weight of each parameter according to the automation level, the weight of the vehicle signal may be reduced as the automation level increases. Further, the weight of the biological signal may be increased as the automation level increases.

  As an example, based on the automation level determined by the automation level determination unit 405, the importance of each parameter is set from the correspondence relationship in which the importance is associated with each combination of the automation level and the parameter. As a specific example, a configuration using a table in which a coefficient indicating importance is associated with each combination of an automation level and a parameter as shown in FIG. Assume that the total value of importance of each parameter is 1. Further, when it is necessary to determine a plurality of types of driver states such as an inoperable state and an awakened state, a configuration in which a correspondence relationship is prepared for each type of driver state may be used. A parameter having a higher importance level than other parameters has a higher weight.

  In addition, at the automation level 3 or higher, the driver basically does not perform the driving operation, and it is considered that the reliability of the estimation result of the driver state based on the vehicle signal is extremely low. Therefore, as shown in FIG. At level 3 or higher, the vehicle signal is preferably weighted.

  The example of FIG. 5 is an example in a mode in which a driver state can be determined using a biological signal in addition to a driver image and a vehicle signal. In the mode in which the driver state is determined without using the biological signal, a configuration in which a table in which the importance of the biological signal is omitted may be used. In this case, in the table, the sum of the coefficient indicating the importance for the driver image and the coefficient indicating the importance for the vehicle signal may be always 1.

  The driver state determination unit 406 determines the driver state based on the estimation result of the driver state based on each parameter and the weight of each parameter corresponding to the automation level. As an example, a coefficient indicating the importance of each parameter is added to a numerical value corresponding to the stage of each driver state estimated from each parameter, and the integrated values are added together. And the stage of the driver state which the numerical value which rounded off the decimal of the obtained value shows is determined as a driver state. The numerical values corresponding to the stage of the driver state are, for example, “0” for “unavailable”, “1” for “unable to drive”, “0” for “waken”, “1” for “quickly”, “ “Drowsiness” may be 2, “Shallow Sleep” may be 3, and “Deep Sleep” may be 4.

  An example of the determination of the driver state will be described using the example of FIG. For example, in the automation level 2, the estimation result of the awakening state by the vehicle signal is “wakefulness”, the estimation result of the awakening state by the driver image is “sleepiness”, and the estimation result of the awakening state by the body signal is “sleepiness” In this case, 0 × 0.2 + 2 × 0.5 + 2 × 0.3 = 1.6. In this case, since the value obtained by rounding off the decimal number of 1.6 is 2, the driver state is determined to be “sleepy”.

  The presentation control unit 407 causes the display device 43 and / or the audio output device 44 to present information when the driver state determined by the driver state determination unit 406 is a state requiring a warning. For example, when it is determined that driving is impossible, information is presented to the passengers informing the driver of the physical condition abnormality. In addition, when the determined arousal state is equal to or less than a predetermined arousal level, for example, a “random state”, information indicating the arousal level of the driver may be presented. It is preferable that the information indicating the driver's arousal level be provided only when the automation level, such as the automation level 3 or less, is a predetermined level or less.

  When the driver state determined by the driver state determination unit 406 is a state that requires a stimulus for improving the driver state, the stimulus control unit 408 has a timing and device corresponding to the automation level determined by the automation level determination unit 405. This creates a stimulus that improves the driver's condition. The mode of stimulation in the stimulation control unit 408 will be described in detail later.

  The request transmission unit 409 transmits a request to the connected wearable device 2 via the short-range communication device 5. As an example of the request, there is a request for starting stimulation at the stimulation unit 25 of the wearable device 2.

<Driver status determination related processing>
Here, an example of the flow of processing related to determination of the driver status in the HCU 40 (hereinafter referred to as driver status determination related processing) will be described using the flowchart of FIG. In FIG. 6, the case where the HCU 40 can acquire a biological signal measured by the wearable device 2 and the awake state is determined as the driver state will be described as an example. The flowchart of FIG. 6 is a configuration that starts when the ignition power supply of the vehicle HV is turned on and when the cooperation status detection unit 401 detects that the wearable device 2 is located in the vehicle interior. do it.

  First, in step S <b> 1, the biological signal acquisition unit 402 starts acquiring a biological signal received from the wearable device 2 by the short-range communication device 5. After starting the acquisition of the biological signal, it is assumed that the biological signal received from the wearable device 2 by the short-range communication device 5 is sequentially acquired.

  In step S2, the vehicle signal acquisition unit 404 starts acquiring vehicle signals such as the vehicle speed detected by the vehicle speed sensor and the steering angle detected by the steering angle sensor. After acquisition of the vehicle signal is started, the vehicle speed detected by the vehicle speed sensor and the steering angle detected by the steering angle sensor are sequentially acquired.

  In step S <b> 3, the image acquisition unit 403 starts acquiring a driver image output from the driver monitor camera 41. After starting the acquisition of the driver image, it is assumed that the driver image output from the driver monitor camera 41 is sequentially acquired. In addition, the process of S1-S3 may be the structure which changed order, and is good also as a structure performed in parallel.

  In step S4, when it is time to determine the driver state by the driver state determination unit 406 (YES in S4), the process proceeds to step S5. On the other hand, if it is not time to determine the driver state (NO in S4), the process proceeds to step S14. For example, the determination of the driver state in the driver state determination unit 406 is periodically performed with a predetermined time interval, and the period for determining the driver state may be set as the timing for determining the driver state.

  In step S5, the automation level determination unit 405 determines the automation level of the driving operation in the vehicle HV. In step S6, the driver state determination unit 406 sets the importance of parameters such as a vehicle signal, a driver image, and a biological signal according to the automation level determined in S5 based on, for example, the table shown in FIG.

  In step S7, parameters such as the biological signal acquired by the biological signal acquisition unit 402, the vehicle signal acquired by the vehicle signal acquisition unit 404, the driver image acquired by the image acquisition unit 403, and the importance of each parameter set in S7 are obtained. First, the driver's arousal state is determined.

  In step S8, the stimulus control unit 408 determines whether or not a stimulus for improving the arousal state is necessary based on the automation level determined in S5 and the awake state determined in S7. If it is determined that stimulation is necessary (YES in S8), the process proceeds to step S9. On the other hand, if it is determined that no stimulation is required (NO in S8), the process proceeds to step S14. As an example, based on the automation level determined in S5 and the arousal state determined in S7, it is determined whether or not a stimulus is necessary from the correspondence that associates the presence or absence of the stimulus for each combination of the automation level and the arousal state. do it.

  In the example of the first embodiment, as shown in FIG. 7, when the automation level is 0 to 2, it is determined that stimulation is necessary when the degree of arousal is “middle” and “sleepiness”, while the degree of arousal is “arousal”. ”,“ Shallow sleep ”, and“ deep sleep ”, it may be determined that no stimulation is required. In the case of automation levels 0 to 2, when the arousal level is “shallow sleep” or “deep sleep”, it is determined that stimulation is not necessary because the arousal level is “shallow sleep” or “deep sleep” This is because the awakening state should be improved by this. In the case of automation level 3 or higher, it is determined that stimulation is necessary when the arousal level is “manage”, “drowsiness”, “shallow sleep”, “deep sleep”, while the arousal level is “awake” It may be determined that no stimulation is necessary. In the case of the automation level 3 or higher, it is determined that the driver needs to take a nap in a situation where driving operation is unnecessary while driving when the degree of arousal is “shallow sleep” or “deep sleep”. This is because it is assumed.

  In step S9, the stimulus control unit 408 determines a stimulus mode based on the automation level determined in S5 and the arousal state determined in S7. In the first embodiment, it is assumed that the mode of determination is the strength of the stimulus, the device that performs the stimulus, and the timing at which the stimulus is started. As an example, based on the automation level determined in S5 and the arousal state determined in S7, the intensity of the stimulus, the device for performing the stimulation, and the timing for starting the stimulation are associated with each combination of the automation level and the awakening state. What is necessary is just to determine the aspect of a stimulus from correspondence.

  In the example of the first embodiment, as shown in FIG. 7, in the case of automation levels 0 to 2, when the arousal level is “random” or “drowsiness”, the device that performs stimulation is actuated by the vehicle side unit 3. The apparatus 45 determines that the stimulus start timing is immediately after the determination of the arousal state. Further, when the arousal level is “manage”, the strength of the stimulus is determined to be a little stronger, and when the arousal level is “sleepy”, the strength of the stimulus is determined to be higher.

  On the other hand, if the automation level is 3 or higher, the device that performs stimulation is determined to be the wearable device 2. When the level of arousal is “random” when the automation level is 3 or higher, the strength of the stimulus is determined to be weak, and the start timing of the stimulus is determined when switching to the automation level 2 or lower or immediately before arrival at the destination. . The destination may be a destination set in the navigation device, for example. The term “immediately before” here may be, for example, 1 second. When the degree of arousal is “sleepiness”, the intensity of the stimulus is determined to be slightly weak, and the start timing of the stimulus is determined at the time of switching to the automation level 2 or lower or several seconds before the arrival at the destination. When the degree of arousal is “light sleep”, the intensity of the stimulus is determined to be slightly stronger, and the start timing of the stimulus is determined to be switched to the automation level 2 or lower or several tens of seconds before arrival at the destination. When the arousal level is “deep sleep”, the strength of the stimulus is determined to be high, and the stimulus start timing is determined at the time of switching to the automation level 2 or lower or the number of arrivals before the destination.

  The reason why the stimulus start timing is not determined immediately after the determination of the arousal state in the case of the automation level 3 or higher is that the driver basically does not perform the driving operation in the automation level 3 or higher, so it is not necessary to immediately improve the awakening state. It is. Therefore, the timing at which the driving operation of the driver becomes necessary after switching to the automation level 2 or lower, or the timing at which the driver is expected to wake up when arriving at the destination is set as the stimulus start timing.

  In addition, when the automation level is 0 to 2, the device that performs the stimulation is determined as the actuation device 45, whereas when the automation level is 3 or more, the device that performs the stimulation is determined as the wearable device 2. This is because when the automation level is 3 or higher, there is a high possibility that the driver does not hold the steering wheel, and there is a high possibility that the actuation device 45 cannot give a stimulus to the driver. Therefore, as described above, in accordance with an increase in the automation level, the device provided on the member that is touched by the driver during driving operation is provided with a member that the driver touches other than during driving operation. It is preferable to switch to a mode in which stimulation is performed by the apparatus.

  In step S10, when it is the timing to start the stimulus determined in S9 (YES in S10), the process proceeds to step S13. On the other hand, if it is not the timing to start the stimulation determined in S9 (NO in S10), the process proceeds to step S11.

  In step S11, similarly to S4, when it is time to determine the driver state by the driver state determination unit 406 (YES in S11), the process proceeds to step S5 and the process is repeated. On the other hand, if it is not time to determine the driver state (NO in S11), the process proceeds to step S12.

  In step S12, if it is the end timing of the driver state determination related process (YES in S12), the driver state determination related process is ended. On the other hand, if it is not the end timing of the driver state determination related process (NO in S12), the process returns to S10 and the process is repeated. Examples of the end timing of the driver state determination-related processing include that the ignition power supply of the vehicle HV is turned off, and that the cooperation status detection unit 401 no longer detects that the wearable device 2 is located in the vehicle interior. There is.

  In step S13, the stimulation control unit 408 causes the driver to perform stimulation with the intensity of the stimulation determined in S9. When the device that performs the stimulation determined in S9 is the actuation device 45, the stimulation control unit 408 instructs the actuation device 45 to generate a stimulus having the strength determined in S9. . The actuation device 45 that has received the instruction improves the awakening state of the driver holding the steering wheel by generating a stimulus of the instructed intensity with the steering wheel. When the device that performs the stimulation determined in S9 is the wearable device 2, the stimulation control unit 408 causes the request transmission unit 409 to generate the stimulation of the stimulation strength determined in S9. Send a request to In the wearable device 2 that has received the request, the main control unit 20 generates a stimulus of the requested strength from the stimulation unit 25, thereby improving the awakening state of the driver wearing the wearable device 2.

  In step S14, when it is the end timing of the driver state determination related process (YES in S14), the driver state determination related process is ended. On the other hand, if it is not the end timing of the driver state determination related process (NO in S14), the process returns to S4 and is repeated.

  In addition, when the ignition power supply of the vehicle HV is turned on and the cooperation state detection unit 401 cannot detect that the wearable device 2 is located in the vehicle interior, the biological signal is not used. The driver state determination unit 406 may be switched to a mode for determining the driver state to determine the driver state. Even in this case, the same processing as that shown in FIG. 6 may be performed except that the body signal is not used.

  In the flowchart of FIG. 6, information presentation by the presentation control unit 407 regarding the arousal state determined by the driver state determination unit 406 is not described. However, the presentation control unit 407 is configured to perform information presentation indicating the determined arousal state. Alternatively, the information presentation may be performed only when the arousal level is equal to or lower than a predetermined level. Moreover, it is good also as a structure which does not perform the information presentation by the presentation control part 407. FIG.

<Summary of Embodiment 1>
According to the configuration of the first embodiment, when automatic driving is performed, the driver state is determined by increasing the weight of parameters such as the driver image and biological signal rather than the vehicle signal, compared to when automatic driving is not performed. It is possible to suppress the difference between the actual driver state and the determination result due to the vehicle signal that does not change due to the driving operation. Therefore, it is possible to suppress a decrease in the accuracy of determining the driver state even during the automatic operation.

  Further, according to the configuration of the first embodiment, as the automation level increases, the weight of the biological signal is changed to be larger than that of the driver image, and the driver state is determined. become. Details are as follows. As the level of automation increases, the driver no longer needs to perform driving operations, so the need for the driver to look ahead is reduced, and the driver's looking away will increase. As the driver's looking away increases, it becomes difficult to detect wrinkles, eyes, etc. from the driver image, and for example, it becomes difficult to determine the arousal state of the driver state. On the other hand, according to the configuration of the first embodiment, as the automation level increases, the influence of using the driver image is suppressed by determining the driver state by changing the weight of the biological signal larger than the driver image. Thus, it is possible to suppress a decrease in the accuracy of determining the driver state.

  Furthermore, according to the configuration of the first embodiment, since the mode of stimulation for improving the driver state is changed according to the automation level, it is possible to perform stimulation according to the automation level. Details are as follows. In the case of the automation level 2 or lower, since it is necessary for the driver to perform the driving operation, when the driver state is likely to hinder the driving operation, a stimulus of strength corresponding to the driver state is given to the driver. On the other hand, when the automation level is 3 or higher, the driver does not need to perform a driving operation basically, so the driver may take a nap. However, the driver needs to be awake at the time of switching to automation level 2 or lower or at the time of arrival at the destination. Therefore, by stimulating at a strength and start timing according to the driver state, it is possible to wake up the driver with as little trouble as possible.

(Embodiment 2)
Further, the configuration may be such that the driver state to be determined is changed or added according to the automation level (hereinafter, Embodiment 2).

  Embodiment 2 of the present invention will be described below with reference to the drawings. The driver state determination system 1 according to the second embodiment is the same as the driver state determination system 1 according to the first embodiment except that the driver state determination system 1 includes the HCU 40a instead of the HCU 40. As shown in FIG. 8, the HCU 40a includes a cooperation status detection unit 401, a biological signal acquisition unit 402, an image acquisition unit 403, a vehicle signal acquisition unit 404, an automation level determination unit 405, a driver state determination unit 406a, a presentation control unit 407, A stimulus control unit 408 and a request transmission unit 409 are provided. The HCU 40a is the same as the HCU 40 of the first embodiment except that a driver state determination unit 406a is provided instead of the driver state determination unit 406.

  The driver state determination unit 406a, except that the type of driver state to be determined is changed or added according to the automation level determined by the automation level determination unit 405, except for the driver state determination unit 406 of the first embodiment. It is the same. For example, the driver state determination unit 406a may be configured not to determine the depth of sleep when the automation level is 2 or less, but to determine the sleep depth when the level is automation level 3 or higher. This is because the driver may take a nap at the automation level 3 or higher, unlike the automation level 2 or lower.

  In addition, the driver state determination unit 406a does not determine the driver's aside state as the driver state when the automation level is equal to or higher than the predetermined level, but determines the aside state when the automation level is lower than the predetermined level. It is good also as a structure. The predetermined level here may be the automation level 2 or the automation level 3. This is because the need for the driver to confirm safety increases as the automation level decreases. According to the configuration of the second embodiment, as described above, it is possible to determine a necessary driver state according to the automation level.

(Modification 1)
In the above-described embodiment, the configuration in which the weight of the parameter used for the determination of the driver state is changed according to the automation level at the time of performing the automatic driving is shown, but the configuration is not necessarily limited thereto. For example, the parameter weights for automatic driving are the same regardless of the automation level for automatic driving, and the parameter weights used to determine the driver status are changed only when automatic driving is not performed. It is good also as composition to do.

(Modification 2)
In the above-described embodiment, the configuration in which the stimulation control unit 408 changes the strength of stimulation and the timing of stimulation as the mode of stimulation has been described, but this is not necessarily limited thereto. For example, the stimulus control unit 408 may be configured to change only the strength of the stimulus among the strength of the stimulus and the timing of the stimulus as the mode of the stimulus, or the stimulus of the strength of the stimulus and the timing of the stimulus. It is good also as a structure which changes only a timing.

(Modification 3)
In the above-described embodiment, the actuator provided on the steering wheel is given as an example of the actuation device 45, but this is not necessarily limited thereto. For example, it may be a device that blows out cold air toward the driver, or may be configured such that the sound output device 44 that outputs an alarm bears the function of the actuation device 45. In the case of adopting these configurations, it is considered that the driver can be stimulated even at an automation level of 3 or higher. Therefore, the mode of stimulation according to the automation level determined by the automation level determination unit 405 is changed. What is necessary is just a structure which does not perform.

(Modification 4)
In the above-described embodiment, the configuration in which the stimulation control unit 408 performs stimulation for improving the driver state is not necessarily limited thereto. For example, it is good also as a structure which does not provide the stimulus control part 408 and does not perform the stimulus which improves a driver state.

(Modification 5)
In the above-described embodiment, the configuration in which the driver state determination is performed by the driver state determination unit 406 of the HCU 40 regardless of the automation level determined by the automation level determination unit 405 has been described. For example, the configuration may be such that the device for determining the driver state is changed according to the automation level.

  For example, at the automation level 2 or lower, the driver status determination unit 406 determines the driver status, while at the automation level 3 or higher, the wearable device 2 may determine the driver status. In this case, the request transmission unit 409 of the HCU 40 may be configured to cause the wearable device 2 to transmit a request for determining the driver state, and the wearable device 2 may determine the driver state. When the wearable device 2 determines the driver state, the driver state may be determined from the biological signal measured by the measurement related unit 21. Then, information may be presented from the presentation unit 24 or stimulated from the stimulation unit 25 in accordance with the determined driver state.

(Modification 6)
In addition, when automatic driving is performed in which one automatic driving function is performed, the weight of the vehicle signal may be set to zero. In other words, when the automatic driving is performed, the driver state may be determined using parameters other than the vehicle signal such as the driver image and the biological signal instead of using the driver state determination unit 406 to determine the driver state. According to this, the driver state is not determined using a vehicle signal that does not change due to the driving operation of the driver. Therefore, it is possible to suppress a decrease in the accuracy of determining the driver state by determining the driver state using a vehicle signal that does not change due to the driving operation of the driver.

(Modification 7)
In addition, the driver state may be determined using only one parameter of the vehicle signal, the driver image, and the biological signal according to the automation level. As the automation level increases, the reliability of the biological signal in the determination of the driver state increases, while the reliability of the vehicle signal decreases, so the vehicle signal, the driver image, and the biological signal in this order as the automation level increases. The configuration may be such that the parameters used for determining the driver state are switched.

(Modification 8)
In the above-described embodiment, the configuration in which the biological signal received from the wearable device 2 is used for the determination of the driver state by the driver state determination unit 406 is not necessarily limited thereto. For example, it is good also as a structure using the biological signal received from the apparatus which measures biological signals other than the wearable device 2. FIG. As an example of an apparatus for measuring a biological signal other than the wearable device 2, an apparatus for measuring a driver's biological signal with an electrode embedded in a driver's seat, an apparatus for measuring a driver's biological signal with an electrode embedded in a steering wheel, etc. There is.

(Modification 9)
In the above-described embodiment, the configuration in which both the biological signal received from the wearable device 2 and the driver image acquired from the driver monitor camera 41 can be used for the determination of the driver state by the driver state determination unit 406. However, this is not necessarily the case. For example, the driver state determination unit 406 may use only one of the biological signal received from the wearable device 2 and the driver image acquired from the driver monitor camera 41 to determine the driver state.

(Modification 10)
The HCU 40 transmits parameters such as a driver image acquired from the driver monitor camera 41, a vehicle signal detected by a sensor mounted on the vehicle HV, and a biological signal received from the wearable device 2 by the short-range communication device 5 to the server. The driver state determined on the server side may be acquired.

(Modification 11)
The wearable device 2 may be configured to exchange information with the HCU 40 via a mobile phone.

  The present invention is not limited to the above-described embodiments and modifications, and various modifications are possible within the scope of the claims, and technical means disclosed in different embodiments and modifications, respectively. Embodiments obtained by appropriately combining the above are also included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Driver state determination system, 2 Wearable device, 3 Vehicle side unit, 5 Near field communication machine, 9 Vehicle control ECU, 25 Stimulation part, 40, 40a HCU (driver state determination apparatus), 45 Actuation apparatus, 402 Biosignal acquisition Unit (determination information acquisition unit), 403 image acquisition unit (determination information acquisition unit), 404 vehicle signal acquisition unit, 406, 406a state determination unit (driver state determination unit), 408 stimulus control unit

Claims (10)

Used in a vehicle capable of performing automatic driving for automatically controlling at least one of acceleration, braking, and steering,
The implementation failure of the automatic operation is switched,
A vehicle signal acquisition unit (404) for acquiring a vehicle signal of the vehicle, which can determine a driver state which is a state of the driver of the vehicle;
A determination information acquisition unit (402, 403) for acquiring determination information that is information other than the vehicle signal as information capable of determining the driver state;
A state determination unit (406, 406a) that determines the driver state using at least one of the vehicle signal acquired by the vehicle signal acquisition unit and the determination information acquired by the determination information acquisition unit; ,
The said state determination part is a driver state determination apparatus which determines the said driver state by making the weight of the said information for determination larger than the said vehicle signal at the time of implementation of the said automatic driving compared with the time of the non-execution of the said automatic driving. In claim 1,
The state determination unit determines the driver state using the vehicle signal when the automatic driving is not performed, and determines the driver state without using the vehicle signal when the automatic driving is performed. Judgment device. In claim 1,
In the automatic operation, the level of automation can be switched to a plurality of stages,
The state determination unit is a driver state determination device that determines the driver state by increasing the weight of the determination information more than the vehicle signal as the level of automation of the automatic driving increases. In claim 3,
The state determination unit (406a) can perform at least one of change and addition of the driver state to be determined, and change the driver state to be determined according to the level of automation of the automatic driving. And a driver state determination device that performs at least one of addition. In claim 3 or 4,
A stimulus control unit (408) for giving the driver a stimulus for improving the driver state is provided to the driver according to a level of automation of the automatic driving. The driver state determination apparatus which changes the aspect to give. In claim 5,
The stimulus control unit is a driver state determination device that changes the strength of the stimulus that improves the driver state according to the level of automation of the automatic driving. In claim 5 or 6,
The said stimulus control part is a driver state determination apparatus which changes the timing which starts the stimulus which improves the said driver state according to the automation level of the said automatic driving | operation. In any one of Claims 3-7,
The determination information acquisition unit includes a biological signal acquisition unit (402) that acquires a biological signal of the driver as information that can determine the driver state, and information of a captured image obtained by imaging the driver as information that can determine the driver state. A driver state determination apparatus including at least one of an image acquisition unit (403) for acquiring In claim 8,
The determination information acquisition unit includes the biological signal acquisition unit and the image acquisition unit,
The said state determination part is a driver state determination apparatus which changes the weight of the said biosignal and the information of the said captured image according to the level of automation of the said automatic driving | operation, and determines the said driver state. In claim 9,
The said state determination part is a driver state determination apparatus which changes the weight of the said biosignal more largely than the information of the said captured image, and determines the said driver state, so that the level of automation of the said automatic driving | operation increases.

Images