JP2012128440A - Voice interactive device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform smooth interaction adapted to speaker's sensitivity in a voice interaction device.SOLUTION: A response control part 7 outputs a response voice corresponding to an utterance content of a speaker that has been recognized by a voice recognition part 4, from a voice synthesis part 2 through a speaker 8. In this case, a response time, between when the voice recognition part 4 detects an end of the utterance of the speaker and when the response voice starts to be supplied, is normally caused to change corresponding to an utterance speed detected by an utterance speed detection part 5.

Description

  The present invention relates to a voice interactive apparatus that provides a response voice according to a recognition result of a speaker's utterance content by a synthesized voice.

  For example, in an in-vehicle system such as a car navigation system or a hands-free telephone system, recognition of a voice command uttered by a user is performed by comparing the uttered voice from the user with a standard voice generation pattern of a recognition target vocabulary stored in advance. And performing a control process according to the voice command has become widespread. When this type of speech recognition technology is used, it is often impossible to uniquely identify the utterance content simply by analyzing a single utterance speech from the user. Will be done. Therefore, as a countermeasure against such a situation, a voice dialogue device that interactively understands the utterance content by outputting a question (response voice) for specifying the utterance content by the user on the in-vehicle system side and obtaining a response. It is considered.

A general voice interaction apparatus has a uniform structure of response voice synthesis speed and response time (time until response start) with respect to the utterance content of a speaker (user). However, there is a situation that the speaking rate of a speaker varies depending on the characteristics of each individual. For this reason, the conversation on the side of the speaker feels unnecessarily waited until the output of the response voice is finished, or the user feels that the response to the question is urgently required. There was a problem that a situation in which it was difficult to (dialogue with less frustrating speaker) was likely to occur. In order to solve such problems, conventionally, as seen in Patent Document 1, for example, the speaking rate by a speaker is measured, and the output rate of the response voice is changed according to the measured speaking rate. A spoken dialogue device is considered.
Japanese Patent Publication No. 7-21759

  In the above conventional voice interaction device, the output speed of the response voice is simply changed according to the speaking speed of the speaker, so the speed of the response voice is constant during the period until the output is completed. Will be retained. For this reason, even if an event (for example, an incoming call) that the speaker wants the response voice to end early in the middle of the output of the response voice occurs, it is unnecessary to wait until the response voice ends. As a result, there is a risk that smooth dialogue based on the sensitivity of the speaker may become difficult, such as increased irritation on the speaker side. In a speech dialogue apparatus, the so-called “between” in conversation is a very important element in conducting a smooth conversation that matches the sensitivity of the speaker. Because it was constant regardless of the speaking speed of the speaker, it was in a situation where smooth dialogue according to the sensitivity of the speaker became difficult.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a voice dialogue apparatus capable of performing a smooth dialogue in accordance with a speaker's sensitivity.

  According to the first aspect, when the speech content of the speaker is recognized by the voice recognition means, the response control means provides the response voice according to the recognition result as synthesized voice. In this case, speech speed detecting means for detecting the speaking speed by the speaker is provided, and the response control means is a response time from when the speech recognition means detects the end of the speaker's utterance until the response speech is provided. That is, the so-called “between” in the conversation is changed according to the utterance speed detected by the speech speed detecting means. In this way, the “interval”, which is a very important factor in conducting a smooth conversation that matches the speaker's sensibility, is controlled to a length corresponding to the speaker's speaking speed. It will be possible to realize a smooth dialogue based on sensitivity.

  According to the means described in claim 2, the response control means is a period until the environmental condition is canceled when the detection signal taken in from the environmental condition detection sensor satisfies a predetermined environmental condition. Only the standby state in which the provision of the response voice is forgotten will be exhibited. For this reason, for example, an environmental condition where it is preferable that the speaker does not respond to the response voice (when the voice interaction device is mounted as an in-vehicle system, In other words, it is possible to provide a response voice, and the reliability of the dialogue is improved.

  According to the means of claim 3, the response control means, when the recognition result by the voice recognition means contains an operation start command or an operation stop command related to the control target device, The timing is adjusted so as to be optimal according to the environmental condition indicated by the detection signal from the environmental condition detection sensor. For this reason, when the control operation of the control target device based on the operation start command or the operation stop command from the speaker is to be executed, it is considered that it is better not to execute the control immediately. If it is for a vehicle, the start of the control operation of the device to be controlled will be postponed under a period during which a steering operation associated with turning right or left of the vehicle is being performed or a period during which the vehicle is traveling backwards. Therefore, the reliability related to the control of the control target device is improved.

Functional block diagram showing the basic configuration of one embodiment of the present invention Sequence diagram for explaining an example of control contents by the response control unit

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 schematically shows a basic configuration example of a voice interactive apparatus 1 and parts related thereto by a combination of functional blocks. In FIG. 1, a voice interactive apparatus 1 is mounted on a vehicle and configured to perform a voice response process with a speaker (vehicle driver) and a control process according to a voice command by the speaker. Although not specifically shown, data is exchanged with a car navigation device, an engine control ECU, a mobile communication ECU, and the like via, for example, an in-vehicle LAN.

  The voice synthesizing unit 2, the speech speed control unit 3 (corresponding to the speech speed control unit), the voice recognition unit 4 (corresponding to the voice recognition unit), and the speech speed detection unit 5 (corresponding to the speech speed detection unit) constituting the voice interactive device 1. ), The dialogue agent unit 6 and the response control unit 7 (corresponding to response control means) are actually realized by the program of the voice dialogue unit ECU, and the following functions are set for each. ing.

  The voice synthesis unit 2 generates a voice signal based on an instruction from the response control unit 7 in real time by a known voice synthesis process, and outputs the voice signal from the in-vehicle speaker 8 as a response voice. The speaker 8 may be a dedicated speaker, but can also be used as a speaker of a car audio device.

  The speech speed control unit 3 has a function of changing the speech rate of the response voice output from the speech synthesis unit 2 in real time and continuously so as to become a speed based on the instruction speed from the response control unit 7. It is. In this case, the speech speed control unit 3 employs, for example, the well-known TDHS (Time Domain Harmonic Scaling) method of compressing and expanding the time axis of the response speech as the speech speed change algorithm, and the pitch of the response speech The utterance speed is changed with the constant.

The voice recognition unit 4 takes in the voice of the speaker input from the in-vehicle microphone 9 and analyzes the voice (that is, the utterance content of the speaker) by, for example, a voice recognition processing method using keyword spotting. Recognize the necessary vocabulary.
The speech speed detection unit 5 detects (predicts) the speech speed of the speaker based on the vocabulary recognized by the speech recognition unit 4 and its speech time.
The dialogue agent unit 6 is provided for dialogue management, and has a configuration in which commands included in the vocabulary recognized by the speech recognition unit 4 are selected and given to the response control unit 7.

  In addition to the control functions of the speech synthesis unit 2 and the speech speed control unit 3 as described above, the response control unit 7 is a mobile communication device (a mobile phone, a car phone, a data communication module (DCM), etc.) mounted on the vehicle. ), Which has a function of executing the operation control of the control target device group 10 such as a car navigation device or a car audio device, and receives various detection signals from the in-vehicle sensor group 11 (corresponding to an environmental condition detection sensor). It has become so. In this case, as components of the in-vehicle sensor group 11, in addition to a vehicle speed sensor, an acceleration sensor, and a steering angle sensor for detecting a running state and an operation state of the vehicle, an operation state of the mobile communication device is detected. For monitoring the operation status of a predetermined in-vehicle device and an interface unit for outputting route guidance information, driver assistance information such as traffic jam information in a car navigation device, etc. Means are also included, and a line-of-sight recognition device for detecting the line-of-sight direction of the vehicle driver and a microphone for monitoring noise and conversation in the passenger compartment are included as necessary. The detection signal from the in-vehicle sensor group 11 may be input from the engine control ECU, the mobile communication ECU, etc. through the in-vehicle LAN.

In the following, a part related to the gist of the present invention in the control contents by the response control unit 7 will be described together with related actions.
That is, the response control unit 7 is in an inactive state in a normal state, but for example, when a dialogue start switch provided at a position where a speaker (vehicle driver) can operate is turned on, or a voice response When a specific voice command requesting the start of operation of the device 1 is given through the voice recognition unit 4 and the dialogue agent unit 6 (when the speaker inputs the specific voice command through the microphone 9), the voice is switched to the active state and the talk is performed. It is configured to start a dialogue with the person.

  When the conversation with the speaker starts, the speech recognition unit 4 analyzes the utterance content of the speaker input through the microphone 9 to recognize the vocabulary necessary for the conversation, and the commands included in the vocabulary are included. It is selected by the dialogue agent unit 6 and given to the response control unit 7.

The response control unit 7 that has received the above command continuously executes the following controls (1) and (2).
(1) A command given to the speech synthesizer 2 when a pause time having a length corresponding to the speech speed of the speaker detected by the speech speed detector 5 is determined and the pause time has elapsed. Control to instruct the creation of an audio signal for performing a response according to the.

(2) For the speech speed control unit 3, the speech speed detected by the speech speed detection unit 5 is the voice signal generation speed in the voice synthesis unit 2, that is, the speech speed of the response voice output through the speaker 8. Control that determines the instruction speed according to the speaking speed of the person and gives the instruction speed to the speaking speed control unit 3.
In this embodiment, the pause time is controlled to be proportionally shorter as the speaking speed of the speaker is higher, and the indicated speed is a normal speed when no event occurs as will be described later. In the state, the speed is controlled so as to increase proportionally as the speaking rate of the speaker increases.

  The response control unit 7 monitors the detection signal from the in-vehicle sensor group 11 to detect a predetermined event, for example, an event that the speaker wants the response voice to end early (incoming call) Etc.) is set. The response control unit 7 executes the following control (3) when the event occurs in the middle of outputting the response voice as described above.

(3) Control for continuously increasing the voice signal generation speed in the voice synthesizer 2, that is, the speech speed of the response voice, compared to the previous speed (the pitch of the response voice does not change).
In addition, the response control unit 7 detects that the detection signal acquired from the in-vehicle sensor group 11 is in a state where a steering operation associated with a predetermined environmental condition (for example, turning right or left of the vehicle) is being performed, If the condition (state) is satisfied, the following controls (4) and (5) are executed.

(4) Control that presents a state in which speech synthesis processing by the speech synthesizer 2 is prohibited only during the period until the above environmental conditions are canceled, that is, a standby state in which the provision of response speech from the speaker 8 is suspended.
(5) When the environmental condition is satisfied in the middle of outputting the response voice, for example, when the output of the response voice is temporarily stopped and then the environmental condition is canceled, the response voice is started from the beginning. Control to output again.

Further, the response control unit 7 performs the following control (6) when the recognition result by the voice recognition unit 4 includes an operation start command or an operation stop command related to the control target device group 10.
(6) The execution timing of the operation start command or the operation stop command is determined based on the environmental condition indicated by the detection signal acquired from the in-vehicle sensor group 11 (for example, a state in which a steering operation accompanying a right or left turn of the vehicle is being performed, Control that adjusts to the optimal timing according to the state of braking (specifically, for example, when the operation start command is a call command for a hands-free phone by a mobile communication device) In the state where the environmental condition is satisfied, the execution of the transmission command is postponed, and then the transmission command is executed only when the environmental condition is canceled).

  In the sequence diagram of FIG. 2, specific examples of the control contents according to the above (1), (2), and (4) to (6) are shown, which will be described below. That is, the example of FIG. 2 corresponds to a situation in which a vehicle driver who is a speaker makes a hands-free phone call to “A”, and the driving state of the vehicle is straight ahead → right turn (or left turn). In-vehicle sensor group 11, speaker, speech recognition unit 4, speech synthesis unit 2, speech speed detection unit 5, speech speed control unit 3, response control unit 7 The transition of the operation contents of the dialogue agent unit 6 and the control target device group 10 is schematically shown.

S1: The speaker speaks “I want to call” while the vehicle is running straight.
S2: The speech recognition unit 4 recognizes the utterance content “I want to call”, the speech speed detection unit 5 detects the speech rate by the speaker, and the dialogue agent unit 6 uses the command according to the speech recognition result by the speech recognition unit 4 (Command for requesting telephone call) is selected and output to the response control unit 7.

  S3: The response control unit 7 performs pause time / instruction speed determination and voice output processing. During this process, the pause time that is proportionally shortened when the speaker's speaking speed is high and the instruction speed that is proportionally increased when the speaker's speaking speed is high are determined, and the pause time elapses. Later, the instruction speed is given to the speech speed control unit 3 and a predetermined voice signal (for example, “Speaker's utterance is accepted” is indicated to the speech synthesizer 2, and “ Control is performed to instruct the generation of the voice signal “Yes, which is?”.

S4: The voice synthesizer 2 generates an instructed voice signal (“Yes, which is?”) And outputs the voice signal from the speaker 8 as a response voice. The utterance speed is controlled to be the instruction speed from the response control unit 7.
S5: The speaker speaks “Mr. A's mobile phone” (the vehicle remains straight).
S6: The speech recognition unit 4 recognizes the utterance content “Mr. A's mobile phone”, the speech speed detection unit 5 detects the speech rate by the speaker, and the dialogue agent unit 6 responds to the speech recognition result by the speech recognition unit 4 Are selected and output to the response control unit 7.

  S7: The response control unit 7 performs pause time / instruction speed determination and voice output processing. During this processing, the pause time and the instruction speed are determined, and after the pause time has elapsed, the instruction speed is given to the speech speed control unit 3 and a predetermined voice signal (for example, a call destination) is given to the voice synthesis unit 2. Control is performed to instruct generation of a voice signal of “calling Mr. A's mobile phone” for notification.

S8: The speaker performs a steering operation to turn the vehicle to the right (or left).
S9: The in-vehicle sensor group 11 (particularly the steering angle sensor) detects a steering operation for a right turn (or a left turn). In addition, when the gaze recognition device for detecting the gaze direction of the vehicle driver is included in the in-vehicle sensor group 11, the detection output by the gaze recognition device is used for the determination of the right turn or left turn operation of the vehicle. Also good.

S10: The response control unit 7 stops speech synthesis (sound signal generation) by the speech synthesis unit 2.
S11: The speaker performs a steering operation for returning the vehicle to the straight traveling state.
S12: The vehicle-mounted sensor group 11 (particularly the steering angle sensor) detects a steering operation for returning the vehicle to a straight traveling state.
S13: The response control unit 7 starts speech synthesis (sound signal generation) by the speech synthesis unit 2. When the vehicle turns right (or left) before the pause time elapses, it is natural that speech synthesis is performed from the beginning, but the vehicle right turn (or left turn) has passed the pause time. Even when it is performed at the timing and while the response voice is being output, the voice synthesis is started from the beginning.

S14: The voice synthesizer 2 generates an instructed voice signal ("calls Mr. A's mobile phone"), outputs the voice signal from the speaker 8 as a response voice, and the speech speed controller 3 Control is made so that the utterance speed of the response voice becomes the instruction speed from the response control section 7.
S15: An operation in which the speaker suddenly brakes the vehicle is performed for a predetermined period.
S16: The vehicle-mounted sensor group 11 detects a sudden braking operation. This detection can be performed based on the output of the vehicle speed sensor or acceleration sensor of the in-vehicle sensor group 11.

S17: The in-vehicle sensor group 11 (particularly, the vehicle speed sensor) detects the restart of the straight traveling state accompanying the release of the sudden brake.
S18: The response control unit 7 sends a telephone call command to Mr. A to the control target device group 10 (particularly, mobile communication device).

  S19: The control target device group 10 (particularly, mobile communication device) makes a call to Mr. A. Note that the telephone number necessary for making a telephone call is obtained from a telephone number database built in advance in the voice interactive apparatus 1 or on the mobile communication device side, for example. Further, when a call is made, the dial tone is notified from, for example, a mobile communication device. For example, the voice synthesizer 2 generates a dial tone and outputs the dial tone from the speaker 8. Also good.

  In short, according to the configuration of the above-described embodiment, when the utterance content of the speaker is recognized by the voice recognition unit 4, the response control unit 7 converts the response voice according to the recognition result into the voice synthesis unit 2. Will be provided by synthesized speech. In this case, a speech speed detector 5 for detecting the speech speed by the speaker and a speech speed controller 3 for adjusting the speech speed of the response voice are provided. In a normal state, the speech speed of the response voice is Without changing the pitch, the speed is controlled to be proportionally increased as the speaking speed of the speaker is higher. As a result, a smooth dialogue according to the sensitivity of the speaker is possible.

  In addition, the response voice is not output immediately when the speech content of the speaker is recognized by the speech recognition unit 4, but is controlled so as to be proportionally shorter as the speech speed of the speaker is higher. A configuration that is output for the first time when the pause time has elapsed, in other words, the response time from when the speech recognition unit 4 recognizes the utterance content of the speaker until the response speech is provided, that is, a so-called “interval” in conversation. Is changed according to the actual speaking speed of the speaker. In this way, the “interval”, which is a very important factor in conducting a smooth conversation that matches the speaker's sensibility, is controlled to a length corresponding to the speaker's speaking speed. It will be possible to realize a smooth dialogue based on sensitivity.

  On the other hand, when an event (such as an incoming call) that the speaker wants the response voice to finish early during the response voice output period occurs, the speaking speed of the response voice does not change the pitch. , It is controlled to continuously speed up from the previous speed. For this reason, when an event in a category that causes the speaker to feel that the response voice is desired to end early, it is possible to prevent a situation where the response voice is unnecessarily waited until the response voice ends. This makes it possible to have a smooth dialogue based on the sensitivity of the speaker, such as eliminating the risk of increased irritation on the part of the speaker. In this case, since the speaking rate of the response voice changes according to the occurrence of the event, the speaker side indirectly determines that the event has occurred based on the change of the speaking rate of the response voice. There is also an advantage that it can be recognized.

  For example, when an event in a category that causes the speaker to feel that the content of the response voice is surely recognized occurs, control is performed to continuously reduce the speed of the response voice without changing the pitch. In such a configuration, it becomes possible for the speaker side to reliably recognize the response voice content, thereby enabling a smooth conversation according to the speaker's request. Is.

  The response control unit 7 waits for provision of response sound only during a period until the environmental condition is canceled when the detection signal acquired from the in-vehicle sensor group 11 satisfies a predetermined environmental condition. Presents a state. For this reason, for example, an environmental condition where it is preferable that the speaker does not respond to the response voice (a state in which a steering operation associated with a vehicle turning right or left as described in the embodiment is being performed, or a vehicle suddenly Under the condition of braking etc.), the provision of response voices will be delayed and the reliability of the dialogue will be improved.

  When the recognition result by the voice recognition unit 4 includes a command related to the operation of the control target device group 10, the response control unit 7 detects the execution timing of the command from the in-vehicle sensor group 11. It adjusts so that it may become the optimal timing according to the environmental condition shown by. For this reason, when trying to execute the control operation of the control target device group 10 based on the command from the speaker, it is considered that it is better not to immediately execute the control operation (as described in the embodiment). Under the period during which the vehicle is suddenly braked or when the vehicle is traveling backward, the start of the control operation of the control target device group 10 is postponed. The reliability related to the control is improved.

(Other embodiments)
The present invention is not limited to the above-described embodiments, and for example, the following modifications or expansions are possible.
Although the embodiment in which the voice interactive device 1 is mounted on a vehicle has been described, the present invention may be applied to a voice interactive device for controlling the operation of a so-called digital home appliance.
The speech speed control unit 3 is configured to change the speech speed of the response voice by the TDHS method. However, a voice time axis compression / decompression algorithm of another method may be adopted, and the voice signal generation speed may be changed. A method of changing the utterance speed of the response voice may be adopted.

  The response control unit 7 is configured to determine the utterance speed of the response voice for responding to the utterance of the speaker so as to become an instruction speed according to the utterance speed of the speaker immediately before the response. A series of utterance speeds on the speaker side after the start of, is sequentially averaged, and an instruction speed corresponding to the average utterance speed may be determined. According to this configuration, even when the speaking speed of the speaker changes due to a temporary factor, the speaking speed of the response voice eventually converges to a state that matches the sensitivity of the speaker. Therefore, it will be useful for smooth dialogue.

  When a plurality of speakers use the voice interactive device 1, the default value of the instruction speed corresponding to the speaker's speech speed detected by the speech speed detecting unit 5 is used as the talk actually using the voice interactive device 1. It is good also as a structure changed for every person. In such a configuration, the average value of the speaking speed of each speaker is stored separately, and the default value of the indicated speed is changed according to the average value.

  A configuration in which when the speaker utters the next utterance before the output of the response voice in response to the utterance of the speaker is finished, the control is performed to increase the pause time and the utterance speed related to the response voice for the utterance. It is also good. According to this configuration, for example, when there is a reason for the speaker to rush to talk, or for a so-called impatient speaker, it becomes possible.

Claims 1 and 2 are effective not only in a speech dialogue apparatus but also in a system that provides information by speech synthesis, such as an ETC system or a car navigation system voice guide.
Further, in this voice interactive device, the gaze recognition device for detecting the gaze direction of the vehicle driver recognizes that the driver gazes at the microphone 9 or the vicinity thereof, or an object symbolizing the microphone 9, and at that time Voice recognition may be possible.
In claim 2, the provision of the response voice is not limited to a standby state, but may be stopped when a preset condition is met. For example, this is a case where the premise of exchanging information is lost, such as when the driver stops driving (stops).

  1 is a speech dialogue apparatus, 2 is a speech synthesizer, 3 is a speech speed controller (speech speed controller), 4 is a speech recognizer (speech recognizer), 5 is a speech speed detector (speech speed detector), 6 Is a dialogue agent unit, 7 is a response control unit (response control means), 10 is a control target device group, and 11 is an in-vehicle sensor group (environmental condition detection sensor).

Claims (3)

In a voice dialogue apparatus comprising voice recognition means for recognizing the utterance content of a speaker, and response control means for providing a response voice corresponding to the recognition result by synthesized voice,
Comprising speech speed detecting means for detecting the speaking speed of the speaker;
The response control means determines a response time from the time when the voice recognition means detects the end of the speaker's utterance to the start of providing the response voice according to the utterance speed detected by the speech speed detection means. A voice interactive device characterized by changing.   The response control means is configured to capture a detection signal from an environmental condition detection sensor, and when the detection signal satisfies a predetermined environmental condition, a period until the environmental condition is canceled The voice interactive apparatus according to claim 1, wherein only a waiting state in which the provision of the response voice is postponed is exhibited.   The response control unit is configured to be able to control the operation of the control target device and is configured to capture a detection signal from an environmental condition detection sensor, and the operation related to the control target device in the recognition result by the voice recognition unit When a start command or an operation stop command is included, the execution timing of the command is adjusted so as to be an optimal timing according to the environmental condition indicated by the detection signal from the environmental condition detection sensor. The voice interactive apparatus according to claim 1 or 2.

Images