JPH10222337A - Computer system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To surely and easily select and operate a window and an application by voice to enhance operation efficiency in a computer system where GUT (graphical user interface) is provided and an data input and an operation instruction by voice are executed. SOLUTION: Voice from a microphone 1 is recognized by a voice recognizing part as a keyword and the keyword is adopted as a voice recognizing interface 2 for executing conversion into the control command of a system software 8 in a converting part 5 so that a window is changed-over by voice, the active window is discriminated from another window so as to be displayed, task list display for discriminating the active application is executed, an input is executed to the application by voice, task change-over is executed as the input to another application by another input device such as a moush, and the keyword registered in a word dictionary 4 or a keyword list being effective at present is display in a screen. A means for facilitating the input of the keyword is also included in the system.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a computer system having a GUI environment and having a man-machine interface for receiving data input and operation instructions by voice.

[0002]

2. Description of the Related Art A man-machine interface for allowing a user to interact with a computer system uses a keyboard or a pointing device (such as a mouse) to input or instruct data from a person to the computer system using a display device. And the like, the computer system displays various data and responds to the person.

[0003] These man-machine interface
In addition to devices, input and output are recently performed by voice. A computer system is provided with a voice synthesizer and a voice generator as voice output devices, and a voice recognizer and a decoder as voice input devices.

[0004]

When data is input or instructed to a computer system by voice, a user utters a predetermined keyword (word or word), and the computer system recognizes this voice by voice. Decrypt it.

Here, the computer system is an X-Wi
In the case of having a graphical user interface (GUI) environment such as an now system or MS-Windows, a plurality of applications (windows) can be handled on the same screen, but one of the applications (windows) is actually active. is there.

For this type of computer system,
When the user performs an operation on a certain application by voice, the application (window) may be in an active state. However, if another application (window) is active, the user uses the operation. Cannot operate on application.

In such a case, conventionally, a window to be operated is activated by a pointing device such as a mouse, and thereafter, an operation instruction is given by voice, and the operation of a mouse or the like is intervened, thereby deteriorating the operation efficiency. Met.

[0008] If an application to be operated is mistaken, there is a possibility that an unrelated application may be erroneously operated.

In a computer system having a GUI environment, a large number of windows are opened for operation. However, since a large number of windows are overlapped with each other, a user can select an active window and a registered application type, an application task list and a keyword list. The structure may not be understood. Also in this case, there is a trouble that the uppermost window is moved and erased with a mouse or the like, and the lower window is moved to the upper layer to search for an operation target window.

[0010] In addition, various operation instructions such as application switching, control content switching, and screen operation require utterance of a keyword corresponding to the instruction, and require voice recognition and decoding. Operational instructions cannot be given by voice to such memories.

An object of the present invention is to make sure and easy selection and operation of windows and applications by voice in a computer system having a GUI environment and having a man-machine interface for performing data input and operation instructions by voice. Another object of the present invention is to provide a computer system capable of improving operation efficiency.

Another object of the present invention is to provide a computer system which facilitates retrieval of a correct keyword even when a keyword stored by a user is ambiguous or forgotten.

[0013]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention has a GUI environment, obtains a keyword by voice recognition of an operation instruction by voice, decodes the keyword, and executes a control command of system software. In a computer system having a voice recognition interface for converting to a window, the system software activates a window corresponding to the control command when receiving a control command for decoding a specific voice from the voice recognition interface. A switching means is provided.

[0014] With this configuration, by performing window switching using a specific keyword by voice, a window switching operation using a mouse or the like becomes unnecessary. In order to switch windows, a system in which the windows with priorities are sequentially activated is set to a different sound at the start and end of the window switching, so that the system automatically starts switching at a fixed time by the switching start sound. And a system in which switching is stopped by a switching end voice, and a system in which both systems are combined. In addition, a window switching method is also provided in which each window is provided with a “bookmark” having a priority order, and this “bookmark” is specified by voice to activate an arbitrary window. Further, a window switching method is also provided in which window information of a plurality of applications is managed, and when each application is designated by voice, the window is switched to the latest active window when the application is designated, and further, the application is switched to a new window in the application with the same voice.

Further, the system software includes means for displaying a window activated by a control command from a voice recognition interface in a window different from other windows and displaying a task list of each window. I do.

With this configuration, the user can recognize the active window and the application by performing the display in which the active window is distinguished from the other windows by voice input and the task list display in which the active application is distinguished. To facilitate.

According to the control command from the voice recognition interface, one of the plurality of applications uses the voice recognition interface as an input device and the other application uses the voice recognition interface as an input device.
One is characterized by comprising a task switching means using a device other than the voice recognition interface as an input device.

According to this configuration, by switching tasks, the input to the application by voice and the input to another application by another input device such as a mouse can be performed.
A plurality of windows can be set as operation targets.

Further, the speech recognition interface prepares a word for each application and a common word in a word dictionary for speech recognition, and when a specific keyword is given, the word is registered in the word dictionary. Means for giving a control command to the system software to display a keyword list on a screen as a list or to display a currently valid keyword list on a screen.

With this configuration, when a user utters a specific keyword such as a keyword list, a keyword registered in the word dictionary or a currently valid keyword list is displayed on the screen, and the user is displayed from the keyword list displayed on the screen. So that you can know the keywords that you need. The display of keywords on the screen is application-specific, hierarchical display, keyword abbreviations are included so that keywords can be input by voice, and keyword display using the macro function is also available. Including.

Further, the voice recognition interface has a table for converting the keyword into a control command. This table associates one control command with a plurality of keywords, and refers to one of the keywords to convert the control command. It is characterized by the table data structure to be obtained.

According to this configuration, even when the user does not remember the keyword accurately, when the user utters a keyword that is close to or related to the keyword, the control command that the user intends by the speech recognition and the conversion of the keyword to the control command. Is obtained.

Further, the voice recognition interface is characterized in that it has a means for displaying a list of keyword candidates obtained by complementing the sound after the long sound from the utterance input in which the ending of the part where the keyword is known is extended.

With this configuration, the user can extend the ending of the part where the keyword is known, and utter the utterance.
A candidate keyword is displayed on a screen, and the correct keyword is known from this screen display so that an operation instruction can be newly performed using the correct keyword.

When the speech recognition interface recognizes a speech accompanied by a prolonged long sound or a speech obtained by adding a long sound to a keyword, the speech recognition interface responds to the keyword according to the length of the long sound. A means for repeatedly generating a control command is provided.

With this configuration, instead of repeatedly uttering the same continuous control command, one keyword accompanied by a long tone is uttered, so that the same control can be repeatedly performed with one keyword utterance. I do.

The voice recognition interface includes a voice operation button for executing voice recognition only when a user inputs a voice, and an input for executing voice recognition only when an actual input voice is at a level or more appropriate for recognition. It is characterized by comprising a level check unit and a recognition result reliability check unit that checks the reliability of the speech recognition result and extracts only the speech that has been accurately recognized.

With this configuration, it is possible to prevent erroneous recognition due to intrusion of noise or voice with insufficient level, and also to check the reliability of the recognition result to prevent erroneous recognition.

Further, according to the present invention, in the system software, a window at a position designated by the pointing device is set as a window with a high degree of user's attention,
A means for increasing the frame rate of the window is provided.

According to this configuration, a window having a high degree of interest of the user can be easily specified without operating the active window.

[0031]

BEST MODE FOR CARRYING OUT THE INVENTION

(First Embodiment) FIG. 1 is a block diagram of a voice recognition interface unit according to an embodiment of the present invention. The microphone 1 is used as voice input means for a user to give an operation instruction to a computer system.

The speech recognition interface unit 2 includes a speech recognition unit 3, a speech recognition word dictionary 4, a keyword control command conversion unit 5, and a keyword-to-control command table 6.
And

The voice recognition unit 3 checks the voice signal from the microphone 1 against the word dictionary 4 for voice recognition, and outputs a keyword whose recognition is matched as a result. For example, when the user utters “open”, a character string “open” is obtained as a keyword.

As shown in the figure, the speech recognition word dictionary 4
Common words for each application executed on the software of the system and for each application are prepared, and the words and the common words of the currently active application are validated among them.

The keyword control command conversion unit 5 refers to the keyword-to-control command table 6 for the keyword from the speech recognition unit 3 and obtains a corresponding control command.

The table 6 describes a control command associated with each keyword on a one-to-one basis. For example, a control command corresponding to the keyword “open” is described by replacing the operation with a keyboard or a mouse.

The system software 8 of the computer main body 7 includes an MS-Window and an X-Window system.
It has a software configuration having a GUI environment such as a stem, and has a function of responding to a control command from the keyword control command converter 5 in addition to a function of responding to an operation instruction from a keyboard or the like.

The display unit 9 has a graphic display function according to display control from the system software 8, and the display screen 9A has a display control function for displaying a plurality of applications (windows) on the same screen. .

Here, in the present embodiment, the display control means by the system software 8 assigns virtual window priorities to all windows displayed on the display screen 9A and is currently active. Make the window the highest priority.

Further, a window located behind the active window is given the second priority, and a priority is set according to the window position where the window behind the third window is the third window.

The system software 8 sets a specific keyword (for example, “window”), and when the specific keyword is received from the keyword control command 5, the currently active application (window) is set. ) Has a window switching function of setting the window located behind the window, that is, the window having the second window priority, as the active window.

At the time of window switching by this window switching function, the window which has been active until then has the lowest window priority.

Therefore, according to the present embodiment, when the target application is in an inactive state, the user utters a keyword for window switching by voice, so that the voice recognition by the voice recognition unit 3 and the keyword control command are performed. The conversion into the control command is performed by the conversion unit 5, whereby the window switching on the screen 9A is performed by the system software 8, and this switching is switched to a window having a higher priority every time a keyword for window switching is uttered. The desired window can be activated.

Thus, the user only needs to speak a specific keyword without operating a pointing device such as a mouse in order to activate a window to be operated. The operation instruction for the window in the state can be given by voice, the operation efficiency can be improved, and erroneous operation can be prevented.

It should be noted that the data may be input using a keyboard or the like, or may be input by recognizing and decoding the data by the voice recognition interface unit 2.

Also, the case where the window is switched in the order of priority every time a specific keyword (control command) is received has been described, but this window switching method is appropriately changed by changing the software on the system software side. it can.

For example, when a keyword for starting window switching is received, the active window is automatically switched at a certain time interval (for example, every one second), and when a keyword for stopping window switching is received, window switching is stopped. And a method combining the present method and the method of the embodiment. Furthermore, a system may be used in which a set of a window switching command and a window number is uttered by voice, and a window having the window number is set as an active window on the system side.

Also, a window switching method is adopted in which each window displayed on the graphic display is provided with a “bookmark” having a priority order and an arbitrary window is activated by designating the “bookmark” by voice. Can be.

For example, the setting or release of “bookmark” is performed by adding a specific keyword such as “set bookmark 1” or “cancel bookmark 1” to the window that is currently active when the user generates it. "Bookmark 1" is set or canceled. And the specification of the window to be activated is
The window is activated by uttering “Bookmark 1” or the like.

In this case, the “bookmark” to be set and canceled
Is preferably displayed in the form of an icon so that it is possible to see at a glance which "bookmark" is to be set or canceled.
Although the use of the "bookmark" is performed by voice, the setting and cancellation can be replaced by another input means such as a keyboard.

The switching of windows by the above-mentioned "bookmark" is different from the switching by keywords such as an editor.
Switching by window-based "bookmarks" enables a window as intended by the user to be activated with a single utterance.

A window management unit is provided in the system software 8, and the management unit manages window information including a displayed window name, an active window name, and an active window history, and switches to the latest active window of each application. You can also.

For example, a plurality of applications A to D are started, and a child window A is
When “Application B” is uttered by voice in a state where “1” and “A2” are displayed, the management unit searches the latest active window of the application B and activates, for example, the application B1.

Further, when "app B" is uttered continuously by voice, the next app B2 is activated.

(Second Embodiment) In the above embodiment, the window is switched by voice and the application is operated. When a large number of windows are overlapped, the user can determine which application is active. The decision may be lost. In addition, the user may not know what child windows are included in each window.

Therefore, in the present embodiment, as shown in FIG. 2, while a plurality of applications are running, the system software 8 is currently running from a control command corresponding to this keyword by, for example, uttering a “task list”. The application list (each window name) is displayed as a task list as shown in FIG.

At this time, the currently active application (for example, application A) performs knitting display and special color / frame display so that it can be easily recognized that the application is active. If there are further child windows in each window, for example, in the application D, the child windows are simultaneously displayed in a hierarchical structure. Further, the active window can be switched in the display state of the task list.

According to the present embodiment, the user does not need to remember the currently active window. To check the active window, a plurality of window names can be displayed at a glance by uttering a specific keyword such as a "task list". Can be judged and easily recognized.

(Third Embodiment) FIG. 4 shows another embodiment of the present invention. 1 differs from FIG. 1 in that a task switching unit 10 is provided.

The task switching section 10 controls the system software 8 to switch input devices.

Normally, one window (for example, application A) is to be operated. When “task switching B” is given to the interface 2 by voice in this state, this control command is given to the task switching unit 10. .

At this time, the task switching unit 10 switches the input device so that the application B is a voice input target and the application A can be input only from a keyboard or a mouse.

The switching state of the input device is such that the application A and the application B are different from the other applications and are different between the applications A and B in the window display screen so that the user can understand.
For example, display by color is performed.

The return from the input device switching state is performed by a return command to the task switching unit 10 by the interface 2 by giving "task switching B" by voice.

According to this embodiment, a plurality of windows can be operated.

(Fourth Embodiment) FIG. 5 shows another embodiment of the present invention. In the present embodiment, when a specific keyword is given to the keyword control commad conversion unit 5, the keyword list registered in the word dictionary 4 can be displayed and controlled as a list.

For example, in the configuration shown in FIG. 5, when the user utters the voice of “keyword list” through the microphone 1, the voice recognition unit 3 gives the voice recognition result to the keyword control commad conversion unit 5. The conversion unit 5
Reads a keyword group registered in the word dictionary 4 for this keyword, gives it as keyword list data, gives a control command of the keyword list to the system software 8, and controls the display by the system software 8. The list is displayed on the screen 9A.

FIG. 6 shows an example of a keyword list screen display, in which keywords of all applications and a common word list are displayed, and a knitting display is performed so that currently effective keywords can be understood.

When there is an individual application in each application, for example, application A
The word list is displayed in a hierarchical structure for each individual application.

The display of the keyword list may be a keyword display of the currently valid application.

According to the present embodiment, currently effective keywords or all keywords are displayed in the form of a list. Therefore, the user needs to remember keywords for calling the keyword list without remembering keywords. Only needs to be done.

In addition, since a hierarchically structured keyword list is displayed, it is easy to recognize keywords for each application. In addition, it is easy to associate keywords between applications.

(Fifth Embodiment) In this embodiment, similarly to the above-described embodiment, when a specific keyword is given to the keyword control command conversion unit 5, a list of currently valid keywords is listed. This is a case where display control is performed as a table.

For example, when the user utters the voice of the “keyword list” through the microphone 1, a currently effective keyword group is read from the word dictionary 4 and displayed on the screen 9 A. At this time, an abbreviation (number or symbol) corresponding to each keyword group is displayed at the same time.

The subsequent operation instruction is made by voice input of a keyword from the displayed keyword group in the same manner as in the above-described embodiment, or by voice input of an abbreviation (number or symbol) to be added to the keyword group list display. You.

FIG. 7 shows an operation example in which the monitoring status of "Fukuoka" is taken out using a certain monitoring control software.

First, when the user utters the "keyword list", the keyword control command conversion unit 5 reads the currently effective keyword group from the word dictionary 4, and uses the control command and the keyword group to execute a menu by the system software 8. Display as screen A1.

Next, from the displayed menu screen A1,
The user speaks “file” to know the keyword of the file. As a result, a group of effective keywords for the file is read from the word dictionary 4 and displayed as a file screen A2. Next, to select "execute" from the file screen A2, it is possible to use the "execute" voice, but in the figure, a keyword group display of the execution screen A3 is obtained by the abbreviation "1" voice. Similarly, the monitoring control screen A4 is obtained from the execution screen A3 by the voice of the abbreviation symbol "2", and the monitoring status of "Fukuoka" can be obtained by the voice of the abbreviation "4" designating "Fukuoka". .

Here, the keyword "macro" displayed on each of the screens A1 to A4 has a function of converting the uttered words and the order of utterance into a macro, and the frequent operations are executed by uttering fewer words. To make it possible.

FIG. 8 shows a case in which a monitoring status using a macro is obtained. A macro screen A5 is obtained by voice input of the macro symbol "11" on the menu screen A1. On this macro screen A5, the place name "Fukuoka" and the monitoring control distinction "monitoring" are displayed at a time, and the monitoring status display of "Fukuoka" is obtained by the voice of the abbreviation "3".

As described above, according to the present embodiment, it is necessary to remember one keyword for calling the keyword list without preparing the keyword list and without the user remembering the keyword. The required keywords can be easily known from the screen display.

Further, according to the present embodiment, by accepting an abbreviation of a keyword in addition to the keyword itself, the number of uttered words of the user can be reduced, the complexity can be reduced, and the speech recognition rate of the system can be improved accordingly. be able to.

Further, according to the present embodiment, by making the utterance words and the order of utterances in the form of a macro into a macro, operation can be simplified, speeded up, and a user's own operation instruction can be realized.

(Sixth Embodiment) FIG. 9 shows another embodiment of the present invention, and shows table data of a keyword-to-control command table.

In the above-described embodiments, since a window is switched or an operation instruction is made in response to a voice that becomes a specific keyword, it is necessary for the user to remember this keyword. For this reason, when the user's memory is ambiguous and the correct keyword is not correctly remembered, window switching and operation instruction cannot be performed.

For this problem, it is conceivable to prepare a keyword list. However, to dynamically switch applications in a GUI environment, preparing a keyword list for each application is cumbersome and not practical. In addition, it is necessary to rewrite the list each time a change occurs in the application.

Therefore, in the present embodiment, the keyword-to-control command table 6 shown in the figure is prepared so that the control command intended by the user can be obtained without preparing a keyword list and even in the ambiguous storage of the user. prepare.

FIG. 9 shows an example in which a plurality of (three in the figure) keywords “open”, “open” and “open” are prepared for the control command description for file opening.

Therefore, in order for the user to open the file by voice, instead of the correct keyword “open”,
When “open” or “open” is uttered in association with the utterance, the conversion into the correct control command is performed by the conversion unit 5 referring to the keyword control command table 6, and the desired control is performed. Similarly, window switching is performed by preparing a plurality of keywords.

According to the present embodiment, it is not necessary for the user to memorize the correct keywords, and without preparing a keyword list, by uttering a close word that can be associated with the operation, the desired keyword can be obtained. Operating instructions can be obtained.

Note that editing such as adding or changing a plurality of keywords to the table 6 can be performed by inputting from a keyboard, but may also be performed by voice.

The editing by voice is performed, for example, by giving “edit of table 6” as a control command to the system software 8 by voice to make the table editable. Then, a keyword for keyword editing (for example,
“Speech of“ registration ”), and search for a portion having the keyword from the table 6 by uttering the correct keyword,
Next, this is realized by registering the alias keyword in the table by uttering the alias keyword.

(Seventh Embodiment) In the above embodiments, it is necessary to correctly remember at least one keyword. However, since the user's memory is ambiguous, the user forgets the correct keyword, and You may know halfway.

Therefore, the present embodiment provides a system for obtaining a target keyword from a part that is known up to the middle of the keyword.

In the present embodiment, the user inputs a part of the keyword which is known partway by voice, and extends the ending of the part which is known by the voice input and utters the word, thereby complementing the part after the long sound. A list of correct keyword candidates to be displayed is displayed.

For example, if the keyword for displaying the contents of the file is “display”, the keyword “display [hyo]” registered in the table 6 is
If you know [Hyo], you will say "Hyo" and extend the ending.

The keyword control command conversion unit 5 that has received the keyword ending with a long tone searches the keyword in the table 6 and lists the correct keyword candidates starting with the recognized keyword “hyo” (for example, display, standard, evaluation). , ...) are given to the system software 8 to obtain the screen display.

[0098] From this screen display, the user utters the correct keyword "display" again to obtain an operation instruction using the correct keyword "display".

According to the present embodiment, even if the user forgets the correct keyword ending, if the user remembers the beginning of the keyword, the system finds the correct keyword from the keyword list displayed by the system, and finds the correct keyword. You can input keywords by voice.

(Eighth Embodiment) From the previous embodiments, when performing voice input by knowing the voice input of a correct keyword, "enlarge", "reduce" of the screen, and "move" up, down, left, and right as keywords. In such a case, the user repeatedly utters a keyword (for example, “enlarge”, “enlarge”, “enlarge”) so as to obtain an expected screen state.

Therefore, the present embodiment provides a system for simplifying the keyword input of the same continuous operation instruction.

In the present embodiment, when a utterance accompanied by a prolonged long sound at the end of the keyword is repeated, the same continuous control command can be repeated according to the length of the long sound, so that the same utterance of the same keyword is repeated. Make it unnecessary.

For example, the utterance of the keyword "enlargement" for enlarging the screen is uttered with "long" with "long". In response to this utterance, the speech recognition unit 3 recognizes that the operation is an enlargement operation by referring to the word dictionary 4 from the portion of the speech “Kakuda”, and as a keyword “Kakudai” and a code indicating that this is continuous. It is provided to a keyword control command converter 5, and the converter 5 repeats generation of a control command for screen enlargement.

When the speech recognition unit 3 recognizes "i" at the end of the keyword "enlargement", it gives a code indicating stop of repetitive control to the conversion unit 5, whereby the conversion unit 5 executes "enlargement". Of the control command is stopped, and the system software 8 stops the enlarged display control of the screen.

The same applies to the reduction and up / down / left / right movement of the screen, and it can also be used for menu selection in a pull-down menu. That is,
When selecting an item in the pull-down menu, move the cursor up and down for each keyword “up” and “down”
, The cursor is moved up and down as long as the long sound continues, and when the utterance ends, the movement of the cursor is stopped, and the item of the stop position can be selected.

According to the present embodiment, in order to give a continuous operation instruction using the same keyword, the user performs utterance accompanied by a long tone as a keyword, thereby performing continuous control such as enlargement of a screen while the long tone continues. The continuous control is stopped by terminating the utterance when the desired screen state is attained, and the screen state expected by the user can be obtained. As a result, the user can reduce the number of utterances of the keyword by one.
The operation instruction can be easily performed continuously.

In the embodiment, in place of a keyword accompanied by a long sound, a keyword (for example, "Kakudai") with a long sound added to the end is uttered, and the operation instruction of the keyword is repeated as long as the long sound continues. To achieve the same effect.

(Ninth Embodiment) In the system configuration shown in FIGS. 1, 4 and 5, speech recognition generally does not achieve a recognition rate of 100%, so that it is expected that this will contribute to erroneous operation of the system.

Therefore, the present embodiment provides a system for reliably preventing erroneous operation even when an error is made in voice recognition.

FIG. 10 shows a voice recognition interface according to this embodiment. The voice operation button 11 is a software button, and gives a voice input permission signal to the voice input interface 12 by this on operation. By providing the button 11, when the button 11 is turned off, voice input is cut off, and noise due to ambient noise or the like is prevented from entering the voice recognition unit 3, thereby preventing erroneous operation due to noise.

The input level check unit 13 checks the level of the audio signal passed through the interface 12 and cuts off the audio signal whose input level is lower than usual, thereby
Prevent erroneous utterances due to low-level speech recognition.

When the input level check section 13 cuts off the audio signal, the input level check section 13 sends a signal indicating that the cutoff has been made to the response function section 14.

The recognition result reliability check unit 15 checks the reliability of the recognition result by the speech recognition unit 3, and stops passing the recognition result to the conversion unit 5 when the reliability is low. As a result, the operation of the speech recognition by the erroneous speech by checking the reliability is prevented.

The checking unit 15 sends a signal indicating that the operation has been stopped to the response function unit 14 when the reliability is low.

The response function unit 14 checks the check unit 13 or 1
When the voice recognition abnormal signal is received from 5, the user is requested to re-utter. In addition, when an abnormal signal is also received in response to the re-speaking of the request, the voice operation button 11 is forcibly turned off, and the user issues an input request through another input interface (mouse, keyboard, etc.).

Therefore, according to the present embodiment, the voice recognition is executed only when the user inputs a voice by using the voice operation button 11, and the actual input voice is at an appropriate level or higher for the recognition by the input level checker 13. Only when the speech is recognized, the reliability of the speech recognition result is checked by the recognition result reliability check unit 15 so that only the recognized speech can be answered.

In addition, when an abnormality occurs in voice input and recognition,
By prompting the user to input again or by inputting through another interface, reliable voice input and recognition can be obtained.

(Tenth Embodiment) In displaying a plurality of windows at the same time, the frame rate of each window is changed according to the degree of the window (interest level) of the user's attention. Optimized allocation of system resources.

The degree of interest can be estimated from the display state of the window. As a rule, as shown in FIG. 11A, a window closer to the active window has a higher degree of interest, and a smaller window area has a lower degree of interest. The frame rate of the window is changed depending on the degree of interest.

In this rule, the degree of interest is automatically determined according to the display state of the window, and the user needs an operation such as changing the active window to arbitrarily change the degree of interest, which complicates the operation.

Therefore, the present embodiment provides a system for reliably and automatically estimating the degree of interest of the user and obtaining a display of the optimal distribution.

In the present embodiment, as shown in FIG. 11B, the window where the pointing device such as a mouse is located is recognized by the system software as the window having high interest of the user, and the frame rate of the window is determined. Enhance. At this time, the active window is not changed.

Therefore, in this embodiment, by changing the pointer position of a pointing device such as a mouse or the like, the degree of interest of the window is automatically changed and the frame rate is also changed. The user can arbitrarily increase the interest level of a specific window. For this reason, the operation of increasing the frame rate of a window of interest to the user is simplified, and a more user-friendly system can be constructed.

[0124]

As described above, according to the present invention, the window switching by voice is performed by using the voice recognition interface, so that the conventional window switching operation by a mouse or the like becomes unnecessary, and the window switching by voice becomes possible. Operation efficiency can be improved, and erroneous operations can be prevented.

Further, the present invention manages window information of a plurality of applications, switches from the window information to the latest active window of an application when each application is specified by voice, and uses the voice of the same application name to update the new application. Since switching is performed in the order of windows, there is an effect of facilitating designation of a frequently used window among windows of the application.

Further, according to the present invention, the display of the active window is distinguished from other windows by voice input, and the task list display of the active application is performed. This has the effect of making it easier to recognize the application.

Further, according to the present invention, a plurality of windows can be operated as input to an application by voice and input to another application by another input device such as a mouse by task switching. .

Further, according to the present invention, when a user utters a specific keyword such as a keyword list, a keyword registered in a word dictionary or a currently valid keyword list is displayed on a screen. A required keyword can be easily known, and the user can know all effective keywords from the screen display only by remembering one keyword such as a keyword list.

Further, according to the present invention, a table for converting a keyword into a control command has a table data configuration in which a plurality of keywords are associated with one control command and a control command is obtained from any one of the keywords. When the user does not remember the keyword correctly, the user can obtain a control command intended by the utterance of a keyword that is close to or related to the keyword, thereby eliminating the need for accurate storage of the keyword and improving the usability.

Also, according to the present invention, the user can extend the ending of the part where the keyword is known, and utter the utterance.
We've added on-screen suggestions to keywords,
The user can know the correct keyword from the keyword candidate screen display, and can know the correct keyword by remembering a part of the keyword.

Also, according to the present invention, the same control is repeated as long as the long sound continues by uttering one keyword accompanied by a long sound instead of repeatedly uttering the same continuous control command. An operation instruction using a keyword requires only one utterance in which a long sound is added to the keyword, and the same continuous operation instruction is simplified.

Further, according to the present invention, the speech recognition is executed only when the user inputs a speech, and the speech recognition is executed only when the actual input speech is at an appropriate level or more for the recognition. Is checked, and only the voice that is accurately recognized can be responded, so that an erroneous operation due to a voice recognition error can be reliably prevented. In addition, when an abnormality occurs in voice input and recognition, a prompt voice input and recognition can be obtained by urging the user to input again or by inputting through another interface.

Further, according to the present invention, the window in which the pointing device such as a mouse is located is recognized by the system software as a window having a high degree of user interest, and the frame rate of the window is increased. The user can arbitrarily increase the interest level of a specific window without performing, and the operation of raising the frame rate of the window having the interest level of the user is simplified, and there is an effect that a more user-friendly system can be constructed. .

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a window image according to another embodiment.

FIG. 3 is a task list window image according to another embodiment.

FIG. 4 is a block diagram of another embodiment.

FIG. 5 is a block diagram of another embodiment.

FIG. 6 is an image of a keyword list window in another embodiment.

FIG. 7 is an operation instruction procedure diagram by voice in another embodiment.

FIG. 8 is an operation instruction procedure diagram by voice in another embodiment.

FIG. 9 is a data configuration example of a keyword-to-control command table according to another embodiment.

FIG. 10 is a configuration example of a voice recognition interface according to another embodiment.

FIG. 11 is an example of a window display based on the degree of interest in another embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Microphone 2 ... Speech recognition interface 3 ... Speech recognition part 4 ... Word dictionary 5 ... Keyword control command conversion part 6 ... Keyword to control command conversion table 7 ... Computer body 8 ... System software 9 ... Display part 10 ... Task switching part DESCRIPTION OF SYMBOLS 11 ... Voice operation button 12 Voice input interface 13 ... Input level check part 14 ... Response function part 15 ... Recognition result reliability check part

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Hiroshi Hara 2-1-1-17 Osaki, Shinagawa-ku, Tokyo Inside the Meidensha Corporation (72) Inventor Hirotoshi Matsuda 2-1-1-17 Osaki, Shinagawa-ku, Tokyo Inside the company Meidensha (72) Inventor Takeshi Ogata 2-1-1-17 Osaki, Shinagawa-ku, Tokyo Co., Ltd. Inside the company Meidensha (72) Kaoru Mochizuki 2-1-1-17 Osaki, Shinagawa-ku, Tokyo Co., Ltd.Meidensha (72) Inventor Isamu Ikeda 2-1-1-17 Osaki, Shinagawa-ku, Tokyo Inside Meidensha Co., Ltd.

Claims (10)

[Claims] 1. A computer system having a GUI environment and having a voice recognition interface that obtains a keyword by voice recognition of an operation instruction by voice, decodes the keyword, and converts the keyword into a control command of system software. A computer system, characterized in that the wearer has window switching means for, when receiving a control command obtained by decoding a specific voice from the voice recognition interface, activating a window corresponding to the control command. 2. The system software manages window information of a plurality of applications, and switches from the window information to the latest active window of the application when each application is designated by a control command from the voice recognition interface.
2. The computer system according to claim 1, further comprising means for switching to a new window order of the application by a voice having the same application name. 3. The system software further comprises means for displaying a window activated by a control command from a voice recognition interface in a window different from other windows and displaying a task list of each window. The computer system of claim 1, wherein 4. A task switching means in which one of a plurality of applications uses a voice recognition interface as an input device and the other application uses a device other than the voice recognition interface as an input device according to a control command from the voice recognition interface. The computer system according to claim 1, further comprising: 5. The speech recognition interface prepares a word for each application and a common word in a word dictionary for speech recognition, and when a specific keyword is given, the word is registered in the word dictionary. 2. A system according to claim 1, further comprising means for giving a control command to the system software for displaying a keyword list on a screen as a list or displaying a currently valid keyword list on a screen.
Or the computer system according to 2. 6. The voice recognition interface has a table for converting the keyword into a control command. This table associates one control command with a plurality of keywords, and references the control command by referring to any one of the keywords. The computer system according to any one of claims 1 to 5, wherein a table data structure to be obtained is obtained. 7. The speech recognition interface according to claim 1, further comprising means for displaying a list of keyword candidates obtained by complementing a long sound after the utterance input obtained by extending the ending of a part where the keyword is known. The computer system according to any one of claims 1 to 5. 8. The speech recognition interface, when recognizing a speech accompanied by a prolonged long sound or a speech obtained by adding a long sound to a keyword, responds to the keyword according to the length of the long sound. The computer system according to any one of claims 1 to 5, further comprising means for repeatedly generating a control command. 9. The voice recognition interface includes a voice operation button for executing voice recognition only when a user performs a voice input, and an input for executing voice recognition only when an actual input voice is at or above a level suitable for recognition. 9. The apparatus according to claim 1, further comprising: a level check unit; and a recognition result reliability check unit for checking the reliability of the speech recognition result and extracting only the speech that has been accurately recognized. A computer system according to any one of the preceding claims. 10. The system software according to claim 1, wherein the system software sets a window at a position designated by a pointing device as a window with a high degree of user's attention, and includes means for increasing a frame rate of the window. 3. The computer system according to 3.