JPH1173274A - Visual line input intention transmitting device and method therefor and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a visual line input intention transmitting device for allowing a heavily physically handicapped person who can not push a step button to easily perform intention decision for character input. SOLUTION: Relating to a visual line input intention transmitting device, the video of the Japanese syllabary (or katakana or alphabets) is displayed through a display controller 31 on a liquid crystal display device in a head mount display and an outside monitor 1009. The visual line position of a user is detected by a visual line detecting circuit, and a character at the visual line position is calculated based on the detected visual line position data. Then, the data for the past 100 times of the selected character are updated. When the condition of a character input is fulfilled by this new character selection, addition to an input character column is operated, and the voice output of the newly inputted character is operated so that the completion of the character input can be reported to the user. Thus, the operator can perform character input by the visual line in any attitude.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a gaze input intention transmitting device, method, and storage medium for transmitting intention by detecting a gaze.

[0002]

2. Description of the Related Art Conventionally, kana characters, alphabets, and the like are displayed on a monitor, an image of a user's eyeball is captured by a camera, and a gaze position is analyzed. A line-of-sight input intention communication device for performing input is known (Japanese Patent Application Laid-Open No. 61-223929).

[0003]

However, in the above-mentioned conventional gaze input communication device, when the user makes a decision on character input, the user needs to press the step button, and the user cannot press the step button. It was difficult for those who used it.

Accordingly, an object of the present invention is to provide a gaze input intention communication device, a method and a storage medium which can easily make a character input decision for a severely disabled person who cannot press the step button. I do.

[0005]

In order to achieve the above object, a gaze input communication device according to claim 1 of the present invention comprises: video display means for displaying an image on a display panel; and an image on the display panel. A gaze detecting means for detecting a gaze of a user facing the gaze, and a gaze input intention transmitting device for transmitting the intention of the user by the detected gaze, wherein the gaze position detected by the gaze detection means, An option determining means for determining an option on the display panel being watched by a user is provided.

According to a second aspect of the present invention, there is provided an eye-gaze input intent communication device including eye-gaze detecting means for detecting a user's eye-gaze facing an image on a display panel, and transmitting the user's intention by the detected eye-gaze. The input intention communicating device is characterized in that it has an option judging means for judging an option on the display panel which the user is gazing at, based on the gaze position detected by the gaze detecting means.

According to a third aspect of the present invention, there is provided a gaze input communication device.
An image display unit for displaying an image on the display panel; and a line-of-sight detection unit for detecting a line of sight of a user facing the image on the display panel, and a line-of-sight input intention for transmitting the user's intention by the detected line of sight. A head position detecting means for detecting a head position of the user; a gaze position correcting means for correcting a gaze position detected by the gaze detecting means with the detected head position; And an option judging means for judging an option on the display panel being watched by the user based on the determined gaze position.

According to a fourth aspect of the present invention, there is provided a gaze input communication device for projecting an image of a display panel onto a screen and displaying the image, and a gaze for detecting a user's line of sight facing the displayed image of the display panel. A gaze input intention transmitting device for transmitting the user's intention by the detected line of sight, the display panel being watched by the user based on the line of sight detected by the line of sight detecting unit. An option determining means for determining the above option is provided.

According to a fifth aspect of the present invention, there is provided a gaze input communication device for projecting an image of a display panel onto a screen for display, and a gaze for detecting a user's line of sight facing the displayed image of the display panel. A gaze input intention transmitting device for transmitting the user's intention by the detected line of sight, comprising: a head position detecting unit for detecting the position of the user's head; Gaze position correction means for correcting a gaze position detected by the gaze detection means at a position, and option judgment means for judging an option on the display panel which the user is gazing at, with the corrected gaze position It is characterized by having.

According to a sixth aspect of the present invention, in the gaze input intention transmitting device according to any one of the first to fifth aspects, the option judging means determines whether the gaze position is the same for a predetermined time or more. , An option to be input is determined.

According to a seventh aspect of the present invention, in the gaze input intention transmitting device according to any one of the first to fifth aspects, the option judging means determines whether or not the gaze position is present on the option. The user may close the eyes for a predetermined time to determine an option to be input.

In the eye-gaze input intention communicating device according to the present invention, in the eye-gaze input intention communication device according to any one of claims 1 to 5, the option judging means may be arranged so that after the gaze position exists on the option. The user may press a switch for a predetermined time to determine an option to be input.

According to a ninth aspect of the present invention, in the sight line input intention communicating device according to any one of the first to fifth aspects, the option determining means determines whether the sight line position is the same for a predetermined time or more. Means for deciding an option to be entered by being located in the area; means for deciding an option to be entered by the user closing his / her eyes for a predetermined time after the gaze position is present on the option; and And means for determining options to be input by the user pressing a switch for a predetermined period of time, and switching means for switching the means for determining these options.

According to a tenth aspect of the present invention, in the gaze input intent communication device according to the sixth aspect, the user can change a predetermined time until determining an input option. It is characterized by having.

The gaze input communication device according to the eleventh aspect of the present invention is configured such that the user keeps his / her eyes closed for a predetermined time until an option to be input is determined in the gaze input communication device according to the seventh aspect. Is provided with changing means that can be changed.

According to a twelfth aspect of the present invention, the user changes a predetermined time during which the switch for determining an input option is pressed in the sight line inputting communicator according to the eighth aspect. Characterized in that it is provided with changing means capable of performing the following.

According to a thirteenth aspect of the present invention, in the gaze input intention communicating device according to any one of the first to fifth aspects, the option judging means comprises: The option to be input is determined by being on the same option at the ratio of.

According to a fourteenth aspect of the present invention, there is provided the gaze input communication device according to the thirteenth aspect, further comprising a change unit that allows the user to change the predetermined ratio. And

In the gaze input communication device according to a fifteenth aspect, in the gaze input communication device according to any one of the first to fifth aspects, the user can change the number of options of the display panel. It is characterized by having a changing means that can be used.

According to a sixteenth aspect of the present invention, in the gaze input intention transmitting device according to any one of the first to fifth aspects, switching between the displayed image of the display panel and the external input image is performed. And a switching means that can be performed by the user.

In a preferred embodiment, the external input image is a television, a video, a computer, a video-on-demand device, a clock, and a security device. , A video phone or the like.

The gaze input intention communicating device according to the eighteenth aspect is characterized in that the gaze input intention communicating device according to the sixteenth aspect is provided with operating means for allowing the user to operate an external environment device. I do.

[0023] In the eye-gaze input communication device according to claim 19, in the eye-gaze input communication device according to claim 18, the external environment device is a lighting device, a television, an audio device, a video, a computer, a video-on-demand. It is a device, a telephone, a videophone, a clock, an electric blind, an electric curtain, an electric window opening / closing device, an air conditioner, a security device, an electric reclining bed, an excrement device, a lifter or a nurse call.

According to a twentieth aspect of the present invention, there is provided a gaze input intention communication device, wherein the gaze input intention communication device according to the first aspect further includes diopter adjustment means for adjusting diopter of the displayed image. .

According to a twenty-first aspect of the present invention, there is provided the gaze input communication device according to any one of the first to fifth aspects, further comprising an erasing means for erasing the image on the display panel. And

A gaze input communication device according to a twenty-second aspect of the present invention is the gaze input communication device according to the twenty-first aspect, further comprising a switching unit that allows the user to switch between displaying and erasing the image on the display panel. It is characterized by the following.

In the eye-gaze input communication device according to claim 23, in the eye-gaze input communication device according to claim 22, the erasing means erases the image on the display board when the user closes his / her eyes for a predetermined time. It is characterized by doing.

According to a twenty-fourth aspect of the present invention, there is provided the gaze input intention communicating device according to the twenty third aspect, further comprising a change unit that can change the predetermined time during which the user keeps his or her eyes closed. It is characterized by the following.

The gaze input communication device according to a twenty-fifth aspect is characterized in that, in the gaze input communication device according to the twenty-second aspect, the user turns the gaze in a predetermined direction to delete the image on the display panel. And

The gaze input communication device according to the twenty-sixth aspect is characterized in that the gaze input communication device according to the twenty-fifth aspect further comprises a change unit that can change a predetermined direction in which the user turns the gaze. And

According to a twenty-seventh aspect of the present invention, there is provided a gaze input communication device according to any one of the first to fifth aspects, further comprising an area limiting means for limiting an area that can be input by the gaze. And

In the eye-gaze input intention communicating device according to claim 28, in the eye-gaze input intention communication device according to claim 27, switching can be performed so as to switch whether or not the user can limit an area that can be input by the eye line. Means are provided.

A sight line input intention communicating device according to a twenty-ninth aspect is the sight line input intention communicating device according to the twenty-eighth aspect, further comprising changing means for changing an area in which the user has limited input by the sight line. Features.

According to a thirtieth aspect of the present invention, in the gaze input intent communication device according to the twenty-seventh aspect, the user can close an eye for a predetermined time to limit an area in which the user can input by the gaze. Features.

A gaze input communication device according to a thirty-first aspect of the present invention is the gaze input communication device according to the thirtieth aspect, further comprising a change unit that can change the predetermined time during which the user keeps his or her eyes closed. It is characterized by the following.

According to a thirty-second aspect, in the gaze input communication device according to the thirty-seventh aspect, the user turns the gaze in a predetermined direction to limit an area where the user can input by the gaze. It is characterized by.

A gaze input intention communicating device according to a thirty-third aspect is characterized in that the gaze input intention communicating device according to the thirty-second aspect is provided with changing means capable of changing a predetermined direction in which the user turns his / her gaze. And

A gaze input intention communicating device according to a thirty-fourth aspect is characterized in that, in the gaze input intention communicating device according to the twenty-seventh aspect, the gaze position is not displayed in an area where the input by the gaze is restricted.

A gaze input communication device according to a thirty-fifth aspect is the gaze input communication device according to the twenty-seventh aspect, wherein the user displays or does not display the gaze position in an area where input by the gaze is restricted. Characterized in that it comprises a changing means capable of changing the setting.

According to a thirty-sixth aspect of the present invention, there is provided a gaze input intention communicating device according to any one of the first to fifth aspects, further comprising changing means for changing the size of the displayed display panel. Features.

A gaze input communication device according to a thirty-seventh aspect of the present invention is the gaze input communication device according to the thirty-sixth aspect, further comprising changing means for allowing the user to change the size of the display panel. .

According to a thirty-eighth aspect of the present invention, there is provided a gaze input intention communicating device including a change unit for changing a cursor position of a created text display section displayed by the gaze input intention communicating device according to any of the first to fifth aspects. It is characterized by having.

A sight line input communication device according to a thirty-ninth aspect of the present invention is the sight line input communication device according to the thirty-eighth aspect, further comprising switching means for switching whether or not the user changes the cursor position based on the sight line. It is characterized by the following.

[0044] In the eye-gaze input intention transmitting device according to claim 40, after the eye-gaze input is determined in the eye-gaze input intention communication device according to any one of claims 1 to 5, the input by the gaze is not accepted for a predetermined time. It is characterized by.

In the eye-gaze input intention communicating device according to claim 41, in the eye-gaze input intention communication device according to claim 40, the user changes a predetermined time during which the eye-gaze input is not accepted after the user confirms the line-of-sight input. A change means is provided.

A sight line input intention communicating device according to a twenty-second aspect is characterized in that the sight line input intention communicating device according to any one of the first to fifth aspects further comprises a printing means for performing printing in accordance with the option based on the sight line. .

A visual line input intention communicating device according to a thirty-second aspect is characterized in that, in the visual line input intention communicating device according to the thirty-second aspect, the paper is not discharged every time the printing is executed.

In the eye-gaze input intention communicating device according to claim 44, in the eye-gaze input intention communication device according to claim 43, the user can change whether to discharge the paper every time the printing is executed. It is characterized by having a changing means that can be used.

A gaze input intention communicating method according to claim 45, wherein an image on a display panel is displayed, a gaze of a user facing the image on the display panel is detected, and the intention of the user is determined by the detected gaze. In the method of communicating a gaze input intention, a choice on the display panel being watched by the user is determined based on the detected gaze position.

The storage medium according to claim 46 is a storage medium storing a program to be executed by a computer in a gaze input intention transmitting device for transmitting a user's intention by a line of sight. Displaying an image, detecting a user's line of sight facing the image on the display panel, and determining an option on the display panel being watched by the user based on the detected line of sight position And characterized in that:

[0051]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A gaze input communication device according to the present invention,
An embodiment of a method and a storage medium will be described.

[First Embodiment] First, the principle of the visual line detection method will be described. FIG. 1 is a diagram illustrating the principle of the eye gaze detection method. FIG. 1A is a top view, and FIG. 1B is a side view.

In the figure, reference numerals 906a and 906b denote light emitting diodes (IR) which emit infrared light insensitive to an observer.
ED), and each light source is disposed substantially symmetrically in the x direction (horizontal direction) with respect to the optical axis of the imaging lens 911, and y
It is arranged slightly below in the direction (vertical direction). Part of the illumination light reflected by the observer's eyeball forms an image on the image sensor 912 through the imaging lens 911. FIG. 2 is a diagram showing an eyeball image projected on the image sensor 912 and the signal intensity thereof.

First, considering a horizontal plane (see FIG. 1A), infrared light emitted from the light source 906b illuminates the cornea 910 of the eyeball 908 of the observer. At this time, the cornea 91
The corneal reflection image d (virtual image) formed by the infrared light reflected on the surface of the image sensor 912 is condensed by the imaging lens 911 and forms an image at a position d 'on the image sensor 912. Similarly, the infrared light emitted from the light source 906a illuminates the cornea 910 of the eyeball. At this time, a corneal reflection image e (virtual image) formed by infrared light reflected on the surface of the cornea 910 is condensed by the imaging lens 911 and forms an image at a position e ′ on the image sensor 912.

Light beams from the ends a and b of the iris 904 form images at positions a 'and b' on the image sensor 912 via the imaging lens 911.

The eyeball 90 with respect to the optical axis of the imaging lens 911
In the case where the rotation angle θ of the optical axis 8 is small, assuming that the x coordinates of the ends a and b of the iris 904 are xa and xb, a large number of points xa and xb are obtained on the image sensor. (See the crosses in FIG. 2A). Therefore, the pupil center x is obtained by the least square method of the circle.
Calculate c. On the other hand, assuming that the x coordinate of the center of curvature o of the cornea 910 is xo, the rotation angle θx with respect to the optical axis of the eyeball 908
Is represented by equation (1). oc · sin θx = xc−xo (1) Further, xo is obtained by the equation (2) in consideration of a predetermined correction value δx at the midpoint k of the corneal reflection images d and e. xk = (xd + xe) / 2 xo = (xd + xe) / 2 + δx (2) Here, δx is a numerical value obtained geometrically from the installation method of the apparatus, the eyeball distance, and the like. Therefore, equation (1),
According to (2), θx is obtained as shown in Expression (3). θx = arcsin [[xc − ｛(xd + xe) / 2 + δx}] / oc] (3) Furthermore, the coordinates of each feature point projected on the image sensor are added with “′ (dash)”. When rewritten, it becomes as shown in Expression (4). θx = arcsin [[xc ′ − {(xd ′ + xe ′) / 2 + δx ′}] / oc / β] (4) where β is the distance sz of the eyeball to the imaging lens 911.
e, and is actually obtained as a function of the interval | xd'-xe '| of the corneal reflection image.

Considering the vertical plane (see FIG. 1B), 2
The corneal reflection images generated by the IREDs 906a and 906b are generated at the same value, and are defined as i. Eye rotation axis θ
The method for calculating y is almost the same as that for the horizontal plane, except for equation (2). If y is the y coordinate of the corneal curvature center o, then yo is obtained by equation (5). yo = yi + δy (5) Here, δy is a numerical value obtained geometrically from the arrangement method of the apparatus, the eyeball distance, and the like. Therefore, the rotation axis θy in the vertical direction is represented by Expression (6). θy = arcsin [[yc ′ − (yi ′ + δy ′)] / oc / β] (6) Further, position coordinates (xn, y) on the screen of a finder or the like.
When n) is a constant m determined by the finder optical system, the values are expressed by equations (7) and (8) on a horizontal plane and a vertical plane, respectively. xn = m · arcsin [[xc ′ − ｛(xd ′ + xe ′) / 2 + δx ′}] / oc / β] (7) yn = m · arcsin [[yc ′ − ｛(yi ′ + δy ′) ] / Oc / β] (8) As is apparent from FIG. 2, the pupil edge is detected using the falling (xb ′) and rising (xa ′) of the output waveform of the image sensor. The coordinates of the corneal reflection image use sharp rising portions (xe ') and (xd').

Next, a personal computer (personal computer) system having a gaze detection function will be described as an example of the gaze input intention communication device.

FIG. 3 is a diagram schematically showing a configuration of a personal computer system having a visual line detection function. The personal computer system includes a personal computer unit 1008 which is a personal computer main body, a head mounted display 1006 for an operator to observe the personal computer screen, and an external monitor 1009 for the operator or a person other than the operator to observe the personal computer screen. Have been.

The head mounted display 1006 is
It is fixed to the vicinity of the operator's eyes by members such as goggles and eyeglass frames.

The head mounted display 1006 is
A liquid crystal display element 1002, a special prism 1003 for realizing a magnifying observation system, a line of sight detection circuit 1064 for detecting the line of sight of the photographer's eye 1005, a liquid crystal display circuit 1007 for displaying a personal computer screen on the liquid crystal display element 1002, Infrared light emitting diodes 1060 and 1061 for irradiating the left and right eyes 1005 with infrared light, imaging lenses 1062a and 1062b for imaging infrared light reflected by the eyeball, and image formation through the imaging lens system 1062 And a photoelectric conversion element 1063 for converting the converted infrared light into an electric signal.

The eye-gaze detecting circuit 1064 is provided with a gazing point detecting circuit for obtaining the gazing point on the liquid crystal display element 1002 of the photographer based on the photographer's eyeball image on the photoelectric conversion element 1063.

The optical function of the observation system in the head mounted display 1006 having such a configuration will be described. The light from the liquid crystal display element 1002 is refracted and transmitted on the third optically active surface c, totally reflected on the first optically active surface a, reflected on the reflective layer of the second optically active surface b, and returned to the first optically active surface b again. The light is refracted and transmitted through the optical working surface a, and is emitted as a light beam having a divergent angle (convergent angle, parallel) suitable for the diopter of the observer.

Here, a line connecting the eye 1005 of the observer and the center of the liquid crystal display element 1002 is shown as the basic optical axis. The adjustment to the diopter of the observer is performed by the liquid crystal display element 100.
2 can be made by moving the prism 2 in parallel along the optical axis of the prism 1003. In order to make the prism 1003 a telecentric system by correcting image performance and distortion, it is preferable that each of the three working surfaces is formed of a three-dimensional curved surface having no rotation target axis, including the basic optical axis, It has a curved surface structure that is symmetric only with respect to a plane parallel to the paper surface.

Subsequently, the head mounted display 1
The optical action of the visual axis detection system in 006 will be described. Light emitted from the infrared light emitting diodes 1060 for the naked eye (two in the depth direction of the paper) and the infrared light emitting diodes 1061 for the eyeglasses (two in the depth direction of the paper) passes through the opening provided in the second optical working surface b. 1012, 1013, 1014,
Through 1015, the observer's eyes are illuminated from a direction different from the optical axis of the visual axis detection system.

The illumination light is reflected and scattered by the cornea and the pupil of the observer, and the light reflected by the cornea forms a corneal reflection image, and the light scattered by the pupil forms a pupil image. These lights are
An image is formed on the image sensor 1063 by the imaging lens system 1062 through the opening 1010 formed in the optically active surface b.

An eye image obtained from the image sensor 1063 is input to a visual axis detection circuit 1064 configured based on the above-described visual axis detection principle, and is supplied to the visual axis detection circuit 106.
Reference numeral 4 outputs gaze point data.

Here, the imaging lens system 1062 is composed of two lenses 1062a and 1062b. Especially,
Since the lens 1062b is a wedge-shaped lens, the imaging lens system can be configured with a small number of lenses, which is suitable for miniaturization. By providing a curvature to the oblique surface of the lens, eccentric aberration generated on the second optically active surface b can be effectively corrected. Further, providing at least one aspheric surface in the imaging lens system is effective in correcting off-axis imaging performance.

When the stop of the imaging lens system is closer to the opening 1010 formed on the second optically active surface b, the opening 1010 can be made smaller, and the center of the observation system can be prevented. Since it is effective, it is desirable that the aperture and the aperture coincide if possible. Setting the opening to a size smaller than 2 mm is smaller than the pupil of the eye, and is effective in preventing the observer from dropping out.

As light for illuminating the eyes, light with low visibility is preferable, so infrared light is used. At this time, if at least one lens serving as a member for cutting visible light is provided in the imaging lens system, the detection accuracy of the line of sight can be improved.

FIG. 4 is a diagram showing the shape of the prism 1003 viewed from the side. Although the mirror surface for reflection is applied to the optically active surface b, the opening 1010 for the imaging lens system 1062 and the infrared light emitting diode 106
The openings for 0 and 1061 are unprocessed portions of mirror coating (1012 and 1013 are for infrared light emitting diodes for the naked eye, 1014 and 1015 are for infrared light emitting diodes for glasses).

Incidentally, these uncoated portions of the mirror coating are so small that they do not affect the finder observation system, and the diameter is preferably about 2 mm or less. Thus, the opening is formed in the mirror coating, and the illumination light source is used as the prism 1
003 on the opposite side of the eye
Even if the field of view is increased by increasing the refractive power of 003, it is possible to appropriately illuminate from a position close to the eye level.

In this embodiment, the infrared light emitting diodes for the naked eye and the eyeglasses are used in different arrangements. Two infrared light emitting diodes 1060 for the naked eye are arranged one by one on the left and right with a narrow width symmetrically with the optical axis at the same height from the lower part slightly away from the optical axis, while two infrared light emitting diodes 1060 for the eyeglasses are used. The light emitting diodes 1061 are arranged one by one on the left and right sides with a wide width symmetrically with respect to the optical axis at the same height from the lower part far away from the optical axis. There are three reasons for this. One is to obtain better illumination conditions according to the eyeball distance, and the position is set so as to evenly illuminate the eye detection area as much as possible. The second is
This is because the corneal reflection image needs to be at a height such that it is not kicked by the eyelid, and the infrared light emitting diode for the naked eye is at a higher position than the infrared light emitting diode for the eyeglasses. The third reason is that infrared light is reflected on the glasses so that ghosts formed on the glasses are less likely to be detected, and the infrared light emitting diodes for the glasses are located on the left, right, and lower than those for the naked eye. , Located at a distance.

Incidentally, the discrimination between the eyeball and the spectacles is made based on the interval | xd'-xe '| of the corneal reflection image.
Is calculated by calculating the distance.

The line-of-sight detection circuit 1064 is a photoelectric conversion element 1
The gazing point on the screen of the liquid crystal display element 1002 of the photographer is obtained according to the above-described principle based on the photographer's eyeball image formed on the image 063.

Next, the personal computer unit 1008 will be described. In the PC unit 1008, 18
Reference numeral 14 denotes a CPU that performs arithmetic processing on programs and data. 1813, a system bus connecting each device;
Reference numeral 18 denotes a memory controller for controlling the ROM 1816 and RAM 1817, and reference numeral 1812 denotes a video graphic controller for controlling the contents written in the video RAM 1811 to be displayed on a display. Reference numeral 1815 denotes an accessory device controller for controlling a pointing device and a keyboard. In the present embodiment, the accessory device controller 1815 is connected to the line-of-sight detection circuit 1064 in the head-mounted display 1006. 1819 is an I for controlling peripheral devices.
The / O channel is connected to the liquid crystal display circuit 1007 of the head mounted display 1006 in this embodiment.

As described above, in the gaze input intention communication device of the present embodiment, the gaze position data of the operator detected by the gaze detection circuit 1064 of the head mounted display 1006 can be applied as the pointing device of the personal computer unit 1008. It can be applied to screen scrolling and menu selection.

At the same time, a personal computer screen can be displayed on the external monitor 1009, so that a person other than the operator can see the personal computer screen. Note that a head mount display for one eye may be used, and in this case, the operator himself can see the external monitor 1009.

The operation of using the eye-gaze input intention communicating device having the above configuration will be described. FIG. 5 is a diagram conceptually showing a use operation of the gaze input intention communication device. In the figure, 2
Is a head-mounted display having a built-in eye-gaze detection circuit.
Consists of The computer 4 includes the personal computer unit 1008 and the external monitor 1009 described above.

The user 1 of the system can observe the image 3 generated by the computer 4 by wearing the head-mounted display 2 in the same manner as the glasses.

On the other hand, the gaze position data detected by the gaze detection circuit in the head mounted display 2 is input to the computer 4. That is, the user 1 can observe the image generated by the computer 4 on the head-mounted display 2 and can instruct any operation by the line of sight (movement of the eyeball). In this embodiment, the user 1 uses the hiragana or katakana, alphabets, and numbers of the display panel image 3 generated by the computer 4 to create a sentence using the watched character as an input character to assist a conversation. Is possible.

FIG. 6 is a flowchart showing the processing procedure of the eye-gaze input intention communication processing program. This processing program is stored in the ROM 1816 in the personal computer unit 1008, and is executed by the CPU 1814.

When the process is started, first, the I / O
F is initialized (step S201). In the present embodiment, the serial port RS232C is opened to enable communication. Subsequently, the voice processing unit is initialized (step S202).

An image of the 50 sounds (or katakana or alphabet) of the hiragana is displayed on the liquid crystal display device 1002 and the external monitor 1009 in the head mounted display (line-of-sight input scouter) via the display controller 31 (step S203). FIG. 7 shows a liquid crystal display device 10.
FIG. 2 is a diagram showing an image of the display panel displayed on the external monitor 02 and an external monitor 1009. Each option of the image on the display panel will be described later.

Subsequently, the gaze position is detected (step S).
204), the character at the line-of-sight position is calculated based on the detected line-of-sight position data (step S205). Then, the data of the past 100 times of the selected character is updated (step S206).

It is determined whether the character input condition is satisfied by the new character selection (step S20).
7). The character input condition is to satisfy the number of times corresponding to the time when the line of sight must be on the same character for the character input, or the number of times corresponding to a set constant rate within a certain time.

If the character input condition is satisfied in step S207, the character string is added to the input character string (step S20).
8) The voice of the newly input character is output (step S209), and the completion of the input is notified to the user. After this,
Returning to the process of step S203, a new input character is displayed on the screen.

On the other hand, if the character input condition is not satisfied in step S207, the process returns to step S204 and repeats character selection by detecting a new line-of-sight position.

As described above, in the first embodiment, the operator can perform character input regardless of the appearance and can easily perform character input for a severely handicapped person.

In this embodiment, whether or not the character input condition is satisfied is determined by measuring the number of times that the same character has a line-of-sight position in step S206. A timer may be started from the point in time when the position comes, and measurement may be performed. Further, the ratio of the line of sight position within a certain period of time may be arbitrarily set by the user, thereby improving usability.

[Second Embodiment] FIG. 8 is a diagram conceptually showing the operation of using the eye-gaze input intention communicating device in the second embodiment. The same components as those in the first embodiment are denoted by the same reference numerals.

In the figure, reference numeral 102 denotes eye-gaze input glasses having a built-in eye-gaze detection circuit, and has an internal configuration in which a portion for displaying an image in the head-mounted display 1006 is removed. The user 1 of the system wears the eye-gaze input glasses 102 and observes the image 3 on the display panel on the monitor 6 of the computer 4.

On the other hand, the gaze position data detected by the gaze detection circuit in the gaze input glasses 102 is input to the computer 4. That is, the user 1 observes the image generated by the computer 4 on the monitor 6 and can instruct any operation by the line of sight (movement of the eyeball). In the present embodiment, using the image 3 of the display board composed of hiragana or katakana, alphabetic characters, and numerals generated by the computer 4 to create a sentence using the characters watched by the user 1 as input characters to assist a conversation. Is possible.

As described above, in the second embodiment, characters can be input while watching the image on the display panel displayed on the monitor 6 while wearing the eye-gaze input glasses 102. Operations can be performed.

[Third Embodiment] FIG. 9 is a diagram conceptually showing the operation of using the eye-gaze input intention communicating device in the third embodiment. The same components as those in the first and second embodiments are denoted by the same reference numerals.

In the figure, reference numeral 102 denotes eye-gaze input glasses having an eye-gaze detecting circuit. Reference numeral 7 denotes a magnetic field generator such as a magnet, and 8 denotes a magnetic sensor. The user 1 of the system wears the line-of-sight input glasses 102 and the magnetic field generator 7, and observes the image 3 on the monitor 6 of the computer 4.

An induced current is generated in the magnetic sensor 8 by the magnetic field generated from the magnetic field generator 7 and the magnitude of the induced current is processed by a computer to measure the position and angle data of the user's head.

The gaze position data detected by the gaze detection circuit in the gaze input glasses 102 and the head position / angle data calculated from the magnetic sensor 8 are input to the computer 4. That is, the user 1 observes the image generated by the computer 4 on the monitor 6, and corrects the error of the line-of-sight position data caused by the movement of the head by using the head position / angle data, thereby correcting the line-of-sight (eyeball Some operations can be instructed by (movement). In the present embodiment, by using a video 3 of a display board composed of hiragana or katakana, alphabetic characters, and numbers generated by the computer 4, a sentence is created by using characters watched by the user 1 as input characters to assist a conversation. Is possible.

In the third embodiment, it is possible to correct an error in the line-of-sight position data caused by the movement of the head, and reduce errors in character input.

[Fourth Embodiment] FIG. 10 is a diagram conceptually showing the operation of using the eye-gaze input intention communicating device in the fourth embodiment. The same components as those in the first and second embodiments are denoted by the same reference numerals.

In the drawing, reference numeral 102 denotes eye-gaze input glasses having a built-in eye-gaze detection circuit. The user 1 of the system wears the eye-gaze input glasses 102 and observes the image 3 projected on the screen 107 through the projection device 106.

On the other hand, the gaze position data detected by the gaze detection circuit in the gaze input glasses 102 is input to the computer 4. That is, the user 1 observes the image generated by the computer 4 on the screen 107, and can instruct any operation by the line of sight (movement of the eyeball). In the present embodiment, by using a video 3 of a display board composed of hiragana or katakana, alphabetic characters, and numbers generated by the computer 4, a sentence is created by using characters watched by the user 1 as input characters to assist a conversation. Is possible.

In the fourth embodiment, since the user 1 sees the image 3 projected on the screen 107,
Eye fatigue can be reduced even if you keep watching for a relatively long time.

[Fifth Embodiment] FIG. 11 is a diagram conceptually showing the operation of using the eye-gaze input intention communicating device in the fifth embodiment. The same components as those of the first, second, and fourth components are denoted by the same reference numerals.

In the figure, reference numeral 102 denotes eye-gaze input glasses including a gaze detection circuit. 7 is a magnetic field generator such as a magnet, and 8 is a magnetic sensor. System user 1
Is an image 3 projected on a screen 107 by the projection device 106, wearing the eye-gaze input glasses 2 and the magnetic field generation device 7.
Observe.

An induced current is generated in the magnetic sensor 8 by the magnetic field generated by the magnetic field generator 7, and the magnitude of the current is processed by a computer to measure the position and angle data of the user's head.

The gaze position data detected by the gaze detection circuit in the gaze input glasses 102 and the head position / angle data detected by the magnetic sensor 8 are input to the computer 4. That is, the user 1 observes the image generated by the computer 4 on the screen 107 via the projection device 106, and corrects the error of the line-of-sight position data caused by the movement of the head by using the head position / angle data. While doing so, it is possible to instruct any operation by gaze (movement of the eyeball).

In the present embodiment, a sentence is created by using the characters watched by the user 1 as input characters using the video 3 of the display board composed of hiragana or katakana, alphabets, and numbers generated by the computer 4, and having a conversation. It is possible to assist.

As described above, in the fifth embodiment, when viewing the image 3 projected on the screen 107, it is possible to correct an error in the line-of-sight position data caused by the movement of the head, and reduce errors in character input. can do.

[Sixth Embodiment] The configuration of the device according to the sixth embodiment is the same as that of the first embodiment. FIG. 12 is a flowchart illustrating a processing procedure of a gaze input intention communication processing program according to the sixth embodiment. This processing program is stored in the ROM 1816 in the personal computer unit 1008, and is executed by the CPU 1814.

First, the interface (I /
F) is initialized (step S801). In the present embodiment, the serial port RS232C is opened to enable communication.

Subsequently, the voice processing unit is initialized (step S802). The 50 hiragana sounds (see FIG. 7), or the katakana and alphabet images are displayed on the liquid crystal display device 1002 and the external monitor 1009 in the head mounted display (line-of-sight input scouter) 2 via the display controller 31 (step S803). .

The line-of-sight position data is detected (step S80).
4) It is determined whether the eyes are closed based on the obtained line-of-sight position data (step S805). If the gaze position data is data indicating that the eyes are not closed, a character at the gaze position is calculated based on the gaze position data obtained in step S804 (step S811).

It is determined whether or not the selected character is the same as the previous selected character (step S812). If the selected character is the same as the previous selected character, the counter value is increased by one (step S813). It is determined whether or not the increased counter value is equal to or greater than the set value (step S814).

If the counter value is equal to or larger than the set value, the input enable flag is set to the value 1 (step S815). The processes in steps S812, S813, S814, and S815 prevent the user from inputting the line of sight at the moment when the eyes are closed and erroneously inputting characters not selected by the user. On the other hand, step S8
If the set value has not been reached in step 14, the process returns to step S804.

If the selected character is different from the previous selected character in step S812, the selected character is newly updated (step S816) and the counter is reset (step S816).
817), and the process returns to step S804.

On the other hand, if the line-of-sight position data is data indicating that the eyes are closed in step S805, the counter value is increased by one (step S806).

It is determined whether or not the counter value is equal to or larger than the set value (step S807). If the counter value is equal to or larger than the set value and the input enable flag is set to 1, the counter is reset (step S808). .

Thereafter, the character input processing is performed in the same manner as the method of holding the eyes for a certain time in the input determination method (step S809), and the voice is output (step S810). Thereafter, the process returns to step S803.

In the sixth embodiment, since the input is determined by closing the eyes, the eye fatigue can be suppressed as compared with the case where the input is determined while keeping the line of sight for a certain period of time. still,
The time during which the eyes are closed, that is, the blink time, can be changed as described later.

[Seventh Embodiment] The configuration of the device according to the seventh embodiment is the same as that of the first embodiment. FIG. 13 is a flowchart illustrating a processing procedure of a gaze input intention communication processing program according to the seventh embodiment. First, the I / F of the communication line is initialized (step S
1401). In this embodiment, the serial port RS23
Open 2C to enable communication. further,
The voice processing unit is initialized (step S1402).

The 50 hiragana characters (see FIG. 7), katakana, and alphabet are displayed on the liquid crystal display device 1002 and the external monitor 1009 in the head mounted display 2 via the display controller 31 (step S1403).

The line-of-sight position data is detected (step S14).
04), it is determined whether the currently selected input determination method is a method of holding the line of sight for a fixed time or a method of closing the eyes (step S1405).

FIG. 14 is a diagram showing a screen for changing the input confirmation method. On this screen, the user can select whether the input determination method is to be an input determination method by closing (blinking) eyes or an input determination method by holding a line of sight for a certain time. When the input determination method is switched, switching is performed using the input determination method that has been set.

When the input determination method is changed, it is also possible to change the gaze holding time, the blinking time, and the number of characters on the display panel (keyboard).

FIG. 15 is a diagram showing a screen for changing the holding time in the input determination method by holding the line of sight for a certain period of time. The user can select the gaze holding time on this screen. The line-of-sight holding time can be set in seconds from 3 seconds to 7 seconds.

FIG. 16 is a diagram showing a screen for changing the blinking time in the input confirmation method by closing the eyes. The blink time can be set in seconds from 1 second to 5 seconds, and the user can change the blink time on this screen. FIG. 17 shows a screen for changing the number of characters on the display panel. The user can select the number of characters on the display panel (6 rows and 6 columns or 6 rows and 7 columns) on this screen.

Next, a case will be described in which the input character is determined in step S1405 by holding the line of sight for a certain period of time. The character at the line-of-sight position is calculated based on the line-of-sight position data obtained in step S1404 (step S1406).

The data for the past 100 times of the selected character is updated (step S1407). It is determined whether this new character selection satisfies the character input condition (step S1408). Here, the condition for character input is the number of times that the number of times of selected character data among the past 100 character data corresponds to the time during which the gaze position must be on the same character for character input, or a fixed time. Exceeds the number of times corresponding to the fixed ratio set in the above.

If the character input condition is satisfied in step S1408, the character string is added to the input character string (step S1408).
1409) The voice of the newly input character is output to notify the user of the completion of the character input (step S141).
0). Thereafter, the process returns to step S1403, and a new input character is displayed on the screen.

On the other hand, if the character input condition is not satisfied in step S1408, the process returns to step S1404, where new line-of-sight position data is detected and character selection is repeated.

Next, a method of determining an input character by closing the eyes will be described. If the method of closing the eyes is selected as the input confirmation method in step S1405,
It is determined whether or not the line-of-sight position data obtained in step S1404 is data indicating that the eyes are closed (step S1411).

If the data indicates that the eyes are not closed,
The character at the gaze position is calculated based on the gaze position data obtained in step S1404 (step S141).
2). It is determined whether or not the character is the same as the previously selected character (step S1413). If the character is the same, the counter value is increased by one (step S1416).

It is determined whether or not the incremented counter value is equal to or greater than the set value (step S1417). If the counter value is equal to or greater than the set value, the input enable flag is set to a value of 1 (step S1418). Step S1413, S1
In the processing of 416, S1417, and S1418, the line of sight is input at the moment when the eyes are closed, thereby preventing erroneous input of characters not selected by the user. If the counter value is not equal to or greater than the set value in step S1417, the process returns to step S1404.

On the other hand, if the selected character is different from the previous selected character in step S1413, the selected character is newly updated (step S1414), the counter is reset (step S1415), and the process returns to step S1404.

On the other hand, if the line-of-sight position data is data indicating that the eyes are closed in step S1411, the counter value is increased by one (step S1419). It is determined whether or not the count value increased in step S1419 is equal to or larger than the set value and the input enabled flag is set to a value of 1 in step S1418 (step S1420).

If the counter value is equal to or greater than the set value and the input enable flag is set to a value of 1, the counter is reset (step S1421), and the flow shifts to the process of step S1409. Similarly, a character input process is performed.

On the other hand, if it is determined in step S1420 that the count value is smaller than the set value and the input enable flag is not set to the value 1, the process returns to step S1404.

In the seventh embodiment, the user can select a method of fixing the input by holding his / her gaze for a certain period of time or a method of fixing the input by closing his / her eyes, thereby improving the usability of the user.

[Eighth Embodiment] The eye-gaze input intention communicating device according to the eighth embodiment can switch from a character selection screen to a TV broadcast screen by a character selection screen option. This is applied to the seventh embodiment.

FIG. 18 is a diagram showing a TV broadcast screen when TV is selected as an option on the character selection screen. By selecting a TV, a TV broadcast screen is displayed as a video in the first to seventh embodiments. Then, the user looks at the upper left corner (or the lower left corner, the upper right corner, or the lower right corner) of the screen for a predetermined time, thereby selecting options 2G to 7G (FIG. 1).
9) is displayed. FIG. 19 shows a T in which options are displayed.
It is a figure showing a V broadcast screen.

When the option 2G is selected, the screen returns to the character selection screen shown in FIG. When option 3G is selected, the TV
Broadcasts change in the direction of smaller channel numbers. Option 4
When G is selected, TV broadcasts change in the direction of larger channel numbers. When the option 5G is selected, the volume is reduced. When the option 6G is selected, the volume increases. When option 7G is selected, a TV image is displayed on the entire monitor.

In the eighth embodiment, a user can select a desired screen by switching between a character input display panel and a TV broadcast screen.

It should be noted that the present invention is not limited to TV images, but includes video, computers, video-on-demand devices, clocks, security devices,
An external input image such as a videophone may be selectable, so that the user can freely select various images.

Further, an external environment control screen for controlling an external environment device may be provided, whereby an illumination device, a television, an audio device, a video, a computer, a video-on-demand device, a telephone, a videophone , Watches, electric blinds, electric curtains, electric window opening and closing devices, air conditioners, security devices, electric reclining beds, excretion devices,
Users can freely control external environmental devices such as lifters and nurse calls.

[Ninth Embodiment] The device configuration of the ninth embodiment is the same as that of the first embodiment. In the eye-gaze input intention communication device according to the ninth embodiment, when the user wants to increase the diopter, the user brings the gaze to the upper part of the screen, and when the user wants to make the diopter shallow, the user brings the gaze to the lower part of the screen. Diopter adjustment can be performed.
When the user wants to end the diopter adjustment, the user can end the diopter adjustment by moving his / her gaze to the center of the screen.

FIG. 20 is a flowchart showing the diopter adjustment processing procedure in the ninth embodiment. This processing program is stored in the ROM 1816,
4 is performed. First, gaze position data is acquired by the gaze detection circuit (step S1701). It is determined whether or not the line-of-sight position data acquired in step S1701 is above a set value (step S1702).

If the line-of-sight position data is higher than the set value, the diopter determination counter used for determining the diopter is set to a value of 0 (step S1703), and the computer 4 makes the diopter deeper with respect to the head mounted display 2. (Step S1704), and the process returns to step S1701.

On the other hand, if the line-of-sight position data is not above the set value in step S1702, it is determined whether or not the line-of-sight position data is below the set value (step S170).
5). If the line-of-sight position data is lower than the set value, the diopter determination counter used when determining the diopter is set to a value of 0 (step S1706), and the computer 4 instructs the head mounted display 2 to reduce the diopter. Forwarding (step S1707), the process returns to S1701.

On the other hand, if the line-of-sight position data is not below the set value in step S1705, it is determined whether or not the line-of-sight position data is within the set area at the center of the screen (step S1708).

If the line-of-sight position data is within the set area at the center of the screen, the diopter determination counter value is increased by one (step S1709). It is determined whether or not the diopter confirmation counter is larger than the set value (step S1710). If the diopter determination counter is larger than the set value, it is determined that diopter adjustment has been completed, and the process ends.

On the other hand, if the diopter adjustment counter is equal to or less than the set value in step S1710, the process returns to step SS1701. On the other hand, if the line-of-sight position data is not within the set area at the center of the screen in step S1708, the process returns to step SS1701.

In the ninth embodiment, the diopter can be adjusted according to the user.

[Tenth Embodiment] The device configuration of the tenth embodiment is almost the same as that of the first embodiment described above. However, in the tenth embodiment, a character input is used to determine a character input. Switch (not shown) is I / O channel 1
819. This processing program is ROM
1816 and executed by the CPU 1814.

FIG. 21 is a flowchart showing a processing procedure of a gaze input intention communication processing program according to the tenth embodiment.

First, the I / F of the communication line is initialized (step S1801). In the present embodiment, the serial port RS232C is opened to enable communication.
Further, the voice processing unit is initialized (Step S180)
2). The 50 hiragana sounds (see FIG. 7), the katakana, and the alphabet are displayed on the liquid crystal display device 1002 and the external monitor 1009 in the head mounted display (line-of-sight input scouter) 2 via the display controller 31 (step S1803).

The line-of-sight position data is detected (step S1).
804). Then, it is determined whether or not the switch for confirming the input has been pressed (step S1805).
If the switch to confirm the input is not pressed,
The character at the gaze position is calculated based on the gaze position data (step S1811).

It is determined whether or not the character is the same as the previously selected character (step S1812). If the character is the same, the counter is incremented by one (step S1813). It is determined whether or not the increased counter value is equal to or greater than the set value (step S1814). If the counter value is equal to or greater than the set value, the input enable flag is set to a value of 1 (step S1815).

If the set value has not been reached in step S1814, the process returns to step S1804. If the selected character is different from the previous selected character in step S1812, the selected character is updated (step S18).
16) and reset the counter (step S181)
7), the process returns to step S1804.

On the other hand, if the switch for confirming the input is pressed in step S1805, the counter value is increased by one (step S1806). Step S180
It is determined whether the counter value increased in step 6 is equal to or larger than the set value and the input enabled flag is set to a value of 1 (step S1).
807) If the counter value is equal to or greater than the set value and the input enable flag is set to the value 1, the counter is reset (step S1808), and the same character input processing as in the input determination method of holding the line of sight for a fixed time is performed (step S1808). Step S180
9) Output audio (step S1810). After that, the process returns to the process of step S1803.

On the other hand, if the counter value is smaller than the set value in step S1814, the process returns to step S1804.

In the tenth embodiment, the input can be determined by the movement of the eyes and the operation of pressing a small switch, and the number of operations by the eyes can be reduced. The switch may be mounted on the head or may be held by hand.
Further, the time during which the switch is pressed until the input of the selected character is confirmed may be arbitrarily set by the user, and the usability can be improved by setting the ON time of the switch according to the user. .

Next, a description will be given of a character selection screen which is an image on the display panel and functions of the options in the first to tenth embodiments.

FIG. 22 is a diagram showing the first page of the hiragana character selection screen. FIG. 23 is a diagram showing the second page of the hiragana character selection screen. FIG. 24 is a diagram showing the third page of the hiragana character selection screen. The options on the character selection screen are as follows.

"Cancel": Cancels the input of the character entered immediately before, cancels the input of "Dakuten" and "Semi-Dakuten" entered last, and moves the cursor of the created sentence display section entered previously. Cancel, cancel previously entered clear, previous delete, cancel delete, cancel previous "yes", "no" input, cancel previous "enter" cancel Do. “Function”: A function item selection screen is displayed, and various functions of the gaze input communication device are selected. “Registration”: Register the created text and delete the registered text.

"Call": Calls a registered sentence.

"Speech": The created sentence is uttered.

"-": In the Hiragana / Katakana input mode, a long tone is input.
Enter a hyphen.

[゛ (Dark point)]: Inputs a cloud point. First of all, the characters that can be used as cloud characters (ka line, sa line, ta line, wa line, c)
Is input, and then the "cloudy point" is input.

[゜ (semi-voiced point)] A semi-voiced point is input. First, enter the semi-voiced dot executable characters (the line), and then
This is performed by inputting a "semi-voiced point".

[△]: Shift from character selection screen page 2 and character selection screen page 3 to character selection screen page 1. No operation is performed on the character selection screen page 1.

[∧]: Shift from character selection screen page 2 to character selection screen page 1. The screen shifts from the character selection screen page 3 to the character selection screen page 2. No operation is performed on the character selection screen page 1.

[∨]: Moves from character selection screen page 2 to character selection screen page 3. The screen shifts from the character selection screen page 1 to the character selection screen page 2. No operation is performed on page 3 of the character selection screen.

[▽]: Shift from character selection screen page 1 and character selection screen page 2 to character selection screen page 3. No operation is performed on page 3 of the character selection screen.

[0175]

[Outside 1] “Clear”: Clears the created text.

"Previous deletion": Deletes the character immediately to the left of the cursor in the created sentence display section.

"Delete": Deletes the character at the same position as the cursor in the created text display section.

"Yes" / "No": Enter "Yes" / "No". This is an effective key when the user receives a question from a third party and answers immediately.

[0179]

[Outside 2] As described above, since many options are provided on the character selection screen by switching pages, the user can select various functions.

FIG. 25 is a diagram showing a function selection screen 1 displayed when the “function” of the option is selected from the character selection screen. FIG. 26 is a diagram showing the function selection screen 2.

The user selects an option to change various setting values of the gaze input communication device. here,
When "Cancel" is selected, the character selection screen [Fig.
Reference). The transition from the function selection screen 1 to the function selection screen 2 can be made by selecting “▽” on the function selection screen 1. The transition from the function selection screen 2 to the function selection screen 1 can be made by selecting “△”.

Next, the function of “calibration” on the function selection screen 1 will be described. FIG. 27 is a flowchart showing the procedure of the calibration process.
First, the calibration status variable (NAC_
STAT) is initialized (step S302). After clearing the screen (step S303), a message for executing the calibration is displayed (step S30).
4).

Theoretically, a left index is displayed at a position where the eyeball is tilted 6 degrees to the left from the front (step S305). The left calibration data is obtained (step S306). It is determined whether or not the acquisition of the left calibration data has been successful (step S307), and the acquisition of the left calibration data has failed. Status variable (N
AC_STAT) (step S30)
8), the process returns to step S305.

On the other hand, if the acquisition of the left calibration data has been successful, this fact is displayed, and the right index is displayed at a position where the eyeball is inclined 6 degrees to the right from the front in theory (step S309). The right calibration data is obtained (step 310). It is determined whether or not the right calibration data has been successfully obtained (step S311).
If acquisition of the right calibration data has failed, the status variable (NAC_STAT) is reinitialized (step S312), and the process returns to step S309.

On the other hand, if the right calibration data has been successfully acquired, this fact is displayed, and in theory, upon successful completion of the left and right calibrations, eyeglasses with a gaze detection function or a head mounted display with a gaze detection function are displayed. The calibration data is written in the EEPROM, and a calibration data output message is displayed (step S313), and the process is terminated.

The calibration is performed when the eye-gaze input intention communication device is started or when “Calibration” on the function item selection screen is selected.

Next, the function of “rest” on the function selection screen 1 will be described. FIG. 28 is a flowchart showing a processing procedure regarding the input when the user turns off the screen display for a break and narrows the range in which the user can input the line of sight. First, gaze position data is detected in conjunction with a timer (step S402). It is determined whether the gaze input is being restricted (step S403).

If the line of sight input is not restricted, normal line of sight input processing is executed (step S404). If the gaze input is restricted, it is determined whether or not the gaze position is within the gaze position displayable range (step S405).
If it is out of the gaze position displayable range, the process ends.

On the other hand, if it is within the range in which the gaze position can be displayed, the gaze position is displayed (step S406). It is determined whether or not the line of sight is within the option cell (step S
407). If the line of sight is outside the option cell, the process ends.

On the other hand, if the line-of-sight position is within the option cell, a cell selection process is performed (step S408), and the process ends.

Here, the line-of-sight position display range in step S405 and the option cell range in step S407 do not always match. FIG. 29 shows a normal input screen in the break mode. In break mode, the only option cell to return to the normal input screen "Back"
Other than the above, the choice cell is not displayed for a break.

When the user wants to input "return" to return to the normal selection screen, for some reason, the user looks at the actual gaze position and the gaze position detected by the gaze detection circuit (display unit). If the gaze position is different from the actual gaze position, and if the gaze position is displayed only in the option cell “return”, the user is not sure how far the detected gaze position is from the actual position, With such control, it is not known whether the position detected by the line-of-sight detection circuit falls within the option cell “return”. In order to prevent this, the gaze position is displayed when the gaze position approaches the option cell “return”.

[0193] A break can be performed by selecting a break on the function selection screen 1 or keeping the eyes closed for a predetermined time. It can also be implemented by inputting according to the confirmation condition. In addition, the direction in which the user turns his or her eyes may be arbitrarily changed, and usability can be improved.

Next, the function of “volume change” on the function selection screen 1 will be described. FIG. 30 is a diagram showing a screen for changing the volume. The user can change the volume on this screen. When "small" is selected, the amplitude of the audio waveform becomes 1/5 that of "normal". Similarly,
In the case of "slightly small", it becomes 1/2 times. If "Slightly large" is selected, the size is doubled. Further, if “large” is selected, the size becomes five times. When changing the volume,
On the function selection screen 1, select "volume change".

Next, the function of “change utterance speed” on the function selection screen 1 will be described. FIG. 31 is a diagram showing a screen for changing the utterance speed. The user can change the utterance speed on this screen. When "slow" is selected, 170 msec / mora is selected. When "slightly late" is selected, 150 msec / mora is selected. When "normal" is selected, 130 msec / mora is selected.
sec / mora, “fast” becomes 90 msec / mora. To change the utterance speed, select “Change utterance speed” on the function selection screen 1.

Next, the function of “key sound setting” on the function selection screen 1 will be described. FIG. 32 is a diagram showing a screen for changing whether or not to enable a function of reading out the name of the option when the option is selected.
When “Yes” is selected, the name of the option is read out each time the option is selected. When "None" is selected, the name of the option is not read out even if the option is selected. However, the utterance of the prepared sentence when "utterance" is selected is performed in both cases.

Next, the function of “change invalid time” on the function selection screen 1 will be described. FIG. 33 is a diagram showing a screen for changing the invalid time. The user can select an invalid time on this screen. To change the invalid time, select “Change invalid time” on the function selection screen 1.

Next, the function of “keyboard size setting” on the function selection screen 2 will be described. FIG. 34 is a diagram showing a screen for changing the size of the keyboard. FIG.
FIG. 5 shows a screen for changing the size of the keyboard in the left-right direction. When the user selects “change keyboard size” from the second page of the function selection screen, the screen is switched to the screen shown in FIG.

Here, whether to change the size of the keyboard (display panel) in the vertical direction or the horizontal direction is selected from the options of “up and down” and “left and right”.

The change in the left-right direction will be described. FIG.
If "left / right" is selected, the screen shown in FIG. 35 is displayed. The numerical value in the second row from the top in the third column from the left of the option indicates how much space is left between the left edge of the screen and the left edge of the keyboard.

In FIG. 35, from the top of the third column from the left of the option,
The numerical value in the row indicates how far the left end of the screen is from the right end of the keyboard (assuming this numerical value is A, it is indicated by a margin 64-A between the right end of the screen and the right end of the keyboard).

In order to change the left and right numerical values, two changing methods are facilitated. The first method is to increase or decrease the numerical value by selecting the arrows on the left and right of each numerical value, and the second method is to select each numerical value (when selected, a red frame 10R is displayed This is a method of directly inputting a numerical value by selecting “0” to “9” below. Determine each numerical value, and select "O
The setting is determined by selecting the "K" cell.

Similarly, the change in the vertical direction will be described. FIG. 36 shows a screen for changing the size of the keyboard in the vertical direction. By selecting "up / down" in FIG. 34, the screen is switched to the screen shown in FIG. Top 3 choices
The numerical value in the second column from the left of the column indicates how much space is provided between the lower end of the menu bar and the upper end of the keyboard in the window of Windows 95 (Microsoft, USA).
In the third row from the top of the options, the numerical value in the fifth column from the left is
Indicates the distance between the lower end of the menu bar and the lower end of the keyboard (assuming this numerical value is A, the margin between the upper end of the input character display section and the lower end of the keyboard is represented by 37-A).

To change the upper and lower numerical values,
Two kinds of change methods are prepared. The first method is to increase or decrease the numerical value by changing the arrows above and below each numerical value, and the second method is to select each numerical value (when selected, a red frame 20R is displayed. This is a method of directly inputting a numerical value by selecting “0” to “9” below. The settings are determined by determining the respective numerical values and selecting the “OK” cell.

FIG. 37 shows the character selection screen after the size of the keyboard has been changed. Here,
The numerical values in the vertical direction are respectively left: 5, right: 59, upper: 5,
Bottom: A character selection screen as a result of selecting 32 is shown. In addition, changing the numerical value of up, down, left and right to "1"
This is equivalent to changing 10 dots on the screen.

Next, the function of “setting of line-of-sight cursor movement” on the function selection screen 2 will be described. FIG. 38 is a diagram showing a screen for changing whether the cursor is moved to or not moved to the line-of-sight position when the line-of-sight position is placed on the created sentence display unit on the character selection screen. When "Yes" is selected, the gaze position is set at an arbitrary position on the created sentence display section on the character selection screen, and when the input is determined according to the input determination method, the cursor can be moved to the gaze position. .

Subsequently, the function of “memo page printing” on the function selection screen 1 will be described. FIG. 39 is a diagram showing a screen for executing printing of the created text. In the selection of “Yes” or “No” for paper discharge, it is possible to set whether or not to discharge paper. This is effective when printing short sentences, etc., rather than exhausting the paper set in the printer every time printing is selected by selecting `` No '', the printed portion is sent from the printer and When the three parties can read the print contents, the paper discharge function of the printer is stopped. The next printing is performed from the point where the paper discharge is stopped. Thereby, paper can be saved.

Next, the function of “registration” on the character selection screen will be described. FIG. 40 is a diagram showing a registration screen for storing the created text in the memory. Here, the user registers the text created on the character selection screen at an arbitrary position.
It is possible to keep it. This screen is displayed by selecting the option “register” on the character selection screen. The uppermost row is a function key related to registration, and the second to fifth rows are registered text display sections.

[0209] At the bottom, a registered text page is displayed. Eight registered sentences can be registered per registered sentence page. There are 10 registered text pages, and a total of 80 texts can be registered.

[0210] The text created on the character selection screen is displayed below the option display section. First, the function keys related to the registration at the top row will be described.

[Execution]: The results of [insert], [delete], and [previous deletion] performed in the registration mode are determined, and the registration mode is terminated.

[Cancel]: Cancels all the results of [Insert], [Delete] and [Previous Delete] performed in the registration mode, and ends the registration mode.

[Insert]: The text created on the character selection screen at the currently selected registered text display position (in FIG. 40, the portion where the first red frame 30R of the registered text display section is displayed). Insert and register.

FIG. 41 is a diagram showing a screen as a result of registration of a sentence. The first row of the registered text display section before implementation,
Registration sentence that was in the second-stage "Hello", "are you Genki whether" the second stage, move to the third stage, is created in the character selection screen in the first stage of the newly registered sentence display unit The sentence "I'm good today" is inserted and registered.

However, at the end of the registration mode, if a cancel is input in FIG. 41, the result of insertion / registration is determined and the screen returns to the character selection screen.

[Delete]: The registered text at the currently selected registered text display position (in FIG. 40, the portion where the first red frame 30R of the registered text display section is displayed) is deleted. When [Delete] is input, the user's intention is confirmed. FIG. 42 is a diagram showing a screen for confirming the user's intention when [Delete] is input. Here, when "yes" is input, deletion is performed. FIG. 43 is a diagram showing a deletion result screen. However, at the end of the registration mode, if a cancel is input in FIG. 43, the deletion result is not fixed and the screen returns to the character selection screen.

[Delete All]: Deletes all registered texts currently registered. When [Delete All] is entered, the user's intention is confirmed. FIG. 44 is a diagram showing a screen for confirming the user's intention when "delete all" is input. Here, if "yes" is input, all deletion is performed. However, when cancel mode is entered at the end of registration mode,
The result of the entire deletion is not determined but returns to the character selection screen.

[▽] ([△]): In FIG. 40, the registered sentence display section displays the registered sentence pages from the first registered sentence to the fourth registered sentence on the first page. By inputting [▽], the registered sentence display unit displays the registered sentence pages from the fifth registered sentence to the eighth registered sentence on the first page. At this time, the display of [▽] changes to [△].
FIG. 45 is a diagram showing a screen in which the display of [▽] is changed to [△]. Further, when [△] is input, the display of the registered sentence display section is changed from the fifth registered sentence to the eighth registered sentence on the first page of the registered sentence page, from the first registered sentence to the fourth sentence on the same page.
Change to registration text.

Next, the lowermost registered text page display section will be described. [P. 1] to [P. 5] correspond to registered text page 1 to registered text page 5, respectively. [P. 1] to [P. 5], the registered text of the corresponding page is displayed on the registered text display section. When [▼] is entered, [P. 6] to [P. 10] (registered sentence page 6 to registered sentence page 10) is displayed, and registered sentence page 10 can be selected from registered sentence page 6. The display of [▼] is changed to [▲]. FIG. 46 is a diagram showing a screen in which the display of [▼] is changed to [▲].

The registered text is managed in a TXT file when the application is not running. When the application is running, it is managed in the registered text management array. The registration sentence management array is in the form of a two-dimensional array, in which the number of first elements is represented on a registration sentence page, and the number of second elements indicates the order of the 10 registered sentences in each registered sentence page.

FIG. 47 is a flowchart showing a processing procedure for reading a TXT file in which a registered sentence is written at the time of starting the application. First, a file in which the registered text is written is opened (step S
1302). Subsequently, all the registered sentences are read from the file in which the registered sentences are written as one-piece sentences (step S1303). FIG. 48 is a diagram showing one continuous file in which the registered text is written. A single registered sentence is divided for each return code ($ n) and written in the registered sentence management array in order from the top (step S1).
304).

FIG. 49 is a flowchart showing a processing procedure for writing a registered sentence in a file when the application is terminated. All the sentences in the registered sentence management array are converted into one sentence (step S1307). The created one-piece sentence (see FIG. 48) is written to a registered sentence TXT file (step S1308). T for registration text
The XT file is closed (step S1309).

Subsequently, the function of “call” on the character selection screen will be described. FIG. 50 is a diagram showing a call screen for calling a registered sentence on the created sentence display section of the character selection screen. Here, the user can call the document at an arbitrary position in the text created on the character selection screen.

This screen is an option "call" on the character selection screen.
Is displayed by inputting. The top row is a function key related to calling, and the second to fifth rows are calling text display sections. At the bottom, a call text page is displayed.

As described above, the call text includes a registered text created by the user and registered using the registration mode, and a fixed text that is usually used in the application and is a daily text commonly used. There are two types of sentences. The fixed text contains 8 pages over 10 pages in the same way as the registered text described above.
0 fixed sentences are prepared. The selected call text is displayed below the option display section.

First, the function keys related to the uppermost call will be described.

[Execute]: The selected call text is called and inserted into the text prepared on the character selection screen, and the call mode is ended. The insertion position is inserted starting from the position of the cursor in the created sentence display section of the character selection screen. FIG. 51 is a diagram showing a screen of a call result. In the figure, "Hello" is being called in the creation sentence.

[Cancel]: Terminates the calling mode without calling and inserting the selected calling text into the text on the character selection screen.

[Fixed sentence]: It is possible to select a fixed sentence that has been made in the application in advance in a commonly used everyday sentence. When [Fixed sentence] is input, the fixed sentence is displayed on the call sentence display section, and the fixed sentence page name is displayed on the call sentence page display section. FIG. 52 is a diagram showing a fixed sentence and a fixed sentence page name as a table.

[Registration Text]: As described above, it enables the user to select a registration text created and registered using the registration mode. When [Registration Text] is input, the registration text is displayed on the calling text display, and the registration text page names ([P.1] to [P.10]) are displayed on the calling text page display.
Is displayed.

[Speech]: Speak the currently selected calling sentence. Here, the user can have a conversation by repeating the selection and utterance of the calling sentence in the calling mode without calling the calling sentence once to the created sentence display section of the character selection screen.

[▽] ([△]): FIG. 53 is a diagram showing a screen on which the fixed sentence page is displayed on the call sentence display unit from the first fixed sentence to the fourth fixed sentence of the first page.
By inputting [▽], the fixed sentence page from the fifth fixed sentence to the eighth fixed sentence of the first page is displayed on the calling sentence display section.

At this time, the display of [▽] changes to [△]. FIG. 54 is a diagram showing a screen in which the display of [▽] is changed to [△]. When [△] is input, the display of the calling sentence display section is changed from the first fixed sentence to the fourth fixed sentence on the first page from the fifth fixed sentence to the eighth fixed sentence on the first page.
Change to fixed text.

Next, the lowermost call text page display section will be described. The operation is the same as that of the registration text page described above. In the case of a fixed sentence call, a fixed sentence page is displayed on the call sentence page display section, and the fixed sentence page name is as shown in FIG. In the case of a registered text call, a registered text page is displayed on the call text page display section, and the registered text page name is [P. 1] ~
[P. 10].

In the option "change of input determination method" on the function selection screen 2, there is a method of determining an option to be input when the line-of-sight position is on the same option for a predetermined time or more, and the line-of-sight position exists on the option. Later, a method of determining an option to be input by the user closing his / her eyes for a predetermined time and a method of determining an option to input by the user pressing a switch for a predetermined time after the gaze position exists on the character Switching may be performed.

The present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of one device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or an apparatus. In this case, by reading out a storage medium storing a program represented by software for achieving the present invention into a system or an apparatus, the system or the apparatus can enjoy the effects of the present invention.

In this embodiment, a ROM is used as a storage medium.
Although not limited to ROM, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD, magnetic tape, nonvolatile memory card, and the like can be used.

[0238]

According to the gaze input communication device of the first aspect of the present invention, the image of the display board is displayed by the image display means, and the gaze of the user facing the image of the display board by the gaze detection means. When the user's intention is conveyed by the detected line of sight, the option on the display panel which the user is gazing at by the option determining unit by the line of sight detected by the line of sight detecting unit Therefore, the user can watch the option (character) that the user wants to input on the display panel of the screen, and make a selection by detecting the line of sight with the line of sight detecting means. In this way, the user can input options only by moving the eyes. Therefore, a severely disabled person who cannot press the step button can easily make a character input decision. Further, the same effect can be obtained in the gaze input intention communication device according to the second aspect. Furthermore, the same effect can be obtained in the gaze input communication method according to claim 45.

[0239] According to the gaze input intention communicating device of the third aspect, the image on the display board is displayed by the image display means, and the gaze of the user facing the image on the display board is detected by the gaze detection means. When the intention of the user is transmitted by the line of sight, the head position of the user is detected by the head position detecting unit, and the head position detected by the line of sight detecting unit is detected by the line of sight detecting unit. The selected gaze position is corrected, and the option on the display panel that the user is gazing at is determined by the option determination unit based on the corrected gaze position. Can input the option correctly by eye movement.

According to the gaze input communication device of the fourth aspect, the user projects the image of the display panel by projecting the image on the screen by the projection means and faces the displayed image of the display panel by the gaze detection means. When detecting the line of sight of the user and communicating the intention of the user by the detected line of sight, the line of sight detected by the line of sight detecting means, on the display panel being watched by the user by the option determining means. Is determined, so that projection on the screen can suppress eye fatigue when inputting the options.

According to the gaze input intention communicating device of the fifth aspect, the user projects the image of the display panel on the screen by projecting the image and displays the image of the display panel by the eye gaze detecting means. When the gaze of the user is detected and the intention of the user is transmitted by the detected gaze, the position of the head of the user is detected by head position detection means, and the head detected by the gaze position correction means is detected. The gaze position detected by the gaze detection unit is corrected based on the position, and the option on the display panel being watched by the user is determined by the option determination unit based on the corrected gaze position. By doing so, the user can input the options correctly by eye movements even if the head position changes, while suppressing eye fatigue when inputting the options.

According to the gaze input intention communicating device of the sixth aspect, the option judging means determines the option to be input when the gaze position is on the same option for a predetermined time or more, so that the operation for viewing the video is performed. Character input can be easily performed only by using

According to the gaze input intention communicating device of the present invention, the option judging means selects the option to be input by the user closing his / her eyes for a predetermined time after the gaze position exists on the option. Since the determination is made, the input can be determined by closing the eyes, and the eye fatigue can be suppressed as compared with the case where the input is determined while holding the line of sight for a certain time.

[0244] According to the gaze input intention communicating device of the eighth aspect, the option judging means selects the option to be input by the user pressing the switch for a predetermined time after the gaze position exists on the option. As a result of the determination, the user can input the options with the movement of the eyes and the slight movement of the body.

[0245] According to the gaze input intention communicating device of the ninth aspect, the option judging means determines the option to be input when the gaze position is on the same option for a predetermined time or more, Means that the user closes his / her eyes for a predetermined time to determine an option to input, and after the gaze position exists on the option, the user presses a switch for a predetermined time. Since there are provided means for determining input options and switching means for switching the means for determining these options, the user can select an input option determination method according to the state, thereby improving operability. .

According to the gaze input communication device of the tenth aspect, since the user is provided with changing means for changing the predetermined time until the option to be input is determined, the user can change the predetermined time. The time until the option is determined can be adjusted according to the state. Thus, the blink time can be set according to the user, and the usability can be improved.

[0247] According to the gaze input intention communication device of the eleventh aspect, a change means capable of changing the predetermined time during which the user keeps his or her eyes closed before deciding an option to be input is provided. With the provision, the user can adjust the time until the option is determined according to the state. In this way, the gaze holding time can be set according to the user, and the usability can be improved.

According to the gaze input intention communication device of the twelfth aspect, since the user is able to change the predetermined time during which the switch for determining the option to be input is pressed, the changing means is provided. The user can adjust the time until the option is determined according to the state. In this way, the time during which the switch is pressed can be set according to the user, and the usability can be improved.

[0249] According to the gaze input intention communicating device of the thirteenth aspect, the option judging means determines the option to be input when the gaze position is on the same option at a certain ratio during a certain time. Therefore, the user can adjust the selection of the option according to the state. As described above, the input can be determined in consideration of the fluctuation of the line of sight, and the usability can be improved.

[0250] According to the gaze input intent communication device of the present invention, since the changing means capable of changing the constant rate by the user is provided, the user can select an option according to the state. The decision can be adjusted. Thus, the ratio can be set according to the user, and the usability can be improved.

According to the gaze input intention communication device of the present invention, the user is provided with a changing means which can change the number of options of the display panel. The number of display panel options can be set, and usability can be improved.

According to the eye-gaze input communication device of the sixteenth aspect, the user is provided with switching means for switching between the image on the display panel and the externally input image. You can watch external input video freely.

According to the gaze input communication device of the present invention, the external input image is an image of a television, a video, a computer, a video-on-demand device, a clock, a security device, a videophone, or the like. The user can freely watch various images.

According to the gaze input intention communication device of the eighteenth aspect, since the external environment device is provided with the operation means capable of operating the user, the user can freely operate the external environment device. .

[0255] According to the eye-gaze input communication device of claim 19, the external environment device is a lighting device, a television, an audio device, a video, a computer, a video-on-demand device, a telephone, a videophone, a clock, Since it is an electric blind, an electric curtain, an electric window opening / closing device, an air conditioner, a security device, an electric reclining bed, an excrement device, a lifter or a nurse call, the user can freely operate these external environmental devices.

According to the eye-gaze input intention communicating device of the twentieth aspect, since the diopter adjusting means for adjusting the diopter of the displayed image is provided, the diopter is adjusted according to the visual acuity of the user. be able to.

According to the gaze input communication device of the present invention, since the erasing means for erasing the image on the display panel is provided, it is unnecessary to display the image on the display panel.

According to the eye-gaze input intention communicating device of the twenty-second aspect, since the user is provided with switching means for switching the display or erasure of the image on the display panel, the user can display the image when it is not necessary. You can erase the video on the board.

According to the eye-gaze input intention communicating device of the twenty-third aspect, the erasing means erases the image on the display board when the user closes his / her eyes for a predetermined time, so that the erasing means can be simply operated by the eyes. The image on the display panel can be deleted.

According to the gaze input communication device of the twenty-fourth aspect, since the user is provided with changing means for changing the predetermined time during which the user keeps his / her eyes closed, the user can adjust the state according to the state. Can be changed to improve usability.

According to the gaze input intention communication device of the twenty-fifth aspect, since the user turns the gaze in a predetermined direction to erase the image on the display panel, the display panel can be easily displayed only by the movement of the eyes. Images can be deleted.

According to the gaze input intention communication device of the twenty-sixth aspect, since the user is provided with the change means capable of changing the predetermined direction in which the user turns his / her gaze, the direction is changed according to the state of the user. it can.

According to the eye-gaze input intention communication device of the twenty-seventh aspect, the user can temporarily stop the eye-gaze input because the gaze input intention communication device is provided with the area restriction means for restricting the area that can be input by the gaze.

[0264] According to the gaze input intention communication device of the twenty-eighth aspect, since the user is provided with the switching means for switching whether or not to limit the area in which the user can input by the gaze, the gaze input is provided. Pause can be freely performed.

[0265] According to the gaze input intention communication device of the twenty-ninth aspect, since the user is provided with the changing means for changing the area where the input by the gaze is restricted, the input can be performed in accordance with the state of the user. You can set a restricted area.

According to the eye-gaze input communication device of the present invention, since the user closes his / her eyes for a predetermined period of time, the area that can be input by the eye-gaze is limited, so that the user can simply operate the eye. The eye-gaze input can be temporarily stopped.

According to the eye-gaze input communication device of the thirty-first aspect, since the user is provided with changing means for changing the predetermined time during which the user keeps his / her eyes closed, the apparatus is adapted to the state of the user. Time can be set and usability can be improved.

According to the gaze input communication device of the thirty-second aspect, since the user turns his / her gaze in a predetermined direction to limit the area that can be input with the gaze, the user can perform the eye movement only by eye movement. Pause can be easily performed by gaze input.

According to the gaze input intention transmitting device of claim 33, since the user is provided with the changing means capable of changing the predetermined direction in which the user turns his / her gaze, the direction is changed according to the state of the user. can do.

According to the gaze input intent communication device of the thirty-fourth aspect, since the gaze position is not displayed in the area where the input by the gaze is restricted, the gaze position is not displayed unnecessarily.

[0271] According to the gaze input intention transmitting device of the thirty-fifth aspect, the changing means capable of changing whether the user displays or does not display the gaze position in an area where input by the gaze is restricted. Is provided, the user can freely display or delete the gaze position.

[0272] According to the gaze input intention communication device of the thirty-sixth aspect, since the change means for changing the size of the displayed display panel is provided, the size can be set to a size that is easy for the user to see.

According to the eye-gaze input communication device of claim 37, since the user is provided with a change means capable of changing the size of the display panel, the user can change the size of the display panel according to the state. Can be set.

According to the gaze input communication device of the thirty-eighth aspect, since there is provided a change means for changing the cursor position of the created sentence display unit by the gaze, the user can change the cursor movement options many times. You do not need to enter

According to the gaze input intention communication device of the thirty-ninth aspect, since the user is provided with the switching means for switching whether or not the cursor position is changed by the line of sight, the user can change the cursor position according to the state of the user. You can set the cursor position.

[0276] According to the gaze input intent communication device of the forty-fourth aspect, after the input by the gaze is determined, the input by the gaze is not accepted for a predetermined time, so that the user can confirm the input result with plenty of time. be able to.

[0277] According to the gaze input intention communication device of the forty-first aspect, after the user determines the input by the gaze,
Since there is provided a changing means for changing the predetermined time during which the input of the line of sight is not accepted, it is possible to provide a margin for confirming the input result according to the user.

According to the eye-gaze input intention communicating device of the present invention, since the printing means for performing printing in accordance with the option based on the eye-gaze is provided, the user can print the created text.

[0279] According to the eye-gaze input intention communicating device of the present invention, paper is not discharged every time the printing is executed, so that paper can be saved.

[0280] According to the gaze input intention communication device of the forty-fourth aspect, the user is provided with a changing means capable of changing whether to discharge the paper every time the printing is executed. Can save paper by printing as needed.

[0281] In the storage medium according to claim 46, a storage medium storing a program to be executed by a computer in a line-of-sight input intention transmitting device for transmitting a user's intention through the line of sight, wherein the program is stored in a display panel. Displaying an image, detecting a user's line of sight facing the image on the display panel, and determining an option on the display panel being watched by the user based on the detected line of sight position Therefore, the versatility and expandability of the eye-gaze input communication device can be improved.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of a gaze detection method.

FIG. 2 is a diagram showing an eyeball image projected on an image sensor 912 and the signal intensity thereof.

FIG. 3 is a diagram schematically showing a configuration of a personal computer system having a line-of-sight detection function.

FIG. 4 is a diagram showing a shape of a prism 1003 viewed from a side.

FIG. 5 is a diagram conceptually showing a use operation of the gaze input intention communication device.

FIG. 6 is a flowchart illustrating a processing procedure of a gaze input intention communication processing program.

FIG. 7 shows a liquid crystal display device 1002 and an external monitor 100.
FIG. 9 is a diagram showing an image of the display panel displayed in 9.

FIG. 8 is a diagram conceptually illustrating a use operation of the gaze input intention communication device according to the second embodiment.

FIG. 9 is a diagram conceptually illustrating a use operation of the gaze input intention communication device according to the third embodiment.

FIG. 10 is a diagram conceptually illustrating a use operation of a gaze input intention communication device according to a fourth embodiment.

FIG. 11 is a diagram conceptually illustrating a use operation of a gaze input intention communication device according to a fifth embodiment.

FIG. 12 is a flowchart illustrating a processing procedure of a gaze input intention communication processing program according to a sixth embodiment.

FIG. 13 is a flowchart illustrating a processing procedure of a gaze input intention communication processing program according to a seventh embodiment.

FIG. 14 is a diagram showing a change screen of an input confirmation method.

FIG. 15 is a diagram showing a screen for changing a holding time in an input determination method by holding a line of sight for a certain time;

FIG. 16 is a diagram illustrating a blink time change screen in an input confirmation method by closing an eye.

FIG. 17 is a diagram showing a screen for changing the number of characters on the display panel.

FIG. 18 is a diagram showing a TV broadcast screen when TV is selected as an option on the character selection screen.

FIG. 19 is a diagram showing a TV broadcast screen on which options are displayed.

FIG. 20 is a flowchart illustrating a diopter adjustment processing procedure according to the ninth embodiment.

FIG. 21 is a flowchart illustrating a processing procedure of a gaze input intention communication processing program according to the tenth embodiment.

FIG. 22 is a diagram showing a first page of a hiragana character selection screen.

FIG. 23 is a diagram showing a second page of the hiragana character selection screen.

FIG. 24 is a diagram showing a third page of the hiragana character selection screen.

FIG. 25 is a diagram showing a function selection screen 1 displayed when an option “function” is selected from the character selection screen.

FIG. 26 is a diagram showing a function selection screen 2.

FIG. 27 is a flowchart illustrating a calibration processing procedure.

FIG. 28 is a flowchart showing a processing procedure for input when the user turns off the screen display for a break and narrows the range in which the user can input a line of sight.

FIG. 29 is a diagram showing a normal input screen in a break mode.

FIG. 30 is a diagram showing a screen for changing a volume.

FIG. 31 is a diagram showing a screen for changing the utterance speed.

FIG. 32 is a diagram showing a screen for changing whether or not to enable a function of reading out the name of an option when the option is selected when the option is selected.

FIG. 33 is a diagram showing a screen for changing an invalid time.

FIG. 34 is a diagram showing a screen for changing the size of the keyboard.

FIG. 35 is a diagram showing a screen for changing the size of the keyboard in the left-right direction.

FIG. 36 is a diagram showing a screen for changing the size of the keyboard in the vertical direction.

FIG. 37 is a diagram showing a character selection screen after the size of the keyboard is changed.

FIG. 38 is a diagram showing a screen for changing whether or not to move the cursor to the line of sight when the line of sight is placed on the created sentence display unit on the character selection screen.

FIG. 39 is a diagram showing a screen for executing printing of created text.

FIG. 40 is a diagram showing a registration screen for storing created sentences in a memory.

FIG. 41 is a diagram showing a screen as a result of registration of a sentence.

FIG. 42 is a diagram showing a screen for confirming a user's intention when [Delete] is input.

FIG. 43 is a diagram showing a deletion result screen.

FIG. 44 is a diagram showing a screen for confirming the user's intention when “delete all” is input.

FIG. 45 is a diagram showing a screen in which the display of [▽] is changed to [△].

FIG. 46 is a diagram showing a screen in which the display of [▼] is changed to [▲].

FIG. 47 is a flowchart showing a processing procedure for reading a TXT file in which a registered sentence is written when the application starts up.

FIG. 48 is a diagram showing one continuous file in which a registration sentence is written.

FIG. 49 is a flowchart showing a processing procedure for writing a registration sentence to a file when an application is terminated.

FIG. 50 is a diagram showing a call screen for calling a registered sentence on a created sentence display section of a character selection screen.

FIG. 51 is a diagram showing a screen of a call result.

FIG. 52 is a diagram showing a fixed sentence and a fixed sentence page name as a table.

FIG. 53 is a diagram showing a screen on which a fixed sentence page is displayed on the call sentence display unit from the first fixed sentence to the fourth fixed sentence of the first page.

FIG. 54 is a diagram showing a screen in which the display of [▽] is changed to [△].

[Explanation of symbols]

2,1006 head-mounted display 3 video 4 computer 7 magnetic field generator 8 magnetic sensor 102 gaze input glasses 106 projection device 107 screen 1002 liquid crystal display element 1007 liquid crystal display circuit 1008 personal computer (personal computer) unit 1009 monitor 1064 gaze detection circuit 1814 CPU 1816 ROM

Claims (46)

[Claims] 1. An image display means for displaying an image on a display panel, and a line-of-sight detecting means for detecting a line of sight of a user facing the image on the display panel, and the intention of the user is determined by the detected line of sight. A gaze input communication device for transmitting, comprising: an option judging means for judging an option on the display panel being watched by the user based on a gaze position detected by the gaze detecting means. Input communication device. 2. A line-of-sight input intention transmitting device, comprising: line-of-sight detecting means for detecting a line of sight of a user facing an image on a display panel; and transmitting the user's intention by the detected line of sight. An eye-gaze input intention communication device, comprising: option judging means for judging an option on the display panel being watched by the user based on the detected eye-gaze position. 3. An image display means for displaying an image on the display panel, and a line-of-sight detecting means for detecting a line of sight of a user facing the image on the display panel, and the intention of the user is determined by the detected line of sight. In a gaze input intention transmitting device for transmitting, a head position detecting means for detecting a head position of the user, and a gaze position correction for correcting a gaze position detected by the gaze detection means with the detected head position. A gaze input intention communicating device, comprising: means for selecting an option on the display panel which the user is gazing at, based on the corrected gaze position. 4. Projection means for projecting and displaying an image of a display panel on a screen, and eye-gaze detecting means for detecting an eye-gaze of a user facing the displayed image of the display panel, wherein the detected eye-gaze is provided. A line-of-sight input intention transmitting device for transmitting the user's intention, comprising: option determining means for determining an option on the display panel being watched by the user, based on a line-of-sight position detected by the line-of-sight detecting means. A gaze input communication device, characterized in that: 5. Projection means for projecting and displaying an image of a display panel on a screen, and eye-gaze detecting means for detecting an eye-gaze of a user facing the displayed image of the display panel, wherein the detected eye-gaze is provided. A head position detecting means for detecting a head position of the user; and a gaze detected by the gaze detecting means at the detected head position. Gaze position correcting means for correcting a position; and option determining means for determining an option on the display panel being watched by the user based on the corrected gaze position. Transmission device. 6. The line of sight according to claim 1, wherein the option determining unit determines an option to be input when the position of the line of sight is on the same option for a predetermined time or more. Input communication device. 7. The method according to claim 1, wherein the option determining unit determines an option to be input by the user closing his / her eyes for a predetermined time after the gaze position exists on the option. 5. The gaze input intention communication device according to any one of 5. 8. The method according to claim 1, wherein the option determining unit determines an option to be input by the user pressing a switch for a predetermined time after the gaze position is present on the option. 5. The gaze input intention communication device according to any one of 5. 9. The option determining means includes: means for determining an option to be input when the gaze position is on the same option for a predetermined period of time or more; Means for determining options to be input by closing the time eye; means for determining options to be input by the user pressing a switch for a predetermined time after the gaze position is present on the options; The gaze input intention communication device according to any one of claims 1 to 5, further comprising a switching unit for switching a determination unit. 10. The eye-gaze input intention communication device according to claim 6, further comprising a change unit that allows the user to change a predetermined time until an input option is determined. 11. The line of sight according to claim 7, further comprising a change unit that allows the user to change a predetermined time during which the user keeps his / her eyes closed until an input option is determined. Input communication device. 12. The eye-gaze input intention communication device according to claim 8, further comprising a change unit that allows the user to change a predetermined time during which a switch for determining an input option is pressed. . 13. The option determining unit according to claim 1, wherein the option determining unit determines an option to be input when the gaze position is on the same option at a certain rate during a certain time. The eye-gaze input communication device according to any one of the above. 14. The gaze input intention communication device according to claim 13, further comprising a change unit capable of changing the fixed ratio by the user. 15. The eye-gaze input intention communication device according to claim 1, further comprising a change unit that can change the number of options of the display panel by the user. . 16. The apparatus according to claim 1, further comprising a switching unit that allows the user to switch between the displayed image on the display panel and the externally input image. 7. A gaze input communication device according to claim 1. 17. The eye-gaze input communication according to claim 16, wherein the externally input image is an image of a television, a video, a computer, a video-on-demand device, a clock, a security device, a videophone, and the like. apparatus. 18. The eye-gaze input intention communication device according to claim 16, further comprising operation means for allowing the user to operate an external environment device. 19. The external environment device includes a lighting device, a television, an audio device, a video, a computer, and a video-on-device.
19. A demand device, telephone, videophone, clock, electric blind, electric curtain, electric window opening / closing device, air conditioner, security device, electric reclining bed, excrement device, lifter or nurse call.
The eye-gaze input communication device according to the above description. 20. The eye-gaze input intention communicating device according to claim 1, further comprising a diopter adjustment unit that adjusts a diopter of the displayed video. 21. The gaze input intention communication device according to claim 1, further comprising an erasing means for erasing the image on the display panel. 22. The eye-gaze input communication device according to claim 21, further comprising switching means for allowing the user to switch between displaying and erasing the image on the display panel. 23. The eye-gaze input communication device according to claim 22, wherein the erasing means erases the image on the display panel when the user closes his / her eyes for a predetermined time. 24. The eye-gaze input communication device according to claim 23, further comprising changing means for changing the predetermined time during which the user keeps his / her eyes closed. 25. The eye-gaze input communication device according to claim 22, wherein the user turns the gaze in a predetermined direction to delete the image on the display panel. 26. The gaze input communication device according to claim 25, further comprising changing means for changing a predetermined direction in which the user turns the gaze. 27. The eye-gaze input intention communication device according to claim 1, further comprising an area limiter that limits an area that can be input by the eye. 28. The eye-gaze input intention communication device according to claim 27, further comprising a switching unit that can switch whether or not the user can limit an input-enabled area by the eye-gaze. 29. The gaze input intent communication device according to claim 28, further comprising changing means for changing an area in which the user is restricted from inputting with the gaze. 30. The eye-gaze input intention communication device according to claim 27, wherein the user can close an eye for a predetermined time to limit an area that can be input by the gaze. 31. The eye-gaze input communication device according to claim 30, further comprising changing means for changing the predetermined time during which the user keeps his / her eyes closed. 32. The gaze input intention communication device according to claim 27, wherein the user turns his / her gaze in a predetermined direction to limit an area where the user can input by the gaze. 33. The eye-gaze input intention communication device according to claim 32, further comprising changing means for changing a predetermined direction in which the user turns his / her gaze. 34. The gaze position is not displayed in an area where input by the gaze is restricted.
7. A gaze input communication device according to claim 7. 35. The apparatus according to claim 27, further comprising changing means for allowing the user to change whether or not to display the line-of-sight position in an area where input by the line of sight is restricted. Gaze input communication device. 36. The gaze input intention communication device according to claim 1, further comprising changing means for changing a size of the displayed display panel. 37. The apparatus according to claim 36, further comprising changing means for changing the size of the display panel by the user.
The eye-gaze input communication device according to the above description. 38. The eye-gaze input intention communication device according to claim 1, further comprising changing means for changing a cursor position of the created sentence display unit by a line of sight. 39. The gaze input intention communication device according to claim 38, further comprising switching means for switching whether or not the user changes the cursor position based on the line of sight. 40. The gaze input intention communication device according to claim 1, wherein the gaze input is not accepted for a predetermined time after the gaze input is determined. 41. The gaze input intention communication device according to claim 40, further comprising changing means for changing a predetermined time during which the user does not accept the gaze input after the user has confirmed the gaze input. 42. The gaze input intention communication device according to claim 1, further comprising printing means for performing printing in accordance with the gaze option. 43. The gaze input communication device according to claim 42, wherein the paper is not discharged every time the printing is executed. 44. The eye-gaze input communication device according to claim 43, further comprising changing means for allowing the user to change whether to discharge the paper every time the printing is executed. 45. A gaze input intention transmitting method for displaying an image on a display panel, detecting a line of sight of a user facing the image on the display panel, and transmitting the user's intention by the detected line of sight. A gaze input intention communication method, wherein an option on the display panel that the user is gazing at is determined based on the detected gaze position. 46. A storage medium storing a program executed by a computer in a line-of-sight input communication device that transmits a user's intention by line of sight, the program comprising: a step of displaying an image on a display panel; A step of detecting a user's line of sight facing the image on the display panel, and a step of determining an option on the display panel being watched by the user based on the detected line of sight position. Storage medium.