JP2000148393A - Visual and touch sense information transmitting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a visual and touch sense information transmitting device by which visual information and touch sense information are presented on the same position at the same time and the information is also inputted at the position. SOLUTION: The touch sense information is displayed by selectively applying voltage to vertical electrodes 1 to 9 and horizontal electrodes A to G, ruggedly deforming or vibrating a piezoelectric material sheet 31 and forming a key top. Since a polarization is caused when external force is applied by a positive piezoelectric effect, when a part of the key top is depressed the position is scanned and detected and the information is inputted. In addition, the visual information is displayed by utilizing an electrooptical effect (a change of a refractive index) of the piezoelectric material sheet 31.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual and tactile information transmitting device which can transmit information to a human through both visual and tactile senses.

More specifically, the present invention relates to the field of technology of a visual and tactile information transmission device capable of presenting both video information on a display and tactile information by deformation and vibration.

[0003] Furthermore, the present invention relates to a visual and tactile information transmitting device which enables a human to transmit information to the device.

[0004]

2. Description of the Related Art At present, displays using a cathode ray tube (CRT) or a liquid crystal panel are mainly used as visual information transmitting devices.

[0005] By combining these devices with a mouse or a touch panel, a picture is displayed on the screen of the visual information transmission device, and the picture is selected using a mouse or a touch panel, that is, a so-called GUI (graphic user interface). Man-machine interfaces are widespread. These are widely used in computers, automatic ticket vending machines at stations, ATMs (automatic teller machines) in banks, and the like.

In addition, CAD (Computer Aided
In the field of design, a method of displaying a designed two-dimensional or three-dimensional drawing on a display and operating elements (points, line segments, etc.) on the drawing using a mouse has become widespread.

However, in these GUIs and CADs,
Elderly or visually impaired people with poor eyesight cannot use small pictures and drawings, and the touch panel does not have a click feeling, so there are many malfunctions such as "pressing twice or pressing twice". The point has.

[0008] As an information transmission device mainly for visually impaired persons, a tactile information transmission device such as a braille display, a tactile display, and a tactile display is known. In these devices, thin pins arranged in a lattice using a piezoelectric element or the like are vibrated up and down to display Braille, characters, pictures, and the like.

However, it is difficult to reduce the size of the device because the piezoelectric element to be driven is large with respect to the necessary pin arrangement interval.
In addition, there is a problem in that a considerable amount of training is required for interpretation, and only a tactile sensation can be presented, so that an untrained healthy person cannot use it.

[0010] On the other hand, in the field of visual and tactile information transmission devices, technologies called tool-mediated or object-oriented for the purpose of achieving an artificial reality are being developed.

The former, ie, tool-mediated type (multi-modal mouse, Institute of Biotechnology, H / I Society 1995)
In / 1), visual information is presented on a flat display such as a CRT or HMD (head mounted display), and tactile information is presented by transmitting vibration to a mouse or joystick.

In the latter case, the object-oriented type (haptic screen, University of Tsukuba, VR Society 1996/1
In (0), a linear actuator is arranged on the back surface, an elastic sheet capable of deforming the surface into irregularities is prepared, and an image is projected on the surface using a projector.

[0013] However, in the tool-mediated type, since the information transmission points are different, a sense of incongruity is caused in the recognition, the absolute coordinate position cannot be transmitted only by the tactile information, and the information can be simultaneously presented to a plurality of persons. Difficulties are pointed out.

In addition, the object-oriented type has problems that there is no information input means, and that the image is blocked by the operator's hand because the image is projected on the surface of the sheet.

[0015]

SUMMARY OF THE INVENTION The present invention solves the above-described problems, and visual information and tactile information are presented at the same time and at the same position. It is an object of the present invention to provide a visual and tactile information transmission device that can perform the operation.

[0016]

The above object is achieved by the following means.

A first aspect of the present invention is a visual / tactile information transmitting device, which presents an image on a surface of the visual / tactile information transmitting device and transmits visual information for transmitting the information through the visual sense. A visual tactile sensation, comprising: a presentation means; and a tactile information presenting means arranged on a video display surface of the visual information presenting means for deforming or vibrating the surface and transmitting information through a tactile sensation. Information transmission device.

[Solution of the Second Invention] In the visual and tactile information transmitting apparatus of the first invention, the visual information presenting means uses a liquid crystal panel, a CRT, or other light wavelength and intensity. A visual and tactile information transmission device for presenting visual information by using a visual sensor.

According to a third aspect of the present invention, in the visual and haptic information transmitting apparatus according to the first aspect, the haptic information presenting means uses a piezoelectric element. apparatus.

"Solution of the Fourth Invention" In the visual-tactile information transmitting device according to the first invention, the tactile information presenting means may include an electrostatic force, an electromagnetic force, a thermal expansion, a magnetostriction, a light distortion, and other mechanical devices. A visual and tactile information transmission device characterized by utilizing a temporal displacement.

A fifth aspect of the present invention is a visual-tactile information transmitting device, which is a visual-tactile information transmitting device that presents an image on its surface and transmits visual information through the visual sense. Presenting means, disposed on the image display surface of the visual information presenting means, deforming or vibrating the surface, and transmitting tactile information through tactile information presenting means, externally applied pressure and its position A visual input unit for inputting information according to a position to which pressure is applied, wherein the information input unit is configured to input information based on a position where pressure is applied.

[Sixth Solution of the Invention] In the visual-tactile information transmitting device according to a fifth invention, the tactile information presenting means and the information input means can transmit light. A visual-tactile information transmission device, which is arranged so as to be superimposed on visual information presentation means.

[Seventh Invention Solution] In the visual-tactile information transmitting device according to a fifth invention, the information input means comprises a piezoelectric element or other reversible electric signal / mechanical displacement conversion means. A visual and tactile information transmission device, characterized in that:

[Eighth Solution to the Invention] In the visual-tactile information transmitting device according to the fifth invention, the tactile information presenting means and the information input means are one device which is composed of a piezoelectric element and is shared. A visual-tactile information transmission device, comprising:

[Ninth Solution of the Ninth Invention] In the visual-tactile information transmission device according to a fifth invention, the information input means may include a capacitance, a strain, a resistance, or other mechanical displacement as an electric signal. A visual-tactile information transmission device, characterized in that the device utilizes an element that converts the information into a visual and tactile information.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention provides a visual / tactile information transmitting apparatus capable of realizing both visual information presentation by video and presentation of tactile information by deformation and vibration with the above-described configuration. .

The visual information presenting means is a liquid crystal panel or a CRT.
For example, a video is created, and visual information is transmitted through the video.

The tactile information presenting means is formed of, for example, a thin sheet using a piezoelectric material such as quartz that can transmit light.
Lattice electrodes are formed on the front and back of the sheet using a dielectric resin or the like that can transmit light, and when an electric field is applied to a predetermined electrode, vibration or distortion is generated at the lattice points due to the inverse piezoelectric effect. The generated vibration and distortion are read by touching with a finger or the like.

The information input means detects the pressure applied from the outside to the sheet and its position by using the electromotive force of the sheet due to the positive piezoelectric effect, and inputs information.

The positive piezoelectric effect and the inverse piezoelectric effect are phenomena in which electric polarization (voltage) occurs when a stress is applied to a piezoelectric material, and mechanical phenomena occur when an electric field is applied to the piezoelectric material. is there.

[0031]

Embodiment 1 FIG. 1 shows the overall configuration of a visual and tactile information transmitting apparatus according to an embodiment of the present invention.

The visual and tactile information transmission device 10 is a liquid crystal panel or C used in a general television or computer.
It comprises a display 20 such as an RT and a tactile / touch panel 30 arranged on the display surface.

An image such as a moving image or a still image is presented on the display 20, and minute irregularities or vibrations corresponding to the image are presented on the tactile / touch panel 30 on the surface of the display 20.

Tactile information is transmitted to the operator by the unevenness and the vibration. If the video is a picture, it represents its three-dimensional shape and outline, and if it is a character, it represents its font and its corresponding braille. Furthermore, by changing the amount of displacement, amplitude, frequency, and the like, it is possible to express the color and shade of the video.

The operator can obtain visual information through the image on the display 20 and also obtain tactile information through displacement and vibration of the touch panel 30.

On the other hand, the tactile / touch panel 30 also serves as an information input means. When a predetermined pressure or more is applied to the surface, the position and the magnitude of the pressure can be detected and transmitted to the apparatus main body. . Thus, the target in the video can be identified by directly touching the target. Also, by displaying selection keys and numeric keys on the display 20 and detecting the presence or absence of pressure at each key position and the order thereof,
The key selection result of the operator can be known.

FIG. 2 is a diagram for explaining the configuration of the touch / touch panel 30.

The tactile / touch panel 30 includes a piezoelectric material sheet 31 and vertical electrodes 1 to 9 and horizontal electrodes A to
G, which intersect in a grid pattern. The number of the electrodes can be increased or decreased as necessary.

The touch and touch panel 30 is made of quartz or PLZ.
It is composed of a piezoelectric material that can transmit light, such as T (lead zirconate titanate). This piezoelectric material is formed into a thin-layer sheet to form a piezoelectric material sheet 31, which has been subjected to a polarization process so that its displacement direction is the thickness direction.

By applying a predetermined electric field in the thickness direction of the piezoelectric material sheet 31, the thickness of the portion to which the electric field is applied can be increased or decreased.

When the applied electric field is direct current, a fixed uneven shape can be given, and when the applied electric field is alternating current, it becomes oscillating.

The vertical and horizontal electrodes for applying an electric field are formed in a grid pattern on the front and back of the piezoelectric material sheet 31 by using a light-transmitting conductive film such as ITO (indium tin oxide) or ZnO (tin oxide). Is formed. By forming vertical electrodes 1 to 7 on the front surface in the vertical direction on the front surface and horizontal electrodes A to E on the rear surface in the horizontal direction on the back surface, and selectively applying an electric field to the vertical and horizontal electrodes, Displacement and vibration are generated at the intersection of the electrodes.

Since the vertical electrodes 1 to 7, the horizontal electrodes A to E, and the piezoelectric material sheet 31 transmit light, the image displayed on the display 20 can be viewed from the front side.

In FIG. 2, a front electrode: 7 and a back electrode: E are selected, and an electric field is applied from the front surface to the back surface, thereby deforming the intersection (position 7-E, d) into a convex shape. Is shown. Further, by selecting a plurality of electrodes and applying a signal, an arbitrary displacement or vibration can be generated at an arbitrary position.

Both the visual information presented on the display 20 and the tactile information presented on the touch panel 30 are transmitted from the control unit of the visual / tactile information transmitting device 10. The contents include video information from a television, text and photo information from a computer, operation panel information from various control devices, and the like, and the control unit of the visual-tactile information transmission device 10 receives information from these external devices. .

Here, visual information is presented by changing the wavelength and brightness of light in a liquid crystal panel or a CRT as in the prior art.

On the other hand, tactile information is presented in different ways depending on the content to be transmitted. In other words, when the transmitted information is textual information, it is more important to transmit the meaning content than its shape, and it is difficult to read even if the fine character shape is expressed by unevenness. For reasons, it is mainly converted into the corresponding braille and displayed. Note that a large character that can be read in a shape such as a title may be displayed as described later.

When the visual / tactile information transmission device 10 can obtain character codes and position information, such as an operation panel of a computer or other equipment, the codes are converted into Braille and displayed in the same position as the characters. . Graphic parts such as tables and ruled lines are processed similarly.

On the other hand, when characters are treated as part of video information as in a television, they cannot be directly converted into Braille information. So, OCR
Using a known character recognition technology such as an (optical character reading device), a character portion is cut out from the video information to obtain its code and position information.

When a character has a color or shading, the character can be presented with a different displacement. Further, when expressing by vibration, it is also possible to change the amplitude and frequency to express.

When the transmitted information is a picture such as a photograph, it is necessary to transmit the shape. As a method of presenting the shape, a method of expressing the contour, a method of expressing the depth, or the like can be adopted.

When a picture is represented by an outline, video information to be displayed on the display 20 is secondarily differentiated to extract a luminance edge portion, which is represented by unevenness.

On the other hand, when expressing by depth,
It receives depth information together with video information from an external device. Computer graphics, the operation panel of the device, and the like receive depth information created artificially, and in the case of a video obtained naturally such as a television camera, receive the depth information measured by a stereoscopic imaging device.

The visual / tactile information transmitting device 10 controls the touch / touch panel 30 using the received depth information, and presents a three-dimensional shape due to the unevenness at a position matching the image. It is also possible to express the size of the unevenness by the difference in amplitude and frequency of vibration.

The above operation is repeated at predetermined time intervals,
By updating the displacement, it is possible to represent a moving image.

Thus, the visually impaired person and the healthy person can obtain the same information (contents and arrangement).

The tactile / touch panel 30 shown in FIG.
It also has the function of a touch panel.

The piezoelectric material has a characteristic of generating an electric field when receiving a pressure from the outside, and the pressure can be detected by accumulating the electric field and extracting it as a voltage.
Even when the tactile panel is displaced, it can be detected in the form of voltage fluctuation.

FIG. 3 is a diagram showing a state where the touch panel is used as a touch panel.

The visual / tactile information transmission device 10 alternately repeats scanning of the vertical (front) electrode and scanning of the horizontal (back) electrode at predetermined time intervals, and detects a potential difference generated between the electrodes.

As shown in FIG. 3, when an external pressure is applied to the coordinate portion of the vertical 3 / horizontal F, the touch and touch panel 3
An electric field is generated in this portion of 0 from the back surface to the front surface.
By detecting this electric field as a potential difference, the vertical 3 / horizontal F
It is detected that an external force is applied to the position. Since the magnitude of the external force and the magnitude of the electric field are almost proportional,
By obtaining and storing the relational expression in advance, the magnitude of the external force can be obtained.

In this embodiment, the panel is made of one piece of piezoelectric material. However, when pressure is applied to one location, a force is transmitted to the surrounding area and an electric field is generated. In this case, the position of the largest value or the position of the center of gravity in the coordinate group where the electric field is detected is detected, and this point is regarded as the center where pressure is applied. If the calculation is performed based on the physical properties such as the elastic modulus of the material of the touch panel and the touch panel, the center position can be obtained more accurately. The same processing is performed even when a force is applied to a plurality of locations.

FIG. 4 shows an example in which a key top to be operated is formed at the center of the piezoelectric material sheet 31 of the tactile / touch panel 30 to display tactile information, and information is input by simultaneously pressing a part of the key top. ing. Here, vertical 3-7 /
A keytop is formed by applying a voltage to the electrodes of the horizontal DF to form a rectangular convex portion on the surface of the panel. If the interval between the electrodes is sufficiently small, it is possible to display the function name in Braille inside or around the key top.

The apparatus sequentially scans the electrodes and detects that an electric field is generated at the coordinate of 5 / E in the vertical direction. When it is determined that the position to which the external force is applied is inside the formed key top, it is determined that the key is selected. The number of key tops may be one or more, and detection is possible even when a plurality of keys are pressed sequentially or simultaneously. In addition, erroneous operation is prevented by detecting that the external force is equal to or greater than a predetermined value based on the magnitude of the electric field and recognizing the key input.

The scanning of the electrodes for detecting the electric field can also serve as the scanning when applying a voltage for presenting tactile information.

FIGS. 5 and 6 are flowcharts showing the operation of the control device.

When the information presentation operation is started (S10),
The control device of the visual-tactile information transmission device 10 receives and stores information to be transmitted from an external device such as a computer or a television receiver (S11). When the information for one screen is obtained, the display coordinates of the display are scanned to control the wavelength and intensity of light in order to display visual information (S12).

When the scanning of all the coordinates is completed and the display of the visual information for one screen is completed (S13, YES), the control device determines whether the display contents are characters, pictures, or both (S1).
4). This attribute determination is performed based on the type of the connected device, the attached information, and the like. If both are mixed, the control device determines whether a character portion is included using a known image processing method (S15). Determination of attributes (S14, S1
When 5) is completed, the character portion is converted into a braille code (S16). In addition, the pattern portion is set according to the setting (S1
7) The contour is extracted (S18) or the depth information is received (S19).

When the information to be presented is created, the control device scans the electrodes of the touch and touch panel 30 and applies a voltage to predetermined coordinates (S20) to generate displacement and vibration (S20).
21). When the control of all the electrodes is completed (S22), the apparatus scans the electrodes again to measure a potential difference at each coordinate (difference from a given potential) (S23), and when this value exceeds a predetermined value (S23). In S24, YES), the coordinate value is stored (S25).

The scanning of all the electrodes is completed (S26).
And whether or not the reception of information from the external device is completed (S
27) If continuing (S27, NO), the process moves to a new information presentation operation (S11). If the information reception has been completed (S27, YES), the process ends (S30).

FIG. 7 is a block diagram of the control device.

The control of the control device is performed by a CPU (Central Processing Unit) 50. CPU
Reference numeral 50 denotes a peripheral device, an external information acquisition and storage unit 51 for acquiring and storing information from an external device, and a content determination unit 5.
2. Braille conversion means 5 for converting a character code into a Braille code
3. Character recognition means 54 for extracting a character portion from video information and converting it to a character code, depth information receiving means 55, contour extraction means 56 for extracting a contour from video information, and coordinate value storage for storing coordinates at which a potential difference occurs. A transmission unit 57 and a time measuring unit 58 for measuring the passage of time are provided.

The CPU 50 includes a CRT control means 60 for controlling the CRT display, a voltage applying means 61 for controlling the driving and detection of the touch / touch panel, and a potential difference measuring means 6.
2 are connected. The voltage application unit 61 sequentially scans the electrodes of the vertical electrode scanning unit 63 and the horizontal electrode scanning unit 65, and the tactile / touch panel 30 is deformed by the voltage application unit 61.
Vibration is performed, and external pressure is detected by the potential difference measuring means 62.

As described above, the visual information and the tactile information are
Since the information can be presented at the same position and information can be input at the same position, all persons can accurately interface with the device.

[Embodiment 2] In the tactile panel, other phenomena such as electrostatic force, electromagnetic force, thermal expansion, magnetostriction, and optical distortion that convert an electric signal into displacement / force can be used.

FIG. 8 shows a second embodiment using the thermal expansion of a gas. The tactile panel is sandwiched between two transparent sheet members, and the sheet on the surface is made of a flexible material. The gap between the two sheets is partitioned into minute chambers (one of which is shown in FIG. 8) sealed by vertical and horizontal lattice members, and a gas having a property of thermally expanding inside. Is filled. A vertical electrode 71 and a horizontal electrode 72 are attached to the vertical and horizontal grid members as in the first embodiment, and a transparent heating member 70 forming a resistor is connected to the intersection of each electrode.

In this device, by selecting predetermined vertical and horizontal electrodes 71 and 72 and applying a potential therebetween, the planar resistor (heat generating member 70) at the intersection is heated to reduce the temperature of the sealed gas. To raise. As the temperature of the gas increases, the volume of the gas increases, so that the flexible sheet on the surface bulges outward to form a convex portion.

In this case, the touch panel is arranged on the surface of the tactile panel, and the touch panel is required to be thin and flexible enough to detect its displacement from the upper part. Can be adopted.

Example 3 PLZT, which was mentioned above as an example of a tactile panel material, has not only a piezoelectric effect but also an electro-optical effect (a property that the refractive index changes when an electric field is applied). Conventionally, the development of optical shutters and optical memories utilizing the latter property has been advanced. Also in the present embodiment, the tactile display and the visual display can be realized by one panel by using both the piezoelectric effect and the electro-optical effect of PLZT.

When an electric field is applied to a predetermined portion of the panel formed by PLZT, the portion is displaced and the refractive index changes. Therefore, if a light source that emits light is arranged behind the light source, the amount of light reaching the surface changes. Visual display can be performed by this property. In addition, R
A color image can be displayed by disposing a GB filter. Thus, in this embodiment, the CRT
Since there is no need to provide a display means such as a liquid crystal display or a liquid crystal display for that purpose, there is an advantage that cost and space can be reduced.

[0081]

According to the visual / tactile information transmitting apparatus of the present invention, visual information and tactile information can be displayed in a superimposed manner. Further, information can be input by operating the display surface. For this reason, many people including a healthy person and a visually impaired person can operate in the same environment, and it becomes possible to interface correctly with the visual-tactile information transmission device.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an overall configuration of a visual and tactile information transmission device according to an embodiment.

FIG. 2 is a diagram for explaining a configuration of a tactile / touch panel 30;

FIG. 3 is a diagram for explaining the operation of the touch panel 30 when the touch panel 30 is used as a touch panel;

FIG. 4 shows a haptic / touch panel 30 displaying haptic information;
It is a figure showing signs that it also functions as a touch panel.

FIG. 5 is a flowchart showing the operation of the control device (first half);
It is.

FIG. 6 is a flowchart showing the operation of the control device (second half);
It is.

FIG. 7 is a block diagram of a control device.

FIG. 8 is a diagram for explaining a second embodiment in which tactile information is displayed using thermal expansion of a gas.

[Explanation of symbols]

 1-9 Vertical electrodes A-G Horizontal electrodes 10 Visual and tactile information transmission device 20 Display 30 Tactile and touch panel 31 Piezoelectric material sheet 50 CPU 51 External information acquisition and storage means 52 Content discrimination means 53 Braille conversion means 54 Character recognition means 55 Depth information reception Means 56 Contour extraction means 57 Coordinate value storage / transmission means 58 Time measurement means 60 CRT control means 61 Voltage application means 62 Potential difference measurement means 63 Vertical electrode scanning means 65 Horizontal electrode scanning means

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (reference) G09F 27/00 G09F 27/00 A

Claims (9)

[Claims] 1. A visual-tactile information transmitting device, comprising: a visual-information presenting means for presenting an image on a surface thereof and transmitting information via a visual sense; A tactile information presenting means disposed on a video display surface of the means for deforming or vibrating the surface and transmitting information through a tactile sensation. 2. The visual and tactile information transmitting device according to claim 1, wherein the visual information presenting means presents visual information using a liquid crystal panel, a CRT, or other light using the wavelength and intensity of light. A visual and tactile information transmission device, characterized in that: 3. The visual and tactile information transmitting device according to claim 1, wherein the tactile information presenting means uses a piezoelectric element. 4. The visual / tactile information transmitting device according to claim 1, wherein the tactile information presenting means utilizes electrostatic force, electromagnetic force, thermal expansion, magnetostriction, optical distortion, and other mechanical displacements. A visual and tactile information transmission device, characterized in that it is provided. 5. A visual / tactile information transmitting device, comprising: a visual information presenting means for presenting an image on a surface thereof and transmitting information via a visual sense; Tactile information presenting means arranged on the image display surface of the means for deforming or vibrating the surface and transmitting information via tactile sense, and information input means capable of detecting externally applied pressure and its position And the information input means is for inputting information according to the position where the pressure is applied. 6. The visual and tactile information transmitting device according to claim 5, wherein the tactile information presenting means and the information input means are capable of transmitting light, and are superimposed on the visual information presenting means. A visual / tactile information transmission device, which is arranged. 7. The visual-tactile information transmitting device according to claim 5, wherein the information input means comprises a piezoelectric element or other reversible electric signal / mechanical displacement converting means. Information transmission device. 8. The visual and tactile information transmitting device according to claim 5, wherein the tactile information presenting means and the information input means are one device composed of a piezoelectric element. Tactile information transmission device. 9. The visual and tactile information transmission device according to claim 5, wherein the information input means uses an element for converting a mechanical displacement into an electric signal, such as capacitance, strain, resistance, or other. A visual and tactile information transmission device, characterized in that: