JP4894905B2 - Information processing system and display processing program - Google Patents

Description

  The present invention relates to an information processing system capable of inputting information using a coding pattern indicating position coordinates, and more particularly to control of an area for a user to enter characters, symbols, and the like.

  In recent years, electronic pens that digitize written information have been developed, and “Anoto Pen” developed by Anoto, Sweden, is known as a representative one. Anotopen is used together with dedicated paper (hereinafter, simply referred to as “exclusive paper”) on which a dot pattern indicating position coordinates patterned by a predetermined algorithm is printed.

  Specifically, the anotopen has a small camera for capturing a dot pattern printed on dedicated paper at the pen tip, a processor for calculating position coordinates on the dedicated paper from the captured dot pattern, and a calculated position. It is equipped with a data communication unit that transmits coordinates etc. to external devices. In addition, when a user writes characters on the special paper with an anotopen, or when a check mark is entered in the image designed on the special paper, the small camera prints on the special paper as the pen moves. The captured dot pattern is imaged, and information (hereinafter referred to as “entry information”) such as characters, symbols, or figures entered by the user is generated from successive position coordinates calculated by the processor. Thereafter, the generated entry information is transmitted from the Anotopen to a terminal device such as a nearby personal computer or mobile phone by the data communication unit (see, for example, Patent Document 1).

  In addition, an information processing system has been proposed in which characters and symbols are entered by touching a display screen of a computer device with a dedicated pen or the like and the information is acquired. As one of them, there is known one using a character entry area for entering a character handwritten by a user in an area on a predetermined screen. For example, as a system using such a character entry area, a character entry area is set in advance, and a person who fills in a character by acquiring a handwritten character or symbol in the character entry area ( The burden at the time of entry is reduced for a so-called user (for example, Patent Document 2).

  Furthermore, what uses such an information processing system as an input interface of the navigation apparatus in moving bodies, such as a car, is also known (for example, patent documents 3). In this system, the inclination angle of the character entry area is set based on the user's instruction, and information on the characters or symbols entered based on the set inclination angle is acquired.

  On the other hand, without setting a character entry area, the user can freely enter characters or symbols, and based on those characters or symbols, the direction as a written character string (writing direction), the size of the character An information processing system that analyzes the direction of each character (character direction) is also known (for example, Patent Document 4).

Japanese translation of PCT publication No. 2003-511761 JP-A-8-50530 JP 2008-250679 A JP 2002-175498 A

  However, none of the systems for inputting handwritten characters described in Patent Documents 2 to 4 is constructed as a system for executing smooth character input.

  For example, in the information processing system described in Patent Document 2, although there is a description that characters are entered in the inclined character entry area, the character entry area such as movement or rotation of the character entry area during character entry There is no description to change the range. Therefore, in this information processing system, it may be difficult for the user to smoothly perform character input during character input depending on the user's habit or the situation during the user's character input.

  In addition, in the navigation device described in Patent Document 3, it is described that the inclination of the character entry area is set by the user before the character is entered and the character entry area is fixed. There is no description to change the range of the character entry area during character entry, such as movement or rotation of the entry area. Therefore, as in Patent Literature 2, depending on the situation of the user's habit or the user's character input, it may be difficult for the user to smoothly execute the character input during character input.

  On the other hand, the information processing system described in Patent Document 4 does not have a function of displaying a character entry area for assisting a character input position and reducing a processing load such as recognition processing. Since the position where the character is entered is not fixed, there is no concept of changing the range of the character entry area.

  The present invention has been made to solve the above-mentioned problems, and its purpose is to allow the user to smoothly input characters during character input in consideration of the user's habit or the situation during character input by the user. Another object is to provide an information processing system and a display processing program that can be used.

  (1) In order to solve the above-described problem, the present invention reads the coded pattern along a movement path when moving on a screen on which a coded pattern indicating position coordinates is formed based on an operation by a user. Reading means, calculation means for calculating the coordinates of the movement path based on the data of the coded pattern read by the reading means, and the shape of the movement path formed based on the calculated coordinates of the movement path, Projection output means for outputting to the projection means for projecting onto the screen as an image, and character entry for processing the shape of the movement path as a character in an area formed in a part of the image projected on the screen Rotation control means for rotating the area based on the shape of the movement path in the character entry area, and the rotation control means is the user A character string composed of the shape of the movement path of the reading means during the entry of the character moving in the character entry area by the reading means based on the operation, and in the character entry area being entered The inclination angle of the character string is calculated, and the character entry area is rotated based on the calculated inclination angle.

  With this configuration, when the reading means is moved by the user and a character is input into the character entry area, the present invention projects the moving path of the reading means onto the screen as the input character string, and It is possible to calculate the inclination angle of the character string input by the user during the entry of, and rotate the character entry area based on the calculated inclination angle. Therefore, according to the present invention, it is possible to allow the user to smoothly input a character during character input in consideration of the user's situation during character input, such as a user's habit or an entry state depending on the user's standing position on the screen.

  (2) In order to solve the above-described problem, the present invention uses the coded pattern as a movement path when moving on a light-transmitting sheet on which a coded pattern indicating position coordinates is formed based on an operation by a user. Reading means for reading along, calculation means for calculating the coordinates of the movement path based on the data of the coded pattern read by the reading means, and the movement path formed based on the calculated coordinates of the movement path Display output means for outputting a shape as an image from a back surface of the light transmissive sheet to a display means for visual display from the front surface of the light transmissive sheet; and one of images displayed on the light transmissive sheet. A character entry area for processing the shape of the movement path as a character is formed based on the shape of the movement path in the character entry area. Rotation control means, and the rotation control means has a shape of a movement path of the reading means while the reading means is entering a character moving in the character entry area based on an operation of the user. The character string is configured to calculate an inclination angle of the character string in the character entry area being entered, and to rotate the character entry area based on the calculated inclination angle. .

  With this configuration, when the reading unit is moved by the user and a character is input into the character entry area, the moving path of the reading unit is displayed on the light transmissive sheet as the input character string. The inclination angle of the character string input by the user during the entry of the character can be calculated, and the character entry area can be rotated based on the calculated inclination angle. Therefore, according to the present invention, it is possible to allow the user to smoothly input a character during character input in consideration of the user's situation during character input, such as a user's habit or an entry state depending on the user's standing position on the screen.

  (3) In the information processing system, the rotation control unit calculates an inclination angle of the character string at every predetermined timing while the character is being entered, and the character is calculated each time the inclination angle is calculated. It is good also as a structure which rotates an entry area.

  With this configuration, when the reading means is moved by the user and a character is input in the character entry area, the inclination angle of the character string input by the user is calculated every predetermined timing, and the calculated inclination angle is set. Based on this, the character entry area can be rotated. Therefore, the character entry area can be rotated in accordance with changes in the user's situation in character input, such as the entry situation depending on the user's habit or the user's standing position on the screen, so that the user's situation changes during character input. Even so, the user can input characters smoothly.

  (4) The information processing system may further include character recognition processing means for executing character recognition based on the shape of the movement path in the character entry area, and the rotation control means may be configured as the predetermined timing. Each time the character recognition processing unit recognizes one character, the inclination angle of the character string may be calculated based on the movement path group used when the character is recognized as the character.

  With this configuration, the inclination angle of the character string can be calculated for each character. Therefore, the inclination angle of the character string can be calculated while accurately grasping the character string entered by the user. Therefore, it is possible to rotate the character entry area so as to accurately correspond to the character string entered by the user.

  (5) Furthermore, in the information processing system, each time the rotation control unit recognizes one character by the character recognition processing unit, a reference point determined based on a movement path group of the recognized character is obtained. The inclination angle of the character string may be calculated based on the obtained reference points.

  With this configuration, the inclination of the character string is based on a point calculated by a predetermined calculation such as the center point or the center of gravity of each character, or a predetermined sample point such as the uppermost point or the lowermost point on the coordinates of each character. Since the angle can be calculated, the inclination angle of the character string can be calculated simply by comparing the reference point such as the calculation point or sample point of each character. Therefore, the inclination angle of this character string can be calculated easily and reliably.

  (6) Further, in the information processing system, when the character recognition processing unit rotates the character entry area, a movement path that forms the character string based on an inclination angle of the character entry area. Is preferably converted into a movement path corresponding to the character entry area before rotation, and character recognition of the character string is executed again.

  With this configuration, since the moving path of the reading means can be converted into a moving path corresponding to the character entry area before the rotation based on the rotation angle of the character entry area, an inclination occurs in each entered character. The character string recognition process can be executed again depending on the status of the character string not being displayed. Therefore, the accuracy of character recognition can be improved.

  (7) In the information processing system, each time the rotation control unit specifies one stroke that is a component of the movement path as the predetermined timing, the coordinates of the movement path included in each stroke It is preferable that the character entry area is rotated by calculating the inclination angle of the character string based on the group.

  With this configuration, the inclination angle of the character string can be calculated for each stroke of the movement path, so that the inclination angle of the character string can be calculated while reliably grasping the shape of the character entered by the user. Since the stroke angle of the character string can be calculated as long as the shape of the stroke is specified, the tilt angle of the character string can be calculated without executing complicated processing such as interlocking with the character recognition processing. .

  (8) Further, in the information processing system, each time the one stroke is specified, the rotation control unit obtains a reference point determined based on a coordinate group of the movement path included in each stroke, and obtains the reference point. The inclination angle of the character string may be calculated based on each reference point.

  With this configuration, the inclination angle of the character string can be calculated based on a point calculated by a predetermined calculation such as the center point of each stroke or a predetermined sample point such as the highest point or the lowest point of each stroke. Therefore, the inclination angle of the character string can be calculated simply by comparing the reference point such as the calculation point or sample point of each character. Therefore, the inclination angle of this character string can be calculated easily and reliably.

  (9) The information processing system may further include character recognition processing means for performing character recognition based on the shape of the movement path in the character entry area.

  With this configuration, the character recognition process can be executed as a process separate from the calculation of the inclination angle of the character string, so there is no need to link the calculation of the inclination angle of the character string and the character recognition. It can be constructed easily.

  (10) Further, in the information processing system, when the character recognition processing means rotates the character entry area, the movement path of the reading means is rotated based on the rotation angle of the character entry area. It is preferable that the character recognition is performed by converting to a movement path corresponding to the character entry area before the character entry.

  With this configuration, since the moving path of the reading means can be converted into a moving path corresponding to the character entry area before the rotation based on the rotation angle of the character entry area, an inclination occurs in each entered character. The character string recognition process can be executed depending on the state of the character that is not. Therefore, the accuracy of character recognition can be improved.

  (11) In the information processing system, when the rotation angle of the character string has already been calculated, the rotation control unit newly calculates one or more first inclination angles that have already been calculated. A new third tilt angle may be calculated based on the second tilt angle and the character entry area may be rotated based on the calculated third tilt angle.

  With this configuration, the inclination angle can be calculated in consideration of the already entered character string, so that the rotation of the character entry area can be reliably rotated in accordance with the user entry situation.

  (12) In the information processing system, the display control unit that performs display control of a display unit that displays the same image as the image projected on the screen, and the coordinates calculated by the calculation unit are displayed on the display unit. It is preferable to further comprise coordinate conversion means for converting the display screen coordinates of the means, wherein the rotation control means rotates the character entry area based on the coordinates converted by the coordinate conversion means.

  With this configuration, the coordinates calculated by the calculation means are converted into the coordinates of the display screen of the display means by the coordinate conversion means, and the character entry area is rotated, so that different coordinate systems for the screen and the display screen of the display means are used. Even if it is used, the rotation process of the character entry area can be surely executed.

  (13) Alternatively, in the information processing system, the information processing system further includes coordinate conversion means for converting the coordinates calculated by the calculation means into coordinates on a display screen of the display means, and the rotation control means is controlled by the coordinate conversion means. The character entry area may be rotated based on the converted coordinates.

  With this configuration, the coordinates calculated by the calculation means are converted into the coordinates of the display screen of the display means by the coordinate conversion means, and the character entry area is rotated, so that different coordinate systems for the screen and the display screen of the display means are used. Even if it is used, the rotation process of the character entry area can be surely executed.

  (14) Further, in order to solve the above-described problem, an electronic pen that reads the coded pattern indicating the position coordinates and calculates the coordinates of the movement path is displayed on the screen on which the coded pattern is formed based on the operation by the user. A display processing program executed by a computer that performs information processing for displaying a shape indicating a movement path of the electronic pen on the screen when moved, the computer displaying the coordinates of the calculated movement path A projection output means for outputting the shape of the movement path formed on the basis of the image to a projection means for projecting the image on the screen as an image, an area formed in a part of the image projected on the screen, and the movement A character entry area for processing the shape of the route as a character is based on the shape of the movement route in the character entry area. It functions as a rotation control means for rotating, and is constituted by the shape of the moving path of the reading means while the reading means is writing a character moving in the character entry area based on the user's operation. The computer is used as the rotation control means so as to calculate an inclination angle of the character string in the character entry area being entered and to rotate the character entry area based on the calculated inclination angle. Furthermore, it has the structure which makes it function.

  An information processing system according to the present invention can be configured by installing the above program in a computer device.

(15) Further, in order to solve the above problem, the electronic pen that reads the coded pattern indicating the position coordinates and calculates the coordinates of the movement path is light transmissive in which the coded pattern is formed based on the operation by the user . A display processing program executed by a computer that performs information processing for displaying a shape indicating a movement path of the electronic pen when moving on a sheet, the computer displaying the coordinates of the calculated movement path Display output means for outputting the shape of the movement path as an image from the back surface of the light transmissive sheet to the display means for visual display from the front surface of the light transmissive sheet, and displayed on the light transmissive sheet. A character entry area for processing the shape of the moving path as a character. The rotation control means for rotating based on the shape of the movement path in the reading section, and the reading means during the writing of the character moving in the character entry area based on the operation of the user. A character string composed of the shape of the movement path of the means, the inclination angle of the character string in the character entry area being entered is calculated, and the character entry area is rotated based on the calculated inclination angle Thus, the computer is further configured to function as the rotation control means.

  An information processing system according to the present invention can be configured by installing the above program in a computer device.

  In the present invention, when a reading means is moved by a user and a character is input into a character entry area, the movement path of the reading means is projected on a screen as an input character string, The inclination angle of the character string input by the user can be calculated, and the character entry area can be rotated based on the calculated inclination angle. Therefore, according to the present invention, it is possible to allow the user to smoothly input a character during character input in consideration of the user's situation during character input, such as a user's habit or an entry state depending on the user's standing position on the screen.

It is a system configuration figure of the information processing system in a 1st embodiment. It is a figure explaining the relationship between the arrangement | positioning of the dot in a dot pattern, and the value into which the dot is converted. It is a figure for demonstrating a dot pattern, (a) shows a dot pattern typically, (b) shows the example of the information corresponding to it. It is the schematic which shows the structure of an electronic pen, and is a block diagram which shows the function. It is a functional block diagram of a computer apparatus. It is a figure for demonstrating the 1st recognition process in 1st Embodiment. It is a figure (I) for explaining rotation control processing of a character entry field in a 1st embodiment, (a) is a figure for explaining the example of calculation in the inclination angle of a character string, and (b). It is an example of the character entry area | region rotated based on the calculated inclination | tilt angle. It is a figure (II) for demonstrating the rotation control process of a character entry area | region in 1st Embodiment, (a) is for demonstrating the example of calculation of the inclination angle in a character string when the 3rd character is entered. FIG. 4B and FIG. 4B are examples of the character entry area rotated based on the calculated tilt angle. It is a flowchart which shows the operation | movement of the display process including the rotation control process accompanying the 1st recognition process in the information processing system of 1st Embodiment. It is a flowchart which shows operation | movement of the 2nd recognition process in the information processing system of 1st Embodiment. It is a figure for demonstrating the rotation control process of the character entry area regarding the 1st modification of 1st Embodiment, Comprising: It is a figure for demonstrating the example of calculation in the inclination angle of a character string. It is a figure for demonstrating the rotation control process of the character entry area regarding the 2nd modification of 1st Embodiment, Comprising: It is a figure for demonstrating the example of calculation in the inclination angle of a character string. It is a figure for demonstrating the rotation control process of the character entry area regarding the 3rd modification of 1st Embodiment, Comprising: It is a figure for demonstrating the example of calculation in the inclination angle of a character string. It is a figure for demonstrating the rotation control process of the character entry area | region regarding the 4th modification of 1st Embodiment, Comprising: (a) is a calculation example of the inclination angle in a character string when the 3rd character is entered. The figure for explanation and (b) are examples of the character entry area rotated based on the calculated inclination angle. It is a figure for demonstrating the rotation control processing of the character entry area regarding the 5th modification of 1st Embodiment, Comprising: The figure for demonstrating the example of calculation of the inclination angle in a character string when the 3rd character is entered It is. It is a figure for demonstrating the rotation control process of a character entry area | region in 2nd Embodiment, Comprising: It is a figure for demonstrating the example of calculation in the inclination angle of a character string. It is a flowchart which shows the operation | movement of the display process containing the rotation control process in the information processing system of 2nd Embodiment. It is a system configuration figure showing the composition of the information processing system in a 3rd embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the embodiment described below, a predetermined image is projected on a screen having a predetermined coding pattern by a projector, and entry information entered by a user using a screen and an electronic pen is superimposed on the image. This is an embodiment when the information processing system and the display processing program of the present invention are applied to the information processing system to be executed.

<First Embodiment>
First, the information processing system 10 according to the first embodiment will be described with reference to FIGS.

[System configuration of information processing system]
First, the configuration of the information processing system 10 will be described. FIG. 1 is a system configuration diagram showing the configuration of the information processing system 10.

  As shown in FIG. 1, the information processing system 10 is an electronic pen 1 that is used by a user and generates entry information according to strokes (handwriting) of characters, symbols, figures, or the like written on the screen 4 by handwriting of the user. The computer apparatus 2 that receives the entry information generated by the electronic pen 1 and executes a predetermined process; and the image signal output from the computer apparatus 2 is received and displayed on the display means 26 of the computer apparatus 2. And a projector 3 that projects the same image as the image to be projected onto the screen 4 in synchronization.

  The electronic pen 1 constitutes reading means and calculation means of the present invention, and the computer apparatus 2 constitutes projection output means, rotation control means, character recognition processing means, display control means, and coordinate conversion means of the present invention. . Further, the projector 3 constitutes a projection unit of the present invention.

[screen]
Next, the screen 4 in the present embodiment will be described.

  The screen 4 includes a magnet plate 401 that is hard enough to apply a sufficient writing pressure to the electronic pen 1 and a sheet 402 on which a dot pattern (coded pattern) described later is formed on the entire surface by printing. In particular, the paper 402 is fixed to the front surface of the magnet plate 401 by pressing it from above the paper 402 with a magnet (not shown) or by adhering it with glue, adhesive or the like. Instead of the magnet plate 401, a white board, a partition board, a wall surface of a room, or the like may be used. Further, instead of the paper 402, a sheet on which a dot pattern is printed may be used, or the dot pattern may be directly formed on the screen 4.

[Dot pattern]
Next, an Anoto dot pattern (coded pattern) printed on the sheet 402 of the screen 4 in this embodiment will be described with reference to FIGS. FIG. 2 is a diagram for explaining the relationship between the dots of the dot pattern and the values to which the dots are converted. FIG. 3 is a diagram for explaining a dot pattern.

  As shown in FIG. 2, each dot of the dot pattern is associated with a predetermined value depending on its position. That is, each dot is associated with a value of 0 to 3 depending on which direction the top, bottom, left, or right is shifted from the reference position of the virtual grid (the intersection of the vertical and horizontal lines). Further, since the value of each dot can be further converted into the first bit value for the X coordinate and the second bit value for the Y coordinate, the position on the screen 4 is determined by the combination of the information thus associated. Coordinates are configured to be determined. The dot pattern is printed with ink containing carbon that absorbs infrared rays.

  FIG. 3A shows an arrangement of dot patterns at a certain position. As shown in FIG. 3 (a), 6 × 6 36 dots (hereinafter referred to as “6 × 6 dots”) within a range of about 2 mm in length and width are from which part on the screen 4 (paper 402). Even if 6 × 6 dots are taken, they are arranged so as to form a unique pattern. A dot pattern formed by these 36 dots holds relative position coordinates on the screen 4 (paper 402). Note that FIG. 3B is a diagram in which each dot shown in FIG. 3A is converted to a value associated with the regularity shown in FIG. 2 according to the shift direction from the reference position of the virtual lattice. is there. This conversion is performed by the electronic pen 1 that captures an image of a dot pattern.

[Electronic pen]
Next, the structure and operation of the electronic pen 1 in this embodiment will be described with reference to FIG. FIG. 4 is a schematic diagram showing the structure of the electronic pen 1 and a block diagram showing its function.

  The electronic pen 1 is used for tapping on the screen 4 on which an image is projected by the projector 3 (light tapping on the screen 4 by the pen tip 103) and entering characters, symbols, or figures. When the user taps the screen 4 using the electronic pen 1 or draws a character, a symbol, or a figure, the electronic pen 1 moves the movement path ( Hereinafter, the dot pattern printed on the paper 402 is read locally and continuously along “handwriting” and “stroke”. Then, the electronic pen 1 calculates the coordinates of the local position on the screen 4 and transmits the calculated position coordinate data to the computer device 2.

  Specifically, as shown in FIG. 4, the electronic pen 1 includes a processor 108 including a pen unit 104, an LED 105, a CMOS camera 106, a pressure sensor 107, a CPU, a ROM, A memory 109 such as a RAM, a real-time clock 110, a communication unit 111 including an antenna and the like, and a battery 112 are provided. The tip of the pen unit 104 is a pen point unit 103, and the pen point unit 103 is brought into contact with the sheet 402 of the screen 4 by the user when writing or tapping characters or the like. The handwriting written on the screen 4 by the electronic pen 1 is projected by the projector 3 by the processing of the computer device 2, so that the pen unit 104 may be not filled with ink, but can be erased. It may be filled with ink.

  The battery 112 is for supplying electric power to each member in the electronic pen 1. For example, the power of the electronic pen 1 itself is turned on / off by detaching a cap (not shown) of the electronic pen 1. It is configured. The real time clock 110 transmits time information indicating the current time (time stamp) and supplies the time information to the processor 108. The pressure sensor 107 detects the pressure applied from the pen tip portion 103 through the pen portion 104 when the user enters a character or the like on the screen 4 with the electronic pen 1 or taps it, that is, the writing pressure. Is transmitted to the processor 108.

  Based on the pen pressure data provided from the pressure sensor 107, the processor 108 performs pen-down of the electronic pen 1 (the stroke is written by touching the screen 4) and pen-up (from the touched state to the pen tip portion 103). And the switch of the LED 105 and the CMOS camera 106 is turned on / off. That is, when the user enters characters or the like on the screen 4 with the electronic pen 1, the pen pressure is applied to the pen tip portion 103, and the pressure sensor 107 detects a writing pressure higher than a predetermined value. The LED 105 and the CMOS camera 106 are operated. Then, when the user releases the electronic pen 1 from the screen 4, the pressure sensor 107 does not detect a writing pressure higher than a predetermined value. At this time, the processor 108 finishes entering one handwriting. And the operation of the LED 105 and the CMOS camera 106 is terminated.

  The LED 105 and the CMOS camera 106 are attached in the vicinity of the pen tip portion 103 of the electronic pen 1, and an opening 102 is formed in a portion of the housing 101 that faces the LED 105 and the CMOS camera 106. The LED 105 irradiates infrared rays toward the vicinity of the pen tip portion 103 on the screen 4. The region irradiated with infrared rays is configured to be slightly shifted from the position where the pen tip portion 103 contacts the screen 4. The CMOS camera 106 captures the above-described dot pattern in the area illuminated by the LED 105 and supplies image data of the dot pattern to the processor 108.

  As described above, since the dot pattern is printed with ink containing carbon that absorbs infrared rays, the infrared rays irradiated by the LED 105 are absorbed at the positions of the dots. As a result, the dot portion has a small amount of infrared reflection, and the portion other than the dot has a large amount of infrared reflection. For this reason, by taking a picture with the CMOS camera 106, by setting a threshold value based on the difference in the amount of reflected infrared rays, it is possible to distinguish the dot area containing carbon from the other areas. Note that the shooting area by the CMOS camera 106 is a range including a size of about 2 mm × about 2 mm as shown in FIG. 3A, and the shooting by the CMOS camera 106 is performed at regular intervals of about 50 to 100 times per second. Is called.

  The processor 108 operates while the characters or the like are written on the screen 4 by the user, that is, when the LED 105 and the CMOS camera 106 are switched on based on the writing pressure data supplied from the pressure sensor 107. For each dot pattern of each image data supplied by, X and Y coordinates (hereinafter also simply referred to as “position coordinates” or “coordinate data”) and dot pattern addresses on the screen 4 are individually calculated. That is, the processor 108 converts the image data of the dot pattern as shown in FIG. 3A supplied by the CMOS camera 106 into the data array shown in FIG. And the Y coordinate bit value, and the X, Y coordinate data and the dot pattern address are calculated from the data array by a predetermined calculation method.

  As described above, since the 6 × 6 dot pattern on the screen 4 does not overlap within the screen 4, when the user enters characters or the like with the electronic pen 1, Whether it corresponds to the position can be specified by coordinate calculation by the processor 108. Then, the processor 108 generates one coordinate attribute information by associating the current time (T) transmitted from the real-time clock 110, the pen pressure data (P), the X and Y coordinate data, and the dot pattern address.

  The memory 109 stores property information such as pen ID (electronic pen identification information) such as “pen01” for identifying the electronic pen 1, pen manufacturer number, and pen software version. Then, the communication unit 111 sequentially transmits each entry information in which the pen ID is associated with the coordinate attribute information or the like to the computer device 2. Transmission to the computer apparatus 2 by the communication unit 111 is immediately and sequentially performed by wireless transmission such as Bluetooth (registered trademark).

  Next, entry information transmitted from the electronic pen 1 to the computer device 2 will be described. When a user writes a stroke (handwriting) on the screen 4 using the electronic pen 1, first, the electronic pen 1 is brought into contact with the screen 4. Then, the pressure sensor 107 of the electronic pen 1 detects the writing pressure applied to the pen tip portion 103. When the processor 108 of the electronic pen 1 determines that the pressure sensor 107 has detected a writing pressure of a predetermined value or more, the pen down information PD indicating the contact of the electronic pen 1 with the paper 402 (screen 4) and the identification of the electronic pen 1 are detected. The entry information associated with the pen ID, which is information, is generated, and the communication unit 111 transmits the entry information to the computer apparatus 2.

  Further, the user draws a stroke by moving the pen tip portion 103 while maintaining the writing pressure after the pen tip portion 103 of the electronic pen 1 is brought into contact with the paper 402 (screen 4). The processor 108 continues to obtain the coordinate information (X, Y) and the dot pattern address obtained by the calculation, the pen pressure data (P) detected by the pressure sensor 107, and the time information (T) transmitted by the real time clock 110. The entry information that associates the coordinate attribute information and the pen ID with each other is sequentially generated according to the imaging cycle of the dot pattern by the CMOS camera 106, and the communication unit 111 sequentially transmits the entry information to the computer apparatus 2.

  When the user finishes drawing the stroke and moves the electronic pen 1 away from the paper 402 (screen 4), the pressure sensor 107 does not detect the writing pressure. Is no longer detected. Then, the processor 108 generates entry information that associates the pen-up information PU indicating that the electronic pen 1 is detached from the paper 402 (screen 4) with the pen ID that is the identification information of the electronic pen 1, and performs communication. The unit 111 is made to send the entry information to the computer apparatus 2. As described above, a set of coordinate attribute information generated by the electronic pen 1 from when the electronic pen 1 is determined to be pen-down to when the electronic pen 1 is determined to be pen-up, that is, when one stroke is entered by the user is referred to as stroke information. .

[Computer device]
Next, the computer apparatus 2 of this embodiment will be described with reference to FIGS. 1 and 5. FIG. 5 is a functional block diagram showing each function of the computer apparatus 2.

  The computer device 2 is, for example, a personal computer, and includes, as hardware, an antenna device capable of data communication with the electronic pen 1, a processor such as a CPU, a memory such as ROM and RAM, a display, a mouse, a keyboard, and the like. The In addition, as shown in FIG. 5, the computer device 2 is functionally composed of an input unit 21, a receiving unit 22, a processing unit 24, a storage unit 25, a display unit 26, and a transmitting unit 27, each of which includes a mouse and a keyboard. And performing predetermined processing based on the entry information received from the electronic pen 1.

  The receiving unit 22 includes an antenna, a receiving circuit, and the like, and sequentially receives each entry information transmitted from the electronic pen 1 and transmits it to the processing unit 24.

  The processing means 24 is constituted by a processor such as a CPU. The processing unit 24 executes a coordinate conversion process for displaying on the display unit 26 based on a predetermined calibration process and a result of the calibration process. As the calibration process, the processing unit 24 obtains a coordinate conversion function for converting the position coordinates of the dot pattern on the screen 4 into the position coordinates on the display area 201 in the display unit 26 of the computer device 2.

  Further, the processing means 24 constitutes a character or symbol entered in the character entry area 406 projected on the screen 4 by controlling the character entry area 206 on the display area 201 of the display means 26. The process of recognizing the coordinate data after the coordinate data of the stroke is transformed by the coordinate transformation function and various display control processes in the display means 26 are executed. In particular, the processing means 24 performs a rotation control process (hereinafter simply referred to as “rotation control process” or “rotation process”) of the character entry area 206 in conjunction with the character recognition process. Specifically, when the user enters a character in the character entry area 406 with the electronic pen 1, the processing unit 24 converts the coordinate data included in the entry information transmitted from the electronic pen 1 with a coordinate conversion function, While performing character recognition for each character, the inclination angle (θ) of the input character string is calculated, and the character entry area 206 is rotated based on the calculated inclination angle (θ).

  As shown in FIG. 5, the processing unit 24 performs a process as described above, as shown in FIG. 5, a calibration processing unit 241, a coordinate conversion unit 242, a character recognition processing unit 243, and a character entry region control unit 244. And a display control unit 245. The details of each part of the processing means 24 including the character recognition processing unit 243 and the character entry area control unit 244 will be described later.

  The storage means 25 is constituted by a hard disk or a memory such as a ROM or a RAM. The storage means 25 stores a program for executing calibration processing (hereinafter referred to as “calibration program”) and a program for executing rotation control processing for rotating the character entry area 206 (hereinafter referred to as “calibration program”). And a character recognition processing program (hereinafter referred to as a “character recognition program”) that is executed in conjunction with the character entry region rotation program.

  Specifically, with regard to the calibration program, information such as position coordinates and shape on the display area 201 of the calibration mark displayed on the display means 26, the coordinate area of the dot pattern printed on the paper 402 of the screen 4 Is stored in the storage means 25. As for the character entry area rotation program, the initial value of the coordinate area of the character entry area 206 displayed on the display means 26 and a mark (hereinafter also referred to as “definition mark”) 207 for demarcating and displaying the range. Each information necessary for the rotation control processing of the initial coordinates of the demarcation mark 207 and other character entry area 206 such as the shape is stored in the storage means 25 in advance. Further, with respect to the character recognition program, a stroke drawn in the character entry area 406 projected on the screen 4 based on the character entry area 206 displayed on the display means 26 and captured as an entry in the character entry area 406. Information on a pattern relating to each character or symbol (hereinafter referred to as “pattern information”) and other information necessary for character recognition processing used for collation of the shape pattern formed by the group is stored in the storage means 25 in advance.

  In addition to the above data, the storage means 25 stores each entry information for each stroke generated by the electronic pen 1 when the user enters characters, symbols, or figures on the screen 4. Each stroke information of the character recognition target stroke ST is stored. Further, the storage means 25 stores the coordinate conversion function obtained by the calibration process, the value of the coordinate area of the character entry area 206 rotated by the rotation control process, and the character input to the character entry area 206 by the coordinate conversion. Each program during or after calibration processing, character recognition processing (first recognition processing and second recognition processing), and rotation control processing, such as column inclination angle (= rotation angle of character entry area) (θ) Various types of data generated in accordance with user instructions in accordance with or during each process are stored.

  The display means 26 is constituted by a display such as a liquid crystal or an organic EL, and displays the contents instructed by the processing means 24. As shown in FIG. The recognized character display position mark 205, the character entry area 206, and the demarcation mark 207 are displayed by the processing of the display control unit 245 of the means 24.

  The transmission unit 27 transmits an image signal used for image display on the display unit 26 to the projector 3 according to an instruction from the processing unit 24. For this reason, the same image as the image displayed in the display area 201 of the computer apparatus 2 is synchronized with the screen 4 and is enlarged and projected onto the image projection area 403 by the projector 3. In the example shown in FIG. 1, the character entry areas 206 defined by the definition marks 207 displayed in the display area 201 are respectively displayed as character entry areas 406 defined by the corresponding definition marks 407 on the screen 4. Projected. The data transmission method to the projector 3 by the transmission unit 27 may be a wired method or a wireless method.

[Processing means]
Next, details of each part in the processing means 24 will be described with reference to FIGS. 6 to 8 together with FIG. FIG. 6 is a diagram for explaining the first recognition process. FIG. 7 is a diagram (I) for explaining the rotation control process of the character entry area. In particular, FIG. 7 (a) is a diagram for explaining an example of calculating the inclination angle (θ) of the character string. FIG. 7 and FIG. 7B are examples of the character entry area that is rotated based on the calculated inclination angle (θ). Further, FIG. 8 is a diagram (II) for explaining the rotation control processing of the character entry area. In particular, FIG. 8A shows an inclination angle (θ in the character string when the third character is entered. ) And FIG. 8B are examples of the character entry area rotated based on the calculated inclination angle (θ).

(1) Calibration processing unit The calibration processing unit 241 converts the position coordinates of the dot pattern on the screen 4 into the position coordinates on the display area 201 in the display unit 26 according to the calibration program stored in the storage unit 25. To obtain a coordinate transformation function. Specifically, the calibration processing unit 241 displays two or more calibration marks in the display area 201 of the display unit 26 and causes the projector 3 to project the calibration marks on the screen 4. When the receiving unit 22 receives coordinate data on the screen 4 generated by tapping the calibration mark projected on the screen 4 by the electronic pen 1, the calibration processing unit 241 uses the calibration mark as a calibration mark. The coordinate conversion function is calculated by acquiring the correspondence between the position coordinates on the display area 201 of the display means 26 and the position coordinates of the dot pattern on the screen 4. The calibration processing unit 241 stores the calculated coordinate conversion function in the storage unit 25.

(2) Coordinate conversion unit The coordinate conversion unit 242 uses the calibration processing unit 241 to calculate the X and Y coordinate data on the screen 4 included in each entry information for each stroke transmitted from the electronic pen 1 and received. Based on the conversion function, the X and Y coordinates of the stroke on the display area 201 of the display means 26 are converted. Then, the coordinate conversion unit 242 stores the converted X and Y coordinates (hereinafter also referred to as “calibration coordinates”) in the storage unit 25 for each stroke.

  At this time, when the coordinate conversion unit 242 recognizes that each calibration coordinate of one stroke is within the range of the coordinate area of the character entry area 206, the stroke (hereinafter referred to as the stroke entered in the coordinate area of the character entry area 406). This is referred to as “character recognition target stroke”.) ST is determined, and each entry information is stored in the storage unit 25 as stroke information of the character recognition target stroke ST. That is, for each character recognition target stroke ST, the coordinate conversion unit 242 uses the time information for specifying the stroke order required for the character recognition process, for each piece of entry information in which the X and Y coordinates are rewritten to the calibration coordinates. It is stored in the storage means 25 together with information such as T).

(3) Character Recognition Processing Unit The character recognition processing unit 243 performs character recognition processing on the stroke of the electronic pen 1 input in the character entry area 206 according to the character recognition processing program stored in the storage unit 25. In particular, the character recognition processing unit 243 is a recognition process for specifying a stroke group for each character according to the stroke order specified based on the time information included in the stroke information in conjunction with the rotation control process during the user's character entry. (Hereinafter referred to as “first recognition process”) and a recognition process (hereinafter referred to as “second recognition process”) for recognizing a character string that has been entered based on a user instruction. The character recognition processing unit 243 performs the character recognition target stroke (character entry area 406) in the same way as a program for performing character recognition such as OCR (Optical Character Reader) in both the first recognition process and the second recognition process. The stroke formed by the electronic pen 1 and input to the character entry area 206) The pattern formed from the coordinate-converted X and Y coordinate groups in ST and the pattern information of each character or symbol stored in advance in the storage means 25 Is used to identify a character or symbol.

  As the first recognition processing, the character recognition processing unit 243, for each character recognition target stroke ST specified by the coordinate conversion unit 242, each entry information of the stroke newly determined as the character recognition target stroke STt according to the stroke order; These strokes (STn to ST (t) are based on the stroke pattern shape pattern formed from each entry information of other strokes already determined as character recognition target strokes (STn to ST (t-1)). -1)), whether the shape pattern of the stroke group can be recognized as one character and whether it can be a part of the character including the newly determined character recognition target stroke STt, in other words, character recognition It is determined whether the target stroke STt is a stroke of the next character. When the character recognition processing unit 243 determines that the shape pattern of the stroke group (STn to ST (t-1)) can be recognized as one character and the character recognition target stroke STt is a stroke of the next character, The character entry area control unit 244 causes the rotation control process to be executed based on the stroke group (STn to ST (t−1)) that forms this one character. On the other hand, when the character recognition processing unit 243 determines that the character cannot be recognized as one character, the stroke group formed by the strokes (STn to STt) when the next character recognition target stroke ST (t + 1) is specified. It is determined whether or not the shape pattern can be recognized as one character. As described above, the character recognition processing unit 243 is configured to determine whether or not one character can be recognized for each character recognition target stroke ST, and to specify a stroke group that forms each character for each character.

  Specifically, as shown in FIG. 6, the character input by the user is “Tomorrow”, and the coordinate conversion unit 242 performs first to ninth character recognition target strokes (hereinafter, simply “ When the ST1 to ST9 are sequentially specified, the character recognition processing unit 243 determines that each stroke from the first stroke ST1 to the eighth stroke ST8 is a new character recognition target stroke STt. If it occurs, it is not recognized as one character based on the first stroke ST1 to the eighth stroke ST8. However, when the ninth stroke ST9 is subsequently determined as the new character recognition target stroke STt, the character recognition processing unit 243 determines that the ninth stroke ST9 is the stroke of the next character and determines the first stroke ST1 to ST1. The eighth stroke ST8 is recognized as one character “bright”. That is, first, when it is determined that the fourth stroke ST4 is a new character recognition target stroke STt, it can be recognized as a “day” character by the stroke group of the first stroke ST1 to the fourth stroke ST4, but the character is Whether it is “day” itself or a part of a different character such as “time” needs to be determined including the next fifth stroke ST5. Further, when it is determined that the eighth stroke ST8 is a new character recognition target stroke STt, it can be recognized as a “bright” character by the stroke group of the first stroke ST1 to the eighth stroke ST8. It is necessary to determine whether it is a part of a different character such as “Correspondence” or “Alliance”, including the next ninth stroke ST9. Therefore, the character recognition processing unit 243 uses the newly specified character recognition target stroke STt as a determination target in order to reliably specify a stroke group of one character, but does not include it in the stroke group specified as one character. 1 recognition processing is executed. Further, the character recognition processing unit 243 performs the stroke group from the nth stroke STn to the mth stroke STm (m> n) already used for character recognition every time one character is recognized for the character recognition processing for the second and subsequent characters. The first recognition process is executed based on the stroke group after the (m + 1) th stroke ST (m + 1) specified thereafter.

  As the second recognition process, the character recognition processing unit 243 performs a recognition start key or a coordinate input area (not shown) such as the computer apparatus 2 after the user has input characters or symbols in the character entry area 206. (Image projection area) When a tap operation in the recognition start area (not shown) in the 403 or the character entry area 406 of the electronic pen 1 is detected, it is determined that the user's entry has been completed, and is included in the character entry area 206 Recognize characters in a batch. In particular, when the user's instruction as described above is input, the character recognition processing unit 243 receives each calibration coordinate of the stroke (that is, the electronic pen 1) stored in the storage unit 25 as entry information in the character entry area 206. The coordinate data group of the stroke on the display screen of the display means 26 corresponding to the coordinate data of the stroke on the screen 4 is reversely rotated based on the rotation angle (θ) and the coordinate data of the rotation center (−θ The character is recognized based on the coordinate data after the coordinate conversion. That is, when it is determined that the character entry area 206 is rotated by the rotation control process (for example, when the rotation flag information is stored by the storage unit 25), the character recognition processing unit 243 2 As a recognition process, for each calibration coordinate group of each character recognition target stroke ST, the rotation angle stored with the coordinates of a point (base point B described later) in the character entry area 206 as the center of rotation is stored. The calibration coordinates of each stroke are normalized by reversely rotating by (θ) (by applying a rotation matrix of −θ), and character recognition processing is performed on the pattern formed by the normalized X and Y coordinate groups. Execute. Thereby, since the character or symbol written with inclination can be changed into the state written horizontally without inclination, the precision regarding identification of a character etc., ie, the recognition precision in character recognition, can be improved. In addition, when the second recognition process is executed, all strokes displayed in the character entry area 206 (ie, character recognition target strokes ST) are identified in order to recognize the character string to be entered next. All strokes stored in the storage means 25), reference point 220 and inclination angle (θ) data to be described later are deleted from the storage means 25. Further, the character recognition processing unit 243 stores the text data of the character string recognized as a result of the second recognition process in the storage unit 25.

(4) Character entry area control section The character entry area control section 244 is based on the initial value of the coordinate area of the character entry area 206 stored in the storage means 25 and information on the demarcation mark 207 for demarcating and displaying the range. As shown in FIG. 1, a demarcation mark 207 is displayed in the display area 201 of the display means 26. In particular, the character entry area control unit 244 displays a horizontally long rectangular character entry area 206 by using a demarcation mark 207 in the lower left corner of the display area 201 of the display means 26 as an initial setting. Then, as shown in FIG. 1, the character entry area control unit 244 transmits an image signal for projecting the same image as this image on the screen 4 to the projector 3 via the transmission unit 27.

  As a result, a character entry area 406 that can be specified by the demarcation mark 407 is displayed in the lower left corner of the screen 4. In FIG. 1, the character entry areas 206 and 406 are displayed in an outer frame, but the character entry areas 206 and 406 may be highlighted using yellow or other colors to prompt entry of entry information. The character entry areas 206 and 406 may be displayed by a ruled line (underline).

  In addition, the character entry area control unit 244, when a character is entered in the character entry area 406 on the screen 4 by the user and input to the character entry area 206, the character entry area control unit 244 is in the process of entering characters according to the character entry area rotation program. Each time a character is recognized, the inclination angle (θ) of the character string entered by the user is calculated, and the character entry area 206 is rotated based on the calculated inclination angle (θ). That is, stroke information related to characters entered in the character entry area 406 by the electronic pen 1 is generated, and coordinate conversion is performed by the coordinate conversion unit 242 based on the stroke information of the entered characters. When the stroke input to is identified as the character recognition target stroke ST, the character entry area control unit 244 is linked with the first recognition processing of the character recognition processing unit 243 and each time one character is recognized by the character recognition processing unit 243. In addition, the reference point 220 of the recognized character is calculated based on the stroke group forming one character. The character entry area control unit 244 further calculates the inclination angle (θ) of the character string based on the calculated reference point 220 of the character, and calculates the inclination angle (θ calculated based on the predetermined base point B). ) Is controlled so as to have the same angle (hereinafter referred to as “rotation angle”) (θ).

  Specifically, as shown in FIG. 7A, the character entry area control unit 244, when the first character W1 of the first character is recognized, is based on the stroke group that forms the recognized first character W1. Thus, the uppermost point 221a, the lowermost point 222a, the leftmost point 223a, and the rightmost point 224a on the coordinates of the display area 201 of the display means 26 after the coordinate conversion are recognized. Then, the character entry area control unit 244 obtains a reference point 220a serving as the center of the character stroke group based on the uppermost point 221a, the lowermost point 222a, the leftmost point 223a, and the rightmost point 224a. Similarly, when the second character W2 of the second character is recognized, the character entry area control unit 244 determines the highest point 221b and the lowest point based on the stroke group that forms the recognized second character W2. The point 222b, the leftmost point 223b, and the rightmost point 224b are recognized, and the reference point 220b from each of these points is obtained. Hereinafter, the reference point 220χ of the nth character Wn is referred to as the nth reference point 220χ. When the second reference point 220b is calculated, the character entry area control unit 244 determines the inclination α on the coordinates of the straight line L formed by the second reference point 220b and the already obtained first reference point 220a. Is calculated as the inclination angle (θ) of the character string. Further, as shown in FIG. 7B, the character entry area control unit 244 uses the upper left corner of the character entry area 206 as a base point B, and the rotation angle (θ (= α)) equal to the calculated inclination angle (θ (= α)). The character entry area 206 is rotated to have θ). Even if the user is entering characters, if the inclination angle (θ) is calculated, the character entry region 206 is controlled to rotate, so that the character entry region control unit 244 is as shown in FIG. Even if the user is inputting the third character W3, if the inclination angle (θ (= α)) is calculated, the character entry area 206 is controlled to rotate.

  If the calculated reference point 220 is one, that is, only the character “bright” is recognized as the first character W1 in FIG. 7A, and the calculated reference point is only 220a, the character Since the entry area control unit 244 cannot calculate the inclination angle (θ), the entry area control unit 244 does not calculate the inclination angle (θ) of the character string and does not perform the rotation control process of the character entry area 206. On the other hand, when a plurality of three or more characters are recognized and the inclination angle (θ) of one or more character strings has already been calculated, the character entry region control unit 244, that is, the character entry region 206 When the rotation has already started from a predetermined position (tilt angle (θ) = 0), the calculated tilt angle (θ) (hereinafter referred to as “first tilt angle (θ1)”) and a new one are newly added. Based on the calculated tilt angle (θ) (hereinafter referred to as “second tilt angle (θ2)”), a new tilt angle (θ) (hereinafter referred to as “third tilt angle (θ3)”). And the character entry area 206 is rotated based on the calculated third inclination angle (θ3). In particular, as shown in FIGS. 8A and 8B, the inclination α (that is, the first inclination angle (θ1 = α)) of the straight line L1 formed by the first character and the second character has already been calculated. If the third reference point 220c is obtained by executing the recognition of the third character, the inclination β on the coordinates of the straight line L2 formed by the first reference point 220a and the third reference point 220c is set to the second inclination. Calculated as the angle (θ2). Then, the character entry area control unit 244 is based on the first inclination angle (θ1 (= α)) (= rotation angle (θ)) and the second inclination angle (θ2 (= β)) calculated this time (formula 1) is performed to calculate a new inclination γ (that is, the third inclination angle (θ3)), and the upper left point of the character entry area 206 is set as a base point B as shown in FIG. The character entry area 206 that has already been rotated by the first tilt angle (θ1 (= α)) so as to have the same rotation angle (θ) as the calculated third tilt angle (θ3 (= γ)) is further provided. Rotate.

  Similarly, in the character recognition processing for the fourth and subsequent characters, the third tilt angle (θ3) is calculated based on the first tilt angle (θ1) and the second tilt angle (θ2), and the character entry area 206 is rotated. . For example, according to the example of FIG. 8, the inclination angle γ calculated by the calculation of (Equation 1) by the character entry area control unit 244 at the time of recognition of the third character is executed when the fourth character is recognized. In the rotation control process, the inclination of the straight line calculated based on the reference point (not shown) of the fourth character corresponds to the second inclination angle (θ2). Become.

(5) Display Control Unit The display control unit 245 displays a calibration mark in the display area 201 of the display unit 26 according to an instruction from the calibration processing unit 241 during the calibration process. Further, the display control unit 245 displays the character entry region 206 and the demarcation mark 207 according to an instruction from the character entry region control unit 244, and rotates and rotates the character entry region 206 by an operation using the electronic pen 1. It is possible to display the later state in the display area 201 of the display means 26 in real time. Further, the display control unit 245 causes the display area 201 of the display unit 26 to display a stroke based on the position coordinates (calibration coordinates) obtained by coordinate conversion of the coordinate data of the entry information transmitted from the electronic pen 1 by the coordinate conversion unit 242. .

  In particular, when the second recognition process is executed, the display control unit 245 erases the display of handwritten strokes such as characters, symbols, and other graphics displayed in the character entry area 206, and the second The text data of the character and the symbol specified by the character recognition in the recognition process is displayed at a place determined as a character input position. Specifically, when the computer device 2 is executing a program such as a text editor and the program window (the recognized character display position mark 205 shown in FIG. 1) is displayed in the display area 201 of the display means 26. The display control unit 245 displays text data such as a character specified by character recognition at a position where the character input cursor is located on the recognized character display position mark 205. At this time, the display control unit 245 temporarily displays the text data candidates after character recognition in the character entry area 206 or a candidate character display area (not shown), and in the confirmation key or coordinate input area 403 (not shown) such as the computer device 2. It may be configured to display at a predetermined character input position by detecting a tap operation in a fixed region (not shown) or a character entry region 406 of the electronic pen 1 or a candidate character display region (not shown).

  In addition, the display control unit 245 projects image signals for projecting the projector 3 in synchronization with the images displayed on the display area 201 in order to project and display various images on the screen 4. Output and send to the projector 3. The projector 3 projects an image similar to the image displayed on the display unit 26 on the image projection area 403 of the screen 4 based on the image signal received from the transmission unit 27 of the computer apparatus 2.

[Rotation control process with first recognition process]
Next, the operation of the rotation control process involving the first recognition process in the information processing system 10 will be described with reference to FIG. 9 together with various processes accompanying the reception of entry information. FIG. 9 is a flowchart showing a display process including a rotation control process with a first recognition process.

  In this operation processing, calibration processing is executed in advance by the calibration processing unit 241, a coordinate conversion function is obtained and stored in the storage unit 25, and coordinate conversion is performed by the coordinate conversion unit 242. Then, the character entry area control unit 244 refers to the initial value of the coordinate area of the character entry area 206 stored in the storage means 25, and as shown in FIG. 1, at the lower left of the display area 201 of the display means 26, A character entry area 206 defined by a rectangular definition mark 207 is displayed. In addition, since an image signal for indicating the same image as this image is transmitted from the transmission unit 27 to the projector 3, the projector 3 defines the image projection area 403 of the screen 4 with a demarcation mark 407. A character entry area 406 is projected and displayed.

  First, when one stroke is entered on the screen 4 by the user who is the writer, the electronic pen 1 sequentially transmits each entry information of the entered stroke to the computer device 2 (step S101). Specifically, when the pen is detected based on the pen pressure detected by the pressure sensor 107 and the start of the stroke is recognized by the user, the electronic pen 1 turns on the LED 105 and the CMOS camera 106, and the pressure sensor 107 pens up. Until the end of the stroke is recognized by the user, the dot pattern on the screen 4 is photographed at predetermined time intervals, and the pen ID, time information (T), and X, Y coordinates are taken for each photographed image. The entry information including the data is generated and sequentially transmitted to the computer apparatus 2.

  Further, when the computer device 2 starts receiving each entry information (stroke information) transmitted from the electronic pen 1, the processing means 24 sequentially stores the entry information in the storage means 25 for each pen ID (step S201). .

  Next, the coordinate conversion unit 242 sequentially converts the received coordinate data of each entry information into coordinates for displaying in the display area 201 of the display unit 26 based on the coordinate conversion function stored in the storage unit 25 ( Step S202). The coordinate conversion unit 242 sequentially stores the calibration coordinate data after coordinate conversion in the storage unit 25 in association with the pen ID for each stroke, and the coordinate coordinate data after coordinate conversion is based on the calibration coordinates. Drawing as a stroke and displaying it in the display area 201 of the display means 26 (step S203). In the process of step S203, the transmission unit 27 projects the same image as the image displayed in the display area 201 of the display unit 26, that is, an image including a drawn stroke on the screen 4. An image signal is transmitted to the projector 3.

  Next, the coordinate conversion unit 242 determines whether or not the coordinates (calibration coordinates) of the entry information coordinate-converted by the coordinate conversion unit 242 are included in the character entry area 206 for each received entry information. (Step S204). That is, the coordinate conversion unit 242 determines whether or not the stroke configured from the received entry information corresponds to the character recognition target stroke ST. In this case, since the character entry area 406 on the screen 4 corresponds to the character entry area 206 of the display area 201 of the display means 26, the user inputs a character or symbol into the character entry area 406 on the screen 4. It can be recognized from the coordinate data in each entry information of the stroke transmitted by that the character entry area 206 in the display area 201 includes this stroke by coordinate conversion by the coordinate conversion unit 242.

  In the process of step S204, when the coordinate conversion unit 242 determines that the stroke composed of the received entry information is included in the character entry area 206 based on each calibration coordinate, that is, the character When it is determined that the stroke is the recognition target stroke ST (step S204: Yes), the coordinate conversion unit 242 stores the character recognition target stroke ST in the storage unit 25, and the character recognition processing unit 243 performs one character of step S205. The process proceeds to a recognition process for specifying each stroke group, that is, a first recognition process. On the other hand, when the coordinate conversion unit 242 determines that the stroke formed from each piece of entry information received based on the calibration coordinates is not included in the character entry area 206 (step S204: No), this operation process End.

  If it is determined that the stroke composed of the received entry information is the character recognition target stroke ST (step S204: Yes), the character recognition processing unit 243 converts the stroke composed of the received entry information into a stroke. Based on the calibration coordinates, the first recognition process is executed (step S205). Specifically, the character recognition processing unit 243, when there are already specified other character recognition target strokes STn to ST (t-1), newly specified character recognition target stroke STt and other character recognition target strokes STt. Character recognition processing is comprehensively executed using the character recognition target strokes STn to ST (t-1).

  Next, the character recognition processing unit 243 determines whether or not the character recognition target stroke ST group can be recognized as a result of the first recognition processing in step S205 (step S206). When the character recognition processing unit 243 determines that one character can be recognized (step S206: Yes), the process proceeds to step S207 by the character entry area control unit 244, and the character recognition processing unit 243 determines that one character cannot be recognized. If it has been performed (step S206: No), the character recognition processing unit 243 ends the operation processing.

  When the character recognition processing unit 243 determines that one character can be recognized (step S206: Yes), the character entry area control unit 244 is recognized based on each calibration coordinate of the stroke group constituting one character recognized in step S206. One character reference point 220t is calculated and stored in the storage means 25 (step S207). Subsequently, the character entry area control unit 244 determines whether or not the calculated reference point 220t is the first reference point 220a of the first character W1 (step S208), and is not the first reference point 220a. If it is determined (step S208: No), the process proceeds to step S209. On the other hand, when the character entry area control unit 244 determines that the reference point 220t calculated in step S207 is the first reference point 220a (step S208: Yes), that is, the character recognized in step S205 is the same. In the case of the first character W1, the operation process is terminated without calculating the inclination angle (θ).

  When the character entry area control unit 244 determines that the reference point 220t calculated in step S207 is not the first reference point 220a (step S208: No), the character entry area control unit 244 determines that the calculated reference point Based on 220t, the inclination angle (θ) of the character string is calculated to rotate the character entry area 206 (step S209). In the example shown in FIGS. 7 and 8, the character entry area control unit 244 determines that the first already calculated first reference point 220t calculated in step 207 is the second reference point 220b of the second character W2. The inclination of the straight line L1 formed based on the first reference point 220a of the character W1 and the second reference point 220b calculated this time is calculated as the first inclination angle (θ1). Further, when the calculated reference point 220 is the third reference point 220c, the character entry area control unit 244 determines whether the calculated first reference point 220a and the newly calculated third reference point 220c are the same. The second inclination angle (θ2) is calculated based on the calculation result, and further, the calculation of (Equation 1) is performed based on the already calculated first inclination angle (θ1) and the newly calculated second inclination angle (θ2). The third inclination angle (θ3) is calculated as the inclination angle (θ) of the character string (step S209).

  Next, the character entry area control unit 244 displays the character entry area 206 in the display area 201 of the display means 26 so that the rotation angle (θ) is the same as the calculated tilt angle (θ). While rotating (step S209). At this time, since the projector 3 projects the same image as the display unit 26 of the computer apparatus 2 onto the image projection area 403 of the screen 4, the character entry area 406 projected on the screen 4 is also the character entry area 206. Rotate according to the rotation.

  Finally, the character entry area control unit 244 causes the storage unit 25 to store the tilt angle (θ) calculated in step S209 and the rotated coordinate area of the character entry area 206 after the rotation (step S210). The operation process is terminated.

[Second recognition process]
Next, the operation of the second recognition process of the information processing system 10 in the first embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing the operation of the second recognition process.

  First, when the computer device 2 recognizes a start instruction for character recognition processing based on a predetermined operation, such as a recognition start key (not shown) provided in the computer device 2 is recognized (step S301), the character recognition processing unit 243 For each calibration coordinate of each character recognition target stroke (stroke entered in the character entry area 406 by the electronic pen 1 and input into the character entry area 206), the rotation angle (θ) calculated in step S209 is calculated. The character recognition target stroke ST is normalized by rotating it in reverse (by applying a rotation matrix of −θ) (step S302). At this time, the character entry area 206 may be returned to the initial value, that is, the position of the initial coordinates stored in the storage means 25, or the position as it is may be held.

  Then, the character recognition processing unit 243 collates the pattern information recorded in the storage unit 25, and the pattern based on the X and Y coordinate groups in the normalized character entry area 206 (for each character recognition target stroke ST group). Recognition processing for recognizing one or more characters or symbols from the shape) is performed (step S303). Next, the character recognition processing unit 243 deletes the characters, symbols, and the like that have been character-recognized from the character entry area 206, and further deletes each data such as the character recognition target stroke ST used for character recognition from the storage means 25, The text data of the characters and symbols specified after the character recognition is stored in the storage means 25 and displayed at a position determined as a character input position in the display area 201 (step S304), and this processing is terminated.

[Effects of First Embodiment]
As described above, in the information processing system 10 according to the present embodiment, when a character is entered in the character entry area 406 on the screen 4 by the user using the electronic pen 1, a stroke (movement path of the electronic pen 1) is generated. The character string inputted to the character entry area 206 of the display means 26 of the computer device 2 is projected on the screen 4 and the character string inputted by the user every time one character is recognized while the character is being entered. The tilt angle (θ) can be calculated, and the character entry area 206 (406) can be rotated based on the calculated tilt angle (θ). Accordingly, the information processing system 10 may cause the user's habit or the user's screen such that the character string rises to the right (upward to the left) or the character string goes down to the right (downward to the left) during character entry. The character entry area can be rotated in accordance with the change of the user's situation in the character entry in consideration of the entry situation depending on the standing position with respect to the character, so that the user can smoothly enter the character on the screen 4 when inputting the character into the computer device 2. Lets you fill in letters.

  In addition, since the information processing system 10 according to the present embodiment can calculate the inclination angle (θ) of the character string for each character, the inclination angle of the character string ( θ) can be calculated. Therefore, the information processing system 10 can rotate the character entry area 206 (406) in an appropriate manner corresponding to the character string entered by the user using the electronic pen 1.

  Further, since the information processing system 10 of the present embodiment can calculate the inclination angle (θ) of the character string simply by comparing the reference points 220 of the respective characters, the inclination angle of the character string can be easily and reliably obtained. (Θ) can be calculated.

  In addition, the information processing system 10 according to the present embodiment, based on the rotation angle (θ) of the character entry area 206 (406), sets each stroke (movement path of the electronic pen 1) to the character entry area 206 (before the rotation) ( 406), the character string recognition process can be executed again according to a state in which no inclination occurs in each entered character. As a result, the information processing system 10 can improve the accuracy of character recognition.

  In addition, when the inclination angle (θ) of the character string has already been calculated, the information processing system 10 according to the present embodiment is newly calculated as one or more first inclination angles (θ1) that have already been calculated. A new third tilt angle (θ3) is calculated based on the second tilt angle (θ2), and the character entry area 206 (406) is rotated based on the calculated third tilt angle (θ3). Therefore, the inclination angle (θ) can be calculated in consideration of the already entered character string. Therefore, the information processing system 10 can reliably rotate the character entry area 206 (406) in accordance with the user entry situation.

  In the information processing system 10 according to the present embodiment, the coordinates on the screen 4 are converted into the coordinates of the display area 201 of the display unit 26 by the coordinate conversion unit 242 and then the character entry area 206 (406) is rotated. Even if different coordinate systems are used for the screen 4 and the display area 201 of the display means 26, the rotation control processing of the character entry area 406 (206) can be reliably executed.

[Modification]
Next, first to fifth modifications of the information processing apparatus according to the first embodiment will be described with reference to FIGS. Each of FIGS. 11 to 13 is a diagram for explaining the rotation control processing of the character entry area regarding the first to third modifications, and the inclination of the character string in each modification. It is a figure for demonstrating the example of calculation in an angle ((theta)). FIG. 14 is a diagram for explaining the rotation control processing of the character entry area according to the fourth modified example. In particular, FIG. 14A shows the inclination angle in the character string when the third character is entered. FIG. 14B and FIG. 14B for explaining an example of calculating (θ) are examples of the character entry area rotated based on the calculated inclination angle (θ). Further, FIG. 15 is a diagram for explaining the rotation control processing of the character entry area regarding the fifth modified example, and explains an example of calculating the inclination angle (θ) in the character string when the third character is entered. It is a figure for doing.

  In the information processing system 10 of the first modification, the character recognition processing unit 243 of the first embodiment calculates the reference point 220 of each character based on a plurality of coordinates obtained by the character recognition target stroke ST belonging to one character. Instead of calculating the reference point 220 for each character based on all the coordinates (sample points) S of each character recognition target stroke ST belonging to one character, the inclination angle (θ) of the character string is calculated. There is a feature. Other configurations are the same as those of the first embodiment.

  Specifically, in the information processing system 10 of the first modified example, the character entry area control unit 244, as shown in FIG. 11, all calibration coordinates (sample points) of each character recognition target stroke ST belonging to one character. Based on S, the average of the X coordinate and the Y coordinate (hereinafter referred to as “average coordinate”) is calculated, and the calculated average coordinates are used as the second reference point 220a of the first character W1 and the second reference point of the second character W2. Set as reference point 220b. Then, as in the first embodiment, the information processing system 10 calculates the inclination α on the coordinates of the straight line L formed by the first reference point 220a and the second reference point 220b as the inclination angle (θ). It is configured. Also, after the third character, the character entry area control unit 244 similarly uses the average coordinates calculated based on all the calibration coordinates (sample points) S of each character recognition target stroke ST belonging to one character as a reference point. 220, and the tilt angle (θ) is calculated based on the calculated reference point 220. With this configuration, the information processing system 10 according to the first modified example can calculate the inclination angle (θ) of the character string using the average coordinates of each character as the reference point 220. The tilt angle (θ) can be calculated.

  In the first modification, the average coordinate calculated for each character is used as the reference point 220. However, based on all the calibration coordinates (sample points) S of each character recognition target stroke ST belonging to one character. It is also possible to calculate coordinates (hereinafter referred to as “centroid coordinates”) that serve as the center of gravity of the character, and use the calculated center of gravity coordinates as the reference points 220 of each character. In calculating the barycentric coordinates, instead of the points calculated based on the calibration coordinates (sample points) S, a polygonal region surrounded by each character recognition target stroke ST belonging to one character is specified, and this specification is performed. Various calculation methods may be used such as obtaining the barycentric coordinates based on the polygonal area.

  In the information processing system 10 of the second modification, the character recognition processing unit 243 of the first embodiment calculates the reference point 220 of each character based on a plurality of coordinates obtained by the character recognition target stroke ST belonging to one character. Instead, the lowest point 222 of each character is used as each reference point 220. Other configurations are the same as those of the first embodiment.

  Specifically, in the information processing system 10 of the second modified example, as shown in FIG. 12, the character entry area control unit 244 sets the lowest point 222a of the first character W1 as the first reference point 220a. In addition, the lowest point 222b of the second character W2 is set as the second reference point 220b, and the coordinates of the straight line L formed by the first reference point 220a (= 222a) and the second reference point 220b (= 222b). The upper inclination α is calculated as the inclination angle (θ). Similarly, after the third character, the character entry area control unit 244 similarly sets the lowest point 222 as the reference point 220 for each character, and the inclination angle (θ) based on the set reference point 220. Is calculated. With this configuration, the information processing system 10 according to the second modification can calculate the inclination angle (θ) of the character string using the lowest point 222 on the coordinates of each character as the reference point 220. The inclination angle (θ) of the character string can be calculated easily and reliably.

  In the information processing system 10 of the third modification, the character recognition processing unit 243 of the first embodiment calculates the reference point 220 of each character based on a plurality of coordinates obtained by the character recognition target stroke ST belonging to one character. Instead, the uppermost point 221 of each character is used as each reference point 220. Other configurations are the same as those of the first embodiment.

  Specifically, in the information processing system 10 of the third modified example, as shown in FIG. 13, the character entry area control unit 244 sets the uppermost point 221a of the first character W1 as the first reference point 220a. The uppermost point 221b of the second character W2 is set as the second reference point 220b, and the coordinates of the straight line L formed by the first reference point 220a (= 221a) and the second reference point 220b (= 221b) are set. The inclination α is calculated as the inclination angle (θ). Similarly, after the third character, the character entry area control unit 244 similarly sets the uppermost point 221 as the reference point 220 for each character, and sets the inclination angle (θ) based on the set reference point 220. calculate. With this configuration, the information processing system 10 according to the third modified example can calculate the inclination angle (θ) of the character string using the uppermost point 221 on the coordinates of each character as the reference point 220. The inclination angle (θ) of this character string can be calculated easily and reliably.

  In the information processing system 10 of the fourth modification example, the character recognition processing unit 243 of the first embodiment replaces the point of calculating the inclination angle (θ) of the third and subsequent characters with (Equation 1), instead of the first inclination angle. The third tilt angle (θ3 (= γ)) after the third character is calculated based on (θ1 (= α)) and the second tilt angle (θ2 (= β)) and (Expression 2). There is. Other configurations are the same as those of the first embodiment.

  Specifically, in the information processing system 10 according to the fourth modified example, as shown in FIGS. 14A and 14B, the character entry area control unit 244 has the reference point 220a for the first character and the second character. And 220b, when the inclination α of the straight line L1 is already calculated as the first inclination angle (θ1), when the third character is recognized and the third reference point 220c is obtained, the second reference point 220c is obtained. The inclination β on the coordinates of the straight line L2 formed by the reference point 220b and the third reference point 220c is calculated as the second inclination angle (θ2). Then, the character entry area control unit 244 executes the calculation of (Equation 2) based on the calculated first inclination angle (θ1 (= α)) and second inclination angle (θ2 (= β)), and performs a new operation. 14 is calculated as a third inclination angle (θ3), and the calculated third inclination angle (θ3 (= γ) with the upper left point of the character entry area 206 as a base point B as shown in FIG. )) Is rotated so that the character entry area 206 that has already been rotated by the inclination angle (θ1 (= α) is rotated so that the rotation angle becomes the entire rotation angle. With this configuration, the information processing system 10 of the fourth modified example When the inclination angle (θ) has already been calculated, a new first angle is calculated based on the already calculated one or more first inclination angles (θ1) and the newly calculated second inclination angle (θ2). Three tilt angles (θ3) are calculated, and based on the calculated third tilt angle (θ3) Since the character entry area 206 is rotated, the inclination angle (θ) can be calculated in consideration of the already entered character string, and the rotation of the character entry area 206 (406) can be reliably performed in accordance with the user's entry situation. Can be rotated.

  Similarly, in the character recognition processing for the fourth and subsequent characters, the third tilt angle (θ3) is calculated based on the first tilt angle (θ1) and the second tilt angle (θ2), and the character entry area 206 is rotated. . For example, according to the example of FIG. 14, the inclination angle γ calculated by the calculation of (Equation 1) by the character entry area control unit 244 at the time of recognition of the third character is executed when the fourth character is recognized. In the rotation control process, the inclination of the straight line calculated based on the reference point (not shown) of the fourth character corresponds to the second inclination angle (θ2). Become.

  In the information processing system 10 according to the fifth modification, the character recognition processing unit 243 according to the first embodiment replaces the point of calculating the inclination angle (θ) of the third character and thereafter with (Equation 1) instead of the first reference point. The characteristic is that the inclination β of the straight line L formed by the reference point 220χ of the last nth character Wn recognized as 220a is calculated as the inclination angle (θ). Other configurations are the same as those of the first embodiment.

  Specifically, in the information processing system 10 of the fifth modified example, as shown in FIG. 15, the reference point 220e of the fifth character W5 is calculated based on each character recognition target stroke ST that forms the fifth character W5. Then, the character entry area control unit 424 calculates the inclination β of the straight line L formed between the calculated fifth reference point 220e and the reference point 220a of the first character W1 recognized first as the inclination angle (θ). And the upper left point of the character entry area 206 is set as the base point B, and the calculated inclination angle (θ (= β)) and the rotation angle (θ) as the entire angle have already been rotated. The character entry area 206 is rotated.

  In addition, in 1st Embodiment including each said modification, a dot pattern (coding pattern) does not need to be restricted to an Anoto system. In the first embodiment, the upper left corner of the character entry area 206 is used as the base point B. However, a predetermined point such as the lower right corner, upper right corner, lower left corner, or center point is used as the base point B of the character entry area 206. May be.

  In addition, the information processing system 10 according to the present embodiment can be configured to identify a character or symbol recognized based on the confirmation instruction of the user who is the writer, and is linked to an external application such as a word processor. It can also be used for entry.

Second Embodiment
Next, an information processing system according to the second embodiment will be described with reference to FIGS. 16 and 17.

  The information processing system 10 according to the second embodiment does not perform the first recognition process instead of performing the rotation control process such as calculating the tilt angle (θ) while performing the first recognition process in the first embodiment. Further, there is a feature in that the inclination angle (θ) is calculated based on the character recognition target stroke ST identified as being input in the character entry area 206. In the present embodiment, the same members and the same processes as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted. In the information processing system 10 of the second embodiment, the character recognition processing unit 243 executes only the second recognition process without executing the first recognition process.

[Character entry area control section]
Next, the rotation control process in the second embodiment will be described with reference to FIG. FIG. 16 is a diagram for explaining the rotation control process.

  When a character is entered in the character entry area 406 by the user and a stroke is entered in the character entry area 206 by transmission of entry information, the character entry area control unit 244 determines whether a character is being entered according to the character entry area rotation program. Each time the character recognition target stroke ST is specified, the coordinates of the center point c of the character recognition target stroke ST are calculated. Then, the character entry area control unit 244 calculates the inclination angle (θ) of the character string entered by the user based on the calculated coordinates of each center point c, and based on the calculated inclination angle (θ). The character entry area 206 is rotated. In particular, the character entry area control unit 244 executes a predetermined calculation such as a least square method based on the newly calculated center point ct and the already calculated center points c1 to c (t−1). Is calculated as the inclination angle (θ) of the character string and has the same rotation angle (θ) as the inclination angle (θ) calculated based on the base point B. Rotation control.

  Specifically, as shown in FIG. 16, the character “Tomorrow” is entered in the character entry area 406 on the screen 4 by the user using the electronic pen 1, and the coordinate data of the entry information transmitted from the electronic pen 1 is used. When the coordinate conversion unit 242 performs coordinate conversion and each stroke input in the character entry area 206 is specified as the character recognition target stroke ST from the first stroke ST1 to the twelfth stroke ST12, the specified first is specified. For each stroke from the stroke ST1 to the twelfth stroke ST12, the coordinates (X, Y coordinates) of each center point c from the first center point c1 to the twelfth center point c12 of each character recognition target stroke based on each entry information, respectively. ) Are sequentially calculated from the average of the coordinate values. That is, each time the coordinates of the center point ct are newly calculated, the character entry area control unit 244 is based on the newly calculated center point ct and the already calculated center points c1 to c (t−1). A predetermined calculation such as a least square method is executed, and the inclination α of one straight line L is calculated as the inclination angle (θ) of the character string. Then, based on the base point B, the character entry area control unit 244 controls the rotation of the character entry area 206 so as to have the same rotation angle (θ) as the calculated inclination angle (θ). For example, when the character entry area control unit 244 newly calculates the tenth center point c10, the first center point c1 to the ninth center point c9 already calculated and the newly calculated tenth center point c10 Then, a predetermined calculation such as a least square method is executed, and the inclination α of one straight line L is calculated as the inclination angle (θ) of the character string.

  Note that the character entry area control unit 244 determines the stroke direction and length for the first few strokes (for example, from the first stroke ST1 to the eighth stroke ST8) of the character string specified as the character recognition target stroke ST. Thus, the inclination α of the straight line L cannot be calculated accurately, and therefore the rotation control processing of the character entry area 206 is not performed without calculating the inclination angle (θ) of the character string. In other words, since characters are usually written from top to bottom, the first few strokes may be recognized as an erroneous downward slope, and in order to eliminate the associated misrecognition, For several strokes at the beginning of writing a character string, the rotation control processing of the character entry area 206 is not performed without calculating the inclination angle (θ) of the character string.

[Rotation control processing]
Next, the operation of the rotation control process in the information processing system 10 of the second embodiment will be described with reference to FIG. 17 together with various processes associated with the reception of entry information. FIG. 17 is a flowchart showing display processing including rotation control processing.

  In this operation processing, calibration processing is executed in advance by the calibration processing unit 241, a coordinate conversion function is obtained and stored in the storage unit 25, and coordinate conversion is performed by the coordinate conversion unit 242. Then, the character entry area control unit 244 refers to the initial value of the coordinate area of the character entry area 206 stored in the storage means 25, and in the lower left of the display area 201 of the display means 26, as in the first embodiment. A character entry area 206 defined by a rectangular demarcation mark 207 is displayed. In addition, since an image signal for indicating the same image as this image is transmitted from the transmission unit 27 to the projector 3, the projector 3 defines the image projection area 403 of the screen 4 with a demarcation mark 407. A character entry area 406 is projected and displayed.

  First, when one stroke is entered on the screen 4 by the user who is the writer, the electronic pen 1 sequentially transmits each entry information of the entered stroke to the computer device 2 (step S401). Specifically, when the pen is detected based on the pen pressure detected by the pressure sensor 107 and the start of the stroke is recognized by the user, the electronic pen 1 turns on the LED 105 and the CMOS camera 106, and the pressure sensor 107 pens up. Until the end of stroke is recognized by the user, the dot pattern on the screen 4 is photographed at predetermined time intervals, and the pen ID, time information (T), and X, Y coordinates are taken for each photographed image. The entry information including the data is generated and sequentially transmitted to the computer apparatus 2.

  When the computer apparatus 2 starts receiving each entry information (stroke information) transmitted from the electronic pen 1, the processing means 24 sequentially stores the entry information in the storage means 25 for each pen ID (step S501). .

  Next, the coordinate conversion unit 242 sequentially converts the received coordinate data of each entry information into coordinates for displaying in the display area 201 of the display unit 26 based on the coordinate conversion function stored in the storage unit 25 ( Step S502). Next, the coordinate conversion unit 242 sequentially stores the calibration coordinate data after the coordinate conversion in the storage unit 25 in association with the pen ID for each stroke, and draws the stroke based on the calibration coordinate to display the display unit 26. Are displayed in the display area 201 (step S503). In the process of step S503, the transmission unit 27 projects the same image as the image displayed in the display area 201 of the display unit 26, that is, an image including a drawn stroke on the screen 4. An image signal is transmitted to the projector 3.

  Next, the coordinate conversion unit 242 determines whether or not the coordinates (calibration coordinates) of the entry information coordinate-converted by the coordinate conversion unit 242 are included in the character entry area 206 for each received entry information. (Step S504). That is, the coordinate conversion unit 242 determines whether or not the stroke configured from the received entry information corresponds to the character recognition target stroke ST. In this case, since the character entry area 406 on the screen 4 corresponds to the character entry area 206 of the display area 201 of the display means 26, the user inputs a character or symbol into the character entry area 406 on the screen 4. It can be recognized from the coordinate data in each entry information of the stroke transmitted by that the character entry area 206 in the display area 201 includes this stroke by coordinate conversion by the coordinate conversion unit 242.

  In the process of step S504, when the coordinate conversion unit 242 determines that the stroke formed from each received entry information is included in the character entry area 206 based on the calibration coordinates, that is, character recognition. If the target stroke ST is identified (step S504: Yes), the coordinate conversion unit 242 stores the character recognition target stroke ST in the storage unit 25, and the character entry region control unit 244 performs the process of step S505. Migrate to On the other hand, if the coordinate conversion unit 242 determines that the stroke formed from each piece of entry information received based on the calibration coordinates is not included in the character entry area 206, that is, it is not the character recognition target stroke ST. If specified (step S504: No), the operation process is terminated.

  When it is determined in the process of step S504 that the stroke formed from the received entry information is the character recognition target stroke ST (step S504: Yes), the character entry area control unit 244 determines the character recognition target stroke. The coordinates of the center point c of ST are calculated and stored in the storage means 25 (step S505). Next, the character entry area control unit 244 determines whether or not a predetermined number or more of character recognition target strokes ST has already been input to the character recognition area 206, in other words, a predetermined number or more of character recognition target strokes ST is stored in the storage unit 25. It is determined whether it is stored (step S506).

  In the process of step S506, if the character entry area control unit 244 determines that a predetermined number or more of character recognition target strokes ST have already been input to the character recognition area 206 (step S506: Yes), the process proceeds to step S506. Transition to processing. On the other hand, when the character entry area control unit 244 determines that a predetermined number or more of character recognition target strokes ST have not yet been input to the character recognition area 206 (step S506: No), the operation processing is terminated.

  When the process proceeds to step S507, the character entry area control unit 244 determines the coordinates of the center point ct calculated as the center point c of the stroke newly specified as the character recognition target stroke STt, and the character recognition target already calculated. Based on the coordinates of the center points cn to c (t-1) of the strokes STn to ST (t-1), a predetermined calculation such as a least square method is executed to calculate an inclination angle (θ) The entry area 206 is rotated (step S507). That is, the character entry area control unit 244 displays the character entry area 206 and the demarcation mark 207 in the display area 201 of the display means 26 so that the rotation angle (θ) is the same as the calculated inclination angle (θ). Rotate while rotating (step S507). At this time, since the projector 3 projects the same image as the display unit 26 of the computer device 2 onto the image projection area 403 of the screen 4, the character entry area 406 projected on the screen 4 is also the character entry area 206. Rotate to rotate.

  Finally, the character entry area control unit 244 stores the tilt angle (θ) calculated in step S507 and the coordinate area of the rotated character entry area 206 in the storage means 25 (step S508), and performs this operation process. Terminate.

[Effects of Second Embodiment]
As described above, the information processing system 10 according to the present embodiment, as in the first embodiment, when a character is entered in the character entry area 406 on the screen 4 by the user using the electronic pen 1, the stroke ( The movement path of the electronic pen 1) is projected onto the screen 4 as a character string input to the character entry area 206 of the display means 26 of the computer device 2, and each time a stroke is specified while the character is being entered. The inclination angle (θ) of the character string input by the user can be calculated, and the character entry area 206 (406) can be rotated based on the calculated inclination angle (θ). Therefore, the information processing system 10 takes into account the user's situation during character entry, such as the entry situation depending on the user's habit or the user's standing position on the screen, or in accordance with the change of the user situation in the character entry. 206 (406) can be rotated, so that the user can smoothly enter characters on the screen 4 when inputting characters to the computer apparatus 2.

  In addition, the information processing system 10 according to the present embodiment can calculate the inclination angle (θ) of the character string for each stroke, and accurately grasp the stroke entered by the user, and the inclination angle (θ) of the character string. Therefore, the character entry area 206 (406) can be rotated in an appropriate manner corresponding to the character string entered by the user.

  Further, since the information processing system 10 of the present embodiment can calculate the inclination angle (θ) of the character string based only on the center point c of each stroke, the inclination angle of the character string can be easily and reliably obtained. (Θ) can be calculated.

  In addition, the information processing system 10 according to the present embodiment does not use the result of the recognition process by the character recognition processing unit 243 for the calculation of the inclination angle (θ) of the character string by the character entry region control unit 244. It is not necessary to link the calculation of the tilt angle (θ) and character recognition, and each process can be easily constructed.

  In addition, the information processing system 10 according to the present embodiment, based on the rotation angle (θ) of the character entry area 206 (406), sets each stroke (movement path of the electronic pen 1) to the character entry area 206 (before the rotation) ( 406) and the character string recognition process can be executed in a state where no inclination is generated in each written character, so that the accuracy of character recognition can be improved efficiently. Can do.

  Further, the information processing system 10 of the present embodiment rotates the character entry area 406 (206) after the coordinates on the screen 4 are converted into the coordinates on the display area 201 of the display means 26 by the coordinate conversion unit 242. Therefore, even if different coordinate systems are used for the screen 4 and the display area 201 of the display means 26, the rotation control process for the character entry area 406 (206) can be reliably executed.

[Modification]
Note that the information processing system 10 of the present embodiment may not be limited to the Anoto method for the dot pattern (coded pattern), as in the first embodiment. In the present embodiment, the upper left corner of the character entry area 206 is used as the base point B. However, a predetermined point such as the lower right corner, upper right corner, lower left corner, or center point is used as the base point B of the character entry area 206. Also good.

  Further, the information processing system 10 of the present embodiment can be configured to identify a character or symbol recognized based on the confirmation instruction of the user who is the writer, as in the first embodiment, and can be configured to be external to a word processor or the like. It can also be used to fill in this application in conjunction with the application.

  In addition, the information processing system 10 of the present embodiment uses the center point c of each stroke when calculating the inclination angle (θ) of the character string, but the uppermost point, the lowermost point, the rightmost point, or the leftmost point. A sample point, such as a point, can be used.

<Third Embodiment>
Next, an information processing system according to the third embodiment will be described with reference to FIG. FIG. 18 is a system configuration diagram showing the configuration of the information processing system 100 in the third embodiment.

  The information processing system 100 according to the third embodiment replaces the computer device 2 in place of displaying an image on the screen 4 using the projector 3 and entering a stroke using the screen 4 in the first and second embodiments. It is characterized in that strokes are entered using a transparent or translucent light-transmitting sheet disposed above. In the present embodiment, the same members and the same processes as those in the first and second embodiments are denoted by the same reference numerals, and the description thereof is omitted.

  As shown in FIG. 18, the information processing system 100 according to the third embodiment includes a light-transmitting sheet 9 on which a dot pattern (coded pattern) that can be read by the electronic pen 1 is printed. The light transmissive sheet 9 is pasted on the front surface of the display area 201 of the computer apparatus 2, and an image displayed on the display area 201 is displayed visually from the front surface of the light transmissive sheet 9. The dot pattern is printed on the light transmissive sheet 9 with ink having a characteristic of selectively reflecting infrared rays.

  In the electronic pen 1, the processor 108 is determined by providing a threshold value from the image data captured by the CMOS camera 106 so as to recognize a partial region with strong infrared reflection as a dot. The threshold value is set so as to be able to distinguish between the case where only the infrared rays transmitted through the light-transmitting sheet 9 from the display and the case where the infrared rays irradiated from the LEDs 105 of the electronic pen 1 are reflected by the dots are included. Is done.

  In order to make it easy to recognize dots from the image data captured by the CMOS camera 106 of the electronic pen 1, an infrared transmission filter is provided in the opening of the electronic pen 1 so that only the infrared light is irradiated to the CMOS camera 106. You may provide near 102. Further, the dot pattern of this embodiment is not limited to the Anoto method. In addition, the information processing system 100 of the third embodiment can be appropriately applied to the modification of the first embodiment and the examples of the second embodiment.

  The information processing system of the present invention can be used for pen input operations in external applications such as school classes, business meetings, lectures, presentations, and word processors.

c ... Center point ST ... Character recognition target stroke W ... Character 1 ... Electronic pen 2 ... Computer device 3 ... Projector 4 ... Screen 9 ... Light transmissive sheet 21 ... Input means 22 ... Receiving means 24 ... Processing means 25 ... Storage means 26 ... Display means 27 ... Transmission means 103 ... Pen tip section 106 ... CMOS camera 107 ... Pressure sensor 108 ... Processor 109 ... Memory 111 ... Communication unit 201 ... Display area 206 ... Character entry area 207 ... Delimitation mark 220 ... Reference point 221 ... Top Point 222 ... Bottom point 223 ... Leftmost point 224 ... Rightmost point 241 ... Calibration processing unit 242 ... Coordinate conversion unit 243 ... Character recognition processing unit 244 ... Character entry area control unit 245 ... Display control unit 401 ... Magnet plate ( Whiteboard)
402 ... Paper 403 ... Coordinate input area (image projection area)
406 ... Character entry area 407 ... Delimitation mark 10, 100 ... Information processing system

Claims (15)

Reading means for reading the coded pattern along a movement path when moving on a screen on which a coded pattern indicating position coordinates is formed based on an operation by a user;
Calculating means for calculating the coordinates of the movement path based on the data of the coded pattern read by the reading means;
A projection output means for outputting the shape of the movement path formed based on the calculated coordinates of the movement path to a projection means for projecting the shape on the screen as an image;
A character entry area for processing the shape of the movement path as a character, which is an area formed in a part of an image projected on the screen, based on the shape of the movement path in the character entry area Rotation control means for rotating;
With
The rotation control means is a character string composed of the shape of the movement path of the reading means while the reading means is writing a character moving in the character entry area based on the operation of the user. An information processing system, wherein an inclination angle of a character string in the character entry area being entered is calculated, and the character entry area is rotated based on the calculated inclination angle. A reading unit that reads the coded pattern along a movement path when moving on a light-transmitting sheet on which a coded pattern indicating position coordinates is formed based on an operation by a user;
Calculating means for calculating the coordinates of the movement path based on the data of the coded pattern read by the reading means;
The shape of the movement path formed based on the calculated coordinates of the movement path is output as an image from the back surface of the light transmissive sheet to a display unit that is visibly displayed from the front surface of the light transmissive sheet. Display output means;
A character entry area for processing the shape of the movement path as a character, which is an area formed in a part of an image displayed on the light transmissive sheet, and the shape of the movement path in the character entry area Rotation control means for rotating based on
With
The rotation control means is a character string composed of the shape of the movement path of the reading means while the reading means is writing a character moving in the character entry area based on the operation of the user. An information processing system, wherein an inclination angle of a character string in the character entry area being entered is calculated, and the character entry area is rotated based on the calculated inclination angle. The information processing system according to claim 1 or 2,
The rotation control means calculates an inclination angle of the character string at every predetermined timing while the character is being entered, and rotates the character entry area every time the inclination angle is calculated. Information processing system. The information processing system according to claim 3,
Further comprising character recognition processing means for performing character recognition based on the shape of the movement path in the character entry area;
Each time the character recognition processing unit recognizes one character as the predetermined timing, the rotation control unit calculates an inclination angle of the character string based on a moving path group used when the character is recognized as the one character. An information processing system characterized by this. The information processing system according to claim 4,
The rotation control means obtains a reference point determined based on a movement path group of the recognized character every time one character is recognized by the character recognition processing means, and based on each of the obtained reference points An information processing system that calculates an inclination angle of a character string. The information processing system according to claim 4 or 5,
In the case where the character entry area is rotated, the character recognition processing means corresponds to the character entry area before the rotation of the moving path constituting the character string based on the inclination angle of the character entry area. An information processing system, wherein the character recognition of the character string is executed again after being converted to a moving route. The information processing system according to claim 3,
The rotation control means determines the inclination angle of the character string based on the coordinate group of the movement path included in each stroke each time one stroke that is a component of the movement path is specified as the predetermined timing. An information processing system for calculating and rotating the character entry area. The information processing system according to claim 7,
The rotation control unit obtains a reference point determined based on a coordinate group of the movement path included in each stroke each time the one stroke is specified, and the character string based on the obtained reference point An information processing system characterized by calculating an inclination angle of. The information processing system according to claim 7 or 8,
An information processing system, further comprising character recognition processing means for executing character recognition based on a shape of the movement path in the character entry area. Oite to the information processing system according to claim 9,
When the character recognition processing means is rotating the character entry area, the movement path corresponding to the character entry area before rotating the movement path of the reading means based on the rotation angle of the character entry area. An information processing system, wherein the character recognition is performed after conversion into a character string. The information processing system according to any one of claims 3 to 10,
When the inclination angle of the character string has already been calculated, the rotation control means is based on the already calculated one or more first inclination angles and the newly calculated second inclination angle. An information processing system that calculates a new third inclination angle and rotates the character entry area based on the calculated third inclination angle. The information processing system according to claim 1,
Display control means for performing display control of display means for displaying the same image as the image projected on the screen;
Coordinate conversion means for converting the coordinates calculated by the calculation means into coordinates on a display screen of the display means;
Further comprising
The information processing system, wherein the rotation control means rotates the character entry area based on the coordinates converted by the coordinate conversion means. The information processing system according to claim 2,
A coordinate conversion means for converting the coordinates calculated by the calculation means into coordinates on a display screen of the display means;
The information processing system, wherein the rotation control means rotates the character entry area based on the coordinates converted by the coordinate conversion means. When the electronic pen that reads the coding pattern indicating the position coordinates and calculates the coordinates of the movement path moves on the screen on which the coding pattern is formed based on an operation by the user, the movement path of the electronic pen is changed. A display processing program executed by a computer that performs information processing for displaying a shape to be displayed on the screen.
Projection output means for outputting the shape of the movement path formed based on the calculated coordinates of the movement path to the projection means for projecting the image on the screen as an image;
A character entry area for processing the shape of the movement path as a character, which is an area formed in a part of an image projected on the screen, based on the shape of the movement path in the character entry area Rotation control means for rotating,
And function as
A character string composed of the shape of the movement path of the reading means while the reading means is writing the character moving in the character entry area based on the user's operation, and the character being entered A display processing program for calculating an inclination angle of a character string in an entry area and further causing the computer to function as the rotation control means so as to rotate the character entry area based on the calculated inclination angle. . When the electronic pen that reads the coding pattern indicating the position coordinates and calculates the coordinates of the movement path moves on the light-transmitting sheet on which the coding pattern is formed based on the operation by the user, the electronic pen A display processing program executed by a computer that performs information processing for displaying a shape indicating a movement path ,
The computer,
Display output means for outputting the shape of the movement path based on the calculated movement path as an image from the back surface of the light transmissive sheet to the display means for visual display from the front surface of the light transmissive sheet,
A character entry area for processing the shape of the movement path as a character, which is an area formed in a part of an image displayed on the light transmissive sheet, and the shape of the movement path in the character entry area Rotation control means for rotating based on
And function as
A character string composed of the shape of the movement path of the reading means while the reading means is writing the character moving in the character entry area based on the user's operation, and the character being entered A display processing program for calculating an inclination angle of a character string in an entry area and further causing the computer to function as the rotation control means so as to rotate the character entry area based on the calculated inclination angle. .

Images