KR20030011271A - A method and system for form recognition and digitized image processing - Google Patents

Abstract

Provided are an automatic identification method and system for the pre-printed mark (45) based on the position of the mark (230) manually formed on the form (45). The method includes receiving ink image data 412; Identifying the presence and location of the mark 230 to determine any misalignment between the received ink data and the stored image of the form; And shifting the ink data to correct misalignment. The system includes a preprinted form 45 that includes a digitizer 35 and one or more preprinted indicators unique to the form indicative of where the user enters one or more identification marks 220, 230. Digitizer 35 includes one or more digital images of marks formed on the form and transmits the image data to a computer processor coupled to the database. The processor 25 identifies the presence and location of one or more identification marks formed by the user and identifies the form based on the location of the one or more marks.

Description

A METHOD AND SYSTEM FOR FORM RECOGNITION AND DIGITIZED IMAGE PROCESSING

Systems are known that automatically identify a form located on a digitizer device (the term "form" used herein refers to an image printed on a page instead of an actual page; that is, when there are two pages printed with the same image). For example, US Pat. No. 5,869,789 (Reid-Green) describes a page recognition system that detects pre-coded page numbers on paper when placing paper on a digitizer system. Doing. The system uses a built-in scanner to detect the precoded page identifier on the back side of the page. US Pat. No. 5,555,101 (Larson et al.) Describes an automated page recognition system for a form received by the facsimile, which uses a preprinted pattern to identify the form.

U.S. Patent No. 5,629,499 (Flickinger et al.) Discloses an electronic clipboard and describes two methods of recognizing a form used thereon. In the first method, a toggle switch is used to toggle the various form identifiers displayed on the clipboard, and in the second method, the barcode reader reads a barcode previously printed on the form. The toggle display requires the clipboard to include a microprocessor that must be reprogrammed each time a new set of forms is used.

Digital notepads are known which record the notation formed in a paper form placed on paper. For example, A.T. Cross Company has marketed CrossPad (35; see FIG. 1), a portable digital notepad that digitizes and records pen movements using wireless transmission. CrossPad sends the recorded pen movements to the personal computer for processing and analysis. However, CrossPad itself cannot identify preprinted forms. The information recorded by CrossPad and transmitted to the computer only includes pen strokes entered and recorded by the user on the form. Preprinted information on the form cannot be detected by CrossPad. CrossPad has no form ID display that can be toggled by the user, nor does it have a barcode reader.

Because devices such as CrossPad only detect user-entered information, these devices are generally not available in systems that identify forms based on preprinted information. If there are multiple forms in the digitizer, even systems such as those described in Reid-Green cannot detect form identification information. For example, Reid-Green only identifies the foam at the bottom of the stack. Flickinger et al. The device described in Figure 2 can read multiple pages, but requires an additional (expensive) form reading mechanism not found in CrossPad-type devices. By using such a barcode reader or other form reading mechanism in the digitizer, the cost and weight of the digitizer is increased.

Another disadvantage of the CrossPad type device is that the spatial relationship of the input marks written on the paper form is not the same as the spatial relationship of the corresponding digital marks on the underlying "digital form." This offset (“digital drift”) may be several millimeters.

Another disadvantage of CrossPad type devices is the fact that when using the form, the information preprinted on the form is not part of the digital ink file that is uploaded. In order to selectively use a digital ink file after uploading, it is necessary to add the contextual information image in the paper form to the digital ink image. The two images must be correctly aligned so that the image of the original form can be reproduced with the information recorded on it.

Thus, there is a need for an automated form recognition system that can recognize forms for use in CrossPad and similar devices without relying on information preprinted on the form.

There is also a need for a system that eliminates the need for integrating unnecessary LCD displays (with programmable microprocessors), toggle switches or other input devices into the CrossPad-type electronic clipboard.

There is also a need for a method and system for correcting digital drift for CrossPad type devices and other digitizers. There is a need for a method of combining and aligning a digital ink image of handwritten information on a form with an underlying form image.

FIELD OF THE INVENTION The present invention relates generally to data entry using digitizer devices, and more particularly to systems and methods for identifying forms located on digitizers and for determining and correcting digital image offsets to digitizer pads. will be.

1 shows a configuration of an embodiment of a preferred digitizer system.

2 and 3 show a data collection form for use in a preferred embodiment of the present invention.

4 is a flowchart showing the operation of software for processing digital data.

5 and 6 are flow charts illustrating the operation of the form recognition software.

Fig. 7 shows the layout of the zones used in the preferred embodiment of the present invention.

8 shows a graphical user interface for use in a preferred embodiment of the present invention.

9 shows the main form alignment steps used in the preferred embodiment.

10 shows coordinate definitions used in the preferred embodiment.

Fig. 11 shows a location-ink-bubble-center process of a preferred embodiment.

12 illustrates a recognized-alined-image process of the preferred embodiment.

Fig. 13 shows a merge-aligned-ink-with-reference-image process of the alignment ink of the preferred embodiment.

14 shows the form identification processing of the preferred embodiment.

15 shows a form identifier template of a preferred embodiment.

16 shows a digitizer pad having a three-sided paper guide.

The present invention includes the steps of identifying the presence and position of a mark (manually) formed on a preprinted form; And identifying the preprinted form by comparing the position of the mark with a corresponding position of the form and the identification mark listed in the database. In addition, the present invention is a digitizer; A preprinted form including one or more preprinted indicators unique to the form indicating where the user should enter one or more identification marks; And receiving one or more digital images of the marks formed on the form, identifying the presence and position of one or more identification marks formed by the user, and corresponding positions of the forms and identification marks listed in the database for the positions of the one or more marks. A pre-printed form identification system, characterized in that it has a computer processor coupled with the database, for comparison. The present invention also provides software for receiving and storing data indicating positions of marks manually formed on a preprinted form, and comparing the positions of the marks with corresponding positions of the forms and identification marks listed in the database. Automatic identification computer software of a preprinted form, characterized in that it comprises software for identifying the form.

The invention also includes a method of correcting digital drift and mapping a digital ink file onto a digital form. Digitizer pads (such as CrossPads or similar devices) typically experience a certain amount of digital drift-the spatial difference between the relative position of the recording on the paper form and the corresponding position of the corresponding digital mark on the underlying digital form. The amount and direction of the digital drift is constant for each pad (and any position on a given form), but generally varies with the pad.

The software of the preferred embodiment determines the amount and direction of the digital drift based on known landmarks on a given form. Preferably, this landmark is a form identification mark as described below. In a preferred embodiment, the form identifier is circular. The digital drift is determined by comparing the known position of the form identifier circle center on the paper form with the center of the corresponding digital mark on the digital form below and determining the change in the horizontal and vertical directions. Once digital drift is determined, it is applied as a calibration factor for all digital markings on the form.

Preferably, this correction factor is used to enable alignment of the handwritten digital ink image with the image of the underlying foam. This alignment is achieved by first identifying the form in which the digital ink data is recorded. Then, the preloaded image of the identified form is retrieved from the database. Calibration factors are used to align the ink data with the digital form. Accurate alignment of digital forms and digital inks allows for the reproduction of paper forms that appear when filling in forms. In addition, correct alignment allows composite images to be processed by optical mark recognition (OMR), optical character recognition (OCR), and image snippet storage into a database.

The present invention also includes a method and system for enabling the identification of forms that did not have a form identification mark previously printed above. In a preferred embodiment of this aspect of the invention, a metal or plastic guide having three millimeter holes located at the positions of the various selected guides is attached to one side of the digitizer pad. Preferably, the guide is installed at a predetermined position relative to the customer. By fixing the guide to the pad surface at a predetermined position, the user can form a mark in the hole, where the position of the hole corresponds to a position that the preprinted mark can have. In other words, the mark has the same position as the mark that has filled the preprinted form identification circle. This makes it possible to use those forms in the rest of the system disclosed by the customer with preprinted forms that are not printed with the form identification mark.

In a preferred embodiment, the present system and method includes the digitizer system shown in FIG. 1 having a digitizer 35 such as CrossPad, a pre-printed form 45, a personal computer 25, and associated software. Use together. For convenience of reference, the term "CrossPad" refers to any digitizer pad that can utilize the present invention. Those skilled in the art will appreciate that the system and method can be used with any digitizer system that can place the form in a digitizer field. In addition, although the term “form” is used herein to refer to a form printed on paper, those skilled in the art can recognize that the disclosed systems and methods may be the same and used in forms embodied in other media. . For example, the disclosed invention is a plastic laminate or "electronic paper" such as "Gyricon" developed by Xerox Corportion, Palo Alto, California, or similar products developed by E Ink Corporation, Massachusetts, Cambridge. electric paper) ".

The general operation of the digitizer system shown in FIG. 1 is as follows. Place the preprinted data collection form 45 on the CrossPad digitizer 35. Using a special pen required by CrossPad 35, the user marks on form 45. The mark formed on the form 45 is recorded by the Crosspad 35 as a digital image. The user connects the CrossPad 35 to the computer 25, and the CrossPad 35 transmits the stored image to the computer 25. In another embodiment, CrossPad is connected to a converter that converts Crosspad data into an Internet-compatible format, and transmits the converted CrossPad data to a remotely located computer via the internet by the converter. In one embodiment, the Crosspad image stored on computer 25 is then processed by software stored on the computer. Forms and software used in the preferred embodiment are described below.

2 shows a first data collection form 45 for use in the preferred embodiment. The left margin area 210 of the form is reserved for foam identification. The "Start Here" instruction 225 reminds the user to fill the indicator 230 for the form identifier (the circle points to an indicator and the mark created by filling the circle is the form identifier). For each different form, form identifier indicator 230 is located in a different zone in area 210.

Forms used in the preferred embodiment are printed so that they can be used in one or more data-entry systems. For example, the cross-hair image 215 at the bottom right of the form and the crosshair image 220 at the bottom right of the form are used to align the image of the form when scanning with a light scanner.

3 shows a second data collection form for use in the preferred embodiment. The foam in Fig. 3 is shown without reference numerals in order to more clearly show the shape of the foam used in the preferred embodiment. The form identifier indicator on the form of FIG. 3 is located in a different zone in area 210 than the form identifier indicator on the form.

Preferably, whenever a user starts recording on the form, the user fills in the form identifier indicator. However, to reduce the error, the first user wants to fill a plurality of indicators before handing the CrossPad to the second user. For example, a first user can use a template that includes indicators for all forms. When using Forms 1, 3, and 6, the first user places the template on CrossPad, fills the indicator for Form 1, presses the CrossPad's Next Page button, fills in the indicator for Form 3, and next Page indicator. Click again to fill in the indicators for Form 6. The first user then presses the Back Page button twice and delivers the CrossPad to the second user with the forms 1, 3, and 6 clipped on. Thereafter, the second user can fill in Forms 1, 3, and 6 without having to fill in the unique identifier indicator (when moving from form to form, the second user must still press the Next and Back page buttons).

4 is a flowchart showing the operation of the software used in the preferred embodiment. When the user of the Crosspad 35 fills in the form identifier indicator 230, the image and location of the form identifier is stored in CrossPad memory, along with the image and location data for any other information the user has written on the form. Although any suitable format may be used to store the data, CrossPad stores the information as "ink data", an IBM proprietary data format. In general, this data is referred to herein as ink data.

In step 410, the user plugs in the CrossPad to a serial connection cable (although infrared, Ethernet, or other connection device may be used) attached to the personal computer, and then presses the button of the CrossPad to transfer the collected ink data to the personal computer. Upload. In another embodiment, ink data is first delivered to a converter, which transmits this data to a computer located remotely, preferably via the Internet. The remotely located computer may be a personal computer, but the computer is preferably a server-class computer (eg, an internet server) because it must receive data from a plurality of sources.

In step 412, the ink delivery program stored in the personal computer receives the delivered ink data, and in step 415 stores the data as an ink data file on the hard drive of the computer. In a preferred embodiment, the IBM Ink Manager Transfer Program supplied with Crosspad performs this step and creates a file known as a "notebook."

After uploading the ink data file, the ink delivery program starts an ink converter program (here called TMDInkExec) in step 417. In step 420, TMDInkExec reads the uploaded ink data file and instructs the ink conversion module included in the IBM-supplied library, IBM Electric Ink SDK, to copy each ink page to a non-exclusive image file (e.g., For example, group 4 TIFF, but other formats can be used). TMDInkExec points to the appropriate ink conversion modules according to the instructions in the document contained in the IBM Electro Ink SDK. In another embodiment, the conversion from notebook format to non-proprietary image format in step 420 can also be performed at the remote computer; For example, the ink data can be delivered to a remote computer using e-mail or a file transfer protocol, and the remote computer (or computers) then examines a number of receive queues (e.g., email boxes). In step 420, the receiving batch of the received ink data is processed.

Upon completion of step 420, in step 422 the TMDInkExec program notifies the second program TMDInkMonitor to further process the TIFF data. In step 424, the TMDInkMonitor program optionally assigns a target ID to a page batch of ink data files (e.g., if the page is a filled form for a particular patient, the patient's ID can be entered). Or display the graphical interface for confirmation. When using a scanner instead of CrossPad, attach the target ID label to the form, recognize it, and assign the recognized number to the system. If you do not assign a target ID (and not recorded by CrossPad), the TMDInkMonitor program assigns a default target ID (preferably the ID of the previous target, incremented by 1).

The batch upload ID is assigned to the corresponding batch of TIFF image file in step 426, and in step 428 TIFF data is recorded in the database under the batch upload ID and the destination ID. The TIFF image file is then copied from the database and processed by the recognition engine to convert the image into recognition data using OMR (optical mark reading or optical mark recognition) at step 430. The recognition engine is software that provides a documented set of services to other software modules or programs that are involved in converting (recognizing) the service image data into computer readable text. Although the Caere Developer's Kit 2000 Recognition Engine, sold by Caere Corporation of Los Gatos, Calofornia, is used in preferred embodiments, a variety of recognition engines (eg, Mitek Systems, Inc., San Diego, Calif., And Minnesota Minneapolis) are used. National Computer Systems, Inc., commercially available, and those skilled in the art can use them in preferred embodiments.

In step 432, the TMDInkMonitor instructs the form ID determination module to sequentially drive the form ID corresponding to the recognition data. 5 and 6 illustrate the steps performed by the form ID determination module.

The form ID determination module uses an OMR recognition module and an OMR filling method. The recognition module is software that conforms to the specifications of the recognition engine and converts different types of image data into computer readable text. The fill method specifies the type of image data expected to occur in the zone. Some fill methods are associated with one recognition module; Some are supported by one or more recognition modules. Some examples of filling methods are multi-font machine printing, OMR, hand print, bar code, dot matrix printing, OCRA, and Braille.

A zone is an area in an image up to the full page size that contains features that interest the user. In a preferred embodiment, the zone is rectangular, although one skilled in the art will appreciate that other shaped zones may be used. Image data covered by zones is handled and processed separately. Common zone types may be graphical (unrecognized), text, or barcodes or OMRs. In order to be able to recognize the image data in the zone areas, each zone is given a property based on the expected information to be included in the zone. Basic zone properties are x-y coordinates, size, recognition module, filter, language dictionary, and user dictionary. A language dictionary is a file that contains the dictionary elements of a language and the rules for generating words from them. This file can be used to mark non-dictionary words of recognized text or to convert non-dictionary words into appropriate vocabulary. Some recognition modules refer to language dictionaries to help determine their recognition. The user dictionary contains a list of regular expressions that define words, strings, and character patterns, and supplements the contents of the language dictionary file. One or more user dictionaries may be associated with a particular zone. The regular expression in the user dictionary defines an acceptable pattern of characters by specifying a character class, range or individual character in which one or more character positions in the zone are effective to improve recognition accuracy. Regular expressions are also called marks or patterns. In general, regular expressions are generally used in some predictable form-like situations.

Form Identification Zone Template (or FIZT) is used to identify forms from images. Zone templates are a set of zone and page features (e.g., corresponding to a form to be recognized) that are specified and stored on media such as disk files or database records for later retrieval and processing. FIZT consists of a series of zones laid out in a grid pattern (see FIG. 7). You can use filters at either the zone level or the form level to improve recognition results. A filter is a specification for removing those that a particular category of characters are considered possible valid results. For example, a filter can contain numbers (digits only), uppercase letters, lowercase letters, and punctuation marks.

In another preferred embodiment, because the present invention uses OMR techniques, non-standard symbols (such as N original characters similar to symbol ©) are used to represent the relevant information of the form dnl. For example, such relevant information includes cut-and-paste areas, information that needs to be corrected, and references to other data files.

Zones for form recognition are arranged left to right and top to bottom, as indicated by the numbers in FIG. 7. Each zone corresponds to the form number of the set to which the form belongs. Zones are laid out with spaces between them to minimize the possibility of false form identification caused by the recorder marking out of the circle or deflecting when scanning the form. Forms in the set are assigned a zone corresponding to the form number, and prefilled unfilled dots in the assigned zones on the form page (see Figures 2 and 3). Each zone, and therefore each dot, is assigned a number. The table ("dotmap") maps dot numbers to form numbers in the form set. For example, the table for dot mapping is:

The "dotmap" is stored for later retrieval in storage media such as database tables or disk files.

Referring to FIG. 5, in step 510, the form ID determination module retrieves data recognized from a mark sense zone. In step 520, a search is performed on the first one filled dot and the corresponding zone, and in step 530, the dot number is determined from the zone definition. In step 540, the corresponding form ID number is retrieved from the dot mapping table based on the dot number (for example, in the above table, the dot number '1' corresponds to the form ID number '00300001-01'). In step 545, if the form ID number is not determined, in step 550 an ink-data-based image of the page is displayed to the user, and in step 555 the user immediately enters the appropriate form ID number. do. In step 560, the user enters a form ID number. While the preferred embodiment uses numbers to identify the form, one of ordinary skill in the art appreciates that symbols other than alphanumeric characters may also be used. The processing shown in FIG. 5 is shown in more detail in FIG.

Referring to Figure 6, after the form ID number is determined by the system or input by the user, step 645 is performed. In step 645, the data is further recognized based on the form ID number (e.g., using optical character recognition, or handprint recognition, also known as Intelligent Character Recognition (ICR)); In step 650, the ID number of the form on which the image is recorded and the recognized data are stored as an unedited file in the database. The recognized data is organized according to the category of the form and is optionally displayed on the user's audit screen. The data is displayed in a format such as a table for convenient inspection. In step 660, the user optionally inspects, verifies, and corrects each recognized data element on the inspection screen, as appropriate. In a preferred embodiment, a graphical user interface (GUI) 810 (see FIG. 8) allows the user to view data in a "contextual orientation": for example, if some data is not recognized, the user Has an image of a portion of the paper form displayed on the screen and an image of the ink data for that page to provide the user with an on-screen image of the portion of the written-on form. On top of the image on the paper form. This allows the user to inspect and verify data elements without a written form and also to see all the recorded images on which the paper form is written. In turn, this allows the operator to perform inspection and verification at a site remote from the site where the paper form is recorded. Also, for example, if the data is a medical report, a contextually-oriented on-screen view, recorded by hand on the form, may indicate that the user is not only the patient's personality but also the recorder when the report is received. To refresh your mood.

In step 665, the checked data is stored in a database.

9 schematically illustrates a preferred form alignment process. In step 910, the associated body and the form identification processing described in FIGS. 5 and 6 are performed. In step 915, if the form identification step 910 does not succeed, in step 920, the form alignment process ends, and the form is marked as "unknown".

If the foam identification step 910 succeeds, then in step 925, a location ink bubble center process (see FIGS. 10 and 11, and related text) is performed in the ink image bubble. In this step, the coordinates (XInkCenter, YInkCenter) of the ink image bubble are calculated.

Subsequent to step 925, the offset (XOffset, YOffset) between the reference bubble and the ink image bubble is XOffset = XInkCenter-XMast; Calculate YOffset = YInkCenter-YMast. XMast and YMast are denoted by X0 and Y0 in FIG.

In step 940, ink alignment is performed; Shift the image by applying the XOffset and YOffset values to the ink image.

In step 950, the aligned ink image is recognized (see FIG. 12 and related text). Finally, in step 960, the recognized alignment ink image is combined with the reference image (stored image of the identified form).

FIG. 10 shows the coordinates and zone definitions used in the preferred Ink Bubble Center Localization process, and FIG. 11 shows the processing steps.

10 shows a coordinate system with y coordinates increasing downwards. The reference search zone 1020 may be determined by a form and a form identification zone template. Since the foam has been identified, a bubble ink circle 1030 centered at (Xink, Yink) is found within a particular reference bubble zone 1010. The reference bubble zone 1010 allows the reference bubble to have a center X0, Y0 which is also the center of the reference bubble ink zone 1010. The reference search zone 1020 is defined by extending the bubble ink zone 1010 with the distance Xexpand horizontally in each direction (left and right) and the distance Yexpand vertically in each direction (up and down).

Accordingly, the reference search zone 1020 has the edges of coordinates (X1, Y1) (upper left corner), (X2, Y1) (upper right corner), (X2, Y2) (lower right corner), and (X1, Y2) is a rectangle having (lower left corner).

11 shows the ink bubble positioning process. In step 1105, the variable is initialized. BubbleZoneLeft is the x coordinate of the left side of the bubble zone 1010. BubbleZoneRight is the x coordinate of the right side of the bubble zone 1010. BubbleZoneTop is the y coordinate of the top of the bubble zone 1010. BubbleZoneBottom is the y coordinate of the bottom of the bubble zone 1010. Set the variable YScan to Y1.

Initially set the variables YTop, YBottom, XLeft, and XRight to Y2, Y1, X2, and X1, respectively. At the end of the ink bubble center localization process, the variable YTop contains the y coordinate value of the bubble ink 1030, YBottom contains the y coordinate value of the bottom of the bubble ink 1030, and XLeft contains the bubble ink 1030. And the x coordinate value of the left side of the XRight contains the x coordinate value of the bubble ink 1030. If the process is certain, even if the bubble ink is not a complete disc, the process determines the center of the bubble ink 1030.

In step 1110, the variable XScan is set to X1. In step 115, it is checked whether the black pixel is in (XScan, YScan), which is initially at (X1, Y1), i.e., in the upper left corner of the reference search zone 1020. If the answer is no, then at step 1160 increase the variable XScan. If the increment is not greater than X2 when checking the value in step 1165 (ie, the scan does not reach the right side of the reference search zone 1020), repeat step 1115. In step 1165, the increment in step 1160 of the variable XScan causes XScan to have a value greater than X2, and in step 1170 the variable Xscan is increased. If the value of YScan is greater than Y2 when the value of YScan is checked at step 1175, at step 1180, XInkCenter and YInKCenter are calculated. In step 1175, if the value of YScan is not greater than Y2, step 1110 is repeated. The software scans from left to right and top to bottom for a reference search zone 1020.

If the response is YES in step 1115, then in step 1120 it is checked whether the current value of the variable YScan is less than the current value of YTop. If YES, in step 1125, YTop is set equal to YScan. Thus, the top of the bubble ink 1030 is found and its y coordinate is determined as the current value of YScan. In step 1130, it follows step 1120 if the response in step 1120 is "no", and in step 1125 if the response in step 1120 is "yes".

In step 1130, it is checked whether the present value of XScan is smaller than the present value of XLeft. If the answer is yes, then at step 1140 set XLeft equal to XScan. If the answer is yes in step 1130, step 1145 follows step 1140, and if the response is no in step 1130, then step 1130.

In step 1145, it is checked if XScan is greater than XRight. If the answer is yes, then at step 1150 set XRight and XScan equally. If the answer to step 1145 is "no", step 1155 follows step 1145 and if the response to step 1145 is "yes", then step 1150 follows. In step 1155, YBottom is set equal to YScan. As described above, step 1160 follows step 1155. As the processing continues, the YTop value decreases until it reflects the y coordinate value of the top pixel of the bubble ink 1030; The YBottom value increases until it reflects the y coordinate value of the lowest pixel of the bubble ink 1030; The XLeft value decreases until it reflects the x coordinate value of the leftmost pixel of the bubble ink 1030; The XRight value is increased until it reflects the x coordinate value of the rightmost pixel in bubble ink 1030. Thus, when finally reaching step 1180, the calculation of XInkCenter (= (XLeft + XRight) / 2) and YInkCenter (= (YTop + YBottom) / 2 yields a bubble when the bubble ink 1030 forms a perfect disk. Accurately determine the center position of the ink 1030, and in all cases reasonably estimate the center of the bubble ink 1030. The present method of determining the center of the bubble ink 1030 is merely a preferred embodiment in the appended claims. It will be appreciated by those skilled in the art that there are various known methods for determining the center of an irregularly shaped two-dimensional object that can be substituted for the method described above without departing from the scope of the disclosed invention. .

9, in step 930, the offset between the ink bubble 1030 and the reference bubble (center (XMast, YMast) is calculated using the equations XOffset = YInkCenter-XMast and YOffset = YInkCenter-YMast. And the value of YOffset are used to shift the ink image, for example, if XOffset is positive and YOffset is negative, the ink image shifts left by the distance | XOffset | and shifts downward by the distance | YOffset |.

Preferably, the distance is measured by the number of pixels. Thus, in a preferred embodiment, the ink image pixels move horizontally by | XOffset | pixels and vertically by | YOffset | pixels. The following C ++ code illustrates a preferred method of performing ink image shift.

The ink pixels are offset by combining the loaded ink with a white page "null" image generated programmatically. A bitwise add algorithm is used to combine the pixels, which sets the pixel result up to 225 (white). The following code does this.

Once the ink image is shifted (aligned), OMR and OCR processing is performed-in step 950 to recognize the aligned image.

12 shows a preferred ink image recognition process. In step 1210, load the field definition file for the form. In step 1220, the ink image is loaded into the memory. In step 1230, the field definition file and the image are sent to the recognition engine for recognition, and in step 1240, the recognized results are stored in a database.

Referring back to FIG. 9, step 960 aligns the aligned recognition ink image with the reference image of the identified (blank) form. This treatment 13 is shown. In step 1310, the reference image of the form is loaded into memory. In step 1320, the ink image is loaded into the memory.

In step 1330, the ink image is combined with the reference image. The bitwise OR algorithm is used programmatically to combine the ink pixels, thereby combining the ink pixels with the reference image pixels. The following code does this.

Finally, in step 1340, the combined image is stored on a disk or other storage medium.

14 shows the form identification processing of the preferred embodiment. Part of this treatment is shown in FIG. 5. In step 1410, the form set form ID zone definition file is loaded into memory. In step 1420, the ink image is loaded into the memory. In step 1425, recognition is instructed and the loaded zone definition and ink image are used. In step 1430, the variable form ID is set equal to one. In step 1435, it is checked whether the recognition result character in the form ID is equal to one. If so, the form is identified and processing ends at step 1450. Otherwise, perform step 1440, where the form ID is incremented. In step 1445, it is checked whether the form ID is greater than the number of forms in the form set. If so, the process ends at step 1455 without identifying the form. Otherwise, processing returns to step 1435.

In another embodiment of the present invention, a preprinted form that does not have a printed form indicator is used. To enable the use of such foams, a template, preferably metal or plastic, is attached to the digitizer pad. The template functions as a stencil (see FIG. 15) and the user fills the holes in the guide; Each hole corresponds to a different form. In this way, the above-described form identification processing is used for a form having no preprinted form indicator.

In a preferred embodiment, a configuration as shown in FIG. 15 is used. Preferably, the template is a plastic overlay 1510 that assembles onto a side paper guide (see FIG. 16 showing the three paper guides 1610 on the left) above the digitizer pad (preferably CrossPad or a derivative thereof). Preferably, template 1510 has three slots 1530 that allow assembly to three raised paper guides on the digitizer pad. Those skilled in the art will appreciate that other installation methods may be substituted for other configurations of digitizer pads. For example, if there is only one raised paper guide above the digitizer pad and to the right of the pad, the template 1510 has only one slot located along its right side.

Referring again to the preferred embodiment shown in FIG. 15, the three slots 1530 are approximately (± 2 mm) 48 mm apart, 1 mm wide, 5 mm from the left edge of the template 1510, and approximately (± 1 m) 48 mm Length. The top of the top slot 1530 is preferably 15 mm from the top of the template 1510 and the bottom of the bottom slot 1530 is preferably 15 mm from the bottom edge of the template 1510.

The template 1310 itself is approximately 2 mm thick, 273 mm long, and 27 mm wide. Preferably, each foam supporter bubble 1520 is 2.5 mm in diameter and the preferred minimum distance between the bubbles 1520 is 4 mm.

Although the described embodiments are not able to fully achieve the object of the present invention, these embodiments are merely illustrative and do not limit the present invention. For example, CrossPad can be easily modified to allow users to skip between pages (eg, page 14 directly to page 3) without repeatedly pressing the page up and page down buttons.

Claims (57)

As an automatic recognition method of the preprinted form, (a) receiving and storing data indicating the position of a manually formed mark on a preprinted form; And (b) identifying the form based on the position of the mark. The method of claim 1, And the data received and stored in step (a) is digital image data received and stored in a computer storage medium by the computer. The method of claim 2, Step (b) comprises determining a form identification zone comprising the position of the mark and determining a form corresponding to the form identification zone. The method of claim 3, wherein Step (b) comprises storing a table that matches form identification zones with form IDs. The method of claim 4, wherein Step (b) is Reformatting the received digital image data into a format to be processed by the recognition engine; Converting the reformatted digital image data into recognition data by processing the reformatted data by the recognition engine; Retrieving recognition data for one or more form identification zones; Searching the retrieved recognition data for a zone containing a mark; And And identifying the form by matching the zone to a form ID. The method of claim 5, And displaying digital image data on a computer so that the user can manually identify the form on which the image was recorded. The method of claim 6, And displaying the recognition data on a user computer monitor in a format convenient for examination. The method of claim 7, wherein Displaying digital image data superimposed on the image of the identified form to provide an on-screen image of the form and the image recorded thereon. The method of claim 8, Examining the recognition data by comparing the recognition data with the displayed digital image data superimposed on an image of the identified form. As the identification system of the preprinted form, a) digitizer; b) a preprinted form comprising one or more preprinted indicators unique to the form indicating a location at which the user enters one or more identification marks; And c) i) receive one or more digital images of the marks formed on the form, ii) identify the presence and location of one or more identification marks formed by a user, iii) a computer processor coupled with a database for identifying a form based on the location of the one or more marks. As automatic identification computer software of preprinted form, (a) software for receiving and storing data indicative of the position of a manually formed mark on a preprinted form; And (b) software for identifying the form based on the position of the mark. The method of claim 11, And software for storing a table that matches form identification zones with form IDs. The method of claim 11, And the received and stored data is digital image data stored on a computer storage medium. The method of claim 13, Further comprising software for management of the digital image data for a subsequent reference. The method of claim 14, The identifying software includes software for determining a form identification zone that includes the location of the mark, and software for determining a form corresponding to the form identification zone. The method of claim 15, Software to identify Reformatting the received digital image data into a format processed by the recognition engine; Converting the reformatted digital image data into recognition data by processing the reformatted image data by the recognition engine; Retrieve recognition data for one or more form identification zones; Search the retrieved recognition data for a zone containing a mark; And And software for identifying the form by matching the zone to a form ID. The method of claim 16, And software for displaying digital image data on a computer monitor so that a user can manually identify the form on which the image was recorded. The method of claim 17, And software for displaying the recognized data on a user computer monitor in a convenient format for inspection. The method of claim 18, And software for displaying digital image data superimposed on the image of the identified form to provide an on-screen image of the form and the image recorded thereon. The method of claim 19, And identifying software for examining the recognition data by comparing the recognition data with the displayed digital image data superimposed on an image of the identified form. The method of claim 20, Further comprising software for recognizing relevant information on the form based on non-standard symbols recorded on the form. As a digital image data processing method, (a) receiving digital image data; (b) determining a position of a first mark included in the digital image data; (c) determining the distance and direction from the position of the first mark to the reference position; And (d) performing a shift of the digital image data based on the distance and direction. The method of claim 22, And said digital image data corresponds to a mark formed on a form located on a digitizer pad. The method of claim 23, And the form is identified by the position of the zone in which the identification mark is formed. The method of claim 24, And the first mark is the identification mark. The method of claim 25, And said identification mark is approximately circular. The method of claim 25, And the position of the first mark is determined by locating the center of the first mark. The method of claim 26, And the position of the first mark is determined by locating the center of the first mark. As a digital image data processing method, (a) receiving digital image data; (b) determining a position of a first mark included in the digital image data; (c) determining the distance and direction from the position of the first mark to a reference position; (d) performing a shift of the digital image data based on the distance and direction; (e) converting the shifted image data into recognition image data by processing the digital image data by a recognition engine; And (f) combining the recognition image data with a reference image. The method of claim 29, And said digital image data corresponds to a mark formed on a form located on a digital pad. The method of claim 30, And the reference image is an image of the form. The method of claim 31, wherein And the form is identified by the position of the zone in which the identification mark is formed. The method of claim 32, And the first mark is the identification mark. The method of claim 33, wherein And said identification mark is approximately circular. The method of claim 33, wherein And the position of the first mark is determined by locating the center of the first mark. The method of claim 34, wherein And the position of the first mark is determined by locating the center of the first mark. As an automatic identification method of the preprinted form, (a) receiving and recording, on the preprinted form, data indicating the position of the form formed manually by filling a hole in the template attached to the digitizer pad; And (b) identifying the form based on the position of the mark. The method of claim 37, And the data received and stored in step (a) is digital image data received and stored in a computer storage medium by the computer. The method of claim 38, Determining in step (b) a form identification zone comprising said position of said mark and determining a form corresponding to said form identification zone. The method of claim 39, Step (b) comprises storing a table that matches form identification zones with form IDs. The method of claim 40, Step (b) is Reformatting the received digital image data into a format processed by the recognition engine; Converting the reformatted digital image data into recognition data by processing the reformatted image data by the recognition engine; Retrieving recognition data for one or more form identification zones; Searching the retrieved recognition data for a zone containing a mark; And Identifying the form by matching the zone to a form ID. 42. The method of claim 41 wherein And displaying digital image data on a computer monitor to allow a user to manually identify the form on which the image was recorded. The method of claim 42, And displaying the recognition data on a user computer monitor in a format convenient for examination. The method of claim 43, And displaying digital image data superimposed on the image of the identified form to provide an on-screen image of the form and the image recorded thereon. The method of claim 44, Examining the recognition data by comparing the recognition data with the displayed digital image data superimposed on an image of the identified form. As the identification system of the preprinted form, a) digitizer; b) preprinted foam; c) a template attached to the digitizer and including one or more holes through which a user can enter one or more identification marks to identify the form; d) i) receive one or more digital images of the marks formed on the foam, ii) identify the presence and location of one or more identification marks formed by a user, iii) a computer processor coupled with a database for identifying a form based on the location of the one or more marks. As automatic identification computer software of preprinted form, (a) software for receiving and storing data indicative of the position of a manually formed mark on a preprinted form by filling in one or more holes in the template; And (b) software for identifying the form based on the position of the mark. The method of claim 47, And software for storing a table that matches form identification zones with form IDs. The method of claim 47, And the received and stored data is digital image data stored in a computer storage medium. The method of claim 49, And further comprising software for digital image data management for the following criteria. 51. The method of claim 50 wherein The identifying software includes software for determining a form identification zone that includes the location of the mark, and software for determining a form corresponding to the form identification zone. The method of claim 51, wherein Software to identify Reformatting the received digital image data into a format processed by the recognition engine; Converting the reformatted digital image data into recognition data by processing the reformatted image data by the recognition engine; Retrieve recognition data for one or more form identification zones; Search the retrieved recognition data for a zone containing a mark; And And software for identifying the form by matching the zone to a form ID. The method of claim 52, wherein Further comprising software for displaying digital image data on a computer monitor so that a user can manually identify the form on which the image was recorded. The method of claim 53, wherein And automatically display the identification data on a user computer monitor in a format convenient for examination. The method of claim 54, wherein And software for displaying the superimposed digital image data of the image of the identified form to provide an on-screen image of the form and the image recorded thereon. The method of claim 55, And identifying the identification data by comparing the identification data with the displayed digital image data superimposed on an image of an identification form. The method of claim 56, wherein Further comprising software for recognizing the relevant information on the form based on the non-standard symbols recorded on the form.