JP2003337941A - Device and method for image recognition, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognizing device of which the size and the cost can be reduced as a device for recognizing characters and images. <P>SOLUTION: The device comprises an image input means 1 of sequentially reading a two-dimensional code and characters represented on a paper surface, part by part, while scanning them, an image data storage means 2 of storing partial images of the two-dimensional code and characters, an image composing means 3 of putting the partial images together to generate the whole image of the two-dimensional code and characters, a binarizing means 4 of converting multi-valued data of the whole image into binary data consisting of 1 for parts of the two-dimensional code and characters and 0 for other background parts, an image cutting-out means 5 of cutting only the two-dimensional code and characters out of the binarized image, a two-dimensional code recognizing means 6 of decoding the cut two-dimensional code, a character recognizing means 7 of recognizing the cut characters, and a process executing means 9 which performs a process prescribed with the two-dimensional code. With this constitution, an optical system and an imaging sensor can be small-sized, so that the device is small-sized and the cost in reduced. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition apparatus for reading and recognizing a character or a two-dimensional code figure displayed on a paper surface, an image recognition method therefor, and a computer program defining the method, in particular. , Which enables image recognition with a small device.

[0002]

2. Description of the Related Art A conventional image recognition device for reading a two-dimensional code printed on a paper surface illuminates the paper surface on which a two-dimensional code figure is printed, as described in JP-A-7-234915. Then, the reflected light from the entire two-dimensional code figure is collected on the light receiving surface of the CCD area sensor, and the CCD area sensor reads the entire two-dimensional code figure according to the intensity of the light in the reflected light and converts it into an electric signal. Has been done.

A multi-valued image of a two-dimensional code read by a CCD sensor is usually subjected to a binarization process for converting it into binary data of white and black, and a recognition process is performed using the binarized image. Is done. The following method is generally used as the binarization method. Binarization is performed by using a threshold value having the highest separability from a histogram having pixel values of the entire image as data (discriminant analysis method). The entire image is divided into small regions, a threshold value is determined for each small region, and binarization is performed (region division method).

[0004]

However, since the conventional image processing apparatus for a two-dimensional code adopts a structure for photographing the figures of the two-dimensional code at one time, it requires a sufficient optical distance, and However, since the CCD area sensor is used as the image pickup element, there is a problem that the device is inevitably increased in size and the cost is increased. In order to add a two-dimensional code recognition function to a small mobile terminal such as a mobile phone, it is necessary to reduce the size and cost of the device. Also, by adding a character recognition function to such a small mobile terminal, various application applications are expanded. Further, the conventional binarization method can accurately perform binarization when the illuminance unevenness is small and the contrast is high, but the brightness unevenness of the light source is large, the influence of external light is large, and the paper surface is dirty. If there is, binarization cannot be performed accurately.

The present invention solves the above-mentioned conventional problems, and can reduce the size and cost of a device for recognizing characters and images, and can accurately binarize an input image. It is an object of the present invention to provide an image recognition apparatus that can perform the image recognition, and also provide an image recognition method and a computer program that defines the method.

[0006]

In view of the above, in the present invention, the image recognition device has an image input means for sequentially reading the two-dimensional code or the characters displayed on the paper while scanning, and an image input means for reading the two-dimensional code or characters. An image data storage means for storing a partial image of a dimensional code or a character, an image synthesizing means for synthesizing a partial image stored in the image data storage means to generate an entire image of a two-dimensional code or a character, and an image synthesizing means. Multi-valued data of the generated whole image,
Binarization means for converting the value of the two-dimensional code or character portion and the value of the other background portion into binary data different from each other,
An image cutting-out unit that cuts out only a two-dimensional code or a character portion from the binarized data generated by the binarizing unit, and decodes the two-dimensional code cut out by the image cutting unit.
The dimensional code recognition means, the character recognition means for recognizing the character cut out by the image cutout means, and the processing execution means for performing the processing defined by the two-dimensional code decoded by the two-dimensional code recognition means are provided.

Further, in the image recognition method of the present invention, the two-dimensional code or character displayed on the paper is sequentially read part by part while scanning the paper, and the partial image of the read two-dimensional code or character is temporarily stored as image data. After storing in the means, the two-dimensional code or the whole image of the character is generated by synthesizing, and the multivalued data of the generated whole image is converted into the value of the two-dimensional code or the character and the value of the other background portion. Convert to different binary data to generate a binary image,
Only the two-dimensional code or character portion is cut out from this binarized image, the cut-out two-dimensional code is decoded, the cut-out character is recognized, and the processing specified by the decoded two-dimensional code is performed. There is.

Further, the computer program of the present invention causes a computer to sequentially read the two-dimensional code or character displayed on the paper one by one while scanning the paper and a partial image of the read two-dimensional code or character. A procedure of once storing in the image data storage means, a procedure of synthesizing a partial image of the two-dimensional code or character stored in the image data storage means to generate an entire image of the two-dimensional code or character, Multivalued data
A procedure for generating a binarized image by converting the value of the two-dimensional code or character portion and the value of the other background portion into different binary data, and a procedure for generating a binarized image from this binarized image. A procedure of cutting out only the part, a step of decoding the cut out two-dimensional code, a step of recognizing the cut out character, and a step of performing the process defined by the decoded two-dimensional code are executed.

In the present invention, since the image is partially acquired and the whole image is obtained by synthesizing the images, the optical system and the image sensor can be downsized, and the size and cost of the device can be reduced. Can be planned. Further, it is possible to read the two-dimensional code displayed on the paper and perform the processing specified by the two-dimensional code.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION (First Embodiment) As shown in FIG. 1, an image recognition apparatus according to a first embodiment of the present invention scans a two-dimensional code or a character printed on a paper surface. An optical reading unit 1 that sequentially takes in partial images of these images, an image data storage unit 2 that temporarily stores partial images input from the optical reading unit 1, and an image data storage unit 2 are stored. An image synthesizing unit 3 that synthesizes partial images to generate a two-dimensional code or an entire image of characters, and a binary value that converts multi-valued data of the entire image generated by the image synthesizing unit 3 into binary data of 1 or 0. A conversion unit 4, a code / character cutout unit 5 that cuts out only a two-dimensional code or a character part from the binary data of the image obtained by the binarization unit 4, and a two-dimensional code cut out by the code / character cutout unit 5. On the other hand, the code type is judged and the 2-dimensional code recognition unit 6 for over de
And a character recognition unit (OCR) 7 that performs character recognition on the characters cut out by the code / character cutout unit 5, and a recognition result output unit 8 that outputs the recognition result of the two-dimensional code recognition unit 6 or the character recognition unit 7. And a processing execution unit 9 that automatically performs necessary processing based on the recognition result output from the recognition result output unit 8. The functions of the respective units can be realized by causing a computer (CPU) incorporated in this apparatus to perform the processing specified by the program.

As shown in FIG. 2, the optical reading unit 1 has two units.
A light source 10 that uniformly emits light onto a paper surface 15 on which a dimension code or characters are printed, and light from the light source 10 and the paper surface 1.
The input window portion 1 provided in a part of the upper surface of the FAP 12 for contacting the paper surface 15 with the fiber array plate (FAP) 12 made of a transparent material that propagates the reflected light from 5
6 and an image sensor 11 that receives the reflected light from the paper surface 15 and converts it into an electric signal.

The light source 10 is composed of an LED or a semiconductor laser for illumination. FAP12 forming optical path of transparent material
Can be replaced by optical components such as glass and plastic. The image sensor 11 is, for example, a CCD sensor or a CMOS sensor that converts light into an electric signal. The light a from the light source 10 reaches the white portion 13 of the paper surface 15, and the light a ′ having a strong light amount reflected here is the image sensor 11.
To reach. Further, the light b from the light source 10 reaches the black portion 14 of the paper surface 15, and since a part of the light is absorbed, the light b ′ with a small reflected light amount reaches the image sensor 11.
The intensity of this light is detected to obtain a printed matter image on the paper surface 15.

In this optical reading unit 1, the light sources 10 are arrayed, the input window 16 is set to a size that is larger than the object to be read, and the image sensor 11 also has a sufficient number of two-dimensional light receiving elements. By doing so, it is possible to read the object at one time, but this increases the size of the device and increases the cost. Therefore, the input window section 16 is set as a small strip-shaped area smaller than the reading object, and the optical reading unit 1 is slid with respect to the object to sequentially acquire partial images of the object and synthesize the partial images. Then, the whole image of the object is obtained. With this configuration, the number of light sources of the light source 10 and the number of light receiving elements of the image sensor 11 are set to the input window section 16
Since the number can be reduced to the number corresponding to the area of the strip-shaped small region, and the required optical distance can be reduced, the size and cost of the device can be reduced.

The image data storage unit 2 stores a partial image of the reading object received by the image sensor 11 of the optical reading unit 1. The image synthesizing unit 3 extracts two time-sequential partial images from the partial images sequentially stored in the image data storage unit 2 and performs matching processing for superimposing the two partial images. As a result, the amount of positional deviation between the two partial images is detected, and one composite image is created from the two partial images. This process is performed for all partial images,
An entire image of the read object is generated.

The binarizing unit 4 converts the multivalued data of the entire image of the read object generated by the image synthesizing unit 3 into binary data in which the code portion or the character portion is 1 and the other background portions are 0. Do the conversion. The code / character cutout unit 5 is 2
2 from the binarized image of the reading object created by the binarization unit 4.
The inclusion area where the dimension code or the character exists is detected. This detection processing can be easily obtained by acquiring the vertical and horizontal histograms of the image and detecting the black pixel existence range in both directions. When the object to be read is a two-dimensional code, the two-dimensional code recognition unit 6 inputs the binary image of the two-dimensional code existing area detected by the code / character cutout unit 5, determines the type of the code, and performs a decoding process. I do. When the object to be read is a character, the character recognition unit 7 inputs the binary image of the character existing area detected by the code / character cutout unit 5, cuts out each character, and performs character recognition processing. The recognition result output unit 8 displays the decoding result of the code processed by the two-dimensional code recognition unit 6 or the character recognition result processed by the character recognition unit 7 on a display device such as a display.

The process execution unit 9 performs a process required according to the result output by the recognition result output unit 8. For example, the recognition result by two-dimensional code or character recognition is URL
If it is an address, it will automatically connect to the home page. If the recognition result by the two-dimensional code or character recognition is a telephone number, FAX number or email address,
Register them in the address book. If the recognition result by the two-dimensional code or character recognition is a telephone number, FAX number or email address,
Start communication. And so on.

As described above, since the image recognition apparatus capable of recognizing the two-dimensional code and the characters acquires and synthesizes the partial images to generate the entire image, the optical system and the image sensor can be downsized. Therefore, the cost can be reduced. Further, this image recognition device can register a telephone number, an address, etc., or start communication based on the recognition result of the two-dimensional code or the character.

Detailed operations of the image synthesizing unit 3, the binarizing unit 4, the code / character cutting unit 5, and the process executing unit 9 of this apparatus will be described in the following embodiments.

(Second Embodiment) In the second embodiment of the present invention, processing for reducing the processing amount and speeding up the processing in the image synthesizing unit 3 will be described. FIG. 3 is a diagram for explaining a synthesizing process of two frame images that are continuous in time series. Of the two continuous images in time series, the image acquired first is the previous frame image 20, and the image acquired later is the current frame image 21. In order to perform image combination, the previous frame image 20 and the current frame image 21
And are overlapped with each other to calculate a difference average value of each pixel in the overlapping area 22 where the two images overlap. This process is performed by shifting the current frame image 21 by one pixel in the scanning direction of the apparatus with the previous frame image 20 as a reference. Then, the position where the difference amount between the two images having the lowest difference average value is small is detected, and the two images are combined at this position.

However, the calculation of the average difference value is performed in the overlapping area 2
If all the pixels of 2 are calculated as samples, enormous processing calculation and processing time are required. Therefore, in such a case, it is general to sample the pixels in the overlapping region 22 and reduce the number of pixels to be calculated. Figure 4
Is an example of a commonly used sample acquisition. Black square samples 26 represent pixels for which the difference calculation is performed. In this case, samples 26 are set every two pixels in the scanning direction and every eight pixels in the direction perpendicular to the scanning direction. When such sampling is performed, the number of samples is reduced to about 1/16 as compared with the case where all the pixels in the overlapping area 22 are taken into the sample, and the processing amount and the processing time can be proportionally reduced.

Next, an example of sample acquisition in the embodiment of the present invention will be described. FIG. 5 shows the image composition unit 3 of this embodiment.
It is a figure which shows the efficient sample acquisition which is performed by. If the number of pixels in the overlapping area 29 between the previous frame image 27 and the current frame image 28 is sufficiently large when the difference is calculated, the overlapping area 2
9 is divided into a sample acquisition region 31 and a sample non-acquisition region 32, and samples are acquired from the pixels of the sample acquisition region 31. The sample acquisition area 31 is the overlap area 2
It is set on the end side of the current frame image 28 in FIG.

The sample is set every few pixels on a line in the vertical direction with respect to the scanning direction, and on the line every several pixels in the scanning direction, it is set by shifting a few pixels in the vertical direction. In the example of FIG. 5, the sample acquisition area 31 is set on the upper 8 lines of the overlapping area 29, and the black square samples 30 are set at every 8 pixels in the direction perpendicular to the scanning direction and 2 pixels in the scanning direction. On the other hand, it is set by shifting two pixels in the vertical direction. By doing so, the number of samples can be reduced as compared to the sample acquisition of FIG.
It is possible to reduce the amount of processing and speed up the processing while maintaining the same degree of image synthesis accuracy as in the above case.

The image synthesizing unit 3 thus creates one synthetic image from the two partial images. Then, the combined image and the partial image read next by the optical reading unit 1 are combined, and this processing is performed on all the partial images read by the optical reading unit 1 to obtain the entire image of the read object. To generate.

As a result, the image recognition apparatus of this embodiment is capable of reducing the processing amount and increasing the speed, even though it is a method of obtaining partial images and synthesizing the entire image.

(Third Embodiment) In the third embodiment of the present invention, in the binarization unit 4, when the luminance unevenness of the light source is large, the influence of external light is large, or the paper surface is dirty. Even in the case, a method for accurately binarizing a code or a character will be described.

FIG. 15 shows a processing flow of the binarizing unit 4. In the drawings showing the processing flow, “step” is indicated by “S”. The binarization unit 4 uses a threshold value (binarization threshold value in the discriminant analysis method) for binarizing this image based on the brightness multivalue data of the image output from the image synthesizing unit 3. Then, the luminance value having the highest separability is calculated (step 110). Specifically, as shown in FIG.
A luminance value that separates the mountain of the background portion from the mountain of the code or the character is calculated from the luminance histogram of the input image, and this is used as the discriminant analysis binarization threshold value m. This calculation can be easily obtained by a mathematical formula of the discriminant analysis method (omitted in this description).

Next, the start address of the memory in which the multi-valued luminance data of the input image is stored is stored in addr (step 111). It is determined whether addr indicates the final address of the memory in which the brightness multi-valued data of the input image is stored (step 112). If the final address has not been reached, the process proceeds to the calculation of average brightness (step 113). ). As shown in FIG. 8, the average luminance is calculated by focusing on the pixel of interest stored in addr.
The average luminance value a of the peripheral 7 × 7 pixels is calculated.

Next, the offset value o is calculated from the average brightness value a based on the preset function of the average brightness value a and the offset value o (step 114). In FIG. 7, the average brightness value a and the offset value o are in a proportional relationship, and are functionalized by “(a straight line passing through the average brightness value, offset value) = (discriminant analysis binarization threshold value m, 0)”. I have an example. Since the offset value o is a function of the average brightness value a, the offset value will be referred to as o (a) hereinafter.

Next, using the brightness average value a calculated in step 113 and the offset value o (a) calculated in step 114, binary determination of the pixel of interest is performed (step 11).
5). If the luminance value of the pixel of interest is p, p ≦ a−o (a)
In the case of, the pixel of interest is binarized into a black pixel (step 11).
6). If p> a-o (a), the pixel of interest is binarized into a white pixel (step 117). 1 is added to addr to advance the address of the memory in which the multi-valued luminance data is stored by 1 (step 118). This process is performed for all the pixels of the input image, and at step 112 add
Repeat until r reaches the final address.

Here, the role of the offset value will be described.
When the binarization is performed by setting the average brightness of 7 × 7 pixels as a threshold value by the area division method without setting the offset value, the area for which the average brightness is obtained is a minute area of about 7 × 7 pixels. However, in the background portion where the luminance change is small, even if the luminance change is small, white and black judgments are mixed and a stable binarized image cannot be obtained. Therefore, by introducing the offset value shown in FIG. 7, black pixels are likely to be formed when the average luminance is small (that is,
When the average brightness a is smaller than m, o (a) becomes negative, and therefore (a-o (a)) becomes larger than a, and the brightness value p of the pixel of interest is p≤a-o (a). In addition, when the average luminance is large, white pixels are easily formed (that is, when the average luminance a is larger than m, o (a) becomes positive, and therefore (a−o (a) ) Is smaller than a, and the luminance value p of the pixel of interest tends to be p> a-o (a)).

As a result, even when the unevenness of the brightness of the light source is large, the influence of the external light is large, or the paper surface is dirty, the binarization can be performed to accurately separate the code or character portion from the background. it can. Although the average luminance value of the peripheral 7 × 7 pixels centering on the pixel of interest is calculated here, the number of peripheral pixels may be set to a different number.

(Fourth Embodiment) In the fourth embodiment of the present invention, a code / character cutout unit 5 for automatically determining whether an object to be cut out from a printed matter is a two-dimensional code or a character. The processing will be described.

FIG. 16 is a processing flow for explaining the printing type determining method of the present invention. The code / character cutout unit 5
The binary image output from the binarizing unit 4 is projected in the horizontal direction to calculate a histogram of the number of black pixels (vertical histogram) (step 120). 2 of FIG.
In the case of the dimension code (QR code), a histogram as shown in FIG. 10 is acquired. Further, in the case of the characters in FIG. 12, a histogram as shown in FIG. 13 is acquired. Next, the binary image output from the binarizing unit 4 is projected in the vertical direction to calculate a histogram of the number of black pixels (horizontal direction histogram) (step 121). In the case of the two-dimensional code (QR code) of FIG. 9, a histogram as shown in FIG. 11 is acquired. Further, in the case of the characters in FIG. 12, a histogram as shown in FIG. 14 is acquired.

Next, the number vn of black pixel peaks is detected from the vertical histogram (step 122), and the number hn of black pixel peaks is detected from the horizontal histogram (step 123). The detection of a mountain is obtained by detecting a continuous block having one or more black pixels. The number of mountain clusters in the histograms of FIGS. 10, 11 and 13 is 1, and the number of mountain clusters in the histograms of FIG. 14 is 30.

Next, the number of peaks vn detected in step 122
And the number of mountains hn detected in step 123,
Using xhn as an evaluation value, it is identified whether it is a character or a two-dimensional code (step 124). If vn × hn is smaller than a predetermined threshold value, it is determined to be a two-dimensional code and the two-dimensional code recognition process is performed (step 125). If it is larger than the predetermined threshold value, a character is detected. The determination is made and the process proceeds to the character recognition process (step 126).

As a concrete example, in the case of the two-dimensional code shown in FIG. 9, vn = 1 and hn = 1, and vn × hn = 1. In the case of the characters in FIG. 12, vn = 1, hn = 30, and vn × hn = 30. When the threshold value is 4, it can be determined that it is a two-dimensional code in the case of FIG. 9 and a character in the case of FIG. Further, when considering that the image is acquired with the character string inclined, the detection of the mountain from the histogram is not detected at the position of 0 of the black pixel number frequency, and the black pixel number is determined according to the assumed inclination angle. The determination can be made by detecting the position at a high frequency.

This method has a smaller processing amount and can be judged at high speed, as compared with the method of judging the existence of the code by detecting the cut-out code which is the characteristic of the two-dimensional code from the entire image. This eliminates the troublesome procedure for the user to select the recognition target, and improves the operability of the image recognition device.

(Fifth Embodiment) In the fifth embodiment of the present invention, the processing of the processing execution unit 9 will be described. First, as a preliminary preparation, as shown in FIG. 18, in the information data section 41 in which original information data for image recognition is described,
A processing type data section 40 in which processing type data representing a processing type of an operation command to the processing execution unit 9 is described is added, and combined data 42 of information data and processing type data is created,
This combined data 42 is encoded into a two-dimensional code.
Further, the correspondence between the operation command executed by the process execution unit 9 and the process type data is also determined in advance.

FIG. 17 shows a processing flow of the image recognition apparatus for reading this two-dimensional code. When the optical reading unit 1 reads the two-dimensional code in which the combined data is encoded, the two-dimensional code recognition unit 6 decodes it to restore the combined data (step 130). The restored combined data is output from the recognition result output unit 8 to the process execution unit 9. The process execution unit 9 extracts the process type data from the combined data (step 131), and executes the operation command corresponding to the process type data (step 132).

Next, description will be given using a specific example of FIG.
Here, the case where the optical image recognition device is provided in a small mobile terminal such as a mobile phone and the small mobile terminal has a two-dimensional code recognition function will be described. Combined data 4
In the second information data section 41, U which is the original information data
Stores data such as RL, mail address, and telephone number. Further, the processing type data section 40 of the combined data 42 stores the following processing type data designating the processing in the processing executing section 9. When performing the process of automatically connecting to the URL → Data:
1 When registering a URL → Data: 2 When registering an email address → Data:
3 When processing to register a telephone number → Data: 4 When processing to call a telephone number → Data: 5 The processing execution unit 9 stores the processing type data 1 in the processing type data unit 40 of the restored combined data 42. Is stored, and the URL is stored in the information data section 41, the UR
Automatically connect to L and display screen of home page.

When the process type data 2 is stored in the process type data section 40 of the restored combined data 42 and the URL is stored in the information data section 41, the U
Register the RL in the address book of the small mobile terminal. Further, when the processing type data part 40 of the restored combined data 42 stores the processing type data 3 and the information data part 41 stores the mail address, the mail address is stored in the mail address book of the small portable terminal. Register with. Also,
If the processing type data section 40 of the restored combined data 42 stores the processing type data 4 and the information data section 41 stores the telephone number, the telephone number is registered in the telephone directory of the small portable terminal. . Further, when the processing type data section 40 of the restored combined data 42 stores the processing type data 5 and the information data section 41 stores the telephone number, the telephone number is automatically transmitted. .

As described above, by providing a small mobile terminal such as a mobile phone with a two-dimensional code recognition function, the two-dimensional code of the combined data including the processing type data of the operation command can be read and various small mobile terminals can be read. The processing of can be automatically performed. Therefore, the user can omit troublesome operations for performing various processes, and can improve the operability of the small mobile terminal. The computer program that defines the operation of the image recognition device of the present invention can be recorded in a recording medium and moved to another image recognition device through this recording medium.

[0043]

As is clear from the above description, in the image recognition apparatus, the image recognition method and the program of the present invention, partial images are acquired and combined to generate the entire image. The sensor can be miniaturized,
The device can be downsized and the cost can be reduced. In addition, since the two-dimensional code can be read and the operation command represented by the two-dimensional code can be automatically executed, the user is freed from the troublesome operations of various processes, and the operability of the apparatus is improved.

Further, when selecting the pixel of the sample only from a part of the overlapping area of the overlapped partial images when synthesizing the whole image from the partial images, a small number of samples are used to generate the partial image. Highly accurate composition is possible.
Therefore, high-speed processing is possible even though a partial image synthesizing procedure is required. When generating a binarized image, the value of the pixel of interest is determined by using a value obtained by subtracting an offset value from a pixel average value of a region including the pixel of interest and peripheral pixels of the pixel as a threshold value. In this case, when the brightness unevenness of the light source is large or when the influence of external light is large,
Even when the paper surface is dirty, the code and characters can be accurately binarized.

When the image is cut out, histograms in the directions parallel and perpendicular to the scanning direction are acquired, and whether the image is a two-dimensional code or a character is determined from the number of peaks and the size of the peaks in the histograms in both directions. When determining whether or not, it becomes possible to automatically identify the recognition target object,
The user is released from the operation of selecting the recognition target object.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an optical image recognition device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of an optical reading unit according to the first embodiment of the present invention.

FIG. 3 is a diagram illustrating a superposing process of two frame images in an image synthesizing process according to the second embodiment of the present invention.

FIG. 4 is a diagram showing an example of sample acquisition generally performed.

FIG. 5 is a diagram showing sample acquisition in image synthesis processing according to the second embodiment of the present invention.

FIG. 6 is a diagram showing a luminance histogram of an input image according to the third embodiment of the present invention.

FIG. 7 is a diagram showing a relationship between an offset value and an average brightness value set in a third embodiment of the present invention.

FIG. 8 is a diagram for explaining an average luminance value calculation method in the third embodiment of the present invention.

FIG. 9 is a diagram showing an example of a two-dimensional code (QR code).

FIG. 10 is a diagram showing a vertical histogram of a two-dimensional code (QR code) according to the fourth embodiment of the present invention.

FIG. 11 is a diagram showing a horizontal direction histogram of a two-dimensional code (QR code) according to the fourth embodiment of the present invention.

FIG. 12 is a diagram showing an example of a character string.

FIG. 13 is a diagram showing a vertical histogram of a character string according to the fourth embodiment of the present invention.

FIG. 14 is a diagram showing a horizontal histogram of a character string according to the fourth embodiment of the present invention.

FIG. 15 is a processing flow illustrating a binarization method according to the third embodiment of the present invention.

FIG. 16 is a process flow illustrating a print type determination method according to the fourth embodiment of the present invention.

FIG. 17 is a process flow showing a process execution procedure of the image processing apparatus according to the fifth embodiment of the present invention.

FIG. 18 is a diagram illustrating data to which operation command processing type data used in the fifth embodiment of the present invention is added.

[Explanation of symbols]

1 Optical reading unit 2 Image data storage 3 Image synthesis section 4 Binarization part 5 Code / Character cutout 6 Two-dimensional code recognition unit 7 Character recognition part 8 Recognition result output section 9 Process execution unit 10 light sources 11 Image sensor 12 FAP 13 Shirobe 14 Kurobe 15 pages 16 Input window 20, 23, 27 Previous frame image 21, 24, 28 Current frame image 22, 25, 29 Overlap area 26,30 samples 31 Sample acquisition area 32 sample non-acquisition area 40 Processing type data section 41 Information Data Division 42 combined data

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G06T 7/00 G06T 7/00 300G 7/40 100 7/40 100A (72) Inventor Chihiro Yokohama Kanagawa Prefecture Matsushita Communication Industrial Co., Ltd., 3-1, Tsunashima Higashi, Kohoku-ku, Yokohama (72) Inventor Tomoyuki Tsutoribe 3-4-1, Tsunashima-Higashi, Kohoku-ku, Yokohama-shi, Kanagawa Matsushita Communication Industry Co., Ltd. (72) Inventor Misa Sano 4-3-1, Tsunashima-higashi, Kohoku-ku, Yokohama-shi, Kanagawa Matsushita Communication Industry Co., Ltd. (72) Inventor Ryosuke Iida 4-3-1, Tsunashima-higashi, Kohoku-ku, Yokohama-shi, Kanagawa Matsushita Communication Industry Co., Ltd. ( 72) Inventor Chie Ide 4-3-1 Tsunashima-higashi, Kohoku-ku, Yokohama-shi, Kanagawa Matsushita Communication Industrial Co., Ltd. (72) Inventor Sachiko Shiina Yokohama, Kanagawa 4-3 Tsunashima Higashi 4-chome, Kohoku Ward Matsushita Communication Industry Co., Ltd. F-term (reference) 5B029 BB02 CC30 DD05 5B057 CA08 CA12 CA16 CB08 CB12 CB16 CE08 CE10 CE12 DA07 DB02 DB09 DC19 DC32 5B072 CC21 CC38 DD01 DD23 DD24 DD25 FF02 LL11LL 5L096 AA06 BA17 BA18 EA43 FA32 FA36 FA42 FA44 FA46 GA51 HA07 JA11

Claims (16)

[Claims] 1. An image input unit for sequentially reading a two-dimensional code or a character displayed on a sheet while scanning, and image data storage for storing a partial image of the two-dimensional code or a character read by the image input unit. Means, an image synthesizing means for synthesizing the partial images stored in the image data storage means to generate an entire image of the two-dimensional code or characters, and multivalued data of the entire image generated by the image synthesizing means. Is converted into binary data in which the value of the two-dimensional code or character portion and the value of the other background portion are different, and the binary data generated by the binarizing means from the binary data An image cutting-out means for cutting out only a two-dimensional code or a character portion, and a two-dimensional code recognizing hand for decoding the two-dimensional code cut out by the image cutting means. And a character recognizing means for recognizing the character cut out by the image cutting means, and a process executing means for performing a process defined by the two-dimensional code decoded by the two-dimensional code recognizing means. Image recognition device. 2. The image synthesizing means combines the preceding partial image with the preceding partial image when synthesizing the preceding partial image, which is the partial image preceding in time, with the succeeding partial image, which is the partial image succeeding in time. The image recognition apparatus according to claim 1, wherein the subsequent partial image is overlapped, sample pixels are selected from a partial area of the overlapped area, and matching processing is performed. 3. The binarizing means calculates a pixel average value representing an average luminance of pixels in a region including a pixel of interest and peripheral pixels of the pixel, and the pixel average value is a function of the pixel average value. The image recognition apparatus according to claim 1, wherein the value of the pixel of interest is determined by using a value obtained by subtracting the offset value as the threshold value. 4. The image cutting means acquires histograms in a direction parallel to and a direction perpendicular to the scanning direction, and determines the number of peaks and the size of the peaks in the histogram in the parallel and vertical directions. The image recognition apparatus according to claim 1, wherein it is determined whether the reading target displayed on the paper is a two-dimensional code or a character. 5. The two-dimensional code recognition means decodes the code of the operation instruction previously encoded in the two-dimensional code, and the processing execution means decodes the operation instruction code decoded by the two-dimensional code recognition means. The image recognition apparatus according to claim 1, wherein the processing is performed according to the following. 6. A two-dimensional code or character displayed on the paper is sequentially read part by part while scanning the paper,
The read partial image of the two-dimensional code or the character is temporarily stored in the image data storage means, and is then combined to form the two-dimensional code.
An entire image of a dimension code or a character is generated, and multi-valued data of the generated entire image is converted into binary data in which the value of the two-dimensional code or character and the value of the other background portion are different. To generate a binarized image, cut out only the two-dimensional code or character portion from the binarized image, decode the cut-out two-dimensional code, character-recognize the cut-out character, and decode the An image recognition method characterized by performing processing specified by a dimension code. 7. When synthesizing the partial images, a preceding partial image which is the partial image preceding in time is overlapped with a subsequent partial image which is the partial image succeeding in time, 7. A matching process is performed by selecting a sample pixel only from a partial area.
The image recognition method described in. 8. When generating the binarized image, a pixel average value representing average luminance of pixels in a region including a pixel of interest and peripheral pixels of the pixel is calculated, and the pixel average value is calculated from the pixel average value. A threshold value is a value obtained by subtracting an offset value that is a function of the value, and the binary data of the pixel of interest is determined by comparing the brightness value of the pixel of interest and the threshold value. The image recognition method according to claim 6. 9. A histogram in a direction parallel to and a direction perpendicular to the scanning direction of the binarized image is obtained, and the number of peaks and the size of the peaks in the histogram in the parallel direction and the vertical direction are used. 7. The image recognition method according to claim 6, wherein it is determined whether the reading target displayed on the paper surface is a two-dimensional code or a character. 10. A method of reading the two-dimensional code displayed on the paper sheet, in which the code of the operation instruction is encoded, decoding the code of the operation instruction, and executing the operation instruction defined by the code. The image recognition method according to claim 6, which is characterized in that. 11. A procedure for a computer to sequentially read the two-dimensional code or character displayed on the paper one by one while scanning the paper, and the partial image of the read two-dimensional code or character is temporarily stored in the image data storage means. A procedure of storing, a procedure of synthesizing the partial image of the two-dimensional code or character stored in the image data storage means to generate an entire image of the two-dimensional code or character, and multi-valued data of the generated entire image Is converted into binary data in which the value of the two-dimensional code or character portion and the value of the other background portion are different to generate a binary image, and the two-dimensional image from the binary image. A procedure for cutting out only the code or character portion, a step for decoding the cut out two-dimensional code, a step for recognizing the cut out character, Program for executing a procedure for performing processing specified by the two-dimensional code over de. 12. In a procedure of synthesizing the partial images of the two-dimensional code or character to generate the entire image of the two-dimensional code or character, the computer is configured to display a preceding partial image which is the partial image preceding in time. 12. The method according to claim 11, wherein a subsequent partial image which is the partial image that is temporally behind is superimposed, and a pixel of a sample is selected from a partial region of the superimposed region to perform a matching process. Program of. 13. A procedure of converting multivalued data of the entire image into binary data to generate the binarized image,
A procedure for calculating, in a computer, a pixel average value that represents the average luminance of pixels in a region including a pixel of interest and peripheral pixels of the pixel, and subtracting an offset value that is a function of the pixel average value from the pixel average value. The program according to claim 11, which executes a procedure of comparing the luminance value of the pixel of interest with the threshold value and determining binary data of the pixel of interest, with the value being a threshold. 14. A procedure of, in the procedure of cutting out the two-dimensional code or character portion from the binarized image, a procedure of causing a computer to obtain histograms in a direction parallel to a scan direction of the binarized image and a direction perpendicular to the scan direction. And a procedure for determining whether the image is a two-dimensional code or a character based on the number of peaks and the size of the peaks in the histogram in the parallel direction and the vertical direction. 11. The program according to item 11. 15. A procedure of performing a process defined by the two-dimensional code, wherein the computer identifies a code of an operation instruction represented by the two-dimensional code,
The program according to claim 11, for executing a procedure of performing processing according to the operation instruction corresponding to the code. 16. A recording medium on which the program according to any one of claims 11 to 15 is recorded.