ITMI20062316A1 - METHOD AND APPARATUS FOR RECOGNIZING TEXT IN A DIGITAL IMAGE. - Google Patents

Description

P02725 / IT / FC Title: "Method and apparatus for recognizing text in a digital image."

DESCRIPTION

The present invention relates to a method and an apparatus for recognizing text in a digital image, in particular to a method and an apparatus in accordance with the preamble of claims 1 and 27.

In today's society, most information can be acquired through sight. Although the laws in force are increasing the number of information available through touch (think for example of the elevator buttons, the Braille writings on medicines, the indications

tactile on the floors of airports or stations), there are still many fields in which a blind or visually impaired user has extreme difficulty in

make use of the information available.

For example, think of drinks contained in bottles or cans, or the contents of food cans, or boxes containing food, or medicines.

In all these cases, a blind or visually impaired user can very easily find himself in the situation of having to obtain information that is only available through visual stimuli, precisely those that the visually impaired user or

blind person cannot decode easily.

For example, all cans have the same shape and almost the same dimensions, the different contents being widely described on the surface of the can; glass bottles can indifferently contain wine, water, oil, vinegar, liqueurs, or plastic bottles

Ing. Aurelio PERANI Registered no. ALBO 277 (on their own or others) with the same shape can contain trichlorethylene or water; cans with the same shape can contain beans, tomato sauce or tuna; the cardboard boxes can contain pasta with extremely different cooking times and, finally, the medicines, have the information relating to the name of the medicine and the expiration date in tactile format only on the package: when the blister is separated from this, for a blind or visually impaired users it can become practically impossible to correctly decode this information starting only from the shape of the blister.

From these examples, it is clear how a mistake in understanding this visual information can have unpleasant consequences, from the mild consequences of eating overcooked pasta with a tuna sauce instead of beans, to poisoning caused by taking the wrong medicine. or a corrosive liquid instead of plain water.

To avoid such problems, blind or visually impaired users generally rely on the help of visually impaired people who can read that important information for them, perhaps giving up external help for less relevant information, such as related information. the brand of detergent or the cooking time of the pasta.

However, it would be desirable to have a system that allows visually impaired and / or blind users to free themselves from the help of other people, so as to be less dependent on them and to be able to exploit even less important information, however fundamental for a better quality of life.

In the art there are known digital image acquisition devices, OCR programs (Optical Character Recognition, i.e. programs capable of converting the text contained in a digital image into textual code) and

Ing. Aurelio PERANi N. lscr. ALBO 277 (on their own and for others) speech synthesizers able to read the OCR result.

Image acquisition devices, whether they are scanners, cameras (digital or not), webcams, planetary scanners, digital video cameras or the like are affected by the lack of precision in setting the image acquisition parameters; the result is, frequently, a blurred and / or blurred image, over- or underexposed, with an incorrect framing, with the presence of unwanted reflections, with an incorrect calibration of color, contrast and / or depth of field .

To all the aforementioned problems relating to the goodness of the starting image is added the problem relating to the shape of the photographed object, which, in the case of a can, for example, could also require a non-sighted user to rotate the can. around its axis to read the brand name in full and therefore know its content.

Starting from these images it is currently impossible for an OCR system to obtain good results, as the OCRs currently available on the market have been substantially developed to read documents containing text printed in white on black, with regular and standard type characters (Arial, Times New Roman, Courier, etc.), with a body of at least 10-12 points and aligned along the vertical direction or the horizontal direction.

Currently possible results are limited in terms of practicality to, for example, reading drug package inserts.

The aforementioned framing, lighting and image stability problems were addressed by the owner of the present invention and solved at least partially by means of the device covered by the patent entitled "ASSISTANCE DEVICE FOR THE PROOFLY VISIBLE TO SHOOT

Ing. Aurelio PERANI Registered no. ALBO 277 (in person and for others) PHOTOGRAPHS ", filed in the name of the same Applicant on 29 November 2006.

In view of the state of the art described, the purpose of the present invention is to provide a method and an apparatus for recognizing text in a digital image that does not have the disadvantages of the prior art.

In accordance with the present invention, this object is achieved by means of a method according to claim 1 and an apparatus according to claim 27.

This object is also achieved by a computer product which, loaded into the memory of a computer and made to work on this computer, allows the method to be implemented in accordance with the present invention.

The characteristics and advantages of the present invention will become evident from the following detailed description of a practical embodiment, illustrated by way of non-limiting example in the accompanying drawings, in which:

figure 1 shows an apparatus according to an embodiment of the present invention;

- Figures 2A, 2B, and 2C show graphic representations of the application of respective recognition strategies, in accordance with the present invention.

In the course of the present invention, some terms will be used, which are defined below.

By "digital image" we mean an image that is presented in digital format, regardless of how it was acquired for the first time; this acquisition will be advantageously carried out by means of a digital camera, but other methods are possible, as indicated

Ing. Aurelio PER ANI N. Insc. ALBO 277 (on his own and for others) in the introductory part. The subject of the image will be clear from the context. By "object" we mean any element that can be represented in a digital image, for example, bottles, cans, jars, but also 1-door books, 2-door books, brochures, newspapers, magazines, envelopes, signs, synoptic tables, etc. ..

With "text in the image" we mean portions of the digital image containing graphic representations of text elements, whether they are alphanumeric symbols or other textual characters of all languages including Asian, Arabic and Cyrillic, but also commercially known symbols, such as for example <S>, et, £ r, ®, hand wash, pure new wool, etc ..; in general, "text present in the image" can be considered anything that is attributable to one of the characters that can be read by a computer as "text" or otherwise attributable to glyphs.

By "OCR means" we mean means capable of analyzing a digital image and recognizing the presence and position of any text present in the image and converting it into text.

Having said all this, with reference to the attached figures, an apparatus 1 for determining the text 2 contained in an image 3 depicting an object 4 is represented.

This apparatus 1 includes:

- a processing device 5 having processing means 6 to process the digital image 3 so as to produce a first modified image 3A;

-means 7 for performing an optical character recognition (OCR) on said first modified image 3A to determine at least one region 3B containing text within said modified image 3A;

Ing. Aurelio PERANI Registered no. ALBO 277 (for oneself and for others) -means 8 to convert said text contained in region 3B into

a first voice signal SI, said voice signal SI being emitted through

an audio interface 9 operatively connected to said processing device 1.

These means for carrying out an optical recognition of the character 7 take the form of a scanner or, advantageously, of a camera (as shown in Figure 1), a webcam, a planetarium scanner or a digital video camera.

It should be noted that such means for carrying out an optical recognition of character 7 provide for associating to the first region 3B a first reliability value Cl, i.e. a value by means of which it is possible to have an estimate

the quality of the text contained in this first region 3B, as well as the position of

this region 3B with reference to the first modified image 3A.

It should be noted that the means 7 for carrying out an optical character recognition (OCR) are operatively connected with said processing device 1, for example, by means of a Bluetooth communication protocol, by USB, by means of a Wi-Fi communication protocol, via

cable, Wireless etc ..

In particular, if a digital camera is used to acquire the digital image 3, it should be emphasized that in addition to transferring the information relating to this digital image 3, it is possible that

also transferred all the information that the camera stores in a so-called "exif file" (or similar proprietary formats), that is

details that contain details about the camera, about the parameters of the

shooting such as, for example, focus distance, aperture, time of

Ing. Aurelio PERANI Registered no. ALBO 277 (on his own and for others) exposure, focal length, etc. and on the burn configuration.

In particular, the processing device 1 takes the form of a computer, be it a PC, a laptop, a palmtop, a PDA or a telecommunications device such as a mobile phone.

Advantageously, a program capable of recognizing the text contained in the region 3B of the modified image is installed in this processing device 1.

To this end, said means for processing 6 are able to:

(a) processing said digital image 3 to produce a first modified image 3A;

(b) performing optical character recognition (OCR) on said first modified image 3A to determine at least one region containing text

3B inside said modified image 3A and associating to said first region a first reliability value C1;

(c) converting said recognized text into a first voice signal S1;

in which said phase of processing (a) includes the further phase of correcting in

automatically the graphic parameters of said image, said graphic parameters including brightness, contrast, saturation and / or equivalent color spaces.

The graphic parameters can also be modified thanks to the information contained in the "exif" parameters of said image 3A.

It should be noted that this automatic modification can also take place by modifying one or more of the aforementioned brightness, contrast and / or saturation parameters or in equivalent color spaces.

Furthermore, such means for processing 6 are also capable of carrying out the following steps:

Ing. Aurelio PERANI Registered no. BOOK 277 (on one's own and for others) (d) compare said first confidence value CI with a predetermined

reliability value C2;

(e) selecting at least one recognition strategy Xj among a plurality "n" of recognition strategies X if said first reliability value CI is lower than said predetermined reliability value C2.

It should be noted that each of said plurality of recognition strategies X is capable of modifying the graphic and geometric / perspective parameters of said first modified image 3A to generate a second modified image 3C to increase said first value Cl.

Advantageously, the previously listed steps (e), (b) and (c) can be repeated as long as said first reliability value Cl is close to or higher than said predetermined reliability value C2.

It should be noted that each X; of said plurality of recognition strategies X is able to modify the graphic parameters of the first modified image 3A to generate the second modified image 3C, by iterating one or more of said plurality of strategies X until reaching a

reliability value C compatible for the purposes of the present invention.

In other words, following the application on the modified image 3A

of one or more of such strategies X it is possible to reach a sufficiently high reliability value for the text contained in the text region 3B of the modified image 3A to be read with sufficient precision thus

to generate the voice signal YES understandable to the user.

The above strategies X for obtaining recognition of the text contained in region 3B will now be illustrated, also with reference to Figures 2A to 2D.

Ing. Aurelio PERANI Registered no. ALBO 277 (on his own and ask him others) A first recognition strategy Xi include the phase of:

(f) select from a predefined list the type to which said object 4 belongs, said list including objects attributable to any geometric primitive having a cylindrical, spherical, conical shape, or also objects attributable to any single, double and / or composite perspective primitive having a shape parallepipedica.

It should be noted that the selection can also be set manually by the user by specifying the specific geometric primitive or perspective primitive to which the object 4 belongs.

If object 4 belongs to the type of objects having a cylindrical, spherical and / or conical shape, strategy X) provides that said phase (f) includes the further phase (f.l) or strategy Xu of applying an inverse cylindrical transformation function to said text region 3B.

In fact, also with reference to Figure 2A, thanks to this inverse cylindrical transformation function we have that the basic lines of the text contained in this region 3B, which appear to be curved due to the perspective, are brought back through said inverse transformation to straight lines 3D.

If the text areas (for example as represented in figure 2d) are two 3E and 3F, both with low reliability, on the two sides of the object 4, it is probable that the object 4 must be rotated by a predetermined angle, to example, 90 degrees.

To this end, phase (f) includes the further phase (f.2) or strategy X] .2 of imposing a predefined angular rotation around the vertical axis of said object 4.

Advantageously, a second signal is generated

Ing. Aurelio PERANI Registered no. ALBO 277 (For oneself and for others) item S2 to specify the default rotation.

In particular, the OCR means 6 return as a value the portions of the modified image 3A to which position and "general" reliability are associated.

Furthermore, within these portions of the modified image 3 ° returned by the OCR means, their position and reliability is also associated with each character.

So in the event that the object 4 depicted in image 3 and has two columns with very low reliability, it is likely that this object 4 belongs to a geometric primitive in the list and that object 4 has been photographed from the wrong side.

The voice signal S2 warns the user and proposes to carry out the angular rotation around the vertical axis of the object 4, for example a rotation equal to 90 °.

If, on the other hand, object 4 depicted in image 3 has two columns with medium-high reliability, it is likely that object 4 belongs to a geometric primitive in the list and that object 4 has been photographed from the right side. In this scenario, object 4 is as wide as the sum of the column widths.

It should be noted that the width of object 4 is calculated as the width of the single column or the sum of the two columns.

The inventive method during the strategy Xu and X (2 sets the inverse cylindrical transformation function at the center of the width and height of the object 4.

Object 4 is attributable to said geometric primitives and is chosen from

Irsg. Aurelio PERANI N. Insc. ALBO 277 (on his own and for others) group comprising bottles, cans, jars and the like.

If object 4 belongs to the typology of objects attributable to single or double perspective primitives having a parallelpipedic shape of the book type with one door (figure 2B) or two wings (figure 2C), phase (f) includes the further phase ( f.3) or strategy X1.3 to apply an inverse perspective function to said text region.

In fact, thanks to this inverse perspective function, the basic lines of the text contained in the region 3b, curved by the structure of the relative geometric primitive, are brought back through an inverse transformation to straight lines, as shown in the 3D area.

Object 4 is attributable to said primitive perspective having a parallepipedic shape and is chosen from the group including 1-leaf books, 2-leaf books, brochures, newspapers, magazines, envelopes, signs, synoptic tables.

A further strategy Xj .4, which can be executed subsequently or independently of the strategies described above, comprises the further step (f.4) of interpolating the native resolution of said first modified image 3A to generate said second modified image 3C having a resolution higher than the resolution native of said first modified image.

Advantageously, thanks to this strategy Xi.4, it is possible to find characters with a small body inside said text region 3B.

A further strategy X1 5, which can be executed subsequently or independently of the strategies described above, includes the further step (f.5) of applying a first vector filter of the type of edge erosion of said region containing text.

Advantageously, thanks to this strategy X1.5 it is possible to distinguish the

Ing. Aurelio PERANI Registered no. ALBO 277 (On his own Θ for the others) written in bold type inside said region of text 3B.

A further strategy Xi.6, which can be executed subsequently or independently of the strategies described above, includes the further phase

(f.6) to apply a second vector or graphic filter of the type of maximum contrast to said region containing text, and / or strategy X] .7 the further phase (f.7)

to apply a third vector or graphic filter of the minimum contrast type is a

said region containing text.

A further strategy Xi, 8, which can be executed subsequently or independently of the strategies described above, includes the further phase

(f.8) to apply a fourth vector or graphic filter having a first value of

threshold for filtering the white to black pixels of said region containing text 3B,

and / or strategy X1.9 of applying a fifth filter having a second threshold value

to filter the black to white pixels of that region containing text.

It should be noted that said threshold value of strategies Xu and X1.9 can be determined as a function of the predetermined reliability value.

A second X2 recognition strategy, which can be performed later

0 regardless of the strategies described above, which comprises the step of interpolating the native resolution of said first modified image to generate said second modified image having a higher resolution

at the native resolution of said first modified image, as already described

previously.

Furthermore, there may be a third recognition strategy X3, which can be executed subsequently or independently of the strategies described above,

which comprises the step of applying said first edge erosion filter a

said region containing text, as already described above.

Ing. Aurelio PERANI Registered no. ALBO 277 (on one's own and for others) A fourth recognition strategy X4, which can be executed subsequently or independently of the strategies described above, comprises the step of applying said second maximum contrast filter to said region containing

text 3B.

A fifth recognition strategy X5 comprises the step of applying said third minimum contrast filter to said text-containing region

3B.

A sixth X6 recognition strategy includes the apply phase

said fourth filter having a first threshold value for filtering the white to black pixels of said region containing text 3B.

Alternatively, the sixth recognition strategy X6 comprises the step of applying said fifth filter having a second threshold value to filter the black to white pixels of said region containing text.

It should be noted that said threshold value for filtering pixels from black to white and / or from white to black can be determined as a function of the predetermined reliability value.

It should also be noted that said at least one region containing text within said modified image if said first reliability value is close to zero coincides with said modified image.

As can be appreciated from what has been described, the method and the apparatus according to the invention allow to meet the needs mentioned in the

introductory part of the present description and to obviate the drawbacks of the methods and apparatuses of the known art.

Obviously, a person skilled in the art, in order to satisfy contingent and specific needs, can make numerous modifications and variations to the

Ing. Aurelio PERANI Registered no. ALBO 277 (in its own right and for the others) configurations described above, all however contained within the scope of protection of the invention as defined by the following claims.

Ing. Aurelio PERANI N. lscr. ALBO 277 (on his own to ask him others)

Claims (31)

CLAIMS 1. Method for recognizing text in an image (3), in which an object (4) is represented in said image (3), comprising the steps of: (a) processing a digital image (3) to produce a first modified image (3A); (b) performing an optical character recognition (OCR) on said first modified image (3A) to determine at least one region containing text (3B) within said modified image (3A) and associating to said first region (3B) a first reliability value (Cl); (c) converting said recognized text into a first voice signal (S1); in which said processing step (a) comprises a further step of automatically correcting the graphic parameters of said image (3), said graphic parameters comprising brightness, contrast, saturation and / or equivalent color spaces. 2. Method for recognizing text in an image in accordance with claim 1, comprising the further steps of: (d) comparing said first reliability value (Cl) with a predetermined reliability value (C2); (e) selecting at least one recognition strategy (Xi; ..., X6) among a plurality of recognition strategies if said first reliability value (Cl) is lower than said predetermined reliability value (C2), each of said plurality recognition strategies (Xi; ..., X6) being adapted to modify the graphic and geometric / perspective parameters of said first modified image (3A) to generate a second modified image (3C) to increase said first reliability value (Cl ). Ing. Aurelio PERANI N. Jscr. ALBO 277 (on his own and for others) 3. Method for recognizing text in an image in accordance with claim 2, wherein said method provides for repeating steps (e), (b) and (c) as long as said first reliability value (Cl) is close to or higher to said predetermined reliability value (C2). 4. Method for recognizing text in an image in accordance with any one of claims 1 to 3, wherein a first recognition strategy (Xj) comprises the step of: (f) select from a predefined list the type to which said object belongs (4), said list including objects attributable to any geometric primitive having a cylindrical, spherical, conical shape, or also objects attributable to any single, double and / or composite perspective primitive having a parallelpipedic shape. 5. Method for recognizing text in an image according to claim 4, wherein if said object (4) belongs to the typology of objects having a cylindrical, spherical and / or conical shape, said phase (f) comprises (XJ.I) 1 a further step (f.l) of applying an inverse cylindrical transformation function to said text region. Method for recognizing text in an image according to claim 4 or 5, wherein said step (f) comprises the further step (f.2) of imposing (X1.2) a predefined angular rotation around the vertical axis of said object. Method for recognizing text in an image according to claim 6, wherein said step (f.2) provides for generating a second voice signal (S2) to specify said predefined angular rotation. 8. Method for recognizing text in an image in accordance with the ! ng. Aurelio PERANI N. Insc. ALBO 277 (on his own and asking others) claim 4, in which if said object (4) belongs to the type of objects attributable to single or double perspective primitives having a parallelpipedic shape, said phase (f) includes the further phase (f.3) of applying (X1.3) an inverse perspective function to said text region (3B). 9. Method for recognizing text in an image in accordance with any one of claims 4 to 8, in which said step (f) includes the further step (f.4) to interpolate (X | .4) the native resolution of said first modified image to generate said second modified image having a resolution higher than the native resolution of said first modified image (3B). Method for recognizing text in an image according to any one of claims 4 to 9, wherein said step (f) comprises the further step (f.5) of applying (Xi.5) a first vector filter or graph of type of edge erosion called region containing text (3B). Method for recognizing text in an image according to any one of claims 4 to 10, wherein said step (f) comprises the further step (f.6) of applying (Xi.e) a second vector filter 0 graph of maximum contrast type to said region containing text (3B). Method for recognizing text in an image according to any one of claims 4 to 10, wherein said step (f) comprises the further step (f.7) of applying (X1.7) a third vector filter or type chart of minimum contrast is to said region containing text (3B). Method for recognizing text in an image according to any one of claims 4 to 12, wherein said step (f) comprises the further step (f.8) of applying (Xi.s) a fourth vector filter or graphic having a first threshold value for filtering the white to black pixels thereof ! ng. Aurelio PERANI Tech. No. ALBO 277 (on his own and for others) region containing text (3B). Method for recognizing text in an image according to any one of claims 4 to 12, wherein said step (f) comprises It is a further step (f.9) of applying (X1.9) a fifth vector or graphic filter having a second threshold value to filter the black to white pixels of said region containing text (3B). 15. Method for recognizing text in an image according to claims 13 and 14, wherein said threshold value can be determined as a function of the predetermined reliability value (C2). Method for recognizing text in an image according to any one of claims 1 to 3, wherein a second recognition strategy (X2) comprises the step of interpolating the native resolution of said first modified image (3A) to generate said second modified image (3C) having a resolution higher than the native resolution of said first modified image (3A). Method for recognizing text in an image according to any one of claims 1 to 4 or 16, wherein a third recognition strategy (X3) comprises the step of applying said first edge erosion filter to said text-containing region (3B). 18. Method for recognizing text in an image according to any one of claims 1 to 4 or 16 to 17, wherein a fourth recognition strategy (X4) comprises the step of applying said second maximum contrast filter to said region containing text (3B). 19. Method for recognizing text in an image in accordance with any of claims 1 to 4 or 16 to 18, wherein a fifth strategy Ing. Aurelio PERANI N. fscr. ALBO 277 (on its own and for others) of recognition (X5) comprises the step of applying said third minimum contrast filter to said region containing text (3B). Method for recognizing text in an image in accordance with any one of claims 1 to 4 or 16 to 19, wherein a sixth recognition strategy (X6) comprises the step of applying said fourth filter having a first threshold value for filtering the pixels from white to black of said region containing text (3B). Method for recognizing text in an image according to any one of claims 1 to 40 from 16 to 19, wherein a sixth recognition strategy (X6) comprises the step of applying said fifth filter having a second threshold value for filtering the black to white pixels of said region containing text (3B). 22. Method for recognizing text in an image in accordance with claims 20 and 21, wherein said threshold value can be determined according to of the predetermined reliability value. 23. Method for recognizing text in an image according to claim 4, wherein said object referable to said geometric primitives is selected from the group comprising bottles, cans, cans. 24. Method for recognizing text in an image according to claim 8, wherein said object with said perspective primitives having parallepipedic form is chosen from the group including 1-leaf books, 1-leaf books 2 panels, brochures, newspapers, magazines, envelopes, signs, synoptic tables. 25. Method for recognizing text in an image in accordance with any of the previous claims 1 to 24, wherein said at least one region containing text (3B) within said modified image (3A) Ing. Aurelio PERANI Registered no. ALBO 277 (on his own and for others) if said first reliability value (Cl) is close to zero, it coincides with said modified image (3A). 26. Computer product that can be directly loaded into the memory of a computer, comprising portions of program code capable of carrying out the method according to any one of claims 1 to 25 when run on said computer. 27. Apparatus for recognizing text in an image (3), in which an object (4) is represented in said image (3), said apparatus comprising: - a processing device (1) having means for processing a digital image (3) to produce a first modified image (3A); - means (7) for carrying out an optical character recognition (OCR) on said first modified image (3A) to determine at least one region containing text (3B) inside said modified image (3A) and to associate with said first region (3B) a first reliability value (C 1); -means (8) for converting said recognized text into a first voice signal (S1), said voice signal (SI) being emitted through an audio interface (9) operatively connected to said processing device (1); characterized in that said means (7) for carrying out an optical character recognition comprise a program in accordance with any one of claims 1 to 25. 28. Apparatus for recognizing text in an image, according to claim 27, wherein said means (7) for acquiring said digital image are operatively connected with said processing device (1). 29. Apparatus for recognizing text in an image, according to any one of claims 27 to 28, wherein said means (7) for performing a Ing. Aurelio PERANI N. lscr. ALBO 277 (on their own and by others) optical character recognition (OCR) on said first modified image (3A) are able to generate a first reliability value (Cl) for said at least one region containing text (3B), 30. Apparatus for recognizing text in an image, according to claim 27, wherein said processing means are adapted to compare said first reliability value (Cl) with a predetermined reliability value (C2) and to select at least one recognition strategy (Xj; ..., X6) among a plurality of recognition strategies if said first reliability value (Cl) is lower than said predetermined reliability value (C2), each of said plurality of recognition strategies (Xs ; ..., X6) being able to modify the graphic parameters of said first modified image (3A) to generate a second modified image (3C) as long as said first reliability value (Cl) is close to or higher than said predetermined value of reliability (C2). 31. Apparatus for recognizing text in an image, according to any one of claims 27 to 30, wherein said processing device (1) comprises a computer, a notebook, a palmtop, a PDA or a telecommunications system such as a mobile phone. Ing. Aurelio PERANI N. lscr. ALBO 277 (on his own and for others)