JP5160366B2 - Pattern matching method for electronic parts - Google Patents

Description

  The present invention relates to an electronic component pattern matching method, and more particularly to an electronic component pattern matching method that is suitable for use in positioning and direction determination by image recognition of an electronic component having a plurality of corners such as a rectangular electronic component.

  In general, in a surface mounting apparatus that mounts electronic components on a substrate, the target electronic component picked up and held by a suction nozzle mounted on an XY moving mounting head is imaged by a lower component recognition camera, and the obtained target Image processing is performed, and alignment is performed by correcting the positional deviation in the XY direction and the rotation direction about the nozzle by image recognition, and the image is accurately mounted on the target position on the substrate.

  As an alignment (position recognition) technique based on such image recognition (image processing), there is template matching. In this template matching, if the matching calculation for evaluating the degree of coincidence is performed on the data of all the target pixels, the calculation amount becomes enormous. Therefore, the calculation speed is increased by narrowing down the target pixels under some conditions. Conventionally, a method for achieving this has been adopted.

  For example, in Patent Document 1, as a method of selecting a strong edge strength, an edge detection filter such as a Sobel is used to obtain the edge strength, and a threshold value is set for the value to thin out the target. A technique for narrowing down the pixels to be is disclosed.

  Patent Document 2 discloses a method in which a portion with a large curvature of an extracted edge point sequence is dense and a portion with a small curvature is rough, and the spatial distribution on the reference image and the directionality of the edge are somewhat uniform. Techniques relating to methods such as thinning out so as to be similar are disclosed.

Japanese Patent No. 2,689,688 Japanese Patent No. 2981382

  However, as disclosed in Patent Document 1, in the method of simply performing the thinning process based on the edge strength, the edge of the characteristic shape portion may be missed when the edge is detected locally. is there.

  In addition, since Patent Document 1 does not consider the change in illumination conditions such as illuminance, the same edge may not be detected even though the same part is targeted, so that the edge detection accuracy is not stable. For this reason, there is a problem that the accuracy of pattern matching is lowered.

  It is conceivable to deal with this by thinning out the edges at equal intervals. In this case, however, edge points are detected uniformly from the entire image. The edge of the shape portion is lost, and matching accuracy does not increase. However, if the thinning interval is narrowed in order to extract the edge of the characteristic shape portion, the speeding up by the thinning process cannot sufficiently function.

  In addition, the edge point sequence disclosed in Patent Document 2 is a method in which a portion with a large curvature is dense and a portion with a small curvature is rough, especially when a density gradient vector is used for evaluation of similarity. The edge point sequence in the portion with a large curvature is not suitable because the accuracy of the vector direction is not good because the accuracy of the vector direction is not good.

  Here, the difference in density gradient vector between the portion with a large curvature and the straight portion will be specifically described with reference to FIG.

  FIG. 8A is an enlarged view of the periphery of a corner obtained by imaging a rectangular part, and FIG. 8B is an image of the same part, where the contrast is lower than that of FIG. It is an example. FIG. 8 shows an image of a multi-value image in which the pixel density changes from black to white.

  In both cases of FIGS. 8A and 8B, the direction of the density gradient vector indicated by the arrow is stably in the same direction at the edge of the straight line portion. However, it can be seen that the edge of the corner portion having a large curvature is unstable because the direction of the gradient vector changes with the change in the surrounding density value due to the difference in contrast.

  The present invention has been made to solve the above-described conventional problems, and performs pattern matching for recognizing the position of an electronic component based on a target image obtained by imaging an electronic component having a plurality of corners such as a rectangular shape. It is an object of the present invention to provide a pattern matching method for an electronic component that can be performed at high speed and with high accuracy.

  The present invention captures an electronic component having a plurality of corners to acquire a reference image, extracts each corner from the acquired reference image, and edge from a predetermined extraction region set based on each extracted corner From the extracted edges, only the straight edges excluding the corner vicinity are selected, the selected straight edges are saved as template data, and the target image is captured by capturing the same type of electronic components. Acquire and extract each corner from the acquired target image, extract an edge from a predetermined extraction area set based on each extracted corner, and from the extracted edge, only the linear edge excluding the corner vicinity And performing the pattern matching for matching the target electronic component based on the selected linear edge and the template data, Resolve those were.

  In the present invention, the selection of the straight edge is performed by using a difference between tangent angles of two points before and after an arbitrary distance with respect to an arbitrary point of an edge extracted from a predetermined extraction region set with reference to the corner. The determination may be made based on the above.

  In the present invention, the degree of coincidence between the linear edge extracted from the target image and the template data may be evaluated based on a density gradient vector at each edge point.

  In the present invention, the rough positioning of the target part is performed based on the corner coordinates extracted from the target image and the corner coordinates extracted from the reference image, and the target part is positioned with high accuracy. You may make it perform based on the density | concentration gradient vector calculated about a linear edge.

  According to the present invention, only a straight edge in the vicinity of a corner is extracted from a target image obtained by imaging an electronic component having a plurality of corner portions such as a rectangular shape, and the edge is compared with template data. Therefore, pattern matching can be performed at high speed and with high accuracy.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing an outline of an image recognition system applied when a pattern matching method according to an embodiment of the present invention is performed.

  This system includes a suction nozzle 1 for sucking and holding electronic components, a standard camera 4 and a high-resolution camera 5 for imaging an electronic component 2 held by the suction nozzle 1 under illumination by a lighting device 3, An image processing device 13 that processes an image obtained by imaging an electronic component set at an imaging position to detect the position and shape of an object existing at an arbitrary position in the image, and together with the image processing device 13 It is comprised by the machine control apparatus 14 which controls operation | movement of the suction nozzle 1, the illuminating device 3, etc. FIG.

  The image processing device 13 controls the designated camera 4 or 5, takes an image of the electronic component 2, digitizes the image signal by the A / D converter 6, and stores it in the image memory 7 as multivalued image data. Let

  Further, the image processing device 13 performs various processes on the data in the image memory 7 by the calculation unit 9 to create template data for positioning and shape discrimination of the electronic component 2 and in the template data storage memory 10. save. The process data generated during the process is stored in the work memory 8. The control unit 11 controls operations on the arithmetic unit 9 and the memories 8 and 10.

  The machine control device 14 selects either the camera 4 or 5 to be used for imaging according to the electrode size or the like of the normal electronic component via the interface 12 and holds the electronic component 2 by suction suction 1 and selects it. Set to the imaging position of the selected camera. Further, the illumination device 3 is moved and lit so that an image is picked up by the selected camera, and the image processing device 13 is instructed to execute processing together with the selected camera channel information via the interface 12.

  Next, referring to the flowchart of FIG. 2, description will be given of the creation of template data as a reference when performing component position recognition by pattern matching. Details of each step will be described later.

  First, a reference rectangular image is set (captured) at a predetermined camera position, and a reference image is captured (acquired) and stored in the image memory 7 (step 1). Next, corner extraction processing is performed on the reference image, and the corner extraction data is stored in the work memory 8 (step 2).

  Next, edge extraction processing is performed only on a certain area centered on the extracted arbitrary corner, and the obtained edge extraction data is stored in the work memory 8 (steps 3 and 4). The edge extraction process in steps 3 and 4 is repeated for the number of detected corners. Edges other than the extracted corner periphery (neighborhood) are selected, and the straight edge formed by the edge point sequence of each selected predetermined area is stored as template data in the template data storage memory 10 (step 6), and data is created. Exit.

  Next, details of individual processing executed by the image processing apparatus 13 when creating template data according to the flowchart of FIG. 2 will be described.

Step 1: Reference Image Capture The image processing device 13 controls the designated standard camera 4 or high-resolution camera 5, captures an image of the electronic component 2, digitizes it with the A / D converter 6, and stores it in the image memory 7. Store as image data.

Step 2: Corner extraction processing Corner extraction processing is performed on the entire area.

  Corner extraction can be performed using a corner detection operator such as Harris or SUSAN.

  Since the corner is an invariant feature, if the same operator is applied to the same part, it can be detected stably without being affected by rotation or scale fluctuation.

  The corner extraction result is stored in the work memory 8 as coordinate data.

Step 3: Edge extraction area setting A constant area having a predetermined radius centered on the detected (extracted) corner coordinates is set as an edge extraction area.

  Thus, by setting an edge extraction region having a predetermined radius and performing edge extraction processing locally, the time required for edge extraction can be shortened.

  In addition, since the edge extraction region is set based on the corner, the edge can be stably extracted without being affected by the change in the illumination condition. That is, by extracting edges based on corners that are invariant features, thinning processing based on edge strength is not required, so that edges can be stably extracted without being affected by changes in illumination conditions. .

  The size of the area (predetermined radius) may be specified by the user, or may be calculated and set for each part from the outer size, the corner curvature, and the like.

Step 4: Edge extraction processing Edge extraction processing is performed on the area specified (set) in Step 3.

  Edge extraction can be performed using an edge detection filter such as a Sobel.

  The edge extraction result is stored in the work memory 8 as two-dimensional binary data.

  The region designation (step 3) and edge extraction processing (step 4) are repeated for the number of detected corners.

Step 5: Edge Selection Processing FIG. 3 shows an enlarged view of the corner portion of the reference image G.

  An edge E within a certain region (in the great circle B) corresponding to the predetermined radius is selected except for the edge (in the small circle A) in the immediate vicinity (in the vicinity of the corner) including the detection corner C.

  In order to distinguish between the immediate vicinity of the corner and the stable area outside the corner, for example, as shown in FIG. 4, for any point on the contour line (edge point sequence), at two points before and after an arbitrary distance, respectively. If the angle difference between the two tangents when calculating the tangent is within the set threshold, it is determined as a point on the straight line, and if it exceeds the threshold, it is determined as a point on the curve. A method of determining a point on a straight line as a straight edge in a stable region can be used by clearly separating the straight line part from the R part of the corner. Incidentally, FIG. 4A shows an image when the difference in the tangent angle exceeds the threshold, and FIG. 4B shows an image when the difference is within the threshold.

Step 6: Template Data FIG. 5 shows a configuration example of template data.

  The density gradient vector constituting the template data is calculated for the edge point sequence of the straight line portion selected from the stable region. The concentration gradient vector can be obtained by the same method as in Patent Document 2.

  This template data is created and stored in the template data storage memory 10.

  The machine control device 14 can control reading and writing of template data via the interface 12.

  Through the processing of each of the above steps, it is possible to prevent missing characteristic shape parts when extracting edges from the reference image.

  FIG. 6 shows an image of an example of an edge detected (extracted) for a rectangular part having four corners captured in the reference image. A total of eight thick line segments for each corner represent a straight edge E.

  After the above template data is created, positioning by pattern matching for the target part will be described with reference to the flowchart of FIG.

  Similarly, the same type of electronic component as a target is set at a predetermined camera position, a target image is captured and stored in an image memory (step 101), and a corner is applied to the target image as in the case of the reference image. Extraction processing is performed, and the corner extraction data is stored in the work memory 8 (step 102).

  A rough search is performed by comparing the corner extracted from the reference image with the corner extracted from the target image, and rough positioning is performed (step 111).

  Next, as in steps 3 and 4, edge extraction processing is performed only within a certain area centered on the corner, and the edge extraction data is stored in the work memory 8 (steps 103 and 104). The edge extraction processing in steps 103 and 104 is repeated for the number of detected corners.

  Next, an edge excluding the corner periphery extracted for each corner is selected (step 105), and the degree of coincidence in the density gradient vector direction between the template data read from the storage memory 10 and the edge selected from the target image is calculated. The position where the degree of coincidence is maximum is set as the matching result (step 112).

  Next, details of individual processing executed by the image processing apparatus 13 when positioning is performed according to the flowchart of FIG. 7 will be described in detail.

  Step 101: Reference image capturing and Step 102: Corner extraction processing are substantially the same as those in Steps 1 and 2, except that the object to be imaged is the same type of electronic component and the reference image is replaced with the target image. Since it is the same, description is abbreviate | omitted.

Step 111: Coarse search Rough positioning is performed based on the relative positional relationship between the corner position extracted from the reference image and the corner position extracted from the target image by a technique such as Geometric Hashing. At this time, only corner coordinate information is used, and no gradient vector is used. Since corners are easily associated with each other, coarse search can be performed at high speed by using the coordinate information of the corners.

  Since Step 103: Edge extraction region setting, Step 104: Edge extraction processing, and Step 105: Edge selection processing are substantially the same as Step 3, Step 4, and Step 5, description thereof will be omitted.

Step 112: Fine search The template data is affine transformed (scale fluctuation, rotation, translation) and superimposed on the density gradient vector data of the target image.

  In the present embodiment, as shown in the image in FIG. 8 above, by extracting only the linear edge points excluding the corners where the density gradient vector is stable, high-precision matching is performed. Can be realized. Furthermore, by using a limited range of edges only near the corner as a feature point, it is possible to perform a fine search at high speed.

  According to the embodiment described above in detail, the following effects can be obtained.

(1) Since edge data in a limited range is used, there is no need for thinning out as in the prior art, and therefore it is possible to prevent missing characteristic shape portions due to thinning out.

(2) Since the edge of the predetermined area is extracted with the corner as a reference, the feature point can be stably obtained regardless of the illumination condition.

(3) Since the edge is extracted with reference to the corner, the feature point can be stably obtained even if the target image is changed, such as rotation or scale fluctuation.

(4) By using the edge point of the straight line portion where the change in the direction of the density gradient vector is small as the feature point, it is possible to obtain a stable degree of coincidence and improve the repeatability of the positioning result.

(5) By narrowing the feature points to a limited range, it is possible to speed up the positioning process.

(6) The feature point extraction process can be speeded up by narrowing down the edge extraction region.

(7) By using the corner coordinates, it is possible to increase the speed of the coarse search for rough alignment.

  In the above embodiment, a rectangular electronic component is taken as a component having a plurality of corners. However, the present invention is not limited to this, and any electronic component having two or more corners may be used as long as the positional relationship of the corners can be specified. That's fine.

The block diagram which shows the outline | summary of the image recognition system applied to the pattern matching of this invention Flow chart showing the procedure for creating template data Image diagram showing edges around extracted corners Explanatory drawing which shows the method of determining the linearity of an edge Chart showing an example of the data structure of template data Image diagram showing detected edge data The flowchart which shows the processing procedure of the positioning by this embodiment Explanatory drawing which shows the characteristic of the density gradient vector in the corner of the imaged image

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Suction nozzle 2 ... Electronic component 3 ... Illuminating device 4 ... Standard camera 5 ... High resolution camera 6 ... A / D converter 7 ... Image memory 8 ... Working memory 9 ... Calculation part 10 ... Template data storage memory 11 ... Control part DESCRIPTION OF SYMBOLS 12 ... Interface 13 ... Image processing apparatus 14 ... Machine control apparatus

Claims (4)

Capture an electronic component having multiple corners to obtain a reference image,
Extract each corner from the acquired reference image,
Edges are extracted from a predetermined extraction area set based on each extracted corner,
From the extracted edges, select only the straight edges excluding the corner neighborhood,
While saving the selected straight edge as template data,
Capture the same type of electronic components to obtain the target image,
Extract each corner from the acquired target image,
Edges are extracted from a predetermined extraction area set based on each extracted corner,
From the extracted edges, select only the straight edges excluding the corner neighborhood,
A pattern matching method for an electronic component, which performs pattern matching for collating a target electronic component based on the selected linear edge and the template data.   The selection of the linear edge is determined based on the difference between the tangent angles of two points before and after an arbitrary distance with respect to an arbitrary point of an edge extracted from a predetermined extraction region set with reference to the corner. The pattern matching method for an electronic component according to claim 1, wherein the pattern matching method is performed.   2. The electronic component pattern matching method according to claim 1, wherein the degree of coincidence between the linear edge extracted from the target image and template data is evaluated based on a density gradient vector at each edge point. Coarse positioning of the target part is performed based on corner coordinates extracted from the target image and corner coordinates extracted from the reference image;
The electronic component pattern matching method according to claim 3, wherein the target component is positioned with high accuracy based on a density gradient vector calculated for the linear edge.