TWI433529B - Method for intensifying 3d objects identification - Google Patents

Description

Enhanced method for identifying 3D objects

The present invention relates to a method for recognizing a 3D object, and more particularly to a method for separating at least two aggregated 3D objects by Gaussian filtering and a watershed segmentation algorithm to enhance recognition of the 3D object.

Using a stereo camera to capture an image containing a 3D object, a disparity map between the 3D object and the background can be obtained. The user can let the stereo camera erect and simulate the plan view to understand the position and movement of the 3D object in space. Therefore, the 3D object recognition system can utilize the above 3D object characteristics to apply to a head count or detect a person flow.

However, when identifying 3D objects (such as people), it is inevitable that two or more 3D objects will always be gathered together. At this time, the 3D objects on the plan view will also be more than two. 3D objects (eg, two people) come together, making the 3D object recognition system easy to misjudge as a single 3D object (single person), causing errors in counting 3D objects.

The invention provides a method for enhancing recognition of a 3D object, comprising: capturing a left eye image and a right eye image by using a left eye camera and a right eye camera; correcting the left eye image and the right eye image to generate a corrected image a left eye image and a corrected right eye image; using the corrected left eye image and the corrected right eye image to generate a disparity map; according to the disparity map, determining the disparity map a 3D object different from the background; projecting the 3D object different from the background onto a top view; filtering out the noise on the top view to generate a de-noising 3D object; determining whether the de-noising 3D object is at least The two 3D objects are gathered; and if the de-noising 3D object is assembled from the at least two 3D objects, the at least two 3D objects are separated.

The invention provides a method for enhancing recognition of a 3D object by projecting a 3D object in a parallax map generated by a left eye image and a right eye image onto a top view. Then, the Gaussian filter is used to remove the noise of the 3D object and enhance the contour features of the 3D object. Finally, at least two or more 3D objects gathered together are separated by a watershed cutting method.

Please refer to FIG. 1. FIG. 1 is a flow chart showing a method for enhancing recognition of a 3D object according to an embodiment of the present invention. The steps of FIG. 1 are detailed as follows: Step 100: Start; Step 102: Use a left-eye camera LC and a right-eye camera RC to capture a left-eye image LI and a right-eye image RI; Step 104: Correct the left eye The image LI and the right eye image RI are used to generate a corrected left eye image CLI and a corrected right eye image CRI; Step 106: using the corrected left eye image CLI and the corrected right eye image CRI to generate a parallax Disparity map; Step 108: Determine a different 3D object in the disparity map according to the disparity map; Step 110: Project the 3D object different from the background onto a plan view; Step 112: Filtering the noise on the top view to generate a de-noising 3D object; step 114: determining whether the de-noising 3D object is aggregated by at least two 3D objects; and step 116: if the de-noising 3D object is at least two When the 3D objects are gathered, at least two 3D objects are separated; Step 118: End.

In step 104, the left eye image LI and the right eye image RI are corrected by using the correction parameter to generate a corrected left eye image CLI and a corrected right eye image CRI, wherein the correction parameter includes the left eye obtained offline. The distance B between the camera LC and the right-eye camera RC, and the left-eye camera LC and the right-eye camera RC capture the left-eye image LI and the right-eye image RI in a synchronized manner. In step 106, a disparity map is generated using the corrected left eye image CLI and the corrected right eye image CRI. In the parallax map, the depth D of the reference line baseline in which the 3D object is located from the left-eye camera LC and the right-eye camera RC can be generated by the distance B between the left-eye camera LC and the right-eye camera RC. Please refer to FIG. 2 and FIG. 3 . FIG. 2 is a schematic diagram showing overlapping of corrected left eye image CLI and corrected right eye image CRI to generate parallax dx, and FIG. 3 illustrates using left eye camera LC and right. The eye camera RC produces a schematic view of the depth of view D. As shown in Fig. 2, the imaging position PL(XL, YL) is the position of a 3D object in the corrected left-eye image CLI, and the imaging position PR (XR, YR) is the corrected right of the 3D object. The position in the ocular image CRI, wherein the 3D object contains the background of the corrected left eye image CLI and the corrected right eye image CRI that does not move. Therefore, the parallax dx can be generated according to the imaging positions PL(XL, YL), PR(XR, YR), and the equation (1).

Dx=XR-XL (1)

As shown in FIG. 3, the view of the 3D object can be generated based on the parallax dx, the distance B between the left-eye camera LC and the right-eye camera RC, the focal length f of the left-eye camera LC and the right-eye camera RC, and the equation (2). Deep D, which is the Z coordinate of the 3D object. The 3D object here includes any 3D object of the corrected left eye image CLI and the corrected right eye image CRI.

D=Z=f*(B/dx) (2)

In step 108, according to the disparity map obtained in step 106, it is determined that the 3D object in the disparity map is different from the background, especially in the corrected left eye image CLI and the corrected right eye image CRI. Background 3D objects. Since the depth of field of the background does not change, the disparity map obtained in step 106 can be used to determine a different 3D object in the disparity map. After obtaining the depth of view Z of the 3D object, the X coordinate and the Y coordinate of the 3D object can be generated according to the formulas (3) and (4). Thus, the stereoscopic information of the 3D object, that is, the three-dimensional coordinates (X, Y, Z) of the 3D object, can be obtained from the two image planes of the left-eye camera LC and the right-eye camera RC, wherein the equation (3) XL and YL of the formula (4) may be substituted by XR and YR.

X=(XL*Z)/f (3)

Y=(YL*Z)/f (4)

In step 110, the user can obtain the positional information of the 3D object on the plane by using the top view. After the left-eye camera LC and the right-eye camera RC are used to generate the stereoscopic information of the 3D object, the 3D object different from the background is first projected onto the 3D plan view, and then the 3D object is observed at a vertical ground angle. Please refer to FIG. 4, which is a schematic view showing a 3D object in a plan view.

In the disparity map, the point of each disparity map gives a projected weight value. The projection weight value calculation method F ( f _x , f _y , Z _cam ) proposed by the present invention gives a point of each projection weight value to each parallax map according to the concept that the weight of the 3D object is farther away from the baseline line. After accumulating these projection weight values, each point on the top view will judge whether it is a noise or a 3D object according to the amount of accumulated amount. The larger the accumulated weight value, the point should be a 3D object.

In step 112, according to the reference, the removal of the noise of the 3D object can be accomplished by using the height information, the projection weight value, and the Gaussian filter, wherein the projection weight value is improved by the step 110, and the user can determine the projection. The point with less weight value should be noise.

In addition, the height information is one of the information of each point on the top view, which is used to indicate the height of each point in space. When a 3D object (such as a person) is projected onto a top view, the height information of the 3D object usually has a shape similar to a mountain, and the Gaussian filter itself has a mountain-like appearance, so the user uses a Gaussian filter instead. In addition to removing noise from 3D objects, it can also be used to enhance the contour features of 3D objects for later identification of 3D objects.

When 3D objects (such as people) are brought together, the objects on the top view will also get together, resulting in frequent misjudgment into the same 3D object. Thus, in step 112, a Gaussian filter has been utilized to enhance the contour features of the 3D object, particularly the "mountain" feature. Therefore, in step 114, the user can use the "find area extreme value" method to find the "mountain" to see if a 3D object may be merged from two or more 3D objects (that is, have more than two " The top of the mountain"). Usually there is only one 3D object, and there is only one "mountain". When two "mountains" are found above, it means that there may be more than two 3D objects being together.

In step 116, when it is determined that a 3D object (e.g., a person) has more than two "mountains", it is possible that at least two or more 3D objects are merged. At this time, the 3D object recognition system may first separate at least two 3D objects by using a watershed segmentation algorithm. Then, according to the range of the 3D object after cutting, it is judged whether it is a 3D object or a noise. Therefore, as long as the range of the 3D object after cutting is large enough, the 3D object recognition system will be considered as a 3D object, otherwise it is noise.

Referring to FIGS. 5A and 5B, FIG. 5A is a schematic diagram of an image presented according to the luminance value of each pixel, and FIG. 5B is a schematic diagram illustrating separation of at least two 3D objects by a watershed cutting method. As shown in Fig. 5A, the white area is a region with a large brightness, and the area with the closest dot is a region with a small brightness. The main concept of the watershed cutting method is to treat the entire image as a topographic map. The brightness of each pixel is the height of the terrain. From this topographic map, the watershed is found to separate the 3D objects. As shown in FIG. 5B, all the watershed lines, that is, the boundaries of at least two 3D objects, can be found according to the brightness values of the pixels of FIG. 5A to distinguish the 3D objects on the top view.

Please refer to FIG. 6. FIG. 6 is a flow chart showing a method for enhancing recognition of a 3D object according to another embodiment of the present invention. The steps of FIG. 6 are detailed as follows: Step 600: Start; Step 602: Use a left-eye camera LC and a right-eye camera RC to capture a left-eye image LI and a right-eye image RI; Step 604: Correct the left eye The image LI and the right eye image RI are used to generate a corrected left eye image CLI and a corrected right eye image CRI; step 606: sharpen the corrected left eye image CLI and the corrected right eye image CRI to generate a sharpened left eye image SLI and a sharpened right eye image SRI; step 608: calculating a variability of the sharpened left eye image SLI and the sharpened right eye image SRI; step 610: utilizing Sharpening the left eye image SLI and the sharpened right eye image SRI to generate a disparity map; step 612: determining, according to the disparity map, a background-like 3D object (eg, a person) Step 614: Projecting a 3D object different from the background onto a plan view; Step 616: Filtering the noise on the top view to generate a de-noising 3D object; Step 618: Determining the noise-free 3D object Whether it is composed of at least two 3D objects; and step 620: If the 3D object to the noise collected by the system from the at least two 3D object, the 3D object is at least two separation; Step 622: End.

The difference between the embodiment of Fig. 6 and the embodiment of Fig. 1 is that the embodiment of Fig. 6 has two more steps. In step 606, the corrected left eye image CLI and the corrected right eye image CRI use a high pass filter to obtain the high frequency features of the corrected left eye image CLI and the corrected right eye image CRI. To enhance the high frequency portion of the corrected left eye image CLI and the corrected right eye image CRI, such as the corrected left eye image CLI and the corrected right eye image CRI edge and/or pattern portion. In step 608, the variability of the sharpened left eye image SLI and the sharpened right eye image SRI is calculated to identify and remove smooth, featureless regions, such as the entire wall in the background. The 3D object recognition system is used to identify 3D objects (such as people), but the smooth featureless area in the background is not a 3D object that needs to be recognized by the 3D object recognition system, so the smooth and featureless area in the background is removed, reducing 3D. The burden of the object identification system. The remaining operating principles of the embodiment of FIG. 6 are the same as those of the embodiment of FIG. 1, and are not described herein again.

In summary, the method for enhancing recognition of a 3D object provided by the present invention projects a 3D object in a parallax map generated by a left eye image and a right eye image onto a top view. Then, the Gaussian filter is used to remove the noise of the 3D object and enhance the contour features of the 3D object. Finally, at least two or more 3D objects gathered together are separated by a watershed cutting method. Therefore, the present invention can be applied to places where a flow of people is required or a flow of people is detected, such as a hypermarket, a movie theater, a department store, and the like. The present invention can also be applied to a platform warning line or other security zone, and as soon as a 3D object (for example, a person) is detected, an alarm is immediately sounded.

The above are only the preferred embodiments of the present invention, and all changes and modifications made to the scope of the present invention should be within the scope of the present invention.

CRI. . . Corrected right eye image

CLI. . . Corrected left eye image

PL, PR. . . Imaging position

B. . . distance

D. . . Depth of field

LC. . . Left eye camera

RC. . . Right eye camera

100-118, 600-622. . . step

1 is a flow chart illustrating a method of enhancing recognition of a 3D object according to an embodiment of the present invention.

Figure 2 is a schematic diagram showing the generation of parallax by superimposing the corrected left eye image and the corrected right eye image.

Fig. 3 is a view showing the use of a left-eye camera and a right-eye camera to generate a depth of view.

Figure 4 is a schematic diagram showing the use of a 3D object on a top view.

Figure 5A is a schematic diagram of an image presented based on the luminance values of each pixel.

Figure 5B is a schematic diagram illustrating the separation of at least two 3D objects using a watershed cutting process.

Figure 6 is a flow chart illustrating a method of enhancing recognition of a 3D object in accordance with another embodiment of the present invention.

600-622. . . step

Claims (9)

A method for enhancing recognition of a 3D object, comprising: using a left eye camera and a right eye camera to capture a left eye image and a right eye image; correcting the left eye image and the right eye image to generate a corrected left An eye image and a corrected right eye image; using the corrected left eye image and the corrected right eye image to generate a disparity map; and determining, according to the disparity map, one of the disparity maps a 3D object that is different from the background; a corresponding weight is given for each point in the 3D object different from the background to project the background-specific 3D object onto a plan view, wherein the corresponding The weight is related to the distance between the left eye camera and the right eye camera and the point; filtering the noise on the top view to generate a denoising 3D object; determining whether the denoising 3D object is at least two 3D objects are gathered; and if the de-noising 3D object is assembled from the at least two 3D objects, the watershed segmentation algorithm is used to separate according to the height information of the de-noising 3D object on the top view. The Less two 3D objects. The method of claim 1, wherein the capturing the left eye image and the right eye image is to capture the left eye image and the right eye image in a synchronized manner. The method of claim 1, wherein the correcting the left eye image and the right eye image is to correct the left eye image and the right eye image according to a distance between the left eye camera and the right eye camera. The method of claim 1, further comprising: sharpening the corrected left eye image and the corrected right eye image to generate a sharpened left eye image and a sharpened right eye image; And calculating the variability of the sharpened left eye image and the sharpened right eye image. The method of claim 4, wherein sharpening the corrected left eye image and the corrected right eye image is to enhance the corrected left eye image and the corrected right eye image by using a high pass filter The high frequency part. The method of claim 1, wherein the corrected left eye image and the corrected right eye image are used to generate the parallax map to overlap the corrected left eye image and the corrected right eye image, The parallax map. The method of claim 1, wherein determining the background-specific 3D object in the disparity map is to determine the disparity map by using a background difference between the background in the disparity map and the 3D object different from the background. This is different from the background 3D object. The method of claim 1, wherein filtering the noise system on the top view utilizes A Gaussian filter filters out the noise on the top view. The method of claim 1, wherein the separating the at least two 3D objects according to the watershed method is to find a boundary of each of the at least two 3D objects according to a brightness value of each pixel on the at least two 3D objects, to separate The at least two 3D objects.