JP6717769B2 - Information processing device and program - Google Patents

Description

The present invention relates to an information processing apparatus and a program capable of obtaining an appropriate feature point from an image for object determination based on a feature amount and other uses.

The technique of determining an object from an image can convert analog information described in a medium that can be easily distributed and presented into digital information, and can improve user convenience. For example, Non-Patent Document 1 is disclosed as the technology. In Non-Patent Document 1, a feature point is detected from an image, a feature amount is calculated from the periphery of the feature point, and then the feature amount is compared with a feature amount that has been accumulated in advance. Specify.

On the other hand, as a technique for further improving the accuracy of the determination based on the characteristic points and the characteristic amounts as described above, Patent Documents 1 to 3 are disclosed. In Patent Document 1, in order to reduce erroneous correspondence of feature points, accuracy is improved by Delaunay triangulation with feature points as vertices and topology evaluation. Patent Document 2 discloses a method of selecting an effective feature amount by evaluating the feature amount discrimination performance. In Patent Document 3, by associating the similarity of the positional relationship between the feature points and the edges, the feature points are highly accurately associated.

JP, 2014-127068, A JP, 2014-134858, A JP, 2014-126893, A

D. G. Lowe, ``Object recognition from local scale-invariant features,'' Proc. of IEEE International Conference on Computer Vision (ICCV), pp.1150-1157, 1999.

However, the conventional techniques described in Non-Patent Document 1 and Patent Documents 1 to 3 described above have a problem in that the distribution of feature points greatly affects the accuracy.

Specifically, when the background has a complicated pattern, the feature points are concentrated on the background, and the determination process does not function at all unless a sufficient number of feature points can be detected from the imaging target as the foreground. For example, considering a case where a framework is adopted in which an upper limit is set for the number of feature points from the viewpoint of processing time and the feature points are defined from the top of the scores (feature point scores), the imaging target is considered. Even if a feature point exists, the feature point to be imaged may not be selected if the feature point score is relatively small. This is called the first case.

On the other hand, if there is a pattern with a large feature point score in the imaged object itself, the feature points are selected up to the upper limit in only some areas, and it is erroneously determined as a similar object that matches only in that area. It is possible. This is called the second case.

FIG. 1 is a diagram showing schematic examples of the above-mentioned first case and second case as [1] and [2].

That is, in the first case shown in [1] in FIG. 1, it is difficult to distinguish two or more different objects given as the images P1 and P2. For example, images P1 and P2 are both covers of books belonging to the same series (series explaining hobbies) of the same publisher, the book of image P1 describes mountaineering, and the book of image P2 describes cycling in the same series. It is assumed that this is a separate volume. In this case, in the images P1 and P2, series title regions R10 and R20 indicating that they are the same series are present in the upper side portion, and these regions R10 and R20 are completely the same pattern (including a character and the like are referred to as a pattern). Areas R11 and R21 that have different patterns and that represent the content of each volume because they are separate volumes in the series are present in the middle and lower stages. I shall.

Here, in both images P1 and P2, it is assumed that the series title regions R10 and R20 having the completely same pattern are configured as regions with high feature point scores, and regions R11 and R21 that configure different patterns. Assuming that it is configured as a region having a low feature point score, all or most of the feature points are detected from the regions R10 and R20 having the same pattern, and no feature points are detected from the different regions R11 and R21. Only a small amount will be detected. Therefore, the images P1 and P2 that should actually image different objects become indistinguishable or difficult to distinguish.

In the second case shown as [2] in FIG. 1, the same image P1 as described in the first case is registered as the reference image, and the query image obtained by actually photographing the same target is like the image P10. If it is configured as described above, it becomes difficult to determine the matching of the images P1 and P10. That is, the query image P10 forms a region R13 that does not exist in the reference image P1 by adding, for example, a “belt” to the lower part of the book cover for the purpose of sales promotion. The area R12 on the middle side of the query image P10 partially matches the area R11 on the middle/lower side of the reference image P1, and the series title areas R10 and R30 on the upper side are the reference image P1 and the query image P10. Is common in.

In such a case, it is assumed that the “belt” region R13 that does not exist in the reference image P1 in the query image P10 is configured as a region having a high feature point score, and all or most of the feature points in the query image P10 are regions. If it is detected from R13, for the same reason as explained in the first example of [1], it is possible to obtain the same judgment of the images P10 and P1 which should actually be capturing the same target. Becomes difficult.

Note that the book cover is shown as a schematic example in FIG. 1, but it should be noted that similar or similar problems may occur not only for the book cover but also for any type of product package and other arbitrary objects. Further, in FIG. 1, the description has been made by focusing on only one determination target such as a book cover as a schematic example, but the same or similar also when one or more targets are imaged in a spatial arrangement in the image. Please note that the following problems may occur.

In view of the problems of the conventional technology as described above, it is a first object of the present invention to provide an information processing apparatus capable of obtaining an appropriate feature point from an image for object determination based on a feature amount and other uses. ..

A second object of the present invention is to provide an information processing device capable of highly accurately determining an object in an image based on the appropriate feature point.

In order to achieve the above-mentioned object, the present invention is an information processing apparatus, which detects a feature point from an image and obtains a feature point score of each feature point, and from among the detected feature points, A first feature is that a selection unit that outputs a feature point obtained by performing selection based on a feature point score and a position of the feature point as a feature point of the image is provided.

Further, in the present invention, the information processing apparatus further refers to a neighborhood area of the feature point in the image with respect to each feature point output as the feature point of the image by the selection unit. In the image, a calculation unit that calculates a feature amount, a feature amount calculated for the image by the calculation unit, and a feature amount calculated from the captured image for each of a plurality of predetermined objects are compared in the image. A second feature is that the image capturing target includes a determination unit that determines which of the plurality of predetermined targets corresponds to the predetermined target.

According to the first feature, an appropriate feature point can be obtained by outputting the feature point obtained by performing selection based on the feature point score and position of each feature point as the feature point of the image. ..

According to the second feature, it is possible to determine which of the plurality of predetermined targets the target imaged in the image corresponds to using the obtained appropriate feature point.

It is a figure which shows the example of a 1st case and 2nd case as a subject of a prior art. It is a functional block diagram of the information processing apparatus which concerns on one Embodiment. It is a figure which shows the captured image and the example of the weight map calculated in inverse proportion to the depth from the captured image. It is a flow chart which shows an example of processing of a sorting part in a second embodiment. It is a figure which shows the example of the effect of this invention as an application example of the flow of FIG. It is a figure which shows the example of the effect of this invention about the case of more characteristic points than FIG.

FIG. 2 is a functional block diagram of the information processing apparatus according to the embodiment of the present invention. As illustrated, the information processing device 10 includes an imaging unit 1, a detection unit 2, a selection unit 3, a calculation unit 4, a determination unit 5, and a storage unit 6. As the information processing device 10, any information terminal including the image pickup unit 1 can be used, and in addition to a mobile terminal, a tablet terminal, a desktop or laptop computer, or the like can be used. In addition, a part or all of the image pickup unit 1 other than the image pickup unit 1 may be installed in the server and information may be appropriately exchanged through communication. The processing content in each part of FIG. 2 is as follows.

The image capturing unit 1 captures an image of an image capturing target and outputs the image to the detecting unit 2, the selecting unit 3, and the calculating unit 4. Here, the user or the like operates the image capturing unit 1 to capture an image so that the image capturing target includes the target determined by the determination unit 5 (one of the targets stored in advance in the storage unit 6). Assumed to be performed. As the imaging unit 1, a digital camera or the like that is standardly equipped in a mobile terminal can be used.

The detection unit 2 detects a plurality of feature points from the image obtained by the image pickup unit 1, obtains a feature point score representing the feature point likelihood for each feature point, and selects the plurality of feature points and the feature point score. Output to part 3. Existing methods such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) can be used to detect the feature points, and a score according to each method may be calculated for calculation of the feature point score. ..

For example, in SIFT, a feature point candidate is obtained as a point that gives the maximum value of a DoG (Difference of Gaussian) image (difference image group), and a stable feature point is detected based on the main curvature and contrast. The DoG output value (maximum value) at the detected feature point may be adopted as the feature point score.

In the selection unit 3, from each feature point with the feature point score obtained in the detection unit 2, a predetermined number of feature points are selected based on the feature point score and the position in the image of the feature point, and the selected feature. The points are output to the calculation unit 4. The sorting unit 3 may further refer to the image obtained by the imaging unit 1 when performing the sorting. Details of the selection processing by the selection unit 3 will be described later.

In the calculation unit 4, for each feature point selected by the selection unit 3, by referring to the region in the local vicinity of each feature point in the image obtained by the imaging unit 1, to calculate the feature amount of each feature point, The characteristic amount is output to the determination unit 5. For the calculation of the feature amount in the calculation unit 4, existing methods such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) can be used, as described in the detection unit 2. The calculation method of the feature amount in the calculation unit 4 and the detection method of the feature point in the detection unit 2 may be different types of methods.

The determination unit 5 compares the feature amount calculated by the calculation unit 4 and the feature amount stored in the storage unit 6 to determine a match, thereby determining the imaging target. That is, it identifies which of the one or more predetermined imaging targets stored in the storage unit 6 corresponds to the target in the captured image.

Here, various well-known methods can be used when comparing the feature amounts. For example, the similarity between histograms of BoVW (Bag of Visual words) may be calculated. Further, the feature information in which the coordinates of the feature points for which the feature amount has been calculated are also associated may be used for comparison. For example, by using RANSAC (Random Sample Consensus) and the like, outliers are eliminated while trying to individually match each feature amount forming the feature information, thereby identifying the best matching item as a whole. Techniques may be used. That is, between the query image (the image obtained by the imaging unit 1) and the reference image (the image in which the feature information stored in the storage unit 6 is extracted), the feature point coordinates associated with each other in the homography matrix The matching may be performed in consideration of the geometrical consistency of.

The storage unit 6 previously calculates and stores the feature amount from each of the captured images of the plurality of image capturing targets, and uses the feature amount as a reference in the determination process of the determination unit 5. Here, artificial transformation such as projective transformation may be added to the imaged target to calculate and store the feature amount. In particular, in the case of deformation in which the size changes, the apparent size of the image captured by the image capturing unit 1 is associated with the depth information (scale information) obtained when the calculating unit 4 calculates the feature amount. Since the effect of improving the amount determination accuracy can be obtained, the feature amount may be stored after being associated with the depth information and used for reference by the determination unit 5. Further, when the feature point coordinates are used in the determination in the determination unit 5 as in the case of the above-described RANSAC, the storage unit 6 also calculates and stores the feature point coordinates and the feature amount as feature information associated with each other. You can do this.

The characteristic amount (and the characteristic point) stored in the storage unit 6 is the same as the characteristic amount (and the characteristic point detected by the detecting unit 2) calculated by the calculating unit 4.

The details of the sorting process in the sorting unit 3 will be described below. First, as a general description, in the selection unit 3, relative to the feature point score of each feature point, relative to the image capturing unit 1 and the image capturing target obtained for each pixel coordinate of the image captured by the image capturing unit 1. By also considering the positional relationship, each feature point can be selected. The relative position relationship to be considered is that the target point reflected at each pixel position (u, v) of the image obtained by the image pickup unit 1 is a camera whose center is the camera center of the camera constituting the image pickup unit 1. It is a relation expressed as a three-dimensional space coordinate position (x,y,z)=(x(u,v),y(u,v),z(u,v)) by coordinates.

By the selection process based on the feature point score and the relative positional relationship in the selection unit 3, as a result, the distribution of the feature points on the image suitable for the determination process in the determination unit 5 on the subsequent stage side is adjusted. .. In particular, in the conventional method, only the feature point score was considered, whereas in the present invention, the distribution adjustment of the feature points suitable for the determination process is realized by further considering the relative positional relationship. Becomes To give a schematic example, the characteristic points are obtained by concentrating only on the regions R10 and R20 in the images P1 and P2 shown in FIG. 1, or the characteristic points are concentrating only on the region R13 in the image P10 of FIG. By preventing such occurrences from occurring, the accuracy of the determination process is improved.

As a first embodiment that also specifically considers the relative positional relationship, the depth (depth) of each pixel position of the captured image is calculated, and then the weight given in the predetermined relationship according to the depth is used as the feature point score. On the other hand, the weight may be additionally used. That is, for each feature point i (i=1,2,...), the depth d(i) at the coordinate position (u(i),v(i)) of that feature point is determined, and the predetermined function F The weight w(i)=F(d(i)) is obtained by using the weighted score w(i)*score( which is obtained by multiplying the feature point score score(i) calculated in the detection unit 2 by the weight. i) is adopted as a score in the present invention for selecting the feature point i, and the weighting score w(i)*score(i) is set to a predetermined number of high-ranking feature points i. The selection result in 3 may be used. The weight w(i) and the feature point score score(i) may be obtained as non-negative values.

Here, for the depth calculation in the captured image, any existing method can be used. For example, an existing method using motion parallax, Markov Random Field (for example, disclosed in Non-Patent Document 2 below), and neural network (for example, disclosed in Non-Patent Document 3 below) can be used. Alternatively, stereo parallax may be used when there are a plurality of cameras forming the image capturing unit 1. Further, it is possible to further use a depth sensor (for example, a depth sensor based on irradiating a marker with infrared rays and capturing an image of the marker with an infrared camera) together with the captured image of the image capturing unit 1.

[Non-Patent Document 2] DA. Saxena and et. al. ``Make3D: Learning 3D Scene Structure from a Single Still Image,'' IEEE Transactions of Pattern Analysis and Machine Intelligence (PAMI), 2008.
[Non-Patent Document 3] D. Eigen and et. al. ``Depth Map Prediction from a Single Image using a Multi-Scale Deep Network,'' Advances in Neural Information Processing Systems, No. 27, pp.2366-2374, 2014.

Further, also regarding the predetermined function F for giving the weight w(i) corresponding to the depth d(i), how the image is captured by the image capturing unit 1 and how the determination is performed by the determining unit 5 (or stored in the storage unit 6). The predetermined value may be set according to the selected feature amount).

For example, if the user is supposed to image an object to be imaged relatively close to the image capturing unit 1, the feature point may be inversely proportional to the depth or may decrease as the depth increases. It is desirable to set a calculation weight (a weight as a positive value under the assumption that the feature point score score(i) is calculated as a non-negative value as described above). FIG. 3 is a diagram showing an example of a captured image and a weight map calculated in inverse proportion to the depth from the captured image, separately from [1] and [2]. The captured image of [1] is an example of an indoor landscape in which various objects (imaging targets) are arranged, and the depth map is small (that is, closer to the front camera) in the corresponding weight map of [2]. A greater weight is given to an object such as a stand or a drawing image, which is expressed as a weight that is closer to white and a weight that is closer to black.

As another weight setting example, the image information may be divided into a foreground and a background according to the depth, and the weight information may be set so that the feature points are selected only from the foreground region. That is, if expressed by an equation, the following binary weights may be set, for example. Here, th is a threshold for distinguishing the foreground and the background, and a predetermined value may be used, or th may be dynamically obtained from a distribution histogram of the depth of the feature points (depth value d(i)). It should be noted that any existing statistical method may be used when dynamically obtaining.
w(i)=1 (d(i)≦th, that is, foreground)
w(i)=0 (d(i)>th, that is, background)

When the method of selecting the feature points only from the foreground is adopted as described above, the following effects can be obtained. That is, on the assumption that the imaged object is imaged in the foreground region due to the relative proximity, the feature points are selected only from the foreground region by eliminating the influence of the background region that is not the imaged target. Therefore, it is possible to obtain the effects of solving the problem of the conventional technique and improving the determination accuracy of the determination unit 5.

As described above, in the first embodiment that also considers the relative positional relationship, the three-dimensional spatial coordinate position (x,y,z)=(x(u,v),y(u,v), as the relative positional relationship, The relationship corresponding to z(u,v)) was obtained by depth and used. As a second embodiment that also considers the relative positional relationship, the spatial coordinate position (x (u, v), y (u, v), z (u, v)) is not directly used, the captured image It is also possible to use only the pixel position (u, v) of (1) to realize the selection that achieves substantially the same effect as the first embodiment.

As a general explanation, in the second embodiment, the positions (u(i), v(i)) on the image of the feature points i to be selected are made uniform without being biased, and then the positions ( The feature point i may be selected based on u(i), v(i)) and the feature point score score(i). At this time, the range in which the positions (u(i), v(i)) are made uniform without being biased is not necessarily the entire captured image, but a part of the captured image (for example, in the vicinity It may be set as the range in which the characteristic points are detected).

FIG. 4 is a flowchart showing an example of processing of the selection unit 3 in the second embodiment. In step S1, the shape and arrangement of the divided areas for dividing the captured image, the selection order of the divided areas, and the maximum number of characteristic points (upper limit of the number of characteristic points to be selected by the selection unit 3) are set, and then the process proceeds to step S2. And proceed.

The divided area of the captured image in step S1 may be divided by a predetermined method. For example, a predetermined number n in the horizontal direction and a predetermined number m in the vertical direction may be equally divided into a total of n×m rectangular regions. That is, if the size (number of pixels) of the captured image is W horizontal and H vertical, it may be divided into n×m rectangular regions of W/n vertical and H/m horizontal, respectively. .. Further, by applying an existing arbitrary area division method based on color information, texture information, and the like to the captured image, the shape of the divided area may not necessarily be a rectangular area.

Further, the selection order for the divided areas in step S1 may be determined as a predetermined order according to the division method. For example, when divided into n×m rectangular areas, the raster scan order may be used. Even in the case of division that is not a rectangular area, the selection order of the divided areas may be determined such that the raster scan order is set at the center of gravity of the divided areas.

Here, in the description of FIG. 4, the divided areas are denoted by R(k) (k=1, 2,...) And the selection order thereof is denoted by O(k). For example, if the selection order O(1)=3, O(2)=2, O(3)=1 for the three divided regions R(1), R(2), R(3), then ``R(3 ) (O(3)=1st)” → “R(2) (O(2)=2nd)” → “R(1) (O(1)=3rd)” in order. Is meant to be done.

When step S1 is performed (for the first time) to step S2, the current value in the selection order is set as the first value.

In step S2, among the divided regions R(k) specified by the current value O(k) in the selection order, the unselected feature points i having the maximum feature point score score(i) are selected. After selecting one, proceed to step S3. If no unselected feature point i remains in the designated divided area R(k) in step S2, the selection may be skipped (omitted) and the process may proceed to step S3. If the maximum value of the feature point scores score(i) among the unselected feature points i is less than the predetermined threshold value in step S2, the selection may be skipped (omitted) and the process may proceed directly to step S3. Good.

In step S3, it is determined whether or not the number of feature points selected in the series of steps S2 up to that point reaches a predetermined number, that is, the upper limit number set in step S1, and the upper limit number is reached. If so, the process proceeds to step S4, and if not reached, the process proceeds to step S5.

In step S4, the feature points selected in the series of steps S2 up to that point are output as the selection result in the selection unit 3, and then the flow of FIG. 4 ends.

In step S5, the current value O(k) of the selection order used in step S2 immediately before reaching step S5 is updated to the next value “O(k)+1” (=O(k′)). After setting the next selection target region R(k'), the process returns to step S2.

Thus, each time the loop processing of steps S2, S3, and S5 in FIG. 4 is repeated, the divided areas whose selection order is set to, for example, the first, the second, and the third in step S2 are set as feature point selection targets, respectively. Then, the feature points are selected from the set divided areas in step S2. When the current value O(k) of the selection order is the last order in step S5, the first selection order may be set again as the next selection target.

FIG. 5 is a diagram showing an application example of the flow of FIG. In FIG. 5, when the flow of FIG. 4 is applied under the setting that the upper limit number of selections is 7, the rectangular region is divided into 3×3, and the selection order is the raster scan order, an image (not shown) is displayed. The position distribution in the image of the feature points selected from is shown in [2]. In FIG. 4, as a contrast to [2], [1] shows a case where the upper limit number of 7 is selected from the same image based on only the feature point score values without applying the method of the present invention. ing.

That is, in both [1] and [2] of FIG. 5, the feature points in the same image are plotted together with the feature point score values. There are 16 feature points in total, and the feature point score is the highest. Examples are shown having each integer value from 100 to a minimum of 85. In both [1] and [2], the selected feature points are indicated by underlining their score values. In the example [1] to which the method of the present invention is not applied, the highest value 100 to the top 7 are simply selected to select the feature point score 94, but as can be seen in [1], the image The position distribution inside is biased to the lower left part. On the other hand, in the example of [2] to which the method of FIG. 4 of the present invention is applied, the bias of [1] is eliminated, and the characteristic points are selected relatively evenly in the image. That is, in [2], three feature point scores 88, 89, 85 are selected in the upper part of the image, two feature point scores 92, 91 are selected in the middle part, and the feature point scores are selected in the lower part. Two of the scores 100 and 99 are selected, and the spatial bias of the selected feature point distribution as seen in [1] is eliminated.

FIG. 6 is a diagram showing an application example of the flow of FIG. 4 regarding a certain image (not shown) under a larger number of feature point upper limit numbers than in the case of FIG. Similar to the case of FIG. 5, in FIG. 6, [1] shows the distribution of selected feature points when the present invention is not applied with the upper limit number of the same feature points in the same image as the proportionality. In contrast, in the application example of the present invention shown in [2], the bias of the feature points to the middle and lower stages is eliminated, and the feature points are selected almost uniformly on the upper stage side. Has been done. Note that in the example of FIG. 6, the region R13 of the “belt” that does not correspond to the original determination target is extracted as shown in [3] of FIG. 1, although many feature points with high feature point scores are extracted. It is an example of the explanation of the effect of the present invention when it exists in the middle lower part.

As described above, according to the present invention, after the image capturing target is captured by the image capturing unit 1, the determining unit 5 can determine the captured image capturing target. At this time, the feature points are selected based on the relative positional relationship between the image pickup unit 1 and the image pickup target, so that the feature point distribution can be optimized and highly accurate determination can be performed. Further, the feature amount at the feature point coordinates is calculated from the captured image, and the feature information under the same condition is selected and determined from the storage unit 6, so that the determination accuracy can be improved.

Hereinafter, supplementary items for explanation in the present invention will be described.

(1) As a first modified example of the second embodiment regarding the sorting processing in the sorting unit 3, the processing of sorting the feature points i one by one from the one having the maximum feature point score score(i) is repeated and predetermined. The selection may be performed until the upper limit number is reached, and the next update process may be performed each time one is selected. That is, for the feature point j whose position on the image is within the predetermined distance from the already selected feature point i, the feature point score score(j) may be updated to a low value. Updating to a lower value may be performed, for example, by multiplying by a predetermined coefficient r (0≦r<1), and it is assumed that N feature points have already been selected within a predetermined distance with respect to an unselected feature point j. If present, the feature point score may be r ^N *score(j) updated cumulatively N times. The setting of the predetermined coefficient r=0 corresponds to completely excluding the feature points within the predetermined distance from the already-selected feature points from the selection target.

The first modified example can be realized by replacing the processing in steps S1, S2, and S5 in FIG. 4 with the following processing. That is, in step S1, area division and division area selection order are not set, but only the maximum number of characteristic points (upper limit number of characteristic points to be selected by the selection unit 3) is set. In step S2, unselected feature points having the largest feature point score are selected from the entire image, not from the selected region. In step S5, the feature point score score(j) is set to a low value for a feature point j that is within a predetermined distance from the feature point i selected in the immediately preceding step S2, instead of selecting the next divided area. To update.

(2) As a second modified example of the second embodiment regarding the sorting process in the sorting unit 3, the following may be performed. That is, the area division of the image is performed in the same manner as described in step S1 of FIG. Then, for each divided region R(k) (k=1, 2,...), the total number of feature points detected by the detection unit 2 in the divided region is Num(k), and each divided region R From (k), the entire number of r*Num(k) number of feature points according to the total number of feature points Num(k) are sorted in descending order of feature point score, and output as the sorting result from the sorting unit 3. .. Here, the coefficient r may be set as a predetermined value (0<r<1) that gives a number corresponding to the total number of feature points Num(k), and in particular, it is selected from each divided region R(k). The coefficient r may be set as a value such that the total sum "Σr*Num(k)" of the numbers coincides with the preset upper limit of the number of feature points. Note that the number r*Num(k), which is the number multiplied by the coefficient r, is not an integer in general, but may be appropriately converted into an integer by a predetermined rounding process. As for the total sum "Σr*Num(k)" of the numbers, any number may be used as long as it matches the predetermined upper limit number of feature points in the threshold determination.

According to the second modified example, it is possible to realize an even distribution of feature points according to the number of feature points detected by the detection unit 2 in the divided area in the selection result output from the selection unit 3. Become. That is, the feature point distribution obtained by thinning out the feature point distribution in the picked-up image detected by the detection unit 2 by the coefficient r in the entire image can be output as the selection result from the selection unit 3.

(3) Regarding the sorting process in the sorting unit 3, the first embodiment and the second embodiment may be combined. For example, in the second embodiment, the feature point i that maximizes the feature point score score(i) is selected in step S2 of FIG. 4, but in step S2, the feature point score score(i) is replaced with the first feature point score score(i). The weighted score w(i)*score(i) in one embodiment may be used to select the feature point i that maximizes the weighted score.

(4) The weight w(i) in the first embodiment does not consider the arrangement in space coordinates such as the depth d(i), and the coordinates (u(i),v( It may be set based only on i)). For example, a weight w(i) having a positive value may be set such that it takes a maximum value at a predetermined position such as the center of the image and becomes smaller as the distance from the predetermined position increases. Also in this case, the feature point derived from the image-capturing target of the determination target is prioritized on the assumption that the image-capturing target serving as the determination target is imaged as being near the center or other predetermined position in the image. Therefore, it is possible to make a selection, which contributes to improvement in accuracy of the judgment in the judgment unit 5.

(5) The information processing device 10 may be configured by only an arbitrary part of each functional unit shown in FIG. For example, the imaging unit 1 may be omitted, and the detection unit 2 may acquire an arbitrary image from any other place on the network. However, at this time, by acquiring the accompanying information such as the camera information regarding the read image, the above-described respective processes such as the sorting process by the sorting unit 3 based on the depth can be performed. And it is sufficient. For example, considering a case where the information processing apparatus 10 is configured to include only the detection unit 2 and the selection unit 3, a device that selects and outputs a feature point suitable for recognition and other determination processing from the image read by the detection unit 2. As a result, the information processing device 10 functions.

(6) The information processing device 10 can be realized as a computer having a general configuration. That is, a CPU (central processing unit), a main storage device that provides a work area for the CPU, an auxiliary storage device that can be configured with a hard disk, SSD, etc., an input interface such as a keyboard, a mouse, a touch panel, etc., a network. The information processing apparatus 10 can be configured by a general computer including a communication interface for connecting to and performing communication, a display for displaying, a camera, and a bus connecting these. Further, the processing of each unit of the information processing apparatus 10 shown in FIG. 2 can be realized by a CPU that reads and executes a program for executing the processing, but an arbitrary part of the processing is performed by a separate dedicated circuit or the like. It may be realized. The image pickup unit 1 can be realized by a camera as the hardware.

10... Information processing device, 1... Imaging unit, 2... Detection unit, 3... Selection unit, 4... Calculation unit, 5... Judgment unit, 6... Storage unit

Claims (18)

A detection unit that detects the feature points from the image and obtains the feature point score of each feature point,
From the detected feature points, a feature point obtained by performing a selection based on the feature point score and position of each feature point, a selection unit that outputs as a feature point of the image,
In the selection unit, selection based on the feature point score and position of each feature point is performed with priority given to a higher weighting score obtained by multiplying the feature point score of each feature point by a weight based on the position of each feature point. Done by
In the selection unit, the feature points are repeatedly selected from the one having the higher weighting score, and each time the selection is performed repeatedly, the weighting score of the feature points determined to have a short distance on the image from the selected feature points. the information processing apparatus according to claim users update to a low value. The information processing apparatus according to claim 1, wherein the updating by the selection unit to the low value at each feature point is a cumulative update. The updating to the low value at each feature point by the selecting unit is to reduce the weighting score at a constant rate and leave it as a selection target by the selecting unit thereafter. Or the information processing device according to 2. Updating to the low value at each feature point by the selection unit is excluding from the selection target by the selection unit thereafter by setting the weighting score to zero. The information processing device described. 5. The selecting section repeats selecting a feature point from the one having the highest weighted score repeatedly until the number of the selected feature points reaches an upper limit number. An information processing device according to claim 2. A detection unit that detects the feature points from the image and obtains the feature point score of each feature point,
From the detected feature points, a feature point obtained by performing a selection based on the feature point score and position of each feature point, a selection unit that outputs as a feature point of the image ,
In the selection unit, selection based on the feature point score and position of each feature point is performed with priority given to a higher weighting score obtained by multiplying the feature point score of each feature point by a weight based on the position of each feature point. Done by
Wherein in sorting unit, the respective divided areas obtained by the image divided into regions selected in a predetermined order, that the sorting of the weighted scores from the selected divided region is selected one feature point having the maximum An information processing apparatus, characterized in that the feature points of the image are output by continuing in the predetermined order until the number of the feature points thus determined reaches an upper limit number . The selecting unit omits selecting feature points from the selected divided area when no unselected feature points remain in the selected divided area in the predetermined order. 6. The information processing device according to 6. In the selection unit, when it is determined that the maximum value of the weighting score in the feature points that have not been selected in the divided region selected in the predetermined order is small, the feature points are selected from the selected divided region. The information processing apparatus according to claim 6, wherein the information processing apparatus is omitted. Wherein in sorting section, the weight, according to any one of claims 1 to 8, characterized in that calculated as a value based on the spatial position of each feature point on the basis of the camera center of the camera that captured the image Information processing device. The information processing apparatus according to claim 9 , wherein the selection unit calculates the weight as a value based on a depth of each feature point. Wherein in sorting section, the information processing apparatus according to claim 10, wherein the value based on the depth of the feature points, and calculates a larger value depth is small. The selection unit divides the image into a foreground region and a background region, and performs the position-based selection by selecting only feature points in the divided foreground region as selection targets. Item 12. The information processing device according to any one of items 1 to 11 . Wherein in sorting section, the information processing apparatus according to any one of the weights, claims 1 and calculates a value based on the pixel coordinates in the image of each feature point 12. Wherein in sorting section, the information processing apparatus according to any one of the weight, to the pixel coordinate position in the image of each feature point claims 1 and calculates as a larger value closer to the predetermined coordinate position 13 .. Regarding each feature point output as the feature point of the image by the selection unit, a calculation unit that further calculates a feature amount at each feature point by referring to a region near the feature point in the image is provided. claims 1 to an information processing apparatus according to any one of 14. By comparing the feature amount calculated for the image by the calculation unit and the feature amount calculated from the captured image for each of the plurality of predetermined targets, the target imaged in the image is The information processing apparatus according to claim 15 , further comprising a determination unit that determines which one of the predetermined targets is applicable. The information processing apparatus according to any one of claims 1 to 16 , further comprising an imaging unit that acquires the image by performing imaging. Program for causing to function as the information processing apparatus according to computer to any one of claims 1 to 17.