CN103177264B - The image classification method that view-based access control model dictionary Global Topological is expressed - Google Patents

Abstract

The invention discloses the image classification method that a kind of view-based access control model dictionary Global Topological is expressed, including training and two processes of identification, specifically include step: the target image having marked classification is carried out feature extraction, the feature extracted is carried out Global Topological coding on visual dictionary, coding result is trained and models；The image of unknown classification is carried out feature extraction, the feature extracted is carried out Global Topological coding on visual dictionary, be input to train the model obtained by coding result, it is thus achieved that the classification of target image.Expressing the manifold for image due to Global Topological to express and have invariance, therefore the present invention uses the Global Topological of view-based access control model dictionary to express the precision improving image recognition, and this technology has great importance for the understanding of dynamic image.The present invention is expressed by the Global Topological of study visual dictionary, can identify the classification of image accurately, and this technology can be widely applied to safety verification, the field such as web search and digital entertainment.

Description

Image classification method based on visual dictionary global topology expression

Technical Field

The invention relates to the field of pattern recognition and the field of computer vision, in particular to an image classification method based on visual dictionary global topological expression.

Background

In computer vision, image classification is a fundamental research problem. In the past decades, researchers have attempted to study classification problems based on the physiological basis of human cognition. As an important basis, researchers believe that a basic element of image classification is the potential manifold representation of images. This manifold expression comes from neuron coding, and researchers propose to exploit the topological relationship between neurons to obtain some topological invariance, which plays an important role in the manifold expression of images. Thus, visual topology is a bridge connecting neuronal coding and manifold expression, and this relational mechanism has been widely applied to feature coding and manifold expression in image classification. A great deal of research in recent years shows that local features can be better reconstructed by describing visual topology, so that the manifold expression of an image is smoother, and the aim of improving the classification precision of the image is fulfilled. Therefore, it is necessary to study visual topology in image classification.

In recent years, more and more researchers have considered topological structures in image classification, especially in the local feature coding part. The topology used for feature coding was originally derived from topology-expressing neural networks, and researchers in the networks utilized topological relationships between neurons to more accurately code the neurons. Inspired by this topological representation, many researchers have begun working on encoding local features using topological relationships of visual words in visual dictionary models. At present, the topological structure mainly considers the topological relation of two visual words, such as the fusion of two visual words with the closest distance, the distance and angle relation of the two visual words, and the like, and the relation of every two topological structures achieves certain effect.

However, one limitation of this two-by-two topology, according to the theory of manifold expression, is that it does not well characterize the manifold invariance of the feature space. Therefore, it is necessary to consider a high-order topology that can make the manifold representation smoother, such as a global topology between multiple visual words. Another limitation is that considering this high order topology is challenging because combinations of visual words grow exponentially as the number of visual words increases. Therefore, in the invention, the image classification method based on the visual dictionary global topology expression is provided to overcome the two limitations so as to achieve a better image recognition effect.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides an image classification method based on visual dictionary global topological expression.

The invention provides an image classification method based on visual dictionary global topology expression, which comprises the following steps:

step 1, collecting a plurality of training images, respectively carrying out local sampling on the plurality of training images, and extracting Scale Invariant Feature Transform (SIFT) features from obtained local sampling blocks so as to obtain an SIFT feature set of the training images;

step 2, clustering the obtained SIFT feature set to generate a plurality of clustersCenter, and forming visual dictionary with the clustering center as visual wordsWhere C denotes a visual dictionary consisting of M visual words C in D dimensions_iThe components of the composition are as follows,representing a subspace formed by M points in the D-dimensional space;

step 3, carrying out global topological coding on each SIFT feature of each training image;

step 4, all SIFT characteristics of each training imageGlobal topology code V of₁，V₂，…，V_NPerforming maximum aggregation operation to generate an image expression F of the training image;

step 5, sending the image expressions of all training images into a classifier for training to generate a training model;

step 6, similar to the step 1, carrying out local sampling on each image to be recognized, and extracting scale-invariant feature transform (SIFT) features from the obtained local sampling blocks to obtain an SIFT feature set of each image to be recognized;

step 7, based on the visual dictionary which is obtained in the step 2 and consists of visual words, carrying out global topological coding on each SIFT feature of each image to be recognized by utilizing the step 3;

step 8, similar to the step 4, performing maximum aggregation operation on the global topological codes of all SIFT features of each image to be identified to generate an image expression of the image to be identified;

and 9, sending the image expression of the image to be recognized obtained in the step 8 into the training model generated in the step 5 for testing, so as to obtain the recognition result of the target category in the image to be recognized.

According to the method, the topological structure of the target image can still be robustly acquired under the complex condition of the target image, so that the robust identification of the image is carried out. In an intelligent visual monitoring system, the technology can be used for identifying the category of a target in a monitoring system scene, so that the monitoring system can well identify the current behavior of a target object.

Drawings

FIG. 1 is a flowchart of an image classification method based on a visual dictionary global topology expression according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to specific embodiments and the accompanying drawings.

Fig. 1 is a flowchart of an image classification method based on a global topological expression of a visual dictionary according to the present invention, and as shown in fig. 1, the image classification method based on a global topological expression of a visual dictionary according to the present invention includes the following steps:

step 1, collecting a plurality of training images, respectively carrying out local sampling on the plurality of training images, wherein the size of pixels of local sampling blocks can be 16 × 16, and extracting Scale Invariant Feature Transform (SIFT) features from the obtained local sampling blocks so as to obtain an SIFT feature set of the training images;

step 2, clustering the obtained SIFT feature set to generate a plurality of clustering centers, and forming a visual dictionary by taking the clustering centers as visual wordsWhere C denotes a visual dictionary consisting of M visual words C in D dimensions_iThe components of the composition are as follows,representing a subspace formed by M points in the D-dimensional space;

in this step, the clustering may use a clustering algorithm commonly used in the art, such as a k-means clustering algorithm.

Step 3, carrying out global topological coding on each SIFT feature of each training image;

the step 3 further comprises three substeps:

step 3.1, calculating the correlation among the visual words in the visual dictionary;

in one embodiment of the present invention, the correlation between visual words is obtained by using a distance and angle based self-increment algorithm, and before introducing the distance and angle based self-increment algorithm, several parameters are defined:

in-vision dictionaryFor visual word c_iIf c is a_jAnd c_iAre correlated, they are defined to form a cone in the feature space:

$c_i\&RightArrow;c_j=\left\{y\right|{\left|\right|y-c_i\left|\right|}_2^2\&le;{\mathrm{\&mu;}}_{i,}\mathrm{\&Delta;}(c_{i,}y,c_j)<\mathrm{\&theta;}\}---\left(1\right)$

wherein, c_i→c_jDenotes c_iAnd c_jA cone in the feature space formed between two visual words, y is any one feature point in the cone, { y } represents this cone,denotes y and c_iEuclidean distance between, mu_iIs c_iC after clustering with k-means_iThe maximum Euclidean distance between the characteristic points of (a), theta is the angle of the control cone; delta (c)_i，y，c_j) Is the vector y-c_iSum vector c_j-c_iThe angle between, defined as follows:

$\mathrm{\&Delta;}(c_{i,}y,c_j)=\arccos \frac{\&lang;y-c_{i,}{c}_j-c_i\&rang;}{{\left|\right|y-c_i\left|\right|}_2\&CenterDot;{\left|\right|c_j-c_i\left|\right|}_2}---\left(2\right)$

wherein,<y-c_i，c_j-c_i>representing a vector y-c_iSum vector c_j-c_iInner product between, "· represents a dot product between two vectors, | | · | luminance |₂Representing the two-norm of the vector.

Then, according to the angle between the cone defined by formula (1) and the vector defined by formula (2), the distance and angle based auto-increment algorithm can obtain the correlation between the visual words, and the basic idea of the distance and angle based auto-increment algorithm is that each related visual word occupies an independent cone area. Specifically, the distance and angle based auto-increment algorithm comprises:

first, for a certain visual word c_iAll visual words according to it to c_iIs sorted from near to far, wherein the closest visual word is taken as the initial relevant visual word c_i1；

Second, remove the visual word c_iOther visual words are processed one by one according to the distance and angle criteriaChecking if a certain visual word c_jUpon examination both the distance and angle criteria are met, the visual word is labeled as related visual word c_ijAnd added to the set S_iFinally, the visual word c is obtained_iOf related visual words S_i；

Wherein the distance criterion is defined as:

||c_i-c_ij||₂＜τ||c_i-c_i1||₂(3)

wherein τ is used to control c_iEuclidean distance to other visual words, i.e. distance c_iVisual words that are too far away are not considered;

the angle criterion is defined as:

Δ(c_i，c_ij，c_k)＞θ， $\&ForAll;\left\{c_k\right\}\&Element;S_i---\left(4\right)$

wherein S is_iIs a visual word c_iC, and c_kRepresenting that has been added to set S_iThe related visual words in (1).

Finally, all the visual words are processed in a traversal mode, and a relevant word set S of the visual dictionary C can be obtained:

where D represents the dimension of each visual word, S_iIs a visual word c_iOf related words, Q_iIs S_iNumber of visual words in (1).

Step 3.2, establishing a global topology model for the nearest K visual words of each SIFT feature by utilizing the correlation among the visual words obtained in the step 3.1;

in this step, it is assumed that the SIFT feature set extracted for each training image is set asWherein D is the feature dimension and N is the feature number. Then for each feature x_jThe global topology model is defined as:

Δ(C，x_j，S)＝[Δ(c₁，x_j，S₁)，...，Δ(c_M，x_j，S_M)](7)

$\&ForAll;\mathrm{\&Delta;}(c_i,x_j,S_i)=[\mathrm{\&Delta;}(c_i,x_j,c_{i1}),...,\mathrm{\&Delta;}(c_i,x_j,c_{{\mathrm{iQ}}_i})]\&Element;R^{1\&times;Q_i}---\left(8\right)$

wherein, Δ (c)_i，x_j，S_i) Is a vector x_j-c_iAnd set S_iEach of the related visual words c_ijAnd c_iFormed vector c_ij-c_iThe angle therebetween:

$\mathrm{\&Delta;}(c_i,x_j,S_i)=[\mathrm{\&Delta;}(c_i,x_j,c_{i1}),...,\mathrm{\&Delta;}(c_i,x_j,c_{{\mathrm{iQ}}_i})],$

$\mathrm{\&Delta;}(c_i,x_j,c_{\mathrm{ij}})=\arccos \frac{\&lang;x_j-x_i,c_{\mathrm{ij}}-c_i\&rang;}{{\left|\right|x_j-c_i\left|\right|}_2\&CenterDot;{\left|\right|c_{\mathrm{ij}}-c_i\left|\right|}_2},$ <x_j-c_i，c_ij-c_i>representing a vector x_j-c_iSum vector c_ij-c_iInner product between, "· represents a dot product between two vectors, | | · | luminance |₂Representing the two-norm of the vector.

And 3.3, performing global topology coding on each SIFT feature by using the global topology model obtained in the step 3.2.

In this step, each SIFT feature x is determined by the following equation_iCarrying out global topological coding:

$\underset{V_i}{\arg \min }{\left|\right|x_i-{\mathrm{SV}}_i^T\left|\right|}_2^2+[\mathrm{\&lambda;T}\left(V_i\right)+\mathrm{\&alpha;}{\left|\right|V_i\&CenterDot;\mathrm{\&Delta;}(C,x_i,S)\left|\right|}_2^2]---\left(9\right)$

where λ is the penalty term coefficient, T (V)_i) Is to V_iα is a penalty term of the global topology model, ". represents a point product between two vectors, V_iIs x_iThe coding on S is defined as follows:

wherein (v)_k1)_iIs the local feature x_iIn the visual word c_kFirst related visual word c of_k1The above response, and so on.

Then, the formula (9) is optimized and solved, and the SIFT feature x can be obtained_iGlobal topology code V of_i。

Step 4, all SIFT characteristics of each training imageGlobal topology code V of₁，V₂，...，V_NPerforming maximum aggregation operation to generate an image expression F of the training image;

in this step, a maximum clustering operation is performed according to the following formula to generate an image expression F of the training image:

F＝max_column[V₁ ^T，V₂ ^T，...，V_N ^T]^T(12)

therein, max_columnMeaning that only the maximum value remains for each column of the matrix.

Step 5, sending the image expressions of all training images into a classifier for training to generate a training model;

in this step, the classifier may use a classifier commonly used in the art, such as a support vector machine.

Step 6, similar to the step 1, carrying out local sampling on each image to be recognized, and extracting scale-invariant feature transform (SIFT) features from the obtained local sampling blocks to obtain an SIFT feature set of each image to be recognized;

step 7, based on the visual dictionary composed of visual words obtained in the step 2, performing global topological coding on each SIFT feature of each image to be recognized by using the step 3 (such as formula (7) -formula (11));

step 8, similar to the step 4, performing maximum aggregation operation on the global topological codes of all SIFT features of each image to be identified by using the image expression defined by the formula (12) to generate the image expression of the image to be identified;

and 9, sending the image expression of the image to be recognized obtained in the step 8 into the training model generated in the step 5 for testing, so as to obtain the recognition result of the target category in the image to be recognized.

From the above, the image classification method based on the global topological expression of the visual dictionary provided by the invention comprises two processes of training and recognition, and then the implementation of the method is described by taking a vehicle detection system in a certain monitoring scene as an example. The vehicle detection system can judge whether a monitored scene contains a vehicle.

A large number of vehicle images (1000) and non-vehicle images (1000) are collected and used to train a vehicle recognition model. The training step S1 is:

step S11: SIFT feature extraction is carried out on 1000 vehicle images (positive samples) and 1000 non-vehicle images (negative samples), 2000 groups of SIFT features are generated, calculation is carried out by averaging that each group contains 2000 SIFT features, and 4000000(2000 multiplied by 2000) SIFT features are extracted in total;

step S12: carrying out k-menas clustering operation on 4000000 SIFT features to generate 1 visual dictionary containing 1000 visual words;

step S13: in an actual application scene, 3 nearest neighbor visual words of each SIFT local feature are taken to carry out global topological coding, and the coding length is 3000 dimensions;

step S14: carrying out maximum aggregation operation on the global topological codes of 2000 SIFT local features of each training image to obtain image expression of the image, wherein the length of the image expression is 3000 dimensions;

step S15: sending the image expressions of all training images into a classifier for training to obtain a training model;

in the identification stage, a camera signal is accessed to a computer through an acquisition card to acquire a test picture, and the specific identification step S2 is as follows:

step S21: inputting a test image, and carrying out SIFT feature extraction operation on the test image to generate 1 group of SIFT features containing 2000 SIFT features.

Step S22: in an actual application scene, 3 nearest neighbor visual words of each SIFT local feature are taken to carry out global topological coding, and the coding length is 3000 dimensions;

step S23: carrying out maximum aggregation operation on the global topological codes of 2000 SIFT local features of each test image to obtain image expression of the image, wherein the length of the image expression is 3000 dimensions;

step S24: and sending the image expression of the image to be recognized obtained in the step S23 to the training model generated in the step S15 for testing, so as to obtain the recognition result of the target class in the image to be recognized.

In summary, the invention provides an effective image classification technology based on visual dictionary global topology expression. The invention is easy to realize, has stable performance, can improve the comprehension capability of the intelligent monitoring system to the monitored scene, and is a key technology in the next generation of intelligent video monitoring system.

The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (8)

1. An image classification method based on visual dictionary global topology expression is characterized by comprising the following steps:

step 1, collecting a plurality of training images, respectively carrying out local sampling on the plurality of training images, and extracting Scale Invariant Feature Transform (SIFT) features from obtained local sampling blocks so as to obtain an SIFT feature set of the training images;

step 2, clustering the obtained SIFT feature set to generate a plurality of clustering centers, and forming a visual dictionary by taking the clustering centers as visual wordsWhere C denotes a visual dictionary consisting of M visual words C in D dimensions_iThe components of the composition are as follows,representing a subspace formed by M points in the D-dimensional space;

step 3, carrying out global topological coding on each SIFT feature of each training image;

step 4, all SIFT characteristics of each training imageGlobal topology code V of₁，V₂，...，V_NPerforming maximum aggregation operation to generate an image expression F of the training image;

step 5, sending the image expressions of all training images into a classifier for training to generate a training model;

step 6, similar to the step 1, carrying out local sampling on each image to be recognized, and extracting scale-invariant feature transform (SIFT) features from the obtained local sampling blocks to obtain an SIFT feature set of each image to be recognized;

step 7, based on the visual dictionary which is obtained in the step 2 and consists of visual words, carrying out global topological coding on each SIFT feature of each image to be recognized by utilizing the step 3;

step 8, similar to the step 4, performing maximum aggregation operation on the global topological codes of all SIFT features of each image to be identified to generate an image expression of the image to be identified;

step 9, sending the image expression of the image to be recognized obtained in the step 8 into the training model generated in the step 5 for testing, so as to obtain a recognition result of the target category in the image to be recognized;

the step 3 further comprises three substeps:

step 3.1, calculating the correlation among the visual words in the visual dictionary;

step 3.2, establishing a global topology model for the nearest K visual words of each SIFT feature by utilizing the correlation among the visual words obtained in the step 3.1;

step 3.3, carrying out global topological coding on each SIFT feature by using the global topological model obtained in the step 3.2;

in the step 3.2, it is assumed that the SIFT feature set extracted from each training image isWhere D is the feature dimension and N is the number of features, then for each feature x_jThe global topology model is expressed as:

Δ(C，x_j，S)＝[Δ(c₁，x_j，S₁)，...，Δ(c_M，x_j，S_M)]，

$\&ForAll;\mathrm{\&Delta;}(c_i,x_j,S_i)=\&lsqb;\mathrm{\&Delta;}(c_i,x_j,c_{i1}),...,\mathrm{\&Delta;}(c_i,x_j,c_{{\mathrm{iQ}}_i})\&rsqb;\&Element;R^{1\&times;Q_i},$

wherein, Δ (c)_i，x_j，S_i) Is a vector x_j-c_iAnd set S_iEach related word c in_ijAnd c_iFormed vector c_ij-c_iThe angle therebetween:

$\mathrm{\&Delta;}(c_i,x_j,S_i)=[\mathrm{\&Delta;}(c_i,x_j,c_{i1}),...,\mathrm{\&Delta;}(c_i,x_j,c_{{\mathrm{iQ}}_i})],$

<x_j-c_i，c_ij-c_i>representing a vector x_j-c_iSum vector c_ij-c_iInner product between, "· represents a dot product between two vectors, | | · | luminance |₂Representing the two-norm of the vector.

2. The method according to claim 1, wherein in the step 2, a k-means clustering algorithm is adopted for clustering.

3. The method of claim 1, wherein the correlations between visual words are derived using a distance and angle based auto-incremental algorithm.

4. The method of claim 3, wherein the distance and angle based auto-increment algorithm comprises:

first, for a certain visual word c_iAll visual words according to it to c_iIs sorted from near to far, wherein the closest visual word is taken as the initial relevant visual word c_i1；

Second, remove the visual word c_iChecking other visual words one by one according to the distance and angle criteria, if a certain visual word c_jUpon examination both the distance and angle criteria are met, the visual word is labeled as related visual word c_ijAnd added to the set S_iFinally, the visual word c is obtained_iOf related visual words S_i；

And finally, traversing all the visual words to obtain a relevant word set S of the visual dictionary C.

5. The method of claim 4, wherein the distance criterion is defined as:

||c_i-c_ij||₂＜τ||c_i-c_i1||2，

wherein τ is used to control c_iEuclidean distances to other visual words;

the angle criterion is defined as:

Δ(c_i，c_ij，c_k)＞θ，

wherein S is_iIs a visual word c_iC, and c_kRepresenting that has been added to set S_iThe visual word in (1).

6. The method of claim 4, wherein the set of related words S of the visual dictionary C is represented as:

where D represents the dimension of each visual word, S_iIs a visual word c_iOf related visual words, Q_iIs S_iNumber of visual words in (1).

7. The method of claim 1, wherein in step 3.3, each SIFT feature x is determined using the following formula_iCarrying out global topological coding:

$\underset{V_i}{\mathrm{argmin}}\left|\right|x_i-{{\mathrm{SV}}_i}^T|{|}_2^2+\&lsqb;\mathrm{\&lambda;}T\left(V_i\right)+\mathrm{\&alpha;}||V_i\&CenterDot;\mathrm{\&Delta;}(C,x_i,S)|{|}_2^2\&rsqb;,$

where λ is the penalty term coefficient, T (V)_i) Is to V_iα is a penalty term of the global topology model, ". represents a point product between two vectors, V_iIs x_iGlobal topology coding on S:

wherein (v)_k1)_iIs each SIFT feature x_iIn the visual word c_kFirst related visual word c of_k1The above response, and so on.

8. The method according to claim 1, wherein in step 4, the maximum clustering operation is performed according to the following formula to generate an image expression F of the training image:

F＝max_column[V₁ ^T，V₂ ^T，...，V_N ^T]^T

therein, max_columnMeaning that only the maximum value remains for each column of the matrix.