CN109993047A - City huddles violation recognition methods, device and the electronic equipment of material - Google Patents
City huddles violation recognition methods, device and the electronic equipment of material Download PDFInfo
- Publication number
- CN109993047A CN109993047A CN201810787496.XA CN201810787496A CN109993047A CN 109993047 A CN109993047 A CN 109993047A CN 201810787496 A CN201810787496 A CN 201810787496A CN 109993047 A CN109993047 A CN 109993047A
- Authority
- CN
- China
- Prior art keywords
- violation
- article element
- machine learning
- feature
- learning model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/192—Recognition using electronic means using simultaneous comparisons or correlations of the image signals with a plurality of references
- G06V30/194—References adjustable by an adaptive method, e.g. learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the present invention discloses violation recognition methods, device and the electronic equipment that a kind of city huddles material, is related to image recognition technology, can be improved the efficiency that city huddles the violation identification of material.The recognition methods includes: to identify to the picture of front end camera shooting using the first machine learning model, export the feature of the article element in the picture；The feature of the article element includes the title of article element and the quantity and/or size of article element；Whether the feature of the article element in the picture of first machine learning model output identifies the article element as the input of the second machine learning model in violation of rules and regulations, and exports recognition result.Described device includes realizing the module of the above method.The present invention is suitable for the violation identification that city huddles material.
Description
Technical field
The present invention relates to violation recognition methods, dresses that image identification technical field more particularly to a kind of city huddle material
It sets and electronic equipment.
Background technique
Administration of city appearance is that the appearance of the city administrative responsibile institution of municipal government supervises troop by the appearance of the city, in accordance with the law to city
Building appearance, landscape light, outdoor advertising setting and production and transport etc. the management activity that carries out of clean and tidy, specification.It is city
Important component in integrated management, and measure one of the key criteria of urban management level height.
Currently, municipal administration's city management such as order to city appearance environment and in the street, do mainly or to scene law enforcement by people and disobey
Rule check that efficiency is lower.
Summary of the invention
It is set in view of this, the embodiment of the present invention provides a kind of violation recognition methods, device and the electronics that city huddles material
It is standby, it can be improved the efficiency that city huddles the violation identification of material.
In a first aspect, the embodiment of the present invention provides the violation recognition methods that a kind of city huddles material, comprising: utilize first
Machine learning model identifies the picture of front end camera shooting, exports the feature of the article element in the picture；Institute
The feature for stating article element includes the title of article element and the quantity and/or size of article element；By first machine
The feature of article element in the picture of learning model output, as the input of the second machine learning model, to the article member
Whether element is identified in violation of rules and regulations, and exports recognition result.
With reference to first aspect, in the first embodiment of first aspect, first machine learning model by with
Lower step obtains: production training sample；Wherein, the label of each training sample includes the name of the article element in training sample
The quantity and/or size of title and article element；Feature extraction is carried out to each training sample, obtains the feature of each training sample
Vector；Feature vector based on each training sample carries out model training, to obtain the first machine learning model.
With reference to first aspect, in second of embodiment of first aspect, the input of second machine learning model
It further include capture rate threshold value；The feature by the article element in the picture of first machine learning model output, as
Whether the input of the second machine learning model identifies the article element in violation of rules and regulations, and exports recognition result, comprising: will
The feature and capture rate threshold value of article element in the picture of the first machine learning model output, as the second machine
The input of learning model substitutes into the second machine learning model；Second machine learning model according to the capture rate threshold value of setting,
And the feature of the article element, whether the article element in the picture is predicted in violation of rules and regulations, and whether in violation of rules and regulations to export
Prediction result.
With reference to first aspect, in the third embodiment of first aspect, second machine learning model by with
Lower step obtains: the spy for the article element in two or more pictures that capture rate threshold value and the first machine learning model are exported
Sign is used as training sample；Wherein, the label of training sample corresponding with the feature of article element is in violation of rules and regulations or not in violation of rules and regulations, and to catch
The label for obtaining the corresponding training sample of rate threshold value is capture rate；The processing of convolution sum pondization is carried out to the training sample, is obtained
The feature vector of the training sample；Based on the feature vector of the training sample, to minimize input capture rate threshold value and defeated
Difference between capture rate out, and maximizing accuracy rate is that optimization aim carries out model training, to obtain the second machine learning
Model.
The third embodiment with reference to first aspect, in the 4th kind of embodiment of first aspect, first machine
The output of device learning model further includes the confidence level of the feature of each article element in picture；First machine learning model is exported
Two or more pictures in article element feature as training sample, comprising: choose the first machine learning model output
The feature of article element in two or more pictures；From the feature of the article element in all pictures of selection, scheme at every
The feature for choosing the article element of predetermined quantity in piece from high to low by confidence level, the training sample as the second machine learning model
This.
Second of embodiment with reference to first aspect, in the 5th kind of embodiment of first aspect, second machine
The output of device learning model further include article element in the picture whether the confidence level of violation.
With reference to first aspect, in the 6th kind of embodiment of first aspect, the feature of the article element further include: object
The syntagmatic between positional relationship and article element between product element.
With reference to first aspect, in the 7th kind of embodiment of first aspect, the violation that the city huddles material is known
Other method, further includes: and preset by the feature of the article element in the picture of first machine learning model output
Judge that the decision rule in library is compared in violation of rules and regulations, whether the article element is identified in violation of rules and regulations, and export recognition result；
Wherein, the decision rule include article element and article element whether the mapping relations of violation.
Second aspect, the embodiment of the present invention provide the violation identification device that a kind of city huddles material, comprising: the first identification
Module identifies the picture of front end camera shooting, exports in the picture for utilizing the first machine learning model
The feature of article element；The feature of the article element includes the title of article element and the quantity of article element and/or big
It is small；Second identification module, the feature of the article element in picture for exporting first machine learning model, as
Whether the input of two machine learning models identifies the article element in violation of rules and regulations, and exports recognition result.
In conjunction with second aspect, in the first embodiment of second aspect, first machine learning model by with
Lower step obtains: production training sample；Wherein, the label of each training sample includes the name of the article element in training sample
The quantity and/or size of title and article element；Feature extraction is carried out to each training sample, obtains the feature of each training sample
Vector；Feature vector based on each training sample carries out model training, to obtain the first machine learning model.
In conjunction with second aspect, in second of embodiment of second aspect, the input of second machine learning model
It further include capture rate threshold value；Second identification module, specifically in the picture that exports first machine learning model
Article element feature and capture rate threshold value, as the second machine learning model input substitute into the second machine learning mould
Type；Second machine learning model is according to the capture rate threshold value of setting and the feature of the article element, to the picture
In article element whether predicted in violation of rules and regulations, and export whether the prediction result of violation.
In conjunction with second aspect, in the third embodiment of second aspect, second machine learning model by with
Lower step obtains: the spy for the article element in two or more pictures that capture rate threshold value and the first machine learning model are exported
Sign is used as training sample；Wherein, the label of training sample corresponding with the feature of article element is in violation of rules and regulations or not in violation of rules and regulations, and to catch
The label for obtaining the corresponding training sample of rate threshold value is capture rate；The processing of convolution sum pondization is carried out to the training sample, is obtained
The feature vector of the training sample；Based on the feature vector of the training sample, to minimize input capture rate threshold value and defeated
Difference between capture rate out, and maximizing accuracy rate is that optimization aim carries out model training, to obtain the second machine learning
Model.
In conjunction with the third embodiment of second aspect, in the 4th kind of embodiment of second aspect, first machine
The output of device learning model further includes the confidence level of the feature of each article element in picture；Second identification module, including
Sample acquisition submodule, the feature of the article element in two or more pictures for choosing the output of the first machine learning model；
From the feature of the article element in all pictures of selection, predetermined quantity is chosen from high to low by confidence level in every picture
Article element feature, the training sample as the second machine learning model.
In conjunction with second of embodiment of second aspect, in the 5th kind of embodiment of second aspect, second machine
The output of device learning model further include article element in the picture whether the confidence level of violation.
In conjunction with second aspect, in the 6th kind of embodiment of second aspect, the feature of the article element further include: object
The syntagmatic between positional relationship and article element between product element.
In conjunction with second aspect, in the 7th kind of embodiment of second aspect, the violation that the city huddles material is known
Other device, further includes: third identification module, the article element in picture for exporting first machine learning model
Whether feature judges that the decision rule in library is compared with preset violation, knows in violation of rules and regulations to the article element
Not, and recognition result is exported；Wherein, the decision rule include article element and article element whether the mapping relations of violation.
The third aspect, the embodiment of the present invention provide a kind of electronic equipment, and the electronic equipment includes: shell, processor, deposits
Reservoir, circuit board and power circuit, wherein circuit board is placed in the space interior that shell surrounds, processor and memory setting
On circuit boards；Power circuit, for each circuit or the device power supply for above-mentioned electronic equipment；Memory is for storing and can hold
Line program code；Processor is run and executable program code pair by reading the executable program code stored in memory
The program answered, for executing method described in aforementioned any embodiment.
Fourth aspect, the embodiment of the present invention provide a kind of computer readable storage medium, the computer-readable storage medium
Matter is stored with one or more program, and one or more of programs can be executed by one or more processor, with reality
Method described in existing aforementioned any embodiment.
City provided in an embodiment of the present invention huddles violation recognition methods, device, electronic equipment and the storage medium of material,
After identifying by the first machine learning model to the article element in picture, further pass through the second machine learning model pair
Whether the article element in the picture carries out identification decision in violation of rules and regulations, and exports recognition result, this way it is not necessary to reach scene
Be able to achieve to city huddle material whether the identification of violation, improve in violation of rules and regulations identification efficiency.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is the violation recognition methods flow diagram that one city of the embodiment of the present invention huddles material；
Fig. 2 is the violation recognition methods flow chart that two city of the embodiment of the present invention huddles material；
Fig. 3 is the violation recognition methods flow chart that three city of the embodiment of the present invention huddles material；
Fig. 4 is the structural schematic diagram for the violation identification device that four city of the embodiment of the present invention huddles material；
Fig. 5 is the structural schematic diagram for the violation identification device that six city of the embodiment of the present invention huddles material；
Six city of Fig. 6 embodiment of the present invention huddles the data flow and control flow diagram of the violation identification device of material；
Fig. 7 is the structural schematic diagram of seven electronic equipment of the embodiment of the present invention
Specific embodiment
The embodiment of the present invention is described in detail with reference to the accompanying drawing.
It will be appreciated that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Base
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts it is all its
Its embodiment, shall fall within the protection scope of the present invention.
Embodiment one
Fig. 1 is the violation recognition methods flow diagram that one city of the embodiment of the present invention huddles material, the present embodiment application
In video monitoring system, whether the material stacked to the place such as the community, street, public place in city carries out judgement knowledge in violation of rules and regulations
Not, the reference frame to enforce the law as municipal administration.The video monitoring system includes front end camera and takes the photograph with the front end
As the violation identification device that head is connected, the front end camera is installed on the positions such as the community, street, public place in city, uses
The material stacked in Target monitoring area carries out Image Acquisition, and the image information of acquisition is sent to identification in violation of rules and regulations and is filled
It sets, whether the violation identification device determines the material stacked in the target area according to described image information in violation of rules and regulations
It identifies and exports recognition result.
As shown in Figure 1, the present embodiment city huddles the violation recognition methods of material, may include:
Step 101, using the first machine learning model, the picture of front end camera shooting is identified, described in output
The feature of article element in picture.
Article element refers in picture captured by the camera of front end shown each article, for example, brick, cement heap,
Carton, desk, bag weaved etc..
The feature of article element is used to that article attribute of an element is described or is demarcated.The feature of article element includes object
The title of product element and the quantity and/or size of article element.
Front end camera is installed on the positions such as the community, street, public place in city, for Target monitoring area Nei Dui
The material put carries out Image Acquisition.
First machine learning model, can by convolutional neural networks (Convolutional Neural Network,
CNN) training of training sample is obtained.
The feature of step 102, the article element for exporting first machine learning model, as the second engineering
Whether the input for practising model, identify the article element, and export recognition result in violation of rules and regulations.
In the present embodiment, after being identified by the first machine learning model to the article element in picture, further lead to
It crosses the second machine learning model and whether identification decision is carried out in violation of rules and regulations to the article element in the picture, and export recognition result,
This way it is not necessary to reach scene be also able to achieve to city huddle material whether the identification of violation, improve in violation of rules and regulations identification efficiency,
And human cost can be saved.
Fig. 2 is the violation recognition methods flow chart that two city of the embodiment of the present invention huddles material, the method for the present embodiment
Scene is applicable in as in the first embodiment, details are not described herein.As shown in Fig. 2, the method for the present embodiment may include:
Step 201 carries out Image Acquisition to the material stacked in Target monitoring area by front end camera.
Front end camera is installed on the positions such as the community, street, public place in city, to what is stacked in Target monitoring area
Material carries out Image Acquisition.
Step 202, using the first machine learning model, the picture of front end camera shooting is identified, described in output
The feature of article element in picture.
Article element refers in picture captured by the camera of front end shown each article, for example, brick, cement heap,
Carton, desk, bag weaved etc..
The feature of article element is used to that article attribute of an element is described or is demarcated.The feature of article element includes object
The title of product element and the quantity and/or size of article element.
In another embodiment, the feature of the article element may also include that between the texture of article element, article element
Positional relationship and article element between syntagmatic etc..Wherein, the positional relationship can further comprise comprising closing again
System and neighbouring relations.Such as: being stacked with multiple bamboo baskets in a picture side by side, multiple bowls have been put in bamboo basket, then position here is closed
System is the neighbouring relations between bamboo basket and bamboo basket, the neighbouring relations between bowl and bowl, inclusion relation and phase between bamboo basket and bowl
Adjacent relationship.
First machine learning model, can by convolutional neural networks (Convolutional Neural Network,
CNN) training of training sample is obtained.
Specifically, first machine learning model is obtained by following steps:
S2021, production training sample.
Photo captured by the daily law enforcement of municipal administration can be collected, every photo of collection as a training sample,
Most training samples form training sample set.
Label for labelling is carried out to each training sample that training sample is concentrated, the label of each training sample includes training sample
In article element title and article element quantity and/or size.
S2022, feature extraction is carried out to each training sample, obtains the feature vector of each training sample.
Convolution (Convolution) He Chihua (max-pooling) processing is carried out to each training sample, obtains each trained sample
This feature vector.
Specifically, for each training sample, object candidate frame can be first oriented, then carries out convolution sum for candidate frame
Pondization processing, extracts the feature vector of picture in candidate frame, to can get the feature vector of each training sample.
When for having multiple objects element in a samples pictures, the object candidate frame of identical quantity can be oriented.For example,
There are three object elements in one samples pictures can then orient three object candidate frames such as brick, cement heap and desk, Gu
Determine the candidate frame of the candidate frame of brick, the candidate frame of fixed cement heap and fixed desk.Accordingly, it is desirable to be directed to each candidate frame
The processing of convolution sum pondization is carried out, the feature vector of picture in each candidate frame is extracted, by each candidate in a samples pictures
The set of the feature vector of picture in frame, the feature vector as this samples pictures.
Due to when carrying out process of convolution, the size to the picture of input be it is fixed, therefore, when in a samples pictures
In when orienting multiple objects candidate frame, if multiple objects candidate frame is of different sizes, need to zoom to each candidate frame
Fixed size, in order to carry out the processing of convolution sum pondization.
S2023, the feature vector based on each training sample carry out model training, to obtain the first machine learning mould
Type.
It should be understood that carrying out model training in the feature vector based on each training sample, the first machine is obtained
After learning model, it is also necessary to be tested using test sample the first machine learning model.The selection or system of test sample
As the prior art, details are not described herein.
In the present embodiment, can by the more frame detectors of single sweep operation (Single Shot MultiBox Detector,
SSD it) is trained and obtains the first machine learning model.The present embodiment is without being limited thereto, can also be by other convolutional neural networks, such as
RCNN, Faster RCNN etc., which are trained, obtains the first machine learning model.
The feature and capture rate threshold of step 203, the article element for exporting first machine learning model
Value, the input as the second machine learning model substitute into the second machine learning model.
By the feature of the article element of first machine learning model output, it is integrated into one-dimensional vector, as the
The input of two machine learning models substitutes into the second machine learning model.
The input of second machine learning model further includes capture rate threshold value.Capture rate refers to the violation quantity of identification pair
With practical violation ratio of number.For example, to 15 article elements, if identifying it after the prediction of the second machine learning model
In have 10 article elements in violation of rules and regulations, in this 10 article elements for being identified as violation, wherein there is 8 to be identified as violation article member
The judgement of element is correctly that another 2 judgements for being identified as violation article element are wrong.In addition, in this 15 article elements
In, the quantity of practical violation is 12, then capture rate=8/12 ≈ 67%.
Capture rate threshold value can be manually set according to actual needs, for example be 80% or 85% etc..
Second machine learning model can obtain the training of training sample by convolutional neural networks.Specifically, institute
Stating the second machine learning model can be obtained by following steps:
S2031, training sample is obtained.
Two or more that the first machine learning model exports are obtained such as the article element in 36,49 or 64 pictures
Feature is as training sample, in addition to this, also using the capture rate threshold value of artificial settings as training sample, that is, is used as described second
The input of machine learning model.
Label for labelling is carried out to each training sample.The label of training sample corresponding with the feature of article element is in violation of rules and regulations
Or not in violation of rules and regulations, the label of training sample corresponding with capture rate threshold value is capture rate.
In another embodiment, the output of first machine learning model may also include each article element in picture
The confidence level of feature；Correspondingly, confidence level can be pressed in every picture from height from the feature of the article element in all pictures
To it is low choose predetermined quantity article element feature as training sample.For example, from the spy of the article element in all pictures
In sign, to the feature of each article element in every picture, it is ranked up according to confidence level, is pressed in every picture from high to low
Confidence level chooses the feature of (such as 64) article element of preset quantity as training sample from high to low.In this way, can reduce
For trained sample size, training speed is improved.
S2032, the processing of convolution sum pondization is carried out to the training sample, obtain the feature vector of the training sample.
The process of convolution sum pondization processing is recyclable multiple.In the present embodiment, the activation primitive of convolutional layer is using SELU (contracting
Put exponential type linear unit), the invention is not limited thereto, and other activation primitives can also be used.
S2033, the feature vector based on the training sample, with minimize input capture rate threshold value and output capture rate it
Between difference, and maximize accuracy rate be optimization aim carry out model training, to obtain the second machine learning model.
Accuracy rate refers to the violation quantity of identification pair and is identified as violation ratio of number.For example, to 15 article elements, if
After the prediction of the second machine learning model, identifying wherein has 10 article elements in violation of rules and regulations, is identified as violation at this 10
In article element, wherein there is 8 judgements for being identified as violation article element to be correctly, another 2 are identified as violation article element
Judgement be it is wrong, then accuracy rate be 80%.
In the present embodiment, based on the feature vector of the training sample, by connecting feedforward network (Fully entirely
Connect Feedforward Network), with minimize input capture rate threshold value and export capture rate between difference and
Maximizing accuracy rate is that optimization aim carries out model training, to obtain the second machine learning model.
In one embodiment, the optimization aim may is that minimum (capture rate threshold value-output capture of input
Rate) * 10+ minimum (1- accuracy rate).
In the present embodiment, under normal circumstances, the relationship between capture rate and accuracy rate are as follows: capture rate is higher, and accuracy rate is got over
It is low.Relationship between the two is just as the relationship between the sensitivity of radar and the accuracy rate of radar advisory, the sensitivity tune of radar
It is sensitiveer, enemy's situation fail to report it is fewer, but report by mistake it is bigger, that is, the accuracy rate reported is lower.
When concrete application, can flexible modulation capture rate threshold value as needed, can be according to the tune of capture rate threshold value by training
Section is automatically found best accuracy rate.In addition, carrying out model training according to the above-mentioned relation between capture rate and accuracy rate, obtain
To the second machine learning model, whether the article element itself in picture can be sentenced in violation of rules and regulations by the second machine learning model
Disconnected, this deterministic process can be relatively independent with the identification process to the article element in picture.
Trained process goes back the 1/3 of usable samples to assess, and every 1000 iterative estimation is primary, if assessment collection is defeated
Result cannot be more preferable out, terminates with regard to training.
It should be understood that after obtaining the second machine learning model, it is also necessary to using test sample to the second machine
Learning model is tested.
Step 204, second machine learning model are according to the capture rate threshold value of setting and the spy of the article element
Sign, whether the article element in the picture is predicted in violation of rules and regulations, and export whether the prediction result of violation.
In the present embodiment, after being identified by the first machine learning model to the article element in picture, further,
By the second machine learning model whether identification decision is carried out to the article element in the picture in violation of rules and regulations, and exports identification knot
Fruit, this way it is not necessary to reach scene be also able to achieve to city huddle material whether the identification of violation, improve in violation of rules and regulations identification effect
Rate, and human cost can be saved.
The recognition result be picture in article element whether the qualitative judgement of violation, can be obtained according to the recognition result
Whether in violation of rules and regulations to know the article element stacked in the picture scene.
In another embodiment, the output of second machine learning model further includes that the article element in the picture is
The confidence level of no violation.The recognition result exported by the confidence level and the second machine learning model, in the picture
Whether the material held in scene makes accurate judgement in violation of rules and regulations.
Embodiment three
Fig. 3 is the violation recognition methods flow chart that three city of the embodiment of the present invention huddles material, referring to Fig. 3, the present embodiment
On the basis of embodiment shown in Fig. 2, further includes:
Step 301, by the feature of the article element in the picture of first machine learning model output, and preset
Violation judge that the decision rule in library is compared, whether the article element is identified in violation of rules and regulations, and export identification knot
Fruit.
Wherein, the decision rule include article element and article element whether the mapping relations of violation.
For example, a judgment rule is the chair then violation for putting 2 or more, if being exported by the first machine learning model
A picture in article element feature be 3 chairs, then determine the chair put in the content scene of the picture disobey
Rule.
In the present embodiment, the second machine learning model and use judge library in violation of rules and regulations, the violation identification side parallel as two kinds
Formula can use parallel, can also select one way in which as needed, so that the more flexible multiplicity of violation identification method.
Example IV
Fig. 4 is the structural schematic diagram for the violation identification device that four city of the embodiment of the present invention huddles material, and the present embodiment is answered
For video monitoring system, whether the material stacked to the place such as the community, street, public place in city determines in violation of rules and regulations
Identification, using the reference frame enforced the law as municipal administration.The video monitoring system include front end camera and with the front end
The connected violation identification device of camera, the front end camera are installed on the positions such as the community, street, public place in city,
For carrying out Image Acquisition to the material stacked in Target monitoring area, and the image information of acquisition is sent to identification in violation of rules and regulations and is filled
It sets, whether the violation identification device determines the material stacked in the target area according to described image information in violation of rules and regulations
It identifies and exports recognition result.
As shown in figure 4, the device of the present embodiment may include: the first identification module 11 and the second identification module 12, wherein
First identification module 11 identifies the picture of front end camera shooting, exports institute for utilizing the first machine learning model
State the feature of the article element in picture；The feature of the article element includes the title and article element of article element
Quantity and/or size；Second identification module 12, the article element in picture for exporting first machine learning model
Feature whether the article element is identified in violation of rules and regulations as the input of the second machine learning model, and export identification knot
Fruit.
The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 1, realization principle and skill
Art effect is similar, and details are not described herein again.
Embodiment five
The device of the present embodiment, on the basis of the embodiment shown in fig. 4, further, first machine learning model
It is obtained by following steps:
Make training sample；Wherein, the label of each training sample include the article element in training sample title, with
And the quantity and/or size of article element；
Feature extraction is carried out to each training sample, obtains the feature vector of each training sample；
Feature vector based on each training sample carries out model training, to obtain the first machine learning model.
The input of second machine learning model further includes capture rate threshold value；
Second identification module, specifically for the article element in the picture that exports first machine learning model
Feature and capture rate threshold value, as the second machine learning model input substitute into the second machine learning model；Described second
Machine learning model is according to the capture rate threshold value of setting and the feature of the article element, to the article member in the picture
Element whether predicted in violation of rules and regulations, and export whether the prediction result of violation.
Second machine learning model is obtained by following steps:
The feature for the article element in two or more pictures that capture rate threshold value and the first machine learning model are exported
As training sample；Wherein, the label of training sample corresponding with the feature of article element is in violation of rules and regulations or not in violation of rules and regulations, with capture
The label of the corresponding training sample of rate threshold value is capture rate；
The processing of convolution sum pondization is carried out to the training sample, obtains the feature vector of the training sample；
Based on the feature vector of the training sample, to minimize input capture rate threshold value and export the difference between capture rate
Value, and maximizing accuracy rate is that optimization aim carries out model training, to obtain the second machine learning model.
The output of first machine learning model further includes the confidence level of the feature of each article element in picture；It is described
Second identification module 12, including sample acquisition submodule, for choosing two or more pictures of the first machine learning model output
In article element feature；From the feature of the article element in all pictures of selection, confidence level is pressed in every picture
The feature of the article element of predetermined quantity, the training sample as the second machine learning model are chosen from high to low.
The output of second machine learning model further include article element in the picture whether the confidence level of violation.
The feature of the article element further include: the group between positional relationship and article element between article element
Conjunction relationship.
The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 2, realization principle and skill
Art effect is similar, and details are not described herein again.
Embodiment six
Fig. 5 is the structural schematic diagram for the violation identification device that six city of the embodiment of the present invention huddles material, and Fig. 6 is the present invention
Six city of embodiment huddles the data flow and control flow diagram of the violation identification device of material, and wherein solid line indicates data flow,
Dotted line indicates control stream.Referring to Fig. 5 and Fig. 6, the device of the present embodiment, on the basis of device shown in Fig. 4, further includes:
Third identification module 13, the spy of the article element in picture for exporting first machine learning model
Whether sign, judge that the decision rule in library is compared with preset violation, identify in violation of rules and regulations to the article element,
And export recognition result；Wherein, the decision rule include article element and article element whether the mapping relations of violation.
The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 3, realization principle and skill
Art effect is similar, and details are not described herein again.
Embodiment seven
Fig. 7 is the structural schematic diagram of seven electronic equipment of the embodiment of the present invention, and Fig. 1-3 any embodiment of the present invention may be implemented
Process, as shown in fig. 7, above-mentioned electronic equipment may include: shell 41, processor 42, memory 43, circuit board 44 and power supply
Circuit 45, wherein circuit board 44 is placed in the space interior that shell 41 surrounds, and processor 42 and memory 43 are arranged in circuit board
On 44；Power circuit 45, for each circuit or the device power supply for above-mentioned electronic equipment；Memory 43 is executable for storing
Program code；Processor 42 is run by reading the executable program code stored in memory 43 and executable program code
Corresponding program huddles the violation recognition methods of material for executing city described in aforementioned any embodiment.
Processor 42 to the specific implementation procedures of above-mentioned steps and processor 42 by operation executable program code come
The step of further executing may refer to the description of Fig. 1-3 illustrated embodiment of the present invention, and details are not described herein.
The electronic equipment exists in a variety of forms, including but not limited to:
(1) mobile communication equipment: the characteristics of this kind of equipment is that have mobile communication function, and to provide speech, data
Communication is main target.This Terminal Type includes: smart phone (such as iPhone), multimedia handset, functional mobile phone and low
Hold mobile phone etc..
(2) super mobile personal computer equipment: this kind of equipment belongs to the scope of personal computer, there is calculating and processing function
Can, generally also have mobile Internet access characteristic.This Terminal Type includes: PDA, MID and UMPC equipment etc., such as iPad.
(3) server: providing the equipment of the service of calculating, and the composition of server includes that processor, hard disk, memory, system are total
Line etc., server is similar with general computer architecture, but due to needing to provide highly reliable service, in processing energy
Power, stability, reliability, safety, scalability, manageability etc. are more demanding.
(4) other electronic equipments with data interaction function.
The embodiment of the present invention also provides a kind of computer readable storage medium, and the computer-readable recording medium storage has
One or more program, one or more of programs can be executed by one or more processor, to realize aforementioned
City described in one embodiment huddles the violation recognition methods of material.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence " including one ... ", it is not excluded that
There is also other identical elements in the process, method, article or apparatus that includes the element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.
For Installation practice, since it is substantially similar to the method embodiment, so the comparison of description is simple
Single, the relevent part can refer to the partial explaination of embodiments of method.
For convenience of description, description apparatus above is to be divided into various units/modules with function to describe respectively.Certainly, exist
Implement to realize each unit/module function in the same or multiple software and or hardware when the present invention.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the program can be stored in a computer-readable storage medium
In, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, the storage medium can be magnetic
Dish, CD, read-only memory (Read-Only Memory, ROM) or random access memory (Random Access
Memory, RAM) etc..
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
In the technical scope disclosed by the present invention, any changes or substitutions that can be easily thought of by those familiar with the art, all answers
It is included within the scope of the present invention.Therefore, protection scope of the present invention should be subject to the protection scope in claims.
Claims (10)
1. the violation recognition methods that a kind of city huddles material characterized by comprising
Using the first machine learning model, the picture of front end camera shooting is identified, the article in the picture is exported
The feature of element；The feature of the article element includes the title of article element and the quantity and/or size of article element；
By the feature of the article element in the picture of first machine learning model output, as the second machine learning model
Whether input, identify the article element, and export recognition result in violation of rules and regulations.
2. the violation recognition methods that city according to claim 1 huddles material, which is characterized in that first engineering
Model is practised to obtain by following steps:
Make training sample；Wherein, the label of each training sample includes the title and object of the article element in training sample
The quantity and/or size of product element；
Feature extraction is carried out to each training sample, obtains the feature vector of each training sample；
Feature vector based on each training sample carries out model training, to obtain the first machine learning model.
3. the violation recognition methods that city according to claim 1 huddles material, which is characterized in that second engineering
The input for practising model further includes capture rate threshold value；
The feature by the article element in the picture of first machine learning model output, as the second machine learning mould
Whether the input of type identifies the article element in violation of rules and regulations, and exports recognition result, comprising:
By the feature and capture rate threshold value of the article element in the picture of first machine learning model output, as the
The input of two machine learning models substitutes into the second machine learning model；
Second machine learning model is according to the capture rate threshold value of setting and the feature of the article element, to the figure
Whether the article element in piece predicted in violation of rules and regulations, and export whether the prediction result of violation.
4. the violation recognition methods that city according to claim 1 huddles material, which is characterized in that second engineering
Model is practised to obtain by following steps:
The feature for the article element in two or more pictures that capture rate threshold value and the first machine learning model are exported as
Training sample；Wherein, the label of training sample corresponding with the feature of article element is in violation of rules and regulations or not in violation of rules and regulations, with capture rate threshold
The label for being worth corresponding training sample is capture rate；
The processing of convolution sum pondization is carried out to the training sample, obtains the feature vector of the training sample；
Based on the feature vector of the training sample, to minimize input capture rate threshold value and export the difference between capture rate,
And maximizing accuracy rate is that optimization aim carries out model training, to obtain the second machine learning model.
5. the violation recognition methods that city according to claim 4 huddles material, which is characterized in that first engineering
The output of habit model further includes the confidence level of the feature of each article element in picture；
Using the feature of the article element in two or more pictures of the first machine learning model output as training sample, comprising:
Choose the feature of the article element in two or more pictures of the first machine learning model output；
From the feature of the article element in all pictures of selection, chooses and make a reservation for from high to low by confidence level in every picture
The feature of the article element of quantity, the training sample as the second machine learning model.
6. the violation recognition methods that city according to claim 3 huddles material, which is characterized in that second engineering
Practise model output further include article element in the picture whether the confidence level of violation.
7. the violation recognition methods that city according to claim 1 huddles material, which is characterized in that the article element
Feature further include: the syntagmatic between positional relationship and article element between article element.
8. the violation recognition methods that city according to claim 1 huddles material, which is characterized in that further include:
By the feature of the article element in the picture of first machine learning model output, library is judged with preset violation
In decision rule be compared, whether the article element is identified in violation of rules and regulations, and export recognition result；Wherein, described
Decision rule include article element and article element whether the mapping relations of violation.
9. the violation identification device that a kind of city huddles material characterized by comprising
First identification module identifies the picture of front end camera shooting, exports for utilizing the first machine learning model
The feature of article element in the picture；The feature of the article element includes the title and article element of article element
Quantity and/or size；
Second identification module, the feature of the article element in picture for exporting first machine learning model, as
Whether the input of the second machine learning model identifies the article element in violation of rules and regulations, and exports recognition result.
10. a kind of electronic equipment, which is characterized in that the electronic equipment includes: shell, processor, memory, circuit board and electricity
Source circuit, wherein circuit board is placed in the space interior that shell surrounds, and processor and memory setting are on circuit boards；Power supply
Circuit, for each circuit or the device power supply for above-mentioned electronic equipment；Memory is for storing executable program code；Processing
Device runs program corresponding with executable program code by reading the executable program code stored in memory, for holding
City described in the aforementioned any claim 1-8 of row huddles the violation recognition methods of material.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711470655 | 2017-12-28 | ||
| CN2017114706555 | 2017-12-28 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109993047A true CN109993047A (en) | 2019-07-09 |
Family
ID=67128424
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810787496.XA Pending CN109993047A (en) | 2017-12-28 | 2018-07-17 | City huddles violation recognition methods, device and the electronic equipment of material |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109993047A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110458082A (en) * | 2019-08-05 | 2019-11-15 | 城云科技（中国）有限公司 | A kind of city management case classification recognition methods |
| CN112861716A (en) * | 2021-02-05 | 2021-05-28 | 深圳市安软科技股份有限公司 | Illegal article placement monitoring method, system, equipment and storage medium |
| CN113221804A (en) * | 2021-05-25 | 2021-08-06 | 城云科技（中国）有限公司 | Disordered material detection method and device based on monitoring video and application |
| CN113378602A (en) * | 2020-03-10 | 2021-09-10 | 顺丰科技有限公司 | Method and device for detecting illegal stacking of articles, server and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102034212A (en) * | 2010-06-21 | 2011-04-27 | 艾浩军 | City management system based on video analysis |
| CN103150579A (en) * | 2013-02-25 | 2013-06-12 | 东华大学 | Abnormal human behavior detecting method based on video sequence |
| CN103676829A (en) * | 2013-09-11 | 2014-03-26 | 无锡加视诚智能科技有限公司 | An intelligent urban integrated management system based on videos and a method thereof |
| CN104508671A (en) * | 2012-06-21 | 2015-04-08 | 菲利普莫里斯生产公司 | System and method for generating biomarker signatures through integrated bias correction and classification prediction |
| CN104834912A (en) * | 2015-05-14 | 2015-08-12 | 北京邮电大学 | Weather identification method and apparatus based on image information detection |
| CN104881674A (en) * | 2015-03-19 | 2015-09-02 | 智联通建筑科技（北京）有限公司 | Identification method and system as well as estimation method and system for lamp on-off state |
| CN106230849A (en) * | 2016-08-22 | 2016-12-14 | 中国科学院信息工程研究所 | A kind of smart machine machine learning safety monitoring system based on user behavior |
| CN106874926A (en) * | 2016-08-04 | 2017-06-20 | 阿里巴巴集团控股有限公司 | Service exception detection method and device based on characteristics of image |
| CN107016356A (en) * | 2017-03-21 | 2017-08-04 | 乐蜜科技有限公司 | Certain content recognition methods, device and electronic equipment |
| CN107133968A (en) * | 2017-06-07 | 2017-09-05 | 武汉洁美雅科技有限公司 | A kind of garbage disposal control system based on Internet of Things |
| CN107222865A (en) * | 2017-04-28 | 2017-09-29 | 北京大学 | The communication swindle real-time detection method and system recognized based on suspicious actions |
| CN107426231A (en) * | 2017-08-03 | 2017-12-01 | 北京奇安信科技有限公司 | A kind of method and device for identifying user behavior |
-
2018
- 2018-07-17 CN CN201810787496.XA patent/CN109993047A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102034212A (en) * | 2010-06-21 | 2011-04-27 | 艾浩军 | City management system based on video analysis |
| CN104508671A (en) * | 2012-06-21 | 2015-04-08 | 菲利普莫里斯生产公司 | System and method for generating biomarker signatures through integrated bias correction and classification prediction |
| CN103150579A (en) * | 2013-02-25 | 2013-06-12 | 东华大学 | Abnormal human behavior detecting method based on video sequence |
| CN103676829A (en) * | 2013-09-11 | 2014-03-26 | 无锡加视诚智能科技有限公司 | An intelligent urban integrated management system based on videos and a method thereof |
| CN104881674A (en) * | 2015-03-19 | 2015-09-02 | 智联通建筑科技（北京）有限公司 | Identification method and system as well as estimation method and system for lamp on-off state |
| CN104834912A (en) * | 2015-05-14 | 2015-08-12 | 北京邮电大学 | Weather identification method and apparatus based on image information detection |
| CN106874926A (en) * | 2016-08-04 | 2017-06-20 | 阿里巴巴集团控股有限公司 | Service exception detection method and device based on characteristics of image |
| CN106230849A (en) * | 2016-08-22 | 2016-12-14 | 中国科学院信息工程研究所 | A kind of smart machine machine learning safety monitoring system based on user behavior |
| CN107016356A (en) * | 2017-03-21 | 2017-08-04 | 乐蜜科技有限公司 | Certain content recognition methods, device and electronic equipment |
| CN107222865A (en) * | 2017-04-28 | 2017-09-29 | 北京大学 | The communication swindle real-time detection method and system recognized based on suspicious actions |
| CN107133968A (en) * | 2017-06-07 | 2017-09-05 | 武汉洁美雅科技有限公司 | A kind of garbage disposal control system based on Internet of Things |
| CN107426231A (en) * | 2017-08-03 | 2017-12-01 | 北京奇安信科技有限公司 | A kind of method and device for identifying user behavior |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110458082A (en) * | 2019-08-05 | 2019-11-15 | 城云科技（中国）有限公司 | A kind of city management case classification recognition methods |
| CN110458082B (en) * | 2019-08-05 | 2022-05-03 | 城云科技（中国）有限公司 | Urban management case classification and identification method |
| CN113378602A (en) * | 2020-03-10 | 2021-09-10 | 顺丰科技有限公司 | Method and device for detecting illegal stacking of articles, server and storage medium |
| CN112861716A (en) * | 2021-02-05 | 2021-05-28 | 深圳市安软科技股份有限公司 | Illegal article placement monitoring method, system, equipment and storage medium |
| CN113221804A (en) * | 2021-05-25 | 2021-08-06 | 城云科技（中国）有限公司 | Disordered material detection method and device based on monitoring video and application |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Visual semantic reasoning for image-text matching | |
| Jiang et al. | Cross-modal video moment retrieval with spatial and language-temporal attention | |
| CN109993047A (en) | City huddles violation recognition methods, device and the electronic equipment of material | |
| Dai et al. | Toyota smarthome untrimmed: Real-world untrimmed videos for activity detection | |
| CN109344787A (en) | A kind of specific objective tracking identified again based on recognition of face and pedestrian | |
| Jiang et al. | Few-shot food recognition via multi-view representation learning | |
| Xu et al. | An HMM-based framework for video semantic analysis | |
| CN104331509A (en) | Picture managing method and device | |
| CN109213882A (en) | Picture sort method and terminal | |
| CN106611151B (en) | A kind of face identification method and device | |
| CN108416314A (en) | The important method for detecting human face of picture | |
| CN110414360A (en) | A kind of detection method and detection device of abnormal behaviour | |
| CN110399822A (en) | Action identification method of raising one's hand, device and storage medium based on deep learning | |
| Truong et al. | Lifelogging retrieval based on semantic concepts fusion | |
| CN110443179A (en) | It leaves the post detection method, device and storage medium | |
| CN106250396B (en) | Automatic image label generation system and method | |
| Guo | Action recognition using log-covariance matrices of silhouette and optical-flow features | |
| Lao et al. | FedVQA: Personalized Federated Visual Question Answering over Heterogeneous Scenes | |
| CN106714099A (en) | Photograph information processing and scenic spot identification method, client and server | |
| CN105512155A (en) | Device and method for multi-layer semantic image retrieval | |
| Damen et al. | Epic-kitchens-100 | |
| Siadari et al. | 4D effect video classification with shot-aware frame selection and deep neural networks | |
| CN111984852B (en) | Generating image acquisition | |
| WO2023039331A1 (en) | Object detection systems and methods including an object detection model using a tailored training dataset | |
| CN103034785A (en) | System and method for acquiring health information by mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190709 |