CN113034546B

CN113034546B - Track merging method and device, electronic equipment and storage medium

Info

Publication number: CN113034546B
Application number: CN202110371563.1A
Authority: CN
Inventors: 陈士辉; 项伯贵; 陈益新
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2021-04-07
Filing date: 2021-04-07
Publication date: 2024-08-02
Anticipated expiration: 2041-04-07
Also published as: CN113034546A

Abstract

The embodiment of the application provides a track merging method, a track merging device, electronic equipment and a storage medium, which relate to the technical field of data processing and comprise the following steps: obtaining a motion trail of a current object generated based on the image acquired by the image acquisition device; obtaining local identity characteristics of the current object according to the image acquired by the image acquisition equipment; judging whether a target object with the similarity of local identity characteristics between the archived object and the current object being greater than or equal to a first preset threshold value exists in the archived object; if the current object exists, determining that the current object and the target object are the same object, and merging the motion trail of the target object and the current object; if not, judging whether the current object is a strange object, if so, creating object information for the current object, and realizing archiving of the current object. By applying the scheme provided by the embodiment of the application, the motion trails belonging to the same object can be combined.

Description

Track merging method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a track merging method, a track merging device, an electronic device, and a storage medium.

Background

In security monitoring scenes, in order to better manage objects such as personnel and vehicles in the scene, it is generally necessary to obtain a motion track of the objects.

In the related art, an image acquisition device may be deployed in the above-described scene, and a motion trail of an object within a monitoring range of the image acquisition device may be obtained based on an image acquired by the image acquisition device. Because the area range of the security monitoring scene is usually larger, and the monitoring area of the image acquisition equipment is usually limited, a plurality of image acquisition equipment needs to be deployed in the security monitoring scene, and different image acquisition equipment is utilized to obtain the motion trail of the object in different monitoring areas.

Since the motion trajectories obtained by using different image capturing devices may be motion trajectories of the same object, a trajectory merging scheme is now needed to merge motion trajectories belonging to the same object.

Disclosure of Invention

The embodiment of the application aims to provide a track merging method, a track merging device, electronic equipment and a storage medium, so as to merge motion tracks belonging to the same object. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a track merging method, where the method includes:

Obtaining a motion trail of a current object generated based on the image acquired by the image acquisition device;

obtaining local identity characteristics of the current object according to the image acquired by the image acquisition equipment;

Judging whether a target object with the similarity of local identity characteristics with the current object is larger than or equal to a first preset threshold exists in the archived object, wherein the archived object is: an object for which object information has been created;

if the current object exists, determining that the current object and the target object are the same object, and merging the motion trail of the target object and the current object;

if not, judging whether the current object is a strange object, if so, creating object information for the current object, and realizing archiving of the current object.

In one embodiment of the present application, after the determining that the current object and the target object are the same object, the method further includes:

marking the obtained motion trail by utilizing the object identification of the target object;

The merging the motion trail of the target object and the current object comprises the following steps:

And merging the motion trail of the object mark marked with the target object.

In one embodiment of the application, the method further comprises:

for any two archived tracks, merging the two archived tracks under the condition that the two archived tracks meet a preset merging condition, merging object information of objects to which the two archived tracks belong, and determining the object information of the objects to which the merged motion track belongs as merged object information, wherein the archived tracks are as follows: the object is the motion trail of the archived object, and the merging condition is as follows: the method meets the preset space reachable condition, and the similarity between the integral identity characteristics of the belonged objects is larger than or equal to a second preset threshold value.

In one embodiment of the present application, the spatially reachable conditions include:

The distance between the end position of the motion track and the initial position of the obtained motion track is smaller than or equal to a preset distance threshold value; and/or

The duration between the ending time of the motion trail and the starting time of the obtained motion trail is smaller than or equal to a preset duration threshold; and/or

And calculating the movement speed of the object based on the termination position, the starting position, the termination time and the starting time to be less than or equal to a preset speed threshold.

In one embodiment of the present application, the determining whether the current object is a strange object includes:

extracting local identity features from the image acquired by the image acquisition equipment to obtain the confidence that the extracted local identity features characterize the identity of the current object;

And judging whether the confidence coefficient of the local identity feature is larger than or equal to a preset confidence coefficient threshold value, if so, determining that the current object is a strange object.

In one embodiment of the application, the method further comprises:

Judging whether a target track meeting a merging condition with the obtained motion track exists in the archived track under the condition that the current object is not a strange object, wherein the archived track is: the object is the motion trail of the archived object, and the merging condition is as follows: the method meets the preset space reachable condition, and the similarity between the integral identity characteristics of the belonged objects is more than or equal to a second preset threshold value;

and if the motion track exists, merging the obtained motion track with the target track.

In one embodiment of the present application, the merging the motion trajectories of the target object and the current object includes:

If a plurality of target objects exist in the archived object, merging object information of the plurality of target objects to obtain merged object information, merging motion trajectories of the plurality of target objects and the obtained motion trajectories, and determining object information of an object to which the merged motion trajectories belong as the merged object information.

In one embodiment of the present application, the merging the object information of the plurality of target objects to obtain merged object information includes:

Determining object information of a target object with highest feature confidence degree in the plurality of target objects as combined object information, wherein the feature confidence degree of each object is characterized by: characterizing the confidence of the identity of the object based on the object features of the object extracted from the image, wherein the object features comprise: local identity features and/or global identity features.

In a second aspect, an embodiment of the present application provides a track merging device, including:

the motion trail obtaining module is used for obtaining the motion trail of the current object generated based on the image acquired by the image acquisition equipment;

The first characteristic obtaining module is used for obtaining the local identity characteristic of the current object according to the image acquired by the image acquisition equipment;

The system comprises a target object judging module, a triggering track merging module and an object archiving module, wherein the target object judging module is used for judging whether a target object with the similarity of local identity characteristics between the current object and the archived object is larger than or equal to a first preset threshold value exists, and if the target object exists, the triggering track merging module triggers the object archiving module, and if the target object does not exist, the archived object is: an object for which object information has been created;

the track merging module is used for determining that the current object and the target object are the same object and merging the motion tracks of the target object and the current object;

and the object archiving module is used for judging whether the current object is a strange object or not, if so, creating object information for the current object, and archiving the current object.

In one embodiment of the application, the apparatus further comprises:

The track marking module is used for marking the obtained motion track by utilizing the object identification of the target object after determining that the current object and the target object are the same object;

the track merging module is specifically configured to:

and determining that the current object and the target object are the same object, and merging the motion trail marked with the object identifier of the target object.

In one embodiment of the present application, the apparatus further includes a first determining module configured to:

In one embodiment of the present application, the object archiving module is specifically configured to:

And judging whether the confidence coefficient of the local identity feature is greater than or equal to a preset confidence coefficient threshold value, if so, determining that the current object is a strange object, and creating object information for the current object to file the current object.

In one embodiment of the present application, the apparatus further includes a second judging module configured to:

In one embodiment of the present application, the track merging module includes:

an object determining unit, configured to determine that the current object and the target object are the same object;

the information merging unit is used for merging object information of a plurality of target objects if the plurality of target objects exist in the archived objects to obtain merged object information;

and the track merging unit is used for merging the motion tracks of the plurality of target objects and the obtained motion tracks, and determining the object information of the object to which the merged motion track belongs as the merged object information.

In one embodiment of the present application, the information merging unit is specifically configured to:

if a plurality of target objects exist in the archived objects, determining object information of a target object with highest feature confidence in the plurality of target objects as combined object information, wherein the feature confidence of each object is characterized by: characterizing the confidence of the identity of the object based on the object features of the object extracted from the image, wherein the object features comprise: local identity features and/or global identity features.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

A memory for storing a computer program;

A processor configured to implement the method of any one of the first aspects when executing a program stored on a memory.

In a fourth aspect, embodiments of the present application provide a computer readable storage medium having a computer program stored therein, which when executed by a processor implements the method of any of the first aspects.

Embodiments of the present application also provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform any of the track merging methods described above.

The embodiment of the application has the beneficial effects that:

In the track merging scheme provided by the embodiment of the application, a motion track of a current object generated based on an image acquired by image acquisition equipment is firstly obtained; obtaining local identity characteristics of a current object according to an image acquired by image acquisition equipment; judging whether a target object with the similarity of local identity characteristics with the current object being greater than or equal to a first preset threshold exists in the archived object, wherein the archived object is: an object for which object information has been created; if the object information exists, the object information is created for the current object before the current object is described, and the target object which is the same as the current object exists in the archived object, so that the current object and the target object can be determined to be the same object, and the motion trail of the target object and the motion trail of the current object are combined, and therefore the combination of different motion trail of the target object can be achieved. If the object information does not exist, the object information is not created for the current object before the current object is described, so that whether the current object is a strange object or not can be judged, if the object information is not created for the current object, the current object is archived, and the movement tracks of the current object can be conveniently combined later. Therefore, by applying the scheme provided by the embodiment of the application, the motion tracks belonging to the same object can be combined.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are necessary for the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the application and that other embodiments may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic flow chart of a track merging method according to an embodiment of the present application;

FIG. 2 is a flowchart illustrating another track merging method according to an embodiment of the present application;

Fig. 3 is a schematic structural diagram of a track merging device according to an embodiment of the present application;

Fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

In order to realize the combination of motion trajectories belonging to the same object, embodiments of the present application provide a trajectory combination method, a device, an electronic device, and a storage medium, which are described in detail below.

Referring to fig. 1, fig. 1 is a flowchart of a track merging method according to an embodiment of the present application, where the method may be applied to electronic devices such as an electronic computer, a mobile phone, an NVR (Network Video Recorder, a network video recorder), a DVR (Digital Video Recorder, a hard disk recorder), and the like. The track merging method comprises the following steps S101-S105:

S101, a motion trajectory of a current object generated based on an image acquired by an image acquisition device is obtained.

The current object may be a person, a vehicle, an animal, or the like.

Specifically, at least one image acquisition device may be disposed in the security monitoring scene, and when an object moves in the monitoring area of the image acquisition device, the image acquisition device may acquire an image of the object, and according to the image, a motion track of the object may be obtained and used as a motion track of the current object.

The security monitoring scene can be a mall, a factory, an office building, a station, a school and the like. And (3) motion trail characterization: changes in the spatial position of an object over a period of time.

In one embodiment of the present application, an image of an object acquired by an image acquisition apparatus may be first obtained, and then a motion trajectory of the object may be determined from the obtained image. In addition, the motion trail of the object may be obtained by an image acquisition device, a trail determination device, or the like based on the acquired image, and then the motion trail may be transmitted to an electronic device, which may directly obtain the motion trail of the object.

In one embodiment of the application, when the motion trail of the object is obtained based on the images, the image position of the object in each frame of images can be obtained, then the actual space position of the object is obtained according to the preset position conversion relation between the image position and the actual space position, and then the actual space position of the object obtained based on each frame of images is sequenced according to the acquisition time sequence among the frames of images, so that the motion trail of the object is obtained.

In addition, the image can be input into a track generation model which is trained in advance, so that the motion track of the object in the image, which is output by the track generation model, can be obtained.

S102, according to the image acquired by the image acquisition equipment, the local identity characteristic of the current object is obtained.

Wherein, the local identity features are: a feature of the object part that characterizes the identity of the object. In the case that the object is a person, the local identity feature may be a facial feature, a chest card feature, or the like; in the case where the object is a vehicle, the local identity feature may be a license plate feature or the like.

Specifically, the local identity characteristic of the object can be obtained by using the image besides the motion trail of the object based on the image acquired by the image acquisition device, so that the target object which is the same as the current object can be conveniently searched for according to the local identity characteristic of the current object.

In one embodiment of the application, when the local identity characteristic of the object is obtained, a local image area of the object in the image can be first identified, and then the image characteristic of the local area is extracted as the local identity characteristic of the object.

In addition, the image can be input into a pre-trained local identity characteristic model to obtain the local identity characteristic of the object output by the model.

In one embodiment of the application, when the local identity feature of the object is obtained, the confidence degree of the local identity feature for representing the identity of the object can be obtained, and the confidence degree can be used as a score to reflect the accuracy of the local identity feature. Therefore, a plurality of images in the object movement process can be obtained, local identity features and confidence degrees of the object are obtained for each image respectively, then the local identity features are fused based on the confidence degrees of the local identity features, and the fusion result is used as the final local identity feature of the object.

For example, for the obtained plurality of local identity features and confidence levels, the local identity feature with the highest confidence level may be selected as the final local identity feature of the object; or selecting local identity features with confidence coefficient higher than a preset threshold value, then fusing the selected local identity features, and taking the fused local identity features as final local identity features of the object.

S103, judging whether a target object with the similarity of the local identity characteristics with the current object in the archived object is larger than or equal to a first preset threshold value, if so, executing S104, and if not, executing S105.

Wherein, the target object can be understood as: an archived object that is the same object as the current object, the archived object being: an object of the object information has been created.

The object information of each object may include: an object identification created for the object. The object identifier may be one or a combination of a number, a chinese character, an english character, a special symbol, etc.

Specifically, the archived object may be an object in an image acquired by another image acquisition device before, the motion track of the object may have been obtained based on the image acquired before in S101, in order to find the motion track of the same object, similarity calculation may be performed with the local identity features of each archived object one by one based on the obtained local identity features, so as to obtain the similarity of the local identity features between the current object and each archived current object, and whether the similarity is greater than or equal to a first preset threshold value is determined, if so, it is stated that object information has been created for the current object before, and the current object may be considered as the archived object. Further, the archived object whose corresponding similarity is equal to or greater than the first preset threshold may be regarded as the target object that is the same object as the current object.

In one embodiment of the present application, when determining whether or not a target object exists in the archived object, it may be determined whether or not a target motion trajectory exists in the motion trajectories of the archived object, and if so, it may be considered that a target object exists in the archived object, and further, the archived object to which the target motion trajectory belongs may be regarded as the target object that is the same object as the current object.

Wherein, the target motion trail is as follows: a motion track which has a partial space-time coincidence track with the obtained motion track and has the similarity between the local identity characteristics of the object and the current object reaching a first preset threshold value, wherein the space-time coincidence refers to: the position and time of the track coincide.

Specifically, since there may be a coincidence region between the monitoring regions of different image capturing apparatuses, there may also be a partial coincidence locus between the motion loci of the same object obtained based on the different image capturing apparatuses. If there is a motion trail of the archived object partially overlapping with the motion trail of the current object and the similarity of local identity features between the archived object and the current object is high, it may be indicated that the archived object and the current object are the same object, i.e., the current object is the archived object.

S104, determining that the current object and the target object are the same object, and merging the motion trail of the target object and the current object.

Specifically, when the similarity of the local identity features between the target object and the current object reaches the first preset threshold, the current object and the target object may be considered to be the same object, so that the motion trail of the target object and the motion trail of the current object obtained in S101 may be combined, thereby implementing combination of different motion trail of the same object.

In one embodiment of the application, when the track combination is performed, the motion tracks of the target object and the current object can be spliced according to time sequence, so that the combination of different motion tracks of the same object is realized.

In addition, when track merging is performed, whether an overlapping portion exists between the motion tracks to be merged or not can be judged first, if so, the overlapping portion can be merged first, and then the merged motion tracks and the motion tracks of the non-overlapping portion are spliced according to a time sequence, so that the merged motion tracks are obtained.

Specifically, since the motion trajectory is obtained based on the image acquired by the image acquisition device, the motion trajectory can be understood as: the track generated when the object moves in the monitoring area of the image acquisition equipment, and overlapping parts possibly exist between the monitoring areas of different image acquisition equipment, so that the obtained motion tracks can also have overlapping parts, the motion tracks of the overlapping parts are fused, then the fused motion tracks and the motion tracks of the non-overlapping parts are spliced, and the accuracy of the obtained motion tracks is higher.

S105, judging whether the current object is a strange object, if so, creating object information for the current object, and realizing archiving of the current object.

Specifically, under the condition that the current object is judged not to belong to the filed object, whether the current object is a strange object or not can be further judged, if so, it can be further determined that object information is not created for the current object, so that object information can be created for the current object, the filing of the current object is realized, the obtained motion trail of the current object is stored, and the obtained motion trail and the motion trail are conveniently combined in the follow-up.

In the track merging scheme provided in the above embodiment, a motion track of a current object generated based on an image acquired by an image acquisition device is first obtained; obtaining local identity characteristics of a current object according to an image acquired by image acquisition equipment; judging whether a target object with the similarity of local identity characteristics with the current object being greater than or equal to a first preset threshold exists in the archived object, wherein the archived object is: an object for which object information has been created; if the object information exists, the object information is created for the current object before the current object is described, and the target object which is the same as the current object exists in the archived object, so that the current object and the target object can be determined to be the same object, and the motion trail of the target object and the motion trail of the current object are combined, and therefore the combination of different motion trail of the target object can be achieved. If the object information does not exist, the object information is not created for the current object before the current object is described, so that whether the current object is a strange object or not can be judged, if the object information is not created for the current object, the current object is archived, and the movement tracks of the current object can be conveniently combined later. Therefore, by applying the scheme provided by the embodiment, the motion tracks belonging to the same object can be combined.

In one embodiment of the present application, after determining that the current object and the target object are the same object in S104, the obtained motion trail may be further marked by using an object identifier of the target object.

Specifically, the motion trail obtained in S101 may be marked as an object identifier of the target object, so as to indicate that the object to which the motion trail belongs is: the marked object identifies the corresponding object.

In this way, when the motion trajectories of the target object and the current object are combined in S104, the motion trajectories of the object identifier marked with the target object may be combined.

Specifically, under the condition that different motion trajectories are marked with the same object identifier, the fact that the different motion trajectories belong to the same object is indicated, so that the different motion trajectories can be combined according to time sequence, and different motion trajectories of the same object can be combined.

In addition, after determining that the object to which the obtained motion trail belongs is the target object in S102, an association relationship between the target object and the obtained motion trail may be established, so that when the trail combination is performed in S103, for each object, each motion trail associated with the object may be combined, thereby implementing the combination of the motion trail of the object. In one embodiment of the present application, for any two archived tracks, if the two archived tracks satisfy a preset merging condition, merging the two archived tracks, merging object information of objects to which the two archived tracks belong, and determining that object information of objects to which the merged motion track belongs is merged object information.

Wherein, the archived track is: the object is the motion trail of the archived object, and the merging condition is as follows: the method meets the preset space reachable condition, and the similarity between the integral identity characteristics of the belonged objects is larger than or equal to a second preset threshold value.

Characterization of the above spatially reachable conditions: between the positions reflected by the different trajectories is temporally and spatially reachable.

The integral identity features are as follows: the identity of the object can be characterized by the characteristics of the whole object. In the case that the object is a person, the overall identity feature may be a human body feature, including a physical feature of the person, a wearing feature, a hairstyle feature, and the like; in the case where the object is a vehicle, the overall identity feature may be a vehicle model feature, a vehicle body color feature, or the like.

Specifically, the archived tracks of the archived objects may be matched in pairs, whether the two archived tracks satisfy the space reachable condition and whether the overall identity characteristics of the objects are matched may be determined, if so, the objects to which the two archived tracks belong may be considered to be actually the same object, so that the object information of the objects to which the two archived tracks belong may be combined, and the two archived tracks may be combined.

In one embodiment of the present application, the archived tracks may be merged at predetermined intervals according to the scheme described above. The predetermined period may be 5 minutes, 10 minutes, 1 hour, etc. And the method can trigger the scheme to merge the archived tracks after receiving the track merging instruction.

In one embodiment of the application, when the integral identity characteristic of the object is obtained, the image characteristic of the integral region of the image can be extracted as the integral identity characteristic of the object.

In addition, the image can be input into a pre-trained integral identity characteristic model to obtain the integral identity characteristic of the object output by the model.

In one embodiment of the application, when the integral identity feature of the object is obtained, the confidence coefficient of the integral identity feature for representing the identity of the object can be obtained, and the confidence coefficient can be used as a score to reflect the accuracy of the integral identity feature.

Therefore, a plurality of images in the object movement process can be obtained, the integral identity characteristics and the confidence coefficient of the object are obtained for each image respectively, then the integral identity characteristics are fused based on the confidence coefficient of each integral identity characteristic, and the fusion result is used as the final integral identity characteristics of the object.

For example, for the obtained plurality of overall identity features and confidence levels, the overall identity feature with the highest confidence level may be selected as the final overall identity feature of the object; or selecting the integral identity features with the confidence coefficient higher than a preset threshold value, then fusing the selected integral identity features, and taking the fused integral identity features as final integral identity features of the object.

In one embodiment of the present application, the above-mentioned space reachable condition may be any one or a combination of the following conditions:

And (2) under the condition 1, the distance between the end position of the motion track and the initial position of the obtained motion track is smaller than or equal to a preset distance threshold.

Specifically, the starting position of the obtained motion trail may be determined, and if the distance between the ending position of the archived trail and the starting position is smaller, the object may be considered to be able to go from the ending position of the archived trail to the starting position of the obtained motion trail, so that the space accessibility condition is considered to be satisfied between the two motion trail.

And 2, the duration between the ending time of the motion trail and the starting time of the obtained motion trail is less than or equal to a preset duration threshold value.

Specifically, the starting time of the obtained motion trail may be determined, and if the duration between the ending time of the archived trail and the starting time is smaller, the object may be considered to be able to go from the end position of the archived trail to the starting position of the obtained motion trail in time, so that the space reachability condition is considered to be satisfied between the two motion trail.

And 3, calculating the movement speed of the object based on the termination position, the starting position, the termination time and the starting time to be less than or equal to a preset speed threshold.

Specifically, based on the end position, the start position, the end time, and the start time, the motion speed of the object in the process of starting from the start position and reaching the end position at the start time can be calculated, if the motion speed is too high, the two archived trajectories are considered to not satisfy the space availability condition, otherwise, the two archived trajectories are considered to satisfy the space availability condition.

In addition, the above space-reachable conditions may be: the monitoring areas of the image acquisition devices on which the method is based are directly accessible. For example, assuming that the archived motion trajectory y1 is obtained based on the image capturing device c1, where the motion trajectory obtained in step S101 is y2, and the image capturing device based on the motion trajectory is c2, if there is a direct connection channel between the monitoring areas j1 and j2 of c1, which indicates that the monitoring areas j1 and j2 are directly reachable, then the motion trajectories y1 and y2 are considered to satisfy the space reachable condition; if a wall or other shielding object exists between the monitoring areas j1 and j2 or a direct connecting channel does not exist between the monitoring areas j1 and j2, the motion tracks y1 and y2 are considered to not meet the space accessibility condition.

In one embodiment of the application, the topological relation among the image acquisition devices can be established according to the communication relation among the monitoring areas of the image acquisition devices in the scene. Therefore, when judging whether the two motion trajectories meet the space reachable condition, judging whether the monitoring area of the image acquisition equipment based on the two motion trajectories is directly reachable according to the topological relation, and if so, determining that the two motion trajectories meet the space reachable condition.

In one embodiment of the present application, when S105 determines whether the current object is a strange object, local identity feature extraction may be performed on an image acquired by the image acquisition device, so as to obtain confidence that the extracted local identity feature characterizes the identity of the current object, and determine whether the confidence of the local identity feature is greater than or equal to a preset confidence threshold, if so, determine that the current object is a strange object.

Specifically, if the confidence of the local identity feature of the current object is higher, it is indicated that the local identity feature can characterize the identity of the current object, and the obtained local identity feature is valid, in this case, the target object that matches the local identity feature of the current object still cannot be found from the archived object, and it may be further determined that no object information has been created for the current object, thereby determining that the current object is a strange object.

In addition, the whole identity characteristic extraction can be carried out on the image acquired by the image acquisition equipment, the confidence degree of the extracted whole identity characteristic representing the identity of the current object is obtained, and then the judgment is carried out by combining the confidence degrees of the whole identity characteristic and the local identity characteristic of the current object.

For example, when the confidence coefficient of the local identity feature of the current object is greater than or equal to a preset first confidence coefficient threshold value and the confidence coefficient of the overall identity feature is greater than or equal to a preset second confidence coefficient threshold value, determining that the current object is a strange object;

Or determining the current object as a strange object under the condition that the sum of the confidence coefficient of the local identity characteristic and the confidence coefficient of the whole identity characteristic of the current object is larger than or equal to a preset third confidence coefficient threshold value.

In one embodiment of the present application, if the current object is not a strange object, it may be determined whether a target track satisfying a merging condition with the obtained motion track exists in the archived tracks, and if so, the obtained motion track and the target track are merged.

Specifically, under the condition that the current object is neither an archived object nor a strange object, the target track of the same object as the motion track of the object can be searched from the archived track directly according to the space reachable condition and the integral identity, and then the motion track is merged into the searched target track.

In one embodiment of the application, when the current object is neither an archived object nor a strange object and the motion trail of the current object does not have a corresponding target trail, a trail mark can be set for the motion trail of the current object, so that the motion trail can be conveniently queried later.

In one embodiment of the present application, if a plurality of target objects exist in the archived object, object information of the plurality of target objects may be merged to obtain merged object information, and motion trajectories of the plurality of target objects and the obtained motion trajectories are merged to determine that object information of an object to which the merged motion trajectories belong is the merged object information.

Specifically, if there are a plurality of target objects, it is described that the plurality of target objects and the current object are actually the same object, and therefore, it is possible to combine the object information of the plurality of target objects into one object information, and combine the motion trajectories of the plurality of target objects and the obtained motion trajectories into one motion trajectory as the motion trajectories of the objects corresponding to the combined object information.

In one embodiment of the present application, when merging object information, object information of a target object with highest feature confidence degree among a plurality of target objects may be determined as merged object information.

Wherein, the feature confidence of each object characterizes: based on the confidence of the object identity represented by the object characteristics of the object obtained by image extraction, the object characteristics comprise: local identity features and/or global identity features.

Specifically, a target object with the highest confidence coefficient of the corresponding local identity feature can be selected from a plurality of target objects, and the object information of the selected target object is used as the combined object information;

the object information of the selected object can be used as the combined object information;

and selecting a target object with the highest sum of the confidence coefficient of the corresponding local identity characteristic and the confidence coefficient of the integral identity characteristic, and taking the object information of the selected target object as the combined object information.

Referring to fig. 2, fig. 2 is a flow chart of another track merging method according to an embodiment of the present application, where the method includes the following steps:

s201, a motion trajectory of a current object generated based on an image acquired by an image acquisition device is obtained.

S202, judging whether the current object is an archived object according to the local identity characteristics of the current object, if yes, executing step S203, otherwise executing step S204.

Specifically, the local identity characteristic of the current object can be obtained according to the image acquired by the image acquisition equipment; and judging whether a target object with the similarity of the local identity characteristics with the current object in the archived object is larger than or equal to a first preset threshold value, and if so, determining that the current object is the archived object.

S203, determining the object to which the obtained motion trail belongs as a target object, marking the obtained motion trail by utilizing the object identifier of the target object, and merging the motion trail marked with the same object identifier.

If a plurality of target objects exist in the archived object, merging object information of the plurality of target objects to obtain merged object information, merging motion trajectories of the plurality of target objects and the obtained motion trajectories, and determining the object information of the object to which the merged motion trajectories belong as merged object information.

S204, judging whether the current object is a strange object, if so, executing a step S205, otherwise, executing a step S206.

Specifically, the local identity feature extraction can be performed on the image acquired by the image acquisition equipment, the confidence coefficient of the extracted local identity feature representing the identity of the current object is obtained, whether the confidence coefficient of the local identity feature is larger than or equal to a preset confidence coefficient threshold value is judged, and if so, the current object is determined to be a strange object.

And S205, creating object information for the current object, and archiving the current object.

S206, judging whether a target track meeting the combination condition with the obtained motion track exists in the archived track, and if so, combining the obtained motion track with the target track.

Specifically, an archived track which meets a preset space reachable condition with the motion track and has a similarity between the overall identity characteristics of the belonged objects greater than or equal to a second preset threshold value can be searched, and then the track is combined with the searched archived track.

Referring to fig. 3, fig. 3 is a schematic structural diagram of a track merging device according to an embodiment of the present application, where the device includes:

A motion trajectory obtaining module 301, configured to obtain a motion trajectory of a current object generated based on an image acquired by an image acquisition device;

A first feature obtaining module 302, configured to obtain a local identity feature of the current object according to an image acquired by the image acquisition device;

The target object determining module 303 is configured to determine whether a target object with a similarity between the local identity features of the archived object and the current object being greater than or equal to a first preset threshold exists, and if so, trigger the track merging module 304, and if not, trigger the object archiving module 305, where the archived object is: an object for which object information has been created;

The track merging module 304 is configured to determine that the current object and the target object are the same object, and merge motion tracks of the target object and the current object;

The object archiving module 305 is configured to determine whether the current object is a strange object, and if so, create object information for the current object, so as to archive the current object.

In one embodiment of the application, the apparatus further comprises:

The track merging module 304 is specifically configured to:

In one embodiment of the present application, the object archiving module 305 is specifically configured to:

In one embodiment of the present application, the track merging module 304 includes:

The embodiment of the application also provides an electronic device, as shown in fig. 4, which comprises a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 complete communication with each other through the communication bus 404,

A memory 403 for storing a computer program;

the processor 401 is configured to implement the track merging method when executing the program stored in the memory 403.

The communication bus mentioned above for the electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the electronic device and other devices.

The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor including a central processing unit (Central Processing Unit, CPU), a layer processor (Network Processor, NP), and the like; but may also be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.

In yet another embodiment of the present application, a computer readable storage medium is provided, in which a computer program is stored, which when executed by a processor, implements the steps of any of the track merging methods described above.

In yet another embodiment of the present application, a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the track merging methods of the above embodiments is also provided.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the apparatus embodiments, the electronic device embodiments, the computer-readable storage medium embodiments, the computer program product embodiments, the description is relatively simple, as it is substantially similar to the method embodiments, and relevant places are referred to in the partial description of the method embodiments.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A track merging method, the method comprising:

obtaining a motion trail of a current object generated based on multi-frame images acquired by image acquisition equipment, wherein the motion trail represents: a change in the spatial position of the current object over a period of time;

Obtaining local identity characteristics of the current object according to the multi-frame images acquired by the image acquisition equipment;

Judging whether a target motion trail exists in motion trail of an archived object, wherein the archived object is: an object of the created object information, the target motion trail is: a motion track which has a partial space-time coincidence track with the motion track of the current object and has the similarity between the local identity characteristics of the current object and the object reaching a first preset threshold value, wherein the space-time coincidence represents the position and time coincidence of the track;

If a target motion trail exists, determining that the current object and the target object are the same object, and combining the motion trail of the target object and the motion trail of the current object, wherein the target object is an archived object to which the target motion trail belongs;

If the target motion track does not exist, extracting local identity features of the image acquired by the image acquisition equipment, and obtaining the confidence coefficient of the extracted local identity features for representing the identity of the current object;

judging whether the confidence coefficient of the local identity feature is larger than or equal to a preset confidence coefficient threshold value,

If the confidence coefficient is larger than or equal to a preset confidence coefficient threshold value, determining that the current object is a strange object, and creating object information for the current object to file the current object;

if the current object is less than the preset confidence threshold, judging whether a target track meeting a merging condition with the obtained motion track exists in the archived track when the current object is not a strange object, wherein the archived track is: the object is the motion trail of the archived object, and the merging condition is as follows: the method meets the preset space reachable condition, and the similarity between the integral identity characteristics of the belonged objects is more than or equal to a second preset threshold value;

And if the target track meeting the combination condition with the obtained motion track exists, combining the obtained motion track with the target track.

2. The method of claim 1, wherein after said determining that the current object and the target object are the same object, the method further comprises:

And merging the motion trail of the object mark marked with the target object.

3. The method according to claim 1, wherein the method further comprises:

4. A method according to claim 3, wherein the spatially reachable condition comprises:

5. The method according to any one of claims 1-4, wherein the merging the motion trajectories of the target object and the current object comprises:

6. The method of claim 5, wherein merging the object information of the plurality of target objects to obtain merged object information comprises:

7. A track merging device, the device comprising:

the motion trail obtaining module is used for obtaining a motion trail of a current object generated based on multi-frame images acquired by the image acquisition equipment, and the motion trail represents: a change in the spatial position of the current object over a period of time;

The first characteristic obtaining module is used for obtaining the local identity characteristic of the current object according to the multi-frame images acquired by the image acquisition equipment;

The system comprises a target object judging module, a track merging module and an object archiving module, wherein the target object judging module is used for judging whether a target motion track exists in motion tracks of archived objects, and if the target motion track exists, the track merging module is triggered, and if the target motion track does not exist, the object archiving module is triggered, wherein the archived objects are: an object of the created object information, the target motion trail is: a motion track which has a partial space-time coincidence track with the motion track of the current object and has the similarity between the local identity characteristics of the current object and the object reaching a first preset threshold value, wherein the space-time coincidence represents the position and time coincidence of the track;

The track merging module is used for determining that the current object and the target object are the same object, merging the motion tracks of the target object and the current object, and the target object is an archived object to which the target motion track belongs;

The object archiving module is used for extracting local identity characteristics of the image acquired by the image acquisition equipment when the target motion track does not exist, and obtaining the confidence that the extracted local identity characteristics represent the identity of the current object; judging whether the confidence coefficient of the local identity feature is larger than or equal to a preset confidence coefficient threshold value, if so, determining that the current object is a strange object, creating object information for the current object, and archiving the current object;

The device also comprises a second judging module for:

When the confidence coefficient threshold value is smaller than a preset confidence coefficient threshold value and the current object is not a strange object, judging whether a target track meeting a merging condition with the obtained motion track exists in the archived track, wherein the archived track is: the object is the motion trail of the archived object, and the merging condition is as follows: the method meets the preset space reachable condition, and the similarity between the integral identity characteristics of the belonged objects is more than or equal to a second preset threshold value; and if the target track meeting the combination condition with the obtained motion track exists, combining the obtained motion track with the target track.

8. The apparatus of claim 7, wherein the apparatus further comprises:

the track merging module is specifically configured to:

determining that the current object and the target object are the same object, and merging the motion trail marked with the object identifier of the target object;

the device further comprises a first judging module for:

for any two archived tracks, merging the two archived tracks under the condition that the two archived tracks meet a preset merging condition, merging object information of objects to which the two archived tracks belong, and determining the object information of the objects to which the merged motion track belongs as merged object information, wherein the archived tracks are as follows: the object is the motion trail of the archived object, and the merging condition is as follows: the method meets the preset space reachable condition, and the similarity between the integral identity characteristics of the belonged objects is more than or equal to a second preset threshold value;

The space reachable conditions include:

The movement speed of the object calculated based on the termination position, the starting position, the termination time and the starting time is smaller than or equal to a preset speed threshold;

the track merging module comprises:

The track merging unit is used for merging the motion tracks of the plurality of target objects and the obtained motion tracks, and determining the object information of the object to which the merged motion track belongs as the merged object information;

The information merging unit is specifically configured to:

9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

A memory for storing a computer program;

A processor for implementing the method of any of claims 1-6 when executing a program stored on a memory.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-6.