CN106251348A - A kind of self adaptation multi thread towards depth camera merges background subtraction method - Google Patents
A kind of self adaptation multi thread towards depth camera merges background subtraction method Download PDFInfo
- Publication number
- CN106251348A CN106251348A CN201610601964.0A CN201610601964A CN106251348A CN 106251348 A CN106251348 A CN 106251348A CN 201610601964 A CN201610601964 A CN 201610601964A CN 106251348 A CN106251348 A CN 106251348A
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- foreground
- depth
- background
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a kind of self adaptation multi thread towards depth camera and merge background subtraction method, step includes: A, some color image frames of input depth camera collection and depth image frame, it is utilized respectively basic background modeling method to initialize, sets up color background model and degree of depth background model;B, input a new frame, respectively obtain color image and the respective foreground image of depth image by background subtraction;C, utilize Weighted Fusion method that the foreground image of color image and depth image is fused to new foreground image, and by threshold value selection technique by its binaryzation;D, use multi thread space-time consistency strategy, carry out self adaptation improvement with the testing result of current color image, depth image and previous frame to foreground image, obtain final result;E, renewal background model.F, repeated execution of steps B are to step E, until having processed all picture frames.The method can effectively detect prospect, obtains higher recall rate and accuracy rate.
Description
Technical field
The present invention relates to technical field of computer vision, more specifically, relate to a kind of self adaptation towards depth camera
Multi thread merges background subtraction method.
Background technology
Background subtraction is a background task of computer vision field, and its main purpose is from ambient field by foreground target
Scape detects, pays close attention to widely having obtained in recent years.It serves substantial amounts of video processing applications, as intelligent video is supervised
Control, target tracking, action recognition also have man-machine interaction etc..The output result of background subtraction is typically higher level computer vision
The input information of task, so its execution directly influences these tasks ensuing execution result.Although having in recent years
Greater advance, but the scene comprising complicated factor for some, such as, comprise violent illumination variation, shade and dynamic background
(such as escalator, rock curtain, the display of flicker) etc., background subtraction remains a relatively difficult task.
The development of technology recently, enabling obtain the depth information of some scenes.These depth informations can utilize and fly
The equipment such as the Kinect of row time (ToF) camera, the Xtion PRO of Asus or Microsoft obtain.Use these equipment, permissible
Obtain comprising the RGB-D video set of color image and deep image information.In the present invention, being obtained in that color image and the degree of depth
Camera and the equipment of image are referred to as depth camera.The effectively combination of these colors and depth information will obtain more preferable background and subtract
The result removed.Therefore, the present invention proposes a kind of self adaptation multi thread towards depth camera and merges background subtraction method.
Summary of the invention
The present invention discloses a kind of self adaptation multi thread towards depth camera and merges background subtraction method.The method proposes face
Color image combines with the information of depth image strategy, can effectively suppress the appearance of noise spot;Multi thread improvement strategy is proposed,
Make to detect the foreground image obtained more accurately complete.
Technical problem solved by the invention can use following technical solution to realize:
The present invention needs using a kind of background modeling method of color image and depth image that can individually be used for as base
Plinth, is referred to as basic background modeling method by this method below.
The invention provides a kind of self adaptation multi thread towards depth camera and merge background subtraction method, including following step
Rapid:
A, some color image frames of input depth camera collection and depth image frame, and it is utilized respectively basic background modeling
Method initializes, and sets up color background model and degree of depth background model;
B, input a new frame, respectively obtain color image and the respective foreground image of depth image by background subtraction;
C, the method utilizing Weighted Fusion, be fused to new foreground picture the foreground image of color image and depth image
Picture, and the foreground image binaryzation that will be merged by threshold value selection technique;
D, use multi thread space-time consistency strategy, tie with the detection of current color image, depth image and previous frame
Fruit carries out self adaptation improvement to foreground image, obtains final result;
Concrete, improvement to as if previous step perform the binaryzation fusion results that obtains, the clue of reference includes currently
The prospect that the color image of frame and depth image, use edge detector detect the edge image that obtains, previous frame finally exports
Image.WithRepresent the binaryzation foreground image that step C is exported, when wherein pixel x is prospectOtherwiseEtIt is defined as current frame color imageUse the edge image that edge detector obtains.EtIt it is a width two-value
Change image, the E when pixel x is edge pixelt(x)=255, otherwise Et(x)=0.?It is defined as finally exporting in t
Foreground image.
The step of self adaptation modification method is as follows:
1) to imageIn foreground pixel point x (it is to say,) use iteration mode.For the first time
Iteration, 8 connected region neighbor set of pixel xProcessed.Specifically, if pixel?In be one
Background pixel point, when meeting following condition, it can be converted into foreground point:
Wherein function D () is for calculating the Euclidean distance of two characteristic vectors, κcAnd κdFor presetting constant.
2) for the second time in iteration, forThe pixel concentrated takes same processing method.Collection is defined as
Wherein ∪ operator represents union of sets computing.Similarly, will process in the T time iterationThe picture concentrated
Element, until iteration ends performs.
3) when pixel x is judged as background pixel in former frame, number of times T (x) of its iteration not can exceed that pre-defined
Constant value γ:
This method uses this time consistency strategy, by a kind of conservative in the way of suppress the incorrect expansion of noise spot.
In this self adaptation improvement steps, traversalIn each foreground pixel and utilize described above multi-thread
Rope space-time consistency strategy processes, and then produces final foreground detection result images
E, renewal background model.
Concrete, updating background model is to come the back of the body according to the more New Policy that selected basic background modeling method is corresponding
Scape model is updated.
F, repeated execution of steps B are to step E, until having processed all picture frames.
It is contemplated that propose a kind of self adaptation multi thread towards depth camera to merge background subtraction method.On initial rank
Section, carries out background modeling by the color image and depth image selecting a kind of basic background modeling method respectively video, from
And obtain color background model and degree of depth background model.In the foreground detection stage, obtain final by two continuous print steps
Result.First, the information exported by Weighted Fusion foreground detection obtains a coarse result, and the both sides of fusion are by respectively will
Coloured image and depth image obtain for associated background model detection.Afterwards, multi thread space-time consistency plan is utilized
Slightly, the present invention uses a kind of method of degree of accuracy that is effectively improved to obtain final testing result.After foreground detection completes,
Background model will update the scene changes adapting to produce in time.Its feature and advantage be:
1. easily produce the defect of more noise for existing background subtraction method, propose a kind of towards depth camera
Self adaptation multi thread merges background subtraction method.
2. the information of color image and depth image is effectively combined by this method, judges by accident during excluding modeling and detection
For the pixel of foreground point, thus it is effectively improved the accuracy rate of testing result.
3. this method is to utilize multi thread to improve foreground image, make use of space-time consistency during improvement,
The foreground target obtained is more complete, and accuracy is higher.
4. the background subtraction method that the present invention provides can utilize multiple different basic background modeling method, versatility
By force.
Accompanying drawing explanation
Fig. 1 is the flow process that a kind of self adaptation multi thread towards depth camera disclosed by the invention merges background subtraction method
Figure;
Fig. 2 is the flow chart of steps using the improvement of multi thread space-time consistency strategy;
Fig. 3 is present invention framework exemplary plot of background modeling method based on ViBe.
Detailed description of the invention
In order to make the purpose of the present invention, technical scheme and advantage clearer, below in conjunction with drawings and Examples, right
The present invention is further elaborated.Should be appreciated that specific embodiment described herein is only used for explaining the present invention, not
For limiting the present invention.
The basic thought of the present invention is as follows: first, selects a kind of basic background modeling method.At initial phase, with this
Color image and the depth image of basic background modeling method respectively video carry out background modeling.In the foreground detection stage, logical
The information crossing the output of Weighted Fusion foreground detection obtains a coarse result, and the both sides of fusion are by respectively by coloured image and deep
Degree image obtains for associated background model detection.Afterwards, utilize multi thread space-time consistency strategy, use one to have
Effect improves the method for degree of accuracy and obtains final testing result.After foreground detection completes, background model will update with
Adapt to the scene changes produced in time.
In following example illustrate, background modeling method based on ViBe method will be used.ViBe method specifically may be used
With reference to following paper document ViBe:A Universal Background Subtraction Algorithm for Video
Sequences.IEEE Transactions Image Processing 20(6):1709-1724(2011).Fig. 3 illustrates
The present invention is the framework exemplary plot of background modeling method based on ViBe method.
Flow chart shown in Figure 1, the present invention discloses a kind of self adaptation multi thread towards depth camera and merges background
Subduction method, specifically comprises the following steps that
A, some color image frames of input depth camera collection and depth image frame, and it is utilized respectively basic background modeling
Method initializes, and sets up color background model and degree of depth background model;
Specifically, this step carries out background modeling with ViBe method.
The initialization of background model completes in the first two field picture.For single pixel x, ViBe method is by its neighborhoodStochastical sampling produces the mode of a sets of pixel values and sets up model.?WithIt is respectively defined as color and the degree of depth
First frame image information of image.By ViBe method, fromMiddle set up one pixel-by-pixel color based on color image the back of the body
Scape model
Also have fromMiddle set up a degree of depth background model based on depth image pixel-by-pixel
WhereinRepresent from pixelThe three-dimensional RGB color characteristic vector extracted,Represent from pixelThe value of the one-dimensional depth image of middle extraction.
B, input a new frame, respectively obtain color image and the respective foreground image of depth image by background subtraction;
In t, based on ViBe method, utilize color background modelTo the color image being currently enteredCarry on the back
Scape reduces, it is possible to obtain a foreground image It is a bianry image, when pixel x is judged as foreground pixelOtherwiseSimilarly, degree of depth background model is utilizedCan be from the depth image being currently enteredMiddle acquisition two-value foreground image
C, the method utilizing Weighted Fusion, be fused to new foreground picture the foreground image of color image and depth image
Picture, and the foreground image binaryzation that will be merged by threshold value selection technique;
Specifically, the method using Weighted Fusion, by foreground imageWithMerge and obtain foreground imageUse formula
Can be expressed as
Wherein [] be used for defining Iverson bracket operator (if statement O is correct, then [O]=1, otherwise [O]
=0).α is the constant set.Then the technology utilizing threshold value to select willBinaryzation, simultaneously the noise spot that may be background
Get rid of:
D, use multi thread space-time consistency strategy, tie with the detection of current color image, depth image and previous frame
Fruit carries out self adaptation improvement to foreground image, obtains final result;
In the step of this self adaptation improvement, the present invention utilizes multi thread space-time consistency strategy to produce more accurate
True foreground detection result.This improvement strategy is to observe based on following, when present scene body occurs in scene, and adjacent prospect
Pixel would generally show a kind of apparent strong time-space relationship.
This method is the feature clue that each pixel has reconnoitred color, the degree of depth and edge.Specifically, EtIt is defined as
To coloured imageCanny edge detector is used to detect the edge image obtained.EtIt is a width binary image, when pixel x
E when being edge pixelt(x)=255, otherwise Et(x)=0.?It is defined as the foreground image finally exported in t.
As in figure 2 it is shown, the step of this self adaptation modification method is as follows.
To imageIn foreground pixel point x (it is to say,) use iteration mode.The most repeatedly
Generation, 8 connected region neighbor set of pixel xProcessed.Specifically, if pixel?In be a back of the body
Scene vegetarian refreshments, when meeting following condition, it can be converted into foreground point (even):
Wherein function D () is for calculating the Euclidean distance of two characteristic vectors, κcAnd κdFor presetting constant.
For the second time in iteration, forThe pixel concentrated takes same processing method.Collection is defined as
Wherein ∪ operator represents union of sets computing.Similarly, will process in the T time iterationThe picture concentrated
Element, until iteration ends performs.
According to the Space Consistency of formula (5), at Fw tThe iteration improvement steps of a foreground pixel can not have
Background context pixel stops when being converted into prospect.But, in order to avoid the foreground point of mistake (is mistaken for the background picture of prospect
Vegetarian refreshments) extension, self adaptation improved, process with the addition of time consistency.Specifically, it is judged as the back of the body when pixel x in former frame
During scene element, number of times T (x) of its iteration not can exceed that predefined constant value γ:
This method uses this time consistency strategy, by a kind of conservative in the way of suppress the incorrect expansion of noise spot.
In this self adaptation improvement steps, traversalIn each foreground pixel and utilize multi thread described above
Space-time consistency strategy processes, and then produces final foreground detection result images
E, renewal background model.
After completing foreground detection, it will update the scene changes that background model can adapt to produce in time.
Based on ViBe method, one being judged as to pixel x of background in t, its model is updated to:
Wherein operatorIt is defined as gathering a unit randomly drawing of M and have the probability quilt of 1/ ΦReplaced.WithIt it is all the pixel randomly selected.
F, repeated execution of steps B are to step E, until having processed all picture frames.
The inventive method embodiment is carried out Setup Experiments explanation herein.N is set as that 20, α are set as 0.5, and δ is set as
127, γ are set as 4, and Φ is set as 16.
The self adaptation multi thread towards depth camera provided the embodiment of the present invention above merges background subtraction method
Being described in detail, principle and the embodiment of the present invention are set forth by specific case used herein, above reality
The explanation executing example is only intended to help to understand method and the core concept thereof of the present invention;General technology simultaneously for this area
Personnel, according to the thought of the present invention, the most all will change, in sum, and this theory
Bright book content should not be construed as limitation of the present invention.
Claims (4)
1. the self adaptation multi thread towards depth camera merges background subtraction method, it is characterised in that described method includes
Following steps:
A, some color image frames of input depth camera collection and depth image frame, and it is utilized respectively basic background modeling method
Initialize, set up color background model and degree of depth background model;
B, input a new frame, respectively obtain color image and the respective foreground image of depth image by background subtraction;
C, the method utilizing Weighted Fusion, be fused to new foreground image the foreground image of color image and depth image, and
The foreground image binaryzation that will be merged by threshold value selection technique;
D, use multi thread space-time consistency strategy, come with the testing result of current color image, depth image and previous frame
Foreground image is carried out self adaptation improvement, obtains final result;
E, renewal background model;
F, repeated execution of steps B are to step E, until having processed all picture frames.
Method the most according to claim 1, it is characterised in that the basic background modeling method described in step A is that one can
Individually for color image and the background modeling method of depth image.
Method the most according to claim 1, it is characterised in that step D use space-time consistency strategy carries out self adaptation and changes
Good step is as follows:
WithWithRepresent current frame color image and the depth image of t respectively, withRepresent the two-value that step C is exported
Change foreground image, when wherein pixel x is prospectOtherwiseEtIt is defined as coloured imageMake
The edge image obtained with edge detector;EtIt is a width binary image, the E when pixel x is edge pixelt(x)=255, no
Then Et(x)=0;?It is defined as the foreground image finally exported in t;
1) to imageInterior foreground pixel point x is (i.e.) use iteration mode;In first time iteration, pixel x
8 connected region neighbor setProcessed;Specifically, if pixel?In be a background pixel point,
When meeting following condition, it can be converted into foreground point:
Wherein function D () is for calculating the Euclidean distance of two characteristic vectors, κcAnd κdFor presetting constant;
2) for the second time in iteration, forThe pixel concentrated takes same processing method;Collection is defined as
Wherein ∪ operator represents union of sets computing;Similarly, will process in the T time iterationThe pixel concentrated,
Until iteration ends performs;
3) when pixel x is judged as background pixel in former frame, number of times T (x) of its iteration not can exceed that predefined often
Value γ:
TraversalIn each foreground pixel and utilize above-described multi thread space-time consistency strategy to process, produce final
Foreground detection result images
Method the most according to claim 1, it is characterised in that the renewal background model described in step E is to use the basis back of the body
Background model is updated by the more New Policy of scape modeling method.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610601964.0A CN106251348B (en) | 2016-07-27 | 2016-07-27 | Self-adaptive multi-cue fusion background subtraction method for depth camera |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610601964.0A CN106251348B (en) | 2016-07-27 | 2016-07-27 | Self-adaptive multi-cue fusion background subtraction method for depth camera |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106251348A true CN106251348A (en) | 2016-12-21 |
CN106251348B CN106251348B (en) | 2021-02-02 |
Family
ID=57604179
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610601964.0A Expired - Fee Related CN106251348B (en) | 2016-07-27 | 2016-07-27 | Self-adaptive multi-cue fusion background subtraction method for depth camera |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106251348B (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107067843A (en) * | 2017-02-10 | 2017-08-18 | 广州动创信息科技有限公司 | Body-sensing touch-control electronic blank tutoring system |
CN109060309A (en) * | 2018-06-28 | 2018-12-21 | 广东工业大学 | A kind of optimal resolution color matching instrument of color difference and its test method |
CN109658441A (en) * | 2018-12-14 | 2019-04-19 | 四川长虹电器股份有限公司 | Foreground detection method and device based on depth information |
CN110111341A (en) * | 2019-04-30 | 2019-08-09 | 北京百度网讯科技有限公司 | Display foreground acquisition methods, device and equipment |
CN111915687A (en) * | 2020-07-13 | 2020-11-10 | 浙江工业大学 | Background extraction method with depth information and color information |
CN113465252A (en) * | 2020-05-29 | 2021-10-01 | 海信集团有限公司 | Intelligent refrigerator and drawer state detection method in intelligent refrigerator |
US12100162B2 (en) | 2018-12-03 | 2024-09-24 | Zhejiang Dahua Technology Co., Ltd. | Method and system for obtaining and processing foreground image and background image |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070122039A1 (en) * | 2005-11-29 | 2007-05-31 | Microsoft Corporation | Segmentation of objects by minimizing global-local variational energy |
US20130084006A1 (en) * | 2011-09-29 | 2013-04-04 | Mediatek Singapore Pte. Ltd. | Method and Apparatus for Foreground Object Detection |
CN104361577A (en) * | 2014-10-20 | 2015-02-18 | 湖南戍融智能科技有限公司 | Foreground detection method based on fusion of depth image and visible image |
CN104408747A (en) * | 2014-12-01 | 2015-03-11 | 杭州电子科技大学 | Human motion detection method suitable for depth image |
-
2016
- 2016-07-27 CN CN201610601964.0A patent/CN106251348B/en not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070122039A1 (en) * | 2005-11-29 | 2007-05-31 | Microsoft Corporation | Segmentation of objects by minimizing global-local variational energy |
US20130084006A1 (en) * | 2011-09-29 | 2013-04-04 | Mediatek Singapore Pte. Ltd. | Method and Apparatus for Foreground Object Detection |
CN104361577A (en) * | 2014-10-20 | 2015-02-18 | 湖南戍融智能科技有限公司 | Foreground detection method based on fusion of depth image and visible image |
CN104408747A (en) * | 2014-12-01 | 2015-03-11 | 杭州电子科技大学 | Human motion detection method suitable for depth image |
Non-Patent Citations (2)
Title |
---|
YOUNG-MIN SONG ET AL: "Background subtraction based on Gaussian mixture models using color and depth information", 《THE 2014 INTERNATIONAL CONFERENCE ON CONTROL, AUTOMATION AND INFORMATION SCIENCES》 * |
孟祥怡: "运动目标的跟踪与识别算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107067843A (en) * | 2017-02-10 | 2017-08-18 | 广州动创信息科技有限公司 | Body-sensing touch-control electronic blank tutoring system |
CN109060309A (en) * | 2018-06-28 | 2018-12-21 | 广东工业大学 | A kind of optimal resolution color matching instrument of color difference and its test method |
US12100162B2 (en) | 2018-12-03 | 2024-09-24 | Zhejiang Dahua Technology Co., Ltd. | Method and system for obtaining and processing foreground image and background image |
CN109658441A (en) * | 2018-12-14 | 2019-04-19 | 四川长虹电器股份有限公司 | Foreground detection method and device based on depth information |
CN110111341A (en) * | 2019-04-30 | 2019-08-09 | 北京百度网讯科技有限公司 | Display foreground acquisition methods, device and equipment |
CN113465252A (en) * | 2020-05-29 | 2021-10-01 | 海信集团有限公司 | Intelligent refrigerator and drawer state detection method in intelligent refrigerator |
CN113465252B (en) * | 2020-05-29 | 2022-06-21 | 海信集团有限公司 | Intelligent refrigerator and drawer state detection method in intelligent refrigerator |
CN111915687A (en) * | 2020-07-13 | 2020-11-10 | 浙江工业大学 | Background extraction method with depth information and color information |
CN111915687B (en) * | 2020-07-13 | 2024-07-05 | 浙江工业大学 | Background extraction method with depth information and color information |
Also Published As
Publication number | Publication date |
---|---|
CN106251348B (en) | 2021-02-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106251348A (en) | A kind of self adaptation multi thread towards depth camera merges background subtraction method | |
US8531484B2 (en) | Method and device for generating morphing animation | |
US10706613B2 (en) | Systems and methods for dynamic occlusion handling | |
KR102153607B1 (en) | Apparatus and method for detecting foreground in image | |
CN102194443B (en) | Display method and system for window of video picture in picture and video processing equipment | |
US20100098331A1 (en) | System and method for segmenting foreground and background in a video | |
JP2019160298A (en) | Image processing apparatus and method for object boundary stabilization in image of sequence of images | |
CN103440664B (en) | Method, system and computing device for generating high-resolution depth map | |
CN108377374B (en) | Method and system for generating depth information related to an image | |
JP2011509451A (en) | Segmentation of image data | |
CN104392468A (en) | Improved visual background extraction based movement target detection method | |
CN102567727A (en) | Method and device for replacing background target | |
CN105069801A (en) | Method for preprocessing video image based on image quality diagnosis | |
CN102202224A (en) | Caption flutter-free method and apparatus used for plane video stereo transition | |
CN106997478B (en) | RGB-D image salient target detection method based on salient center prior | |
CN102034247A (en) | Motion capture method for binocular vision image based on background modeling | |
CN109063659A (en) | The detection and tracking and system of moving target | |
EP3376470B1 (en) | Moving body tracking method, moving body tracking device, and program | |
CN102420985A (en) | Multi-view video object extraction method | |
CN103942756B (en) | A kind of method of depth map post processing and filtering | |
US9436996B2 (en) | Recording medium storing image processing program and image processing apparatus | |
KR101125061B1 (en) | A Method For Transforming 2D Video To 3D Video By Using LDI Method | |
CN102625028A (en) | Method and apparatus for detecting static logo existing in video | |
CN111179281A (en) | Human body image extraction method and human body action video extraction method | |
CN102724530B (en) | Three-dimensional method for plane videos based on feedback control |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210202 Termination date: 20210727 |