KR101537559B1 - Device for detecting object, device for detecting object for vehicle and method thereof - Google Patents

Abstract

An object detection method is disclosed. The method includes calculating an average movement amount between a previous frame and a current frame for all the pixels using the optical flow method and compensating a background movement amount of the background frame of the previous frame by a calculated average movement amount, Calculating a movement amount of each pixel between the current frame and the configured background frame by using the optical flow method; calculating a movement amount of each pixel between the current frame and the configured background frame by using the optical flow method; Calculating coordinates of each pixel, and detecting the detected coordinate value as an object region in the current frame.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object detecting apparatus,

The present invention relates to an object detecting apparatus, a vehicle object detecting apparatus and a method thereof, and more particularly, to an object detecting apparatus and method for detecting an object from a background in an image acquired in an environment in which an image acquiring means such as a camera moves .

In general, optical flow technique and background subtraction technique are typical methods for detecting an object such as a moving object or a person using only image information.

The optical flow technique is a method of detecting the object by grouping vectors having similar characteristics by calculating the direction and distance vector of each pixel between two image frames.

In the background separation method, the image frame is sequentially received, the pixel value information of each image frame is accumulated on a frame basis, a background frame is created, and a moving object part is defined as a part having a large difference between the current frame and the background frame. It is a method to detect an object based on

However, all of the above-described conventional techniques can find an object moving only on a camera installed in a place, for example, a CCTV, without errors, and for a general image in which a background moves as a camera moves, The background frame can not be accumulated due to the difference of the pixel values, and the object can not be accurately searched.

Accordingly, it is an object of the present invention to provide an object detection apparatus, an object detection apparatus for a vehicle, and a method thereof, which detect an object without error in an image acquired in an environment in which an image acquisition means such as a camera moves.

According to another aspect of the present invention, there is provided an object detecting method comprising: calculating an average moving amount of each pixel between a previous frame and a current frame using an optical flow method; The method comprising the steps of: constructing the background frame compensated for background movement amount as a background frame of the current frame by compensating a background movement amount of the background frame; calculating a movement amount of each pixel between the current frame and the configured background frame Calculating a coordinate value of each pixel corresponding to a movement amount exceeding a preset threshold value, and detecting the detected coordinate value as an object region in the current frame.

According to another aspect of the present invention, there is provided an apparatus for detecting an object, comprising: an image acquisition unit that captures a surrounding situation in a mobile environment and sequentially outputs a previous frame and a current frame; A background extracting unit for calculating an average movement amount, compensating a background movement amount of the background frame of the previous frame by the calculated average movement amount, and extracting the background frame compensated for the background movement amount as a background frame of the current frame; Calculates a movement amount of each pixel between the current frame and the compensated background frame, calculates a coordinate value of each pixel corresponding to a movement amount exceeding a preset threshold value, detects the detected coordinate value as an object area in the current frame And an object detection unit.

In the vehicle object detecting apparatus according to another aspect of the present invention, there is provided an apparatus for detecting an object for a vehicle, the apparatus comprising: an image acquiring unit for photographing a circumstance of the vehicle during driving of the vehicle and successively outputting a previous frame and a current frame; And a background extracting unit for calculating a background movement amount of the background frame of the previous frame by the calculated average movement amount and extracting the background frame compensated for the background movement amount as a background frame of the current frame, Calculating an amount of movement of each pixel between the current frame and the compensated background frame using the optical flow method, determining a threshold value according to a running speed value of the vehicle provided from an electronic control unit in the vehicle, Corresponding to the movement amount exceeding the threshold value And an object detection unit for calculating coordinate values of the respective pixels and detecting the detected coordinate values as the object region in the current frame.

A method of detecting an object using a vehicle object detecting apparatus according to another aspect of the present invention, the method comprising: capturing a surrounding situation of the vehicle during driving of the vehicle, sequentially outputting a previous frame and a current frame; Calculates the average movement amount of each pixel between the previous frame and the current frame, compensates the background movement amount of the background frame of the previous frame by the calculated average movement amount, extracts the background frame with the background movement amount compensated as the background frame of the current frame Calculating a movement amount of each pixel between the current frame and the compensated background frame using the optical flow method, determining a threshold value according to a running speed value of the vehicle provided from an electronic control unit in the vehicle, The movement amount exceeding the threshold value By calculating the coordinate values of the respective pixels, the coordinate values of the detected current includes the step of detecting the object region in the frame.

According to the present invention, the amount of movement of the background is calculated from the image acquired by the moving camera, the image is corrected by the calculated amount of movement, and the background is extracted by compensating the background error due to the movement of the camera. The object detection error due to the extraction of the object in a state in which the motion of the object is reflected as it is can be reduced. In addition, by combining optical flow and background separation methods, the accuracy of moving object detection can be efficiently increased.

1 is a block diagram schematically illustrating an internal configuration of an object detecting apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the configuration of the background extracting unit shown in FIG. 1. FIG.
3 is a block diagram showing the configuration of the object detection unit shown in FIG.
4 is a block diagram showing another embodiment of the object detection unit shown in FIG.
FIG. 5 is a block diagram showing another embodiment of the object detection unit shown in FIG. 1. FIG.
6 is a flowchart illustrating an object detection method according to an embodiment of the present invention.

The present invention calculates a background movement amount in an image captured by a moving camera,

The present invention provides a motion compensated background and a method of detecting an object moving using an input image, unlike the prior art in which motion of a background is directly reflected by extracting a background compensating a motion of a camera by moving an image by a movement amount. It also provides a way to increase the accuracy of moving object detection by combining optical flow and background separation methods. Accordingly, it is possible to detect a moving object while compensating for the movement of the background even when the camera's viewpoint is moving. Therefore, it is possible to detect a moving object in a video captured in an environment where a camera moves, such as a CCTV having a pan- And may be utilized in various technical fields for subsequent processing such as warning.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the invention to those skilled in the art. And the present invention is defined by the description of the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. It is noted that " comprises, " or "comprising," as used herein, means the presence or absence of one or more other components, steps, operations, and / Do not exclude the addition.

1 is a block diagram schematically showing the configuration of an object detecting apparatus according to an embodiment of the present invention.

The object detecting apparatus 100 according to an embodiment of the present invention compensates for the amount of background movement occurring between frames in an environment in which a camera moves using an optical flow method, And detects an object in the image based on the movement amount between the background frames configured according to the movement amount.

1, the object detection apparatus 100 includes an image acquisition unit 110, a frame memory 120, a background extraction unit 130, and an object detection unit 140).

The image acquiring unit 110 photographs the surrounding situation and sequentially displays the images of the previous image frame I (n-1) (hereafter referred to as the previous frame) and the current image frame I (n: And may be a camera having a pan / tilt function or a front / rear camera mounted on a vehicle. The present invention is not particularly limited to this, and includes all kinds of photographing means capable of photographing a surrounding situation in a moving environment.

The frame memory 120 is a kind of buffer for receiving a current frame I (n) from the image acquisition unit 110 and storing the current frame I (n) temporarily. The previous frame I (n) And outputs the frame I (n-1).

The background extraction unit 130 receives the current frame I (n) from the image acquisition unit 110 and the previous frame I (n-1) from the frame memory 120 and outputs the previous frame I (n-1) included in the previous frame I (n-1) using the optical flow between the current frame I (n-1) and the current frame I (n) (Separates or configures) the compensated previous background frame B (n-1) 'as the background frame B (n) of the current frame. A specific description thereof will be described in detail with reference to FIG. 2 below.

The object detection unit 140 detects the current frame I (n) using the optical flow between the compensated background frame B (n-1) 'or B (n) from the background extraction unit 130 and the current frame I And detects an object area in the frame I (n). A detailed description thereof will be made in detail with reference to FIGS. 3 to 5 below.

First, an embodiment of the background extraction unit 130 shown in FIG. 1 will be described in detail.

FIG. 2 is a block diagram showing the configuration of the background extracting unit shown in FIG. 1. FIG.

2, the background extraction unit 130 includes a first optical flow (OF) calculation unit 132, a background OF reference value generation unit 134, a background compensation unit 136, and a frame memory 138 .

The first OF calculator 132 calculates the amount of movement (optical flow value or motion vector value) between the previous frame I (n-1) and the current frame I (n) And outputs the calculated result as the first OF frame OF1 (n) for the current frame I (n). At this time, the output first OF frame (OF1 (n)) is outputted in a form including both the background movement amount and the object movement amount. For example, the calculation of the travel can be calculated according to the "High accuracy optical flow estimation based on a theory for warping" method proposed by T. Brox.

The background OF reference value generator 134 separates (extracts or detects) the background movement amount from the background movement amount and the object movement amount included in the first OF frame OF1 (n) and outputs it to the previous frame I (n-1) And generates a reference value REF for compensating the background frame B (n-1). To this end, the background OF reference value generator 134 arranges the movement amounts of all the pixels in the first OF frame (OF1 (n)) in order of magnitude, and divides the movement amounts arranged in the order of magnitude into equal intervals . Then, an interval in which the largest amount of movement is distributed among the divided sections is selected, and an average value (or an average movement amount) of the movement amounts distributed in the selected section is generated as the background OF reference value RFE. For example, the interval between the maximum movement amount (maximum optical flow value or maximum movement vector value) and the minimum movement amount (minimum optical flow value or minimum movement vector value) in the first OF frame OF1 (n) (N) of the previous frame (I (n-1)) by dividing the moving distance of the previous frame I (n-1) into a plurality of divided sections and then selecting a section including the largest number of movement amounts. n-1)) is used as the reference value RFE. This is the median of the maximum frequency interval on the histogram.

The background compensating unit 136 uses the background OF reference value RFE from the background OF reference value generator 134 to calculate the background frame of the previous frame I (n-1) from the frame memory 138 (N-1)) of the previous frame I (n-1) by an average movement amount that is the background OF reference value RFE, Thereby compensating the movement amounts of the pixels. If there is no previously compensated background frame in the frame memory 138, i. E. The initial frame itself is regarded as a previously configured (created or detached) background frame. The background compensator 136 composing the compensated background frame B (n-1) 'further receives the current frame I (n) from the image acquiring unit 110, (Mapping or compensating) the pixels of B (n-1) ') with the pixel values of the corresponding pixels of the input current frame I (n). By doing so, the final background frame, in which the background movement amount is compensated, is separately extracted. At this time, instead of directly replacing (mapping or compensating) the pixels of the compensated background frame B (n-1) 'with the pixel values of the corresponding pixels of the current frame I (n) (n-1) ') by a learning value (for example, 0.1) multiplied by a predetermined learning coefficient (e.g., 0.1), and supplies the pixels of the compensated background frame B (Mapped or compensated) by the pixel value compensated by the compensated pixel value. That is, the pixel value of B (n) is obtained by adding the pixel value at the corresponding position of I (n) to the pixel value at the specific position with respect to the moved B (n-1) It is equal to the value obtained by multiplying the difference between the pixel value in the frame and the learning coefficient. This can be expressed by the following equation (1).

Here, k is a learning coefficient.

Hereinafter, the object extracting unit 140 for extracting the object region using the background frame separated and extracted by the background extracting unit 130 will be described in detail.

3 is a block diagram showing the configuration of the object detection unit shown in FIG.

Referring to FIG. 3, the object detecting unit 140 includes a second OF calculating unit 142 and an object region separating unit 144.

The second OF calculator 142 calculates the background frame B (n) compensated for the background movement amount from the background extractor 130 and the current frame I (n) from the image obtaining unit 110 Calculates an amount of movement between the background frame B (n) and the current frame I (n) with respect to all the pixels using the optical flow to calculate the amount of movement between the background frame B (n) and the current frame I (n) And outputs an OP frame OF2 (n). Unlike the first OP frame OF2 (n) output from the first OF calculator 132 described above, the second OP frame OF2 (n) is a background frame B (n) Can be regarded as a background frame of the current frame I (n), and thus the amount of movement of the pixels constituting the object of the current frame I (n) will be output in a prominent manner.

The object region separator 144 separates and extracts the object region in the second OP frame OF2 (n) using the threshold value TH set by the designer. For example, the coordinate value of each pixel corresponding to the threshold value or more may be calculated, and the detected coordinate value may be detected as an object region in the current frame. At this time, the accuracy of object detection can be increased by using a background separation mask. The description will be made with reference to FIG. 4 below.

4 is a block diagram showing another embodiment of the object detection unit shown in FIG.

The object detecting unit 140 according to another embodiment shown in FIG. 4 has the same operation as the embodiment of FIG. 3 except that the mask generating unit 146 and the filter unit 148 are added to the embodiment of FIG. And functions.

The mask generation unit 146 receives the background frame B (n) in which the current frame I (n) and the background movement amount are compensated, and generates a background separation mask in order to increase the accuracy of object detection, Generates a mask frame 47 corresponding to the difference between these frames, and outputs the mask frame 47 as the background separation mask.

The filter unit 148 filters the noise included in the object region output from the object region separation unit 144 using the mask frame 47 from the mast generation unit 146. By doing so, the accuracy of object detection can be increased.

On the other hand, by setting the threshold value TH applied to the object region separation unit 144 in consideration of the moving speed of the camera, the accuracy of object detection can be increased. This will be described with reference to FIG.

FIG. 5 is a block diagram showing another embodiment of the object detection unit shown in FIG. 1. FIG.

5 illustrates an example in which the threshold value TH is adjusted according to the running speed of the vehicle, that is, the moving speed of the camera mounted on the vehicle, when the object detecting apparatus 100 of the present invention is mounted on a vehicle. When the object detecting apparatus 100 of the present invention is applied to a vehicle, the object detecting unit 140 according to another embodiment of the present invention detects the presence or absence of electrons in the vehicle through the network communication 40 in the vehicle such as CAN communication or LIN communication. The driving speed of the vehicle can be provided from the control unit 200. [

The object detection unit 140 receiving the vehicle traveling speed from the electronic control unit 200 may further include a threshold value generation unit 149 in addition to the configurations shown in FIG. The threshold value generator 149 compares the reference speed (for example, 10 Km / h) set by the designer with the traveling speed of the vehicle received from the electronic control unit 200, and determines whether the traveling speed of the vehicle is greater than the reference speed And outputs the first threshold TH1 to the object region separator 144. If the traveling speed of the vehicle is higher than the reference speed, the second threshold TH2, which is larger than the first threshold TH1, And outputs it to the object region separator 144. Then, the object region separation unit 144 may adaptively detect the object region in the second OF frame (OF2 (n)) by adjusting the threshold value according to the current vehicle running speed, that is, the moving speed of the camera. In the case where the speed of the vehicle is high, that is, in the case of the frames obtained in an environment in which the moving speed of the camera is high, the shift amount of the optical flow between the adjacent frames is large. In this case, when a low threshold is applied, Detection is difficult. Conversely, in the case of frames obtained in an environment in which the moving speed of the camera is low, the shift amount of the optical flow between adjacent frames is small, and therefore, if a high threshold value is applied in this case, accurate object detection is similarly difficult. However, in the present invention, as described above, a high threshold is applied when the moving speed of the camera is high, and a low threshold is applied when the moving speed of the camera is high.

In this embodiment, although two embodiments of FIGS. 4 and 5 are shown as methods for improving the accuracy of object detection, the accuracy of object detection can be further improved through the combination of these embodiments. Those skilled in the art will appreciate that such a combined embodiment can be sufficiently implemented through the description of FIGS. 4 and 5, so a detailed description thereof will be omitted.

FIG. 6 is a flowchart showing an object detection method according to an embodiment of the present invention. The subject of each step shown in FIG. 6 can be clearly understood by referring to FIG. 1 to FIG. 5, It does not refer to the person performing the step.

Referring to FIG. 6, in step S610, an average movement amount between a previous frame and a current frame is calculated for all pixels using an optical flow scheme. Calculating an average movement amount includes: calculating a movement amount of each pixel between a previous frame and a current frame, and arranging the movement amounts of the pixels in order of magnitude; dividing the movement amount arranged in the size order into a predetermined unit interval, And a step of calculating an average value of the movement amounts distributed in the predetermined interval as the average movement amount.

Next, in step S620, the process of compensating the background movement amount of the background frame of the previous frame by the calculated average movement amount and constructing the background frame with the background movement amount compensated as the background frame of the current frame is performed. The method includes the steps of extracting a pixel value of each pixel in a background frame of the current frame, performing a predetermined learning on a difference between the extracted pixel value and a pixel value of the compensated background frame corresponding to a position of the pixel value, And a step of replacing the pixel value according to the calculation result with the pixel value of each pixel in the compensated background frame.

In step S630, a background frame replaced with the pixel value is constructed as a background frame in the current frame.

In step S640, the process of computing the movement amount of each pixel between the current frame and the configured background frame is performed using the optical flow method. In step S650, the coordinate value of each pixel corresponding to the movement amount exceeding the preset threshold value And detecting the detected coordinate value as the object region in the current frame is performed. In order to increase the detection accuracy of the object region, a mask frame corresponding to the difference between the compensated background frame and the current frame is generated. The detected object region is filtered using the mask frame, And detecting the object region as a final object region in the current frame. Alternatively, the threshold value may be adaptively adjusted according to the moving speed value of the camera. For example, in the case of a camera mounted on a vehicle, a coordinate value of each pixel corresponding to a moving amount exceeding the first threshold value TH1 is detected as an object area, When the reference speed is higher than the reference speed, the coordinate value of each pixel corresponding to the movement amount equal to or larger than the second threshold value TH2 larger than the first threshold value TH1 can be detected as the object area.

As described above, the present invention provides a background separation method using an optical flow, in which two consecutive frames are input from an image and a movement amount (or a motion vector) of the entire image pixel is calculated through an optical flow, For example, a moving amount occupying more than 70% is applied to the previous frame in reverse based on the same number of movement amounts to compensate for background motion. Then, the background frame is separated using the compensated previous frame and the current frame. We obtain clustering movement vector (vector) for separated background frame and objects moving through the optical flow back to the current frame. The reference value (vector boundary value) of the movement amount that linearly divides the moving object is automatically determined according to the number of clusters, and only a small number of clusters having the largest movement amount are determined. Image noise reduction filters such as a background separation mask are applied to the determined clusters, and the background separation mask obtained through background separation of the background frame and the current frame is applied to the determined clusters to increase the accuracy. The vector of the movement of the clusters is transformed into the coordinate system of the original image and outputted, thereby detecting the final object area.

The embodiments of the present invention have been described above. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

Claims (8)

Calculating an average movement amount between a previous frame and a current frame for all pixels using an optical flow scheme;
Compensating the background movement amount of the background frame of the previous frame by the calculated average movement amount and constructing the background frame in which the background movement amount is compensated as the background frame of the current frame;
Calculating a movement amount of each pixel between the current frame and the configured background frame using the optical flow scheme; And
Calculating a coordinate value of each pixel corresponding to a movement amount equal to or greater than a predetermined threshold value and detecting the detected coordinate value as an object region in the current frame,
The step of calculating the average movement amount includes:
Calculating a movement amount of each pixel between a previous frame and a current frame, and arranging movement amounts of the pixels in order of magnitude;
A step of equally dividing the movement amount arranged in the order of magnitude into a predetermined unit interval, and selecting an interval in which the greatest amount of movement is distributed among the equally divided intervals; And
Calculating an average value of the movement amounts distributed in the predetermined interval as the average movement amount;
/ RTI >
delete 2. The method of claim 1, wherein the background frame comprises:
Extracting a pixel value of each pixel in a background frame of the current frame;
Calculating a compensated pixel value by calculating a predetermined learning coefficient for the extracted pixel value; And
Replacing the compensated pixel value with a pixel value of each pixel in the background frame compensated for by the background movement amount;
/ RTI >
The method of claim 1, wherein the step of detecting an object region in the current frame comprises:
Generating a mask frame corresponding to a difference between the compensated background frame and the current frame;
Filtering the detected object region using the mask frame; And
Detecting the filtered object region as a final object region in the current frame;
/ RTI >
2. The apparatus of claim 1,
The current frame is determined according to a moving speed value of a camera that acquires the previous frame and the current frame.
An image acquiring unit that photographs a surrounding situation in a mobile environment and sequentially outputs a previous frame and a current frame;
Calculates an average movement amount between the previous frame and the current frame for all the pixels using the optical flow method, compensates the background movement amount of the background frame of the previous frame by the calculated average movement amount, To a background frame of the current frame; And
Calculating a movement amount of each pixel between the current frame and the compensated background frame using the optical flow method, calculating coordinate values of each pixel corresponding to a movement amount exceeding a preset threshold value, And an object detection unit for detecting the object region in the object region,
The background extracting unit extracts,
The amount of movement of each pixel between the previous frame and the current frame is calculated, the movement amounts of the pixels are arranged in order of magnitude, the movement amount arranged in the order of magnitude is equally divided into a predetermined unit interval, And selecting an interval and calculating an average value of the movement amounts distributed in the selected interval as the average movement amount.
A vehicle object detecting apparatus mounted on a vehicle,
An image acquiring unit that photographs a surrounding situation of the vehicle while the vehicle is running, and sequentially outputs a previous frame and a current frame;
Calculates an average movement amount between the previous frame and the current frame for all the pixels using the optical flow method, compensates the background movement amount of the background frame of the previous frame by the calculated average movement amount, To a background frame of the current frame; And
Calculating a movement amount of each pixel between the current frame and the compensated background frame using the optical flow method, determining a threshold value according to a running speed value of the vehicle provided from an electronic control unit in the vehicle, And an object detecting unit for calculating coordinate values of each pixel corresponding to the movement amount and detecting the detected coordinate value as an object region in the current frame,
The background extracting unit extracts,
The amount of movement of each pixel between the previous frame and the current frame is calculated, the movement amounts of the pixels are arranged in order of magnitude, the movement amount arranged in the order of magnitude is equally divided into a predetermined unit interval, And selecting an interval and calculating an average value of the movement amounts distributed in the selected interval as the average movement amount. A method of detecting an object using a vehicle object detecting apparatus mounted on a vehicle,
A step of photographing a surrounding situation of the vehicle during driving of the vehicle and successively outputting a previous frame and a current frame
Calculates an average movement amount between the previous frame and the current frame for all the pixels using the optical flow method, compensates the background movement amount of the background frame of the previous frame by the calculated average movement amount, Extracting a current frame as a background frame of the current frame; And
Calculating a movement amount of each pixel between the current frame and the compensated background frame using the optical flow method, determining a threshold value according to a running speed value of the vehicle provided from an electronic control unit in the vehicle, Calculating a coordinate value of each pixel corresponding to the movement amount and detecting the detected coordinate value as an object region in the current frame,
Wherein the step of calculating the average movement amount in the process of extracting the background frame of the current frame includes:
Calculating a movement amount of each pixel between a previous frame and a current frame, and arranging movement amounts of the pixels in order of magnitude;
A step of equally dividing the movement amount arranged in the order of magnitude into a predetermined unit interval, and selecting an interval in which the greatest amount of movement is distributed among the equally divided intervals; And
And calculating an average value of the movement amounts distributed in the predetermined interval as the average movement amount.

Images