KR101921608B1 - Apparatus and method for generating depth information - Google Patents

Abstract

An apparatus for generating depth information, comprising: a patch extracting unit for extracting a boundary component of an input image and extracting a multi-scale image patch according to the intensity of the boundary component; A blur information acquiring unit that calculates blur information of the image patch using the feature information, and a blur information acquiring unit that acquires depth information of the image patches and color information of the input image And a depth information obtaining unit for obtaining depth information of the entire image using the depth information.

Description

[0001] APPARATUS AND METHOD FOR GENERATING DEPTH INFORMATION [0002]

The present invention relates to an apparatus and method for generating depth information.

Generally, the method of estimating the degree of blur and depth information in an image acquires information based on an image obtained by varying the focal distance, so that a plurality of images must be acquired.

In the general method of estimating depth information in a single image, there is uncertainty in the degree of blur in a homogeneous region, and in the case of a patch-based study, the uncertainty of the patch size causes a depth uncertainty It is difficult to acquire accurate depth information.

An object of the present invention is to provide an apparatus and method for estimating accurate depth information from a single image of one frame.

In one embodiment of the present invention, a depth information generating apparatus includes a patch extracting unit for extracting a boundary component of an input image and extracting a multi-scale image patch according to the intensity of the boundary component, A feature information extracting unit for extracting at least one feature information for determining blur information of an input image, a blur information acquiring unit for calculating blur information of the image patch using the feature information, And a depth information obtaining unit for obtaining depth information of the entire image using the color information of the input image.

The patch extracting unit may use a canny edge detector to determine a weak boundary between pixels included in the image whose boundary value is less than or equal to a reference value and determine a strong boundary to be a portion that exceeds the reference value.

The reference value may include a first reference value for determining the strong boundary and a second reference value for determining the weak boundary.

Wherein the feature information extracting unit includes a DCT feature information extracting unit that calculates DCT feature information based on a 2D discrete cosine transform (DCT) value of the image patch, a DCT feature information extracting unit that extracts DCT feature information based on a Gradient value A slope feature information extracting unit for extracting a slope histogram and calculating slope feature information using the slope histogram, extracting a singular value from a singular value decomposition (SVD) of the image patch, An SVD feature information extracting unit for calculating SVD feature information using the specific value, and a deep feature information extracting unit for calculating deep feature information using RGB values of the image patch.

The blur information acquiring unit may calculate blur extent of the image patch through a classifier that is learned as learning data based on the DCT feature information, the slant feature information, the SVD feature information, and the deep feature information.

The depth information obtaining unit obtains depth information of a pixel included in the input image by using a depth propagation method for estimating surrounding depth information using the degree of blur of the image patch and the color information of the image patch as initial depth information, Can be obtained.

According to an aspect of the present invention, there is provided an apparatus for generating depth information, the apparatus including a memory for storing a program, a processor for executing an operation implemented in the program in association with the memory, Wherein the program extracts a multi-size image patch according to the intensity of a boundary component of the input image, extracts at least one feature information from the image patch, And calculating depth information of the entire image using the degree of blur and the color information of the input image.

The program uses a canny edge detector to determine a weak boundary between a portion of the pixels included in the image and a boundary of which the boundary value is less than a reference value, .

The reference value may include a first reference value for determining the strong boundary and a second reference value for determining the weak boundary.

The program calculates DCT feature information based on a 2D discrete cosine transform (DCT) value for the image patch, calculates gradient feature information based on a gradient value inside the image patch, A SVD feature information is calculated using the singular value by singular value decomposition (SVD) of the image patch, and the SVD feature information is calculated using the singular value, And may include instructions for computing information.

The program may include instructions for calculating a fog degree of the image patch based on the DCT feature information, the slope feature information, the SVD feature information, and the deep feature information through a classifier learned by learning data have.

The program may further comprise a depth propagation method for estimating surrounding depth information using the degree of blur of the image patch and the color information of the image patch as initial depth information to calculate a depth of a pixel included in the input image And may include instructions for computing information.

A method of generating depth information of an apparatus according to an embodiment of the present invention includes receiving a single image of one frame, extracting a multi-scale image patch according to the intensity of the image boundary, extracting at least one feature Extracting image blur information based on the feature information, and acquiring depth information of all image pixels based on the image blur information.

Wherein the step of extracting the image patch uses a canny edge detection method to determine a portion of the pixels included in the image whose boundary value is less than or equal to a reference value as a weak boundary, .

The step of extracting the feature information may include calculating DCT feature information based on a 2D discrete cosine transform (DCT) value of the image patch, calculating a slope based on a gradient value in the image patch, Extracting a histogram and calculating inclination feature information using the inclination histogram; extracting a singular value through singular value decomposition (SVD) of the image patch; Calculating SVD feature information, and calculating deep feature information using the RGB values of the image patch.

The step of extracting the image blur information may include calculating a degree of blur of the image patch through a classifier learned as learning data based on the DCT feature information, the slant feature information, the SVD feature information, and the deep feature information .

Wherein the step of acquiring the depth information comprises the steps of: obtaining depth information of the image patch by using a depth propagation method for estimating surrounding depth information using the degree of blur of the image patch and the color information of the image patch as initial depth information; The depth information of the included pixels can be obtained.

According to the embodiment of the present invention, accurate and high-quality depth information can be obtained from a single image of one frame.

1 is a configuration diagram of an apparatus for generating depth map information according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating extraction of weak boundaries and strong boundaries from an input image according to an embodiment of the present invention.
FIG. 3 illustrates a process of extracting DCT feature information from an input image patch according to an embodiment of the present invention. Referring to FIG.
4 is a diagram illustrating a process of extracting gradient feature information from an input image patch according to an embodiment of the present invention.
5 is a diagram illustrating SVD feature information extracted from an input image patch according to an embodiment of the present invention.
6 is a diagram illustrating a process of extracting deep feature information from an input image patch according to an embodiment of the present invention.
7 is a diagram illustrating deep feature information extracted from an input image patch according to an embodiment of the present invention.
8 is a diagram illustrating a process of extracting blur information from feature information of an input image patch according to an embodiment of the present invention.
FIG. 9 is a diagram showing that feature information includes size information of an image patch from which the feature is extracted.
10 is a flowchart of a depth information generating method according to an embodiment of the present invention.
11 is a diagram illustrating a depth information generating method according to an embodiment of the present invention.
FIGS. 12 to 15 are examples of a technology field in which depth information obtained according to an embodiment of the present invention can be applied.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as " comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise. Also, the terms " part, " " module, " and " module ", etc. in the specification mean a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software have.

A single image of one frame may be, for example, an image photographed with a digital camera such as a mobile terminal or DSLR. It is not easy to acquire accurate depth information in one image. In the following, a method of generating a dense depth map from an image of one frame will be described.

1 is a configuration diagram of an apparatus for generating depth map information according to an embodiment of the present invention.

Referring to FIG. 1, the depth information generation apparatus 100 extracts multi-size patches from one image, acquires image feature information from a multi-size patch, and acquires depth information on the entire image using the acquired image feature information . To this end, the depth information generation apparatus 100 includes an image input unit 110, a patch extraction unit 130, a feature information extraction unit 150, a blur information acquisition unit 170, and a depth information acquisition unit 190 do. The depth information generation apparatus 100 includes at least one processor and at least one memory, and uses a processor and a memory to drive a program that implements the operations described in the present invention.

The image input unit 110 receives an image used for acquiring depth information. It is assumed that the image input unit 110 receives one frame image.

The patch extracting unit 130 extracts patches of various sizes according to the intensity of the boundary of the image to estimate blur of the input image. Generally, strong boundary includes boundary component and texture component which can provide much information for estimation of cloudiness and depth information. Therefore, patches of small size are extracted at strong boundaries, whereas in case of weak boundary, Therefore, we extract a lot of information that helps to estimate cloudiness and depth information by extracting large patches. That is, the patch extraction unit 130 extracts only the patches of the textured portion because the homogeneous portion without the texture does not include the information for grasping the blurring degree in the input image.

The patch extracting unit 130 detects edge information of an image and distinguishes a strong boundary portion from a weak boundary portion. FIG. 2 is a diagram illustrating extraction of weak boundaries and strong boundaries from an input image according to an embodiment of the present invention. Referring to FIG. 2, the patch extracting unit 130 uses a canny edge detector to determine a weak boundary between pixels included in the image and a strong boundary .

The reference value may include a first reference value for determining a strong boundary and a second reference value for determining a weak boundary, and the reference value may be adjusted to extract a stronger boundary value. The patch extracting unit 130 extracts a portion determined to be a strong boundary with a patch of a small scale and a portion determined to be a weak boundary with a patch having a large scale.

The feature information extraction unit 150 extracts feature information from the input image. The feature information refers to features of the extracted image in order to obtain depth information from the input image, and the feature information extraction unit 150 extracts feature information using an essential matrix of the image.

The depth information generation apparatus 100 according to the present invention extracts feature points of an image in various ways in order to analyze high frequency components, boundary values, and details of an image patch in order to acquire depth information from one image. The depth information generation apparatus 100 includes a DCT feature information extraction unit 151, a gradient feature information extraction unit 153, an SVD feature information extraction unit 155, and a deep feature information extraction unit 157.

FIG. 3 illustrates a process of extracting DCT feature information from an input image patch according to an embodiment of the present invention. Referring to FIG.

3, the DCT feature information extracting unit 151 extracts discrete cosine transform (DCT) feature values to analyze how much high frequency components are included in the image patch extracted by the patch extracting unit 130 . The DCT feature value f _D can be expressed by Equation (1).

The value obtained by the DCT characteristic information extracting unit 151 from Equation (1) is f _D. Here, f _D denotes DCT feature information represented by a vector of n _D -dimensions.

는 극 좌표(Polar Coordinates)로 표현된 DCT 패치이다. DCT 패치는 패치 추출부(130)에서 추출한 패치에 대한 2D DCT를 계산하여 획득할 수 있다.

는 Radial axis의 k번째 경계값을 의미하고,

는

와

로 둘러싸인 영역의 넓이, W_D는 특징의 합을 1로 만들기 위한 표준화 값, 그리고 n_D는 특징의 차원을 나타낸다.

Is a DCT patch expressed in Polar Coordinates. The DCT patch can be obtained by calculating the 2D DCT for the patch extracted by the patch extracting unit 130. [

Denotes the kth boundary of the radial axis,

The

Wow

W _D is the normalized value to make the sum of the features equal to 1, and n _D is the dimension of the feature.

4 is a diagram illustrating a process of extracting gradient feature information from an input image patch according to an embodiment of the present invention.

4, the gradient feature information extracting unit 153 extracts a gradient histogram using the gradient values inside the image patch (FIG. 4A) in order to analyze how strong a gradient is included in the image, To calculate a slope feature value (Fig. 4B). The slant feature point f _G can be expressed by Equation (2).

H _G is the Gradient histogram of the image patch, W _G is the normalized value to make the sum of the features equal to 1, and n _D is the dimension of the feature.

5 is a diagram illustrating SVD feature information extracted from an input image patch according to an embodiment of the present invention.

The SVD feature information extracting unit 155 derives a Singular Value through Singular Value Decomposition (SVD) of the image patch in order to analyze whether the detail of the image is included, and uses the SVD feature value . The SVD feature value (f _s ) can be expressed as Equation (3).

는 k번째 특이값(singular value)이고, W_s는 특징의 합을 1로 만들기 위한 표준화 값, 그리고 n_S는 특징의 차원을 나타낸다.

Is the k-th singular value decomposition (singular value), W _s is a standard value for making the sum of the characteristics with 1, and _S n represents the dimension of the feature.

)으로 사용하고, [27x27]일 경우 27개를 얻을 수 있으며, 그 중 크기가 큰 순서대로 25개를 계산하여 특징 계산의 입력(

)으로 사용한다. 이것을 입력으로 계산한 SVD 특징은 f_s로 나타내며, 이것은 n_S-차원의 벡터로 나타낸다. 즉,

의 출력을 얻게 된다.If the input image size is [15x15], 15 outlier values can be obtained,

), 27 for [27x27] and 27 for 27 [27x27], and 25 of them are calculated in order of size,

). The SVD feature, calculated as an input, is denoted by f _s , which is represented by a vector of n _S -dimensions. In other words,

.

FIG. 6 is a diagram illustrating a process of extracting deep feature information from an input image patch according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating deep feature information extracted from an input image patch according to an embodiment of the present invention .

The deep feature point extracting unit 157 extracts deep feature information using R, G, and B image patches.

The deep feature extraction unit 157 uses a convolutional network that is trained with a large number of learning data. That is, the R, G, and B image patches are extracted through the Convolutional Network, which consists of a convolutional layer and a pooling layer.

Referring to FIG. 6, a feature map is formed through convolution operations using 25 different 3x3 convolution kernels in the first convolution layer. The next feature map moves by 2 spaces and performs 3x3 Max Pooling. In the next convolution layer, a convolution using 50 3x3 kernels is performed to form a feature map. This feature map also applies stride 2, 3x3 Max Pooling. We then apply convolution operations using 50 1x1 kernels and convolution operations using 25 1x1 kernels in the last layer. After each convolution, a ReLU (Rectified Linear Unit) rectifier was used. The last 25 1x1 kernels can be used to obtain deep feature information (fc) represented by a 25-dimensional vector.

FIG. 8 is a diagram illustrating a process of extracting blur information from feature information of an input image patch according to an embodiment of the present invention, and FIG. 9 is a flowchart illustrating a process of extracting blur information from feature information, FIG.

Referring to FIG. 8, the blur information obtaining unit 170 calculates blur information of the image patch using the feature information extracted by the feature information extracting unit 150. FIG. The blur information obtaining unit 170 determines blur information of an image using a learned classifier using training data. The training data consists of image patches and labels for the degree of blur corresponding to the patches. The image patches are extracted at strong boundaries in the image with the correct focus, and a randomly generated blur kernel is applied To create a blurred patch. The blurred image patch can be expressed as Equation (4).

는 초점이 정확히 맞은 영상 내의 강한 경계에서 추출된 영상 패치이고,

는 흐림 커널이다.

Is an image patch extracted from a strong boundary in an image with the correct focus,

Is a cloudy kernel.

)은 수학식 5와 같이 표현될 수 있다. Cloudy kernel (

Can be expressed as Equation (5).

은 l번째 흐림 커널의 표준편차 값이다.x, y is the distance from the center of the image patch

Is the standard deviation value of the lth blur kernel.

를 얻을 수 있다.

에 대하여 각각 다른 값을 할당할 수 있고, 따라서 만일 어떤 영상 패치가 몇 번째 l 에 해당하는 흐림 커널로 생성된 흐려진 영상 패치임을 판별해 낼 수 있다면, 해당하는 흐림 커널인

를 바로 알아낼 수 있다. Through this operation, blurred image patch

Can be obtained.

So that if it is possible to determine which image patch is a blurred image patch generated by a blur kernel corresponding to a certain number of l,

.

The classifier learned by the learning data as described above determines the degree of blur of the image using the feature information extracted by the feature information extracting unit 150. The blur information obtaining unit 170 obtains the blur information using the vectors connecting the DCT feature information, the gradient feature information, the SVD feature information, and the deep feature information extracted by the feature information extracting unit 150, It returns the probability that it will correspond to the label from 1 to L finally through the Fully Connected Layer, and assigns the label having the highest probability to the label of the corresponding input. This label is a measure of the fuzziness of the input patches.

Referring to FIG. 8, the first layer has 300 nodes, and the input is delivered to the next layer using a rectified linear unit (ReLU) rectifier. The second layer has 150 Nodes and also uses the ReLU rectifier to pass the input to the next layer. The third layer has 11 nodes (the same number as L), and the ReLU rectifier and Softmax function are used to find the probabilities belonging to each label.

At this time, since the size of the image patch is not uniform but multi-scale according to the characteristics of the boundary value, learning may not converge when creating the learning data. Therefore, as shown in FIG. 9, in order to include the size information in the feature information itself, a position for recording the feature extracted in each size is determined, and a constant position for replacing the Bias term of the corresponding Neural Network is additionally allocated. In addition, the weight corresponding to the assigned constant position replaces the role of Bias, and enables the convergence of learning using features extracted from the multi-scale.

In this way, the blur information acquiring unit 170 can acquire the reconstructed depth information, but extracts only the depth information of a rare extraction patch, so that it is not sufficient to generate accurate depth information of the entire image. Next, a method of acquiring dense depth information of the entire image will be described.

The depth information obtaining unit 190 restores the depth information of all the pixels of the image based on the blur information of the image patch in the blur information obtaining unit 170. The depth information obtaining unit 190 can obtain the depth information of the entire image using the depth propagation method using the blur information of the image patch extracted by the blur information obtaining unit 170 as the initial depth information .

The depth information obtaining unit 190 uses the color information and the blur information of the image to obtain depth information of the entire image. In this case, if the input image is used as it is, a small texture included in the image patch may cause noise during the propagation process, so that an edge-preserving smoothing filter can be applied.

In order to propagate the blurring information obtained at the boundary of the input image to neighboring pixels, a color line model in which neighboring pixels exist on a straight line of RGB space is applied, and a matting Laplacian The matting Laplacian matrix can be expressed as Equation (6).

는 Kronecker 델타 함수,

는 관심 영역의 평균 및 분산값,

는 단위행렬, 그리고

는 정규화 상수이다. 경계 보존형 평활 필터를 적용한 입력 영상

의 각 픽셀 위치(i,j,k)에서의 RGB값([r,g,b])을 이용하여 관심 영역의 평균 및 분산값을 계산하고, 이를 통해 매팅 라플라시안 매트릭스(L)을 계산한다.In Equation (6), i, j and k denote the linear index values of the pixels, w _k denotes the region of interest with the center k,

The Kronecker delta function,

Is the mean and variance of the region of interest,

Is the unit matrix, and

Is a normalization constant. Input image with boundary preservation type smoothing filter

The average and variance values of the ROIs are calculated using the RGB values ([r, g, b]) at the respective pixel positions (i, j, k) of the Laplacian matrix L and the matched Laplacian matrix L is calculated.

Finally, the depth information obtaining unit 190 calculates the depth information I _F of the entire image based on the matched Laplacian matrix L of the input image and the blur accuracy value at the boundary acquired by the blur information acquiring unit 170 , The depth information of the entire image can be expressed by Equation (7).

는 사용자 변수, 그리고

는 I_S(t)가 0이 아닐 경우 D(i,j)가

인 행렬이다. In Equation (7), I _S is the blur information at the image boundary,

Is a user variable, and

If the I _S (t) is not 0 D (i, j) is

.

FIG. 10 is a flowchart of a depth information generating method according to an embodiment of the present invention, and FIG. 11 is a diagram illustrating a depth information generating method according to an embodiment of the present invention.

Referring to FIGS. 10 and 11, the depth information generation apparatus 100 receives a single image of one frame (S110).

The depth information generation apparatus 100 extracts a multi-scale image patch according to the intensity of the boundary of the input image (S120). The depth information generation apparatus 100 detects edge information of an image to discriminate a strong boundary portion from a weak boundary portion. The portion determined as a strong boundary is a patch of a small scale and is determined as a weak boundary The extracted part is extracted with a patch of a large scale.

The depth information generation apparatus 100 obtains the sparse feature information from the extracted multi-scale image patch (S130). The feature information is a feature point of the extracted image to obtain depth information from the input image. The depth information generation apparatus 100 calculates DCT feature information, gradient feature information, SVD feature information, and deep feature information of an image using Equations (1), (2) and (3) That is, the depth information generation apparatus 100 calculates feature information of an image in various ways in order to acquire depth information from a single image.

The depth information generation apparatus 100 calculates the blur extent of the image by combining the feature information (S140). The depth information generation apparatus 100 extracts the degree of blur of the image based on the feature information extracted using the classifier learned by the learning data.

The depth information generation apparatus 100 restores depth information of all the pixels of the image based on the blur information (S150). The depth information generation apparatus 100 can acquire dense depth information from the flow information using a depth propagation method. The depth information generation apparatus 100 may use three-dimensional coordinates of feature points representing color information and flow information of an image in order to acquire depth information on the entire image.

In this way, the depth information generation apparatus 100 can acquire depth information as shown in FIG. 11 (h) from a single image of one frame.

Next, a description will be given of a field that can be applied by using the depth information generated by the depth information generating apparatus 100. FIG.

FIGS. 12 to 15 are examples of a technology field in which depth information obtained according to an embodiment of the present invention can be applied.

Referring to FIG. 12, the present invention can extract depth and blur information from a single input image and apply it to synthesis with other images. At this time, the image synthesis can be performed using the foreground and background areas extracted through the estimated blur and depth information.

Referring to FIG. 13, the present invention can be used to emphasize the foreground of an image by extracting depth and blur information from a single input image and increasing blurring degree.

Referring to FIG. 14, the present invention can be used for extracting depth and blur information from a single input image and restoring deblurring, that is, focus.

Referring to FIG. 15, the present invention can be used to extract depth and blur information from a single input image and restore it to a 3D image.

Figure 15 compares depth information and 3D images generated according to one embodiment of the present invention with depth maps and 3D images generated by the prior art.

In general, conventionally, when estimating the degree of blur using a single image, the ratio of the edge gradient magnitude of the original image to the blurred boundary magnitude by the arbitrarily generated blur kernel is used Estimate the degree of cloudiness. The degree of blur estimated at the boundary is propagated to the entire image using the color information of the input image. The accuracy of this method depends largely on the accuracy of edge alignment. In the case of radio waves using an input image, it is often propagated to a wrong value by a small texture. Therefore, the reliability of the entire blur degree is very low.

Referring to FIG. 15, the present invention can acquire high-quality depth information and acquire an accurate three-dimensional image.

Thus, the present invention can acquire accurate depth information from one single image.

The embodiments of the present invention described above are not implemented only by the apparatus and method, but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (10)

As a depth information generating apparatus,
A patch extracting unit for extracting multi-scale image patches with a patch size inversely proportional to a boundary strength in an input image,
A feature information extracting unit for extracting feature information of the image patch from each of the image patches,
A blur information acquiring unit for calculating blur information corresponding to the feature information extracted by the feature information extracting unit based on the blur information classifier learned to output blur information from the feature information of the training image patches,
A depth information acquiring unit for acquiring the depth information of the input image by depth propagating the blur information of the input image obtained by the blur information acquiring unit and the color information of the input image,
And a depth information generating unit for generating depth information. The method of claim 1,
The patch extracting unit
And determines a boundary strength of the boundary component based on a boundary value of the boundary component included in the input image. The method of claim 1,
The feature information extracting unit
A DCT feature information extracting unit for calculating DCT feature information based on a 2D discrete cosine transform (DCT) value for each image patch,
A slope feature information extracting unit for extracting a slope histogram based on a gradient value of each image patch and calculating slope feature information using the slope histogram,
An SVD feature information extracting unit for extracting a singular value through singular value decomposition (SVD) of each image patch and calculating SVD feature information using the singular value;
And a deep feature information extracting unit for calculating deep feature information using RGB values of each image patch,
The blur information obtaining unit
Wherein at least one feature information of each image patch extracted by the feature information extracting unit is input to the blur information classifier to calculate blur information of the corresponding image patch. The method of claim 1,
The depth information obtaining unit
A depth information propagation method for estimating neighboring depth information using blur information of the image patches and color information of the image patches as initial depth information in a convolution network, Device. As a depth information generating apparatus,
A memory for storing a program for generating depth information from an input image, and
And a processor for executing an operation implemented in the program in association with the memory,
The program including a depth information calculation program,
The depth information calculation program
Extracting multi-scale image patches with a patch size inversely proportional to a boundary strength in the input image,
Extracting feature information of the image patch from each of the image patches,
Calculating blur information corresponding to the feature information extracted from each of the image patches based on the blur information classifier learned to output blur information from the feature information of the training image patches, and
Calculating depth information of the input image by depth propagation of the blur information of the image patches calculated in the input image and color information of the input image,
The depth information generating apparatus comprising: The method of claim 5,
The program further comprises a learning program for learning the blur information classifier used in the depth information calculation program,
The learning program
Extracting an image patch having a strong boundary focused on the learning images, applying a blur kernel to the image patch of the strong boundary to generate a blurred patch to generate the learning image patches, and
A step of learning the blur information classifier by receiving the learning image patches
The depth information generating apparatus comprising: The method of claim 5,
The depth information calculation program
In the step of extracting the feature information,
Discrete Cosine Transform (DCT) feature information of each image patch, gradient feature information of each image patch, Singular Value Decomposition (SVD) feature information of each image patch, and each image patch And calculating the deep feature information using the RGB values of the depth information,
In the step of calculating the blur information,
And calculating blur information of the corresponding image patch from feature information of each image patch. A method of generating depth information of an apparatus,
Extracting multi-scale image patches with a patch size inversely proportional to the boundary strength in the input image,
Extracting feature information of the image patch from each of the image patches,
Calculating blur information corresponding to the feature information extracted from each of the image patches based on the blur information classifier learned to output blur information from the feature information of the training image patches, and
Acquiring the depth information of the input image by performing depth propagation on the blur information of the image patches calculated in the input image and the color information of the input image,
And generating a depth information. 9. The method of claim 8,
The step of extracting the feature information
Extracting at least one feature information from each image patch,
The at least one feature information may include at least one of 2D discrete cosine transform (DCT) feature information, gradient feature information, singular value decomposition (SVD) feature information, and deep feature information using RGB values Wherein the depth information comprises at least one of the depth information. 9. The method of claim 8,
Further comprising learning the blur information classifier prior to calculating the blur information,
The step of learning the blur information classifier
Extracting an image patch having a strong boundary focused on the learning images, applying a blur kernel to the image patch of the strong boundary to generate a blurred patch to generate the learning image patches, and
And receiving the training image patches and learning the blur information classifier.

Images