CN107371016A - Based on asymmetric distortion without with reference to 3D stereo image quality evaluation methods - Google Patents

Based on asymmetric distortion without with reference to 3D stereo image quality evaluation methods Download PDF

Info

Publication number
CN107371016A
CN107371016A CN201710613185.7A CN201710613185A CN107371016A CN 107371016 A CN107371016 A CN 107371016A CN 201710613185 A CN201710613185 A CN 201710613185A CN 107371016 A CN107371016 A CN 107371016A
Authority
CN
China
Prior art keywords
gabor
left view
right view
view
pyramid
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710613185.7A
Other languages
Chinese (zh)
Inventor
沈丽丽
彭科
雷锦艺
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN201710613185.7A priority Critical patent/CN107371016A/en
Publication of CN107371016A publication Critical patent/CN107371016A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N17/00Diagnosis, testing or measuring for television systems or their details
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074Stereoscopic image analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

The present invention relates to it is a kind of based on asymmetric distortion without refer to 3D stereo image quality evaluation methods, comprise the following steps:Using Gabor filter component solution stereo pairs, left view and right view Gabor pyramids are respectively obtained.The one-eyed figure being added after being merged is put pixel-by-pixel with right view Gabor pyramids by left view.Setting value figure is subtracted each other by left view and right view Gabor pyramid absolute differences.One-eyed figure and differential chart are fitted with generalized gaussian model GGD, and extracts statistical nature parameter after fitting, includes average, variance and the form parameter of GGD distributions.The structural similarity SSIM fractions between left view and right view after Gabor filter is decomposed are calculated, in this, as the asymmetrical information of stereo-picture, the statistical nature parameter extracted with reference to previous step, with support vector regression SVR come prognostic chart picture mass fraction.

Description

Non-reference 3D stereo image quality evaluation method based on asymmetric distortion
The technical field is as follows:
the invention relates to the field of objective quality evaluation of 3D images.
Background art:
with the development of the science and technology and the update of the science and technology, the products of the digital age gradually enter the lives of people. Under the wave of the digital era, the field of 3D stereoscopic images is rapidly developing and applying, and compared with two-dimensional planar images, stereoscopic images enable people to experience stereoscopic sensation and presence in real time, and gradually become the mainstream direction of multimedia research. However, the stereo image is often distorted in different forms during encoding, compressing, transmitting, etc., and these distortions all cause the visual quality to be degraded. Therefore, how to accurately and effectively evaluate the image quality has become a research hotspot in the field of image processing.
The 3D image utilizes the binocular parallax principle of human eyes, the two eyes respectively receive images from the same scene, and parallax is formed through brain fusion, so that the stereoscopic impression and the sense of reality are given to a user. In the process, the binocular vision system plays an important role, and because the human eye vision system has a very complex physiological mechanism and relates to multiple interdisciplines from psychology to physics of eyes, the establishment of a three-dimensional image quality evaluation model conforming to human vision characteristics is a difficult point of research. Conventionally, Stereoscopic Image Quality Assessment (SIQA) is mainly classified into two methods: one method is to separately evaluate the left view and the right view, and then obtain objective values by weighting; the other method is to add parallax or depth information to perform comprehensive quality evaluation on the basis of left and right view evaluation. The first method has simple implementation scheme, low algorithm complexity and easy implementation; the second method considers parallax information, and the realization effect is more consistent with that of a human visual system.
Despite the current 3D stereoscopic image quality evaluation methods and the great advances made, most SIQA algorithms ignore some facts: 1) some SIQA algorithms separately consider the statistical features and spatial frequency domain features of natural scene images; 2) the existing SIQA method is more suitable for symmetrical stereo image distortion, but the effect obtained by non-symmetrical distortion is not ideal. Therefore, under the condition of asymmetric distortion, the invention provides a brand-new non-reference 3D stereo image quality evaluation method.
The invention content is as follows:
the technical problems solved by the invention are as follows: aiming at the distortion of an asymmetric 3D image, a novel non-reference 3D stereoscopic image quality evaluation method is provided, a Gabor filter is designed to carry out pyramid decomposition on left and right views of a stereoscopic image, the left and right images are fused according to a binocular competition and binocular fusion principle to obtain a single eye diagram and a difference diagram, then normalization processing is carried out on the two obtained images, generalized Gaussian model fitting is carried out, statistical characteristics are respectively extracted, finally, Support Vector Regression (SVR) is used for quality score prediction, and the experiment result shows that the provided SIQA algorithm achieves a very good effect. The technical scheme of the invention is as follows:
a non-reference 3D stereo image quality evaluation method based on asymmetric distortion comprises the following steps:
1) and decomposing the stereo image pair by using a Gabor filter bank to respectively obtain a left view Gabor pyramid and a right view Gabor pyramid.
2) And adding pixel points by pixel points of the Gabor pyramid of the left view and the Gabor pyramid of the right view to obtain the fused monocular image.
3) And subtracting the Gabor pyramid absolute difference values of the left view and the right view to generate a difference value map.
4) And fitting the single eye diagram and the difference diagram by using a generalized Gaussian model GGD, and extracting statistical characteristic parameters after fitting, including the mean, the variance and the shape parameters of GGD distribution.
5) And calculating the structural similarity SSIM fraction between the left view and the right view after the decomposition of the Gabor filter, taking the fraction as the asymmetric information of the stereo image, and using the support vector regression SVR to predict the image quality fraction by combining the statistical characteristic parameters extracted in the last step.
Description of the drawings:
the implementation steps and advantages of the present invention can be more prominent, and the flow and operation of the present invention can be more easily understood through the attached drawings.
FIG. 1(a) a left view of an asymmetrically distorted 3D image, and FIG. 1(b) a right view of an asymmetrically distorted 3D image;
FIG. 2(a) is a left view of the second layer after decomposition of the Gabor filter, and FIG. 2(b) is a right view of the second layer after decomposition of the Gabor filter;
FIG. 3 is a fused single eye view;
FIG. 4 generates a difference map;
FIG. 5 is a normalized difference plot GGD fit plot;
figure 6 difference map GGD fit map for different distortion types.
The specific implementation mode is as follows:
in order to make the technical solution of the present invention more clear and easy to implement, so as to further highlight the advantages and objects of the present invention, the embodiments of the present invention will be further described and explained in detail with reference to the accompanying drawings.
101: gabor pyramid decomposition
The Gabor filter can accurately describe the spatial frequency domain information and the local correlation of the image. Moreover, the Gabor filter response is very similar to the human visual cortex stimulus response, so the invention adopts a multi-scale Gabor filter bank to decompose the image.
First, for a given stereo image, we select a white noise stereo image in LIVE 3D image database phase-II, which is asymmetrically distorted and whose left and right views are not distorted to the same degree. Fig. 1(a) shows a left view of an asymmetrically distorted 3D image, and fig. 1(b) shows a right view of an asymmetrically distorted 3D image. Decomposing the left view and the right view of the image by a Gabor filter respectively to obtain a Gabor pyramid, wherein the expression of the Gabor filter is shown as a formula (1):
x'=x cosθ+y sinθ;
y'=-x sinθ+y cosθ.
where x, y represent the spatial coordinates of the pixels in each view, x ', y' are the coordinates after rotation, λ is the wavelength, θ is the direction, σ is a parameter related to the bandwidth, γ is a Gabor filter shape parameter, and Ψ is the phase offset.
In theory, by setting different filter bandwidths and directions, Gabor filter banks with different scales and different directions can be generated, however, in order to reduce the computational complexity, in an experiment, the direction of the Gabor filter is selected to be the horizontal direction, the bandwidth is set to be a variable parameter, and the bandwidths are respectively set to be {1, 1.64, 2.68 }. Thus, after each view is decomposed by the multi-scale Gabor filter bank, the result forms a Gabor pyramid consisting of three layers of channels, including left and right views. Finally, the Gabor pyramid decomposition is performed on the graph (1), and as shown in fig. 2, fig. 2(a) is a left view of the second layer after the Gabor filter decomposition, and fig. 2(b) is a right view of the second layer after the Gabor filter decomposition. Since the first, third and second level channels result similarly, we only draw the second level Gabor pyramid decomposed image for non-redundant description.
102: fused monocular picture
In the stereoscopic view, important attributes in human visual characteristics are displayed by binocular competition and binocular fusion, and the two factors are considered for effective 3D stereoscopic image quality evaluation. Therefore, a single-eye map (a "cyclic" map) is used to simulate binocular fusion and binocular competition of a human eye visual system, namely, the left view and the right view decomposed by the Gabor filter are fused to obtain an image which is the single-eye map, and the fusion process is to add the left view and the right view decomposed by the Gabor pyramid pixel by pixel. Because the Gabor filter is formed by 3 layers of channels after decomposition, 3 pieces of fused single-eye images are obtained, namely the fused single-eye images
WhereinEach representing a single eye diagram consisting of 3 layers of channels. As shown in fig. 3, namely, on the second layer channel, the fused single eye diagram is obtained by fusing the left view of fig. 2(a) and the right view of fig. 2 (b).
103: generating a difference map
The difference graph is obtained by calculating a difference value from a left view and a right view of the Gabor pyramid, in order to obtain an absolute difference graph, the global energy of each layer of the Gabor pyramid, namely the pixel-by-pixel point square sum of an image, is firstly calculated, taking a channel of a second layer as an example, the global energy of the left view and the right view is calculated to be respectively ELAnd ER. When E isL>ERTime, difference diagramThat is, the corresponding right view pixel point is subtracted from the left view pixel point of the layer 2 channel of the Gabor pyramid, and when E isL<ERTime, difference diagramThat is, the right view pixel point of the layer 2 channel of the Gabor pyramid subtracts the corresponding left view pixel point. Briefly, the difference map is the absolute difference of the left and right views.
Similarly, the difference maps of the first channel and the third channel are respectively calculatedAndas shown in fig. 4, the difference map is generated from the left and right views of the second layer in fig. 2(a) and 2 (b).
DM-GGD feature extraction
In order to effectively evaluate the quality score of the distorted stereo image pair, the IQA algorithm needs to extract good statistical features of the image. And a Generalized Gaussian Distribution (GGD) is a prior model that can well describe the statistical characteristics of an image. Therefore, we fit the single eye diagram and the difference diagram with the GGD (DM-GGD), and select parameters from them to extract image features.
First, we need to map the cyclopeanSum and difference mapCarrying out normalization processing, wherein the calculation formulas are shown as formulas (2) and (3):
wherein I'C(x,y,fn),I'D(x,y,fn) Normalized one-eye and difference plots, respectively, with a constant C of 0.01 to maintain stability, μ (x, y, f)n) And ρ (x, y, f)n) Respectively, the mean and variance of the image, and the calculation formulas are shown in formulas (4) and (5):
Ik,l(x,y,fn) Is represented by a cyclopiaOr difference mapCircularly symmetric gaussian weighting function w ═ wk,l|k=-3,…3,l=-3,…3}。
Then, we used the GGD model to fit I'C(x,y,fn) Or l'D(x,y,fn) The GGD fitting expression is shown in formula (6):
parameter of the above formula (μ, σ)2And γ) are the mean, variance and shape parameters, respectively, of the GGD distribution. Wherein,a=βγ/2(1/γ),
the black bars show the difference plots after normalization, the red curves represent the difference plots of the GGD fit after normalization, and fig. 6 depicts the difference plot distribution for five different distortion types in the LIVE database, as shown in fig. 5. It can be seen that the distorted GGD fit curves of different types are different while satisfying a 0-mean gaussian distribution. In addition, the calculation formula of Skewness (Skewness) in the statistical characteristics is s ═ E (x- μ)33Kurtosis (Kurtosis) calculation formula k ═ E (x- μ)44. Therefore, we finally selected 4 parameters (σ) after GGD fitting2γ, s, k) to represent the statistical features of the 3D stereo image, and at the same time, the single eye diagram and the difference diagram pyramid have 3 layers of channels respectively, so that the total number of feature vectors extracted by us is 24(3 × (4+4)) dimensions.
104: SVR predicted image quality score
We predict image quality using support vector regression SVR in MATLABL environment LIBSVM toolbox. SVR is mainly used for solving the optimization problem, and the expression form is shown as formula (7):
zj,z'j≥0,ξ,μ>0 j=1,…m
wherein x isjIs a feature vector, yjIs the DMOS value corresponding to the training sample, w is the weight, φ (x)j) Is a mapping function, parameter zjAnd z'jIs a relaxation variable, Ω is a bias, ξ and η are related to training samples in training, we use Radial Basis Function (RBF) as a kernel to measure the distance between 2 samples in high-dimensional vector spaceAnd (5) separating.
After extracting the features of each 3D stereo image, a feature vector is obtained. We performed SVR regression prediction experiments on LIVE 3D image database phase-II. The LIVE 3D database phase-II generates 360 pictures from 8 reference stereo images with 5 distortion types, and comprises the following steps: white noise, JPEG compression, JP2K compression, gaussian blur and fast fading. In an experiment, 80% of three-dimensional pictures in a database are randomly selected as training data, 20% of data are used as test data, the iteration frequency is 1000, and the intermediate value of the test data is taken as a final quality evaluation score.
105: results of the experiment
In order to evaluate the performance of the non-reference 3D stereoscopic image quality evaluation proposed by the invention, 3 evaluation indexes of a Spearman Rank Order Correlation Coefficient (SROCC), a Pilson Linear Correlation Coefficient (PLCC) and a Root Mean Square Error (RMSE) are selected to evaluate the performance of the algorithm. The evaluation values of PLCC, SROCC and RMSE corresponding to the DM-GGD algorithm provided by the invention are shown in Table 1. The larger the values of PLCC and SROCC are, the more consistent the values with the subjective evaluation value DMOS are, the better the algorithm effect is, and the smaller the RMSE value is, the better the algorithm effect is. It can be seen from the figure that the values of the DM-GGD algorithm PLCC can reach 0.920 respectively, the evaluation results are high, the consistency with a human eye visual angle system is good, and the performance effect is good under the condition of a single distortion type.
In summary, the invention provides a new non-reference 3D stereo image quality evaluation method DM-GGD mainly for asymmetric 3D stereo image distortion. Through experimental results, the provided evaluation method can well reflect the asymmetric information of the asymmetric 3D stereo image, and the performance of the evaluation method exceeds that of most SIQA algorithms with reference and without reference. Looking forward in the future, further work in the future is to establish a model more conforming to a human visual system and extract more accurate image characteristics to evaluate the quality of a stereo image.
TABLE 1 evaluation index PLCC, SROCC, RMSE results of DM-GGD algorithm

Claims (1)

1. A non-reference 3D stereo image quality evaluation method based on asymmetric distortion comprises the following steps:
1) and decomposing the stereo image pair by using a Gabor filter bank to respectively obtain a left view Gabor pyramid and a right view Gabor pyramid.
2) And adding pixel points by pixel points of the Gabor pyramid of the left view and the Gabor pyramid of the right view to obtain the fused monocular image.
3) And subtracting the Gabor pyramid absolute difference values of the left view and the right view to generate a difference value map.
4) And fitting the single eye diagram and the difference diagram by using a generalized Gaussian model GGD, and extracting statistical characteristic parameters after fitting, including the mean, the variance and the shape parameters of GGD distribution.
5) And calculating the structural similarity SSIM fraction between the left view and the right view after the decomposition of the Gabor filter, taking the fraction as the asymmetric information of the stereo image, and using the support vector regression SVR to predict the image quality fraction by combining the statistical characteristic parameters extracted in the last step.
CN201710613185.7A 2017-07-25 2017-07-25 Based on asymmetric distortion without with reference to 3D stereo image quality evaluation methods Pending CN107371016A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710613185.7A CN107371016A (en) 2017-07-25 2017-07-25 Based on asymmetric distortion without with reference to 3D stereo image quality evaluation methods

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710613185.7A CN107371016A (en) 2017-07-25 2017-07-25 Based on asymmetric distortion without with reference to 3D stereo image quality evaluation methods

Publications (1)

Publication Number Publication Date
CN107371016A true CN107371016A (en) 2017-11-21

Family

ID=60308628

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710613185.7A Pending CN107371016A (en) 2017-07-25 2017-07-25 Based on asymmetric distortion without with reference to 3D stereo image quality evaluation methods

Country Status (1)

Country Link
CN (1) CN107371016A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108156451A (en) * 2017-12-11 2018-06-12 江苏东大金智信息系统有限公司 A kind of 3-D view/video without reference mass appraisal procedure
CN108460752A (en) * 2017-12-20 2018-08-28 宁波大学 A kind of asymmetric more distortion objective evaluation method for quality of stereo images
CN108648186A (en) * 2018-05-11 2018-10-12 北京理工大学 Based on primary vision perception mechanism without with reference to stereo image quality evaluation method
CN109345552A (en) * 2018-09-20 2019-02-15 天津大学 Stereo image quality evaluation method based on region weight
CN110634126A (en) * 2019-04-04 2019-12-31 天津大学 No-reference 3D stereo image quality evaluation method based on wavelet packet decomposition
CN112233089A (en) * 2020-10-14 2021-01-15 西安交通大学 No-reference stereo mixed distortion image quality evaluation method
CN112950629A (en) * 2021-04-02 2021-06-11 上海大学 No-reference panoramic image quality evaluation method and system
CN113362315A (en) * 2021-06-22 2021-09-07 中国科学技术大学 Image quality evaluation method and evaluation model based on multi-algorithm fusion

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011109218A (en) * 2009-11-13 2011-06-02 Victor Co Of Japan Ltd Stereoscopic video imaging device, and stereoscopic video imaging method
CN102722888A (en) * 2012-05-22 2012-10-10 天津大学 Stereoscopic image objective quality evaluation method based on physiological and psychological stereoscopic vision
CN105407349A (en) * 2015-11-30 2016-03-16 宁波大学 No-reference objective three-dimensional image quality evaluation method based on binocular visual perception
CN105654142A (en) * 2016-01-06 2016-06-08 上海大学 Natural scene statistics-based non-reference stereo image quality evaluation method
CN106651835A (en) * 2016-10-20 2017-05-10 天津大学 Entropy-based double-viewpoint reference-free objective stereo-image quality evaluation method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011109218A (en) * 2009-11-13 2011-06-02 Victor Co Of Japan Ltd Stereoscopic video imaging device, and stereoscopic video imaging method
CN102722888A (en) * 2012-05-22 2012-10-10 天津大学 Stereoscopic image objective quality evaluation method based on physiological and psychological stereoscopic vision
CN105407349A (en) * 2015-11-30 2016-03-16 宁波大学 No-reference objective three-dimensional image quality evaluation method based on binocular visual perception
CN105654142A (en) * 2016-01-06 2016-06-08 上海大学 Natural scene statistics-based non-reference stereo image quality evaluation method
CN106651835A (en) * 2016-10-20 2017-05-10 天津大学 Entropy-based double-viewpoint reference-free objective stereo-image quality evaluation method

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108156451A (en) * 2017-12-11 2018-06-12 江苏东大金智信息系统有限公司 A kind of 3-D view/video without reference mass appraisal procedure
CN108460752A (en) * 2017-12-20 2018-08-28 宁波大学 A kind of asymmetric more distortion objective evaluation method for quality of stereo images
CN108460752B (en) * 2017-12-20 2020-04-10 宁波大学 Objective evaluation method for quality of asymmetric multi-distortion stereo image
CN108648186A (en) * 2018-05-11 2018-10-12 北京理工大学 Based on primary vision perception mechanism without with reference to stereo image quality evaluation method
CN108648186B (en) * 2018-05-11 2021-11-19 北京理工大学 No-reference stereo image quality evaluation method based on primary visual perception mechanism
CN109345552A (en) * 2018-09-20 2019-02-15 天津大学 Stereo image quality evaluation method based on region weight
CN110634126A (en) * 2019-04-04 2019-12-31 天津大学 No-reference 3D stereo image quality evaluation method based on wavelet packet decomposition
CN112233089A (en) * 2020-10-14 2021-01-15 西安交通大学 No-reference stereo mixed distortion image quality evaluation method
CN112950629A (en) * 2021-04-02 2021-06-11 上海大学 No-reference panoramic image quality evaluation method and system
CN113362315A (en) * 2021-06-22 2021-09-07 中国科学技术大学 Image quality evaluation method and evaluation model based on multi-algorithm fusion

Similar Documents

Publication Publication Date Title
CN107371016A (en) Based on asymmetric distortion without with reference to 3D stereo image quality evaluation methods
Shao et al. Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties
Zhang et al. Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network
Yang et al. 3D panoramic virtual reality video quality assessment based on 3D convolutional neural networks
Fang et al. Saliency detection for stereoscopic images
CN109360178B (en) Fusion image-based non-reference stereo image quality evaluation method
Chen et al. No-reference quality assessment of natural stereopairs
Ding et al. No-reference stereoscopic image quality assessment using convolutional neural network for adaptive feature extraction
CN110060236B (en) Stereoscopic image quality evaluation method based on depth convolution neural network
Yue et al. Blind stereoscopic 3D image quality assessment via analysis of naturalness, structure, and binocular asymmetry
Shao et al. Learning blind quality evaluator for stereoscopic images using joint sparse representation
Shao et al. Learning receptive fields and quality lookups for blind quality assessment of stereoscopic images
Xu et al. Binocular rivalry oriented predictive autoencoding network for blind stereoscopic image quality measurement
Liu et al. Blind stereoscopic image quality assessment based on hierarchical learning
Lv et al. No-reference stereoscopic image quality assessment using binocular self-similarity and deep neural network
Su et al. Color and depth priors in natural images
CN103780895B (en) A kind of three-dimensional video quality evaluation method
CN108389189B (en) Three-dimensional image quality evaluation method based on dictionary learning
Kim et al. Binocular fusion net: deep learning visual comfort assessment for stereoscopic 3D
Tian et al. Quality assessment of DIBR-synthesized views: An overview
CN108520510B (en) No-reference stereo image quality evaluation method based on overall and local analysis
CN109788275A (en) Naturality, structure and binocular asymmetry are without reference stereo image quality evaluation method
CN110246111A (en) Based on blending image with reinforcing image without reference stereo image quality evaluation method
Han et al. Stereoscopic video quality assessment model based on spatial-temporal structural information
CN111915589A (en) Stereo image quality evaluation method based on hole convolution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20171121