- Email: [email protected]
Reduced-reference stereoscopic image quality assessment based on view and disparity zero-watermarks
Signal Processing: Image Communication 29 (2014) 167–176
Contents lists available at ScienceDirect
Signal Processing: Image Communication journal homepage: www.elsevier.com/locate/image
Reduced-reference stereoscopic image quality assessment based on view and disparity zero-watermarks Wujie Zhou a,b, Gangyi Jiang a,c,n, Mei Yu a,c, Feng Shao a, Zongju Peng a a b c
Faculty of Information Science and Engineering, Ningbo University, Ningbo 315211, China School of Information and Electronic Engineering, Zhejiang University of Science &Technology, Hangzhou 310023, China National Key Lab of Software New Technology, Nanjing University, Nanjing 210093, China
a r t i c l e i n f o
abstract
Article history: Received 22 May 2012 Received in revised form 21 October 2013 Accepted 23 October 2013 Available online 6 November 2013
As a practical and novel application of watermarking, this paper presents a zerowatermarking based objective reduced-reference stereoscopic image quality assessment (RR-SIQA) method. In the proposed method, two kinds of zero-watermarks are constructed according to the characteristics of image structure and stereoscopic perception. Concretely, two view zero-watermarks, which are constructed by judging the relation of the horizontal and vertical components of gradient vectors with respect to the two views, are used to reflect the image structure variation of the stereoscopic image. Meanwhile, a disparity zero-watermark, which is constructed with disparity map of the stereoscopic image, is used to reflect the stereoscopic perception quality variation. Then, the quality of stereoscopic image is objectively assessed by pooling the recovering rates of the detected zero-watermarks. The experimental results show that the stereoscopic image quality evaluation results assessed with the proposed RR-SIQA method are well consistent with subjective assessment, and the proposed method achieves better performance than the widely used full-reference stereoscopic image quality assessment method PSNR in assessing quality of stereoscopic images compressed with JPEG and JPEG2000. & 2013 Elsevier B.V. All rights reserved.
Keywords: Three dimensional TV Stereoscopic image processing Image quality Reduced-reference Zero-watermark
1. Introduction With the technological advancement made in computer graphics, computer vision and network communication [1,2], stereoscopic image processing is becoming increasingly popular and would possibly have many applications [3]. In these applications, stereoscopic images may have gone through various processes, including compression, communication, printing, display, restoration, segmentation, and fusion [4], any of which can introduce distortions that may impair visual qualities of the images. On the other hand, with the evolution towards new multimedia n Corresponding author. Tel.: þ86 574 87600411; fax: þ86 574 87600940. E-mail addresses: [email protected] (W. Zhou), [email protected] (G. Jiang).
0923-5965/$ - see front matter & 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.image.2013.10.005
communication systems and services, users now expect services to be delivered according to their demands on quality [5]. Recently, the concept of quality of service (QoS) has been extended to the new concept of quality of experience (QoE), combining user perception, experience, and expectations with non-technical and technical parameters such as application- and network-level QoS [6]. However, for wireless systems the possible limitations due to the features of the devices and of the transmission channel may result in perceivable impairments that influence the user's perception on images. Therefore, stereoscopic image quality assessment (SIQA) is necessary since it enables to adjust the parameters of stereoscopic image processing so as to optimize stereoscopic image quality or make it to reach a given quality [7]. SIQA methods can be classified as subjective and objective methods. Subjective SIQA method is closer to
168
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
human visual perception [8]. However, it is time consuming and expensive, and cannot be done in real time. By contrast, the objective SIQA method needs less time and is convenient to be implemented. Simple fidelity methods such as mean square error (MSE) or peak signal-to-noise ratio (PSNR) are often used to evaluate the quality of a processed stereoscopic image in the light of the original ones [9–11]. These methods belong to the so-called fullreference stereoscopic image quality assessment (FR-SIQA) methods, which require full information of the original stereoscopic image [12–15]. However, since the end-user usually cannot obtain the original image through the wired or wireless network, it is difficult to implement fullreference image quality assessment in real applications. The opposite is the No Reference (NR) model which does not need any information about the original image, but these kind of methods are useful only when the types of distortions that the stereoscopic images suffered by are fixed and known [16,17]. Thus, they may be limited in modern wired or wireless network, where the distortions could be a combination of lossy compression, network delay and packet loss, and various types of pre and postprocessing filtering. On the other hand, general-purpose NR model is still at its immature stage. Reduced Reference (RR) model is another alternative method, the concept of which is basically proposed in [18]. In this manner, an image is transmitted to the receiver side via a transmission channel, which may introduce distortions to the received image. Meanwhile, RR features extracted from the original image at the transmitter side, which usually have much lower data rate than the image data, are sent to the receiver side through an ancillary channel, and RR features extracted from the distorted image at the receiver side are then compared with the RR features of the original image to yield a quality score for the distorted image. Benefitting from some known features of the original images, RR models usually can achieve more accurate assessment than the NR models. On the other hand, RR image quality assessment (IQA) methods require only a reduced amount of information from the original images. This makes them more useful in multimedia communications over wired or wireless networks compared with Full Reference (FR) models, where the full information of the original images is often not available. For example, Atzori et al. proposed a source rate control scheme for streaming video sequences over wireless channels by resorting on a reduced-reference quality estimation approach [19]. Over the years, some scholars contributed significant research in the design of RR mono-scopic image quality assessment methods [20–27]. The researches on RR monoscopic video quality assessment have also been made some progress. Soundararajan et al. presented a family of reduced reference video quality assessment models that utilize spatial and temporal entropic differences for video quality assessment [28]. A Gaussian scale mixture model for the wavelet coefficients of frames and frame differences is used to measure the amount of spatial and temporal information differences between the reference and distorted videos, respectively. Recently, the reduced-reference stereoscopic image quality assessment (RR-SIQA) also has been made some progresses [29–31]. Maalorf et al. proposed a RR-SIQA for color stereoscopic images by studying the relation between
the disparity maps of the original and the distorted stereoscopic images and comparing the sensitivity coefficients of the stereoscopic images [29]. Hewage et al. proposed a RR-SIQA for color plus depth based 3D video transmission [30,31]. As a well known information hiding technologies, watermarking are proposed for applications such as copyright protection, certification in the past dozen years [32–35]. But nowadays, fragile watermarks are also adopted to measure distortion degree of an image, thus some watermarking based monoscopic IQA methods have become an active research topic [36–38]. Wang et al. proposed a digital watermarking based mono-scopic IQA method in which a watermark is embedded into the discrete wavelet transform (DWT) domain of the original image using a quantization method [36]. Similarly, watermarking based method is also used to assess the quality of video [37]. Bhattacharya et al. proposed a novel approach which makes use of both fragile and robust watermarking techniques. The embedded fragile watermark is used to assess the degradation undergone by the transmitted images, while robust image features are used to construct the reference watermark from the received image, for assessing the amount of degradation of the fragile watermark [38]. However, since a watermark signal is inserted into the host image, visual quality degradation is introduced. By contrast, zero-watermark techniques extract some characteristics from the host image and use them for watermark detection, instead of watermark embedding. Thus, the distortion to the host image due to watermark embedding is eliminated [39]. In this paper, a new RR-SIQA method using view and disparity zero-watermarks is proposed. Since stereoscopic image degradation deforms image structure and descends quality of stereoscopic perception, zero-watermarks are constructed according to characteristics of image structure and stereoscopic perception, so that the recovering rates of the detected zero-watermarks can be used to evaluate the quality of stereoscopic image. 2. The proposed RR-SIQA method In this paper, we evaluate quality of stereoscopic image from the viewpoint of image quality and stereoscopic perception quality. Fig. 1. shows diagram of the proposed SIQA method. View zero-watermark is constructed by judging the relation of horizontal and vertical components of gradient vectors, because the variation of gradient can reflect the change in image structure. Disparity zerowatermark, which is constructed with disparities between the left and right views of stereoscopic image, is used to describe the stereoscopic perception quality of stereoscopic image. Then, RR-SIQA score is obtained by pooling the zero-watermark recovering rates of the view zerowatermarks and the disparity zero-watermark to assess the quality of stereoscopic image. 2.1. Construction and detection of view zero-watermark Image degradation deforms the image structure, thus analysis of structural distortions of image is beneficial to IQA. Gradient vector, as a good representation of visual quality variations in sharpness, can be used for gauging structural changes in the left and right views of stereoscopic image.
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
169
Fig. 1. Diagram of the proposed RR-SIQA method by using gradient vector and disparity based zero-watermarks.
In this paper, Sobel operator is used as gradient filters due to its effectiveness and easy to be realized. Let Il ¼{L(i, j)} and Ir ¼{R(i, j)} be the left and right views of a stereoscopic image, where 1rirM, 1rjrN, M and N are the width and height of the stereoscopic image. Here, we take the left view as an example to illustrate the generation and detection of view zero-watermark. The left view zero-watermark is constructed as follows:
The constructed view zero-watermark is sensitive in flat regions but less sensitive in highly structured regions. This feature is consistent with the human perception, since HVS is less sensitive to details in the areas with high amount of texture activities [40].
(1) Calculate the horizontal and vertical components of gradient vector Gl (i, j) ¼{gh,l (i, j), gv,l (i, j)} of each pixel in the left view of the stereoscopic image
Stereoscopic image is captured using two cameras located in slightly shifted positions. The displacement between the left and right image coordinates is known as the disparity. Because the depth is perceived through retinal disparities, visual sensitivity to depth can be evaluated in terms of the variation in disparity. For a distorted stereoscopic image, the more similar between the disparities of original and distorted stereoscopic images, the better the quality of its stereoscopic perception is. Therefore, disparity has a significant impact on stereoscopic image quality assessment [41]. Literature [41] used a belief propagation based method to estimate the disparity map, so as to assess perceptual stereoscopic quality of stereoscopic image. Literature [42] calculated a 3D depth map of the right eye image, by using a depth estimation algorithm Depth Estimation Reference Software (DERS), to include 3D depth information in perceptual quality metric for stereoscopic crosstalk perception. In this paper, disparity is also used to construct zero-watermark so as to measure the stereoscopic perception. The disparity zero-watermark is constructed as follows:
g h;l ði; jÞ ¼ Lði 1; j 1Þ þ 2Lði; j 1Þ þLðiþ 1; j 1Þ Lði 1; j þ 1Þ 2Lði; j þ1Þ Lði þ1; j þ1Þ
ð1Þ
g v;l ði; jÞ ¼ Lði 1; j 1Þ þ2Lði 1; jÞ þ Lði 1; j þ1Þ Lði þ 1; j 1Þ 2Lði þ1; jÞ Lði þ1; j þ1Þ
ð2Þ
(2) The left view zero-watermark W g;l ¼ wg;l ði; jÞ with total M N bits is constructed by judging the relation of absolute values of the horizontal and vertical components of the gradient vectors, and the left view zero-watermark at the point (i, j) named wg,l(i, j) is constructed by ( wg;l ði; jÞ ¼
1; 0;
if g v;l ði; jÞ 4 g h;l ði; jÞ otherwise
ð3Þ
The detection of view zero-watermark is the same as that of the construction, except that the left view zerowatermark W′g;l ¼ w′g;l ði; jÞ is detected from the left view of distorted stereoscopic image, instead of the left view of the original stereoscopic image. The construction and detection of right view zerowatermark are similar with the construction and detection of left view zero-watermark, and the constructed and detected right view zero-watermarks with respect to the original and distorted stereoscopic images are denoted as W g;r ¼ wg;r ði; jÞ and W′g;r ¼ w′g;r ði; jÞ hereinafter.
2.2. Construction and detection of disparity zero-watermark
(1) Let (x, y) be the coordinate of the left-bottom corner of the current block to be estimated in the left view, dv be the candidate of the disparity vector. The cost function E(x, y, dv) for disparity estimation can be written as Eðx; y; dv Þ ¼ Edata ðx; y; dv Þ þ λEsmooth ðx; y; dv Þ
ð4Þ
∑ Lðx þp; y þqÞ
n1 n1
Edata ðx; y; dv Þ ¼ ∑
p¼0q¼0
Rðx þ p þ dv ; y þqÞ
ð5Þ
170
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176 3 Esmooth ðx; y; dv Þ ¼ ∑ dv dk k¼0
ð6Þ
The data item Edata is the displaced frame difference between the current block in the left view and its possible corresponding block in the right view, while the smoothness item Esmooth utilizes disparity vectors of the neighboring blocks to enhance smoothness of disparity map. The d0–d3 in Eq. (6) are disparity vectors of the left-top, top, right-top and left neighboring blocks of the current block. Factor λ is a constant which controls the influence of the smoothness item on the cost function. The dv which achieves the minimal cost E within the search window of disparity estimation will be regarded as the optimal disparity vector of the current block. For the occluded blocks in the left view of stereoscopic image which have no corresponding blocks in the right view, their disparity vectors are simply set to 0. The obtained disparity map of the stereoscopic image is denoted as D¼{d(s, t)}, 1rs rM/n, 1 rtrN/n. (2) Calculate the average disparity of the disparity map D S¼
n2 M=n N=n ∑ ∑ dðs; tÞ MN s ¼ 1 t ¼ 1
ð7Þ
MN (3) Disparity zero-watermark W d ¼ wd ðs; tÞ with total nn bits is constructed by judging the relation of d(s, t) and S, and wd(s, t) is constructed by ( 1; if dðs; tÞ 4S wd ðs; tÞ ¼ ð8Þ 0; otherwise
recovering rates of the detected zero-watermarks are used to assess the quality of the distorted stereoscopic image. The quality of the left and right views of a stereoscopic image is assessed by gauging the change of their gradient vectors in terms of the original ones because gradient reflects the structure of an image. The view zero-watermark recovering rate, HCview, is calculated by HC view ¼
HC view;l þHC view;r 2
ð9Þ
HC view;l ¼ 1
1 M N ∑ ∑ w ði; jÞ w′g;l ði; jÞ MN i ¼ 1 j ¼ 1 g;l
ð10Þ
HC view;r ¼ 1
1 M N ∑ ∑ wg;r ði; jÞ w′g;r ði; jÞ MN i ¼ 1 j ¼ 1
ð11Þ
where denotes the exclusive-OR. It is well known that the stereoscopic perception can be artificially induced by presenting two different images of the same scene, shifting one with respect to the other, thus mimicking two different views, namely the left and right views of stereoscopic image, to the left and right eyes. The difference in the views generates disparity in the stereoscopic image. When stereoscopic image is degraded, distortions may lead to modifications to disparity between left and right views. Therefore, it is useful to introduce this modification to assess the quality of stereoscopic perception. Since the constructed disparity zero-watermark reflects characteristic of disparity to some extent, the recovering rate of detected disparity zero-watermark is used to evaluate quality of stereoscopic perception, and it is defined by HC disp ¼ 1
n2 M=n N=n ∑ ∑ ½w ðs; tÞ w′d ðs; tÞ MN s ¼ 1 t ¼ 1 d
ð12Þ
The detection of disparity zero-watermark is the same as that of its construction, except that the disparity zerowatermark W′d ¼ w′d ðs; tÞ is detected from disparity map of the distorted stereoscopic image, instead of disparity map of the original stereoscopic image, and the threshold S is used as the same with the construction, that is to say, S is received from the sender. The designed disparity zero-watermark is more sensitive to disparity distortion at the middle level of the scene due to the use of average disparity as a threshold. If an object is far from the viewer, it is usually less interested by the viewer, thus its distortion may not be noticed by the viewer. But if an object is very close to the viewer, its disparity will be very large. Thus, if the distortion of its disparity is not so great compared with the original value, that is, the proportion of the change is not big, the distortion of the object may be less noticeable by the viewer.
The recovery ratio of the view zero-watermark reflects the quality of the left and right views of a stereoscopic image, while the recovery ratio of the disparity zero-watermark reflects the stereoscopic perception quality of the stereoscopic image. Thus, a nonlinear function is used to pool the two evaluation results into a score as RR-SIQA value of the stereoscopic image. The stereoscopic image quality score, HCs, is defined by
2.3. Zero-watermark based quality assessment for stereoscopic image
To test the performance of the proposed RR-SIQA method, some experiments are implemented. The VCIP database used in the experiments is introduced in [8], which is composed of totally 370 distorted stereoscopic images generated with 10 original high-resolution color stereoscopic images, as shown in Fig. 2. The distortions include JPEG2000 (100 stereoscopic images), JPEG
Because the constructed zero-watermarks are associated with some inherent characteristics of stereoscopic image, the integrality of zero-watermark varies as the quality of the distorted stereoscopic image. Thus, the
HC s ¼ aðHC view Þb þð1 aÞðHC disp Þ2 b
ð13Þ
where the parameters a and b are determined by training. In the proposed RR-SIQA method, the two view zerowatermarks and the disparity zero-watermark will be transmitted to the receiver. Since all of them are binary maps, they can be losslessly compressed efficiently so as to reduce the required side channel bandwidth. 3. Experimental results and analysis
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
(70 stereoscopic images), Gaussian white noise (100 stereoscopic images) and Gaussian blur (100 stereoscopic images) with different distortion degrees. The database also provides subjective evaluation results, e.g., differential mean opinion score (DMOS), for evaluating the consistency of objective SIQA methods against the human perception. To assess these color stereoscopic images objectively, their luminance component is used to construct the zerowatermarks. Parameters a and b in Eq. (13) are determined by training with a subset of the database including all 111
171
distorted stereoscopic images of “Bowling1”, “Computer” and “Dolls”. We employed the training algorithm to find the optimum parameters in Eq. (13) with respect to the highest correlation between HCs and DMOS, and finally they are set as a¼0.5004 and b¼1.1668. Other parameters used for disparity estimation are set as λ¼ 0.25, and 38 for the searching window. To balance contradiction between overhead and disparity accuracy, n¼8 was used. The evaluation is performed by comparing stereoscopic image quality score HCs with the subjective DMOS of the
Fig. 2. Left views of the original stereoscopic images [8]. (a)Art, (b) Bowling1, (c) Computer, (d) Dolls, (e) Drumsticks, (f) Dwarves, (g) Laundry, (h) Mobius, (i) Reindeer, and (j) Rocks1.
Fig. 3. Scatter plots of DMOS vs. HCs obtained with the proposed method. (a) JPEG2000 compression, (b) JPEG compression, (c) Gaussian white noise, and (d) Gaussian blur.
172
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
stereoscopic images. The linear correlation coefficient (CC), the rank order CC (ROCC) and the root mean-squared (RMS) error between DMOS and objective score after nonlinear regression are utilized to test the consistency of the proposed RR-SIQA method and the subjective perception. CC expresses the accuracy of an objective method, ROCC indicates the monotonic property of the objective method, while RMS error expresses the validity of the objective method. The higher the CC and ROCC between an image quality assessment score and DMOS value is, the better the assessment method is. The smaller RMS error is, the better the method performs. The nonlinearity chosen for regression is a five-parameter logistic function [43] written as DMOSp ¼ β1 logisticðβ2 ; ðx β3 ÞÞ þβ4 x þ β5
ð14Þ
logisticðτ; xÞ ¼
1 1 2 1 þexpðτxÞ
ð15Þ
where β1, β2, β3, β4, and β5 are determined by using the subjective scores and the objective scores. Fig. 3 presents scatter plots of DMOS values vs. HCs, with respect to the four different types of distortions. The curves in the figure are obtained with the logistic function. If the points in scatter plot are close to the fitted curve, it means that the SIQA method coincides with the DMOS. The scatter plots of the metric in [29] are also given in Fig. 4. Table 1 shows the CC, ROCC and RMS error of the proposed RR-SIQA method with respect to the four different types of distortions as well as the results of the metric in [29]. It is seen that the proposed method performs quite well for wide range of distortion types. For stereoscopic images distorted by the
Fig. 4. Scatter plots of DMOS vs. scores obtained with the metric in [29]. (a) JPEG2000 compression, (b) JPEG compression, (c) Gaussian white noise, and (d) Gaussian blur.
Table 1 Performance of the proposed objective method compared with metric in [29]. Proposed method
CC ROCC RMS error
Metric in [29]
JPEG2000
JPEG
Gaussian white noise
Gaussian blur
JPEG2000
JPEG
Gaussian white noise
Gaussian blur
0.961 0.765 4.35
0.950 0.884 3.88
0.962 0.950 3.55
0.947 0.887 1.85
0.955 0.746 5.11
0.947 0.833 4.71
0.954 0.941 3.56
0.940 0.853 2.09
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
173
Fig. 5. HCs vs. the degree of distortion with respect to “Art”. (a) JPEG2000 compression, (b) JPEG compression, (c) Gaussian white noise, and (d) Gaussian blur.
Table 2 Comparison of computation time using database [8]. Model
Wang et al. [20]
Li et al. [22]
Zhang et al. [24]
Ma et al. [27]
Proposed RR-SIQA
Time (s/image)
14.66
20.89
8.04
41.41
38.15
four types of distortions, the proposed RR-SIQA method provides better prediction accuracy (larger CC), stricter monotonicity (larger ROCC) and smaller prediction error (smaller RMS error). Therefore, the proposed method is a reasonable and useful choice in practical RR-SIQA systems. Fig. 5 shows HCs versus the degree of distortion of the “Art” stereoscopic image with respect to the four different types of distortions. The horizontal axes represent the degree of distortion under different types of distortions. It is noticed that the curves of HCs versus the degree of distortion are monotonous. Moreover, for stereoscopic images distorted with Gaussian white noise and Gaussian blur, the curves tend to be more consistent with its tangents, that is, HCs is more close to a constant as the degree of distortion grows. While for stereoscopic images distorted with JPEG2000 and JPEG, HCs drops quickly at the points with high compression rate. This feature is well consistent with HVS. We also compare the computational complexity of the proposed RR-SIQA method with other four RR-IQA methods
without any optimization of the code. The results are reported in Table 2, where we list the average time taken for each image, over all stereoscopic images in the database [8], using a computer with Intel i3 processor at 3.10 GHz. It can be seen that the proposed method takes slightly more time than most of the other methods under comparison, mainly due to computation of disparity, but it is comparable to metric in [27]. Nowadays, many stereoscopic images have been usually compressed to JPEG or JPEG2000 format to reduce the bandwidth and storage requirements. As a result, the objective quality assessment for compressed stereoscopic images is necessary in various applications. PSNR (FR method) is often used to evaluate the quality of a reconstructed stereoscopic image in term of the original one. However, it is noticed that the consistency between PSNR and subjective perception is not satisfied in some situations, for example, when stereoscopic images are distorted by JPEG compression, as shown in Fig. 6, which presents the scatter plots of the DMOS values versus PSNR. In Fig. 6, the PSNR
174
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
Fig. 6. Scatter plots of DMOS vs. PSNR. (a) JPEG2000 compression and (b) JPEG compression.
Table 3 Performance of the proposed objective method compared with PSNR.
CC ROCC RMS error
Proposed method
PSNR
JPEG2000
JPEG
JPEG2000
JPEG
0.961 0.765 4.35
0.950 0.884 3.88
0.957 0.761 4.01
0.915 0.848 5.10
values are the average PSNR of the left and right views of a stereoscopic image. 2D metrics are considered to fail for 3D images [44]. Literature [45] also gives some examples that images with same MSE have quite different subjective perception. Moreover, in wireless multimedia systems, the original stereoscopic image is often not available. Clearly, the RR method is useful in such situations. The comparison results on CC, ROCC and RMS error of the proposed RR-SIQA method and PSNR metric are shown in Table 3. It is noticed that the proposed method achieves better performance than PSNR metric in terms of better prediction accuracy (higher CC), stricter monotonicity (higher ROCC) and similar prediction error (lower RMS error) when assessing quality of stereoscopic image compressed with JPEG2000 and JPEG. We also implement the proposed SIQA method and the metric in [29] on the LIVE 3D Image Quality Database [46], which contains five types of distortions across 365 distorted stereoscopic images. The parameters a, b, λ, n, and the searching window of the proposed SIQA are the same with that used for VCIP database. Table 4 provides the CC, SROCC and RMS error with respect to the two metrics for each type of distortion on the LIVE 3D Image Quality Database. It is clear that the proposed method also outperforms the metric in [29] for this database. Additionally, to assess the statistical significance of the performance difference between two SIQA methods, an F-test based on the errors between the DMOS scores and the method predictions (obtained from the non-linear regression process that was used to compute the linear correlation coefficient on the entire dataset) is made, which could show the statistically superior objective model between two SIQA objective methods. The detail
of the statistical evaluation can be found in [43,47]. The residual errors between the objective method prediction and the DMOS value are defined as Ei ¼ P i DMOSi ;
i ¼ 1; 2; …; N
ð16Þ
where P is the objective method prediction and N is the number of experimental stereoscopic images. An F-test is performed on the ratio of the variance of the residual error from one objective to that of another objective method at 95% significance level. The F-ratio is always formed by placing the objective method with larger residual error variance in the numerator. An F-ratio ratio larger than a threshold indicates that the performance of the SIQA method in the numerator of the F-ratio is statistically inferior to that of the SIQA method in the denominator. The results of the statistical significance test are listed in Table 5. Each entry in the table includes two characters which correspond to the two publicly databases in the order of {VCIP Database and the LIVE 3D Image Quality Database}, The symbol “-” denotes that the two SIQA methods are statistically indistinguishable, “1” denotes that the SIQA method of the row is statistically better than that of the column, and “0” denotes that the SIQA method of the column is better than that of the row. It is seen that the proposed method outperforms the metric in [29] and PSNR. 4. Conclusions In this paper, a reduced-reference stereoscopic image quality assessment (RR-SIQA) method is proposed based on zero-watermarks. Two kinds of zero-watermarks, that is, view zero-watermark and disparity zero-watermark, are constructed to reflect the image structure and stereoscopic perception. Then, objective quality score of stereoscopic image are obtained by pooling the zero-watermark recovering rates of the view zero-watermarks and the disparity zero-watermark. Experimental results show that the stereoscopic image quality evaluation results assessed with the proposed objective method are well consistent with subjective assessment, and the proposed RR-SIQA method achieves better performance than the widely used FF-SIQA method PSNR in assessing quality of stereoscopic image compressed with JPEG2000 and JPEG. The proposed
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
175
Table 4 Performance of the proposed objective method compared with metric in [29] on the LIVE 3D Image Quality Database. Proposed method
CC ROCC RMS error
Metric in [29]
JPEG2000
JPEG
WN
blur
FF
JPEG2000
JPEG
WN
Blur
FF
0.907 0.906 5.08
0.697 0.702 3.72
0.926 0.929 6.01
0.920 0.889 4.71
0.639 0.547 8.67
0.904 0.856 5.53
0.531 0.550 5.54
0.896 0.896 7.41
0.798 0.690 8.75
0.670 0.545 9.23
Table 5 Statistical significance matrix base on SIQA-DMOS residuals. Model
PSNR
Metric in [29]
Proposed method
PSNR Metric in [29] Proposed method
–– 0– 11
1– –– 11
00 00 ––
method for automatic SIQA can be used to monitor stereoscopic image quality for three dimensional image/ video broadcasting and multimedia systems. Future works will focus on the features of HVS especially stereoscopic visual masking and extension to other more effective SIQA methods utilizing different filters as well as reducedreference stereoscopic video quality assessment methods.
Acknowledgments This research is supported by the Natural Science Foundation of China (Grant nos. 61071120, 61171163, 6127120, 61271021, and 61111140392), the Innovative Scientific Research Project for Postgraduates in Zhejiang Province (Grant no. YK2011049), the Research Foundation of Education Department of Zhejiang Province (Grant no. Y201224839) and the Outstanding (Postgraduate) Dissertation Growth Foundation of Ningbo University (Grant no. PY20110002). It was also sponsored by the K.C. Wong Magna Fund in Ningbo University. References [1] Pedro F. Felzenszwalb, R. Zabih, Dynamic programming and graph algorithms in computer vision, IEEE Trans. Pattern Anal. Mach. Intell. 33 (4) (2011) 721–740. [2] W. Wang, D. Peng, H. Wang, et al., A multimedia quality-driven network resource management architecture for wireless sensor networks with stream authentication, IEEE Trans. Multimedia 12 (5) (2010) 439–447. [3] M. Tanimoto, FTV: free-viewpoint television, Signal Process.: Image Commun. 27 (2012). [4] Y. Fan, Y. Kung, B. Lin, Three-dimensional auto-stereoscopic image recording, mapping and synthesis system for multiview 3D display, IEEE Trans. Magn. 47 (3) (2011) 683–686. [5] F. Shao, G. Jiang, X. Wang, M. Yu, K. Chen, Stereoscopic video coding with asymmetric luminance and chrominance qualities, IEEE Trans. Consum. Electron. 56 (4) (2010) 2460–2468. [6] M. Fiedler, T. Hossfeld, T.G. Phuoc, A generic quantitative relationship between quality of experience and quality of service, IEEE Network 24 (2) (2010) 36–41. [7] A.K. Moorthy, A.C. Bovik, Visual quality assessment algorithms: what does the future hold? Multimedia Tools Appl. 51 (2) (2010) 675–696. [8] X. Wang, M. Yu, Y. Yang, G. Jiang, Research on subjective stereoscopic image quality assessment, in: Proceedings of SPIE-IS&T Electronic Imaging, San Jose, USA, 2009, pp. 1–10.
[9] L. Jiang, J. He, N. Zhang, T. Huang, An overview of 3D video representation and coding, 3D Res. 1 (1) (2010) 43–47. [10] C. Jung, L. Jiao, Y. Oh, J. Kim, Depth-preserving DIBR based on disparity map over T-DMB, Electron. Lett. 46 (9) (2010) 628–629. [11] T.R. Coffman, A.C. Bovik, Efficient stereoscopic ranging via stochastic sampling of match quality, IEEE Trans. Image Process. 19 (2) (2010) 451–460. [12] D. Kim, K. Sohn, Visual fatigue prediction for stereoscopic image, IEEE Trans. Circuits Syst. Video Technol. 21 (2) (2011) 231–236. [13] A. Benoit, P. Callet, P. Campisi, R. Cousseau, Quality assessment of stereoscopic image, EURASIP J. Image Video Process. (2008) 1–13. [14] J. Yang, C. Hou, R. Xu, J. Lei, New metric for stereo image quality assessment based on HVS, Int. J. Imaging Syst. Technol. 20 (4) (2010) 301–307. [15] M. Lambooij, W. Iisselsteijn, D.G. Bouwhuis, I. Heynderickx, Evaluation of stereoscopic image: beyond 2D quality, IEEE Trans. Broadcast. 57 (2) (2011) 432–444. [16] R. Akhter, Z. Parvez Sazzad, Y. Horita, J. Baltes, No-reference stereoscopic image quality assessment, in: Proceedings of SPIE-IS&T Electronic Imaging SPIE, San Jose, USA, vol. 7524, 2010. [17] Z.M.P. Sazzad, S. Yamanaka, Y. Horita, Spatio-temporal segmentation based continuous no-reference stereoscopic video quality prediction, in: Proceedings of the Second International Workshop on Quality of Multimedia Experience (QoMEX), Trondheim, June 21–23, 2010, pp. 106–111. [18] M. Pinson, S. Wolf, A new standardized method for objectively measuring video quality, IEEE Trans. Broadcast. 50 (3) (2004) 312–322. [19] Luigi Atzori, Alessandro Floris, Giaime Ginesu, Daniele Giusto, Streaming video over wireless channels: exploiting reducedreference quality estimation at the user-side, Signal Process.: Image Commun. 27 (2012) 1049–1065. [20] Z. Wang and, E.P. Simonceli, Reduced-reference image quality assessment using a wavelet domain natural image statistic mode1, in: Proceedings of the Human Vision and Electronic Imaging X, San Jose, CA, vol. 5666(1), 2005, pp. 149–159. [21] Z. Wang, G. Wu, H.R. Sheikh, E.P. Simoncelli, E.-H. Yang, A.C. Bovik, Quality-aware images, IEEE Trans. Image Process. 15 (6) (2006) 1680–1689. [22] Q. Li, Z. Wang, Reduced-reference image quality assessment using divisive normalization-based image representation, IEEE Journal of Selected Topics in Signal Processing 3 (2) (2009) 202–211. (Special issue on Visual Media Quality Assessment). [23] X. Gao, W. Lu, D. Tao, X. Li, Image quality assessment based on multiscale geometric analysis, IEEE Trans. Image Process. 18 (7) (2009) 1409–1423. [24] M. Zhang, W. Xue, X. Mou, Reduced reference image quality assessment based on statistics of edge, in: Proceedings of the SPIE, vol. 7876, 2011, pp. 787611-1–787611-7. [25] J.A. Redi, P. Gastaldo, I. Heynderickx, R. Zunino, Color distribution information for reduced-reference assessment of perceived image quality, IEEE Trans. Circuits Syst. Video Technol. 20 (12) (2010) 1757–1769. [26] Z. Wang, A.C. Bovik, Reduced- and no-reference image quality assessment: the natural scene statistic model approach, IEEE Signal Process. Mag. 28 (2011) 29–40. [27] L. Ma, et al., Reduced-reference image quality assessment in reorganized DCT domain, Signal Process.-Image Commun. (2012). [28] Rajiv Soundararajan, Alan C. Bovik, Video quality assessment by reduced reference spatio-temporal entropic differencing, IEEE Trans. Circuits Syst. Video Technol. 23 (4) (2013) 684–694. [29] A. Maalouf, M.C. Larabi, CYCLOP: a stereo color image quality assessment metric, in: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Prague, May 22–27, 2011, pp. 1520–6149.
176
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
[30] C.T.E.R. Hewage, M.G. Martini, Reduced-reference quality assessment for 3D video compression and transmission, IEEE Trans. Consum. Electron. 57 (3) (2011) 1185–1193. [31] C.T.E.R. Hewage, M.G. Martini, Reduced-reference quality metric for 3D depth map transmission, 3DTV-conference: the true visioncapture, Transmission and Display of 3D Video (3DTV-CON), Tampere, June 7–9, 2010, pp. 1–4. [32] J. Cox, J. Kilian, F.T. Leighton, T. Shamoon, Secure spread spectrum watermarking for multimedia, IEEE Trans. Image Process. 6 (12) (1997) 1673–1687. [33] L.X. Luo, Z.Y. Chen, M. Chen, X. Zeng, Reversible image watermarking using interpolation technique, IEEE Trans. Inf. Forensics Secur. 5 (1) (2010) 187–193. [34] T.Y. Lee, S. Lin, Dual watermark for image tamper detection and recovery, Pattern Recognition 41 (2008) 3497–3506. [35] C.L. Li, Y.H. Wang, B. Ma, Z.X. Zhang, Tamper detection and selfrecovery of biometric images using salient region-based authentication watermarking scheme, Comput. Stand. Interfaces 34 (4) (2012) 367–379. [36] S. Wang, D. Zhang, J. Zhao, W.J. Tam, F. Speranza, An image quality evaluation method based on digital watermarking, IEEE Trans. Circuits Syst. Video Technol. 171 (2007) 98–105. [37] M.S. El-Mahallawy, A. Hashad, H.H. Ali, H.S. Zaky, Quality estimation of video transmitted over an additive WGN channel based on digital watermarking and wavelet transform, World Acad. Sci. Eng. Technol. 57 (2009) 419–423. [38] A. Bhattachrya, S. Palit, N. Chatterjee, G. Roy, Blind assessment of image quality employing fragile watermarking, in: Proceedings of the 7th International Symposium on Image and Signal Processing and Analysis (ISPA), 2011, pp. 431–436.
[39] J. Sang, X.F. Liao, M.S. Alam, Neural-network-based zerowatermark scheme for digital images, Opt. Eng. 45 (9) (2006) 0970061–0970069. [40] K. Minoo, T.Q. Nguyen, Perceptual Video Coding with H.264, in: Proceedings of the 39th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, USA, October 2005, pp. 741–745. [41] J. You, L. Xing, A. Perkis, X. Wang, Perceptual quality assessment for stereoscopic images based on 2D image quality metrics and disparity analysis, in: Proceedings of the 5th International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Scottsdale, AZ, USA, January 2010. [42] L. Xing, J. You, T. Ebrahimi, A. Perkis, A perceptual quality metric for stereoscopic crosstalk perception, in: Proceedings of IEEE International Conference on Image Processing (ICIP), Hong Kong, China, September 2010, pp. 4033–4036. [43] H. Sheikh, M. Sabir, A. Bovik, A statistical evaluation of recent full reference image quality assessment algorithms, in: Proceedings of IEEE Transactions on Image Processing, vol. 15(11), 2006, pp. 3441– 3452. [44] Atanas Boev, Maija Poikela, Atanas Gotchev, Anil Aksay, Modelling of the stereoscopic HVS, Report on Mobile 3DTV 〈http://sp.cs.tut.fi/ mobile3dtv/results/〉. [45] Z. Wang, A.C. Bovik, Mean squared error: love it or leave it? A new look at signal fidelity measures, IEEE Signal Process. Mag. 26 (1) (2009) 98–117. [46] A.K. Moorthy, C. Su, A. Mittal, A.C. Bovik, Subjective evaluation of stereoscopic image quality, Signal Process.: Image Commun. 28 (8) (2013) 870–883. [47] X. Fei, L. Xiao, Y. Sun, Z. Wei, Perceptual image quality assessment based on structural similarity and visual masking, Signal Process.: Image Commun. 27 (7) (2012) 772–783.
Contents lists available at ScienceDirect
Signal Processing: Image Communication journal homepage: www.elsevier.com/locate/image
Reduced-reference stereoscopic image quality assessment based on view and disparity zero-watermarks Wujie Zhou a,b, Gangyi Jiang a,c,n, Mei Yu a,c, Feng Shao a, Zongju Peng a a b c
Faculty of Information Science and Engineering, Ningbo University, Ningbo 315211, China School of Information and Electronic Engineering, Zhejiang University of Science &Technology, Hangzhou 310023, China National Key Lab of Software New Technology, Nanjing University, Nanjing 210093, China
a r t i c l e i n f o
abstract
Article history: Received 22 May 2012 Received in revised form 21 October 2013 Accepted 23 October 2013 Available online 6 November 2013
As a practical and novel application of watermarking, this paper presents a zerowatermarking based objective reduced-reference stereoscopic image quality assessment (RR-SIQA) method. In the proposed method, two kinds of zero-watermarks are constructed according to the characteristics of image structure and stereoscopic perception. Concretely, two view zero-watermarks, which are constructed by judging the relation of the horizontal and vertical components of gradient vectors with respect to the two views, are used to reflect the image structure variation of the stereoscopic image. Meanwhile, a disparity zero-watermark, which is constructed with disparity map of the stereoscopic image, is used to reflect the stereoscopic perception quality variation. Then, the quality of stereoscopic image is objectively assessed by pooling the recovering rates of the detected zero-watermarks. The experimental results show that the stereoscopic image quality evaluation results assessed with the proposed RR-SIQA method are well consistent with subjective assessment, and the proposed method achieves better performance than the widely used full-reference stereoscopic image quality assessment method PSNR in assessing quality of stereoscopic images compressed with JPEG and JPEG2000. & 2013 Elsevier B.V. All rights reserved.
Keywords: Three dimensional TV Stereoscopic image processing Image quality Reduced-reference Zero-watermark
1. Introduction With the technological advancement made in computer graphics, computer vision and network communication [1,2], stereoscopic image processing is becoming increasingly popular and would possibly have many applications [3]. In these applications, stereoscopic images may have gone through various processes, including compression, communication, printing, display, restoration, segmentation, and fusion [4], any of which can introduce distortions that may impair visual qualities of the images. On the other hand, with the evolution towards new multimedia n Corresponding author. Tel.: þ86 574 87600411; fax: þ86 574 87600940. E-mail addresses: [email protected] (W. Zhou), [email protected] (G. Jiang).
0923-5965/$ - see front matter & 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.image.2013.10.005
communication systems and services, users now expect services to be delivered according to their demands on quality [5]. Recently, the concept of quality of service (QoS) has been extended to the new concept of quality of experience (QoE), combining user perception, experience, and expectations with non-technical and technical parameters such as application- and network-level QoS [6]. However, for wireless systems the possible limitations due to the features of the devices and of the transmission channel may result in perceivable impairments that influence the user's perception on images. Therefore, stereoscopic image quality assessment (SIQA) is necessary since it enables to adjust the parameters of stereoscopic image processing so as to optimize stereoscopic image quality or make it to reach a given quality [7]. SIQA methods can be classified as subjective and objective methods. Subjective SIQA method is closer to
168
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
human visual perception [8]. However, it is time consuming and expensive, and cannot be done in real time. By contrast, the objective SIQA method needs less time and is convenient to be implemented. Simple fidelity methods such as mean square error (MSE) or peak signal-to-noise ratio (PSNR) are often used to evaluate the quality of a processed stereoscopic image in the light of the original ones [9–11]. These methods belong to the so-called fullreference stereoscopic image quality assessment (FR-SIQA) methods, which require full information of the original stereoscopic image [12–15]. However, since the end-user usually cannot obtain the original image through the wired or wireless network, it is difficult to implement fullreference image quality assessment in real applications. The opposite is the No Reference (NR) model which does not need any information about the original image, but these kind of methods are useful only when the types of distortions that the stereoscopic images suffered by are fixed and known [16,17]. Thus, they may be limited in modern wired or wireless network, where the distortions could be a combination of lossy compression, network delay and packet loss, and various types of pre and postprocessing filtering. On the other hand, general-purpose NR model is still at its immature stage. Reduced Reference (RR) model is another alternative method, the concept of which is basically proposed in [18]. In this manner, an image is transmitted to the receiver side via a transmission channel, which may introduce distortions to the received image. Meanwhile, RR features extracted from the original image at the transmitter side, which usually have much lower data rate than the image data, are sent to the receiver side through an ancillary channel, and RR features extracted from the distorted image at the receiver side are then compared with the RR features of the original image to yield a quality score for the distorted image. Benefitting from some known features of the original images, RR models usually can achieve more accurate assessment than the NR models. On the other hand, RR image quality assessment (IQA) methods require only a reduced amount of information from the original images. This makes them more useful in multimedia communications over wired or wireless networks compared with Full Reference (FR) models, where the full information of the original images is often not available. For example, Atzori et al. proposed a source rate control scheme for streaming video sequences over wireless channels by resorting on a reduced-reference quality estimation approach [19]. Over the years, some scholars contributed significant research in the design of RR mono-scopic image quality assessment methods [20–27]. The researches on RR monoscopic video quality assessment have also been made some progress. Soundararajan et al. presented a family of reduced reference video quality assessment models that utilize spatial and temporal entropic differences for video quality assessment [28]. A Gaussian scale mixture model for the wavelet coefficients of frames and frame differences is used to measure the amount of spatial and temporal information differences between the reference and distorted videos, respectively. Recently, the reduced-reference stereoscopic image quality assessment (RR-SIQA) also has been made some progresses [29–31]. Maalorf et al. proposed a RR-SIQA for color stereoscopic images by studying the relation between
the disparity maps of the original and the distorted stereoscopic images and comparing the sensitivity coefficients of the stereoscopic images [29]. Hewage et al. proposed a RR-SIQA for color plus depth based 3D video transmission [30,31]. As a well known information hiding technologies, watermarking are proposed for applications such as copyright protection, certification in the past dozen years [32–35]. But nowadays, fragile watermarks are also adopted to measure distortion degree of an image, thus some watermarking based monoscopic IQA methods have become an active research topic [36–38]. Wang et al. proposed a digital watermarking based mono-scopic IQA method in which a watermark is embedded into the discrete wavelet transform (DWT) domain of the original image using a quantization method [36]. Similarly, watermarking based method is also used to assess the quality of video [37]. Bhattacharya et al. proposed a novel approach which makes use of both fragile and robust watermarking techniques. The embedded fragile watermark is used to assess the degradation undergone by the transmitted images, while robust image features are used to construct the reference watermark from the received image, for assessing the amount of degradation of the fragile watermark [38]. However, since a watermark signal is inserted into the host image, visual quality degradation is introduced. By contrast, zero-watermark techniques extract some characteristics from the host image and use them for watermark detection, instead of watermark embedding. Thus, the distortion to the host image due to watermark embedding is eliminated [39]. In this paper, a new RR-SIQA method using view and disparity zero-watermarks is proposed. Since stereoscopic image degradation deforms image structure and descends quality of stereoscopic perception, zero-watermarks are constructed according to characteristics of image structure and stereoscopic perception, so that the recovering rates of the detected zero-watermarks can be used to evaluate the quality of stereoscopic image. 2. The proposed RR-SIQA method In this paper, we evaluate quality of stereoscopic image from the viewpoint of image quality and stereoscopic perception quality. Fig. 1. shows diagram of the proposed SIQA method. View zero-watermark is constructed by judging the relation of horizontal and vertical components of gradient vectors, because the variation of gradient can reflect the change in image structure. Disparity zerowatermark, which is constructed with disparities between the left and right views of stereoscopic image, is used to describe the stereoscopic perception quality of stereoscopic image. Then, RR-SIQA score is obtained by pooling the zero-watermark recovering rates of the view zerowatermarks and the disparity zero-watermark to assess the quality of stereoscopic image. 2.1. Construction and detection of view zero-watermark Image degradation deforms the image structure, thus analysis of structural distortions of image is beneficial to IQA. Gradient vector, as a good representation of visual quality variations in sharpness, can be used for gauging structural changes in the left and right views of stereoscopic image.
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
169
Fig. 1. Diagram of the proposed RR-SIQA method by using gradient vector and disparity based zero-watermarks.
In this paper, Sobel operator is used as gradient filters due to its effectiveness and easy to be realized. Let Il ¼{L(i, j)} and Ir ¼{R(i, j)} be the left and right views of a stereoscopic image, where 1rirM, 1rjrN, M and N are the width and height of the stereoscopic image. Here, we take the left view as an example to illustrate the generation and detection of view zero-watermark. The left view zero-watermark is constructed as follows:
The constructed view zero-watermark is sensitive in flat regions but less sensitive in highly structured regions. This feature is consistent with the human perception, since HVS is less sensitive to details in the areas with high amount of texture activities [40].
(1) Calculate the horizontal and vertical components of gradient vector Gl (i, j) ¼{gh,l (i, j), gv,l (i, j)} of each pixel in the left view of the stereoscopic image
Stereoscopic image is captured using two cameras located in slightly shifted positions. The displacement between the left and right image coordinates is known as the disparity. Because the depth is perceived through retinal disparities, visual sensitivity to depth can be evaluated in terms of the variation in disparity. For a distorted stereoscopic image, the more similar between the disparities of original and distorted stereoscopic images, the better the quality of its stereoscopic perception is. Therefore, disparity has a significant impact on stereoscopic image quality assessment [41]. Literature [41] used a belief propagation based method to estimate the disparity map, so as to assess perceptual stereoscopic quality of stereoscopic image. Literature [42] calculated a 3D depth map of the right eye image, by using a depth estimation algorithm Depth Estimation Reference Software (DERS), to include 3D depth information in perceptual quality metric for stereoscopic crosstalk perception. In this paper, disparity is also used to construct zero-watermark so as to measure the stereoscopic perception. The disparity zero-watermark is constructed as follows:
g h;l ði; jÞ ¼ Lði 1; j 1Þ þ 2Lði; j 1Þ þLðiþ 1; j 1Þ Lði 1; j þ 1Þ 2Lði; j þ1Þ Lði þ1; j þ1Þ
ð1Þ
g v;l ði; jÞ ¼ Lði 1; j 1Þ þ2Lði 1; jÞ þ Lði 1; j þ1Þ Lði þ 1; j 1Þ 2Lði þ1; jÞ Lði þ1; j þ1Þ
ð2Þ
(2) The left view zero-watermark W g;l ¼ wg;l ði; jÞ with total M N bits is constructed by judging the relation of absolute values of the horizontal and vertical components of the gradient vectors, and the left view zero-watermark at the point (i, j) named wg,l(i, j) is constructed by ( wg;l ði; jÞ ¼
1; 0;
if g v;l ði; jÞ 4 g h;l ði; jÞ otherwise
ð3Þ
The detection of view zero-watermark is the same as that of the construction, except that the left view zerowatermark W′g;l ¼ w′g;l ði; jÞ is detected from the left view of distorted stereoscopic image, instead of the left view of the original stereoscopic image. The construction and detection of right view zerowatermark are similar with the construction and detection of left view zero-watermark, and the constructed and detected right view zero-watermarks with respect to the original and distorted stereoscopic images are denoted as W g;r ¼ wg;r ði; jÞ and W′g;r ¼ w′g;r ði; jÞ hereinafter.
2.2. Construction and detection of disparity zero-watermark
(1) Let (x, y) be the coordinate of the left-bottom corner of the current block to be estimated in the left view, dv be the candidate of the disparity vector. The cost function E(x, y, dv) for disparity estimation can be written as Eðx; y; dv Þ ¼ Edata ðx; y; dv Þ þ λEsmooth ðx; y; dv Þ
ð4Þ
∑ Lðx þp; y þqÞ
n1 n1
Edata ðx; y; dv Þ ¼ ∑
p¼0q¼0
Rðx þ p þ dv ; y þqÞ
ð5Þ
170
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176 3 Esmooth ðx; y; dv Þ ¼ ∑ dv dk k¼0
ð6Þ
The data item Edata is the displaced frame difference between the current block in the left view and its possible corresponding block in the right view, while the smoothness item Esmooth utilizes disparity vectors of the neighboring blocks to enhance smoothness of disparity map. The d0–d3 in Eq. (6) are disparity vectors of the left-top, top, right-top and left neighboring blocks of the current block. Factor λ is a constant which controls the influence of the smoothness item on the cost function. The dv which achieves the minimal cost E within the search window of disparity estimation will be regarded as the optimal disparity vector of the current block. For the occluded blocks in the left view of stereoscopic image which have no corresponding blocks in the right view, their disparity vectors are simply set to 0. The obtained disparity map of the stereoscopic image is denoted as D¼{d(s, t)}, 1rs rM/n, 1 rtrN/n. (2) Calculate the average disparity of the disparity map D S¼
n2 M=n N=n ∑ ∑ dðs; tÞ MN s ¼ 1 t ¼ 1
ð7Þ
MN (3) Disparity zero-watermark W d ¼ wd ðs; tÞ with total nn bits is constructed by judging the relation of d(s, t) and S, and wd(s, t) is constructed by ( 1; if dðs; tÞ 4S wd ðs; tÞ ¼ ð8Þ 0; otherwise
recovering rates of the detected zero-watermarks are used to assess the quality of the distorted stereoscopic image. The quality of the left and right views of a stereoscopic image is assessed by gauging the change of their gradient vectors in terms of the original ones because gradient reflects the structure of an image. The view zero-watermark recovering rate, HCview, is calculated by HC view ¼
HC view;l þHC view;r 2
ð9Þ
HC view;l ¼ 1
1 M N ∑ ∑ w ði; jÞ w′g;l ði; jÞ MN i ¼ 1 j ¼ 1 g;l
ð10Þ
HC view;r ¼ 1
1 M N ∑ ∑ wg;r ði; jÞ w′g;r ði; jÞ MN i ¼ 1 j ¼ 1
ð11Þ
where denotes the exclusive-OR. It is well known that the stereoscopic perception can be artificially induced by presenting two different images of the same scene, shifting one with respect to the other, thus mimicking two different views, namely the left and right views of stereoscopic image, to the left and right eyes. The difference in the views generates disparity in the stereoscopic image. When stereoscopic image is degraded, distortions may lead to modifications to disparity between left and right views. Therefore, it is useful to introduce this modification to assess the quality of stereoscopic perception. Since the constructed disparity zero-watermark reflects characteristic of disparity to some extent, the recovering rate of detected disparity zero-watermark is used to evaluate quality of stereoscopic perception, and it is defined by HC disp ¼ 1
n2 M=n N=n ∑ ∑ ½w ðs; tÞ w′d ðs; tÞ MN s ¼ 1 t ¼ 1 d
ð12Þ
The detection of disparity zero-watermark is the same as that of its construction, except that the disparity zerowatermark W′d ¼ w′d ðs; tÞ is detected from disparity map of the distorted stereoscopic image, instead of disparity map of the original stereoscopic image, and the threshold S is used as the same with the construction, that is to say, S is received from the sender. The designed disparity zero-watermark is more sensitive to disparity distortion at the middle level of the scene due to the use of average disparity as a threshold. If an object is far from the viewer, it is usually less interested by the viewer, thus its distortion may not be noticed by the viewer. But if an object is very close to the viewer, its disparity will be very large. Thus, if the distortion of its disparity is not so great compared with the original value, that is, the proportion of the change is not big, the distortion of the object may be less noticeable by the viewer.
The recovery ratio of the view zero-watermark reflects the quality of the left and right views of a stereoscopic image, while the recovery ratio of the disparity zero-watermark reflects the stereoscopic perception quality of the stereoscopic image. Thus, a nonlinear function is used to pool the two evaluation results into a score as RR-SIQA value of the stereoscopic image. The stereoscopic image quality score, HCs, is defined by
2.3. Zero-watermark based quality assessment for stereoscopic image
To test the performance of the proposed RR-SIQA method, some experiments are implemented. The VCIP database used in the experiments is introduced in [8], which is composed of totally 370 distorted stereoscopic images generated with 10 original high-resolution color stereoscopic images, as shown in Fig. 2. The distortions include JPEG2000 (100 stereoscopic images), JPEG
Because the constructed zero-watermarks are associated with some inherent characteristics of stereoscopic image, the integrality of zero-watermark varies as the quality of the distorted stereoscopic image. Thus, the
HC s ¼ aðHC view Þb þð1 aÞðHC disp Þ2 b
ð13Þ
where the parameters a and b are determined by training. In the proposed RR-SIQA method, the two view zerowatermarks and the disparity zero-watermark will be transmitted to the receiver. Since all of them are binary maps, they can be losslessly compressed efficiently so as to reduce the required side channel bandwidth. 3. Experimental results and analysis
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
(70 stereoscopic images), Gaussian white noise (100 stereoscopic images) and Gaussian blur (100 stereoscopic images) with different distortion degrees. The database also provides subjective evaluation results, e.g., differential mean opinion score (DMOS), for evaluating the consistency of objective SIQA methods against the human perception. To assess these color stereoscopic images objectively, their luminance component is used to construct the zerowatermarks. Parameters a and b in Eq. (13) are determined by training with a subset of the database including all 111
171
distorted stereoscopic images of “Bowling1”, “Computer” and “Dolls”. We employed the training algorithm to find the optimum parameters in Eq. (13) with respect to the highest correlation between HCs and DMOS, and finally they are set as a¼0.5004 and b¼1.1668. Other parameters used for disparity estimation are set as λ¼ 0.25, and 38 for the searching window. To balance contradiction between overhead and disparity accuracy, n¼8 was used. The evaluation is performed by comparing stereoscopic image quality score HCs with the subjective DMOS of the
Fig. 2. Left views of the original stereoscopic images [8]. (a)Art, (b) Bowling1, (c) Computer, (d) Dolls, (e) Drumsticks, (f) Dwarves, (g) Laundry, (h) Mobius, (i) Reindeer, and (j) Rocks1.
Fig. 3. Scatter plots of DMOS vs. HCs obtained with the proposed method. (a) JPEG2000 compression, (b) JPEG compression, (c) Gaussian white noise, and (d) Gaussian blur.
172
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
stereoscopic images. The linear correlation coefficient (CC), the rank order CC (ROCC) and the root mean-squared (RMS) error between DMOS and objective score after nonlinear regression are utilized to test the consistency of the proposed RR-SIQA method and the subjective perception. CC expresses the accuracy of an objective method, ROCC indicates the monotonic property of the objective method, while RMS error expresses the validity of the objective method. The higher the CC and ROCC between an image quality assessment score and DMOS value is, the better the assessment method is. The smaller RMS error is, the better the method performs. The nonlinearity chosen for regression is a five-parameter logistic function [43] written as DMOSp ¼ β1 logisticðβ2 ; ðx β3 ÞÞ þβ4 x þ β5
ð14Þ
logisticðτ; xÞ ¼
1 1 2 1 þexpðτxÞ
ð15Þ
where β1, β2, β3, β4, and β5 are determined by using the subjective scores and the objective scores. Fig. 3 presents scatter plots of DMOS values vs. HCs, with respect to the four different types of distortions. The curves in the figure are obtained with the logistic function. If the points in scatter plot are close to the fitted curve, it means that the SIQA method coincides with the DMOS. The scatter plots of the metric in [29] are also given in Fig. 4. Table 1 shows the CC, ROCC and RMS error of the proposed RR-SIQA method with respect to the four different types of distortions as well as the results of the metric in [29]. It is seen that the proposed method performs quite well for wide range of distortion types. For stereoscopic images distorted by the
Fig. 4. Scatter plots of DMOS vs. scores obtained with the metric in [29]. (a) JPEG2000 compression, (b) JPEG compression, (c) Gaussian white noise, and (d) Gaussian blur.
Table 1 Performance of the proposed objective method compared with metric in [29]. Proposed method
CC ROCC RMS error
Metric in [29]
JPEG2000
JPEG
Gaussian white noise
Gaussian blur
JPEG2000
JPEG
Gaussian white noise
Gaussian blur
0.961 0.765 4.35
0.950 0.884 3.88
0.962 0.950 3.55
0.947 0.887 1.85
0.955 0.746 5.11
0.947 0.833 4.71
0.954 0.941 3.56
0.940 0.853 2.09
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
173
Fig. 5. HCs vs. the degree of distortion with respect to “Art”. (a) JPEG2000 compression, (b) JPEG compression, (c) Gaussian white noise, and (d) Gaussian blur.
Table 2 Comparison of computation time using database [8]. Model
Wang et al. [20]
Li et al. [22]
Zhang et al. [24]
Ma et al. [27]
Proposed RR-SIQA
Time (s/image)
14.66
20.89
8.04
41.41
38.15
four types of distortions, the proposed RR-SIQA method provides better prediction accuracy (larger CC), stricter monotonicity (larger ROCC) and smaller prediction error (smaller RMS error). Therefore, the proposed method is a reasonable and useful choice in practical RR-SIQA systems. Fig. 5 shows HCs versus the degree of distortion of the “Art” stereoscopic image with respect to the four different types of distortions. The horizontal axes represent the degree of distortion under different types of distortions. It is noticed that the curves of HCs versus the degree of distortion are monotonous. Moreover, for stereoscopic images distorted with Gaussian white noise and Gaussian blur, the curves tend to be more consistent with its tangents, that is, HCs is more close to a constant as the degree of distortion grows. While for stereoscopic images distorted with JPEG2000 and JPEG, HCs drops quickly at the points with high compression rate. This feature is well consistent with HVS. We also compare the computational complexity of the proposed RR-SIQA method with other four RR-IQA methods
without any optimization of the code. The results are reported in Table 2, where we list the average time taken for each image, over all stereoscopic images in the database [8], using a computer with Intel i3 processor at 3.10 GHz. It can be seen that the proposed method takes slightly more time than most of the other methods under comparison, mainly due to computation of disparity, but it is comparable to metric in [27]. Nowadays, many stereoscopic images have been usually compressed to JPEG or JPEG2000 format to reduce the bandwidth and storage requirements. As a result, the objective quality assessment for compressed stereoscopic images is necessary in various applications. PSNR (FR method) is often used to evaluate the quality of a reconstructed stereoscopic image in term of the original one. However, it is noticed that the consistency between PSNR and subjective perception is not satisfied in some situations, for example, when stereoscopic images are distorted by JPEG compression, as shown in Fig. 6, which presents the scatter plots of the DMOS values versus PSNR. In Fig. 6, the PSNR
174
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
Fig. 6. Scatter plots of DMOS vs. PSNR. (a) JPEG2000 compression and (b) JPEG compression.
Table 3 Performance of the proposed objective method compared with PSNR.
CC ROCC RMS error
Proposed method
PSNR
JPEG2000
JPEG
JPEG2000
JPEG
0.961 0.765 4.35
0.950 0.884 3.88
0.957 0.761 4.01
0.915 0.848 5.10
values are the average PSNR of the left and right views of a stereoscopic image. 2D metrics are considered to fail for 3D images [44]. Literature [45] also gives some examples that images with same MSE have quite different subjective perception. Moreover, in wireless multimedia systems, the original stereoscopic image is often not available. Clearly, the RR method is useful in such situations. The comparison results on CC, ROCC and RMS error of the proposed RR-SIQA method and PSNR metric are shown in Table 3. It is noticed that the proposed method achieves better performance than PSNR metric in terms of better prediction accuracy (higher CC), stricter monotonicity (higher ROCC) and similar prediction error (lower RMS error) when assessing quality of stereoscopic image compressed with JPEG2000 and JPEG. We also implement the proposed SIQA method and the metric in [29] on the LIVE 3D Image Quality Database [46], which contains five types of distortions across 365 distorted stereoscopic images. The parameters a, b, λ, n, and the searching window of the proposed SIQA are the same with that used for VCIP database. Table 4 provides the CC, SROCC and RMS error with respect to the two metrics for each type of distortion on the LIVE 3D Image Quality Database. It is clear that the proposed method also outperforms the metric in [29] for this database. Additionally, to assess the statistical significance of the performance difference between two SIQA methods, an F-test based on the errors between the DMOS scores and the method predictions (obtained from the non-linear regression process that was used to compute the linear correlation coefficient on the entire dataset) is made, which could show the statistically superior objective model between two SIQA objective methods. The detail
of the statistical evaluation can be found in [43,47]. The residual errors between the objective method prediction and the DMOS value are defined as Ei ¼ P i DMOSi ;
i ¼ 1; 2; …; N
ð16Þ
where P is the objective method prediction and N is the number of experimental stereoscopic images. An F-test is performed on the ratio of the variance of the residual error from one objective to that of another objective method at 95% significance level. The F-ratio is always formed by placing the objective method with larger residual error variance in the numerator. An F-ratio ratio larger than a threshold indicates that the performance of the SIQA method in the numerator of the F-ratio is statistically inferior to that of the SIQA method in the denominator. The results of the statistical significance test are listed in Table 5. Each entry in the table includes two characters which correspond to the two publicly databases in the order of {VCIP Database and the LIVE 3D Image Quality Database}, The symbol “-” denotes that the two SIQA methods are statistically indistinguishable, “1” denotes that the SIQA method of the row is statistically better than that of the column, and “0” denotes that the SIQA method of the column is better than that of the row. It is seen that the proposed method outperforms the metric in [29] and PSNR. 4. Conclusions In this paper, a reduced-reference stereoscopic image quality assessment (RR-SIQA) method is proposed based on zero-watermarks. Two kinds of zero-watermarks, that is, view zero-watermark and disparity zero-watermark, are constructed to reflect the image structure and stereoscopic perception. Then, objective quality score of stereoscopic image are obtained by pooling the zero-watermark recovering rates of the view zero-watermarks and the disparity zero-watermark. Experimental results show that the stereoscopic image quality evaluation results assessed with the proposed objective method are well consistent with subjective assessment, and the proposed RR-SIQA method achieves better performance than the widely used FF-SIQA method PSNR in assessing quality of stereoscopic image compressed with JPEG2000 and JPEG. The proposed
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
175
Table 4 Performance of the proposed objective method compared with metric in [29] on the LIVE 3D Image Quality Database. Proposed method
CC ROCC RMS error
Metric in [29]
JPEG2000
JPEG
WN
blur
FF
JPEG2000
JPEG
WN
Blur
FF
0.907 0.906 5.08
0.697 0.702 3.72
0.926 0.929 6.01
0.920 0.889 4.71
0.639 0.547 8.67
0.904 0.856 5.53
0.531 0.550 5.54
0.896 0.896 7.41
0.798 0.690 8.75
0.670 0.545 9.23
Table 5 Statistical significance matrix base on SIQA-DMOS residuals. Model
PSNR
Metric in [29]
Proposed method
PSNR Metric in [29] Proposed method
–– 0– 11
1– –– 11
00 00 ––
method for automatic SIQA can be used to monitor stereoscopic image quality for three dimensional image/ video broadcasting and multimedia systems. Future works will focus on the features of HVS especially stereoscopic visual masking and extension to other more effective SIQA methods utilizing different filters as well as reducedreference stereoscopic video quality assessment methods.
Acknowledgments This research is supported by the Natural Science Foundation of China (Grant nos. 61071120, 61171163, 6127120, 61271021, and 61111140392), the Innovative Scientific Research Project for Postgraduates in Zhejiang Province (Grant no. YK2011049), the Research Foundation of Education Department of Zhejiang Province (Grant no. Y201224839) and the Outstanding (Postgraduate) Dissertation Growth Foundation of Ningbo University (Grant no. PY20110002). It was also sponsored by the K.C. Wong Magna Fund in Ningbo University. References [1] Pedro F. Felzenszwalb, R. Zabih, Dynamic programming and graph algorithms in computer vision, IEEE Trans. Pattern Anal. Mach. Intell. 33 (4) (2011) 721–740. [2] W. Wang, D. Peng, H. Wang, et al., A multimedia quality-driven network resource management architecture for wireless sensor networks with stream authentication, IEEE Trans. Multimedia 12 (5) (2010) 439–447. [3] M. Tanimoto, FTV: free-viewpoint television, Signal Process.: Image Commun. 27 (2012). [4] Y. Fan, Y. Kung, B. Lin, Three-dimensional auto-stereoscopic image recording, mapping and synthesis system for multiview 3D display, IEEE Trans. Magn. 47 (3) (2011) 683–686. [5] F. Shao, G. Jiang, X. Wang, M. Yu, K. Chen, Stereoscopic video coding with asymmetric luminance and chrominance qualities, IEEE Trans. Consum. Electron. 56 (4) (2010) 2460–2468. [6] M. Fiedler, T. Hossfeld, T.G. Phuoc, A generic quantitative relationship between quality of experience and quality of service, IEEE Network 24 (2) (2010) 36–41. [7] A.K. Moorthy, A.C. Bovik, Visual quality assessment algorithms: what does the future hold? Multimedia Tools Appl. 51 (2) (2010) 675–696. [8] X. Wang, M. Yu, Y. Yang, G. Jiang, Research on subjective stereoscopic image quality assessment, in: Proceedings of SPIE-IS&T Electronic Imaging, San Jose, USA, 2009, pp. 1–10.
[9] L. Jiang, J. He, N. Zhang, T. Huang, An overview of 3D video representation and coding, 3D Res. 1 (1) (2010) 43–47. [10] C. Jung, L. Jiao, Y. Oh, J. Kim, Depth-preserving DIBR based on disparity map over T-DMB, Electron. Lett. 46 (9) (2010) 628–629. [11] T.R. Coffman, A.C. Bovik, Efficient stereoscopic ranging via stochastic sampling of match quality, IEEE Trans. Image Process. 19 (2) (2010) 451–460. [12] D. Kim, K. Sohn, Visual fatigue prediction for stereoscopic image, IEEE Trans. Circuits Syst. Video Technol. 21 (2) (2011) 231–236. [13] A. Benoit, P. Callet, P. Campisi, R. Cousseau, Quality assessment of stereoscopic image, EURASIP J. Image Video Process. (2008) 1–13. [14] J. Yang, C. Hou, R. Xu, J. Lei, New metric for stereo image quality assessment based on HVS, Int. J. Imaging Syst. Technol. 20 (4) (2010) 301–307. [15] M. Lambooij, W. Iisselsteijn, D.G. Bouwhuis, I. Heynderickx, Evaluation of stereoscopic image: beyond 2D quality, IEEE Trans. Broadcast. 57 (2) (2011) 432–444. [16] R. Akhter, Z. Parvez Sazzad, Y. Horita, J. Baltes, No-reference stereoscopic image quality assessment, in: Proceedings of SPIE-IS&T Electronic Imaging SPIE, San Jose, USA, vol. 7524, 2010. [17] Z.M.P. Sazzad, S. Yamanaka, Y. Horita, Spatio-temporal segmentation based continuous no-reference stereoscopic video quality prediction, in: Proceedings of the Second International Workshop on Quality of Multimedia Experience (QoMEX), Trondheim, June 21–23, 2010, pp. 106–111. [18] M. Pinson, S. Wolf, A new standardized method for objectively measuring video quality, IEEE Trans. Broadcast. 50 (3) (2004) 312–322. [19] Luigi Atzori, Alessandro Floris, Giaime Ginesu, Daniele Giusto, Streaming video over wireless channels: exploiting reducedreference quality estimation at the user-side, Signal Process.: Image Commun. 27 (2012) 1049–1065. [20] Z. Wang and, E.P. Simonceli, Reduced-reference image quality assessment using a wavelet domain natural image statistic mode1, in: Proceedings of the Human Vision and Electronic Imaging X, San Jose, CA, vol. 5666(1), 2005, pp. 149–159. [21] Z. Wang, G. Wu, H.R. Sheikh, E.P. Simoncelli, E.-H. Yang, A.C. Bovik, Quality-aware images, IEEE Trans. Image Process. 15 (6) (2006) 1680–1689. [22] Q. Li, Z. Wang, Reduced-reference image quality assessment using divisive normalization-based image representation, IEEE Journal of Selected Topics in Signal Processing 3 (2) (2009) 202–211. (Special issue on Visual Media Quality Assessment). [23] X. Gao, W. Lu, D. Tao, X. Li, Image quality assessment based on multiscale geometric analysis, IEEE Trans. Image Process. 18 (7) (2009) 1409–1423. [24] M. Zhang, W. Xue, X. Mou, Reduced reference image quality assessment based on statistics of edge, in: Proceedings of the SPIE, vol. 7876, 2011, pp. 787611-1–787611-7. [25] J.A. Redi, P. Gastaldo, I. Heynderickx, R. Zunino, Color distribution information for reduced-reference assessment of perceived image quality, IEEE Trans. Circuits Syst. Video Technol. 20 (12) (2010) 1757–1769. [26] Z. Wang, A.C. Bovik, Reduced- and no-reference image quality assessment: the natural scene statistic model approach, IEEE Signal Process. Mag. 28 (2011) 29–40. [27] L. Ma, et al., Reduced-reference image quality assessment in reorganized DCT domain, Signal Process.-Image Commun. (2012). [28] Rajiv Soundararajan, Alan C. Bovik, Video quality assessment by reduced reference spatio-temporal entropic differencing, IEEE Trans. Circuits Syst. Video Technol. 23 (4) (2013) 684–694. [29] A. Maalouf, M.C. Larabi, CYCLOP: a stereo color image quality assessment metric, in: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Prague, May 22–27, 2011, pp. 1520–6149.
176
W. Zhou et al. / Signal Processing: Image Communication 29 (2014) 167–176
[30] C.T.E.R. Hewage, M.G. Martini, Reduced-reference quality assessment for 3D video compression and transmission, IEEE Trans. Consum. Electron. 57 (3) (2011) 1185–1193. [31] C.T.E.R. Hewage, M.G. Martini, Reduced-reference quality metric for 3D depth map transmission, 3DTV-conference: the true visioncapture, Transmission and Display of 3D Video (3DTV-CON), Tampere, June 7–9, 2010, pp. 1–4. [32] J. Cox, J. Kilian, F.T. Leighton, T. Shamoon, Secure spread spectrum watermarking for multimedia, IEEE Trans. Image Process. 6 (12) (1997) 1673–1687. [33] L.X. Luo, Z.Y. Chen, M. Chen, X. Zeng, Reversible image watermarking using interpolation technique, IEEE Trans. Inf. Forensics Secur. 5 (1) (2010) 187–193. [34] T.Y. Lee, S. Lin, Dual watermark for image tamper detection and recovery, Pattern Recognition 41 (2008) 3497–3506. [35] C.L. Li, Y.H. Wang, B. Ma, Z.X. Zhang, Tamper detection and selfrecovery of biometric images using salient region-based authentication watermarking scheme, Comput. Stand. Interfaces 34 (4) (2012) 367–379. [36] S. Wang, D. Zhang, J. Zhao, W.J. Tam, F. Speranza, An image quality evaluation method based on digital watermarking, IEEE Trans. Circuits Syst. Video Technol. 171 (2007) 98–105. [37] M.S. El-Mahallawy, A. Hashad, H.H. Ali, H.S. Zaky, Quality estimation of video transmitted over an additive WGN channel based on digital watermarking and wavelet transform, World Acad. Sci. Eng. Technol. 57 (2009) 419–423. [38] A. Bhattachrya, S. Palit, N. Chatterjee, G. Roy, Blind assessment of image quality employing fragile watermarking, in: Proceedings of the 7th International Symposium on Image and Signal Processing and Analysis (ISPA), 2011, pp. 431–436.
[39] J. Sang, X.F. Liao, M.S. Alam, Neural-network-based zerowatermark scheme for digital images, Opt. Eng. 45 (9) (2006) 0970061–0970069. [40] K. Minoo, T.Q. Nguyen, Perceptual Video Coding with H.264, in: Proceedings of the 39th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, USA, October 2005, pp. 741–745. [41] J. You, L. Xing, A. Perkis, X. Wang, Perceptual quality assessment for stereoscopic images based on 2D image quality metrics and disparity analysis, in: Proceedings of the 5th International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), Scottsdale, AZ, USA, January 2010. [42] L. Xing, J. You, T. Ebrahimi, A. Perkis, A perceptual quality metric for stereoscopic crosstalk perception, in: Proceedings of IEEE International Conference on Image Processing (ICIP), Hong Kong, China, September 2010, pp. 4033–4036. [43] H. Sheikh, M. Sabir, A. Bovik, A statistical evaluation of recent full reference image quality assessment algorithms, in: Proceedings of IEEE Transactions on Image Processing, vol. 15(11), 2006, pp. 3441– 3452. [44] Atanas Boev, Maija Poikela, Atanas Gotchev, Anil Aksay, Modelling of the stereoscopic HVS, Report on Mobile 3DTV 〈http://sp.cs.tut.fi/ mobile3dtv/results/〉. [45] Z. Wang, A.C. Bovik, Mean squared error: love it or leave it? A new look at signal fidelity measures, IEEE Signal Process. Mag. 26 (1) (2009) 98–117. [46] A.K. Moorthy, C. Su, A. Mittal, A.C. Bovik, Subjective evaluation of stereoscopic image quality, Signal Process.: Image Commun. 28 (8) (2013) 870–883. [47] X. Fei, L. Xiao, Y. Sun, Z. Wei, Perceptual image quality assessment based on structural similarity and visual masking, Signal Process.: Image Commun. 27 (7) (2012) 772–783.