Reduction of blocking artifacts in JPEG compressed images

Digital Signal Processing 17 (2007) 225–243 www.elsevier.com/locate/dsp

Reduction of blocking artifacts in JPEG compressed images Sukhwinder Singh, Vinod Kumar ∗ , H.K. Verma Instrumentation and Signal Processing Laboratory, Department of Electrical Engineering, Indian Institute of Technology Roorkee, Roorkee 247667 (Uttaranchal), India Available online 12 September 2005

Abstract Image compression is a very important issue for several applications in the area of multimedia communications, the objective being reduction of storage and transmission costs. Many efficient coding techniques have been developed for various applications. Amongst them, the Joint Photographic Experts Group (JPEG) has been recommended for compression of continuous tone still images. However, the reconstructed images from JPEG compression produce annoying blocking artifacts near block boundaries, particularly in highly compressed images, because each block is transformed and quantized independently. Several techniques/algorithms have been proposed by researchers, both in spatial and frequency domains, for reduction of these artifacts with varied degree of success. These are briefly overviewed here. A new technique working in frequency domain, is proposed here by authors. This paper puts forth a method and an algorithm, working in frequency domain, for the detection and reduction of such blocking artifacts. These artifacts are modeled here as 2-D step functions between two neighboring blocks. Presence of the blocking artifacts is detected by using block activity based on human visual system (HVS) and block statistics. The boundary regions between blocks are identified as either smooth or non-smooth regions. The blocking artifacts in smooth regions are removed by modifying a few DCT coefficients appropriately, whilst an edge-preserving smoothing filter is applied to the non-smooth regions, i.e., genuine edges. The algorithm has been applied to variety of JPEG compressed images and results are compared with other postprocessing algorithms. The reduction in the blocking artifacts for each image have been evaluated using three indices, namely peak signal-to-noise ratio (PSNR), mean structure similarity (MSSIM) index based on human visual perception, and a new index, called here block boundary measure (BBM), applied

* Corresponding author. Fax: +91 1332 273560.

E-mail address: [email protected] (V. Kumar). 1051-2004/$ – see front matter © 2005 Elsevier Inc. All rights reserved. doi:10.1016/j.dsp.2005.08.003

226

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

to both vertical and horizontal block boundaries. The results show that the proposed method is very effective in detecting and reducing the blocking artifacts in JPEG compressed images. © 2005 Elsevier Inc. All rights reserved. Keywords: Image compression; Blocking artifacts; Image quality; Human visual perception

1. Introduction 1.1. Background To meet the growing demand for video communication, many efficient image compression methods have been developed and standardized for example JPEG and MPEG [1,2]. The purpose of the data compression is to reduce the storage and transmission costs while maintaining the image quality. High quality image communication with low-bitrate is gaining special importance in the relatively established applications like video conferencing, videophone, and interactive TV and newer applications like telemedicine, picture archiving, and communication systems (PACS). The block discrete cosine transform (BDCT) is amongst the most widely used techniques for compression of still and moving images. In a typical DCT compression scheme, the input image is divided into small blocks, each block being transformed independently to convert the image pixels into DCT coefficients. The DCT coefficients are then quantized using scalar quantization defined by a quantization matrix. The main drawback of the BDCT based compression techniques is, the introduction of blocking artifacts, which represents the artificial discontinuity between adjacent blocks resulting from the independent processing of the blocks without taking into account the interblock pixel correlations. There is an obvious need of removing these blocking artifacts in the low-bit-rate transform compressed images. 1.2. Literature review Over the past several years, many techniques have been applied to reduce the blocking artifacts in block DCT-coded images. Two approaches are generally adopted. In the first approach, the reduction of blocking artifacts is carried out at the encoding side [3,4], but the methods based on this approach do not conform to the existing standards such as JPEG and MPEG. In the second approach, the reconstructed image is postprocessed aimed at improving its visual quality without any modification in the encoding or decoding mechanisms, making it compatible with the aforesaid coding standards. Because of this advantage, most of the recently proposed algorithms follow the second approach. Postprocessing of the decoded image may be carried out in spatial domain or in frequency domain. 1.2.1. Spatial domain techniques Reeve and Lim proposed a symmetric, two-dimensional 3 × 3 Gaussian spatial filtering method for the pixels along the block boundaries [5]. However, it causes blurring of the image due to its low-pass nature. Ramamurthi and Gersho proposed nonlinear space-variant

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

227

filter which adapts to the varying shape of the local signal spectrum, and reduces only the locally out-of-band noise [6]. The algorithm employs a two-dimensional (2-D) filter in the areas away from edges, and for near edges, one-dimensional (1-D) filter aligned parallel to edge so as to minimize the blocking artifacts. Hsu and Chen proposed an adaptive separable median filter (ASMF) [7]. The proposed filter not only reduced the blocking artifacts, but also preserved the edges. Meier et al. presented a region-based method for enhancement of images degraded by blocking effects [8]. In this method, the degraded image is segmented by a region growing algorithm, and each region is the filtered using a low-pass filter. It preserves the edges, as filtering is not applied to region boundaries. Lee et al. proposed a postprocessing algorithm to reduce the blocking artifacts in JPEG compressed images after classifying them into edge area and monotone area according to the edge map which is obtained after thresholding the gradient absolute image [9]. The signal adaptive filtering consists of a 1-D directional smoothing filtering for edge area and 2-D adaptive average filtering for monotone area. A corner outlier detection/replacement scheme is also given to remove the corner outlier. Chou et al. remove blockiness by performing a simple nonlinear smoothing of pixels [10]. They first form the maximum likelihood estimation of quantization noise to differentiate between artificial and actual edges. Many researchers proposed iterative methods based on the theory of projections onto convex sets (POCS) [11–14]. In these methods, initially closed convex constraint sets are defined which correspond to all of the available data on the original uncoded image. Iterative computations of alternating projections onto these convex sets recover the original image from the coded image. However, these methods usually have high computational complexity, and thus are difficult to adapt to real-time image processing applications. Rourke et al. gave an algorithm to remove blocking artifacts by maximizing the a posteriori probability (MAP) of the unknown image [15]. A Markov random field (MRF), and the Huber minimax function is selected as a prospective function model the probability function of the decompressed image. Luo et al. proposed a two step approach for blocking artifacts reduction based on MAP [16]. First, a DC calibration is performed in a block-by-block fashion based on gradient continuity constraints over the block boundaries. Then, a modified Huber–Markov random field model is employed in order to differentiate the pixels on the block boundary from those inside the block. Finally a local optimization technique, iterative conditional mode (ICM) is applied to employ smoothing algorithms. Meier et al. proposed a method to remove blocking artifacts by first segmenting the degraded image into regions by an MRF segmentation algorithm, and then each region is enhanced separately using an MRF model [17]. Coudoux et al. proposed a method based on a nonlinear, space-variant filtering [18]. A visibility parameter is computed for each artifact using several characteristics of the human vision system (HVS). Then this information is used to steer the selection of an adaptive nonlinear, space-variant smoothing operation at block boundaries. 1.2.2. Frequency domain techniques Minami et al. gave a new approach for reducing the blocking effect in frequency domain [19]. A new index to measure the blocking effects namely the mean squared difference of slope (MSDS) is introduced. It is shown that the expected value of the MSDS increases after quantizing the DCT coefficients. This approach removes the blocking effect

228

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

by minimizing the MSDS, while imposing linear constraints corresponding to quantization bounds. Lakhani et al. also reduce blocking effects using MSDS [20]. However, a different solution of minimizing the MSDS is used. Recently, Triantafyllidis et al. have proposed another method of minimizing MSDS, which involves diagonal neighboring pixels in addition to horizontal and vertical neighboring pixels [21]. Liu et al. proposed a DCT-domain method for blind measurement of blocking artifacts, by modeling the artifacts as 2-D step functions in shifted blocks [22]. A fast DCT-domain algorithm extracts all the parameters required to detect the presence of blocking artifacts, by using HVS properties. Artifacts are then reduced by using an adaptive method. Zeng proposed a simple DCT-domain method for blocking effect reduction, applying a zero masking to the DCT coefficients of some shifted image blocks [23]. However, the loss of edge information caused by the zero-masking scheme is noticeable. Luo and Ward gave a technique, which preserved the edge information [24]. The technique is based on reducing the blocking artifacts in the smooth regions of the image. The correlation between the intensity values of the boundary pixels of two neighboring blocks in the DCT domain is used to distinguish between smooth and non-smooth regions. 1.3. Present work An attempt has been made in the present paper to further improve the approach presented in [24] by adding the concept of visual perception to detect the blocking artifacts. In most of the image processing systems, the final observer of the processed image is the human. Thus the retention of visual quality is important from the point of view of observer [25]. Therefore, it would be logical to incorporate the HVS characteristics to measure the blocking artifacts [18,26]. Visibility of blocking artifacts is dependent on the local content of a given image. Due to the masking effect in the HVS artifacts in the regions of the high activity are less perceptible than those on low activity regions. Blocking artifacts are detected by using the activity masking property of HVS alongwith the local information content of image in terms of energy. A differentiation between the actual edge and artificial discontinuity arising from blocking artifacts is made with a view to preserve the actual edges in the image while reducing the artificial discontinuity. This difference separates the blocks into smooth and non-smooth regions. For the smooth region, reduction of blocking artifacts is carried out by modifying few DCT coefficients. Whereas for actual edge blocks, an edge preserving smoothing sigma filter is used because of its low computational complexity [27]. This will enable the use of the proposed technique in real time applications. From the experimentation with different sizes of window of sigma filter, it was found that 5 × 5 window is the best choice both in terms of performance and computational burden. 2. Measurement of blocking artifacts 2.1. Model of blocking artifacts Consider two horizontal adjacent 8 × 8 blocks b1 and b2 , with mean values μ1 and  μ2 . When the corresponding DCT blocks of b1 and b2 are μ2 , respectively, where μ1 =

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

229

Fig. 1. Structure of a new block b, consisting of right half of b1 and the left half of b2 .

quantized using a large quantization parameter, most of the DCT coefficients become zero. Accordingly, a 2-D step function between b1 and b2 (due to μ1 = μ2 ) may become visible, creating a blocking artifact. A new block b composed of the right half of b1 and the left half of b2 is formed, as shown in Fig. 1. Blocking artifact between b1 and b2 can be modeled as 2-D step function in block b defined as defined as follows:  − 18 , ∀i ∈ [0, 7], j ∈ [0, 3], (1) s(i, j ) = 1 8 , ∀i ∈ [0, 7], j ∈ [4, 7]. Hence b(i, j ) = β s(i, j ) + μ + r(i, j ),

∀i, j ∈ [0, 7],

(2)

where |β| is the amplitude of 2-D step function s, μ is the average value of the block b, indicating the local background brightness, and r is the residual block, which describes the local activity around the block edge. If the value of |β| is large, the blocking effect is taken to be more serious, provided that the background brightness and local activity remain unchanged. 2.2. Proposed technique of blocking artifact detection It is observed that the 8 × 8 DCT transform of the 2-D step block s has only four nonzero elements in the first row because s is constant in the vertical direction and anti-symmetric in the horizontal direction. Let the vector v = [v0 , v1 , . . . , v7 ] be the first row of 8 × 8 DCT transform of the 2-D step function s. Then vj = 0, ∀j = 2n, n ∈ [0, 3]. From the unitary property of DCT transform, it can be shown that    7  7 7    2  vj =  s 2 (x, y) = 1. (3) v2 = j =0

x=0 y=0

If B is the DCT transform of block b, the parameters in Eq. (2) can be computed as follows: μ = B(0, 0)/8, β=

7  j =0

vj B(0, j ).

(4) (5)

230

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

If R be the DCT transform of residual block r. Then R can be computed as follows: R = B, R(0, 0) = 0, R(0, i) = R(0, i) − βvi ,

∀i ∈ [0, 7].

(6)

The activity masking and brightness masking are two well-established properties of the HVS that are highly significant to the observation of blocking artifacts [22]. HVS based model to compute visibility of blocking artifacts (η) can be as follows [22]: η=

|β| (1 + Ahtotal )(1 + (μ/μ0 )γ )

,

(7)

where Ahtotal is the total horizontal activity in the block b, and μ0 = 150, γ = 2. For a given DCT block if its visibility (η) is less than a certain threshold (τ ), the no processing is applied to block. If η  τ , then it shows that the reduction of blocking artifact is required. It can be found that 8 × 8 DCT transform of b, i.e., B only has five non-zero elements in the first row, thus:  0, i = 0, j ∈ [0, 7], B(i, j ) = (8) 0, i = 0, j = 2n, ∀n ∈ [1, 3], B(i, j ) = non-zero,

i = 0, j = 0, and j = 2n + 1, ∀n ∈ [0, 3].

(9)

This shows that the noise energy of the horizontal boundary discontinuity of block is represented by only a few coefficients of the first row of the DCT coefficient array. If there is heavier blockiness, then there are larger ripples in these nonzero values.

3. Blocking artifacts reduction In order to reduce the blocking artifacts between two horizontally adjacent (8 × 8) blocks, first row of B(i, j ) for i = 0, j = 0, and j = 2n + 1, ∀n ∈ [0, 3] should be modified. If modifications of these coefficients are carried out without considering the nature of the image in the neighborhood, it may result in additional artifacts. Hence modifications are to be carried out on the basis of the local information content of the image. However, before applying the modification, it must be decided whether the edge is genuine due to horizontal change in pixel intensity values at that position or it has resulted from blocking artifact between blocks b1 and b2 . In other words, it needs to be checked whether two blocks belong to smooth or non-smooth regions as under: (1) Smooth regions: If the two neighboring blocks b1 and b2 have similar frequency properties and the 8 × 8 pixel area around the block boundary, i.e., block b (Fig. 1) does not have high frequency, then the latter area is considered to be of smooth nature. (2) Non-smooth regions: If the nature of frequencies of two neighboring 8 × 8 blocks b1 and b2 differ from each other then the regions are termed as non-smooth regions. The

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

231

presence of texture or strong diagonal edges would result in relatively high values of the high order DCT coefficients. For mathematical modeling of the decision of whether the area in question formed by blocks b1 and b2 is a smooth region, the following criteria are used: (i) Blocks b1 and b2 have similar horizontal frequency properties, i.e., the energy lying in the low frequency regions of the blocks in the frequency domain are close in values, and (ii) Blocks b1 and b2 have boundary between them belonging to a relatively smooth region, i.e., the new shifted block b is of low frequency content. In addition, the visibility of blocking artifact should be more than the specified threshold, i.e., η  τ . Thus, energyB1 − energyB2 < Te ,

energyB < Ts ,

η  τ,

(10)

where 1  B(i, j )2 , mn 3

energy =

3

m, n = 4.

i=0 j =0

The values of Te , Ts , and τ need to be predetermined experimentally. The proposed algorithm has been applied to different images and reduction in the blocking artifacts is observed. After experimentation it was found that, Te = 150, Ts = 250, and τ = 50 gave the best results for all types of images. 3.1. Reduction in smooth regions If the above constraints are satisfied, the blocking artifact reduction is performed in the DCT domain by modifying the relevant coefficients of B. The first row of the DCT coefficient matrix of block b is modified by the weighted average of blocks b1 , b2 , and b. The advantage of using weighted average of adjacent block coefficients to modify the AC coefficients is that it is more adaptive to image content. Let MB be the modified DCT coefficient of block b. The modifications are carried out as follows:  (11) MB (0, j ) = α0 B(0, j ) + β0 B1 (0, j ) + B2 (0, j ) , ∀j ∈ [0, 1],  MB (0, j ) = α1 B(0, j ) + β1 B1 (0, j ) + B2 (0, j ) , ∀j = 2n + 1, n ∈ [1, 3], (12) MB (0, j ) = B(0, j ),

∀j = 2n, n ∈ [1, 3],

(13)

where α0 + 2β0 = 1 and α1 + 2β1 = 1. The actual values chosen on the basis of trials are: α0 = 0.6, β0 = 0.2 and α1 = 0.5, β1 = 0.25, where α0 , β0 and α1 , β1 are varied from 0.0 to 1.0. To remove the boundary discontinuity between two vertically adjacent blocks, a similar procedure can be used. But in this case, the first column of the DCT coefficient array of block b is modified.

232

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

Table 1 Comparison of PSNR for different window sizes of sigma filter when proposed technique is applied to different JPEG compressed images Image Lena Boat Peppers Bird Test pattern

Bit rate (bpp) 0.156 0.247 0.169 0.157 0.351

PSNR for various window sizes 3×3

5×5

7×7

29.34 30.85 27.82 36.23 31.61

29.45 31.14 28.02 36.38 32.57

29.35 29.78 27.95 35.92 30.75

3.2. Reduction in non-smooth regions For the strong texture and edge regions the above filtering is not performed so as to avoid the appearance of artifacts. In order to remove the blocking effects that might have remained and improve on other encoding artifacts such as ringing effects and uneven appearance in stationary areas which are due to quantization, an edge-preserving smoothing spatial Sigma filter [25] is applied. This filter is based on sigma probability of Gaussian distribution, and it smoothes the image noise by averaging only those neighborhood pixels which have the intensities within a fixed sigma range of the center pixel. As a result, image edges are preserved and subtle details are retained. From the experimentation with different sizes of window of sigma filter, it was found that 5 × 5 window is the best choice both in terms of performance and computational burden. The comparison of peak signalto-noise ratio (PSNR) for different window sizes of sigma filter applied to different JPEG compressed images is presented in Table 1. It is clear that the 5 × 5 window size gave best results in terms of PSNR at different bit rates. Applying such a filter preserves the edges and smoothes the noise, resulting in a more agreeable appearance.

4. Results and discussion In order to evaluate the performance of the proposed algorithm, it has been applied to a variety of JPEG compressed images and the results are compared with other postprocessing techniques in DCT domain proposed by [22,24]. The results are also compared with the technique proposed by [9]. The first experiment conducted to demonstrate the performance of these techniques for various bit rates. The objective results measured in peak signal-to-noise ratio (PSNR) of postprocessing of Lena image compressed with compression ratios ranging from 30:1 to 50:1 are shown in Fig. 2. It can be seen form Fig. 2 that blocking artifact reduction technique proposed by [24] only improves the PSNR marginally, and sometimes even has lower PSNR than the decoded JPEG image. The results obtained from technique given by [22] are also similar. The proposed technique achieves nearly the same PSNR as that proposed by [9] for higher bit rates whereas the proposed technique gave best results at lower bit rates. Table 2 shows

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

233

Fig. 2. PSNR vs bit rate over JPEG coded Lena image post processed by different deblocking methods. Table 2 Comparison of PSNR for different post processing techniques Image

Bit rate (bpp)

JPEG

Method [9]

Method [22]

Method [24]

Proposed method

Boat Peppers Bird Test pattern

0.247 0.169 0.157 0.351

31.14 27.13 35.35 31.09

31.67 27.86 35.92 31.93

31.02 27.01 35.14 31.01

30.05 27.69 36.39 30.95

31.14 28.02 36.38 32.57

that proposed technique gives best/or same PSNR values as compared to all three methods for different images. It is well known that the PSNR is not always a good measure to reflect the subjective image quality, even though it is one of the most popular criteria employed in image processing. In the last three decades, a great deal of effort has gone into the development of quality assessment methods that take advantage of known characteristics of the human visual system (HVS). Very recently Wang et al. evolved a measure of structural similarity (SSIM) that compares local patterns of pixel intensities after they have been normalized for luminance and contrast [28]. It is based on the assumption that the HVS is highly adapted to extract structural information from the viewing field. 4.1. Mean structure similarity (MSSIM): Mathematical definition Suppose a and b are two nonnegative image signals, if one of the signals is considered to have perfect quality, then the similarity measure can be used as a quantitative measurement of the quality of the second signal and is computed as SSIM(a, b) =

(2μa μb + C1 )(2σab + C2 ) , (μ2a + μ2b + C1 )(σa2 + σb2 + C2 )

(14)

where μa , μb and σa , σb are mean intensities and standard deviations for a and b, respectively. C1 and C2 are constants. In discrete form σab can be estimated as

234

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

1  (ai − μa )(bi − μb ). N −1 N

σab =

(15)

i=1

For image quality assessment, it is useful to apply the SSIM index locally rather than globally. The local statistics are computed within a local w × w square window, which moves pixel-by-pixel over the entire image. At each step, the local statistics and SSIM index are calculated within the local window. In practice, a single overall quality measure of the entire image is required. A mean SSIM (MSSIM) index to evaluate the overall image quality is computed as MSSIM(A, B) =

M 1  SSIM(ai , bi ), M

(16)

i=1

where A and B are the original and reconstructed images respectively; ai and bi are the image contents at the ith local window; and M is the number of local windows of the image. For the two identical images the mean structural similarity (MSSIM) index is equal to one. This, being an objective method for assessing perceptual image quality has been used in present evaluation. In the present study, 11 × 11 window size has been used as proposed by [28]. In addition, the extent of reduction of blocking artifacts by the proposed method is measured by a new index, the block boundary measure (BBM) defined below. 4.2. Block boundary measure (BBM): Mathematical definition This index is used here to measure the quantum of blocking artifact at block boundaries. Given an image ⎧ ⎫ 0,0 0,1 0,tb−1 bi,j bi,j . . . bi,j ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ 1,0 1,1 1,tb−1 ⎪ ⎪ ⎨ ⎬ bi,j bi,j . . . bi,j , f= .. ⎪ ⎪ . ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ tb−1,0 tb−1,1 ⎭ tb−1,tb−1 ⎪ bi,j bi,j . . . bi,j p,q

where bi,j represents (i, j )th pixel intensity value in (p, q)th block, tb is the number of blocks in the image along horizontal or vertical direction as in Fig. 3. For the vertical boundaries between blocks of 8 × 8 pixels, it is computed as follows: tb−2  tb−2   p,q  1 b − bp,q+1 , BBMver = 7,j 0,j (tb − 1)2

∀j ∈ [0, 7],

(17)

p=0 q=0

where  ·  is the l2 norm. Similarly, the BBM for the horizontal block boundaries can be obtained as BBMhor =

tb−2  tb−2   p,q  1 b − bp,q+1 , i,7 i,0 2 (tb − 1) p=0 q=0

∀i ∈ [0, 7].

(18)

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

235

Fig. 3. Dividing the image into blocks of size 8 × 8. Table 3 Comparison of MSSIM index for different post processing techniques applied to different JPEG compressed images Image

Bit rate (bpp)

JPEG

Method [9]

Method [22]

Method [24]

Proposed method

Lena Boat Peppers Bird Test pattern

0.156 0.247 0.169 0.157 0.351

0.8108 0.8627 0.8423 0.9219 0.8959

0.8217 0.8842 0.8670 0.9417 0.9214

0.7951 0.8581 0.8271 0.9190 0.8945

0.8095 0.8570 0.8565 0.9521 0.9322

0.8249 0.8754 0.8692 0.9431 0.9381

4.3. Evaluation by MSSIM index The numerical values of the MSSIM index for different images are presented in Table 3. It is observed that there is an improvement in MSSIM index for almost all different images as compared to other blocking artifact reduction techniques. The reduction in MSSIM index is observed for the post processing technique [22]. This indicates that this method is not retaining the structural similarity as of the original images. Where as the method given by [24] showed improvement except for Lena and Boat images. The results obtained from the method [9] are almost comparable with that of the proposed method. 4.4. Evaluation by BBM index The BBM index values for vertical block boundary and horizontal block boundary for different post processed JPEG compressed images by different blocking artifact reduction methods are presented in Table 4. It is observed that after removing the blocking artifacts the value of BBM index becomes very close to that of the original image. This shows that the effect of the blocking

236

Image

Bit rate (bpp)

Original

JPEG

Method [9]

Method [22]

Method [24]

Proposed method

BBMhor

BBMver

BBMhor

BBMver

BBMhor

BBMver

BBMhor

BBMver

BBMhor

BBMver

BBMhor

BBMver

Lena Boat Peppers Bird Test pattern

0.156 0.247 0.169 0.157 0.351

2.21 2.15 1.72 0.79 2.90

1.72 1.96 1.55 0.65 2.91

3.16 3.95 3.69 1.72 5.86

2.84 3.89 4.46 1.61 5.77

2.15 3.29 2.58 1.56 4.72

1.96 3.31 3.27 1.47 4.75

3.41 4.04 3.87 1.87 5.97

3.09 4.01 4.64 1.72 5.90

1.31 1.63 1.60 0.81 3.26

1.14 1.69 1.88 0.74 3.25

1.63 2.03 1.84 1.10 3.33

1.41 2.06 2.18 0.99 3.45

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

Table 4 Comparison of BBM indices for different post processing techniques applied to different JPEG compressed images

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

(a)

(b)

(c)

(d)

237

Fig. 4. Subjective quality comparison of the Lena image compressed at 0.298 bpp and post filtered by different methods. (a) original and (b) decoded image. Images processed by: (c) method of [9], (d) method of [22], (e) method of [24], and (f) proposed method.

artifacts has been effectively reduced by the technique. As evident from the Table 3 the method given by [24] gave best performance in terms of reduction of blocking artifacts. But this technique fails in terms of PSNR and MSSIM index. The technique suggested by [9], which gives comparable performance with proposed technique in terms of PSNR and MSSIM index could not also reduce the artifacts to greater extent. Similar results have been obtained for the technique proposed by [22]. Hence it is observed that the proposed method reduces the artifacts to great extent while preserving the image information, which is important from human observer’s point of view. In Fig. 4, the central portion of respective Lena images processed using different post filters are presented to show the visual quality. In the images processed by postprocessed

238

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

(e)

(f) Fig. 4. (continued)

methods [9] and [22] (i.e., Figs. 4c and 4d), blocking artifacts are not sufficiently reduced. This has also been indicated in Table 3 with high values for BBM indices. Although the method proposed by [24] reduces the blockiness more sufficiently in smooth regions, it blurs the details or retains blocking artifacts perceptible in some high-activity areas probably due to non-consideration of block activity based on HVS as shown in Fig. 4e. The proposed method yields good results, achieving favorably both aspects of reducing blocking artifacts and preserving image details. The decoded image of Lena by the proposed in Fig. 4f is visually more pleasing than the images produced by other methods. As well, the best values of PSNR, MSSIM, and BBM have been obtained using the proposed postprocessing technique for low-bit-rate transform coded images. Figure 5 shows the zoomed portion of Lena image near eye area before and after the reduction of blocking artifacts. Face region below eye belongs to the smooth region and eyebrow portion is the non-smooth region. For smooth regions of face blocking artifacts have been reduced by method in Section 3.1 while for the eyebrow region, an edge preserving smoothing filter works. Similar observations are true for Peppers image shown in Fig. 6. From these images it is very clear and visible that blocking artifacts have reduced. 4.5. Complexity The method proposed by [9] computes global and local edge map of image. This needs the convolution of the image with Sobel kernel to find the gradient image. Then 1-D filter is applied to reduce the staircase noise for all points on the global edge map. After that the resultant image is convolved 2-D kernel of 5 × 5 window. This whole process involves many multiplications and finally the complexity is further increased when the corner outlier detection and replacement algorithm is applied. The method given in [22] considers the each edge block to be classified into three types. Here also first operation is computation of gradient image to get edge information. For

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

(a)

239

(b)

(c) Fig. 5. Subjective quality comparison of the Lena image compressed at 0.298 bpp and post filtered by proposed method. (a) JPEG compressed image, (b) zoomed portion near eye area with blocking artifacts, (c) zoomed portion near eye area after reduction of blocking artifacts with proposed method. (Note: Portion below eye belongs to the smooth regions and the above it is the non-smooth region. For the eyebrow region edge preserving smoothing filter works and for smooth region blocking artifacts has been reduced by method in Section 3.1.)

Type I edge blocks, no further processing is done. For Type II, the initial process involves at most 16 multiplications. Finally, DCT domain filtering is applied to edge blocks of Type II or Type III. Here filtering window size is taken to be 3 × 3 that involve 8 shifted blocks for each block to be processed. This makes the method to be complex as it involves so many multiplications and the application of quantization constraint to each processed block further enhances the complexity. The proposed method involves only first row of two blocks for reducing the blocking artifacts in the smooth regions and the 5 × 5 widow sigma filter is applied for non-smooth regions. As the number of complex operations of multiplication of the proposed method

240

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

(a)

(b)

(c)

(d)

Fig. 6. Subjective quality comparison of the Peppers image compressed at 0.202 bpp and post filtered by different methods. (a) original and (b) decoded image. Images processed by: (c) method of [9], (d) method of [22], (e) method of [24], and (f) proposed method.

is less as compared to method [9] and [22], it makes the suggested method less complex. Moreover the processing time of the proposed method is less than that of [9] and [22] where as it is almost same for [24]. Thus the proposed method is better than other three methods in terms of complexity and processing time.

5. Conclusions An attempt has been made in this paper to remove the annoying blocking artifacts from low-bit-rate JPEG compressed images. In the proposed technique the blocking artifacts are modeled as 2-D step functions. A frequency domain algorithm extracts all the parameters

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

(e)

241

(f) Fig. 6. (continued)

needed to detect the presence of blocking artifacts by using visual perception along with the block statistics. The boundary regions between blocks are identified as smooth and non-smooth regions. To demonstrate the performance of the proposed algorithm, PSNR, SSIM index based on HVS perception and a new index “block boundary measure (BBM)” have been used. It is found that there is a significant improvement in the perceptual quality of the JPEG compressed images after removal of blocking artifact by the proposed method. The values of BBM index, both for vertical and horizontal boundaries between the blocks, are also seen to move very close to the corresponding values of the original images after application of the new algorithm. Thus, the three evaluation methods establish the efficacy of the proposed technique of detecting and reducing blocking artifacts in different images compressed using JPEG. Due to the low computational requirement the method can be integrated into real time image/video applications, that process image/video in the DCT domain, for online quality monitoring.

Acknowledgments The authors are thankful to the Department of Electrical Engineering, Indian Institute of Technology Roorkee, for providing the facilities to carry out this work. Sukhwinder Singh is also grateful to Sant Longowal Institute of Engineering and Technology, Longowal (PB) for sponsoring him for doctoral research work, and to the Ministry of Human Resources and Development, Government of India for providing financial assistance.

References [1] G.K. Wallace, The JPEG still-picture compression standard, Commun. ACM 34 (1991) 30–44.

242

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

[2] J.L. Mitchell, W.B. Pennebaker, C.E. Fogg, D.J. LeGall, MPEG Video Compression Standard, Chapman & Hall, New York, 1997. [3] H.S. Malvar, D.H. Staelin, The LOT: Transform coding without blocking effects, IEEE Trans. Accoust. Speech Signal Process. ASSP-37 (1989) 553–559. [4] H.S. Malvar, Biorthogonal and nonuniform lapped transforms for transform coding with reduced blocking and ringing artifacts, IEEE Trans. Signal Process. 46 (1998) 1043–1053. [5] H. Reeve, J. Lim, Reduction of blocking effect in image coding, Proc. ICASSP 83 (1983) 1212–1215. [6] B. Ramamurthi, A. Gersho, Nonlinear space-variant postprocessing of block coded images, IEEE Trans. Accoust. Speech Signal Process. ASSP-34 (1986) 1258–1268. [7] Y.-F. Hsu, Y.-C. Chen, A new adaptive separable median filter for removing blocking effects, IEEE Trans. Consum. Electron. (1993) 510–513. [8] T. Meier, K.N. Ngan, G. Crebbin, A region-based algorithm for enhancement of images degraded by blocking effects, Proc. IEEE Tencon 1 (1996) 405–408. [9] Y.L. Lee, H.C. Kim, H.W. Park, Blocking effect reduction of JPEG images by signal adaptive filtering, IEEE Trans. Image Process. 7 (1998) 229–234. [10] J. Chou, M. Crouse, K. Ramchadran, A simple algorithm for removing blocking artifacts in block transform coded images, IEEE Signal Process. Lett. 5 (1998) 33–35. [11] A. Zakhor, Iterative procedure for reduction of blocking effects in transform image coding, IEEE Trans. Circuits Syst. Video Technol. 2 (1992) 91–95. [12] Y. Yang, N.P. Galatsanos, A.K. Katsaggelos, Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images, IEEE Trans. Circuits Syst. Video Technol. 3 (1993) 421–432. [13] H. Paek, R.-C. Kim, S.-U. Lee, On the POCS-based postprocessing technique to reduce the blocking artifacts in transform coded images, IEEE Trans. Circuits Syst. Video Technol. 8 (1998) 358–367. [14] H. Paek, R.-C. Kim, S.-U. Lee, A DCT-based spatially adaptive post processing technique to reduce the blocking artifacts in transform coded images, IEEE Trans. Circuits Syst. Video Technol. 10 (2000) 36–41. [15] T.P. Rourke, R.L. Stevenson, Improved image decompression for reduced transform coding artifacts, IEEE Trans. Circuits Syst. Video Technol. 5 (1995) 490–499. [16] J. Luo, C.W. Chen, K.J. Parker, T.S. Huang, Artifact reduction in low bit rate DCT-based image compression, IEEE Trans. Image Process. 5 (1996) 1363–1368. [17] T. Meier, K.N. Ngan, G. Crebbin, Reduction of blocking artifacts in image and video coding, IEEE Trans. Circuits Syst. Video Technol. 9 (1999) 490–500. [18] F.X. Coudoux, M. Gzalet, P. Corlay, Reduction of blocking effect in DCT-coded images based on a visual perception criterion, Signal Process. Image Commun. 11 (1998) 179–186. [19] S. Minami, A. Zakhor, An optimization approach for removing blocking effects in transform coding, IEEE Trans. Circuits Syst. Video Technol. 5 (1995) 74–82. [20] G. Lakhani, N. Zhong, Derivation of prediction equations for blocking effect reduction, IEEE Trans. Circuits Syst. Video Technol. 9 (1999) 415–418. [21] G.A. Triantafyllidis, D. Tzovaras, M.G. Strintzis, Blocking artifact detection and reduction in compressed data, IEEE Trans. Circuits Syst. Video Technol. 12 (2002) 877–890. [22] S. Liu, A.C. Bovik, Efficient DCT-domain blind measurement and reduction of blocking artifacts, IEEE Trans. Circuits Syst. Video Technol. 12 (2002) 1139–1149. [23] B. Zeng, Reduction of blocking effect in DCT-coded images using zero-masking techniques, Signal Process. 79 (1999) 205–211. [24] Y. Luo, R.K. Ward, Removing the blocking artifacts of block-based dct compressed images, IEEE Trans. Image Process. 12 (2003) 838–842. [25] D.J. Granrath, The role of human visual models in image processing, Proc. IEEE 69 (1981) 552–561. [26] S.A. Karunasekera, N.G. Kingsbury, A Distortion measure for blocking artifacts in images based on human visual sensitivity, IEEE Trans. Image Process. 4 (6) (1995) 713–724. [27] J.-S. Lee, Digital image smoothing and the sigma filter, Comput. Vision Graph. Image Process. 24 (1983) 255–269. [28] Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: From error visibility to structure similarity, IEEE Trans. Image Process. 13 (2004) 600–612.

S. Singh et al. / Digital Signal Processing 17 (2007) 225–243

243

Sukhwinder Singh obtained his B.Tech. (Computer Eng.) degree from GNDU Amritsar (Punjab) in 1991 and M.E. (Computer Sci. & Eng.) Hons. degree from Thapar Institute of Engineering and Technology, Patiala in 1999. He joined the Department of Computer Sci. & Eng. at Sant Longowal Institute of Engineering and Technology, Longowal (Punjab) in 1992. Presently he is serving as Asst. Professor in the Department of Computer Sci. & Eng. He is the life member of ISTE, member of other professional bodies. He is now working towards the Ph.D. degree at the Electrical Engineering Department of the Indian Institute of Technology, Roorkee. His research interests include Medical Image Compression and Analysis, Telemedicine and Network Security. Dr. Vinod Kumar obtained his B.Sc. (Electrical Engineering) Hons degree from Punjab University in 1973, M.E. (Measurement & Instrumentation) Hons and Ph.D. degrees from the University of Roorkee, Roorkee in 1975 and 1984, respectively. He joined the Electrical Engineering Department of the University of Rookee (presently, IIT Roorkee) in 1975 and is presently Professor in Electrical Engineering Department of IIT Roorkee. He has guided 12 doctoral and more than 70 Master’s thesis and has more than 125 research publications in internationally reputed journals and conference proceedings. He has undertaken large number of consultancy and sponsored projects from industries and government departments. He holds membership of many professional bodies. He is a fellow of the Institute of Engineers (I), Institution of Electronics and Telecommunication Engineers and Biomedical Engineering Society of India. He is a Senior Member of IEEE, USA. He has many honors and awards to his credit, namely IETE K.S. Krishna Memorial Award, Khosla Cash Award & Prize, Khosla Cash Prize, Khosla Annual Research Prize, Certificate of Merit for research papers by Institution of Engineers (I). He has also conducted several courses, workshops for the benefit of faculty and field engineers. Dr. Kumar served the institute as Associate Dean Academic, Director/Coordinator AVRC and Coordinator Information Super Highway Centre. He is presently Head Continuing Education Centre. His areas of interest are Measurement and Instrumentation, Medical Instrumentation, Medical Image Processing, Digital Signal Processing and Telemedicine. Dr. H.K. Verma is Professor of Instrumentation in the Department of Electrical Engineering at the Indian Institute of Technology, Roorkee. He was Head of this Department from September 1991 to September 1994, Dean of Sponsored Research and Industrial Consultancy of the Institute from May 2000 to June 2003 and Dean of Faculty of the Institute from August 2003 to August 2004. Born in 1946, H.K. Verma graduated in Electrical Engineering in 1967 from University of Jodhpur and obtained Master of Eng. and Ph.D. degrees in 1969 and 1977, respectively, from the University of Roorkee. He has been at the Electrical Eng. Department of the University of Roorkee/Indian Institute of Technology Roorkee since September 1969 continuously except for a short spell of two years, 1980 to 1982, when he worked as R&D Manager of a public limited company. Prof. Verma has published over 150 research papers and guided 11 Ph.D. theses and 89 M.E./M.Tech. dissertations. He is member of several professional bodies. Dr. Verma’s current research interests include intelligent, distributed and biomedical instrumentation. He is deeply involved in industrial/professional consultancy in the areas of instrumentation, hydroelectric power and egovernance.