- Email: [email protected]
SNLM: A switching non-local means filter for removal of high density salt and pepper noise
Scientia Iranica D (
)
( ),
–
Sharif University of Technology Scientia Iranica Transactions D: Computer Science & Engineering and Electrical Engineering www.sciencedirect.com
SNLM: A switching non-local means filter for removal of high density salt and pepper noise M. Nasri ∗ , S. Saryazdi, H. Nezamabadi-pour Department of Electrical Engineering, Shahid Bahonar University of Kerman, Kerman, P.O. Box 76169-133, Iran Received 28 February 2012; revised 1 July 2012; accepted 23 December 2012
KEYWORDS Image denoising; Non local means filter; High-density salt and pepper noise.
Abstract A Switching Non-Local Means (SNLM) filter is presented for high-density salt and pepper noise reduction. Firstly, the impulse noises are detected, based on the fact that their values must be the extreme gray-level of the image. Then, at the filtering stage, the noise-free pixels remain unchanged and noisy pixels are restored using a modified non-local means filter. However, to calculate the weights of the filter, only noise-free pixels are considered. It means that in a search window around the noisy pixel, some small patches are taken into account around noise-free pixels and the similarity between these patches and the central patch determines the weights. Experimental results show that the proposed method can provide better performance than many of the existing impulse denoising methods in high-density impulse noise in terms of PSNR, and MAE. © 2013 Sharif University of Technology. Production and hosting by Elsevier B.V. All rights reserved.
1. Introduction IMAGES are frequently contaminated with different types of noise in acquisition, storage, or transmission. An important noise type that mixes with an original image neither in an additive manner nor multiplicative is impulse noise [1]. As a list of problems that may lead to this type of noise, we have: Quick transients, such as faulty switching occurring during imaging [2], malfunctioning camera sensor pixels and faulty memory locations in storage [3], sensor motion during exposure [4], timing errors in analog-to-digital conversion, external disturbance in a noisy environment, and transmitting in the noisy channel [5], Erroneous synchronization in digital recording [6] and bit errors in transmission [7]. In this noise type, some image pixels are replaced with noise values that can be the extreme intensity values of the original image (salt and pepper impulse noise (SNP)) or any values in the dynamic range of the image (Random-Valued impulse Noise (RVN)).
∗
Corresponding author. Tel.: +98 341 323 5900; fax: +98 341 323 5900. E-mail addresses: [email protected] (M. Nasri), [email protected] (S. Saryazdi), [email protected] (H. Nezamabadi-pour). Peer review under responsibility of Sharif University of Technology.
The well known standard median filter is the first method in impulse reduction, and it is still in the core of mostrecent methods [8–11]. But, its performance in noise-reduction decreases as noise density becomes high, since it performs identically on all image pixels and leads to the blurring of image details. Some modifications of this filter, incorporating a switching strategy, leads to an improvement in noise reduction and detail preservation of images simultaneously [8,12]. The switching framework consists of detection and filtering stages; the pixels that are labeled noise-free in the detection stage remain unchanged and the noisy-pixels are altered differently in the filtering process [12]. The impulse reduction methods can be categorized into low to medium density (<70%) [13] and high-density impulse noise (HDIN) (>70%) reduction [7,9,10]. When an image encounters more than one impulse fault type in acquisition, storage or transmission, it may be polluted with HDIN. HDIN is a challenging and most important task, since, in some cases, up to 90% of the image pixels are missing and must be restored based upon the remaining pixels of the image. In [9], a DecisionBased Algorithm (DBA) is proposed by Srinivasan and Ebenezer. In this method, for cases where the median of the 3 × 3 window around a noisy pixel is also a noisy value in HDIN, the neighboring pixel is used for replacement. Nevertheless, this repeated replacement results in artifacts in the image. In [7], a modified decision based unsymmetric trimmed median filter (MDBUTMF) is proposed for removal of HDIN. In this switching method, if the 3 × 3 window around a noisy pixel contains all
1026-3098 © 2013 Sharif University of Technology. Production and hosting by Elsevier B.V. All rights reserved. doi:10.1016/j.scient.2013.01.001
2
M. Nasri et al. / Scientia Iranica, Transactions D: Computer Science & Engineering and Electrical Engineering (
)
( ),
–
Figure 1: Block diagram of the SNLM method and the switching concept.
0’s and 255’s, the mean value of all elements of the window replaces the noisy pixel. In this case, this mean value has no relation to the original pixel. So, the restored image contains several noisy spots in HDIN. The main shortcoming of all these methods is that they only consider the local information of the image and discard the correlation between image pixels globally in natural images. This local information is inadequate for yielding satisfactory results, particularly in HDIN. In only a few methods in the literature, the global image information is exploited in the impulse noise reduction [11,14]. In [11], a global-local noise detector is proposed, but the filtering stage is still local and based on an adaptive median filter. Impulse noise reduction, based on LongRange Correlation (LRC), is proposed by Wang et al., which considers some remote region’s information rather than local information [14]. One of the most important and powerful methods that considers all the image information to restore a noisy image is the non-local means (NLM) filter. The NLM method, first proposed by Buades et al. [15], shows superior results in Gaussian noise removal, and it is at the core of many additive noise reduction methods over the last five years [16,17]. In this paper, to consider global information of the image in impulse noise reduction, the basic NLM filter is integrated to a switching framework, and a Switching Non-Local Means filter (SNLM) is proposed for removal of impulse noise, especially in HDIN. In the first stage of the proposed method, the noisy pixels are detected, based on the fact that their values must be the extreme gray level of the image. Then, the image is restored, based on noise-free pixels in a non-local manner. The organization of this paper is as follows. After a brief review on the non-local means denoising method in Section 2, the proposed algorithm (SNLM) is described in Section 3. Section 4 describes the simulation and experimental results and, finally, the concluding marks are drawn in Section 5. 2. Non-local means denoising filter NLM denoising is based on the fact that for a given image, X , the filtered value, Xˆ (i, j), at pixel (i, j) is obtained as a weighted average of all the pixels in the image (or a search window, ΩS , around the pixel), that is: Xˆ (i, j) =
w ((i, j), (k, l)) · X (i, j) ,
(1)
∀k,l∈Ωs
0 ≤ w((i, j) , (k, l)) ≤ 1
w((i, j), (k, l)) = 1.
(2)
∀i,j∈Ωs
The weights, w((i, j), (k, l)), are based on the similarity between the neighborhoods, Ni,j and Nk,l , of pixels (i, j) and (k, l), where NP is a square sub-image window centered pixel, p, with radius RP . These weights are calculated as: K Ni,j − Nk,l 2 /h
w ((i, j) , (k, l)) =
∀k,l∈Ωs
K Ni,j − Nk,l 2 /h
.
(3)
K (x) is kernel function and the default choice is the Gaussian kernel with zero mean and standard deviation, σ , that is K (x) = exp −x2 /2σ 2 . h is exponential decay control and ∥·∥2 is the Euclidean norm. 3. The proposed SNLM method The proposed method consists of two different stages, detection and filtering. The output of the detection stage is a binary noise map matrix (b) with 1’s representing noisy pixels and 0’s representing noise-free pixels. As shown in Figure 1, image pixels enter the filtering stage with a switch element that switches between noisy and noise-free cases and eventually outputs restored image after the filtering operation. The algorithm of the SNLM method is as follows: Stage 1-Detection: The proposed method can be used in denoising SNP and RVN cases, but with different detection strategies. There are effective detection procedures in both noise types. In the SNP case, BDND (boundary discriminative noise detection) [18] and simple detection [7,9] and, in RVN, DWM (directional median filter) [19] and ACWMF (adaptive center weighted median filter) [20] are examples of effective impulse detectors. The main focus of this paper is SNP noise, and in this case, noisy pixels must be the extreme gray-level values of the image, and noise-free pixels must lie between these two extreme values. It means that for an 8-bit monochrome image, the noisy pixels must be either 0 or 255. So, we mark them as ‘‘1’’ in the binary noise map. The other elements of this matrix are filled with ‘‘0’’ [7,9]. By combining the proposed SNLM method with one of the RVN detectors [19,20], this method can be applied on RVN effectively. However, this modification does not change the proposed filtering stage of the method. Stage 2-Filtering: Similar to other switching methods, the noise-free pixels remain unchanged. Noisy pixels (pixels with mark ‘‘1’’ in the binary noise map) are altered as follows: (a) Two different windows are considered around each noisy pixel; a bigger window with radius RS , which is called the search window, and a smaller window with radius Rc , which is called the central patch. (b) Around each noise-free pixel in the search window, consider a window with radius Rc that is called the remote patch. (c) The restored value of the pixel is obtained as the weighted sum of all noise-free pixels in the search window. The weights are based on the similarity between the central patch and remote patches. However, unlike the NLM method, we do not have all pixels in each patch, as some of them are noisy pixels, and we must discard them. The similarity is based on the Euclidean norm between noisefree pixels in two patches, as in Eq. (4). Note that pixel (i, j) is the pixel to be restored and pixel (k, l) is a noise-free pixel in the search window. The kernel matrix in Eq. (4) is a Gaussian kernel matrix, with size equal to 2∗ Rc + 1, and its standard
M. Nasri et al. / Scientia Iranica, Transactions D: Computer Science & Engineering and Electrical Engineering (
)
( ),
–
3
w ((i, j) , (k, l)) exp −d ((i, j) , (k, l)) /h′2
=
exp −d ((i, j) , (k, l)) /h′2
∀(k,l)∈Rs ,b(k,l)=0
.
(5)
(d) Finally, the restored value of the pixel is calculated as in Eq. (6). As shown in this equation, when the binary noise map b(i, j) equals zero or, in other words, the pixel is noisefree, the original pixel is retained. However, when the pixel is noisy, the restored value is calculated as a weighted sum of all non-noisy pixels in the search window:
Xˆ (i, j) =
Figure 2: The SNLM method graphical description.
deviation equals 1. Xi,j and bi,j are (i, j) pixels of the noisy image and the binary noise map, respectively. Based on the defined similarity in Eq. (4), the weights are calculated as Eq. (5). d((i,j),(k,l)) =
Rc Rc
kernel(Rc + m, Rc + n)
w ((i, j), (k, l)) · X (i, j)
∀(k,l)∈Ωs ,b(k,l)=0
X (i, j)
(6)
if b(i, j) = 1 if b(i, j) = 0.
Figure 2 shows this procedure in denoising a noisy pixel. The central patch and four typical remote patches in the binary noise map are labeled, ‘‘o’’ and ‘‘a’’–‘‘d’’, respectively. The values of patches ‘‘o’’ and ‘‘b’’ are shown in the image. Only the pixels are used in the calculation of similarity between two patches that are noise-free in both patches. For instance, only two data sets, (192, 164) and (210, 168), are used in calculation of the similarity between ‘‘o’’ and ‘‘b’’ patches. 4. Simulation results
m=−Rc n=−Rc
· 1 − bi+m,j+n · 1 − bk+m,l+n 2 · xi+m,j+n − xk+m,l+n
d
(4)
To evaluate the performance of the proposed method, several standard images, such as Lena, girl and Baboon, are used
a
b
c
e
f
g
h
i
Figure 3: Simulation results of different methods. (a) Original ‘‘Baboon’’ image; (b) Noisy with 90% SNP; (c) Noisy with 70% SNP; (d) PSMF method; (e) GP method; (f) LRC method; (g) DBA method; (h) MDBUTMF; and (i) Proposed method (SNLM). 2nd row shows the output of different methods in 90% noise and 3rd row shows the output of different methods in 70% noise.
4
M. Nasri et al. / Scientia Iranica, Transactions D: Computer Science & Engineering and Electrical Engineering (
d
a
b
c
e
f
g
h
)
( ),
–
i
Figure 4: Simulation results of different methods. (a) Original ‘‘girl’’ image; (b) Noisy with 90% SNP; (c) Noisy with 70% SNP; (d) PSMF method; (e) GP method; (f) LRC method; (g) DBA method; (h) MDBUTMF; and (i) Proposed method (SNLM). 2nd row shows the output of different methods in 90% noise and 3rd row shows the output of different methods in 70% noise.
with different SNP impulse noise levels varying from 10% to 90% with increment 10%. The performance criteria used in the paper is PSNR and MAE, as defined in Eqs. (7) and (9), respectively, [1]:
PSNR = 10 log10
2552 MSE
,
M N 2 1 MSE = I (i, j) − ˆI (i, j) , MN i=1 j=1
MAE =
M N 1 I (i, j) − ˆI (i, j) . MN i=1 j=1
(7)
a
b
(8) Figure 5: Comparison of LRC and proposed method (SNLM) in denoising Lena eyes with 70% SNP noise. (a) LRC; and (b) SNLM.
(9)
The size of the original image, I, is (M ×N ) and the reconstructed image is ˆI. In order to compare the results, several recent methods are implemented in MATLAB, with the mentioned parameters in their papers and applied to noisy images. These methods are PSMF [10], GP [21], LRC [14], DBA [9], and MDBUTMF [7]. The SNLM method has three different parameters: search window’s radius, RS , central and remote patches radius, Rc , and control parameter, h′ . Although the results of noise reduction can be improved by tuning these parameters, due to noise density and image characteristics, for simplicity and as our focus is on the HDIN case, we fix these parameters in 7, 2, and 10, respectively. PSNR and MAE results of these methods in denoising Lena and girl images are shown in Tables 1 and 2. As can be seen, the PSNR result of the SNLM method is somehow lower than DBA, MDBUTMF, and LRC in low density (<30%) impulse noise, but it is higher than PSMF and GP and has acceptable results. The SNLM outperforms all methods in terms of PSNR and MAE in
a wide range of HDIN, and it can effectively suppress impulse noise. For instance, its PSNR is more than two dB higher than other methods in 90% SNP noise. Figures 3 and 4 show the results of denoising Baboon and girl images with two different noise levels. It can be seen that PSMF and GP filters output images with severe noisy spots, MDBUTMF and DBA method has artifacts, and both LRC and SNLM methods have acceptable visual appearances, but in LRC, some small fluctuations are seen in the image edges. To better visually compare the SNLM and LRC methods, Figure 5 shows these methods in denoising the zoomed Lena image with 70% SNP noise. 5. Conclusion and future works In this paper, a Switching Non-Local Means (SNLM) denoising method is proposed for high density impulse noise reduction. The basic Non-Local Means (NLM) filter exploits all the pixels in the restoration of the image and yields good results in Gaussian noise reduction. By using this filter in conjunction with a switching strategy, a promising result is obtained
M. Nasri et al. / Scientia Iranica, Transactions D: Computer Science & Engineering and Electrical Engineering ( Table 1: Comparison of PSNR and MAE values of different methods for ‘‘Lena’’ image noise reduction at different noise levels. Noise density (%)
‘‘Lena’’ – PSNR (dB) PSMF
GP
LRC
DBA
MDBUTMF
10 20 30 40 50 60 70 80 90
37.26 32.84 28.95 24.82 20.67 12.25 9.96 8.10 6.63
32.49 29.62 27.59 25.74 23.63 20.61 16.75 12.65 9.12
42.85 39.02 36.15 33.28 30.52 26.89 23.66 22.32 21.27
41.57 37.54 34.78 32.37 30.14 27.92 25.62 23.14 19.71
42.42 37.95 35.05 32.16 29.47 27.46 22.10 17.89 18.48
10 20 30 40 50 60 70 80 90
0.58 1.17 2.11 3.77 7.30 29.94 48.34 71.80 98.60
0.75 1.41 2.23 3.37 5.30 9.53 19.20 39.12 70.73
0.39 0.82 1.35 2.02 2.97 4.62 7.29 9.20 11.20
SNLM 39.25 36.65 35.13 33.66 31.96 29.93 27.75 25.61 23.47
‘‘Lena’’ – MAE 0.40 0.86 1.42 2.09 2.93 4.04 5.54 7.92 13.07
0.36 0.83 1.39 2.12 3.16 4.42 6.42 10.34 19.75
0.51 1.02 1.53 2.07 2.86 3.95 5.51 7.70 11.12
Table 2: Comparison of PSNR and MAE values of different methods for ‘‘girl’’ image noise reduction at different noise levels. Noise density (%)
10 20 30 40 50 60 70 80 90
‘‘girl’’ – PSNR (dB) PSMF
GP
LRC
DBA
MDBUTMF
39.53 34.36 30.52 26.66 21.76 17.03 10.00 8.12 6.71
32.63 29.58 27.81 26.32 24.40 21.52 17.60 13.14 9.31
45.32 41.70 38.87 36.12 33.12 29.95 26.98 25.70 24.46
44.64 40.75 38.18 35.70 33.52 31.19 28.80 26.06 22.13
45.29 41.29 38.21 36.07 33.89 31.58 28.92 25.50 20.93
43.20 40.23 38.30 36.25 34.62 32.78 30.76 28.87 26.61
SNLM
0.29 0.64 1.21 1.61 2.27 3.19 4.71 7.71 15.63
0.38 0.76 1.19 1.57 2.20 3.10 4.22 5.71 8.41
‘‘girl’’ – MAE 10 20 30 40 50 60 70 80 90
0.46 1.08 1.91 3.27 6.09 12.76 46.79 70.70 98.07
0.56 1.12 1.78 2.68 4.32 7.98 16.72 36.04 67.96
0.32 0.67 1.13 1.70 2.52 3.81 5.70 7.14 8.95
0.33 0.69 1.20 1.62 2.26 3.11 4.27 6.25 10.96
in high-density impulse noise reduction. The results of denoising different standard images show that the proposed method outperforms several recent methods in this area of research. As future work, there are fast and efficient variants of the NLM method that can be used. Moreover, this paper can be extended in mixed Gaussian and impulse noise reduction. Another future work may be optimizing the tuning parameters of the proposed method to remove impulse noise effectively for all noise densities. References [1] Bovik, A., Handbook of Image and Video Processing, Academic Press, New York (2000). [2] Gonzalez, R.C. and Woods, R.E., Digital Image Processing, 2nd Edn., Prentice Hall Press (2002). [3] Dong, Y., Chan, R.H. and Xu, S. ‘‘A detection statistic for randomvalued impulse noise’’, IEEE Transactions on Image Processing, 16(4), pp. 1112–1120 (2007).
)
( ),
–
5
[4] Civicioglu, P. ‘‘Removal of random-valued impulsive noise from corrupted images’’, IEEE Transactions on Consumer Electronics, 55(4), pp. 2097–2104 (2009). [5] Toh, K.K.V. and Isa, N.A.M. ‘‘Cluster-based adaptive fuzzy switching median filter for universal impulse noise reduction’’, IEEE Transactions on Consumer Electronics, 56(4), pp. 2560–2568 (2010). [6] Awad, A.S. ‘‘Standard deviation for obtaining the optimal direction in the removal of impulse noise’’, IEEE Signal Processing Letters, 18(7), pp. 407–410 (2011). [7] Esakkirajan, S., Veerakumar, T., Subramanyam, A.N. and PremChand, C.H. ‘‘Removal of high density salt and pepper noise through modified decision based unsymmetric trimmed median filter’’, IEEE Signal Processing Letters, 18(5), pp. 287–290 (2011). [8] Akkoul, S., Ledee, R., Leconge, R. and Harba, R. ‘‘A new adaptive switching median filter’’, IEEE Signal Processing Letters, 17(6), pp. 587–590 (2010). [9] Srinivasan, K.S. and Ebenezer, D. ‘‘A new fast and efficient decisionbased algorithm for removal of high-density impulse noises’’, IEEE Signal Processing Letters, 14(3), pp. 189–192 (2007). [10] Zhou, W. and Zhang, D. ‘‘Progressive switching median filter for the removal of impulse noise from highly corrupted images’’, IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 46(1), pp. 78–80 (1999). [11] Yuan, S.-Q. and Tan, Y.-H. ‘‘Impulse noise removal by a global-local noise detector and adaptive median filter’’, Signal Processing, 86(8), pp. 2123–2128 (2006). [12] Sun, T. and Neuvo, Y. ‘‘Detail-preserving median based filters in image processing’’, Pattern Recognition Letters, 15, pp. 341–347 (1994). [13] Ko, S.J. and Lee, Y.H. ‘‘Center weighted median filters and their applications to image enhancement’’, IEEE Transactions on Circuits and Systems, 38(9), pp. 984–993 (1991). [14] Wang, Z. and Zhang, D. ‘‘Restoration of impulse noise corrupted images using long-range correlation’’, IEEE Signal Processing Letters, 5(1), pp. 4–7 (1998). [15] Buades, A., Coll, B. and Morel, J. ‘‘Image denoising by non-local averaging’’, IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP ’05, pp. 25–28 (2005). [16] Vignesh, R., Byung Tae, O. and Kuo, C.C.J. ‘‘Fast non-local means (NLM) computation with probabilistic early termination’’, IEEE Signal Processing Letters, 17(3), pp. 277–280 (2010). [17] Doré, V. and Cheriet, M. ‘‘Robust NL-means filter with optimal pixel-wise smoothing parameter for statistical image denoising’’, IEEE Transactions on Signal Processing, 57(5), pp. 1703–1716 (2009). [18] Ng, P.-E. and Ma, K.-K. ‘‘A switching median filter with boundary discriminative noise detection for extremely corrupted images’’, IEEE Transactions on Image Processing, 15(6), pp. 1506–1516 (2006). [19] Dong, Y. and Xu, S. ‘‘A new directional weighted median filter for removal of random-valued impulse noise’’, IEEE Signal Processing Letters, 14(3), pp. 193–196 (2007). [20] Chen, T. and Hong Ren, W. ‘‘Adaptive impulse detection using centerweighted median filters’’, IEEE Signal Processing Letters, 8, pp. 1–3 (2001). [21] Petrovic, N.I. and Crnojevic, V. ‘‘Universal impulse noise filter based on genetic programming’’, IEEE Transactions on Image Processing, 17(7), pp. 1109–1120 (2008).
Mehdi Nasri was born in Isfahan, Iran, in 1982. He received his B.S. degree in Biomedical Engineering from the University of Isfahan, Iran, in 2004, and received his M.S. degree in Electrical Engineering from Shahid Bahonar University of Kerman, Iran, in 2007, where he is currently pursuing his Ph.D. degree in Electrical Engineering. His research interests include: computer vision, image processing and soft computing.
Saeid Saryazdi received B.S. and M.S. degrees in Electrical Engineering from Isfahan University of Technology, Iran, in 1985 and 1987, respectively, and DEA and Ph.D. degrees in Electrical Engineering from Rennes1 University, France, in 1994 and 1997, respectively. In 1997, he joined the Department of Electrical Engineering at Shahid Bahonar University of Kerman as Assistant Professor, and was promoted to Associate Professor in 2007. From 2005 to 2006, he was a visiting professor at the École de Technologie Supérieure (University of Quebec), Montréal, QC, Canada. His research interests include: PDE based image denoising and inpainting, steganography, watermarking, and image retrieval. Hossein Nezamabadi-pour was born in Iran, in 1976. He received his B.S. degree in Electrical Engineering from Shahid Bahonar University of Kerman, Iran, in 1998, and his M.S. and Ph.D. degrees, also in Electrical Engineering, from Tarbait Modarres University, Tehran, Iran, in 2000 and 2004, respectively. In 2004, he joined the Department of Electrical Engineering at Shahid Bahonar University of Kerman, Iran, as Assistant Professor, and was promoted to Associate Professor in 2008, and Full Professor in 2012. Dr. Nezamabadipour is author and co-author of more than 250 peer reviewed journals and conference papers. His research interests include: soft computing, evolutionary computation, image processing and pattern recognition.
)
( ),
–
Sharif University of Technology Scientia Iranica Transactions D: Computer Science & Engineering and Electrical Engineering www.sciencedirect.com
SNLM: A switching non-local means filter for removal of high density salt and pepper noise M. Nasri ∗ , S. Saryazdi, H. Nezamabadi-pour Department of Electrical Engineering, Shahid Bahonar University of Kerman, Kerman, P.O. Box 76169-133, Iran Received 28 February 2012; revised 1 July 2012; accepted 23 December 2012
KEYWORDS Image denoising; Non local means filter; High-density salt and pepper noise.
Abstract A Switching Non-Local Means (SNLM) filter is presented for high-density salt and pepper noise reduction. Firstly, the impulse noises are detected, based on the fact that their values must be the extreme gray-level of the image. Then, at the filtering stage, the noise-free pixels remain unchanged and noisy pixels are restored using a modified non-local means filter. However, to calculate the weights of the filter, only noise-free pixels are considered. It means that in a search window around the noisy pixel, some small patches are taken into account around noise-free pixels and the similarity between these patches and the central patch determines the weights. Experimental results show that the proposed method can provide better performance than many of the existing impulse denoising methods in high-density impulse noise in terms of PSNR, and MAE. © 2013 Sharif University of Technology. Production and hosting by Elsevier B.V. All rights reserved.
1. Introduction IMAGES are frequently contaminated with different types of noise in acquisition, storage, or transmission. An important noise type that mixes with an original image neither in an additive manner nor multiplicative is impulse noise [1]. As a list of problems that may lead to this type of noise, we have: Quick transients, such as faulty switching occurring during imaging [2], malfunctioning camera sensor pixels and faulty memory locations in storage [3], sensor motion during exposure [4], timing errors in analog-to-digital conversion, external disturbance in a noisy environment, and transmitting in the noisy channel [5], Erroneous synchronization in digital recording [6] and bit errors in transmission [7]. In this noise type, some image pixels are replaced with noise values that can be the extreme intensity values of the original image (salt and pepper impulse noise (SNP)) or any values in the dynamic range of the image (Random-Valued impulse Noise (RVN)).
∗
Corresponding author. Tel.: +98 341 323 5900; fax: +98 341 323 5900. E-mail addresses: [email protected] (M. Nasri), [email protected] (S. Saryazdi), [email protected] (H. Nezamabadi-pour). Peer review under responsibility of Sharif University of Technology.
The well known standard median filter is the first method in impulse reduction, and it is still in the core of mostrecent methods [8–11]. But, its performance in noise-reduction decreases as noise density becomes high, since it performs identically on all image pixels and leads to the blurring of image details. Some modifications of this filter, incorporating a switching strategy, leads to an improvement in noise reduction and detail preservation of images simultaneously [8,12]. The switching framework consists of detection and filtering stages; the pixels that are labeled noise-free in the detection stage remain unchanged and the noisy-pixels are altered differently in the filtering process [12]. The impulse reduction methods can be categorized into low to medium density (<70%) [13] and high-density impulse noise (HDIN) (>70%) reduction [7,9,10]. When an image encounters more than one impulse fault type in acquisition, storage or transmission, it may be polluted with HDIN. HDIN is a challenging and most important task, since, in some cases, up to 90% of the image pixels are missing and must be restored based upon the remaining pixels of the image. In [9], a DecisionBased Algorithm (DBA) is proposed by Srinivasan and Ebenezer. In this method, for cases where the median of the 3 × 3 window around a noisy pixel is also a noisy value in HDIN, the neighboring pixel is used for replacement. Nevertheless, this repeated replacement results in artifacts in the image. In [7], a modified decision based unsymmetric trimmed median filter (MDBUTMF) is proposed for removal of HDIN. In this switching method, if the 3 × 3 window around a noisy pixel contains all
1026-3098 © 2013 Sharif University of Technology. Production and hosting by Elsevier B.V. All rights reserved. doi:10.1016/j.scient.2013.01.001
2
M. Nasri et al. / Scientia Iranica, Transactions D: Computer Science & Engineering and Electrical Engineering (
)
( ),
–
Figure 1: Block diagram of the SNLM method and the switching concept.
0’s and 255’s, the mean value of all elements of the window replaces the noisy pixel. In this case, this mean value has no relation to the original pixel. So, the restored image contains several noisy spots in HDIN. The main shortcoming of all these methods is that they only consider the local information of the image and discard the correlation between image pixels globally in natural images. This local information is inadequate for yielding satisfactory results, particularly in HDIN. In only a few methods in the literature, the global image information is exploited in the impulse noise reduction [11,14]. In [11], a global-local noise detector is proposed, but the filtering stage is still local and based on an adaptive median filter. Impulse noise reduction, based on LongRange Correlation (LRC), is proposed by Wang et al., which considers some remote region’s information rather than local information [14]. One of the most important and powerful methods that considers all the image information to restore a noisy image is the non-local means (NLM) filter. The NLM method, first proposed by Buades et al. [15], shows superior results in Gaussian noise removal, and it is at the core of many additive noise reduction methods over the last five years [16,17]. In this paper, to consider global information of the image in impulse noise reduction, the basic NLM filter is integrated to a switching framework, and a Switching Non-Local Means filter (SNLM) is proposed for removal of impulse noise, especially in HDIN. In the first stage of the proposed method, the noisy pixels are detected, based on the fact that their values must be the extreme gray level of the image. Then, the image is restored, based on noise-free pixels in a non-local manner. The organization of this paper is as follows. After a brief review on the non-local means denoising method in Section 2, the proposed algorithm (SNLM) is described in Section 3. Section 4 describes the simulation and experimental results and, finally, the concluding marks are drawn in Section 5. 2. Non-local means denoising filter NLM denoising is based on the fact that for a given image, X , the filtered value, Xˆ (i, j), at pixel (i, j) is obtained as a weighted average of all the pixels in the image (or a search window, ΩS , around the pixel), that is: Xˆ (i, j) =
w ((i, j), (k, l)) · X (i, j) ,
(1)
∀k,l∈Ωs
0 ≤ w((i, j) , (k, l)) ≤ 1
w((i, j), (k, l)) = 1.
(2)
∀i,j∈Ωs
The weights, w((i, j), (k, l)), are based on the similarity between the neighborhoods, Ni,j and Nk,l , of pixels (i, j) and (k, l), where NP is a square sub-image window centered pixel, p, with radius RP . These weights are calculated as: K Ni,j − Nk,l 2 /h
w ((i, j) , (k, l)) =
∀k,l∈Ωs
K Ni,j − Nk,l 2 /h
.
(3)
K (x) is kernel function and the default choice is the Gaussian kernel with zero mean and standard deviation, σ , that is K (x) = exp −x2 /2σ 2 . h is exponential decay control and ∥·∥2 is the Euclidean norm. 3. The proposed SNLM method The proposed method consists of two different stages, detection and filtering. The output of the detection stage is a binary noise map matrix (b) with 1’s representing noisy pixels and 0’s representing noise-free pixels. As shown in Figure 1, image pixels enter the filtering stage with a switch element that switches between noisy and noise-free cases and eventually outputs restored image after the filtering operation. The algorithm of the SNLM method is as follows: Stage 1-Detection: The proposed method can be used in denoising SNP and RVN cases, but with different detection strategies. There are effective detection procedures in both noise types. In the SNP case, BDND (boundary discriminative noise detection) [18] and simple detection [7,9] and, in RVN, DWM (directional median filter) [19] and ACWMF (adaptive center weighted median filter) [20] are examples of effective impulse detectors. The main focus of this paper is SNP noise, and in this case, noisy pixels must be the extreme gray-level values of the image, and noise-free pixels must lie between these two extreme values. It means that for an 8-bit monochrome image, the noisy pixels must be either 0 or 255. So, we mark them as ‘‘1’’ in the binary noise map. The other elements of this matrix are filled with ‘‘0’’ [7,9]. By combining the proposed SNLM method with one of the RVN detectors [19,20], this method can be applied on RVN effectively. However, this modification does not change the proposed filtering stage of the method. Stage 2-Filtering: Similar to other switching methods, the noise-free pixels remain unchanged. Noisy pixels (pixels with mark ‘‘1’’ in the binary noise map) are altered as follows: (a) Two different windows are considered around each noisy pixel; a bigger window with radius RS , which is called the search window, and a smaller window with radius Rc , which is called the central patch. (b) Around each noise-free pixel in the search window, consider a window with radius Rc that is called the remote patch. (c) The restored value of the pixel is obtained as the weighted sum of all noise-free pixels in the search window. The weights are based on the similarity between the central patch and remote patches. However, unlike the NLM method, we do not have all pixels in each patch, as some of them are noisy pixels, and we must discard them. The similarity is based on the Euclidean norm between noisefree pixels in two patches, as in Eq. (4). Note that pixel (i, j) is the pixel to be restored and pixel (k, l) is a noise-free pixel in the search window. The kernel matrix in Eq. (4) is a Gaussian kernel matrix, with size equal to 2∗ Rc + 1, and its standard
M. Nasri et al. / Scientia Iranica, Transactions D: Computer Science & Engineering and Electrical Engineering (
)
( ),
–
3
w ((i, j) , (k, l)) exp −d ((i, j) , (k, l)) /h′2
=
exp −d ((i, j) , (k, l)) /h′2
∀(k,l)∈Rs ,b(k,l)=0
.
(5)
(d) Finally, the restored value of the pixel is calculated as in Eq. (6). As shown in this equation, when the binary noise map b(i, j) equals zero or, in other words, the pixel is noisefree, the original pixel is retained. However, when the pixel is noisy, the restored value is calculated as a weighted sum of all non-noisy pixels in the search window:
Xˆ (i, j) =
Figure 2: The SNLM method graphical description.
deviation equals 1. Xi,j and bi,j are (i, j) pixels of the noisy image and the binary noise map, respectively. Based on the defined similarity in Eq. (4), the weights are calculated as Eq. (5). d((i,j),(k,l)) =
Rc Rc
kernel(Rc + m, Rc + n)
w ((i, j), (k, l)) · X (i, j)
∀(k,l)∈Ωs ,b(k,l)=0
X (i, j)
(6)
if b(i, j) = 1 if b(i, j) = 0.
Figure 2 shows this procedure in denoising a noisy pixel. The central patch and four typical remote patches in the binary noise map are labeled, ‘‘o’’ and ‘‘a’’–‘‘d’’, respectively. The values of patches ‘‘o’’ and ‘‘b’’ are shown in the image. Only the pixels are used in the calculation of similarity between two patches that are noise-free in both patches. For instance, only two data sets, (192, 164) and (210, 168), are used in calculation of the similarity between ‘‘o’’ and ‘‘b’’ patches. 4. Simulation results
m=−Rc n=−Rc
· 1 − bi+m,j+n · 1 − bk+m,l+n 2 · xi+m,j+n − xk+m,l+n
d
(4)
To evaluate the performance of the proposed method, several standard images, such as Lena, girl and Baboon, are used
a
b
c
e
f
g
h
i
Figure 3: Simulation results of different methods. (a) Original ‘‘Baboon’’ image; (b) Noisy with 90% SNP; (c) Noisy with 70% SNP; (d) PSMF method; (e) GP method; (f) LRC method; (g) DBA method; (h) MDBUTMF; and (i) Proposed method (SNLM). 2nd row shows the output of different methods in 90% noise and 3rd row shows the output of different methods in 70% noise.
4
M. Nasri et al. / Scientia Iranica, Transactions D: Computer Science & Engineering and Electrical Engineering (
d
a
b
c
e
f
g
h
)
( ),
–
i
Figure 4: Simulation results of different methods. (a) Original ‘‘girl’’ image; (b) Noisy with 90% SNP; (c) Noisy with 70% SNP; (d) PSMF method; (e) GP method; (f) LRC method; (g) DBA method; (h) MDBUTMF; and (i) Proposed method (SNLM). 2nd row shows the output of different methods in 90% noise and 3rd row shows the output of different methods in 70% noise.
with different SNP impulse noise levels varying from 10% to 90% with increment 10%. The performance criteria used in the paper is PSNR and MAE, as defined in Eqs. (7) and (9), respectively, [1]:
PSNR = 10 log10
2552 MSE
,
M N 2 1 MSE = I (i, j) − ˆI (i, j) , MN i=1 j=1
MAE =
M N 1 I (i, j) − ˆI (i, j) . MN i=1 j=1
(7)
a
b
(8) Figure 5: Comparison of LRC and proposed method (SNLM) in denoising Lena eyes with 70% SNP noise. (a) LRC; and (b) SNLM.
(9)
The size of the original image, I, is (M ×N ) and the reconstructed image is ˆI. In order to compare the results, several recent methods are implemented in MATLAB, with the mentioned parameters in their papers and applied to noisy images. These methods are PSMF [10], GP [21], LRC [14], DBA [9], and MDBUTMF [7]. The SNLM method has three different parameters: search window’s radius, RS , central and remote patches radius, Rc , and control parameter, h′ . Although the results of noise reduction can be improved by tuning these parameters, due to noise density and image characteristics, for simplicity and as our focus is on the HDIN case, we fix these parameters in 7, 2, and 10, respectively. PSNR and MAE results of these methods in denoising Lena and girl images are shown in Tables 1 and 2. As can be seen, the PSNR result of the SNLM method is somehow lower than DBA, MDBUTMF, and LRC in low density (<30%) impulse noise, but it is higher than PSMF and GP and has acceptable results. The SNLM outperforms all methods in terms of PSNR and MAE in
a wide range of HDIN, and it can effectively suppress impulse noise. For instance, its PSNR is more than two dB higher than other methods in 90% SNP noise. Figures 3 and 4 show the results of denoising Baboon and girl images with two different noise levels. It can be seen that PSMF and GP filters output images with severe noisy spots, MDBUTMF and DBA method has artifacts, and both LRC and SNLM methods have acceptable visual appearances, but in LRC, some small fluctuations are seen in the image edges. To better visually compare the SNLM and LRC methods, Figure 5 shows these methods in denoising the zoomed Lena image with 70% SNP noise. 5. Conclusion and future works In this paper, a Switching Non-Local Means (SNLM) denoising method is proposed for high density impulse noise reduction. The basic Non-Local Means (NLM) filter exploits all the pixels in the restoration of the image and yields good results in Gaussian noise reduction. By using this filter in conjunction with a switching strategy, a promising result is obtained
M. Nasri et al. / Scientia Iranica, Transactions D: Computer Science & Engineering and Electrical Engineering ( Table 1: Comparison of PSNR and MAE values of different methods for ‘‘Lena’’ image noise reduction at different noise levels. Noise density (%)
‘‘Lena’’ – PSNR (dB) PSMF
GP
LRC
DBA
MDBUTMF
10 20 30 40 50 60 70 80 90
37.26 32.84 28.95 24.82 20.67 12.25 9.96 8.10 6.63
32.49 29.62 27.59 25.74 23.63 20.61 16.75 12.65 9.12
42.85 39.02 36.15 33.28 30.52 26.89 23.66 22.32 21.27
41.57 37.54 34.78 32.37 30.14 27.92 25.62 23.14 19.71
42.42 37.95 35.05 32.16 29.47 27.46 22.10 17.89 18.48
10 20 30 40 50 60 70 80 90
0.58 1.17 2.11 3.77 7.30 29.94 48.34 71.80 98.60
0.75 1.41 2.23 3.37 5.30 9.53 19.20 39.12 70.73
0.39 0.82 1.35 2.02 2.97 4.62 7.29 9.20 11.20
SNLM 39.25 36.65 35.13 33.66 31.96 29.93 27.75 25.61 23.47
‘‘Lena’’ – MAE 0.40 0.86 1.42 2.09 2.93 4.04 5.54 7.92 13.07
0.36 0.83 1.39 2.12 3.16 4.42 6.42 10.34 19.75
0.51 1.02 1.53 2.07 2.86 3.95 5.51 7.70 11.12
Table 2: Comparison of PSNR and MAE values of different methods for ‘‘girl’’ image noise reduction at different noise levels. Noise density (%)
10 20 30 40 50 60 70 80 90
‘‘girl’’ – PSNR (dB) PSMF
GP
LRC
DBA
MDBUTMF
39.53 34.36 30.52 26.66 21.76 17.03 10.00 8.12 6.71
32.63 29.58 27.81 26.32 24.40 21.52 17.60 13.14 9.31
45.32 41.70 38.87 36.12 33.12 29.95 26.98 25.70 24.46
44.64 40.75 38.18 35.70 33.52 31.19 28.80 26.06 22.13
45.29 41.29 38.21 36.07 33.89 31.58 28.92 25.50 20.93
43.20 40.23 38.30 36.25 34.62 32.78 30.76 28.87 26.61
SNLM
0.29 0.64 1.21 1.61 2.27 3.19 4.71 7.71 15.63
0.38 0.76 1.19 1.57 2.20 3.10 4.22 5.71 8.41
‘‘girl’’ – MAE 10 20 30 40 50 60 70 80 90
0.46 1.08 1.91 3.27 6.09 12.76 46.79 70.70 98.07
0.56 1.12 1.78 2.68 4.32 7.98 16.72 36.04 67.96
0.32 0.67 1.13 1.70 2.52 3.81 5.70 7.14 8.95
0.33 0.69 1.20 1.62 2.26 3.11 4.27 6.25 10.96
in high-density impulse noise reduction. The results of denoising different standard images show that the proposed method outperforms several recent methods in this area of research. As future work, there are fast and efficient variants of the NLM method that can be used. Moreover, this paper can be extended in mixed Gaussian and impulse noise reduction. Another future work may be optimizing the tuning parameters of the proposed method to remove impulse noise effectively for all noise densities. References [1] Bovik, A., Handbook of Image and Video Processing, Academic Press, New York (2000). [2] Gonzalez, R.C. and Woods, R.E., Digital Image Processing, 2nd Edn., Prentice Hall Press (2002). [3] Dong, Y., Chan, R.H. and Xu, S. ‘‘A detection statistic for randomvalued impulse noise’’, IEEE Transactions on Image Processing, 16(4), pp. 1112–1120 (2007).
)
( ),
–
5
[4] Civicioglu, P. ‘‘Removal of random-valued impulsive noise from corrupted images’’, IEEE Transactions on Consumer Electronics, 55(4), pp. 2097–2104 (2009). [5] Toh, K.K.V. and Isa, N.A.M. ‘‘Cluster-based adaptive fuzzy switching median filter for universal impulse noise reduction’’, IEEE Transactions on Consumer Electronics, 56(4), pp. 2560–2568 (2010). [6] Awad, A.S. ‘‘Standard deviation for obtaining the optimal direction in the removal of impulse noise’’, IEEE Signal Processing Letters, 18(7), pp. 407–410 (2011). [7] Esakkirajan, S., Veerakumar, T., Subramanyam, A.N. and PremChand, C.H. ‘‘Removal of high density salt and pepper noise through modified decision based unsymmetric trimmed median filter’’, IEEE Signal Processing Letters, 18(5), pp. 287–290 (2011). [8] Akkoul, S., Ledee, R., Leconge, R. and Harba, R. ‘‘A new adaptive switching median filter’’, IEEE Signal Processing Letters, 17(6), pp. 587–590 (2010). [9] Srinivasan, K.S. and Ebenezer, D. ‘‘A new fast and efficient decisionbased algorithm for removal of high-density impulse noises’’, IEEE Signal Processing Letters, 14(3), pp. 189–192 (2007). [10] Zhou, W. and Zhang, D. ‘‘Progressive switching median filter for the removal of impulse noise from highly corrupted images’’, IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 46(1), pp. 78–80 (1999). [11] Yuan, S.-Q. and Tan, Y.-H. ‘‘Impulse noise removal by a global-local noise detector and adaptive median filter’’, Signal Processing, 86(8), pp. 2123–2128 (2006). [12] Sun, T. and Neuvo, Y. ‘‘Detail-preserving median based filters in image processing’’, Pattern Recognition Letters, 15, pp. 341–347 (1994). [13] Ko, S.J. and Lee, Y.H. ‘‘Center weighted median filters and their applications to image enhancement’’, IEEE Transactions on Circuits and Systems, 38(9), pp. 984–993 (1991). [14] Wang, Z. and Zhang, D. ‘‘Restoration of impulse noise corrupted images using long-range correlation’’, IEEE Signal Processing Letters, 5(1), pp. 4–7 (1998). [15] Buades, A., Coll, B. and Morel, J. ‘‘Image denoising by non-local averaging’’, IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP ’05, pp. 25–28 (2005). [16] Vignesh, R., Byung Tae, O. and Kuo, C.C.J. ‘‘Fast non-local means (NLM) computation with probabilistic early termination’’, IEEE Signal Processing Letters, 17(3), pp. 277–280 (2010). [17] Doré, V. and Cheriet, M. ‘‘Robust NL-means filter with optimal pixel-wise smoothing parameter for statistical image denoising’’, IEEE Transactions on Signal Processing, 57(5), pp. 1703–1716 (2009). [18] Ng, P.-E. and Ma, K.-K. ‘‘A switching median filter with boundary discriminative noise detection for extremely corrupted images’’, IEEE Transactions on Image Processing, 15(6), pp. 1506–1516 (2006). [19] Dong, Y. and Xu, S. ‘‘A new directional weighted median filter for removal of random-valued impulse noise’’, IEEE Signal Processing Letters, 14(3), pp. 193–196 (2007). [20] Chen, T. and Hong Ren, W. ‘‘Adaptive impulse detection using centerweighted median filters’’, IEEE Signal Processing Letters, 8, pp. 1–3 (2001). [21] Petrovic, N.I. and Crnojevic, V. ‘‘Universal impulse noise filter based on genetic programming’’, IEEE Transactions on Image Processing, 17(7), pp. 1109–1120 (2008).
Mehdi Nasri was born in Isfahan, Iran, in 1982. He received his B.S. degree in Biomedical Engineering from the University of Isfahan, Iran, in 2004, and received his M.S. degree in Electrical Engineering from Shahid Bahonar University of Kerman, Iran, in 2007, where he is currently pursuing his Ph.D. degree in Electrical Engineering. His research interests include: computer vision, image processing and soft computing.
Saeid Saryazdi received B.S. and M.S. degrees in Electrical Engineering from Isfahan University of Technology, Iran, in 1985 and 1987, respectively, and DEA and Ph.D. degrees in Electrical Engineering from Rennes1 University, France, in 1994 and 1997, respectively. In 1997, he joined the Department of Electrical Engineering at Shahid Bahonar University of Kerman as Assistant Professor, and was promoted to Associate Professor in 2007. From 2005 to 2006, he was a visiting professor at the École de Technologie Supérieure (University of Quebec), Montréal, QC, Canada. His research interests include: PDE based image denoising and inpainting, steganography, watermarking, and image retrieval. Hossein Nezamabadi-pour was born in Iran, in 1976. He received his B.S. degree in Electrical Engineering from Shahid Bahonar University of Kerman, Iran, in 1998, and his M.S. and Ph.D. degrees, also in Electrical Engineering, from Tarbait Modarres University, Tehran, Iran, in 2000 and 2004, respectively. In 2004, he joined the Department of Electrical Engineering at Shahid Bahonar University of Kerman, Iran, as Assistant Professor, and was promoted to Associate Professor in 2008, and Full Professor in 2012. Dr. Nezamabadipour is author and co-author of more than 250 peer reviewed journals and conference papers. His research interests include: soft computing, evolutionary computation, image processing and pattern recognition.