- Email: [email protected]
Decision-based non-local means filter for removing impulse noise from digital images
Signal Processing 93 (2013) 517–524
Contents lists available at SciVerse ScienceDirect
Signal Processing journal homepage: www.elsevier.com/locate/sigpro
Fast communication
Decision-based non-local means filter for removing impulse noise from digital images Xuming Zhang a, Yi Zhan a, Mingyue Ding a, Wenguang Hou a,n, Zhouping Yin b a b
School of Life Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
a r t i c l e i n f o
abstract
Article history: Received 30 September 2011 Received in revised form 28 August 2012 Accepted 29 August 2012 Available online 7 September 2012
The decision-based non-local means filter is proposed to remove fixed-value impulse noise from the corrupted digital images. The proposed filter first identifies the corrupted pixels using the local statistics based noise detector and then removes the detected impulses using the reference image-based non-local means filter while keeping the uncorrupted pixels unaltered. Extensive simulations demonstrate that the proposed filter can remove impulse noise from the corrupted images effectively while preserving image details very well at the various noise ratios, which leads to its significantly better image restoration performance than numerous state-of-the-art switching-based filters. & 2012 Elsevier B.V. All rights reserved.
Keywords: Impulse noise Noise detector Non-local means Image restoration
1. Introduction Images are often corrupted by impulse noise in the process of transmission over noisy communication channels or recording by noisy sensors [1]. The median filter has been widely used for removing impulse noise because of its superior performance in noise suppression and edge preservation in comparison with the linear filters. However, the standard median (MED) filter is implemented uniformly across the entire image without taking account of whether a pixel is corrupted or not. Inevitably, the MED filter will modify both noise pixels and undisturbed good pixels, thus resulting in blurring or loss of image details and edges. To prevent the alteration of undisturbed pixels, switchingbased filters have been proposed. In the switching filtering scheme, the noise detector is first used to classify the pixels in the images as noise pixels and noise-free pixels and then filtering is activated for the detected noise
n Corresponding author. Tel.: þ86 15927152858; fax: þ 86 27 87792072. E-mail addresses: [email protected] (X. Zhang), [email protected] (Y. Zhan), [email protected] (M. Ding), [email protected] (W. Hou), [email protected] (Z. Yin).
0165-1684/$ - see front matter & 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.sigpro.2012.08.022
pixels. Among the recently proposed switching-based filters are difference-type noise detection based cost function-type filter [2], second-order difference analysis based median filter [3], global–local noise detectionbased adaptive median (GLAM) filter [4], switching median filter with boundary discriminative noise detection (BDND) [5], opening closing sequence (OCS) filter [6], fast switching median (FSM) filter [7], efficient edgepreserving (EEP) filter [8], switching adaptive weighted mean (SAWM) filter [9], convolutional noise detectionbased switching median (CNDSM) filter [10], noise adaptive fuzzy switching median (NAFSM) filter [11] and switching-based filter using nonmonotone adaptive gradient method (NAGM) [12]. Although these switchingbased filters perform better than the MED filter due to the adoption of noise detection mechanism, they only use the local statistics within a small neighborhood of pixels for image denoising and thus tend to damage image details at high noise ratios. The non-local means (NLM) filter, recently proposed by Buades [13], replaces the considered pixel by the weighted mean of all the pixels in the whole image or the surrounding neighborhoods. This filter can preserve image details better than the point-wise filters in that it relies on the global self-redundancy of the images and
518
X. Zhang et al. / Signal Processing 93 (2013) 517–524
measures the similarity between two pixels by evaluating the Gaussian weighted Euclidean distance between two image patches surrounding these pixels. Although the NLM filter has been widely used for removing Gaussian noise [14,15], speckle noise [16], Rician noise [17,18] and Poisson noise [19] in the images, it performs badly in suppressing impulse noise which follows the long-tailed non-Gaussian distribution. To fully utilize the advantage of the NLM filter in preserving image details and overcome its drawback in removing impulse noise, the decision-based non-local means (DNLM) filter is proposed in this paper. The DNLM filter combines this local statistics based noise detector with the reference image-based non-local means filter to remove impulse noise. The proposed filter can restore the images corrupted by impulse noise effectively and it outperforms many well-known switching-based filters in terms of noise reduction and detail preservation. This paper is organized as follows: Section 2 presents the local statistics based noise detector and the reference image based non-local means filter. Comparisons of denoised results on the standard test images and real images are made in Section 3. Finally, a brief conclusion is given in Section 4. 2. The DNLM filter The framework of the DNLM filter is shown in Fig. 1. In the proposed filter, the local statistics based noise detector is first adopted to classify the pixels in the corrupted images as noise pixels and noise-free pixels. The noise pixels are processed by the weighted mean filter while the identity filter is applied to noise-free pixels, which will produce the reference image. Based on the noise detection results and the reference image, the non-local means filter is adopted to restore the original intensity values of noise pixels. The restored image will be obtained by combining the identity filtering result with the non-local means filtering result. Obviously, the crucial components of the DNLM filter involve the noise detector and the non-local means filter. 2.1. Local statistics based noise detector Let f i,j and oi,j denote the intensity value of the image corrupted by impulse noise and that of the original image at pixel location ði,jÞ, respectively. The intensity value f i,j will be given by: 8 with probability u, > <0 with probability ð1uvÞ, o ð1Þ f i,j ¼ i,j > : 255 with probability v:
When u ¼v, the noise model given by (1) describes the so-called salt-and-pepper noise. It has been well known that the pixel corrupted by fixed valued impulse noise will generally take by far higher or lower intensity value than its neighboring uncorrupted pixels. Based on the above noise characteristics, the pixel at ði,jÞ will be regarded as noise candidate if it takes the maximum max min intensity value f i,j or minimum one f i,j of all the pixels L L in the Lf Lf detection window W i,jf , where W i,jf ¼ fði þp, j þ qÞ : ðLf 1Þ=2 rp,q r ðLf 1Þ=2g. However, if only extreme intensity values are used to determine whether the considered pixel is corrupted or not, some noise-free edge pixels and flat-region pixels will be misclassified as noise candidates although the corrupted pixels can be identified very effectively. To dismiss the misclassified noise-free pixels from noise candidates, the additional statistics will be adopted. Let F i,j denote the set of noise-free pixels in the Ld Ld detecmax tion window W Li,jd , i.e., F i,j ¼ fði þp,j þqÞ : f i þ p,j þ q af i,j , min f i þ p,j þ q a f i,j ,ðLd 1Þ=2r p,q rðLd 1Þ=2g. To ensure accurate noise detection, Ld will be determined adaptively in the following way. Starting with Ld ¼3, this detection window iteratively extends outward by one pixel in its four sides (i.e., Ld ’Ld þ 2) until the number of elements in F i,j is not less than 3 or Ld 4 Lf . The weighted mean mi,j of all the pixels in F i,j is defined as P ði þ p,j þ qÞ2F i,j oi þ p,j þ q f i þ p,j þ q P mi,j ¼ , ð2Þ ði þ p,j þ qÞ2F i,j
oi þ p,j þ q
where oi þ p,j þ q means the weight of f i þ p,j þ q . Because the median di,j of the pixels in F i,j has the least probability to be the value of the corrupted pixel, di,j is utilized to determine oi þ p,j þ q . It is easy to understand that the more f i þ p,j þ q and di,j are similar, the greater value oi þ p,j þ q should take to strengthen the contribution of f i þ p,j þ q to mi,j . Accordingly, oi þ p,j þ q is a decreasing function of the absolute difference between f i þ p,j þ q and di,j . Through extensive simulations, we choose oi þ p,j þ q as 1
oi þ p,j þ q ¼ 1þ
!2 : 9f i þ p,j þ q di,j 9 max
ð3Þ
min
f i,j f i,j
By comparing the absolute difference between f i,j and mi,j with the detection threshold Td, the pixel at (i,j) will be classified as noise pixel with bi,j ¼ 1 or noise-free pixel with bi,j ¼ 0 ( 1 ði,jÞ= 2F i,j and 9f i,j mi,j 9 4 T d , bi,j ¼ ð4Þ 0 otherwise:
2.2. Reference image based non-local means filter The detected impulses will be removed by the reference image-based non-local means filter. Here the reference image is produced based on the noise detection results and it is estimated as the denoised version of the input noisy image. The intensity value of any pixel at (i,j) in the reference image is represented by: Fig. 1. Framework of the decision-based non-local means filter.
r i,j ¼ bi,j mi,j þ ð1bi,j Þ f i,j :
ð5Þ
X. Zhang et al. / Signal Processing 93 (2013) 517–524
For the noise pixel at (i,j) in the input noisy image, the corresponding non-local means denoised intensity NLi,j is obtained based on the reference image. Let Oi,j denote the set of the selected pixels within the Ls Ls search window W Li,js centered at (i,j) in the reference image. The intensity NLi,j will be computed as the weighted average of all the pixels in Oi,j , i.e. P ðk,lÞ2O si,j,k,l r k,l , ð6Þ NLi,j ¼ P i,j ðk,lÞ2Oi,j si,j,k,l where the weight si,j,k,l measures the similarity between two pixels at (i,j) and (k,l) in the reference image. For the set Oi,j , it is the same to the search window in the traditional NLM filter while in the DNLM filter it will be determined adaptively based on the local characteristics of the pixels in W Li,js . Let U i,j be the number of unaltered pixels in W Li,js whose intensity values are the same to those of the corresponding pixels in the input noisy image. If U i,j is less than half of the search window size, Oi,j is presented as the set of all the pixels in W Li,js to achieve good image smoothing effect. Otherwise, Oi,j is defined as the set of unaltered pixels in W Li,js to ensure excellent detail preservation performance 8 Ls Ls > > , U i,j o < fðp,qÞ : 9pi9 r Ls ,9qj9 r Ls g, 2 Oi,j ¼ L L > s s > : : fðp,qÞ : r p,q ¼ f p,q ,9pi9 r Ls ,9qj9 r Ls g, U i,j Z 2 ð7Þ To determine the weight si,j,k,l in (6) accurately is the key to removing impulse noise by the DNLM filter. In the traditional NLM filter, si,j,k,l is defined as the exponential function of the Gaussian weighted Euclidean distance between two image patches centered at the two pixels (i,j) and (k,l) in the noisy image. Because the weights computed by the exponential function will still take positive values when the Euclidean distances are quite large, the traditional NLM filter cannot attenuate the unfavorable influence of highly dissimilar image patches on the denoised results. A feasible solution to this problem is to dismiss such image patches from the filtering process by setting their weights to zero. Based on the above analysis, the DNLM filter uses the piecewise function for computing si,j,k,l and it is defined as 8 sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi!4 > > < 1 JVðQ i,j ÞVðQ k,l ÞJ2 , si,j,k,l ¼ hi,j > > : 0
JVðQ i,j ÞVðQ k,l ÞJ2 rhi,j , otherwise,
ð8Þ where hi,j is the decay parameter controlling the filtering degree, J J denotes the Euclidean norm, VðQ i,j Þ and VðQ k,l Þ are the vectors of pixel intensities taken from the Lp Lp image patches Q i,j and Q k,l centered at (i,j) and (k,l) in the reference image, respectively. It can be seen from (8) that the DNLM filter determines the weight si,j,k,l based on the reference image rather than the noisy image, which lays the core foundation for the DNLM filter to suppress impulse noise in the noisy image effectively. The reason will be explained in this way. For the detected noise pixel at (i,j) and any other pixel at (k,l)
519
in the noisy image, the two image patches Ni,j and Nk,l centered at the two pixels are likely to include noise pixels. Because of the disadvantageous influence of these noise pixels, it is difficult to accurately determine si,j,k,l using the Euclidean distance between N i,j and Nk,l . However, no or very few noise pixels will remain in two image patches Q i,j and Q k,l in the reference image. Exclusion of noise pixels from the compared two image patches ensures that si,j,k,l can be computed more effectively using JVðQ i,j ÞVðQ k,l ÞJ2 than using JVðNi,j ÞVðNk,l ÞJ2 . To illustrate this point better, the standard image Lena will be used as an example. The original Lena image, the image corrupted with 10% impulse noise and the reference image restored by (5) are shown in Fig. 2. Two pairs of 15 15 image patches surrounded by the white boxes and the black boxes are chosen from each of the three images. The Euclidean distances between the two image patches in the white boxes in the three images are 76.20, 114.69 and 74.32, respectively. For the two image patches in the black boxes, their Euclidean distances in the three images are 145.41, 166.51, 143.93, respectively. Obviously, the Euclidean distances between image patches in the reference image are much closer to those in the original image than those in the noisy image. The above example demonstrates that for the corrupted image, the DNLM filter can determine similarity between two pixels effectively by means of the Euclidean distance between two image patches in the reference image. The decay parameter hi,j in (8) plays an important role in the DNLM filter. In the traditional NLM filter, the same decay parameter is chosen for all the pixels in the corrupted image. However, it is difficult to suppress noise effectively while preserving intricate image details very well using the globally fixed decay parameter for the whole image. To address this problem, hi,j will be adaptively determined based on the local image structure in a pixel-wise way. Experimentally, it has been shown that the decay parameter should take a large value for a smooth-region pixel to facilitate noise suppression and take a relatively small value for the edge-region pixel to preserve image details. The gradient magnitude g i,j of the pixel at (i,j) in the reference image can indicate if it is an edge pixel or not. It follows that the pixel with the large gradient magnitude will take smaller decay parameter than that with the small gradient magnitude. It is obvious that hi,j is a decreasing function of g i,j , which can be estimated using Sobel operators. Meanwhile, simulations on a broad variety of gray-level images show that hi,j is also dependent on the noise ratio in the noisy image and it should increase with the increasing noise ratio. Based on the extensive simulations, hi,j is chosen as ð1 þ R2 Þ
hi,j ¼ bi,j
bi,j ¼ d
,
1 , g i,j 2 1þ g max
ð9Þ
ð10Þ
where R is the noise ratio which is estimated as the ratio of the number of detected noise pixels to the total number
520
X. Zhang et al. / Signal Processing 93 (2013) 517–524
3. Experiments on standard test images and real images 3.1. Simulation on standard test images In this section, 512 512 gray-level images such as Lena, Mandrill, Goldhill, Boat and Barbara and Bridge are used as test images. The salt-and-pepper noise with uniform distribution is added to these images, which means that the pixel has equal probability of being corrupted by either a positive impulse (with value 255) or a negative impulse (with value 0). Simulations are conducted on test images corrupted by salt-andpepper noise with the noise ratio varying from 10% to 80%. The noise detection accuracy of the DNLM filter is appreciated and its restoration performance is compared with that of other state-of-the-art switchingbased filters. 3.1.1. Noise detection accuracy The noise detection performance of the DNLM filter depends on the detection window W Lf and the detection threshold Td. For W Lf , a big window contains more pixels and has higher statistical reliability than a small window and thus facilitates accurate noise detection. As regards Td, a very large value will produce numerous misclassifications of noise pixels as noise-free ones while many uncorrupted pixels will be misdetected as noise pixels if Td takes a very small value. The above analysis shows that using a very small value of Lf or Td is likely to cause damage to image details and edges due to the participation of misdetected noise-free pixels into the filtering process. Meanwhile, using a very small Lf or a very large Td results in the remaining of undetected noise pixels in the reference image and the final denoised image, which will attenuate the denoising performance of the DNLM filter because the remaining noise pixels are disadvantageous for the accurate computation of similarity between their neighboring pixels in the reference image. Extensive simulations indicate that good noise detection results can be obtained when Td takes a value in the range [5,20] and Lf is not less than 9. Therefore, we set Td ¼5 and Lf ¼9 for noise detection. Table 1 lists the noise detection results of the DNLM filter operating on all the test images in terms of the total number of undetected noise pixels and misdetected noise-free pixels. We can see from Table 1 that the DNLM filter can realize noise detection with very few or no mistakes at the various noise ratios, which provides the DNLM filter with a solid foundation to achieve the outstanding restoration performance.
Fig. 2. The image Lena used to illustrate the effectiveness of the DNLM filter in determining the similarity between two pixels: (a) original image Lena, (b) noisy image with 10% impulse noise, and (c) reference image.
of pixels in the noisy image; d is a predefined constant and g max is the maximum gradient magnitude of the pixels in the reference image.
3.1.2. Restoration performance The restoration performance of the DNLM filter depends on the search window size Ls Ls , the image patch size Lp Lp and the predefined constant d. As regards Ls and Lp, the small values should be chosen to preserve image details because very few impulses will remain in the reference image. For the constant d, image details will be damaged if it takes a very large value while the image will be under-smoothed if it takes a very small value.
X. Zhang et al. / Signal Processing 93 (2013) 517–524
Table 1 Noise detection results of the DNLM filter operating on all the test images corrupted by salt-and-pepper noise with the various noise ratios. Test images Lena Mandrill Goldhill Boat Barbara Bridge
10% 32 84 55 55 31 151
20% 0 0 0 0 0 57
30% 0 0 0 0 0 96
40% 0 0 0 0 0 130
50% 0 0 0 0 0 161
60% 0 0 0 0 0 172
70% 0 0 0 0 0 182
80% 0 0 0 0 0 173
PB MSSIMðO,EÞ ¼
SSIMðok ,ek Þ ¼
k¼1
521
SSIMðok ,ek Þ , B
ð2mok mek þ C 1 Þð2sok ek þ C 2 Þ ðm2ok þ m2ek þ C 1 Þðs2ok þ s2ek þ C 2 Þ
ð12Þ
,
ð13Þ
where D is the dynamic range of the pixel intensities (255 for 8-bit gray-level images); O and E denote the original image and the filtered image, respectively; ok and ek are the image contents at the k-th local window in the original and filtered images; B is the total number of local windows in the image; mok and mek are the mean intensity of ok and ek; sok and sek are the standard deviation of ok and ek; sok ek is the covariance between ok and ek; C 1 ¼ ðK 1 DÞ2 and C 2 ¼ ðK 2 DÞ2 are small constants to stabilize SSIM using the following parameter settings: K 1 ¼ 0:01 and K 2 ¼ 0:03. Figs. 3 and 4 show PSNR and MSSIM values of all the evaluated filters operating on the corrupted images Lena and Mandrill, respectively. Obviously, the DNLM filter produces higher PSNR and MSSIM values than the other filters at the various noise ratios. To demonstrate the
Fig. 3. Restoration results in PSNR (dB) of the various filters operating on the image Lena and the image Mandrill: (a) PSNR for the image Lena and (b) PSNR for the image Mandrill.
Simulations on all the test images show that d in the range [160,200] can ensure satisfactory denoised results. Therefore, we set Ls ¼3, Lp ¼3 and d ¼ 180 for noise removal. Restoration performance is quantitatively evaluated by peak signal-to-noise ratio (PSNR) and mean structural similarity (MSSIM) [20] which are defined as PSNR ¼ 10 log10
D2 dB, 1 PM PN 2 i¼1 j ¼ 1 ðoi,j ei,j Þ MN
ð11Þ
Fig. 4. Restoration results in MSSIM of the various filters operating on the image Lena and the image Mandrill: (a) MSSIM for the image Lena and (b) MSSIM for the image Mandrill.
522
X. Zhang et al. / Signal Processing 93 (2013) 517–524
Table 2 Restoration results in PSNR (dB) of the various filters operating on the four test images corrupted by 40% and 80% salt-and-pepper noise. Filters
GLAM BDND filter OCS filter FSM filter EEP filter SAWM filter CNDSM filter NAFSM filter NAGM filter DNLM filter
Goldhill
Boat
Barbara
Bridge
40%
80%
40%
80%
40%
80%
40%
80%
32.45 32.47 29.37 30.55 32.66 33.12 31.64 32.85 33.38 34.58
26.81 26.33 25.63 25.23 26.46 27.65 26.52 27.41 27.82 29.07
31.54 31.78 27.93 29.60 32.01 32.45 30.57 32.10 32.88 34.27
25.38 24.59 24.12 23.97 24.86 26.40 25.23 26.24 26.78 27.92
26.50 26.89 24.21 25.19 26.59 27.46 26.06 26.92 27.76 28.96
22.34 23.15 21.85 20.91 22.28 22.60 21.76 22.54 22.82 23.97
27.73 26.30 24.40 25.85 27.73 28.17 26.91 28.12 28.21 29.42
22.49 22.10 21.32 21.24 21.93 23.01 21.98 22.99 23.18 24.40
Table 3 Restoration results in MSSIM of the various filters operating on the four test images corrupted by 40% and 80% salt-and-pepper noise. Filters
GLAM BDND filter OCS filter FSM filter EEP filter SAWM filter CNDSM filter NAFSM filter NAGM filter DNLM filter
Goldhill
Boat
Barbara
Bridge
40%
80%
40%
80%
40%
80%
40%
80%
0.9210 0.9211 0.8013 0.8838 0.9243 0.9331 0.9093 0.9259 0.9395 0.9413
0.7294 0.6977 0.6959 0.6837 0.7194 0.7738 0.7291 0.7634 0.7816 0.7993
0.9346 0.9363 0.8425 0.9032 0.9376 0.9443 0.9227 0.9387 0.9491 0.9532
0.7756 0.7457 0.7399 0.7373 0.7655 0.8125 0.7738 0.8045 0.8174 0.8338
0.9005 0.9021 0.7299 0.8612 0.9019 0.9151 0.8875 0.9065 0.9214 0.9321
0.6937 0.7052 0.6558 0.6322 0.7009 0.7361 0.6916 0.7249 0.7320 0.7532
0.8782 0.8628 0.6649 0.8217 0.8792 0.8912 0.8595 0.8866 0.8985 0.9042
0.6056 0.5565 0.5381 0.5583 0.5959 0.6608 0.6069 0.6548 0.6553 0.6871
generalization of the proposed filter, the performance to restore other four corrupted images is compared among these switching-based filters. Tables 2 and 3 list PSNR and MSSIM values of the various filters for the four images corrupted by 40% and 80% salt-and-pepper noise, respectively. Likewise, the DNLM filter outperforms the other filters in terms of the above metrics. The objective evaluation based on PSNR and MSSIM measurement indicates that the DNLM filter has the best restoration performance among all the compared filters. To verify the advantage of the DNLM filter over the other evaluated filters, the subjective visual comparisons among the restoration results of the various filters are also made. Figs. 5 and 6 show the enlarged portions of the restored results of all the evaluated filters for the image Lena corrupted by 80% impulse noise and those of the most competitive NAGM filter and the DNLM filter for the image Mandrill corrupted by 80% impulse noise, respectively. We can see from Figs. 5 and 6 that the BDND filter, the OCS filter and the NAGM filter lead to image oversmoothing. The FSM filter produces artifacts near the edges and in the textural areas. The EEP filter, the SAWM filter, the CNDSM filter and the NAFSM filter damage image details to some extent. By comparison, the DNLM filter provides the best restoration results among all these filters in that it preserves image details very well while removing impulse noise effectively. The reason why the DNLM filter can restore the corrupted images more effectively than other evaluated
filters will be simply discussed here. The difference between the DNLM filter and the compared filters lies in the strategy to restore the original intensity values of the detected noise pixels. The DNLM filter exploits the selfsimilarity of the images in the non-local neighborhoods and suppress impulse noise by averaging the noise-free pixels in the duplicated structures. However, the compared filters are generally the point-wise filters which attenuate the influence of impulse noise only by averaging the pixels in the same structure. It is because of the utilization of redundancy from similar patterns in the image that the DNLM filter provides better restoration performance than these point-wise filters. 3.2. Test on real images To demonstrate the applicability of the DNLM filter to denoising real images, the denoised results of such images as the TV image, the panoramic radiograph and the cephalometric radiograph are shown in Fig. 7. The observation from Fig. 7 shows that the DNLM filter has successfully suppressed impulse noise in these real images with image details and edges being preserved very well. 4. Conclusion In this paper, we have presented a novel decisionbased non-local means filter for impulse noise removal.
X. Zhang et al. / Signal Processing 93 (2013) 517–524
523
Fig. 6. Comparisons of restoration results using the NAGM filter and the DNLM filter for the image Mandrill corrupted by 80% salt-and-pepper noise: (a) original image, (b) noisy image, (c) NAGM filter, and (d) DNLM filter. Fig. 5. Comparisons of restoration results using the various filters for the image Lena corrupted by 80% salt-and-pepper noise: (a) original image, (b) noisy image, (c) GLAM filter, (d) BDND filter, (e) OCS filter, (f) FSM filter, (g) EEP filter, (h) SAWM filter, (i) CNDSM filter, (j) NAFSM filter, (k) NAGM filter, and (l) DNLM filter.
The proposed filter realizes noisy image restoration by identifying the corrupted pixels using the local statistics based noise detector and removing them using the
524
X. Zhang et al. / Signal Processing 93 (2013) 517–524
panoramic and cephalometric radiographs. This work was partly supported by Major Program of National Natural Science Foundation of China (Grant No.: 51035002), National Natural Science Foundation of China (Grant No.: 30911120497), the National 973 project (Grant No.: 2011CB933103) and the Project of the National 12th-Five Year Research Program of China (Grant No.: 2012BAI13B02). References
Fig. 7. The denoised results of the DNLM filter operating on the real corrupted images: (a) corrupted TV image, (b) corrupted panoramic radiograph, (c) corrupted cephalometric radiograph, (d) denoised TV image, (e) denoised panoramic radiograph, (f) denoised cephalometric radiograph.
distinctive reference image based non-local means filter, which computes the similarity between two pixels by the piecewise weight function with the adaptive decay parameters. Comparisons of restoration performance among the proposed filter and numerous switching-based techniques show that it outperforms the compared filters in suppressing noise and retaining image details.
Acknowledgments We would like to thank professor Iuri Frosio from University of Milan in Italy to provide us with the
[1] H.L. Eng, K.K. Ma, Noise adaptive soft-switching median filter, IEEE Transactions on Image Processing 10 (2) (2001) 242–251. [2] S.Q. Yuan, Y.H. Tan, H.L. Sun, Impulse noise removal by the difference-type noise detector and the cost function-type filter, Signal Processing 87 (10) (2007) 2417–2430. [3] D. Dang, W. Luo, Impulse noise removal utilizing second-order difference analysis, Signal Processing 87 (9) (2007) 2017–2025. [4] S.Q. Yuan, Y.H. Tan, Impulse noise removal by a global–local noise detector and adaptive median filter, Signal Processing 86 (8) (2006) 2123–2128. [5] P.-E. Ng, K.-K. Ma, A switching median filter with boundary discriminative noise detection for extremely corrupted image, IEEE Transactions on Image Processing 15 (6) (2006) 1506–1516. [6] Z.F. Deng, Z.P. Yin, Y.L. Xiong, High probability impulse noiseremoving algorithm based on mathematical morphology, IEEE Signal Processing Letters 14 (1) (2007) 31–34. [7] K.S. Srinivasan, D. Ebenezer, A new fast and efficient decision-based algorithm for removal of high-density impulse noises, IEEE Signal Processing Letters 14 (3) (2007) 189–192. [8] P.Y. Chen, C.Y. Lien, An efficient edge-preserving algorithm for removal of salt-and-pepper noise, IEEE Signal Processing Letters 15 (2008) 833–836. [9] X.M. Zhang, Y.L. Xiong, Impulse noise removal using directional weighted noise detector and adaptive weighted mean filter, IEEE Signal Processing Letters 16 (4) (2009) 295–298. [10] S.S. Wang, C.H. Wu, A new impulse detection and filtering method for removal of wide range impulse noises, Pattern Recognition 42 (9) (2009) 2194–2202. [11] K.K.V. Toh, N.A.M. Isa, Noise adaptive fuzzy switching median filter for salt-and-pepper noise reduction, IEEE Signal Processing Letters 17 (3) (2010) 281–284. [12] G. Yu, L. Qi, Y. Sun, Y. Zhou, Impulse noise removal by a nonmonotone adaptive gradient method, Signal Processing 90 (10) (2010) 2891–2897. [13] A. Buades, B. Coll, J.M. Morel, A review of image denoising algorithms, with a new one, Multiscale Modeling and Simulation 4 (2) (2005) 490–530. [14] T. Brox, O. Kleinschmidt, D. Cremers, Efficient nonlocal means for denoising of textural patterns, IEEE Transactions on Image Processing 17 (7) (2008) 1083–1092. [15] Q. Chen, Q.S. Sun, D.S. Xia, Homogeneity similarity based image denoising, Pattern Recognition 43 (12) (2010) 4089–4100. [16] P. Coup, P. Hellier, C. Barillot, Nonlocal means-based speckle filtering for ultrasound images, IEEE Transactions on Image Processing 18 (10) (2009) 2221–2229. [17] P. Coupe, P. Yger, S. Prima, P. Hellier, C. Kervrann, C. Barillot, An optimized blockwise nonlocal means denoising filter for 3-D magnetic resonance images, IEEE Transactions on Medical Imaging 27 (4) (2008) 425–441. [18] H. Liu, C. Yang, N. Pan, E. Song, R. Green, Denoising 3D MR images by the enhanced non-local means filter for Rician noise, Magnetic Resonance Imaging 28 (2010) 1485–1496. [19] C. Deledalle, F. Tupin, L. Denis, Poisson NL-means: unsupervised non-local means for Poisson noise, in: IEEE International Conference on Image Processing (ICIP), Hong Kong, China, September 2010, pp. 801–804. [20] Z. Wang, A.C. Bovik, H.R. Sheik, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing 13 (4) (2004) 1–14.
Contents lists available at SciVerse ScienceDirect
Signal Processing journal homepage: www.elsevier.com/locate/sigpro
Fast communication
Decision-based non-local means filter for removing impulse noise from digital images Xuming Zhang a, Yi Zhan a, Mingyue Ding a, Wenguang Hou a,n, Zhouping Yin b a b
School of Life Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China
a r t i c l e i n f o
abstract
Article history: Received 30 September 2011 Received in revised form 28 August 2012 Accepted 29 August 2012 Available online 7 September 2012
The decision-based non-local means filter is proposed to remove fixed-value impulse noise from the corrupted digital images. The proposed filter first identifies the corrupted pixels using the local statistics based noise detector and then removes the detected impulses using the reference image-based non-local means filter while keeping the uncorrupted pixels unaltered. Extensive simulations demonstrate that the proposed filter can remove impulse noise from the corrupted images effectively while preserving image details very well at the various noise ratios, which leads to its significantly better image restoration performance than numerous state-of-the-art switching-based filters. & 2012 Elsevier B.V. All rights reserved.
Keywords: Impulse noise Noise detector Non-local means Image restoration
1. Introduction Images are often corrupted by impulse noise in the process of transmission over noisy communication channels or recording by noisy sensors [1]. The median filter has been widely used for removing impulse noise because of its superior performance in noise suppression and edge preservation in comparison with the linear filters. However, the standard median (MED) filter is implemented uniformly across the entire image without taking account of whether a pixel is corrupted or not. Inevitably, the MED filter will modify both noise pixels and undisturbed good pixels, thus resulting in blurring or loss of image details and edges. To prevent the alteration of undisturbed pixels, switchingbased filters have been proposed. In the switching filtering scheme, the noise detector is first used to classify the pixels in the images as noise pixels and noise-free pixels and then filtering is activated for the detected noise
n Corresponding author. Tel.: þ86 15927152858; fax: þ 86 27 87792072. E-mail addresses: [email protected] (X. Zhang), [email protected] (Y. Zhan), [email protected] (M. Ding), [email protected] (W. Hou), [email protected] (Z. Yin).
0165-1684/$ - see front matter & 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.sigpro.2012.08.022
pixels. Among the recently proposed switching-based filters are difference-type noise detection based cost function-type filter [2], second-order difference analysis based median filter [3], global–local noise detectionbased adaptive median (GLAM) filter [4], switching median filter with boundary discriminative noise detection (BDND) [5], opening closing sequence (OCS) filter [6], fast switching median (FSM) filter [7], efficient edgepreserving (EEP) filter [8], switching adaptive weighted mean (SAWM) filter [9], convolutional noise detectionbased switching median (CNDSM) filter [10], noise adaptive fuzzy switching median (NAFSM) filter [11] and switching-based filter using nonmonotone adaptive gradient method (NAGM) [12]. Although these switchingbased filters perform better than the MED filter due to the adoption of noise detection mechanism, they only use the local statistics within a small neighborhood of pixels for image denoising and thus tend to damage image details at high noise ratios. The non-local means (NLM) filter, recently proposed by Buades [13], replaces the considered pixel by the weighted mean of all the pixels in the whole image or the surrounding neighborhoods. This filter can preserve image details better than the point-wise filters in that it relies on the global self-redundancy of the images and
518
X. Zhang et al. / Signal Processing 93 (2013) 517–524
measures the similarity between two pixels by evaluating the Gaussian weighted Euclidean distance between two image patches surrounding these pixels. Although the NLM filter has been widely used for removing Gaussian noise [14,15], speckle noise [16], Rician noise [17,18] and Poisson noise [19] in the images, it performs badly in suppressing impulse noise which follows the long-tailed non-Gaussian distribution. To fully utilize the advantage of the NLM filter in preserving image details and overcome its drawback in removing impulse noise, the decision-based non-local means (DNLM) filter is proposed in this paper. The DNLM filter combines this local statistics based noise detector with the reference image-based non-local means filter to remove impulse noise. The proposed filter can restore the images corrupted by impulse noise effectively and it outperforms many well-known switching-based filters in terms of noise reduction and detail preservation. This paper is organized as follows: Section 2 presents the local statistics based noise detector and the reference image based non-local means filter. Comparisons of denoised results on the standard test images and real images are made in Section 3. Finally, a brief conclusion is given in Section 4. 2. The DNLM filter The framework of the DNLM filter is shown in Fig. 1. In the proposed filter, the local statistics based noise detector is first adopted to classify the pixels in the corrupted images as noise pixels and noise-free pixels. The noise pixels are processed by the weighted mean filter while the identity filter is applied to noise-free pixels, which will produce the reference image. Based on the noise detection results and the reference image, the non-local means filter is adopted to restore the original intensity values of noise pixels. The restored image will be obtained by combining the identity filtering result with the non-local means filtering result. Obviously, the crucial components of the DNLM filter involve the noise detector and the non-local means filter. 2.1. Local statistics based noise detector Let f i,j and oi,j denote the intensity value of the image corrupted by impulse noise and that of the original image at pixel location ði,jÞ, respectively. The intensity value f i,j will be given by: 8 with probability u, > <0 with probability ð1uvÞ, o ð1Þ f i,j ¼ i,j > : 255 with probability v:
When u ¼v, the noise model given by (1) describes the so-called salt-and-pepper noise. It has been well known that the pixel corrupted by fixed valued impulse noise will generally take by far higher or lower intensity value than its neighboring uncorrupted pixels. Based on the above noise characteristics, the pixel at ði,jÞ will be regarded as noise candidate if it takes the maximum max min intensity value f i,j or minimum one f i,j of all the pixels L L in the Lf Lf detection window W i,jf , where W i,jf ¼ fði þp, j þ qÞ : ðLf 1Þ=2 rp,q r ðLf 1Þ=2g. However, if only extreme intensity values are used to determine whether the considered pixel is corrupted or not, some noise-free edge pixels and flat-region pixels will be misclassified as noise candidates although the corrupted pixels can be identified very effectively. To dismiss the misclassified noise-free pixels from noise candidates, the additional statistics will be adopted. Let F i,j denote the set of noise-free pixels in the Ld Ld detecmax tion window W Li,jd , i.e., F i,j ¼ fði þp,j þqÞ : f i þ p,j þ q af i,j , min f i þ p,j þ q a f i,j ,ðLd 1Þ=2r p,q rðLd 1Þ=2g. To ensure accurate noise detection, Ld will be determined adaptively in the following way. Starting with Ld ¼3, this detection window iteratively extends outward by one pixel in its four sides (i.e., Ld ’Ld þ 2) until the number of elements in F i,j is not less than 3 or Ld 4 Lf . The weighted mean mi,j of all the pixels in F i,j is defined as P ði þ p,j þ qÞ2F i,j oi þ p,j þ q f i þ p,j þ q P mi,j ¼ , ð2Þ ði þ p,j þ qÞ2F i,j
oi þ p,j þ q
where oi þ p,j þ q means the weight of f i þ p,j þ q . Because the median di,j of the pixels in F i,j has the least probability to be the value of the corrupted pixel, di,j is utilized to determine oi þ p,j þ q . It is easy to understand that the more f i þ p,j þ q and di,j are similar, the greater value oi þ p,j þ q should take to strengthen the contribution of f i þ p,j þ q to mi,j . Accordingly, oi þ p,j þ q is a decreasing function of the absolute difference between f i þ p,j þ q and di,j . Through extensive simulations, we choose oi þ p,j þ q as 1
oi þ p,j þ q ¼ 1þ
!2 : 9f i þ p,j þ q di,j 9 max
ð3Þ
min
f i,j f i,j
By comparing the absolute difference between f i,j and mi,j with the detection threshold Td, the pixel at (i,j) will be classified as noise pixel with bi,j ¼ 1 or noise-free pixel with bi,j ¼ 0 ( 1 ði,jÞ= 2F i,j and 9f i,j mi,j 9 4 T d , bi,j ¼ ð4Þ 0 otherwise:
2.2. Reference image based non-local means filter The detected impulses will be removed by the reference image-based non-local means filter. Here the reference image is produced based on the noise detection results and it is estimated as the denoised version of the input noisy image. The intensity value of any pixel at (i,j) in the reference image is represented by: Fig. 1. Framework of the decision-based non-local means filter.
r i,j ¼ bi,j mi,j þ ð1bi,j Þ f i,j :
ð5Þ
X. Zhang et al. / Signal Processing 93 (2013) 517–524
For the noise pixel at (i,j) in the input noisy image, the corresponding non-local means denoised intensity NLi,j is obtained based on the reference image. Let Oi,j denote the set of the selected pixels within the Ls Ls search window W Li,js centered at (i,j) in the reference image. The intensity NLi,j will be computed as the weighted average of all the pixels in Oi,j , i.e. P ðk,lÞ2O si,j,k,l r k,l , ð6Þ NLi,j ¼ P i,j ðk,lÞ2Oi,j si,j,k,l where the weight si,j,k,l measures the similarity between two pixels at (i,j) and (k,l) in the reference image. For the set Oi,j , it is the same to the search window in the traditional NLM filter while in the DNLM filter it will be determined adaptively based on the local characteristics of the pixels in W Li,js . Let U i,j be the number of unaltered pixels in W Li,js whose intensity values are the same to those of the corresponding pixels in the input noisy image. If U i,j is less than half of the search window size, Oi,j is presented as the set of all the pixels in W Li,js to achieve good image smoothing effect. Otherwise, Oi,j is defined as the set of unaltered pixels in W Li,js to ensure excellent detail preservation performance 8 Ls Ls > > , U i,j o < fðp,qÞ : 9pi9 r Ls ,9qj9 r Ls g, 2 Oi,j ¼ L L > s s > : : fðp,qÞ : r p,q ¼ f p,q ,9pi9 r Ls ,9qj9 r Ls g, U i,j Z 2 ð7Þ To determine the weight si,j,k,l in (6) accurately is the key to removing impulse noise by the DNLM filter. In the traditional NLM filter, si,j,k,l is defined as the exponential function of the Gaussian weighted Euclidean distance between two image patches centered at the two pixels (i,j) and (k,l) in the noisy image. Because the weights computed by the exponential function will still take positive values when the Euclidean distances are quite large, the traditional NLM filter cannot attenuate the unfavorable influence of highly dissimilar image patches on the denoised results. A feasible solution to this problem is to dismiss such image patches from the filtering process by setting their weights to zero. Based on the above analysis, the DNLM filter uses the piecewise function for computing si,j,k,l and it is defined as 8 sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi!4 > > < 1 JVðQ i,j ÞVðQ k,l ÞJ2 , si,j,k,l ¼ hi,j > > : 0
JVðQ i,j ÞVðQ k,l ÞJ2 rhi,j , otherwise,
ð8Þ where hi,j is the decay parameter controlling the filtering degree, J J denotes the Euclidean norm, VðQ i,j Þ and VðQ k,l Þ are the vectors of pixel intensities taken from the Lp Lp image patches Q i,j and Q k,l centered at (i,j) and (k,l) in the reference image, respectively. It can be seen from (8) that the DNLM filter determines the weight si,j,k,l based on the reference image rather than the noisy image, which lays the core foundation for the DNLM filter to suppress impulse noise in the noisy image effectively. The reason will be explained in this way. For the detected noise pixel at (i,j) and any other pixel at (k,l)
519
in the noisy image, the two image patches Ni,j and Nk,l centered at the two pixels are likely to include noise pixels. Because of the disadvantageous influence of these noise pixels, it is difficult to accurately determine si,j,k,l using the Euclidean distance between N i,j and Nk,l . However, no or very few noise pixels will remain in two image patches Q i,j and Q k,l in the reference image. Exclusion of noise pixels from the compared two image patches ensures that si,j,k,l can be computed more effectively using JVðQ i,j ÞVðQ k,l ÞJ2 than using JVðNi,j ÞVðNk,l ÞJ2 . To illustrate this point better, the standard image Lena will be used as an example. The original Lena image, the image corrupted with 10% impulse noise and the reference image restored by (5) are shown in Fig. 2. Two pairs of 15 15 image patches surrounded by the white boxes and the black boxes are chosen from each of the three images. The Euclidean distances between the two image patches in the white boxes in the three images are 76.20, 114.69 and 74.32, respectively. For the two image patches in the black boxes, their Euclidean distances in the three images are 145.41, 166.51, 143.93, respectively. Obviously, the Euclidean distances between image patches in the reference image are much closer to those in the original image than those in the noisy image. The above example demonstrates that for the corrupted image, the DNLM filter can determine similarity between two pixels effectively by means of the Euclidean distance between two image patches in the reference image. The decay parameter hi,j in (8) plays an important role in the DNLM filter. In the traditional NLM filter, the same decay parameter is chosen for all the pixels in the corrupted image. However, it is difficult to suppress noise effectively while preserving intricate image details very well using the globally fixed decay parameter for the whole image. To address this problem, hi,j will be adaptively determined based on the local image structure in a pixel-wise way. Experimentally, it has been shown that the decay parameter should take a large value for a smooth-region pixel to facilitate noise suppression and take a relatively small value for the edge-region pixel to preserve image details. The gradient magnitude g i,j of the pixel at (i,j) in the reference image can indicate if it is an edge pixel or not. It follows that the pixel with the large gradient magnitude will take smaller decay parameter than that with the small gradient magnitude. It is obvious that hi,j is a decreasing function of g i,j , which can be estimated using Sobel operators. Meanwhile, simulations on a broad variety of gray-level images show that hi,j is also dependent on the noise ratio in the noisy image and it should increase with the increasing noise ratio. Based on the extensive simulations, hi,j is chosen as ð1 þ R2 Þ
hi,j ¼ bi,j
bi,j ¼ d
,
1 , g i,j 2 1þ g max
ð9Þ
ð10Þ
where R is the noise ratio which is estimated as the ratio of the number of detected noise pixels to the total number
520
X. Zhang et al. / Signal Processing 93 (2013) 517–524
3. Experiments on standard test images and real images 3.1. Simulation on standard test images In this section, 512 512 gray-level images such as Lena, Mandrill, Goldhill, Boat and Barbara and Bridge are used as test images. The salt-and-pepper noise with uniform distribution is added to these images, which means that the pixel has equal probability of being corrupted by either a positive impulse (with value 255) or a negative impulse (with value 0). Simulations are conducted on test images corrupted by salt-andpepper noise with the noise ratio varying from 10% to 80%. The noise detection accuracy of the DNLM filter is appreciated and its restoration performance is compared with that of other state-of-the-art switchingbased filters. 3.1.1. Noise detection accuracy The noise detection performance of the DNLM filter depends on the detection window W Lf and the detection threshold Td. For W Lf , a big window contains more pixels and has higher statistical reliability than a small window and thus facilitates accurate noise detection. As regards Td, a very large value will produce numerous misclassifications of noise pixels as noise-free ones while many uncorrupted pixels will be misdetected as noise pixels if Td takes a very small value. The above analysis shows that using a very small value of Lf or Td is likely to cause damage to image details and edges due to the participation of misdetected noise-free pixels into the filtering process. Meanwhile, using a very small Lf or a very large Td results in the remaining of undetected noise pixels in the reference image and the final denoised image, which will attenuate the denoising performance of the DNLM filter because the remaining noise pixels are disadvantageous for the accurate computation of similarity between their neighboring pixels in the reference image. Extensive simulations indicate that good noise detection results can be obtained when Td takes a value in the range [5,20] and Lf is not less than 9. Therefore, we set Td ¼5 and Lf ¼9 for noise detection. Table 1 lists the noise detection results of the DNLM filter operating on all the test images in terms of the total number of undetected noise pixels and misdetected noise-free pixels. We can see from Table 1 that the DNLM filter can realize noise detection with very few or no mistakes at the various noise ratios, which provides the DNLM filter with a solid foundation to achieve the outstanding restoration performance.
Fig. 2. The image Lena used to illustrate the effectiveness of the DNLM filter in determining the similarity between two pixels: (a) original image Lena, (b) noisy image with 10% impulse noise, and (c) reference image.
of pixels in the noisy image; d is a predefined constant and g max is the maximum gradient magnitude of the pixels in the reference image.
3.1.2. Restoration performance The restoration performance of the DNLM filter depends on the search window size Ls Ls , the image patch size Lp Lp and the predefined constant d. As regards Ls and Lp, the small values should be chosen to preserve image details because very few impulses will remain in the reference image. For the constant d, image details will be damaged if it takes a very large value while the image will be under-smoothed if it takes a very small value.
X. Zhang et al. / Signal Processing 93 (2013) 517–524
Table 1 Noise detection results of the DNLM filter operating on all the test images corrupted by salt-and-pepper noise with the various noise ratios. Test images Lena Mandrill Goldhill Boat Barbara Bridge
10% 32 84 55 55 31 151
20% 0 0 0 0 0 57
30% 0 0 0 0 0 96
40% 0 0 0 0 0 130
50% 0 0 0 0 0 161
60% 0 0 0 0 0 172
70% 0 0 0 0 0 182
80% 0 0 0 0 0 173
PB MSSIMðO,EÞ ¼
SSIMðok ,ek Þ ¼
k¼1
521
SSIMðok ,ek Þ , B
ð2mok mek þ C 1 Þð2sok ek þ C 2 Þ ðm2ok þ m2ek þ C 1 Þðs2ok þ s2ek þ C 2 Þ
ð12Þ
,
ð13Þ
where D is the dynamic range of the pixel intensities (255 for 8-bit gray-level images); O and E denote the original image and the filtered image, respectively; ok and ek are the image contents at the k-th local window in the original and filtered images; B is the total number of local windows in the image; mok and mek are the mean intensity of ok and ek; sok and sek are the standard deviation of ok and ek; sok ek is the covariance between ok and ek; C 1 ¼ ðK 1 DÞ2 and C 2 ¼ ðK 2 DÞ2 are small constants to stabilize SSIM using the following parameter settings: K 1 ¼ 0:01 and K 2 ¼ 0:03. Figs. 3 and 4 show PSNR and MSSIM values of all the evaluated filters operating on the corrupted images Lena and Mandrill, respectively. Obviously, the DNLM filter produces higher PSNR and MSSIM values than the other filters at the various noise ratios. To demonstrate the
Fig. 3. Restoration results in PSNR (dB) of the various filters operating on the image Lena and the image Mandrill: (a) PSNR for the image Lena and (b) PSNR for the image Mandrill.
Simulations on all the test images show that d in the range [160,200] can ensure satisfactory denoised results. Therefore, we set Ls ¼3, Lp ¼3 and d ¼ 180 for noise removal. Restoration performance is quantitatively evaluated by peak signal-to-noise ratio (PSNR) and mean structural similarity (MSSIM) [20] which are defined as PSNR ¼ 10 log10
D2 dB, 1 PM PN 2 i¼1 j ¼ 1 ðoi,j ei,j Þ MN
ð11Þ
Fig. 4. Restoration results in MSSIM of the various filters operating on the image Lena and the image Mandrill: (a) MSSIM for the image Lena and (b) MSSIM for the image Mandrill.
522
X. Zhang et al. / Signal Processing 93 (2013) 517–524
Table 2 Restoration results in PSNR (dB) of the various filters operating on the four test images corrupted by 40% and 80% salt-and-pepper noise. Filters
GLAM BDND filter OCS filter FSM filter EEP filter SAWM filter CNDSM filter NAFSM filter NAGM filter DNLM filter
Goldhill
Boat
Barbara
Bridge
40%
80%
40%
80%
40%
80%
40%
80%
32.45 32.47 29.37 30.55 32.66 33.12 31.64 32.85 33.38 34.58
26.81 26.33 25.63 25.23 26.46 27.65 26.52 27.41 27.82 29.07
31.54 31.78 27.93 29.60 32.01 32.45 30.57 32.10 32.88 34.27
25.38 24.59 24.12 23.97 24.86 26.40 25.23 26.24 26.78 27.92
26.50 26.89 24.21 25.19 26.59 27.46 26.06 26.92 27.76 28.96
22.34 23.15 21.85 20.91 22.28 22.60 21.76 22.54 22.82 23.97
27.73 26.30 24.40 25.85 27.73 28.17 26.91 28.12 28.21 29.42
22.49 22.10 21.32 21.24 21.93 23.01 21.98 22.99 23.18 24.40
Table 3 Restoration results in MSSIM of the various filters operating on the four test images corrupted by 40% and 80% salt-and-pepper noise. Filters
GLAM BDND filter OCS filter FSM filter EEP filter SAWM filter CNDSM filter NAFSM filter NAGM filter DNLM filter
Goldhill
Boat
Barbara
Bridge
40%
80%
40%
80%
40%
80%
40%
80%
0.9210 0.9211 0.8013 0.8838 0.9243 0.9331 0.9093 0.9259 0.9395 0.9413
0.7294 0.6977 0.6959 0.6837 0.7194 0.7738 0.7291 0.7634 0.7816 0.7993
0.9346 0.9363 0.8425 0.9032 0.9376 0.9443 0.9227 0.9387 0.9491 0.9532
0.7756 0.7457 0.7399 0.7373 0.7655 0.8125 0.7738 0.8045 0.8174 0.8338
0.9005 0.9021 0.7299 0.8612 0.9019 0.9151 0.8875 0.9065 0.9214 0.9321
0.6937 0.7052 0.6558 0.6322 0.7009 0.7361 0.6916 0.7249 0.7320 0.7532
0.8782 0.8628 0.6649 0.8217 0.8792 0.8912 0.8595 0.8866 0.8985 0.9042
0.6056 0.5565 0.5381 0.5583 0.5959 0.6608 0.6069 0.6548 0.6553 0.6871
generalization of the proposed filter, the performance to restore other four corrupted images is compared among these switching-based filters. Tables 2 and 3 list PSNR and MSSIM values of the various filters for the four images corrupted by 40% and 80% salt-and-pepper noise, respectively. Likewise, the DNLM filter outperforms the other filters in terms of the above metrics. The objective evaluation based on PSNR and MSSIM measurement indicates that the DNLM filter has the best restoration performance among all the compared filters. To verify the advantage of the DNLM filter over the other evaluated filters, the subjective visual comparisons among the restoration results of the various filters are also made. Figs. 5 and 6 show the enlarged portions of the restored results of all the evaluated filters for the image Lena corrupted by 80% impulse noise and those of the most competitive NAGM filter and the DNLM filter for the image Mandrill corrupted by 80% impulse noise, respectively. We can see from Figs. 5 and 6 that the BDND filter, the OCS filter and the NAGM filter lead to image oversmoothing. The FSM filter produces artifacts near the edges and in the textural areas. The EEP filter, the SAWM filter, the CNDSM filter and the NAFSM filter damage image details to some extent. By comparison, the DNLM filter provides the best restoration results among all these filters in that it preserves image details very well while removing impulse noise effectively. The reason why the DNLM filter can restore the corrupted images more effectively than other evaluated
filters will be simply discussed here. The difference between the DNLM filter and the compared filters lies in the strategy to restore the original intensity values of the detected noise pixels. The DNLM filter exploits the selfsimilarity of the images in the non-local neighborhoods and suppress impulse noise by averaging the noise-free pixels in the duplicated structures. However, the compared filters are generally the point-wise filters which attenuate the influence of impulse noise only by averaging the pixels in the same structure. It is because of the utilization of redundancy from similar patterns in the image that the DNLM filter provides better restoration performance than these point-wise filters. 3.2. Test on real images To demonstrate the applicability of the DNLM filter to denoising real images, the denoised results of such images as the TV image, the panoramic radiograph and the cephalometric radiograph are shown in Fig. 7. The observation from Fig. 7 shows that the DNLM filter has successfully suppressed impulse noise in these real images with image details and edges being preserved very well. 4. Conclusion In this paper, we have presented a novel decisionbased non-local means filter for impulse noise removal.
X. Zhang et al. / Signal Processing 93 (2013) 517–524
523
Fig. 6. Comparisons of restoration results using the NAGM filter and the DNLM filter for the image Mandrill corrupted by 80% salt-and-pepper noise: (a) original image, (b) noisy image, (c) NAGM filter, and (d) DNLM filter. Fig. 5. Comparisons of restoration results using the various filters for the image Lena corrupted by 80% salt-and-pepper noise: (a) original image, (b) noisy image, (c) GLAM filter, (d) BDND filter, (e) OCS filter, (f) FSM filter, (g) EEP filter, (h) SAWM filter, (i) CNDSM filter, (j) NAFSM filter, (k) NAGM filter, and (l) DNLM filter.
The proposed filter realizes noisy image restoration by identifying the corrupted pixels using the local statistics based noise detector and removing them using the
524
X. Zhang et al. / Signal Processing 93 (2013) 517–524
panoramic and cephalometric radiographs. This work was partly supported by Major Program of National Natural Science Foundation of China (Grant No.: 51035002), National Natural Science Foundation of China (Grant No.: 30911120497), the National 973 project (Grant No.: 2011CB933103) and the Project of the National 12th-Five Year Research Program of China (Grant No.: 2012BAI13B02). References
Fig. 7. The denoised results of the DNLM filter operating on the real corrupted images: (a) corrupted TV image, (b) corrupted panoramic radiograph, (c) corrupted cephalometric radiograph, (d) denoised TV image, (e) denoised panoramic radiograph, (f) denoised cephalometric radiograph.
distinctive reference image based non-local means filter, which computes the similarity between two pixels by the piecewise weight function with the adaptive decay parameters. Comparisons of restoration performance among the proposed filter and numerous switching-based techniques show that it outperforms the compared filters in suppressing noise and retaining image details.
Acknowledgments We would like to thank professor Iuri Frosio from University of Milan in Italy to provide us with the
[1] H.L. Eng, K.K. Ma, Noise adaptive soft-switching median filter, IEEE Transactions on Image Processing 10 (2) (2001) 242–251. [2] S.Q. Yuan, Y.H. Tan, H.L. Sun, Impulse noise removal by the difference-type noise detector and the cost function-type filter, Signal Processing 87 (10) (2007) 2417–2430. [3] D. Dang, W. Luo, Impulse noise removal utilizing second-order difference analysis, Signal Processing 87 (9) (2007) 2017–2025. [4] S.Q. Yuan, Y.H. Tan, Impulse noise removal by a global–local noise detector and adaptive median filter, Signal Processing 86 (8) (2006) 2123–2128. [5] P.-E. Ng, K.-K. Ma, A switching median filter with boundary discriminative noise detection for extremely corrupted image, IEEE Transactions on Image Processing 15 (6) (2006) 1506–1516. [6] Z.F. Deng, Z.P. Yin, Y.L. Xiong, High probability impulse noiseremoving algorithm based on mathematical morphology, IEEE Signal Processing Letters 14 (1) (2007) 31–34. [7] K.S. Srinivasan, D. Ebenezer, A new fast and efficient decision-based algorithm for removal of high-density impulse noises, IEEE Signal Processing Letters 14 (3) (2007) 189–192. [8] P.Y. Chen, C.Y. Lien, An efficient edge-preserving algorithm for removal of salt-and-pepper noise, IEEE Signal Processing Letters 15 (2008) 833–836. [9] X.M. Zhang, Y.L. Xiong, Impulse noise removal using directional weighted noise detector and adaptive weighted mean filter, IEEE Signal Processing Letters 16 (4) (2009) 295–298. [10] S.S. Wang, C.H. Wu, A new impulse detection and filtering method for removal of wide range impulse noises, Pattern Recognition 42 (9) (2009) 2194–2202. [11] K.K.V. Toh, N.A.M. Isa, Noise adaptive fuzzy switching median filter for salt-and-pepper noise reduction, IEEE Signal Processing Letters 17 (3) (2010) 281–284. [12] G. Yu, L. Qi, Y. Sun, Y. Zhou, Impulse noise removal by a nonmonotone adaptive gradient method, Signal Processing 90 (10) (2010) 2891–2897. [13] A. Buades, B. Coll, J.M. Morel, A review of image denoising algorithms, with a new one, Multiscale Modeling and Simulation 4 (2) (2005) 490–530. [14] T. Brox, O. Kleinschmidt, D. Cremers, Efficient nonlocal means for denoising of textural patterns, IEEE Transactions on Image Processing 17 (7) (2008) 1083–1092. [15] Q. Chen, Q.S. Sun, D.S. Xia, Homogeneity similarity based image denoising, Pattern Recognition 43 (12) (2010) 4089–4100. [16] P. Coup, P. Hellier, C. Barillot, Nonlocal means-based speckle filtering for ultrasound images, IEEE Transactions on Image Processing 18 (10) (2009) 2221–2229. [17] P. Coupe, P. Yger, S. Prima, P. Hellier, C. Kervrann, C. Barillot, An optimized blockwise nonlocal means denoising filter for 3-D magnetic resonance images, IEEE Transactions on Medical Imaging 27 (4) (2008) 425–441. [18] H. Liu, C. Yang, N. Pan, E. Song, R. Green, Denoising 3D MR images by the enhanced non-local means filter for Rician noise, Magnetic Resonance Imaging 28 (2010) 1485–1496. [19] C. Deledalle, F. Tupin, L. Denis, Poisson NL-means: unsupervised non-local means for Poisson noise, in: IEEE International Conference on Image Processing (ICIP), Hong Kong, China, September 2010, pp. 801–804. [20] Z. Wang, A.C. Bovik, H.R. Sheik, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing 13 (4) (2004) 1–14.