- Email: [email protected]
ANFIS filter for efficient restoration of digital images corrupted by impulse noise
Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637 www.elsevier.de/aeue
A median/ANFIS filter for efficient restoration of digital images corrupted by impulse noise M. Emin Yüksel∗ Digital Signal and Image Processing Laboratory, Department of Electrical and Electronics Engineering, Erciyes University, Kayseri 38039, Turkey Received 22 April 2005
Abstract A new operator for removing impulse noise from digital images is presented. The proposed operator is a hybrid filter constructed by combining four center-weighted median filters (CWMF) with a simple adaptive neuro-fuzzy inference system (ANFIS). The internal parameters of the ANFIS are optimized by training. The training is performed by using simple artificial images which can easily be generated by computer. The fundamental advantage of the proposed operator over other operators in the literature is that it efficiently removes impulse noise while at the same time effectively preserves image details and texture. Performance of the proposed operator is tested at various noise densities and for different test images, and also compared with conventional as well as state-of-the-art impulse noise removal operators. Experimental results show that the proposed operator significantly outperforms the other operators and efficiently removes impulse noise from digital images without distorting image details and texture. 䉷 2005 Elsevier GmbH. All rights reserved. Keywords: Image processing; Noise filtering; Median filter; Neuro-fuzzy systems
1. Introduction Digital images are valuable and important sources of information for a variety of research and application areas including astronomy, biology, medicine, remote sensing, materials science and so on. Due to a number of nonidealities encountered in image sensors and communication channels, recorded images are often corrupted by impulse noise during image acquisition and/or transmission. In most applications, it is very important to remove the noise from the image data because the performances of subsequent image processing tasks, such as edge detection, image segmentation, object recognition, etc., are severely degraded by noise. Efficient removal of noise from image data is a difficult task for any image processing system because the useful ∗ Tel.: +90 352 4374901/32204; fax: +90 352 4375784.
E-mail address: [email protected] 1434-8411/$ - see front matter 䉷 2005 Elsevier GmbH. All rights reserved. doi:10.1016/j.aeue.2005.12.005
information in the original image must not be distorted by the noise filter. Conventional noise filters usually have the drawback of introducing undesirable distortions and blurring effects into the output image during noise removal process [1,2]. In the last decade, removal of impulse noise from digital images has been one of the popular research topics in image processing and many filtering methods have been proposed. These methods may mainly be divided into two groups. The first group of methods exploits the rank order information (i.e. order statistics) of the noisy pixels in a given filtering window. These are usually based on the standard median filter [1,2] and its derivatives. The standard median filter is a simple rank selection filter and attempts to remove impulse noise by changing the luminance value of the noisy pixel with the median of the luminance values of the pixels contained within the filtering window. This simple filtering approach yields a reasonable noise removal performance,
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
but introduces significant blurring effects into image details even at low-noise densities. The median filter has been of high research interest since its application to impulse noise removal and a number of median-based filtering approaches trying to avoid the inherent disadvantages of the standard median filter have been proposed. The weighted median filter [3] and the centerweighted median filter (CWMF) [4] are modified median filters capable of controlling the tradeoff between the noise suppression and detail preservation ability by giving more weight to the appropriate pixels of the filtering window. Hence, they may be designed to offer better detail preservation performance than the median filter at the cost of reduced noise removal performance. A method for optimal design of these filters is presented in [5]. A number of methods [6–12] combine the median filter with a decision mechanism which aims to determine whether the center pixel of a given filtering window is corrupted or not. If the center pixel is classified by the decision mechanism as a corrupted pixel, its restored value is obtained by processing the pixels in the filtering window by the median filter. Otherwise, i.e. if the center pixel is classified as uncorrupted, it is left unchanged. Although this approach significantly enhances the performance of the median filter by reducing its distortion effects, its performance inherently depends on the performance of the impulse detector. As a consequence, several different impulse detection approaches exploiting median filters [6], multiple CWMFs with different center weights [7], edge detection kernels [8], histogram-based methods [9], iterative order statistic filters [10], statistical tests [11] and boolean functions [12] have been proposed. The progressive switching median filter [13] is a modified switching median filter in which detection and removal of impulse noise are iteratively done in two separate stages. This filter exhibits improved noise suppression performance than some other median-based filters but it has a very high computational complexity due to its iterative nature. Some more complicated derivatives of the basic switching median filter employing multiple median-based filters in the structure have also been proposed. The tri-state median filter [14] is an improved switching median filter that is obtained by adding a CWMF into the basic switching median filter structure. The multi-state median filter [15] is a further extended version of the tri-state median filter. These two filters exhibit better filtering performance at the expense of increased computational complexity. The signal-dependent rank-ordered mean filter [16] is another kind of switching filter exploiting rank order information for impulse noise cancellation. The structure of the filter is very similar to that of a basic switching median filter. The only difference is that the median filter is replaced with a rank-ordered mean filter. The signaldependent rank-ordered mean filter has significantly better noise removal and detail preservation properties than some conventional as well as state-of-the-art impulse noise filters
629
for both gray scale [16] and color [17] images. The minimum/maximum exclusive mean filter [18] is another type of switching rank-ordered mean filter. This filter exhibits good filtering performance at the cost of increased computational complexity. All of the above-mentioned methods more or less have the drawback of distorting the details and texture in the input image during noise removal process. For this reason, there has been a growing research interest in the last few years in the application of new nonlinear techniques, such as neural networks and fuzzy systems, to the problems in image processing [19–27]. Indeed, neuro-fuzzy (NF) systems combine in a single system the ability of neural networks to learn from examples and the capability of fuzzy systems to model the uncertainty which is encountered when both preservation of the details and removal of the noise are jointly required. Therefore, NF systems may be employed as very powerful tools for the removal of impulse noise from digital images provided that appropriate network structures and processing strategies are utilized. Hence, impulse noise removal methods in the second group comprise nonlinear filtering operators based on neural networks and fuzzy systems. One important subset of this family of operators is fuzzy filters based on fuzzy IF–THEN and IF–THEN–ELSE rules [19,22]. These operators usually employ a set of fuzzy rules for detecting the impulse noise at the center pixel of a given filtering window, and an appropriate inference mechanism for removing it. Although they perform relatively better than the median-based operators regarding noise removal, they are inherently heuristic and the determination of the fuzzy rules may be quite difficult especially for high noise densities. In order to avoid this problem, NF approaches allowing the determination of the rule base and the parameters of the fuzzy filter by utilizing a set of training data have also been proposed [23,26]. Although these methods exhibit relatively better noise suppression and detail preservation performance, they usually have much higher complexity than the other methods. In this paper, a novel impulse noise filtering operator is presented. The fundamental advantage of the proposed operator over other operators in the literature is that it efficiently removes impulse noise from digital images while at the same time effectively preserves image details and texture. The proposed operator is a hybrid filter constructed by combining four CWMFs with a simple adaptive neuro-fuzzy inference system (ANFIS). The internal parameters of the ANFIS are optimized by training. The training is performed by using a simple artificial training image which can easily be generated by computer. Performance of the proposed operator is tested at various noise densities and for different test images, and also compared with conventional as well as state-of-the-art impulse noise removal operators. Experimental results show that the proposed operator significantly outperforms the other operators and efficiently removes impulse noise from digital images without distorting image details and texture.
630
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
2. Method 2.1. The proposed operator Fig. 1 shows the structure of the proposed impulse noise removal operator. The operator is a hybrid filter constructed by combining four CWMFs with a simple ANFIS. Each CWMF in the structure has a different order, i.e. a different center weight. The operator uses a minimal 3 × 3 pixel filtering window, which is also the filtering window for the four CWMFs in the structure. The orders of the CWMFs increase from 0 to 3. Hence the notation CWMF(k) in Fig. 1 represents a kth-order CWMF, which has a center weighting of (2k + 1).
2.2. The center-weighted median filter Let x[r, c] denote the luminance value of the pixel at location (r, c) of the noisy input image having a size of R × C pixels. Here, r and c are the row and the column indices, respectively, with 1 r R and 1 c C. Let W [r, c] represent the group of pixels contained in a filtering window with the size of 3 × 3 pixels and centered at location (r, c) of the noisy input image: W [r, c] = {x[r + p, c + q] | (p, q) ∈ [−1, 0, 1]},
(1)
where p and q are integer indices each individually ranging from −1 to 1. Similarly, let W (w) [r, c] represent the group of pixels contained in a center-weighted filtering window having the same size and location as W [r, c]: W (w) [r, c] = {W [r, c], (w − 1) ♦ x[r, c]}
(w 1),
(2)
where ♦ represents the replication operator, i.e. the expression (w − 1) ♦ x[r, c] produces (w − 1) copies of the center pixel x[r, c]. Hence, W (w) [r, c] contains the same pixels as in W [r, c] except that there are w copies of the center pixel x[r, c] in W (w) [r, c]. The kth-order CWMF is a simple rank selection filter outputting the median of the pixels contained in a centerweighted filtering window with a center weight of (2k + 1): (k)
ycwmf [r, c] = Median(W (2k+1) [r, c]) (k 0),
(3)
(k)
where ycwmf [r, c] is the output of the kth-order CWMF and k is an integer number (k 0). The CWMF is equivalent to a standard MF for k = 0: (0)
ycwmf [r, c] = Median(W (1) [r, c]) = Median(W [r, c]).
(4)
Similarly, for k > 3, the CWMF is equivalent to an identity filter, which directly outputs the center pixel of the filtering window, for a 3 × 3 pixel filtering window: (k)
ycwmf [r, c] = x[r, c] for k > 3.
(5)
Fig. 1. Structure of the proposed impulse noise removal operator. The operator comprises four center-weighted median filters (CWMF) each with a different order and a simple adaptive neuro-fuzzy inference system (ANFIS).
It is shown in [4] that a CWMF with a smaller center weight provides better noise suppression performance than a CWMF with a larger center weight. It is also shown that a CWMF with a larger center weight provides better detail preservation performance than a CWMF with a smaller center weight. As a consequence, the center weight of the CWMF can be appropriately varied to trade off between noise suppression and detail preservation.
2.3. The adaptive neuro-fuzzy inference system The ANFIS used in the structure of the proposed impulse noise removal operator is a first-order Sugeno-type fuzzy system with five inputs and one output [28]. Each input has two generalized bell-type membership functions whereas the output has a linear membership function. The input-output relationship of the ANFIS is as follows: Let X1 · · · X5 denote the inputs of the ANFIS and Y denote its output. Each possible combination of inputs and their associated membership functions is represented by a rule in the rule base of the ANFIS. Since the ANFIS has 5 inputs and each input has 2 membership functions, the rule base contains a total of 32 (25 ) rules, which are as follows: 1. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M31 ) and (X4 is M41 ) and (X5 is M51 ), then R1 = F1 (X1 , X2 , X3 , X4 , X5 ). 2. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M31 ) and (X4 is M41 ) and (X5 is M52 ), then R2 = F2 (X1 , X2 , X3 , X4 , X5 ). 3. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M31 ) and (X4 is M42 ) and (X5 is M51 ), then R3 = F3 (X1 , X2 , X3 , X4 , X5 ). 4. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M31 ) and (X4 is M42 ) and (X5 is M52 ), then R4 = F4 (X1 , X2 , X3 , X4 , X5 ). 5. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M32 ) and (X4 is M41 ) and (X5 is M51 ), then R5 = F5 (X1 , X2 , X3 , X4 , X5 ). .. .
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
631
32. if (X1 is M12 ) and (X2 is M22 ) and (X3 is M32 ) and (X4 is M42 ) and (X5 is M52 ), then R32 = F32 (X1 , X2 , X3 , X4 , X5 ). where Mij denotes the jth membership function of the ith input, Rk denotes the output of the kth rule, and Fk denotes the kth output membership function. The input membership functions are generalized bell type and the output membership functions are linear: Mij (u) =
1 , u − aij 2cij 1+ bij
Fk (u1 , u2 , u3 , u4 , u5 ) = dk1 u1 + dk2 u2 + dk3 u3 + dk4 u4 + dk5 u5 + dk6 ,
(6)
(7)
where i = 1, 2, . . . , 5, j = 1, 2 and k = 1, 2, . . . , 32. Here the parameters a, b, c and d are constants that characterize the shape of the membership functions. The optimal values of these parameters are determined by training, which will be discussed in detail later on. The output of the ANFIS is the weighted average of the individual rule outputs. The weighting factor, wk , of each rule is calculated by evaluating the membership expressions in the antecedent of the rule. This is accomplished by first converting the input values to fuzzy membership values by utilizing the input membership functions and then applying the and operator to these membership values. The and operator corresponds to the multiplication of input membership values. Hence, the weighting factors of the rules are calculated as follows: w1 w2 w3 w4 w5 .. . w32
= = = = = =
M11 (X1 )M21 (X2 )M31 (X3 )M41 (X4 )M51 (X5 ) M11 (X1 )M21 (X2 )M31 (X3 )M41 (X4 )M52 (X5 ) M11 (X1 )M21 (X2 )M31 (X3 )M42 (X4 )M51 (X5 ) M11 (X1 )M21 (X2 )M31 (X3 )M42 (X4 )M52 (X5 ) M11 (X1 )M21 (X2 )M32 (X3 )M41 (X4 )M51 (X5 ) .. . M12 (X1 )M22 (X2 )M32 (X3 )M42 (X4 )M52 (X5 ). (8)
Once the weighting factors are obtained, the output of the ANFIS can be found by calculating the weighted average of the individual rule outputs: 32 k=1 wk Rk . (9) Y= 32 k=1 wk Readers interested in the details of fuzzy systems may refer to an excellent book on this subject [28].
2.4. Training of the ANFIS The internal parameters of the ANFIS are optimized by training. Fig. 2 represents the setup used for training. Here, the parameters of the ANFIS are iteratively optimized so that its output converges to the output of the ideal noise filter
Fig. 2. Training of the ANFIS.
Fig. 3. The training images: (a) original training image and (b) noisy training image.
which, by definition, can completely remove the noise from the image. The ideal noise filter is conceptual only and does not necessarily exist in reality. It is only the output of the ideal noise filter that is necessary for training, and this is represented by the original (noise-free) training image. Fig. 3 shows the images used for training. The image shown in Fig. 3a is the original (noise-free) training image, which is a 128 × 128 pixel artificial image that can easily be generated in a computer. Each square box in this image has a size of 4 × 4 pixels and the 16 pixels contained within each box have the same luminance value, which is a random integer number uniformly distributed in [0, 255]. The image in Fig. 3b is the noisy training image and is obtained by corrupting the original training image by impulse noise of 30% noise density. Although the density of the corrupting noise is not very critical regarding training performance, it is experimentally observed that the operator exhibits the best performance when the noise density of the input training image is equal to the noise density of the noisy input image to be restored. It is also observed that the performance of the operator slightly decreases as the difference between the two noise densities increases. Hence, in order to obtain a stable filtering performance for a wide range of filtering noise densities, very low and very high values for training noise density should be avoided since it is usually impossible to know the actual noise density of a corrupted image in a real practical application. Results of extensive simulation experiments indicate that very good filtering performance is easily obtained for all kinds of images corrupted by impulse
632
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
noise with a noise density between 0% and 80% provided that the input training image has a noise density between 20% and 50%. The images in Figs. 3a and b are employed as the target (desired) and the input images during training, respectively. The parameters of the ANFIS are then iteratively tuned by using the Levenberg–Marquardt optimization algorithm [28] so as to minimize the learning error. Once the training of the ANFIS is completed, its internal parameters are fixed and it is combined with the CWMFs to construct the proposed operator, as shown in Fig. 1.
2.5. Filtering of the noisy image The noisy input image is processed by sliding a 3 × 3 filtering window on the image. The window is started from the upper-left corner of the image and moved sideways and progressively downwards in a raster scanning fashion. For each filtering window, the pixels contained within the window are fed to the CWMFs in the structure. Each CWMF appropriately weights the filtering window and generates at its output the restored value of the center pixel of the window by employing the filtering scheme discussed before. Hence, each CWMF in the structure generates a different restored subimage for the same noisy input image. As discussed before, the lower the order of a given CWMF in the structure, the higher the degree of noise suppression and the lower the degree of detail preservation in the subimage generated by this CWMF, and vice versa. Therefore, each subimage represents a different balance between detail preservation and noise suppression. These subimages are fed to the inputs of the ANFIS. The ANFIS generates the final output image of the proposed operator by employing the fuzzy inference mechanism discussed before.
3. Results The proposed operator discussed in the previous section is implemented. The performance of the operator is tested under various noise conditions and on several popular images from the literature including Boats, Bridge, Lena, Pentagon and Peppers. These images are shown in Fig. 4. All images are 8-bit gray level images having the same size of 256 × 256 pixels. The test images used in the experiments are generated by contaminating the original images by impulse noise with an appropriate noise density depending on the experiment. For comparison, the corrupted test images are also filtered by using several conventional and state-of-the-art impulse noise removal operators including the standard median filter (MF) [1], the edge-detecting median filter (EDMF) [8], the multi-state median filter (MSMF) [15], the progressive switching median filter (PSMF) [13], the signal-dependent rank-ordered mean filter (SDROMF) [16] and a fuzzy filter
Fig. 4. Test images: (a) Boats, (b) Bridge, (c) Lena, (d) Pentagon and (e) Peppers.
(FF) [19]. These filters are representative implementations of different approaches to impulse noise filtering problem. The EDMF, MSMF, PSMF, SDROMF and FF operators have a number of tuning parameters. Unfortunately, there are no analytical methods available in the literature to determine the optimal values for these parameters that yield the best results for a given filtering experiment. Hence, the values of these parameters are heuristically determined. In the experiments presented in this paper, the values suggested in the corresponding references are used. These are as follows: For EDMF, T = 116 [8]. For MSMF, wmax = 5 and T = 20 [15]. For PSMF, ND = 3 and NF = 3 [13]. For SDROMF, 1 = 0, 2 = 1 and the thresholds {T1 , T2 , T3 , T4 } are {8, 20, 40, 50} [16]. For FF, L = 256, a = 40 and b = 32 [19]. All filters including the proposed filter operate on a 3 × 3 filtering window, except the EDMF filter which performs better with a 5 × 5 window [8]. The performances of all operators are evaluated by using the mean-squared error (MSE) criterion, which is defined as MSE =
C R 1 (s[r, c] − y[r, c])2 , RC r=1 c=1
(10)
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
633
Table 1. Comparison of the overall filtering performances of the operators
Noisy MF MSMF EDMF SDROMF FF PSMF Proposed
Boats
Bridge
Lena
Pentagon
Peppers
Avrg.
4749 370 270 265 236 211 279 104
4818 376 258 233 223 193 205 115
4953 293 229 187 170 180 162 82
4361 389 269 214 227 169 191 117
4807 253 197 162 131 136 123 69
4738 336 245 212 197 178 192 102
The test images are corrupted by 25% impulse noise. The MSE values are calculated for all pixels of the input images.
Table 2. Comparison of the noise suppression performances of the operators
Noisy MF MSMF EDMF SDROMF FF PSMF Proposed
Boats
Bridge
Lena
Pentagon
Peppers
Avrg.
19 015 649 862 648 473 516 546 428
18 311 650 859 642 495 550 501 456
19 854 537 777 512 380 484 449 329
17 420 643 771 575 424 421 383 403
19 245 505 724 419 307 355 348 262
18 769 597 799 559 416 465 445 376
The test images are corrupted by 25% impulse noise. The MSE values are calculated for only the corrupted pixels of the input images.
Table 3. Comparison of the detail preservation performances of the operators
Noisy MF MSMF EDMF SDROMF FF PSMF Proposed
Boats
Bridge
Lena
Pentagon
Peppers
Avrg.
0 277 72 137 157 109 190 1
0 285 58 97 133 74 106 12
0 212 47 79 100 79 67 22
0 304 102 93 162 84 127 4
0 169 21 76 72 63 48 17
0 249 60 96 125 82 108 11
The test images are corrupted by 25% impulse noise. The MSE values are calculated for only the uncorrupted pixels of the input images.
where s[r, c] and y[r, c] represent the original and the restored versions of a corrupted test image, respectively. Several experiments are performed to measure and compare the noise suppression and detail preservation performances of all operators. The experiments are especially designed to reveal the performances of the operators for different image properties and noise conditions. Since all experiments are related with noise and noise is a random process, every realization of the same experiment yields different results even if the experimental conditions are the same. Therefore, each individual filtering experiment presented in this paper is repeated for 10 times yielding 10 different MSE values for the same experiment. The average of these values is then taken as the representative MSE value for that experiment.
3.1. Experiment Set 1 (constant noise density) In the first set of experiments, the original images shown in Fig. 4 are corrupted by impulse noise having a noise density of 25%. The corrupted images are filtered by all operators and the MSE values of the output images are calculated. In order to obtain a deeper analysis of the filtering behavior and to separately evaluate the noise suppression and detail preservation ability of the operators, the MSE values are separately calculated (i) for all pixels, (ii) for corrupted pixels only and (iii) for uncorrupted pixels only. These are given in Tables 1–3, respectively. The MSE values listed in Table 1 are calculated for all pixels and reflect the overall filtering performances of the operators. As it is seen from this table, performance of the
634
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
MF is the worst of all for all test images. The MSMF is considerably better than the MF but worse than the others. The performances of the EDMF, PSMF, SDROMF and FF are very close to each other. The proposed filter, however, exhibits the best performance for all test images. Table 2 lists the MSE values calculated for only the corrupted pixels of the test images for a comparative evaluation of the noise removal performances of the operators. This time the performance of the MF is better than the MSMF, as expected. This is because the MSMF is based on a certain number of CWMFs with different center weights. As discussed in Section 1, the CWMF demonstrates better detail preservation performance than the standard median filter at the cost of reduced noise removal performance. Hence, the noise removal performance of the MSMF is worse than that of the MF. The performances of the SDROMF, PSMF and FF are again very close to each other, whereas the EDMF yields relatively higher MSE values. This is because the EDMF uses edge detection kernels to detect impulses. The decrease in the performance of this operator may be attributed to the false classification of a number of impulses as edge pixels. As a consequence, these pixels are left unfiltered, which results in a decrease in the noise suppression performance of the EDMF operator. On the other hand, the performance of the proposed filter is considerably better than the other filters for all images except the Pentagon image for which the PSMF operator performs better. The real power of the proposed filter is revealed by Table 3, which lists the MSE values calculated for only the uncorrupted pixels of the test images for a comparative evaluation of the detail preservation performances of the filters. An ideal noise filter would only process the corrupted pixels of a noisy input image and leave the uncorrupted pixels unchanged. Therefore, the MSE value calculated for only the uncorrupted pixels of a given test image would be zero for an ideal noise filter. However, it is easily seen from Table 3 that the MSE values of all filters except the proposed filter are significantly higher than zero. This obviously implies that these filters significantly distort the uncorrupted regions of the test images during restoration of the corrupted regions, resulting in undesirable blurring effects in the details of the input image and loss of useful information within the image. On the other hand, the MSE values of the proposed filters are significantly lower than those of the other filters and much closer to zero. This clearly indicates that the filtering behavior of the proposed filter is much closer to that of an ideal filter regarding detail preservation when compared to the other filters. Hence, the proposed filter effectively preserves the useful information contained within the uncorrupted regions of the input image during restoration of the corrupted regions. The output images of all operators for the Lena image corrupted by impulse noise with 25% noise density are shown in Fig. 5 for a visual evaluation of the noise removal
Fig. 5. Output images of the operators for the Lena image corrupted by impulse noise with 25% noise density: (a) Noisy Lena image, (b) MF, (c) MSMF, (d) EDMF, (e) SDROMF, (f) FF, (g) PSMF and (h) Proposed.
and detail preservation performances of the operators. It is observed from this figure that the performance of the MF and MSMF are very close to each other. Some noise blotches are easily visible in the output images of these two filters. The EDMF exhibits better detail preservation performance than the other operators at the cost of slightly reduced noise suppression performance. The output images of the SDROMF, FF and PSMF are almost indistinguishable from each other and they are significantly better than those of the MF, MSMF and EDMF regarding noise
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
635
Table 4. Performances of all operators for varying noise density
Noisy MF MSMF EDMF SDROMF FF PSMF Proposed
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
948 136 64 71 83 47 51 12
1897 157 85 90 102 71 76 27
2839 189 115 119 126 99 103 47
3799 245 163 159 158 135 133 66
4738 336 245 212 197 178 192 97
5698 491 372 282 249 231 241 128
6649 733 575 371 311 294 281 170
7593 1077 867 489 398 378 326 236
8534 1572 1291 647 515 491 381 304
9488 2226 1865 873 677 629 442 398
suppression. However, it is observed that these operators introduce some amount of blur distortions into the details of the image. The proposed filter, however, demonstrates very good noise suppression and excellent detail preservation performance. The difference, especially in detail preservation, can easily be observed by carefully comparing the appearance of the eyes and the face of the lady as well as the feathers on the hat.
3.2. Experiment Set 2 (varying noise density) The second set of experiments are designed to evaluate the dependency of the performances of the operators on noise density. For this purpose, the noise density is increased from 5% to 50% with 5% steps. For each noise density step, the five test images shown in Fig. 4 are corrupted by impulse noise with that noise density. This produces five different experimental images each having the same noise density. These images are filtered by one of the operators and the MSE values are calculated for all pixels of the output images, producing five different MSE values representing the filtering performance of that operator under different image properties. These values are then averaged to obtain the representative MSE value of that operator for that noise density. This procedure is separately repeated for all noise densities to obtain the variation of the average MSE value of that operator as a function of noise density. Finally, the overall experimental procedure is individually repeated for each operator. Table 4 shows the variation of the average MSE values of the operators as a function of noise density. As it is seen from this table, the performances of the MF and MSMF operators are very poor, with the MSMF being better than the MF. The EDMF performs better than the MF and MSMF but worse than the others. As discussed in the previous subsection (Experiment Set 1), the EDMF operator utilizes edge detection kernels to distinguish between corrupted, uncorrupted and edge pixels. In this way, the smoothing effect of the filter, which is desirable for noisy pixels but undesirable for edge pixels, is suitably controlled. But, the fundamental drawback of this approach is that an increasing number of noisy pixels are incorrectly detected as edge pixels as the
noise density increases. As a result, the performance of the EDMF operator relatively decreases with increasing noise density. This is observed in Table 4. The performances of the SDROMF and FF operators are very close to each other especially for higher noise densities. The FF operator exhibits considerably better filtering performance than the MF, MSMF and EDMF operators for all noise densities. The filtering performance of the PSMF operator is relatively better at higher noise densities than at lower noise densities. As discussed in the previous subsection, this operator exhibits very good noise suppression performance but significantly distorts image details and texture. As the noise density increases, the number of corrupted pixels in the input images increases. As a consequence, the contribution of the noise suppression performance to the general filtering performance becomes more dominant than the contribution of the detail preservation performance. Therefore, the performance of the PSMF operator becomes better than the other operators for relatively higher noise densities, as observed in Table 4. The proposed operator, however, demonstrates the best filtering performance of all. Its MSE values are significantly lower than those of the other filters for all noise densities.
4. Discussion and conclusion A novel operator for removing impulse noise from digital images is presented. The fundamental superiority of the proposed operator over other operators is that it efficiently removes impulse noise from digital images while successfully preserving the details and texture in the original image. The advantages of the new operator over other operators in the literature may be summarized as follows: 1. It has a very simple structure. It is constructed by appropriately combining four CWMFs with a simple ANFIS. The structure of the ANFIS is also very simple. The neuro-fuzzy-based impulse noise filtering operators in the literature usually have complicated network structures [23,26].
636
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
2. It requires no performance tuning parameters. However, majority of the filtering operators available in the literature [3–19] need one or more parameters that are heuristically determined and externally supplied by the user. In most cases, it is difficult to determine the optimal set of these parameters that yields the best filtering performance for a given image and/or given noise density since there is no analytical method for this purpose. 3. The proposed operator uses a minimal 3 × 3 pixel filtering window whereas some other operators in the literature require filtering windows with larger sizes [8,15,23]. The larger the size of the filtering window, the higher the number of pixels to be processed by the operator to calculate the restored value of the center pixel, hence the longer the time required for the restoration of the noisy image. 4. The ANFIS used in the structure of the proposed operator uses one-dimensional fuzzy membership functions, which simplifies implementation. On the other hand, some neuro-fuzzy systems related with impulse noise filtering use two-dimensional fuzzy membership functions [23]. 5. The ANFIS is trained by using very simple artificial images that can easily be generated in a computer. However, contrary to its simplicity in implementation and convenience in training, it may be used for efficiently filtering any image corrupted by impulse noise of virtually any noise density. It is concluded that the proposed operator can be used as a simple but powerful tool for efficient removal of impulse noise from digital images without distorting the useful information within the image.
References [1] Umbaugh SE. Computer vision and image processing. Englewood Cliffs, NJ: Prentice-Hall; 1998. [2] Gonzales RC, Woods RE. Digital image processing. 2nd ed., Reading, MA: Addison-Wesley; 2002. [3] Yli-Harja O, Astola J, Neuvo Y. Analysis of the properties of median and weighted median filters using threshold logic and stack filter representation. IEEE Trans Signal Process 1991;39(2):395–410. [4] Ko SJ, Lee YH. Center weighted median filters and their applications to image enhancement. IEEE Trans Circuits Syst 1991;38(9):984–93. [5] Jeong B, Lee YH. Design of weighted order statistic filters using the perceptron algorithm. IEEE Trans Signal Process 1994;42(11):3264–9. [6] Sun T, Neuvo Y. Detail-preserving median based filters in image processing. Pattern Recogn Lett 1994;15(4):341–7. [7] Chen T, Wu HR. Adaptive impulse detection using centerweighted median filters. IEEE Signal Proc Lett 2001;8(1): 1–3. [8] Zhang S, Karim MA. A new impulse detector for switching median filters. IEEE Signal Proc Lett 2002;9(11):360–3.
[9] Pok G, Liu Y, Nair AS. Selective removal of impulse noise based on homogeneity level information. IEEE Trans Image Process 2003;12(1):85–92. [10] Crnojevic V, Senk V, Trpovski Z. Advanced impulse detection based on pixel-wise MAD. IEEE Signal Proc Lett 2004;11(7):589–92. [11] Be¸sdok E, Yüksel ME. Impulsive noise rejection from images with Jarque–Berra test based median filter. Int J Electron Commun (AEÜ) 2005;59(2):105–10. [12] Aizenberg I, Butakoff C, Paliy D. Impulsive noise removal using threshold boolean filtering based on the impulse detecting functions. IEEE Signal Proc Lett 2005;12(1):63–6. [13] Wang Z, Zhang D. Progressive switching median filter for the removal of impulse noise from highly corrupted images. IEEE Trans Circuits Syst 1999;46(1):78–80. [14] Chen T, Ma KK, Chen LH. Tri-state median filter for image denoising. IEEE Trans Image Process 1999;8(12):1834–8. [15] Chen T, Wu HR. Space variant median filters for the restoration of impulse noise corrupted images. IEEE Trans Circuits Syst—II 2001;48(8):784–9. [16] Abreu E, Lightstone M, Mitra SK, Arakawa K. A new efficient approach for the removal of impulse noise from highly corrupted images. IEEE Trans Image Process 1996;5(6): 1012–25. [17] Moore MS, Gabbouj M, Mitra SK. Vector sd-rom filter for removal of impulse noise from color images. In: Proceedings of the ECMCS’99 EURASIP conference on DSP for multimedia communications and services, Krakow, 1999. [18] Han WY, Lin JC. Minimum–maximum exclusive mean (MMEM) filter to remove impulse noise from highly corrupted images. Electron Lett 1997;33(2):124–5. [19] Russo F, Ramponi G. A fuzzy filter for images corrupted by impulse noise. IEEE Signal Proc Lett 1996;3(6):168–70. [20] Choi YS, Krishnapuram R. A robust approach to image enhancement based on fuzzy logic. IEEE Trans Image Process 1997;6(6):808–25. [21] Pham DL, Prince JL. Adaptive fuzzy segmentation of magnetic resonance images. IEEE Trans Med Imag 1999;18(9):737–52. [22] Russo F. FIRE operators for image processing. Fuzzy Set Syst 1999;103(2):265–75. [23] Russo F. Noise removal from image data using recursive neurofuzzy filters. IEEE Trans Instrum Meas 2000;49(2): 307–14. [24] Park DC, Woo YJ. Weighted centroid neural network for edge preserving image compression. IEEE Trans Neural Networks 2001;12(5):1134–46. [25] Boskowitz V, Guterman H. An adaptive neuro-fuzzy system for automatic image segmentation and edge detection. IEEE Trans Fuzzy Syst 2002;10(2):247–62. [26] Yüksel ME, Ba¸stürk A. Efficient removal of impulse noise from highly corrupted digital images by a simple neuro-fuzzy operator. Int J Electron Commun (AEÜ) 2003;57(3):214–9. [27] Yüksel ME, Yıldırım MT. A simple neuro-fuzzy edge detector for digital images corrupted by impluse noise. Int J Electron Commun (AEÜ) 2004;58(1):72–5. [28] Jang JSR, Sun CT, Mizutani E. Neuro-fuzzy and soft computing. Englewood Cliffs, NJ: Prentice-Hall; 1997.
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
M. Emin Yüksel received his B.Sc. degree in Electronics and Communications Engineering from the Technical University of Istanbul, Istanbul, Turkey, in July 1990. In February 1991, he joined as a Research Assistant to the Department of Electronics Engineering, Erciyes University, Kayseri. He received his M.Sc. and Ph.D. degrees in Electronics Engineering from
637
Erciyes University in February 1993 and September 1996, respectively. Between March- and December 1995, he had been an academic visitor to Signal Processing Section, Department of Electrical Engineering, Imperial College, London, UK, where he conducted research on time-varying parametric modeling of nonstationary signals. Currently, he is with the Department of Electronics Engineering of Erciyes University. His general research interests include signal processing, image processing, neural networks, fuzzy systems and applications of these techniques.
A median/ANFIS filter for efficient restoration of digital images corrupted by impulse noise M. Emin Yüksel∗ Digital Signal and Image Processing Laboratory, Department of Electrical and Electronics Engineering, Erciyes University, Kayseri 38039, Turkey Received 22 April 2005
Abstract A new operator for removing impulse noise from digital images is presented. The proposed operator is a hybrid filter constructed by combining four center-weighted median filters (CWMF) with a simple adaptive neuro-fuzzy inference system (ANFIS). The internal parameters of the ANFIS are optimized by training. The training is performed by using simple artificial images which can easily be generated by computer. The fundamental advantage of the proposed operator over other operators in the literature is that it efficiently removes impulse noise while at the same time effectively preserves image details and texture. Performance of the proposed operator is tested at various noise densities and for different test images, and also compared with conventional as well as state-of-the-art impulse noise removal operators. Experimental results show that the proposed operator significantly outperforms the other operators and efficiently removes impulse noise from digital images without distorting image details and texture. 䉷 2005 Elsevier GmbH. All rights reserved. Keywords: Image processing; Noise filtering; Median filter; Neuro-fuzzy systems
1. Introduction Digital images are valuable and important sources of information for a variety of research and application areas including astronomy, biology, medicine, remote sensing, materials science and so on. Due to a number of nonidealities encountered in image sensors and communication channels, recorded images are often corrupted by impulse noise during image acquisition and/or transmission. In most applications, it is very important to remove the noise from the image data because the performances of subsequent image processing tasks, such as edge detection, image segmentation, object recognition, etc., are severely degraded by noise. Efficient removal of noise from image data is a difficult task for any image processing system because the useful ∗ Tel.: +90 352 4374901/32204; fax: +90 352 4375784.
E-mail address: [email protected] 1434-8411/$ - see front matter 䉷 2005 Elsevier GmbH. All rights reserved. doi:10.1016/j.aeue.2005.12.005
information in the original image must not be distorted by the noise filter. Conventional noise filters usually have the drawback of introducing undesirable distortions and blurring effects into the output image during noise removal process [1,2]. In the last decade, removal of impulse noise from digital images has been one of the popular research topics in image processing and many filtering methods have been proposed. These methods may mainly be divided into two groups. The first group of methods exploits the rank order information (i.e. order statistics) of the noisy pixels in a given filtering window. These are usually based on the standard median filter [1,2] and its derivatives. The standard median filter is a simple rank selection filter and attempts to remove impulse noise by changing the luminance value of the noisy pixel with the median of the luminance values of the pixels contained within the filtering window. This simple filtering approach yields a reasonable noise removal performance,
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
but introduces significant blurring effects into image details even at low-noise densities. The median filter has been of high research interest since its application to impulse noise removal and a number of median-based filtering approaches trying to avoid the inherent disadvantages of the standard median filter have been proposed. The weighted median filter [3] and the centerweighted median filter (CWMF) [4] are modified median filters capable of controlling the tradeoff between the noise suppression and detail preservation ability by giving more weight to the appropriate pixels of the filtering window. Hence, they may be designed to offer better detail preservation performance than the median filter at the cost of reduced noise removal performance. A method for optimal design of these filters is presented in [5]. A number of methods [6–12] combine the median filter with a decision mechanism which aims to determine whether the center pixel of a given filtering window is corrupted or not. If the center pixel is classified by the decision mechanism as a corrupted pixel, its restored value is obtained by processing the pixels in the filtering window by the median filter. Otherwise, i.e. if the center pixel is classified as uncorrupted, it is left unchanged. Although this approach significantly enhances the performance of the median filter by reducing its distortion effects, its performance inherently depends on the performance of the impulse detector. As a consequence, several different impulse detection approaches exploiting median filters [6], multiple CWMFs with different center weights [7], edge detection kernels [8], histogram-based methods [9], iterative order statistic filters [10], statistical tests [11] and boolean functions [12] have been proposed. The progressive switching median filter [13] is a modified switching median filter in which detection and removal of impulse noise are iteratively done in two separate stages. This filter exhibits improved noise suppression performance than some other median-based filters but it has a very high computational complexity due to its iterative nature. Some more complicated derivatives of the basic switching median filter employing multiple median-based filters in the structure have also been proposed. The tri-state median filter [14] is an improved switching median filter that is obtained by adding a CWMF into the basic switching median filter structure. The multi-state median filter [15] is a further extended version of the tri-state median filter. These two filters exhibit better filtering performance at the expense of increased computational complexity. The signal-dependent rank-ordered mean filter [16] is another kind of switching filter exploiting rank order information for impulse noise cancellation. The structure of the filter is very similar to that of a basic switching median filter. The only difference is that the median filter is replaced with a rank-ordered mean filter. The signaldependent rank-ordered mean filter has significantly better noise removal and detail preservation properties than some conventional as well as state-of-the-art impulse noise filters
629
for both gray scale [16] and color [17] images. The minimum/maximum exclusive mean filter [18] is another type of switching rank-ordered mean filter. This filter exhibits good filtering performance at the cost of increased computational complexity. All of the above-mentioned methods more or less have the drawback of distorting the details and texture in the input image during noise removal process. For this reason, there has been a growing research interest in the last few years in the application of new nonlinear techniques, such as neural networks and fuzzy systems, to the problems in image processing [19–27]. Indeed, neuro-fuzzy (NF) systems combine in a single system the ability of neural networks to learn from examples and the capability of fuzzy systems to model the uncertainty which is encountered when both preservation of the details and removal of the noise are jointly required. Therefore, NF systems may be employed as very powerful tools for the removal of impulse noise from digital images provided that appropriate network structures and processing strategies are utilized. Hence, impulse noise removal methods in the second group comprise nonlinear filtering operators based on neural networks and fuzzy systems. One important subset of this family of operators is fuzzy filters based on fuzzy IF–THEN and IF–THEN–ELSE rules [19,22]. These operators usually employ a set of fuzzy rules for detecting the impulse noise at the center pixel of a given filtering window, and an appropriate inference mechanism for removing it. Although they perform relatively better than the median-based operators regarding noise removal, they are inherently heuristic and the determination of the fuzzy rules may be quite difficult especially for high noise densities. In order to avoid this problem, NF approaches allowing the determination of the rule base and the parameters of the fuzzy filter by utilizing a set of training data have also been proposed [23,26]. Although these methods exhibit relatively better noise suppression and detail preservation performance, they usually have much higher complexity than the other methods. In this paper, a novel impulse noise filtering operator is presented. The fundamental advantage of the proposed operator over other operators in the literature is that it efficiently removes impulse noise from digital images while at the same time effectively preserves image details and texture. The proposed operator is a hybrid filter constructed by combining four CWMFs with a simple adaptive neuro-fuzzy inference system (ANFIS). The internal parameters of the ANFIS are optimized by training. The training is performed by using a simple artificial training image which can easily be generated by computer. Performance of the proposed operator is tested at various noise densities and for different test images, and also compared with conventional as well as state-of-the-art impulse noise removal operators. Experimental results show that the proposed operator significantly outperforms the other operators and efficiently removes impulse noise from digital images without distorting image details and texture.
630
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
2. Method 2.1. The proposed operator Fig. 1 shows the structure of the proposed impulse noise removal operator. The operator is a hybrid filter constructed by combining four CWMFs with a simple ANFIS. Each CWMF in the structure has a different order, i.e. a different center weight. The operator uses a minimal 3 × 3 pixel filtering window, which is also the filtering window for the four CWMFs in the structure. The orders of the CWMFs increase from 0 to 3. Hence the notation CWMF(k) in Fig. 1 represents a kth-order CWMF, which has a center weighting of (2k + 1).
2.2. The center-weighted median filter Let x[r, c] denote the luminance value of the pixel at location (r, c) of the noisy input image having a size of R × C pixels. Here, r and c are the row and the column indices, respectively, with 1 r R and 1 c C. Let W [r, c] represent the group of pixels contained in a filtering window with the size of 3 × 3 pixels and centered at location (r, c) of the noisy input image: W [r, c] = {x[r + p, c + q] | (p, q) ∈ [−1, 0, 1]},
(1)
where p and q are integer indices each individually ranging from −1 to 1. Similarly, let W (w) [r, c] represent the group of pixels contained in a center-weighted filtering window having the same size and location as W [r, c]: W (w) [r, c] = {W [r, c], (w − 1) ♦ x[r, c]}
(w 1),
(2)
where ♦ represents the replication operator, i.e. the expression (w − 1) ♦ x[r, c] produces (w − 1) copies of the center pixel x[r, c]. Hence, W (w) [r, c] contains the same pixels as in W [r, c] except that there are w copies of the center pixel x[r, c] in W (w) [r, c]. The kth-order CWMF is a simple rank selection filter outputting the median of the pixels contained in a centerweighted filtering window with a center weight of (2k + 1): (k)
ycwmf [r, c] = Median(W (2k+1) [r, c]) (k 0),
(3)
(k)
where ycwmf [r, c] is the output of the kth-order CWMF and k is an integer number (k 0). The CWMF is equivalent to a standard MF for k = 0: (0)
ycwmf [r, c] = Median(W (1) [r, c]) = Median(W [r, c]).
(4)
Similarly, for k > 3, the CWMF is equivalent to an identity filter, which directly outputs the center pixel of the filtering window, for a 3 × 3 pixel filtering window: (k)
ycwmf [r, c] = x[r, c] for k > 3.
(5)
Fig. 1. Structure of the proposed impulse noise removal operator. The operator comprises four center-weighted median filters (CWMF) each with a different order and a simple adaptive neuro-fuzzy inference system (ANFIS).
It is shown in [4] that a CWMF with a smaller center weight provides better noise suppression performance than a CWMF with a larger center weight. It is also shown that a CWMF with a larger center weight provides better detail preservation performance than a CWMF with a smaller center weight. As a consequence, the center weight of the CWMF can be appropriately varied to trade off between noise suppression and detail preservation.
2.3. The adaptive neuro-fuzzy inference system The ANFIS used in the structure of the proposed impulse noise removal operator is a first-order Sugeno-type fuzzy system with five inputs and one output [28]. Each input has two generalized bell-type membership functions whereas the output has a linear membership function. The input-output relationship of the ANFIS is as follows: Let X1 · · · X5 denote the inputs of the ANFIS and Y denote its output. Each possible combination of inputs and their associated membership functions is represented by a rule in the rule base of the ANFIS. Since the ANFIS has 5 inputs and each input has 2 membership functions, the rule base contains a total of 32 (25 ) rules, which are as follows: 1. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M31 ) and (X4 is M41 ) and (X5 is M51 ), then R1 = F1 (X1 , X2 , X3 , X4 , X5 ). 2. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M31 ) and (X4 is M41 ) and (X5 is M52 ), then R2 = F2 (X1 , X2 , X3 , X4 , X5 ). 3. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M31 ) and (X4 is M42 ) and (X5 is M51 ), then R3 = F3 (X1 , X2 , X3 , X4 , X5 ). 4. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M31 ) and (X4 is M42 ) and (X5 is M52 ), then R4 = F4 (X1 , X2 , X3 , X4 , X5 ). 5. if (X1 is M11 ) and (X2 is M21 ) and (X3 is M32 ) and (X4 is M41 ) and (X5 is M51 ), then R5 = F5 (X1 , X2 , X3 , X4 , X5 ). .. .
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
631
32. if (X1 is M12 ) and (X2 is M22 ) and (X3 is M32 ) and (X4 is M42 ) and (X5 is M52 ), then R32 = F32 (X1 , X2 , X3 , X4 , X5 ). where Mij denotes the jth membership function of the ith input, Rk denotes the output of the kth rule, and Fk denotes the kth output membership function. The input membership functions are generalized bell type and the output membership functions are linear: Mij (u) =
1 , u − aij 2cij 1+ bij
Fk (u1 , u2 , u3 , u4 , u5 ) = dk1 u1 + dk2 u2 + dk3 u3 + dk4 u4 + dk5 u5 + dk6 ,
(6)
(7)
where i = 1, 2, . . . , 5, j = 1, 2 and k = 1, 2, . . . , 32. Here the parameters a, b, c and d are constants that characterize the shape of the membership functions. The optimal values of these parameters are determined by training, which will be discussed in detail later on. The output of the ANFIS is the weighted average of the individual rule outputs. The weighting factor, wk , of each rule is calculated by evaluating the membership expressions in the antecedent of the rule. This is accomplished by first converting the input values to fuzzy membership values by utilizing the input membership functions and then applying the and operator to these membership values. The and operator corresponds to the multiplication of input membership values. Hence, the weighting factors of the rules are calculated as follows: w1 w2 w3 w4 w5 .. . w32
= = = = = =
M11 (X1 )M21 (X2 )M31 (X3 )M41 (X4 )M51 (X5 ) M11 (X1 )M21 (X2 )M31 (X3 )M41 (X4 )M52 (X5 ) M11 (X1 )M21 (X2 )M31 (X3 )M42 (X4 )M51 (X5 ) M11 (X1 )M21 (X2 )M31 (X3 )M42 (X4 )M52 (X5 ) M11 (X1 )M21 (X2 )M32 (X3 )M41 (X4 )M51 (X5 ) .. . M12 (X1 )M22 (X2 )M32 (X3 )M42 (X4 )M52 (X5 ). (8)
Once the weighting factors are obtained, the output of the ANFIS can be found by calculating the weighted average of the individual rule outputs: 32 k=1 wk Rk . (9) Y= 32 k=1 wk Readers interested in the details of fuzzy systems may refer to an excellent book on this subject [28].
2.4. Training of the ANFIS The internal parameters of the ANFIS are optimized by training. Fig. 2 represents the setup used for training. Here, the parameters of the ANFIS are iteratively optimized so that its output converges to the output of the ideal noise filter
Fig. 2. Training of the ANFIS.
Fig. 3. The training images: (a) original training image and (b) noisy training image.
which, by definition, can completely remove the noise from the image. The ideal noise filter is conceptual only and does not necessarily exist in reality. It is only the output of the ideal noise filter that is necessary for training, and this is represented by the original (noise-free) training image. Fig. 3 shows the images used for training. The image shown in Fig. 3a is the original (noise-free) training image, which is a 128 × 128 pixel artificial image that can easily be generated in a computer. Each square box in this image has a size of 4 × 4 pixels and the 16 pixels contained within each box have the same luminance value, which is a random integer number uniformly distributed in [0, 255]. The image in Fig. 3b is the noisy training image and is obtained by corrupting the original training image by impulse noise of 30% noise density. Although the density of the corrupting noise is not very critical regarding training performance, it is experimentally observed that the operator exhibits the best performance when the noise density of the input training image is equal to the noise density of the noisy input image to be restored. It is also observed that the performance of the operator slightly decreases as the difference between the two noise densities increases. Hence, in order to obtain a stable filtering performance for a wide range of filtering noise densities, very low and very high values for training noise density should be avoided since it is usually impossible to know the actual noise density of a corrupted image in a real practical application. Results of extensive simulation experiments indicate that very good filtering performance is easily obtained for all kinds of images corrupted by impulse
632
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
noise with a noise density between 0% and 80% provided that the input training image has a noise density between 20% and 50%. The images in Figs. 3a and b are employed as the target (desired) and the input images during training, respectively. The parameters of the ANFIS are then iteratively tuned by using the Levenberg–Marquardt optimization algorithm [28] so as to minimize the learning error. Once the training of the ANFIS is completed, its internal parameters are fixed and it is combined with the CWMFs to construct the proposed operator, as shown in Fig. 1.
2.5. Filtering of the noisy image The noisy input image is processed by sliding a 3 × 3 filtering window on the image. The window is started from the upper-left corner of the image and moved sideways and progressively downwards in a raster scanning fashion. For each filtering window, the pixels contained within the window are fed to the CWMFs in the structure. Each CWMF appropriately weights the filtering window and generates at its output the restored value of the center pixel of the window by employing the filtering scheme discussed before. Hence, each CWMF in the structure generates a different restored subimage for the same noisy input image. As discussed before, the lower the order of a given CWMF in the structure, the higher the degree of noise suppression and the lower the degree of detail preservation in the subimage generated by this CWMF, and vice versa. Therefore, each subimage represents a different balance between detail preservation and noise suppression. These subimages are fed to the inputs of the ANFIS. The ANFIS generates the final output image of the proposed operator by employing the fuzzy inference mechanism discussed before.
3. Results The proposed operator discussed in the previous section is implemented. The performance of the operator is tested under various noise conditions and on several popular images from the literature including Boats, Bridge, Lena, Pentagon and Peppers. These images are shown in Fig. 4. All images are 8-bit gray level images having the same size of 256 × 256 pixels. The test images used in the experiments are generated by contaminating the original images by impulse noise with an appropriate noise density depending on the experiment. For comparison, the corrupted test images are also filtered by using several conventional and state-of-the-art impulse noise removal operators including the standard median filter (MF) [1], the edge-detecting median filter (EDMF) [8], the multi-state median filter (MSMF) [15], the progressive switching median filter (PSMF) [13], the signal-dependent rank-ordered mean filter (SDROMF) [16] and a fuzzy filter
Fig. 4. Test images: (a) Boats, (b) Bridge, (c) Lena, (d) Pentagon and (e) Peppers.
(FF) [19]. These filters are representative implementations of different approaches to impulse noise filtering problem. The EDMF, MSMF, PSMF, SDROMF and FF operators have a number of tuning parameters. Unfortunately, there are no analytical methods available in the literature to determine the optimal values for these parameters that yield the best results for a given filtering experiment. Hence, the values of these parameters are heuristically determined. In the experiments presented in this paper, the values suggested in the corresponding references are used. These are as follows: For EDMF, T = 116 [8]. For MSMF, wmax = 5 and T = 20 [15]. For PSMF, ND = 3 and NF = 3 [13]. For SDROMF, 1 = 0, 2 = 1 and the thresholds {T1 , T2 , T3 , T4 } are {8, 20, 40, 50} [16]. For FF, L = 256, a = 40 and b = 32 [19]. All filters including the proposed filter operate on a 3 × 3 filtering window, except the EDMF filter which performs better with a 5 × 5 window [8]. The performances of all operators are evaluated by using the mean-squared error (MSE) criterion, which is defined as MSE =
C R 1 (s[r, c] − y[r, c])2 , RC r=1 c=1
(10)
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
633
Table 1. Comparison of the overall filtering performances of the operators
Noisy MF MSMF EDMF SDROMF FF PSMF Proposed
Boats
Bridge
Lena
Pentagon
Peppers
Avrg.
4749 370 270 265 236 211 279 104
4818 376 258 233 223 193 205 115
4953 293 229 187 170 180 162 82
4361 389 269 214 227 169 191 117
4807 253 197 162 131 136 123 69
4738 336 245 212 197 178 192 102
The test images are corrupted by 25% impulse noise. The MSE values are calculated for all pixels of the input images.
Table 2. Comparison of the noise suppression performances of the operators
Noisy MF MSMF EDMF SDROMF FF PSMF Proposed
Boats
Bridge
Lena
Pentagon
Peppers
Avrg.
19 015 649 862 648 473 516 546 428
18 311 650 859 642 495 550 501 456
19 854 537 777 512 380 484 449 329
17 420 643 771 575 424 421 383 403
19 245 505 724 419 307 355 348 262
18 769 597 799 559 416 465 445 376
The test images are corrupted by 25% impulse noise. The MSE values are calculated for only the corrupted pixels of the input images.
Table 3. Comparison of the detail preservation performances of the operators
Noisy MF MSMF EDMF SDROMF FF PSMF Proposed
Boats
Bridge
Lena
Pentagon
Peppers
Avrg.
0 277 72 137 157 109 190 1
0 285 58 97 133 74 106 12
0 212 47 79 100 79 67 22
0 304 102 93 162 84 127 4
0 169 21 76 72 63 48 17
0 249 60 96 125 82 108 11
The test images are corrupted by 25% impulse noise. The MSE values are calculated for only the uncorrupted pixels of the input images.
where s[r, c] and y[r, c] represent the original and the restored versions of a corrupted test image, respectively. Several experiments are performed to measure and compare the noise suppression and detail preservation performances of all operators. The experiments are especially designed to reveal the performances of the operators for different image properties and noise conditions. Since all experiments are related with noise and noise is a random process, every realization of the same experiment yields different results even if the experimental conditions are the same. Therefore, each individual filtering experiment presented in this paper is repeated for 10 times yielding 10 different MSE values for the same experiment. The average of these values is then taken as the representative MSE value for that experiment.
3.1. Experiment Set 1 (constant noise density) In the first set of experiments, the original images shown in Fig. 4 are corrupted by impulse noise having a noise density of 25%. The corrupted images are filtered by all operators and the MSE values of the output images are calculated. In order to obtain a deeper analysis of the filtering behavior and to separately evaluate the noise suppression and detail preservation ability of the operators, the MSE values are separately calculated (i) for all pixels, (ii) for corrupted pixels only and (iii) for uncorrupted pixels only. These are given in Tables 1–3, respectively. The MSE values listed in Table 1 are calculated for all pixels and reflect the overall filtering performances of the operators. As it is seen from this table, performance of the
634
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
MF is the worst of all for all test images. The MSMF is considerably better than the MF but worse than the others. The performances of the EDMF, PSMF, SDROMF and FF are very close to each other. The proposed filter, however, exhibits the best performance for all test images. Table 2 lists the MSE values calculated for only the corrupted pixels of the test images for a comparative evaluation of the noise removal performances of the operators. This time the performance of the MF is better than the MSMF, as expected. This is because the MSMF is based on a certain number of CWMFs with different center weights. As discussed in Section 1, the CWMF demonstrates better detail preservation performance than the standard median filter at the cost of reduced noise removal performance. Hence, the noise removal performance of the MSMF is worse than that of the MF. The performances of the SDROMF, PSMF and FF are again very close to each other, whereas the EDMF yields relatively higher MSE values. This is because the EDMF uses edge detection kernels to detect impulses. The decrease in the performance of this operator may be attributed to the false classification of a number of impulses as edge pixels. As a consequence, these pixels are left unfiltered, which results in a decrease in the noise suppression performance of the EDMF operator. On the other hand, the performance of the proposed filter is considerably better than the other filters for all images except the Pentagon image for which the PSMF operator performs better. The real power of the proposed filter is revealed by Table 3, which lists the MSE values calculated for only the uncorrupted pixels of the test images for a comparative evaluation of the detail preservation performances of the filters. An ideal noise filter would only process the corrupted pixels of a noisy input image and leave the uncorrupted pixels unchanged. Therefore, the MSE value calculated for only the uncorrupted pixels of a given test image would be zero for an ideal noise filter. However, it is easily seen from Table 3 that the MSE values of all filters except the proposed filter are significantly higher than zero. This obviously implies that these filters significantly distort the uncorrupted regions of the test images during restoration of the corrupted regions, resulting in undesirable blurring effects in the details of the input image and loss of useful information within the image. On the other hand, the MSE values of the proposed filters are significantly lower than those of the other filters and much closer to zero. This clearly indicates that the filtering behavior of the proposed filter is much closer to that of an ideal filter regarding detail preservation when compared to the other filters. Hence, the proposed filter effectively preserves the useful information contained within the uncorrupted regions of the input image during restoration of the corrupted regions. The output images of all operators for the Lena image corrupted by impulse noise with 25% noise density are shown in Fig. 5 for a visual evaluation of the noise removal
Fig. 5. Output images of the operators for the Lena image corrupted by impulse noise with 25% noise density: (a) Noisy Lena image, (b) MF, (c) MSMF, (d) EDMF, (e) SDROMF, (f) FF, (g) PSMF and (h) Proposed.
and detail preservation performances of the operators. It is observed from this figure that the performance of the MF and MSMF are very close to each other. Some noise blotches are easily visible in the output images of these two filters. The EDMF exhibits better detail preservation performance than the other operators at the cost of slightly reduced noise suppression performance. The output images of the SDROMF, FF and PSMF are almost indistinguishable from each other and they are significantly better than those of the MF, MSMF and EDMF regarding noise
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
635
Table 4. Performances of all operators for varying noise density
Noisy MF MSMF EDMF SDROMF FF PSMF Proposed
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
948 136 64 71 83 47 51 12
1897 157 85 90 102 71 76 27
2839 189 115 119 126 99 103 47
3799 245 163 159 158 135 133 66
4738 336 245 212 197 178 192 97
5698 491 372 282 249 231 241 128
6649 733 575 371 311 294 281 170
7593 1077 867 489 398 378 326 236
8534 1572 1291 647 515 491 381 304
9488 2226 1865 873 677 629 442 398
suppression. However, it is observed that these operators introduce some amount of blur distortions into the details of the image. The proposed filter, however, demonstrates very good noise suppression and excellent detail preservation performance. The difference, especially in detail preservation, can easily be observed by carefully comparing the appearance of the eyes and the face of the lady as well as the feathers on the hat.
3.2. Experiment Set 2 (varying noise density) The second set of experiments are designed to evaluate the dependency of the performances of the operators on noise density. For this purpose, the noise density is increased from 5% to 50% with 5% steps. For each noise density step, the five test images shown in Fig. 4 are corrupted by impulse noise with that noise density. This produces five different experimental images each having the same noise density. These images are filtered by one of the operators and the MSE values are calculated for all pixels of the output images, producing five different MSE values representing the filtering performance of that operator under different image properties. These values are then averaged to obtain the representative MSE value of that operator for that noise density. This procedure is separately repeated for all noise densities to obtain the variation of the average MSE value of that operator as a function of noise density. Finally, the overall experimental procedure is individually repeated for each operator. Table 4 shows the variation of the average MSE values of the operators as a function of noise density. As it is seen from this table, the performances of the MF and MSMF operators are very poor, with the MSMF being better than the MF. The EDMF performs better than the MF and MSMF but worse than the others. As discussed in the previous subsection (Experiment Set 1), the EDMF operator utilizes edge detection kernels to distinguish between corrupted, uncorrupted and edge pixels. In this way, the smoothing effect of the filter, which is desirable for noisy pixels but undesirable for edge pixels, is suitably controlled. But, the fundamental drawback of this approach is that an increasing number of noisy pixels are incorrectly detected as edge pixels as the
noise density increases. As a result, the performance of the EDMF operator relatively decreases with increasing noise density. This is observed in Table 4. The performances of the SDROMF and FF operators are very close to each other especially for higher noise densities. The FF operator exhibits considerably better filtering performance than the MF, MSMF and EDMF operators for all noise densities. The filtering performance of the PSMF operator is relatively better at higher noise densities than at lower noise densities. As discussed in the previous subsection, this operator exhibits very good noise suppression performance but significantly distorts image details and texture. As the noise density increases, the number of corrupted pixels in the input images increases. As a consequence, the contribution of the noise suppression performance to the general filtering performance becomes more dominant than the contribution of the detail preservation performance. Therefore, the performance of the PSMF operator becomes better than the other operators for relatively higher noise densities, as observed in Table 4. The proposed operator, however, demonstrates the best filtering performance of all. Its MSE values are significantly lower than those of the other filters for all noise densities.
4. Discussion and conclusion A novel operator for removing impulse noise from digital images is presented. The fundamental superiority of the proposed operator over other operators is that it efficiently removes impulse noise from digital images while successfully preserving the details and texture in the original image. The advantages of the new operator over other operators in the literature may be summarized as follows: 1. It has a very simple structure. It is constructed by appropriately combining four CWMFs with a simple ANFIS. The structure of the ANFIS is also very simple. The neuro-fuzzy-based impulse noise filtering operators in the literature usually have complicated network structures [23,26].
636
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
2. It requires no performance tuning parameters. However, majority of the filtering operators available in the literature [3–19] need one or more parameters that are heuristically determined and externally supplied by the user. In most cases, it is difficult to determine the optimal set of these parameters that yields the best filtering performance for a given image and/or given noise density since there is no analytical method for this purpose. 3. The proposed operator uses a minimal 3 × 3 pixel filtering window whereas some other operators in the literature require filtering windows with larger sizes [8,15,23]. The larger the size of the filtering window, the higher the number of pixels to be processed by the operator to calculate the restored value of the center pixel, hence the longer the time required for the restoration of the noisy image. 4. The ANFIS used in the structure of the proposed operator uses one-dimensional fuzzy membership functions, which simplifies implementation. On the other hand, some neuro-fuzzy systems related with impulse noise filtering use two-dimensional fuzzy membership functions [23]. 5. The ANFIS is trained by using very simple artificial images that can easily be generated in a computer. However, contrary to its simplicity in implementation and convenience in training, it may be used for efficiently filtering any image corrupted by impulse noise of virtually any noise density. It is concluded that the proposed operator can be used as a simple but powerful tool for efficient removal of impulse noise from digital images without distorting the useful information within the image.
References [1] Umbaugh SE. Computer vision and image processing. Englewood Cliffs, NJ: Prentice-Hall; 1998. [2] Gonzales RC, Woods RE. Digital image processing. 2nd ed., Reading, MA: Addison-Wesley; 2002. [3] Yli-Harja O, Astola J, Neuvo Y. Analysis of the properties of median and weighted median filters using threshold logic and stack filter representation. IEEE Trans Signal Process 1991;39(2):395–410. [4] Ko SJ, Lee YH. Center weighted median filters and their applications to image enhancement. IEEE Trans Circuits Syst 1991;38(9):984–93. [5] Jeong B, Lee YH. Design of weighted order statistic filters using the perceptron algorithm. IEEE Trans Signal Process 1994;42(11):3264–9. [6] Sun T, Neuvo Y. Detail-preserving median based filters in image processing. Pattern Recogn Lett 1994;15(4):341–7. [7] Chen T, Wu HR. Adaptive impulse detection using centerweighted median filters. IEEE Signal Proc Lett 2001;8(1): 1–3. [8] Zhang S, Karim MA. A new impulse detector for switching median filters. IEEE Signal Proc Lett 2002;9(11):360–3.
[9] Pok G, Liu Y, Nair AS. Selective removal of impulse noise based on homogeneity level information. IEEE Trans Image Process 2003;12(1):85–92. [10] Crnojevic V, Senk V, Trpovski Z. Advanced impulse detection based on pixel-wise MAD. IEEE Signal Proc Lett 2004;11(7):589–92. [11] Be¸sdok E, Yüksel ME. Impulsive noise rejection from images with Jarque–Berra test based median filter. Int J Electron Commun (AEÜ) 2005;59(2):105–10. [12] Aizenberg I, Butakoff C, Paliy D. Impulsive noise removal using threshold boolean filtering based on the impulse detecting functions. IEEE Signal Proc Lett 2005;12(1):63–6. [13] Wang Z, Zhang D. Progressive switching median filter for the removal of impulse noise from highly corrupted images. IEEE Trans Circuits Syst 1999;46(1):78–80. [14] Chen T, Ma KK, Chen LH. Tri-state median filter for image denoising. IEEE Trans Image Process 1999;8(12):1834–8. [15] Chen T, Wu HR. Space variant median filters for the restoration of impulse noise corrupted images. IEEE Trans Circuits Syst—II 2001;48(8):784–9. [16] Abreu E, Lightstone M, Mitra SK, Arakawa K. A new efficient approach for the removal of impulse noise from highly corrupted images. IEEE Trans Image Process 1996;5(6): 1012–25. [17] Moore MS, Gabbouj M, Mitra SK. Vector sd-rom filter for removal of impulse noise from color images. In: Proceedings of the ECMCS’99 EURASIP conference on DSP for multimedia communications and services, Krakow, 1999. [18] Han WY, Lin JC. Minimum–maximum exclusive mean (MMEM) filter to remove impulse noise from highly corrupted images. Electron Lett 1997;33(2):124–5. [19] Russo F, Ramponi G. A fuzzy filter for images corrupted by impulse noise. IEEE Signal Proc Lett 1996;3(6):168–70. [20] Choi YS, Krishnapuram R. A robust approach to image enhancement based on fuzzy logic. IEEE Trans Image Process 1997;6(6):808–25. [21] Pham DL, Prince JL. Adaptive fuzzy segmentation of magnetic resonance images. IEEE Trans Med Imag 1999;18(9):737–52. [22] Russo F. FIRE operators for image processing. Fuzzy Set Syst 1999;103(2):265–75. [23] Russo F. Noise removal from image data using recursive neurofuzzy filters. IEEE Trans Instrum Meas 2000;49(2): 307–14. [24] Park DC, Woo YJ. Weighted centroid neural network for edge preserving image compression. IEEE Trans Neural Networks 2001;12(5):1134–46. [25] Boskowitz V, Guterman H. An adaptive neuro-fuzzy system for automatic image segmentation and edge detection. IEEE Trans Fuzzy Syst 2002;10(2):247–62. [26] Yüksel ME, Ba¸stürk A. Efficient removal of impulse noise from highly corrupted digital images by a simple neuro-fuzzy operator. Int J Electron Commun (AEÜ) 2003;57(3):214–9. [27] Yüksel ME, Yıldırım MT. A simple neuro-fuzzy edge detector for digital images corrupted by impluse noise. Int J Electron Commun (AEÜ) 2004;58(1):72–5. [28] Jang JSR, Sun CT, Mizutani E. Neuro-fuzzy and soft computing. Englewood Cliffs, NJ: Prentice-Hall; 1997.
M.E. Yüksel / Int. J. Electron. Commun. (AEÜ) 60 (2006) 628 – 637
M. Emin Yüksel received his B.Sc. degree in Electronics and Communications Engineering from the Technical University of Istanbul, Istanbul, Turkey, in July 1990. In February 1991, he joined as a Research Assistant to the Department of Electronics Engineering, Erciyes University, Kayseri. He received his M.Sc. and Ph.D. degrees in Electronics Engineering from
637
Erciyes University in February 1993 and September 1996, respectively. Between March- and December 1995, he had been an academic visitor to Signal Processing Section, Department of Electrical Engineering, Imperial College, London, UK, where he conducted research on time-varying parametric modeling of nonstationary signals. Currently, he is with the Department of Electronics Engineering of Erciyes University. His general research interests include signal processing, image processing, neural networks, fuzzy systems and applications of these techniques.