- Email: [email protected]
Local-statistics algorithm for smoothing noisy images with low signal-to-noise ratio
Volume 59, number 1
OPTICS COMMUNICATIONS
1 August 1986
LOCAL-STATISTICS ALGORITHM FOR SMOOTHING NOISY IMAGES WITH LOW SIGNAL-TO-NOISE RATIO Junji M A E D A and Kazumi M U R A T A
Department of Applied Physics, Faculty of Engineering~ Hokkaido University, Sapporo 060, Japan Received 10 March 1986
A digital method for smoothing noisy images is described. The smoothing procedure is a local-statistics algorithm that is composed of median filtering and mean filtering. This method is suitable for noisy images which have low signal-to-noiseratio and strong edges. We present some results of computer simulations that demonstrate the effectiveness of the proposed algorithm.
1. Introduction
Lee's local-statistics algorithm (or the James-Stein estimator) has recently been used for smoothing noisy images [ 1 - 3 ]. Since this technique assumes a non-stationary mean, nonstationary variance image model [4], the algorithm provides a better adaptation to local changes in image statistics than conventional methods based on a stationary image model. This local-statistics algorithm also has the feature of a suitability for real-time processing or parallel processing. However, Lee's algorithm has two drawbacks: One is the necessity of the variance of noise itself, though this value can be estimated in some cases. The other shortcoming, which has not been pointed out so far, is that the algorithm has an inadequate smoothing ability for noisy images with low signal-to-noise ratio (SNR) when the images include strong edges. This is caused by an inherent nature of the algorithm, which will be discussed in more detail later. In this paper we investigate a new local-statistics algorithm that is especially suitable for smoothing Iow-SNR images with strong edges. We incorporate median filtering into our method, which provides the significant noise-tolerance for a new algorithm. Moreover, the proposed algorithm has another advantage that it requires no information about the variance of noise. Our technique also preserves the characteristic of nearly real-time processing. We present results of 0 030-4018/86/$03.50 © Elsevier Science Publishers B.V. (North-Holland Physics Publishing Division)
computer simulations in which some comparisons are made between Lee's algorithm and the proposed one.
2. Smoothing procedures We consider the case that an original image is degraded only by noise (no spatial blur). The noise can be signal-independent or signal-dependent and includes additive noise, multiplicative noise, fill-grain noise, and Poisson noise. As for multiplicative noise and film-grain noise, they can be made additive and signal-independent by homomorphic transformation [3]. Thus this paper deals with additive noise and Poisson noise. For additive noise, the measured degraded image pixel gi] can be expressed as
gif = fij + nij,
(1)
where f i s the original image and n is usually taken to be zero-mean gaussian noise with a variance o2. At low light levels, the emission of photons is described by a Poisson process, and the resulting measurement uncertainty is called Poisson noise [5]. Let gi/be an integer random variable whose value is the count of the photon counter for a givenfi/. This random variable is a conditional Poisson process with the probability density function
11
Volume 59, number 1
OPTICS COMMUNICATIONS
(~fi/)gi/ exp(-Xfij) P(gij [ft'/')=
gu!
,
(2)
where X is a constant rate parameter. Both the conditional mean and variance ofgi] for given 8j become )~fii, which demonstrates the signal-dependent nature of Poisson noise. By using the transformation proposed by Anscombe [6] :
gij
2(gij + 3/8) 1/2
(3)
Poisson distribution can be approximately expressed as normal distribution with mean equals 2(3, + 1/8) 1/2 and variance 1. This transformation will be used in the experiments in section 3.
2.1. Lee's local-statistics algorithm Lee's algorithm is based on a nonstationary mean, nonstationary variance image model, and was developed mainly for additive noise. The estimated value for additive noise by this algorithm is given by
(5)
o;j -- 1 - o.2/o;j.
and the local mean gii and the local variance vi/are obtained from the measured image by calculating over a moving window of size (2m + 1) X (2n + 1) centered on the pixel (i,j):
i+m
j+n
~
~
1
gij = (2m + 1)(2n + 1) k=i-m l=j-n
l
(2m + 1)(2n + 1)k
i+m m
(6)
gkl,
j+n l=/-n d ,
)-
(7)
The locally adaptive properties of Lee's algorithm are interpreted as follows: For a high-SNR region, Qij ~ 1 and f/! ~ gij. This means that the estimated pixel is equal to the measured pixel itself. On the other hand, for a low-SNR region, Qii ~ O, and /ij ~ g i j , which means that the estimated pixel is equal to the local mean value. The resultant estimate is a balance between the observation gij and the local mean gij. This will be better understood if we rewrite eq. (4) in the next form: 12
(8)
Though this interpretation is in general valid, it emphasizes only the SNR aspect of an image and overlooks the analysis based on the local structures of the image. An image usually contains the relatively flat regions and the regions which include various edges. The previous interpretation can hold for (i) a flat region with high SNR, (ii) a flat region with low SNR, and (iii) an edge region with high SNR. Thus Lee's algorithm is fully effective for these three types of region. However, for an edge region with low SNR, Qi/ never becomes zero because vii in this region takes a relatively large value owing to the presence of the edges. If a region contains very strong edges and has low SNR, Q6 approaches unity and the estimated pixel gets close to the measured noisy pixel. Therefore, for a low-SNR image that includes strong edges, the estimated result by Lee's algorithm may have noisy appearance at or near the edges. A new algorithm was developed to overcome this defect.
2.2..New local-statistics algorithm
where
(,
Qij)gij.
(4)
/ij = gij + Qij(gij - gij),
.,,j:
fly = Qi/gij + (1
1 August 1986
Our algorithm also assumes a nonstationary mean, nonstationary variance image model. A new algorithm aims at a suitability for smoothing low-SNR images containing strong edges and was heuristically derived. For this purpose we incorporate median filtering into our algorithm. It is well known that a median filter is efficient in reducing the effect of spiky or impulse noise without blurring sharp edges [7-10]. The new algorithm is the simple combination of a median filter and a local mean filter. Since the shapes and sizes of the moving window used in each filter are in general different, this technique needs to scan the moving window twice. At first scan, the local mean and the local variance of each pixel are calculated throughout the image using the first window. Then the local variances are normalized to unity. At second scan, median filtering is carried out using the second window, and the smoothed estimate of each pixel is produced by:
f ij = "°ijM (gij } + (1 -- "oij)g,ij,
(9)
where M{ } represents median filtering and "@ is the normalized local variance (0 ~ "oij ~ 1). Eq. (9) means
Volume 59, number 1
OPTICS COMMUNICATIONS
that the median filter works well where the local variance is large, i.e., in the edge regions, while the local mean filter dominates in the flat regions. Consequently, the smoothed estimate by the proposed algorithm is a balance between the median filtered value M{gi/) and the local mean gij. The difference between Lee's algorithm and the new one is apparent by comparing eq. (8) with eq. (9). The features of the proposed algorithm are summarized below. (1) The new algorithm is especially fit for smoothing low-SNR images including strong edges. The problem of noisy appearance at or near edges in Lee's algorithm is mitigated in the proposed algorithm. (2) No information about the variance of noise is required. This enables the new algorithm to apply to wider situations than Lee's algorithm. Poisson noise is one of such examples. (3) Our algorithm needs double scanning of the moving window, while Lee's algorithm requires single scanning. Thus the proposed technique is only adapted to nearly real-time processing.
3. Experimental results We applied Lee's algorithm and the proposed one to images degraded by additive noise or Poisson noise, and some comparisons were made between two algorithms. Fig. l(a) shows an original object which has
1 August 1986
128 X 128 sampling points. Zero mean white gaussian noise was added to this object, and the signalto-noise variance ratio was taken to be 5 dB. The resultant degraded image is shown in fig. l(b) which contains a considerable amount of noise. The smoothed estimate obtained by Lee's algorithm with a moving window of 5 × 5 pixels is shown in fig. 2(a). The variance of noise is assumed to be known. In this picture, we can observe noticeable residual noise near or at the edge regions as was expected in the previous section. The smoothed result obtained by the proposed algorithm is shown in fig. 2(b). A moving window with 5 × 5 pixels were used both for the median filter and for calculating the local mean and the local variance. Fig. 2(b) obviously represents less noisy appearance than fig. 2(a), and demonstrates that our algorithm resuits in a better restoration than Lee's algorithm. The second example is Poisson noise in the context of nuclear medical imaging. Fig. 3(a) shows a model object with its sectional plot (P - Q) which represents a thyroid phantom and has 64 X 64 sampling points. This phantom is used to evaluate the performance of the radioisotope imaging systems. The degraded image by Poisson noise is shown in fig. 3(b) which has quite noisy appearance because of the effect of quantum fluctuations. (The exact simulation of radioisotope images should be accompanied by the low resolution of imaging system as well as Poisson noise. However, only Poisson noise is dealt with in the present work to emphasize the noise smoothing ability of algorithms.)
Fig. 1. (a) An originalobject; (b) degradedimage by additive noise at 5-dB SNR. 13
Fig. 2. Smoothed images obtained by using (a) Lee's algorithm and (b) the proposed algorithm.
g d
;
1
Q.
S I~
I
o.oo
!
!
16.oo
S~.O0 '
4BI • O0 i
8~I. O0
I
I, !
lfl.Ofl
Fig. 3. (a) A model object representing a thyroid phantom with its sectional plot (P with its sectional plot.
I
'!
~2.00
! :
I
4 '
48.00
64.00
Q); (b) degraded image by Poisson noise
Volume 59, number 1
OPTICS COMMUNICATIONS
1 August 1986
!
18
r ~ i_¢ !
! i
! J i
Fig. 4. Smoothed images obtained by using (a) Lee's algorithm and (b) the proposed algorithm with their sectional plots.
First, we use an Anscombe's transformation to make Poisson noise approximately signal-independent. Then, smoothed algorithm is applied to the transformed image. After smoothing, the inverse transformation is applied to the smoothed estimate. The final restored images with their sectional plots obtained by using Lee's algorithm and our algorithm are shown in figs. 4(a) and 4(b), respectively. A 5 X 5 pixels-moving window was used for calculating the local mean and the local variance in both algorithms, and a 3 × 3-
pixels window for the median filter. These pictures and the sectional plots clearly represent that Lee's algorithm results in strong residual noise especially at the edge regions, while the proposed algorithm provides a significantly improved result throughout the phantom. The optimal sizes of moving windows depend on both the sampling points and structures of an image, and were empirically determined in our experiments.
15
Volume 59, number 1
OPTICS COMMUNICATIONS
1 August 1986
4. Conclusion
References
We have described the local-statistics algorithm for smoothing noisy images which contain strong edges. This algorithm is based on a nonstationary mean, nonstationary variance image model, and consists o f median filtering and mean filtering. The usefulness of the proposed algorithm in the presence of a considerable amount of noise has been confirmed through computer simulations. In particular, the proposed algorithm has provided more noticeable improvement in the edge regions than Lee's algorithm when the SNR is low.
[1 ] J.S. Lee, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2 (1980) 165. [2] G.K. Froehlich, J.F. Walkup and T.F. Krile, Appl. Optics 20 (1981) 3619. [3] H.H. Arsenault and M. Levesque, Appl. Optics 23 (1984) 845. [4] D.T. Kuan, A.A. Sawchuk, T.C. Strand and P. Chavel, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-7 (1985) 165. [5] C.M. Lo and A.A. Sawchuk, Proc. SPIE 207 (1979) 84. [6] F.J. Anscombe, Biometrika 35 (1948) 246. [7l J.W. Tukey, Exploratory data analysis (Addison-Wesley, Reading, Mass., 1971 ). [8] B.R. Frieden, J. Opt. Soc. Am. 66 (1976) 280. [9] J. Maeda and K. Murata, Appl. Optics 23 (1984) 857. [10] J. Maeda, Appl. Optics 24 (1985) 751.
16
OPTICS COMMUNICATIONS
1 August 1986
LOCAL-STATISTICS ALGORITHM FOR SMOOTHING NOISY IMAGES WITH LOW SIGNAL-TO-NOISE RATIO Junji M A E D A and Kazumi M U R A T A
Department of Applied Physics, Faculty of Engineering~ Hokkaido University, Sapporo 060, Japan Received 10 March 1986
A digital method for smoothing noisy images is described. The smoothing procedure is a local-statistics algorithm that is composed of median filtering and mean filtering. This method is suitable for noisy images which have low signal-to-noiseratio and strong edges. We present some results of computer simulations that demonstrate the effectiveness of the proposed algorithm.
1. Introduction
Lee's local-statistics algorithm (or the James-Stein estimator) has recently been used for smoothing noisy images [ 1 - 3 ]. Since this technique assumes a non-stationary mean, nonstationary variance image model [4], the algorithm provides a better adaptation to local changes in image statistics than conventional methods based on a stationary image model. This local-statistics algorithm also has the feature of a suitability for real-time processing or parallel processing. However, Lee's algorithm has two drawbacks: One is the necessity of the variance of noise itself, though this value can be estimated in some cases. The other shortcoming, which has not been pointed out so far, is that the algorithm has an inadequate smoothing ability for noisy images with low signal-to-noise ratio (SNR) when the images include strong edges. This is caused by an inherent nature of the algorithm, which will be discussed in more detail later. In this paper we investigate a new local-statistics algorithm that is especially suitable for smoothing Iow-SNR images with strong edges. We incorporate median filtering into our method, which provides the significant noise-tolerance for a new algorithm. Moreover, the proposed algorithm has another advantage that it requires no information about the variance of noise. Our technique also preserves the characteristic of nearly real-time processing. We present results of 0 030-4018/86/$03.50 © Elsevier Science Publishers B.V. (North-Holland Physics Publishing Division)
computer simulations in which some comparisons are made between Lee's algorithm and the proposed one.
2. Smoothing procedures We consider the case that an original image is degraded only by noise (no spatial blur). The noise can be signal-independent or signal-dependent and includes additive noise, multiplicative noise, fill-grain noise, and Poisson noise. As for multiplicative noise and film-grain noise, they can be made additive and signal-independent by homomorphic transformation [3]. Thus this paper deals with additive noise and Poisson noise. For additive noise, the measured degraded image pixel gi] can be expressed as
gif = fij + nij,
(1)
where f i s the original image and n is usually taken to be zero-mean gaussian noise with a variance o2. At low light levels, the emission of photons is described by a Poisson process, and the resulting measurement uncertainty is called Poisson noise [5]. Let gi/be an integer random variable whose value is the count of the photon counter for a givenfi/. This random variable is a conditional Poisson process with the probability density function
11
Volume 59, number 1
OPTICS COMMUNICATIONS
(~fi/)gi/ exp(-Xfij) P(gij [ft'/')=
gu!
,
(2)
where X is a constant rate parameter. Both the conditional mean and variance ofgi] for given 8j become )~fii, which demonstrates the signal-dependent nature of Poisson noise. By using the transformation proposed by Anscombe [6] :
gij
2(gij + 3/8) 1/2
(3)
Poisson distribution can be approximately expressed as normal distribution with mean equals 2(3, + 1/8) 1/2 and variance 1. This transformation will be used in the experiments in section 3.
2.1. Lee's local-statistics algorithm Lee's algorithm is based on a nonstationary mean, nonstationary variance image model, and was developed mainly for additive noise. The estimated value for additive noise by this algorithm is given by
(5)
o;j -- 1 - o.2/o;j.
and the local mean gii and the local variance vi/are obtained from the measured image by calculating over a moving window of size (2m + 1) X (2n + 1) centered on the pixel (i,j):
i+m
j+n
~
~
1
gij = (2m + 1)(2n + 1) k=i-m l=j-n
l
(2m + 1)(2n + 1)k
i+m m
(6)
gkl,
j+n l=/-n d ,
)-
(7)
The locally adaptive properties of Lee's algorithm are interpreted as follows: For a high-SNR region, Qij ~ 1 and f/! ~ gij. This means that the estimated pixel is equal to the measured pixel itself. On the other hand, for a low-SNR region, Qii ~ O, and /ij ~ g i j , which means that the estimated pixel is equal to the local mean value. The resultant estimate is a balance between the observation gij and the local mean gij. This will be better understood if we rewrite eq. (4) in the next form: 12
(8)
Though this interpretation is in general valid, it emphasizes only the SNR aspect of an image and overlooks the analysis based on the local structures of the image. An image usually contains the relatively flat regions and the regions which include various edges. The previous interpretation can hold for (i) a flat region with high SNR, (ii) a flat region with low SNR, and (iii) an edge region with high SNR. Thus Lee's algorithm is fully effective for these three types of region. However, for an edge region with low SNR, Qi/ never becomes zero because vii in this region takes a relatively large value owing to the presence of the edges. If a region contains very strong edges and has low SNR, Q6 approaches unity and the estimated pixel gets close to the measured noisy pixel. Therefore, for a low-SNR image that includes strong edges, the estimated result by Lee's algorithm may have noisy appearance at or near the edges. A new algorithm was developed to overcome this defect.
2.2..New local-statistics algorithm
where
(,
Qij)gij.
(4)
/ij = gij + Qij(gij - gij),
.,,j:
fly = Qi/gij + (1
1 August 1986
Our algorithm also assumes a nonstationary mean, nonstationary variance image model. A new algorithm aims at a suitability for smoothing low-SNR images containing strong edges and was heuristically derived. For this purpose we incorporate median filtering into our algorithm. It is well known that a median filter is efficient in reducing the effect of spiky or impulse noise without blurring sharp edges [7-10]. The new algorithm is the simple combination of a median filter and a local mean filter. Since the shapes and sizes of the moving window used in each filter are in general different, this technique needs to scan the moving window twice. At first scan, the local mean and the local variance of each pixel are calculated throughout the image using the first window. Then the local variances are normalized to unity. At second scan, median filtering is carried out using the second window, and the smoothed estimate of each pixel is produced by:
f ij = "°ijM (gij } + (1 -- "oij)g,ij,
(9)
where M{ } represents median filtering and "@ is the normalized local variance (0 ~ "oij ~ 1). Eq. (9) means
Volume 59, number 1
OPTICS COMMUNICATIONS
that the median filter works well where the local variance is large, i.e., in the edge regions, while the local mean filter dominates in the flat regions. Consequently, the smoothed estimate by the proposed algorithm is a balance between the median filtered value M{gi/) and the local mean gij. The difference between Lee's algorithm and the new one is apparent by comparing eq. (8) with eq. (9). The features of the proposed algorithm are summarized below. (1) The new algorithm is especially fit for smoothing low-SNR images including strong edges. The problem of noisy appearance at or near edges in Lee's algorithm is mitigated in the proposed algorithm. (2) No information about the variance of noise is required. This enables the new algorithm to apply to wider situations than Lee's algorithm. Poisson noise is one of such examples. (3) Our algorithm needs double scanning of the moving window, while Lee's algorithm requires single scanning. Thus the proposed technique is only adapted to nearly real-time processing.
3. Experimental results We applied Lee's algorithm and the proposed one to images degraded by additive noise or Poisson noise, and some comparisons were made between two algorithms. Fig. l(a) shows an original object which has
1 August 1986
128 X 128 sampling points. Zero mean white gaussian noise was added to this object, and the signalto-noise variance ratio was taken to be 5 dB. The resultant degraded image is shown in fig. l(b) which contains a considerable amount of noise. The smoothed estimate obtained by Lee's algorithm with a moving window of 5 × 5 pixels is shown in fig. 2(a). The variance of noise is assumed to be known. In this picture, we can observe noticeable residual noise near or at the edge regions as was expected in the previous section. The smoothed result obtained by the proposed algorithm is shown in fig. 2(b). A moving window with 5 × 5 pixels were used both for the median filter and for calculating the local mean and the local variance. Fig. 2(b) obviously represents less noisy appearance than fig. 2(a), and demonstrates that our algorithm resuits in a better restoration than Lee's algorithm. The second example is Poisson noise in the context of nuclear medical imaging. Fig. 3(a) shows a model object with its sectional plot (P - Q) which represents a thyroid phantom and has 64 X 64 sampling points. This phantom is used to evaluate the performance of the radioisotope imaging systems. The degraded image by Poisson noise is shown in fig. 3(b) which has quite noisy appearance because of the effect of quantum fluctuations. (The exact simulation of radioisotope images should be accompanied by the low resolution of imaging system as well as Poisson noise. However, only Poisson noise is dealt with in the present work to emphasize the noise smoothing ability of algorithms.)
Fig. 1. (a) An originalobject; (b) degradedimage by additive noise at 5-dB SNR. 13
Fig. 2. Smoothed images obtained by using (a) Lee's algorithm and (b) the proposed algorithm.
g d
;
1
Q.
S I~
I
o.oo
!
!
16.oo
S~.O0 '
4BI • O0 i
8~I. O0
I
I, !
lfl.Ofl
Fig. 3. (a) A model object representing a thyroid phantom with its sectional plot (P with its sectional plot.
I
'!
~2.00
! :
I
4 '
48.00
64.00
Q); (b) degraded image by Poisson noise
Volume 59, number 1
OPTICS COMMUNICATIONS
1 August 1986
!
18
r ~ i_¢ !
! i
! J i
Fig. 4. Smoothed images obtained by using (a) Lee's algorithm and (b) the proposed algorithm with their sectional plots.
First, we use an Anscombe's transformation to make Poisson noise approximately signal-independent. Then, smoothed algorithm is applied to the transformed image. After smoothing, the inverse transformation is applied to the smoothed estimate. The final restored images with their sectional plots obtained by using Lee's algorithm and our algorithm are shown in figs. 4(a) and 4(b), respectively. A 5 X 5 pixels-moving window was used for calculating the local mean and the local variance in both algorithms, and a 3 × 3-
pixels window for the median filter. These pictures and the sectional plots clearly represent that Lee's algorithm results in strong residual noise especially at the edge regions, while the proposed algorithm provides a significantly improved result throughout the phantom. The optimal sizes of moving windows depend on both the sampling points and structures of an image, and were empirically determined in our experiments.
15
Volume 59, number 1
OPTICS COMMUNICATIONS
1 August 1986
4. Conclusion
References
We have described the local-statistics algorithm for smoothing noisy images which contain strong edges. This algorithm is based on a nonstationary mean, nonstationary variance image model, and consists o f median filtering and mean filtering. The usefulness of the proposed algorithm in the presence of a considerable amount of noise has been confirmed through computer simulations. In particular, the proposed algorithm has provided more noticeable improvement in the edge regions than Lee's algorithm when the SNR is low.
[1 ] J.S. Lee, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2 (1980) 165. [2] G.K. Froehlich, J.F. Walkup and T.F. Krile, Appl. Optics 20 (1981) 3619. [3] H.H. Arsenault and M. Levesque, Appl. Optics 23 (1984) 845. [4] D.T. Kuan, A.A. Sawchuk, T.C. Strand and P. Chavel, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-7 (1985) 165. [5] C.M. Lo and A.A. Sawchuk, Proc. SPIE 207 (1979) 84. [6] F.J. Anscombe, Biometrika 35 (1948) 246. [7l J.W. Tukey, Exploratory data analysis (Addison-Wesley, Reading, Mass., 1971 ). [8] B.R. Frieden, J. Opt. Soc. Am. 66 (1976) 280. [9] J. Maeda and K. Murata, Appl. Optics 23 (1984) 857. [10] J. Maeda, Appl. Optics 24 (1985) 751.
16