- Email: [email protected]
Edge location shifts produced by median filters: theoretical bounds and experimental results
Signal Processing 16 (1989) 83-96 North-Holland
83
EDGE L O C A T I O N SHIFTS P R O D U C E D BY M E D I A N FILTERS: THEORETICAL B O U N D S AND EXPERIMENTAL RESULTS E.R. DAVIES Machine Vision Group, Department of Physics, Royal Holloway & BedJbrd New College, Egham Hill, Egham, Surrey TW20 OEX, U.K. Received 10 June 1987 Revised 30 November 1987 and 21 April 1988
Abstract. This paper studies size and shape distortions resulting from the use of median filters. The starting point is the filtering of binary objects by operators acting within circular neighbourhoods in a continuous space. This is found to lead to predictable shifts of edges towards the local centres of curvature. In a discrete lattice the effect is modified by the stability of certain objects against edge distortions, small objects generally being less stable than large objects. In grey-scale images, these effects do not disappear but are less size-critical, and are clearly visible with real images when significant noise is present.
Zusammenfassung. In dieser Arbeit werden Verzerrungen von Gr613e und Form untersucht, wie sie durch die A n w e n d u n g von Median-Filtern entstehen. Ausgegangen wird von der Filterung binfirer Objekte durch Operatoren, die kreisf6rmige U m g e b u n g e n in einem kontinuierlichen Raum bearbeiten. Es l~il3t sich zeigen, dab dies zu vorhersagbaren Kantenverschiebungen in Richtung a u f die lokalen Zentren der Kurvenziige ffihrt. In einem diskreten Gitter wird dieser Effekt insofern modifiziert, als gewisse Objekte gegen Kantenverzerrungen unempfindlich sind, wobei kleine Objekte i.a. weniger robust sind als grol3e. In Grauton-Bildern verschwinden diese Effekte nicht, sind jedoch weniger kritisch beziiglich der Gro/3e; sie sind hier in echten Bildern klar zu erkennen, sobald ein nennenswerter Rauschanteil vorhanden ist. R6sumd. Cet article dtudie les distorsions de taille et de forme r6sultant de l'utilisation des filtres m6dians. Le point de ddpart est le filtrage des objets binaires par des opdrateurs agissant sur des voisinages circulaires dans l'espace continu. I1 a 6td trouv6 que ceci conduit/~ des d6calages prdvisibles des contours vers des centres locaux de courbure. Dans une trame discrete I'effet est modif6 par la stabilit6 de certains objets vis fi vis des distorsions de contours, les petits objets dtant moins stables que les grands objets. Dans les images fi niveaux de gris, ces effets ne disparaissent pas mais sont moins sensible ti la taille et sont clairement visibles dans des images rdelles en pr6sence du bruit.
Keywords. Shape distortions, edge shifts, median filter, noise suppression. "The analysis t?["median filters remains intractable Jor all but the most simple assumptions"--Bovik et al. [ 1].
1. Introduction Over the past 10-12 years the median filter has become widely accepted as a general purpose but simple means for removing noise from digital images: indeed, it is probably the most widely used single technique for image noise suppression. Not only is it highly effective, especially with impulse noise, but also it does not suffer from the main drawback of serious image blurring introduced by the Gaussian smoothing operator [2, 8].
More recently, there has been some worry that the median filter may shift edges even if it is successful in not causing blurring [1, 10]. In part these worries stem from the fact that the median filter is a non-linear operator, so that its properties are not entirely obvious: nor are they easy to analyse mathematically [1] (see quotation at the head of this paper). As a result a number of highly mathematical studies, coupled with complicated experimental investigations, have tried to probe the situation, taking account for example of the effects of
0165-1684/89/$3.50 (c~ 1989, Elsevier Science Publishers B.V. (North-Holland)
84
E.R. Davies / Edge location shifts in median filtering
noise which might induce the filter to shift object boundaries [1, 10]. Although thorough, such investigations do not seem to have led to great insight in describing the properties of the median filter. Hence further consideration of the principles underlying median filters in the two-dimensional case that is of interest in dealing with images seemed to be required. As a start, it seemed worthwhile to consider the effects of applying median filters in a continuous space in order to specifically simplify the situation. Such a model should approximate to the situation for large objects and large neighbourhood operators, but should also throw light on what happens in the opposite approximation of a discrete lattice using the usual small neighbourhood operators. In Section 2 we apply this model in the case of binary images, and in Section 3 we study grey-scale images. In Section 4 we consider the situation for a discrete lattice, and in Sections 5 and 6 we report experiments on (discrete) binary and grey-scale images. In Section 7 we consider the edge-shifting characteristics of hybrid median filters. Finally, we summarise our current understanding of the median filter's edge-shifting properties.
2. Median filters in continuous binary images In this section we take the case of a continuous image (i.e. a non-discrete lattice), here assuming for simplicity (1) that the image is binary, (2) that neighbourhoods are exactly circular, and (3) that images are noise-free. We relax these three conditions in the next section. To proceed, we notice that binary edges have symmetrical cross-sections, while straight edges extend this symmetry into two dimensions: hence applying a median filter in a circular neighbourhood cannot pull a straight edge to one side or the other. Now let us see what happens when the filter is applied to an edge that is not straight. If, for example, the edge is circular, the local intensity distribution will contain two peaks whose relative sizes will vary with the precise position of the Signal Processing
a
i.II i[
j i
iI..
I
Fig. 1. Variation in the local intensity distribution with the position of the neighbourhood. (a) shows a neighbourhood of radius a overlapping a dark circular object of radius b. (b)-(d) show, respectively, the intensity distributions f when the separations of the centres are less than, equal to, or greater than the distance d for which the object bisects the area of the neighbourhood.
neighbourhood (Fig. 1). At some position the sizes of the two peaks will be identical. Clearly, this happens when the centre of the neighbourhood is at a unique distance from the centre of a circular object: this is the position at which the output of the median filter changes from dark to light (or vice versa). Thus the median filter produces an inwards shift towards the centre of the circular object (or the centre of curVature), whether the object is dark on a light background or light.on a dark background.
E.R. Davies / Edge location shifts in median filtering
Next suppose that the edge is irregular with several " b u m p s " (i.e. prominences or indentations) within the filter neighbourhood: clearly the filter will now tend to average out the bumps and straighten the edge, since it acts in such a way as to equalise the amounts of dark and light within the neighbourhood (Fig. 2). This means that the edge will be locally biased, but only by a small amount since the various bumps will tend to pull the final edge in opposite directions. On the other hand, if there is one gross b u m p within the neighbourhood--i.e, if the curvature has the same sign and is roughly constant at all points on the edge within the filter n e i g h b o u r h o o d - - t h e n all these parts of the edge will act in consort and it will be pulled sideways a significant amount by the filter. We now see that a circular section of the boundary constitutes a "worst case" situation, for which the filter produces the largest bias in the position of the edge. (Mathematically, the lateral deviation per unit length is greatest for a circle, for a given maximum magnitude of the local curvature.) It is clearly worth finding the size of the worst-case shift, and for this reason we concentrate attention below on circular objects, in the knowledge that all other shapes will give less serious shifts and distortions. The "worst case" calculation is a matter of elementary geometry: we need to find at what distance d from the centre of a circular object (of radius b) the area of a circular neighbourhood (of radius a) is bisected by the object boundary. For small neighbourhoods (a << b), it is possible to find an approximation by averaging the lateral displacement on a circular arc. The result (proved in the Appendix) is:
d / b ~- l - a 2 / 6 b 2 - a4/40b 4.
85
a
b
C
(1)
This analytic solution is useful because of its simplicity and surprisingly high accuracy over the whole range a ~ b. Here we examine the problem from a different viewpoint: this gives a set of equations which can then be solved numerically. From Fig. 3 the area of the sector of angle 2/3 is /3b 2, while the area of the triangle of angle 2/3
Fig. 2. Edge smoothing property of the median filter. (b) shows how the median filter smooths out irregularities in an image (a), and in particular those around the boundaries of objects. Notice how the threads on the screws are virtually eliminated, though detail larger in scale than half the filter area is preserved. In this case the median filter was a 21-element filter operating within a 5 × 5 neighbourhood on a 128x128 pixel image of 6-bit grey-scale. (c) shows the effect of a new type of "'detailpreserving" filter (see Section 7). Vo] 16, No. 2, February 1989
E.R. Davies / Edge location shifts in median filtering
86
Table 1 Estimated edge shifts for filtering binary circles in a continuous space
a/b
d'/b
d/b
E'/b
E/b
0.050 O.1O0 0.150 0.200 0.250 0.300 0.350 0.400 0.450 0.500 0.550 0.600 0.650 0.700 0.750 0.800 0.850 0.900 0.950 1,000 1.050 1.1O0 1.150 1.200 1.250 1.300 1.350 1.400 1.414
1,000 0,998 0,996 0,993 0,989 0,985 0,979 0,973 0,965 0.957 0.947 0.937 0.925 0.912 0.898 0.883 0.867 0.849 0,829 0.808 0.786 0.762 0.736 0,708 0.679 0.647 0,613 0.577 0.567
1.000 0,998 0.996 0.993 0.990 0,985 0.979 0.973 0.965 0.957 0.948 0.937 0.926 0.913 0.899 0.884 0.868 0,849 0.830 0.808 0.784 0.758 0,728 0.695 0.658 0.613 0.557 0.472 0.414
0.000 0.002 0.004 0.007 0.011 0.015 0.021 0.027 0.035 0.043 0,053 0,063 0.075 0.088 0,102 0.117 0.133 O.151 0.171 0.192 0.214 0.238 0.264 0.292 0.321 0,353 0,387 0.423 0.433
0.000 0,002 0,004 0.007 0.010 0.015 0.021 0.027 0,035 0.043 0.052 0,063 0.074 0.087 0.101 0.116 0.132 O.151 0.170 0.192 0,216 0.242 0.272 0.305 0.342 0,387 0.443 0.528 0.586
m
I I
!I
Fig. 3. Geometry for calculating neighbourhood and object overlap. is b 2 s i n / 3 cos/3. H e n c e t h e a r e a o f the s e g m e n t s h o w n s h a d e d is:
B = b2(/3 - s i n / 3 c o s / 3 ) .
(2)
M a k i n g a s i m i l a r c a l c u l a t i o n o f the a r e a A o f a circular segment of radius a and angle 2a, we now d e d u c e t h e a r e a o f o v e r l a p (Fig. 3) b e t w e e n t h e circular neighbourhood
o f r a d i u s a a n d t h e cir-
c u l a r o b j e c t o f r a d i u s b as:
C=A+B.
(3)
F o r a m e d i a n filter this is e q u a l to ~ra2/2. H e n c e
F = a2(a - s i n a cos a ) +b2(/3 - s i n / 3 c o s / 3 ) - 7 r a 2 / 2 = 0,
(4)
where
a 2= b Z + d 2 - 2 b d c o s / 3
(5)
b 2 = a 2 + d 2 - 2 a d c o s a.
(6)
and
T o s o l v e this set o f e q u a t i o n s , we t a k e a g i v e n
d' is the approximate value of d estimated using equation ( 1): the range of validity of this equation is a/b ~ 1.000. Note the excellent agreement between d' and d over this entire range. E-values are derived from corresponding d-values using equation (7). Note that the object disappears when a~ b > 1.414. approaches
a n d t h e n e x c e e d s b. N o t e , h o w e v e r ,
t h a t w h e n a > ,,/2b, t h e o b j e c t is i g n o r e d , b e i n g
v a l u e o f d, d e d u c e v a l u e s o f a a n d /3, c a l c u l a t e
s m a l l e n o u g h to be r e g a r d e d as i r r e l e v a n t n o i s e by
t h e v a l u e o f F, a n d t h e n a d j u s t the v a l u e o f d until F = 0. T h e results o f d o i n g this n u m e r i c a l l y are
the filter: b e y o n d this p o i n t it has no effect at all o n the final i m a g e . T h e m a x i m u m e d g e shift b e f o r e
s u m m a r i s e d in T a b l e 1. In fact d is t h e m o d i f i e d
t h e o b j e c t finally d i s a p p e a r s is ( 2 - x/2)b-~ 0.586b.
v a l u e o f b o b t a i n e d a f t e r filtering, a n d t h e shift p r o d u c e d by t h e filtering p r o c e s s is: E = b - d. As e x p e c t e d , E ~ 0
(7) Con-
T o e x t e n d the a b o v e results to g r e y - s c a l e i m a g e s ,
v e r s e l y , t h e shift b e c o m e s v e r y large as a first
we first c o n s i d e r t h e effect o f a p p l y i n g a m e d i a n
Signal
Processing
as b ~ o o
o r as a ~ 0 .
3. Extension to grey-scale images
87
E.R. Davies/ Edge location shifts in median filtering
filter near a smooth step edge in one dimension. Here the median filter gives zero shift, since for equal distances from the centre to either end of the neighbourhood there are equal numbers of higher and lower intensity values and hence equal areas under the corresponding portions of the intensity histogram. Clearly, this is always valid where the intensity increases monotonically from one end of the neighbourhood to the o t h e r - - a property first pointed out by Gallagher and Wise [6]. (See also [5,7] for recent discussions on related "'root" (invariance) properties of signals under median filtering.) Here it is useful that this is a rather general result. Next we extend the result to two dimensions in the vicinity of a straight edge. We assume here that the intensity values do not vary in the direction parallel to the straight edge, so the situation remains highly symmetrical. Then for an edge with monotonically increasing intensity profile, the numbers of higher intensity values on the one side of the centre of the neighbourhood (looking along the edge direction) will again balance the numbers of lower intensity values on the other side, and the median filter will give zero shift. Implicit in this proof is the use of a circular neighbourhood. However, the median filter will produce zero shift of a straight edge for a n y symmetrically shaped neighbourhood, including square or other neighbourhoods. The proof of this is quite simple. Part of the area of the neighbourhood can be eliminated if the corresponding area on the diametrically opposite end of the neighbourhood is also eliminated, since there is no intensity variation parallel to the straight edge. The underlying reason for this insensitivity to both the shape of the neighbourhood and the shape of the edge intensity profile is that the median filter can be thought of as ignoring equal areas on either side of the local intensity histogram, where the intensities will be either greater or less than the intensity at the c e n t r e - - h o w much greater or less being irrelevant. The arguments presented above are based partly on symmetry: shifts can therefore start appearing for edges that are not straight. It is fairly clear that
irregular boundaries will again tend to be averaged out by a median filter, and that circular boundaries will again be a "worst case". In what follows we therefore concentrate again on the case of circular objects. However, a full analysis requires some consideration of the fact that grey-scale edges are unlike binary edges in having finite slope. This means that we should take account of the exact form of the intensity function within the neighbourhood. When boundaries are roughly circular, contours of constant intensity will often appear as in Fig. 4.
( 1 Fig. 4. Appearance of contours of constant intensity. This figure shows contours of constant intensity on the edge of a large circular object, as seen within a small circular neighbourhood. To find how a median filter acts we merely need to identify the contour of median intensity (in two dimensions the median intensity value labels a whole contour) which divides the area of the neighbourhood into two equal parts. The geometry of the situation is identical to that already examined in Section 2: the main difference here is that for every position of the neighbourhood, there is a corresponding median contour with its own particular value of shift depending on the curvature. Fig. 4 shows an idealised case in which the contours of constant intensity have similar curvature, so that they are all moved inwards by similar amounts. This means that, to a first approximation, the edges of the object retain their cross-sectional profile as it becomes smaller. The size of the object is probably best defined from one particular contour such as that at which the intensity is the mean of the m a x i m u m and minimum intensities--which should be unchanged by the median filter. Vol. 16, No. 2, February 1989
88
E.R. Davies / Edge location shifts in median filtering
We next consider the effects of noise. For simplicity we assume that the noise is additive and of symmetrical (non-skew) intensity distribution: this is valid for Gaussian noise and is also likely to be true for many types of impulse noise. Now recall that the median contour divides the neighbourhood into two equal parts. Hence, adding noise of symmetrical intensity distribution will on average not change the area on either side of the original median contour: this means that noise will not on average cause edges to shift any differently as a result of applying the median filter--i.e, the shifts of edges caused by noise or by applying a median filter are, tofirst order, additive. In particular, noise
does not affect the general conclusions of this paper concerning the shifts of edges introduced by median filters. Though specific experiments have not been performed in this work to introduce noise and check this result quantitatively, it is generally supported by our observations on real images containing noise. This section has generalised the results of Section 2 to cover grey-scale images. It has also shown that the effects of noise should not materially affect the conclusion that median filters shift edges inwards towards local centres of curvature, the "worst case" situation arising for circular objects. Finally, it has shown that straight edges remain unshifted for any symmetrically shaped neighbourhood. However, the detailed shift calculations (Section 2) assumed a circular neighbourhood: though this is not general, it will produce an ideal isotropic response which could not be guaranteed for any other shape of neighbourhood. This fact justifies our concentration on circular neighbourhoods in the above analysis. For neighbourhoods of other shapes it seems simplest to confirm experimentally that the theory makes substantially correct predictions of edge shifts.
4. Extension to discrete neighbourhoods In the previous two sections we have developed theory showing how median filters can introduce Signal Processing
edge shifts in a continuous space. Here we consider the effect of applying median filters in discrete lattices of pixels: specifically, median filters need to be applied in square neighbourhoods of p x p pixels. In the continuous case covered earlier, p was essentially infinite. Unfortunately, it is difficult to see how to extend the theory accurately to typical cases such as 3 × 3 and 5 × 5 neighbourhoods. However, it is trivial to cover the case of p = 1, since in this case the median filter leaves the image unchanged. For intermediate values o f p we expect that edge shifts will fail between these two cases as upper and lower bounds, and indeed that there will be a steady progression from the one to the other bound as p varies (Fig. 5). As we shall see below, this situation is generally confirmed by our experimental data. E/a
0
b/a
Fig. 5. Edge shift curves for various sizes of neighbourhood. This diagram indicates h o w the edge shift curves would be expected to change as p moves between the two limiting values of I and infinity: (a) p = 1, (b) p = 3, (c) p = 5, (d) p = ~ .
5. Median filters in discrete binary images In this section we present experimental results for binary images. For reasons outlined earlier, we concentrate on the worst-case situation of small circular objects. Clearly, in a binary image it is only possible to approximate to (filled) circles, and in these tests radii ranged from 0.5 to about 9 pixels. It seemed sufficient to perform experiments
E.R. D a v i e s / E d g e location shifts in median filtering
using s t a n d a r d m e d i a n filters in a 3 × 3 ( s q u a r e ) n e i g h b o u r h o o d . In these e x p e r i m e n t s , a p r o b l e m arose since the effective r a d i u s o f such a n e i g h b o u r h o o d is not k n o w n a c c u r a t e l y : for the p r e s e n t p u r p o s e we a s s u m e it to be such as to e q u a l i s e the
89
!
E
08/
0 0 0
+
*
-_-+
1
@
•
+ + 0 + _ . _
ll
__+_
0 o o
o oo
O
7
a
6
8
Fig. 7. Edge shifts for binary circles. In this graph the plots represent the experimental results and the continuous curve is derived from Table 1, following the theory of Section 2. It is also of interest to compare the experimental plots with the model of Section 6 (see lower curve in Fig. 8).
areas o f discrete a n d i d e a l i s e d n e i g h b o u r h o o d s .
b
+
+ 1
+
o
m
e
•
•
O
O
OOOO
/
Fig. 6. Binary circles before and after filtering. (a) shows a set of binary circles of radii ranging from 0.5 to 9 pixels. (b) shows the result of applying a median filter operating in a 3 × 3 (square) neighbourhood.
H e n c e we t o o k the r a d i u s as x/9/~r = 1.693 pixels. The results are s h o w n in Figs. 6 a n d 7. W h e n c o m p a r e d with the results o f Table 1 they s h o w s o m e interesting features. In p a r t i c u l a r : (1) T h e r e are a limited n u m b e r o f p o s s i b l e r a d i u s values. (2) F o r a large p r o p o r t i o n o f r a d i u s values no c h a n g e in r a d i u s occurs on a p p l y i n g a m e d i a n filter. (3) F o r very small r a d i u s values the circle d i s a p pears completely. (4) F o r o t h e r small r a d i u s values the circle b e c o m e s m u c h smaller. (5) F o r certain i s o l a t e d larger r a d i u s values there is a r e d u c t i o n in circle size, but (a) the n u m b e r o f i n s t a n c e s b e c o m e s rarer as r a d i u s increases, a n d (b) the r e d u c t i o n in circle size b e c o m e s s m a l l e r as radius increases. (6) In general, r e p e a t e d a p p l i c a t i o n o f a m e d i a n filter to circles a b o v e a certain critical size m u s t l e a d to a small r e d u c t i o n in size f o l l o w e d by stability, w h e r e a s r e p e a t e d a p p l i c a t i o n to circles b e l o w the critical size must l e a d to their e l i m i n a t i o n : the critical r a d i u s is - 2 . 5 pixels. Vol. 16, No. 2, F e b r u a r y 1989
90
E.R. D a v i e s / E d g e location shifts in median filtering
Ignoring result (1) as obvious, we interpret these results as follows. When applied to discrete binary images, the median filter has the properties predicted in Section 2. However, median filters in small neighbourhoods do not have the resolution to detect accurately the curvature of large circles: hence these either become resistant to any change in their size and shape or seem rather unstable and ready to shed their outermost pixels. The latter situation clearly happens for those circles whose boundaries are irregular, since they have some relatively sharp corners which are eliminated by the median filter. Such corners become increasingly rare as circle size increases (see Fig. 6). We return to this point in Section 6, with the aim of building a more realistic model of the action of the median filter in the discrete case. The stability properties we have observed are related to the root behaviour noticed by other workers when median filters are applied repeatedly (mainly to one-dimensional signals) until no further change occurs [5-7]. HOwever, we are here less interested in root behaviour than in mean edge shifts, for a single application of a median filter, as curvatures vary. Hence it is instructive to average out the rather random responses that occur for various radii b. It is seen that the resulting curve (Fig. 7) is similar in shape to the theoretical curve of Section 2, but lies below it and indeed between it and the identity curve corresponding to p = 1--as predicted in Section 4: in no case does the stability effect cause the predicted change in size of an object to be reversed in sign, though it is frequently reduced to zero.
6. Median filters in discrete grey-scale images
The results of the previous section immediately suggest using better approximations to circles, with the jagged binary edges interpolated by appropriate grey-scale values. For each size of circle this was achieved by permitting the intensity to vary linearly from black to white over a range of 1 pixel, and then smoothing the resulting shapes using the Signal Processing
following well-known convolution mask: 1
4
ig
2
.
This procedure was successful in giving a realistic approximation to a smoothed step-edge. (Note, however, that other edge models, such as edges which vary linearly over 1 or 2 pixels, seemed to give essentially the same edge shifts--i.e, the edge shift behaviour was relatively insensitive to the exact type of edge model chosen.) In the experiments described here, circular objects again varied between - 1 and 9 pixels, and pixel intensities were permitted to vary over an 8-bit range. The original and the modified radii were measured by taking the integrated intensities over the circle region and deducing the radii, this approach being used because it overcame problems due to irregularities in circle boundaries. Finally, the experiment was in this case performed for 3 × 3 and 5 × 5 neighbourhoods, though in the latter case an attempt was made at approximating to the more ideal circular neighbourhood by omitting the four corner pixels, and using the pattern: X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
(Note that in some earlier work on edge detection operators, the author found that this pattern not only required less computation than a full 5 × 5 operator, but also gave increased edge orientation accuracy [3].) The results of these experiments are shown in Figs. 8 and 9. In these cases certain features that were present in the previous experiment on binary circles are now absent. In particular, the variation of edge shift E with initial radius b is very smooth, and there is no evidence of stability of large circles against the median filter: i.e. the median filter is
E.R. Davies / Edge location shifts in median filtering
91
a b l e to r e d u c e t h e size o f all circles by an a m o u n t
E 0.8-
w h i c h t e n d s to z e r o m u c h as e x p e c t e d as r a d i u s increases. It s e e m s i m p o r t a n t to a n a l y s e this case r e a s o n a b l y t h o r o u g h l y , as it c o n s t i t u t e s a c o m m o n p r a c tical s i t u a t i o n . H e n c e it was c o m p a r e d q u a n t i t a t i v e l y w i t h t h e t h e o r y o f S e c t i o n 2. A g a i n t h e p r o b lem a r o s e t h a t the e f f e c t i v e r a d i u s o f a d i s c r e t e
b
m e d i a n filter in a 3 × 3 o r 5 × 5 n e i g h b o u r h o o d
o
is
not known accurately: adopting the equal areas s t r a t e g y o f the p r e v i o u s s e c t i o n , we o b t a i n t h e respective
radii
as x / 9 / ~ r = 1.693
and
x/~-/~r=
2.585. T h e r e s u l t i n g t h e o r e t i c a l g r a p h s a r e s h o w n i
i
i
2
4
i
i
i
6
o"o+~++ i
i
8
in Figs. 8 a n d 9. T h e s e s h o w t h a t t h e r e are s o m e b
Fig. 8. Edge shifts for 3 × 3 median filter--grey-scale case. In this graph the plots represent the experimental results and the upper continuous curve is derived from Table 1. The lower continuous curve is derived from the model of Section 6.
o d d results f o r v e r y l o w v a l u e s o f circle radius. O n t h e w h o l e t h e s e c a n b e e x p l a i n e d r a t h e r n e a t l y by a p p e a l i n g to t h e d / a vs. b / a c u r v e (Fig. 10). H e r e we see t h a t t h e effect o f h a v i n g an e d g e profile w h i c h v a r i e s f r o m b l a c k to w h i t e o v e r several p i x e l s is to b r i n g in a r a n g e o f r a d i u s v a l u e s : h e n c e it is n e c e s s a r y to a v e r a g e t h e g r a p h
over a suitable
r a n g e o f values. As a r e s u l t t h e e x p e r i m e n t a l c u r v e g o e s s m o o t h l y d o w n to z e r o b e l o w t h e critical r a d i u s , a n d in the 5 × 5 c a s e cuts slightly a c r o s s the E
t h e o r e t i c a l u p p e r b o u n d c u r v e a b o v e t h e critical
o
o8-
d//a
o6
~o o
°°°
o
04
,/ 77
°°oo °
O.2
///
°o r
~ 2
oo i
~ oo ...... ~r---'r 4
I 6
I
I
I
8
b
Fig. 9. Edge shifts for 5 x 5 median filter--grey-scale case. In this graph the upper set of plots represents the experimental results and the upper continuous curve is derived from Table 1. The lower Continuous curve is derived from the model of Section 6. The lower set of plots represents the much reduced shifts obtained with the detail-preserving type of filter (see Section 7).
b/a Fig. 10. Method of averaging required for small circles. This diagram shows how averaging over the various contour radii appearing within the neighbourhood should be performed. The two main effects are: (1) raising of the circle size (reduction of the edge shift) for circular objects below the critical size; (2) lowering of the circle size (increase in the edge shift) for objects above the critical size. VoL 16, No. 2, F e b ru a ry 1989
E.R. Davies / Edge location shifts in median filtering
92
radius. Other minor inaccuracies can be explained as due to the particular (non-circular) shape of the filter neighbourhood and remanent stability effects. Finally, the expected progression from p = 1 through p = 3 and p = 5 to p = ~ is obeyed, though the intermediate edge shift curves both appear to approach zero more rapidly than would have been predicted on the basis of simple interpolation. We conclude that discrete neighbourhoods impart an additional stability on the edge positions. This can be explained in general terms as follows. For a 3 × 3 median filter, there are essentially only - 8 positions around the boundary of a large object which the median filter can erode. (This is approximately correct for binary images where large "circles" tend to be octagonal (see Fig. 6); however, it must also be approximately correct for grey-level images, since we can consider each greylevel outline as a separate binary image which is then eroded on its own.) Hence the effectiveness of the median filter at eroding large objects will in principle fall off by an additional factor proportional to a/b (i.e. relative to the upper bound given by the theory of Section 2): note, however, that this additional factor will not apply for small objects, so the result can never be larger than the upper bound value of E. For the 5 x 5 median filter, which is more sensitive to the curvature of large objects, the same general model seems to apply, but with a different constant of proportionality. However, to fully explain the observed variations we need to model the p-variation. Following Section 4 and considering vertical lines through the E/a vs. b/a graph (Fig. 5), we try modelling the vertical variation as ( ( 1 / a o - l / a ) , since this quantity automatically approaches a constant value as a tends to infinity, and approaches zero as a approaches ao. (Here ao is the effective radius of a 1 x 1 neighbourhood, assumed in this paper to be equal to v / ] ~ = 0.564.) Taking both variations into account gives the overall model: /~ = min[E, cE(1/ao- 1/a)(a/b)] = min[E, cE(a/ao- 1)/b],
(8)
where the constant c will have to be found empiriSignal Processing
cally. As will be clear from Figs. 8 and 9, this formula gives excellent agreement with the observed results for 3 ×3 and 5 x 5 neighbourhoods, when the constant c is made equal to 1.0. Thus it is now known with fair accuracy (even if partly on a semi-empirical basis) how the upper bound form (Section 2) adapts to a discrete lattice. At this stage there can be no doubt that the median filter gives definite and measurable edge shifts via a specific type of neighbourhood averaging process.
7. Use of hybrid median filters
Although median filters preserve edges in digital images, they are also known to remove fine image detail such as lines. For example, 3 ×3 median filters will remove lines one pixel wide, and 5 x 5 median filters will remove lines two pixels wide. In many applications such as remote sensing and X-ray imaging this is exceedingly important, and efforts have been made to develop filters that overcome the problem. Recently, Nieminen et al. have reported a new class of "detail-preserving" filters [9]; these employ linear sub-filters whose outputs are combined by median operations. There are a great variety of such filters, employing different sub-filter shapes and having the possibility of several layers of median operations. Hence it is not possible to describe them fully here in the space available. Although these filters were aimed particularly at retention of line detail, and are easily understood in this context, they also appear to have some corner-preserving properties and to be resistant to the edge shifts noted earlier in this paper that arise when there is a non-zero horizontal curvature. Perhaps the best of the filters in the new class, from the point of view of preserving edge position, is the two-level "bidirectional" linear-median hybrid filter termed " 2 L H + " in [9]. We here illustrate its operation in a 5 × 5 neighbourhood. It employs the sub-filters A-I in the 5 × 5 region:
E.R. D a v i e s / E d g e location shifts in median filtering
E
-
D
-
C:
-
E
D
C
-
F
FA
-
G
H
B B ; I
-
G
-
H
-
I
-pixels m a r k e d as being in the same sub-filter having their intensities averaged, and dashed pixels being ignored. Non-linear filtering then proceeds using two levels o f median filtering, the final centre-pixel intensity being taken as: A ' = med(A, med(A, B, D, F, H ) , med(A, C, E, G, I)).
(9)
We hereby ignore the line-preserving properties o f this filter and concentrate on its cornerpreserving, low edge-shift characteristics. It is quite easy to see that the 5 x 5 regions 0 0 0 0 0
0 0 0 0 0
0 0 1 1 1
0 0 1 1 1
0 0 1 1 1
0 0 0 0 1
0 0 0 1 1
0 0 1 1 1
0 0 0 1 1
0 0 0 0 1
are preserved by this filter, t h o u g h these examples represent limiting cases which could be disrupted by minor amounts o f noise or slight changes o f orientation. Thus the filter seems g u a r a n t e e d to preserve corners only if the internal angle is greater than 135 °. This figure should be c o m p a r e d with the 180 ° obtained using similar arguments for the normal median filter in 5 x 5 regions such as 0 0 0 0 0
0 0 0 0 0
0 0 1 1 1
1 1 1 1 1
Z
1 1 1 1 1
93
Fig. 9 shows plots obtained with this filter u n d e r the same conditions as for the experiments on 5 × 5 median filters. Clearly, it always gives at least a four-fold i m p r o v e m e n t in edge shift over that for the median filter, and this performance improves with increasing radius o f curvature b until there is zero shift for b > 4 . ( N o t e that b = 4 is approximately the figure that would be expected from the corner angle o f 135 ° noted above, within a 5 x 5 n e i g h b o u r h o o d . ) Hence such detail-preserving filters improve the situation dramatically, but do not completely overcome the underlying problem described in Sections 1-6. In addition, this improvement may not have been obtained without cost, since in some cases the filter seems to i n s e r t structure where none exists: this behaviour is shown in Fig. 11 for some o f the small circular disks used in obtaining Figs. 8 and 9. The result is to cast some doubt on the usefulness o f this type o f filter in all possible situations. Nevertheless, its effect on real images appears to be generally very g o o d (see Figs. 2(c) and 12(c)). Finally, it is worth remarking on the difference in root behaviour of the two types of filter. In general terms, the median filter tends first to straighten out irregular edges, and then gradually to make objects smaller by contracting them preferentially at places o f highest curvature: in the end all objects would be reduced to zero size and eliminated (assuming the stability effects noted earlier are ignored). In a sense this means that there are no root signals for a (two-dimensional) median filter other than straight edges going right across an image. On the other hand, the detailpreserving filters seem able to preserve all but the smallest objects, and to be able to do this without first changing object shapes very m u c h - - s t r a i g h t
O
Fig. 11. Effect of detail-preserving filter on small circles. This figure shows the effect of the detail-preserving filter on grey-scale circular disks with smoothed slanted edges. The four circles are the ones initially having radii b - 1.6, 1.8, 2.1, 2.3 pixels and used to produce the graphs in Figs. 8 and 9. The result is to produce structure where none exists, in particular making small disks appear squarer and over-enhancing the corners. Vol. 16, No. 2. February 1989
94
E.R. Davies
Edge location shifts in median filtering
8
edges and curved edges being equally stable. In this case noise is characterised as features of size smaller than the mask size (i.e. < - 4 pixels diameter for the 2LH+ filter studied above). Thus the detail-preserving filter is more finely tuned in its definition of noise, and a great variety of root shapes are possible.
8. Concluding remarks
b
f
Fig. 12. Circular holes in metal objects before and after filtering. (a) shows the original 128× 128 pixel image with 6-bit grey-scale. (b) shows the 5 × 5 median-filtered image. The diminution in size of the holes is clearly visible in (b): such distortions would have to be corrected for when taking measurements from real filtered images of this type. Even when a detailpreserving filter is used (c), some distortions are present, though the overall result is much better than in (b). Signal Processing
This paper has developed a basic theory predicting the edge shifts produced by median filters in continuous binary and grey-scale images. Experiments and the accompanying rationale have extended this theory to the case of discrete binary and grey-scale images. There seems no doubt, either theoretically or in practice, that the use of the median filter results in size and shape distortions that can be serious (see Figs. 2 and 12): indeed, the noise-removing capabilities of the median filter are seen to be special cases of these more general size and shape distortions. Basically, the distortions have analogue rather than discrete effects, and as a result are much more easily estimated than would previously have been expected. The discrete distortion properties of the median filter are in some ways less serious than predicted by the analogue model, since the lattice digitisation tends to give a size stability to objects. Basically this is because a median filter in a small neighbourhood is relatively insensitive to the curvatures of large objects, even when grey-scale resolution is quite high. A model of this situation which explains within - 1 0 % the observed edge shifts in the discrete case has been outlined (equation (8)), making it clear that the initial concept on which this work is based (Section 2) is sound. None of the work described here invalidates other work on shifts generated by median filters in the presence of noise, etc. [1, 10]. Rather, this work describes shifts and distortions that are present even when there is no noise and when edges are not asymmetrical. We have shown that for certain common types of noise these effects should
95
E.R. Davies/ Edge location shifts in median filtering
be a d d i t i v e , a n d this is b o r n e out by e x p e r i m e n t a l d a t a (Figs. 2 a n d 12). The w o r k d e c r i b e d here shows that care s h o u l d be taken in the use o f i m a g e filters in situations such as i n d u s t r i a l i n s p e c t i o n , where a c c u r a t e m e a s u r e m e n t s are to be m a d e - - s e e , for e x a m p l e , [4]. In p a r t i c u l a r , m e d i a n filters s h o u l d not be a p p l i e d unless this is essential: if they d o have to be a p p l i e d , suitable c o r r e c t i o n s will need to be m a d e . In a n y case it is r e c o m m e n d e d that alternatives such as the new t y p e s o f d e t a i l - p r e s e r v i n g filter [9] s h o u l d be c o n s i d e r e d .
We then find: = b - ( b / 2 ) [ 1 - a 2 / b 2] ,/2 - ( b 2 / 2 a ) sin ' ( a / b ) .
To p r o c e e d , we use the p o w e r series e x p a n s i o n o f the inverse sine function: sin -~ v = v + v 3 / 6 + 3 v S / 4 0 +
this research.
Appendix: Finding a formula for edge shift in a circular neighbourhood To find an a p p r o x i m a t e f o r m u l a for the edge shift in a c i r c u l a r n e i g h b o u r h o o d , we here a d o p t the strategy o f finding the m e a n lateral d i s p l a c e m e n t o f the edge within the n e i g h b o u r h o o d . The lateral d i s p l a c e m e n t x o f the edge o f the circle is given by the e q u a t i o n :
which l e a d s to y2) '/2
(A.2)
A s s u m i n g that a ~ b, the m e a n value o f x over the n e i g h b o u r h o o d is:
)
x dy
/f;
t
dy
[b - (b 2 - y 2 ) ,/2] d y / a .
(A.3)
To p e r f o r m the i n t e g r a t i o n , substitute y = b sin u.
(A.7)
= a 2 / 6 b + a 4 / 4 0 b 3.
E q u a t i n g this to the o b s e r v e d edge shift (A.8)
d ~- b - a : / 6 b - a 4 / 4 0 b 3,
(A.9)
as q u o t e d in Section 2. It is interesting that this f o r m u l a is essentially exact o v e r the entire range a ~ b (see T a b l e 1). In view o f the a p p r o x i m a t e nature o f the a b o v e m o d e l , this degree o f a c c u r a c y must be s o m e w h a t fortuitous, t h o u g h it can be put to g o o d use in m a k i n g s i m p l e c o r r e c t i o n s to m e a s u r e m e n t s m a d e from images that have been p r o c e s s e d b y m e d i a n filters.
References
(A.1)
(X -- b ) 2 + y2 = b 2,
~'~- I(a)
(A.6)
we finally o b t a i n :
The a u t h o r is grateful to the A C M E D i r e c t o r a t e o f the U.K. Science a n d E n g i n e e r i n g R e s e a r c h C o u n c i l for financial s u p p o r t d u r i n g the course o f
x= b-(b2-
... ,
whence
E = b - d,
Acknowledgement
(A.5)
(A.4)
[1] A.C. Bovik, T.S. Huang and D.C. Munson, "The effect of median filtering on edge estimation and detection", IEEE Trans. Patt. Anal. Mach. IntelL, Vol. PAM[-9, No. 2, 1987, pp. 181-194. [2] E.R. Davies, "The median filter: an appraisal and a new truncated version", Proc. 7th Int. Conf. on Pattern Recogn., Montreal, 30 July-2 August 1984, pp. 590-592. [3] E.R. Davies, "Circularity--a new principle underlying the design of accurate edge orientation operators", Image and Vision Computing, Vol. 2, No. 3, 1984, pp. 134-142. [4] E.R. Davies, "'Radial histograms as an aid in the inspection of circular objects", IEE Proc 19, Vol. 132, No. 4, Special Issue on Robotics, 1985, pp. 158-163. [5] J.P. Fitch, E.J. Coyle and N.C. Gallagher, "'Root properties and convergence rates of median filters", IEEE Trans. Acoust., Speech, Signal Process., Vol. ASSP-33, No. 1, 1985, pp. 230-239. Vol. 16. No. 2. FebruaD 1989
96
E.R. Davies / Edge location shifts in median filtering
[6] N.C. Gallagher and G.L. Wise, "'A theoretical analysis of the properties of median filters", IEEE Trans. Acoust., Speech, Signal Process., Vol. ASSP-29, No. 6, 1981, pp. 1136-1141. [7] P. Heinonen and Y. Neuvo, "FIR-median hybrid filters", IEEE Trans. Aeoust., Speech, Signal Process., Vol. ASSP35, No. 6, 1987, pp. 832-838. [8] R.M. Hodgson, D.G. Bailey, M.J. Naylor, A.L.M. Ng and S.J. McNeil, "'Properties, implementations, and applications of rank filters", Image and Vision Computing, Vol. 3, No. 1, 1985, pp. 4-14.
Signal Processing
[9] A. Nieminen, P. Heinonen and Y. Neuvo, "A new class of detail-preserving filters for image processing", IEEE Trans. Part. Anal. Math. lntelL, Vol. PAMI-9, No. 1, 1987, pp. 74-90. [ 10] G.J. Yang and T.S. Huang, "The effect of median filtering on edge location estimation", Comput. Graph. Image Process., Vol. 15, 1981, pp. 224-245.
83
EDGE L O C A T I O N SHIFTS P R O D U C E D BY M E D I A N FILTERS: THEORETICAL B O U N D S AND EXPERIMENTAL RESULTS E.R. DAVIES Machine Vision Group, Department of Physics, Royal Holloway & BedJbrd New College, Egham Hill, Egham, Surrey TW20 OEX, U.K. Received 10 June 1987 Revised 30 November 1987 and 21 April 1988
Abstract. This paper studies size and shape distortions resulting from the use of median filters. The starting point is the filtering of binary objects by operators acting within circular neighbourhoods in a continuous space. This is found to lead to predictable shifts of edges towards the local centres of curvature. In a discrete lattice the effect is modified by the stability of certain objects against edge distortions, small objects generally being less stable than large objects. In grey-scale images, these effects do not disappear but are less size-critical, and are clearly visible with real images when significant noise is present.
Zusammenfassung. In dieser Arbeit werden Verzerrungen von Gr613e und Form untersucht, wie sie durch die A n w e n d u n g von Median-Filtern entstehen. Ausgegangen wird von der Filterung binfirer Objekte durch Operatoren, die kreisf6rmige U m g e b u n g e n in einem kontinuierlichen Raum bearbeiten. Es l~il3t sich zeigen, dab dies zu vorhersagbaren Kantenverschiebungen in Richtung a u f die lokalen Zentren der Kurvenziige ffihrt. In einem diskreten Gitter wird dieser Effekt insofern modifiziert, als gewisse Objekte gegen Kantenverzerrungen unempfindlich sind, wobei kleine Objekte i.a. weniger robust sind als grol3e. In Grauton-Bildern verschwinden diese Effekte nicht, sind jedoch weniger kritisch beziiglich der Gro/3e; sie sind hier in echten Bildern klar zu erkennen, sobald ein nennenswerter Rauschanteil vorhanden ist. R6sumd. Cet article dtudie les distorsions de taille et de forme r6sultant de l'utilisation des filtres m6dians. Le point de ddpart est le filtrage des objets binaires par des opdrateurs agissant sur des voisinages circulaires dans l'espace continu. I1 a 6td trouv6 que ceci conduit/~ des d6calages prdvisibles des contours vers des centres locaux de courbure. Dans une trame discrete I'effet est modif6 par la stabilit6 de certains objets vis fi vis des distorsions de contours, les petits objets dtant moins stables que les grands objets. Dans les images fi niveaux de gris, ces effets ne disparaissent pas mais sont moins sensible ti la taille et sont clairement visibles dans des images rdelles en pr6sence du bruit.
Keywords. Shape distortions, edge shifts, median filter, noise suppression. "The analysis t?["median filters remains intractable Jor all but the most simple assumptions"--Bovik et al. [ 1].
1. Introduction Over the past 10-12 years the median filter has become widely accepted as a general purpose but simple means for removing noise from digital images: indeed, it is probably the most widely used single technique for image noise suppression. Not only is it highly effective, especially with impulse noise, but also it does not suffer from the main drawback of serious image blurring introduced by the Gaussian smoothing operator [2, 8].
More recently, there has been some worry that the median filter may shift edges even if it is successful in not causing blurring [1, 10]. In part these worries stem from the fact that the median filter is a non-linear operator, so that its properties are not entirely obvious: nor are they easy to analyse mathematically [1] (see quotation at the head of this paper). As a result a number of highly mathematical studies, coupled with complicated experimental investigations, have tried to probe the situation, taking account for example of the effects of
0165-1684/89/$3.50 (c~ 1989, Elsevier Science Publishers B.V. (North-Holland)
84
E.R. Davies / Edge location shifts in median filtering
noise which might induce the filter to shift object boundaries [1, 10]. Although thorough, such investigations do not seem to have led to great insight in describing the properties of the median filter. Hence further consideration of the principles underlying median filters in the two-dimensional case that is of interest in dealing with images seemed to be required. As a start, it seemed worthwhile to consider the effects of applying median filters in a continuous space in order to specifically simplify the situation. Such a model should approximate to the situation for large objects and large neighbourhood operators, but should also throw light on what happens in the opposite approximation of a discrete lattice using the usual small neighbourhood operators. In Section 2 we apply this model in the case of binary images, and in Section 3 we study grey-scale images. In Section 4 we consider the situation for a discrete lattice, and in Sections 5 and 6 we report experiments on (discrete) binary and grey-scale images. In Section 7 we consider the edge-shifting characteristics of hybrid median filters. Finally, we summarise our current understanding of the median filter's edge-shifting properties.
2. Median filters in continuous binary images In this section we take the case of a continuous image (i.e. a non-discrete lattice), here assuming for simplicity (1) that the image is binary, (2) that neighbourhoods are exactly circular, and (3) that images are noise-free. We relax these three conditions in the next section. To proceed, we notice that binary edges have symmetrical cross-sections, while straight edges extend this symmetry into two dimensions: hence applying a median filter in a circular neighbourhood cannot pull a straight edge to one side or the other. Now let us see what happens when the filter is applied to an edge that is not straight. If, for example, the edge is circular, the local intensity distribution will contain two peaks whose relative sizes will vary with the precise position of the Signal Processing
a
i.II i[
j i
iI..
I
Fig. 1. Variation in the local intensity distribution with the position of the neighbourhood. (a) shows a neighbourhood of radius a overlapping a dark circular object of radius b. (b)-(d) show, respectively, the intensity distributions f when the separations of the centres are less than, equal to, or greater than the distance d for which the object bisects the area of the neighbourhood.
neighbourhood (Fig. 1). At some position the sizes of the two peaks will be identical. Clearly, this happens when the centre of the neighbourhood is at a unique distance from the centre of a circular object: this is the position at which the output of the median filter changes from dark to light (or vice versa). Thus the median filter produces an inwards shift towards the centre of the circular object (or the centre of curVature), whether the object is dark on a light background or light.on a dark background.
E.R. Davies / Edge location shifts in median filtering
Next suppose that the edge is irregular with several " b u m p s " (i.e. prominences or indentations) within the filter neighbourhood: clearly the filter will now tend to average out the bumps and straighten the edge, since it acts in such a way as to equalise the amounts of dark and light within the neighbourhood (Fig. 2). This means that the edge will be locally biased, but only by a small amount since the various bumps will tend to pull the final edge in opposite directions. On the other hand, if there is one gross b u m p within the neighbourhood--i.e, if the curvature has the same sign and is roughly constant at all points on the edge within the filter n e i g h b o u r h o o d - - t h e n all these parts of the edge will act in consort and it will be pulled sideways a significant amount by the filter. We now see that a circular section of the boundary constitutes a "worst case" situation, for which the filter produces the largest bias in the position of the edge. (Mathematically, the lateral deviation per unit length is greatest for a circle, for a given maximum magnitude of the local curvature.) It is clearly worth finding the size of the worst-case shift, and for this reason we concentrate attention below on circular objects, in the knowledge that all other shapes will give less serious shifts and distortions. The "worst case" calculation is a matter of elementary geometry: we need to find at what distance d from the centre of a circular object (of radius b) the area of a circular neighbourhood (of radius a) is bisected by the object boundary. For small neighbourhoods (a << b), it is possible to find an approximation by averaging the lateral displacement on a circular arc. The result (proved in the Appendix) is:
d / b ~- l - a 2 / 6 b 2 - a4/40b 4.
85
a
b
C
(1)
This analytic solution is useful because of its simplicity and surprisingly high accuracy over the whole range a ~ b. Here we examine the problem from a different viewpoint: this gives a set of equations which can then be solved numerically. From Fig. 3 the area of the sector of angle 2/3 is /3b 2, while the area of the triangle of angle 2/3
Fig. 2. Edge smoothing property of the median filter. (b) shows how the median filter smooths out irregularities in an image (a), and in particular those around the boundaries of objects. Notice how the threads on the screws are virtually eliminated, though detail larger in scale than half the filter area is preserved. In this case the median filter was a 21-element filter operating within a 5 × 5 neighbourhood on a 128x128 pixel image of 6-bit grey-scale. (c) shows the effect of a new type of "'detailpreserving" filter (see Section 7). Vo] 16, No. 2, February 1989
E.R. Davies / Edge location shifts in median filtering
86
Table 1 Estimated edge shifts for filtering binary circles in a continuous space
a/b
d'/b
d/b
E'/b
E/b
0.050 O.1O0 0.150 0.200 0.250 0.300 0.350 0.400 0.450 0.500 0.550 0.600 0.650 0.700 0.750 0.800 0.850 0.900 0.950 1,000 1.050 1.1O0 1.150 1.200 1.250 1.300 1.350 1.400 1.414
1,000 0,998 0,996 0,993 0,989 0,985 0,979 0,973 0,965 0.957 0.947 0.937 0.925 0.912 0.898 0.883 0.867 0.849 0,829 0.808 0.786 0.762 0.736 0,708 0.679 0.647 0,613 0.577 0.567
1.000 0,998 0.996 0.993 0.990 0,985 0.979 0.973 0.965 0.957 0.948 0.937 0.926 0.913 0.899 0.884 0.868 0,849 0.830 0.808 0.784 0.758 0,728 0.695 0.658 0.613 0.557 0.472 0.414
0.000 0.002 0.004 0.007 0.011 0.015 0.021 0.027 0.035 0.043 0,053 0,063 0.075 0.088 0,102 0.117 0.133 O.151 0.171 0.192 0.214 0.238 0.264 0.292 0.321 0,353 0,387 0.423 0.433
0.000 0,002 0,004 0.007 0.010 0.015 0.021 0.027 0,035 0.043 0.052 0,063 0.074 0.087 0.101 0.116 0.132 O.151 0.170 0.192 0,216 0.242 0.272 0.305 0.342 0,387 0.443 0.528 0.586
m
I I
!I
Fig. 3. Geometry for calculating neighbourhood and object overlap. is b 2 s i n / 3 cos/3. H e n c e t h e a r e a o f the s e g m e n t s h o w n s h a d e d is:
B = b2(/3 - s i n / 3 c o s / 3 ) .
(2)
M a k i n g a s i m i l a r c a l c u l a t i o n o f the a r e a A o f a circular segment of radius a and angle 2a, we now d e d u c e t h e a r e a o f o v e r l a p (Fig. 3) b e t w e e n t h e circular neighbourhood
o f r a d i u s a a n d t h e cir-
c u l a r o b j e c t o f r a d i u s b as:
C=A+B.
(3)
F o r a m e d i a n filter this is e q u a l to ~ra2/2. H e n c e
F = a2(a - s i n a cos a ) +b2(/3 - s i n / 3 c o s / 3 ) - 7 r a 2 / 2 = 0,
(4)
where
a 2= b Z + d 2 - 2 b d c o s / 3
(5)
b 2 = a 2 + d 2 - 2 a d c o s a.
(6)
and
T o s o l v e this set o f e q u a t i o n s , we t a k e a g i v e n
d' is the approximate value of d estimated using equation ( 1): the range of validity of this equation is a/b ~ 1.000. Note the excellent agreement between d' and d over this entire range. E-values are derived from corresponding d-values using equation (7). Note that the object disappears when a~ b > 1.414. approaches
a n d t h e n e x c e e d s b. N o t e , h o w e v e r ,
t h a t w h e n a > ,,/2b, t h e o b j e c t is i g n o r e d , b e i n g
v a l u e o f d, d e d u c e v a l u e s o f a a n d /3, c a l c u l a t e
s m a l l e n o u g h to be r e g a r d e d as i r r e l e v a n t n o i s e by
t h e v a l u e o f F, a n d t h e n a d j u s t the v a l u e o f d until F = 0. T h e results o f d o i n g this n u m e r i c a l l y are
the filter: b e y o n d this p o i n t it has no effect at all o n the final i m a g e . T h e m a x i m u m e d g e shift b e f o r e
s u m m a r i s e d in T a b l e 1. In fact d is t h e m o d i f i e d
t h e o b j e c t finally d i s a p p e a r s is ( 2 - x/2)b-~ 0.586b.
v a l u e o f b o b t a i n e d a f t e r filtering, a n d t h e shift p r o d u c e d by t h e filtering p r o c e s s is: E = b - d. As e x p e c t e d , E ~ 0
(7) Con-
T o e x t e n d the a b o v e results to g r e y - s c a l e i m a g e s ,
v e r s e l y , t h e shift b e c o m e s v e r y large as a first
we first c o n s i d e r t h e effect o f a p p l y i n g a m e d i a n
Signal
Processing
as b ~ o o
o r as a ~ 0 .
3. Extension to grey-scale images
87
E.R. Davies/ Edge location shifts in median filtering
filter near a smooth step edge in one dimension. Here the median filter gives zero shift, since for equal distances from the centre to either end of the neighbourhood there are equal numbers of higher and lower intensity values and hence equal areas under the corresponding portions of the intensity histogram. Clearly, this is always valid where the intensity increases monotonically from one end of the neighbourhood to the o t h e r - - a property first pointed out by Gallagher and Wise [6]. (See also [5,7] for recent discussions on related "'root" (invariance) properties of signals under median filtering.) Here it is useful that this is a rather general result. Next we extend the result to two dimensions in the vicinity of a straight edge. We assume here that the intensity values do not vary in the direction parallel to the straight edge, so the situation remains highly symmetrical. Then for an edge with monotonically increasing intensity profile, the numbers of higher intensity values on the one side of the centre of the neighbourhood (looking along the edge direction) will again balance the numbers of lower intensity values on the other side, and the median filter will give zero shift. Implicit in this proof is the use of a circular neighbourhood. However, the median filter will produce zero shift of a straight edge for a n y symmetrically shaped neighbourhood, including square or other neighbourhoods. The proof of this is quite simple. Part of the area of the neighbourhood can be eliminated if the corresponding area on the diametrically opposite end of the neighbourhood is also eliminated, since there is no intensity variation parallel to the straight edge. The underlying reason for this insensitivity to both the shape of the neighbourhood and the shape of the edge intensity profile is that the median filter can be thought of as ignoring equal areas on either side of the local intensity histogram, where the intensities will be either greater or less than the intensity at the c e n t r e - - h o w much greater or less being irrelevant. The arguments presented above are based partly on symmetry: shifts can therefore start appearing for edges that are not straight. It is fairly clear that
irregular boundaries will again tend to be averaged out by a median filter, and that circular boundaries will again be a "worst case". In what follows we therefore concentrate again on the case of circular objects. However, a full analysis requires some consideration of the fact that grey-scale edges are unlike binary edges in having finite slope. This means that we should take account of the exact form of the intensity function within the neighbourhood. When boundaries are roughly circular, contours of constant intensity will often appear as in Fig. 4.
( 1 Fig. 4. Appearance of contours of constant intensity. This figure shows contours of constant intensity on the edge of a large circular object, as seen within a small circular neighbourhood. To find how a median filter acts we merely need to identify the contour of median intensity (in two dimensions the median intensity value labels a whole contour) which divides the area of the neighbourhood into two equal parts. The geometry of the situation is identical to that already examined in Section 2: the main difference here is that for every position of the neighbourhood, there is a corresponding median contour with its own particular value of shift depending on the curvature. Fig. 4 shows an idealised case in which the contours of constant intensity have similar curvature, so that they are all moved inwards by similar amounts. This means that, to a first approximation, the edges of the object retain their cross-sectional profile as it becomes smaller. The size of the object is probably best defined from one particular contour such as that at which the intensity is the mean of the m a x i m u m and minimum intensities--which should be unchanged by the median filter. Vol. 16, No. 2, February 1989
88
E.R. Davies / Edge location shifts in median filtering
We next consider the effects of noise. For simplicity we assume that the noise is additive and of symmetrical (non-skew) intensity distribution: this is valid for Gaussian noise and is also likely to be true for many types of impulse noise. Now recall that the median contour divides the neighbourhood into two equal parts. Hence, adding noise of symmetrical intensity distribution will on average not change the area on either side of the original median contour: this means that noise will not on average cause edges to shift any differently as a result of applying the median filter--i.e, the shifts of edges caused by noise or by applying a median filter are, tofirst order, additive. In particular, noise
does not affect the general conclusions of this paper concerning the shifts of edges introduced by median filters. Though specific experiments have not been performed in this work to introduce noise and check this result quantitatively, it is generally supported by our observations on real images containing noise. This section has generalised the results of Section 2 to cover grey-scale images. It has also shown that the effects of noise should not materially affect the conclusion that median filters shift edges inwards towards local centres of curvature, the "worst case" situation arising for circular objects. Finally, it has shown that straight edges remain unshifted for any symmetrically shaped neighbourhood. However, the detailed shift calculations (Section 2) assumed a circular neighbourhood: though this is not general, it will produce an ideal isotropic response which could not be guaranteed for any other shape of neighbourhood. This fact justifies our concentration on circular neighbourhoods in the above analysis. For neighbourhoods of other shapes it seems simplest to confirm experimentally that the theory makes substantially correct predictions of edge shifts.
4. Extension to discrete neighbourhoods In the previous two sections we have developed theory showing how median filters can introduce Signal Processing
edge shifts in a continuous space. Here we consider the effect of applying median filters in discrete lattices of pixels: specifically, median filters need to be applied in square neighbourhoods of p x p pixels. In the continuous case covered earlier, p was essentially infinite. Unfortunately, it is difficult to see how to extend the theory accurately to typical cases such as 3 × 3 and 5 × 5 neighbourhoods. However, it is trivial to cover the case of p = 1, since in this case the median filter leaves the image unchanged. For intermediate values o f p we expect that edge shifts will fail between these two cases as upper and lower bounds, and indeed that there will be a steady progression from the one to the other bound as p varies (Fig. 5). As we shall see below, this situation is generally confirmed by our experimental data. E/a
0
b/a
Fig. 5. Edge shift curves for various sizes of neighbourhood. This diagram indicates h o w the edge shift curves would be expected to change as p moves between the two limiting values of I and infinity: (a) p = 1, (b) p = 3, (c) p = 5, (d) p = ~ .
5. Median filters in discrete binary images In this section we present experimental results for binary images. For reasons outlined earlier, we concentrate on the worst-case situation of small circular objects. Clearly, in a binary image it is only possible to approximate to (filled) circles, and in these tests radii ranged from 0.5 to about 9 pixels. It seemed sufficient to perform experiments
E.R. D a v i e s / E d g e location shifts in median filtering
using s t a n d a r d m e d i a n filters in a 3 × 3 ( s q u a r e ) n e i g h b o u r h o o d . In these e x p e r i m e n t s , a p r o b l e m arose since the effective r a d i u s o f such a n e i g h b o u r h o o d is not k n o w n a c c u r a t e l y : for the p r e s e n t p u r p o s e we a s s u m e it to be such as to e q u a l i s e the
89
!
E
08/
0 0 0
+
*
-_-+
1
@
•
+ + 0 + _ . _
ll
__+_
0 o o
o oo
O
7
a
6
8
Fig. 7. Edge shifts for binary circles. In this graph the plots represent the experimental results and the continuous curve is derived from Table 1, following the theory of Section 2. It is also of interest to compare the experimental plots with the model of Section 6 (see lower curve in Fig. 8).
areas o f discrete a n d i d e a l i s e d n e i g h b o u r h o o d s .
b
+
+ 1
+
o
m
e
•
•
O
O
OOOO
/
Fig. 6. Binary circles before and after filtering. (a) shows a set of binary circles of radii ranging from 0.5 to 9 pixels. (b) shows the result of applying a median filter operating in a 3 × 3 (square) neighbourhood.
H e n c e we t o o k the r a d i u s as x/9/~r = 1.693 pixels. The results are s h o w n in Figs. 6 a n d 7. W h e n c o m p a r e d with the results o f Table 1 they s h o w s o m e interesting features. In p a r t i c u l a r : (1) T h e r e are a limited n u m b e r o f p o s s i b l e r a d i u s values. (2) F o r a large p r o p o r t i o n o f r a d i u s values no c h a n g e in r a d i u s occurs on a p p l y i n g a m e d i a n filter. (3) F o r very small r a d i u s values the circle d i s a p pears completely. (4) F o r o t h e r small r a d i u s values the circle b e c o m e s m u c h smaller. (5) F o r certain i s o l a t e d larger r a d i u s values there is a r e d u c t i o n in circle size, but (a) the n u m b e r o f i n s t a n c e s b e c o m e s rarer as r a d i u s increases, a n d (b) the r e d u c t i o n in circle size b e c o m e s s m a l l e r as radius increases. (6) In general, r e p e a t e d a p p l i c a t i o n o f a m e d i a n filter to circles a b o v e a certain critical size m u s t l e a d to a small r e d u c t i o n in size f o l l o w e d by stability, w h e r e a s r e p e a t e d a p p l i c a t i o n to circles b e l o w the critical size must l e a d to their e l i m i n a t i o n : the critical r a d i u s is - 2 . 5 pixels. Vol. 16, No. 2, F e b r u a r y 1989
90
E.R. D a v i e s / E d g e location shifts in median filtering
Ignoring result (1) as obvious, we interpret these results as follows. When applied to discrete binary images, the median filter has the properties predicted in Section 2. However, median filters in small neighbourhoods do not have the resolution to detect accurately the curvature of large circles: hence these either become resistant to any change in their size and shape or seem rather unstable and ready to shed their outermost pixels. The latter situation clearly happens for those circles whose boundaries are irregular, since they have some relatively sharp corners which are eliminated by the median filter. Such corners become increasingly rare as circle size increases (see Fig. 6). We return to this point in Section 6, with the aim of building a more realistic model of the action of the median filter in the discrete case. The stability properties we have observed are related to the root behaviour noticed by other workers when median filters are applied repeatedly (mainly to one-dimensional signals) until no further change occurs [5-7]. HOwever, we are here less interested in root behaviour than in mean edge shifts, for a single application of a median filter, as curvatures vary. Hence it is instructive to average out the rather random responses that occur for various radii b. It is seen that the resulting curve (Fig. 7) is similar in shape to the theoretical curve of Section 2, but lies below it and indeed between it and the identity curve corresponding to p = 1--as predicted in Section 4: in no case does the stability effect cause the predicted change in size of an object to be reversed in sign, though it is frequently reduced to zero.
6. Median filters in discrete grey-scale images
The results of the previous section immediately suggest using better approximations to circles, with the jagged binary edges interpolated by appropriate grey-scale values. For each size of circle this was achieved by permitting the intensity to vary linearly from black to white over a range of 1 pixel, and then smoothing the resulting shapes using the Signal Processing
following well-known convolution mask: 1
4
ig
2
.
This procedure was successful in giving a realistic approximation to a smoothed step-edge. (Note, however, that other edge models, such as edges which vary linearly over 1 or 2 pixels, seemed to give essentially the same edge shifts--i.e, the edge shift behaviour was relatively insensitive to the exact type of edge model chosen.) In the experiments described here, circular objects again varied between - 1 and 9 pixels, and pixel intensities were permitted to vary over an 8-bit range. The original and the modified radii were measured by taking the integrated intensities over the circle region and deducing the radii, this approach being used because it overcame problems due to irregularities in circle boundaries. Finally, the experiment was in this case performed for 3 × 3 and 5 × 5 neighbourhoods, though in the latter case an attempt was made at approximating to the more ideal circular neighbourhood by omitting the four corner pixels, and using the pattern: X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
(Note that in some earlier work on edge detection operators, the author found that this pattern not only required less computation than a full 5 × 5 operator, but also gave increased edge orientation accuracy [3].) The results of these experiments are shown in Figs. 8 and 9. In these cases certain features that were present in the previous experiment on binary circles are now absent. In particular, the variation of edge shift E with initial radius b is very smooth, and there is no evidence of stability of large circles against the median filter: i.e. the median filter is
E.R. Davies / Edge location shifts in median filtering
91
a b l e to r e d u c e t h e size o f all circles by an a m o u n t
E 0.8-
w h i c h t e n d s to z e r o m u c h as e x p e c t e d as r a d i u s increases. It s e e m s i m p o r t a n t to a n a l y s e this case r e a s o n a b l y t h o r o u g h l y , as it c o n s t i t u t e s a c o m m o n p r a c tical s i t u a t i o n . H e n c e it was c o m p a r e d q u a n t i t a t i v e l y w i t h t h e t h e o r y o f S e c t i o n 2. A g a i n t h e p r o b lem a r o s e t h a t the e f f e c t i v e r a d i u s o f a d i s c r e t e
b
m e d i a n filter in a 3 × 3 o r 5 × 5 n e i g h b o u r h o o d
o
is
not known accurately: adopting the equal areas s t r a t e g y o f the p r e v i o u s s e c t i o n , we o b t a i n t h e respective
radii
as x / 9 / ~ r = 1.693
and
x/~-/~r=
2.585. T h e r e s u l t i n g t h e o r e t i c a l g r a p h s a r e s h o w n i
i
i
2
4
i
i
i
6
o"o+~++ i
i
8
in Figs. 8 a n d 9. T h e s e s h o w t h a t t h e r e are s o m e b
Fig. 8. Edge shifts for 3 × 3 median filter--grey-scale case. In this graph the plots represent the experimental results and the upper continuous curve is derived from Table 1. The lower continuous curve is derived from the model of Section 6.
o d d results f o r v e r y l o w v a l u e s o f circle radius. O n t h e w h o l e t h e s e c a n b e e x p l a i n e d r a t h e r n e a t l y by a p p e a l i n g to t h e d / a vs. b / a c u r v e (Fig. 10). H e r e we see t h a t t h e effect o f h a v i n g an e d g e profile w h i c h v a r i e s f r o m b l a c k to w h i t e o v e r several p i x e l s is to b r i n g in a r a n g e o f r a d i u s v a l u e s : h e n c e it is n e c e s s a r y to a v e r a g e t h e g r a p h
over a suitable
r a n g e o f values. As a r e s u l t t h e e x p e r i m e n t a l c u r v e g o e s s m o o t h l y d o w n to z e r o b e l o w t h e critical r a d i u s , a n d in the 5 × 5 c a s e cuts slightly a c r o s s the E
t h e o r e t i c a l u p p e r b o u n d c u r v e a b o v e t h e critical
o
o8-
d//a
o6
~o o
°°°
o
04
,/ 77
°°oo °
O.2
///
°o r
~ 2
oo i
~ oo ...... ~r---'r 4
I 6
I
I
I
8
b
Fig. 9. Edge shifts for 5 x 5 median filter--grey-scale case. In this graph the upper set of plots represents the experimental results and the upper continuous curve is derived from Table 1. The lower Continuous curve is derived from the model of Section 6. The lower set of plots represents the much reduced shifts obtained with the detail-preserving type of filter (see Section 7).
b/a Fig. 10. Method of averaging required for small circles. This diagram shows how averaging over the various contour radii appearing within the neighbourhood should be performed. The two main effects are: (1) raising of the circle size (reduction of the edge shift) for circular objects below the critical size; (2) lowering of the circle size (increase in the edge shift) for objects above the critical size. VoL 16, No. 2, F e b ru a ry 1989
E.R. Davies / Edge location shifts in median filtering
92
radius. Other minor inaccuracies can be explained as due to the particular (non-circular) shape of the filter neighbourhood and remanent stability effects. Finally, the expected progression from p = 1 through p = 3 and p = 5 to p = ~ is obeyed, though the intermediate edge shift curves both appear to approach zero more rapidly than would have been predicted on the basis of simple interpolation. We conclude that discrete neighbourhoods impart an additional stability on the edge positions. This can be explained in general terms as follows. For a 3 × 3 median filter, there are essentially only - 8 positions around the boundary of a large object which the median filter can erode. (This is approximately correct for binary images where large "circles" tend to be octagonal (see Fig. 6); however, it must also be approximately correct for grey-level images, since we can consider each greylevel outline as a separate binary image which is then eroded on its own.) Hence the effectiveness of the median filter at eroding large objects will in principle fall off by an additional factor proportional to a/b (i.e. relative to the upper bound given by the theory of Section 2): note, however, that this additional factor will not apply for small objects, so the result can never be larger than the upper bound value of E. For the 5 x 5 median filter, which is more sensitive to the curvature of large objects, the same general model seems to apply, but with a different constant of proportionality. However, to fully explain the observed variations we need to model the p-variation. Following Section 4 and considering vertical lines through the E/a vs. b/a graph (Fig. 5), we try modelling the vertical variation as ( ( 1 / a o - l / a ) , since this quantity automatically approaches a constant value as a tends to infinity, and approaches zero as a approaches ao. (Here ao is the effective radius of a 1 x 1 neighbourhood, assumed in this paper to be equal to v / ] ~ = 0.564.) Taking both variations into account gives the overall model: /~ = min[E, cE(1/ao- 1/a)(a/b)] = min[E, cE(a/ao- 1)/b],
(8)
where the constant c will have to be found empiriSignal Processing
cally. As will be clear from Figs. 8 and 9, this formula gives excellent agreement with the observed results for 3 ×3 and 5 x 5 neighbourhoods, when the constant c is made equal to 1.0. Thus it is now known with fair accuracy (even if partly on a semi-empirical basis) how the upper bound form (Section 2) adapts to a discrete lattice. At this stage there can be no doubt that the median filter gives definite and measurable edge shifts via a specific type of neighbourhood averaging process.
7. Use of hybrid median filters
Although median filters preserve edges in digital images, they are also known to remove fine image detail such as lines. For example, 3 ×3 median filters will remove lines one pixel wide, and 5 x 5 median filters will remove lines two pixels wide. In many applications such as remote sensing and X-ray imaging this is exceedingly important, and efforts have been made to develop filters that overcome the problem. Recently, Nieminen et al. have reported a new class of "detail-preserving" filters [9]; these employ linear sub-filters whose outputs are combined by median operations. There are a great variety of such filters, employing different sub-filter shapes and having the possibility of several layers of median operations. Hence it is not possible to describe them fully here in the space available. Although these filters were aimed particularly at retention of line detail, and are easily understood in this context, they also appear to have some corner-preserving properties and to be resistant to the edge shifts noted earlier in this paper that arise when there is a non-zero horizontal curvature. Perhaps the best of the filters in the new class, from the point of view of preserving edge position, is the two-level "bidirectional" linear-median hybrid filter termed " 2 L H + " in [9]. We here illustrate its operation in a 5 × 5 neighbourhood. It employs the sub-filters A-I in the 5 × 5 region:
E.R. D a v i e s / E d g e location shifts in median filtering
E
-
D
-
C:
-
E
D
C
-
F
FA
-
G
H
B B ; I
-
G
-
H
-
I
-pixels m a r k e d as being in the same sub-filter having their intensities averaged, and dashed pixels being ignored. Non-linear filtering then proceeds using two levels o f median filtering, the final centre-pixel intensity being taken as: A ' = med(A, med(A, B, D, F, H ) , med(A, C, E, G, I)).
(9)
We hereby ignore the line-preserving properties o f this filter and concentrate on its cornerpreserving, low edge-shift characteristics. It is quite easy to see that the 5 x 5 regions 0 0 0 0 0
0 0 0 0 0
0 0 1 1 1
0 0 1 1 1
0 0 1 1 1
0 0 0 0 1
0 0 0 1 1
0 0 1 1 1
0 0 0 1 1
0 0 0 0 1
are preserved by this filter, t h o u g h these examples represent limiting cases which could be disrupted by minor amounts o f noise or slight changes o f orientation. Thus the filter seems g u a r a n t e e d to preserve corners only if the internal angle is greater than 135 °. This figure should be c o m p a r e d with the 180 ° obtained using similar arguments for the normal median filter in 5 x 5 regions such as 0 0 0 0 0
0 0 0 0 0
0 0 1 1 1
1 1 1 1 1
Z
1 1 1 1 1
93
Fig. 9 shows plots obtained with this filter u n d e r the same conditions as for the experiments on 5 × 5 median filters. Clearly, it always gives at least a four-fold i m p r o v e m e n t in edge shift over that for the median filter, and this performance improves with increasing radius o f curvature b until there is zero shift for b > 4 . ( N o t e that b = 4 is approximately the figure that would be expected from the corner angle o f 135 ° noted above, within a 5 x 5 n e i g h b o u r h o o d . ) Hence such detail-preserving filters improve the situation dramatically, but do not completely overcome the underlying problem described in Sections 1-6. In addition, this improvement may not have been obtained without cost, since in some cases the filter seems to i n s e r t structure where none exists: this behaviour is shown in Fig. 11 for some o f the small circular disks used in obtaining Figs. 8 and 9. The result is to cast some doubt on the usefulness o f this type o f filter in all possible situations. Nevertheless, its effect on real images appears to be generally very g o o d (see Figs. 2(c) and 12(c)). Finally, it is worth remarking on the difference in root behaviour of the two types of filter. In general terms, the median filter tends first to straighten out irregular edges, and then gradually to make objects smaller by contracting them preferentially at places o f highest curvature: in the end all objects would be reduced to zero size and eliminated (assuming the stability effects noted earlier are ignored). In a sense this means that there are no root signals for a (two-dimensional) median filter other than straight edges going right across an image. On the other hand, the detailpreserving filters seem able to preserve all but the smallest objects, and to be able to do this without first changing object shapes very m u c h - - s t r a i g h t
O
Fig. 11. Effect of detail-preserving filter on small circles. This figure shows the effect of the detail-preserving filter on grey-scale circular disks with smoothed slanted edges. The four circles are the ones initially having radii b - 1.6, 1.8, 2.1, 2.3 pixels and used to produce the graphs in Figs. 8 and 9. The result is to produce structure where none exists, in particular making small disks appear squarer and over-enhancing the corners. Vol. 16, No. 2. February 1989
94
E.R. Davies
Edge location shifts in median filtering
8
edges and curved edges being equally stable. In this case noise is characterised as features of size smaller than the mask size (i.e. < - 4 pixels diameter for the 2LH+ filter studied above). Thus the detail-preserving filter is more finely tuned in its definition of noise, and a great variety of root shapes are possible.
8. Concluding remarks
b
f
Fig. 12. Circular holes in metal objects before and after filtering. (a) shows the original 128× 128 pixel image with 6-bit grey-scale. (b) shows the 5 × 5 median-filtered image. The diminution in size of the holes is clearly visible in (b): such distortions would have to be corrected for when taking measurements from real filtered images of this type. Even when a detailpreserving filter is used (c), some distortions are present, though the overall result is much better than in (b). Signal Processing
This paper has developed a basic theory predicting the edge shifts produced by median filters in continuous binary and grey-scale images. Experiments and the accompanying rationale have extended this theory to the case of discrete binary and grey-scale images. There seems no doubt, either theoretically or in practice, that the use of the median filter results in size and shape distortions that can be serious (see Figs. 2 and 12): indeed, the noise-removing capabilities of the median filter are seen to be special cases of these more general size and shape distortions. Basically, the distortions have analogue rather than discrete effects, and as a result are much more easily estimated than would previously have been expected. The discrete distortion properties of the median filter are in some ways less serious than predicted by the analogue model, since the lattice digitisation tends to give a size stability to objects. Basically this is because a median filter in a small neighbourhood is relatively insensitive to the curvatures of large objects, even when grey-scale resolution is quite high. A model of this situation which explains within - 1 0 % the observed edge shifts in the discrete case has been outlined (equation (8)), making it clear that the initial concept on which this work is based (Section 2) is sound. None of the work described here invalidates other work on shifts generated by median filters in the presence of noise, etc. [1, 10]. Rather, this work describes shifts and distortions that are present even when there is no noise and when edges are not asymmetrical. We have shown that for certain common types of noise these effects should
95
E.R. Davies/ Edge location shifts in median filtering
be a d d i t i v e , a n d this is b o r n e out by e x p e r i m e n t a l d a t a (Figs. 2 a n d 12). The w o r k d e c r i b e d here shows that care s h o u l d be taken in the use o f i m a g e filters in situations such as i n d u s t r i a l i n s p e c t i o n , where a c c u r a t e m e a s u r e m e n t s are to be m a d e - - s e e , for e x a m p l e , [4]. In p a r t i c u l a r , m e d i a n filters s h o u l d not be a p p l i e d unless this is essential: if they d o have to be a p p l i e d , suitable c o r r e c t i o n s will need to be m a d e . In a n y case it is r e c o m m e n d e d that alternatives such as the new t y p e s o f d e t a i l - p r e s e r v i n g filter [9] s h o u l d be c o n s i d e r e d .
We then find: = b - ( b / 2 ) [ 1 - a 2 / b 2] ,/2 - ( b 2 / 2 a ) sin ' ( a / b ) .
To p r o c e e d , we use the p o w e r series e x p a n s i o n o f the inverse sine function: sin -~ v = v + v 3 / 6 + 3 v S / 4 0 +
this research.
Appendix: Finding a formula for edge shift in a circular neighbourhood To find an a p p r o x i m a t e f o r m u l a for the edge shift in a c i r c u l a r n e i g h b o u r h o o d , we here a d o p t the strategy o f finding the m e a n lateral d i s p l a c e m e n t o f the edge within the n e i g h b o u r h o o d . The lateral d i s p l a c e m e n t x o f the edge o f the circle is given by the e q u a t i o n :
which l e a d s to y2) '/2
(A.2)
A s s u m i n g that a ~ b, the m e a n value o f x over the n e i g h b o u r h o o d is:
)
x dy
/f;
t
dy
[b - (b 2 - y 2 ) ,/2] d y / a .
(A.3)
To p e r f o r m the i n t e g r a t i o n , substitute y = b sin u.
(A.7)
= a 2 / 6 b + a 4 / 4 0 b 3.
E q u a t i n g this to the o b s e r v e d edge shift (A.8)
d ~- b - a : / 6 b - a 4 / 4 0 b 3,
(A.9)
as q u o t e d in Section 2. It is interesting that this f o r m u l a is essentially exact o v e r the entire range a ~ b (see T a b l e 1). In view o f the a p p r o x i m a t e nature o f the a b o v e m o d e l , this degree o f a c c u r a c y must be s o m e w h a t fortuitous, t h o u g h it can be put to g o o d use in m a k i n g s i m p l e c o r r e c t i o n s to m e a s u r e m e n t s m a d e from images that have been p r o c e s s e d b y m e d i a n filters.
References
(A.1)
(X -- b ) 2 + y2 = b 2,
~'~- I(a)
(A.6)
we finally o b t a i n :
The a u t h o r is grateful to the A C M E D i r e c t o r a t e o f the U.K. Science a n d E n g i n e e r i n g R e s e a r c h C o u n c i l for financial s u p p o r t d u r i n g the course o f
x= b-(b2-
... ,
whence
E = b - d,
Acknowledgement
(A.5)
(A.4)
[1] A.C. Bovik, T.S. Huang and D.C. Munson, "The effect of median filtering on edge estimation and detection", IEEE Trans. Patt. Anal. Mach. IntelL, Vol. PAM[-9, No. 2, 1987, pp. 181-194. [2] E.R. Davies, "The median filter: an appraisal and a new truncated version", Proc. 7th Int. Conf. on Pattern Recogn., Montreal, 30 July-2 August 1984, pp. 590-592. [3] E.R. Davies, "Circularity--a new principle underlying the design of accurate edge orientation operators", Image and Vision Computing, Vol. 2, No. 3, 1984, pp. 134-142. [4] E.R. Davies, "'Radial histograms as an aid in the inspection of circular objects", IEE Proc 19, Vol. 132, No. 4, Special Issue on Robotics, 1985, pp. 158-163. [5] J.P. Fitch, E.J. Coyle and N.C. Gallagher, "'Root properties and convergence rates of median filters", IEEE Trans. Acoust., Speech, Signal Process., Vol. ASSP-33, No. 1, 1985, pp. 230-239. Vol. 16. No. 2. FebruaD 1989
96
E.R. Davies / Edge location shifts in median filtering
[6] N.C. Gallagher and G.L. Wise, "'A theoretical analysis of the properties of median filters", IEEE Trans. Acoust., Speech, Signal Process., Vol. ASSP-29, No. 6, 1981, pp. 1136-1141. [7] P. Heinonen and Y. Neuvo, "FIR-median hybrid filters", IEEE Trans. Aeoust., Speech, Signal Process., Vol. ASSP35, No. 6, 1987, pp. 832-838. [8] R.M. Hodgson, D.G. Bailey, M.J. Naylor, A.L.M. Ng and S.J. McNeil, "'Properties, implementations, and applications of rank filters", Image and Vision Computing, Vol. 3, No. 1, 1985, pp. 4-14.
Signal Processing
[9] A. Nieminen, P. Heinonen and Y. Neuvo, "A new class of detail-preserving filters for image processing", IEEE Trans. Part. Anal. Math. lntelL, Vol. PAMI-9, No. 1, 1987, pp. 74-90. [ 10] G.J. Yang and T.S. Huang, "The effect of median filtering on edge location estimation", Comput. Graph. Image Process., Vol. 15, 1981, pp. 224-245.