COMPUTER
VISION,
GRAPHICS,
AND
IMAGE
PROCESSING
49,
85-94 (1990)
NOTE A Morphological Transformation for Sharpening Edges of Features before Segmentation C. J. MORAN CSIRO
Division
of Soils, P.O. Box 669, Canberra 2601 and School of Sydney, N.S. W. 2006, Australia
of Crop Sciences,
University
Received May 3,1988; revised March 1,1989 A gradient-determined grey level morphological opening procedure for edge enhancement is presented. First, the maximum gradient, in the local neighborhood, forms the contribution to the erosion of the centre pixel of that neighborhood. The gradients of the transformed image are then used as contributions to the subsequent dilation of the eroded image. The enhancement procedure is illustrated on sample images representing multiple phases of soil pore structure. Histogram partitioning is applied to the sample images before and after edge enhancement. The resulting segmentation is visually better on the images after enhancement. Stereological measurements made on the segmented images show that the quantity of each phase is preserved but the degree of contact between phases is altered. @1990 Academic press, Inc.
1. INTRODUCTION
The segmentation of multiphase digital images is hindered by variation in brightness across individual features within the image. When feature brightness is uniform the boundaries between features will be clear and separation of the phases is simply a matter of detecting the appropriate threshold for each phase present. However, if a gradual change in brightness from one phase to another occurs, segmentation is more difficult. Intermediate brightness features may lie within the grey level range spanning the boundaries between the brightest and darkest phases. Thus when boundaries are fuzzy, grey level histogram partitioning results in regions of intermediate grey level adjacent to bright features in the segmented image. Bergholm [l] has described mathematical methods for edge focussing after blurring, to remove noise, at varying scales of spatial resolution. The images dealt within this work are not noisy, but are fuzzy at the phase boundaries because of limited spatial resolution and fluorescent flaring. The degree of fuzziness is dependent on feature size and phase contact within any region of an image. This implies that a local edge detection technique would be appropriate for correct pixel allocation at phase boundaries. The local neighborhood gradient has been widely used in template matching methods for edge detection [2-51. The residual of morphological erosions and dilations have also been proposed as local neighborhood edge detectors [6]. Blurring methods have been used to reduce the noise sensitivity of these operators [7]. Recently [8], a history of edge enhancement prior to actual edge detection has been summarized and a nonlinear laplacian approach proposed for enhancement of edges in noisy images. This paper describes, and gives an algorithm for, a morphological function [9] that sharpens the boundary between features. It is derived using the concept of a morphological opening (erosion 85 0734-189X/90
$3.00
Copyri.&t 0 1990 by Academic Press, Inc. All rights of reprcduction in any form resewed.
86
C. J. MORAN
followed by dilation) from mathematical morphology (e.g., [lo, 111) combined with estimation of the local neighborhood gradient. Edge sharpening is based on the premise that the brightness gradient between phases is manifest in the local 3 x 3 neighborhood. In the first instance the grey level of a pixel is decreased by an amount that is derived from the magnitude of the gradient. As this is an erosion operation, a second pass is made across the image that increases the pixel value ensuring the feature size is maintained after segmentation. The main thrust in this paper is the use of the operator for image segmentation but its potential for edge extraction is also briefly explored. This morphological function is illustrated by showing the procedure for an image representing a vertical planar face of soil macropore structure in which pores were filled with two different colored resins. The use of edge sharpening is shown for both feature segmentation and edge extraction. 2. THEORY
AND
When designing the edge sharpening account the nature of the edges. Many considered such as linear ramp and spike such approaches were found inadequate
FIG. 1. Perspective plot of a 170 x 170 pixel subjecting the grey level image to the edge sharpening
PROCEDURE
operator it was necessary to take into one-dimensional edge models have been [12], step [13], and pulse and staircase [14]; here. Figure la is a perspective plot of a
image before operator.
edge
sharpening
(a)
and
(b)
after
A MORPHOLOGICAL
TRANSFORMATION
87
170 x 170 square pixel window from a typical image dealt with in this work. Hills and plateaux represent the bright regions and dark areas appear as the valleys. This figure shows that in the images considered here, the edges are two-dimensional concave or convex surfaces. The rate of change of grey level, the gradient, changes as one moves across an edge either away from or toward a bright region. Also, the gradient is not necessarily constant along an edge. The edge sharpening operator makes use of both the direction and magnitude of the gradient and is successful because of the shape of the edge surfaces. Gradients were defined in the horizontal and vertical directions using the means of grey levels of the local 3 X 3 neighborhood. Six means were computed for each of three pixels, viz., 1. the neighbors to the left, 2. down the middle, 3. to the right, 4. above, 5. across the middle and 6. below. A horizontal gradient exists if the mean across the middle falls between those above and those below. A vertical gradient exists if the mean down the middle falls between those to the left and those to the right. If a pixel, P, is on a gradient it is assumed to be on an edge. If it is on a gradient in both directions, the steeper of the two gradients is used. Edge sharpening is a two pass operation. On the llrst pass the value of a pixel on an edge is decreased by the difference in grey level between the two appropriate means. If it does not lie on an edge no action is taken. The second pass is performed on the image resulting from the first pass. The principle is the same except that the difference between the neighboring means is added to the pixel in the center. The second pass restores the original size of the features in the image. The image was framed by duplicating the outermost pixels around the boundary. The edge sharpening operation was performed inside this frame. The effect of the edge sharpening can be seen in Fig. lb, where the surface relief is rather more abrupt than that in Fig. la. Both Figs. la and b are views from the same perspective. An algorithm for the operator is presented in the Appendix. 3. EXAMPLE
3.1. Feature Segmentation The problem of separating features with the same grey level arose when dealing with the processing of images for the examination of soil pore structure. Undisturbed specimens of soil were impregnated with resin containing an ultraviolet fluorescent dye. A bright dye was used to infiltrate into in situ soil and a less bright dye used to complete impregnation in the laboratory thus, the solid/pore structure is treated as three-phase. The two resins were used to get information about the pore structure that was actually involved in the infiltration process. Video images were then digitized at 512 X 512 square pixel resolution directly from the surface of the blocks housed in an ultraviolet light box. Finally, stereological pore structure attributes describing the distribution, quantity, and relationship between each of the three phases [15] were generated. The first step was to segment the grey level image to generate a three-phase, or ternary, image from which structure attributes could be generated. The edge sharpening operator was developed to aid this segmentation procedure. The edge of bright field pores in contact with the solid phase overlapped the grey level range of the darker laboratory pores. Consequently, when the grey level histogram was tripartitioned bright pores were surrounded by some pixels represent-
C. J. MORAN
FIG. 2. (a) Grey level image of a vertical planar face representing soil pore structure. Black represents pore space filled with resin from field infiltration, grey is resin from laboratory completion of the impregnation, and white represents the soil solid material; (b) its ternary representation after segmentation.
ing the darker pores. Figure 2a is the original grey-level image and Fig. 2b the tripartitioned ternary image, where black represents the bright pores, grey the dull pores, and white the solid. The edge sharpening operator was then applied to the image shown in Fig. 2a. The result is shown in Fig. 3a which was then tripartitioned at the same thresholds as for Fig. 2b. This is shown in Fig. 3b. Figure 2b shows less detail of the smaller features. This is most evident in the large white area to the left
A MORPHOLOGICAL
TRANSFORMATION
89
FIG. 3. Grey level image shown in Fig. 2a: (a) after edge sharpening and (b) its ternary representation after segmentation.
of region A. If the threshold was lowered to detect this detail the outlining of black regions by grey was found to increase. This trade-off between incorrect pixel allocation and detail is not such a problem after edge sharpening. The improved extraction of detail is possible because the contrast between the light background, soil solid, and the small pore features is increased. This would contribute to the increase in surface area after sharpening and partitioning that is shown in Fig. 4b. It follows that segmentation, where the result is a binary image, may also be improved,
90
C. J. MORAN
z0
I 10
20
30
40
50
Depth /mm
FIG. 4. Pore structure attribute data shown as functions of depth. The bold function in each plot shows data before sharpening and the other after sharpening: (a) laboratory (prey) pore/solid surface area; (b) field (black) pore/solid surface area (mm2 mm-3); and (c) total porosity (m& mme3).
in terms of extraction of details, if the image is subjected to the sharpening transformation before other segmentation transformation(s) are applied. When black features (pores filled during field infiltration) are completely surrounded by grey pixels there is obviously a problem because resin cannot infiltrate a pore without touching the sides. Region A in Fig. 2b shows two black features that are surrounded by grey. The same region segmented in Fig. 3b appears more like that perceived in Figs. 2a and 3a. Some grey pixels around black features can still be seen in Fig. 3b. This is acceptable, however, because pore structure may be convoluted; also resin drainage may occur. Thus, some pores may not be completely filled by the field infiltration but are impregnated in the laboratory. Comparison of the same figures for region B indicates improved feature allocation, to either black or grey, in Fig. 3b compared to Fig. 2b. The allocation of pixels to either blsk or grey is nut perfect even after sharpening but these figures indicate an improvement. Here, the aim of image segmentation was to allow the generation of info& about the structure depicted in the image. Figure 4 is presented to illustrate the diBerence between data generated from Figs. 2b and 3b. Data are generated in a
A MORPHOLOGICAL
TRANSFORMATION
91
FIG. 5. Phase: boundaries extracted by segmenting the gradient functicJn of (a) the origin al image shown in Fig. 2a, (b) the sharpened image shown in Fig. 3a and (c) from the bened gradient fuunction of the sharpened image in Fig. 3a.
92
C. J. MORAN
depthwise fashion as the images represent a vertical section through the structure. The structure is assumed to be random isotropic in the horizontal plane and heterogeneous in the vertical. A value is generated for each horizontal line across the image. Even though the images do not appear very different, Fig. 4 shows that the structural attributes are sensitive to the difference. The surface area density or surficial contact between black and white phases (Fig. 4b) is greater at all depths in the ternary image resulting from edge sharpening. Figure 4a also shows that the surface area between white and grey phases is disimilar with and without edge sharpening. It is interesting to note the difference that these segmentation procedures have on the total quantity of pores and solid material. Figure 4c shows that the total porosity (black + grey) is similar with and without edge sharpening. This indicates that edge sharpening does not alter the position of the phase boundaries. 3.2. Edge Detection
There has been much work on detection of edges in digital images. In the case of multiphase images it may be important to segment the edges into phase classes. The edge sharpener may be a useful initial step to avoid the halo effect around bright edges. In the example given, the modulus of the gradient function given in [lo] is used as the edge detector. This is not necessarily advocated over other detection operators but it is effective and reasonably quick. An attempt to extract the phase boundaries of the original image, shown in Fig. 2, using the modulus of the gradient function, without sharpening, is shown in Fig. 5a. If the edges in the gradient image are first sharpened this extraction is as shown in Fig. 5b. There is a marked decrease in the amount of grey pixels at either side of black edges in this figure. The result can be taken a step further by applying edge sharpening to the gradient function of the previously sharpened image in Fig. 2b. Figure 5c shows the extracted edges partitioned after double sharpening. 4. CONCLUSIONS
The edge sharpening operator illustrates that it can be useful to consider edges as two-dimensional surfaces. This allows the combination of gradient direction and magnitude information when transforming digital images. It has been illustrated that morphological concepts can be mixed with what may be considered more conventional neighborhood transformations. Edge sharpening before histogram partitioning is useful for extraction of phase regions, whether two-phase or multiphase, and before detection of the edges. It has been shown that some structural attributes generated from the segmented images are sensitive to the edge sharpening, but others such as the total quantity of pore and solid were not effected. APPENDIX Begin I:/* Edge Decrease define neighbors: NW w SW for each pixel, P,
*/ N P s
NE E SE
A MORPHOLOGICAL
TRANSFORMATION
93
suml=NW+N+NE sum2= W+P+ E sum3=SW+S+SE suml= NW+ W+SW sum5 = N + P + S sum6 = NE + E + SE if(sumI < sum2 < sum3) difserence = sum3 - sum1 else if(suml > sum2 > sum3) difference = sum1 - sum3 if(sum4 C sum5 < sum6) temp = sum6 - sum4 else if(sum4 > sum5 > sum6) temp = sum4 - sum6 ij(temp > difference) dtflerence = temp pixel _ out = P - (dtzerence / 3) #pixel out < 0) pixel-out = 0 i:/* Edge Increase * / Repeat I on image resulting from 1. Except: pixel out = P + deference if (pixel _ out > 255) pixel-out = 255 End
ACKNOWLEDGMENTS The author was supported by a CSIRO Division of Soils postgraduate studentship. Dr. M. E. Probert, CSIRO Division of Soils is thanked for helpful discussion and review of the typescript. I should like to thank a referee for providing useful constructive criticism.
REFERENCES 1. F. Bergholm, Edge focusing, IEEE Trans. Pattern Anal. Mach. Intell. 9, 1987, 726-741. 2. J. M. S. Prewitt, Object enhancement and extraction, in Picture Processing and Psychopictorics (B. S. Lipkin and A. Rosenfeld, Eds), Academic Press, New York, 1970. 3. W. K. Kirsch, Computer determination of the constituent structure of biological images, Comput. Biomed. Res. 4, 1971, 315-328. 4. G. S. Robinson, Edge detection by compass gradient masks, Comput. Vision Graphics Image Process. 6, 1977,492-501. 5. J. Fiiglein, On edge gradient approximations, Pattern Recognit. Lett. 1, 1983, 429-434. 6. V. Goetcherian, From binary to grey tone image processing using fuzzy logic concepts, Pattern Recognit. 12, 1980, 7-15. 7. J. S. J. Lee, R. M. Haralick, and L. G. Shapiro, Morphologic edge detection, IEEE J. Rob. Autom. R&3,1987,142-156. 8. L. J. Van Vliet, I. T. Young, and G. L. Beckers, A non-linear laplace operator as edge detector in noisy images, Comput. Vision Graphics Image Process. 45, 1989, 167-195. 9. R. Nawrath and J. Serra, Quantitative image analysis: Applications using sequential transformations, Microsc. Acta 82, 1979, 113-128. 10. J. Serra, Image Analysis and Mathematical Morphology, Academic Press, London, 1982. 11. S. R. Stemberg, Greyscale morphology, Comput. Vision Graphics Image Process. 35, 1986, 333-355.
94
C. J. MORAN
12. L. S. Davis, A survey of edge detection techniques, Comput.
Gruphics
Image
Process.
4,
1975.
248-270.
13. B. J. Schachter and A. Rosenfeld, Some new methods of detecting step edges in digital pictures, Comm. ACM, Graphics and Image Process. 28, 1978, 172-176. 14. M. Shah, S. Sood, and R. Jain, Pulse and staircase edge models, Comput. Vision Graphics Imuge Process. 34, 1986, 321-343. 15. C. J. Moran, A. J. Koppi, B. W. Murphy, and A. B. McBratney, Comparison of the macropore structure of a sandy loam surface soil horizon subjected to two tillage treatments, Soil Use Manag. 4,1988, 96-101.