- Email: [email protected]
Nucleus and cytoplast contour detector of cervical smear image
Pattern Recognition Letters 29 (2008) 1441–1453
Contents lists available at ScienceDirect
Pattern Recognition Letters journal homepage: www.elsevier.com/locate/patrec
Nucleus and cytoplast contour detector of cervical smear image Meng-Husiun Tsai a, Yung-Kuan Chan a,*, Zhe-Zheng Lin a, Shys-Fan Yang-Mao b, Po-Chi Huang c a b c
Department of Management Information Systems, National Chung Hsing University, 250, Kuokuang Road, Taichung 402, Taiwan Department of Management Information Systems, Central Taiwan University of Science and Technology, No.11, Buzih Lane, Beitun District, Taichung 40601, Taiwan Department of Pathology, Taichung Hospital, Department of Health, Executive Yuan, Taiwan
a r t i c l e
i n f o
Article history: Received 14 March 2007 Received in revised form 27 November 2007 Available online 18 March 2008 Communicated by W. Zhao Keywords: Cervical smear screening Cervical cancer Image segmentation Salt and pepper noise Gaussian noise Contour detection
a b s t r a c t This paper develops a cytoplast and nucleus contour (CNC) detector to sever the nucleus and cytoplast from a cervical smear image. This paper proposes the bi-group enhancer to make a clear-cut separation for the pixels laid between two objects, and the maximal color difference (MCD) method to draw the aptest nucleus contour. The CNC detector adopts a median filter to sweep off noises, the bi-group enhancer to suppress the noises and brighten the object contours, the K-mean algorithm to discern the cytoplast from the background, and the MCD method to extract the nucleus contour. The experimental results show that the CNC detector can give an impressive performance. Besides cervical smear images, these proposed techniques can be utilized in segmenting objects from other images. Ó 2008 Elsevier B.V. All rights reserved.
1. Introduction An estimated 11,150 cases of invasive cervical cancer are expected to be diagnosed in the United States in 2007 and 3670 women are expected to die from the disease. Incidence and mortality rates have decreased steadily over the past five decades, largely due to the widespread use of the Pap smear which detects cervical cancer and precancerous lesions. The Pap smear has made cervical cancer one of the most preventable cancers, but older, poorer, and less educated women are less likely to be screened and screening is not available in many low-resource regions of the world. Worldwide, cervical cancer has a significant impact, with nearly 500,000 new cases and nearly 250,000 deaths reported annually (Women’s Health Report, 2007). Unlike other cancers that cause pain, noticeable lumps, or other early symptoms, cervical cancer has no telltale symptom until it is so advanced that it is usually unresponsive to treatment (Frable, 1982). Only in its latest stage, cervical cancer causes pain in the lower abdominal or back regions. However, most cervical cancer takes many years to develop from normal to dangerous stages. Cervical cancer is a preventable disease and, unlike most cancers, can be easily detected by a routine screening test. Currently, cervical smear screening is the most popular method to detect the presence of abnormal cells arising from the cervix. With a small brush, cotton stick or wooden stick, a specimen is taken from the uterine cer* Corresponding author. Tel.: +886 4 22840422; fax: +886 4 22857173. E-mail address: [email protected] (Y.-K. Chan). 0167-8655/$ - see front matter Ó 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2008.02.024
vix, smeared onto a thin, rectangular glass plate (a slide), and dyed, making it easier to examine the cells under a microscope. The purpose of smear screening is to diagnose pre-malignant cell changes before they become cancerous. Dysplastic cells have undergone precancerous changes. They generally have longer as well as darker nuclei, and a tendency to cling together in large clusters. Mildly dysplastic cells have enlarged and bright nuclei. Moderately dysplastic cells have larger and darker nuclei. The nuclei may start to deteriorate and to be granulation. Severe dysplastic cells have large, dark, and often oddly shaped nuclei; its cytoplast is relatively dark and small (Martin, 2003). Hence, pre-cancers and cancers are associated with a variety of morphologic and architectural alterations, including the textures, sizes, and shapes of cytoplast and nucleus, hyperchromasia and pleomorphism. It also increases nuclear–cytoplastic ratio. Fig. 1 shows the superficial squamous cells stained to enhance the contrast. Current manual screening methods are costly and sometimes result in inaccurate diagnosis caused by human error. The introduction of machine assisted screening will bring significant benefits to the community, which can reduce financial costs and increase screening accuracy. An effective boundary detection algorithm locating the contours of the cytoplast and nucleus plays an important role in developing a useful computer-assisted diagnostic system. Wu et al. (1998) introduced a parametric optimal segmentation approach which is suitable for the images of non-overlapping cells with smooth cell boundaries or contours. However, dealing with
1442
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
B
A
Fig. 1. Superficial squamous cells stained to enhance contrast.
the segmentation of a cell image, a priori knowledge of the nuclear characteristics should be fully used; these characteristics include the cell shape, size, and intensities relative to its background. Mat-Isa et al. (2005) utilize the potential use of thresholding the region growing algorithm as a feature extraction technique. The proposed algorithm is called seeded region growing features extraction (SRGFE). The SRGFE is used to extract the size and grey level of a certain region of interest on a digital image. In the SRGFE algorithm, the user needs to determine the region of interest by clicking the mouse on any pixels in the region and to specify the threshold value, which makes the system impractical. Walker (1997) uses a series of automated fast morphological transforms with octagonal structuring elements. Each gray-scale cell image is first globally thresholded, resulting in an incomplete segmentation of the nucleus in binary form. The cytoplastic background is removed by performing a closing of the image using a structured element which is smaller than the smallest nucleus, and the nuclear heterogeneity is corrected by an opening of a similar size. However, it is more appropriate for local thresholding and it cannot be fully automated. Many other cytoplast and nucleus morphological segmentation methods have also been proposed in the related literatures (Busam et al., 2001; Collier et al., 2002; Corcuff et al., 1996; Inoue et al., 2000; Langley et al., 2001; Masters et al., 1997; Mat-Isa et al., 2005; Rajadhyaksha et al., 1999; Rajadhyaksha et al., 1995; Walker, 1997; Wu et al., 1998). However, the results are based on tedious hand-segmentation of images. Martin (2003) and Norup (2005) also take the CHAMP software to segment and classify cervical smear images. Unfortunately, the CHAMP software cannot provide a satisfying segmentation performance, especially for abnormal cervical cells. The aim of this paper is to develop an automated image segmentation system to sever the cytoplast and nucleus from a cervical smear image, without a priori knowledge of the image objects. Generally, the accuracy of an object contour detector depends on the quality of an image. The heavily stained cervical smear may be masked by menstrual blood, vaginal discharge, air artifacts, etc., thus obscuring the abnormal cervical cells. Sometimes, overexposing or underexposing under the microscope light may also blur the cervical smear images. These problems may cause difficulties in extracting the cytoplast and nucleus of a cervical cell. This paper proposes a bi-group enhancer to eliminate the noise on an image and to sharpen the contours of objects before extracting the object. Since the cytoplast and background on a cervical smear image can apparently be distinguished by their colors, this paper will employ K-mean algorithm to discern between the cytoplast and background on the image. Mostly, two different adjacent objects have dissimilar color distributions. Consider an image with only two different objects, as shown in the image in Fig. 2 for example. In this image, the red1 circle indicates the boundary which separates object A from object B where the color difference of the pixels on the inside and outside of the circle is maximal. Based on the previously mentioned properties, this paper proposes a cytoplast 1 For interpretation of color in Fig. 2, the reader is referred to the web version of this article.
Fig. 2. An image with only two objects.
and nucleus contour (CNC) detector to perceive the cytoplast and nucleus contours of a cervical cell. Besides cervical smear images, these techniques can be employed to segment the objects of other images too. Tsai et al. (2007) briefly introduces the concept of the detector. This paper will describe it in more details and give more experiments and discussions. 2. The CNC detector Edge detection is considered fundamental in image processing and computer vision, with a large number of studies having been published in the last two decades, i.e., the Canny edge detector (Canny, 1986), the Sobel edge detector (Davies, 1990), the Prewitt edge detector (Davies, 1990), and the Roberts Cross edge detector (Davies, 1990). Segmentation is the process of dividing an image into its constituent parts for further analysis. We commonly refer to such parts as regions of interest (ROI). Edge detection based segmentation is a frequently used technique which segments an image on the basis of dissimilarity or heterogeneity within image pixels or regions. Fig. 3 shows the edges (marked by red2 lines) obtained by the above mentioned detectors. Fig. 3 illustrates that the above mentioned methods are only suitable to detecting the objects whose contours are obvious. The Canny edge detector can provide a better performance, but it is susceptible to noise. The active contour model (ACM) (Xu and Prince, 1998) is adapted to a wide range of extracting the objects with vague, complex and/or irregular shape boundary, inhomogeneous and noisy interior, as well as contour with small gaps. However, Fig. 4 states that ACM cannot give a good segmentation when the quality of the image is poor. It is also difficult to decide the weighted parameters of the snake’s tension and rigidity, since the contour of cytoplast is varied. In addition, ACM is very sensitive to the initial contour. Most image segmentation methods perform well while the image has good quality and the object contours are distinct. However, cervical smear images are frequently contaminated and the cytoplast and nucleus contours of cervical cells are often vague, especially for abnormal cervical cells. This paper, hence, adopts a bi-group enhancer to intensify the contours of objects in an image. This paper also presents a maximal color difference (MCD) method to segment one object from others based on their color differences. The CNC detector contains three approaches: bi-group, cytoplast contour detection, and nucleus contour detection approaches. The bi-group approach is to suppress the noises and emphasize the edge pixels; the cytoplast contour detection approach utilizes
2 For interpretation of color in Fig. 3, the reader is referred to the web version of this article.
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
1443
Fig. 3. The edges obtained by Canny, Sobel, Prewitt and Roberts cross-edge detectors.
Fig. 4. The contours obtained by ACM.
a K-means algorithm to separate the cytoplast from the background; and the nucleus contour detection approach adopts the MCD method to draw the contour of nucleus. This section will introduce the three approaches in great detail. 2.1. Bi-group approach Image segmentation task directly depends on the quality of the image. The generation of an accurate edge map becomes a very crucial issue when the images are corrupted by noises. There have been several denoising techniques presented in the past studies, such as, the mean filter (Gonzalez and Woods, 2002), the median filter (Gonzalez and Woods, 2002), the Gaussian filter (Gonzalez and Woods, 2002), and the type-B filter (Russo and Lazzari, 2005). Fig. 5a is the image Lena created by adding some impulse noises, and Fig. 6a is a cervical image with some Gaussian noises. Figs. 5 and 6 demonstrate that only the median filter can eliminate both impulse noises and Gaussian noises. Therefore, this paper adopts the median filter to discard impulse and Gaussian noises. The term ‘‘edge” stands for a local luminance change of sufficient strength which is considered important in a given task. Many contextual edge detection techniques based on suppression have been previously proposed. Russo and Lazzari (2005) provide a type-A filter to enhance the contours of objects. It takes into account the differences between the pixel to be processed and its neighbors as follows: small differences are considered to be the
gradients of noises which should be reduced; large differences are considered to be those of edge points which should be preserved. A two-step procedure is applied to the image channels in order to increase the effectiveness of the smoothing action. Hence, type-A pixels are those corrupted by noise with amplitude which is not too different from the neighbors. Yin et al. (2004) also present an automatically adaptive window-level selection algorithm (so-called adaptive image optimization (AIO)) for achieving on improved performance. In this algorithm, first, region of interest (ROI) is extracted by using the variance change and the integral projection; second, the image statistical values, such as, maximum, minimum, and average, can be obtained in the detected ROI; finally, the window and level can be determined from the image statistical values, and a contrast transfer function can be obtained by using the cubic spline interpolation. Via AIO, the quality of an image can be improved by increasing the maximum dynamic range adaptively. Figs. 7 and 8 illustrate the images processed by the type-A prefiltering and the AIO algorithm. However, in either of these methods does suppression have an effect on nearby peripheral pixels which are definitely considered to be the background, cytoplasts, or nucleus. This paper therefore proposes bi-group enhancer to effectively isolate the object pixels from other object pixels. Let pi,j be the pixel located at the coordinates (i, j) on a cervical smear image f0, and Wi,j be the corresponding window of pi,j which is the central pixel of Wi,j consisting of the m m pixels
1444
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
Fig. 5. (a) Noisy image Lena; the images obtained by (b) mean filter, (c) Gaussian filter, (d) type-B filter and (e) median filter.
Fig. 6. (a) A cervical image; the images obtained by (b) mean filter, (c) Gaussian filter, (d) type-B filter and (e) median filter.
Fig. 7. (a) Another original cell image; (b)–(d) the cell images processed by AIO, type-A filter and bi-group enhancers.
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
1445
Fig. 8. (a) Another original cell image; (b)–(d) the cell images processed by AIO, type-A filter and bi-group enhancers.
pibm1c;jbm1c , pibm1c;jbm1cþ1 ; . . . ; pibm1c;j1 , pibm1c;j , pibm1c;j 2 2 2 2 2 2 2 þ1; . . . ; pibm1c;jþbm1c , pibm1cþ1;jbm1c , pibm1cþ1;jbm1cþ1 ; . . . ; 2
2
2
2
2
2
pibm1cþ1;j1 , pibm1cþ1;j , pibm1cþ1;jþ1 ; . . . ; pibm1cþ1;jþbm1c ; . . . ; piþ 2 2 2 2 m12 ; j m1 , piþbm1c;jbm1cþ1 ; . . ., piþbm1c;j1 , piþbm1c;j , piþbm1c;jþ 2 2 2
2
2
2
2
1; . . . ; piþbm1c;jþbm1c . 2
2
After denoising, fo becomes an image ft. Let pi,j be the pixel located at the coordinates (i, j) in ft, and Wi,j be the corresponding window of pi,j where pi,j is the central pixel of Wi,j consisting of m m pixels. Assume that C i;j ¼ fc1 ; c2 ; . . . ; cm2 g are the colors of the pixels on Wi,j, and the color values in Ci,j are sorted increasingly, that is, c1 6 c2 6 6 cm2 . Pm2 þ1 Bi-group enhancer defines the interval between m22þ1 i¼12 ci and cm2 þ1 as well as the interval between cm2 þ1þ1 and m221 2 2 Pm2 i¼ðm2 þ3Þ=2 ci as indefinite intervals since it is difficult to recognize whether pi,j is in an object or in background while the color c of 2
pi,j lies in both intervals. Set mid ¼ m 2þ1 with m considered to be an odd number, then the two intervals are defined as [c1, cmid] and ½cmidþ1 ; cm2 . Hence, the bi-group approach replaces c with c0 , where c0 is defined as follows: 8 m id m id P P > 1 1 > > ci ; if mid ci 6 c 6 cmid ; > mid > > i¼1 i¼1 < m2 m2 c0 ¼ P P 1 1 > ci ; if cmid < c 6 mid1 ci ; > mid1 > > i¼mid1 i¼midþ1 > > : c; otherwise: If c is in the indefinite intervals, the bi-group enhancer changes c into the average color cf of the first half of Ci,j’s when c is closer to cf, or c is supplanted by the average color of the later half of Ci,j’s. Figs. 7 and 8 illustrate that the bi-group enhancer can more effectively separate the pixels of an object from the pixels of other object. In the real world, there are many images with vague objects, such as the images shown in Figs. 7a and 8a. Besides, denoising operations often contain severe blurring. The purpose of a bi-group enhancer is to discriminate the pixels on one object from those on other objects, which are near to the edges of the objects. Fig. 9 shows the color histograms of four cervical smear images after processed by AIO, type-A filter, and bi-group enhancers. Obviously, one can observe that the color histograms of the images obtained by the bi-group enhancer are more distinct separated into three clusters than other two enhancers. 2.2. Cytoplast contour detection approach The region of nucleus is generally much smaller than those of a cytoplast and the background on a cervical smear image. Fig. 10a is a cervical smear image; Fig. 10b shows its color histogram. In Fig. 10b, the color bins which are close to the two highest peaks are the color distributions of the cytoplast and the background;
the color distribution of the nucleus is on the left of the color bins. The color distributions of the cytoplast and the background on a cervical smear image can apparently be distinguished from the color histogram of the image. The CNC detector hence takes the Kmean algorithm (Su and Chou, 2001) to discern between the pixels on a cytoplast and those on a background. Assume that the pixel colors of a cervical smear image are from n0 to n1, where 0 5 n0, n1 5 255. Since there are only a small quantity of nucleus pixels on a cervical smear image and the pixel colors of a nucleus are darker. In order to prevent the clustering via Kmean algorithm from being influenced by the pixel colors of a nucleus, the CNC detector divides the pixels with colors from Ci + e to Cn into two groups, where e is a given constant and Ci as well as Cn is the minimal and maximal pixel colors. It then considers the pixels in the group with a higher pixel color to be the background pixels, and the others to be the cytoplast pixels. The following steps explain how to use the K-mean algorithm to partition the pixels with colors between Ci + e and Cn into two groups: Step 1: Randomly select 2 different values from the interval between Ci + e and Cn to be the respective values of the two groups. Step 2: Categorize each pixel of the image with color within Ci + e and Cn into one of the two groups according to its distance compared to the representative value of each group. The representative value of each group is then substituted for the average color of the pixels in the group. After that, Step 2 is repeated until each group is unchanged or the iteration count is greater than a given constant. 2.3. Nucleus contour detection approach The regions of most nuclei are quite small and they have high variations in color intensity. Thus, K-mean algorithm cannot precisely discriminate them from those of cytoplasts. The CNC detector hence proposes a maximal color difference (MCD) method to detect the nucleus contour. The MCD method can be used to discern an object B from another object A. In this method, an initial contour S is first specified to partition an image into two regions inside and outside S. When given an initial contour S, the MCD method repeatedly moves the pixels on S inward or outward according to the color difference D of the regions inside and outside S until D is maximal. Here, we call D the color difference of S. As a result, S is the expected contour when D of S is maximal. Fig. 11 demonstrates three S represented by three red3 circles, respectively, located inside, on the boundary of, and outside object B, where A and B are two different objects and p is one pixel on S. 3 For interpretation of color in Fig. 11, the reader is referred to the web version of this article.
1446
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
Images
Bi-group
AIO
Fig. 9. The color histograms of four cervical smear images after processed by AIO, type-A filter and bi-group enhancers.
1447
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
Type-A
Fig. 9 (continued)
Fig. 10. A cervical smear image and it color.
A
A
p pO
(a) S is outside B
p
p
B pI
A
B pO
pI
(b) S is on the boundary of B Fig. 11. Three different location of S.
pO
pI
B
(c) S is inside B
1448
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
pI p pO Fig. 12. The related location of p, pI and pO.
The color difference of S depicted by a red circle in Fig. 11b is maximal among those delineated by red, blue and yellow circles. Apparently, S in Fig. 11b is the circle closest to the boundary of B. In Fig. 11a, the color difference of S marked by a yellow circle is maximal; in Fig. 11c, it is indicated by a blue circle which is maximal. Hence, the MCD method, respectively, moves the pixel p in Fig. 11a and c to pI and pO. Given an initial contour S, let nO and nI be the numbers of pixels outside and inside S; D, DO, and DI the color differences of S, SI, and SO; p a pixel on S; pO and pI the two pixels which are closest to p on the normal line of S at p; C, CO, and CI the colors of p, pO, and pI; SI (resp. SO) the same as S, only that p is moved to pI (resp. pO). Following the procedure mentioned previously, the MCD method continually moves each pixel p on S until the color difference of S is maximal; moves p to pI, if the color difference of SI is greater than that of SO; or else, move p to pO. Let L be the tangent line of S at p. Fig. 12 shows the related locations of p, pI and pO. When given an initial contour S, the MCD method repeatedly moves each pixel p on S, until the color difference of S is maximal by the following approaches: repeat I þC C O nO C I C C O nO þC DO ¼ CðnI nI þ1Þ ðnO 1Þ ; DI ¼ CðnI nI 1Þ ðnO þ1Þ , if DO = max(D, DO, DI) then nO ¼ nO 1
Bi-group
(a) The average gradients on the target edges of nuclei
Bi-group
(b) The average gradients on the target edges of cytoplasts Fig. 13. The average gradients of the target: (a) nucleus and (b) cytoplast contour pixels after processed by AIO, type-A filter and bi-group enhancers.
nI ¼ nI þ 1 D ¼ DO ; S ¼ SO move p to pO, else if DI = max(D, DO, DI) then nI ¼ nI 1; nO ¼ nO þ 1 D ¼ DI ; S ¼ SI move p to pI, try to move the next pixel on S until no pixel on S is moved The CNC detector moves each pixel on the extracted cytoplast contour, detected in cytoplast contour detection approach, t pixels inward to generate the initial contour S. It then uses the MCD method to extract the nucleus contour from the extracted cytoplast.
3. Experiments The purpose of this section is to investigate the performance of the CNC detector by experiments compared to those obtained by the GVF–ACM method (Xu and Prince, 1998) and CHAMP software (Martin, 2003; Norup, 2005). The experiments use 26 cervical smear images of 64 64 pixels, which were provided by Taichung Hospital in Taiwan, ROC, as the test data. Each of these images is transformed into a gray-level image. Fig. 14 displays these test images and their target cytoplast and nucleus contours which were manually drawn by an experienced doctor. The first experiment is to evaluate the usefulness of the bi-group approach compared to the AIO and type-A filter enhancers. A sharp edge generally has higher pixel gradients. Most of segmentation methods can give a better performance in cutting off the object with sharper contour. Fig. 13a (resp. Fig. 13b) describes the distribution of the average gradient of the target nucleus (resp. cytoplast) contour pixels of each of the 26 test images after processed by AIO, type-A filter, and bi-group enhancers where m is set to 5 for the bi-group enhancer. Fig. 13a and b tells that the bi-group enhancer can give more definite object contour than other two enhancers. The following experiments are to scrutinize the performance of the CNC detector. The next experiment first employs the CNC detector to extract the cytoplast and nucleus contours of the test images where m, e, and t are given to be 5, 30 and 5. This experiment then takes the GVF–ACM method (Xu and Prince, 1998) to sever the cytoplasts and nucleuses from the test images where all the parameters a, b, j are given to be 1. This experiment also adopts the CHAMP software (Martin, 2003; Norup, 2005) to separate the cytoplasts and nucleuses from the test images. Fig. 14 displays the cytoplast and nucleus contours cut by CNC detector, GVF– ACM method, and the CHAMP software. The performance criteria, misclassification error (ME), edge mismatch (EMM), relative foreground area error (RAE), and modified Hausdorff distance (MHD) (Sezgin and Sankur, 2004), are often used to put into evidence which shows the differing performance features of the segmentation methods. This experiment also evaluates the performances of the CHAMP software (Martin, 2003; Norup, 2005), the GVF–ACM method (Xu and Prince, 1998), and the CNC detector via ME, EMM, MHD and RAE. Table 1, respectively, lists their average measurements for the extracted cytoplast and nucleus contours on the 26 tested images cut by the CHAMP software, the GVF–ACM method, and the CNC detector; Fig. 14 shows the extracted cytoplast and nucleus contours. In this table, all ME, EMM, REA and MHD tell that the CNC detector can give a much better performance than the GVF–ACM method and the CHAMP software. Most of segmentation methods can give a better performance in cutting off the object with a quite different color from its adjacent object. Let AC1 and AC2 be the average pixel colors of object O1 and
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
1449
Fig. 14. The original images, the target cytoplast and nucleus contours of the test image, and the cytoplast and nucleus contours cut by CNC detector, GVF ACM method and CHAMP software.
O2, respectively, and CDiff = |AC1 AC2|. We call CDiff the color difference between O1 and O2. Fig. 15a (resp. Fig. 15b) describes the distribution of the color difference between the nucleus and cyto-
plast regions of each original image to the EMM (resp. MHD) of the nucleus contour on the image extracted by the CHAMP software (Martin, 2003; Norup, 2005), GVF–ACM method (Xu and Prince,
1450
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
Fig. 14 (continued)
1998), and CNC detector for all the 26 test images. Fig. 15a and b demonstrates that the CNC detector provides better segmentation than the CHAMP software and GVF–ACM method in severing the nucleus because it has higher color contrast to its adjacent objects; that is, the MCD method can perform well when the object highly contrasts chromatically with adjacent objects.
Fig. 15c (resp. Fig. 15d) depicts the distribution of the color difference between the background and cytoplast regions of each original image to the EMM (resp. MHD) of the cytoplast contour on the image extracted by the CHAMP software, GVF–ACM method, and CNC detector for all the 26 test images. Fig. 15c and d shows that the CNC detector is more indifferent to the color difference
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
1451
Fig. 14 (continued)
when isolating cytoplasts since the CNC detector uses the K-mean algorithm, which only considers the color similarity of pixels, to separate the background pixels and cytoplast pixels. More unevenness or more noise in an object’s texture generally makes it more difficult to segment that object. The standard deviation of the pixel colors of an object can depict the texture and noises on the object. Let Std1 be the standard deviation of the pixel colors of one object, Std2 be that of other object, and Std = (Std1 + Std2)/2. We call Std the average standard deviation of the two objects. Fig. 15e (resp. Fig. 15f) illustrates the relation between the EMM (resp. MHD) of each image and the average standard deviation of the cytoplast and nucleus regions on the im-
age for all the test images. Fig. 15g (resp. Fig. 15h) displays the relation between the EMM (resp. MHD) of each image and the average standard deviation of the background region and cytoplast region on this image. Fig. 15e and f illustrates that largely the CHAMP software, GVF– ACM method, and MCD method provide a worse performance in splitting the cytoplast when the standard deviation of the pixel colors on an object is larger since the pixel colors of the cytoplast and background are variable or many noises appear on the images. Fig. 15e and f also shows that mostly the CHAMP software and the MCD method is better than the GVF–ACM method in severing the nucleus from a cervical cell image; overall the MCD method
1452
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
the CHAMP software cannot give a good segmentation for the images in images 6, 7 and 15 in Fig. 14 since they contain many noises. The GVF–ACM method is susceptible to noises (i.e. the extracted cytoplasts of the images 9, 12 and 13 in Fig. 14). In addition, the GVF–ACM method takes the internal force to smooth the contour which may constrain it effectively to segment the object with irregular object contours (i.e. the extracted nuclei of images 8, 12, 13, 14 in Fig. 14). It is difficult appropriately to control the internal force following different object shapes. Hence, Table 1 and Fig. 14 indicate that the GVF–ACM method gives the worse performance than two others. From Table 1 and Fig. 14, on the whole, the CNC detector provides a better performance than two others on cytoplast segmentation. In nucleus segmentation, the MCD method and the
Table 1 The average of error measurements Method
Object
MHD
EMM
RAE
ME
CHAMP
Nucleus Cytoplast
0.2836 0.0841
0.2583 0.3850
0.2472 0.0900
0.0090 0.0413
GVF–ACM
Nucleus Cytoplast
0.3848 0.1960
0.4518 0.4219
0.2595 0.0891
0.0077 0.0465
CNC
Nucleus Cytoplast
0.1690 0.0617
0.2278 0.1914
0.2060 0.0325
0.0064 0.0244
and the CHAMP software can obtain a close segmentation effect in nucleus segmentation but the MCD method mostly gives a better performance than the CHAMP software when Std is greater than 21.5 since the CHAMP software is more sensitive to noises. Hence,
a
e
Nucleus (CDiff, EMM) GVF-ACM
Nucleus (Std, EMM) CHAMP
CNC
0.90
0.90
0.72
0.72
0.54
0.54
EMM
EMM
CHAMP
0.36
0.00 20
0.36
35
50
65
80
0.00 8.0
95
12.5
17.0
Nucleus (CDiff, MHD)
b
GVF-ACM
CNC
CHAMP
1. 75
GVF-ACM
CNC
1. 40
MHD
1.40
MHD
26.0
Nucleus (Std, MHD)
f
CHAMP
1.75
1.05
1. 05
0.70
0. 70
0.35
0. 35
0.00 20
35
50
65
80
0. 00 8.0
95
12.5
7.0
CDiff
c
21.5
26.0
Std
g
Cytoplast (CDiff, EMM) CHAMP
0.90
GVF-ACM
CNC
Cytoplast (Std, EMM)
0.72
0.72
0.54
0.54
0.36
CHAMP
0.90
EMM
EMM
21.5
Std
CDiff
0.18
GVF-ACM
CNC
0.36 0.18
25
35
45
55
65
75
0.00 3
85
6
9
d
Cytoplast (CDiff, MHD) CHAMP
2.5
12
15
18
Std
CDiff
Cytoplast (Std, MHD)
h GVF-ACM
CNC
CHAMP
2.5
2.0
GVF-ACM
CNC
2.0
MHD
MHD
CNC
0.18
0.18
0.00 15
GVF-ACM
1.5
1.5
1.0
1.0
0.5
0.5
0.0 15
29
43
57
CDiff
71
85
0.0 3
6
9
12
15
18
Std
Fig. 15. The relations of the EMM and MHD of the extracted contours to the color differences and the average standard deviations of the target nuclei and cytoplasts.
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
CHAMP software can generally offer close segmentation effect. However, Fig. 15a, b, e and f tell that the MCD method can give a better performance than the CHAMP software generally when the color difference or the average standard deviation of the cytoplast and nucleus is larger. However, there is a trifling defect in the CNC detector. From the extracted nucleus contours on images 10 and 11 in Fig. 14, one can easily observe that the CNC detector cannot quite effectively distinguish two adjacent objects if their colors are very similar. 4. Conclusions This paper develops an automatic method, called a cytoplast and nucleus contour (CNC) detector, to segment the nucleus and cytoplast from a cervical smear image. In this paper, a bi-group enhancer is proposed to isolate the pixels on one object from those on another object; it can effectively suppress the noises and enhance the object contours. This paper also uses the MCD method to segment one object from another based on their color differences. The MCD method can precisely move the contour of an object to its boundary. Additionally, the CNC detector uses K-mean algorithm to partition the nucleus and cytoplast on a cervical smear image. The results of experiments reveal that the CNC detector gives a more impressive performance of object segmentation than the GVF–ACM method and the CHAMP software. Besides cervical smear images, these proposed techniques can still be employed by the object segmentation of other images. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.patrec.2008.02.024. References Busam, K.J., Hester, K., Charles, C., Sachs, D.L., Antonescu, C.R., Gonzalez, S., Halpern, A., 2001. Detection of clinically amelanotic malignant melanoma and assessment of its margins by in vivo confocal scanning laser microscopy. Arch. Dermatol. 137 (7), 923–929. Canny, J., 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8 (6), 679–698. Collier, T., Lacy, A., Malpica, A., Follen, M., Richards-Kortum, R., 2002. Near real-time confocal microscopy of amelanotic tissue: detection of dysplasia in ex vivo cervical tissue. Acad. Radiol. 9 (5), 504–512.
1453
Corcuff, P., Gonnord, G., Pierard, G.E., Leveque, J.L., 1996. In vivo confocal microscopy of human skin: A new design from cosmetology and dermatology. Scanning 18 (5), 351–355. Davies, E., 1990. Machine Vision: Theory, Algorithms and Practicalities. Academic Press (Chapter 5). Frable, W.J., 1982. Needle aspiration biopsy of pulmonary tumors. Semin. Respirat. Med. 4(2), 161–169. Gonzalez, R., Woods, R., 2002. Digital Image Processing. Prentice-Hall. Inoue, H., Igari, I., Nishikage, T., Ami, K., Yoshida, T., Iwai, T., 2000. A novel method of virtual histopathology using laser scanning confocal microscopy in vitro with untreated fresh specimens from the gastrointestinal mucosa. Endoscopy 32 (6), 439–443. Langley, R.G.B., Rajadhyaksha, M., Dwyer, P.J., Sober, A.J., Flotte, T.J., Anderson, R.R., 2001. Confocal scanning laser microcopy of benign and malignancy melanocytic skin lesions in vivo. J. Am. Acad. Dermatol. 45 (3), 365–376. Martin, E., 2003. Pap-smear classification. Master’s Thesis, Technical University of Denmark, Oersted-DTU, Automation. Masters, B.R., Aziz, D.J., Gmitro, A.F., Kerr, J.H., O’Grady, T.C., Goldman, L., 1997. Rapid observation of unfixed unstained human skin biopsy specimens with confocal microscopy and visualization. J. Biomed. Opt. 2 (4), 437–445. Mat-Isa, N.A., Mashor, M.Y., Othman, N.H., 2005. Seeded region growing features extraction algorithm: Its potential use in improving screening for cervical cancer. Int. J. Comput. Internet Manage. 13 (1), 61–70. Norup, J., 2005. Classification of pap-smear data by transductive neuro-fuzzy methods. Master’s Thesis, Technical University of Denmark: Oersted-DTU, Automation. Rajadhyaksha, M., Grossman, M., Esterowitz, D., Webb, R.H., Anderson, R.R., 1995. In vivo confocal scanning laser microscopy of human skin: Melanin provides strong contrast. J. Invest. Dermatol. 104 (6), 946–952. Rajadhyaksha, M., Gonzalez, S., Zavislan, J.M., Anderson, R.R., Webb, R.H., 1999. In vivo confocal scanning laser microscopy of human skin. II: Advances in instrumentation and comparison with histology. J. Invest. Dermatol. 113 (3), 293–303. Russo, F., Lazzari, A., 2005. Color edge detection in presence of gaussian noise using nonlinear prefiltering. IEEE Trans. Instrum. Measure. 54 (1), 352–358. Sezgin, M., Sankur, B., 2004. Survey over image thresholding techniques and quantitative performance evaluation. J. Electron. Imaging 13 (1), 146–165. Su, M.C., Chou, C.H., 2001. A modified version of K-means algorithm with a distance based on cluster symmetry. IEEE Trans. Pattern Anal. Mach. Intell. 23 (6), 674– 680. Tsai, M.H., Chan, Y.K., Lin, Z.Z., Chen, Y.J., Chen, S.C., Yang-Mao, S.F., Huang, P.C., 2007. Nucleus and cytoplasm contour detector of cervical smear image. Proceedings of 4th High-End Visualization Workshop, pp. 127–136. Walker, R.F., 1997. Adaptive multi-scale texture analysis with application to automated cytology. Dissertation, Department of Electrical and Computer Engineering, University of Queensland. NCI Women’s Health Report FY 2005–2006, 2007. National Cancer Institute.
Contents lists available at ScienceDirect
Pattern Recognition Letters journal homepage: www.elsevier.com/locate/patrec
Nucleus and cytoplast contour detector of cervical smear image Meng-Husiun Tsai a, Yung-Kuan Chan a,*, Zhe-Zheng Lin a, Shys-Fan Yang-Mao b, Po-Chi Huang c a b c
Department of Management Information Systems, National Chung Hsing University, 250, Kuokuang Road, Taichung 402, Taiwan Department of Management Information Systems, Central Taiwan University of Science and Technology, No.11, Buzih Lane, Beitun District, Taichung 40601, Taiwan Department of Pathology, Taichung Hospital, Department of Health, Executive Yuan, Taiwan
a r t i c l e
i n f o
Article history: Received 14 March 2007 Received in revised form 27 November 2007 Available online 18 March 2008 Communicated by W. Zhao Keywords: Cervical smear screening Cervical cancer Image segmentation Salt and pepper noise Gaussian noise Contour detection
a b s t r a c t This paper develops a cytoplast and nucleus contour (CNC) detector to sever the nucleus and cytoplast from a cervical smear image. This paper proposes the bi-group enhancer to make a clear-cut separation for the pixels laid between two objects, and the maximal color difference (MCD) method to draw the aptest nucleus contour. The CNC detector adopts a median filter to sweep off noises, the bi-group enhancer to suppress the noises and brighten the object contours, the K-mean algorithm to discern the cytoplast from the background, and the MCD method to extract the nucleus contour. The experimental results show that the CNC detector can give an impressive performance. Besides cervical smear images, these proposed techniques can be utilized in segmenting objects from other images. Ó 2008 Elsevier B.V. All rights reserved.
1. Introduction An estimated 11,150 cases of invasive cervical cancer are expected to be diagnosed in the United States in 2007 and 3670 women are expected to die from the disease. Incidence and mortality rates have decreased steadily over the past five decades, largely due to the widespread use of the Pap smear which detects cervical cancer and precancerous lesions. The Pap smear has made cervical cancer one of the most preventable cancers, but older, poorer, and less educated women are less likely to be screened and screening is not available in many low-resource regions of the world. Worldwide, cervical cancer has a significant impact, with nearly 500,000 new cases and nearly 250,000 deaths reported annually (Women’s Health Report, 2007). Unlike other cancers that cause pain, noticeable lumps, or other early symptoms, cervical cancer has no telltale symptom until it is so advanced that it is usually unresponsive to treatment (Frable, 1982). Only in its latest stage, cervical cancer causes pain in the lower abdominal or back regions. However, most cervical cancer takes many years to develop from normal to dangerous stages. Cervical cancer is a preventable disease and, unlike most cancers, can be easily detected by a routine screening test. Currently, cervical smear screening is the most popular method to detect the presence of abnormal cells arising from the cervix. With a small brush, cotton stick or wooden stick, a specimen is taken from the uterine cer* Corresponding author. Tel.: +886 4 22840422; fax: +886 4 22857173. E-mail address: [email protected] (Y.-K. Chan). 0167-8655/$ - see front matter Ó 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.patrec.2008.02.024
vix, smeared onto a thin, rectangular glass plate (a slide), and dyed, making it easier to examine the cells under a microscope. The purpose of smear screening is to diagnose pre-malignant cell changes before they become cancerous. Dysplastic cells have undergone precancerous changes. They generally have longer as well as darker nuclei, and a tendency to cling together in large clusters. Mildly dysplastic cells have enlarged and bright nuclei. Moderately dysplastic cells have larger and darker nuclei. The nuclei may start to deteriorate and to be granulation. Severe dysplastic cells have large, dark, and often oddly shaped nuclei; its cytoplast is relatively dark and small (Martin, 2003). Hence, pre-cancers and cancers are associated with a variety of morphologic and architectural alterations, including the textures, sizes, and shapes of cytoplast and nucleus, hyperchromasia and pleomorphism. It also increases nuclear–cytoplastic ratio. Fig. 1 shows the superficial squamous cells stained to enhance the contrast. Current manual screening methods are costly and sometimes result in inaccurate diagnosis caused by human error. The introduction of machine assisted screening will bring significant benefits to the community, which can reduce financial costs and increase screening accuracy. An effective boundary detection algorithm locating the contours of the cytoplast and nucleus plays an important role in developing a useful computer-assisted diagnostic system. Wu et al. (1998) introduced a parametric optimal segmentation approach which is suitable for the images of non-overlapping cells with smooth cell boundaries or contours. However, dealing with
1442
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
B
A
Fig. 1. Superficial squamous cells stained to enhance contrast.
the segmentation of a cell image, a priori knowledge of the nuclear characteristics should be fully used; these characteristics include the cell shape, size, and intensities relative to its background. Mat-Isa et al. (2005) utilize the potential use of thresholding the region growing algorithm as a feature extraction technique. The proposed algorithm is called seeded region growing features extraction (SRGFE). The SRGFE is used to extract the size and grey level of a certain region of interest on a digital image. In the SRGFE algorithm, the user needs to determine the region of interest by clicking the mouse on any pixels in the region and to specify the threshold value, which makes the system impractical. Walker (1997) uses a series of automated fast morphological transforms with octagonal structuring elements. Each gray-scale cell image is first globally thresholded, resulting in an incomplete segmentation of the nucleus in binary form. The cytoplastic background is removed by performing a closing of the image using a structured element which is smaller than the smallest nucleus, and the nuclear heterogeneity is corrected by an opening of a similar size. However, it is more appropriate for local thresholding and it cannot be fully automated. Many other cytoplast and nucleus morphological segmentation methods have also been proposed in the related literatures (Busam et al., 2001; Collier et al., 2002; Corcuff et al., 1996; Inoue et al., 2000; Langley et al., 2001; Masters et al., 1997; Mat-Isa et al., 2005; Rajadhyaksha et al., 1999; Rajadhyaksha et al., 1995; Walker, 1997; Wu et al., 1998). However, the results are based on tedious hand-segmentation of images. Martin (2003) and Norup (2005) also take the CHAMP software to segment and classify cervical smear images. Unfortunately, the CHAMP software cannot provide a satisfying segmentation performance, especially for abnormal cervical cells. The aim of this paper is to develop an automated image segmentation system to sever the cytoplast and nucleus from a cervical smear image, without a priori knowledge of the image objects. Generally, the accuracy of an object contour detector depends on the quality of an image. The heavily stained cervical smear may be masked by menstrual blood, vaginal discharge, air artifacts, etc., thus obscuring the abnormal cervical cells. Sometimes, overexposing or underexposing under the microscope light may also blur the cervical smear images. These problems may cause difficulties in extracting the cytoplast and nucleus of a cervical cell. This paper proposes a bi-group enhancer to eliminate the noise on an image and to sharpen the contours of objects before extracting the object. Since the cytoplast and background on a cervical smear image can apparently be distinguished by their colors, this paper will employ K-mean algorithm to discern between the cytoplast and background on the image. Mostly, two different adjacent objects have dissimilar color distributions. Consider an image with only two different objects, as shown in the image in Fig. 2 for example. In this image, the red1 circle indicates the boundary which separates object A from object B where the color difference of the pixels on the inside and outside of the circle is maximal. Based on the previously mentioned properties, this paper proposes a cytoplast 1 For interpretation of color in Fig. 2, the reader is referred to the web version of this article.
Fig. 2. An image with only two objects.
and nucleus contour (CNC) detector to perceive the cytoplast and nucleus contours of a cervical cell. Besides cervical smear images, these techniques can be employed to segment the objects of other images too. Tsai et al. (2007) briefly introduces the concept of the detector. This paper will describe it in more details and give more experiments and discussions. 2. The CNC detector Edge detection is considered fundamental in image processing and computer vision, with a large number of studies having been published in the last two decades, i.e., the Canny edge detector (Canny, 1986), the Sobel edge detector (Davies, 1990), the Prewitt edge detector (Davies, 1990), and the Roberts Cross edge detector (Davies, 1990). Segmentation is the process of dividing an image into its constituent parts for further analysis. We commonly refer to such parts as regions of interest (ROI). Edge detection based segmentation is a frequently used technique which segments an image on the basis of dissimilarity or heterogeneity within image pixels or regions. Fig. 3 shows the edges (marked by red2 lines) obtained by the above mentioned detectors. Fig. 3 illustrates that the above mentioned methods are only suitable to detecting the objects whose contours are obvious. The Canny edge detector can provide a better performance, but it is susceptible to noise. The active contour model (ACM) (Xu and Prince, 1998) is adapted to a wide range of extracting the objects with vague, complex and/or irregular shape boundary, inhomogeneous and noisy interior, as well as contour with small gaps. However, Fig. 4 states that ACM cannot give a good segmentation when the quality of the image is poor. It is also difficult to decide the weighted parameters of the snake’s tension and rigidity, since the contour of cytoplast is varied. In addition, ACM is very sensitive to the initial contour. Most image segmentation methods perform well while the image has good quality and the object contours are distinct. However, cervical smear images are frequently contaminated and the cytoplast and nucleus contours of cervical cells are often vague, especially for abnormal cervical cells. This paper, hence, adopts a bi-group enhancer to intensify the contours of objects in an image. This paper also presents a maximal color difference (MCD) method to segment one object from others based on their color differences. The CNC detector contains three approaches: bi-group, cytoplast contour detection, and nucleus contour detection approaches. The bi-group approach is to suppress the noises and emphasize the edge pixels; the cytoplast contour detection approach utilizes
2 For interpretation of color in Fig. 3, the reader is referred to the web version of this article.
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
1443
Fig. 3. The edges obtained by Canny, Sobel, Prewitt and Roberts cross-edge detectors.
Fig. 4. The contours obtained by ACM.
a K-means algorithm to separate the cytoplast from the background; and the nucleus contour detection approach adopts the MCD method to draw the contour of nucleus. This section will introduce the three approaches in great detail. 2.1. Bi-group approach Image segmentation task directly depends on the quality of the image. The generation of an accurate edge map becomes a very crucial issue when the images are corrupted by noises. There have been several denoising techniques presented in the past studies, such as, the mean filter (Gonzalez and Woods, 2002), the median filter (Gonzalez and Woods, 2002), the Gaussian filter (Gonzalez and Woods, 2002), and the type-B filter (Russo and Lazzari, 2005). Fig. 5a is the image Lena created by adding some impulse noises, and Fig. 6a is a cervical image with some Gaussian noises. Figs. 5 and 6 demonstrate that only the median filter can eliminate both impulse noises and Gaussian noises. Therefore, this paper adopts the median filter to discard impulse and Gaussian noises. The term ‘‘edge” stands for a local luminance change of sufficient strength which is considered important in a given task. Many contextual edge detection techniques based on suppression have been previously proposed. Russo and Lazzari (2005) provide a type-A filter to enhance the contours of objects. It takes into account the differences between the pixel to be processed and its neighbors as follows: small differences are considered to be the
gradients of noises which should be reduced; large differences are considered to be those of edge points which should be preserved. A two-step procedure is applied to the image channels in order to increase the effectiveness of the smoothing action. Hence, type-A pixels are those corrupted by noise with amplitude which is not too different from the neighbors. Yin et al. (2004) also present an automatically adaptive window-level selection algorithm (so-called adaptive image optimization (AIO)) for achieving on improved performance. In this algorithm, first, region of interest (ROI) is extracted by using the variance change and the integral projection; second, the image statistical values, such as, maximum, minimum, and average, can be obtained in the detected ROI; finally, the window and level can be determined from the image statistical values, and a contrast transfer function can be obtained by using the cubic spline interpolation. Via AIO, the quality of an image can be improved by increasing the maximum dynamic range adaptively. Figs. 7 and 8 illustrate the images processed by the type-A prefiltering and the AIO algorithm. However, in either of these methods does suppression have an effect on nearby peripheral pixels which are definitely considered to be the background, cytoplasts, or nucleus. This paper therefore proposes bi-group enhancer to effectively isolate the object pixels from other object pixels. Let pi,j be the pixel located at the coordinates (i, j) on a cervical smear image f0, and Wi,j be the corresponding window of pi,j which is the central pixel of Wi,j consisting of the m m pixels
1444
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
Fig. 5. (a) Noisy image Lena; the images obtained by (b) mean filter, (c) Gaussian filter, (d) type-B filter and (e) median filter.
Fig. 6. (a) A cervical image; the images obtained by (b) mean filter, (c) Gaussian filter, (d) type-B filter and (e) median filter.
Fig. 7. (a) Another original cell image; (b)–(d) the cell images processed by AIO, type-A filter and bi-group enhancers.
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
1445
Fig. 8. (a) Another original cell image; (b)–(d) the cell images processed by AIO, type-A filter and bi-group enhancers.
pibm1c;jbm1c , pibm1c;jbm1cþ1 ; . . . ; pibm1c;j1 , pibm1c;j , pibm1c;j 2 2 2 2 2 2 2 þ1; . . . ; pibm1c;jþbm1c , pibm1cþ1;jbm1c , pibm1cþ1;jbm1cþ1 ; . . . ; 2
2
2
2
2
2
pibm1cþ1;j1 , pibm1cþ1;j , pibm1cþ1;jþ1 ; . . . ; pibm1cþ1;jþbm1c ; . . . ; piþ 2 2 2 2 m12 ; j m1 , piþbm1c;jbm1cþ1 ; . . ., piþbm1c;j1 , piþbm1c;j , piþbm1c;jþ 2 2 2
2
2
2
2
1; . . . ; piþbm1c;jþbm1c . 2
2
After denoising, fo becomes an image ft. Let pi,j be the pixel located at the coordinates (i, j) in ft, and Wi,j be the corresponding window of pi,j where pi,j is the central pixel of Wi,j consisting of m m pixels. Assume that C i;j ¼ fc1 ; c2 ; . . . ; cm2 g are the colors of the pixels on Wi,j, and the color values in Ci,j are sorted increasingly, that is, c1 6 c2 6 6 cm2 . Pm2 þ1 Bi-group enhancer defines the interval between m22þ1 i¼12 ci and cm2 þ1 as well as the interval between cm2 þ1þ1 and m221 2 2 Pm2 i¼ðm2 þ3Þ=2 ci as indefinite intervals since it is difficult to recognize whether pi,j is in an object or in background while the color c of 2
pi,j lies in both intervals. Set mid ¼ m 2þ1 with m considered to be an odd number, then the two intervals are defined as [c1, cmid] and ½cmidþ1 ; cm2 . Hence, the bi-group approach replaces c with c0 , where c0 is defined as follows: 8 m id m id P P > 1 1 > > ci ; if mid ci 6 c 6 cmid ; > mid > > i¼1 i¼1 < m2 m2 c0 ¼ P P 1 1 > ci ; if cmid < c 6 mid1 ci ; > mid1 > > i¼mid1 i¼midþ1 > > : c; otherwise: If c is in the indefinite intervals, the bi-group enhancer changes c into the average color cf of the first half of Ci,j’s when c is closer to cf, or c is supplanted by the average color of the later half of Ci,j’s. Figs. 7 and 8 illustrate that the bi-group enhancer can more effectively separate the pixels of an object from the pixels of other object. In the real world, there are many images with vague objects, such as the images shown in Figs. 7a and 8a. Besides, denoising operations often contain severe blurring. The purpose of a bi-group enhancer is to discriminate the pixels on one object from those on other objects, which are near to the edges of the objects. Fig. 9 shows the color histograms of four cervical smear images after processed by AIO, type-A filter, and bi-group enhancers. Obviously, one can observe that the color histograms of the images obtained by the bi-group enhancer are more distinct separated into three clusters than other two enhancers. 2.2. Cytoplast contour detection approach The region of nucleus is generally much smaller than those of a cytoplast and the background on a cervical smear image. Fig. 10a is a cervical smear image; Fig. 10b shows its color histogram. In Fig. 10b, the color bins which are close to the two highest peaks are the color distributions of the cytoplast and the background;
the color distribution of the nucleus is on the left of the color bins. The color distributions of the cytoplast and the background on a cervical smear image can apparently be distinguished from the color histogram of the image. The CNC detector hence takes the Kmean algorithm (Su and Chou, 2001) to discern between the pixels on a cytoplast and those on a background. Assume that the pixel colors of a cervical smear image are from n0 to n1, where 0 5 n0, n1 5 255. Since there are only a small quantity of nucleus pixels on a cervical smear image and the pixel colors of a nucleus are darker. In order to prevent the clustering via Kmean algorithm from being influenced by the pixel colors of a nucleus, the CNC detector divides the pixels with colors from Ci + e to Cn into two groups, where e is a given constant and Ci as well as Cn is the minimal and maximal pixel colors. It then considers the pixels in the group with a higher pixel color to be the background pixels, and the others to be the cytoplast pixels. The following steps explain how to use the K-mean algorithm to partition the pixels with colors between Ci + e and Cn into two groups: Step 1: Randomly select 2 different values from the interval between Ci + e and Cn to be the respective values of the two groups. Step 2: Categorize each pixel of the image with color within Ci + e and Cn into one of the two groups according to its distance compared to the representative value of each group. The representative value of each group is then substituted for the average color of the pixels in the group. After that, Step 2 is repeated until each group is unchanged or the iteration count is greater than a given constant. 2.3. Nucleus contour detection approach The regions of most nuclei are quite small and they have high variations in color intensity. Thus, K-mean algorithm cannot precisely discriminate them from those of cytoplasts. The CNC detector hence proposes a maximal color difference (MCD) method to detect the nucleus contour. The MCD method can be used to discern an object B from another object A. In this method, an initial contour S is first specified to partition an image into two regions inside and outside S. When given an initial contour S, the MCD method repeatedly moves the pixels on S inward or outward according to the color difference D of the regions inside and outside S until D is maximal. Here, we call D the color difference of S. As a result, S is the expected contour when D of S is maximal. Fig. 11 demonstrates three S represented by three red3 circles, respectively, located inside, on the boundary of, and outside object B, where A and B are two different objects and p is one pixel on S. 3 For interpretation of color in Fig. 11, the reader is referred to the web version of this article.
1446
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
Images
Bi-group
AIO
Fig. 9. The color histograms of four cervical smear images after processed by AIO, type-A filter and bi-group enhancers.
1447
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
Type-A
Fig. 9 (continued)
Fig. 10. A cervical smear image and it color.
A
A
p pO
(a) S is outside B
p
p
B pI
A
B pO
pI
(b) S is on the boundary of B Fig. 11. Three different location of S.
pO
pI
B
(c) S is inside B
1448
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
pI p pO Fig. 12. The related location of p, pI and pO.
The color difference of S depicted by a red circle in Fig. 11b is maximal among those delineated by red, blue and yellow circles. Apparently, S in Fig. 11b is the circle closest to the boundary of B. In Fig. 11a, the color difference of S marked by a yellow circle is maximal; in Fig. 11c, it is indicated by a blue circle which is maximal. Hence, the MCD method, respectively, moves the pixel p in Fig. 11a and c to pI and pO. Given an initial contour S, let nO and nI be the numbers of pixels outside and inside S; D, DO, and DI the color differences of S, SI, and SO; p a pixel on S; pO and pI the two pixels which are closest to p on the normal line of S at p; C, CO, and CI the colors of p, pO, and pI; SI (resp. SO) the same as S, only that p is moved to pI (resp. pO). Following the procedure mentioned previously, the MCD method continually moves each pixel p on S until the color difference of S is maximal; moves p to pI, if the color difference of SI is greater than that of SO; or else, move p to pO. Let L be the tangent line of S at p. Fig. 12 shows the related locations of p, pI and pO. When given an initial contour S, the MCD method repeatedly moves each pixel p on S, until the color difference of S is maximal by the following approaches: repeat I þC C O nO C I C C O nO þC DO ¼ CðnI nI þ1Þ ðnO 1Þ ; DI ¼ CðnI nI 1Þ ðnO þ1Þ , if DO = max(D, DO, DI) then nO ¼ nO 1
Bi-group
(a) The average gradients on the target edges of nuclei
Bi-group
(b) The average gradients on the target edges of cytoplasts Fig. 13. The average gradients of the target: (a) nucleus and (b) cytoplast contour pixels after processed by AIO, type-A filter and bi-group enhancers.
nI ¼ nI þ 1 D ¼ DO ; S ¼ SO move p to pO, else if DI = max(D, DO, DI) then nI ¼ nI 1; nO ¼ nO þ 1 D ¼ DI ; S ¼ SI move p to pI, try to move the next pixel on S until no pixel on S is moved The CNC detector moves each pixel on the extracted cytoplast contour, detected in cytoplast contour detection approach, t pixels inward to generate the initial contour S. It then uses the MCD method to extract the nucleus contour from the extracted cytoplast.
3. Experiments The purpose of this section is to investigate the performance of the CNC detector by experiments compared to those obtained by the GVF–ACM method (Xu and Prince, 1998) and CHAMP software (Martin, 2003; Norup, 2005). The experiments use 26 cervical smear images of 64 64 pixels, which were provided by Taichung Hospital in Taiwan, ROC, as the test data. Each of these images is transformed into a gray-level image. Fig. 14 displays these test images and their target cytoplast and nucleus contours which were manually drawn by an experienced doctor. The first experiment is to evaluate the usefulness of the bi-group approach compared to the AIO and type-A filter enhancers. A sharp edge generally has higher pixel gradients. Most of segmentation methods can give a better performance in cutting off the object with sharper contour. Fig. 13a (resp. Fig. 13b) describes the distribution of the average gradient of the target nucleus (resp. cytoplast) contour pixels of each of the 26 test images after processed by AIO, type-A filter, and bi-group enhancers where m is set to 5 for the bi-group enhancer. Fig. 13a and b tells that the bi-group enhancer can give more definite object contour than other two enhancers. The following experiments are to scrutinize the performance of the CNC detector. The next experiment first employs the CNC detector to extract the cytoplast and nucleus contours of the test images where m, e, and t are given to be 5, 30 and 5. This experiment then takes the GVF–ACM method (Xu and Prince, 1998) to sever the cytoplasts and nucleuses from the test images where all the parameters a, b, j are given to be 1. This experiment also adopts the CHAMP software (Martin, 2003; Norup, 2005) to separate the cytoplasts and nucleuses from the test images. Fig. 14 displays the cytoplast and nucleus contours cut by CNC detector, GVF– ACM method, and the CHAMP software. The performance criteria, misclassification error (ME), edge mismatch (EMM), relative foreground area error (RAE), and modified Hausdorff distance (MHD) (Sezgin and Sankur, 2004), are often used to put into evidence which shows the differing performance features of the segmentation methods. This experiment also evaluates the performances of the CHAMP software (Martin, 2003; Norup, 2005), the GVF–ACM method (Xu and Prince, 1998), and the CNC detector via ME, EMM, MHD and RAE. Table 1, respectively, lists their average measurements for the extracted cytoplast and nucleus contours on the 26 tested images cut by the CHAMP software, the GVF–ACM method, and the CNC detector; Fig. 14 shows the extracted cytoplast and nucleus contours. In this table, all ME, EMM, REA and MHD tell that the CNC detector can give a much better performance than the GVF–ACM method and the CHAMP software. Most of segmentation methods can give a better performance in cutting off the object with a quite different color from its adjacent object. Let AC1 and AC2 be the average pixel colors of object O1 and
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
1449
Fig. 14. The original images, the target cytoplast and nucleus contours of the test image, and the cytoplast and nucleus contours cut by CNC detector, GVF ACM method and CHAMP software.
O2, respectively, and CDiff = |AC1 AC2|. We call CDiff the color difference between O1 and O2. Fig. 15a (resp. Fig. 15b) describes the distribution of the color difference between the nucleus and cyto-
plast regions of each original image to the EMM (resp. MHD) of the nucleus contour on the image extracted by the CHAMP software (Martin, 2003; Norup, 2005), GVF–ACM method (Xu and Prince,
1450
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
Fig. 14 (continued)
1998), and CNC detector for all the 26 test images. Fig. 15a and b demonstrates that the CNC detector provides better segmentation than the CHAMP software and GVF–ACM method in severing the nucleus because it has higher color contrast to its adjacent objects; that is, the MCD method can perform well when the object highly contrasts chromatically with adjacent objects.
Fig. 15c (resp. Fig. 15d) depicts the distribution of the color difference between the background and cytoplast regions of each original image to the EMM (resp. MHD) of the cytoplast contour on the image extracted by the CHAMP software, GVF–ACM method, and CNC detector for all the 26 test images. Fig. 15c and d shows that the CNC detector is more indifferent to the color difference
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
1451
Fig. 14 (continued)
when isolating cytoplasts since the CNC detector uses the K-mean algorithm, which only considers the color similarity of pixels, to separate the background pixels and cytoplast pixels. More unevenness or more noise in an object’s texture generally makes it more difficult to segment that object. The standard deviation of the pixel colors of an object can depict the texture and noises on the object. Let Std1 be the standard deviation of the pixel colors of one object, Std2 be that of other object, and Std = (Std1 + Std2)/2. We call Std the average standard deviation of the two objects. Fig. 15e (resp. Fig. 15f) illustrates the relation between the EMM (resp. MHD) of each image and the average standard deviation of the cytoplast and nucleus regions on the im-
age for all the test images. Fig. 15g (resp. Fig. 15h) displays the relation between the EMM (resp. MHD) of each image and the average standard deviation of the background region and cytoplast region on this image. Fig. 15e and f illustrates that largely the CHAMP software, GVF– ACM method, and MCD method provide a worse performance in splitting the cytoplast when the standard deviation of the pixel colors on an object is larger since the pixel colors of the cytoplast and background are variable or many noises appear on the images. Fig. 15e and f also shows that mostly the CHAMP software and the MCD method is better than the GVF–ACM method in severing the nucleus from a cervical cell image; overall the MCD method
1452
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
the CHAMP software cannot give a good segmentation for the images in images 6, 7 and 15 in Fig. 14 since they contain many noises. The GVF–ACM method is susceptible to noises (i.e. the extracted cytoplasts of the images 9, 12 and 13 in Fig. 14). In addition, the GVF–ACM method takes the internal force to smooth the contour which may constrain it effectively to segment the object with irregular object contours (i.e. the extracted nuclei of images 8, 12, 13, 14 in Fig. 14). It is difficult appropriately to control the internal force following different object shapes. Hence, Table 1 and Fig. 14 indicate that the GVF–ACM method gives the worse performance than two others. From Table 1 and Fig. 14, on the whole, the CNC detector provides a better performance than two others on cytoplast segmentation. In nucleus segmentation, the MCD method and the
Table 1 The average of error measurements Method
Object
MHD
EMM
RAE
ME
CHAMP
Nucleus Cytoplast
0.2836 0.0841
0.2583 0.3850
0.2472 0.0900
0.0090 0.0413
GVF–ACM
Nucleus Cytoplast
0.3848 0.1960
0.4518 0.4219
0.2595 0.0891
0.0077 0.0465
CNC
Nucleus Cytoplast
0.1690 0.0617
0.2278 0.1914
0.2060 0.0325
0.0064 0.0244
and the CHAMP software can obtain a close segmentation effect in nucleus segmentation but the MCD method mostly gives a better performance than the CHAMP software when Std is greater than 21.5 since the CHAMP software is more sensitive to noises. Hence,
a
e
Nucleus (CDiff, EMM) GVF-ACM
Nucleus (Std, EMM) CHAMP
CNC
0.90
0.90
0.72
0.72
0.54
0.54
EMM
EMM
CHAMP
0.36
0.00 20
0.36
35
50
65
80
0.00 8.0
95
12.5
17.0
Nucleus (CDiff, MHD)
b
GVF-ACM
CNC
CHAMP
1. 75
GVF-ACM
CNC
1. 40
MHD
1.40
MHD
26.0
Nucleus (Std, MHD)
f
CHAMP
1.75
1.05
1. 05
0.70
0. 70
0.35
0. 35
0.00 20
35
50
65
80
0. 00 8.0
95
12.5
7.0
CDiff
c
21.5
26.0
Std
g
Cytoplast (CDiff, EMM) CHAMP
0.90
GVF-ACM
CNC
Cytoplast (Std, EMM)
0.72
0.72
0.54
0.54
0.36
CHAMP
0.90
EMM
EMM
21.5
Std
CDiff
0.18
GVF-ACM
CNC
0.36 0.18
25
35
45
55
65
75
0.00 3
85
6
9
d
Cytoplast (CDiff, MHD) CHAMP
2.5
12
15
18
Std
CDiff
Cytoplast (Std, MHD)
h GVF-ACM
CNC
CHAMP
2.5
2.0
GVF-ACM
CNC
2.0
MHD
MHD
CNC
0.18
0.18
0.00 15
GVF-ACM
1.5
1.5
1.0
1.0
0.5
0.5
0.0 15
29
43
57
CDiff
71
85
0.0 3
6
9
12
15
18
Std
Fig. 15. The relations of the EMM and MHD of the extracted contours to the color differences and the average standard deviations of the target nuclei and cytoplasts.
M.-H. Tsai et al. / Pattern Recognition Letters 29 (2008) 1441–1453
CHAMP software can generally offer close segmentation effect. However, Fig. 15a, b, e and f tell that the MCD method can give a better performance than the CHAMP software generally when the color difference or the average standard deviation of the cytoplast and nucleus is larger. However, there is a trifling defect in the CNC detector. From the extracted nucleus contours on images 10 and 11 in Fig. 14, one can easily observe that the CNC detector cannot quite effectively distinguish two adjacent objects if their colors are very similar. 4. Conclusions This paper develops an automatic method, called a cytoplast and nucleus contour (CNC) detector, to segment the nucleus and cytoplast from a cervical smear image. In this paper, a bi-group enhancer is proposed to isolate the pixels on one object from those on another object; it can effectively suppress the noises and enhance the object contours. This paper also uses the MCD method to segment one object from another based on their color differences. The MCD method can precisely move the contour of an object to its boundary. Additionally, the CNC detector uses K-mean algorithm to partition the nucleus and cytoplast on a cervical smear image. The results of experiments reveal that the CNC detector gives a more impressive performance of object segmentation than the GVF–ACM method and the CHAMP software. Besides cervical smear images, these proposed techniques can still be employed by the object segmentation of other images. Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at doi:10.1016/j.patrec.2008.02.024. References Busam, K.J., Hester, K., Charles, C., Sachs, D.L., Antonescu, C.R., Gonzalez, S., Halpern, A., 2001. Detection of clinically amelanotic malignant melanoma and assessment of its margins by in vivo confocal scanning laser microscopy. Arch. Dermatol. 137 (7), 923–929. Canny, J., 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8 (6), 679–698. Collier, T., Lacy, A., Malpica, A., Follen, M., Richards-Kortum, R., 2002. Near real-time confocal microscopy of amelanotic tissue: detection of dysplasia in ex vivo cervical tissue. Acad. Radiol. 9 (5), 504–512.
1453
Corcuff, P., Gonnord, G., Pierard, G.E., Leveque, J.L., 1996. In vivo confocal microscopy of human skin: A new design from cosmetology and dermatology. Scanning 18 (5), 351–355. Davies, E., 1990. Machine Vision: Theory, Algorithms and Practicalities. Academic Press (Chapter 5). Frable, W.J., 1982. Needle aspiration biopsy of pulmonary tumors. Semin. Respirat. Med. 4(2), 161–169. Gonzalez, R., Woods, R., 2002. Digital Image Processing. Prentice-Hall. Inoue, H., Igari, I., Nishikage, T., Ami, K., Yoshida, T., Iwai, T., 2000. A novel method of virtual histopathology using laser scanning confocal microscopy in vitro with untreated fresh specimens from the gastrointestinal mucosa. Endoscopy 32 (6), 439–443. Langley, R.G.B., Rajadhyaksha, M., Dwyer, P.J., Sober, A.J., Flotte, T.J., Anderson, R.R., 2001. Confocal scanning laser microcopy of benign and malignancy melanocytic skin lesions in vivo. J. Am. Acad. Dermatol. 45 (3), 365–376. Martin, E., 2003. Pap-smear classification. Master’s Thesis, Technical University of Denmark, Oersted-DTU, Automation. Masters, B.R., Aziz, D.J., Gmitro, A.F., Kerr, J.H., O’Grady, T.C., Goldman, L., 1997. Rapid observation of unfixed unstained human skin biopsy specimens with confocal microscopy and visualization. J. Biomed. Opt. 2 (4), 437–445. Mat-Isa, N.A., Mashor, M.Y., Othman, N.H., 2005. Seeded region growing features extraction algorithm: Its potential use in improving screening for cervical cancer. Int. J. Comput. Internet Manage. 13 (1), 61–70. Norup, J., 2005. Classification of pap-smear data by transductive neuro-fuzzy methods. Master’s Thesis, Technical University of Denmark: Oersted-DTU, Automation. Rajadhyaksha, M., Grossman, M., Esterowitz, D., Webb, R.H., Anderson, R.R., 1995. In vivo confocal scanning laser microscopy of human skin: Melanin provides strong contrast. J. Invest. Dermatol. 104 (6), 946–952. Rajadhyaksha, M., Gonzalez, S., Zavislan, J.M., Anderson, R.R., Webb, R.H., 1999. In vivo confocal scanning laser microscopy of human skin. II: Advances in instrumentation and comparison with histology. J. Invest. Dermatol. 113 (3), 293–303. Russo, F., Lazzari, A., 2005. Color edge detection in presence of gaussian noise using nonlinear prefiltering. IEEE Trans. Instrum. Measure. 54 (1), 352–358. Sezgin, M., Sankur, B., 2004. Survey over image thresholding techniques and quantitative performance evaluation. J. Electron. Imaging 13 (1), 146–165. Su, M.C., Chou, C.H., 2001. A modified version of K-means algorithm with a distance based on cluster symmetry. IEEE Trans. Pattern Anal. Mach. Intell. 23 (6), 674– 680. Tsai, M.H., Chan, Y.K., Lin, Z.Z., Chen, Y.J., Chen, S.C., Yang-Mao, S.F., Huang, P.C., 2007. Nucleus and cytoplasm contour detector of cervical smear image. Proceedings of 4th High-End Visualization Workshop, pp. 127–136. Walker, R.F., 1997. Adaptive multi-scale texture analysis with application to automated cytology. Dissertation, Department of Electrical and Computer Engineering, University of Queensland. NCI Women’s Health Report FY 2005–2006, 2007. National Cancer Institute.