- Email: [email protected]
Using hard and soft models for classification of medical images
Chemometrics and Intelligent Laboratory Systems 88 (2007) 100 – 106 www.elsevier.com/locate/chemolab
Using hard and soft models for classification of medical images Sergei Kucheryavski Department of Applied Physics and Electronics, Altai State University, Lenin str., 61, 656049, Barnaul, Russia Received 31 March 2006; received in revised form 17 July 2006; accepted 28 August 2006 Available online 10 October 2006
Abstract Methods for processing and analysis of medical images are observed. A method for recognition and classification of blood cells, based on the black-box model, is proposed. Two approaches (wavelet transformation and angle measure technique) are used for construction of feature vectors and PLS discriminant analysis is used for discrimination and classification. The method is tested on the low-quality images of blood smears and shows high level of cell type recognition. © 2006 Elsevier B.V. All rights reserved. Keywords: Image processing and analysis; Angle measure technique (AMT); Wavelet transformation; PLS discriminant analysis
1. Introduction Image processing and analysis is a very important tool for medical investigations and diagnostics. In the present day, there are a lot of diagnostic techniques that result in images, e.g., magnetic resonance tomography, X-ray and ultrasound tomography, optical and confocal microscopy, etc. [1–4]. There is also a wide area of tasks that should be solved with the help of image analysis and classification methods. First of all, it is the segmentation of twoand three-dimensional images of body tissues and internals [5], image registration that allows to identify a geometrical transformation of investigated objects [6] and many others. The image classification has a special place among these methods. It allows to solve such very important problems as determination of cancer cells, blood and marrows cells recognition, revelation of declination in tissues and so on. Preliminary analysis of existing methods showed that most of them are based on the strict relations between some physical properties of investigated objects and etalon [7]. Usually, these properties consist of different geometrical features (i.e., length, diameters, areas, etc.) and color features like hue and intensity. With minor reservations, these methods could be considered as the hard model-based approaches. Such approaches have very rigid requirements to the analyzed data. In the case of images, it means low noise level, good E-mail address: [email protected]. 0169-7439/$ - see front matter © 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.chemolab.2006.08.012
resolution, high level of contrast, etc. This leads to high demands for the equipment that is used for image acquisition (microscopes, cameras, etc.), and very often, the software, which is based on these methods, is developed for the concrete types of equipment and is delivered as a unit with a computer, microscope, camera and so on. The price of such complexes starts from tens of thousands of dollars. From another point of view, in numerous works, it has been shown that image analysis approach, based on the soft models, usually allows to analyze and to classify images with middle and low quality including noised images [8–10]. Here soft modeling means analysis of the common properties of an image, instead of physical properties of the investigated objects, and uses the black-box approach for describing the dependencies between image properties and investigated features. One of the ways to construct the vector of image properties is to use the color components of image pixels. The multivariate image analysis (MIA) is the most known example of such approach among chemometricians. MIA considers an image as a 3D W × H × K matrix, where W and H are the width and height of the image and K is the number of variables, describing each pixel on the image. In the simplest case these variables are the intensity of color components of the pixels (e.g., R, G and B values) [11], but MIA is most effective when images are hyperspectral, i.e., each pixel has an intensity in different wavelength bands. Satellite images that are collected using Multi-Spectral Scanners (MSS) are one of the examples [12].
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
101
Fig. 1. Image classification algorithm.
Hyperspectral imaging can be used also in a laboratory with the help of different imaging systems based on the spectrometry (e.g., near-infrared spectrometry, ion mass spectrometry, etc.) [13,14]. The number of wavelength bands and, accordingly, the size of K-vector for each pixel in this case could by varied from tens to hundreds. The hyperspectral imaging is very usable for analysis of biological and organic samples but in spite of the availability of spectrometers and imaging system nowadays, hyperspectral imaging is still slow and expensive. The second way to construct the image properties vector is to use special methods (e.g., statistical analysis, texture analysis, Fourier transform, etc.) that allow to obtain different integral, scaling dependent and other types of the image features. These methods are quite fast and not resource demanding, which makes them attractive for medical diagnostic purposes. In the present paper, a soft model-based image classification approach was applied to the problem of recognition of blood cell type. Two different ways for feature vector extracting were used: wavelet analysis and angle measure technique (AMT). The result of the comparison of these methods for recognition of “unknown” blood cells is presented. 2. Theory 2.1. Classification algorithm The method of image analysis and classification that was used in the present work is traditional for black-box methods and consists of four steps (Fig. 1). In the first step, image acquisition, the ordinary laboratory optical microscope with cheap VGA digital camera, was used. In the second step, the contrast stretching algorithm was used to enhance the quality of obtained images. After that, the images were transformed into Lab color model, and then pictures of white blood cells were extracted from the image using cluster segmentation algorithm in the Lab variable space. The third step, features extraction, is the most important one, because the main problem that must be decided during classification is to define the most valuable features in the different tasks. In the present work, two methods (wavelet transformation and angle measure technique (AMT)) were chosen for this purpose. This decision was based on the results of other works devoted to the image classification problem [15,16]. Principal component analysis (PCA) [17] was chosen as an unsupervised classification approach for preliminary analysis and discriminant analysis based on the projection to latent structures (PLS-DA) [18] was chosen for the recognition of new “unknown” cells on the fourth step of the classification algorithm.
2.2. Construction of feature vector using wavelet transformation From the signal processing point of view, the wavelet transformation of one-dimensional signal is a multistep or multilevel process [19]. At each step, two filters H and G are used. Filter H gives a smoothed version of a signal, and filter G gives the details (difference between row signal and its smoothed version). In the first step, these filters are applied to the raw signal and the details of the first level of transformation are obtained and stored. In the next step, the raw signal is substituted with its smoothed version and the filtration process is repeated and yields second level details and so on. The coefficients of the filters H and G could be easily obtained using the iterative algorithm with a proper wavelet function [20]. The wavelet transformation of images assumes the successive application of the filters to the image rows and columns on the first level and its smoothed versions on the next [21]. On each level, first on the rows of the image or its smoothed version I, the filters H and G are applied. Two matrices (HrI and GrI) are obtained. Then on the columns of these matrices, filters H and G are applied again and the four resulting matrices (HcHrI, GcHrI, HcGrI, GcGrI) are obtained (Fig. 2). The matrix HcHrI is the average or smoothed version of the raw image, while the matrices GcHrI, HcGrI and GcGrI are vertical, horizontal and diagonal details of the given transformation level also known as transformation or wavelet coefficients.
Fig. 2. Wavelet transformation of two-dimensional signal.
102
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
Fig. 5. The examples of experimental images with different blood cells.
Fig. 3. The AMT algorithm.
After the transformation is completed, image features are generated from the wavelet coefficients using different statistic functions [22]. Denote the horizontal, vertical and diagonal details on ith level of transformation as dhi, dvi and ddi. Then, the features vector could be regarded as {f(dh1), f(dv1), f(dd1), …, f(dhm), f(dvm), f(ddm)}, where f is a statistic function and 1, …, m are the numbers of the transformation level. Based on the preliminary experiments, the energy of wavelet coefficients was chosen as a function f in the present work. 2.3. Construction of a feature vector using AMT Angle measure technique was chosen as a second method for constructing of feature vector. The AMT was developed in 1994 by Robert Andrle for the description of the complex geomorphic lines [23]. AMT transforms an image into a onedimensional spectrum without losses of the structure information. It is highly sensitive to changing typical scales of objects on images. The algorithm of AMT is very simple. In the first stage, the raw image is unfolded into the one-dimensional signal or profile. Different ways of unfolding could be used. The most simple, line by line, is usually usable for analysis of textures or other homogeneous images, but in some cases, unfolding in two directions (horizontal and vertical) gives better results.
After performing the unfolding procedure, a number of points are chosen randomly along the unfolded signal. Than, circles with radius s and with center in these points are constructed. In the next step, the intersections of the profile with the circle are found for each point. The scheme of this procedure is shown in Fig. 3. Here A is the point that was randomly chosen on the profile, C and B are points of intersection of this profile and the circle with the radius s. The supplement to the angle CAB is calculated and stored for all points following which the mean value of angles is obtained. The mean angle value reflects the peculiarities of the profile and accordingly of the image on the scale that corresponds to the radius s. By repeating these calculations for different scales 1 ≤ s ≤ sM, the complex AMTspectrum, which can be regarded as a vector of images features on the set of scales, is constructed. Originally, the graphic representation of AMT spectrum is used for image analysis [23]. The presence of inflection points, peaks, local extrema and other peculiarities on the AMT spectrum curve is very important and gives useful information about investigated signal or image. The example of AMT spectrum of test image is shown in Fig. 4. The axis of abscissa shows radius s in pixels, while the ordinate axis shows the corresponding mean angle in radian. As is easy to see, there is a peak on the curve that corresponds to the typical size of objects in the image. If the several images are compared, the slope and the radius of curvature of the spectrum curves should be also taken into account. But in many cases, the visual analysis of graphic representation of the AMT spectra does not allow to get the required result, because of the overlapping of the curves. Esbensen et al. introduced AMT into chemometrics as a general
Fig. 4. The example of the AMT-spectrum of test image.
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
103
Fig. 6. The results of PLS-DA analysis of calibration data set. Top plot corresponds to AMT-based features vectors building, bottom to Wavelet based.
approach for analysis of measurement series [24]. The core idea was to consider mean angle spectrum as a feature vector that describes the scale features of investigated signal or image and
Fig. 8. The images of lymphocyte (top) and neutrophil (bottom) as surface.
to use PCA or PLS for analysis and classification purposes. In the present work, we use the same approach. 3. Experimental 3.1. Objects for investigations From elementary biology, it is well known that blood consists of plasma and cells, which for their turn could be
Fig. 7. The results of PLS-DA analysis of test data set. Top plot corresponds to AMT-based features vectors building, bottom to Wavelet based.
Fig. 9. Spiral unfolding of the image.
104
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
Fig. 10. The profiles of spiral-like unfolded images of the lymphocytes (plots on the left part) and of the neutrophils (plots on the right part).
divided into the red and white cells [25]. Erythrocytes, lymphocytes, granulocytes and monocytes are the main mature cells. The neutrophils, monocytes and lymphocytes are the most prolate of them. Using good equipment and chemicals for the preparation of blood smears, it is possible to obtain high-quality images where the blood cells can be differentiated very easy. But the quality of images, which are obtained with an ordinary optical microscope and cheap digital camera, are too far from the requirements of commercial cells recognition software. The main goal of the experiment was to find out how the methods described in the previous section will recognize blood cell types on the low-quality images. For this purpose, images of the specially prepared blood smears were taken from the optical microscope with a VGA digital camera. Then the pictures of white blood cells were automatically segmented from the initial images using cluster discrimination algorithm in Lab color space. The pictures had a resolution 128 × 128 pixels and a white background. The examples of pictures with different blood cells are shown in Fig. 5. All pictures were divided randomly to the test set and calibration set. The calibration set consisted of 60 samples of lymphocytes and neutrophils that were taken from different patients in different days and the test set consisted of 96 samples that were taken in other days and from other people than the samples from the calibration set. For every cell picture, two feature vectors were calculated: one using the statistics of details of the wavelet transformation and another using the angle measure technique. Then, PCA was used for preliminary analysis and PLS discriminant analysis was used for calibration of the model and class prediction for new “unknown” samples from the test set. The quality of recognition was estimated as a ratio of the number of incorrectly recognized samples to the total number of samples of each cell type and evaluated in percent.
4. Results The results of preliminary PCA analysis showed the presence of groups on the scores plots for both methods used for features vector building, but the clouds of points that correspond to the different cell types were overlapped. It was
Fig. 11. The results of PLS-DA analysis of the calibration data set (top plot) and of the test data set (bottom plot) using spiral unfolding.
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
decided to apply PLS discriminant analysis in order to estimate the discrimination power of each method. The results of PLS discriminant analysis for the calibration set are shown on the classification plots in Fig. 6. The centering and scaling were used for data preprocessing. Classification plot on the top of Fig. 6 was obtained using wavelet transformation for feature vector building, the plot on the top of Fig. 6–to AMT. Here and on the other plots, the lymphocytes are drawn by squares and the neutrophils by circles. Points that lie to the upper horizontal line at 0.5 are classified by PLS-DA as the lymphocytes, whereas points that lie below this line are classified as the neutrophils. Here the use of the wavelet transformation for feature vector building gives better results than the AMT. This is common also for the classification of the unknown samples from the test set, the results of PLS discriminant analysis for the test set are shown in Fig. 7. Obviously, the AMT-based algorithm did not allow to recognize new unknown samples correctly while the wavelet transformation-based algorithm of feature vectors construction gave about 75% of the correct recognized cells. If we will take into account the quality of the images, this result is acceptable from the image analysis point of view, but the quality of recognition is not enough for medical diagnostics. The experiments with other wavelet functions and metrics did not give any improvement. What was wrong with AMT? If one looks at the images of blood cells as a surface using one of the RGB channels or (which is better) using the L channel from Lab color model as a third variable, it would be easy to notice that such surfaces have a fractal nature. The example of such pictures for a lymphocyte and neutrophil is shown in Fig. 8. It is obvious that the lymphocyte has a smoother surface than the neutrophil and, of course, the segmented neutrophil surface will be more fractured than the surface of the row neutrophil. On the other hand, the AMT was developed for the description of complex geometric structures where the standard fractal analysis technique has failed. It was shown that the AMT is very sensitive and allows to discriminate surfaces with the different fractal dimension [26]. Evidently, the problem was in the wrong preprocessing of the images. It could be asserted that most surfaces as well as cells have a kind of radial symmetry. For the use of this additional a priori knowledge, it was decided to change the unfolding scheme at the first step of the AMT algorithm in the spiral favor (Fig. 9). The analysis of the spiral unfolded images profiles gave interesting results that are shown in Fig. 10. The first (zero) half of profiles that correspond to white background pixels around the cell picture have been cut in order to make plots more informative. The profiles on the left part of the figure correspond to the lymphocytes and profiles on right side to the neutrophils. Obviously, the profiles correlate with the size of the cells and their kernels, but what is more important is that one can see the common tendency in the profile shape for each class. It was decided to apply the angle measure technique to the profiles of images, obtained by spiral unfolding. The results of PLS discriminant analysis of the AMT spectra of the spiral-like unfolded images are shown in Fig. 11. The
105
classification plot on the top of Fig. 11 corresponds to the calibration set and the classification plot on the bottom of Fig. 11 to the test set. Correctly recognized unknown samples from the test set were 97% for one class and 96% for the other. 5. Discussion and conclusion Image processing and analysis is a valuable part of medical practice and diagnostics. The habitual methods of analysis, which are based on the hard modeling of studied objects, are spread very wide in medicine. Such methods usually use morphology information about investigated objects that leads to rigid requirements to the quality of images and accordingly demand expensive equipment. On the other hand, soft model-based image analysis approaches that deal with common properties of an image, instead of physical properties of objects, allow to obtain good results even for middle- or low-quality images. In these approaches, the black-box modeling is used for establishing the dependencies between the image properties and investigated features. In this work, it was shown that such approaches give excellent results in the task of blood cell recognition. Moreover, this allowed for low-quality images of blood smears that was obtained with ordinary, inexpensive equipment. It allows us to understand that even increasing digital camera resolution from VGA to 1-2Mpix will give better result and will permit to recognize more subtle differences in the blood cells. Also, using the angle measure technique, earlier mainly for texture analysis, gave a good account of oneself in the task of heterogeneous image recognition and classification. But to make the AMT more efficient, in practice, the preliminary investigations of morphology of analyzed objects are necessary. The results of these investigations should be taken into account when selecting the type of image unfolding on the first step of AMT algorithm. As it was shown, the correct selection of the unfolding scheme has considerable influence on the final result. References [1] M. Haglund, Neurosurg. Clin. N. Am. 8 (1997) 287–291. [2] D. Ropers, M. Regenfus, G. Wasmeier, S. Achenbach, S. Minerva, Cardioangiology 52 (2004) 407–417. [3] G.T. Clement, Ultrasonics 42 (2004) 1087–1093. [4] J.W. Schopf, A.B. Tripathi, A.B. Kudryavtsev, Astrobiology 6 (2006) 1–16. [5] T.W. Nattkemper, Medinfo 11 (2004) 847–851. [6] L. Zöllei, J. Fisher, W.M. Wells, Information Processing in Medical Imaging (IPMI), LNCS 2732 (2003) 366–377. [7] W. Shitong, W. Min, IEEE Trans. Inf. Technol. Biomed. 10 (2006) 5–10. [8] R.C. Gonzalez, R.E. Woods, Digital Image Processing, Addison Wesley, 1993. [9] Y. Fu, Handbook of Pattern Recognition and Image Processing, Academic Press, 1986. [10] Z.Q. Hong, Pattern Recogn. 24 (1991) 211–219. [11] T. Lied, P. Geladi, K. Esbensen, J. Chemom. 14 (2000) 585–598. [12] D. Landgrebe, IEEE Signal Process. Mag. 19 (2002) 17–28. [13] P. Geladi, J. Burgerb, T. Lestander, Chemometr. Intell. Lab. Syst. 72 (2004) 209–217. [14] G. Payne, et al., Talanta 67 (2005) 334–344. [15] M.H. Bharati, J. Jay Liu, J.F. MacGregor, Chemometr. Intell. Lab. Syst. 72 (2004) 57–71.
106
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
[16] J. Huang, K.H. Esbensen, Chemometr. Intell. Lab. Syst. 54 (2000) 1–19. [17] L. Eriksson, E. Johansson, N. Kettaneh-Wold, S. Wold, Introduction to Multi-and Megavariate Data Analysis using Projection Methods (PCA and PLS), Umetrics, Umea, Sweden, 1999. [18] M. Sjostrom, S. Wold, B. Soderstrom, in: E.S. Gelsema, L.N. Kanal (Eds.), Pattern Recognition in Practice, Elsevier, Amsterdam, 1986. [19] S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, 1999. [20] C.K. Chui, Wavelets: A Tutorial in Theory and Applications, Academic Press, 1992. [21] E.J. Stollnitz, T. DeRose, D.H. Salesin, Wavelets for Computer Graphics, Theory and Applications, Morgan Kaufmann Publishers Inc., San Francisco, California, 1996.
[22] G. Van de Wouwer, Wavelets for Multiscale Texture Analysis, PhD thesis, University of Antwerp, Antwerp, Belgium, 1998. [23] R. Andrle, Math. Geol. 16 (1996) 83-79. [24] K.H. Esbensen, K. Kvaal, K.H. Hjelmen, J. Chemom. 10 (1996) 569–590. [25] S.C. Anderson, K.B. Poulsen, Anderson's Atlas of Hematology, Lippincott Williams & Wilkins, 2003. [26] S. Kucheryavski, V. Polyakov, A. Govorov, Analysis of simulated fracture surfaces using AMT and fractal geometry methods, in: A.L. Pomerantsev (Ed.), Progress in Chemometrics Research, Nova Science Publishers Inc., USA, 2005.
Using hard and soft models for classification of medical images Sergei Kucheryavski Department of Applied Physics and Electronics, Altai State University, Lenin str., 61, 656049, Barnaul, Russia Received 31 March 2006; received in revised form 17 July 2006; accepted 28 August 2006 Available online 10 October 2006
Abstract Methods for processing and analysis of medical images are observed. A method for recognition and classification of blood cells, based on the black-box model, is proposed. Two approaches (wavelet transformation and angle measure technique) are used for construction of feature vectors and PLS discriminant analysis is used for discrimination and classification. The method is tested on the low-quality images of blood smears and shows high level of cell type recognition. © 2006 Elsevier B.V. All rights reserved. Keywords: Image processing and analysis; Angle measure technique (AMT); Wavelet transformation; PLS discriminant analysis
1. Introduction Image processing and analysis is a very important tool for medical investigations and diagnostics. In the present day, there are a lot of diagnostic techniques that result in images, e.g., magnetic resonance tomography, X-ray and ultrasound tomography, optical and confocal microscopy, etc. [1–4]. There is also a wide area of tasks that should be solved with the help of image analysis and classification methods. First of all, it is the segmentation of twoand three-dimensional images of body tissues and internals [5], image registration that allows to identify a geometrical transformation of investigated objects [6] and many others. The image classification has a special place among these methods. It allows to solve such very important problems as determination of cancer cells, blood and marrows cells recognition, revelation of declination in tissues and so on. Preliminary analysis of existing methods showed that most of them are based on the strict relations between some physical properties of investigated objects and etalon [7]. Usually, these properties consist of different geometrical features (i.e., length, diameters, areas, etc.) and color features like hue and intensity. With minor reservations, these methods could be considered as the hard model-based approaches. Such approaches have very rigid requirements to the analyzed data. In the case of images, it means low noise level, good E-mail address: [email protected]. 0169-7439/$ - see front matter © 2006 Elsevier B.V. All rights reserved. doi:10.1016/j.chemolab.2006.08.012
resolution, high level of contrast, etc. This leads to high demands for the equipment that is used for image acquisition (microscopes, cameras, etc.), and very often, the software, which is based on these methods, is developed for the concrete types of equipment and is delivered as a unit with a computer, microscope, camera and so on. The price of such complexes starts from tens of thousands of dollars. From another point of view, in numerous works, it has been shown that image analysis approach, based on the soft models, usually allows to analyze and to classify images with middle and low quality including noised images [8–10]. Here soft modeling means analysis of the common properties of an image, instead of physical properties of the investigated objects, and uses the black-box approach for describing the dependencies between image properties and investigated features. One of the ways to construct the vector of image properties is to use the color components of image pixels. The multivariate image analysis (MIA) is the most known example of such approach among chemometricians. MIA considers an image as a 3D W × H × K matrix, where W and H are the width and height of the image and K is the number of variables, describing each pixel on the image. In the simplest case these variables are the intensity of color components of the pixels (e.g., R, G and B values) [11], but MIA is most effective when images are hyperspectral, i.e., each pixel has an intensity in different wavelength bands. Satellite images that are collected using Multi-Spectral Scanners (MSS) are one of the examples [12].
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
101
Fig. 1. Image classification algorithm.
Hyperspectral imaging can be used also in a laboratory with the help of different imaging systems based on the spectrometry (e.g., near-infrared spectrometry, ion mass spectrometry, etc.) [13,14]. The number of wavelength bands and, accordingly, the size of K-vector for each pixel in this case could by varied from tens to hundreds. The hyperspectral imaging is very usable for analysis of biological and organic samples but in spite of the availability of spectrometers and imaging system nowadays, hyperspectral imaging is still slow and expensive. The second way to construct the image properties vector is to use special methods (e.g., statistical analysis, texture analysis, Fourier transform, etc.) that allow to obtain different integral, scaling dependent and other types of the image features. These methods are quite fast and not resource demanding, which makes them attractive for medical diagnostic purposes. In the present paper, a soft model-based image classification approach was applied to the problem of recognition of blood cell type. Two different ways for feature vector extracting were used: wavelet analysis and angle measure technique (AMT). The result of the comparison of these methods for recognition of “unknown” blood cells is presented. 2. Theory 2.1. Classification algorithm The method of image analysis and classification that was used in the present work is traditional for black-box methods and consists of four steps (Fig. 1). In the first step, image acquisition, the ordinary laboratory optical microscope with cheap VGA digital camera, was used. In the second step, the contrast stretching algorithm was used to enhance the quality of obtained images. After that, the images were transformed into Lab color model, and then pictures of white blood cells were extracted from the image using cluster segmentation algorithm in the Lab variable space. The third step, features extraction, is the most important one, because the main problem that must be decided during classification is to define the most valuable features in the different tasks. In the present work, two methods (wavelet transformation and angle measure technique (AMT)) were chosen for this purpose. This decision was based on the results of other works devoted to the image classification problem [15,16]. Principal component analysis (PCA) [17] was chosen as an unsupervised classification approach for preliminary analysis and discriminant analysis based on the projection to latent structures (PLS-DA) [18] was chosen for the recognition of new “unknown” cells on the fourth step of the classification algorithm.
2.2. Construction of feature vector using wavelet transformation From the signal processing point of view, the wavelet transformation of one-dimensional signal is a multistep or multilevel process [19]. At each step, two filters H and G are used. Filter H gives a smoothed version of a signal, and filter G gives the details (difference between row signal and its smoothed version). In the first step, these filters are applied to the raw signal and the details of the first level of transformation are obtained and stored. In the next step, the raw signal is substituted with its smoothed version and the filtration process is repeated and yields second level details and so on. The coefficients of the filters H and G could be easily obtained using the iterative algorithm with a proper wavelet function [20]. The wavelet transformation of images assumes the successive application of the filters to the image rows and columns on the first level and its smoothed versions on the next [21]. On each level, first on the rows of the image or its smoothed version I, the filters H and G are applied. Two matrices (HrI and GrI) are obtained. Then on the columns of these matrices, filters H and G are applied again and the four resulting matrices (HcHrI, GcHrI, HcGrI, GcGrI) are obtained (Fig. 2). The matrix HcHrI is the average or smoothed version of the raw image, while the matrices GcHrI, HcGrI and GcGrI are vertical, horizontal and diagonal details of the given transformation level also known as transformation or wavelet coefficients.
Fig. 2. Wavelet transformation of two-dimensional signal.
102
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
Fig. 5. The examples of experimental images with different blood cells.
Fig. 3. The AMT algorithm.
After the transformation is completed, image features are generated from the wavelet coefficients using different statistic functions [22]. Denote the horizontal, vertical and diagonal details on ith level of transformation as dhi, dvi and ddi. Then, the features vector could be regarded as {f(dh1), f(dv1), f(dd1), …, f(dhm), f(dvm), f(ddm)}, where f is a statistic function and 1, …, m are the numbers of the transformation level. Based on the preliminary experiments, the energy of wavelet coefficients was chosen as a function f in the present work. 2.3. Construction of a feature vector using AMT Angle measure technique was chosen as a second method for constructing of feature vector. The AMT was developed in 1994 by Robert Andrle for the description of the complex geomorphic lines [23]. AMT transforms an image into a onedimensional spectrum without losses of the structure information. It is highly sensitive to changing typical scales of objects on images. The algorithm of AMT is very simple. In the first stage, the raw image is unfolded into the one-dimensional signal or profile. Different ways of unfolding could be used. The most simple, line by line, is usually usable for analysis of textures or other homogeneous images, but in some cases, unfolding in two directions (horizontal and vertical) gives better results.
After performing the unfolding procedure, a number of points are chosen randomly along the unfolded signal. Than, circles with radius s and with center in these points are constructed. In the next step, the intersections of the profile with the circle are found for each point. The scheme of this procedure is shown in Fig. 3. Here A is the point that was randomly chosen on the profile, C and B are points of intersection of this profile and the circle with the radius s. The supplement to the angle CAB is calculated and stored for all points following which the mean value of angles is obtained. The mean angle value reflects the peculiarities of the profile and accordingly of the image on the scale that corresponds to the radius s. By repeating these calculations for different scales 1 ≤ s ≤ sM, the complex AMTspectrum, which can be regarded as a vector of images features on the set of scales, is constructed. Originally, the graphic representation of AMT spectrum is used for image analysis [23]. The presence of inflection points, peaks, local extrema and other peculiarities on the AMT spectrum curve is very important and gives useful information about investigated signal or image. The example of AMT spectrum of test image is shown in Fig. 4. The axis of abscissa shows radius s in pixels, while the ordinate axis shows the corresponding mean angle in radian. As is easy to see, there is a peak on the curve that corresponds to the typical size of objects in the image. If the several images are compared, the slope and the radius of curvature of the spectrum curves should be also taken into account. But in many cases, the visual analysis of graphic representation of the AMT spectra does not allow to get the required result, because of the overlapping of the curves. Esbensen et al. introduced AMT into chemometrics as a general
Fig. 4. The example of the AMT-spectrum of test image.
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
103
Fig. 6. The results of PLS-DA analysis of calibration data set. Top plot corresponds to AMT-based features vectors building, bottom to Wavelet based.
approach for analysis of measurement series [24]. The core idea was to consider mean angle spectrum as a feature vector that describes the scale features of investigated signal or image and
Fig. 8. The images of lymphocyte (top) and neutrophil (bottom) as surface.
to use PCA or PLS for analysis and classification purposes. In the present work, we use the same approach. 3. Experimental 3.1. Objects for investigations From elementary biology, it is well known that blood consists of plasma and cells, which for their turn could be
Fig. 7. The results of PLS-DA analysis of test data set. Top plot corresponds to AMT-based features vectors building, bottom to Wavelet based.
Fig. 9. Spiral unfolding of the image.
104
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
Fig. 10. The profiles of spiral-like unfolded images of the lymphocytes (plots on the left part) and of the neutrophils (plots on the right part).
divided into the red and white cells [25]. Erythrocytes, lymphocytes, granulocytes and monocytes are the main mature cells. The neutrophils, monocytes and lymphocytes are the most prolate of them. Using good equipment and chemicals for the preparation of blood smears, it is possible to obtain high-quality images where the blood cells can be differentiated very easy. But the quality of images, which are obtained with an ordinary optical microscope and cheap digital camera, are too far from the requirements of commercial cells recognition software. The main goal of the experiment was to find out how the methods described in the previous section will recognize blood cell types on the low-quality images. For this purpose, images of the specially prepared blood smears were taken from the optical microscope with a VGA digital camera. Then the pictures of white blood cells were automatically segmented from the initial images using cluster discrimination algorithm in Lab color space. The pictures had a resolution 128 × 128 pixels and a white background. The examples of pictures with different blood cells are shown in Fig. 5. All pictures were divided randomly to the test set and calibration set. The calibration set consisted of 60 samples of lymphocytes and neutrophils that were taken from different patients in different days and the test set consisted of 96 samples that were taken in other days and from other people than the samples from the calibration set. For every cell picture, two feature vectors were calculated: one using the statistics of details of the wavelet transformation and another using the angle measure technique. Then, PCA was used for preliminary analysis and PLS discriminant analysis was used for calibration of the model and class prediction for new “unknown” samples from the test set. The quality of recognition was estimated as a ratio of the number of incorrectly recognized samples to the total number of samples of each cell type and evaluated in percent.
4. Results The results of preliminary PCA analysis showed the presence of groups on the scores plots for both methods used for features vector building, but the clouds of points that correspond to the different cell types were overlapped. It was
Fig. 11. The results of PLS-DA analysis of the calibration data set (top plot) and of the test data set (bottom plot) using spiral unfolding.
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
decided to apply PLS discriminant analysis in order to estimate the discrimination power of each method. The results of PLS discriminant analysis for the calibration set are shown on the classification plots in Fig. 6. The centering and scaling were used for data preprocessing. Classification plot on the top of Fig. 6 was obtained using wavelet transformation for feature vector building, the plot on the top of Fig. 6–to AMT. Here and on the other plots, the lymphocytes are drawn by squares and the neutrophils by circles. Points that lie to the upper horizontal line at 0.5 are classified by PLS-DA as the lymphocytes, whereas points that lie below this line are classified as the neutrophils. Here the use of the wavelet transformation for feature vector building gives better results than the AMT. This is common also for the classification of the unknown samples from the test set, the results of PLS discriminant analysis for the test set are shown in Fig. 7. Obviously, the AMT-based algorithm did not allow to recognize new unknown samples correctly while the wavelet transformation-based algorithm of feature vectors construction gave about 75% of the correct recognized cells. If we will take into account the quality of the images, this result is acceptable from the image analysis point of view, but the quality of recognition is not enough for medical diagnostics. The experiments with other wavelet functions and metrics did not give any improvement. What was wrong with AMT? If one looks at the images of blood cells as a surface using one of the RGB channels or (which is better) using the L channel from Lab color model as a third variable, it would be easy to notice that such surfaces have a fractal nature. The example of such pictures for a lymphocyte and neutrophil is shown in Fig. 8. It is obvious that the lymphocyte has a smoother surface than the neutrophil and, of course, the segmented neutrophil surface will be more fractured than the surface of the row neutrophil. On the other hand, the AMT was developed for the description of complex geometric structures where the standard fractal analysis technique has failed. It was shown that the AMT is very sensitive and allows to discriminate surfaces with the different fractal dimension [26]. Evidently, the problem was in the wrong preprocessing of the images. It could be asserted that most surfaces as well as cells have a kind of radial symmetry. For the use of this additional a priori knowledge, it was decided to change the unfolding scheme at the first step of the AMT algorithm in the spiral favor (Fig. 9). The analysis of the spiral unfolded images profiles gave interesting results that are shown in Fig. 10. The first (zero) half of profiles that correspond to white background pixels around the cell picture have been cut in order to make plots more informative. The profiles on the left part of the figure correspond to the lymphocytes and profiles on right side to the neutrophils. Obviously, the profiles correlate with the size of the cells and their kernels, but what is more important is that one can see the common tendency in the profile shape for each class. It was decided to apply the angle measure technique to the profiles of images, obtained by spiral unfolding. The results of PLS discriminant analysis of the AMT spectra of the spiral-like unfolded images are shown in Fig. 11. The
105
classification plot on the top of Fig. 11 corresponds to the calibration set and the classification plot on the bottom of Fig. 11 to the test set. Correctly recognized unknown samples from the test set were 97% for one class and 96% for the other. 5. Discussion and conclusion Image processing and analysis is a valuable part of medical practice and diagnostics. The habitual methods of analysis, which are based on the hard modeling of studied objects, are spread very wide in medicine. Such methods usually use morphology information about investigated objects that leads to rigid requirements to the quality of images and accordingly demand expensive equipment. On the other hand, soft model-based image analysis approaches that deal with common properties of an image, instead of physical properties of objects, allow to obtain good results even for middle- or low-quality images. In these approaches, the black-box modeling is used for establishing the dependencies between the image properties and investigated features. In this work, it was shown that such approaches give excellent results in the task of blood cell recognition. Moreover, this allowed for low-quality images of blood smears that was obtained with ordinary, inexpensive equipment. It allows us to understand that even increasing digital camera resolution from VGA to 1-2Mpix will give better result and will permit to recognize more subtle differences in the blood cells. Also, using the angle measure technique, earlier mainly for texture analysis, gave a good account of oneself in the task of heterogeneous image recognition and classification. But to make the AMT more efficient, in practice, the preliminary investigations of morphology of analyzed objects are necessary. The results of these investigations should be taken into account when selecting the type of image unfolding on the first step of AMT algorithm. As it was shown, the correct selection of the unfolding scheme has considerable influence on the final result. References [1] M. Haglund, Neurosurg. Clin. N. Am. 8 (1997) 287–291. [2] D. Ropers, M. Regenfus, G. Wasmeier, S. Achenbach, S. Minerva, Cardioangiology 52 (2004) 407–417. [3] G.T. Clement, Ultrasonics 42 (2004) 1087–1093. [4] J.W. Schopf, A.B. Tripathi, A.B. Kudryavtsev, Astrobiology 6 (2006) 1–16. [5] T.W. Nattkemper, Medinfo 11 (2004) 847–851. [6] L. Zöllei, J. Fisher, W.M. Wells, Information Processing in Medical Imaging (IPMI), LNCS 2732 (2003) 366–377. [7] W. Shitong, W. Min, IEEE Trans. Inf. Technol. Biomed. 10 (2006) 5–10. [8] R.C. Gonzalez, R.E. Woods, Digital Image Processing, Addison Wesley, 1993. [9] Y. Fu, Handbook of Pattern Recognition and Image Processing, Academic Press, 1986. [10] Z.Q. Hong, Pattern Recogn. 24 (1991) 211–219. [11] T. Lied, P. Geladi, K. Esbensen, J. Chemom. 14 (2000) 585–598. [12] D. Landgrebe, IEEE Signal Process. Mag. 19 (2002) 17–28. [13] P. Geladi, J. Burgerb, T. Lestander, Chemometr. Intell. Lab. Syst. 72 (2004) 209–217. [14] G. Payne, et al., Talanta 67 (2005) 334–344. [15] M.H. Bharati, J. Jay Liu, J.F. MacGregor, Chemometr. Intell. Lab. Syst. 72 (2004) 57–71.
106
S. Kucheryavski / Chemometrics and Intelligent Laboratory Systems 88 (2007) 100–106
[16] J. Huang, K.H. Esbensen, Chemometr. Intell. Lab. Syst. 54 (2000) 1–19. [17] L. Eriksson, E. Johansson, N. Kettaneh-Wold, S. Wold, Introduction to Multi-and Megavariate Data Analysis using Projection Methods (PCA and PLS), Umetrics, Umea, Sweden, 1999. [18] M. Sjostrom, S. Wold, B. Soderstrom, in: E.S. Gelsema, L.N. Kanal (Eds.), Pattern Recognition in Practice, Elsevier, Amsterdam, 1986. [19] S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, 1999. [20] C.K. Chui, Wavelets: A Tutorial in Theory and Applications, Academic Press, 1992. [21] E.J. Stollnitz, T. DeRose, D.H. Salesin, Wavelets for Computer Graphics, Theory and Applications, Morgan Kaufmann Publishers Inc., San Francisco, California, 1996.
[22] G. Van de Wouwer, Wavelets for Multiscale Texture Analysis, PhD thesis, University of Antwerp, Antwerp, Belgium, 1998. [23] R. Andrle, Math. Geol. 16 (1996) 83-79. [24] K.H. Esbensen, K. Kvaal, K.H. Hjelmen, J. Chemom. 10 (1996) 569–590. [25] S.C. Anderson, K.B. Poulsen, Anderson's Atlas of Hematology, Lippincott Williams & Wilkins, 2003. [26] S. Kucheryavski, V. Polyakov, A. Govorov, Analysis of simulated fracture surfaces using AMT and fractal geometry methods, in: A.L. Pomerantsev (Ed.), Progress in Chemometrics Research, Nova Science Publishers Inc., USA, 2005.