- Email: [email protected]
Wavelet transform based image texture analysis for size estimation applied to the sorting of tea granules
Journal of Food Engineering 79 (2007) 629–639 www.elsevier.com/locate/jfoodeng
Wavelet transform based image texture analysis for size estimation applied to the sorting of tea granules S. Borah a, E.L. Hines a
a,*
, M. Bhuyan
b
Electrical and Electronics Division, School of Engineering, University of Warwick, Coventry CV4 7AL, UK b Department of Electronics, Tezpur University, Tezpur 784 028, Assam, India Received 11 October 2005; accepted 13 February 2006 Available online 29 March 2006
Abstract This paper describes a new texture feature estimation technique for discriminating images of eight different grades of CTC (cutting, tearing, and curling) tea. This new set of feature vectors can discriminate the images of different sized tea granules with more efficiency than the statistical feature vectors do. The technique conjugates the feature information of one group of images along with the information of rest of the groups. This is executed by considering range of different groups of images of the same granule size. Indeed, ranges are estimated using the existing statistical texture features, namely variance, entropy and energy, in difference form. Daubechies’ wavelets transform (WT) based sub-band images are utilized for calculating these statistical features. The techniques, for estimating these ranges and calculating the final feature set, adopt a simplified version of Mahalanobis distance calculation. Later, the data visualization method, principal component analysis (PCA), which is used to visualize the existing classes of textures, has found distinguishable characteristics among the new feature sets. It is further observed that the unsupervised clustering algorithm self organizing map (SOM) can classify the images efficiently into appropriate clusters. Two neural networks, namely multi-layer perceptron (MLP) network and learning vector quantization (LVQ) were used for texture classifications. The classification accuracy, for example 74.67% and 80% in MLP and LVQ, respectively, outperforms the other results obtained by using existing statistical texture features. 2006 Elsevier Ltd. All rights reserved. Keywords: Tea granule size; Wavelet transform; Texture feature; Mahalanobis distance
1. Introduction Efficient feature extraction is one of the most significant aspects of texture analysis in computer vision applications. Most existing feature extraction methods provide efficient tool for shape and pattern classification in the images (Chen, Nixon, & Thomas, 1994; Pietikainen, Ojala, & Xu, 2000; Tianhorng & Kuo, 1993). But the situation becomes complicated, while both size and shape of particles in the images come into the limelight. Such phenomenon was envisioned in discriminating images of different * Corresponding author. Tel.: +44 (0) 24 765 23246; fax: +44 (0) 24 764 18922. E-mail addresses: [email protected] (S. Borah), E.L.Hines@ warwick.ac.uk (E.L. Hines), [email protected] (M. Bhuyan).
0260-8774/$ - see front matter 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.jfoodeng.2006.02.022
grades of tea during the tea grading process in tea industry. Statistical studies reveal that the size of tea granules is the only distinguishable parameters for sorting tea into different grades. As a result, the made-tea images are characterized as consisting of natural stochastic texture in accordance with variable sizes of tea granules. 1.1. Tea grading Sorting of CTC black tea into different grades in accordance with its size is a significant process in tea industries before quality evaluation. Tea is sorted in accordance with different sized tea granules by passing it over a series of vibrating sieves of different mesh sizes. Various grades of tea are resulted from different outlets of these sieves. These tea grades are categorized into four main groups, namely,
630
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
leaf, brokens, fannings and dust in descending order of particle size. They are traded under a wide variety of traditional names, for example BOP (Broken Orange Pekoe), BOPL (Broken Orange Pekoe Large), BOPSM (Broken Orange Pekoe Small), BP (Broken Pekoe), OF (Orange Fannings), PF (Pekoe Fannings), PD (Pekoe Dust), and Dust, etc. The whole sorting process is carefully monitored so as to assess the appropriate grades. Human sensory panel, supported by visual approximation have been traditionally used for this purpose in the tea industries. Certain linguistic terms are used in describing the sorted tea, which are significant in the monitoring process. Some of the most commonly used terms are as follows: • attractive or well-made: well made, uniform colour and size, • even: comprised of equal sizes of tea granules, • mixed: presence of different grades in one, • bold: pieces of leaves, which are too big for a grade, • stalky: Undue presence of stalk. Although it is widely pronounced as black tea, but the colour of tea is not black in true sense, rather it is almost blackish brown. A bright colour is always desirable as it represents the good quality tea and on the other hand unexpected dull colour is the sign of poor quality. The tea term ‘even’ means uniform sized tea granules in a particular tea grade; this is desirable in most of the cases. In some other cases different grades are intentionally mixed together to make some other variety and this is termed ‘mixed’. But the stalks are unwanted, termed as ‘stalky’, and these are the result of coarse plucking of tea shoots. Similarly the ‘bold’ is also unwanted as it is due to the presence of some big leaf in a particular grade. Such complicated phenomena demand the careful monitoring of the grading process to ensure the desired quality product.
Computer vision based analysis is one of the useful aspects in the event of discrimination of the textures. It employs certain advanced techniques for characterizing the complex size, shape, colour, texture, etc. The literatures suggest that the computer vision is advantageous in many applications of computer image analysis for classification, detection, or segmentation of images based on local spatial variations of intensity or colour (Pietikainen et al., 2000). The events of application of computer vision in food products for various prospects were reviewed by Brosnan and Sun (2002). It has been reported elsewhere that texture analysis plays a critical role in automatic grading systems, where the size of the object is one of the most important parameters. For example, Wu et al. (1995) used computer vision technique for on-line grading of herring roe. Similarly, foreign object detection in some food items is also an important texture analysis application in this area (Patel, Hannah, & Davies, 1994). A series of imaging and texture classifications techniques are explored using eight different ‘even’ tea grades from different tea gardens of Assam, India. Indeed the key aim of the work is to estimate the tea granules sizes in the images; work is carried out to differentiate the textures from one another. The phenomenon is subdivided into four sequential stages: image acquisition, image preprocessing, feature extraction and finally classification. A charge couple device (CCD) camera is used to capture the images maintaining the same distance from the objects, which makes the system computationally less complex. Moreover, the tea granules are spread with a special way for imaging. The images are stored in eight corresponding databases for off line analysis. Image texture features are extracted for discriminating the images in terms of the granule sizes of tea grades. An idea of conjugation of texture features along with the other classes of images is developed and implemented. The imaging techniques, feature extractions and classification results of the first phase of experiments are reported in this paper.
1.2. Tea image texture analysis 2. Materials and methods In the same way as in other food processing industries, tea researchers seek to modernize their grading monitoring process by applying ‘scientific methods’. The reason behind this is to satisfy a market, driven by customer demands, with their products with greater differentiation with enhanced quality. This is not always achievable using the conventional quality evaluations processes, which are typically based on judgments made by the sensory panel. In this respect, the tea industry is interested in the possibility of on-line monitoring of the sorting/grading process using for example computer vision, artificial olfaction and artificial tasting methods, etc. In the phenomenon of computer vision, while imaging different tea grades, images contain some patterns (texture). These textures are accomplished mainly due to various tea granule sizes, presence of different grades, presence of unexpected sized granule, and presence of stalk, etc.
2.1. Materials Eight varieties of the most common cut–tear–curl (CTC) tea grades produced in Assam, India are used for the experiments. The grades are BOPL, BOP BOPSM, BP, PF, PD, OF and Dust as described earlier. Table 1 gives details of the selected grades along with their approximate granule sizes. These grades are as per the norms of the Tocklai Tea Experimental Station, Jorhat, Assam, of India. 2.2. Imaging methods In order to get the best performance it is vital that the computer vision system is set up in the best way for the specific application. In the tea industry, for tea granule size estimation purposes, a large numbers of parameters need
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
631
Table 1 Tea grades along with approximate granule size (diameter) Grade names Approximate granule size (diameter)/mm
BOPL 2.0
BOP 1.7
to be accounted for. Some important physical parameters include the viewing distance, viewing direction, lighting condition, etc. Besides, some other parameters such as efficient data analysis capability, ability to make complex decisions such as dealing with overlapping tea granules, etc. are also important. In the case of human visual perception, these parameters are being adjusted automatically by the unique human ability during tea grading. The same parameters also need to be accounted for the case of computer vision based tea granule size estimation. For example, the granule sizes of the same granules may appear to be different in the images if the viewing distance is different, and different granule sizes may appear to be same for the same reason. Moreover, the variation in viewing direction and lighting conditions may make such variations in the images for the same tea granules. Therefore, these parameters are to be adjusted manually to increase the efficiency of the proposed computer vision system. In order to achieve this, the viewing distance, viewing direction and lighting conditions are held constant during the tea imaging process. On the other hand, the arrangement of the tea granules in the images is the other important part of the system performance and so, some adjustments are to be made during image capture. Special care is taken during the imaging of tea granules and their impacts on the image analysis are discussed in this section.
BOPSM 1.3
BP 1.0
PF 0.5
PD 0.355
OF 0.25
Dust Not specific
2.3. Arrangement of tea granules in the images While very few tea granules are spread on the floor or in a tray without any special effort to arrange them, they arrange themselves without having any definite order. In such particular cases, the sizes of the granules can be estimated by calculating the contour by considering each individual granule separately. But the key problem relates to the cases of overlapping tea granules, which is a common event in such imaging. Overlapping makes it very probable that wrong interpretations of the actual sizes of the granules will be made. This is because the computer vision system calculates the effective contour of the overlapped granules, which is different from the actual contour. Moreover, the extent of the overlapping of the granules or how many granules overlap in a particular position is quiet unpredictable. Therefore the system becomes complicated for estimating the sizes of the granules by using such scheme. Fig. 1 shows various possible arrangements of the tea granules (also showing some overlapping phenomenon) and segmented images of tea granules and estimation of the contours of the granules. It is observed in the figures that the contour lengths are different though the images are of the same sized tea granules. One more apparent disadvantage of this method of imaging is the non-uniformity of the tea granules size.
Fig. 1. (a) Tea granules; (b) segmented images; (c) contours of granules.
632
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
2.4. Image surface roughness due to tea granules Another method of imaging for the tea granule size estimation is by considering large numbers of tea granules and estimating the surface roughness. This is realized by using image texture analysis techniques. Fig. 2 shows images of the eight selected varieties of tea grades. Near uniform arrangements of the tea granules play an important role in estimating the surface roughness using texture analysis. Therefore, utmost care has been taken to ensure the maximum uniformity on the surface of overall arrangement of tea granules. The advantage of using such images is to compensate for any variations in the size of the tea granules and thus to minimize chances of misinterpretation of granule sizes. 2.5. Texture analysis method selection The texture analysis problems, such as texture segmentation, texture classification and texture primitive detection, etc., have been tried to be solved in many different ways in recent years. A wide variety of texture analysis techniques have been developed over the years (Reed & du Buf, 1993; Tuceryan & Jain, 1998). It was indicated in the introduction of the paper that the texture analysis method typically comprises of four sequential stages. These are image acquisition, image preprocessing, feature extraction and finally classification. Among them, the third stage is to compute a characteristic detail of an image that able to numerically describe its textual properties. This is the most significant stage, since it has the major impact on the overall performance of the texture analysis technique. In fact, the various approaches of texture analysis differ from each other mainly in terms of these methods for extracting textual features from the images.
It is found that various approaches to texture analysis are very diverse and in this respect, four categories can be defined (Tuceryan & Jain, 1998), which are namely statistical, geometrical, model based and signal processing. The first three texture analysis methods are found suitable for the regular and near regular texture analysis. But such regularity and near regularity is not achievable in the natural texture such as in the case of tea image textures. Literatures suggest that the signal processing methods are comparatively well suited to analyze the natural textures. Of these methods, Fourier transform (FT) based texture analysis approach considers only the global frequency content of images. It does not infer any spatial information about the image surface. But it is advantageous to obtain the spatial information of the texture along with the structure of the frequency content in the image for efficient texture classification. This phenomenon is achievable in some modified version of transforms. For example, in the case of windowed Fourier transform (WFT) or short-term Fourier transform (STFT), the image signal is multiplied by a window function, which generates the spatial information also. The Gabor transform (Wang & Asundi, 2000) is the one which is the most widely used in this category; where the window function is Gaussian. But the disadvantage of this technique is that it is not very flexible as the window size is fixed (fixed length). Once the window size is chosen for the WFT, the space–frequency resolution is fixed over the entire space–frequency plane. Therefore the spatial resolution at small scale and scale resolution at large scales are limited. Moreover, such a technique is computationally complex for applications such as the natural texture application as discussed here. On the other hand, the wavelet transform based technique performs the space–frequency decomposition with low computational complexity. This technique maintains
Fig. 2. Images if eight different tea grades with different granule sizes.
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
a flexible window width while frequency changes in the neighbourhood locations. The wavelet transform based texture analysis method has proven to be an efficient method for texture analysis due to its property of both space and frequency localization (Mallat, 1989) in this specified manner. The technique deals with the analysis of image data of different resolution. Subsequently, a wavelet-based method, which is often called wavelet texture analysis (WTA), is considered to be the state of the art texture analysis technique in the experiments. It shows better performance than other methods in many cases when applied to natural texture classifications in various different types of applications. Various wavelet bases can be found in literature in different texture analysis applications. These include Haar, Daubechies, Gabor, etc. Daubechies’ wavelet (Daubechies, 1988) is selected as bases due to its orthonormal characteristics. In comparison to the Haar wavelet, the Daubechies’ wavelet has continuous derivatives that respond well to discontinuities in the texture; while Haar wavelet does not allow the sharp transitions and fast attenuation. Moreover, Haar wavelet can not efficiently separate the image signals into low and high frequency sub-bands. On the other hand the Gabor filter uses the Gaussian window function, which is of fixed width for all the frequencies. One more disadvantage of Gabor filter is its high computational cost. But, in comparison to these, Daubechies constructed smooth scaling functions of compact support having orthonormal shifts and then applied the DWT method to obtain smooth orthogonal wavelets. Therefore these advantageous characteristic of Daubechies’ wavelet allow a compact coding of the image. Textured image analysis, comparison and segmentation have already been shown using Daubechies’ wavelets as a successful technique (Manian & Va´squez, 1998; Salari & Ling, 1995; Wang, Wiederhold, & Firschein, 1997). Salari and Ling (1995) showed better performance in texture segmentation by using Daubechies’ WT based technique. Wang et al. (1997) uses it for image indexing on the basis of texture variability and got better performance. Similarly, Manian and Va´squez (1998) showed that the Daubechies WT based texture analysis produces the best result in invariant texture classification. Based on these review, the fast wavelet transform (FWT), with a set of Daubechies’ wavelets, are used to decompose the gray valued tea images into different sub-band images. This is considered in the following sections of this paper.
633
bands of the pyramidal decomposed images. Four-level pyramidal decomposition of a tea image into its sub-band images is adopted. The variance, entropy, and energy are calculated for all the sub-band images of the different tea grades and used as feature vectors. They contain the information about the texture variations in the images. Thence having the information regarding the texture, in terms of feature vectors, intelligent system techniques are applied to model the classification system for discriminating between them. 3.1. Feature extraction The experimental procedure for tea image texture feature extraction method is as follows: • Tea images are preprocessed and the sizes of the images are made to be same for all images to make the system computationally less complex. • The Daubechies wavelet based low pass and high pass filters are designed. • A two dimensional FWT is applied to decompose the image into its sub-band images. Four-level pyramidal decomposition is used in this work. • All the sub-band images are stored for calculating the statistical features from the sub-bands. Fig. 3 shows Daubechies’ wavelet based decomposed four sub-bands that are created from one 128 · 128 tea image (gray scale). The sizes of the lower sub-bands have become 64 · 64, 32 · 32, 16 · 16 and 8 · 8, respectively, down the order. It is observed during the decomposition that the sub-bands such as dLH and dHL contain significantly much less energy, so they are ignored. Therefore dHH is the only sub-band used for the feature calculation. Therefore, there will be four sub-band images from four levels at different resolutions will be available for feature
3. Tea image texture analysis The two main stages of texture analysis namely feature extraction and analyses are discussed in this section. The algorithm described in this paper is developed in MATLAB using a combination of the Image Processing Toolbox, Wavelet toolbox (The Math works) for MATLAB; and Uvi_Wave version 3.0 for MATLAB. In the context of different texture analysis approaches, the technique employed adopts feature extraction from different sub-
Fig. 3. Four sub-bands are created from a 128 · 128 gray scale tea image.
634
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
vector calculation. During statistical feature calculation, if the decomposed sub-band image is f(x, y) with dimension (X, Y), then the various statistical features like mean, variance, entropy and energy feature vectors of the particular sub-band are calculated using the standard notations (Laine & Fan, 1993; Wang, Chen, Chein, & Tsai, 1998); which are as follows: Mean ¼
X X Y 1 X f ðx; yÞ XY x¼1 y¼1
Variance ¼ Energy ¼
X X Y X
1 ðXY Þ
Entropy ¼
jf ðx; yÞ Meanj2
ð2Þ
x¼1 y¼1
X X Y X
1 ðXY Þ
2
ð1Þ
2
jf ðx; yÞj
2
ð3Þ
x¼1 y¼1 X X Y X
1 ðXY Þ
2
2
jf ðx; yÞj logff ðx; yÞgj
reduce the vector dimension of the data set and thus considers only the most distinguishing patterns (principal components). The experiment is carried out among the eight different grades (refer Fig. 2) of tea and 160 different images are considered 20 each of each grade. Fig. 4 shows the three dimensional PCA plot of 160 different samples of eight different categories of tea grades in three dimensional (3D) spaces. It is observed from the figure that the PCA can not find any sharp distinction between the feature vectors though they are calculated from eight different sized tea granules.
2
ð4Þ
x¼1 y¼1
Of the four features above, the mean does not tell much about the variations of the elements in the matrices as different matrices might produce the same mean. But, on the other hand, the other statistical features, namely variance, energy and entropy are distinctive in nature for different matrices of different variations in elements. Therefore, these three features are considered to be potentially beneficial to be used as the texture features and are then calculated from all the selected sub-bands of the tea images.
3.2.2. SOM of the data The feature set is then used to find any possible clusters using Kohonen’s self organizing map (SOM) (Kohonen, 1990) based data clustering technique. This method adopts the competitive and unsupervised learning techniques. The basic idea of the method is to map the data pattern onto an ‘N dimensional grid’ of neurons or units. Fig. 5 represents the surf of the codebook generated by the SOM training
3.2. Data visualization 3.2.1. PCA of the data The principal component analysis (PCA) approach is selected for the visualization of the selected features and to help analysis the discriminating properties in them. There are four sub-band images of four different resolutions to be considered for feature vector calculation. Two different features energy and entropy are considered as the feature set in the first instance. The PCA seeks to
Fig. 5. SOM clustering results using the data.
Fig. 4. PCA plots of eight tea grades in three dimensional (3D) spaces.
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
using the same input data that were used for PCA calculation. It is observed that the SOM based technique has also failed to find any sharp distinction among the data set. The SOM clusters the dataset into more than 11 clusters though the features were extracted from only eight different categories of the images. 3.3. New feature extraction method It is observed in Section 3.2 that the feature set that is calculated in the specified manner is not distinctly separable by either data visualization technique PCA or clustering technique SOM. It is then worthwhile to try some other method for feature extraction technique to efficiently classify the data. Therefore a novel texture feature extraction technique is proposed here. The new feature extraction method consists of two distinct steps prior to the final feature extraction step. The first step is the existing feature extraction from the wavelet transform based sub-band images (the same as in Section 3.1) and the second is the estimate of the range of different groups of images. Three feature vectors – variance, entropy and energy are calculated for all the gray scale tea images in the database. The technique is developed using the same eight selected tea grades. Therefore eight different tea image databases are created consisting 20 images in each database. The Mahalanobis distances among the features of each images of each group are calculated for threshold measurement to estimate the range of the groups. 3.3.1. Mahalanobis distance measurement The Mahalanobis distance is a distance measure, which was introduced by P.C. Mahalanobis in 1936. It can be defined as the dissimilarity of two random vectors ~ x and ~ y of the same distribution and if C is their covariance matrix (Eq. (5)): qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 0 dð~ x;~ yÞ ¼ ð~ x ~ yÞ C 1 ð~ x; ~ yÞ ð5Þ Recently, Mahalanobis distance has become a common tool used in computer vision systems for comparing feature vectors, whose elements are quantities having different ranges and amounts of variation. It has been shown to be as a useful measure of similarity if some statistical properties of the texture features are known (Chang & Kuo, 1993). This is found to be one of the most effective methods for determining the dissimilarity of the set of features from an unknown image to the set of features measured from the collection of known images. This distance classifier does not infer which feature is the most important for discrimination of the textured images, but measures the dissimilarity between two different images. Since the Mahalanobis distance is measured in terms of standard deviation from the mean of the samples, the matching values provides a statistical measure of how the features of the test image match. If el denotes L feature out of the three adopted, mi is the mean of the decomposed sub-band of the image
635
of class ‘i’ and ci,l is the covariance of feature ‘l’ of class ‘i’ then the Mahalanobis distance (Md) is defined as 2 L X ðel mi Þ Md ¼ ð6Þ ci;l l¼1 Eq. (6) is used to measure the dissimilarity between two images in terms of extracted features. This provides the difference between two images in terms of texture variations. This dissimilarity measurement is carried out among the images of same group of the eight tea databases. 3.3.2. Threshold calculation for a group of images As the Mahalanobis distance measures similarity among the feature vectors, it has become useful for measuring the differences among the images. That is, to measure the amount of dissimilarity. The technique that is proposed here considers a range of different groups of images of the same granule size and finds the dissimilarities. Having the dissimilarity values (Mahalanobis distances) of a particular group of images, a threshold value is selected (either minimum or maximum). This threshold value is utilized to select the ranges of the groups and thus select the most significant images from the group. While choosing the minimum as the threshold, then it gives the significant image that lies in the middle of the group. On the other hand, choosing the maximum gives two images, which lie in the two extreme ends of the group. The minimum is considered as threshold in this paper, which gives only one image to be considered from each group of images. The technique can be described as follows. The first step is to calculate the ‘Md’ of every image with respect to the rest of the images of the same group. In doing so, a set of distance values is formed. The number of elements in this set to be calculated can easily be defined by using the combination formula, which calculates the number of ways of picking ‘k’ unordered outcomes from ‘n’ possibilities. In this case k = 2 and n = ‘number of images considered in a specific groups’. Then the method tries to determine the two most significant images having minimum ‘Md’ value, i.e., threshold. The significance of finding these images is that these images will be treated as the standard images of a particular database. Moreover, they have the minimum dissimilarity, which means they are the most similar images in the database. So either one image will be used to calculate the final feature set. The other standard images from other databases are also identified in the same manner and finally they are used to calculate the final feature set. Table 2 shows sample threshold calculation scheme considering five images in a particular group. Here images are considered to be I1, I2, I3, I4 and I5. The ‘Md’ is calculated for all the five images with respect to the rest of the images. The first step of this method is to calculate the ‘Md’ among the images using the existing texture features. ‘Mdxy’, in Table 2, indicates this calculation of ‘Md’ between images x and y, where x, y = 1, 2, . . . , 5 in this case.
636
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
Table 2 Sample threshold calculation schematic in a group of images I1 I1 I2 I3 I4 I5
I2
I3
I4
I5
Md12
Md13 Md23
Md14 Md24 Md34
Md15 Md25 Md35 Md45
The number of elements in the set is 5C2 = 10. The lower part of the table (opposite to the diagonal elements) also indicates the same set of values and so is ignored. The next step is to find the lowest valued element. This value is treated as the threshold and takes the corresponding two images are selected. For example if ‘Md23’ is the lowest value in the set then images I2 and I3 will be the two images to be considered as the most significant images in the group. Either I2 or I3 can be considered for the subsequent steps in the algorithm. So, if ‘m’ different groups of images are considered then ‘m’ numbers of images are to be considered for the final feature calculation. The value of ‘m’ is 8 (eight) in this paper. The experiment is carried out among the same 160 different images those were used for the PCA and SOM techniques (Sections 3.2.1 and 3.2.2). Group 1
Group 2
Group 3
Group 4
3.3.3. Final feature set calculation The last step of the method is the final feature calculation. The same ‘Md’ calculation is also carried out for this step but in this case with respect to the selected images only. Therefore, to calculate the texture feature for a particular image the following two steps are considered: First the statistical texture features are calculated. Then secondly, ‘Md’ is calculated with respect to each of the eight numbers of selected images from eight different groups. These resulting ‘Md’ values are considered as the final feature vectors of the particular image. The vector length itself will be eight for this feature extraction technique. Fig. 6 shows the schematic of this feature estimation technique. The significance of this feature set is that, all vectors contain information of entire databases of all different groups and at the same time contains fewer dimensions. Moreover, the advantage of such approach is that it enhances information about the texture significantly and minimizes any chance of misclassification. The PCA and SOM experiments for the new feature set are shown Section 3.3.4, which clarifies the above asserts. 3.3.4. Visualization of new feature set 3.3.4.1. PCA of new feature set. Fig. 7 shows the PCA plot of 160 different samples of eight different categories of tea Group 5
Group 6
Group 7
Group 8
Test image Fig. 6. Scheme of this new feature estimation technique.
Fig. 7. PCA plots of eight tea grades in 3D spaces using the new feature set.
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
637
new feature set. The performances of the networks are described in this section. 4.1. MLP
Fig. 8. SOM clustering of the new feature set.
grades in three dimensional (3D) spaces using the new feature set calculated in the previous section. The plot indicates the better performance of the new feature set in comparison to the previous feature set. It is observed that some categories of images form specific groups in the plot though some other groups are still not distinctly separable. 3.3.4.2. SOM clustering. The SOM technique is also applied to the new feature set to explore the clustering in them. Fig. 8 represents the surf of the codebook generated by the SOM. It is observed that the performance of the SOM based technique is enriched by using the new dataset. That is, the SOM can find eight distinct clusters in the dataset. This reveals the better performance of the new feature set in comparison to the original feature set as described in Section 3.1. Some small peaks are also found in the surf plot and these peaks may be the misclustering of some of the feature vectors due to their nearby similar characteristics.
The MLP uses the supervised training process phase as it is presented with training vectors together with the associated targets. A MLP network learns from the input data by adjusting the weights in the network using a specific learning algorithm. Many different training algorithms exist that can be applied to the MLP, but the most commonly used algorithm is error back propagation (Rumelhart, Hinton, & Williams, 1986). The purpose of this algorithm is to minimize the difference between the generated network output and the desired output, termed as error. The MLP network transforms the 8 input neurons (features) to 8 output neurons (8::8 network) using Bayesian Regulation back propagation. The network structure is shown in Fig. 9. The weights are trained with the error feed-forward back propagation algorithm. The activation functions for the neurons in the hidden layers (eight neurons in this case) employ the sigmoid function. It is observed that the network has very low computational complexity as training using 700 samples took <15 min on a PC with a 3 GHz CPU. While using 150 testing samples, the network results in 74.67% correct classification. On the other hand, while testing with the original feature set the accuracy achieved was just 46% with the same training and test samples. 4.2. LVQ Kohonen’s LVQ (Kohonen, 1990) is also tested, as with the MLP, using the same sample of images using the new
4. Data classification It was observed in Section 3.3.4 that the new feature set has efficiently distinguishable characteristics among them. The existing clusters can be distinctly visualized by PCA technique. It is also observed that the new feature set can be successfully clustered by the SOM. It is observed that though the data set were constructed for eight different categories of samples, some of them are nearly identical in nature. These facts are depicted for the classification section by using artificial neural network (ANN) technique. A final decision making stage is used to check if any application specific knowledge is available in the system, such as confidence thresholds, or risk associated with different misclassifications errors. This stage is also helpful to model a system to represent the knowledge of the feature set directly to the system. Two different algorithms, namely MLP and LVQ are selected from the literature (due to their reported ability to classify data) and implemented with the
Fig. 9. Architecture of MLP networks (8::8 networks).
638
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
Fig. 10. Architecture of the LVQ network.
feature set. The LVQ network is also a supervised classification algorithm. As a supervised method, LVQ uses known target output classifications for each input pattern of the form. This algorithm does not approximate density functions of class samples, but directly defines class boundaries based on prototypes, a nearest-neighbour rule and a winner-takes-it-all paradigm. It consists of two definite steps, the clustering step and the training step. The clustering step uses SOM architecture, which classifies the data with competitive learning. Then the network is trained with the winning neurons in supervised manner. Fig. 10 shows the LVQ network reproduced from neural network toolbox documentation of MATLAB [www.mathworks.com], which uses R: number of elements for the input vector (8); S1: number of competitive neurons (21); S2: number of linear neurons (8). While using the same 700 training samples and 150 testing samples, as used in MLP, the network results in 80% correct classification. This result outperforms the result obtained by the MLP network in the previous section. 5. Discussion and conclusions Tea sorting process into different grades in accordance with the size of tea granules is a very important process in tea processing industries. Human sensory panel, supported by visual approximation, has traditionally been maintaining the grading standard, and no computer vision approach has so far been reported. Tea research associations seek to modernize their quality monitoring process in scientific ways to satisfy a market driven by customer demands with products with greater differentiation. As a consequence, researchers explore the possibilities of on-line monitoring of the sorting/grading process using scientific methods. In view of the disadvantages of manual methods, a novel approach by using computer vision for tea granule size estimation is carried out in an on going project. This paper reports on work concerned with the analysis of tea image texture classification. In particular, this paper is dedicated towards finding an efficient texture feature as the existing features are not very efficient in estimating the size of the objects in the images. This size classifier
method is based on the surface roughness of the images. The method uses the Daubechies’ wavelet based decomposed sub-band images and existing useful feature vectors for calculating the new set of features. Eight different databases of images of eight different grades of tea are used to develop this system. The performance of the system is satisfactory as evident by the results obtained; although it appears to compromise computational complexity with higher number of distinctive groups. But such feature does not make much complexity with the lesser number of groups as the case of tea grades. This is because; there is not very high number of grades to be classified in tea industry. The method is found to be advantageous in terms of both accuracy and complexity in the problem specified. It is observed from the clustering techniques that some of the groups can not be clustered as definite cluster points, i.e., these groups merge with each other. This phenomenon was observed during the feature extraction step also. Some images fall into some other categories of images though they are grouped as different from each other. This is due to the phenomenon of having almost the same texture pattern though the size of the tea granules is slightly different from each other. Moreover, sub-bands with the first and second scale are found to be more sensitive to the nature of texture as evident from the energy contents. All the features have produced distinctive numerical values, which are effective in the discrimination of one class of image from other. Finally, such work aims towards exploring an efficient model for the determination of the sizes of tea granules. In this context the new feature set is being trained and classified using two well known neural network techniques, namely MLP and LVQ. It is observed that the system performance is satisfactory as evident from the experiments. For example up to 80% (highest) accuracy is obtained using the Kohonen’s LVQ network. On the other hand, the MLP resulted with 74.67% accurate classifications. But these results were significantly better than the results obtained while the usual feature sets are being used for classifications. That is to say that the new feature set resulted in significantly improved performance.
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
Acknowledgements The authors wish to thank the Commonwealth Scholarship Commission in London for their funding, support, etc. for S. Borah, without which the work would not have been possible. References Brosnan, T., & Sun, D. W. (2002). Inspection and grading of agricultural and food products by computer vision systems – a review. Computers and Electronics in Agriculture, 36(2–3), 193–213. Chang, T., & Kuo, C. C. J. (1993). Texture analysis and classification with tree-structured wavelet transform. IEEE Transactions on Image Processing, 2(4), 429–441. Chen, Y. Q., Nixon, M. S., & Thomas, D. W. (1994). Neural networks and texture classification. Applications of neural networks to signal processing (Digest No. 1994/248), IEE. Daubechies, I. (1988). Orthonormal bases of compactly supported wavelets. Communications on Pure and Applied Mathematics, 41, 909–996. http://www.mathworks.com. Kohonen, T. (1990). The self organising map. In Proceedings of the IEEE (special issue on Neural Network), 78(9), 1464–1480. Laine, A., & Fan, J. (1993). Texture classifications by wavelet packet signatures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(11), 1186–1191. Mallat, S. G. (1989). A theory for multi resolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7), 674–693. Manian, V., & Va´squez, R. (1998). Scaled and rotated texture classification using a class of basis functions. Pattern Recognition, 31(12), 1937–1948.
639
Patel, D., Hannah, I., & Davies, E. R. (1994). Foreign object detection via texture analysis. In International conference on pattern recognition, October 9–13 (Vol. 1, pp. 586–588). Pietikainen, M., Ojala, T., & Xu, Z. (2000). Rotation invariant texture classification using feature distributions. Pattern Recognition, 33, 43–52. Reed, T. R., & du Buf, J. M. H. (1993). Image understanding. Computer Vision, Graphics, and Image Processing, 57, 359–372. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536. Salari, E., & Ling, Z. (1995). Texture segmentation using hierarchical wavelet decomposition. Pattern Recognition, 28(12), 1819–1824. Tianhorng, C., & Kuo, C. C. J. (1993). Texture analysis and classification with tree-structured wavelet transform. IEEE Transactions on Image Processing, 2(4), 429–441. Tuceryan, M., & Jain, A. K. (1998). Texture analysis. In C. H. Chen, L. F. Pau, & P. S. P. S. P. Wang (Eds.), The handbook of pattern recognition and computer vision (2nd ed., pp. 207–248). World Scientific Publishing Co. Wang, J., & Asundi, A. K. (2000). A computer vision system for wineglass defect inspection via Gabor-filter based texture features. Information Sciences, 127, 157–171. Wang, J. W., Chen, C. H., Chein, W. M., & Tsai, C. M. (1998). Texture classification using non-separable two dimensional wavelets. Pattern Recognition Letters, 19, 1225–1234. Wang, J. Ze., Wiederhold, G., & Firschein, O. (1997). Wavelet based image indexing techniques with partial sketch retrieval capability. In Proceedings of the 4th forum on research and technology advances in digital libraries (pp. 13–24). Wu, Q. M., de Silva, C. W., Beatty, A., Cao, L. X., Gamage, L., Gosine, R., et al. (1995). Knowledge-based system for on-line grading of herring roe. In IEEE international conference on intelligent systems for the 21st century, 22–25 October (Vol. 4, pp. 3742–3747).
Wavelet transform based image texture analysis for size estimation applied to the sorting of tea granules S. Borah a, E.L. Hines a
a,*
, M. Bhuyan
b
Electrical and Electronics Division, School of Engineering, University of Warwick, Coventry CV4 7AL, UK b Department of Electronics, Tezpur University, Tezpur 784 028, Assam, India Received 11 October 2005; accepted 13 February 2006 Available online 29 March 2006
Abstract This paper describes a new texture feature estimation technique for discriminating images of eight different grades of CTC (cutting, tearing, and curling) tea. This new set of feature vectors can discriminate the images of different sized tea granules with more efficiency than the statistical feature vectors do. The technique conjugates the feature information of one group of images along with the information of rest of the groups. This is executed by considering range of different groups of images of the same granule size. Indeed, ranges are estimated using the existing statistical texture features, namely variance, entropy and energy, in difference form. Daubechies’ wavelets transform (WT) based sub-band images are utilized for calculating these statistical features. The techniques, for estimating these ranges and calculating the final feature set, adopt a simplified version of Mahalanobis distance calculation. Later, the data visualization method, principal component analysis (PCA), which is used to visualize the existing classes of textures, has found distinguishable characteristics among the new feature sets. It is further observed that the unsupervised clustering algorithm self organizing map (SOM) can classify the images efficiently into appropriate clusters. Two neural networks, namely multi-layer perceptron (MLP) network and learning vector quantization (LVQ) were used for texture classifications. The classification accuracy, for example 74.67% and 80% in MLP and LVQ, respectively, outperforms the other results obtained by using existing statistical texture features. 2006 Elsevier Ltd. All rights reserved. Keywords: Tea granule size; Wavelet transform; Texture feature; Mahalanobis distance
1. Introduction Efficient feature extraction is one of the most significant aspects of texture analysis in computer vision applications. Most existing feature extraction methods provide efficient tool for shape and pattern classification in the images (Chen, Nixon, & Thomas, 1994; Pietikainen, Ojala, & Xu, 2000; Tianhorng & Kuo, 1993). But the situation becomes complicated, while both size and shape of particles in the images come into the limelight. Such phenomenon was envisioned in discriminating images of different * Corresponding author. Tel.: +44 (0) 24 765 23246; fax: +44 (0) 24 764 18922. E-mail addresses: [email protected] (S. Borah), E.L.Hines@ warwick.ac.uk (E.L. Hines), [email protected] (M. Bhuyan).
0260-8774/$ - see front matter 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.jfoodeng.2006.02.022
grades of tea during the tea grading process in tea industry. Statistical studies reveal that the size of tea granules is the only distinguishable parameters for sorting tea into different grades. As a result, the made-tea images are characterized as consisting of natural stochastic texture in accordance with variable sizes of tea granules. 1.1. Tea grading Sorting of CTC black tea into different grades in accordance with its size is a significant process in tea industries before quality evaluation. Tea is sorted in accordance with different sized tea granules by passing it over a series of vibrating sieves of different mesh sizes. Various grades of tea are resulted from different outlets of these sieves. These tea grades are categorized into four main groups, namely,
630
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
leaf, brokens, fannings and dust in descending order of particle size. They are traded under a wide variety of traditional names, for example BOP (Broken Orange Pekoe), BOPL (Broken Orange Pekoe Large), BOPSM (Broken Orange Pekoe Small), BP (Broken Pekoe), OF (Orange Fannings), PF (Pekoe Fannings), PD (Pekoe Dust), and Dust, etc. The whole sorting process is carefully monitored so as to assess the appropriate grades. Human sensory panel, supported by visual approximation have been traditionally used for this purpose in the tea industries. Certain linguistic terms are used in describing the sorted tea, which are significant in the monitoring process. Some of the most commonly used terms are as follows: • attractive or well-made: well made, uniform colour and size, • even: comprised of equal sizes of tea granules, • mixed: presence of different grades in one, • bold: pieces of leaves, which are too big for a grade, • stalky: Undue presence of stalk. Although it is widely pronounced as black tea, but the colour of tea is not black in true sense, rather it is almost blackish brown. A bright colour is always desirable as it represents the good quality tea and on the other hand unexpected dull colour is the sign of poor quality. The tea term ‘even’ means uniform sized tea granules in a particular tea grade; this is desirable in most of the cases. In some other cases different grades are intentionally mixed together to make some other variety and this is termed ‘mixed’. But the stalks are unwanted, termed as ‘stalky’, and these are the result of coarse plucking of tea shoots. Similarly the ‘bold’ is also unwanted as it is due to the presence of some big leaf in a particular grade. Such complicated phenomena demand the careful monitoring of the grading process to ensure the desired quality product.
Computer vision based analysis is one of the useful aspects in the event of discrimination of the textures. It employs certain advanced techniques for characterizing the complex size, shape, colour, texture, etc. The literatures suggest that the computer vision is advantageous in many applications of computer image analysis for classification, detection, or segmentation of images based on local spatial variations of intensity or colour (Pietikainen et al., 2000). The events of application of computer vision in food products for various prospects were reviewed by Brosnan and Sun (2002). It has been reported elsewhere that texture analysis plays a critical role in automatic grading systems, where the size of the object is one of the most important parameters. For example, Wu et al. (1995) used computer vision technique for on-line grading of herring roe. Similarly, foreign object detection in some food items is also an important texture analysis application in this area (Patel, Hannah, & Davies, 1994). A series of imaging and texture classifications techniques are explored using eight different ‘even’ tea grades from different tea gardens of Assam, India. Indeed the key aim of the work is to estimate the tea granules sizes in the images; work is carried out to differentiate the textures from one another. The phenomenon is subdivided into four sequential stages: image acquisition, image preprocessing, feature extraction and finally classification. A charge couple device (CCD) camera is used to capture the images maintaining the same distance from the objects, which makes the system computationally less complex. Moreover, the tea granules are spread with a special way for imaging. The images are stored in eight corresponding databases for off line analysis. Image texture features are extracted for discriminating the images in terms of the granule sizes of tea grades. An idea of conjugation of texture features along with the other classes of images is developed and implemented. The imaging techniques, feature extractions and classification results of the first phase of experiments are reported in this paper.
1.2. Tea image texture analysis 2. Materials and methods In the same way as in other food processing industries, tea researchers seek to modernize their grading monitoring process by applying ‘scientific methods’. The reason behind this is to satisfy a market, driven by customer demands, with their products with greater differentiation with enhanced quality. This is not always achievable using the conventional quality evaluations processes, which are typically based on judgments made by the sensory panel. In this respect, the tea industry is interested in the possibility of on-line monitoring of the sorting/grading process using for example computer vision, artificial olfaction and artificial tasting methods, etc. In the phenomenon of computer vision, while imaging different tea grades, images contain some patterns (texture). These textures are accomplished mainly due to various tea granule sizes, presence of different grades, presence of unexpected sized granule, and presence of stalk, etc.
2.1. Materials Eight varieties of the most common cut–tear–curl (CTC) tea grades produced in Assam, India are used for the experiments. The grades are BOPL, BOP BOPSM, BP, PF, PD, OF and Dust as described earlier. Table 1 gives details of the selected grades along with their approximate granule sizes. These grades are as per the norms of the Tocklai Tea Experimental Station, Jorhat, Assam, of India. 2.2. Imaging methods In order to get the best performance it is vital that the computer vision system is set up in the best way for the specific application. In the tea industry, for tea granule size estimation purposes, a large numbers of parameters need
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
631
Table 1 Tea grades along with approximate granule size (diameter) Grade names Approximate granule size (diameter)/mm
BOPL 2.0
BOP 1.7
to be accounted for. Some important physical parameters include the viewing distance, viewing direction, lighting condition, etc. Besides, some other parameters such as efficient data analysis capability, ability to make complex decisions such as dealing with overlapping tea granules, etc. are also important. In the case of human visual perception, these parameters are being adjusted automatically by the unique human ability during tea grading. The same parameters also need to be accounted for the case of computer vision based tea granule size estimation. For example, the granule sizes of the same granules may appear to be different in the images if the viewing distance is different, and different granule sizes may appear to be same for the same reason. Moreover, the variation in viewing direction and lighting conditions may make such variations in the images for the same tea granules. Therefore, these parameters are to be adjusted manually to increase the efficiency of the proposed computer vision system. In order to achieve this, the viewing distance, viewing direction and lighting conditions are held constant during the tea imaging process. On the other hand, the arrangement of the tea granules in the images is the other important part of the system performance and so, some adjustments are to be made during image capture. Special care is taken during the imaging of tea granules and their impacts on the image analysis are discussed in this section.
BOPSM 1.3
BP 1.0
PF 0.5
PD 0.355
OF 0.25
Dust Not specific
2.3. Arrangement of tea granules in the images While very few tea granules are spread on the floor or in a tray without any special effort to arrange them, they arrange themselves without having any definite order. In such particular cases, the sizes of the granules can be estimated by calculating the contour by considering each individual granule separately. But the key problem relates to the cases of overlapping tea granules, which is a common event in such imaging. Overlapping makes it very probable that wrong interpretations of the actual sizes of the granules will be made. This is because the computer vision system calculates the effective contour of the overlapped granules, which is different from the actual contour. Moreover, the extent of the overlapping of the granules or how many granules overlap in a particular position is quiet unpredictable. Therefore the system becomes complicated for estimating the sizes of the granules by using such scheme. Fig. 1 shows various possible arrangements of the tea granules (also showing some overlapping phenomenon) and segmented images of tea granules and estimation of the contours of the granules. It is observed in the figures that the contour lengths are different though the images are of the same sized tea granules. One more apparent disadvantage of this method of imaging is the non-uniformity of the tea granules size.
Fig. 1. (a) Tea granules; (b) segmented images; (c) contours of granules.
632
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
2.4. Image surface roughness due to tea granules Another method of imaging for the tea granule size estimation is by considering large numbers of tea granules and estimating the surface roughness. This is realized by using image texture analysis techniques. Fig. 2 shows images of the eight selected varieties of tea grades. Near uniform arrangements of the tea granules play an important role in estimating the surface roughness using texture analysis. Therefore, utmost care has been taken to ensure the maximum uniformity on the surface of overall arrangement of tea granules. The advantage of using such images is to compensate for any variations in the size of the tea granules and thus to minimize chances of misinterpretation of granule sizes. 2.5. Texture analysis method selection The texture analysis problems, such as texture segmentation, texture classification and texture primitive detection, etc., have been tried to be solved in many different ways in recent years. A wide variety of texture analysis techniques have been developed over the years (Reed & du Buf, 1993; Tuceryan & Jain, 1998). It was indicated in the introduction of the paper that the texture analysis method typically comprises of four sequential stages. These are image acquisition, image preprocessing, feature extraction and finally classification. Among them, the third stage is to compute a characteristic detail of an image that able to numerically describe its textual properties. This is the most significant stage, since it has the major impact on the overall performance of the texture analysis technique. In fact, the various approaches of texture analysis differ from each other mainly in terms of these methods for extracting textual features from the images.
It is found that various approaches to texture analysis are very diverse and in this respect, four categories can be defined (Tuceryan & Jain, 1998), which are namely statistical, geometrical, model based and signal processing. The first three texture analysis methods are found suitable for the regular and near regular texture analysis. But such regularity and near regularity is not achievable in the natural texture such as in the case of tea image textures. Literatures suggest that the signal processing methods are comparatively well suited to analyze the natural textures. Of these methods, Fourier transform (FT) based texture analysis approach considers only the global frequency content of images. It does not infer any spatial information about the image surface. But it is advantageous to obtain the spatial information of the texture along with the structure of the frequency content in the image for efficient texture classification. This phenomenon is achievable in some modified version of transforms. For example, in the case of windowed Fourier transform (WFT) or short-term Fourier transform (STFT), the image signal is multiplied by a window function, which generates the spatial information also. The Gabor transform (Wang & Asundi, 2000) is the one which is the most widely used in this category; where the window function is Gaussian. But the disadvantage of this technique is that it is not very flexible as the window size is fixed (fixed length). Once the window size is chosen for the WFT, the space–frequency resolution is fixed over the entire space–frequency plane. Therefore the spatial resolution at small scale and scale resolution at large scales are limited. Moreover, such a technique is computationally complex for applications such as the natural texture application as discussed here. On the other hand, the wavelet transform based technique performs the space–frequency decomposition with low computational complexity. This technique maintains
Fig. 2. Images if eight different tea grades with different granule sizes.
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
a flexible window width while frequency changes in the neighbourhood locations. The wavelet transform based texture analysis method has proven to be an efficient method for texture analysis due to its property of both space and frequency localization (Mallat, 1989) in this specified manner. The technique deals with the analysis of image data of different resolution. Subsequently, a wavelet-based method, which is often called wavelet texture analysis (WTA), is considered to be the state of the art texture analysis technique in the experiments. It shows better performance than other methods in many cases when applied to natural texture classifications in various different types of applications. Various wavelet bases can be found in literature in different texture analysis applications. These include Haar, Daubechies, Gabor, etc. Daubechies’ wavelet (Daubechies, 1988) is selected as bases due to its orthonormal characteristics. In comparison to the Haar wavelet, the Daubechies’ wavelet has continuous derivatives that respond well to discontinuities in the texture; while Haar wavelet does not allow the sharp transitions and fast attenuation. Moreover, Haar wavelet can not efficiently separate the image signals into low and high frequency sub-bands. On the other hand the Gabor filter uses the Gaussian window function, which is of fixed width for all the frequencies. One more disadvantage of Gabor filter is its high computational cost. But, in comparison to these, Daubechies constructed smooth scaling functions of compact support having orthonormal shifts and then applied the DWT method to obtain smooth orthogonal wavelets. Therefore these advantageous characteristic of Daubechies’ wavelet allow a compact coding of the image. Textured image analysis, comparison and segmentation have already been shown using Daubechies’ wavelets as a successful technique (Manian & Va´squez, 1998; Salari & Ling, 1995; Wang, Wiederhold, & Firschein, 1997). Salari and Ling (1995) showed better performance in texture segmentation by using Daubechies’ WT based technique. Wang et al. (1997) uses it for image indexing on the basis of texture variability and got better performance. Similarly, Manian and Va´squez (1998) showed that the Daubechies WT based texture analysis produces the best result in invariant texture classification. Based on these review, the fast wavelet transform (FWT), with a set of Daubechies’ wavelets, are used to decompose the gray valued tea images into different sub-band images. This is considered in the following sections of this paper.
633
bands of the pyramidal decomposed images. Four-level pyramidal decomposition of a tea image into its sub-band images is adopted. The variance, entropy, and energy are calculated for all the sub-band images of the different tea grades and used as feature vectors. They contain the information about the texture variations in the images. Thence having the information regarding the texture, in terms of feature vectors, intelligent system techniques are applied to model the classification system for discriminating between them. 3.1. Feature extraction The experimental procedure for tea image texture feature extraction method is as follows: • Tea images are preprocessed and the sizes of the images are made to be same for all images to make the system computationally less complex. • The Daubechies wavelet based low pass and high pass filters are designed. • A two dimensional FWT is applied to decompose the image into its sub-band images. Four-level pyramidal decomposition is used in this work. • All the sub-band images are stored for calculating the statistical features from the sub-bands. Fig. 3 shows Daubechies’ wavelet based decomposed four sub-bands that are created from one 128 · 128 tea image (gray scale). The sizes of the lower sub-bands have become 64 · 64, 32 · 32, 16 · 16 and 8 · 8, respectively, down the order. It is observed during the decomposition that the sub-bands such as dLH and dHL contain significantly much less energy, so they are ignored. Therefore dHH is the only sub-band used for the feature calculation. Therefore, there will be four sub-band images from four levels at different resolutions will be available for feature
3. Tea image texture analysis The two main stages of texture analysis namely feature extraction and analyses are discussed in this section. The algorithm described in this paper is developed in MATLAB using a combination of the Image Processing Toolbox, Wavelet toolbox (The Math works) for MATLAB; and Uvi_Wave version 3.0 for MATLAB. In the context of different texture analysis approaches, the technique employed adopts feature extraction from different sub-
Fig. 3. Four sub-bands are created from a 128 · 128 gray scale tea image.
634
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
vector calculation. During statistical feature calculation, if the decomposed sub-band image is f(x, y) with dimension (X, Y), then the various statistical features like mean, variance, entropy and energy feature vectors of the particular sub-band are calculated using the standard notations (Laine & Fan, 1993; Wang, Chen, Chein, & Tsai, 1998); which are as follows: Mean ¼
X X Y 1 X f ðx; yÞ XY x¼1 y¼1
Variance ¼ Energy ¼
X X Y X
1 ðXY Þ
Entropy ¼
jf ðx; yÞ Meanj2
ð2Þ
x¼1 y¼1
X X Y X
1 ðXY Þ
2
ð1Þ
2
jf ðx; yÞj
2
ð3Þ
x¼1 y¼1 X X Y X
1 ðXY Þ
2
2
jf ðx; yÞj logff ðx; yÞgj
reduce the vector dimension of the data set and thus considers only the most distinguishing patterns (principal components). The experiment is carried out among the eight different grades (refer Fig. 2) of tea and 160 different images are considered 20 each of each grade. Fig. 4 shows the three dimensional PCA plot of 160 different samples of eight different categories of tea grades in three dimensional (3D) spaces. It is observed from the figure that the PCA can not find any sharp distinction between the feature vectors though they are calculated from eight different sized tea granules.
2
ð4Þ
x¼1 y¼1
Of the four features above, the mean does not tell much about the variations of the elements in the matrices as different matrices might produce the same mean. But, on the other hand, the other statistical features, namely variance, energy and entropy are distinctive in nature for different matrices of different variations in elements. Therefore, these three features are considered to be potentially beneficial to be used as the texture features and are then calculated from all the selected sub-bands of the tea images.
3.2.2. SOM of the data The feature set is then used to find any possible clusters using Kohonen’s self organizing map (SOM) (Kohonen, 1990) based data clustering technique. This method adopts the competitive and unsupervised learning techniques. The basic idea of the method is to map the data pattern onto an ‘N dimensional grid’ of neurons or units. Fig. 5 represents the surf of the codebook generated by the SOM training
3.2. Data visualization 3.2.1. PCA of the data The principal component analysis (PCA) approach is selected for the visualization of the selected features and to help analysis the discriminating properties in them. There are four sub-band images of four different resolutions to be considered for feature vector calculation. Two different features energy and entropy are considered as the feature set in the first instance. The PCA seeks to
Fig. 5. SOM clustering results using the data.
Fig. 4. PCA plots of eight tea grades in three dimensional (3D) spaces.
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
using the same input data that were used for PCA calculation. It is observed that the SOM based technique has also failed to find any sharp distinction among the data set. The SOM clusters the dataset into more than 11 clusters though the features were extracted from only eight different categories of the images. 3.3. New feature extraction method It is observed in Section 3.2 that the feature set that is calculated in the specified manner is not distinctly separable by either data visualization technique PCA or clustering technique SOM. It is then worthwhile to try some other method for feature extraction technique to efficiently classify the data. Therefore a novel texture feature extraction technique is proposed here. The new feature extraction method consists of two distinct steps prior to the final feature extraction step. The first step is the existing feature extraction from the wavelet transform based sub-band images (the same as in Section 3.1) and the second is the estimate of the range of different groups of images. Three feature vectors – variance, entropy and energy are calculated for all the gray scale tea images in the database. The technique is developed using the same eight selected tea grades. Therefore eight different tea image databases are created consisting 20 images in each database. The Mahalanobis distances among the features of each images of each group are calculated for threshold measurement to estimate the range of the groups. 3.3.1. Mahalanobis distance measurement The Mahalanobis distance is a distance measure, which was introduced by P.C. Mahalanobis in 1936. It can be defined as the dissimilarity of two random vectors ~ x and ~ y of the same distribution and if C is their covariance matrix (Eq. (5)): qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 0 dð~ x;~ yÞ ¼ ð~ x ~ yÞ C 1 ð~ x; ~ yÞ ð5Þ Recently, Mahalanobis distance has become a common tool used in computer vision systems for comparing feature vectors, whose elements are quantities having different ranges and amounts of variation. It has been shown to be as a useful measure of similarity if some statistical properties of the texture features are known (Chang & Kuo, 1993). This is found to be one of the most effective methods for determining the dissimilarity of the set of features from an unknown image to the set of features measured from the collection of known images. This distance classifier does not infer which feature is the most important for discrimination of the textured images, but measures the dissimilarity between two different images. Since the Mahalanobis distance is measured in terms of standard deviation from the mean of the samples, the matching values provides a statistical measure of how the features of the test image match. If el denotes L feature out of the three adopted, mi is the mean of the decomposed sub-band of the image
635
of class ‘i’ and ci,l is the covariance of feature ‘l’ of class ‘i’ then the Mahalanobis distance (Md) is defined as 2 L X ðel mi Þ Md ¼ ð6Þ ci;l l¼1 Eq. (6) is used to measure the dissimilarity between two images in terms of extracted features. This provides the difference between two images in terms of texture variations. This dissimilarity measurement is carried out among the images of same group of the eight tea databases. 3.3.2. Threshold calculation for a group of images As the Mahalanobis distance measures similarity among the feature vectors, it has become useful for measuring the differences among the images. That is, to measure the amount of dissimilarity. The technique that is proposed here considers a range of different groups of images of the same granule size and finds the dissimilarities. Having the dissimilarity values (Mahalanobis distances) of a particular group of images, a threshold value is selected (either minimum or maximum). This threshold value is utilized to select the ranges of the groups and thus select the most significant images from the group. While choosing the minimum as the threshold, then it gives the significant image that lies in the middle of the group. On the other hand, choosing the maximum gives two images, which lie in the two extreme ends of the group. The minimum is considered as threshold in this paper, which gives only one image to be considered from each group of images. The technique can be described as follows. The first step is to calculate the ‘Md’ of every image with respect to the rest of the images of the same group. In doing so, a set of distance values is formed. The number of elements in this set to be calculated can easily be defined by using the combination formula, which calculates the number of ways of picking ‘k’ unordered outcomes from ‘n’ possibilities. In this case k = 2 and n = ‘number of images considered in a specific groups’. Then the method tries to determine the two most significant images having minimum ‘Md’ value, i.e., threshold. The significance of finding these images is that these images will be treated as the standard images of a particular database. Moreover, they have the minimum dissimilarity, which means they are the most similar images in the database. So either one image will be used to calculate the final feature set. The other standard images from other databases are also identified in the same manner and finally they are used to calculate the final feature set. Table 2 shows sample threshold calculation scheme considering five images in a particular group. Here images are considered to be I1, I2, I3, I4 and I5. The ‘Md’ is calculated for all the five images with respect to the rest of the images. The first step of this method is to calculate the ‘Md’ among the images using the existing texture features. ‘Mdxy’, in Table 2, indicates this calculation of ‘Md’ between images x and y, where x, y = 1, 2, . . . , 5 in this case.
636
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
Table 2 Sample threshold calculation schematic in a group of images I1 I1 I2 I3 I4 I5
I2
I3
I4
I5
Md12
Md13 Md23
Md14 Md24 Md34
Md15 Md25 Md35 Md45
The number of elements in the set is 5C2 = 10. The lower part of the table (opposite to the diagonal elements) also indicates the same set of values and so is ignored. The next step is to find the lowest valued element. This value is treated as the threshold and takes the corresponding two images are selected. For example if ‘Md23’ is the lowest value in the set then images I2 and I3 will be the two images to be considered as the most significant images in the group. Either I2 or I3 can be considered for the subsequent steps in the algorithm. So, if ‘m’ different groups of images are considered then ‘m’ numbers of images are to be considered for the final feature calculation. The value of ‘m’ is 8 (eight) in this paper. The experiment is carried out among the same 160 different images those were used for the PCA and SOM techniques (Sections 3.2.1 and 3.2.2). Group 1
Group 2
Group 3
Group 4
3.3.3. Final feature set calculation The last step of the method is the final feature calculation. The same ‘Md’ calculation is also carried out for this step but in this case with respect to the selected images only. Therefore, to calculate the texture feature for a particular image the following two steps are considered: First the statistical texture features are calculated. Then secondly, ‘Md’ is calculated with respect to each of the eight numbers of selected images from eight different groups. These resulting ‘Md’ values are considered as the final feature vectors of the particular image. The vector length itself will be eight for this feature extraction technique. Fig. 6 shows the schematic of this feature estimation technique. The significance of this feature set is that, all vectors contain information of entire databases of all different groups and at the same time contains fewer dimensions. Moreover, the advantage of such approach is that it enhances information about the texture significantly and minimizes any chance of misclassification. The PCA and SOM experiments for the new feature set are shown Section 3.3.4, which clarifies the above asserts. 3.3.4. Visualization of new feature set 3.3.4.1. PCA of new feature set. Fig. 7 shows the PCA plot of 160 different samples of eight different categories of tea Group 5
Group 6
Group 7
Group 8
Test image Fig. 6. Scheme of this new feature estimation technique.
Fig. 7. PCA plots of eight tea grades in 3D spaces using the new feature set.
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
637
new feature set. The performances of the networks are described in this section. 4.1. MLP
Fig. 8. SOM clustering of the new feature set.
grades in three dimensional (3D) spaces using the new feature set calculated in the previous section. The plot indicates the better performance of the new feature set in comparison to the previous feature set. It is observed that some categories of images form specific groups in the plot though some other groups are still not distinctly separable. 3.3.4.2. SOM clustering. The SOM technique is also applied to the new feature set to explore the clustering in them. Fig. 8 represents the surf of the codebook generated by the SOM. It is observed that the performance of the SOM based technique is enriched by using the new dataset. That is, the SOM can find eight distinct clusters in the dataset. This reveals the better performance of the new feature set in comparison to the original feature set as described in Section 3.1. Some small peaks are also found in the surf plot and these peaks may be the misclustering of some of the feature vectors due to their nearby similar characteristics.
The MLP uses the supervised training process phase as it is presented with training vectors together with the associated targets. A MLP network learns from the input data by adjusting the weights in the network using a specific learning algorithm. Many different training algorithms exist that can be applied to the MLP, but the most commonly used algorithm is error back propagation (Rumelhart, Hinton, & Williams, 1986). The purpose of this algorithm is to minimize the difference between the generated network output and the desired output, termed as error. The MLP network transforms the 8 input neurons (features) to 8 output neurons (8::8 network) using Bayesian Regulation back propagation. The network structure is shown in Fig. 9. The weights are trained with the error feed-forward back propagation algorithm. The activation functions for the neurons in the hidden layers (eight neurons in this case) employ the sigmoid function. It is observed that the network has very low computational complexity as training using 700 samples took <15 min on a PC with a 3 GHz CPU. While using 150 testing samples, the network results in 74.67% correct classification. On the other hand, while testing with the original feature set the accuracy achieved was just 46% with the same training and test samples. 4.2. LVQ Kohonen’s LVQ (Kohonen, 1990) is also tested, as with the MLP, using the same sample of images using the new
4. Data classification It was observed in Section 3.3.4 that the new feature set has efficiently distinguishable characteristics among them. The existing clusters can be distinctly visualized by PCA technique. It is also observed that the new feature set can be successfully clustered by the SOM. It is observed that though the data set were constructed for eight different categories of samples, some of them are nearly identical in nature. These facts are depicted for the classification section by using artificial neural network (ANN) technique. A final decision making stage is used to check if any application specific knowledge is available in the system, such as confidence thresholds, or risk associated with different misclassifications errors. This stage is also helpful to model a system to represent the knowledge of the feature set directly to the system. Two different algorithms, namely MLP and LVQ are selected from the literature (due to their reported ability to classify data) and implemented with the
Fig. 9. Architecture of MLP networks (8::8 networks).
638
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
Fig. 10. Architecture of the LVQ network.
feature set. The LVQ network is also a supervised classification algorithm. As a supervised method, LVQ uses known target output classifications for each input pattern of the form. This algorithm does not approximate density functions of class samples, but directly defines class boundaries based on prototypes, a nearest-neighbour rule and a winner-takes-it-all paradigm. It consists of two definite steps, the clustering step and the training step. The clustering step uses SOM architecture, which classifies the data with competitive learning. Then the network is trained with the winning neurons in supervised manner. Fig. 10 shows the LVQ network reproduced from neural network toolbox documentation of MATLAB [www.mathworks.com], which uses R: number of elements for the input vector (8); S1: number of competitive neurons (21); S2: number of linear neurons (8). While using the same 700 training samples and 150 testing samples, as used in MLP, the network results in 80% correct classification. This result outperforms the result obtained by the MLP network in the previous section. 5. Discussion and conclusions Tea sorting process into different grades in accordance with the size of tea granules is a very important process in tea processing industries. Human sensory panel, supported by visual approximation, has traditionally been maintaining the grading standard, and no computer vision approach has so far been reported. Tea research associations seek to modernize their quality monitoring process in scientific ways to satisfy a market driven by customer demands with products with greater differentiation. As a consequence, researchers explore the possibilities of on-line monitoring of the sorting/grading process using scientific methods. In view of the disadvantages of manual methods, a novel approach by using computer vision for tea granule size estimation is carried out in an on going project. This paper reports on work concerned with the analysis of tea image texture classification. In particular, this paper is dedicated towards finding an efficient texture feature as the existing features are not very efficient in estimating the size of the objects in the images. This size classifier
method is based on the surface roughness of the images. The method uses the Daubechies’ wavelet based decomposed sub-band images and existing useful feature vectors for calculating the new set of features. Eight different databases of images of eight different grades of tea are used to develop this system. The performance of the system is satisfactory as evident by the results obtained; although it appears to compromise computational complexity with higher number of distinctive groups. But such feature does not make much complexity with the lesser number of groups as the case of tea grades. This is because; there is not very high number of grades to be classified in tea industry. The method is found to be advantageous in terms of both accuracy and complexity in the problem specified. It is observed from the clustering techniques that some of the groups can not be clustered as definite cluster points, i.e., these groups merge with each other. This phenomenon was observed during the feature extraction step also. Some images fall into some other categories of images though they are grouped as different from each other. This is due to the phenomenon of having almost the same texture pattern though the size of the tea granules is slightly different from each other. Moreover, sub-bands with the first and second scale are found to be more sensitive to the nature of texture as evident from the energy contents. All the features have produced distinctive numerical values, which are effective in the discrimination of one class of image from other. Finally, such work aims towards exploring an efficient model for the determination of the sizes of tea granules. In this context the new feature set is being trained and classified using two well known neural network techniques, namely MLP and LVQ. It is observed that the system performance is satisfactory as evident from the experiments. For example up to 80% (highest) accuracy is obtained using the Kohonen’s LVQ network. On the other hand, the MLP resulted with 74.67% accurate classifications. But these results were significantly better than the results obtained while the usual feature sets are being used for classifications. That is to say that the new feature set resulted in significantly improved performance.
S. Borah et al. / Journal of Food Engineering 79 (2007) 629–639
Acknowledgements The authors wish to thank the Commonwealth Scholarship Commission in London for their funding, support, etc. for S. Borah, without which the work would not have been possible. References Brosnan, T., & Sun, D. W. (2002). Inspection and grading of agricultural and food products by computer vision systems – a review. Computers and Electronics in Agriculture, 36(2–3), 193–213. Chang, T., & Kuo, C. C. J. (1993). Texture analysis and classification with tree-structured wavelet transform. IEEE Transactions on Image Processing, 2(4), 429–441. Chen, Y. Q., Nixon, M. S., & Thomas, D. W. (1994). Neural networks and texture classification. Applications of neural networks to signal processing (Digest No. 1994/248), IEE. Daubechies, I. (1988). Orthonormal bases of compactly supported wavelets. Communications on Pure and Applied Mathematics, 41, 909–996. http://www.mathworks.com. Kohonen, T. (1990). The self organising map. In Proceedings of the IEEE (special issue on Neural Network), 78(9), 1464–1480. Laine, A., & Fan, J. (1993). Texture classifications by wavelet packet signatures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(11), 1186–1191. Mallat, S. G. (1989). A theory for multi resolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7), 674–693. Manian, V., & Va´squez, R. (1998). Scaled and rotated texture classification using a class of basis functions. Pattern Recognition, 31(12), 1937–1948.
639
Patel, D., Hannah, I., & Davies, E. R. (1994). Foreign object detection via texture analysis. In International conference on pattern recognition, October 9–13 (Vol. 1, pp. 586–588). Pietikainen, M., Ojala, T., & Xu, Z. (2000). Rotation invariant texture classification using feature distributions. Pattern Recognition, 33, 43–52. Reed, T. R., & du Buf, J. M. H. (1993). Image understanding. Computer Vision, Graphics, and Image Processing, 57, 359–372. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536. Salari, E., & Ling, Z. (1995). Texture segmentation using hierarchical wavelet decomposition. Pattern Recognition, 28(12), 1819–1824. Tianhorng, C., & Kuo, C. C. J. (1993). Texture analysis and classification with tree-structured wavelet transform. IEEE Transactions on Image Processing, 2(4), 429–441. Tuceryan, M., & Jain, A. K. (1998). Texture analysis. In C. H. Chen, L. F. Pau, & P. S. P. S. P. Wang (Eds.), The handbook of pattern recognition and computer vision (2nd ed., pp. 207–248). World Scientific Publishing Co. Wang, J., & Asundi, A. K. (2000). A computer vision system for wineglass defect inspection via Gabor-filter based texture features. Information Sciences, 127, 157–171. Wang, J. W., Chen, C. H., Chein, W. M., & Tsai, C. M. (1998). Texture classification using non-separable two dimensional wavelets. Pattern Recognition Letters, 19, 1225–1234. Wang, J. Ze., Wiederhold, G., & Firschein, O. (1997). Wavelet based image indexing techniques with partial sketch retrieval capability. In Proceedings of the 4th forum on research and technology advances in digital libraries (pp. 13–24). Wu, Q. M., de Silva, C. W., Beatty, A., Cao, L. X., Gamage, L., Gosine, R., et al. (1995). Knowledge-based system for on-line grading of herring roe. In IEEE international conference on intelligent systems for the 21st century, 22–25 October (Vol. 4, pp. 3742–3747).