- Email: [email protected]
Burning state recognition of rotary kiln using ELMs with heterogeneous features
Neurocomputing 102 (2013) 144–153
Contents lists available at SciVerse ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Burning state recognition of rotary kiln using ELMs with heterogeneous features Weitao Li a, Dianhui Wang b,a,n, Tianyou Chai a a b
State Key Laboratory of Synthetical Automation of Process Industries, Northeastern University, Shenyang 110004, China Department of Computer Science and Computer Engineering, La Trobe University, Melbourne, VIC 3086, Australia
a r t i c l e i n f o
a b s t r a c t
Available online 19 June 2012
Image based burning state recognition plays an important role in sintering process control of rotary kiln. Although many efforts on dealing with this problem have been made over the past years, the recognition performance cannot be satisfactory due to the disturbance from smoke and dust inside the kiln. This work aims to develop a reliable burning state recognition system using extreme learning machines with heterogeneous features. The recorded flame images are firstly represented by various low-level features, which characterize the distribution of the temperature field and the flame color, the local and global configurations. To learn the merits of our proposed flame image-based burning state recognition system, four learner models (ELM, MLP, PNN and SVM) are examined by a typical flame image database with 482 images. Simulation results demonstrate that the heterogeneous features based ELM classifiers outperform other classifiers in terms of both recognition accuracy and computational complexity. & 2012 Elsevier B.V. All rights reserved.
Keywords: Burning state Multivariate image analysis Eigen-flame image Latent semantic analysis ELM
1. Introduction Rotary kiln, as a large-scale sintering facility, is widely used in metallurgical, cement, chemical, and environment protection industries. The main control objective of the rotary kiln sintering process is to achieve consistent product quality, which is often referred to as the key performance index. However, practically, the measurement of the product quality index is done by manual sampling with 1-h period. Therefore, indirect control is employed to replace online control, i.e. keeping key process parameters that can be measured online and are closely related to the product quality index into their preset ranges means satisfied product quality index. Based on the analysis of rotary kiln mechanism process, the fact that burning zone temperature directly determines the characteristics of the clinker is widely acknowledged. Thus, the accurate measurement for such temperature is the most critical issue for the rotary kiln sintering control process [1–3]. However, due to the harsh environment inside the kiln, the accurate measurement through thermocouple is still a challenging task. Recently, we have developed and implemented a hybrid control system in No. 3 rotary kiln at Shanxi Aluminum Corp. [4]. In such system, burning state is recognized based on the
n Corresponding author at: Department of Computer Science and Computer Engineering, La Trobe University, Melbourne, VIC 3086, Australia. E-mail address: [email protected] (D. Wang).
0925-2312/$ - see front matter & 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.neucom.2011.12.047
clustering of temperatures from the non-contact colorimetric measuring device. Because of the rich and reliable visual information, for operators, burning zone flame image is considered to be more reliable than the burning zone temperature to estimate the burning state. Flame image-based state recognition has already been studied in the past, where a flame image is first segmented into regions of interest (ROIs), features for the representation of the color and configuration characteristics of these regions are then extracted, burning state recognition is performed based on the features extracted [5–7]. However, due to the poor image quality caused by smoke and dust inside the kiln, accurate segmentation of ROIs is quite challenging and therefore unreliable. This will in turn result in inaccurate feature extraction and poor state recognition. To avoid the above problems, we have tried to extract features to represent the color and configuration characteristics of ROIs of flame image without segmentation, with the goal of improving the burning state recognition. From operators point of view, more discriminable ROIs will facilitate the subsequent feature extraction and burning state recognition. Motivated by the knowledge that flame and material zones are with distinct texture characteristics, Gabor filter is employed as a pre-processing step to discriminate them [8]. Practically, its parameters are often set by trial and error. However, we believe most of a filter bank offer little improvement to (or even reduce) the discriminative power due to the peaking phenomenon [9]. Hence, we propose to incorporate Mahalanobis
W. Li et al. / Neurocomputing 102 (2013) 144–153
measure [10] and forward selection technique [9] to automatically generate a compact Gabor filter bank to enhance the separability of ROIs to facilitate the sequel. Flame color indicates the combustion region and the distribution of the temperature field, and hence exhibits an intuitive impression for the burning state. Appropriate flame region with similar color corresponds normal burning state. Unlike other flame image analysis methods that track the turbulent flame in the image space to extract features to represent the flame color, the multivariate image analysis (MIA) technique [11] shows its efficiency to feature the flame color without the difficult locating flame step. MIA lies in projecting image pixels with similar color in a common region of the score space independently of their spatial location, and retrieving the locations of pixels with similar color in the image space after detecting feature in the score space. Such extracted feature will be used to represent the flame color. The configuration of ROIs characterizes the heat source, disturbance from smoke and dust, and clinker sintering status. Especially, the configuration of the flame zone and the height of the material zone are the key factors to recognize the burning state [5,6]. Flame zone with good circularity and appropriate material height mean normal burning state. Due to the difficult segmentation of ROIs, instead, global features are firstly extracted to represent the global configuration of the flame image. Eigenflame images are obtained using principal component analysis (PCA), and global features are then produced by correlating each flame image with the eigen-flame images to represent the global configuration. Unlike traditional selection criterion for eigen-images [12], a new selection procedure is used, with the goal of selecting global features that possess the maximum discriminative power. Generally, local configuration is considered to contain more valuable details to complement the global configuration. Scale invariant feature transform (SIFT) operator [13] is hence employed to extract key points of flame image to avoid the segmentation. The dimension of a SIFT descriptor is 1 128. Exploring research in image and text retrieval, ‘‘bag of visual words’’ (BoVW) [14] and term frequency-inverse document frequency weight [15] are applied to vector quantize the descriptors into clusters and form a visual word-image table to reduce feature representation dimension. For such table, latent semantic analysis (LSA) [16] is used to map such visual word-image space to a latent semantic space by taking advantage of some implicit higher-order structure in associations of visual words with images to mitigate potential zero-frequency problem and reduce feature dimension further [17]. Now, semantic vector as local feature conceptually represents the local configuration. Previously, semantics are explicitly assumed to have same saliency. In our work, a new semantic selection procedure is introduced in order to consider the saliency of semantics to select local features with maximum discriminative power. To imitate the fact that the integration of multi-feature is used to estimate the final burning state, the above individual features are concatenate and normalized, and a single-hidden-layer feed forward neural networks (SLFNs) classifier with extreme learning machine (ELM) algorithm [18] is employed to recognize the burning state. Different from conventional learning algorithms for neural networks, with randomly chosen input weights and hidden bias and calculated output weights, ELM not only trains much faster with higher generalization ability, but also overcomes many issues faced by gradient-based algorithms such as stopping criteria, learning rate, learning epochs, and local minima. Many types of hidden nodes including additive/RBF hidden nodes, multiplicative nodes, and non-neural alike nodes, can be used as long as they are piecewise nonlinear. Readers may refer to a recent survey paper for more details on ELM [19]. In ELM, since different hidden node parameters correspond to different
145
classification performance, the selection of the hidden node number is the most critical issue. Recently, due to the universal approximation capability, the minimum training error and weight norm, diverse modification for ELM has been successfully applied to many classification problems [20] but has never been used in flame image recognition before. The advantages of our new flame image-based burning state recognition method are fourfold. Firstly, our new method is computationally more efficient and more accurate and robust than the image segmentation-based and temperature-based methods. Secondly, MIA is effective to feature the flame color to avoid the difficult flame tracking. Thirdly, eigen-flame image-based method is feasible to feature the global configuration to avoid the difficult segmentation. Fourthly, without segmentation, local configuration is effectively featured by semantic vector. Numerous experimental studies show that, with feasible ELM classifier, our new method outperforms the image segmentation-based methods and temperaturebased method. As we can expect, more consistent product quality index can be achieved if the new burning state recognition method is incorporated into our previously developed hybrid control system. The rest of the paper is organized as follows. The rotary kiln sintering process and weaknesses of previous burning state recognition methods are presented in Section 2. Section 3 gives our new flame image multi-feature-based burning state recognition method. Experimental studies, conclusions and future work are given in Sections 4 and 5 respectively.
2. Rotary kiln sintering process and previous burning state recognition methods 2.1. Rotary kiln sintering process A schematic diagram of the rotary kiln sintering process is shown in Fig. 1, where raw material slurry is sprayed into the rotary kiln the upper end, i.e. kiln tail. At the lower end, i.e. kiln head, coal powders from the coal feeder and primary air from the air blower are mixed into a bi-phase fuel flow and then are sprayed into the kiln head hood and combust with secondary air from the cooler. The heated gas is brought to the kiln tail by the induced draft fan, while the material moves to the kiln head by the rotation of kiln and its gravity, in counter direction of the gas flow. After the material passes through drying zone, pre-heating zone, decomposing zone, burning zone, and cooling zone in sequence, the final product of the sintering process of rotary kiln, namely clinker, is generated and is fed downstream for further processing [21]. Taking alumina sintering process for instance, during burning zone, with 1200–1300 1C, the following chemical reaction arises [21]: Na2 O Al2 O3 2Si O2 þ4CaO-Na2 O Al2 O3 þ2ð2CaO Si O2 Þ cyclone dust collector coal feeder primary air
air blower
temperature measure
electric dust collector
induced draft fan returned dust rotary kiln pre-heating zone
burning kiln zone head decomposing hood cooling zone zone
cooler
kiln tail hood drying zone
secondary air sinter, to the next procedure
Fig. 1. Schematic diagram of rotary kiln sintering process.
ð1Þ
to the chimney
from raw material slurry feeder
146
W. Li et al. / Neurocomputing 102 (2013) 144–153
Si O2 þ2CaO-2CaO Si O2
ð2Þ
Ti O2 þCaO-CaO Ti O2
ð3Þ
At the outlet of the cooler, the final product quality index, i.e. clinker unit weight CW, is measured manually. With appropriate burning zone temperatures TBZ, satisfied CW (1.20–1.45 kg/L [22]) will be produced. Since both higher and lower TBZ will adversely affect CW [22], hence, the accurate measurement of TBZ becomes the most critical issue for the product quality control of the rotary kiln sintering process. 2.2. Previous burning state recognition methods In previous studies, the recognition of the burning state for rotary kiln is mainly based on three ways. Firstly, primitively, via the peephole of kiln head, operators observe the burning zone to estimate burning state. However, the performance of such way is limited by operators’ experience, mental state and responsibility. As a result, inconsistent estimation result by different operators will lead to inconsistent clinker quality. Secondly, a number of thermocouples are inserted through the kiln shell along the kiln to evaluate the burning state. However, this method is considered unreliable by operators, since fouling occurs on the thermocouples located in the most interesting region (i.e. burning zone). Recently, colorimetric measuring device is used in our developed control system [4]. Unfortunately, due to the disturbance from dust and smoke, such a single point temperature is often far from its true value and often exhibits large fluctuation and severe noise. To alleviate the problems involved in the temperature measurement and imitate the burning state estimation mode of experienced operators, the third way based on the burning zone flame
image is studied recently. Because of a wealth of temperature field and clinker sintering information, flame image is considered to be more reliable for estimating burning state by operators. As shown in Fig. 2, a flame image consists of a few parts including kiln wall, coal zone, material zone and flame zone. Coal zone is formed by coal powders from the coal burner; flame zone is formed by instant explosion and combustion of mixed coal powders and air; material zone is formed by the sintering of raw material slurry. For operators, the flame color and configuration of ROIs are the key factors in the recognition of three burning states, i.e. overburning state, under-burning state, and normal-burning state [5,6]. Previously, before adopting features to represent the above factors, ROIs are firstly obtained by image segmentation techniques. However, since turbulent flame bounces around, brightness of material zone results from flame zone, and dust and smoke couple ROIs, tracking the accurate boundary of ROIs is a challenging task. Fig. 3 shows the experimental studies based on several image segmentation methods [5,23–28]. As we can see, difficult segmentation will in turn lead to inaccurate feature extraction and poor burning state recognition result. To avoid the problems listed above, we try to extract features to represent the color and configuration of flame image without the difficult step of segmentation. In this paper, three feature extraction methods corresponding to flame color, global configuration of ROIs, and local configuration of ROIs are employed respectively. The above individual features will be concatenate to a feature vector, and then fed into the pattern classifier to obtain the final burning state recognition result. Flow chart of our proposed method is shown in Fig. 4. Details of each method are described as follows.
3. Robust burning state recognition based on heterogeneous feature 3.1. Pre-processing of flame image
Fig. 2. A burning zone flame image.
For the Gabor filter pre-processing phase, due to the camera placement providing a rough estimate for the range of ROIs, two 25 25 fixed windows are used to sample the flame and material zones to avoid the segmentation issue as shown in Fig. 5. Assume a total of 2T r flame and material texture images sampled from Tr grayscale transformed images of the training RGB flame images: T 1 ,T 2 , . . . ,T 2T r . Let z1 ,z2 , . . . ,znG denote feature groups extracted from filtered texture images by the nG ðnG ¼ 64Þ initial Gabor filters, where zk ¼ ½z1,k ,z2,k , . . . ,z2T r ,k T , k ¼ 1, . . . ,nG , and p parameters of initial Gabor ffiffiffiffiffiffiffiffiffiffiffi filter bank are set as f m ¼ g=ð2g þ 2 log 2=pÞ, nf ¼4, no ¼4,
Fig. 3. Performance of image segmentation methods: (a) flame image; (b) Ostu [23]; (c) Fuzzy c mean and Gabor wavelet (FCMG) [5]; (d) Fuzzy c mean (FCM) [24]; (e) normalized cuts [25]; (f) multistage adaptive threshold (MAT) [26]; (g) B-snake [27]; (h) minimax [28].
W. Li et al. / Neurocomputing 102 (2013) 144–153
Input flame images
Class 1 . . .
Class 3
Pre–processing
Gabor filter
Optimal filter bank
Gabor filter subset
Feature extraction
147
Classification
MIA MIA (for color)
f1
PCA (forPCA global configuration)
f2
SIFT+BoVW+LSA SIFT+BoW+LSA (for local configuration)
f3
L e a r n i n g
ELM
Feature vector F
Feature extraction
Trained classifier
New images
T e s Burning t state i n g
Fig. 4. Flow chart of the burning state recognition method.
of the unstructured noise and extract more meaningful ROIs from the original flame image. MIA is equivalent to unfold the threeway matrix I into a two-way matrix I without considering the spatial coordinates of pixels, and then performs PCA on it unfold
T I F r F c 3 - I ðF r F c Þ3 ¼ SPC a ¼ 1 t a pa þ E
Fig. 5. Fixed windows of flame image.
g ¼ 0:5,1:0, and Z ¼ 0:5,1:0 based on [29]. For each texture image, the mean m and standard deviation s features are extracted, i.e. zw,k ¼ ½mw,k , sw,k . Mahalanobis separability measure JM(k) is employed as the metric function to evaluate and sort the discriminative power of zk and associated filters defined as follows: T J M ðkÞ ¼ ðmf ,k mm,k ÞC 1 k ðmf ,k mm,k Þ
ð4Þ
where mf ,k , mm,k , and C k denote mean vector and covariance matrix of flame class and material class in feature space along feature group zk respectively. In our study, such metric is combined with a forward selection technique to automatically select uncorrelated feature groups and associated Gabor filters to best distinguish flame and material zones much more to facilitate the sequel. This method has been successfully applied to standard texture image analysis, but not on flame image before. See [30] for details. Once the compact bank with ns filters for training gray-scale image dataset is selected, they will be applied to the R, G, and B channels sub-image of each original flame image respectively. Then, for each training image I e , the mean image I e 0 of ns filtered images is used to replace I e to form a new training image for further processing. 3.2. Sub-feature to represent flame color An RGB flame image I has F r F c 3 pixels, and each pixel is quantized by the 0–255 integers in its R, G, and B channels. MIA technique is based on multiway PCA, which can eliminate much
ð5Þ
where a (PC ¼3 for red, green, and blue spectral channels) indexes the principal components, t a is a ðF r F c Þ 1 score vector, and pa is a 3 1 loading vector. After scaling and rounding off from 0 to 255, t a is denoted as sa and can be refolded into the original image size and displayed as an image. For RGB image, the first two score vectors usually explain 99% of the total variance. Inspection of the t 1 t 2 score plot is a common tool in PCA to detect clusters or outliers. However, due to the numerous pixels, an analysis from the compressed score space is more important [31]. A 256 256 score plot histogram SS obtained from s1 and s2 is hence employed to describe the compressed score space, where each element is computed as SSm,n ¼ St 1
ð8t,s1, t ¼ m,s2, t ¼ n,m,n ¼ 0, . . . ,255Þ
ð6Þ
In such plot, pixels with similar color in the image space are clustered together in the score space, and the locations of pixels in the score plots change significantly as the burning states change, which enables us to use masking to obtain ROIs of flame image whose pixels have similar color in the image space to distinguish various burning states shown in Fig. 6. According to [31], a 256 256 binary matrix M as shown in Fig. 6 is constructed to extract the area feature f 1 to characterize the color of the training flame image, where each element is defined as follows: f 1 ¼ Sm,n SSm,n
ð8m,n,M m,n ¼ 1Þ
ð7Þ
3.3. Sub-feature to represent global configuration of ROIs Eigen-flame image decomposition based on PCA is applied to extract global features. Suppose E1 ,E2 , . . . ,ET r denote eigen-flame images after applying PCA to the new generated training dataset. Due to the correlation coefficients between an image and the eigenflame images can be considered as global feature to represent the global configuration, the selection for the eigen-flame images is hence quite important. Unlike traditional selection criterion that is
148
W. Li et al. / Neurocomputing 102 (2013) 144–153
Fig. 6. (a) Original flame image. (b) Score plot of flame image with masking (yellow). (c) Flame image with overlay of highlighted pixels. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this article.)
LSA is hence employed to merge visual words with similar meanings to mitigate such problem. LSA requires the singular value decomposition (SVD) to generate a semantic space that represents conceptual visual word-image associations, which can be written as N Nv T r ¼ U Nv Nv RNv Nv V TT r Nv
ð9Þ
where U, V, and R are the matrices of visual word vectors, image vectors, and singular values. Previously, the best approximation of N with rank-l is given by selecting the first l values of R and associated vectors from U and V ~ V~ T U RV T ¼ N N~ l ¼ U~ ll R ll T r l Fig. 7. Detected SIFT keypoints for a flame image.
optimal from a low dimensional reconstruction viewpoint, Fisher ratio [32] is employed as the metric function to evaluate the class separability of Eq defined as PN C
i¼1
J F ðqÞ ¼
PN C
j¼1
9g i,q g j,q 9 2
2
2
hi,q þ hj,q N C ðNC 1Þ
ðq ¼ 1, . . . ,T r Þ
ð8Þ
where g i,q , g j,q , hi,q , hj,q , and NC denote mean and standard deviation of flame image class i and class j correlated with Eq and the image class number respectively. Once JF of all Eq are evaluated, the eigen-flame images selection, i.e. extraction of global feature f 2 , can be carried on. Followed by the same procedure, a testing flame image is handled to extract the global feature. This eigen-flame image method has been used to flame image, but without the pre-processing step. Refer to [33] for details. 3.4. Sub-feature to represent local configuration of ROIs SIFT operator is superior to others in local feature detection and description. Fig. 7 shows the detected SIFT keypoints for a flame image. To avoid the high-dimensional SIFT descriptor, ‘‘BoVW’’ is employed to construct the vocabulary via K-means clustering from D SIFT descriptors of Tr training images. Such Nv ðN v 5DÞ clustered vectors is considered as ‘‘visual words’’, and then descriptors of every image are assigned to the nearest cluster to form a co-occurrence table N Nv T r . Essentially, N records the frequency of ‘‘visual words’’ actually appearing in each flame image. The tf-idf weight is said to be superior to the frequency list by N to evaluate the importance of a visual word, and is hence applied to form a new visual word-image table N Nv T r , where each row xr for a visual word whilst each column ye for an image. In the matrix N, many zero entrices present and those nonzero entrices represent the content of images. Thus, similarity metrics between images will be evaluated by counting the synonymy given in the table lists of visual words in images.
ð10Þ
Since local configuration can be considered to be conceptually featured by semantic vectors and the essence of LSA is from a low dimensional reconstruction viewpoint, semantic selection should be carried out in order to take into account the saliency of various semantics to generate the best local feature. For each column of V, Mahalanobis measure is used to evaluate and sort the discriminative power of semantic vector vr , r ¼ 1, . . . ,Nv in the sense of Nv visual words: PNC PNC ~ 1 ~ ~ ~ ~ T i¼1 j ¼ 1 ðm i,r m j,r ÞC i,j,r ðm i,r m j,r Þ ð11Þ JM ðrÞ ¼ NC ðNC 1Þ ~ i,r , m ~ j,r , and C~ i,j,r denote mean vector and covariance where m matrix of flame image class i and j in semantic space along vr respectively. This metric is also combined with a forward selection technique to select salient semantics. Such method will handle Nv cases of visual word number and each case is similar to our previous Gabor filter design algorithm. Once the optimal dictionary size q, corresponding semantic number t, and asso~ tt are selected for training dataset, the ciated V~ T r t , U~ tt , and R local feature f 3 of a training image can be represented by T
1
~ f 3 ¼ yT U~ tt R tt
ð12Þ
In the same way, the feature vector of a testing image is T
T ~ 1 d~ ¼ d U~ tt R tt
ð13Þ
3.5. Classifier designing for burning state recognition ELM is a new learning algorithm for single layer feed-forward neural network. In ELM, instead of tuned, the input weights and hidden biases are randomly generated. The hidden layer maps the inputs into a new space via a fixed nonlinear transformation, whilst the output layer performs like a linear combiner in this new space. After the hidden node number is selected, the output weights are adjusted via a generalized inverse operation. Suppose, we have Tr training samples fF e ,Y e g, activation function GðFÞ, and hidden node number NH, where F e ¼ ½f e,1 ,f e,2 ,f e,3 is the concatenate features of the flame image sample e extracted
W. Li et al. / Neurocomputing 102 (2013) 144–153
above-mentioned, and Y e ¼ ½Y e,1 ,Y e,2 ,Y e,3 A R3 is its coded class label vector (for our three classes case). The ELM algorithm consisting of three steps can be summarized as (1) Randomly assign input weight wp and bias bp, p ¼ 1; 2, . . . ,NH . (2) Calculate the hidden layer output matrix H, where each element he,p ¼ Gðwp F e þ bp Þ,e ¼ 1; 2, . . . ,T r denotes the output of the pth hidden neuron with respect to F e ; wp ¼ ½wp,1 ,wp,2 , . . . ,wp,T r T and bp denote the weight vector and the bias of pth hidden neuron respectively; wp F e is the inner product of wp and F e . (3) Calculate the output weight b ¼ H y Y, where b ¼ ½b1 , b2 , . . . , bNH T is the output weight matrix and bp ¼ ½bj,1 , bp,2 , bp,3 T denotes the weight vector between the pth hidden neuron and output neurons; Y ¼ ½Y 1 ,Y 2 , . . . ,Y T r T is the expected output matrix. H y is the Moore–Penrose generalized inverse of H, and SVD is used to calculate H y . If H is nonsingular, an alternative formula for computing b [34] is
b ¼ ðH T HÞ1 H T Y
T
T
JðbÞ ¼ Y Yb H Y
4.1. Experiments for Gabor filter pre-processing As described in Section 3.1, Fig. 9 shows the classification result of the training flame and material texture images with respect to the candidate Gabor filter bank used during one sampling experiment, where each point is obtained based on ELM classifier with optimal hidden node number and sigmoidal activation function. As shown in Fig. 9, high-dimensional feature representation leads to a gradual degradation performance. Gabor filter bank designing is not a subjective parameter setting procedure, but filter selection should be an indispensable part. Such compact filter bank with four most discriminative Gabor filters is selected to filter the training and testing flame images throughout this sampling experiment. The effectiveness of our Gabor filter bank designing method for discriminating ROIs is tested compared with other designing methods with 2000 repeats listed in Table 1. From Table 1, as we can see, our filter bank design approach not only selects the uncorrelated filters and leads to a more compact filter bank, but also distinguishes the flame and material zones much more to facilitate the sequel.
ð14Þ
And the associated minimal cost function is T
149
4.2. Experiments for global feature selection ð15Þ
The above two equations are used to implement ELM in the present study.
After the pre-processing by the optimal compact Gabor filter bank, Fig. 10 shows the classification accuracy corresponding to the candidate eigen-flame image combinations used, i.e. the dimension of global feature, where each point is also based on
4. Experiments 0.945 0.94 0.935
Classification performance
In order to validate the presented method, flame images of the burning zone under various conditions are collected from No. 3 rotary kiln at Shanxi Aluminum Corp. A color CCD camera (Panasonic WV-CP450) and a non-contact colorimetric temperature measure device are installed outside the peephole of the kiln head. The output signal of CCD is digitized using an image grabber card (Matrox Meteor II). Each digital image has a size of 512 384 pixels, and each pixel is composed of red (R), green (G), and blue (B) components. The sampling period for flame image and burning zone temperature is set to 10 s. A total of 482 typical flame images, including 86 over-burning state images, 193 under-burning state images, and 203 normalburning state images are selected from 4150 flame images to form the sample dataset. Based on bootstrapping [35] with 2000 replica, training and testing dataset are performed to estimate the accuracy of the flame image burning state recognition. Some flame image examples are shown in Fig. 8. The labeling of these images is done by rotary kiln operational experts at Shanxi Aluminum Corp.
0.93 0.925 0.92 0.915 0.91 0.905
Optimal
0.9 0.895 0
5
10
15
Canadiate Gabor filter bank Fig. 9. Classification performance vs. candidate Gabor filter bank.
Fig. 8. Some typical flame images: (a) over-burning; (b) under-burning; (c) normal-burning.
20
150
W. Li et al. / Neurocomputing 102 (2013) 144–153
4.3. Experiments for local feature selection
Table 1 Comparison results for various Gabor filter designing methods. Gabor filter design methods
Average accuracy (%)
No. of Gabor filters
Presented method Fisher ratio-based Clausi-based [36] Li-based [37] Bianconi-based [29]
93.74 7 1.4 93.23 7 1.4 90.58 71.9 89.34 7 2.0 90.75 71.9
5.77 2.0 6.67 3.0 24 36 288
Following the procedure in Section 3.4, Fig. 11 gives the classification result with respect to the candidate semantic subsets with 15 visual words during one sampling experiment, where each point is also obtained based on ELM classifier with optimal hidden node number and sigmoidal activation function. In Fig. 11, the phenomenon that excessive insignificant semantics leading to the overfitting shows semantic selection should be an indispensable part in order to take into account their saliency. Therefore, such subset with four most discriminative semantics is selected in the sense of 15 visual words for the local feature extraction. Fig. 12 shows the classification accuracy corresponding to the number of visual words, where each point is obtained based on the result of the optimal local feature in the corresponding visual
1 0.95
Classification performance
For our flame images, the number of keypoints is typically 16. Thus, the maximum dictionary size Nv is set as 15 and the number of semantics Ns in the sense of Nv words is defined as ( N v 1, 2 rN v r10 Ns ¼ ð16Þ 10, 10 rNv r 15
0.9 0.85 0.8 0.75
Optimal
0.7
0.92 0.65
0.9 10
20
30
40
50
Number of Principal Components Fig. 10. Classification performance vs. number of eigen-flame images.
Table 2 Comparison results for various global feature extraction methods. Accuracy and No. of PC
With eigen-images selection
Without eigen-images selection
Accuracy (with filter) Accuracy (without filter) No. of PC (with filter) No. of PC (without filter)
88.53 72.0 84.25 71.9 9.37 2.0 11.47 2.1
84.63 7 2.2 79.42 7 2.7 10.2 7 2.2 11.7 7 1.8
Classification performance
0
0.88
0.86
0.84
Optimal 0.82
0.8 1
2
3
4
5
6
7
8
9
10
Number of semantics Fig. 11. Classification performance vs. candidate semantic subsets.
(a) Independent of eigen-flame image selection method, our Gabor filter bank pre-processing approach achieves the discrimination of the flame and material zones much more to facilitate the subsequent global feature extraction. (b) By selecting the eigen-flame images, the global feature obtained substantially outperforms those without selection. This is because PCA is optimal from a low-dimensional reconstruction viewpoint and is non-optimal from the pattern classification point of view.
0.92 0.91
Classification performance
ELM classifier with optimal hidden node number and sigmoidal activation function. In Fig. 10, peaking phenomenon also occurs. Thus, eigen-flame images need to be selected and such 11 most discriminative eigen-flame images are used to form global feature for the training and testing dataset throughout this sampling experiment. The significant degree of our eigen-flame image selection algorithm for global feature extraction is tested compared with traditional eigen-image selection method with 2000 repeats listed in Table 2 respectively. From Table 2, we have the following observations:
0.9 0.89 0.88 0.87 0.86
Optimal
0.85 0.84 0
2
4
6
8
10
12
Number of visual words Fig. 12. Classification performance vs. number of visual words.
14
W. Li et al. / Neurocomputing 102 (2013) 144–153
151
1
Table 3 Comparison results for various local feature extraction methods. Average accuracy (%)
No. of visual words
No. of semantics
Presented method Standard LSA-based Stop list-based Tf-idf table-based Presented method (without filter)
89.54 71.6 89.21 71.2 87.46 71.4 85.62 71.8 89.24 71.6
9.17 1.4 9 9 9 9
8.27 1.2 8 – – –
word number. In Fig. 12, such seven visual words are selected to construct the dictionary throughout this sampling experiment. In order to validate the effectiveness of our local feature extraction method, we have also tested the performance of standard LSA-based method, stop list analogy-based method [14], and tf-idf indexing table-based method with 2000 repeats listed in Table 3. From Table 3, we have the following observations: (a) Our Gabor filter bank pre-processing approach also facilitates the local feature extraction. Moreover, our method is feasible to extract local feature directly from the flame image, and can select more salient and discriminative semantics than the standard LSA-based method to form the more meaningful local feature. (b) LSA can extract more meaningful semantics than visual words, which not only generates low-dimensional local feature representation to mitigate the problem of synonymy. (c) During the stop list analogy, frequency is used to remove the most frequent and infrequent visual words to form better feature vector. However, evaluation of the discriminative power of visual words is not involved in such method, and the performance is hence interior.
4.4. Recognition performance based on concatenate feature After acquiring the above-mentioned individual feature, such features will be concatenate and normalized to recognize the final burning state based on ELM classifier. Again, sigmoidal activation function is used in ELM. The hidden node number has great effect on the recognition performance. Fig. 13 shows the recognition result with respect to the number of hidden node used during one sampling experiment. In such experiment, 57 is selected as the optimal hidden node number with 18-dimensional concatenate feature. Such hidden node number selection procedure is also used in the global and local feature selection. In order to validate the effectiveness of the used ELM classifier, we have also tested the performance of other classifiers, i.e. probabilistic neural network classifier (PNN) [38], neural network classifier (NN), and support vector machine classifier (SVM). PNN is constructed based on the well-known Bayesian classification technique, which considers the probability characteristic of sample space and uses typical samples as hidden layer nodes; NN, as a classifier, maps the feature space to a class label space. In the present study, the node number of the hidden layer is selected as two times the dimension of input features and the sigmoidal activation function is used; SVM is constructed based on the structural risk minimization principle. The basic idea of SVM is to map patterns from the input space to a transformed space, and then perform pattern classification in the transformed space using linear SVM. The recognition results and time-costing with 2000 repeats are listed in Table 4.
0.9
Classification performance
Various methods
0.8
0.7
0.6
Optimal 0.5
0.4 0
10
20
30
40
50
60
70
Number of hidden node Fig. 13. Recognition result vs. number of hidden node.
Table 4 Comparison results for various classifiers. Various classifiers
Training time (s)
Average accuracy (%)
ELM PNN NN SVM
1.781 4.391 61.766 8.344
92.75 7 2.8 90.49 7 3.3 88.92 7 4.0 90.12 7 4.3
As we can see in Table 4, various classifiers offer different contributions to the recognition result. For NN classifiers, the performance of multi-feature does not improve the final result and is even inferior to the individual feature because of the possible peak phenomenon. In our study, due to its characteristic, based on the selected optimal features, ELM shows its feasibility and the best recognition result for the burning state recognition of flame images compared to other classifiers. It can also be seen from Table 4, the standard deviation of the generalization performance of ELM is much smaller than other classifiers, meaning that ELM may run much more stable. Moreover, another important advantage of ELM classifier is the training speed, which is also included in Table 4. 4.5. Contrasted experiments Hereto, flame image has exhibited its power to recognize the burning state. Hereinafter, we will explore the effectiveness of our flame image-based method compared with other existing burning state recognition methods. Firstly, we compared our method with the temperature-based burning state recognition method. Taking the labeling by kiln operational experts as ground truth, the recognition accuracy of the two methods is 92.75% and 81.34% at the same time. Again, our flame image-based method outperforms the temperature-based method and is more reliable even with the disturbance of dust and smoke inside the kiln. Then, we compared our flame image-based method with four feasible image segmentation-based methods, including Ostu [23], fuzzy c mean and Gabor wavelet (FCMG) [5], dual fast marching (DFM) [6], and multistage adaptive threshold (MAT) [26]. Based on the experience of kiln operational experts, the following 16 features are extracted to characterize the flame color and the configuration of ROIs, namely average brightness of R sub-image, average brightness and its variance of the flame and material zones of R sub-image, area, length, width, circularity, and barycentric coordinates of the flame
152
W. Li et al. / Neurocomputing 102 (2013) 144–153
Table 5 Burning state recognition results for different image segmentation-based methods. Methods
PNN
NN
SVM
ELM
Ostu FCMG DFM MAT
42.67 711.4 44.72 712.3 47.23 711.5 52.19 711.3
40.26 7 11.1 41.62 7 12.5 44.31 7 12.3 49.29 7 12.3
41.79 7 11.2 42.83 7 12.1 46.32 7 13.2 51.78 7 11.3
42.13 711.2 46.45 79.1 49.63 710.1 55.79 79.2
zone, and height, width, area, and barycentric coordinates of the material zone. All these features are constructed to a feature vector to send to the ELM pattern classifier to obtain the recognition result. The average classification accuracy of 2000 repeats with different classifiers is shown in Table 5. Again, in our study, ELM gives the best recognition result except Ostu-based method. This is probably because Ostu is a general-purpose image segmentation method whilst the other methods are special methods for flame image or better image segmentation method. Moreover, obviously, although image segmentation based feature extraction has been successfully applied to many image recognition applications, in our application, the flame images are of poor quality due to the smoke and dust inside the kiln, and this in turn results in inaccurate ROIs segmentation and feature extraction. The above image segmentation-based methods are substantially inferior to our flame image-based method that is without the image segmentation procedure, and could work well only when the flame image is of high quality.
5. Conclusions and future work In this study, we have explored the feasibility of applying multifeature and ELM classifier to recognize the burning state of flame image for the rotary kiln without the difficult segmentation procedure. Firstly, MIA is employed to feature the color of flame image. Then, Gabor filter bank as the pre-processing step combined with the eigen-flame image decomposition is used to extract global features to represent the global configuration of flame image. Again, SIFT operator combined with the BoVW image descriptor and LSA is applied to extract local features to characterize the local configuration. The final recognition result is based on the integration of the above multi-feature and ELM classifier. For global and local features, we carry out extensive experiments to validate the effectiveness of feature extraction and necessity of feature selection, and then for the integrated optimal feature vector, we conduct comparative study of various classifier methods, temperature-based method, and image segmentation-based methods. Experimental results have demonstrated the feasibility and effectiveness of our feature method and ELM classifier. Exploring potentially more reasonable features and improving the existing ELM classifier to achieve better flame image burning state recognition result are our future work.
Acknowledgments This work is supported by the National Basic Research Program of China (2009CB320601), the Plan for University Subject Innovation and Introducing Intelligence (B08015), and Natural Science Foundation of China (61020106003), Matching grant for 1000 Talent Program (201100020). References [1] C.W. Ruby, A new approach to expert kiln control, in: XXXIX IEEE Conference on Cement Industry Technical, Hershey, USA, 1997, pp. 339–412.
[2] M. Akalp, A.L. Dominguez, R. Longchamp, Supervisory fuzzy control of a rotary cement kiln, in: 7th IEEE Conference on Electrotechnical, Antalya, Turkey, 1994, pp. 754–757. [3] M. Jarvensivu, K. Saari, S.L. Jamsa-Jounela, Intelligent control system of an industrial lime kiln process, Control Eng. Pract. 9 (6) (2001) 589–606. [4] X.J. Zhou, T.Y. Chai, Pattern-based hybrid intelligent control for rotary kiln process, in: IEEE International Conference on Control Applications, Singapore, 2007, pp. 31–35. [5] P. Sun, T.Y. Chai, X.J. Zhou, Rotary kiln flame image segmentation based on FCM and Gabor wavelet based texture coarseness, in: 7th World Congress on Intelligent Control and Automation, Chongqing, China, 2008, pp. 7615–7620. [6] H.Y. Jiang, X.L. Cui, X.J. Zhou, T.Y. Chai, Image segmentation based on improved dual fast marching method, J. Syst. Simulation 3 (20) (2008) 803–810. [7] L. Bertucco, A. Fichera, G. Nunnari, A. Pagano, A cellular neural networks approach to flame image analysis for combustion monitoring, in: Proceedings of the 6th IEEE International Workshop on Cellular Neural Networks and their Applications, Catania, Italy, 2000, pp. 455–459. [8] M. Idrissa, M. Acheroy, Texture classification using Gabor filters, Pattern Recognition Lett. 23 (9) (2002) 1095–1102. [9] A.K. Jain, R.P.W. Duin, J.C. Mao, Statistical pattern recognition: a review, IEEE Trans. Pattern Anal. Mach. Intell. 22 (1) (2000) 4–37. [10] M. Dash, H. Liu, Feature selection for classification, Intell. Data Anal. 1 (1) (1997) 131–156. [11] P. Geladi, H. Grahn, Multivariate Image Analysis, Wiley, New York, 1996. [12] J. Yang, D. Zhang, A.F. Frangi, J.Y. Yang, Two-dimensional PCA: a new approach to appearance-based face representation and recognition, IEEE Trans. Pattern Anal. Mach. Intell. 26 (1) (2004) 131–137. [13] D.G. Lowe, Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vision 60 (2) (2004) 91–110. [14] J. Sivic, A. Zisserman, A. Video google: a text retrieval approach to object matching in videos, in: Ninth IEEE International Conference on Computer Vision, Nice, France, 2003, pp. 1470–1477. [15] R.B. Yates, B.R. Neto, Modern Information Retrieval, ACM Press, New York, 1999. [16] S.C. Deerwester, S.T. Dumais, T.K. Landauer, G.W. Furnas, R.A. Harshman, Indexing by latent semantic analysis, J. Am. Soc. Inf. Sci. 41 (6) (1990) 391–407. [17] H. Witten, T.C. Bell, The zero-frequency problem: estimating the probabilities of novel events in adaptive text compression, IEEE Trans. Inf. Theory 37 (4) (1991) 1085–1094. [18] G.B. Huang, Q.Y. Zhu, C.K. Siew, Extreme learning machine: theory and applications, Neurocomputing 70 (2006) 489–501. [19] G.B. Huang, D.H. Wang, Y. Lan, Extreme learning machines: a survey, Int. J. Mach. Learn. Cybern. 2 (2) (2011) 107–122. [20] G.B. Huang, X.J. Ding, H.M. Zhou, Optimization method based extreme learning machine for classification, Neurocomputing 74 (1–3) (2010) 155–163. [21] N.Y. Chen, Physics and Chemistry of Alumina Production, Science and Technology Press, Shanghai, 1962. [22] Y.M. Wang, D.O. Yan, C.Y. Yang, C.Z. Gao, Alumina Production Process and Equipment, Metallurgical Industry Press, Beijing, 1987. [23] N. Otsu, A threshold selection method from gray-level histogram, IEEE Trans. SMC 9 (1) (1979) 62–66. [24] J.C. Dunn, A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters, J. Cybern. 3 (3) (1973) 32–57. [25] J.B. Shi, J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell. 22 (8) (2000) 888–905. [26] F.X. Yan, H. Zhang, C.R. Kube, A multistage adaptive thresholding method, Pattern Recognition Lett. 26 (8) (2005) 1183–1191. [27] G. Dong, N. Ray, S.T. Acton, Intravital leukocyte detection using the gradient inverse coefficient of variation, IEEE Trans. Med. Imaging 24 (7) (2005) 910–924. [28] B.N. Saha, N. Ray, Image thresholding by variational minimax optimization, Pattern Recognition 42 (5) (2009) 843–856. [29] F. Bianconi, A. Ferna´ndez, Evaluation of the effects of Gabor filter parameters on texture classification, Pattern Recognition 40 (12) (2007) 3325–3335. [30] W.T. Li, K.Z. Mao, H. Zhang, T.Y. Chai, Designing compact Gabor filter banks for efficient texture feature extraction, in: International Conference on Control, Automation, Robotics and Vision, Singapore, 2010, pp. 1193–1198. [31] H.L. Yu, J.F. MacGregor, Monitoring flames in an industrial boiler using multivariate image analysis, AIChE J. 50 (7) (2004) 1474–1483. [32] R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, Wiley New York, 1973. [33] W.T. Li, K.Z. Mao, X.J. Zhou, T.Y. Chai, H. Zhang, Eigen-flame image-based robust recognition of burning states for sintering process control of rotary kiln, in: IEEE Conference on Decision and Control, Shanghai, China, 2009, pp. 398–403. [34] T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer, New York, 2001. [35] M.R. Chemick, Bootstrap Methods: A Practitioner’s Guide, Wiley, New York, 1999. [36] D.A. Clausi, H. Deng, Design-based texture feature fusion using Gabor filters and co-occurrence probabilities, IEEE Trans. Image Process. 14 (7) (2005) 925–936. [37] S. Li, J.S. Taylor, Comparison and fusion of multiresolution features for texture classification, Pattern Recognition Lett. 26 (5) (2004) 633–638. [38] D.F. Specht, Probabilistic neural network, Neural Networks 3 (1) (1990) 109–118.
W. Li et al. / Neurocomputing 102 (2013) 144–153 Weitao Li received the BSc degree and the MSc degree in Hefei University of Technology in 2004 and 2007, respectively. He is currently working toward the PhD degree in Northeastern University. His research interests include image processing and pattern recognition.
Dianhui Wang received the PhD degree from Northeastern University, Shenyang, China, in 1995. From 1995 to 2001, he worked as a Postdoctoral Fellow with Nanyang Technological University, Singapore, and a Researcher with The Hong Kong Polytechnic University, Hong Kong, China. He is currently a Reader and Associate Professor with the Department of Computer Science and Computer Engineering, La Trobe University, Melbourne, Victoria, Australia. He is also associated with the Key Laboratory of Synthetical Automation of Process Industries, Ministry of Education, Northeastern University. His current research interests include data mining and computational intelligence systems for bioinformatics and engineering applications.
153 Tianyou Chai received the PhD degree in control theory and engineering from Northeastern University, Shenyang, China, in 1985. He is the director of the Key Laboratory of Synthetical Automation of Process Industries, Ministry of Education, Northeastern University. His current research interests include adaptive control, intelligent decoupling control, and integrated automation of industrial process. Dr. Chai was elected as a member of the Chinese Academy of Engineering in 2003, an Academician of the International Eurasian Academy of Sciences in 2007, and an International Federation of Automatic Control Fellow in 2008.
Contents lists available at SciVerse ScienceDirect
Neurocomputing journal homepage: www.elsevier.com/locate/neucom
Burning state recognition of rotary kiln using ELMs with heterogeneous features Weitao Li a, Dianhui Wang b,a,n, Tianyou Chai a a b
State Key Laboratory of Synthetical Automation of Process Industries, Northeastern University, Shenyang 110004, China Department of Computer Science and Computer Engineering, La Trobe University, Melbourne, VIC 3086, Australia
a r t i c l e i n f o
a b s t r a c t
Available online 19 June 2012
Image based burning state recognition plays an important role in sintering process control of rotary kiln. Although many efforts on dealing with this problem have been made over the past years, the recognition performance cannot be satisfactory due to the disturbance from smoke and dust inside the kiln. This work aims to develop a reliable burning state recognition system using extreme learning machines with heterogeneous features. The recorded flame images are firstly represented by various low-level features, which characterize the distribution of the temperature field and the flame color, the local and global configurations. To learn the merits of our proposed flame image-based burning state recognition system, four learner models (ELM, MLP, PNN and SVM) are examined by a typical flame image database with 482 images. Simulation results demonstrate that the heterogeneous features based ELM classifiers outperform other classifiers in terms of both recognition accuracy and computational complexity. & 2012 Elsevier B.V. All rights reserved.
Keywords: Burning state Multivariate image analysis Eigen-flame image Latent semantic analysis ELM
1. Introduction Rotary kiln, as a large-scale sintering facility, is widely used in metallurgical, cement, chemical, and environment protection industries. The main control objective of the rotary kiln sintering process is to achieve consistent product quality, which is often referred to as the key performance index. However, practically, the measurement of the product quality index is done by manual sampling with 1-h period. Therefore, indirect control is employed to replace online control, i.e. keeping key process parameters that can be measured online and are closely related to the product quality index into their preset ranges means satisfied product quality index. Based on the analysis of rotary kiln mechanism process, the fact that burning zone temperature directly determines the characteristics of the clinker is widely acknowledged. Thus, the accurate measurement for such temperature is the most critical issue for the rotary kiln sintering control process [1–3]. However, due to the harsh environment inside the kiln, the accurate measurement through thermocouple is still a challenging task. Recently, we have developed and implemented a hybrid control system in No. 3 rotary kiln at Shanxi Aluminum Corp. [4]. In such system, burning state is recognized based on the
n Corresponding author at: Department of Computer Science and Computer Engineering, La Trobe University, Melbourne, VIC 3086, Australia. E-mail address: [email protected] (D. Wang).
0925-2312/$ - see front matter & 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.neucom.2011.12.047
clustering of temperatures from the non-contact colorimetric measuring device. Because of the rich and reliable visual information, for operators, burning zone flame image is considered to be more reliable than the burning zone temperature to estimate the burning state. Flame image-based state recognition has already been studied in the past, where a flame image is first segmented into regions of interest (ROIs), features for the representation of the color and configuration characteristics of these regions are then extracted, burning state recognition is performed based on the features extracted [5–7]. However, due to the poor image quality caused by smoke and dust inside the kiln, accurate segmentation of ROIs is quite challenging and therefore unreliable. This will in turn result in inaccurate feature extraction and poor state recognition. To avoid the above problems, we have tried to extract features to represent the color and configuration characteristics of ROIs of flame image without segmentation, with the goal of improving the burning state recognition. From operators point of view, more discriminable ROIs will facilitate the subsequent feature extraction and burning state recognition. Motivated by the knowledge that flame and material zones are with distinct texture characteristics, Gabor filter is employed as a pre-processing step to discriminate them [8]. Practically, its parameters are often set by trial and error. However, we believe most of a filter bank offer little improvement to (or even reduce) the discriminative power due to the peaking phenomenon [9]. Hence, we propose to incorporate Mahalanobis
W. Li et al. / Neurocomputing 102 (2013) 144–153
measure [10] and forward selection technique [9] to automatically generate a compact Gabor filter bank to enhance the separability of ROIs to facilitate the sequel. Flame color indicates the combustion region and the distribution of the temperature field, and hence exhibits an intuitive impression for the burning state. Appropriate flame region with similar color corresponds normal burning state. Unlike other flame image analysis methods that track the turbulent flame in the image space to extract features to represent the flame color, the multivariate image analysis (MIA) technique [11] shows its efficiency to feature the flame color without the difficult locating flame step. MIA lies in projecting image pixels with similar color in a common region of the score space independently of their spatial location, and retrieving the locations of pixels with similar color in the image space after detecting feature in the score space. Such extracted feature will be used to represent the flame color. The configuration of ROIs characterizes the heat source, disturbance from smoke and dust, and clinker sintering status. Especially, the configuration of the flame zone and the height of the material zone are the key factors to recognize the burning state [5,6]. Flame zone with good circularity and appropriate material height mean normal burning state. Due to the difficult segmentation of ROIs, instead, global features are firstly extracted to represent the global configuration of the flame image. Eigenflame images are obtained using principal component analysis (PCA), and global features are then produced by correlating each flame image with the eigen-flame images to represent the global configuration. Unlike traditional selection criterion for eigen-images [12], a new selection procedure is used, with the goal of selecting global features that possess the maximum discriminative power. Generally, local configuration is considered to contain more valuable details to complement the global configuration. Scale invariant feature transform (SIFT) operator [13] is hence employed to extract key points of flame image to avoid the segmentation. The dimension of a SIFT descriptor is 1 128. Exploring research in image and text retrieval, ‘‘bag of visual words’’ (BoVW) [14] and term frequency-inverse document frequency weight [15] are applied to vector quantize the descriptors into clusters and form a visual word-image table to reduce feature representation dimension. For such table, latent semantic analysis (LSA) [16] is used to map such visual word-image space to a latent semantic space by taking advantage of some implicit higher-order structure in associations of visual words with images to mitigate potential zero-frequency problem and reduce feature dimension further [17]. Now, semantic vector as local feature conceptually represents the local configuration. Previously, semantics are explicitly assumed to have same saliency. In our work, a new semantic selection procedure is introduced in order to consider the saliency of semantics to select local features with maximum discriminative power. To imitate the fact that the integration of multi-feature is used to estimate the final burning state, the above individual features are concatenate and normalized, and a single-hidden-layer feed forward neural networks (SLFNs) classifier with extreme learning machine (ELM) algorithm [18] is employed to recognize the burning state. Different from conventional learning algorithms for neural networks, with randomly chosen input weights and hidden bias and calculated output weights, ELM not only trains much faster with higher generalization ability, but also overcomes many issues faced by gradient-based algorithms such as stopping criteria, learning rate, learning epochs, and local minima. Many types of hidden nodes including additive/RBF hidden nodes, multiplicative nodes, and non-neural alike nodes, can be used as long as they are piecewise nonlinear. Readers may refer to a recent survey paper for more details on ELM [19]. In ELM, since different hidden node parameters correspond to different
145
classification performance, the selection of the hidden node number is the most critical issue. Recently, due to the universal approximation capability, the minimum training error and weight norm, diverse modification for ELM has been successfully applied to many classification problems [20] but has never been used in flame image recognition before. The advantages of our new flame image-based burning state recognition method are fourfold. Firstly, our new method is computationally more efficient and more accurate and robust than the image segmentation-based and temperature-based methods. Secondly, MIA is effective to feature the flame color to avoid the difficult flame tracking. Thirdly, eigen-flame image-based method is feasible to feature the global configuration to avoid the difficult segmentation. Fourthly, without segmentation, local configuration is effectively featured by semantic vector. Numerous experimental studies show that, with feasible ELM classifier, our new method outperforms the image segmentation-based methods and temperaturebased method. As we can expect, more consistent product quality index can be achieved if the new burning state recognition method is incorporated into our previously developed hybrid control system. The rest of the paper is organized as follows. The rotary kiln sintering process and weaknesses of previous burning state recognition methods are presented in Section 2. Section 3 gives our new flame image multi-feature-based burning state recognition method. Experimental studies, conclusions and future work are given in Sections 4 and 5 respectively.
2. Rotary kiln sintering process and previous burning state recognition methods 2.1. Rotary kiln sintering process A schematic diagram of the rotary kiln sintering process is shown in Fig. 1, where raw material slurry is sprayed into the rotary kiln the upper end, i.e. kiln tail. At the lower end, i.e. kiln head, coal powders from the coal feeder and primary air from the air blower are mixed into a bi-phase fuel flow and then are sprayed into the kiln head hood and combust with secondary air from the cooler. The heated gas is brought to the kiln tail by the induced draft fan, while the material moves to the kiln head by the rotation of kiln and its gravity, in counter direction of the gas flow. After the material passes through drying zone, pre-heating zone, decomposing zone, burning zone, and cooling zone in sequence, the final product of the sintering process of rotary kiln, namely clinker, is generated and is fed downstream for further processing [21]. Taking alumina sintering process for instance, during burning zone, with 1200–1300 1C, the following chemical reaction arises [21]: Na2 O Al2 O3 2Si O2 þ4CaO-Na2 O Al2 O3 þ2ð2CaO Si O2 Þ cyclone dust collector coal feeder primary air
air blower
temperature measure
electric dust collector
induced draft fan returned dust rotary kiln pre-heating zone
burning kiln zone head decomposing hood cooling zone zone
cooler
kiln tail hood drying zone
secondary air sinter, to the next procedure
Fig. 1. Schematic diagram of rotary kiln sintering process.
ð1Þ
to the chimney
from raw material slurry feeder
146
W. Li et al. / Neurocomputing 102 (2013) 144–153
Si O2 þ2CaO-2CaO Si O2
ð2Þ
Ti O2 þCaO-CaO Ti O2
ð3Þ
At the outlet of the cooler, the final product quality index, i.e. clinker unit weight CW, is measured manually. With appropriate burning zone temperatures TBZ, satisfied CW (1.20–1.45 kg/L [22]) will be produced. Since both higher and lower TBZ will adversely affect CW [22], hence, the accurate measurement of TBZ becomes the most critical issue for the product quality control of the rotary kiln sintering process. 2.2. Previous burning state recognition methods In previous studies, the recognition of the burning state for rotary kiln is mainly based on three ways. Firstly, primitively, via the peephole of kiln head, operators observe the burning zone to estimate burning state. However, the performance of such way is limited by operators’ experience, mental state and responsibility. As a result, inconsistent estimation result by different operators will lead to inconsistent clinker quality. Secondly, a number of thermocouples are inserted through the kiln shell along the kiln to evaluate the burning state. However, this method is considered unreliable by operators, since fouling occurs on the thermocouples located in the most interesting region (i.e. burning zone). Recently, colorimetric measuring device is used in our developed control system [4]. Unfortunately, due to the disturbance from dust and smoke, such a single point temperature is often far from its true value and often exhibits large fluctuation and severe noise. To alleviate the problems involved in the temperature measurement and imitate the burning state estimation mode of experienced operators, the third way based on the burning zone flame
image is studied recently. Because of a wealth of temperature field and clinker sintering information, flame image is considered to be more reliable for estimating burning state by operators. As shown in Fig. 2, a flame image consists of a few parts including kiln wall, coal zone, material zone and flame zone. Coal zone is formed by coal powders from the coal burner; flame zone is formed by instant explosion and combustion of mixed coal powders and air; material zone is formed by the sintering of raw material slurry. For operators, the flame color and configuration of ROIs are the key factors in the recognition of three burning states, i.e. overburning state, under-burning state, and normal-burning state [5,6]. Previously, before adopting features to represent the above factors, ROIs are firstly obtained by image segmentation techniques. However, since turbulent flame bounces around, brightness of material zone results from flame zone, and dust and smoke couple ROIs, tracking the accurate boundary of ROIs is a challenging task. Fig. 3 shows the experimental studies based on several image segmentation methods [5,23–28]. As we can see, difficult segmentation will in turn lead to inaccurate feature extraction and poor burning state recognition result. To avoid the problems listed above, we try to extract features to represent the color and configuration of flame image without the difficult step of segmentation. In this paper, three feature extraction methods corresponding to flame color, global configuration of ROIs, and local configuration of ROIs are employed respectively. The above individual features will be concatenate to a feature vector, and then fed into the pattern classifier to obtain the final burning state recognition result. Flow chart of our proposed method is shown in Fig. 4. Details of each method are described as follows.
3. Robust burning state recognition based on heterogeneous feature 3.1. Pre-processing of flame image
Fig. 2. A burning zone flame image.
For the Gabor filter pre-processing phase, due to the camera placement providing a rough estimate for the range of ROIs, two 25 25 fixed windows are used to sample the flame and material zones to avoid the segmentation issue as shown in Fig. 5. Assume a total of 2T r flame and material texture images sampled from Tr grayscale transformed images of the training RGB flame images: T 1 ,T 2 , . . . ,T 2T r . Let z1 ,z2 , . . . ,znG denote feature groups extracted from filtered texture images by the nG ðnG ¼ 64Þ initial Gabor filters, where zk ¼ ½z1,k ,z2,k , . . . ,z2T r ,k T , k ¼ 1, . . . ,nG , and p parameters of initial Gabor ffiffiffiffiffiffiffiffiffiffiffi filter bank are set as f m ¼ g=ð2g þ 2 log 2=pÞ, nf ¼4, no ¼4,
Fig. 3. Performance of image segmentation methods: (a) flame image; (b) Ostu [23]; (c) Fuzzy c mean and Gabor wavelet (FCMG) [5]; (d) Fuzzy c mean (FCM) [24]; (e) normalized cuts [25]; (f) multistage adaptive threshold (MAT) [26]; (g) B-snake [27]; (h) minimax [28].
W. Li et al. / Neurocomputing 102 (2013) 144–153
Input flame images
Class 1 . . .
Class 3
Pre–processing
Gabor filter
Optimal filter bank
Gabor filter subset
Feature extraction
147
Classification
MIA MIA (for color)
f1
PCA (forPCA global configuration)
f2
SIFT+BoVW+LSA SIFT+BoW+LSA (for local configuration)
f3
L e a r n i n g
ELM
Feature vector F
Feature extraction
Trained classifier
New images
T e s Burning t state i n g
Fig. 4. Flow chart of the burning state recognition method.
of the unstructured noise and extract more meaningful ROIs from the original flame image. MIA is equivalent to unfold the threeway matrix I into a two-way matrix I without considering the spatial coordinates of pixels, and then performs PCA on it unfold
T I F r F c 3 - I ðF r F c Þ3 ¼ SPC a ¼ 1 t a pa þ E
Fig. 5. Fixed windows of flame image.
g ¼ 0:5,1:0, and Z ¼ 0:5,1:0 based on [29]. For each texture image, the mean m and standard deviation s features are extracted, i.e. zw,k ¼ ½mw,k , sw,k . Mahalanobis separability measure JM(k) is employed as the metric function to evaluate and sort the discriminative power of zk and associated filters defined as follows: T J M ðkÞ ¼ ðmf ,k mm,k ÞC 1 k ðmf ,k mm,k Þ
ð4Þ
where mf ,k , mm,k , and C k denote mean vector and covariance matrix of flame class and material class in feature space along feature group zk respectively. In our study, such metric is combined with a forward selection technique to automatically select uncorrelated feature groups and associated Gabor filters to best distinguish flame and material zones much more to facilitate the sequel. This method has been successfully applied to standard texture image analysis, but not on flame image before. See [30] for details. Once the compact bank with ns filters for training gray-scale image dataset is selected, they will be applied to the R, G, and B channels sub-image of each original flame image respectively. Then, for each training image I e , the mean image I e 0 of ns filtered images is used to replace I e to form a new training image for further processing. 3.2. Sub-feature to represent flame color An RGB flame image I has F r F c 3 pixels, and each pixel is quantized by the 0–255 integers in its R, G, and B channels. MIA technique is based on multiway PCA, which can eliminate much
ð5Þ
where a (PC ¼3 for red, green, and blue spectral channels) indexes the principal components, t a is a ðF r F c Þ 1 score vector, and pa is a 3 1 loading vector. After scaling and rounding off from 0 to 255, t a is denoted as sa and can be refolded into the original image size and displayed as an image. For RGB image, the first two score vectors usually explain 99% of the total variance. Inspection of the t 1 t 2 score plot is a common tool in PCA to detect clusters or outliers. However, due to the numerous pixels, an analysis from the compressed score space is more important [31]. A 256 256 score plot histogram SS obtained from s1 and s2 is hence employed to describe the compressed score space, where each element is computed as SSm,n ¼ St 1
ð8t,s1, t ¼ m,s2, t ¼ n,m,n ¼ 0, . . . ,255Þ
ð6Þ
In such plot, pixels with similar color in the image space are clustered together in the score space, and the locations of pixels in the score plots change significantly as the burning states change, which enables us to use masking to obtain ROIs of flame image whose pixels have similar color in the image space to distinguish various burning states shown in Fig. 6. According to [31], a 256 256 binary matrix M as shown in Fig. 6 is constructed to extract the area feature f 1 to characterize the color of the training flame image, where each element is defined as follows: f 1 ¼ Sm,n SSm,n
ð8m,n,M m,n ¼ 1Þ
ð7Þ
3.3. Sub-feature to represent global configuration of ROIs Eigen-flame image decomposition based on PCA is applied to extract global features. Suppose E1 ,E2 , . . . ,ET r denote eigen-flame images after applying PCA to the new generated training dataset. Due to the correlation coefficients between an image and the eigenflame images can be considered as global feature to represent the global configuration, the selection for the eigen-flame images is hence quite important. Unlike traditional selection criterion that is
148
W. Li et al. / Neurocomputing 102 (2013) 144–153
Fig. 6. (a) Original flame image. (b) Score plot of flame image with masking (yellow). (c) Flame image with overlay of highlighted pixels. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this article.)
LSA is hence employed to merge visual words with similar meanings to mitigate such problem. LSA requires the singular value decomposition (SVD) to generate a semantic space that represents conceptual visual word-image associations, which can be written as N Nv T r ¼ U Nv Nv RNv Nv V TT r Nv
ð9Þ
where U, V, and R are the matrices of visual word vectors, image vectors, and singular values. Previously, the best approximation of N with rank-l is given by selecting the first l values of R and associated vectors from U and V ~ V~ T U RV T ¼ N N~ l ¼ U~ ll R ll T r l Fig. 7. Detected SIFT keypoints for a flame image.
optimal from a low dimensional reconstruction viewpoint, Fisher ratio [32] is employed as the metric function to evaluate the class separability of Eq defined as PN C
i¼1
J F ðqÞ ¼
PN C
j¼1
9g i,q g j,q 9 2
2
2
hi,q þ hj,q N C ðNC 1Þ
ðq ¼ 1, . . . ,T r Þ
ð8Þ
where g i,q , g j,q , hi,q , hj,q , and NC denote mean and standard deviation of flame image class i and class j correlated with Eq and the image class number respectively. Once JF of all Eq are evaluated, the eigen-flame images selection, i.e. extraction of global feature f 2 , can be carried on. Followed by the same procedure, a testing flame image is handled to extract the global feature. This eigen-flame image method has been used to flame image, but without the pre-processing step. Refer to [33] for details. 3.4. Sub-feature to represent local configuration of ROIs SIFT operator is superior to others in local feature detection and description. Fig. 7 shows the detected SIFT keypoints for a flame image. To avoid the high-dimensional SIFT descriptor, ‘‘BoVW’’ is employed to construct the vocabulary via K-means clustering from D SIFT descriptors of Tr training images. Such Nv ðN v 5DÞ clustered vectors is considered as ‘‘visual words’’, and then descriptors of every image are assigned to the nearest cluster to form a co-occurrence table N Nv T r . Essentially, N records the frequency of ‘‘visual words’’ actually appearing in each flame image. The tf-idf weight is said to be superior to the frequency list by N to evaluate the importance of a visual word, and is hence applied to form a new visual word-image table N Nv T r , where each row xr for a visual word whilst each column ye for an image. In the matrix N, many zero entrices present and those nonzero entrices represent the content of images. Thus, similarity metrics between images will be evaluated by counting the synonymy given in the table lists of visual words in images.
ð10Þ
Since local configuration can be considered to be conceptually featured by semantic vectors and the essence of LSA is from a low dimensional reconstruction viewpoint, semantic selection should be carried out in order to take into account the saliency of various semantics to generate the best local feature. For each column of V, Mahalanobis measure is used to evaluate and sort the discriminative power of semantic vector vr , r ¼ 1, . . . ,Nv in the sense of Nv visual words: PNC PNC ~ 1 ~ ~ ~ ~ T i¼1 j ¼ 1 ðm i,r m j,r ÞC i,j,r ðm i,r m j,r Þ ð11Þ JM ðrÞ ¼ NC ðNC 1Þ ~ i,r , m ~ j,r , and C~ i,j,r denote mean vector and covariance where m matrix of flame image class i and j in semantic space along vr respectively. This metric is also combined with a forward selection technique to select salient semantics. Such method will handle Nv cases of visual word number and each case is similar to our previous Gabor filter design algorithm. Once the optimal dictionary size q, corresponding semantic number t, and asso~ tt are selected for training dataset, the ciated V~ T r t , U~ tt , and R local feature f 3 of a training image can be represented by T
1
~ f 3 ¼ yT U~ tt R tt
ð12Þ
In the same way, the feature vector of a testing image is T
T ~ 1 d~ ¼ d U~ tt R tt
ð13Þ
3.5. Classifier designing for burning state recognition ELM is a new learning algorithm for single layer feed-forward neural network. In ELM, instead of tuned, the input weights and hidden biases are randomly generated. The hidden layer maps the inputs into a new space via a fixed nonlinear transformation, whilst the output layer performs like a linear combiner in this new space. After the hidden node number is selected, the output weights are adjusted via a generalized inverse operation. Suppose, we have Tr training samples fF e ,Y e g, activation function GðFÞ, and hidden node number NH, where F e ¼ ½f e,1 ,f e,2 ,f e,3 is the concatenate features of the flame image sample e extracted
W. Li et al. / Neurocomputing 102 (2013) 144–153
above-mentioned, and Y e ¼ ½Y e,1 ,Y e,2 ,Y e,3 A R3 is its coded class label vector (for our three classes case). The ELM algorithm consisting of three steps can be summarized as (1) Randomly assign input weight wp and bias bp, p ¼ 1; 2, . . . ,NH . (2) Calculate the hidden layer output matrix H, where each element he,p ¼ Gðwp F e þ bp Þ,e ¼ 1; 2, . . . ,T r denotes the output of the pth hidden neuron with respect to F e ; wp ¼ ½wp,1 ,wp,2 , . . . ,wp,T r T and bp denote the weight vector and the bias of pth hidden neuron respectively; wp F e is the inner product of wp and F e . (3) Calculate the output weight b ¼ H y Y, where b ¼ ½b1 , b2 , . . . , bNH T is the output weight matrix and bp ¼ ½bj,1 , bp,2 , bp,3 T denotes the weight vector between the pth hidden neuron and output neurons; Y ¼ ½Y 1 ,Y 2 , . . . ,Y T r T is the expected output matrix. H y is the Moore–Penrose generalized inverse of H, and SVD is used to calculate H y . If H is nonsingular, an alternative formula for computing b [34] is
b ¼ ðH T HÞ1 H T Y
T
T
JðbÞ ¼ Y Yb H Y
4.1. Experiments for Gabor filter pre-processing As described in Section 3.1, Fig. 9 shows the classification result of the training flame and material texture images with respect to the candidate Gabor filter bank used during one sampling experiment, where each point is obtained based on ELM classifier with optimal hidden node number and sigmoidal activation function. As shown in Fig. 9, high-dimensional feature representation leads to a gradual degradation performance. Gabor filter bank designing is not a subjective parameter setting procedure, but filter selection should be an indispensable part. Such compact filter bank with four most discriminative Gabor filters is selected to filter the training and testing flame images throughout this sampling experiment. The effectiveness of our Gabor filter bank designing method for discriminating ROIs is tested compared with other designing methods with 2000 repeats listed in Table 1. From Table 1, as we can see, our filter bank design approach not only selects the uncorrelated filters and leads to a more compact filter bank, but also distinguishes the flame and material zones much more to facilitate the sequel.
ð14Þ
And the associated minimal cost function is T
149
4.2. Experiments for global feature selection ð15Þ
The above two equations are used to implement ELM in the present study.
After the pre-processing by the optimal compact Gabor filter bank, Fig. 10 shows the classification accuracy corresponding to the candidate eigen-flame image combinations used, i.e. the dimension of global feature, where each point is also based on
4. Experiments 0.945 0.94 0.935
Classification performance
In order to validate the presented method, flame images of the burning zone under various conditions are collected from No. 3 rotary kiln at Shanxi Aluminum Corp. A color CCD camera (Panasonic WV-CP450) and a non-contact colorimetric temperature measure device are installed outside the peephole of the kiln head. The output signal of CCD is digitized using an image grabber card (Matrox Meteor II). Each digital image has a size of 512 384 pixels, and each pixel is composed of red (R), green (G), and blue (B) components. The sampling period for flame image and burning zone temperature is set to 10 s. A total of 482 typical flame images, including 86 over-burning state images, 193 under-burning state images, and 203 normalburning state images are selected from 4150 flame images to form the sample dataset. Based on bootstrapping [35] with 2000 replica, training and testing dataset are performed to estimate the accuracy of the flame image burning state recognition. Some flame image examples are shown in Fig. 8. The labeling of these images is done by rotary kiln operational experts at Shanxi Aluminum Corp.
0.93 0.925 0.92 0.915 0.91 0.905
Optimal
0.9 0.895 0
5
10
15
Canadiate Gabor filter bank Fig. 9. Classification performance vs. candidate Gabor filter bank.
Fig. 8. Some typical flame images: (a) over-burning; (b) under-burning; (c) normal-burning.
20
150
W. Li et al. / Neurocomputing 102 (2013) 144–153
4.3. Experiments for local feature selection
Table 1 Comparison results for various Gabor filter designing methods. Gabor filter design methods
Average accuracy (%)
No. of Gabor filters
Presented method Fisher ratio-based Clausi-based [36] Li-based [37] Bianconi-based [29]
93.74 7 1.4 93.23 7 1.4 90.58 71.9 89.34 7 2.0 90.75 71.9
5.77 2.0 6.67 3.0 24 36 288
Following the procedure in Section 3.4, Fig. 11 gives the classification result with respect to the candidate semantic subsets with 15 visual words during one sampling experiment, where each point is also obtained based on ELM classifier with optimal hidden node number and sigmoidal activation function. In Fig. 11, the phenomenon that excessive insignificant semantics leading to the overfitting shows semantic selection should be an indispensable part in order to take into account their saliency. Therefore, such subset with four most discriminative semantics is selected in the sense of 15 visual words for the local feature extraction. Fig. 12 shows the classification accuracy corresponding to the number of visual words, where each point is obtained based on the result of the optimal local feature in the corresponding visual
1 0.95
Classification performance
For our flame images, the number of keypoints is typically 16. Thus, the maximum dictionary size Nv is set as 15 and the number of semantics Ns in the sense of Nv words is defined as ( N v 1, 2 rN v r10 Ns ¼ ð16Þ 10, 10 rNv r 15
0.9 0.85 0.8 0.75
Optimal
0.7
0.92 0.65
0.9 10
20
30
40
50
Number of Principal Components Fig. 10. Classification performance vs. number of eigen-flame images.
Table 2 Comparison results for various global feature extraction methods. Accuracy and No. of PC
With eigen-images selection
Without eigen-images selection
Accuracy (with filter) Accuracy (without filter) No. of PC (with filter) No. of PC (without filter)
88.53 72.0 84.25 71.9 9.37 2.0 11.47 2.1
84.63 7 2.2 79.42 7 2.7 10.2 7 2.2 11.7 7 1.8
Classification performance
0
0.88
0.86
0.84
Optimal 0.82
0.8 1
2
3
4
5
6
7
8
9
10
Number of semantics Fig. 11. Classification performance vs. candidate semantic subsets.
(a) Independent of eigen-flame image selection method, our Gabor filter bank pre-processing approach achieves the discrimination of the flame and material zones much more to facilitate the subsequent global feature extraction. (b) By selecting the eigen-flame images, the global feature obtained substantially outperforms those without selection. This is because PCA is optimal from a low-dimensional reconstruction viewpoint and is non-optimal from the pattern classification point of view.
0.92 0.91
Classification performance
ELM classifier with optimal hidden node number and sigmoidal activation function. In Fig. 10, peaking phenomenon also occurs. Thus, eigen-flame images need to be selected and such 11 most discriminative eigen-flame images are used to form global feature for the training and testing dataset throughout this sampling experiment. The significant degree of our eigen-flame image selection algorithm for global feature extraction is tested compared with traditional eigen-image selection method with 2000 repeats listed in Table 2 respectively. From Table 2, we have the following observations:
0.9 0.89 0.88 0.87 0.86
Optimal
0.85 0.84 0
2
4
6
8
10
12
Number of visual words Fig. 12. Classification performance vs. number of visual words.
14
W. Li et al. / Neurocomputing 102 (2013) 144–153
151
1
Table 3 Comparison results for various local feature extraction methods. Average accuracy (%)
No. of visual words
No. of semantics
Presented method Standard LSA-based Stop list-based Tf-idf table-based Presented method (without filter)
89.54 71.6 89.21 71.2 87.46 71.4 85.62 71.8 89.24 71.6
9.17 1.4 9 9 9 9
8.27 1.2 8 – – –
word number. In Fig. 12, such seven visual words are selected to construct the dictionary throughout this sampling experiment. In order to validate the effectiveness of our local feature extraction method, we have also tested the performance of standard LSA-based method, stop list analogy-based method [14], and tf-idf indexing table-based method with 2000 repeats listed in Table 3. From Table 3, we have the following observations: (a) Our Gabor filter bank pre-processing approach also facilitates the local feature extraction. Moreover, our method is feasible to extract local feature directly from the flame image, and can select more salient and discriminative semantics than the standard LSA-based method to form the more meaningful local feature. (b) LSA can extract more meaningful semantics than visual words, which not only generates low-dimensional local feature representation to mitigate the problem of synonymy. (c) During the stop list analogy, frequency is used to remove the most frequent and infrequent visual words to form better feature vector. However, evaluation of the discriminative power of visual words is not involved in such method, and the performance is hence interior.
4.4. Recognition performance based on concatenate feature After acquiring the above-mentioned individual feature, such features will be concatenate and normalized to recognize the final burning state based on ELM classifier. Again, sigmoidal activation function is used in ELM. The hidden node number has great effect on the recognition performance. Fig. 13 shows the recognition result with respect to the number of hidden node used during one sampling experiment. In such experiment, 57 is selected as the optimal hidden node number with 18-dimensional concatenate feature. Such hidden node number selection procedure is also used in the global and local feature selection. In order to validate the effectiveness of the used ELM classifier, we have also tested the performance of other classifiers, i.e. probabilistic neural network classifier (PNN) [38], neural network classifier (NN), and support vector machine classifier (SVM). PNN is constructed based on the well-known Bayesian classification technique, which considers the probability characteristic of sample space and uses typical samples as hidden layer nodes; NN, as a classifier, maps the feature space to a class label space. In the present study, the node number of the hidden layer is selected as two times the dimension of input features and the sigmoidal activation function is used; SVM is constructed based on the structural risk minimization principle. The basic idea of SVM is to map patterns from the input space to a transformed space, and then perform pattern classification in the transformed space using linear SVM. The recognition results and time-costing with 2000 repeats are listed in Table 4.
0.9
Classification performance
Various methods
0.8
0.7
0.6
Optimal 0.5
0.4 0
10
20
30
40
50
60
70
Number of hidden node Fig. 13. Recognition result vs. number of hidden node.
Table 4 Comparison results for various classifiers. Various classifiers
Training time (s)
Average accuracy (%)
ELM PNN NN SVM
1.781 4.391 61.766 8.344
92.75 7 2.8 90.49 7 3.3 88.92 7 4.0 90.12 7 4.3
As we can see in Table 4, various classifiers offer different contributions to the recognition result. For NN classifiers, the performance of multi-feature does not improve the final result and is even inferior to the individual feature because of the possible peak phenomenon. In our study, due to its characteristic, based on the selected optimal features, ELM shows its feasibility and the best recognition result for the burning state recognition of flame images compared to other classifiers. It can also be seen from Table 4, the standard deviation of the generalization performance of ELM is much smaller than other classifiers, meaning that ELM may run much more stable. Moreover, another important advantage of ELM classifier is the training speed, which is also included in Table 4. 4.5. Contrasted experiments Hereto, flame image has exhibited its power to recognize the burning state. Hereinafter, we will explore the effectiveness of our flame image-based method compared with other existing burning state recognition methods. Firstly, we compared our method with the temperature-based burning state recognition method. Taking the labeling by kiln operational experts as ground truth, the recognition accuracy of the two methods is 92.75% and 81.34% at the same time. Again, our flame image-based method outperforms the temperature-based method and is more reliable even with the disturbance of dust and smoke inside the kiln. Then, we compared our flame image-based method with four feasible image segmentation-based methods, including Ostu [23], fuzzy c mean and Gabor wavelet (FCMG) [5], dual fast marching (DFM) [6], and multistage adaptive threshold (MAT) [26]. Based on the experience of kiln operational experts, the following 16 features are extracted to characterize the flame color and the configuration of ROIs, namely average brightness of R sub-image, average brightness and its variance of the flame and material zones of R sub-image, area, length, width, circularity, and barycentric coordinates of the flame
152
W. Li et al. / Neurocomputing 102 (2013) 144–153
Table 5 Burning state recognition results for different image segmentation-based methods. Methods
PNN
NN
SVM
ELM
Ostu FCMG DFM MAT
42.67 711.4 44.72 712.3 47.23 711.5 52.19 711.3
40.26 7 11.1 41.62 7 12.5 44.31 7 12.3 49.29 7 12.3
41.79 7 11.2 42.83 7 12.1 46.32 7 13.2 51.78 7 11.3
42.13 711.2 46.45 79.1 49.63 710.1 55.79 79.2
zone, and height, width, area, and barycentric coordinates of the material zone. All these features are constructed to a feature vector to send to the ELM pattern classifier to obtain the recognition result. The average classification accuracy of 2000 repeats with different classifiers is shown in Table 5. Again, in our study, ELM gives the best recognition result except Ostu-based method. This is probably because Ostu is a general-purpose image segmentation method whilst the other methods are special methods for flame image or better image segmentation method. Moreover, obviously, although image segmentation based feature extraction has been successfully applied to many image recognition applications, in our application, the flame images are of poor quality due to the smoke and dust inside the kiln, and this in turn results in inaccurate ROIs segmentation and feature extraction. The above image segmentation-based methods are substantially inferior to our flame image-based method that is without the image segmentation procedure, and could work well only when the flame image is of high quality.
5. Conclusions and future work In this study, we have explored the feasibility of applying multifeature and ELM classifier to recognize the burning state of flame image for the rotary kiln without the difficult segmentation procedure. Firstly, MIA is employed to feature the color of flame image. Then, Gabor filter bank as the pre-processing step combined with the eigen-flame image decomposition is used to extract global features to represent the global configuration of flame image. Again, SIFT operator combined with the BoVW image descriptor and LSA is applied to extract local features to characterize the local configuration. The final recognition result is based on the integration of the above multi-feature and ELM classifier. For global and local features, we carry out extensive experiments to validate the effectiveness of feature extraction and necessity of feature selection, and then for the integrated optimal feature vector, we conduct comparative study of various classifier methods, temperature-based method, and image segmentation-based methods. Experimental results have demonstrated the feasibility and effectiveness of our feature method and ELM classifier. Exploring potentially more reasonable features and improving the existing ELM classifier to achieve better flame image burning state recognition result are our future work.
Acknowledgments This work is supported by the National Basic Research Program of China (2009CB320601), the Plan for University Subject Innovation and Introducing Intelligence (B08015), and Natural Science Foundation of China (61020106003), Matching grant for 1000 Talent Program (201100020). References [1] C.W. Ruby, A new approach to expert kiln control, in: XXXIX IEEE Conference on Cement Industry Technical, Hershey, USA, 1997, pp. 339–412.
[2] M. Akalp, A.L. Dominguez, R. Longchamp, Supervisory fuzzy control of a rotary cement kiln, in: 7th IEEE Conference on Electrotechnical, Antalya, Turkey, 1994, pp. 754–757. [3] M. Jarvensivu, K. Saari, S.L. Jamsa-Jounela, Intelligent control system of an industrial lime kiln process, Control Eng. Pract. 9 (6) (2001) 589–606. [4] X.J. Zhou, T.Y. Chai, Pattern-based hybrid intelligent control for rotary kiln process, in: IEEE International Conference on Control Applications, Singapore, 2007, pp. 31–35. [5] P. Sun, T.Y. Chai, X.J. Zhou, Rotary kiln flame image segmentation based on FCM and Gabor wavelet based texture coarseness, in: 7th World Congress on Intelligent Control and Automation, Chongqing, China, 2008, pp. 7615–7620. [6] H.Y. Jiang, X.L. Cui, X.J. Zhou, T.Y. Chai, Image segmentation based on improved dual fast marching method, J. Syst. Simulation 3 (20) (2008) 803–810. [7] L. Bertucco, A. Fichera, G. Nunnari, A. Pagano, A cellular neural networks approach to flame image analysis for combustion monitoring, in: Proceedings of the 6th IEEE International Workshop on Cellular Neural Networks and their Applications, Catania, Italy, 2000, pp. 455–459. [8] M. Idrissa, M. Acheroy, Texture classification using Gabor filters, Pattern Recognition Lett. 23 (9) (2002) 1095–1102. [9] A.K. Jain, R.P.W. Duin, J.C. Mao, Statistical pattern recognition: a review, IEEE Trans. Pattern Anal. Mach. Intell. 22 (1) (2000) 4–37. [10] M. Dash, H. Liu, Feature selection for classification, Intell. Data Anal. 1 (1) (1997) 131–156. [11] P. Geladi, H. Grahn, Multivariate Image Analysis, Wiley, New York, 1996. [12] J. Yang, D. Zhang, A.F. Frangi, J.Y. Yang, Two-dimensional PCA: a new approach to appearance-based face representation and recognition, IEEE Trans. Pattern Anal. Mach. Intell. 26 (1) (2004) 131–137. [13] D.G. Lowe, Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vision 60 (2) (2004) 91–110. [14] J. Sivic, A. Zisserman, A. Video google: a text retrieval approach to object matching in videos, in: Ninth IEEE International Conference on Computer Vision, Nice, France, 2003, pp. 1470–1477. [15] R.B. Yates, B.R. Neto, Modern Information Retrieval, ACM Press, New York, 1999. [16] S.C. Deerwester, S.T. Dumais, T.K. Landauer, G.W. Furnas, R.A. Harshman, Indexing by latent semantic analysis, J. Am. Soc. Inf. Sci. 41 (6) (1990) 391–407. [17] H. Witten, T.C. Bell, The zero-frequency problem: estimating the probabilities of novel events in adaptive text compression, IEEE Trans. Inf. Theory 37 (4) (1991) 1085–1094. [18] G.B. Huang, Q.Y. Zhu, C.K. Siew, Extreme learning machine: theory and applications, Neurocomputing 70 (2006) 489–501. [19] G.B. Huang, D.H. Wang, Y. Lan, Extreme learning machines: a survey, Int. J. Mach. Learn. Cybern. 2 (2) (2011) 107–122. [20] G.B. Huang, X.J. Ding, H.M. Zhou, Optimization method based extreme learning machine for classification, Neurocomputing 74 (1–3) (2010) 155–163. [21] N.Y. Chen, Physics and Chemistry of Alumina Production, Science and Technology Press, Shanghai, 1962. [22] Y.M. Wang, D.O. Yan, C.Y. Yang, C.Z. Gao, Alumina Production Process and Equipment, Metallurgical Industry Press, Beijing, 1987. [23] N. Otsu, A threshold selection method from gray-level histogram, IEEE Trans. SMC 9 (1) (1979) 62–66. [24] J.C. Dunn, A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters, J. Cybern. 3 (3) (1973) 32–57. [25] J.B. Shi, J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell. 22 (8) (2000) 888–905. [26] F.X. Yan, H. Zhang, C.R. Kube, A multistage adaptive thresholding method, Pattern Recognition Lett. 26 (8) (2005) 1183–1191. [27] G. Dong, N. Ray, S.T. Acton, Intravital leukocyte detection using the gradient inverse coefficient of variation, IEEE Trans. Med. Imaging 24 (7) (2005) 910–924. [28] B.N. Saha, N. Ray, Image thresholding by variational minimax optimization, Pattern Recognition 42 (5) (2009) 843–856. [29] F. Bianconi, A. Ferna´ndez, Evaluation of the effects of Gabor filter parameters on texture classification, Pattern Recognition 40 (12) (2007) 3325–3335. [30] W.T. Li, K.Z. Mao, H. Zhang, T.Y. Chai, Designing compact Gabor filter banks for efficient texture feature extraction, in: International Conference on Control, Automation, Robotics and Vision, Singapore, 2010, pp. 1193–1198. [31] H.L. Yu, J.F. MacGregor, Monitoring flames in an industrial boiler using multivariate image analysis, AIChE J. 50 (7) (2004) 1474–1483. [32] R.O. Duda, P.E. Hart, Pattern Classification and Scene Analysis, Wiley New York, 1973. [33] W.T. Li, K.Z. Mao, X.J. Zhou, T.Y. Chai, H. Zhang, Eigen-flame image-based robust recognition of burning states for sintering process control of rotary kiln, in: IEEE Conference on Decision and Control, Shanghai, China, 2009, pp. 398–403. [34] T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer, New York, 2001. [35] M.R. Chemick, Bootstrap Methods: A Practitioner’s Guide, Wiley, New York, 1999. [36] D.A. Clausi, H. Deng, Design-based texture feature fusion using Gabor filters and co-occurrence probabilities, IEEE Trans. Image Process. 14 (7) (2005) 925–936. [37] S. Li, J.S. Taylor, Comparison and fusion of multiresolution features for texture classification, Pattern Recognition Lett. 26 (5) (2004) 633–638. [38] D.F. Specht, Probabilistic neural network, Neural Networks 3 (1) (1990) 109–118.
W. Li et al. / Neurocomputing 102 (2013) 144–153 Weitao Li received the BSc degree and the MSc degree in Hefei University of Technology in 2004 and 2007, respectively. He is currently working toward the PhD degree in Northeastern University. His research interests include image processing and pattern recognition.
Dianhui Wang received the PhD degree from Northeastern University, Shenyang, China, in 1995. From 1995 to 2001, he worked as a Postdoctoral Fellow with Nanyang Technological University, Singapore, and a Researcher with The Hong Kong Polytechnic University, Hong Kong, China. He is currently a Reader and Associate Professor with the Department of Computer Science and Computer Engineering, La Trobe University, Melbourne, Victoria, Australia. He is also associated with the Key Laboratory of Synthetical Automation of Process Industries, Ministry of Education, Northeastern University. His current research interests include data mining and computational intelligence systems for bioinformatics and engineering applications.
153 Tianyou Chai received the PhD degree in control theory and engineering from Northeastern University, Shenyang, China, in 1985. He is the director of the Key Laboratory of Synthetical Automation of Process Industries, Ministry of Education, Northeastern University. His current research interests include adaptive control, intelligent decoupling control, and integrated automation of industrial process. Dr. Chai was elected as a member of the Chinese Academy of Engineering in 2003, an Academician of the International Eurasian Academy of Sciences in 2007, and an International Federation of Automatic Control Fellow in 2008.