- Email: [email protected]
Soft Computing of Biochemical Oxygen Demand Using an Improved T–S Fuzzy Neural Network
Chinese Journal of Chemical Engineering 22 (2014) 1254–1259
Contents lists available at ScienceDirect
Chinese Journal of Chemical Engineering journal homepage: www.elsevier.com/locate/CJCHE
Process Systems Engineering and Process Safety
Soft Computing of Biochemical Oxygen Demand Using an Improved T–S Fuzzy Neural Network☆ Junfei Qiao ⁎, Wei Li, Honggui Han College of Electronic Information and Control Engineering, Beijing University of Technology, Beijing 100124, China
a r t i c l e
i n f o
Article history: Received 3 January 2014 Received in revised form 21 February 2014 Accepted 8 March 2014 Available online 18 September 2014 Keywords: Biochemical oxygen demand Wastewater treatment T–S fuzzy neural network K-means clustering
a b s t r a c t It is difficult to measure the online values of biochemical oxygen demand (BOD) due to the characteristics of nonlinear dynamics, large lag and uncertainty in wastewater treatment process. In this paper, based on the knowledge representation ability and learning capability, an improved T–S fuzzy neural network (TSFNN) is introduced to predict BOD values by the soft computing method. In this improved TSFNN, a K-means clustering is used to initialize the structure of TSFNN, including the number of fuzzy rules and parameters of membership function. For training TSFNN, a gradient descent method with the momentum item is used to adjust antecedent parameters and consequent parameters. This improved TSFNN is applied to predict the BOD values in effluent of the wastewater treatment process. The simulation results show that the TSFNN with K-means clustering algorithm can measure the BOD values accurately. The algorithm presents better approximation performance than some other methods. © 2014 The Chemical Industry and Engineering Society of China, and Chemical Industry Press. All rights reserved.
1. Introduction Wastewater treatment process is a complex system including a variety of physical and biochemical reactions. Due to the nonlinear characteristics, delay-time and uncertainty [1], it is a challenge to measure the online values for effluent qualities in the wastewater treatment process. Biochemical oxygen demand (BOD), one of the most important effluent qualities, is significant to be measured accurately in time [2]. However, the values of BOD are obtained by 5 day measurement of field sampling according to conventional chemical methods. It needs a long time to analyze the BOD values, and the results cannot be used for monitoring the reaction process. Moreover, the BOD online monitoring instrument is very expensive and difficult to be implemented in the wastewater treatment plant. Recently, soft computing method was proposed as a useful tool to model complex systems for process industries [3–5]. In the last decades, the research on soft computing methods such as fuzzy set and neural network has become one of the most important topics in industrial applications [6,7]. In particular, integrating knowledge representation ability of fuzzy logic and great learning ability of neural network, fuzzy neural network (FNN) has attracted great interests of many researchers. It has been widely applied to system identification [8], pattern ☆ Supported by the National Natural Science Foundation of China (61203099, 61034008, 61225016), Beijing Science and Technology Project (Z141100001414005), Beijing Science and Technology Special Project (Z141101004414058), Ph.D. Program Foundation from Ministry of Chinese Education (20121103120020), Beijing Nova Program (Z131104000413007), and Hong Kong Scholar Program (XJ2013018). ⁎ Corresponding author. E-mail address: [email protected] (J. Qiao).
recognition [9], intelligent control [10] and other areas [11]. Especially, FNN is popular for modeling and controlling in the wastewater treatment process. Pan and Wang introduced a FNN-based chemical oxygen demand (COD) soft measuring technique to realize the COD monitoring online [2]. Li et al. proposed an improved FNN control method combined with the particle swarm optimization algorithm and back-propagation algorithm to control dissolved oxygen (DO) concentration of the wastewater treatment process [12]. The results play a significant role in the subsequent study for adjusting the parameters of FNN. However, most of the initial structures of FNN are determined by random selection, so it presents some uncertainty and limitation. Moreover, the dynamics of the wastewater treatment process is highly non-linear. Thus a reasonable initial structure of FNN is needed to give satisfactory performance. In this study, K-means clustering method, one of the most commonly used clustering methods [13], is used based on [14] to choose a suitable initial structure of FNN. A TS-type FNN model (TSFNN), including the number of fuzzy rules and initial values of parameters, is designed. With the initial structure, a gradient descent method is employed to adjust the antecedent parameters and consequent parameters of TSFNN in the parameter learning process. Then this TSFNN is applied to predict BOD values in the wastewater treatment process by the soft computing method. The performance of TSFNN is compared with that of different methods. 2. Wastewater Treatment Process With increasingly serious water pollution, wastewater treatment has become more and more important. Biological wastewater treatment is
http://dx.doi.org/10.1016/j.cjche.2014.09.023 1004-9541/© 2014 The Chemical Industry and Engineering Society of China, and Chemical Industry Press. All rights reserved.
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259
an effective method to remove organic pollutants and the secondary treatment part to dispose industrial and municipal domestic wastewater [15,16]. The mathematic model of a biological wastewater treatment process reflects the features and reaction processes of a group of bacteria. To study this complex ecosystem, the mathematical model plays an important role in analyzing the process and predicting the output of the process so as to implement necessary operations. As defined by the benchmark simulation model No. 1 [17], a biological wastewater treatment process is composed of five activated sludge reactors and a secondary settler, using the activated sludge model No. 1 [18] and a double-exponential settling velocity function [19] to describe the whole process. However, these mathematical models can only reflect some behavior in the whole biological wastewater treatment process due to its complexity. Recently, intelligent model methods are considered as an effective tool to describe the process. Neural networks, as a typical intelligent method, are used to model the wastewater treatment process [20,21]. There are lots of effluent qualities, such as total nitrogen, COD, ammonia nitrogen, total suspended solids, and BOD5 [17], all of which must meet the national standard. BOD is one of the most important effluent quality indexes for the biological wastewater treatment process. BOD5 means the measurement result of field sampling of 5 days with chemical method. This method needs much time to obtain the result. It is necessary to establish feature model with the help of easily measured variables.
3. The Improved TSFNN To predict BOD values, a soft computing method based on the TSFNN is employed for the wastewater treatment process in this paper.
1255
3.1. TSFNN The structure of TSFNN used here is shown in Fig. 1. This TSFNN is a multiple-input single-output system and consists of two parts including antecedent network and consequent network [22]. Each neuron in Layer 1 of the antecedent network represents an input variable xi (i = 1, 2, …, n), with the number of input variables n. Layer 2 of the antecedent network is the membership function layer. The Gaussian function is chosen as the membership function in this TSFNN, 2 2 − xi −ci j bi j ; i ¼ 1; 2; ⋯; n; j ¼ 1; 2; ⋯; m
j
μ i ¼ exp
ð1Þ
where μ ij is the jth membership function in the ith neuron, cij and bij are the center and width of the Gaussian function, respectively. Here we assume that the fuzzy segmentation number of each input variable xi is m, which is different from [22]. Layer 3 of the antecedent network is the fuzzy rule layer. Each neuron calculates the firing strength by an algebraic operation, j
j
j
ξ j ¼ μ 1 μ 2 ⋯μ n
ð2Þ
where ξj (j = 1, 2, …, m) is the firing strength as well as the adaptability of each rule, and m is the number of fuzzy rules. Layer 4 of the antecedent network is the normalized calculation, which is calculated as follows ξj ¼ ξj
X m γ¼1
Fig. 1. The structure of TSFNN.
ξγ ; j ¼ 1; 2; ⋯; m:
ð3Þ
1256
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259
There are three layers in the consequent network. Layer 1 is the input layer, which is almost the same as Layer 1 of the antecedent network. Layer 2 of the consequent network is the fuzzy rule layer. It is used to match the consequent part of fuzzy rule y j ¼ p j0 þ p j1 x1 þ ⋯ þ p jn xn ¼
n X
p ji xi ; j ¼ 1; 2; ⋯; m
ð4Þ
i¼0
where yj is the jth consequent part of fuzzy rule, and pji is parameters. Layer 3 of the consequent network is the output layer. It calculates the output of the whole system as follows y¼
m X
ξj yj
ð5Þ
j¼0
where y represents the network output, ξ j is the normalized output of the antecedent network, which is the same as Eq. (3), and yj is equal to Layer 2 of the consequent network. The learning process for this TSFNN is described in the next section.
Step 4: Repeat Step 2 and Step 3 until the cluster centers are no longer changed. Step 5: End and obtain k cluster. The result of this K-means clustering algorithm can be used to initialize the structure of TSFNN. The center of each cluster is used as the center of membership function and the minimum distance between clusters is used as the width of membership function. 3.3. Parameter adjustment To make the TSFNN more accurate, an appropriate method is introduced to adjust the parameters in the network. A gradient descent method with the momentum item [22] is used to update the center and width of membership function in the antecedent network and the connecting coefficient in the consequent network in the learning process. The cost function, E, is set as E¼
2 1 d y −y 2
ð8Þ
3.2. K-means clustering method for initial structure design The adjusting method for TSFNN contains structure designing algorithm and parameter learning algorithm [23], corresponding to the structure initialization and adjustment for the antecedent parameters and consequent parameters, respectively. A good initial structure of TSFNN can lead to satisfactory performance. Referring to [14], a Kmeans clustering method is employed to initialize the structure of TSFNN. The K-means clustering method is used to assign the input data points to cluster by the Euclidean distance computation. The formula of Euclidean distance is defined as d¼
N X
!1=2 2
kX i −Y i k
ð6Þ
where yd is the expected output of the system and y is the real output of the network. The goal of the parameter learning algorithm is to minimize E. Parameter pji is updated using the gradient descent method with the momentum item [22] ∂E þ α p ji ðt Þ−p ji ðt−1Þ ∂p ji ∂E ∂y ∂y j þ α p ji ðt Þ−p ji ðt−1Þ ¼ p ji ðt Þ−η ∂y ∂y j ∂p ji d ¼ p ji ðt Þ þ η y −y ξ j xi þ α p ji ðt Þ−p ji ðt−1Þ
p ji ðt þ 1Þ ¼ p ji ðt Þ−η
ð9Þ
ð j ¼ 1; 2; ⋯; m; i ¼ 1; 2; ⋯; nÞ
i¼1
where d represents the Euclidean distance between the two vectors X = [X1, X2, …, XN]T and Y = [Y1, Y2, …, YN]T, and N is the number of variables in the vector. Firstly, k data points are selected stochastically from the input data space as the initial clustering center. Secondly, the Euclidean distance between other data points and the initial clustering center is calculated, and the point with the minimum Euclidean distance to the cluster is assigned. Then the cluster center is updated in terms of the mean of data objects. Finally, the Euclidean distance between other data points and the new clustering center is calculated, and the data points to the nearest cluster are assigned. The clustering process will continue until all the data points belong to the cluster. The steps of the K-means clustering algorithm are as follows [24]. Input: the number of cluster k and data set D = {D1, D2, …, Dq} containing q data objects. Output: k cluster. Step 1: Select k data points from the data set as the initial clustering center. Step 2: Calculate the Euclidean distance between other data points and the initial clustering center, and assign these points to the nearest cluster. Step 3: Recalculate the mean of data objects for each cluster to update every cluster center j 1 X D ; j ¼ 1; 2; ⋯; k N j j¼1 j
N
Mj ¼
ð7Þ
where Mj represents the mean of cluster j, Nj is the number of data objects in cluster j, and Dj denotes the data point in cluster j.
where η is the learning rate, α is the momentum coefficient, m is the number of fuzzy rules, and n is the number of input variables. When adjusting the center and width of the membership function, we consider parameter pji to be known. The center cij and width bij of the Gaussian function are updated according to gradient descent algorithm with a momentum term [22] ci j ðt þ 1Þ ¼ ci j ðt Þ−η
∂E þ α ci j ðt Þ−ci j ðt−1Þ ∂ci j
ð10Þ
∂E þ α bi j ðt Þ−bi j ðt−1Þ : ∂bi j
ð11Þ
bi j ðt þ 1Þ ¼ bi j ðt Þ−η
The parameters of the membership function are updated as follows ∂E þ α ci j ðt Þ−ci j ðt−1Þ ∂ci j j ∂E ∂y ∂ξ j ∂ξ j ∂μ i þ α ci j ðt Þ−ci j ðt−1Þ ¼ ci j ðt Þ−η j ∂y ∂ξ j ∂ξ j ∂μ ∂ci j i 0 1 ! m X d l ¼ ci j ðt Þ þ 2η y −y y j @ ξi A ∏ μ h xi −ci j h≠i&l≠ j 0 2 1,i≠ j !2 ! m − x −c X i i j 2 B C exp @ ξ þ α ci j ðt Þ−ci j ðt−1Þ b A i i j b2i j i¼1
ci j ðt þ 1Þ ¼ ci j ðt Þ−η
ð12Þ
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259
1257
∂E þ α bi j ðt Þ−bi j ðt−1Þ ∂bi j j ∂E ∂y ∂ξ j ∂ξ j ∂μ i þ α bi j ðt Þ−bi j ðt−1Þ ¼ bi j ðt Þ−η ∂y ∂ξ j ∂ξ j ∂μ j ∂bi j 0i 1 ! m 2 X d l @ A ∏ μ h xi −ci j ξi ¼ bi j ðt Þ þ η y −y y j
bi j ðt þ 1Þ ¼ bi j ðt Þ−η
i≠ j
h≠i&l≠ j
0 2 1, !2 ! m X 3 B− xi −ci j C b þ α bi j ðt Þ−bi j ðt−1Þ exp @ ξ A i ij 2 bi j i¼1 ð13Þ where yj is the same as that in Eq. (4) and ξj is calculated by Eq. (2). 3.4. Improved TSFNN The learning procedure for applying TSFNN to calculate the BOD values by the soft computing method is summarized as follows. Step 1: Determine the input variables of the model according to the analysis for a wastewater treatment process. Step 2: Determine the number of cluster according to the real situation. Use K-means clustering algorithm to partition the input data into several clusters. Step 3: Initialize the structure of TSFNN. The number of cluster is equal to that of fuzzy rules. The center of each cluster is set to the center of the membership function, and the width of the membership function is determined by the minimum distance between each cluster. Step 4: Update the center and width of the membership function as well as the connecting coefficient by the gradient descent method with the momentum item. Step 5: Calculate the BOD values by the soft computing method with the improved TSFNN. Step 6: Stop when catching the stop time. Because K-means clustering algorithm is employed to initialize the network structure, this method can reduce the uncertainty from random selection.
Fig. 2. BOD values by the improved TSFNN in the training process.
Fig. 3. Error of BOD outputs by the improved TSFNN in the training process.
4. Simulation Experiment The improved TSFNN using K-means clustering algorithm is applied to predict the BOD values by the soft computing method in the wastewater treatment process. According to the analysis [17] and related researches [25,26], four variables COD, suspended solids (SS), pH and DO are selected as the input variables. The output is the BOD values. Then the TSFNN has four input variables and one output variable. The sample data for this experiment are from a small scale wastewater treatment plant in Beijing. 150 collected data are used as the training sample, and the other 50 are used as the testing sample. For the convenience of comparison, all the data are normalized to [− 1, 1]. The learning rate is set to 0.002, the momentum coefficient is set to 0.05, the maximum number of iterations is 20000, and the number of K-means cluster is set to 8. Other parameters are provided stochastically. This experiment is carried out under the MATLAB environment. In order to evaluate the availability of this method, the results are compared with other methods. The simulation results of BOD values for different methods are shown in Figs. 2–5. Fig. 2 compares BOD values between the real output and the output of improved TSFNN in the training process. Fig. 3 is the corresponding training error. Figs. 4 and 5 compare BOD predictions
Fig. 4. BOD predictions by the improved TSFNN in the testing process.
1258
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259 Table 1 Performance comparison of different methods Model
Epoch
Training NRMSE
Testing NRMSE
Running time/s
TSFNN with Kmeans clustering TSFNN KSOM model [27]
20000
0.2158
0.4207
79.62
20000 –
0.3717 0.5200
0.5193 0.5470
77.50 –
5. Conclusions
Fig. 5. BOD predictions by TSFNN in the testing process.
between the real output and the outputs of improved TSFNN and TSFNN, respectively, in the testing process. With improved TSFNN, the BOD output is almost coincident with the real output in the training process; although it has some deviation in the testing process, it is better than that with the TSFNN. Thus the TSFNN with K-means clustering presents better approximation performance. The K-means clustering algorithm is useful in initializing the structure of TSFNN. Fig. 6 shows the root mean square error (RMSE) obtained by the improved TSFNN in the training process. Table 1 gives the results with different methods for comparison, where NRMSE represents the normalized root mean square error. The results with the KSOM model are the same as those in the original paper [27]. The NRMSE for improved TSFNN with K-means clustering is 0.4207, which is smaller than that of the other two methods. The improved TSFNN presents better approximation performance in calculating BOD values in the biological wastewater treatment process than TSFNN, but at the cost of a little longer time spent on clustering method at the beginning. The cost time in seconds can be accepted for the wastewater treatment process because of the characteristic of large lag.
Fig. 6. RMSE values with the improved TSFNN in the training process.
This paper introduced a TSFNN to measure the BOD values by the soft computing method for the wastewater treatment process. The K-means clustering method was used to initialize the structure of TSFNN, including the number of fuzzy rules and parameters of the membership function. Simulation results show that the TSFNN using K-means clustering has better approximation performance in predicting BOD values than the other two methods, while the running time is a little longer than TSFNN because of the clustering algorithm. Only the structure identification is considered here for network performance. The parameter learning method is under study.
References [1] H.G. Han, J.F. Qiao, Q.L. Chen, Model predictive control of dissolved oxygen concentration based on a self-organizing RBF neural network, Control. Eng. Pract. 20 (2012) 465–476. [2] Z.R. Pan, Q. Wang, Based on fuzzy neural network COD soft measuring technique's research, Comput. Meas. Control 19 (7) (2011) 1572–1574. [3] F. Xu, Y.Y. Wang, X.L. Luo, Soft sensor for inputs and parameters using nonlinear singular state observer in chemical processes, Chin. J. Chem. Eng. 21 (9) (2013) 1038–1047. [4] W.M. Shao, X.M. Tian, P. Wang, Local partial least squares based online soft sensing method for multi-output processes with adaptive process states division, Chin. J. Chem. Eng. 22 (7) (2014) 828–836. [5] Y. Lü, H.Z. Yang, A multi-model approach for soft sensor development based on feature extraction using weighted kernel fisher criterion, Chin. J. Chem. Eng. 22 (2) (2014) 146–152. [6] H.S. Wong, L. Guan, A neural learning approach for adaptive image restoration using a fuzzy model-based network architecture, IEEE Trans. Neural Netw. 12 (3) (2001) 516–531. [7] Y.R. Huang, S.D. Zhang, Dissolved oxygen intelligent optimization control system in the aeration tank of wastewater treatment, Inf. Control. 40 (3) (2011) 393–400. [8] X.S. Deng, X.Z. Wang, Incremental learning of dynamic fuzzy neural networks for accurate system modeling, Fuzzy Set. Syst. 160 (2009) 972–987. [9] H.G. Han, J.F. Qiao, A self-organizing fuzzy neural network based on growing-andpruning algorithm, IEEE Trans. Fuzzy Syst. 18 (6) (2010) 1129–1143. [10] J.S. Wang, C.S.G. Lee, Self-adaptive recurrent neuro fuzzy control of an autonomous underwater vehicle, IEEE Trans. Robot. Autom. 19 (2) (2003) 283–295. [11] H.G. Han, X.L. Wu, J.F. Qiao, Nonlinear systems modeling based on self-organizing fuzzy-neural-network with adaptive computation algorithm, IEEE Trans. Cybern. 44 (4) (2014) 554–564. [12] X.J. Li, J.X. Luo, Z.Q. Huang, W.J. Wang, Fuzzy neural network controller of dissolved oxygen in the wastewater treatment process, Control Eng. China 17 (2010) 64–67 (Suppl.). [13] J. MacQueen, Some Methods for Classification and Analysis of Multivariate Observations, Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1, 1967, pp. 281–297. [14] R.L. Liu, H.Y. Su, J. Chu, Fuzzy neural network based on hybrid learning algorithm and its application to soft sensor, J. Syst. Simul. 17 (12) (2005) 2878–2881. [15] T. Takagi, M. Sugeno, Fuzzy identification of systems and its applications to modeling and control, IEEE Trans. Syst. Man Cybern. SMC-15 (1) (1985) 116–132. [16] S.X. Du, Modeling and control of biological wastewater treatment processes, Control Theory Appl. 19 (5) (2002) 660–666. [17] J. Alex, L. Benedetti, J. Copp, K.V. Gernaey, U. Jeppsson, I. Nopens, M.N. Pons, C. Rosen, J.P. Steyer, P. Vanrolleghem, J. Alex, L. Benedetti, J. Copp, K.V. Gernaey, U. Jeppsson, I. Nopens, M.N. Pons, C. Rosen, J.P. Steyer, P. Vanrolleghem, Benchmark simulation model no. 1 (BSM1), IWA Taskgroup on Benchmarking of Control Strategies for WWTPs, 2008, pp. 1–61. [18] M. Henze, C.P.L. Grady, W. Gujor, G.V.R. Marais, T. Mastuo, Activated sludge model no. 1, Int. Association on Water Pollution Research and Control (IAWPRC), Sci. and Tech. Repts., No. 1, London, England2002. [19] I. Takacs, G.G. Patry, D. Nolasco, A Dynamic Model of the Clarification Thickening Process, Water Res. 25 (1991) 1263–1271. [20] Z.H. Tong, M.W. Xu, Y.M. Shen, L.Z. Yu, Study on artificial neural network in the sewage disposal model, Chin. J. Sc. Instrum. 26 (8 Suppl.) (2005) 347–348. [21] Q.M. Cong, T.Y. Chai, W. Yu, Modeling wastewater treatment plant via hierarchical neural networks, Control Theory Appl. 26 (1) (2009) 8–14.
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259 [22] G.Y. Li, Neural Fuzzy Control Theory and Applications, Electronic Industry Press, Beijing, 2009. 291–296. [23] G. Leng, G. Prasad, T.M. McGinnity, An on-line algorithm for creating self-organizing fuzzy neural networks, Neural Netw. 17 (2004) 1477–1493. [24] N. Shi, X.M. Liu, Y. Guan, Research on K-means clustering algorithm, Third International Symposium on Intelligent Information Technology and Security Informatics, 2010, pp. 63–67. [25] X.Y. Wang, Z.W. Liu, Z. Su, Research on BOD soft-sensing compensation method based on neural network, J. Syst. Simul. 21 (Suppl. 2) (2009) 83–85.
1259
[26] M.N. Zhang, H.G. Han, J.F. Qiao, Research on dynamic feed-forward neural network structure based on growing and pruning methods, CAAI Trans. Intell. Syst. 6 (2) (2011) 101–106. [27] R. Rabee, J.A. Adebayo, S. Miklas, Applying Kohonen self-organizing map as a software sensor to predict biochemical oxygen demand, Water Environ. Res. 80 (1) (2008) 32–40.
Contents lists available at ScienceDirect
Chinese Journal of Chemical Engineering journal homepage: www.elsevier.com/locate/CJCHE
Process Systems Engineering and Process Safety
Soft Computing of Biochemical Oxygen Demand Using an Improved T–S Fuzzy Neural Network☆ Junfei Qiao ⁎, Wei Li, Honggui Han College of Electronic Information and Control Engineering, Beijing University of Technology, Beijing 100124, China
a r t i c l e
i n f o
Article history: Received 3 January 2014 Received in revised form 21 February 2014 Accepted 8 March 2014 Available online 18 September 2014 Keywords: Biochemical oxygen demand Wastewater treatment T–S fuzzy neural network K-means clustering
a b s t r a c t It is difficult to measure the online values of biochemical oxygen demand (BOD) due to the characteristics of nonlinear dynamics, large lag and uncertainty in wastewater treatment process. In this paper, based on the knowledge representation ability and learning capability, an improved T–S fuzzy neural network (TSFNN) is introduced to predict BOD values by the soft computing method. In this improved TSFNN, a K-means clustering is used to initialize the structure of TSFNN, including the number of fuzzy rules and parameters of membership function. For training TSFNN, a gradient descent method with the momentum item is used to adjust antecedent parameters and consequent parameters. This improved TSFNN is applied to predict the BOD values in effluent of the wastewater treatment process. The simulation results show that the TSFNN with K-means clustering algorithm can measure the BOD values accurately. The algorithm presents better approximation performance than some other methods. © 2014 The Chemical Industry and Engineering Society of China, and Chemical Industry Press. All rights reserved.
1. Introduction Wastewater treatment process is a complex system including a variety of physical and biochemical reactions. Due to the nonlinear characteristics, delay-time and uncertainty [1], it is a challenge to measure the online values for effluent qualities in the wastewater treatment process. Biochemical oxygen demand (BOD), one of the most important effluent qualities, is significant to be measured accurately in time [2]. However, the values of BOD are obtained by 5 day measurement of field sampling according to conventional chemical methods. It needs a long time to analyze the BOD values, and the results cannot be used for monitoring the reaction process. Moreover, the BOD online monitoring instrument is very expensive and difficult to be implemented in the wastewater treatment plant. Recently, soft computing method was proposed as a useful tool to model complex systems for process industries [3–5]. In the last decades, the research on soft computing methods such as fuzzy set and neural network has become one of the most important topics in industrial applications [6,7]. In particular, integrating knowledge representation ability of fuzzy logic and great learning ability of neural network, fuzzy neural network (FNN) has attracted great interests of many researchers. It has been widely applied to system identification [8], pattern ☆ Supported by the National Natural Science Foundation of China (61203099, 61034008, 61225016), Beijing Science and Technology Project (Z141100001414005), Beijing Science and Technology Special Project (Z141101004414058), Ph.D. Program Foundation from Ministry of Chinese Education (20121103120020), Beijing Nova Program (Z131104000413007), and Hong Kong Scholar Program (XJ2013018). ⁎ Corresponding author. E-mail address: [email protected] (J. Qiao).
recognition [9], intelligent control [10] and other areas [11]. Especially, FNN is popular for modeling and controlling in the wastewater treatment process. Pan and Wang introduced a FNN-based chemical oxygen demand (COD) soft measuring technique to realize the COD monitoring online [2]. Li et al. proposed an improved FNN control method combined with the particle swarm optimization algorithm and back-propagation algorithm to control dissolved oxygen (DO) concentration of the wastewater treatment process [12]. The results play a significant role in the subsequent study for adjusting the parameters of FNN. However, most of the initial structures of FNN are determined by random selection, so it presents some uncertainty and limitation. Moreover, the dynamics of the wastewater treatment process is highly non-linear. Thus a reasonable initial structure of FNN is needed to give satisfactory performance. In this study, K-means clustering method, one of the most commonly used clustering methods [13], is used based on [14] to choose a suitable initial structure of FNN. A TS-type FNN model (TSFNN), including the number of fuzzy rules and initial values of parameters, is designed. With the initial structure, a gradient descent method is employed to adjust the antecedent parameters and consequent parameters of TSFNN in the parameter learning process. Then this TSFNN is applied to predict BOD values in the wastewater treatment process by the soft computing method. The performance of TSFNN is compared with that of different methods. 2. Wastewater Treatment Process With increasingly serious water pollution, wastewater treatment has become more and more important. Biological wastewater treatment is
http://dx.doi.org/10.1016/j.cjche.2014.09.023 1004-9541/© 2014 The Chemical Industry and Engineering Society of China, and Chemical Industry Press. All rights reserved.
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259
an effective method to remove organic pollutants and the secondary treatment part to dispose industrial and municipal domestic wastewater [15,16]. The mathematic model of a biological wastewater treatment process reflects the features and reaction processes of a group of bacteria. To study this complex ecosystem, the mathematical model plays an important role in analyzing the process and predicting the output of the process so as to implement necessary operations. As defined by the benchmark simulation model No. 1 [17], a biological wastewater treatment process is composed of five activated sludge reactors and a secondary settler, using the activated sludge model No. 1 [18] and a double-exponential settling velocity function [19] to describe the whole process. However, these mathematical models can only reflect some behavior in the whole biological wastewater treatment process due to its complexity. Recently, intelligent model methods are considered as an effective tool to describe the process. Neural networks, as a typical intelligent method, are used to model the wastewater treatment process [20,21]. There are lots of effluent qualities, such as total nitrogen, COD, ammonia nitrogen, total suspended solids, and BOD5 [17], all of which must meet the national standard. BOD is one of the most important effluent quality indexes for the biological wastewater treatment process. BOD5 means the measurement result of field sampling of 5 days with chemical method. This method needs much time to obtain the result. It is necessary to establish feature model with the help of easily measured variables.
3. The Improved TSFNN To predict BOD values, a soft computing method based on the TSFNN is employed for the wastewater treatment process in this paper.
1255
3.1. TSFNN The structure of TSFNN used here is shown in Fig. 1. This TSFNN is a multiple-input single-output system and consists of two parts including antecedent network and consequent network [22]. Each neuron in Layer 1 of the antecedent network represents an input variable xi (i = 1, 2, …, n), with the number of input variables n. Layer 2 of the antecedent network is the membership function layer. The Gaussian function is chosen as the membership function in this TSFNN, 2 2 − xi −ci j bi j ; i ¼ 1; 2; ⋯; n; j ¼ 1; 2; ⋯; m
j
μ i ¼ exp
ð1Þ
where μ ij is the jth membership function in the ith neuron, cij and bij are the center and width of the Gaussian function, respectively. Here we assume that the fuzzy segmentation number of each input variable xi is m, which is different from [22]. Layer 3 of the antecedent network is the fuzzy rule layer. Each neuron calculates the firing strength by an algebraic operation, j
j
j
ξ j ¼ μ 1 μ 2 ⋯μ n
ð2Þ
where ξj (j = 1, 2, …, m) is the firing strength as well as the adaptability of each rule, and m is the number of fuzzy rules. Layer 4 of the antecedent network is the normalized calculation, which is calculated as follows ξj ¼ ξj
X m γ¼1
Fig. 1. The structure of TSFNN.
ξγ ; j ¼ 1; 2; ⋯; m:
ð3Þ
1256
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259
There are three layers in the consequent network. Layer 1 is the input layer, which is almost the same as Layer 1 of the antecedent network. Layer 2 of the consequent network is the fuzzy rule layer. It is used to match the consequent part of fuzzy rule y j ¼ p j0 þ p j1 x1 þ ⋯ þ p jn xn ¼
n X
p ji xi ; j ¼ 1; 2; ⋯; m
ð4Þ
i¼0
where yj is the jth consequent part of fuzzy rule, and pji is parameters. Layer 3 of the consequent network is the output layer. It calculates the output of the whole system as follows y¼
m X
ξj yj
ð5Þ
j¼0
where y represents the network output, ξ j is the normalized output of the antecedent network, which is the same as Eq. (3), and yj is equal to Layer 2 of the consequent network. The learning process for this TSFNN is described in the next section.
Step 4: Repeat Step 2 and Step 3 until the cluster centers are no longer changed. Step 5: End and obtain k cluster. The result of this K-means clustering algorithm can be used to initialize the structure of TSFNN. The center of each cluster is used as the center of membership function and the minimum distance between clusters is used as the width of membership function. 3.3. Parameter adjustment To make the TSFNN more accurate, an appropriate method is introduced to adjust the parameters in the network. A gradient descent method with the momentum item [22] is used to update the center and width of membership function in the antecedent network and the connecting coefficient in the consequent network in the learning process. The cost function, E, is set as E¼
2 1 d y −y 2
ð8Þ
3.2. K-means clustering method for initial structure design The adjusting method for TSFNN contains structure designing algorithm and parameter learning algorithm [23], corresponding to the structure initialization and adjustment for the antecedent parameters and consequent parameters, respectively. A good initial structure of TSFNN can lead to satisfactory performance. Referring to [14], a Kmeans clustering method is employed to initialize the structure of TSFNN. The K-means clustering method is used to assign the input data points to cluster by the Euclidean distance computation. The formula of Euclidean distance is defined as d¼
N X
!1=2 2
kX i −Y i k
ð6Þ
where yd is the expected output of the system and y is the real output of the network. The goal of the parameter learning algorithm is to minimize E. Parameter pji is updated using the gradient descent method with the momentum item [22] ∂E þ α p ji ðt Þ−p ji ðt−1Þ ∂p ji ∂E ∂y ∂y j þ α p ji ðt Þ−p ji ðt−1Þ ¼ p ji ðt Þ−η ∂y ∂y j ∂p ji d ¼ p ji ðt Þ þ η y −y ξ j xi þ α p ji ðt Þ−p ji ðt−1Þ
p ji ðt þ 1Þ ¼ p ji ðt Þ−η
ð9Þ
ð j ¼ 1; 2; ⋯; m; i ¼ 1; 2; ⋯; nÞ
i¼1
where d represents the Euclidean distance between the two vectors X = [X1, X2, …, XN]T and Y = [Y1, Y2, …, YN]T, and N is the number of variables in the vector. Firstly, k data points are selected stochastically from the input data space as the initial clustering center. Secondly, the Euclidean distance between other data points and the initial clustering center is calculated, and the point with the minimum Euclidean distance to the cluster is assigned. Then the cluster center is updated in terms of the mean of data objects. Finally, the Euclidean distance between other data points and the new clustering center is calculated, and the data points to the nearest cluster are assigned. The clustering process will continue until all the data points belong to the cluster. The steps of the K-means clustering algorithm are as follows [24]. Input: the number of cluster k and data set D = {D1, D2, …, Dq} containing q data objects. Output: k cluster. Step 1: Select k data points from the data set as the initial clustering center. Step 2: Calculate the Euclidean distance between other data points and the initial clustering center, and assign these points to the nearest cluster. Step 3: Recalculate the mean of data objects for each cluster to update every cluster center j 1 X D ; j ¼ 1; 2; ⋯; k N j j¼1 j
N
Mj ¼
ð7Þ
where Mj represents the mean of cluster j, Nj is the number of data objects in cluster j, and Dj denotes the data point in cluster j.
where η is the learning rate, α is the momentum coefficient, m is the number of fuzzy rules, and n is the number of input variables. When adjusting the center and width of the membership function, we consider parameter pji to be known. The center cij and width bij of the Gaussian function are updated according to gradient descent algorithm with a momentum term [22] ci j ðt þ 1Þ ¼ ci j ðt Þ−η
∂E þ α ci j ðt Þ−ci j ðt−1Þ ∂ci j
ð10Þ
∂E þ α bi j ðt Þ−bi j ðt−1Þ : ∂bi j
ð11Þ
bi j ðt þ 1Þ ¼ bi j ðt Þ−η
The parameters of the membership function are updated as follows ∂E þ α ci j ðt Þ−ci j ðt−1Þ ∂ci j j ∂E ∂y ∂ξ j ∂ξ j ∂μ i þ α ci j ðt Þ−ci j ðt−1Þ ¼ ci j ðt Þ−η j ∂y ∂ξ j ∂ξ j ∂μ ∂ci j i 0 1 ! m X d l ¼ ci j ðt Þ þ 2η y −y y j @ ξi A ∏ μ h xi −ci j h≠i&l≠ j 0 2 1,i≠ j !2 ! m − x −c X i i j 2 B C exp @ ξ þ α ci j ðt Þ−ci j ðt−1Þ b A i i j b2i j i¼1
ci j ðt þ 1Þ ¼ ci j ðt Þ−η
ð12Þ
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259
1257
∂E þ α bi j ðt Þ−bi j ðt−1Þ ∂bi j j ∂E ∂y ∂ξ j ∂ξ j ∂μ i þ α bi j ðt Þ−bi j ðt−1Þ ¼ bi j ðt Þ−η ∂y ∂ξ j ∂ξ j ∂μ j ∂bi j 0i 1 ! m 2 X d l @ A ∏ μ h xi −ci j ξi ¼ bi j ðt Þ þ η y −y y j
bi j ðt þ 1Þ ¼ bi j ðt Þ−η
i≠ j
h≠i&l≠ j
0 2 1, !2 ! m X 3 B− xi −ci j C b þ α bi j ðt Þ−bi j ðt−1Þ exp @ ξ A i ij 2 bi j i¼1 ð13Þ where yj is the same as that in Eq. (4) and ξj is calculated by Eq. (2). 3.4. Improved TSFNN The learning procedure for applying TSFNN to calculate the BOD values by the soft computing method is summarized as follows. Step 1: Determine the input variables of the model according to the analysis for a wastewater treatment process. Step 2: Determine the number of cluster according to the real situation. Use K-means clustering algorithm to partition the input data into several clusters. Step 3: Initialize the structure of TSFNN. The number of cluster is equal to that of fuzzy rules. The center of each cluster is set to the center of the membership function, and the width of the membership function is determined by the minimum distance between each cluster. Step 4: Update the center and width of the membership function as well as the connecting coefficient by the gradient descent method with the momentum item. Step 5: Calculate the BOD values by the soft computing method with the improved TSFNN. Step 6: Stop when catching the stop time. Because K-means clustering algorithm is employed to initialize the network structure, this method can reduce the uncertainty from random selection.
Fig. 2. BOD values by the improved TSFNN in the training process.
Fig. 3. Error of BOD outputs by the improved TSFNN in the training process.
4. Simulation Experiment The improved TSFNN using K-means clustering algorithm is applied to predict the BOD values by the soft computing method in the wastewater treatment process. According to the analysis [17] and related researches [25,26], four variables COD, suspended solids (SS), pH and DO are selected as the input variables. The output is the BOD values. Then the TSFNN has four input variables and one output variable. The sample data for this experiment are from a small scale wastewater treatment plant in Beijing. 150 collected data are used as the training sample, and the other 50 are used as the testing sample. For the convenience of comparison, all the data are normalized to [− 1, 1]. The learning rate is set to 0.002, the momentum coefficient is set to 0.05, the maximum number of iterations is 20000, and the number of K-means cluster is set to 8. Other parameters are provided stochastically. This experiment is carried out under the MATLAB environment. In order to evaluate the availability of this method, the results are compared with other methods. The simulation results of BOD values for different methods are shown in Figs. 2–5. Fig. 2 compares BOD values between the real output and the output of improved TSFNN in the training process. Fig. 3 is the corresponding training error. Figs. 4 and 5 compare BOD predictions
Fig. 4. BOD predictions by the improved TSFNN in the testing process.
1258
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259 Table 1 Performance comparison of different methods Model
Epoch
Training NRMSE
Testing NRMSE
Running time/s
TSFNN with Kmeans clustering TSFNN KSOM model [27]
20000
0.2158
0.4207
79.62
20000 –
0.3717 0.5200
0.5193 0.5470
77.50 –
5. Conclusions
Fig. 5. BOD predictions by TSFNN in the testing process.
between the real output and the outputs of improved TSFNN and TSFNN, respectively, in the testing process. With improved TSFNN, the BOD output is almost coincident with the real output in the training process; although it has some deviation in the testing process, it is better than that with the TSFNN. Thus the TSFNN with K-means clustering presents better approximation performance. The K-means clustering algorithm is useful in initializing the structure of TSFNN. Fig. 6 shows the root mean square error (RMSE) obtained by the improved TSFNN in the training process. Table 1 gives the results with different methods for comparison, where NRMSE represents the normalized root mean square error. The results with the KSOM model are the same as those in the original paper [27]. The NRMSE for improved TSFNN with K-means clustering is 0.4207, which is smaller than that of the other two methods. The improved TSFNN presents better approximation performance in calculating BOD values in the biological wastewater treatment process than TSFNN, but at the cost of a little longer time spent on clustering method at the beginning. The cost time in seconds can be accepted for the wastewater treatment process because of the characteristic of large lag.
Fig. 6. RMSE values with the improved TSFNN in the training process.
This paper introduced a TSFNN to measure the BOD values by the soft computing method for the wastewater treatment process. The K-means clustering method was used to initialize the structure of TSFNN, including the number of fuzzy rules and parameters of the membership function. Simulation results show that the TSFNN using K-means clustering has better approximation performance in predicting BOD values than the other two methods, while the running time is a little longer than TSFNN because of the clustering algorithm. Only the structure identification is considered here for network performance. The parameter learning method is under study.
References [1] H.G. Han, J.F. Qiao, Q.L. Chen, Model predictive control of dissolved oxygen concentration based on a self-organizing RBF neural network, Control. Eng. Pract. 20 (2012) 465–476. [2] Z.R. Pan, Q. Wang, Based on fuzzy neural network COD soft measuring technique's research, Comput. Meas. Control 19 (7) (2011) 1572–1574. [3] F. Xu, Y.Y. Wang, X.L. Luo, Soft sensor for inputs and parameters using nonlinear singular state observer in chemical processes, Chin. J. Chem. Eng. 21 (9) (2013) 1038–1047. [4] W.M. Shao, X.M. Tian, P. Wang, Local partial least squares based online soft sensing method for multi-output processes with adaptive process states division, Chin. J. Chem. Eng. 22 (7) (2014) 828–836. [5] Y. Lü, H.Z. Yang, A multi-model approach for soft sensor development based on feature extraction using weighted kernel fisher criterion, Chin. J. Chem. Eng. 22 (2) (2014) 146–152. [6] H.S. Wong, L. Guan, A neural learning approach for adaptive image restoration using a fuzzy model-based network architecture, IEEE Trans. Neural Netw. 12 (3) (2001) 516–531. [7] Y.R. Huang, S.D. Zhang, Dissolved oxygen intelligent optimization control system in the aeration tank of wastewater treatment, Inf. Control. 40 (3) (2011) 393–400. [8] X.S. Deng, X.Z. Wang, Incremental learning of dynamic fuzzy neural networks for accurate system modeling, Fuzzy Set. Syst. 160 (2009) 972–987. [9] H.G. Han, J.F. Qiao, A self-organizing fuzzy neural network based on growing-andpruning algorithm, IEEE Trans. Fuzzy Syst. 18 (6) (2010) 1129–1143. [10] J.S. Wang, C.S.G. Lee, Self-adaptive recurrent neuro fuzzy control of an autonomous underwater vehicle, IEEE Trans. Robot. Autom. 19 (2) (2003) 283–295. [11] H.G. Han, X.L. Wu, J.F. Qiao, Nonlinear systems modeling based on self-organizing fuzzy-neural-network with adaptive computation algorithm, IEEE Trans. Cybern. 44 (4) (2014) 554–564. [12] X.J. Li, J.X. Luo, Z.Q. Huang, W.J. Wang, Fuzzy neural network controller of dissolved oxygen in the wastewater treatment process, Control Eng. China 17 (2010) 64–67 (Suppl.). [13] J. MacQueen, Some Methods for Classification and Analysis of Multivariate Observations, Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1, 1967, pp. 281–297. [14] R.L. Liu, H.Y. Su, J. Chu, Fuzzy neural network based on hybrid learning algorithm and its application to soft sensor, J. Syst. Simul. 17 (12) (2005) 2878–2881. [15] T. Takagi, M. Sugeno, Fuzzy identification of systems and its applications to modeling and control, IEEE Trans. Syst. Man Cybern. SMC-15 (1) (1985) 116–132. [16] S.X. Du, Modeling and control of biological wastewater treatment processes, Control Theory Appl. 19 (5) (2002) 660–666. [17] J. Alex, L. Benedetti, J. Copp, K.V. Gernaey, U. Jeppsson, I. Nopens, M.N. Pons, C. Rosen, J.P. Steyer, P. Vanrolleghem, J. Alex, L. Benedetti, J. Copp, K.V. Gernaey, U. Jeppsson, I. Nopens, M.N. Pons, C. Rosen, J.P. Steyer, P. Vanrolleghem, Benchmark simulation model no. 1 (BSM1), IWA Taskgroup on Benchmarking of Control Strategies for WWTPs, 2008, pp. 1–61. [18] M. Henze, C.P.L. Grady, W. Gujor, G.V.R. Marais, T. Mastuo, Activated sludge model no. 1, Int. Association on Water Pollution Research and Control (IAWPRC), Sci. and Tech. Repts., No. 1, London, England2002. [19] I. Takacs, G.G. Patry, D. Nolasco, A Dynamic Model of the Clarification Thickening Process, Water Res. 25 (1991) 1263–1271. [20] Z.H. Tong, M.W. Xu, Y.M. Shen, L.Z. Yu, Study on artificial neural network in the sewage disposal model, Chin. J. Sc. Instrum. 26 (8 Suppl.) (2005) 347–348. [21] Q.M. Cong, T.Y. Chai, W. Yu, Modeling wastewater treatment plant via hierarchical neural networks, Control Theory Appl. 26 (1) (2009) 8–14.
J. Qiao et al. / Chinese Journal of Chemical Engineering 22 (2014) 1254–1259 [22] G.Y. Li, Neural Fuzzy Control Theory and Applications, Electronic Industry Press, Beijing, 2009. 291–296. [23] G. Leng, G. Prasad, T.M. McGinnity, An on-line algorithm for creating self-organizing fuzzy neural networks, Neural Netw. 17 (2004) 1477–1493. [24] N. Shi, X.M. Liu, Y. Guan, Research on K-means clustering algorithm, Third International Symposium on Intelligent Information Technology and Security Informatics, 2010, pp. 63–67. [25] X.Y. Wang, Z.W. Liu, Z. Su, Research on BOD soft-sensing compensation method based on neural network, J. Syst. Simul. 21 (Suppl. 2) (2009) 83–85.
1259
[26] M.N. Zhang, H.G. Han, J.F. Qiao, Research on dynamic feed-forward neural network structure based on growing and pruning methods, CAAI Trans. Intell. Syst. 6 (2) (2011) 101–106. [27] R. Rabee, J.A. Adebayo, S. Miklas, Applying Kohonen self-organizing map as a software sensor to predict biochemical oxygen demand, Water Environ. Res. 80 (1) (2008) 32–40.