- Email: [email protected]
Modelling and optimal control of fed-batch processes using a novel control affine feedforward neural network
Neurocomputing 61 (2004) 317 – 337
www.elsevier.com/locate/neucom
Modelling and optimal control of fed-batch processes using a novel control a$ne feedforward neural network Zhihua Xiong, Jie Zhang∗ Centre for Process Analytics and Control Technology, School of Chemical Engineering and Advanced Materials, University of Newcastle, Newcastle upon Tyne NE1 7RU, UK Received 13 March 2003; received in revised form 26 September 2003; accepted 19 November 2003
Abstract Many fed-batch processes can be considered as a class of control a$ne nonlinear systems. In this paper, a new type of neural network for modelling fed-batch processes, called as control a$ne feedforward neural network (CAFNN), is proposed. For constrained nonlinear optimal control of fed-batch processes, CAFNN o3ers an e3ective and simple optimal control strategy by sequential quadratic programming (SQP) where the gradient information can be computed directly from CAFNN. Thus the nonlinear programming problem can then be solved more accurately and e$ciently. The proposed modelling and optimal control scheme are illustrated on a nonlinear system and a simulated fed-batch ethanol fermentation process. c 2003 Elsevier B.V. All rights reserved. Keywords: Neural networks; Control a$ne nonlinear systems; Fed-batch processes; Optimal control
1. Introduction In recent years, there has been a growing interest in the optimal control of fed-batch processes that are suitable for responsive manufacturing. In a fed-batch process, our interests are concerned with determining the feeding policy to the reactor that will give the maximum amount of the desired product. Although only one single control variable in the form of the feed rate may appear to present a simple optimal control ∗
Corresponding author. Department of Chemical and Process Engineering, University of Newcastle, Newcastle upon Tyne NE1 7RU, UK. Tel.: +44-191-2227240; fax: +44-191-2225292. E-mail address: [email protected] (J. Zhang). c 2003 Elsevier B.V. All rights reserved. 0925-2312/$ - see front matter doi:10.1016/j.neucom.2003.11.006
318
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
problem, considerable di$culties have been reported in the determination of the optimal feed rate policy for fed-batch processes, because fed-batch processes, in general, are nonlinear dynamic systems. Most of the optimisation studies on batch processes have reported the use of detailed and complicated Hrst principle models [11,13]. However, it is usually very di$cult to obtain those models due to the complexity of the processes, and it is also di$cult to implement them for optimal control. When building Hrst-principles models is very costly and di$cult, empirical models based on process input–output data can provide a useful alternative [14–16]. Neural networks (NN) have been proposed as a promising tool for identifying empirical models [2,18]. Neural networks with their massive parallelism and learning capabilities have been shown to be capable of approximating any continuous nonlinear functions [4]. If properly trained and validated, these neural network models can be used to accurately predict steady-state and dynamic process behaviour, hence, leading to improved process optimisation and control performance [3,8,19]. In general, optimal control of a fed-batch process can be described as a constrained end-point optimal control problem of a nonlinear control a$ne process [17]. It is usually very di$cult to solve this kind of problem by using the conventional Pontryagin’s Maximum Principle (PMP). For a control a$ne fed-batch process optimal control problem whose Hamiltonian (H ) is independent of the control (u), the partial derivative of the Hamiltonian with respect to the control variable, @H =@u, is equal to zero and, therefore, singular arcs exist. The di$culty of directly applying PMP is due to the existence of singular arcs and the low sensitivity of the control policy on the yield of the batch process. Alternatively, gradient-free methods, also called as direct search methods, are feasible computational methods for this kind of problems and they can possibly obtain the global optimum. Many direct search methods have been reported, such as iterative dynamic programming (IDP) [10,12] and heuristic random optimisation [9]. However, most of the direct search methods are very time consuming and ine$cient because they usually require a large number of iterations to search for the global minimum. On the other hand, gradient-based methods, such as sequential quadratic programming (SQP), are more e3ective for constrained optimisation problems. The SQP method is constructed by solving a quadratic programming (QP) sub-problem at each major iteration. An approximation is made of the Hessian of the Lagrangian function using a quasi-Newton updating method. This is then used to generate a QP sub-problem whose solution is used to form a search direction for a line search procedure. However, this kind of computational method has to cope with problems of numerical evaluation of derivatives and feasibility issues. In general, the analytic partial derivatives of the objective function as well as partial derivatives of the constraints are very di$cult to be obtained in many nonlinear systems. Then the gradients have to be calculated by Hnite di3erence approximation in the SQP method. This procedure systematically perturbs each of the variables in order to calculate the partial derivatives of the objective function and the constraints with respect to the control. If the analytic partial derivatives are provided directly, the problem will be solved more accurately and e$ciently. Along with the developments in the area of neural networks, tremendous strides have been made in the area of nonlinear control analysis using di3erential geometric and
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
319
algebraic techniques [5,6]. In this paper, according to the special structure of control a$ne systems, we propose a novel neural network, called as Control A$ne Feedforward Neural Network (CAFNN), for the modelling and optimal control of fed-batch processes. It can be trained easily by a modiHed Levenberg–Marquardt (LM) algorithm and o3ers an e3ective and simple optimal control strategy by SQP in which the gradient information can be computed analytically in a recursive fashion. The computation cost of the optimisation procedure is signiHcantly reduced and improved performance of optimal control can be obtained. The rest of this paper is structured as follows: the structure, the training algorithm and model validation of CAFNN are presented in Section 2. The proposed CAFNN is demonstrated on the modelling of a simulated nonlinear single-input–single-output (SISO) plant in Section 3. In Section 4, an optimal control strategy for fed-batch processes based on a CAFNN model is described in detail, where the algorithm for the recursive computation of the gradient information is also presented. An application of the proposed modelling and optimal control technique to a simulated fed-batch ethanol fermentation process is presented in Section 5. Finally Section 6 draws some concluding remarks. 2. Control ane feedforward neural network 2.1. The structure of CAFNN Many fed-batch processes can be considered as a class of control a$ne nonlinear systems, mathematically formulated as x˙ = f(x) + g(x)u;
x(0) = x0 ;
(1)
p
where x ∈ R is a vector of process states, u ∈ R is the process input (the feed rate), f and g are p-dimensional analytic vector Helds. A discrete time model for such a control a$ne nonlinear system can be described as y(t + 1) = f(y(t); : : : ; y(t − n + 1)) + g(y(t); : : : ; y(t − n + 1))u(t);
(2)
where y is the process output and n ¿ p is the order of the nonlinear system. To model this kind of nonlinear systems, a special neural network, CAFNN, is proposed. The topology of a CAFNN is shown in Fig. 1. It contains two parts: an inner feedforward neural network and a synthesis layer with lateral connection. The dashed frame in Fig. 1 is the inner network, which is a single hidden layer feedforward network. Within this inner network, hidden layer neurons use the sigmoidal function as the neuron activation function whilst the activation function in the output layer neurones is linear. It is now well known that feedforward neural network with one hidden layer can approximate any nonlinear function with arbitrary degree of accuracy [4,14]. This inner network only has as its inputs a number of past system outputs to approximate the nonlinear functions f(•) and g(•) in Eq. (2). The synthesis layer calculates the Hnal output of the CAFNN model as y(t ˆ + 1) = fˆ + gu(t), ˆ where fˆ and
320
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
W1
W2
y(t)
y(t-n+1)
f ( χ,W)
y(t+1)
g( χ ,W )
1 1
u(t) synthesis layer
inner network
Fig. 1. The structure of a CAFNN.
gˆ are the two outputs of the inner network. The system input, u(t), is applied as an input to the synthesis layer where it is multiplied to one of the inner network output, ˆ through g, ˆ and then the result is added to the other output of the inner network, f, the lateral connection. This particular structure represents the control a$ne structure in Eq. (2). There are no adjustable weights in this synthesis layer. Calculations in the CAFNN can be described as follows. DeHne the input vector of the inner network, which only includes past system outputs, as
(t) = [y(t); : : : ; y(t − n + 1)]T
(3)
and deHne predictions of the inner network as ˆ ˆ F( (t); W ) = [f( (t); W ); g( (t); ˆ W )]T ; where Fˆ j ( (t); W ) =
nh m=1
2 wjm a
n I
(4)
1 wml
l (t)
+
m1
+ j2 ;
j = 1; 2:
(5)
l=1
1 In the Eqs. (4) and (5), W is a matrix of the network weights, wml is the connection 1 weight from the lth input node to the mth hidden node, m is the threshold of the mth 2 hidden node, wjm is the connection weight from the mth hidden node to the jth inner NN output-layer node, j2 is the threshold of the jth inner NN output-layer node, nh and nI are the numbers of nodes in the hidden layer and input layer of the inner NN model, respectively. The neurones within the hidden layer use the sigmoidal function as the neuron activation function, i.e. a(z) = 1=(1 + e−z ). In the synthesis layer, the output of CAFNN is calculated as ˆ y(t ˆ + 1; W ) = f( (t); W ) + g( (t); ˆ W )u(t): (6)
For the synthesis layer neuron receiving the system input, u(t), the neuron activation function is the product function, i.e. the neuron output is a product of its inputs. For the other synthesis layer neuron, the neuron activation function is a linear function. In view of its special structure, a CAFNN can be considered similar to a two-hiddenlayer network, in which the two output neurons in the inner network are considered
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
321
as two hidden neurons in the second hidden layer of CAFNN. For a CAFNN with nI inputs and nh hidden neurons, its structure can be symbolised as CAFNN: nI − nh –2–1. The network shown in Fig. 1 is considered here as a feedforward type network in the sense that the network is trained as a feedforward network. In the training of a feedforward network, the one-step-ahead prediction errors of the network are minimised. In calculating one-step-ahead predictions, process output at the previous time step is used to predict the process output at the current time step. However, this feedforward type of network is used as a recurrent network in the optimal control applications where multi-step-ahead predictions are required. In calculating multi-step-ahead predictions, model predictions are fed back to the model input through time delay units for predicting future model outputs. Since training of a feedforward network requires much less e3ort than training a recurrent network, a feedforward type network is used here. For the control a$ne nonlinear system Eq. (2), a general multi-layer neural network (GMNN) can also be used to approximate the nonlinear function between input u and output y. Compared with the input vector of the inner network of CAFNN, Eq. (3), the input vector of GMNN includes both past system outputs and inputs, which can be described as
(t) = [y(t); : : : ; y(t − n + 1); u(t)]T :
(7)
Di3erent structures of input vector and hidden layer of GMNN can be used to model the nonlinear function. To analyse the similarity between CAFNN and GMNN model in this study, GMNN models using the above input vector, Eq. (7), and including two hidden layers in which there are two hidden neurons in the second hidden layer are compared to CAFNN in this study. Similar to the CAFNN, the neurones within the two hidden layers also use the sigmoidal function as the neuron activation function whilst the activation function in the output layer neurones is linear. So this GMNN model is symbolised as GMNN: (nI + 1) − nh –2–1, and the GMNN model output is deHned as y(t ˆ + 1; W ) = GMNN ( (t); W ) n n 2 h I 3 2 1 1 2 wj a + 3 ; = wjm a wml l (t) + m + j j=1
m=1
(8)
l=1
1 2 where wml ; wjm and wj3 are the connection weights between the three layers, m1 ; j2 3 and are thresholds. The key di3erence between CAFNN and GMNN is that in CAFNN only past system outputs y are used as inputs in the inner network to approximate the nonlinear functions f(y) and g(y), and the input (control) signal u is only involved in the synthesis layer to calculate the Hnal CAFNN model output. But GMNN directly uses past outputs y and control signal u to predict the future output. Because the control signal is only used in the synthesis layer of CAFNN, the gradient information of a CAFNN model can be computed analytically in a recursive fashion and can be directly used in the optimisation algorithm such as SQP. Thus a CAFNN model can o3er an e3ective and simple optimal control strategy with signiHcantly reduced computation cost and improved performance.
322
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
The CAFNN can be trained easily by a modiHed LM algorithm, which is described in the following section. 2.2. Training of CAFNN In general, a GMNN model can be trained using any of a number of training methods, such as the back-propagation method, the conjugate gradient method, LM optimisation, or methods based on genetic algorithms. A CAFNN model can also be trained by using these training algorithms. In this study, both GMNN and CAFNN models are trained using the LM algorithm with regularisation [19]. The training objective function can be deHned as L
J (W ) =
1 (y(t; ˆ W ) − y(t))2 + W 2 ; 2L
(9)
t=1
where L is the number of data points, y(t; ˆ W ) is the network prediction, y is the target data, t presents the discrete time, W is a vector of network weights, and is the regularisation parameter. The appropriate value of is obtained through a cross-validation procedure. Data for building both GMNN and CAFNN models are partitioned into a training data set and a testing data set. Several values of are considered, and the one resulting in the smallest error on the testing data is adopted. Regularisation has been widely used in statistical model building and a variety of techniques have been developed, such as ridge regression, principal component regression, and partial least-squares regression [19]. Due to the di3erent model structure of CAFNN from that of GMNN, a modiHed LM training algorithm for CAFNN is described as follows: W (k + 1) = W (k) + (W (k)); (W (k)) = − ∇JW (W (k)) = (t; W (k)) =
L
1 (t; W (k)) T (t; W (k)) + ( + 2)I L
1 L
t=1 L
(10) −1
(t; W (k))(y(t; ˆ W (k) − y(t)) + 2W (k);
∇JW (W (k)); (11) (12)
t=1
ˆ @F(x(t); W) @y(t; ˆ W (k)) = {In ⊗ [1u(t)]} ; @W (k) @W
(13)
where ⊗ represents Kronecker tensor product, is the learning rate, is a factor to control the magnitude and direction of searching, and 2 1 wjm dhm (t)xl (t); Wi = wml ; 1 6 m 6 nh ; 1 6 l 6 nI ; 2 1 Wi = m ; 1 6 m 6 nh ; wjm dhm (t); @Fˆ j (x(t); W ) 2 = Ohm (t); Wi = wjm ; 1 6 j 6 2; 1 6 m 6 nh ; (14) @Wi 1; Wi = j2 ; 1 6 j 6 2; 0; otherwise;
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
323
where j = 1; 2; Ohm (t) = a(•) is the output of the mth hidden node, and dhm (t) = Ohm (t)[1 − Ohm (t)] is the derivative of Ohm (t). A cross validation based “early stopping” mechanism is used to minimise the problem of over-Htting during network training. During the training process, both training and testing errors are continuously monitored to detect the appropriate stopping point. Training is terminated at the point where the network error on the testing data cannot be further reduced. Early stopping is an implicit way to implement regularisation in order to improve model robustness [14]. In this study, both GMNN and CAFNN use this technique to limit the over-Htting problem during training. 2.3. Model validation Many model validity tests for nonlinear models have been developed [19], for example, the Akaike Information Criterion (AIC), the statistical 2 tests, the predicted squared error criterion, and the high-order correlation tests. If the identiHed model based on CAFNN is adequate, the prediction errors should satisfy the following conditions of high-order correlation tests [1]: &'' (() = E['(t − ()'(t)] = ((); &u' (() = E[u(t − ()'(t)] = 0;
∀(;
(15)
∀(;
&u2 ' (() = E{[u2 (t − () − u2 (t)]'(t)} = 0; &u2 '2 (() = E{[u2 (t − () − u2 (t)]'2 (t)} = 0;
(16) ∀(;
(17)
∀(;
&'('u) (() = E['(t)'(t − 1 − ()u(t − 1 − ()] = 0;
(18) ∀(;
(19)
where &xz (() indicates the cross-correlation function between x(t) and z(t), ' is the model residual, and u2 (t) represents the time average or mean value of u2 (t). These tests look into the cross correlation amongst model residuals and inputs. Normalisation to give all tests √ a range of plus and minus one and approximate the 95% conHdence bounds at 1:96= N ; N being the number of testing data, makes the tests independent of signal amplitudes and easy to interpret [1]. If these correlation tests are satisHed then the model residuals are a random sequence and are not predictable from the model inputs. This provides additional evidence of the validity of the identiHed model. 3. Modelling of a nonlinear SISO system A nonlinear SISO system is studied which is described as follows: y(t + 1) =
y(t) + [y(t) − 0:3][y(t) + 0:5]u(t); 1 + y(t)2
(20)
324
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
Table 1 Performance comparison for modelling an SISO plant SSE
CAFNN GMNN
Training CPU time (s)
S1
S2
S3
S4
1.878 1.810
0.924 0.959
0.960 1.128
1.563 1.612
76.970 107.455
where y(0)=0. For this SISO system, a CAFNN is used to model the nonlinear relationship between input u(t) and output y(t), and is also compared to a GMNN model. Data for modelling were generated by using a random sequence of input u(t) uniformly distributed in the interval [ − 2; 2]. A data set was generated and divided into training data S1 , consisting of 200 samples, testing data S2 , consisting of 100 samples, and unseen validation data S3 , consisting of 100 samples. The training and testing data sets are used to develop the neural network models and the unseen validation data is used to further evaluate the Hnal developed neural network models. During the training process, both training and testing data were used simultaneously by the “early stopping” mechanism to limit the over-Htting problem. In the “early stopping” mechanism, neural network prediction errors on both the training and testing data are continuously monitored during the training process and training is stopped at the point beyond which the network prediction errors on the testing data could not be further reduced. The appropriate topology (such as the number of neurons in the hidden layer) and weights of CAFNN and GMNN models were determined by examining a number of network topologies and selecting the one with the least sum of squared error (SSE) on the testing data. A number of network topologies are considered and the one giving the least SSE on the testing data is considered as the appropriate network topology. The CAFNN model was trained using the above-modiHed LM algorithm with regularisation, whereas the GMNN model was trained using the LM algorithm with regularisation. Both CAFNN and GMNN models were trained under the same conditions, including the same training and testing data sets, and the same criterion for early stopping. Network weights for both CAFNN and GMNN were initialised as random numbers uniformly distributed over the range (−0:1; 0:1). After training, the structures of the two models were selected as CAFNN: 1– 6 –2–1 and GMNN: 2– 6 –2–1, and the performance of the CAFNN model is compared to that of the GMNN model as shown in Table 1. It can be seen from Table 1 that the CAFNN requires less training CPU time than the GMNN. This phenomenon is also observed in the training of other topologies of CAFNN and GMNN. This is probably because the number of required weights of a CAFNN is less than that of a GMNN with the same topology. The results in Table 1 also indicate that the CAFNN has better generalisation performance (better performance on the unseen validation data) than the GMNN. This could be due to special structure of CAFNN, which is more suitable for modelling control a$ne systems than GMNN. Because of its special structure, CAFNN would be less prone to over-Htting than GMNN. The plant outputs and predictions of
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
325
0.4 0.3 0.2
y (t)
0.1 0 -0.1 -0.2 model CAFNN GMNN
-0.3 -0.4 -0.5
0
20
40
60
80
100
t
Fig. 2. Predictions of the CAFNN and GMNN models on the unseen validation data set S3 .
model: - CAFNN: --- GMNN: -.0.4
0.2
y (t)
0
-0.2
-0.4
-0.6 0
20
40
60
80
100 t
120
140
160
180
200
Fig. 3. Predictions of the CAFNN and the GMNN models on the unseen validation data set S4 .
both the CAFNN and the GMNN models on the unseen validation data are shown in Fig. 2. In Fig. 2, the solid, dashed, and dash-dotted lines represent, respectively, the plant outputs, predictions from the CAFNN model, and predictions from the GMNN model. It can be seen from Fig. 2 that predictions from CAFNN are more accurate than those from GMNN. In order to further verify the generalisation of both CAFNN and GMNN models, di3erent inputs were used. One such input is a composition of sine waves
2,t 2,t + sin ; (21) u(t) = sin 25 10 where t = [0; 200]. The data produced from the inputs given by Eq. (21) is deHned as the unseen validation data set S4 . Fig. 3 shows the outputs of the plant and the outputs
326
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337 1 0.8 0.6 0.4
Φεε
0.2 0 -0.2 -0.4 -0.6 -0.8
Φ u 2ε 2
(b)
(d)
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -20
0
Φ u 2ε
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -20
-10
-10
0
10
20
(c)
Φεεu
Φuε
(a) -1-20
-10
0
10
20
(e)
10
20
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -20
-10
0
10
20
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -20
-10
0
10
20
Fig. 4. Model validation tests of the CAFNN model.
of both CAFNN and GMNN models for the above input. In Fig. 3, the solid, dashed, and dash-dotted lines represent, respectively, the plant outputs, predictions from the CAFNN model, and predictions from the GMNN model. The SSE of both networks on this data set are given in Table 1. It can be seen from Fig. 3 and Table 1 that the predictions of both networks are quite accurate. By using the above test outputs of CAFNN, the correlation-based model validation results for the CAFNN model were calculated according to Eqs. (15)–(19) and shown in Fig. 4, where plots (a) – (e) are &'' ((); &u' ((); &u2 ' ((); &u2 '2 ((), and &''u ((), √ respectively. The dash-dot lines in each plot are the 95% conHdence bounds (±1:96= 200). It can be seen that only a small number of points are slightly outside the conHdence bounds. This demonstrates that the model can be considered as being adequate for modelling this plant.
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
327
4. Optimal control of fed-batch processes In fed-batch processes, the ultimate interest lies in the Hnal product quality and quantity at the end of a batch. In the optimal control of a fed-batch process, the maximum amount of the desired product is obtained by determining the optimal feed rate to the reactor. This leads to an endpoint optimal control problem of a nonlinear system, which can be mathematically formulated as min
u(k); k=1;:::;N
J [u(k)] = -[y(N )];
(22)
s.t. ˆ y(t + 1) = f[y(t); : : : ; y(t − n + 1)] + g[y(t); ˆ : : : ; y(t − n + 1)]u(t);
(23)
H (y; u) 6 0;
(24)
where - is a scalar objective function, H (y; u) are constraint functions, h is the sampling interval, the Hnal time tf = Nh is given, time k = 0 corresponds to the start of a batch, and k = N corresponds to the end of a batch. To reduce the complexity of the optimisation problem, the set of possible control trajectories is restricted to a Hnite-dimensional space. The control input is here parameterised as a piecewise constant function u0 ; k ∈ [0; h) k ∈ [h; 2h) u1 ; u(k) = (25) .. . uN −1 ; k ∈ [(N − 1)h; Nh): The optimal control problem was solved using SQP. According to Eq. (8) of GMNN, it is di$cult to compute the gradients of the objective function with respect to the controls analytically. On the contrary, according to the special structure of CAFNN in Eq. (23), the gradient information, ∇J [u(k)], can be computed easily. With the gradient information, the control policy can then be solved more accurately and e$ciently by SQP. From the optimal control problem, Eq. (22), the gradients of the objective function with respect to the controls can be calculated as @- @y(N ) ∇J [u(k)] = ; k = 0; 1; : : : ; N − 1; (26) @y(N ) @u(k) where
n ˆ @y(N − i) @y(N − i) @f(y) @y(N ) @g(y) ˆ u(N − 1) ; = + @u(k) @y(N − i) @u(k) @y(N − i) @u(k) i=1
=
n i=1
ˆ @g(y) ˆ @f(y) + u(N − 1) @y(N − i) @y(N − i)
@y(N − i) ; @u(k)
k ¡ N − 1; (27)
328
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
@y(N ) = g(y); ˆ @u(k)
k = N − 1:
(28)
ˆ @f(y) @g(y) ˆ @y(N ) = + u(N − 1) @y(N − i) @y(N − i) @y(N − i)
(29)
DeHne
then Eqs. (27) and (28) can be rewritten as ˆ g(y); @y(N ) n = @y(N ) @y(N − i) @u(k) ; @y(N − i) @u(k)
k = N − 1; k ¡ N − 1:
(30)
i=1
It should be noted that the following partial derivatives are zero when k ¿ N − i: @y(N − i) = 0; @u(k)
k ¿ N − i; i = 1; 2; : : : ; n:
(31)
By recursively computing @y(j)=@u(k) from j = k + 1 to j = N , the gradients ∇J [u(k)] (k = 0; 1; : : : ; N − 1) are obtained. The recursive algorithm for computing the gradient information @y(j)=@u(k) can be outlined as follows: ˆ k = j − 1; g(y); @y(j) n = @y(j) @y(j − i) (32) @u(k) ; k ¡ j − 1; @y(j − i) @u(k) i=1
@y(j − i) = 0; @u(k)
k ¿ j − i;
(33)
ˆ @f(y) @g(y) ˆ @y(j) = + u(j − 1); @y(j − i) @y(j − i) @y(j − i)
(34)
n
h ˆ @f(y) 2 1 = w1m d( (j))wmi ); @y(j − i)
(35)
m=1 n
h @g(y) ˆ 2 1 w2m d( (j))wmi ); = @y(j − i)
(36)
d( (j)) = O( (j))(1 − O( (j)));
(37)
m=1
O( (j)) = a
n I
1 wmi y(j
− i) +
m1
;
(38)
i=1
where (j) = [y(j − 1); : : : ; y(j − n + 1)]T ; k = 0; 1; : : : ; N − 1; i = 1; 2; : : : ; n, and j = 1; 2; : : : ; N .
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
329
From Eq. (32) to Eq. (38), @y(N )=@u(k) (k = 0; 1; : : : ; N − 1) are all obtained, then ∇J [u(k)] (k = 0; 1; : : : ; N − 1) can be calculated from Eq. (26). In general, constraint functions H (y; u) are much simple and the partial derivatives of the constraints @H (y; u)=@u(k) are always computed analytically, as shown below in the simulation example.
5. Optimal control of a fed-batch process A simulated ethanol fermentation process is studied here and the di3erential equations for this process are described as follows: d x1 x1 = Cx1 − u; dt x4
(39)
(150 − x2 ) d x2 u; = −10Cx1 + dt x4
(40)
x3 d x3 = Dx1 − u; dt x4
(41)
d x4 = u; dt
(42)
where C=
0:408x2 ; (1 + x3 =16)(0:22 + x2 )
(43)
D=
x2 : (1 + x3 =71:5)(0:44 + x2 )
(44)
In the above equations, x1 is the cell mass concentration, x2 is the substrate concentration, x3 is the product concentration, and x4 is the liquid volume of the reactor. The initial condition is speciHed as x0 = [1; 150; 0; 10]T , with the feed rate to the reactor u constrained by 0 l=h 6 u 6 12 l=h, the liquid volume of the reactor is limited by the 200 l vessel size, and tf is Hxed to be 63:0 h. In this fed-batch process, the feed rate u is used as the control variable. There is no outVow, so the feed rate must be chosen so that the batch volume does not exceed the physical volume of the reactor. The fermentation process was formulated and used for optimal control studies by Hong [7]. This typical biochemical system exhibiting singular control was also studied for global optimisation by Luus [12]. The performance index (PI), that is to be maximised by choosing the feed rate u in the time interval 0 6 t 6 tf , is the yield of the reactor PI = x3 (tf )x4 (tf ):
(45)
In this study, we assume that the above mechanistic model, Eqs. (39)–(44), is not available, thus a NN-based empirical model has to be utilised. Since the main interest
330
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
Table 2 Performance comparison for modelling a fed-batch process SSE
CAFNN GMNN
Training CPU time (s)
S1
S2
S3
632.474 440.307
435.983 530.852
35.399 50.075
52.96 73.08
of the fed-batch process is on the end-of-batch yield of batch process production, the NN model should o3er accurate long-range predictions. Because the performance index only includes variables x3 and x4 , we use a CAFNN to model the nonlinear relationship between x3 and u. Since x4 is the liquid volume of the reactor, x4 (tf ) can be obtained by integrating u easily. Thus the CAFNN model is described as ˆ y(k + 1) = f(y(k); y(k − 1); y(k − 2)) + g(y(k); ˆ y(k − 1); y(k − 2))u(t) V (N ) =
N −1
u(k)h + V0 ;
(46) (47)
k=0
where y = x3 ; V = x4 ; V0 = 10 l; N = 10, and h = tf =N is interval time. In this study, 52 batches of the process operation under di3erent feeding policies were simulated from the mechanistic model to produce the data set for CAFNN and GMNN modelling. Data from these batch runs were generated by choosing u(t) as a random sequence from a uniform distribution in the interval [0, 12]. All the batch runs start from the same initial condition x0 = [1; 150; 0; 10]T . A data set was generated from Eq. (39) to Eq. (44) and divided into training data S1 , consisting of 30 batches, testing data S2 , consisting of 20 batches, and unseen validation data S3 , consisting of 2 batches. Like the procedure in Section 3, the CAFNN model was trained by using the modiHed LM algorithm with regularisation, whereas the GMNN model was trained by using the LM algorithm with regularisation. Network weights were initialised as random numbers uniformly distributed over the range (−0:1; 0:1) for both networks. To compare the performance of CAFNN with GMNN models, the same training conditions were used, including the same training and testing data sets, and the same criterion for early stopping. The appropriate topology and weights of CAFNN and GMNN models were determined by examining the least SSE on the validation data. After di3erent numbers of neurons in the hidden layer were considered, the structure of the CAFNN was selected as 3–10 –2–1 and that of the GMNN was 4 –10 –2–1. After both NN models were trained, the performance of the CAFNN identiHcation model is compared to that of the GMNN model as shown in Table 2. It has been shown that training CPU time of CAFNN is much less than that of GMNN. Although the performance of the CAFNN model on the training data is not as good as that of the GMNN model, the performance of CAFNN model on the testing data and unseen validation data is better than that of the GMNN model. This demonstrates that the CAFNN model has better generalisation capability than the GMNN model. This is
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
331
ymdl(tf) = 103.7319 ymCAFNN(tf) = 98.7852 110 Product Concentration x3 (g/L)
100 90 80 70 60 50 40 30 Model CAFNN
20 10 0
0
10
20
30 time (h)
40
50
60
Fig. 5. Predictions of the CAFNN model and the mechanistic model.
ymdl(tf) = 103.7319 ymGMNN(tf) = 94.5695 110
Product Concentration x3 ( g / L )
100 90 80 70 60 50 40 30 Model GMNN
20 10 0
0
10
20
30
40
50
60
time (h)
Fig. 6. Predictions of the GMNN model and the mechanistic model.
probably due to the more appropriate structure of the CAFNN model for this fed-batch process and the CAFNN model is less prone to over-Htting than the GMNN model. Predictions from the CAFNN and the GMNN models on one of the unseen validation batch are shown in Figs. 5 and 6, respectively. It can be seen that the predictions from the CAFNN are more accurate than those from the GMNN model, especially towards the end of the batch.
332
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
The optimal control objective for this process is to maximise the product yield at the end of a batch, i.e. at time tf . Based on the trained CAFNN model, this optimal control problem is a Hnal state constrained nonlinear dynamic optimisation, formulated as min
u(k); k=1;:::;N
J (u) = −y(N )V (N );
(48)
s.t. ˆ y(k + 1) = f[y(k); y(k − 1); y(k − 2)] + g[y(k); ˆ y(k − 1); y(k − 2)]u(k); N −1
u(k)h 6 200 − V0 ;
(49) (50)
k=0
0 6 u(k) 6 12:
(51)
The optimal control trajectory based on the CAFNN model is obtained using the MATLAB Optimisation Toolbox function “constr” with the SQP algorithm being selected, and a function is provided to return the analytic partial derivatives of the objective function equation (48) as @J=@u(k) = −@y(N )=@u(k)V (N ) − y(N )h;
(52)
where @y(N )=@u(k) is computed according to Eqs. (32)–(38). The partial derivatives of the constraint, Eq. (50), can be calculated analytically as @H (u)=@u(k) = h:
(53)
The optimal control policy based on the GMNN model is also obtained using the MATLAB function “constr” with the SQP algorithm being selected, but the gradient information is not provided. To cross-check these optimal control results, the IDP algorithm was used to calculate the optimal control proHle based on the rigorous mechanistic model, i.e. Eqs. (39)–(44). The parameters in the IDP algorithm are chosen as follows: time stage P = 10, number of grid points N = 41, number of allowable control values M = 7, number of iterations T = 20, and a reduction factor = 0:7, which are the same as used in [12]. The piecewise constant optimal control policies of all three cases are shown in Fig. 7, and the corresponding outputs V (k) under these optimal control policies in all cases are shown in Fig. 8. In Figs. 7 and 8, the solid lines represent the results from the mechanistic model with IDP, the dashed lines represent the results from the CAFNN model with SQP, and the dash-dotted lines represent the results from the GMNN model with SQP. The Hnal volume V (N ) in all cases are exactly 200 l. Figs. 9 and 10 compare the actual (solid lines) and the NN model predicted (dashed lines) values of output y(k) under the optimal feeding policies based on the CAFNN and the GMNN models, respectively. Fig. 11 shows the actual process outputs (simulated values from the mechanistic model) under the three optimal feeding policies. In Fig. 11, the solid line represents the results from the mechanistic model with IDP, the dashed line represents the results from the CAFNN model with SQP, and the dash-dotted line represents the results from the GMNN model with SQP. Note that the
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
333
Optimal Control 12
10
IDP CAFNN GMNN
Feed rate (L/h)
8
6
4
2
0 0
10
20
30 time (h)
40
50
60
50
60
Fig. 7. Optimal feeding policies. 200 180 IDP CAFNN GMNN
160
Volume (L)
140 120 100 80 60 40 20 0
0
10
20
30 time (h)
40
Fig. 8. Trajectories of the reactor volume.
“optimal” control policies from the CAFNN and the GMNN models are optimal on the respective neural network models but may not be optimal on the actual process due to model–plant mismatches. Therefore, they should be considered as sub-optimal policies on the actual process.
334
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
ymdl(tf) = 101.7767 ymCAFNN(tf) = 103.1358 110
Product Concentration x3(g / L)
100 90 80 70 60 50 40 30 20
Model CAFNN
10 0
0
10
20
30 time (h)
40
50
60
Fig. 9. Actual and predicted values of output y(k) under the sub-optimal feeding policy based on the CAFNN model. ymdl(tf) = 100.9962 ymGMNN(tf) = 103.2124 110
Product Concentration x3 (g /L)
100 90 80 70 60 50 40 30 Model GMNN
20 10 0
0
10
20
30 time (h)
40
50
60
Fig. 10. Actual and predicted values of output y(k) under the sub-optimal feeding policy based on the GMNN model.
The performances of all three cases are shown in Table 3, where the performance indices were calculated according to Eq. (45), i.e. PI = y(N )V (N ). From Table 3 it can be seen that the optimal feeding policy based on IDP with the mechanistic model
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
335
ymdl(tf) = 103.7319 ymCAFNN(tf) = 101.7767 ymGMNN(tf) = 100.9962 110
Product Concentration x3 (g/L)
100 90 80 70 60 50 40 30 IDP CAFNN GMNN
20 10 0
0
10
20
30 time (h)
40
50
60
Fig. 11. Process outputs y(k) under di3erent optimal control policies.
Table 3 Performance comparison on the optimal control of the fed-batch process
CAFNN GMNN Mechanistic model
Performance index on the corresponding model
Performance index on process
Optimisation CPU time (min)
20,627 20,642 20,706
20,355 20,199 20,706
3.72 5.19 142.38
give the best performance, but it takes much longer CPU time than both CAFNN and GMNN model based optimal control because it has to solve those di3erential equations of the rigorous mechanistic model at each iteration. It is therefore very di$cult to apply the rigorous mechanistic model based IDP optimisation to this fed-batch process. Furthermore, the sub-optimal feeding policy based on the CAFNN model gives good performance on the process (only 1.9% less than that based on the mechanistic model), and its performance is also better than that based on the GMNN model. This could be due to the better generalisation capability of the CAFNN model in modelling fed-batch processes and the ease of optimisation with SQP when the gradient information is supplied based on the CAFNN model. 6. Conclusions In order to solve the control a$ne fed-batch process optimal control problem more accurately and e$ciently, a novel neural network structure, CAFNN, is proposed in this
336
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
paper. The training algorithm of CAFNN is obtained easily using a modiHed LM algorithm. According to the special topology of CAFNN, the analytic gradient information can be computed directly o3ering an e3ective and simple optimal control strategy by SQP for constrained nonlinear fed-batch process optimal control problems. The proposed modelling and optimal control scheme are illustrated on an application to an ethanol fermentation process. It is shown that a CAFNN has better generalisation capability than a conventional feedforward network when modelling a fed-batch process. Due to enhanced model generalisation and the availability of model output gradients with respect to controls form the CAFNN model, an accurate and e$cient optimal policy is obtained. Compared to the GMNN and IDP with mechanistic model results, the results of optimal control based on a CAFNN model demonstrate the e3ectiveness of the proposed modelling and optimal control scheme for fed-batch processes. Acknowledgements The work was supported by the UK EPSRC through the Grant GR/N13319— “Nonlinear Optimising Control in Agile Batch Manufacturing”. The authors thank the anonymous reviewers for their constructive comments which helped to improve the quality and presentation of this paper.
References [1] S.A. Billings, W.S.F. Voon, Correlation based model validity tests for non-linear models, Int. J. Control 44 (1986) 235–244. [2] N. Bhat, T. McAvoy, Use of neural nets for dynamic modelling and control of chemical process systems, Comput. Chem. Eng. 14 (1990) 573–583. [3] S. Chen, S.A. Billings, Neural networks for non-linear dynamic system modeling and identiHcation, Int. J. Control 56 (1992) 319–346. [4] G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signal Syst. 2 (1989) 303–314. [5] A. Delgado, C. Kambhampati, K. Warwick, Dynamic recurrent neural network for system identiHcation and control, IEE Proc.-Control Theory Appl. 142 (4) (1995) 307–314. [6] V.I. Elkin, Reduction of Nonlinear Control Systems: Di3erential-Geometric Approach, Moscow, 1997. [7] J. Hong, Optimal substrate feeding policy for fed batch fermentation with substrate and product inhibition kinetics, Biotechnol. Bioeng. 27 (1986) 1421–1431. [8] K.J. Hunt, D. Sbarbaro, R. Zbikowski, P.J. Gawthrop, Neural networks for control systems—a survey, Automatica 28 (1992) 1083–1112. [9] J. Li, R.R. Rhinehart, Heuristic random optimisation, Comput. Chem. Eng. 22 (1998) 427–444. [10] R. Luus, Optimal control by dynamic programming using systematic reduction in grid size, Int. J. Control 51 (1990) 995–1013. [11] R. Luus, On the application of iterative dynamic programming to singular optimal control problems, IEEE Trans. Autom. Control 37 (1992) 1802–1806. [12] R. Luus, Application of dynamic programming to di3erential—algebraic process systems, Comput. Chem. Eng. 17 (1993) 373–377. [13] S. Park, W.F. Ramirez, Optimal production of secreted protein in fed-batch reactors, AIChE J. 34 (1998) 1550–1558.
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
337
[14] J. Sjoberg, Q. Zhang, L. Ljung, A. Benveniste, B. Delyon, P.Y. Glorennec, H. Hjalmarsson, A. Juditsky, Non-linear black-box modeling in system identiHcation: a uniHed overview, Automatica 31 (1995) 1691–1724. [15] Y. Tian, J. Zhang, J. Morris, Modelling and optimal control of a batch polymerisation reactor using a hybrid stacked recurrent neural network model, Ind. Eng. Chem. Res. 40 (2001) 4525–4535. [16] Y. Tian, J. Zhang, J. Morris, Optimal control of a batch emulsion copolymerisation reactor based on recurrent neural network models, Chem. Eng. Processing 41 (2002) 531–538. [17] E. Visser, B. Srinivasan, S. Palanki, D. Bonvin, A feedback-based implementation scheme for batch process optimisation, J. Process Control 10 (2000) 399–410. [18] J. Zhang, Developing robust neural network models by using both dynamic and static process operating data, Ind. Eng. Chem. Res. 40 (2001) 234–241. [19] J. Zhang, A.J. Morris, Recurrent neuro-fuzzy networks for non-linear process modeling, IEEE Trans. Neural Networks 10 (1999) 313–326. Zhihua Xiong received his B.Sc. and M.Sc. degrees in automatic control from Northeastern University, Shenyang, China, in 1993 and 1996, respectively, and the Ph.D. degree in control theory and control engineering from Tsinghua University, Beijing, China, in 2000. He has been a postdoctoral Research Associate in the School of Chemical Engineering and Advanced Materials, University of Newcastle, UK, since 2000. His research interests include neural networks, intelligent control system, optimal control of batch process, model predictive control, and iterative learning control.
Jie Zhang is a Lecturer in the School of Chemical Engineering & Advanced Materials, University of Newcastle, England. He received his B.Sc. degree in Control Engineering from Hebei University of Technology, Tianjin, China, in 1986 and his Ph.D. degree in Control Engineering from City University, London, in 1991. His research interests include: neural networks, neuro-fuzzy systems, fault detection and diagnosis, intelligent control systems, genetic algorithms, optimal control of batch processes, and multivariate statistical process control. He has published over 120 papers in international journals, books, and conferences. He served as a reviewer for many prestigious international journals including IEEE Transactions on Neural Networks, IEEE Transactions on Fuzzy Systems, Neural Networks, Automatica, Chemical Engineering Science, and IEE Proceedings. He is a Senior Member of IEEE and a member of the IEEE Control Systems Society, IEEE Neural Network Society, IEEE Systems Man and Cybnetics Society, and IEEE Instrumentation and Measurement Society. His biography is included in the 4th edition of Marquis Who’s Who in Science and Engineering.
www.elsevier.com/locate/neucom
Modelling and optimal control of fed-batch processes using a novel control a$ne feedforward neural network Zhihua Xiong, Jie Zhang∗ Centre for Process Analytics and Control Technology, School of Chemical Engineering and Advanced Materials, University of Newcastle, Newcastle upon Tyne NE1 7RU, UK Received 13 March 2003; received in revised form 26 September 2003; accepted 19 November 2003
Abstract Many fed-batch processes can be considered as a class of control a$ne nonlinear systems. In this paper, a new type of neural network for modelling fed-batch processes, called as control a$ne feedforward neural network (CAFNN), is proposed. For constrained nonlinear optimal control of fed-batch processes, CAFNN o3ers an e3ective and simple optimal control strategy by sequential quadratic programming (SQP) where the gradient information can be computed directly from CAFNN. Thus the nonlinear programming problem can then be solved more accurately and e$ciently. The proposed modelling and optimal control scheme are illustrated on a nonlinear system and a simulated fed-batch ethanol fermentation process. c 2003 Elsevier B.V. All rights reserved. Keywords: Neural networks; Control a$ne nonlinear systems; Fed-batch processes; Optimal control
1. Introduction In recent years, there has been a growing interest in the optimal control of fed-batch processes that are suitable for responsive manufacturing. In a fed-batch process, our interests are concerned with determining the feeding policy to the reactor that will give the maximum amount of the desired product. Although only one single control variable in the form of the feed rate may appear to present a simple optimal control ∗
Corresponding author. Department of Chemical and Process Engineering, University of Newcastle, Newcastle upon Tyne NE1 7RU, UK. Tel.: +44-191-2227240; fax: +44-191-2225292. E-mail address: [email protected] (J. Zhang). c 2003 Elsevier B.V. All rights reserved. 0925-2312/$ - see front matter doi:10.1016/j.neucom.2003.11.006
318
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
problem, considerable di$culties have been reported in the determination of the optimal feed rate policy for fed-batch processes, because fed-batch processes, in general, are nonlinear dynamic systems. Most of the optimisation studies on batch processes have reported the use of detailed and complicated Hrst principle models [11,13]. However, it is usually very di$cult to obtain those models due to the complexity of the processes, and it is also di$cult to implement them for optimal control. When building Hrst-principles models is very costly and di$cult, empirical models based on process input–output data can provide a useful alternative [14–16]. Neural networks (NN) have been proposed as a promising tool for identifying empirical models [2,18]. Neural networks with their massive parallelism and learning capabilities have been shown to be capable of approximating any continuous nonlinear functions [4]. If properly trained and validated, these neural network models can be used to accurately predict steady-state and dynamic process behaviour, hence, leading to improved process optimisation and control performance [3,8,19]. In general, optimal control of a fed-batch process can be described as a constrained end-point optimal control problem of a nonlinear control a$ne process [17]. It is usually very di$cult to solve this kind of problem by using the conventional Pontryagin’s Maximum Principle (PMP). For a control a$ne fed-batch process optimal control problem whose Hamiltonian (H ) is independent of the control (u), the partial derivative of the Hamiltonian with respect to the control variable, @H =@u, is equal to zero and, therefore, singular arcs exist. The di$culty of directly applying PMP is due to the existence of singular arcs and the low sensitivity of the control policy on the yield of the batch process. Alternatively, gradient-free methods, also called as direct search methods, are feasible computational methods for this kind of problems and they can possibly obtain the global optimum. Many direct search methods have been reported, such as iterative dynamic programming (IDP) [10,12] and heuristic random optimisation [9]. However, most of the direct search methods are very time consuming and ine$cient because they usually require a large number of iterations to search for the global minimum. On the other hand, gradient-based methods, such as sequential quadratic programming (SQP), are more e3ective for constrained optimisation problems. The SQP method is constructed by solving a quadratic programming (QP) sub-problem at each major iteration. An approximation is made of the Hessian of the Lagrangian function using a quasi-Newton updating method. This is then used to generate a QP sub-problem whose solution is used to form a search direction for a line search procedure. However, this kind of computational method has to cope with problems of numerical evaluation of derivatives and feasibility issues. In general, the analytic partial derivatives of the objective function as well as partial derivatives of the constraints are very di$cult to be obtained in many nonlinear systems. Then the gradients have to be calculated by Hnite di3erence approximation in the SQP method. This procedure systematically perturbs each of the variables in order to calculate the partial derivatives of the objective function and the constraints with respect to the control. If the analytic partial derivatives are provided directly, the problem will be solved more accurately and e$ciently. Along with the developments in the area of neural networks, tremendous strides have been made in the area of nonlinear control analysis using di3erential geometric and
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
319
algebraic techniques [5,6]. In this paper, according to the special structure of control a$ne systems, we propose a novel neural network, called as Control A$ne Feedforward Neural Network (CAFNN), for the modelling and optimal control of fed-batch processes. It can be trained easily by a modiHed Levenberg–Marquardt (LM) algorithm and o3ers an e3ective and simple optimal control strategy by SQP in which the gradient information can be computed analytically in a recursive fashion. The computation cost of the optimisation procedure is signiHcantly reduced and improved performance of optimal control can be obtained. The rest of this paper is structured as follows: the structure, the training algorithm and model validation of CAFNN are presented in Section 2. The proposed CAFNN is demonstrated on the modelling of a simulated nonlinear single-input–single-output (SISO) plant in Section 3. In Section 4, an optimal control strategy for fed-batch processes based on a CAFNN model is described in detail, where the algorithm for the recursive computation of the gradient information is also presented. An application of the proposed modelling and optimal control technique to a simulated fed-batch ethanol fermentation process is presented in Section 5. Finally Section 6 draws some concluding remarks. 2. Control ane feedforward neural network 2.1. The structure of CAFNN Many fed-batch processes can be considered as a class of control a$ne nonlinear systems, mathematically formulated as x˙ = f(x) + g(x)u;
x(0) = x0 ;
(1)
p
where x ∈ R is a vector of process states, u ∈ R is the process input (the feed rate), f and g are p-dimensional analytic vector Helds. A discrete time model for such a control a$ne nonlinear system can be described as y(t + 1) = f(y(t); : : : ; y(t − n + 1)) + g(y(t); : : : ; y(t − n + 1))u(t);
(2)
where y is the process output and n ¿ p is the order of the nonlinear system. To model this kind of nonlinear systems, a special neural network, CAFNN, is proposed. The topology of a CAFNN is shown in Fig. 1. It contains two parts: an inner feedforward neural network and a synthesis layer with lateral connection. The dashed frame in Fig. 1 is the inner network, which is a single hidden layer feedforward network. Within this inner network, hidden layer neurons use the sigmoidal function as the neuron activation function whilst the activation function in the output layer neurones is linear. It is now well known that feedforward neural network with one hidden layer can approximate any nonlinear function with arbitrary degree of accuracy [4,14]. This inner network only has as its inputs a number of past system outputs to approximate the nonlinear functions f(•) and g(•) in Eq. (2). The synthesis layer calculates the Hnal output of the CAFNN model as y(t ˆ + 1) = fˆ + gu(t), ˆ where fˆ and
320
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
W1
W2
y(t)
y(t-n+1)
f ( χ,W)
y(t+1)
g( χ ,W )
1 1
u(t) synthesis layer
inner network
Fig. 1. The structure of a CAFNN.
gˆ are the two outputs of the inner network. The system input, u(t), is applied as an input to the synthesis layer where it is multiplied to one of the inner network output, ˆ through g, ˆ and then the result is added to the other output of the inner network, f, the lateral connection. This particular structure represents the control a$ne structure in Eq. (2). There are no adjustable weights in this synthesis layer. Calculations in the CAFNN can be described as follows. DeHne the input vector of the inner network, which only includes past system outputs, as
(t) = [y(t); : : : ; y(t − n + 1)]T
(3)
and deHne predictions of the inner network as ˆ ˆ F( (t); W ) = [f( (t); W ); g( (t); ˆ W )]T ; where Fˆ j ( (t); W ) =
nh m=1
2 wjm a
n I
(4)
1 wml
l (t)
+
m1
+ j2 ;
j = 1; 2:
(5)
l=1
1 In the Eqs. (4) and (5), W is a matrix of the network weights, wml is the connection 1 weight from the lth input node to the mth hidden node, m is the threshold of the mth 2 hidden node, wjm is the connection weight from the mth hidden node to the jth inner NN output-layer node, j2 is the threshold of the jth inner NN output-layer node, nh and nI are the numbers of nodes in the hidden layer and input layer of the inner NN model, respectively. The neurones within the hidden layer use the sigmoidal function as the neuron activation function, i.e. a(z) = 1=(1 + e−z ). In the synthesis layer, the output of CAFNN is calculated as ˆ y(t ˆ + 1; W ) = f( (t); W ) + g( (t); ˆ W )u(t): (6)
For the synthesis layer neuron receiving the system input, u(t), the neuron activation function is the product function, i.e. the neuron output is a product of its inputs. For the other synthesis layer neuron, the neuron activation function is a linear function. In view of its special structure, a CAFNN can be considered similar to a two-hiddenlayer network, in which the two output neurons in the inner network are considered
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
321
as two hidden neurons in the second hidden layer of CAFNN. For a CAFNN with nI inputs and nh hidden neurons, its structure can be symbolised as CAFNN: nI − nh –2–1. The network shown in Fig. 1 is considered here as a feedforward type network in the sense that the network is trained as a feedforward network. In the training of a feedforward network, the one-step-ahead prediction errors of the network are minimised. In calculating one-step-ahead predictions, process output at the previous time step is used to predict the process output at the current time step. However, this feedforward type of network is used as a recurrent network in the optimal control applications where multi-step-ahead predictions are required. In calculating multi-step-ahead predictions, model predictions are fed back to the model input through time delay units for predicting future model outputs. Since training of a feedforward network requires much less e3ort than training a recurrent network, a feedforward type network is used here. For the control a$ne nonlinear system Eq. (2), a general multi-layer neural network (GMNN) can also be used to approximate the nonlinear function between input u and output y. Compared with the input vector of the inner network of CAFNN, Eq. (3), the input vector of GMNN includes both past system outputs and inputs, which can be described as
(t) = [y(t); : : : ; y(t − n + 1); u(t)]T :
(7)
Di3erent structures of input vector and hidden layer of GMNN can be used to model the nonlinear function. To analyse the similarity between CAFNN and GMNN model in this study, GMNN models using the above input vector, Eq. (7), and including two hidden layers in which there are two hidden neurons in the second hidden layer are compared to CAFNN in this study. Similar to the CAFNN, the neurones within the two hidden layers also use the sigmoidal function as the neuron activation function whilst the activation function in the output layer neurones is linear. So this GMNN model is symbolised as GMNN: (nI + 1) − nh –2–1, and the GMNN model output is deHned as y(t ˆ + 1; W ) = GMNN ( (t); W ) n n 2 h I 3 2 1 1 2 wj a + 3 ; = wjm a wml l (t) + m + j j=1
m=1
(8)
l=1
1 2 where wml ; wjm and wj3 are the connection weights between the three layers, m1 ; j2 3 and are thresholds. The key di3erence between CAFNN and GMNN is that in CAFNN only past system outputs y are used as inputs in the inner network to approximate the nonlinear functions f(y) and g(y), and the input (control) signal u is only involved in the synthesis layer to calculate the Hnal CAFNN model output. But GMNN directly uses past outputs y and control signal u to predict the future output. Because the control signal is only used in the synthesis layer of CAFNN, the gradient information of a CAFNN model can be computed analytically in a recursive fashion and can be directly used in the optimisation algorithm such as SQP. Thus a CAFNN model can o3er an e3ective and simple optimal control strategy with signiHcantly reduced computation cost and improved performance.
322
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
The CAFNN can be trained easily by a modiHed LM algorithm, which is described in the following section. 2.2. Training of CAFNN In general, a GMNN model can be trained using any of a number of training methods, such as the back-propagation method, the conjugate gradient method, LM optimisation, or methods based on genetic algorithms. A CAFNN model can also be trained by using these training algorithms. In this study, both GMNN and CAFNN models are trained using the LM algorithm with regularisation [19]. The training objective function can be deHned as L
J (W ) =
1 (y(t; ˆ W ) − y(t))2 + W 2 ; 2L
(9)
t=1
where L is the number of data points, y(t; ˆ W ) is the network prediction, y is the target data, t presents the discrete time, W is a vector of network weights, and is the regularisation parameter. The appropriate value of is obtained through a cross-validation procedure. Data for building both GMNN and CAFNN models are partitioned into a training data set and a testing data set. Several values of are considered, and the one resulting in the smallest error on the testing data is adopted. Regularisation has been widely used in statistical model building and a variety of techniques have been developed, such as ridge regression, principal component regression, and partial least-squares regression [19]. Due to the di3erent model structure of CAFNN from that of GMNN, a modiHed LM training algorithm for CAFNN is described as follows: W (k + 1) = W (k) + (W (k)); (W (k)) = − ∇JW (W (k)) = (t; W (k)) =
L
1 (t; W (k)) T (t; W (k)) + ( + 2)I L
1 L
t=1 L
(10) −1
(t; W (k))(y(t; ˆ W (k) − y(t)) + 2W (k);
∇JW (W (k)); (11) (12)
t=1
ˆ @F(x(t); W) @y(t; ˆ W (k)) = {In ⊗ [1u(t)]} ; @W (k) @W
(13)
where ⊗ represents Kronecker tensor product, is the learning rate, is a factor to control the magnitude and direction of searching, and 2 1 wjm dhm (t)xl (t); Wi = wml ; 1 6 m 6 nh ; 1 6 l 6 nI ; 2 1 Wi = m ; 1 6 m 6 nh ; wjm dhm (t); @Fˆ j (x(t); W ) 2 = Ohm (t); Wi = wjm ; 1 6 j 6 2; 1 6 m 6 nh ; (14) @Wi 1; Wi = j2 ; 1 6 j 6 2; 0; otherwise;
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
323
where j = 1; 2; Ohm (t) = a(•) is the output of the mth hidden node, and dhm (t) = Ohm (t)[1 − Ohm (t)] is the derivative of Ohm (t). A cross validation based “early stopping” mechanism is used to minimise the problem of over-Htting during network training. During the training process, both training and testing errors are continuously monitored to detect the appropriate stopping point. Training is terminated at the point where the network error on the testing data cannot be further reduced. Early stopping is an implicit way to implement regularisation in order to improve model robustness [14]. In this study, both GMNN and CAFNN use this technique to limit the over-Htting problem during training. 2.3. Model validation Many model validity tests for nonlinear models have been developed [19], for example, the Akaike Information Criterion (AIC), the statistical 2 tests, the predicted squared error criterion, and the high-order correlation tests. If the identiHed model based on CAFNN is adequate, the prediction errors should satisfy the following conditions of high-order correlation tests [1]: &'' (() = E['(t − ()'(t)] = ((); &u' (() = E[u(t − ()'(t)] = 0;
∀(;
(15)
∀(;
&u2 ' (() = E{[u2 (t − () − u2 (t)]'(t)} = 0; &u2 '2 (() = E{[u2 (t − () − u2 (t)]'2 (t)} = 0;
(16) ∀(;
(17)
∀(;
&'('u) (() = E['(t)'(t − 1 − ()u(t − 1 − ()] = 0;
(18) ∀(;
(19)
where &xz (() indicates the cross-correlation function between x(t) and z(t), ' is the model residual, and u2 (t) represents the time average or mean value of u2 (t). These tests look into the cross correlation amongst model residuals and inputs. Normalisation to give all tests √ a range of plus and minus one and approximate the 95% conHdence bounds at 1:96= N ; N being the number of testing data, makes the tests independent of signal amplitudes and easy to interpret [1]. If these correlation tests are satisHed then the model residuals are a random sequence and are not predictable from the model inputs. This provides additional evidence of the validity of the identiHed model. 3. Modelling of a nonlinear SISO system A nonlinear SISO system is studied which is described as follows: y(t + 1) =
y(t) + [y(t) − 0:3][y(t) + 0:5]u(t); 1 + y(t)2
(20)
324
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
Table 1 Performance comparison for modelling an SISO plant SSE
CAFNN GMNN
Training CPU time (s)
S1
S2
S3
S4
1.878 1.810
0.924 0.959
0.960 1.128
1.563 1.612
76.970 107.455
where y(0)=0. For this SISO system, a CAFNN is used to model the nonlinear relationship between input u(t) and output y(t), and is also compared to a GMNN model. Data for modelling were generated by using a random sequence of input u(t) uniformly distributed in the interval [ − 2; 2]. A data set was generated and divided into training data S1 , consisting of 200 samples, testing data S2 , consisting of 100 samples, and unseen validation data S3 , consisting of 100 samples. The training and testing data sets are used to develop the neural network models and the unseen validation data is used to further evaluate the Hnal developed neural network models. During the training process, both training and testing data were used simultaneously by the “early stopping” mechanism to limit the over-Htting problem. In the “early stopping” mechanism, neural network prediction errors on both the training and testing data are continuously monitored during the training process and training is stopped at the point beyond which the network prediction errors on the testing data could not be further reduced. The appropriate topology (such as the number of neurons in the hidden layer) and weights of CAFNN and GMNN models were determined by examining a number of network topologies and selecting the one with the least sum of squared error (SSE) on the testing data. A number of network topologies are considered and the one giving the least SSE on the testing data is considered as the appropriate network topology. The CAFNN model was trained using the above-modiHed LM algorithm with regularisation, whereas the GMNN model was trained using the LM algorithm with regularisation. Both CAFNN and GMNN models were trained under the same conditions, including the same training and testing data sets, and the same criterion for early stopping. Network weights for both CAFNN and GMNN were initialised as random numbers uniformly distributed over the range (−0:1; 0:1). After training, the structures of the two models were selected as CAFNN: 1– 6 –2–1 and GMNN: 2– 6 –2–1, and the performance of the CAFNN model is compared to that of the GMNN model as shown in Table 1. It can be seen from Table 1 that the CAFNN requires less training CPU time than the GMNN. This phenomenon is also observed in the training of other topologies of CAFNN and GMNN. This is probably because the number of required weights of a CAFNN is less than that of a GMNN with the same topology. The results in Table 1 also indicate that the CAFNN has better generalisation performance (better performance on the unseen validation data) than the GMNN. This could be due to special structure of CAFNN, which is more suitable for modelling control a$ne systems than GMNN. Because of its special structure, CAFNN would be less prone to over-Htting than GMNN. The plant outputs and predictions of
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
325
0.4 0.3 0.2
y (t)
0.1 0 -0.1 -0.2 model CAFNN GMNN
-0.3 -0.4 -0.5
0
20
40
60
80
100
t
Fig. 2. Predictions of the CAFNN and GMNN models on the unseen validation data set S3 .
model: - CAFNN: --- GMNN: -.0.4
0.2
y (t)
0
-0.2
-0.4
-0.6 0
20
40
60
80
100 t
120
140
160
180
200
Fig. 3. Predictions of the CAFNN and the GMNN models on the unseen validation data set S4 .
both the CAFNN and the GMNN models on the unseen validation data are shown in Fig. 2. In Fig. 2, the solid, dashed, and dash-dotted lines represent, respectively, the plant outputs, predictions from the CAFNN model, and predictions from the GMNN model. It can be seen from Fig. 2 that predictions from CAFNN are more accurate than those from GMNN. In order to further verify the generalisation of both CAFNN and GMNN models, di3erent inputs were used. One such input is a composition of sine waves
2,t 2,t + sin ; (21) u(t) = sin 25 10 where t = [0; 200]. The data produced from the inputs given by Eq. (21) is deHned as the unseen validation data set S4 . Fig. 3 shows the outputs of the plant and the outputs
326
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337 1 0.8 0.6 0.4
Φεε
0.2 0 -0.2 -0.4 -0.6 -0.8
Φ u 2ε 2
(b)
(d)
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -20
0
Φ u 2ε
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -20
-10
-10
0
10
20
(c)
Φεεu
Φuε
(a) -1-20
-10
0
10
20
(e)
10
20
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -20
-10
0
10
20
1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 -20
-10
0
10
20
Fig. 4. Model validation tests of the CAFNN model.
of both CAFNN and GMNN models for the above input. In Fig. 3, the solid, dashed, and dash-dotted lines represent, respectively, the plant outputs, predictions from the CAFNN model, and predictions from the GMNN model. The SSE of both networks on this data set are given in Table 1. It can be seen from Fig. 3 and Table 1 that the predictions of both networks are quite accurate. By using the above test outputs of CAFNN, the correlation-based model validation results for the CAFNN model were calculated according to Eqs. (15)–(19) and shown in Fig. 4, where plots (a) – (e) are &'' ((); &u' ((); &u2 ' ((); &u2 '2 ((), and &''u ((), √ respectively. The dash-dot lines in each plot are the 95% conHdence bounds (±1:96= 200). It can be seen that only a small number of points are slightly outside the conHdence bounds. This demonstrates that the model can be considered as being adequate for modelling this plant.
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
327
4. Optimal control of fed-batch processes In fed-batch processes, the ultimate interest lies in the Hnal product quality and quantity at the end of a batch. In the optimal control of a fed-batch process, the maximum amount of the desired product is obtained by determining the optimal feed rate to the reactor. This leads to an endpoint optimal control problem of a nonlinear system, which can be mathematically formulated as min
u(k); k=1;:::;N
J [u(k)] = -[y(N )];
(22)
s.t. ˆ y(t + 1) = f[y(t); : : : ; y(t − n + 1)] + g[y(t); ˆ : : : ; y(t − n + 1)]u(t);
(23)
H (y; u) 6 0;
(24)
where - is a scalar objective function, H (y; u) are constraint functions, h is the sampling interval, the Hnal time tf = Nh is given, time k = 0 corresponds to the start of a batch, and k = N corresponds to the end of a batch. To reduce the complexity of the optimisation problem, the set of possible control trajectories is restricted to a Hnite-dimensional space. The control input is here parameterised as a piecewise constant function u0 ; k ∈ [0; h) k ∈ [h; 2h) u1 ; u(k) = (25) .. . uN −1 ; k ∈ [(N − 1)h; Nh): The optimal control problem was solved using SQP. According to Eq. (8) of GMNN, it is di$cult to compute the gradients of the objective function with respect to the controls analytically. On the contrary, according to the special structure of CAFNN in Eq. (23), the gradient information, ∇J [u(k)], can be computed easily. With the gradient information, the control policy can then be solved more accurately and e$ciently by SQP. From the optimal control problem, Eq. (22), the gradients of the objective function with respect to the controls can be calculated as @- @y(N ) ∇J [u(k)] = ; k = 0; 1; : : : ; N − 1; (26) @y(N ) @u(k) where
n ˆ @y(N − i) @y(N − i) @f(y) @y(N ) @g(y) ˆ u(N − 1) ; = + @u(k) @y(N − i) @u(k) @y(N − i) @u(k) i=1
=
n i=1
ˆ @g(y) ˆ @f(y) + u(N − 1) @y(N − i) @y(N − i)
@y(N − i) ; @u(k)
k ¡ N − 1; (27)
328
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
@y(N ) = g(y); ˆ @u(k)
k = N − 1:
(28)
ˆ @f(y) @g(y) ˆ @y(N ) = + u(N − 1) @y(N − i) @y(N − i) @y(N − i)
(29)
DeHne
then Eqs. (27) and (28) can be rewritten as ˆ g(y); @y(N ) n = @y(N ) @y(N − i) @u(k) ; @y(N − i) @u(k)
k = N − 1; k ¡ N − 1:
(30)
i=1
It should be noted that the following partial derivatives are zero when k ¿ N − i: @y(N − i) = 0; @u(k)
k ¿ N − i; i = 1; 2; : : : ; n:
(31)
By recursively computing @y(j)=@u(k) from j = k + 1 to j = N , the gradients ∇J [u(k)] (k = 0; 1; : : : ; N − 1) are obtained. The recursive algorithm for computing the gradient information @y(j)=@u(k) can be outlined as follows: ˆ k = j − 1; g(y); @y(j) n = @y(j) @y(j − i) (32) @u(k) ; k ¡ j − 1; @y(j − i) @u(k) i=1
@y(j − i) = 0; @u(k)
k ¿ j − i;
(33)
ˆ @f(y) @g(y) ˆ @y(j) = + u(j − 1); @y(j − i) @y(j − i) @y(j − i)
(34)
n
h ˆ @f(y) 2 1 = w1m d( (j))wmi ); @y(j − i)
(35)
m=1 n
h @g(y) ˆ 2 1 w2m d( (j))wmi ); = @y(j − i)
(36)
d( (j)) = O( (j))(1 − O( (j)));
(37)
m=1
O( (j)) = a
n I
1 wmi y(j
− i) +
m1
;
(38)
i=1
where (j) = [y(j − 1); : : : ; y(j − n + 1)]T ; k = 0; 1; : : : ; N − 1; i = 1; 2; : : : ; n, and j = 1; 2; : : : ; N .
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
329
From Eq. (32) to Eq. (38), @y(N )=@u(k) (k = 0; 1; : : : ; N − 1) are all obtained, then ∇J [u(k)] (k = 0; 1; : : : ; N − 1) can be calculated from Eq. (26). In general, constraint functions H (y; u) are much simple and the partial derivatives of the constraints @H (y; u)=@u(k) are always computed analytically, as shown below in the simulation example.
5. Optimal control of a fed-batch process A simulated ethanol fermentation process is studied here and the di3erential equations for this process are described as follows: d x1 x1 = Cx1 − u; dt x4
(39)
(150 − x2 ) d x2 u; = −10Cx1 + dt x4
(40)
x3 d x3 = Dx1 − u; dt x4
(41)
d x4 = u; dt
(42)
where C=
0:408x2 ; (1 + x3 =16)(0:22 + x2 )
(43)
D=
x2 : (1 + x3 =71:5)(0:44 + x2 )
(44)
In the above equations, x1 is the cell mass concentration, x2 is the substrate concentration, x3 is the product concentration, and x4 is the liquid volume of the reactor. The initial condition is speciHed as x0 = [1; 150; 0; 10]T , with the feed rate to the reactor u constrained by 0 l=h 6 u 6 12 l=h, the liquid volume of the reactor is limited by the 200 l vessel size, and tf is Hxed to be 63:0 h. In this fed-batch process, the feed rate u is used as the control variable. There is no outVow, so the feed rate must be chosen so that the batch volume does not exceed the physical volume of the reactor. The fermentation process was formulated and used for optimal control studies by Hong [7]. This typical biochemical system exhibiting singular control was also studied for global optimisation by Luus [12]. The performance index (PI), that is to be maximised by choosing the feed rate u in the time interval 0 6 t 6 tf , is the yield of the reactor PI = x3 (tf )x4 (tf ):
(45)
In this study, we assume that the above mechanistic model, Eqs. (39)–(44), is not available, thus a NN-based empirical model has to be utilised. Since the main interest
330
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
Table 2 Performance comparison for modelling a fed-batch process SSE
CAFNN GMNN
Training CPU time (s)
S1
S2
S3
632.474 440.307
435.983 530.852
35.399 50.075
52.96 73.08
of the fed-batch process is on the end-of-batch yield of batch process production, the NN model should o3er accurate long-range predictions. Because the performance index only includes variables x3 and x4 , we use a CAFNN to model the nonlinear relationship between x3 and u. Since x4 is the liquid volume of the reactor, x4 (tf ) can be obtained by integrating u easily. Thus the CAFNN model is described as ˆ y(k + 1) = f(y(k); y(k − 1); y(k − 2)) + g(y(k); ˆ y(k − 1); y(k − 2))u(t) V (N ) =
N −1
u(k)h + V0 ;
(46) (47)
k=0
where y = x3 ; V = x4 ; V0 = 10 l; N = 10, and h = tf =N is interval time. In this study, 52 batches of the process operation under di3erent feeding policies were simulated from the mechanistic model to produce the data set for CAFNN and GMNN modelling. Data from these batch runs were generated by choosing u(t) as a random sequence from a uniform distribution in the interval [0, 12]. All the batch runs start from the same initial condition x0 = [1; 150; 0; 10]T . A data set was generated from Eq. (39) to Eq. (44) and divided into training data S1 , consisting of 30 batches, testing data S2 , consisting of 20 batches, and unseen validation data S3 , consisting of 2 batches. Like the procedure in Section 3, the CAFNN model was trained by using the modiHed LM algorithm with regularisation, whereas the GMNN model was trained by using the LM algorithm with regularisation. Network weights were initialised as random numbers uniformly distributed over the range (−0:1; 0:1) for both networks. To compare the performance of CAFNN with GMNN models, the same training conditions were used, including the same training and testing data sets, and the same criterion for early stopping. The appropriate topology and weights of CAFNN and GMNN models were determined by examining the least SSE on the validation data. After di3erent numbers of neurons in the hidden layer were considered, the structure of the CAFNN was selected as 3–10 –2–1 and that of the GMNN was 4 –10 –2–1. After both NN models were trained, the performance of the CAFNN identiHcation model is compared to that of the GMNN model as shown in Table 2. It has been shown that training CPU time of CAFNN is much less than that of GMNN. Although the performance of the CAFNN model on the training data is not as good as that of the GMNN model, the performance of CAFNN model on the testing data and unseen validation data is better than that of the GMNN model. This demonstrates that the CAFNN model has better generalisation capability than the GMNN model. This is
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
331
ymdl(tf) = 103.7319 ymCAFNN(tf) = 98.7852 110 Product Concentration x3 (g/L)
100 90 80 70 60 50 40 30 Model CAFNN
20 10 0
0
10
20
30 time (h)
40
50
60
Fig. 5. Predictions of the CAFNN model and the mechanistic model.
ymdl(tf) = 103.7319 ymGMNN(tf) = 94.5695 110
Product Concentration x3 ( g / L )
100 90 80 70 60 50 40 30 Model GMNN
20 10 0
0
10
20
30
40
50
60
time (h)
Fig. 6. Predictions of the GMNN model and the mechanistic model.
probably due to the more appropriate structure of the CAFNN model for this fed-batch process and the CAFNN model is less prone to over-Htting than the GMNN model. Predictions from the CAFNN and the GMNN models on one of the unseen validation batch are shown in Figs. 5 and 6, respectively. It can be seen that the predictions from the CAFNN are more accurate than those from the GMNN model, especially towards the end of the batch.
332
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
The optimal control objective for this process is to maximise the product yield at the end of a batch, i.e. at time tf . Based on the trained CAFNN model, this optimal control problem is a Hnal state constrained nonlinear dynamic optimisation, formulated as min
u(k); k=1;:::;N
J (u) = −y(N )V (N );
(48)
s.t. ˆ y(k + 1) = f[y(k); y(k − 1); y(k − 2)] + g[y(k); ˆ y(k − 1); y(k − 2)]u(k); N −1
u(k)h 6 200 − V0 ;
(49) (50)
k=0
0 6 u(k) 6 12:
(51)
The optimal control trajectory based on the CAFNN model is obtained using the MATLAB Optimisation Toolbox function “constr” with the SQP algorithm being selected, and a function is provided to return the analytic partial derivatives of the objective function equation (48) as @J=@u(k) = −@y(N )=@u(k)V (N ) − y(N )h;
(52)
where @y(N )=@u(k) is computed according to Eqs. (32)–(38). The partial derivatives of the constraint, Eq. (50), can be calculated analytically as @H (u)=@u(k) = h:
(53)
The optimal control policy based on the GMNN model is also obtained using the MATLAB function “constr” with the SQP algorithm being selected, but the gradient information is not provided. To cross-check these optimal control results, the IDP algorithm was used to calculate the optimal control proHle based on the rigorous mechanistic model, i.e. Eqs. (39)–(44). The parameters in the IDP algorithm are chosen as follows: time stage P = 10, number of grid points N = 41, number of allowable control values M = 7, number of iterations T = 20, and a reduction factor = 0:7, which are the same as used in [12]. The piecewise constant optimal control policies of all three cases are shown in Fig. 7, and the corresponding outputs V (k) under these optimal control policies in all cases are shown in Fig. 8. In Figs. 7 and 8, the solid lines represent the results from the mechanistic model with IDP, the dashed lines represent the results from the CAFNN model with SQP, and the dash-dotted lines represent the results from the GMNN model with SQP. The Hnal volume V (N ) in all cases are exactly 200 l. Figs. 9 and 10 compare the actual (solid lines) and the NN model predicted (dashed lines) values of output y(k) under the optimal feeding policies based on the CAFNN and the GMNN models, respectively. Fig. 11 shows the actual process outputs (simulated values from the mechanistic model) under the three optimal feeding policies. In Fig. 11, the solid line represents the results from the mechanistic model with IDP, the dashed line represents the results from the CAFNN model with SQP, and the dash-dotted line represents the results from the GMNN model with SQP. Note that the
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
333
Optimal Control 12
10
IDP CAFNN GMNN
Feed rate (L/h)
8
6
4
2
0 0
10
20
30 time (h)
40
50
60
50
60
Fig. 7. Optimal feeding policies. 200 180 IDP CAFNN GMNN
160
Volume (L)
140 120 100 80 60 40 20 0
0
10
20
30 time (h)
40
Fig. 8. Trajectories of the reactor volume.
“optimal” control policies from the CAFNN and the GMNN models are optimal on the respective neural network models but may not be optimal on the actual process due to model–plant mismatches. Therefore, they should be considered as sub-optimal policies on the actual process.
334
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
ymdl(tf) = 101.7767 ymCAFNN(tf) = 103.1358 110
Product Concentration x3(g / L)
100 90 80 70 60 50 40 30 20
Model CAFNN
10 0
0
10
20
30 time (h)
40
50
60
Fig. 9. Actual and predicted values of output y(k) under the sub-optimal feeding policy based on the CAFNN model. ymdl(tf) = 100.9962 ymGMNN(tf) = 103.2124 110
Product Concentration x3 (g /L)
100 90 80 70 60 50 40 30 Model GMNN
20 10 0
0
10
20
30 time (h)
40
50
60
Fig. 10. Actual and predicted values of output y(k) under the sub-optimal feeding policy based on the GMNN model.
The performances of all three cases are shown in Table 3, where the performance indices were calculated according to Eq. (45), i.e. PI = y(N )V (N ). From Table 3 it can be seen that the optimal feeding policy based on IDP with the mechanistic model
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
335
ymdl(tf) = 103.7319 ymCAFNN(tf) = 101.7767 ymGMNN(tf) = 100.9962 110
Product Concentration x3 (g/L)
100 90 80 70 60 50 40 30 IDP CAFNN GMNN
20 10 0
0
10
20
30 time (h)
40
50
60
Fig. 11. Process outputs y(k) under di3erent optimal control policies.
Table 3 Performance comparison on the optimal control of the fed-batch process
CAFNN GMNN Mechanistic model
Performance index on the corresponding model
Performance index on process
Optimisation CPU time (min)
20,627 20,642 20,706
20,355 20,199 20,706
3.72 5.19 142.38
give the best performance, but it takes much longer CPU time than both CAFNN and GMNN model based optimal control because it has to solve those di3erential equations of the rigorous mechanistic model at each iteration. It is therefore very di$cult to apply the rigorous mechanistic model based IDP optimisation to this fed-batch process. Furthermore, the sub-optimal feeding policy based on the CAFNN model gives good performance on the process (only 1.9% less than that based on the mechanistic model), and its performance is also better than that based on the GMNN model. This could be due to the better generalisation capability of the CAFNN model in modelling fed-batch processes and the ease of optimisation with SQP when the gradient information is supplied based on the CAFNN model. 6. Conclusions In order to solve the control a$ne fed-batch process optimal control problem more accurately and e$ciently, a novel neural network structure, CAFNN, is proposed in this
336
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
paper. The training algorithm of CAFNN is obtained easily using a modiHed LM algorithm. According to the special topology of CAFNN, the analytic gradient information can be computed directly o3ering an e3ective and simple optimal control strategy by SQP for constrained nonlinear fed-batch process optimal control problems. The proposed modelling and optimal control scheme are illustrated on an application to an ethanol fermentation process. It is shown that a CAFNN has better generalisation capability than a conventional feedforward network when modelling a fed-batch process. Due to enhanced model generalisation and the availability of model output gradients with respect to controls form the CAFNN model, an accurate and e$cient optimal policy is obtained. Compared to the GMNN and IDP with mechanistic model results, the results of optimal control based on a CAFNN model demonstrate the e3ectiveness of the proposed modelling and optimal control scheme for fed-batch processes. Acknowledgements The work was supported by the UK EPSRC through the Grant GR/N13319— “Nonlinear Optimising Control in Agile Batch Manufacturing”. The authors thank the anonymous reviewers for their constructive comments which helped to improve the quality and presentation of this paper.
References [1] S.A. Billings, W.S.F. Voon, Correlation based model validity tests for non-linear models, Int. J. Control 44 (1986) 235–244. [2] N. Bhat, T. McAvoy, Use of neural nets for dynamic modelling and control of chemical process systems, Comput. Chem. Eng. 14 (1990) 573–583. [3] S. Chen, S.A. Billings, Neural networks for non-linear dynamic system modeling and identiHcation, Int. J. Control 56 (1992) 319–346. [4] G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signal Syst. 2 (1989) 303–314. [5] A. Delgado, C. Kambhampati, K. Warwick, Dynamic recurrent neural network for system identiHcation and control, IEE Proc.-Control Theory Appl. 142 (4) (1995) 307–314. [6] V.I. Elkin, Reduction of Nonlinear Control Systems: Di3erential-Geometric Approach, Moscow, 1997. [7] J. Hong, Optimal substrate feeding policy for fed batch fermentation with substrate and product inhibition kinetics, Biotechnol. Bioeng. 27 (1986) 1421–1431. [8] K.J. Hunt, D. Sbarbaro, R. Zbikowski, P.J. Gawthrop, Neural networks for control systems—a survey, Automatica 28 (1992) 1083–1112. [9] J. Li, R.R. Rhinehart, Heuristic random optimisation, Comput. Chem. Eng. 22 (1998) 427–444. [10] R. Luus, Optimal control by dynamic programming using systematic reduction in grid size, Int. J. Control 51 (1990) 995–1013. [11] R. Luus, On the application of iterative dynamic programming to singular optimal control problems, IEEE Trans. Autom. Control 37 (1992) 1802–1806. [12] R. Luus, Application of dynamic programming to di3erential—algebraic process systems, Comput. Chem. Eng. 17 (1993) 373–377. [13] S. Park, W.F. Ramirez, Optimal production of secreted protein in fed-batch reactors, AIChE J. 34 (1998) 1550–1558.
Z. Xiong, J. Zhang / Neurocomputing 61 (2004) 317 – 337
337
[14] J. Sjoberg, Q. Zhang, L. Ljung, A. Benveniste, B. Delyon, P.Y. Glorennec, H. Hjalmarsson, A. Juditsky, Non-linear black-box modeling in system identiHcation: a uniHed overview, Automatica 31 (1995) 1691–1724. [15] Y. Tian, J. Zhang, J. Morris, Modelling and optimal control of a batch polymerisation reactor using a hybrid stacked recurrent neural network model, Ind. Eng. Chem. Res. 40 (2001) 4525–4535. [16] Y. Tian, J. Zhang, J. Morris, Optimal control of a batch emulsion copolymerisation reactor based on recurrent neural network models, Chem. Eng. Processing 41 (2002) 531–538. [17] E. Visser, B. Srinivasan, S. Palanki, D. Bonvin, A feedback-based implementation scheme for batch process optimisation, J. Process Control 10 (2000) 399–410. [18] J. Zhang, Developing robust neural network models by using both dynamic and static process operating data, Ind. Eng. Chem. Res. 40 (2001) 234–241. [19] J. Zhang, A.J. Morris, Recurrent neuro-fuzzy networks for non-linear process modeling, IEEE Trans. Neural Networks 10 (1999) 313–326. Zhihua Xiong received his B.Sc. and M.Sc. degrees in automatic control from Northeastern University, Shenyang, China, in 1993 and 1996, respectively, and the Ph.D. degree in control theory and control engineering from Tsinghua University, Beijing, China, in 2000. He has been a postdoctoral Research Associate in the School of Chemical Engineering and Advanced Materials, University of Newcastle, UK, since 2000. His research interests include neural networks, intelligent control system, optimal control of batch process, model predictive control, and iterative learning control.
Jie Zhang is a Lecturer in the School of Chemical Engineering & Advanced Materials, University of Newcastle, England. He received his B.Sc. degree in Control Engineering from Hebei University of Technology, Tianjin, China, in 1986 and his Ph.D. degree in Control Engineering from City University, London, in 1991. His research interests include: neural networks, neuro-fuzzy systems, fault detection and diagnosis, intelligent control systems, genetic algorithms, optimal control of batch processes, and multivariate statistical process control. He has published over 120 papers in international journals, books, and conferences. He served as a reviewer for many prestigious international journals including IEEE Transactions on Neural Networks, IEEE Transactions on Fuzzy Systems, Neural Networks, Automatica, Chemical Engineering Science, and IEE Proceedings. He is a Senior Member of IEEE and a member of the IEEE Control Systems Society, IEEE Neural Network Society, IEEE Systems Man and Cybnetics Society, and IEEE Instrumentation and Measurement Society. His biography is included in the 4th edition of Marquis Who’s Who in Science and Engineering.