Discrete state space modeling and control of nonlinear unknown systems

ISA Transactions 52 (2013) 795–806

Contents lists available at ScienceDirect

ISA Transactions journal homepage: www.elsevier.com/locate/isatrans

Research Article

Discrete state space modeling and control of nonlinear unknown systems Aydogan Savran n Department of Electrical and Electronics Engineering, Ege University, 35100 Bornova, Izmir, Turkey

art ic l e i nf o Article history: Received 24 February 2012 Received in revised form 29 June 2013 Accepted 7 July 2013 Available online 24 August 2013 This paper was recommended for publication by Dr. Ahmad B. Rad Keywords: Neural network State-space Linearization LM BPTT MIMO State feedback LQ Pole placement

a b s t r a c t A novel procedure for integrating neural networks (NNs) with conventional techniques is proposed to design industrial modeling and control systems for nonlinear unknown systems. In the proposed approach, a new recurrent NN with a special architecture is constructed to obtain discrete-time state-space representations of nonlinear dynamical systems. It is referred as the discrete state-space neural network (DSSNN). In the DSSNN, the outputs of the hidden layer neurons of the DSSNN represent the system's (pseudo) state. The inputs are fed to output neurons and the delayed outputs of the hidden layer neurons are fed to their inputs via adjustable weights. The discrete state space model of the actual system is directly obtained by training the DSSNN with the input–output data. A training procedure based on the backpropagation through time (BPTT) algorithm is developed. The Levenberg–Marquardt (LM) method with a trust region approach is used to update the DSSNN weights. Linear state space models enable to use well developed conventional analysis and design techniques. Thus, building a linear model of a system has primary importance in industrial applications. Thus, a suitable linearization procedure is proposed to derive the linear state space model from the nonlinear DSSNN representation. The controllability, observability and stability properties are examined. The state feedback controllers are designed with both the linear quadratic regulator (LQR) and the pole placement techniques. The regulator and servo control problems are both addressed. A full order observer is also designed to estimate the state variables. The performance of the proposed procedure is demonstrated by applying for both single-input single-output (SISO) and multiple-input multiple-output (MIMO) nonlinear control problems. & 2013 ISA. Published by Elsevier Ltd. All rights reserved.

1. Introduction Neural networks (NNs) have learning, adaptation, and powerful nonlinear mapping capabilities. They are able to learn the system dynamics without assuming much knowledge about the internal structure of the system under consideration. The research on NN based control and modeling has great interest in the past few decades [1–8]. Although, there are a lot of good results have been reported in the literature [9–14], the real world applications of these techniques are not enough. The much effort is required to bridge the gap between theory and application. The state feedback control technique can achieve magnificent results with state-space models. However, in many cases, to obtain a first-principal model of a plant is too difficult and time consuming. The NNs are directly trained with the input–output data to form a model. Thus, the modeling process can be simplified. However, a typical feedforward NN model does not have a state-space

n

Tel.: +90 232 3111664; fax: +90 232 3886024. E-mail address: [email protected]

realization [15]. There are several attempts to obtain conventional model by means of NNs [16–18]. A so-called state space NN model is developed and applied to some problems in a series of papers [17,19]. This special recurrent neural network is composed of five layers: the input layer, two hidden layers, one state layer, and the output layer. They proposed a linearization procedure by supposing the activation functions are linear and setting the biases to zero. In [18] a state-space dynamic neural network method for modeling the transient behaviors of high-speed nonlinear circuits is presented. The SSDNN technique extends the existing dynamic neural network (DNN) approaches into a more generalized and robust formulation. In [20], a network, called model based recurrent neural network, is defined to initially emulate a linearized state-space model of the plant. It can then be trained to accommodate the nonlinearities of the plant by modifying its activation functions, which are defined as contours of RBF's comprising each node. But, the applications of this approach are limited by the requirement of the analytical knowledge of the plant. In [21] a feedforward NN based state space modeling and control of discrete systems are considered. They claim that their approach does not rely on a physical principle model of the dynamic system. But, the network

0019-0578/$ - see front matter & 2013 ISA. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.isatra.2013.07.005

796

A. Savran / ISA Transactions 52 (2013) 795–806

outputs have been considered as the state variables which have to be available to train the network. So, their approach requires already a physical principle model of the plant. The focus of this paper is on the development of a NN based control procedure for real-world systems using minimal underlying assumptions. Thus, a NN based state-space modeling and control procedure is developed by integrating neural networks with conventional techniques. In this procedure, a new recurrent neural network (RNN) with a special architecture is constructed to obtain discrete time state-space models of nonlinear dynamic systems. Also, linear state space representation is derived by linearizing it. The proposed network is referred as the discrete state-space neural network (DSSNN). The proposed network architecture and linearization procedure differ from the other approaches in the literature. The inputs are fed to output neurons and the delayed outputs of the hidden layer neurons are fed to their inputs via adjustable weights. The outputs of the hidden layer neurons represent the state variables. The Levenberg–Marquardt (LM) method with a trust region approach is used to adjust the DSSNN weights [22]. After obtaining the nonlinear DSSNN model, a linearization procedure is applied to derive a linear state space representation of the system. Thus, a linear state space model of the nonlinear system, which enables to design well developed state space controllers, is obtained from the input– output measurements of the plant. The state feedback controllers are designed by using the linearized model with both the LQ control and the pole placement methods. All of the design procedure is demonstrated through several nonlinear systems. The remaining of the paper is organized as follows. The state space method is introduced in Section 2. The DSSNN architecture and training procedure is given in Section 3. Section 4 describes the linearization procedure. In Section 5, the control system structure is given. The illustrative examples are given in Section 6. Section 7 concludes the paper.

The linearized equations make possible to use well developed analysis and design techniques. However, if the knowledge about the system is limited, it is not easy to obtain a linear representation with conventional techniques. Therefore, we propose a NN based approach to obtain a state space representation by means of the input–output data of the system. 2.1. Stability, controllability, observability Two types of stabilities are considered for linear state space systems: BIBO stability and internal stability. The BIBO stability can be determined by means of the transfer function poles. We obtain the discrete-time transfer function (or matrix) of a system from the state space model as GðzÞ ¼

YðzÞ ¼ C½zIA1 B þ D UðzÞ

If every pole of G(z) has a magnitude less than 1, the discrete time system given in (4) is BIBO stable. Also, the internal stability of any discrete time system can be determined by means of the eigenvalues. The system is called marginally stable if all eigenvalues of A have magnitudes less than or equal to 1. If all eigenvalues of A have magnitudes less than 1, the system is called asymptotically stable. Also note that asymptotically stability implies BIBO stability but not the converse. The controllability and observability specify the state space control systems. The controllability of a LTI system is characterized in terms of the matrix below h i M ¼ B AB … An1 B ð6Þ

2. State space representation and analysis The state space representation of systems has several advantages over input–output representation. State space models convey much knowledge about the internal relations of systems. The discrete time state space representation of a nonlinear system is given by xðk þ 1Þ ¼ fðxðkÞ; uðkÞÞ yðkÞ ¼ gðxðkÞ; uðkÞÞ

ð1Þ

n

m

p

where x∈R is the state vector, u∈R and y∈R represent the input and output of the system, respectively. f and g are the nonlinear vector functions. A linear description of the system is obtained by linearizing Eq. (1) about a certain operating point ðx; uÞ. We define the partial derivatives of the vector functions f and g with respect to the vector x and u by f x ðx; uÞ ¼

∂fðx;uÞ ∂x ;

gx ðx; uÞ ¼

∂gðx; uÞ ; ∂x

f u ðx;uÞ ¼

∂fðx;uÞ ∂u

gu ðx;uÞ ¼

∂gðx; uÞ ∂u

ð2Þ

Thus, the constant matrices A∈R nn , B∈Rnm , C∈R pn , and D∈R pm are defined as A ¼ f x ðx; uÞ;

B ¼ f u ðx; uÞ

C ¼ f x ðx; uÞ;

D ¼ f u ðx; uÞ

ð3Þ

and the linearized state space equations are obtained as xðk þ 1Þ ¼ AxðkÞ þ BuðkÞ yðkÞ ¼ CxðkÞ þ DuðkÞ

ð4Þ

ð5Þ

Fig. 1. Structure of the DSSNN.

A. Savran / ISA Transactions 52 (2013) 795–806

797

The matrix M is called the controllability matrix. The system is completely state controllable, if and only if the rank of M is equal to the number of the state variables n. Similarly, the observability is tested by means of the observability matrix below h i n n ð7Þ N ¼ Cn A Cn … ðA Þn1 Cn The system is completely observable, if and only if the rank of N is equal to n.

3. Discrete state space neural network The proposed state space NN architecture is depicted in Fig. 1, where uðkÞ, and yðkÞ represent the input and output of the RNN, respectively, k is the discrete time index, and z1 represents the time delay operators. xðkÞ represents the hidden layer output which corresponds to the state vector. W1, W2 and W4 are the weights between the input and hidden layers, the hidden and output layers, and the input and output layers respectively. The weighed sums of the delayed outputs of the hidden layer are applied to the hidden layer neurons via the weights (W3 ). The bias connections to the neurons are omitted to simplify the presentation in the figure. Fig. 2 depicts the details of the network where each layer is simply represented by only one of its neurons. To train the recurrent systems, the BPTT like derivative calculation is required. But, the calculation of the derivatives by using the chain rule or by the unfolding in time is very complicated. So we

Fig. 3. The adjoint model of the DSSNN.

Fig. 4. Regulator with full state feedback.

Fig. 2. Connections of the DSSNN.

built the adjoint model of the DSSNN, which is depicted in Fig. 3, to simplify the computations. It is constructed by reversing the branch directions, replacing summing junctions with branching points and vice versa, and replacing the time delay operators with time advance operators. The Jacobian matrix or the gradient vector is easily computed by means of the adjoint model of the DSSNN. Since the weights are updated by the LM method, the calculation of the Jacobian matrix is required. The elements of the Jacobian matrix for an output of the DSSNN are computed by feeding 1 instead of the corresponding error value e in the adjoint model

798

A. Savran / ISA Transactions 52 (2013) 795–806

and 0 for others. The backward phase computations from k ¼T to k ¼1 are performed by means of the adjoint model of the DSSNN. When the forward and backward phases of the computations are completed, the sensitivities for each weight, which form the Jacobian matrix, are obtained as in the BPTT algorithm [8]. The elements of the Jacobian matrix are computed in two stages which are referred to as the forward and backward phases. In the forward phase, the DSSNN actions are computed and stored from k¼ 1 to k¼ T through the trajectory. The errors at every k are determined as the differences between the desired outputs and the DSSNN outputs. The initial values for the output of the hidden layer (x) are set to zeros. xð0Þ ¼ 0

ð8Þ

The net quantities (netx ) of the hidden layer neurons and their outputs (x) are computed by netx ðkÞ ¼ W3 xðk1Þ þ W1 uðk1Þ þ B1 xðkÞ ¼ ψðnetx ðkÞÞ

ð9Þ

where W1 represents the weights between the input and hidden layers, W3 represents the weights on the feedback path, and B1 the biases applied to the hidden layer neurons. ψ represents the hidden layer activation functions. Similarly, the net quantities (nety ) of the output layer neurons and their outputs (y) are computed by nety ðkÞ ¼ W2 xðkÞ þ W4 uðkÞ þ B2 yðkÞ ¼ φðnety ðkÞÞ

ð10Þ

where W2 , W4 , B2 , and φ represent the weights between the hidden and output layers, the weights between the input and output layers, the biases applied to the output layer neurons, and the output layer activation functions, respectively. The error signal (e) is defined as the difference between the DSSNN output (y) and the desired output (yd). eðkÞ ¼ ðyðkÞyd ðkÞÞ

ð11Þ

We define the instantaneous value of the error energy, which is a function of all the free parameters, as EðkÞ ¼

1 T ðe ðkÞeðkÞÞ 2

ð12Þ

Then the cost function defined as E total ¼

1 T ∑ EðkÞ Tk¼1

ð13Þ

The weights are adjusted to minimize the cost function. So the sensitivities with respect to each weight have to be computed. At every k, the sensitivity for each weight is computed by multiplying the values scaled by the weight in the DSSNN and the adjoint model. Therefore, after completing the forward phase computations, the backward phase computation is carried out through the adjoint model of DSSNN from k¼ T to k ¼1. The local sensitivities at k ¼T+1 are set to zeros. δ1 ðT þ 1Þ ¼ 0

ð14Þ

adjoint model as follows: ∂eðkÞ ∂eðkÞ ¼ δ2 ðkÞxT ðkÞ; ¼ δ2 ðkÞ ∂W2 ∂B2 ∂eðkÞ ∂eðkÞ ¼ δ1 ðkÞuT ðk1Þ; ¼ δ1 ðkÞ ∂W1 ∂B1 ∂eðkÞ ∂eðkÞ ¼ δ1 ðkÞxT ðk1Þ; ¼ δ2 ðkÞuT ðkÞ ∂W3 ∂W4

The overall sensitivity for each weight is obtained by summing the related sensitivity in (17) over the trajectory. The Jacobian matrix which is required to train the DSSNN is h i ∂e ∂e ∂e ∂e ∂e ∂e ð18Þ J ¼ ∂W1 ∂B1 ∂W2 ∂B2 ∂W3 ∂W4 The gradient vector is computed from the Jacobian matrix (J) by g ¼ JT e

ð19Þ

The network weight vector w is defined as w ¼ ½W1 ; B1 ; W2 ; B2 ; W3 ; W4 

ð20Þ

The change in the weight vector Δwn at the nth iteration is computed by the LM method ðJTn Jn þ μn IÞΔwn ¼ JTn en

ð21Þ

where μn ≥0 is a scalar and I is the identity matrix. For a sufficiently large value of μn, the matrix (JTn Jn þ μn I) is positive definite and Δwn is a descent direction. When μn ¼0, Δwn is the Gauss–Newton T vector. Asμn -1, μnI term dominates so that Δwn -μ1 n Jn en represents an infinitesimal step in the steepest descent direction [22]. We used the trust region approach of Fletcher to determine μn. 4. Linearization procedure We obtain a discrete-time, nonlinear, state-space representation of the system by the DSSNN. However, our aim is to synthesize a linear control system based on a linear state-space model. Therefore, a linear model from the DSSNN model is derived with an appropriate linearization process. The DSSNN equations are netx ðkÞ ¼ W3 xðk1Þ þ W1 uðk1Þ þ B1 xðkÞ ¼ ψðnetx ðkÞÞ nety ðkÞ ¼ W2 xðkÞ þ W4 uðkÞ þ B2 yðkÞ ¼ φðnety ðkÞÞ

xðkÞ ¼ ψðxðk1Þ; uðk1ÞÞyðkÞ ¼ φðxðkÞ; uðkÞÞ

∂ψðx; uÞ ∂ψðnetx Þ ∂netx ¼ ; ∂x ∂netx ∂x

φx ðx; uÞ ¼

∂φðnety Þ ∂nety ; ∂nety ∂x

The local sensitivities are obtained as

ψx ðx; uÞ ¼ ψ ðnetx Þ⋅W3 ;

δ2 ðkÞ ¼ φ′ðnety ðkÞÞeðkÞ

φy ðx; uÞ ¼ φ ðnety Þ⋅W2 ;

ψu ðx; uÞ ¼

∂ψðnetx Þ ∂netx ∂netx ∂u

∂φðnety Þ ∂nety ∂nety ∂u

φu ðx; uÞ ¼

ð24Þ

We can rewrite the equations inserting the equivalents of the derivatives 0

0

In the case of the calculation of the Jacobian matrix, e(k) is set to 1 in (11). Then, the sensitivity for each weight is computed by multiplying the values scaled by this weight in the DSSNN and the

ð23Þ

Our objective is to obtain a linear state space description of the system. We define the partial derivative of the vector function ψ and φ with respect to the vector x and u with the chain rule as ψx ðx; uÞ ¼

ð16Þ

ð22Þ

where x and u are the independent variables, and W1 ; W2 ; W3 ; W4 ; B1 ; and B2 are the constant matrices. We can express (22) with the standard notation as

The derivatives of the activation functions of each layer with respect to their inputs are computed as ∂φðnetðkÞÞ  φ′ðnety ðkÞÞ ¼  ∂netðkÞ net ¼ nety ðkÞ ∂ψðnetðkÞÞ  ψ′ðnetx ðkÞÞ ¼ ð15Þ  ∂netðkÞ net ¼ netx ðkÞ

δ1 ðkÞ ¼ ψ′ðnetx ðkÞÞ½W3 T δ1 ðk þ 1Þ þ WT2 δ2 ðkÞ

ð17Þ

0

ψu ðx; uÞ ¼ ψ ðnetx Þ⋅W1 0

φu ðx; uÞ ¼ φ ðnety Þ⋅W4

ð25Þ

The constant matrices are defined about a certain operating point ðx; uÞ as A ¼ ψx ðx; uÞ; C ¼ φx ðx; uÞ;

B ¼ ψu ðx; uÞ D ¼ φu ðx; uÞ

ð26Þ

A. Savran / ISA Transactions 52 (2013) 795–806

The output of the tangent hyperbolic function is in the range [  1,1]. Also, the input and output data which is applied to a NN having tangent hyperbolic functions are scaled into [ 1,1]. So, the zero corresponding the center of the interval could be good choice as the operating point to linearize the model. Thus, the operating point is properly determined as ðu; xÞ ¼ ð0; 0Þ. The linearized equations are xðkÞ ¼ Axðk1Þ þ Buðk1Þ yðkÞ ¼ CxðkÞ þ DuðkÞ

ð27Þ

We are able to use well developed analysis and design techniques for the linear state space model.

5. Control system structure

^ þ 1Þ ¼ AxðkÞ ^ ^ xðk þ BuðkÞ þ KE ½yp ðkÞyðkÞ ð28Þ

^ where KE is the observer gain matrix, xðkÞ is the observed state vector. The control signal is computed by ^ uðkÞ ¼ KxðkÞ

Then the control signal is computed as ^ uðkÞ ¼ KxðkÞ þ K i ei ðkÞ

ð29Þ

where K is the state feedback gain matrix. In order to complete the control procedure, we must just compute the gain matrices K and KE by means of the matrices A, B, C, and D. Integral control is useful for eliminating the steady state errors due constant disturbance and reference input commands. Therefore most of the control systems include some form of integral control. The integral control scheme is designed with the full-state feedback as shown in Fig. 5. In this scheme, an integrator is inserted on the forward path. In this case, the integrator gain K i is calculated in addition to the state feedback gain matrix K. The A and B matrices are augmented as     0 ^¼ A ^¼ B A ; B ð30Þ C 1 0

where ei ðkÞ is the discrete integral of the errors. 6. Illustrative examples In this section, the proposed modeling and control procedure is tested through two nonlinear plants. Firstly, the DSSNNs are trained to model the plants. Then, the nonlinear DSSNN models are linearized to obtain linear state space realizations. The stability, controllability, and observability properties are examined. Finally, the state feedback control systems are designed and their performances are tested.

As a first example, a SISO nonlinear plant has been chosen to test the modeling and control performance of the proposed approach. The plant is described by the following difference equation yp ðkÞ ¼

yp ðk1Þup ðkÞ  tan ðup ðkÞÞ 1 þ y2p ðk1Þ

6.1.1. Modeling Firstly, the state space modeling of the plant by the DSSNN is considered. The DSSNN has 1 input, 1 output, and 4 hidden layer neurons. The hyperbolic tangent activation functions are used in the hidden layer. The linear activation functions are used in the output layer. The weights are initialized randomly within the interval ½0:5; 0:5. The LM method with the trust region approach is used to update the DSSNN weights. The training and testing data sets are obtained by applying independent and identically distributed (iid) uniform sequence over [  1,1] for 1000 and 100 samples to plant input, respectively. Fig. 6 shows the MSE curves in the logarithmic scale for both the training and testing data sets. It indicates that the most of the learning was done in the first 15 epochs. The modeling performance of the DSSNN for the testing data is shown in Figs. 7 and 8. The plant and DSSNN output values are essentially the same in Fig. 7. Therefore, Fig. 8 was plotted to distinguish the difference between the plant and the DSSNN outputs.

ð31Þ

Fig. 5. Integral control with full state feedback.

ð33Þ

where k is the discrete time step, and up(k) and yp(k) are the input and the output of the plant, respectively.

Thus, the augmented gain matrix is obtained with the pole placement and the LQ control method as ^ ¼ ½K; K i  K

ð32Þ

6.1. SISO nonlinear plant

The state feedback control system is synthesized using the linearized state space model. Both regulator and integral control systems are considered. The pole placement and the linear quadratic (LQ) control techniques are used to compute the state feedback gain matrices. The overall structure of the regulator– observer system is shown in Fig. 3. In the state feedback control techniques, all state variables have to be available for feedback to compute the control signal. However, typically, not all state variables are measured. Then, we need to estimate unavailable state variables. We designed a full-order state observer to estimate these variables based on the measurement of the output

^ ^ yðkÞ ¼ CxðkÞ þ DuðkÞ

799

Fig. 6. MSE in the logarithmic scale.

800

A. Savran / ISA Transactions 52 (2013) 795–806

GðzÞ ¼

0:0002z4 1:0015z3 0:0409z2 0:0169z0:0024 z4 0:0010z3 0:0166z2 þ 0:0012z þ 0:0001

ð35Þ

Then, we can easily determine the BIBO stability of the system by means of the TF poles. The poles are obtained as 

0:0971 þ 0:0645i;

0:09710:0645i;

0:1412;

0:0521

 ð36Þ

Since the magnitudes of the each poles are less than 1, the system is BIBO stable. The internal stability of the system can also be determined by the eigenvalues of A. Since there is no canceled pole in the TF, the eigenvalues are the same with the poles and their magnitudes are less than 1. Therefore, the system is also asymptotically stable. The controllability of the by writing the h system is determined i controllability matrix M ¼ B BA BA2 BA3 as in (6). The rank of M is 4 which equals to the number of the state variables. Therefore, the system The obserh is completely state controllable. i n n n vability matrix N ¼ Cn A Cn ðA Þ2 Cn ðA Þ3 Cn is also written as in (7). The rank of N is 4. Therefore, the system is completely observable.

Fig. 7. Outputs of the plant and the DSSNN.

6.1.3. Control We synthesized the state feedback control systems with the pole placement and the LQ control method using the linear state space realization of the plant. The regulator and the integral control problems are both addressed.

6.1.3.1. Design with the pole placement method. For linear timeinvariant systems, pole locations specify both the stability and the transient responses of closed-loop systems. In the pole placement technique, all of the system poles can be placed to the desired places. The control signal is computed by multiplying the state feedback gain matrix with the instantaneous state as uðkÞ ¼ KxðkÞ Fig. 8. The instantaneous modeling error.

6.1.2. Analysis We need to obtain a linear state-space model of the plant to design a state feedback control system. So, after training, the DSSNN model is linearized as explained in Section 4. Therefore, we obtain a discrete-time, linear, time-invariant, state-space model of the plant with the matrices: 2 3 0:0283 0:0509 0:7998 0:7350 6 0:0420 0:1046 0:0373 0:0287 7 6 7 A¼6 ð34aÞ 7 4 0:0319 0:1222 0:0726 0:2212 5 0:0312 2

2:9923

0:1272

0:0970

In order to compute the control signal, all of the state variables have to available at every k. Therefore, an observer is required to estimate the unmeasured state variables. We designed the observer with the pole placement method. The observer gain matrix KE is obtained

0:2044

3

6 0:1320 7 7 6 B¼6 7 4 0:5865 5

ð34bÞ

0:6292  C ¼ 0:0745

0:5966

D ¼ ½2:4577e004

0:5027

0:8943



ð34cÞ ð34dÞ

The transfer function (TF) is computed from the state space matrices A, B, C, and D as in (5)

ð37Þ

Fig. 9. The state variables with the pole placement method.

A. Savran / ISA Transactions 52 (2013) 795–806

with the pole placement method as 2 3 4:1232 6 0:3564 7 6 7 KE ¼ 6 7 4 1:7668 5

801

ð38Þ

1:1549 After checking the controllability of the system, the state feedback gain matrix K is obtained with pole placement method as   ð39Þ K ¼ 0:0170 0:1279 0:0978 0:3299 Using these gain matrices, the regulator is configured as shown in Fig. 4. The performance of the regulator is shown in Figs. 9 and 10. We see that the state variables and the system output converge to zeros as desired. The integral control system structure is given in Section 5. In this case, the integral gain Ki has to be computed in addition to the ^ and B^ state feedback gain matrix. So, the augmented matrices A are written as described in (30) 2 0:0283 0:0509 0:7998 6 0:0420 0:1046 0:0373 6 6 ^ ¼ 6 0:0319 0:1222 0:0726 A 6 6 4 0:0312 0:1272 0:0970 0:0745

0:5966

0:5027

0:7350

0

0:0287

0

0:2212 0:2044

0 0

0:8943

1:0000

3 7 7 7 7 7 7 5

3 2:9923 6 0:1320 7 6 7 6 7 ^ ¼ 6 0:5865 7 B 6 7 6 7 4 0:6292 5 0

Fig. 11. The integral control system response for the pole placement method.

ð40aÞ

2

ð40bÞ

Then the gains are computed with the pole placement method using these matrices   ^ ¼ ½K; K i  ¼ 0:0025 0:0028 0:0185 0:1064 0:1681 K ð41Þ Using these gain matrixes, the integral control system is configured as shown in Fig. 5. The response of the control system is shown in Fig. 11. We see that the system output track the reference very closely. 6.1.3.2. Design with the LQ control method. The LQ control has a linear feedback law of the following form: uðkÞ ¼ KxðkÞ

Fig. 10. The plant output with the pole placement method.

Fig. 12. The state variables with the LQ control method.

ð42Þ

Fig. 13. The plant output with the LQ control method.

802

A. Savran / ISA Transactions 52 (2013) 795–806

where K is the feedback gain matrix. In the LQ control method, the K matrix is determined to minimize a quadratic cost function J¼

1 N ∑ ðxT ðkÞQ xðkÞ þ uT ðkÞRuðkÞÞ 2k¼0

ð43Þ

In this example, we have simply taken the weighting matrices Q and R as identity matrices with the proper sizes. The state feedback gain matrix with LQ control method is obtained as   K ¼ 0:0073 0:0157 0:2156 0:2005 ð44Þ The regulator is configured as shown in Fig. 4. The same observer gain matrix of (38) is used. The regulator performance is shown in Figs. 12 and 13. We see that the states and the system output reach zeros as desired. The integral control system given in Section 5 is also designed with the LQ control approach. The integral gain Ki is computed in addition to the state feedback gain matrix using the matrices ^ and B ^ of (40) A   ^ ¼ ½K; K i  ¼ 0:0299 0:1488 0:3104 0:3843 0:2637 K

Fig. 15. Outputs of the plant and the DSSNN.

ð45Þ The performance of the LQ control system is shown in Fig. 14. The system output track the reference very closely. 6.2. MIMO nonlinear plant As a second example, the proposed procedure is applied for modeling and control of a multiple-input multiple-output (MIMO) nonlinear dynamical system with two inputs and two outputs. The plant is described by the following difference equation: 2 3 yp1 ðk1Þ " # 2 ðk1Þ þ 0:2yp2 ðk1Þ þ 0:8up1 ðkÞ0:5up2 ðkÞ 1þy yp1 ðkÞ 6 7 p2 7 ¼6 4 yp1 ðk1Þ yp2 ðk1Þ 5 yp2 ðkÞ þ 0:2y ðk1Þ þ 0:4u ðkÞ þ 0:4u ðkÞ p1 p2 2 p2 1þy ðk1Þ p2

ð46Þ where k is the discrete time step, and up1(k), up2(k), and yp1(k), yp2(k) are the inputs and the outputs of the plant, respectively. 6.2.1. Modeling In this example, the DSSNN with 2 inputs, 2 outputs, and 4 hidden layer neurons is used to model the plant. The hyperbolic tangent activation functions are used in the hidden layer. The linear

Fig. 16. The instantaneous modeling error.

activation functions are used in the output layer. The weights are initialized to small random values around zero. The LM method with the trust region approach is used to update the DSSNN weights. The training and testing data sets are obtained by applying independent and identically distributed (iid) uniform sequence over [ 0.5,0.5] for 1000 and 100 samples to both plant inputs, respectively. The modeling performance of the DSSNN for the testing data is shown in Figs. 15 and 16. The plant and DSSNN output values are essentially the same in Fig. 15. Therefore, Fig. 16 was plotted to distinguish the difference between the desired and the network outputs.

6.2.2. Analysis The DSSNN model of the MIMO plant is linearized as explained in Section 4. The discrete-time, linear, time-invariant, state-space model is obtained as 2 3 0:4110 0:2969 0:6113 0:1624 6 1:3301 0:7834 1:9043 0:1518 7 7 6 A¼6 ð47aÞ 7 4 0:3388 0:1359 0:3842 0:2183 5 Fig. 14. The integral control system response for the LQ control method.

0:1081

0:0588

1:1151

0:5556

A. Savran / ISA Transactions 52 (2013) 795–806

2

0:3597

0:0130

6 0:9776 6 B¼6 4 0:4968

 C¼  D¼

3

0:3381 7 7 7 0:1129 5

ð47bÞ

1:5728

0:9671

0:0415

0:0296

0:4026

0:3604

0:8272

0:4830

1:0414

0:0182

0:8010

0:4984

0:4002

0:4002

 ð47cÞ

 ð47dÞ

The transfer matrix is obtained from the state space representation. We checked the poles of the transfer matrix elements. Since the magnitudes of each pole are less than 1, the system is BIBO stable. Also, the internal stability of the plant is determined by the eigenvalues of A. Since the magnitudes of all eigenvalues are less than 1, the plant is asymptotically stable. The controllability and observability of the model are checked by defining the controllability and observability matrices as in (6) and (7). We see that the plant is completely state controllable and observable.

Fig. 18. The plant outputs with the pole placement method.

6.2.3. Regulator design with the pole placement method The regulator is designed using the pole placement and LQ control methods. The control signal is computed by multiplying the state feedback gain matrix with the instantaneous state as in (42). In order to estimate the unmeasured state variables, an observer is designed as explained in Section 5. The observer gain matrix is computed with the pole placement technique as 2 3 0:0579 0:3846 6 0:3296 1:1047 7 6 7 KE ¼ 6 ð48Þ 7 4 0:2038 0:2157 5 0:7353

0:2359

After checking the controllability of the system, the state feedback gain matrix K is obtained with the pole placement method as   0:4831 0:5353 1:7147 0:0216 K¼ ð49Þ 6:7271 2:9171 0:3883 1:7356 Then the overall system is configured as in Fig. 4. The performance of the regulator is shown in Figs. 17 and 18. The state variables and the system output converge to zeros quickly as desired. The control signals for both inputs are also shown in Fig. 19.

Fig. 17. The state variables with the pole placement method.

Fig. 19. The control signals with the pole placement method.

Fig. 20. The state variables with the LQ control method.

803

804

A. Savran / ISA Transactions 52 (2013) 795–806

6.2.4. Regulator design with the LQ control method We repeat the regulator design procedure with the LQ control method. The same observer gain matrix of (48) is used. The weighting matrices Q and R are taken as the identity matrices. The state feedback gain matrix with LQ control method is obtained as  K¼

0:4263

0:2450

0:9241

0:2135

0:5421

0:3288

0:4050

0:1482

 ð50Þ

The regulator performance is shown in Figs. 20 and 21. The state variables and the system output converge to zeros quickly as desired. The control signal is also shown in Fig. 22.

6.2.5. Integral control system design with the LQ control method We designed an integral control system for the MIMO plant to test the tracking performance. In this case, the control problem is much more difficult than the SISO case. The integral control system is configured as given in Section 5. The same observer gain matrix of (48) is used. The integral gain matrix Ki has to be computed in addition to the state feedback gain matrix K. So, the

^ and B ^ are written as described in (30) augmented matrices A 3 2 0:4110 0:2969 0:6113 0:1624 0 0 6 1:3301 0:7834 1:9043 0:1518 0 0 7 7 6 7 6 7 6 0:3388 0:1359 0:3842 0:2183 0 0 ^ ¼6 7 ð51aÞ A 6 0:1081 0:0588 1:1151 0:5556 0 0 7 7 6 6 7 4 0:0415 0:0296 0:4026 0:3604 1 0 5 0:8272 2

0:0130 6 0:9776 6 6 6 ^ ¼ 6 0:4968 B 6 1:5728 6 6 4 0 0

0:4830

1:0414

0:0182

0

1

3 0:3597 0:3381 7 7 7 0:1129 7 7 0:9671 7 7 7 5 0

ð51bÞ

0

In this example, we have taken the weighting matrices as Q ¼ 0:1⋅I66 and R ¼ I22 by means of the identity matrix I. Then, ^ ¼ ½K; K i  are computed with the LQ control method the gains K using these matrices. Thus, the state feedback gain matrix   0:2662 0:1308 0:6554 0:2383 K¼ ð52Þ 0:2885 0:1753 0:1779 0:1186 and the integral gain matrix   0:1845 0:1232 Ki ¼ 0:0805 0:2782

"

Then, the control vector uðkÞ ¼

up1 ðkÞ up1 ðkÞ

is computed as

"

#

^ uðkÞ ¼ KxðkÞ þ Ki ei ðkÞ where the integral vector ei ðkÞ ¼

ð53Þ

#

ð54Þ ei1 ðkÞ ei2 ðkÞ

includes the integrals of

the errors between the references and the plant outputs. The integral control system is configured as like in Fig. 5. The performance of the control system is shown in Figs. 23–25. We see that the system outputs track the references very closely.

6.3. Comparative example

Fig. 21. The plant outputs with the LQ control method.

The performance of the proposed control system is also tested with a comparative study on a nonlinear MIMO discrete-time

Fig. 22. The control signals with the LQ control method.

Fig. 23. The integral control system responses for the LQ control method.

A. Savran / ISA Transactions 52 (2013) 795–806

Fig. 24. The state variables with the LQ control method.

805

Fig. 26. Tracking performance for y1.

Fig. 25. The control signals with the LQ control method.

system given in [23]. x1 ðk þ 1Þ ¼

x2 ðk þ 1Þ ¼

x3 ðk þ 1Þ ¼

1

x21 ðkÞ þ x21 ðkÞ

1þ

x22 ðkÞ

Fig. 27. Tracking performance for y2.

þ 0:3x2 ðkÞ

x21 ðkÞ þ u1 ðkÞ þ 0:1 cos ð0:05kÞ cos ðx1 ðkÞÞ þ x23 ðkÞ þ x24 ðkÞ

x23 ðkÞ x23 ðkÞ þ 0:2x4 ðkÞx4 ðk þ 1Þ ¼ 1 þ x23 ðkÞ 1 þ x21 ðkÞ þ x22 ðkÞ þ x24 ðkÞ

u21 ðkÞ þ u2 ðkÞ þ 0:1 cos ð0:05kÞ cos ðx3 ðkÞÞy1 ðkÞ ¼ x1 ðkÞy2 ðkÞ ¼ x3 ðkÞ ð55Þ where k is the discrete time step, x1 ðkÞ; x2 ðkÞ; x3 ðkÞ; x4 ðkÞ and u1(k), u2(k), and y1(k), y2(k) are the inputs and the outputs of the plant, respectively. The desired reference signals are given as     1 πTk 1 πTk yd1 ðkÞ ¼ 0:5 þ cos þ sin 4 4 4 2     1 πTk 1 πTk þ cos ð56Þ yd2 ðkÞ ¼ 0:5 þ sin 4 4 4 2

Fig. 28. The control inputs u1 and u2.

806

A. Savran / ISA Transactions 52 (2013) 795–806

a discrete-time, linear, time-invariant realization from the input– output data, and to use the well developed state space analysis and design techniques. The simulation studies show that the proposed approach enables to design the state space modeling and control systems for both the nonlinear SISO and MIMO plants, efficiently. References

Fig. 29. Tracking errors.

where T is taken as 0.01. The error terms are defined as ð57Þ e1 ðkÞ ¼ y1 ðkÞyd1 ðkÞ; e2 ðkÞ ¼ y2 ðkÞyd2 ðkÞ  T The initial conditions are set to xð0Þ ¼ 0 0 0 0 . An integral control system as in Section 6.2.5 is designed. The simulation results are shown in Figs. 26–29. As shown in the figures, we obtained a better tracking performance than obtained in [23]. 7. Conclusion In this paper, a procedure for integrating neural networks with conventional techniques has been proposed to design state space modeling and control systems. The architecture and training procedure of a new recurrent neural network (NN), called the discrete state space neural network (DSSNN), have been described. The DSSNN has provided a nonlinear state space model. A linear, time invariant realization has been obtained by linearizing the DSSNN model. Based on this realization, we have designed the state feedback controllers with both the pole placement and the LQ control methods. The regulator and the integral control problems have been considered. A full order observer has been also designed with the pole placement method to estimate the states. The adjoint model of the DSSNN has been built to simplify the back propagation through time (BPTT) like derivative calculation. The fast convergence of the DSSNN weights has been obtained by the Levenberg–Marquardt algorithm. The performance of the proposed procedure has been tested by applying for both SISO and MIMO nonlinear plants. It has been shown that the DSSNN provides an efficient alternative to the first principal methods to model dynamic systems. It enables to obtain

[1] Narendra KS, Parthasarathy K. Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks 1990;1:4–27. [2] Miller WT, Sutton RS, Werbos PJ. Neural networks for control.London: MIT Press; 1992. [3] Park JH, Huh SH, Kim SH, Seo SJ, Park GT. Direct adaptive controller for nonaffine nonlinear systems using self-structuring neural networks. IEEE Transactions on Neural Networks 2005;16:414–22. [4] Akpan VA, Hassapis GD. Nonlinear model identification and adaptive model predictive control using neural networks. ISA Transactions 2011;50:177–94. [5] Li HX, Deng H. An approximate internal model-based neural control for unknown nonlinear discrete processes. IEEE Transactions on Neural Networks 2006;17:659–70. [6] Ge SS, Wang C. Adaptive neural control of uncertain MIMO nonlinear systems. IEEE Transactions on Neural Networks 2004;15:674–92. [7] Denaï MA, Palis F, Zeghbib A. Modeling and control of non-linear systems using soft computing techniques. Applied Soft Computing 2007;7:728–38. [8] Savran A. Multifeedback-layer neural network. IEEE Transactions on Neural Networks 2007;18:373–84. [9] Singh HP, Sukavanam N. Simulation and stability analysis of neural network based control scheme for switched linear systems. ISA Transactions 2012;51:105–10. [10] Zhang BT, Pi YG. Enhanced robust fractional order proportional-plus-integral controller based on neural network for velocity control of permanent magnet synchronous motor. ISA Transactions 2013;52:510–6. [11] Cheng L, Hou ZG, Tan M. Adaptive neural network tracking control for manipulators with uncertain kinematics, dynamics and actuator model. Automatica 2009;45:2312–8. [12] Poultangari I, Shahnazi R, Sheikhan M. RBF neural network based PI pitch controller for a class of 5-MW wind turbines using particle swarm optimization algorithm. ISA Transactions 2012;51:641–8. [13] Chen WS. Adaptive NN control for discrete-time pure-feedback systems with unknown control direction under amplitude and rate actuator constraints. ISA Transactions 2009;48:304–11. [14] Peng J, Dubay R. Identification and adaptive neural network control of a DC motor system with dead-zone characteristics. ISA Transactions 2011;50:588–98. [15] Kotta U, Chowdhury FN, Nomm S. On realizability of neural networks-based input– output models in the classical state-space form. Automatica 2006;42:1211–6. [16] Jin L, Nikiforuk PN, Gupta MM. Approximation of discrete-time state-space trajectories using dynamic recurrent neural networks. IEEE Transactions on Automatic Control 1995;40:1266–70. [17] Zamarreno JM, Vega P. State space neural network, properties and application. Neural Networks 1998;11:1099–112. [18] Cao Y, Ding R, Zhang QJ. State-space dynamic neural network technique for high-speed IC applications: modeling and stability analysis. IEEE Transactions on Microwave Theory and Techniques 2006;54:2398–409. [19] Zamarreno JM, Vega P. Neural predictive control. Application to a highly non-linear system. Engineering Applications of Artificial Intelligence 1999;12:149–58. [20] Gan C, Danai K. Model-based recurrent neural network for modeling nonlinear dynamic systems. IEEE Transactions on Systems, Man, and Cybernetics— Part B: Cybernetics 2000;30:344–51. [21] Canelon JI, Shieh LS, Karayiannis NB. A new approach for neural control of nonlinear discrete dynamic systems. Information Sciences 2005;174:177–96. [22] Scales LE. Introduction to non-linear optimization.Inc. New York: SpringerVerlag; 115–8. [23] Ge SS, Zhang J, Lee TH. Adaptive neural network control for a class of MIMO nonlinear systems with disturbances in discrete-time. IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics 2004;34:1630–45.