A new method for adaptation of the dynamic transition Matrix

A New Methodfor Adaptationof the Dynamic TransitionMatrix by A.M.MORSHEDI Department of Chemical Engineering Pahlavi University, Shiraz, Iran and R.H.LUECKE Department of Chemical Engineering University of Missouri, Columbia, Missouri

A new method is proposed for adaptation of the plant transition matrix of sampled data systems. Correction matrices are based on the difference between predicted and observed values of the state vector. They are constructed adaptively in a way analogous to the Davidon method for finding the Jacobians in nonlinear equation solving. Two numerical examples are presented. Convergence was obtained on a simulation of a simple two-dimensional process as well as on a six-dimensional process. Two adaptive algorithms are presented, one that is efficient for large initial error and one that converges more slowly but gives smooth behavior when initial error is small.

ABSTRACT:

Notation A B C, D G H h L N m 4 t T U

V W k Y Z a4 P 8

n X n state transition matrix n x r control transition matrix specific heat partial correction matrix correction matrix vapor holdup liquid holdup liquid flow rate matrix norm r X 1 control vector transferred heat time sample period catenated state and control vectors vapor flow rate water flow rate n X 1 state vector concentration in liquid concentration in vapor an arbitrary vector volatility constants density temperature

A. M. Morshedi and R. H. Luecke I. Introduction

There has always been a definite need for engineers to obtain good mathematical models for plants which they desire to control. Advancements in digital control hardware have, in recent years, enabled consideration of more complex problems and techniques than ever before. Many types of calculations with a high degree of sophistication are no longer limited by control system capability. The problem of process control has several phases to its solution. The first phase involves finding, either experimentally or theoretically, a dynamic model which represents the process to be controlled. The second phase involves designing a control policy which when used on the chosen model will give satisfactory performance. In addition, there may be a third phase involving adjustment of the model and the control policy on the basis of the actual control performance. In this work, emphasis is on development of a new technique for this third phase. Linear discrete process models are chosen and control policies are designed in terms of the assumed parameters in the models. Sequential adjustments are then made to parameters in the process model to serve as a basis for improving control performance. The adjustments are found using a modification of the techniques described by Barnes (1) and Broyden (2). In those papers the gradient matrix for the search for solutions to non-linear algebraic equations were constructed on the basis of the search results. Convergence to the gradient matrix was proved to within a linear approximation of the system in the operating region. The problems of dynamic estimation and identification are not new to the literature. One of the earliest papers was that of Yule (3) on a model for sun spots. There has been since then a large number of papers regarding estimation and identification spread over a number of statistical, econometrics and engineering journals. Work on the dynamical behavior of economic models by econometricians and statisticians has lead to incidental cross-correlation with engineering (4). A bibliography by Wold (5) covers papers up to 1965 in the field of identification. There have been other surveys in recent years (6-8) and a comprehensive paper by Astrom and Eykhoff (9) which covers identification of deterministic as well as stochastic systems. Many techniques have been proposed for automatic model adjustment. Narenda and McBride (10)and Donalson and Leondes (11) describe a class of methods based on least squares analysis of an error signal between process and model outputs when both are subjected to the same input. Roberts (12) discusses conditions which guarantee stability in linear systems. Butler and Bohn (13) extend the least squares method to non-linear systems. The method described in this work, on the other hand, is not a least squares technique. The actual value of the model-system error is utilized in the development of the transition matrix. In its present form, the method is directly applicable only to processes which are completely state observable and which 2

Journal of The

Franklin Institute

A New Method for Adaptation of the Dynamic

Transition Matrix

have only modest levels of noise or error. Further work is in progress that would extend applications to noisy processes and to processes which are merely observable in the usual technical sense. Although the examples are implemented with time optimal control policies, other control policies may also be used without change to the algorithm. II. Description of Adaptation Process Consider a linear discrete multivariable state being observable:

system with each component

xk+r=A(T)&+B(T)mk.

of the (1)

The key dynamic properties of this system are the transition matrices A and B. These may be estimated from a theoretical derivation based on the process system or may be the result of direct experimental measurements. In either case, in a real process some inaccuracies would be expected both from imprecise measurements and from parameter interactions not explored in the measurement process or not represented in the model. Other model inaccuracies could occur from parameters that have been assumed constant but which actually change slowly (compared to the rate of change in the state). Thus, a second equation may be generated: x:+r=A*(T)xk+B*(T)mk

(2)

where the asterisks (*) denote estimated or assumed properties. The difference between these two equations gives the level of error in the estimate for the (k + l)st state due to errors in the model parameters: L?LXk+l=

A&Xk

+ A&mk

(3)

where the A represents the estimated values (Eq. (2)) subtracted from the true values (Eq. (1)). To simplify subsequent notation, Eq. (3) is rearranged into an augmented form: hXk+l =

[AAkL\Bk]

(4)

or A&+, = Gkuk

(5)

where the definitions for the error matrix C& and catenated vector uk are implied by Eq. (4). We assume that the vector xk is available as current process information and that the control vector mk is also a known quantity or can be measured. In this work, an unconstrained minimum time control law is computed for specification of 1.4~.The actual control policy is, of course, non-optimal since the true value of the transition matrix is not known. Any control policy can be used since only the value of the control vector is use‘d in the identification procedure. Vol. 304. No. I. July

1977

3

A. M. Morshedi and R. H. Luecke The new state vector estimate x*k+l can be predicted from the present state xk and the control law mk Using the estimated transition matriCeS. Application of the control vector mk through the subsequent sample period will carry the process to a new state xk+i that differs from the predicted state x:+~ as a function of the transition matrix error Gk as shown by Eq. (5). The problem is, then, to compute the transition matrix error Gk knowing the vectors hxk+i and uk. Note that the matrix Gk has n X (n + T) unknown elements while there are at most n independent equations. Thus, only partial adjustment of the matrix Gk can be accomplished with information from one sample interval. An updating technique can be based on the quasi-Newton procedures as used by Barnes (1)and Broyden (2) for non-linear equation solving. Consider the matrix D _A%+&k-

(6) z&k

where zk is an arbitrary vector restricted (6) does not vanish. For the vector uk DkUk

only so that the denominator

= A&_+&-Uk = AXk+l. z&k

in Eq.

(7)

Thus, the matrix Dk transforms the vector t.& into Axk+r as does the correction matrix Gk in Eq. (5). These matrices are not necessarily equal but an algorithm is suggested where sequential computations of Dk are added to the current estimate matrix [A*, ES*]. Barnes and Broyden showed that for special cases of the arbitrary vectors &, this procedure leads to convergence to the true matrix [A, B]. Barnes showed convergence in n + r steps when zk is chosen to be orthogonal to the (n + r - 1) vectors uj used in previous steps. Broyden proved convergence with the selection of &. as

Broyden also indicated another choice for & convergence performance and was not considered

but it gave unsatisfactory in this work.

III. System Models for Numerical Examples Although rigorous mathematical proofs for convergence have been given for the Barnes and Broyden algorithms in non-linear equation solving, introduction of the control input alters this relationship in identification methods. Furthermore, it is of interest to examine the rate and paths of convergence for some typical process systems. In equation solving convergence of the state is of sole interest and values for the transition matrix are unimportant. In this work there is increased interest in the values of the transition matrix itself. Two process system examples that have been previously used by others to illustrate discrete optimal control policies were used here to examine the adaptive control algorithms. Journal of The Franklin Institute

A New Method for Adaptation of the Dynamic Transition Matrix

In the two tank water heating system shown in Fig. 1, as presented by Koppel (14), the objective is to control the temperature of the fluid leaving the second tank & by manipulation of the heat input to the first tank. Both tanks are perfectly mixed, have constant volume and all transport delays are negligible. The differential equations describing the system are

PVI d& wdr+e,=eo+L

(9)

WC,

Wdt+e2=e, pv2

de2

where values were chosen such that PVl -= W

1

PV2 7=

l/2

wc,=2. These lead to a discrete system as given by Eq. (1) where

A second numerical example, a six stage absorber as described by Lapidus and Luus (15), is diagrammed in Fig. 2. Perfect mixing with complete vaporliquid equilibrium is assumed for each stage and the holdup of inert liquid and vapor in each stage is assumed to be constant. Material balances lead to the equation

--__-----ti-w

IIdyi+ ~-L(Xj-I-Xj)+V(Yj+1-Yj) dXi_

(11)

h

dt

for j = 1,2,. . . ,6. To

T,

--I

T

r,

r,

I q I

FIG. 1. Two tank water heating system.

Vol.

304, No. I. July 1977

A. M. Morshedi and R. H. Luecke X0.L

Yi

I

t I

H h

FIG. 2. Gas adsorber-six Vapor-liquid

equilibrium

stages.

of the form Yi=CtXj+p

(12)

allows consolidation into a six-dimensional state vector equation of the form of Eq. (1) with a two dimensional control variable. Numerical values of the transition matrices for these systems are given in Tables I and II.

IV. Numerical Results Evaluation of convergence the root mean square value matrix A would be

rates is presented of the elements.

using a matrix norm which is That is, the norm of an n x r

In what follows, starting estimates of the transition matrices A* and B* were formed by adding random numbers to the elements of the true values. The root mean square of the added random numbers is given as a multiple of the norm N of the catenated error free matrix [A, B]. Five levels of error-lN, 5N, lON, 30N and 40N were considered for each model and for each convergence method. Typical performance characteristics are summarized in what follows.

6

Journal of The

Franklin

Institute

A New Method for Adaptation of the Dynamic Transition Matrix TARLEI

Actual Transition Matrix for Two Tank Water Heating System A

B

0.2500

0.2500

0.1250

0.0000

0.5000

0.5000

TABLE

II

Actual Transition Matrix for Six Plate Gas Absorber System A

0.3653 0.1866 0.0489 0.0086 0.0011 0.0001

0.2195 0.4229 0.1967 0.0527 0.0087 0.0011

0.0677 0.23 14 0.4245 0.1969 0.0502 0.0086

B 0.0140 0.0695 0.23 16 0.4245 0.1967 0.0489

0.0022 0.0143 0.0695 0.2314 0.4229 0.1866

0.0002 0.0022 0.0140 0.0677 0.2195 0.3653

0.3308 0.07253 0.0116 0.0014 0.0001 0.0000

0.0000 0.0003 0.0028 0.0189 0.1006 0.3891

V. The Barnes Method The response of the Barnes method at low (IN) and medium (10N) error levels in the two dimensional systems are shown in Figs. 3 and 4. Convergence

of the transition matrix was achieved after four iterations; that is, three time intervals. This is consistent with the convergence proof for the Barnes method which shows that convergence must be attained after at most (n + r+ 1) iterations. That proof does not place limits on intermediate error values, however, and note that substantial increases in the norm occurred before final convergence was attained. Convergence of the state and control vectors occurred several intervals after convergence of the transition matrix because of inaccuracies in the time optimal path computed using the estimated transition matrices. Equivalent results were obtained in the six dimensional case shown for low and medium error levels in Figs. 5 and 6. The transition matrix converged after nine (equal to n + r + 1) iterations; the state vector converged three or four iterations thereafter. Even at the very high error level (40N) similar convergence was obtained. In that case even higher intermediate error levels in the transition matrix were generated and resulted in control vectors that in practice might be impossible to realize. VZ. The Broyden Method In the proof for the Broyden method, it is shown that there must be a monotonic decrease in error, but that full convergence is not guaranteed for Vol.304.No. 1.July1977

7

A. M. Morshedi and R. H. Luecke 2.0

;!!!I \J i 4

E

c”

\

1.5

9,

6

:: L

Control

vector

State

vector

\

E

L/

A

+--x

\

\

/Error

in transltlon

Actual

transition

matrix matrix \

Time

FIG. 3. Performance

of Barnes method: two-dimensional

system; error level = 1N.

any given number of iterations. Thus, it is seen in Figs. 7-10 that, although the error reduction is steady, convergence of the transition matrix can be very slow. As shown in Fig. 7, at the low error level (1N) with the two-dimensional system, convergence of the state and control vectors occurred after six time intervals. Since error was still present in transition matrix at this time, the process was reinitiated with another state perturbation. Little additional convergence of the transition matrix occurred although good behavior of the state and control vectors was attained. A third perturbation produced like results. At the medium error level on the two-dimensional model (lON, Fig. 8), slow convergence of the transition matrix again occurred. In fact, satisfactory convergence of state was not obtained because of error in the transition matrix. The six-dimensional case showed similar behavior at low error levels (lN, Fig. 9); low level error in the transition matrix persisted even though the process was reinitiated. Note, however, that rapid convergence of the state and control vectors was obtained. At high error levels (lON, Fig. lo), satisfactory convergence of state and control vectors was not obtained. In fact, there is a disturbingly divergent look about the control vector.

VZZ.Some Difficulties The error vector Ax~+~ in Eq. (6) must be error free in order to generate the true correction matrix Dk. In a real process, observation error must be 8

Journal

of The Franklin

Institute

A New Method for Adaptation of the Dynamic Transition Matrix

6.0 t

Error

m

State

transition

matrix

vector

Control

vector

Time

FIG. 4. Performance

of Barnes method: two-dimensional

Control

system; error level = 1ON.

vector

State ror

vector in transitton

matrix

TI me

FIG. 5. Performance

Vol. 304, No. 1. July 1977

of Barnes method: six-dimensional

system; error level = IN.

9

A. M. Morshedi and R. H. Luecke

4

:” I 1I

20 -

15 -

E 2 z

Control

I I

1 J

’ r’

I 1

’ I I

I ;

vector

rror

‘O

State

In

transition

matrix

vector

5-

0 Time

FIG.

6. Performance

of Barnes

method:

six-dimensional

error level = 1ON.

system;

2.5

tronsltion

0

I

23456123412345 Time

FIG.

10

7. Performance

of Broyden

method:

two-dimensional

system;

error level = 1 N. Journal

of The Franklin

institute

A New Method for Adaptation of the Dynamic Transition Matrix

Control

vector

30 State

vector

E k z

Error

0

IO

in

20

tronsltlon

matrix

30

Time

FIG. 8. Performance

of Broyden

I

\

method:

Control

two-dimensional

2

4

6

error level = 1ON.

i

vector

6

0

system;

8

i

\

IO

2

4

6

Time

FIG. vol. 304,

9. Performance No.

1. July

,977

of Broyden

method:

six-dimensional

system;

error level = 1N.

11

A. M. Morshedi and R. H. Lwecke 12

A

Control

vector

9

E

State

vector

6

2 2

IO

20

30

Time

FIG. 10. Performance

of Broyden method: six-dimensional

system; error level = 1ON.

expected in this value. Work is presently in progress which employs the methods of stochastic approximation so that convergence with probability one is obtained for the matrix [A*, B*]. That is, the current estimate is updated using the equation [&+I,

B:+,] = [A:, B*k]+ akDk

(14)

where ak is a weighting coefficient satisfying certain conditions that promote stochastic convergence. Some problems have been encountered but the approach seems to be feasible. Another important problem occurs when every component of the state vector is not directly observable. For technically observable systems, the state vector can, in principle, be recovered by repeated differentiation of a single component of the state. However, this operation accentuates noise and error to such a degree as to be essentially impractical for recovery of more than one or two state components. For most large dimensional cases it will be necessary to use some form of a Kalman filter to generate the entire state vector and even so large uncertainties can be expected in some of these generated values. Fortunately moderate error in the unmeasured state variables can be tolerated in the identification procedure. To see this, partition the state vector into 12

Journal

of The Franklin

Institute

A New Method for Adaptation of the Dynamic Transition Matrix measured

and unmeasured

parts:

x, = Xr,

II

(15)

X;

where the components of XL are measured and those of XL are not. Equation (1) can then be conformably partitioned

(16) For most control problems, interest is concentrated on the first part of Eq. (16) that describes the dynamic behavior of those variables that are directly observed. Thus matrices AlI, A12, and B1 used in predicting future values of these variables, are of greatest interest. Similarly partitioning the correction matrix in Eq. (16) gives

DII

&2

013

DZI

D22

023

h,,Z, I

AxZ+1Z,l

AXLIZ

=

&xi

+

z2ri

+

(17)

Z3mk

where Z1, Z2 and Z3 are suitable partitions of Zk. Note that Dll, D12 and D13, the corrections to All, Al2 and Br, involve the unmeasured state variables X; (and not the error in its prediction) only through the inner product in the denominator. Thus adaptation of the transition matrices for the observed variables has a low dependence on the values of the derived state variables. Estimates for transition matrices for unmeasured state variables are difficult to obtain using this procedure but in many applications the initial estimates from these unmeasured values is sufficiently accurate. Work on this aspect of the problem is continuing. VIII. Conclusions and Recommendations This general approach represents a simple method for transition matrix adaptation for systems that are completely state observable and which have relatively low noise in the observations. The Barnes method for adaptation of the transition matrix converges rapidly even for high order systems and at high levels of initial error. It has the disadvantage of generating intermediate values possibly more erroneous than the starting values. The Broyden method on the other hand, guarantees a decrease in error for each successive estimate of the transition matrix. However, the rate of convergence may be slow and high noise levels in high dimensional systems, may lead to generation of large control vectors. A combination of the two approaches seems desirable. Periodic use of the Barnes method would correct large errors and keep the system error low. Use of the Broyden method between these corrections would allow response to slow process changes and adaptation to non-linearities. Both methods are simple to apply compared to other identification techniques. A matrix inversion for both methods was used, but only for computing Vol.

304, No. 1, July 1977

13

A. M. Morshedi and R. H. Luecke the control policy. The Barnes method requires an orthogonalization procedure for the Z vector. Other than that, the corrections are prepared by simple multiplication and addition of matrices. A problem occurs when some components of the state vector are not directly observable. For technically observable systems, the state vector can, in effect, be recovered by repeated differentiation of a single component of the state. However, this operation accentuates noise and error to such a degree as to be essentially impractical for recovery of more than one or two state components. In such cases the effect of the unknown components on measured values can be shown to be small but it would be necessary in such cases to base control upon the observed components only.

Acknowledgement Use of the technical facilities such as computer and drafting services at the Engineering School of Pahlavi University is greatly appreciated.

References (1) J. G. P. Barnes, “An Algorithm for Solving Nonlinear Equations Based on The Computer J., Vol. 8, pp. 66-72, 1965. Secant Method”, (2) C. G. Broyden, “A Class of Methods for Solving Nonlinear Simultaneous Equations”, Math. Comp., Vol. 19, 577-593, 1965. (3) G. U. Yule, “On the Method of Investigating Periodicities in Disturbed Series with Special Reference to Walfer’s Sun Spat Numbers”, Phil. Trans. Roy. Sot., Lind., Vol. A226, 1927. (4) F. M. Fisher, “The Identification Problem in Econometrics”, McGraw-Hill, New York, 1965. (5) H. Wold, “Analysis of Stationary Time Series”, Uppsala, Sweden, Almquist and Wissel, 1938. (6) M. Cuenod and A. Sage, IFAC Symp. Identification in Autom. Control Systems, Prague, survey paper; also in Automatica, Vol. 4, pp. 235-269, 1968. (7) P. Eykhoff, IFAC Symp. Identification in Autom. Control Systems, Prague, survey paper; also in Automatica, Vol. 4, pp. 205-233, 1968. (8) P. Eykhoff, P. M. Van Der Grinter, H. Kwakernaak and B. P. Veltman, Proc. Third IFAC Cong., London, survey paper, 1966. (9) K. J. Astrom and P. Eykhoff, survey paper, Automatica, Vol. 7, pp. 123-162, 1971. “Multiparameter Self-Optimizing Systems Using (10) K. Narendra and L. McBride, Correction Techniques”, IEEE Trans. on Automatic Control. Vol. AC-9, 88, pp. 241-252, 1963. Parameter Tracking (11)D. Donalson and C. Leondes, “A Method Referenced IEEE Trans. on Application and Technique for Adaptive Control Systems”, Industry, Vol. 82, pp. 241-252, 1963. (12) J. Roberts, “A Method of Optimizing Adjustable Parameters in a Control System”, Proc. IEEE Lond., Pt. B, Vol. 109, pp. 519-528, 1962. Identification Technique for a ClaSS (13) R. E. Butler and E. V. Bohn, “An Automatic IEEE Trans. on Automatic Control, Vol. AC-11, pp. of Nonlinear Systems”, 292-296, 1966. (14) L. B. Koppel, “Introduction to Control Theory”, Prentice-Hall, Englewood Cliffs, N.J., 1968. (15) L. Lapidus and R. Luus, “Optimal Control of Engineering Processes”, Blaisdell, London, 1967.

14

Journal ofThe

Franklin Institute