Identification and Tracking of Uncertain Systems by Multiple Objective Optimization

Copyright © IFAC Control Applications of Optimization HaifJ. Israel. 1995

Identification and Tracking of Uncertain Systems by Multiple Objective Optimization Dan Rusnak RAFAEL (88), P.O.Box 2250, Haifa 31021, Israel.

Abstract: The problem of optimal simultaneous identification and tracking of stochastic uncertain linear systems on finite time interval is formulated and solved. The tracking objective is minimization of a quadratic criterion. The identification criterion is maximization of the Fisher information matrix. The combined objective is the simultaneous minimization of the quadratic criterion and maximization of the information matrix. The inherent conflict between tracking and identification, as they are competing for the only available resource, namely the input to the plant, is posed and solved. Examples of application of the approach to a second order system are presented Keywords: Multiple-criterion optimization, Uncertain Linear Systems, Adaptive Systems. 1. TIIE CONFLICT BElWEEN IDENTIFICATION

control performance.

ANDCON1ROL 2. MOOP APPROACH TO CON1ROL OF UNCERTAIN SYSTEMS

In has been recognized that estimation-identification and tracking-control are conflicting (Guez, et a/., 1992; Polderman, 1989; Koussoulass and Dimitriadis, 1989; Rusnak, et al., 1992a,b). That is, the objective of the control is to minimize the tracking error which is usually the input to the plant. However, if the input to the plant is zero, or if it is constant, one can not explore the plant and identify its parameters. In other words, only after realizing this basic conflict of competition, on the only available resource, the input to the plant between the two objectives, the control and identification, one can formulate a problem that will give a framework and attempt to solve it by a compromise.

The observations made in section 1 immediately suggest the employment of Multiple Objective Optimization Theory (MOOP) (Koussoulass and Dimitriadis, 1989; Kbargonekar and Rotea, 1991) to replace the heuristics (e.g. dual control, dither, persistent excitation, etc.), used in previous treatments of control of uncertain systems. The use of the Multiple Objective Optimization theory advises how to combine several conflicting objectives. The formulation of the problem and the technique of solution immediately provide the costprice paid to perform each of the objectives, thus enabling the required trade-off between these criteria. The theory of multiple objective optimization approach to adaptive control for general nonlinear system is presented in (Guez, et al., 1992). Application of this methodology and a suboptimal approach to discrete linear time-invariant system is in (Rusnak, et al., 1992a). Application of a suboptimal approach to on-line identification and control of linearized aircraft dynamics can be found in (Rusnak, et al., 1992b). Preliminary results with this approach were presented in (Rusnak, et al., 1993a,b). In the present paper we apply the general approach to stochastic linear time-invariant system.

In the current literature several approaches are proposed for providing on line estimation, such as application of dither, dual control (Feldbaum, 1965), switching of controllers (Feldbaum, 1965; Gerencser, 1990). These references lack a discussion on how much is "paid" for the estimation, in terms of tracking error and no rigorous rationalization was supplied for the application of these heuristics. Recently, the issue of joint design of identification and control is emerging (Guez, et al., 1992; Rusnak, 1992b; Gevers, 1993). It is realized that the only justification for identification is achieving better 5

3. MULTIPLE OBJECITVE OPTIMIZATIONREVIEW

Theorem 3.1: Suppose that X is a normed linear space and that each component of f:= (it ,h,···.!s) is a convex function on X. Let A be the simplex, i.e.

In this appendix a short review of the multiple objective optimization theory - MOOP is presented.

s

3.1 Multiple Objective Optimization Problem

s

A = { A E R. I Ai~, LAi = I}, i=1

In this section we present the definitions and basiC concepts for the Multi-Objective Optimization problem. We follow closely (Koussoulass and Dimitriadis, 1989), see also (Abmad and Guez, 1992).

and for each A E A consider the following scalarvalued optimization problem: inf{ AT f(x) I x EX}.

In single objective or scalar optimization we are given a functional f(x), XE X, X eRn

(3.1.1)

f(x).

(3.1.2)

Remarks: 1) From this theorem it follows that Pare to optimal points always solve the optimization problem posed in section 3.1. When for each A E A, the infunum in (3.2.2) is either achieved at a unique

Consider now the case where we are given a (fmite) set of functionals fi (x), i=I,2, ... ,N,

(3.2.2)

Suppose that XOE X is Pareto optimal with respect to the vector valued criterion f(x), then there exists A E A such that xO is a solution to (3.2.2). Conversely, given A E A, if (3.2.2) has at most one solution xO then xO is Pare to optimal with respect to

to be minimized under constraints of the general form g(x) ::; O,h(x) = O.

(3.2.1)

(3.1.3)

0

x E X (or it is not achieved at all) one gets an exact parametrization of all the Pareto optimal set. 2) Note that the two standing assumptions of Theorem 3.1 are the linear structure of the space X and the convexity of the individual functionals f. 3) If the convexity condition of the individual functionals is not met then the first part of Theorem 3.1:" Suppose that XOE X is Pareto optimal with respect to the vector valued criterion f(x), then there exists A E A such that x O is a solution to (3.2.2)." is still correct, however, the second (converse) part is not. That is, if f are not convex then the infimum in (3.2.2) does not necessarily lead to a Pareto optimal solution. For the theoretical background see (po-Lung, 1985) and a practical example (Ahmad and Guez, 1992).

under the same constraints. The notion of optimality has to be now redefined. The most widely accepted multi-objective optimality criterion has been introduced by Pareto. Its main principle is that a solution can be called optimal if we cannot find another one that will improve the performance of one objective, without damaging at least one of the others. In technical terms, X*E X is called "Pareto optimal" if there is no x E X, x;tx*, such that fi (x)$Ji (x*) for all (i) with fi (x);tfi (x*) for at least one i. Except for trivial cases the number of Paretooptimal solutions is greater than one and usually infinite. The solutions are elements of the set of all Pareto-optimal solutions, which is also called, the non inferior set, non dominant, unimprovable, Npoint or non-dominated

4. PROBLEM STATEMENT

3.2 Solution Methods of MOOP Problems In this section we pose a control problem of stochastic linear uncertain time-invariant systems. We consider the nth order continuous stochastic linear time-invariant single-input single-output system

Several methods of solution of the Multi-Objective Optimization problem are discussed in (Koussoulass and Dimitriadis, 1989; Salukvadze, 1979; Lin, 1976) and many others. We concentrate on the conditions when the problem can be stated as a scalar optimization problem. The reason / rational for this is that we want to apply the well known existing standard optimal control design techniques to find the optimal control. Next, we state a scalarization result as presented in the following form in (Khargonekar, and Rotea, 1991). Let X denote an arbitrary nonempty set and let

xp(t) = ApXp(t) + bpup(t) + wpl(t),xP(to) = xpo, (4. la) yp(t) =cpXp(t) + wp2(t), where the input up(t) E L2[to,tl]; ApE Rnxn; xp, c T, bpE R nx 1; x(t)E R nX 1 is the state; and l yp(t)ER is the output; wpl(t)E Rnxl is the process

fi:

x~ JR+, i=1,2, .., s be s nonnegative functionals

driving noise; and wp2(t)E RI is the output measurement noise. The noises are mutually

defined on X.

6

~ndependent,

al., 1993dl for state space system model, that the linear time invariant uncertain plant (4.1) can be represented in the following form

zero mean, white stochastic processes,

I.e.

and VpiE R nXn , Vp2E R 1 are given. The initial state xpo ~s a random vector, with given mean Xpo and covanance Qpo. We assume that the system is observa~l~ and without loss of generality that (Ap , bp ,cp) IS m the observer canonical form. That is

Ap=

[

~:~ ~ . -a n-l 0 -an 0

1

oo

Oh(t)

o

&(t)

o

o o

I

Oh(t)

o

ob(t)

wpl(t) -Ohw p 2(t)

~ l' ~ l' o

o

wh(t)

bp =[ hr.-I bn

(4.lb)

(5.1)

cp = Co =[1 00 .. 01. The unknown parameters are 9p = [-al -a2 ... -an bl b2 ... bnl T E R 2n and 9p is a random vector with mean 9p and covariance

:rep.

yp(t) = [co

0 0]

I The controlled variable, zp(t)E R , can be expressed as

oh(t)

+ W p 2(t),

ob(t)

(4.lc) where dpTERnxl is known.

Zp(t) = [d p

We wish to fmd the functional

oh(t) &(t)

where (Ao'c o ) is the Brunovsky canonical fonn; hpi=-ai, E[hpl=ho' E[bp]=bo , Oh=hp-ho and Ob=hpbo ' and wh(t), wb(t) are noises that represent the drift of the parameters. This representation is called the State and Parameters Observability Canonical Form (SPOF), that is written as

such that the criterion

I T Jc- 2E[(Zp(tl)-ZD(:tI» G(Zp(tl)-zo(tl»

0 0]

(4.3)

tl

+ j(Zp(t)_zo(t»TOc(t)(Zp(t)-Zo(t»+up(t)TR(t)Up(t)]dtl to

Xp(t) = Ap(t)Xp(t) + bp up(t) + wpl(t), XP(to) = XI><>' (5.2) yp(t) = cp Xp(t) + W p 2(t). zp(t) = dp Xp(t).

is minimized subject to the differential equation constraints (4.1, 4.2), where ZD(t) is the desired trajectory, fo~ given tl, and such that Up(t)E Up, where Up IS the admissible input set. The expectation is taken with respect to the stochastic processes, initial conditions and the vector of parameters.

6. SIMULTANEOUS OPTIMAL STATE ESTIMATION AND PARAMETERS IDENTIFICAnON The states and parameters observability canonical form immediately suggests the use of the well known theory of observers for linear time-variant syste?,s. The optimal estimator of the augmented s~te IS presented in (Kwakemaak, and Sivan, 1972). It IS essentially the same solution for ARMA system model as rederived in (peterka, 1981; Peterka, 1986; Havlena, 1993). The solution is

5. STATE AND PARAMETERS OBSERVERIDENTIFIER It has been shown in (peterka, 1981; Peterka, 1986, Havlena, 1993) for ARMA system model, and in (Rusnak, et al., 1992c; Rusnak, I. 1993c; Rusnak, et 7

3) It is proved in (Rusnak, 1993c) that uniform complete observability of (Ap(t),cp) is almost always equivalent to persistency of excitation.

7. OPTIMAL INPUTS FOR IDENTIFICATION In this section we consider design of the input such that the observability condition is taken care off with respect to an optimality objective. The optimal input is based on the maximization of the Fisher information matrix. This section follows closely the approach by (Mehra, 1974a,b).

This is the optimal estimator, by what we mean that given {yp(t), up(t), ~t~t} this algorithm derives the best mean square error estimate of the augmented state. These equations are easily solved since up to time t, Ap(t), bp and cp are known and the integration goes forward. The issue of how the selection of the input to the system, up(t), affects the quality of the estimation is dealt with in (Rusnak, et al. , 1992c;Rusnak, 1993c). For example the quality that we may look is the convergence of the estimation error, namely, asymptotic, exponential or in Lyapunov sense convergence, the magnitude of the estimation error covariance, etc ..

We deal with the linear single-input single-output stochastic system (4.1). The problem of finding the "optimal" input for identification, as formulated in (Mehra, 1974a), is to select the input {u(t), 0 ~ t ~ t I} to maximize a suitable norm of the Fisher Information matrix, M, subject to a suitable constraint. The Fisher Information matrix M for the unknown parameters e p is shown to be (Nahi and Wallis, 1969)

The state observer and parameters identifier that is discussed in this section is the optimal estimator. However, it is emphasized here that this optimality is with respect to a very specific experiment. This experiment is that the input driving noise and the input and output measurement noises are white stochastic processes, and the true parameters of the system in the observer canonical form are random variables as well. Namely, the optimal state observer and parameters identifier presented, gives the smallest second order statistics on the average when the average is taken with respect to all systems within the distribution of the parameters, all initial conditions and all noises. This means that when dealing with a specific system which has constant parameters, although unknown, this approach does not necessarily lead to better performance with respect to other identification algorithms.

tl

f

M = E{

V eYpT V;2 VeYp dt }

10

(7. la)

.

where

xp(t) = Ap xp(t) + bpup(t) (7.1b) + Kp(t)[ yp(t) - cp xp(t)], xp(1o) = xpo, yp(t) = cp xp(t) (7.1c)

T

Kp(t) = Qp(t)clt Vp2

-I

(7.1d)

Qp(t) = Qp(t)Ap +Ap9J,(t)

(7.1e) 1 + Vp l-Qp(t)cpTVp f cpQp(t), Qp(1o) = Qpo.

where VeYp is the gradient of the estimate of yp(t) with respect to the unknown parameters, Op, Veyp = dYp ayp lx2n . l' 2' ...... ]ER . To denve a scalar measure

The following presents a result on the performance of the optimal estimator-identifier.

[ae ae

Theorem 6.1: If the input to the plant up(t) is such that (Ap(t),cp) is uniformly completely observable T 1/2 and Vp l is such that (Ap (t),V ) is uniformly pI completely controllable then the optimal observer (5.13) is exponentially stable, and the minimal mean square estimation error is Qp(t). Proof: Direct outcome of (Kwakemaak and Sivan, 1972; theorem 4.10). Q.E.D.

of the Fisher information matrix there are several matrix norms that can be used (Mebra, 1974a). In this work we use the weighted trace norm: Je

=trace(SM)

(7.2)

where S>O is a weighting matrix. We further assume, for simplicity, that S is diagonal, so that the scalar objective, called the estimation objective, can be written as

Remarks: 1) The result in theorem 5.1 means that the dynamic equation

tl

Je = E{ ~(t) = [ Ap(t) - Kp(t) cp] ~(t), ~(to) = ~,

J

VeYp W VeypT dt }

(7.3)

10

(6.2)

-1 where S is diagonal such that Wii = V p2 Sii,

is exponentially stable. 2) The results in theorem 6.1 are sufficient conditions.

i=I,2, ... ,2n. Notice that J e is not the Fisher 8

information matrix.

"

axp Ohl

Remark: Although the vector of unknown quantities for, linear system (4.1) is [ xpoT hpT bpT]T. A simple analysis shows that the initial condition, xpo, is not controllable and therefore the sensitivity of the output with respect 10 the initial condition cannot be affected by the input. For this reason it is not considered in the following.

"

Ep(t) =

We would like to maximize J e subject to the dynamic constraint of the differential equations of the sensitivities. We proceed as in (Mebra, 1974a), and we have

axp Oh n

" axp abl

6n2x1

E R

,

"

a xp abn (7.6)

However, this is a complicated nonlinear problem. To continue we use the states and parameters observability canonical form presented in section 5 and we have

Q..]

Cp {

Cp Q ... Q cp..... Q 2nx6n2 . . . . . ER Q Q ... cp

.

~ de

The system (Rp(t),C p) is not minimal. We do not deal here with the issue of minimallity of this realization. We denote the minimal realization the same (Rp(t),C p) and in the rest of this work when we write (Rp(t),C p) we mean the minimal realization unless otherwise specified.

(7.5) _ ~a:Qp( _ - (Ap(t)-Kp(t)cp) ae' de to) - 0,

where Ap(t),cp are defined in (4.1,4.2), Kp(t) is

ae"

. aX 3nx2n gIven by (5.1) and PE R .

The problem of maximization of the norm of the Fisher information matrix is not well defined without a further constraint For example, in (Mehra, 1974a) an input energy constraint is imposed. Another approach, which is used in this work, is that in addition to maximization of (7.3) to minimize the objective (4.4). Following the MOOP theory joint minimization of both objective is to minimize their weighted sum. Then one has to check if the solution is the optimal, i.e. belongs to the Pareto set (Koussoulass and Dirnitriadis, 1989).

This can be written as (Bonivento, 1973)

= Rp(t) :::p(t), Ve9pT = Cp :::p(t)

:::p(t)

:::p(to) =Epo .

where

Even if the problem of maximizing the Fisher information matrix would have been well defmed by introducing appropriate constraint, the formal solution is not causal. This is since in order to find the optimal input for identification we need the knowledge of the parameters. In absence of the true parameters the a priori estimate is used Mebra, 1974b; Goodwin, 1977). This may be viewed as a causal approximation, or application of the certainty equivalence principle to derive the causal 9

approximation. In spite of this non causality. this approach is adopted here as 1) the optimal solution of the optimal control of uncertain system is noncausal by itself (Rusnak. 1993c); 2) the addition of the additional objective. the Fisher Information matrix does not increases the "amount of non causality" in the problem; 3) the noncausal optimal solution of simultaneous minimization of the tracking objective (4.4) and maximization of a norm of the Fisher Information matrix will imply. as shown in the following chapters. a structure of a suboptimal causal solution. The control objective is given in (4.4) and the input design objective is given in (7.3). Here we formulate the combined problem. that is to minimize 1

T

Jc- 2E[(Zp(t1)-Zo(t1» G(Zp(t1)-zo(t 1» and w2(t) is defined in (4.1).

t1 + f[(Zp(t)-zo(t»TOc(t)(Zp(t)-zo(t)

(8.1)

This is the standard Linear Optimal regulation problem with incomplete and noisy measurements and uncorrelated state excitation and observation noises defined in (Kwakemaak and Sivan. 1972; chapter 5.3) whose solution is given in (Kwakemaak and Sivan. 1972; theorem 5.3). The solution is

to -VeYpWVeYpT +np(t) TR(t)np(t)]dt] which can be written

J=¥:{ (~(t})_~])(t1»Tr(~(t})-~tl»

(8.2)

up(t) =

~(t) = _R- 1(t)bTll(t).

t1 +

f[(~(t)-~o(t» T9cCt)(~(t)-~o(t»+np(t)TR(t)np(t)]dt}.

(8.5) -net) = ll(t)A(t)+A(t)Tn(t) 1 + dTecCt)d-ll(t)bR- (t)bT ll(t). il(t1) =dTr d. -L(t) = (A(t)-bR- 1(t)bT ll(t))TL (t)

10

It is minimized subject to the dynamic constraints (4.3.7.6) which can be written as 3(t) = A(t)3(t) + bup(t) +w1(t). yp(t) =c 3(t) + wp 2(t). ~(t)

~(t) ~(t) + R- 1(t)bT L(t).

T

T

+ d ecCt)~o(t).

3(10) =3 0 •

L(t1) = d ecCt1)~o(t1)'

where

(8.3)

= d 3(t).

o

1\

1\

3(t) = A(t) 3(t) + bup(t) where

1\

+ 'V(t)[ yp(t) - c 3(t) ].

(8.4)

~t{

Ap(t) 0

~t) J.{:J.~[

'V(t)=e(t)cTy2 -1 c

o

p

+ U1 - e(t)cTy2-1 c 6(t). 6(10) =9 0 . Remarks: 1) The matrix eeCt) is symmetric and not positive semidefmite. it is indeftnite. The following are sufficient condition so that (8.5) will have solution [Bryson and Ho. 1975. Ch. 6.3]: i) the convexity condition - R>O; ii) the normality condition - trivially satisfted in our case as we do not constrain the terminal state to have a prescribed value; ill) the condition that no conjugate points exists on the path - that is that the solution of the Riccati equation (8.5) is finite in the time interval 10515tl. The solution of (8.5) is derived by integrating backward. Thus this solution will be valid as long as it is bounded.

=[

.=.p(t)

': " _': " {X(1o) ]{X(1o)] -: _[X(t)] 3p(1o) 0 0

-(to)~o

• -0 -

(8.6)

9(t) = 9(t)A(t)T +A(t)9(t)

J.

i ~ 1~(t) ~(t) J. [0

1\_

3(10)=20.

•

10

9. CERTAINTY EQUIVALENCE BASED . CONTROL

2) The part of the control objective, Qc, tends to stabilize the system. The part of the identification, -V ~12 S, tends to destabilize the

The solution derived in the preceding section can not be implemented in real time. Some causal approximation is needed. The optimal solution suggests the kind of approximation required. The observer part is exact. Certainty Equivalence principle applies with respect to the state and parameters. We have to approximate the controller part. Therefore, we use here as a causal approximation, of the controller part, the controller of the simultaneous identification and tracking problem solution for deterministic systems that is solved with the best estimate of the plant and state at the relevant time instants. This is the same kind of approach as used in the design of optimal inputs for identification in (Mehra, 1974a,b). This approach has been presented as a pragmatic solution in (Rusnak, 1993a,b). The solution for deterministic systems is presented, for the completeness of this representation, in the appendix.

system. This complies with our intuition as application of only control policy creates stable system, and application of only identification policy creates, for infinite time interval, marginally stable system with at most 2n poles on the imaginary axis (Mehra, 1974b). 3) The optimal solution is not causal. This is since we need for solution the future inputs and outputs (up(t), yp(t), t :5 t :5 td, where t is the current time in equation (8.5). It means that it can not be realized in real time. 4) Only the control law is not causal. 5) The observer part is causal. 6) We see that the Certainty Equivalence Principle applies. The Certainty Equivalence Principle states (Bryson and Ho, 1975) that when the true state is not available for full state feedback then the estimated state derived by a Kalman Filter may be applied instead and the stability and performance is preserved (almost). We see here that with respect to linear uncertain systems the Certainty Equivalence Principle still applies with respect to the state and the parameters as well. 7) With respect to the formal noncausal solution the Separation Theorem applies (Bryson and Ho, 1975). The Separation Theorem states that the solution of the optimal controller and the optimal estimator are independent. However, this is true only "with" noncausality. Therefore, one may expect that when some suboptimal causal approximation will be applied the two designs will not be independent 8) Although the representation of the solution of the optimal control of stochastic uncertain systems has an important theoretical implications, it does not have a direct practical meaning. It means that, no matter, what so ever, we can not solve in real time the problem of optimal control of uncertain systems. Any solution is necessarily causal approximation, i.e. suboptimal. The noncausality is not surprising. In previous explicit solution of the optimal control of uncertain systems, by (Hijab, 1983), he derived in the context of the problem of this paper a causal infinite dimensional solution, (Hijab, 1983) defined the problem on a finite uncertainty set. Therefore, if the uncertainty set is not finite Hijab's solution is infinite dimensional. In our solution by the SPOC we replaced the causal infinite dimensional solution with finite dimensional (3n) but noncausal solution. This explains why there are so many control algorithms, adaptive control algorithms. Each of the them can be viewed as a "projection", from a different point of view, from the space of the noncausal optimal solution into the space of causal solutions.

We have:

~(t) = Ao+(ho+5b(t)co '

Gp(t) =bo+Ob(t),

(9.1)

2:(t) = ~(t) ~(t) + ~t)us(t),

~(lo) = ~o' (9.2)

~(t) ~p(t) ~(t)

E -

Q

Q..

Q..

bd(~p(t»

Q

Q

Q , 'G(t) =

Q el

bd(~p(t)

en

° ] [x&t)' (0]

~ [ In 0 0 0 .:.(t) = 0 0 0 0 I2n2 up(t)

(9.3)

=R- 1lip(t)TIIcei(t)2:(t) + R-l~t?Lcei(t), ti:5t
us(t) = -R-l~(t)TIlcei
-!\ei(t) =I\ei(t)~(tj)+~(tD T l\ei(t) + dTecd-l\ei(t)'G(ti)R-l~(ti)Tncei(t), n cei(tl) = dTr d,

-Lcei(t) = (~p(tD-lip(tDR-llip(ti?ncei(t»TLcei(t) T +dTecCo(t), Lcei(tl)= d e c Co(tl). The times ti= iAt, i=O,I,2,3, .. ,tt, are the update times at which the certainty equivalence optimal feedback, rrcei(t), and feedforward, Lcei(t), are recomputed. There are other options to synthesize 11

identification, the parameters converge is fast enough and thus when the triangular part arrives there is almost no transient due to unknown parameters. The respective figures (c) and (d) present the tracking error and the input to the plant. The tracking error is a measure of the quality of the control qualities of the algorithm. The input of the plant demonstrates the effort that is spent on control and identification.

suboptimal causal approximation. Such options are not dealt with in this paper. 10. EXAMPLE Here we present simulation results that demonstrate the ideas of this paper. The results are presented for the following models of the plant -1

Hll(S) = co(sI-AP>

H12(s) = co(sI-AP>

-1

H23(S) = co(sI-Ap)

H24(s) = co(sI-Ap)

-1

-1

s + 2 2s + 3

bp = s2 + bp = -

s + 2 s + 2s + 3

bp =

2

2

s

bp = -

s - 2 - 2s + 3

s - 2 s - 2s + 3 2

11. CONCLUSIONS

(11.1)

The problem of optimal simultaneous identification and control of stochastic uncertain linear systems on fmite time interval has been formulated and solved. The combined objective is the simultaneous minimization of the quadratic control criterion and maximization of the information matrix. The solution is explicit and non causal. It suggests the structure of a causal approximation and clearly states the tradeoff between the control and identification costs. An example demonstrates the performance of a causal control algorithm.

(11.2)

(11.3)

(11.4)

Transfer function H 11 is a stable minimum phase system. Transfer function HI2 is a stable minimum phase system with reversed sign. It will help to demonstrate that the presented algorithm does not need the knowledge of the DC or high frequency gain sign. Transfer function H23 is an unstable nonminimum phase system with two poles in the RHP. Transfer function H24 is an unstable nonminimum phase system with two poles in the RHP with reversed sign.

12. REFERENCES Ahmad, A. and Guez, A. (1992): Multiple Objective Optimization for Learning Control, Proc. SPIE, Orlando, Florida, April 1992. Bonivento, C. (1973): Structural Insensitivity versus Identifiability, IEEE Transactions on Automatic Control, AC-18, No. 2, pp. 190-193. Bryson, A.E. and Ho, Y.C. (1975): Applied Optimal Control, Hemisphere Publishing Corporation, Washington, D.C .. Feldbaum, A. A. (1965): Optimal Control Systems, Academic Press Inc., New York. Gerencser, L. (1990): Closed Loop Parameter Identifiability and Adaptive Control of a Linear Stochastic System, Systems & Control Letters, IS, pp. 411-416. Gevers, M. (1993):Towards a Joint Design of Identification and Control?, In: Essays on

The state estimator and parameters identifier is the presented in section 6 and the control algorithm is the presented in section 9. The desired trajectory is YD(t) = 0, YD(t) = 2t-2, YD(t) = 4-21.,

O::;t<1, l::;t<1.5, 1.5::;t<2.

Control: Perspectives in the theory and its Applications, (Trentelman, H.L. and Willems, J.C., (Ed.», chapter 5, Birkhauser Boston. Goodwin, G. C. and Payne, R. L. (1977): Dynamic System Identification: Experiment Design and Data Analysis, Academic Press, New York. Guez, A., Rusnak, I. and BarKana, I. (1992):

The results are derived for Qc
The examples has been derived for initial model of the plant H(s) =1/s 2 .

Multiple Objective Optimization Approach to Adaptive Learning Control, International Journal of Control, Vol. 56, No. 2, pp. 469-482. Havlena, V. (1993): Simultaneous Parameter Tracking and State Estimation in Linear Systems, Automatica 29, No. pp. 1041-1052, 1993. Hijab, O. (1983): The Adaptive LQG Problem - Part

Figures lUa, 11.2a, 11.3a and 11.4a present the desired and the actual trajectory, and the actual and estimated states of the plant. Figures lUb, 11.2b, 11.3b and 11.4b present the actual and the estimated parameters of the plant. We can see that the control (derived with wo=50) has good qualities for

Vol. AC-28, No. 2, pp. 171-178. Khargonekar, P. P. and Rotea, M. A. (1991): Multiple Objective Optimal Control of Linear Systems: The Quadratic Norm Case, IEEE Transactions on Automatic Control, Vol. 36, No. 1.

QO = diag(1, 1, 106,106 ,106 ,106), W(t) = wol4,Q::;t
1, IEEE Transactions on Automatic Control,

12

Control Conference, ACC 93, San Francisco. Rusnak, I. (1993c): Optimal Adaptive Control of Uncertain Stochastic Linear Systems , Ph.D. thesis, Drexel University. Rusnak, I., Guez, A. and BarKana, I. (1993d): State Observability and Parameters Identifiability of Stochastic Linear Systems, 1993 IEEE Regional Conference on Aerospace Control Systems, CACS 93, Thousand Oaks, Los Angeles. Salukvadze, M.E . (1979) : Vector-Valued Optimization Problems in Control Theory , Academic Press, Inc. New York, 1979.

Koussoulass, N. T. and Dimitriadis, E. K. (1989): Computational Techniques for Multicriteria Stochastic Optimization and Control, In: Control and Dynamic Systems, Advances in Theory and Applications, (Leondes C. T. (Ed», Vol. 30, part 3, Academic Press. Kwakemaak, H. and Sivan, R. (1972): Linear Optimal Control, New York: Wiley, 1972. Lin, J.G. (1976): Multiple Objective Problems: Pareto Optimal Solutions by Methods of Proper Constraints, IEEE Transctions on Automatic Control, Vol. AC-21, No. 5. Ljung, L. (1977): System Identification, PrenticeHall, Inc .. Mehra, R. K. (1974a): Optimal Inputs for Linear Systems Identification, IEEE Transactions on Automatic Control, Vol. AC-19, No. 3. Mehra, R. K. (1974b): Optimal Input Signals for Parameter Estimation in Dynamic Systems Survey and New Results. IEEE Transactions on Automatic Control, Vol. AC-19, No. 6. Nahi, N. E. and Wallis, D. E. Jr. (1969): Optimal Inputs for Parameter Estimation in Dynamic Systems with White Observation Noise, Joint Automatic Control Conference, Boulder, Colorado. Peterka, V. (1981): Bayesian Approach to System Identification, In: Trends and Progress in System Identification, (P. Eykhoff (Ed.», Pergamon Press, Oxford. Peterka, V. (1986): Control of Uncertain Processes: Applied Theory and Algorithms, Supplement to Kybernetica 22, No. 3-6. Poldennan, J. W. (1989): Adaptive Control and Identification: Conflict or Conflux, Stichting Mathematisch Centrum, Amsterdam , Netherlands. Po-Lung, Yu (1985): Multiple-Criteria Decision Making : Concepts, Techniques, and Extensions, Plenum Press, New York, 1985. Rusnak, I., Guez, A. and BarKana, I. (1992a) Multiple Objective Approach to Adaptive and Learning Control of Discrete Linear TimeInvariant Systems, 7-th Workshop on Adaptive and Learning Control, Yale University, May 2022. Rusnak, I., Steinberg, M. , Guez, A. and BarKana, I. (1992b): On-Line Identification and Control of Linearized Aircraft Dynamics, IEEE on Aerospace and Electronic Systems Magazine, Vol. 7, No. 7, pp. 56-60. Rusnak, I., Guez , A. and BarKana, I. (1992c): Necessary and Sufficient Conditions on the Observability and Identifiability of Linear Systems, ACC 1992, American Control Conference, Chicago, Illinois. Rusnak, I., Guez, A. and BarKana, I. (1993a): Control of Stochastic Linear Uncertain Systems by Multiple Objective Optimization, 1993 IEEE Regional Conference on Aerospace Control Systems, CACS 93, Thousand Oaks, Los Angeles. Rusnak, I., Guez, A. and BarKana, I. (1993b): Multiple Objective Optimization Approach to Adaptive Control of Linear Systems, American

APPENDIX A In this appendix we derive the solution of the optimal simultaneous identification and tracking of certain systems. By itself this in not and interesting problem since if the system is certain, i.e. its parameters are known we do not need identification. It is presented here as it is used in the paper as the basis for the certainty equivalence algorithm. A.I . Problem Statement In this section we pose a control problem of linear certain time-invariant systems. We consider the nth order continuous linear time-invariant single-input single-output system xp(t) = ApXp(t) + bpup(t),

xp(to) = Xpo , (A.Lla)

yp(t) = cpXp(t),

t

~

to,

where the input up(t) E L2[to,tl]; ApE Rnxn; xp, c T , bpE Rnxl; x(t)E Rnxl is the state; and I yp(t)E R is the output. We assume that the system is observable and without loss of generality that (Ap, bp ,cp) is in the observer canonical fonn. That is

AP=[::

~

1 0

-an_lOO

-30 0

g],bp=[ ~], 1

o

cp = Co =[1 00 .. 0].

ho-I bn

(A. Llb)

The "unknown" parameters are 9p = [al a2 ... an bl T 2n b2 ... b n] ER. We wish to find the functional

(A. 1.2) such that the criterion (A. 1.3)

13

tl xp

Xp

+ J(Yp(t)-YD
to is minimized subject to the differential equation constraints (A.I.l), where YD(t) is the desired trajectory, and such that Up(t)E Up, where Up is the admissible input set.

Oxp

Oxp

ahl

ahl

Q.

Oxp

dt

ohn

Ap

Q.

Q.

= g bd(Ap)

Oxp

Q

abl

Oxp

Q

ahn

Oxp

Q bd(Ap)

obl

A.2. Optimal Inputs!or Identification In this section we consider design of the input such that it will optimal with respect to the identification problem. This section follows closely the approach by (Mehra, 1974a,b). We deal with the linear singleinput single-output system (4.1). The problem of finding the "optimal" input for identification, as formulated in (Mehra, 1974a), is to select the input (u(t), to :s: t:s: tl} to maximize a suitable norm of the Fisher Information matrix, M, subject to a suitable constraint.

Oxp

Oxp

abn

abn

bp

Q +

Q

up(t).

el

The Fisher Information matrix M for the unknown parameters h and b can be shown to be (Nahi, et al., 1969)

en

tl M = J (Veyp? W 2 -1 (Veyp) dt,

Xp

(A.2.1)

to

Oxp Q

Oxp

oh n (to) =

Oxp Obl

tl tr(MW) = J (Veyp) W (V eyp)T dt,

Q Q Q

Oxp

(A.2.2)

Obn

to

=

xpO

ahl

The norm used in (Rusnak, et al., 1993b) is the tr(M). Here we consider as norm the trace of the weighted information matrix tr(MW'), where W' is the weighting matrix. Here we assume that W' is diagonal, then

=

where W W2- 1 W' diag(0)1,ffi2, ...•0>2n). To find the optimal input for identification we proceed as in (Rusnak, et al .• 1993b):

Xp

V~Yp = Ve cO xp = Co Ve xp = cO[ Vh xp Vb xp]

~ _ ~ oA Oxp( _ oh - Ap oh.. + oh xPoh to) - 0, .p .'1' p P

~

Obp

~~Oxp(

= Ap obp +

Obp up.

(A.2.3)

t7

Q.]

T_

veYp -

~

abp to) = 0, obp = I,

[

!LCOQ. . . Q. Q. cO Q. . Q.

~

Q. Q. . . • cO

~

Ohn abl

2 which we can written as linear (2n +n)-th order single-input system with 2n outputs: (A.2.4)

Oxp

abn where bd(Ap) =block diagonal(Ap,Ap,... ,Ap)E R

14

n2xn2

,

where

(A3.4)

et Q. ... Q. Q ... Q

~

E=

Ap

2 e Rn xn,

E bd(Ap)

A=

en

Q

Q ... Q

ei is the common unit length column, and Q denotes a zero matrix, column or row.

Q..

Q.. Q

Q bd(Ap)

bp Q

A.3. Joint Optima/Identification and Tracking

b=

In this section we derive the solution of the optimal simultaneous identification and tracking of certain systems. By itself this in not and interesting problem since if the system is certain, i.e. its parameters are known we do not need identification. It is presented here as it is used further in the paper as the basis for the certainty equivalence algorithm.

Q el

' c=

[COH' '~l Q. cO Q. .. Q. Q. Q. cO Q. . Q.

,

Q. Q. . .. cO

en Xp

We simultaneously optimize the tracking objective (AI.3) and the estimation objective (A.2.2) subject to (A. Lt, A.2.4). Following the MOOP theory joint minimization of both objectives is to minimized their weighted sum. Then one has to check if the solution is the optimal, i.e. belongs to the Pare to set (Koussoulass and Dimitriadis, 1989).

dxp aht =:(t) =

The control objective is given in (4.3) and the input design objective is given in (5.2). Here we formulate the combined problem, that is to minimize

XPo Q

~ ,.::.10 -( ) ='::'0= ah n

Q

~

Q

abl

Q

~

abn 1 T Jc- [(Yp(tl)-YD(J}» G(Yp(tl)-yo(tl»

(A.3.1)

2

tl + J[(Yp(t)_yo(t»TQc(Yp(t)-YO(t»

10

T

T

-VeypWVeYp +Up(t) RUp(t»)dt). subject to the dynamic constraints (ALl, A.2.4). This can be written as minimization of This is the standard Linear Optimal regulation problem whose solution is given in [K wakernaak and Sivan, 1972; Bryson and Ho, 1975).

(A.3.2)

tt

+

J[(~(t)-~o(t» TecC~(t)-~o(t)+up(t)TRUp(t»)dt}

The solution is (A.3.5)

to

up(t) = -R-1bTn(t) =:(t) + R- 1bTL(t),

subject to the dynamic constraint that can be written

-IT(t) = n(t)A+ATn(t)+ cTScc-n(t)bR-lbTll(t), T n(tt) = c rc,

as 2(t) = A3(t) + bUp(t), ~(t)

2(to) = 2 0 ,

-L(t) = (A-bR-1bTn(t))TL(t) + cTec~D(t), T L(tl) = c ec~D(tl)·

(A.3.3)

=c 2(t), 15

0.8 0.5

0.6 0.4

0 0.2 0

-C.5

-Col -C.4

0

·1

0.5

0

1.5

0.5

lime [...,)

2

lime [sec)

Fig. 11.1a: The desired(input) and the actual trajectory(output) and the actual and estimated states for stable minimum phase plant, H 11.

Fig. 11.2a: The desired(input) and the actual trajectory(output) and the actual and estimated states for stable minimum phase with reversed sign plant, H 12.

-10 '120~-------='0.""'S------~------,...,1.5;:-----'-----:! time [oee)

lime [oee)

Fig. l1.1b: The actual and the estimated parameters for stable minimum phase plant, H 11.

Fig. 11.2b: The actual and the estimated parameters for stable minimum phase with reversed sign plant, H12. traclciDg Cm><

0.08 .------------....::::==~--__,_------_,

0.08

0.06

0.06

0.04

0.04

0.02

0.02 0 -C.02

-C.04

·0.04 .0.06

-C.06

0

.0·080~----~0.5;:---------:--------:1;-;.5;---------:2

1.5

0.5

lime [oec)

time[...,)

Fig. 11.2c: The tracking error for stable minimum phase with reversed sign plant, H12.

Fig. l1.1c: The tracking error for stable minimum phase plant, H 11. ~5,-----------~~~~--

2,,----------~~~=-----------_,

__------_,

2 1.5

-1 -0.5

-2

-1 .1.5

0

0.5

.3 0

1.5

1.5

0.5 time[_)

time [oec)

Fig, 11.2d: The input to the plant for stable minimum phase with reversed sign plant, R12.

Fig. 11.ld: The input to the plant for stable minimum phase plant, H 11. 16

2

O~

__----~__- -__~ ~0~----~0~5------~---------:l~5---------;! time 1_]

time

Fig. 11.3a: The desired(input) and the actual trajectory(output) and the actual and estimated states for unstable nonminirnum phase plant, H23·

lsec]

Fig. 11.4a: The desired(input) and the actual trajectory(output) and the actual and estimated states for unstable nonminirnum phase with reversed sign plant, H24. 4r-________~==~ud~~~ · =HmN= · =~~

.ISO~-------:O:"':.S'-----~--------:-"\'s"'-----~2

__------,

\.S

.me lICe]

2

time [occ]

Fig. II.4b: The actual and the estimated parameters for unstable nonminirnum phase with reversed sign plant, H24.

Fig. 11.3b: The actual and the estimated parameters for unstable nonminimum phase plant. H23. .

0.05 r -_______----=:=~
0.05

0 ·0.05

·0.1 ·O.IS

-0.2 -0.25

0

0.5

I.S

-0.20~-------=0"""::".S-------7"---------:-c\.S;---------:!2

2

.meloec]

time I..c]

Fig. 11.3c: The tracking error for unstable nonrninirnum phase plant, H23.

Fig. l1.4c: The tracking error for unstable nonminimum phase with reversed sign plant, H12· 4r-__________~=t=ro~~~bm=-____________,

4r-----------~~~c=~--------__.

2

o ~0~----~0~ .S ----------------:-"I.S;------~2

-I L - - - - - - - - :""""::.S,--------------:-"I.S" .--------;!2 O 0

.me 1_]

time I..c]

Fig. 11.3d: The input to the plant for unstable nonrninimum phase plant, H23.

Fig. 11.4d: The input to the plant for unstable nonminimum phase with reversed sign plant, H24. 17