- Email: [email protected]
Combining model reference adaptive controllers and stochastic self-tuning regulators
000.~.-1098/S2/010077-0g$03.0010 PerlpzmonPressLM © 1982InternationalFederationof AutomaticControl
Automatica, Vol. 18, No. 1, pp, 77--.84, 1982 Printed in Great Britain
Brief Paper Combining Model Reference Adaptive Controllers and Stochastic Self-tuning Regulators* I. D. L A N D A U t Key Word~--Adaptive control; stochastic control; stability; parameter estimation.
Model reference adaptive control schemes (implicit or explicit) and stochastic self-tuning regulators use parameter adaptation algorithms which have a similar structure. Positive real conditions appear when the stability analysis in a deterministic context and convergence analysis in a stochastic context are done for the two types of schemes, respectively (Ljung and Landau; 1978; Landau and Lozano, 1981; Narendra and Valavani, 1978; Egardt, 1980; Dngard and Landau, 1980c; Dugard, Egardt and Landau, 1980d). This paper goes further than that of Landau (19?9b) in the sense that the equivalence between implicit and explicit schemes and the asymptotic duality established between various model reference adaptive control sFhemes and stochastic self-tuning regulators are used in order to give a positive answer to the question: is it possible to design an adaptive control scheme which can operate in a deterministic environment as a MRAC with desired performances and in a stochastic environement as a S-STURE with desired performances (tracking and regulation)? Partial answers to this question have been given in Gawthrop (1977) and Weilstead (1979) but the solutions proposed have not been fully proved (the convergence analysis is absent). A more complete answer together with the convergence analysis which allows one to specify clearly the conditions of correct operation is given in this paper.
Al~traet--The similarities between model reference adaptive controllers (MRAC) for minimum phase plants where the control objectives are specified by reference models and stochastic self-tuning regulators (S-STURE) for minimum phase plants where the control objectives are specified by autoregressive moving average (ARMA) stochastic models are investigated. These similarities permit the extension, for these two classes of adaptive systems, of the duality existing in the linear case with known parameters between modal control and minimum variance control. The convergence analysis of these schemes in a combined deterministic and stochastic environment allows one finally to define schemes which behave as a desired MRAC in a deterministic environment and as a desired S-STURE in a stochastic environment. The problem of removing the positive real conditions for convergence in deterministic and stochastic environment is also discussed, 1. Introduction
IN THE last few years a growing effort has been made in an attempt to establish connections, similarities or equivalence of various adaptive control schemes for minimum phase SISO systems (Ljung and Landau, 1978; Narendra and Valavani, 1978; Silveira, 1978; Egardt, 1979, 1980: Landau, 1979b; Gawthrop, 1977; Johnson, 1979). Model reference adaptive controllers (MRAC) for deterministic systems using an explicit reference model (see Fig. 1) can be equivalent to adaptive control schemes using an adaptive predictor as an intermediate step for building an adaptive control (Ljung and Landau; 1978; Narendra and Valavani, 1978; Silveira, 1978; Landau, 1979b). The first condition which should be satisfied in order that the two schemes be equivalent is that the control strategy be such that the output of predictor should behave identically to that of the explicit reference model (i.e. the adaptive predictor plus the controller form an implicit reference model; see Fig. 2). The stochastic self-tuniog regulators (S-STURE) where the objectives are specified in terms of ARMA models have a structure similar to that of the implicit model reference adaptive controllers (Ljung and Landau, 1978; Egardt, 1980; Landau, 1979b; Gawthrop, 1977). Minimum variance STURE is a particular form of SSTURE considered above (Landau, 1979b). From the equivalence of implicit and explicit model reference adaptive control, stochastic self-tuning regulators with explicit prediction models can be defined. They are equivalent to the known stochastic self-tuning regulators (with implicit prediction models) and feature structural similarities with explicit model reference adaptive control schemes (Landau, 1979b, c).
Adaptation mechanism
Explicit -I
yM(k~)
1
I
-I
reference m°de' I
FiG. 1. Explicit model reference adaptive control scheme.
• *Received 11 February 1980; revised 16 September 1980; revised 31 March 1981. The original version of this paper was presented at the 8th IFAC Congress which was held in Kyoto, Japan during August 1981. This paper was recommended for publicationin revised form by associate editor K. J. Astr6m. t'Laboratoire d'Automatique de Grenoble (C.N.R.S.), E.N.S.I.E.G.. B.P. 46, 38 402, St. Martin d'Heres, France.
• Tmphcif reference model
+!r
,<->,
-O
FIG. 2. Implicit model reference adaptive control scheme. 77
78
Brief Paper
The paper is organized as follows: Section 2; the design of a class of linear controllers for tracking and regulation in a deterministic and stochastic environment is briefly reviewed. This will provide better understanding of the similarities between MRAC and S-STURE. Section 3 presents MRAC and S-STURE schemes which are extensions of the schemes given in Section 2 for the case when the parameters of the plant are unknown. In Section 4 the structural similarities between S-STURE and MRAC and their behaviour in a combined environment are examined. In Section 5, a scheme which behaves as a MRAC in a deterministic environment and as a S-STURE in a stochastic environment is presented. In Section 6, the problem of removing the positivity condition required for convergence of these schemes is briefly discussed. In order to reduce the technical details and to make the presentation clearer, the plant to be controlled in the absence of disturbances is assumed to be described by a discrete rational transfer function with a basic delay of one sample, throughout the paper. All the results can be extended for the case of a general delay.
2. Combined tracking and regulation for plant with known parameters 2.1 Deterministic environment. The plant to be controlled is described by
Compute the control as follows:
u(k) = C2(q -I )y M(k + 1)-R(q-t)y(k) - B*(q-t)u(k) b0 = C2.(q-t)yM(k + I) - p~4po(k) (lO) bo where
R(q -I) =
n C2(q -I) - A ( q -I) = ~ (c~ - oa)q -i*l
= rl+ r:q-t...+ r~q-"+l
(11)
B * ( q - ' ) = B(O -t) - bo
(12)
pot = [bl...b,~, rl...r,]
(13)
4h(k) = [u(k - 1)...u(k - m), y(k)...y(k - n + 1)]. 04) Introducing (10) into (1) and defining the plant-model error e(k) = y(k) - y ~ ( k )
(15)
one obtains A(q-~)y(k + 1) = B(q-~)u(k); y(0) # 0
(1)
C2(q-t),(k + I) = O. where y(k) is the measured process output u(k) is the control, q-t is the backward shift operator and A(q -~)= 1 +a~q -~+. . . + a,q-"
B(q-t)=bo+b~q-t+...+b,~q -=
bo~O.
(2)
Ct(q-I)y(k) = q-JD(q-l)uM(k)
(4)
C,(q - l ) = l + c { q - l + . . . + c~q -~
(5)
where
is an asymptotically stable polynomial D(q -~) = do + dlq -~ + . . . + &q-in
Vk ~, 0
C2(q-~)yU(R + 1) = p Tcb(k)
(17)
o r = [be, por]
(18)
4,(k) = [u(k), <~,(k)]
(19)
where
(b) Strategy 2: implicit reference model This strategy is directly inspired by the separation theorem: one first designs an appropriate predictor for the plant output an~ then a control will be computed such that the output of the predictor behaves as the desired output in tracki~. Predictor design C2(q-I)y(k + 1) = p r cb(k) = bou(k)+ R(q-l)y(k)
(6)
and a~(k) is a bounded reference sequence. Rexulation: The control should be such that, in regulation, [u" (k) m O] an initial disturbance [y(O) ~ O] is eliminated with the dynamics defined by CAq-~)y(k + 1) = 0
(7)
where
+ B*(q-~)u(k)
~(k + 1) = y(k + 1)- 9(k + 1)
(21)
one notes that it satisfies
(22)
(8)
is an asymptotically stable polynomial. The solution for this control problem can be obtained in two ways: either by using an explicit reference model as part of the control system or by using a one-step ahead predictor of the plant output which together with the controller will form an implicit reference model. (a) Strategy 1: explicit reference model Define the explicit reference model
Cffq-')yU(k + 1) = D(q-I)um(k)
(20)
where R(¢ -I) , B*(q-I), p r ¢~(k) are given by (11), (12), (18) and (19), respectively. Defining the prediction error as
C2(q-l)~(k + 1) = O, k ~ O. C2(q-') = 1 + c~¢ -1 + . . . + c:,q-"
(16)
and one concludes that both control objectives are achieved. Note that the control given by (10) can be expressed in the equivalent form
(3)
The polynomial B(z -~) is assumed to have all zeroes in Izl < 1 (z-complex variable) therefore they can be cancelled without leading to an unbounded control. The objectives of the control which we are considering in the deterministic environment are given below. Tracking: The control u(k) should be such that, in tracking, the output of the process satisfies the difference equation
k~ 0
(9)
where yU(k) is the output of the explicit reference model.
Computation of the control. The control is computed such that ~(k + 1) = yU(k + l) which leads to C2(q-*)9(k + 1) = C2(q-t)yM(k + 1) = p ro(k)
(23)
and the control is given by (10) as expected. The resulting control schemes can be interpreted as a pole placement by feedback combined with a feedforward control of the dynamics and an introduction by feedforward of new dynamics. The feedforward control will cancel the closed loop dynamics and will introduce the desired dynamics for tracking (Landau and Lozano, 1981).
Brief Paper
2.2 Stochastic environment. The plant and the disturbance are described by • A(q - I )y(k + 1) = B(q-~)u(k) + C(q-~)~a(k + 1)
The design presented for deterministic and stochastic environment, respectively with the control objectives considered are dual in the sense that the same control provides a solution to two different problems.
(24)
where C(q -~) = 1 + cq -~ + . . . c . q - "
79
3. The case of unknown plant parameters 3.1 Deterministic environment. When the plant parameters are unknown it is possible to build adaptive control schemes which will use an explicit or an implicit reference model following the design methodology given in Landau and Lozano 0981); Lozano and Landau (1981).
(25)
is an asymptotically stable polynomial, to(k) is a sequence of equally distributed independent normal (0, ~) random variables, A(q -~) and B(q -~) are given by (2) and (3). The control objectives which are considered in the stochastic environment are generalizations of the minimum variance control strategy (they are specified in terms of ARMA models) and the solutions are obta/ned via an algebraic approach. For more details concerning the algebraic approach to linear stochastic control problems see Kucera (1975). The objectives of the control which we are considering in the stochastic environment are Regulation: The plant and the disturbance are given by (24); the control should be such that
( a ) Explicit model reference adaptive control In the design given in Section 2.1. a one replaces the fixed parameter vector in (17) by an adjustable parameter vector /~r(k) = [~e(k),/io(k)]
(34)
and the control law will be given by u ( k ) = C2(q-')YU(k + 1 ) - P°T(k)#°(k) /~o(k)
(35)
or equivalently by C2(q-~)y(k + 1) = C(q-~)to(k + I)
(26) T(k)tb(k) = C2(q-')yU(k + I).
where C2(q -~) defines in fact the poles of the closed loop system. For C2(q-m)= C(q -~) the control objective corresponds to the minimum variance control (Astr6m and Wittenmark, 1973). Tracking: Given a reference trajectory defined by (9) the control u(k) should be such that
One defines the objectives of the adaptive control system as limk.=C:(q-t)E(k + 1) = fim~e°(k + 1) = 0
(27)
For C2(q -~) = C(q -~) the control objective corresponds to minimum variance tracking of the reference trajectory in the presence of a stochastic disturbance. The appropriate control satisfying both objectives can be obtained by using an explicit or implicit prediction reference model.
.
~b(k)[I ~ M < =, for all k.
O(k + 1) =/~(k) + Frcb(k)v(k + 1)
F i ~ , = A z(k)F 7cI + Az(k)da(kNa(k) r
F0>O; 0 < k~(k) ~ 1; 0 ~ k2(k)<2
C1(q-l)~:U(k + 1) = C~(q-l)yU(k + l) -- D(q-t)uU(k). (28)
!-/,( -i ) v(k + 1) = ~ [p - l~(k + l)]r~(k) (29)
where p0 and tb0(k) are given in (13) and (14). Defining the plant-model error as in (15) one obtains using (29) that
(41)
where H~(q-') and H2(q -I) are monic polynomials on q-' (q-Lshifting operator) and v(k + 1) is obtained as follows: ,,
H}(q-')
v(k + =j = ~ e
(30)
,.. tK + 1) = H2(q- )
x (e°(k + 1) + i~(k + 1)]
which is the desired result.
(42)
where
(b) Implicit prediction reference model
e*(k + l) = [p -/~(k + l)]T~b(k) i(k + I) = [~(k) - p(k + l ) ] ~ ( k )
Predictor design (31)
where p and ~(k) are given in (18) and (19). Note that the prediction error defined as in (22) satisfies (32)
Computation of the control. The control is computed such that ~(k + 1) = yU(k + 1) which leads to C:(q-;)~(k + 1) = C2(q-')yU(k + 1) = pT4,(k)
(40)
and v(k + 1) is the a posteriori adaptation error (residual) which satisifies the equation
Compute the control as follows:
C2(q-I)~.(k + 1) = C(q-J)to(k + 1).
(39)
where FK is th adaptation gain matrix given in general by
Define the prediction reference model
C2(q-J)~(k + 1) - pr~b(k)
(38)
Using a stability type design (Landau and Lozano, 1981) one obtains the following adaptation algorithm
(a) Explicit prediction reference model
C : ( q - = ) e ( k + I) = C(q-I)to(k + l)
(37)
where e°(k + l) = C2(q")e(k + 1) and
C2(q-~)ytk + I) = C2(q-~)yU(k + 1)+ C(q-~)w(k + I).
C2(q-~)yU(k + l) - pToc~o(k) u(k) = bo
(36)
(33)
(43)
eO(k + I) = [p -/~(k)]r~b(k) ~(k + I) is a transient adaptation signal also called the'anxiliary error' (Ionesfu and Monopoli, 1977). The objectives of (37) and (38) are achieved if H,(z -~) ~, H2---~'~-2; 2 > x ~maxX~(k), is strictly positive real.
(44)
80
Brief Paper
Note that v(k + 1) can always be expressed in terms of e°(k + 1) which is a measurable quantity and the previous values of v(k + 1). If we assume (as for stochastic self-tuning regulators) that the parameters of the process are unknown but constant, decreasing adaptation gains can be used and (40) becomes
F;~t=F-~'+Xd~(k)da(k)r;
Fo>O;
0
(45)
For the purpose of comparison with two types of S-STURE two prototype schemes which are particular forms of the general design indicated above are considered.
3.2. Stochastic environment. When the plant parameters and the disturbance model are unknown, the controller parameter p in the designs given in Section 2.2 (with explicit or implicit prediction reference model) should be replaced by an adjustable parameter vector. Then an appropriate adaptation algorithm should be defined such that the control objectives in the linear case with known parameters be asymptotically achieved. This can be expressed in general by the condition that a certain filtered error becomes asymptotically a white sequence. For the objective considered in Section 2.2 this can be expressed as Prob { ~ ! mC2(-~) ~,(k
Prototype 1 [Ht(q -1) = 1, H~,(q-t) = C:(q-l)] 1
v(k + 1) = ~
L..21,q
)
e*(k + I),
(46)
For this case the explicit expression for computation of
v(k +'1) is E(k + l ) -
v(k + 1) =
(47)
where
(52)
Given a certain adaptation algorithm the asymptotic properties of the scheme can be examined using the ODE method. Theorem A.I given in the Appendix which is derived from ODE method allows a straightforward analysis of these schemes. For the purpose of this paper we will consider two types of objectives leading to two prototypes schemes
~ c ~ ( k + 1 - i)
I + ~(k)'Fx~(k)
+ 1) = ca(k + 1)} = 1.
(1)
Problime(k+l)=oJ(k+l)}=l;C(q-~)~l. k--*ffi
(2)
Problim{Cz(q-1)e(k + 1) = oJ(k + 1)} = 1; C(q -~) = I.
(53)
k~
(54) ~(k + 1) = C ~ q - )[lO(k) - lO(k + l)]T~(k)
(48)
The first objective corresponds in regulation to minimum variance self-tuning control [i.e. y ( k ) ~ oJ(k)] and the second objective corresponds in regulation to C:(q-~)y(k)-~cu(k) for C(q -~) = 1. Both objectives are ARMA type control objectives. The corresponding adaptation algorithms will be
and e(k + i) is given by (15). One also has v(k + I) = E(k + l)+~(k + I).
The stability of the scheme is assured if 1
Prototype 1 (minimum variance STURE)
A
C:(z-'~-) 2
(49)
is a strictly positive real transfer function (which also implies that lirn $(k) = 0).
p(k + 1) - lO(k)+ FKfb(k)v(k + 1)
(55)
e(k + l) v(k ÷ 1) = 1 + ~b(k)rF~c4~(K)
(56)
F~.~=F~+hd~(k)tb(k)r;Fo>O;O
(57)
where
k--,ffi
Prototype2 and [Ht(q -l) = bI2(q -I) = 1]
v(k + l) = e*(k + 1) = Cz(q-~)e(k + 1) 1 + ~(k)rF~(k)"
(50)
For this case, stability is assured without any positive real condition to be satisfied. (b) Implicit model reference adaptive control The implicit model reference adaptive control results immediately from the linear design in which the linear predictor is replaced by an adaptive predictor
C,.(q-~)~(k + 1) = Or(k)~(k)
(51)
and designing the control such that 9(k+ 1)= yM(k + 1) which leads to the control given by (35). If for updating the adjustable parameter if(k), one uses the same algorithm as given by (39) and (40) where the plant-model error which enters in the computation of v(k + 1) is replaced by the prediction error, the two schemes are equivalent (Lozano and Landau, 1981). One can state the conditions for this equivalence in general as follows: De]inition 3.1. (Equivalence between explicit and implicit MRAC) An explicit MRAC and an implicit MRAC are equivalent if and only if (1) The equations for the a posteriori adaptation error v(k) ate identical. (2) The parametric adaptation algorithms are identical. As a consequence: the positivity conditions for asymptotic stability are identical.
Applying Theorem A.1 or using the results given in Ljung (1977) one concludes that the objective of (53) in regulation will he achieved ff 1
1
C(z-7~)- ~
(ss)
will be strictly positive real.
Prototype 2 l~(k) is updated using (55) with F(k) given by (57) and v(k + 1) is given by Cz(q-t)e(k + 1) v(k + 1) -- 1 + 4~(k)rFKcb(K)"
(59)
Using Theorem A.1 one concludes that the objectives of (54) in regulation will be achieved without any positive real condition to be satisfied. The important remark to be made is that the S-STURE can be implemented either using an explicit or an implicit prediction model, the two implementation being equivalent [only the significance of E(k + 1) differs]. Similar to MRAC, conditions for equivalence of implicit and explicit S-STURE can be given, Landau (1979b, c). Note that in the STURE litterature only the implicit type implementation has been mostly considered.
Brief Paper
81
4. Structural similarities and analysis of S-STURE and MRAC in a combined environment Consider the prototype schemes 1 and 2 for MRAC (with implicit or explicit reference model) when using the gain updating given in (45) and the prototype schemes 1 and 2 for S-STURE (with implicit or explicit prediction reference model) respectively. One observes that the prototype schemes 1 for MRAC and S-STURE, respectively use the same control law, the adaptation algorithms have the same structure, the convergence conditions are of the same type; the only difference occurs for the a posteriori adaptation error during transients [compare (47) and (56) since [ ( k ) - , 0 as k--,®]. A similar situation holds also if one compares the prototype schemes 2 for MRAC and S-STURE with the remark that in this case the a posteriori adaptation errors are identical for both schemes [compare (50) and (59)]. However despite these similarities (structural equivalence) the MRAC and S-STURE achieve different tasks, which justifies the introduction of the following definition.
De/inition 4.1. (Asymptotic) Duality between MRAC and S-STURE (Landau, 1979b, c). A MRAC (implicit or explicit) designed for a deterministic environment is (asymptotically) dual with respect to a S-STURE (implicit or explicit) designed for a stochastic environment if and only if (1) The adjustable parameter vectors are updated by (asymptotically) identical adaptation algorithms [same structure, same observation vector (~b) same a posteriori adaptation error (as k -, ®)]. (2) The positivity conditions for the global asymptotic stability of the MRAC and for w.p.! convergence of the S-STURE are the same. (3) The control laws are (asymptotically) the same. Remark: If both control laws and a posteriori adaptation error are identical for any k, they will he called dual. After emphasizing the similarities between MRAC and S-STURE the basic questions to which this paper wants to ask and answer come in view. They are (1) What is the behaviour of (asymptotic) dual MRAC and S-STURE in a stochastic and deterministic environment, respectively? (2) Can a scheme be designed which behaves as a desired MRAC in a deterministic environment and as a desired S-STURE in a stochastic environment? We will limit ourselves in this section to answer the first question in regulation [ u S ( k ) - 0]. Consider the prototype 1 for MRAC. The analysis in a stochastic environmement can be made using Theorem A. 1. The plant in this case will be described by (24). For the particular case C:(q -~) = C(q -~) a straightforward analysis shows that because of duality this scheme will behave as the prototype 1 of S-STURE (minimum variance STURE) [the transients terms in (47) depending on # ( k ) /~(k- 1) disappear when using ODE method since 10(k)-# ( k - 1 ) = / ~ ] . Therefore we will consider next the case C~(q-*)# C(q -~) and we will sketch briefly the analysis using Theorem A.I given in the appendix. In the presence of stochastic disturbance the a posteriori adaptation error v(k + 1) for prototype I considered in Section 3 satisfies the equation
+ C(q'%,(k
+ l) = [ p * - p l Y ( k ,
#)
+ C(q-~)to(k + 1)
(62)
where [p*] = [b0...bin, cl-
at . . . .
c. - a.]
(63)
corresponds to the controller parameter vector assuring that the closed loop poles are defined by C(q -~) instead of C2(q-I). Equation (62) can be re-written as
~(k+l,/~)=C---~[
p -~]
~(k,/~)+totk+i)
(64)
and applying Theorem A.I one concludes that e(k + l)--* o ( k + l ) and #(k) will converge to p® such that [ p * p.~]~0(k) = 0 where p* corresponds to the closed loop poles defined by C(q -~) if
C(q=r)
2
(65)
is a strictly positive real transfer function. The convergence point and the convergence condition are identical to that of the dual S-STURE (in this case the minimum yariance STURE). This is also true for the prototYpe scheme 2 and in general for dual MRAC and S-STURE. One has then the following important conclusion (for regulation).
Theorem 4.1. A MRAC in the presence of a stochastic disturbance of the same structure as that consi~lered for its dual S-STURE will behave asymptotically as its dual SSTURE (in regulation). Consider now the prototype I S-STURE in a deterministic environment [ w ( k ) = 0, c~ ---0, i = i,, . . . n]. One immediately observes that the algorithm is identical to the prototype I MRAC for C,(q -~) = 1 and one concludes that the scheme will be globally asymptotically stable (the condition of (49) is automatically satisfied). However, the disadvantage of the S-STURE is such a situation is that it will drive all the Closed loop poles to z = 0 and this means a one step response which, depending on the sampling per!od, can become in many cases undesirable because of the magnitude of the resulting control.. Simulations have shown also that large transients occur in this case during adaptation process (Landau and Lozano, 1981). Therefore in a certain way, the desired performance of S-STURE in a deterministic environment should he specified by the designer without modifying the behaviour of the S-STURE in a stochastic environment. This can be obtained by replacing the S-STURE by its dual MRAC. On the other hand since the poles of the closed loop will in general depend on the characteristics of the stochastic disturbance it will be necessary for tracking purposes that the control law (35) be modified in order to take in account the drift of the closed loop poles. This will be discussed in the next section. 5. A combined M R A C - S - T U R E scheme
C2(q-~)v(k + 1) = [# - #(k + l)]T<~(k) + C(q-')oJ(k + 1). (60) Defining now the stationary sequences. ~(k+ 1, p), O(k,l~) for l~(k) =/~ and taking in account that uU(k)= 0 one has
~(k + 1, p) = t(k + l, p) -- y(k + l, #).
(6l)
We will give next a scheme which illustrates how an adaptive scheme behaving as a MRAC in a deterministic environment and a S-STURE in a stochastic environment can be obtained for the joint tracking-regulation problem. The implementation can be done using an implicit or explicit form. In the designs given in Section 2.1 the control law is replaced by
u(k)
One obtains then from (60) for #(k) = # the following equation
C~(q-I)YU(k + l)-#0~(k)T~b0,(k) =
(66)
bo(k)
where
C(q-')[(k + 1, p ) = [p - P]r~(k, .~) n
- ,~. (c~ - c , ) ~ ( k + 1 - i, ~)
AtJT
18:1
-
/~o¢(k) '= [~o(k) T, c(k) T ]
(67)
4~ore(k) = [tboT(d), y U ( k ) . . . y U ( k + 1 - n)].
(68)
82
Brief P a p e r
The vector ~(k) is formed by the estimates of the coefficients of the polynomial [C2(q -~) - C(q-~)]. The ~_4a_pmtionalgorithm is given by
/~.(k+ I) =
:.(k) + Fx~b,,(k)v(k +
1)
(69)
where
0,(k) r = [6o(k), .~o.(k) r]
(70)
,t,~(k) = [u(k), ,~T(k)]
(71)
Theanalysis of this scheme in the deterministic environment is straightforward and it is omitted [consider (69) and (73) and apply the results of Landau and Lozano (198t) which yields the condition of (75)]. The analysis in a stochastic environment using Theorem A.I is clone as following: from (24) one obtains C(q-1)y(k ~- 1) = p*r~(k) + C(q-~)oJ(k + 1)
(77)
p.T = [bo, p~r]
(78)
where
F~L, = F i ~+ X6,(k), ~bT(k); F0>0; 0<~, < 2 u(k + 1) = ~
1
(72)
is given by (63). Defining now the stationary sequences
e* (k + 1)
~(k + 1, p,), ~(k, ~.) for p,(k) = p,
e(k + I ) - ~ d ~ ( k + t - i)
and taking in account that
I.I
I + ~b.(k)rFx~.(k) =
~~ z[t qp , /
-
#(k + 1,/~,) = i(k + 1,/~)
p.(k + 1)]T~k,(k)
(79)
(73) one obtains
C(q-t)~(k + 1, ~,) = C(q-')i.(k + 1, ~0,) -- C(q-~)[y(k + 1, .0,)
and
- yU(k + 1)] = p~r,~,o(k, ~,) + floe(k, ~,)
~(k + 1) = C~---~.p_ ,(k)- p,(k + 1)]~',b(k)
(74)
The block diagram corresponding to this adaptive control system is 8ivan in Fig. 3. The above scheme'has the following properties:
+ (bo- bo)~(k, ~ , ) - C(q-~)y.(k + 1) + C(q-~)~(k + 1). (80) Introducing now the control given by (66), for p,(k) = p one finally obtains
(I) In a datmmlnhfic environment the objectives for the linear duign (tracking and regulation) are asymptotically achieved if
~ ( k + l , p , ) = ~ ( k + 1 , p.)= C-~'~'~[p. I . - Po,] r
~,(kl,t~,)+ o~(k+ 1)
C2( z-"~
2
(75)
is strictly positive real. (2) In a stochastic environnmnt, the minimum variance t r a c k / ~ of the reference trajectory is asymptotically achlavnd (w.p.1) if
2"
(76)
(81)
where p*, = [p*, c~- c~ ..... c2.- c.].
(82)
Applying now Theorem A. I, the convergence condition (76) is obtained. Generalizations of this scheme for delays greater than one and various types of stochastic disturbances as well as simulations can be found in Dullard and Landau (1980) and D u p r d , Egardt and Landau (1980).
.....
it,,°,
I
1
~(k) FIo. 3. Adaptive control scheme for tracking and regulation in a combined environment.
Brief Paper 6. The positivity problem The global asymptotic stability of the prototype 1 MRAC can be assured if the transfer function given in (49) is strictly positive real. This condition is restrictive since it limits drastically the region of the allowed poles in the z-domain. In the MRAC designs two solutions can be considered in order to overcome this problem (Landau and Lozano, 1981). (1) Introduction of linear compensators acting on the plant model error. (2) Introduction of appropriate filters for generating the observation vector ~(k). Note that some of the filters introduced on the plant model error have an effect equivalent to the change of the reference model structure (Landau and Lozano, 1981). The elimination of the positive real condition is illustrated in prototype 2 MRAC where the plant-model error is passed through the linear compensator defined by Cz(q-~), see (50). However, if one consider the prototype 2, S-STURE which is dual to prototype 2 MRAC, one observes that the positive real condition for convergence has been removed (see Section 3.2) but in the meantime the nature of the disturbance and the control objective have been changed with respect to prototype 1 S-STURE. This type of observation made also on other prototype schemes leads to the conclusion that positive real condition can be removed in the deterministic context while maintaining the control objective, but in the stochastic environment, removing of the positive real condition implies a modification of the control problem (different type of disturbance and/or different control objective). 7. Conclusions In the context of minimum phase plants the similarities between MRAC where the control objectives are specified by reference models and S-STURE where the objectives are specified by ARMA models have been investigated both from structural and theoretical (convergence analysis) point of view. It was shown that MRAC and S-STURE feature a duality character which extends the duality existing in the linear case with known parameters between linear model following control and stochastic control w/th ARMA type objective. As a result of this investigation, a method for constructing adaptive control schemes which behaves as a given MRAC in a deterministic environment and as a given S-STURE in a stochastic environment is given and illustrated by an example. The convergence analysis has shown the importance of the positivity conditions and of the transient adaptation signals which are specific to MRAC design. The positivity condition which appears as convergence condition for both MRAC and S-STURE can always be removed in a deterministic context without modifying the control objectives while in a stochastic context removing the positivity conditions leads to a change of the nature of the disturbance and/or of the control objectives. The transient adaptation signals play a crucial role for obtaining good performances from the combined MRAC S-STURE in a deterministic environment and in some situations can improve the performances in the stochastic case with respect to a S-STURE.
References ~strOm, K. J. and B. Wittenmark (1973). On self-tuning Aregulators. Automatica, 9, 185. strtm, K. J., V. Borisson, L. Ljung and B. Wittenmark (1977). Theory and applications of self-tuning regulators. Automatica, 13, 457. AstrOm, K. L and B. Wittenmark (1980). Self-tuning controllers based on pole-zero placement. Proc. IEE, 127, 120. Dugard, L. and I. D. Landau (1980a). Recursive output error identification algorithms theory and evaluation. Automatica, 16, 443.
s3
Dugard L., I. D. Landau and H. M. Silveira (1980b). Adaptative state estimation using M.R.A.S techniques convergence analysis and evaluation. IEEE Trans Aut. Control, AC-25, 1169. Dugard, L. and I. D. Landau (1980c). Stochastic model reference adaptive controllers. Proc. 19th IEEE CDC Conference, Alburquerque, U.S.A. Dugard, L., Egardt B. and Landau I. D. (1980). Design and convergence analysis of stochastic model reference adaptive controllers. IEEE Trans. Aut. Control (to be published). Egardt, B. (1979). Unification of some continuous time adaptive control schemes. IEEE Trans Aut. Control. AC24, 588. Egardt, B. (1980). Unification of some discrete time adaptive control schemes. IEEE Trans. Aut. Control, AC-2& 693. Gawthrop, P. J. (1977). Some interpretations of the selftuning controller. Proc IEEE, 124, 889. Goodwin, G., P. Ramadge and P. Caines (1980). Discrete time multivariable adaptive control. IEEE Trans Aut. Control. AC-25, 449. Ionescu, I. and R. Monopoli (1977). Discrete model reference adaptive control with an augmented error signal. Automatica, 12, 507. Johnson, C. R. (1979). Input matching, error augmentation, self-tuning and output error identification: algorithm in discrete adaptive model following. Proc. 18th IEEE-CDC Conference, Fort Lauderdale, U.S.A. Kucera, V. (1975). Algebraic approach to discrete stochastic control. Kybernetica, 11. Landau, I. E. (1979a). Adaptive Control, The Model Reference Approach. Dekker, New York. Landau, I. D. (1979b). Adaptive controllers with explicit and implicit reference models and stochastic self-tuning regulators--equivalence and duality aspects. Proc. 17th IEEE-CDC Conference, San Diego, U,S.A. Landau, I. D. (1979c). Model reference adaptive control and stochastic self-tuning regulators--towards cross fertilization. Proc. of AFOSR Workshop: Adaptive Control. Champaign, U.S.A. Landau, I. D. and R. Lozano (1981). Unification and evaluation of discrete time explicit model reference adaptive control designs. Automatica, 17, 593. Ljung, L. (1977). On positive real transfer functions and the convergence of some recursive schemes. IEEE Trans Aut. Cont, AC-22, 539. Ljung, L. (1978). Convergence of a self-tuning regulator for systems with general delay. Rport LITH-ISY-I-0252, Linktping University. Ljung, L. and I. D. Landau (1978). Model reference adaptive systems and self-tuning regulators--some connections. Proc. 7th IFAC Congress, Vol. 3, pp. 1973-1980. Lozano, R. and I. D. Landau (1981). Redesign of explicit and implicit discrete time model reference adaptive control schemes. Int. J. Control, 33. Narendra, K. S. and L. S. Valavani (1978). Direct and indirect adaptive control. Proc. 7th IFAC Congress, Vol. 3, pp. 1981-1988. Silveira, H. M. (1978). Contributions ~ la synthtse des systtmes adaptatifs avec mod/.~iesans accts aux variables d'~tat. Thtse es-Sciences Physiques I.N.P.G., Grenoble. Wellstead, P. E. and P. Zanker (1979). Servo Self-Tuners. Int. J. Control 30, 27. Appendix: a theorem for convergence analysis in stochastic environment Consider the parametric adaption algorithm /5(k + 1) = ~(k) + Frcb(k)v(k + 1)
(AI)
F[]+l=F~L~-Ack(k)ck(k)r: F0>0; 0 < k <2.
(A2)
where
Assume that the following relation exists between v(k + 1) and ~b(k) v(k + 1) = H(q-~)[p - IJ(k + l)]r~b(k) + w(k + 1)
(A3)
84
Brief Paper
where ~(k + 1) is the immage of a stochastic disturbance in the equation governing the a posteriori adaptation error. One has then the following result.
where
Theorem A.I. Consider the adaptation algorithm of (A1) and (A2). Assume that stationary processes $(k, p) and ~(k + l, 1~) can be defined for I~(k) = p and that i~(k) belongs infinitely often to the domain for which these stationary processes can be defined. Assume in addition that for 10(k) =
and
U(k ÷ I, p) = H(q-I)~pT(k, ~)[~*- f)l÷ ¢o(k + I) (A4) where {oJ(k+l)} is a white random sequence [or uncorrelated with @(k, i~)]Then if A H ' ( z -~) = H (z -~) --~
(A5)
is strictly positive real Prob{~m~(k)~Dc}ffi l
(A6)
o~ :{Pl[p* - p ] % ( k ) = o)
(A7)
Prob{~!m v(k -~ I} = c~(k + l)}= I.
(A8)
The proof of this theorem can be found in Dugaxd and Landau (1980) or Dugard, Landau and Silveira(1980) and is ' based on the application of O D E method Ljung (1977) to (AI)-(A3). Remark: The application of the O D E method for the convergence analysis assume that 0(k) belongs infinitelyoften to the domain for which the stationary processes ~,(k+ l, ~), ~(k, i~)can be defined. For this reason the resultsin the case of adaptive control schemes have a local validityaround the possible convergence points of the algorithms.
Automatica, Vol. 18, No. 1, pp, 77--.84, 1982 Printed in Great Britain
Brief Paper Combining Model Reference Adaptive Controllers and Stochastic Self-tuning Regulators* I. D. L A N D A U t Key Word~--Adaptive control; stochastic control; stability; parameter estimation.
Model reference adaptive control schemes (implicit or explicit) and stochastic self-tuning regulators use parameter adaptation algorithms which have a similar structure. Positive real conditions appear when the stability analysis in a deterministic context and convergence analysis in a stochastic context are done for the two types of schemes, respectively (Ljung and Landau; 1978; Landau and Lozano, 1981; Narendra and Valavani, 1978; Egardt, 1980; Dngard and Landau, 1980c; Dugard, Egardt and Landau, 1980d). This paper goes further than that of Landau (19?9b) in the sense that the equivalence between implicit and explicit schemes and the asymptotic duality established between various model reference adaptive control sFhemes and stochastic self-tuning regulators are used in order to give a positive answer to the question: is it possible to design an adaptive control scheme which can operate in a deterministic environment as a MRAC with desired performances and in a stochastic environement as a S-STURE with desired performances (tracking and regulation)? Partial answers to this question have been given in Gawthrop (1977) and Weilstead (1979) but the solutions proposed have not been fully proved (the convergence analysis is absent). A more complete answer together with the convergence analysis which allows one to specify clearly the conditions of correct operation is given in this paper.
Al~traet--The similarities between model reference adaptive controllers (MRAC) for minimum phase plants where the control objectives are specified by reference models and stochastic self-tuning regulators (S-STURE) for minimum phase plants where the control objectives are specified by autoregressive moving average (ARMA) stochastic models are investigated. These similarities permit the extension, for these two classes of adaptive systems, of the duality existing in the linear case with known parameters between modal control and minimum variance control. The convergence analysis of these schemes in a combined deterministic and stochastic environment allows one finally to define schemes which behave as a desired MRAC in a deterministic environment and as a desired S-STURE in a stochastic environment. The problem of removing the positive real conditions for convergence in deterministic and stochastic environment is also discussed, 1. Introduction
IN THE last few years a growing effort has been made in an attempt to establish connections, similarities or equivalence of various adaptive control schemes for minimum phase SISO systems (Ljung and Landau, 1978; Narendra and Valavani, 1978; Silveira, 1978; Egardt, 1979, 1980: Landau, 1979b; Gawthrop, 1977; Johnson, 1979). Model reference adaptive controllers (MRAC) for deterministic systems using an explicit reference model (see Fig. 1) can be equivalent to adaptive control schemes using an adaptive predictor as an intermediate step for building an adaptive control (Ljung and Landau; 1978; Narendra and Valavani, 1978; Silveira, 1978; Landau, 1979b). The first condition which should be satisfied in order that the two schemes be equivalent is that the control strategy be such that the output of predictor should behave identically to that of the explicit reference model (i.e. the adaptive predictor plus the controller form an implicit reference model; see Fig. 2). The stochastic self-tuniog regulators (S-STURE) where the objectives are specified in terms of ARMA models have a structure similar to that of the implicit model reference adaptive controllers (Ljung and Landau, 1978; Egardt, 1980; Landau, 1979b; Gawthrop, 1977). Minimum variance STURE is a particular form of SSTURE considered above (Landau, 1979b). From the equivalence of implicit and explicit model reference adaptive control, stochastic self-tuning regulators with explicit prediction models can be defined. They are equivalent to the known stochastic self-tuning regulators (with implicit prediction models) and feature structural similarities with explicit model reference adaptive control schemes (Landau, 1979b, c).
Adaptation mechanism
Explicit -I
yM(k~)
1
I
-I
reference m°de' I
FiG. 1. Explicit model reference adaptive control scheme.
• *Received 11 February 1980; revised 16 September 1980; revised 31 March 1981. The original version of this paper was presented at the 8th IFAC Congress which was held in Kyoto, Japan during August 1981. This paper was recommended for publicationin revised form by associate editor K. J. Astr6m. t'Laboratoire d'Automatique de Grenoble (C.N.R.S.), E.N.S.I.E.G.. B.P. 46, 38 402, St. Martin d'Heres, France.
• Tmphcif reference model
+!r
,<->,
-O
FIG. 2. Implicit model reference adaptive control scheme. 77
78
Brief Paper
The paper is organized as follows: Section 2; the design of a class of linear controllers for tracking and regulation in a deterministic and stochastic environment is briefly reviewed. This will provide better understanding of the similarities between MRAC and S-STURE. Section 3 presents MRAC and S-STURE schemes which are extensions of the schemes given in Section 2 for the case when the parameters of the plant are unknown. In Section 4 the structural similarities between S-STURE and MRAC and their behaviour in a combined environment are examined. In Section 5, a scheme which behaves as a MRAC in a deterministic environment and as a S-STURE in a stochastic environment is presented. In Section 6, the problem of removing the positivity condition required for convergence of these schemes is briefly discussed. In order to reduce the technical details and to make the presentation clearer, the plant to be controlled in the absence of disturbances is assumed to be described by a discrete rational transfer function with a basic delay of one sample, throughout the paper. All the results can be extended for the case of a general delay.
2. Combined tracking and regulation for plant with known parameters 2.1 Deterministic environment. The plant to be controlled is described by
Compute the control as follows:
u(k) = C2(q -I )y M(k + 1)-R(q-t)y(k) - B*(q-t)u(k) b0 = C2.(q-t)yM(k + I) - p~4po(k) (lO) bo where
R(q -I) =
n C2(q -I) - A ( q -I) = ~ (c~ - oa)q -i*l
= rl+ r:q-t...+ r~q-"+l
(11)
B * ( q - ' ) = B(O -t) - bo
(12)
pot = [bl...b,~, rl...r,]
(13)
4h(k) = [u(k - 1)...u(k - m), y(k)...y(k - n + 1)]. 04) Introducing (10) into (1) and defining the plant-model error e(k) = y(k) - y ~ ( k )
(15)
one obtains A(q-~)y(k + 1) = B(q-~)u(k); y(0) # 0
(1)
C2(q-t),(k + I) = O. where y(k) is the measured process output u(k) is the control, q-t is the backward shift operator and A(q -~)= 1 +a~q -~+. . . + a,q-"
B(q-t)=bo+b~q-t+...+b,~q -=
bo~O.
(2)
Ct(q-I)y(k) = q-JD(q-l)uM(k)
(4)
C,(q - l ) = l + c { q - l + . . . + c~q -~
(5)
where
is an asymptotically stable polynomial D(q -~) = do + dlq -~ + . . . + &q-in
Vk ~, 0
C2(q-~)yU(R + 1) = p Tcb(k)
(17)
o r = [be, por]
(18)
4,(k) = [u(k), <~,(k)]
(19)
where
(b) Strategy 2: implicit reference model This strategy is directly inspired by the separation theorem: one first designs an appropriate predictor for the plant output an~ then a control will be computed such that the output of the predictor behaves as the desired output in tracki~. Predictor design C2(q-I)y(k + 1) = p r cb(k) = bou(k)+ R(q-l)y(k)
(6)
and a~(k) is a bounded reference sequence. Rexulation: The control should be such that, in regulation, [u" (k) m O] an initial disturbance [y(O) ~ O] is eliminated with the dynamics defined by CAq-~)y(k + 1) = 0
(7)
where
+ B*(q-~)u(k)
~(k + 1) = y(k + 1)- 9(k + 1)
(21)
one notes that it satisfies
(22)
(8)
is an asymptotically stable polynomial. The solution for this control problem can be obtained in two ways: either by using an explicit reference model as part of the control system or by using a one-step ahead predictor of the plant output which together with the controller will form an implicit reference model. (a) Strategy 1: explicit reference model Define the explicit reference model
Cffq-')yU(k + 1) = D(q-I)um(k)
(20)
where R(¢ -I) , B*(q-I), p r ¢~(k) are given by (11), (12), (18) and (19), respectively. Defining the prediction error as
C2(q-l)~(k + 1) = O, k ~ O. C2(q-') = 1 + c~¢ -1 + . . . + c:,q-"
(16)
and one concludes that both control objectives are achieved. Note that the control given by (10) can be expressed in the equivalent form
(3)
The polynomial B(z -~) is assumed to have all zeroes in Izl < 1 (z-complex variable) therefore they can be cancelled without leading to an unbounded control. The objectives of the control which we are considering in the deterministic environment are given below. Tracking: The control u(k) should be such that, in tracking, the output of the process satisfies the difference equation
k~ 0
(9)
where yU(k) is the output of the explicit reference model.
Computation of the control. The control is computed such that ~(k + 1) = yU(k + l) which leads to C2(q-*)9(k + 1) = C2(q-t)yM(k + 1) = p ro(k)
(23)
and the control is given by (10) as expected. The resulting control schemes can be interpreted as a pole placement by feedback combined with a feedforward control of the dynamics and an introduction by feedforward of new dynamics. The feedforward control will cancel the closed loop dynamics and will introduce the desired dynamics for tracking (Landau and Lozano, 1981).
Brief Paper
2.2 Stochastic environment. The plant and the disturbance are described by • A(q - I )y(k + 1) = B(q-~)u(k) + C(q-~)~a(k + 1)
The design presented for deterministic and stochastic environment, respectively with the control objectives considered are dual in the sense that the same control provides a solution to two different problems.
(24)
where C(q -~) = 1 + cq -~ + . . . c . q - "
79
3. The case of unknown plant parameters 3.1 Deterministic environment. When the plant parameters are unknown it is possible to build adaptive control schemes which will use an explicit or an implicit reference model following the design methodology given in Landau and Lozano 0981); Lozano and Landau (1981).
(25)
is an asymptotically stable polynomial, to(k) is a sequence of equally distributed independent normal (0, ~) random variables, A(q -~) and B(q -~) are given by (2) and (3). The control objectives which are considered in the stochastic environment are generalizations of the minimum variance control strategy (they are specified in terms of ARMA models) and the solutions are obta/ned via an algebraic approach. For more details concerning the algebraic approach to linear stochastic control problems see Kucera (1975). The objectives of the control which we are considering in the stochastic environment are Regulation: The plant and the disturbance are given by (24); the control should be such that
( a ) Explicit model reference adaptive control In the design given in Section 2.1. a one replaces the fixed parameter vector in (17) by an adjustable parameter vector /~r(k) = [~e(k),/io(k)]
(34)
and the control law will be given by u ( k ) = C2(q-')YU(k + 1 ) - P°T(k)#°(k) /~o(k)
(35)
or equivalently by C2(q-~)y(k + 1) = C(q-~)to(k + I)
(26) T(k)tb(k) = C2(q-')yU(k + I).
where C2(q -~) defines in fact the poles of the closed loop system. For C2(q-m)= C(q -~) the control objective corresponds to the minimum variance control (Astr6m and Wittenmark, 1973). Tracking: Given a reference trajectory defined by (9) the control u(k) should be such that
One defines the objectives of the adaptive control system as limk.=C:(q-t)E(k + 1) = fim~e°(k + 1) = 0
(27)
For C2(q -~) = C(q -~) the control objective corresponds to minimum variance tracking of the reference trajectory in the presence of a stochastic disturbance. The appropriate control satisfying both objectives can be obtained by using an explicit or implicit prediction reference model.
.
~b(k)[I ~ M < =, for all k.
O(k + 1) =/~(k) + Frcb(k)v(k + 1)
F i ~ , = A z(k)F 7cI + Az(k)da(kNa(k) r
F0>O; 0 < k~(k) ~ 1; 0 ~ k2(k)<2
C1(q-l)~:U(k + 1) = C~(q-l)yU(k + l) -- D(q-t)uU(k). (28)
!-/,( -i ) v(k + 1) = ~ [p - l~(k + l)]r~(k) (29)
where p0 and tb0(k) are given in (13) and (14). Defining the plant-model error as in (15) one obtains using (29) that
(41)
where H~(q-') and H2(q -I) are monic polynomials on q-' (q-Lshifting operator) and v(k + 1) is obtained as follows: ,,
H}(q-')
v(k + =j = ~ e
(30)
,.. tK + 1) = H2(q- )
x (e°(k + 1) + i~(k + 1)]
which is the desired result.
(42)
where
(b) Implicit prediction reference model
e*(k + l) = [p -/~(k + l)]T~b(k) i(k + I) = [~(k) - p(k + l ) ] ~ ( k )
Predictor design (31)
where p and ~(k) are given in (18) and (19). Note that the prediction error defined as in (22) satisfies (32)
Computation of the control. The control is computed such that ~(k + 1) = yU(k + 1) which leads to C:(q-;)~(k + 1) = C2(q-')yU(k + 1) = pT4,(k)
(40)
and v(k + 1) is the a posteriori adaptation error (residual) which satisifies the equation
Compute the control as follows:
C2(q-I)~.(k + 1) = C(q-J)to(k + 1).
(39)
where FK is th adaptation gain matrix given in general by
Define the prediction reference model
C2(q-J)~(k + 1) - pr~b(k)
(38)
Using a stability type design (Landau and Lozano, 1981) one obtains the following adaptation algorithm
(a) Explicit prediction reference model
C : ( q - = ) e ( k + I) = C(q-I)to(k + l)
(37)
where e°(k + l) = C2(q")e(k + 1) and
C2(q-~)ytk + I) = C2(q-~)yU(k + 1)+ C(q-~)w(k + I).
C2(q-~)yU(k + l) - pToc~o(k) u(k) = bo
(36)
(33)
(43)
eO(k + I) = [p -/~(k)]r~b(k) ~(k + I) is a transient adaptation signal also called the'anxiliary error' (Ionesfu and Monopoli, 1977). The objectives of (37) and (38) are achieved if H,(z -~) ~, H2---~'~-2; 2 > x ~maxX~(k), is strictly positive real.
(44)
80
Brief Paper
Note that v(k + 1) can always be expressed in terms of e°(k + 1) which is a measurable quantity and the previous values of v(k + 1). If we assume (as for stochastic self-tuning regulators) that the parameters of the process are unknown but constant, decreasing adaptation gains can be used and (40) becomes
F;~t=F-~'+Xd~(k)da(k)r;
Fo>O;
0
(45)
For the purpose of comparison with two types of S-STURE two prototype schemes which are particular forms of the general design indicated above are considered.
3.2. Stochastic environment. When the plant parameters and the disturbance model are unknown, the controller parameter p in the designs given in Section 2.2 (with explicit or implicit prediction reference model) should be replaced by an adjustable parameter vector. Then an appropriate adaptation algorithm should be defined such that the control objectives in the linear case with known parameters be asymptotically achieved. This can be expressed in general by the condition that a certain filtered error becomes asymptotically a white sequence. For the objective considered in Section 2.2 this can be expressed as Prob { ~ ! mC2(-~) ~,(k
Prototype 1 [Ht(q -1) = 1, H~,(q-t) = C:(q-l)] 1
v(k + 1) = ~
L..21,q
)
e*(k + I),
(46)
For this case the explicit expression for computation of
v(k +'1) is E(k + l ) -
v(k + 1) =
(47)
where
(52)
Given a certain adaptation algorithm the asymptotic properties of the scheme can be examined using the ODE method. Theorem A.I given in the Appendix which is derived from ODE method allows a straightforward analysis of these schemes. For the purpose of this paper we will consider two types of objectives leading to two prototypes schemes
~ c ~ ( k + 1 - i)
I + ~(k)'Fx~(k)
+ 1) = ca(k + 1)} = 1.
(1)
Problime(k+l)=oJ(k+l)}=l;C(q-~)~l. k--*ffi
(2)
Problim{Cz(q-1)e(k + 1) = oJ(k + 1)} = 1; C(q -~) = I.
(53)
k~
(54) ~(k + 1) = C ~ q - )[lO(k) - lO(k + l)]T~(k)
(48)
The first objective corresponds in regulation to minimum variance self-tuning control [i.e. y ( k ) ~ oJ(k)] and the second objective corresponds in regulation to C:(q-~)y(k)-~cu(k) for C(q -~) = 1. Both objectives are ARMA type control objectives. The corresponding adaptation algorithms will be
and e(k + i) is given by (15). One also has v(k + I) = E(k + l)+~(k + I).
The stability of the scheme is assured if 1
Prototype 1 (minimum variance STURE)
A
C:(z-'~-) 2
(49)
is a strictly positive real transfer function (which also implies that lirn $(k) = 0).
p(k + 1) - lO(k)+ FKfb(k)v(k + 1)
(55)
e(k + l) v(k ÷ 1) = 1 + ~b(k)rF~c4~(K)
(56)
F~.~=F~+hd~(k)tb(k)r;Fo>O;O
(57)
where
k--,ffi
Prototype2 and [Ht(q -l) = bI2(q -I) = 1]
v(k + l) = e*(k + 1) = Cz(q-~)e(k + 1) 1 + ~(k)rF~(k)"
(50)
For this case, stability is assured without any positive real condition to be satisfied. (b) Implicit model reference adaptive control The implicit model reference adaptive control results immediately from the linear design in which the linear predictor is replaced by an adaptive predictor
C,.(q-~)~(k + 1) = Or(k)~(k)
(51)
and designing the control such that 9(k+ 1)= yM(k + 1) which leads to the control given by (35). If for updating the adjustable parameter if(k), one uses the same algorithm as given by (39) and (40) where the plant-model error which enters in the computation of v(k + 1) is replaced by the prediction error, the two schemes are equivalent (Lozano and Landau, 1981). One can state the conditions for this equivalence in general as follows: De]inition 3.1. (Equivalence between explicit and implicit MRAC) An explicit MRAC and an implicit MRAC are equivalent if and only if (1) The equations for the a posteriori adaptation error v(k) ate identical. (2) The parametric adaptation algorithms are identical. As a consequence: the positivity conditions for asymptotic stability are identical.
Applying Theorem A.1 or using the results given in Ljung (1977) one concludes that the objective of (53) in regulation will he achieved ff 1
1
C(z-7~)- ~
(ss)
will be strictly positive real.
Prototype 2 l~(k) is updated using (55) with F(k) given by (57) and v(k + 1) is given by Cz(q-t)e(k + 1) v(k + 1) -- 1 + 4~(k)rFKcb(K)"
(59)
Using Theorem A.1 one concludes that the objectives of (54) in regulation will be achieved without any positive real condition to be satisfied. The important remark to be made is that the S-STURE can be implemented either using an explicit or an implicit prediction model, the two implementation being equivalent [only the significance of E(k + 1) differs]. Similar to MRAC, conditions for equivalence of implicit and explicit S-STURE can be given, Landau (1979b, c). Note that in the STURE litterature only the implicit type implementation has been mostly considered.
Brief Paper
81
4. Structural similarities and analysis of S-STURE and MRAC in a combined environment Consider the prototype schemes 1 and 2 for MRAC (with implicit or explicit reference model) when using the gain updating given in (45) and the prototype schemes 1 and 2 for S-STURE (with implicit or explicit prediction reference model) respectively. One observes that the prototype schemes 1 for MRAC and S-STURE, respectively use the same control law, the adaptation algorithms have the same structure, the convergence conditions are of the same type; the only difference occurs for the a posteriori adaptation error during transients [compare (47) and (56) since [ ( k ) - , 0 as k--,®]. A similar situation holds also if one compares the prototype schemes 2 for MRAC and S-STURE with the remark that in this case the a posteriori adaptation errors are identical for both schemes [compare (50) and (59)]. However despite these similarities (structural equivalence) the MRAC and S-STURE achieve different tasks, which justifies the introduction of the following definition.
De/inition 4.1. (Asymptotic) Duality between MRAC and S-STURE (Landau, 1979b, c). A MRAC (implicit or explicit) designed for a deterministic environment is (asymptotically) dual with respect to a S-STURE (implicit or explicit) designed for a stochastic environment if and only if (1) The adjustable parameter vectors are updated by (asymptotically) identical adaptation algorithms [same structure, same observation vector (~b) same a posteriori adaptation error (as k -, ®)]. (2) The positivity conditions for the global asymptotic stability of the MRAC and for w.p.! convergence of the S-STURE are the same. (3) The control laws are (asymptotically) the same. Remark: If both control laws and a posteriori adaptation error are identical for any k, they will he called dual. After emphasizing the similarities between MRAC and S-STURE the basic questions to which this paper wants to ask and answer come in view. They are (1) What is the behaviour of (asymptotic) dual MRAC and S-STURE in a stochastic and deterministic environment, respectively? (2) Can a scheme be designed which behaves as a desired MRAC in a deterministic environment and as a desired S-STURE in a stochastic environment? We will limit ourselves in this section to answer the first question in regulation [ u S ( k ) - 0]. Consider the prototype 1 for MRAC. The analysis in a stochastic environmement can be made using Theorem A. 1. The plant in this case will be described by (24). For the particular case C:(q -~) = C(q -~) a straightforward analysis shows that because of duality this scheme will behave as the prototype 1 of S-STURE (minimum variance STURE) [the transients terms in (47) depending on # ( k ) /~(k- 1) disappear when using ODE method since 10(k)-# ( k - 1 ) = / ~ ] . Therefore we will consider next the case C~(q-*)# C(q -~) and we will sketch briefly the analysis using Theorem A.I given in the appendix. In the presence of stochastic disturbance the a posteriori adaptation error v(k + 1) for prototype I considered in Section 3 satisfies the equation
+ C(q'%,(k
+ l) = [ p * - p l Y ( k ,
#)
+ C(q-~)to(k + 1)
(62)
where [p*] = [b0...bin, cl-
at . . . .
c. - a.]
(63)
corresponds to the controller parameter vector assuring that the closed loop poles are defined by C(q -~) instead of C2(q-I). Equation (62) can be re-written as
~(k+l,/~)=C---~[
p -~]
~(k,/~)+totk+i)
(64)
and applying Theorem A.I one concludes that e(k + l)--* o ( k + l ) and #(k) will converge to p® such that [ p * p.~]~0(k) = 0 where p* corresponds to the closed loop poles defined by C(q -~) if
C(q=r)
2
(65)
is a strictly positive real transfer function. The convergence point and the convergence condition are identical to that of the dual S-STURE (in this case the minimum yariance STURE). This is also true for the prototYpe scheme 2 and in general for dual MRAC and S-STURE. One has then the following important conclusion (for regulation).
Theorem 4.1. A MRAC in the presence of a stochastic disturbance of the same structure as that consi~lered for its dual S-STURE will behave asymptotically as its dual SSTURE (in regulation). Consider now the prototype I S-STURE in a deterministic environment [ w ( k ) = 0, c~ ---0, i = i,, . . . n]. One immediately observes that the algorithm is identical to the prototype I MRAC for C,(q -~) = 1 and one concludes that the scheme will be globally asymptotically stable (the condition of (49) is automatically satisfied). However, the disadvantage of the S-STURE is such a situation is that it will drive all the Closed loop poles to z = 0 and this means a one step response which, depending on the sampling per!od, can become in many cases undesirable because of the magnitude of the resulting control.. Simulations have shown also that large transients occur in this case during adaptation process (Landau and Lozano, 1981). Therefore in a certain way, the desired performance of S-STURE in a deterministic environment should he specified by the designer without modifying the behaviour of the S-STURE in a stochastic environment. This can be obtained by replacing the S-STURE by its dual MRAC. On the other hand since the poles of the closed loop will in general depend on the characteristics of the stochastic disturbance it will be necessary for tracking purposes that the control law (35) be modified in order to take in account the drift of the closed loop poles. This will be discussed in the next section. 5. A combined M R A C - S - T U R E scheme
C2(q-~)v(k + 1) = [# - #(k + l)]T<~(k) + C(q-')oJ(k + 1). (60) Defining now the stationary sequences. ~(k+ 1, p), O(k,l~) for l~(k) =/~ and taking in account that uU(k)= 0 one has
~(k + 1, p) = t(k + l, p) -- y(k + l, #).
(6l)
We will give next a scheme which illustrates how an adaptive scheme behaving as a MRAC in a deterministic environment and a S-STURE in a stochastic environment can be obtained for the joint tracking-regulation problem. The implementation can be done using an implicit or explicit form. In the designs given in Section 2.1 the control law is replaced by
u(k)
One obtains then from (60) for #(k) = # the following equation
C~(q-I)YU(k + l)-#0~(k)T~b0,(k) =
(66)
bo(k)
where
C(q-')[(k + 1, p ) = [p - P]r~(k, .~) n
- ,~. (c~ - c , ) ~ ( k + 1 - i, ~)
AtJT
18:1
-
/~o¢(k) '= [~o(k) T, c(k) T ]
(67)
4~ore(k) = [tboT(d), y U ( k ) . . . y U ( k + 1 - n)].
(68)
82
Brief P a p e r
The vector ~(k) is formed by the estimates of the coefficients of the polynomial [C2(q -~) - C(q-~)]. The ~_4a_pmtionalgorithm is given by
/~.(k+ I) =
:.(k) + Fx~b,,(k)v(k +
1)
(69)
where
0,(k) r = [6o(k), .~o.(k) r]
(70)
,t,~(k) = [u(k), ,~T(k)]
(71)
Theanalysis of this scheme in the deterministic environment is straightforward and it is omitted [consider (69) and (73) and apply the results of Landau and Lozano (198t) which yields the condition of (75)]. The analysis in a stochastic environment using Theorem A.I is clone as following: from (24) one obtains C(q-1)y(k ~- 1) = p*r~(k) + C(q-~)oJ(k + 1)
(77)
p.T = [bo, p~r]
(78)
where
F~L, = F i ~+ X6,(k), ~bT(k); F0>0; 0<~, < 2 u(k + 1) = ~
1
(72)
is given by (63). Defining now the stationary sequences
e* (k + 1)
~(k + 1, p,), ~(k, ~.) for p,(k) = p,
e(k + I ) - ~ d ~ ( k + t - i)
and taking in account that
I.I
I + ~b.(k)rFx~.(k) =
~~ z[t qp , /
-
#(k + 1,/~,) = i(k + 1,/~)
p.(k + 1)]T~k,(k)
(79)
(73) one obtains
C(q-t)~(k + 1, ~,) = C(q-')i.(k + 1, ~0,) -- C(q-~)[y(k + 1, .0,)
and
- yU(k + 1)] = p~r,~,o(k, ~,) + floe(k, ~,)
~(k + 1) = C~---~.p_ ,(k)- p,(k + 1)]~',b(k)
(74)
The block diagram corresponding to this adaptive control system is 8ivan in Fig. 3. The above scheme'has the following properties:
+ (bo- bo)~(k, ~ , ) - C(q-~)y.(k + 1) + C(q-~)~(k + 1). (80) Introducing now the control given by (66), for p,(k) = p one finally obtains
(I) In a datmmlnhfic environment the objectives for the linear duign (tracking and regulation) are asymptotically achieved if
~ ( k + l , p , ) = ~ ( k + 1 , p.)= C-~'~'~[p. I . - Po,] r
~,(kl,t~,)+ o~(k+ 1)
C2( z-"~
2
(75)
is strictly positive real. (2) In a stochastic environnmnt, the minimum variance t r a c k / ~ of the reference trajectory is asymptotically achlavnd (w.p.1) if
2"
(76)
(81)
where p*, = [p*, c~- c~ ..... c2.- c.].
(82)
Applying now Theorem A. I, the convergence condition (76) is obtained. Generalizations of this scheme for delays greater than one and various types of stochastic disturbances as well as simulations can be found in Dullard and Landau (1980) and D u p r d , Egardt and Landau (1980).
.....
it,,°,
I
1
~(k) FIo. 3. Adaptive control scheme for tracking and regulation in a combined environment.
Brief Paper 6. The positivity problem The global asymptotic stability of the prototype 1 MRAC can be assured if the transfer function given in (49) is strictly positive real. This condition is restrictive since it limits drastically the region of the allowed poles in the z-domain. In the MRAC designs two solutions can be considered in order to overcome this problem (Landau and Lozano, 1981). (1) Introduction of linear compensators acting on the plant model error. (2) Introduction of appropriate filters for generating the observation vector ~(k). Note that some of the filters introduced on the plant model error have an effect equivalent to the change of the reference model structure (Landau and Lozano, 1981). The elimination of the positive real condition is illustrated in prototype 2 MRAC where the plant-model error is passed through the linear compensator defined by Cz(q-~), see (50). However, if one consider the prototype 2, S-STURE which is dual to prototype 2 MRAC, one observes that the positive real condition for convergence has been removed (see Section 3.2) but in the meantime the nature of the disturbance and the control objective have been changed with respect to prototype 1 S-STURE. This type of observation made also on other prototype schemes leads to the conclusion that positive real condition can be removed in the deterministic context while maintaining the control objective, but in the stochastic environment, removing of the positive real condition implies a modification of the control problem (different type of disturbance and/or different control objective). 7. Conclusions In the context of minimum phase plants the similarities between MRAC where the control objectives are specified by reference models and S-STURE where the objectives are specified by ARMA models have been investigated both from structural and theoretical (convergence analysis) point of view. It was shown that MRAC and S-STURE feature a duality character which extends the duality existing in the linear case with known parameters between linear model following control and stochastic control w/th ARMA type objective. As a result of this investigation, a method for constructing adaptive control schemes which behaves as a given MRAC in a deterministic environment and as a given S-STURE in a stochastic environment is given and illustrated by an example. The convergence analysis has shown the importance of the positivity conditions and of the transient adaptation signals which are specific to MRAC design. The positivity condition which appears as convergence condition for both MRAC and S-STURE can always be removed in a deterministic context without modifying the control objectives while in a stochastic context removing the positivity conditions leads to a change of the nature of the disturbance and/or of the control objectives. The transient adaptation signals play a crucial role for obtaining good performances from the combined MRAC S-STURE in a deterministic environment and in some situations can improve the performances in the stochastic case with respect to a S-STURE.
References ~strOm, K. J. and B. Wittenmark (1973). On self-tuning Aregulators. Automatica, 9, 185. strtm, K. J., V. Borisson, L. Ljung and B. Wittenmark (1977). Theory and applications of self-tuning regulators. Automatica, 13, 457. AstrOm, K. L and B. Wittenmark (1980). Self-tuning controllers based on pole-zero placement. Proc. IEE, 127, 120. Dugard, L. and I. D. Landau (1980a). Recursive output error identification algorithms theory and evaluation. Automatica, 16, 443.
s3
Dugard L., I. D. Landau and H. M. Silveira (1980b). Adaptative state estimation using M.R.A.S techniques convergence analysis and evaluation. IEEE Trans Aut. Control, AC-25, 1169. Dugard, L. and I. D. Landau (1980c). Stochastic model reference adaptive controllers. Proc. 19th IEEE CDC Conference, Alburquerque, U.S.A. Dugard, L., Egardt B. and Landau I. D. (1980). Design and convergence analysis of stochastic model reference adaptive controllers. IEEE Trans. Aut. Control (to be published). Egardt, B. (1979). Unification of some continuous time adaptive control schemes. IEEE Trans Aut. Control. AC24, 588. Egardt, B. (1980). Unification of some discrete time adaptive control schemes. IEEE Trans. Aut. Control, AC-2& 693. Gawthrop, P. J. (1977). Some interpretations of the selftuning controller. Proc IEEE, 124, 889. Goodwin, G., P. Ramadge and P. Caines (1980). Discrete time multivariable adaptive control. IEEE Trans Aut. Control. AC-25, 449. Ionescu, I. and R. Monopoli (1977). Discrete model reference adaptive control with an augmented error signal. Automatica, 12, 507. Johnson, C. R. (1979). Input matching, error augmentation, self-tuning and output error identification: algorithm in discrete adaptive model following. Proc. 18th IEEE-CDC Conference, Fort Lauderdale, U.S.A. Kucera, V. (1975). Algebraic approach to discrete stochastic control. Kybernetica, 11. Landau, I. E. (1979a). Adaptive Control, The Model Reference Approach. Dekker, New York. Landau, I. D. (1979b). Adaptive controllers with explicit and implicit reference models and stochastic self-tuning regulators--equivalence and duality aspects. Proc. 17th IEEE-CDC Conference, San Diego, U,S.A. Landau, I. D. (1979c). Model reference adaptive control and stochastic self-tuning regulators--towards cross fertilization. Proc. of AFOSR Workshop: Adaptive Control. Champaign, U.S.A. Landau, I. D. and R. Lozano (1981). Unification and evaluation of discrete time explicit model reference adaptive control designs. Automatica, 17, 593. Ljung, L. (1977). On positive real transfer functions and the convergence of some recursive schemes. IEEE Trans Aut. Cont, AC-22, 539. Ljung, L. (1978). Convergence of a self-tuning regulator for systems with general delay. Rport LITH-ISY-I-0252, Linktping University. Ljung, L. and I. D. Landau (1978). Model reference adaptive systems and self-tuning regulators--some connections. Proc. 7th IFAC Congress, Vol. 3, pp. 1973-1980. Lozano, R. and I. D. Landau (1981). Redesign of explicit and implicit discrete time model reference adaptive control schemes. Int. J. Control, 33. Narendra, K. S. and L. S. Valavani (1978). Direct and indirect adaptive control. Proc. 7th IFAC Congress, Vol. 3, pp. 1981-1988. Silveira, H. M. (1978). Contributions ~ la synthtse des systtmes adaptatifs avec mod/.~iesans accts aux variables d'~tat. Thtse es-Sciences Physiques I.N.P.G., Grenoble. Wellstead, P. E. and P. Zanker (1979). Servo Self-Tuners. Int. J. Control 30, 27. Appendix: a theorem for convergence analysis in stochastic environment Consider the parametric adaption algorithm /5(k + 1) = ~(k) + Frcb(k)v(k + 1)
(AI)
F[]+l=F~L~-Ack(k)ck(k)r: F0>0; 0 < k <2.
(A2)
where
Assume that the following relation exists between v(k + 1) and ~b(k) v(k + 1) = H(q-~)[p - IJ(k + l)]r~b(k) + w(k + 1)
(A3)
84
Brief Paper
where ~(k + 1) is the immage of a stochastic disturbance in the equation governing the a posteriori adaptation error. One has then the following result.
where
Theorem A.I. Consider the adaptation algorithm of (A1) and (A2). Assume that stationary processes $(k, p) and ~(k + l, 1~) can be defined for I~(k) = p and that i~(k) belongs infinitely often to the domain for which these stationary processes can be defined. Assume in addition that for 10(k) =
and
U(k ÷ I, p) = H(q-I)~pT(k, ~)[~*- f)l÷ ¢o(k + I) (A4) where {oJ(k+l)} is a white random sequence [or uncorrelated with @(k, i~)]Then if A H ' ( z -~) = H (z -~) --~
(A5)
is strictly positive real Prob{~m~(k)~Dc}ffi l
(A6)
o~ :{Pl[p* - p ] % ( k ) = o)
(A7)
Prob{~!m v(k -~ I} = c~(k + l)}= I.
(A8)
The proof of this theorem can be found in Dugaxd and Landau (1980) or Dugard, Landau and Silveira(1980) and is ' based on the application of O D E method Ljung (1977) to (AI)-(A3). Remark: The application of the O D E method for the convergence analysis assume that 0(k) belongs infinitelyoften to the domain for which the stationary processes ~,(k+ l, ~), ~(k, i~)can be defined. For this reason the resultsin the case of adaptive control schemes have a local validityaround the possible convergence points of the algorithms.