- Email: [email protected]
Optimal control of nonlinear time-lag systems
Automatica, Vol. 8, pp. 793-795. Pergamon Press, 1972. Printed in Great Britain.
Correspondence Item Optimal Control of Nonlinear Time-Lag Systems* Contr61e Optimal de Syst~mes Non-Lin6aires ~. Dalai de Temps Optimale Kontrolle Nichtlinearer Zeitlicher Nacheilungssysteme OrlTHMa.rIbHbI~ XOHTpO3-IbaearlHefinbbx CHCTeMOTCTaBamta rio 8peMeaH M. A. C O N N O R I "
Let
Summary--This note considers the optimal control of a system represented by a nonlinear differential-difference equation with a general cost function. A second order iterative method of solution based on the fixed point contraction mapping principle is proposed.
Q(x(t:), tf) = S + d/rM~ where M is an appropriate positive definite weighting matrix. We now consider the modified cost function,
Introduction THE ov-rit,~L control problem for a system represented by differential-difference equations has been studied by many authors. However, most authors direct their attention to deriving necessary conditions for optimality and devote little effort to obtaining a computational algorithm for solving these equations. Exceptions to this generalisation are SEBESTA[1], who develops a gradient method, and MACK1Nr~ON[2] who presents a second variation technique. The method proposed in [2] is based on the approximation of a functional relationship by a finite series; this approach has its obvious drawbacks. This note proposes a second variation method coupled with a functional analytic treatment similar to the one proposed by FR~r,IAN [3] for a linear system without time lag. Second variation. We shall consider the following problem: minimise the functional,
J = f l s F(x(t), u(t), t)dt + S(x(tf), tf)
O)
(4)
+ ~,rM~b.
For convenience, we shall denote x(t-O) by xo(t). Then
J = Q(x(t:), t:)+
f
t! o F(x, u, t)
+ ; t r { f ( x , xo, u, t ) - ~ } d t .
(5)
Taking variations ~(t), G0(t), r/(t)in x(t), xo(t), u(t)respectively, and writing H=F+2Tt, gives, to second order in the variations,
+ ½< Qx~(t/)~(ts), ~(t:) > "tt +dt- <2(t:), ~(t:)>
:c =f(x(t), x ( t - 0 ) , u(t), t), x(t) k n o w n for
(2) ~b(x(t:), t f ) = 0
o 2r(t){f(x(t)' x(t-O), u(t), t ) - ~ } d t
AJ = < O.x(t:), ~(t:) >
subject to the constraints,
- o ~ t< o
~t! 3=J
o
°to
(3)
+
where
x(t) is an n-vector
+
o bt¢ o
< H,,(t) +,~(t), ~(t) > dtdt
u(t) is an m-vector +
Hxx(t)~(t), ~(t) >
is a q-vector, q<<.n
+ < Heo(t)~o(t), Go(t) >
t! is assumed fixed.
+ < H,.(0,7(t), ,l(t)> * Received 27 January 1972; revised 9 May 1972. The original version of this note was not presented at any IFAC meeting. It was recommended for publication in revised form by Associate Editor A. Wierzbicki. ? Department of Mathematics, University of Technology, Loughborough, Leicestershire, England.
+ 2 < H~x(t)~(t), tl(t) > + 2 < H~(t)~o(t), tl(t) > dt + 2 < H~o(t)~o(t), ~(t) > }dt 793
(6)
794
Correspondence item where Qz~(ts), Hu(t), Hx~(t), etc. are evaluated along the nominal (xo, 2o, uo) solution. We now seek to minimise (11) subject to the constraints,
where 02H
t32H
H xo = ~--7-~ , cX~Xo
H"°-~U~Xo' etc.
~( t) =.fx~( t) + A~( t - O) +f~q(t),
We also have the following identity:
f
t, < Ho(t), ~o(t) > dt
~ ( t ) = 0 for - 0 ~ < t ~ < 0
(7)
(12)
0
=
where again, fx, fo, fu are evaluated along the nominal trajectory. It can be shown [4] that the solution of (12) can be written in the following form:
d t -o
+ f t f - o d t .
~(t)=
Using the condition ~(t)=0, te[--O, 0] eliminates the first term on the R.H.S. of (7). Using (7), the 5th and 6th terms on the R.H.S. of (6) can be combined into the following form: tY -- 0
fo
+
[" ,It!- 0
We now define 2(0 such that the following relationships hold:
)t(t) = - Hx(t),
0 <~t <~tf-- 0
tf -- 0 <~t <~tf
A(tf)=Qx(tf).
(8) (9)
If we now assume a nominal control law uo(t), equation (2) can be solved in forward time to give xo(t), and equations (8)-(10) can be solved in backward time to give AdO. In general, these solutions will not satisfy condition (3), nor will they satisfy the usual optimality condition Hu(t)=O. Consequently it is necessary to devise an iterative technique which will converge to the optimal solution. This problem is studied in the next section. A fixedpoint method. We now proceed along similar lines to that of FREEMAN[3]. In order to save space the reader is assumed to be familiar with the Hiibert space framework adopted in [3]. Using the assumed relationships (8)-(10) in (6) gives:
+
f',
,io
fi-'
N(a, t - O)f~(a)q(a)da,
~(t-- O) =
fo,O
t - O)f.(a)q(a)da,
t>~O.
(14)
~(ts) >
t t> 0, (15)
where ~r(a, t--O)=N(a, t-O) for O<~a<~t--O and ~(a, t - - 0 ) = 0 for a>t--O. Following Freeman's approach, we now write (13) and (15) in the form, ~(t) = L q
(16)
~o(t)= Lq
(17)
(10)
AJ = ½< Qxx(tf)~(tf),
(13)
Equation (14) can be written in the following equivalent form:
d t .
,~(t) = - H x ( t ) - Ho(t + 0),
N(a, t)f~(o)~l(a)da o
where the matrix N(a, t) is the solution of a certain differential equation. From (13) we have
~ ( t - O) =
< {(H~(t) + Ho(t + O) + ~(t)}, ~(t) > dt
f
t
where L and £ are mappings defined on Hilbert spaces. It will be assumed that these mappings are such that they map the control Hilbert space into the state Hilbert space. Conditions necessary for this assumption to hold can be found as in [3]. We have previously noted that the nominal trajectory will not, in general, satisfy condition (3). Let the vector ~, have a specified change A~,. Then we have,
A~I = G(tf)~(tf)
(I 8)
where G(q) is the matrix
< H~(t), ~(t) > dt We now impose equation (18) as an additional constraint on the minimisation of (11). This is most easily done by adjoining the following term to (11):
+ < Hoo(t)¢o(t), Co(t) > ~["AI//-- G(tf)~(tf)] T W [ A ~ --
G(tf)~(tf)'[
+ where W is a suitable positive definite weighting matrix. Let
+ 2 < H,~,(t)~(t), tl(t) >
~(t) = Hxx(t ) + Qxx(tf)b(t- tf)
+ 2 < H~(t)~o(t), ~i(t) > + 2 < Hxo(t)~o(t), ~(t) > }dt
(11)
G(t) = G ( t f ) b ( t - - ts)
Correspondence item
795
Finally,
where $(t) is the Dirac function. The new problem is now that of minimising J*, where
o { < Hu(t), ~(0 > + ½< ~(O~(t), ~0) >
q* = n ~ t K - ½ H ~ I ( R + R*)r/*.
(20)
Equation (20) is the required integral equation for t/*, and can be written in the following form:
+½ +½ rl*=Cq* .
+ < Hxo(t)~o(t), ~(t) > -- < t~T(t)WA~k, ~(0 > + ½< dr(t)WG(t)~(t), ~(t) > } d t . Using (16) and (17), J* can be written in the form, 2 J * = I(R + Huu)q,r/l - 2]L*GrWA~k,~/[ +
2[Hu l
(19)
where
Assuming that C is a contraction operator then the contraction mapping principle can be invoked to provide an iterative solution to (20) as discussed in [3]. Necessary conditions for C to be a contraction operator can be derived as in [3]. Similar results can be found in [5], where an equation analogous to (20) is discussed and the existence and uniqueness of the solution is proven without invoking the contraction condition. This completes the proposed solution to the problem.
Conclusions
R = L*QL + L*Hoo£ + 2 H ~ L + 2H~0L + 2L*H~o£ + L *~jTWGL, 1", "1 denotes inner product in the control Hilbert space, and L*, L* denote the adjoint operators of L and L respectively. Let F/* be the optimum value for r/. We now give a small variation ~* to r/* and determine a necessary condition for optimality of r/*. 2(J* + As*) = [(R + H..)(q* + fl*), q* + 0"1 - 2 1 K , tl* +0"1
It is known that gradient methods tend to have slow convergence near the optimum. It is thought that a combination of the gradient method, and the above second variation technique might provide a more rapid overall convergence. Much remains to be done to test this assertion.
References [1] H. R. SEBESTA" Design and analysis of optimal control
[2] [3]
where [4] K = L*t~r W A S - H~. We then have 2A J * = [(R + Huu)q*, ~*1 + l(R + H . ) O * , ~t*l - 2 1 K , ~*l + I(R+Huu)O *, ~'1" To first order in the variation r~* this becomes,
2AJ*=I{(R+H..)+(R+H.u)*}rt *, 0*l-2l K, 0*l where (R+Huu)* is the adjoint operator for (R+Huu). Hence a first order condition for r/* to be the optimum is given by,
[5]
systems for dynamical processes with time leg. Ph.D. Dissertation, University of Texas, Austin (1966). D. MAcKINNON: Optimal control of systems with pure time delays using a variational programming approach. IEEE Trans. Aut. Control 255-262 (1967). E. A. FREEMAN: O n the optimization of linear, timeinvariant, muitivariable control systems using the contraction mapping principle. JOTM 3, 416-443 (1969). M. N. OGUZTORELI: Time Lag Control Systems, p. 91. Academic Press, New York (1966). A. MA~aaos: Optimum control of linear lime lag systems with quadratic performance indexes. Preprints of the IVth IFAC Congress, Vol. 7, Warsaw (1969).
R~umr---Dans cette notice, on considrre le contr61e optimal d'un systrme reprrsent6 par une 6quation de diffrrence diffrrentielle non-linraire avec une fonction g6n6rale de coCtt. On propose une mrthode de solution itrrative du second ordre fondre sur le principe de marquage en contraction de point fixe. Zusammenfassung--Dieser Bericht behandelt die optimale Kontrolle eines Systems, welches dutch eine nichtlineare Differential-Differenzgleichungmit einer allgemeinen Kostenfunktion dargestellt wird. Es wird ein Iterativverfahren zweiten Grades zur Lfsung auf Grund des Kontraktionkartographieprinzips ftir den festen Punkt vorgeschlagen.
{(R + H.,) + (R + H..)*}r/* = 2 K . Pe3mMe---B aTOl~ paroTe paccMaTpnBaeTc~ onTnMaJIbHbU~ Since Huu=Huu* this last expression can be written in the following form:
2H.Ji* = 2 K - (R + R*)q* .
KOHTpOJIb CHCTeMbI, Hpe~GTaBJIeHHOI~ ypaBHeHHeM HeY[HHell~IOl~ ~i~¢l~peI-ii~a.rlbHOl~ pa3Hl~/~I C o6mefl ~yI-iKt~e~ 3aTpaTH 3HeprHH. ~JLq pemermg npe~-laraeTc~ HTepaI~OHMeTOR BTOpOFO n o p g ~ a , OCHOBaHHI~i~ Ha IIpHHHHHe OTorpaxem~ 37~aTILq HeHO~IBH~KH01~TOtlKH.
Correspondence Item Optimal Control of Nonlinear Time-Lag Systems* Contr61e Optimal de Syst~mes Non-Lin6aires ~. Dalai de Temps Optimale Kontrolle Nichtlinearer Zeitlicher Nacheilungssysteme OrlTHMa.rIbHbI~ XOHTpO3-IbaearlHefinbbx CHCTeMOTCTaBamta rio 8peMeaH M. A. C O N N O R I "
Let
Summary--This note considers the optimal control of a system represented by a nonlinear differential-difference equation with a general cost function. A second order iterative method of solution based on the fixed point contraction mapping principle is proposed.
Q(x(t:), tf) = S + d/rM~ where M is an appropriate positive definite weighting matrix. We now consider the modified cost function,
Introduction THE ov-rit,~L control problem for a system represented by differential-difference equations has been studied by many authors. However, most authors direct their attention to deriving necessary conditions for optimality and devote little effort to obtaining a computational algorithm for solving these equations. Exceptions to this generalisation are SEBESTA[1], who develops a gradient method, and MACK1Nr~ON[2] who presents a second variation technique. The method proposed in [2] is based on the approximation of a functional relationship by a finite series; this approach has its obvious drawbacks. This note proposes a second variation method coupled with a functional analytic treatment similar to the one proposed by FR~r,IAN [3] for a linear system without time lag. Second variation. We shall consider the following problem: minimise the functional,
J = f l s F(x(t), u(t), t)dt + S(x(tf), tf)
O)
(4)
+ ~,rM~b.
For convenience, we shall denote x(t-O) by xo(t). Then
J = Q(x(t:), t:)+
f
t! o F(x, u, t)
+ ; t r { f ( x , xo, u, t ) - ~ } d t .
(5)
Taking variations ~(t), G0(t), r/(t)in x(t), xo(t), u(t)respectively, and writing H=F+2Tt, gives, to second order in the variations,
+ ½< Qx~(t/)~(ts), ~(t:) > "tt +
:c =f(x(t), x ( t - 0 ) , u(t), t), x(t) k n o w n for
(2) ~b(x(t:), t f ) = 0
o 2r(t){f(x(t)' x(t-O), u(t), t ) - ~ } d t
AJ = < O.x(t:), ~(t:) >
subject to the constraints,
- o ~ t< o
~t! 3=J
o
°to
(3)
+
where
x(t) is an n-vector
+
o bt¢ o
< H,,(t) +,~(t), ~(t) > dt
u(t) is an m-vector +
Hxx(t)~(t), ~(t) >
is a q-vector, q<<.n
+ < Heo(t)~o(t), Go(t) >
t! is assumed fixed.
+ < H,.(0,7(t), ,l(t)> * Received 27 January 1972; revised 9 May 1972. The original version of this note was not presented at any IFAC meeting. It was recommended for publication in revised form by Associate Editor A. Wierzbicki. ? Department of Mathematics, University of Technology, Loughborough, Leicestershire, England.
+ 2 < H~x(t)~(t), tl(t) > + 2 < H~(t)~o(t), tl(t) > dt + 2 < H~o(t)~o(t), ~(t) > }dt 793
(6)
794
Correspondence item where Qz~(ts), Hu(t), Hx~(t), etc. are evaluated along the nominal (xo, 2o, uo) solution. We now seek to minimise (11) subject to the constraints,
where 02H
t32H
H xo = ~--7-~ , cX~Xo
H"°-~U~Xo' etc.
~( t) =.fx~( t) + A~( t - O) +f~q(t),
We also have the following identity:
f
t, < Ho(t), ~o(t) > dt
~ ( t ) = 0 for - 0 ~ < t ~ < 0
(7)
(12)
0
=
where again, fx, fo, fu are evaluated along the nominal trajectory. It can be shown [4] that the solution of (12) can be written in the following form:
+ f t f - o
~(t)=
Using the condition ~(t)=0, te[--O, 0] eliminates the first term on the R.H.S. of (7). Using (7), the 5th and 6th terms on the R.H.S. of (6) can be combined into the following form: tY -- 0
fo
+
[" ,It!- 0
We now define 2(0 such that the following relationships hold:
)t(t) = - Hx(t),
0 <~t <~tf-- 0
tf -- 0 <~t <~tf
A(tf)=Qx(tf).
(8) (9)
If we now assume a nominal control law uo(t), equation (2) can be solved in forward time to give xo(t), and equations (8)-(10) can be solved in backward time to give AdO. In general, these solutions will not satisfy condition (3), nor will they satisfy the usual optimality condition Hu(t)=O. Consequently it is necessary to devise an iterative technique which will converge to the optimal solution. This problem is studied in the next section. A fixedpoint method. We now proceed along similar lines to that of FREEMAN[3]. In order to save space the reader is assumed to be familiar with the Hiibert space framework adopted in [3]. Using the assumed relationships (8)-(10) in (6) gives:
+
f',
,io
fi-'
N(a, t - O)f~(a)q(a)da,
~(t-- O) =
fo,O
t - O)f.(a)q(a)da,
t>~O.
(14)
~(ts) >
t t> 0, (15)
where ~r(a, t--O)=N(a, t-O) for O<~a<~t--O and ~(a, t - - 0 ) = 0 for a>t--O. Following Freeman's approach, we now write (13) and (15) in the form, ~(t) = L q
(16)
~o(t)= Lq
(17)
(10)
AJ = ½< Qxx(tf)~(tf),
(13)
Equation (14) can be written in the following equivalent form:
,~(t) = - H x ( t ) - Ho(t + 0),
N(a, t)f~(o)~l(a)da o
where the matrix N(a, t) is the solution of a certain differential equation. From (13) we have
~ ( t - O) =
< {(H~(t) + Ho(t + O) + ~(t)}, ~(t) > dt
f
t
where L and £ are mappings defined on Hilbert spaces. It will be assumed that these mappings are such that they map the control Hilbert space into the state Hilbert space. Conditions necessary for this assumption to hold can be found as in [3]. We have previously noted that the nominal trajectory will not, in general, satisfy condition (3). Let the vector ~, have a specified change A~,. Then we have,
A~I = G(tf)~(tf)
(I 8)
where G(q) is the matrix
< H~(t), ~(t) > dt We now impose equation (18) as an additional constraint on the minimisation of (11). This is most easily done by adjoining the following term to (11):
+ < Hoo(t)¢o(t), Co(t) > ~["AI//-- G(tf)~(tf)] T W [ A ~ --
G(tf)~(tf)'[
+
+ 2 < H,~,(t)~(t), tl(t) >
~(t) = Hxx(t ) + Qxx(tf)b(t- tf)
+ 2 < H~(t)~o(t), ~i(t) > + 2 < Hxo(t)~o(t), ~(t) > }dt
(11)
G(t) = G ( t f ) b ( t - - ts)
Correspondence item
795
Finally,
where $(t) is the Dirac function. The new problem is now that of minimising J*, where
o { < Hu(t), ~(0 > + ½< ~(O~(t), ~0) >
q* = n ~ t K - ½ H ~ I ( R + R*)r/*.
(20)
Equation (20) is the required integral equation for t/*, and can be written in the following form:
+½
+ < Hxo(t)~o(t), ~(t) > -- < t~T(t)WA~k, ~(0 > + ½< dr(t)WG(t)~(t), ~(t) > } d t . Using (16) and (17), J* can be written in the form, 2 J * = I(R + Huu)q,r/l - 2]L*GrWA~k,~/[ +
2[Hu l
(19)
where
Assuming that C is a contraction operator then the contraction mapping principle can be invoked to provide an iterative solution to (20) as discussed in [3]. Necessary conditions for C to be a contraction operator can be derived as in [3]. Similar results can be found in [5], where an equation analogous to (20) is discussed and the existence and uniqueness of the solution is proven without invoking the contraction condition. This completes the proposed solution to the problem.
Conclusions
R = L*QL + L*Hoo£ + 2 H ~ L + 2H~0L + 2L*H~o£ + L *~jTWGL, 1", "1 denotes inner product in the control Hilbert space, and L*, L* denote the adjoint operators of L and L respectively. Let F/* be the optimum value for r/. We now give a small variation ~* to r/* and determine a necessary condition for optimality of r/*. 2(J* + As*) = [(R + H..)(q* + fl*), q* + 0"1 - 2 1 K , tl* +0"1
It is known that gradient methods tend to have slow convergence near the optimum. It is thought that a combination of the gradient method, and the above second variation technique might provide a more rapid overall convergence. Much remains to be done to test this assertion.
References [1] H. R. SEBESTA" Design and analysis of optimal control
[2] [3]
where [4] K = L*t~r W A S - H~. We then have 2A J * = [(R + Huu)q*, ~*1 + l(R + H . ) O * , ~t*l - 2 1 K , ~*l + I(R+Huu)O *, ~'1" To first order in the variation r~* this becomes,
2AJ*=I{(R+H..)+(R+H.u)*}rt *, 0*l-2l K, 0*l where (R+Huu)* is the adjoint operator for (R+Huu). Hence a first order condition for r/* to be the optimum is given by,
[5]
systems for dynamical processes with time leg. Ph.D. Dissertation, University of Texas, Austin (1966). D. MAcKINNON: Optimal control of systems with pure time delays using a variational programming approach. IEEE Trans. Aut. Control 255-262 (1967). E. A. FREEMAN: O n the optimization of linear, timeinvariant, muitivariable control systems using the contraction mapping principle. JOTM 3, 416-443 (1969). M. N. OGUZTORELI: Time Lag Control Systems, p. 91. Academic Press, New York (1966). A. MA~aaos: Optimum control of linear lime lag systems with quadratic performance indexes. Preprints of the IVth IFAC Congress, Vol. 7, Warsaw (1969).
R~umr---Dans cette notice, on considrre le contr61e optimal d'un systrme reprrsent6 par une 6quation de diffrrence diffrrentielle non-linraire avec une fonction g6n6rale de coCtt. On propose une mrthode de solution itrrative du second ordre fondre sur le principe de marquage en contraction de point fixe. Zusammenfassung--Dieser Bericht behandelt die optimale Kontrolle eines Systems, welches dutch eine nichtlineare Differential-Differenzgleichungmit einer allgemeinen Kostenfunktion dargestellt wird. Es wird ein Iterativverfahren zweiten Grades zur Lfsung auf Grund des Kontraktionkartographieprinzips ftir den festen Punkt vorgeschlagen.
{(R + H.,) + (R + H..)*}r/* = 2 K . Pe3mMe---B aTOl~ paroTe paccMaTpnBaeTc~ onTnMaJIbHbU~ Since Huu=Huu* this last expression can be written in the following form:
2H.Ji* = 2 K - (R + R*)q* .
KOHTpOJIb CHCTeMbI, Hpe~GTaBJIeHHOI~ ypaBHeHHeM HeY[HHell~IOl~ ~i~¢l~peI-ii~a.rlbHOl~ pa3Hl~/~I C o6mefl ~yI-iKt~e~ 3aTpaTH 3HeprHH. ~JLq pemermg npe~-laraeTc~ HTepaI~OHMeTOR BTOpOFO n o p g ~ a , OCHOBaHHI~i~ Ha IIpHHHHHe OTorpaxem~ 37~aTILq HeHO~IBH~KH01~TOtlKH.