Singular Perturbations for Deterministic Control Problems

CopHight © 1FAC 9th ~I ricnnwl World Budapest. lIun gan, 198-1

SI :\CCLAR I'ERTCRBAT I O:\S I I\' A:\D STOCHASTIC CO:\TROL

Congre~~

OI'T I ~IAL

SINGULAR PERTURBATIONS FOR DETERMINISTIC CONTROL PROBLEMS A. Be n soussan ['W1 'I' n/'.'

()f

Pan\~Daup"Hlt'

and ISRIA.

D()mol1lP

dp

\'olllceall. RoCQllfllCO",I,

,8153

I .f Chl'wu\,

Fra,/u

INTRODUCTION. We give in this article a short presentation (without proofs) of a paper to appear in a book edi ted by G. BLANKENSHIP, P . V. KOKOTOVIC and the author (see references), (see also A. BENSOUSSAN

meaning that it admits a unique optimal control lie. This situation occurs when the limit control problem is not too far from convexity. Note that everything can be localized, hence just local optimality is sufficient . On the other hand, a problem minimum of assumptions is made on the itself , in particular we do not assume the existence of an optimal control for the E problem. Therefore a natural "good" control to use is lie itself. We prove the convergence of

[1]) .

The problems we are interested in are of the following type. Consider a dynamic system whose evolution is governed by E dx xE(O) f(x E ,yE ,v) x (it 0

infJE(v(.)) tOJ(u )' o

(1 )

~ dt

= g(xE,yE,v)

yE(O)

v(.)

yo

in a general case , including constraints. At this stage, no estimate of the error is given. When more regularity on the limit problem is available, an estimate of order E is given .

in which v(t) represents a control. The parameter E tends to O. The state of the system (xE(t) ,yE(t)) contains one part x E which varies slowly , and one part yE which varies fastly.

It is possible to improve this estimate. To build a control which is better than lie (i . e. which yields an approximation of inf JE(v(.)) which is

Such a situation is common in the applications . It appears for example in economic models to take into account long term and short term variations, but also in many problems of engineering, biology,

v(. )

of higher order) requires the introduction of boundary layer terms .

mechanics ......

We give an expansion uE which approximates the infimum as accurately as desired. Two expansions are needed (inner and outer expansions), namely one of regular perturbation type and one of boundary layer type (at 0 and T) . The improvement of accuracy is as follows : to obtain E2 one needs to add to lie, boundary layer terms at 0 and T, to obtain E3 one needs to add a term EUl (t) (regular perturbation type), to get £I one needs to correct the boundary layers terms and so on . . . All these additional terms are solutions of control problems. Starting with ul' all control problems are linear quadratic. On the other hand, the first boundary layer terms are solutions of non linear non quadratic control problems . They are control problems on an infinite horizon. It is thus necessary to check that these problems are well posed and that the corresponding control, state decay exponentially in time . It is possible to give assumptions with guarantee these facts, and which relate basically to the limit problem, and thus can be checked easily.

The terminology "singular" explains as follows the problem corresponding to E=O, namely: :

= f(x,y,v) (2)

g(x,y,v) = 0 if of different type from the case E > 0 (an algebraic equation replaces a differential equation equation). An other way of expressing the same idea is to say that, in the limit, the state ' s size shrinks to x, the slow system . The control problem consists in minimizing the cost

(3)

The problems of interest are two fold. We want to study the behaviour of the quantity inf JE(v(.)), v(.)

as E

~

O. Moreover, we want to construct "good" if

The nonlinear control problems arising in the boundary layers at 0 and T have interesting duality considerations and lead to questions of independent interest.

not optimal controls for the E problem. The general philosophy of the approach developed in this article, is that the limit problem is simpler than the E problem. Note that this may not be the case, nothwithstanding the reduction of the size, since the E problem is more regular than the Hmi t problem .

Finally, the point of view of Dynamic Programming is considered. This introduces two main differences. Firstly, the cost function must be considered as a function of the initial conditions. Secondly, the controls are necessarily g i ven by feedbacks. The convergence is demonstrated and an estimate of the rate of convergence is given. However, it is not possible to design a feedback which leads to an accuracy of the approximation similar to that

This underlying philosophy legitimates the assumption that the algebraic equation entering in (2) can be solved in y, in a unique way . \~e also assume that the limit problem is well posed, 1515

A. Bensoussan

1516

developed in the open loop theory. There are difficulties to introduce the boundary layer terms in the feedback point of view. We treat both finite and infinite horizon problems.

h(x)

twice continuously differentiable in x,y , v : the derivatives of f,g are bounded: the 2nd derivatives of ~, h are bounded.

The litterature in the domain of singular perturbations for deterministic systems contains numerous

works. Fortunately excellent surveys exist with extensive lists of reference (P.V. KOKOTOVIC, R. E. O'MALLEY, P. SANNUTI [lJ, for the period before 1976 ; P.V. KOKOTOVIC, V.R. SAKSENA [lJ and V. R. SAKSENA, J. O'REILLY, P.V. KOKOTOVIC [lJfor the period after 1976 ; P.V. KOKOTOVIC [lJ). With respect to the questions treated in this paper, the situation is as follows. The linear quadratic control problem has been well analyzed from the open loop as well as from the feedback point of view (cf . P . SANNUTI - P.V. KOKOTOVIC [lJ, A.H. HADDAD - P . V. KOKOTOVIC [lJ, P . V. KOKOTOVIC R.A. YACKEL [lJ, R.E. O'MALLEY [lJ,[2J, [4J, R.E. O'MALLEY - C.F. KUNG [lJ, [2J, R. R. WILDE P.V . KOKOTOVIC [lJ among the main contributions). In this case, one can guarantee that the E problem has a unique optimal control and thus expansions are easily done . The boundary layer terms lead to Riccati equations in duality. These equations appear also in different contexts (cf. P. FAURRE M. CLERGET - F. GE~~IN [lJ) and the structure of the set of solutions is interesting . The nonlinear case (often referred as the trajectory optimization in the litterature) has been considered in particular by P. SANNUTI [lJ, [2J, R.E. O'MALLEY [3J, [5J, P. SANNUTI - P.V. KOKOTOVIC [lJ, C. R. HADLOCK [1] M. 1. FREEDl1AN - B. GRANOFF [1 J , M.I. FREED~~N - J. KAPLAN [lJ, A. B. VASILEVA, V.A. ANIKEEVA [lJ, P. HABETS [lJ, M. ARDE~ [lJ, [2 J

: Rn ... R ,

(1.2)

11 > 0

gy(x,y,v) '; - In , 2

k Let v(.) E L (0 ,T;R ) . For E given, one solves the differential equations (1.3)

dx

E

E E f(x ,y ,v)

;:it EdyE

~

=

xE(O)

E E .(x ,y ,v)

yE(O)

x

0

= Yo

There is one and only one solution of (1 . 3) such that x E E H1(0,T;R n ), yE E H1 (O,T;Rm). One then considers the cost function (1.4)

T

JE(v(.»

=

E

E

~(x (t) , y (t) ,v(t»dt +

Jo

which is well defined since ~ has quadratic growth . An admissible control satisfies the constraints (1.5)

v(t)

U convex closed non empty ad subset of Rk . E

1 . 2 . The limit problem .

... ). Consider the algebraic equation y

In general the point of view is to write the necessary conditions of optimality and to find expansions . A problem which is considered is to solve the necessary conditions of optimality for the E problem by perturbation techniques . This problem is not treated here. On the other hand the evaluation of the cost function for "good"

controls does not seem very much considered in the litterature, nor the expansion of the optimal cost. The fact that the control Uo itself yields an approximation of order E was known at least in the L.Q. case, although the proof given relies on the boundary layer analysis. We show this fact in general without using the boundary layer. The

(1.6)

g(x , y , v) =

°

in which x, v are parameters. It has a unique solution y(x,v) , which is Cl with bounded derivatives . Consider then the system, for v(.) E L2(0,T;Rk ) (1 . 7)

dx dt

= f(x,y(x,v) ,v)

x(O) = x

o

It admits a unique solution x(.) in H1 (0 , T;R n ). The limit problem consists in minimizing (1.8)

J(v( . »

= fT

~(x(t) ,v(t) ,y(t) ,v(t) )dt

o

presentation of the convergence in the "constraints"

+ h(x(T»

case (lack of regularity) has not either appeared in the litterature. in which we have set : The study of Bellman equations in duality seems also original. It should be interesting to study the complete structure of the set of solutions. In the Dynamic Programming approach, the main concept is that of composite feedback, due to J. CHOW - P.V. KOKOTOVIC [lJ. We extend this work and prove in particular that the decomposition of the composite feedback as the swn of the limit feedback and complementary term involving the fast system is general, and not restricted to a quasi - linear structure of the dynamics .

(1.9)

y(t) = y(x(t) , v(t».

We make now additional asswnptions on the limit problem . Define the Hamiltonian (1.10 )

H(X,y,v,p,q) =

~(x , y,v)

+ p.f(x,y,v) +

+ q.g(x,y,v)

and note w (t) o

°o (t)

1 . OPEN LOOP CONTROL PROBLEMS .

(x (t) 'Yo (t) ,uo (t» o

.

We write the first and second order optimality 1.1. Setting of the problem . Let us consider functions f, g, that:

conditions : ~,

h such

(1. 11)

dx o

;:it

f(O ) o

x

o

(0)

g(Oo) = 0 (1.1 )

f(x,y,v) dpo

g(x,y,v)

- ;:it

= Hx(w o )

p

o

(T)

H (w ) = 0, H (w (t». (v - u (t» Y 0 v 0 0

2 0

+

Singular Perturbations for Deterministic Control Problems

1517

(2 . 1 )

x

o

E h (x (T» x

_ EdqE dt

o

We consider an expansion of the form (1 . 13)

h

(2.2)

,, 0

xx

These conditions ~mply that u is the un~que optimal control for the problgm (1 . 7), (1 . 8). with the notation 1.3. Convergence. We have the following convergence result: Theorem 1.1 : Assume (1.1), (1.2) , (1.13) and the existence of wo(t) such that (1 . 11), (1.12) hold. Then one has (1.14) If u

E

(1.15)

Inf JE(v( . »

(U , Zl ,1;1 , K ,Q1)·

... inf J(v( . » .

o

T

, El

satisfies JE(uE )

"

0

... 0

in

2 k L (0 , T;R )

E Y - Yo ... 0

ill

m 2 L (0,T;R )

in

n 1 H (0 , T;R )

u

x

E

- u

E

- x

0

as the three first

JE(u ) Expanding we obtain the following systems of relations : dx 0 (2.3 ) f(a ) x (0) = x (it

then ( 1.16)

o

... 0 0

0

g(a )

,

0

0

0

0

dPo

- (it

H (w ) x 0

p (T)

h (x (T» x 0

0

0 The convergence result can be improved when some additional regularity properties hold . Assume that

H (w )

Y

dY

(1.17)

(2.4)

0

o

g(a (0) + EO(T»

dT dL

then one has

o

, Y

0

0

(0)

o

Theorem 1 . 2 : We make the assumptions of Theorem Theorem 1 . 1 , as well as the regularity assumptions ~ Then we have :

dZ (1.18)

IInf J E (v( . »

- inf J(v(.»

1 " c

(2.5)

dQ

IfuEsatisfies (1.15) then (1.19)

lu

Ix

E

E

- uo l

-

L2

xo IH1

Iy

E

" c/E

o

- dT o

dT

-

Yo l

,

"

L2

lyE(t) I

0

H

y

(w

0

(T)

T

+ w

0

-q (T)

Q (0)

(T»

o

o

CE

" c.

2. BOUNDARY LAYER ANALYSIS. We now look for terms to correct ~. To guess their form, the easiest method is to write the necessary conditions of optimality for the E problem and make expansions . But this cannot be a method of proof, since we do not assume that the E problem has a solution. 2 . 1 . Formal expansions. Set wE(t) = (xE(t) ,yE(t), uE(t) , pE(t), qE(t» and aE(t) represents the three first compcnents. The necessary conditions write :

2 . 2 . Optimization problems. The elements of the expansion are determined by optimization problems. Let us consider: dY (2.6)

o

dT Y

o

(0)

yo - yo( O)

f""o

[H(x (0 ) ,y (0) + Y ,u (0) + o

0

0

0

1518

A . Bensoussan (2 . 17)

Similarly ,

I;} I 2 '

IyE I 2 S CE 2 ,

L (2.7) I

L

xE IC (0 , T)

o

,; CE 2

A complete expansion can be found in the detailed a r ticle , namely (2 . 18) The systems (2 . 4) , (2 . 5) express the necessary conditions of optimality for the problems (2.6),(2 . 7) . o T . The terms w , W , W are obta1ned by solving linear 1 1 1 quadratic control problems.

-

(2 .

J 1 (u 1 ) and X3 involves

3.1 . Setting of the problem H

(t),q (t)+L (L))"SI 0

0

0

Consider the family of control problems

vv (3.1) yV H )

0

0

E

0

vv

(3.2) H

H ) _ (H xxxyxv

Hyy Hyv ) ( HH vy vv

-1 ( Hyx) H vx

These conditions contain (1 . 12) , taking L=+OO. One a l so asSUIIES

(3.3)

, Z , ~)

bounded in

dyE

~

E E E = g(x , y , v) , y (t) = y

E

J x , y , t (v ( . )) =

JT t

a nd define

* gv(Xo(T)

E E E = f(x , y , v) , x (t) = x

,,0

with the same arguments as in (2 . 18) , I,1 x , y , v , t , L.

(2.10)

E dx

~

(x , y,v,p (t) ,q (t)+Q (L) )"SI

H

9)

~u(~o), X2 involves

~l(l;l) ' ~l(~l) '

3. DYNAMIC PROGRAMMING

8)

HyV ) (X'y , V' p

(2.

810 (1;0) '

where Xl involves

We assum= the existence of solutions for the systems (2.3) , (2.4), (2 . 5) and make in addition the assumptions of 2nd order type

EX31 ,; C 4,

has a left inverse which is

Z,~ .

epE ( x, y , t)

JE (v( . )) . x,y , t

inf v( . )

Ass u me besides (1.1) ,

( 1. 2) ,

(1.3)

that

( 3.4)

These assumptions are sufficient to imply the exponential decay

(3.5)

(2 . 11)

nd the 2 derivatives of f , g are bounded Co by ~ ' where Co is not too large compa r ed to Y .

and from this (2 . 12)

I XO(L) I ,

IUO(L) I ,

(2 . 13)

I W~(L)

I W~(L)

I,

IMo(L) I ,

I ,; ce -

IK (L) I,; ce - YL o

YL

The function epE can be characterized as the maximum solution of the Hamilton Jaccobi Bellman equation (3 . 6)

+ ! D epE . g(x , y , v) E y

2 . 3 . Convergence results

epE(x , T)

Let us list the assumptions . We need (2 . 14)

We of tence (2.5) there

4

f , g , £ , h are c in x , y,v ; all derivatives of f,g are bounded; all derivatives of ~ , h starting from the 2 nd order are bounded

course assum= (1.2), (1.13), then the exis of solutions for the systems (2 . 3) , (2.4) , satisfying also (2 . 8) , (2 . 9) , (2.10). We can state the following

aepE + Inf [ D epE . f(x,y,v) at x

+

~(x , y , v)

+

]

o

= h(x) .

Consider next the limi t control problem (3 . 7)

dx ds = f(x,y , v), x(t) g(x , y, v)

= x , s E (t,T)

= 0

(3.8)

(x(s)) , y(x(s) , v(s)) , v(s))ds + h(x(T))

Theorem 2 . 1. : Make the assumptions just listed. If E u is a control satis fying

and define (3 . 9)

(2.15)

ep(x , t)

= Inf Jx,t(V(.)).

By virtue of (3.4) , of then one has

(3.10)

and defining xE , yE , liE tima tes hold

~:

+ Inf[!l . f(x , Y(x , v)

£l

(2 . 2) , the following es -

at -

2

,v)+ ~( x, y(x, v)

, v)J=O

2 ep(x,T) = h(x), epEC , 0 '; D ep ,; c, 2 a ep I ,; C(l+ l x l 2 ) . I D aep I < C(l+ I x I ) , IatL v

(2.16)

(3.5) , ep is the unique solution

Sin gu l ar Per turbations for Deterministic Con t rol Problems

3 . 2 . Convergence An additional technical assumption is necessary , namely (3.11)

the 3

rd

derivatives of

bounded by the 3

rd

~

Paris.

de r ivati ves of f , g in y , v are

Freedman M. I. , B. Grannoff [1] (1976) . Formal Asynptotic solution of a singularly perturbed non linear optimal control problem. JOTA 19, pp . 301 - 325.

C

Theorem 3.1: Assume (1.1) , (1.2) , (3 . 4) , (3.5) , (3 . 11) then on~the estimate I4>E(x , y , t) - 4>(X , t)I "C

E [1 + IxI

3

+

+ Iyllxl]+ cElyl2 + CE 2 lyl3 3. 3 . Conposite feedback Let u(x , t) be the unique optimal feedback in (3.10) and y(x , t) = y(x , u(x , t)) and define q(x , t) by R. (x , y(x , t) , u(x , t)) + f * (x , y(x, t) , y y ii (x , t))D + g*(x , y(x , t) , u(x , y))q(x , t) = 0 y

Consider the problem , whose unknown function is 4>1 (3 . 13)

~:

+ Inf [D4>.f(x , y , V)+Dy 4>1 . g( x, y , v) + v

+ R.(x,y , v)] = 0 in which x, t are parameters. It is possible to find a solution o f (3.13) of the form

q(x , t)

4>1 ( x, y , t) +

c

Habets P . rl1 Singular Perturbations in Non linear Systems and Optimal Control. in M. Ardena (editor , see above) , pp. 103- 143. Haddad A. H. , P . V. Kokotovic [IJ (1971). Note on singular perturbation of linear state regulato r s . IEEE Trans . Auto Control , AC- 16 , 3 , pp. 279- 281. Hadlock C. A. rl1 (1973) . Existence and dependence on a parameter of solutions of a non linear two point boundary value problem. J. Diff . Equat ., 14 , pp. 498- 517 . Kokotovic P . V. [11 (1984). Applications of singular perturbation techniques to control problems , SIAM Review . KokotovicP.V. , R.E. O'Malley Jr , R. Sannuti [ I J (1976) . Singula r perturbations and order Aut omatica, 12, pp. 123- 132.

X( x, t ; y - y(x, t))

u (x , y , t) = u(x , t) + ~(x , t

Freedman M.I . , J . L. Kaplan rl] (1976). Singular pe r turbations of two point boundary val ue problem arising in optimal control. SIAM Contr ol . Opt. , 14 , pp . 189- 215.

reduction in contr ol theory , an overview.

(y - y (t)) +

where X is the Bellman function of an infi nite horizon p r oblem. Moreover the infimum in (3 . 13) is attained by a function (3 . 14)

Chow J . H. , P . V. Kokotovic rl] (1978). Near Optimal feedback stabilization of a class of non linear singularity perturbed systems. SIAM J . Control Opt . 16, pp . 756- 770 . Faurre P ., M. Clerget , F . Germain [11 (1978) . eper ateurs rationnels positifs. Dunod ,

in y , v are

C

bounded by - - 2 . 1+lx l we can then prove the following

(3.12)

1519

y - y (x ,

t))

called the conposite feedback . It permits a better =ntrol of the sys tern than the limi t feedback u(x , t) . It i s even IJX)re iIlportant i n a prcblem with infinite horizon (T -100), where it provides a stabilizable control (cf. CHOW- KOKOTOVI C [ 1]) . Remark 3.1 . A conplete expansion of 4>E similar to the open loop case is still not established

REFERENCES The extended version of this article will appear in A. BENSOUSSAN - G. BLANKENSHIP - P . V. KOKOTOVIC, ed . Singular Pe r turbations and Optimal Control, Springer Verlag Lectures Notes. Ardema M. [1] (Sept . Oc t. 1979). Linearization of the boundary layer equations of the minimum time to clinb prcblem. Journal Guidance and Control , Vol. 2, nO 5. Ardema M. [2] (1983) . Singular Perturbati ons in Sys tems and Control (ed) . CISM Courses and Lectures nO 280 , Springer Verlag . Bensoussan A. [ 1] Perturbations Methods in Optimal Contr ol. To be published. Bertran P. [1] Calcul formel et perturbations, These a paraitre .

Kokotovic P . V., V. R. Saksena i lJ (1982). Singular perturbations in control theory. Survey 1976 . Kokotovic P .V. , R. A. Yackel [IJ (1972) . Singular pertur bation of linear regulators. IEEE Trans . Auto . Control, AC-17, pp . 29-37. Lions J . L. [1] (1969). Quelques methodes de resolution des problemes aux limi tes non lineaires . Ounod, Paris .

Minty G. J . [ 11 (1962) . Monotone (non linear) operators in Hilbert spaces. Duke Math. Journal , 29, pp . 341 - 346. O' Malley Jr R. E. [ I J (1972) . The singularly perturbed linear state regulator problem. SIAM J . Control , 10, pp. 399- 41 3 . O' Malley Jr R. E. [2J (1972) . Singular perturbation of the time invariant linear state regulator problem. J . Diff . Equat . 12, pp. 117- 128 . O' Malley Jr R. E. [3J (1974). Boundary layer methods fo r certain non linear singularly perturbed optimal control problems . J. Math Anal. App . 45, pp . 468-484. O' Malley Jr R.E . [4] (1974) . Introduction to Singular Perturbations . Academic Press, N. Y. O'Malley Jr R. E. [5] ( ) . Singular perturbations i n optimal control. In Mathematical Control Theory, Lectures Notes in Mathematics, 680 , Springer , N.Y.

1520

A. Bensoussan

O'Malley Jr R.E., C.F. Kung [ lJ (1974). The matrix Riccati approcah to a singularly perturbed regulator problem. J. Diff. Equat. 17, pp. 413-427. O'Malley Jr R.E. [2J (1974). The singularly perturbed linear state regulator problem, SIAM Cont., 13, pp. 327-337. Saksena V.R., J'O'Reilly, P.V. Kokotovic [l J (1982). Singular perturbations and time-scale methods in control theory : Survey 1976. Sannuti P. [lJ (1974). Asymptotic solution of singularly perturbed optimal control problem, Automatica, 10, pp. 183-194. Sannuti P. [2J (1975). Asymptotic expansions of singularly perturbed quasi linear optimal systems, SIAM Cont., 13, 3, pp. 572-592. Sannuti P., P.V. Kokotovic [ l J (1969). Near optimum design of linear systems by a singular perturbation method. IEEE Trans. Auto. Control, AC-14, pp. 15-22. Sannuti P., P.V. Kokotovic [2J (1969). Singular perturbation method for near optimum design of high order non linear systems. Automatica, 5, pp. 773-779. Wilde R.R., P.V. Kokotovic [lJ (1973). Optimal open and closed loop control of singularly perturbed linear systems, IEEE Trans. Auto. Cont., AC-18, pp. 616-625.