On the Computation of Optimal Nonlinear Feedback Controls

CopYright © IFAC Control .-\pplications of :\()nlint>ar Programming and Optimization. Capri. 11«1~ ' , 1~)Hj

ON THE COMPUTATION OF OPTIMAL NONLINEAR FEEDBACK CONTROLS H. Bourdache-Siguerdidjane and M. Fliess La/Jomloirl's des Sigllnllx 1'1 S),slhIlI'S, CNRSIESE, PIIIII'IIII dll ,\lOll 1011 , 91190 Gif~\lIr-}'l'flll' , FmllO'

Abstract. Problems of optimal control, without constraints and in fixed time, are considered. We show that the feedback law satisfies a system of quasi-linear partial differential equations which can be integrated by the method of characteristics. We illustrate the advantages of our approach by two examples ; one is a realistic application in heat transfer of buildings. Keywords. Nonlinear systems, optimal control, feedback, Lie brackets. OPTIMAL FEEDBACK CONTROL LAW

INTRODUCTION As we know, iJ nonlinear control theory, difficulties often arise when computine optimal feedback control loops for many problems. In the early 60's, the problem of optimal feedback regulation for linear systems was studied by Kalman. Some attempts to treat the problem of feedback regulation for nonlinear systems were done by Lukes (1969), Willemstein (1977) and others. They proposed (Lukes, Willemstein), upon reexamining the works of AL'Brekht (1962) and Brunovsky (1967) a technique thereby the optimal feedback loops are represented as Taylor series.

where the state x = (xl , ... ,xN) belongs to lliN and the control u = (ul," .u m) belongs tollim. F = (Fl" .. ,FN) : lli N+m+1.... lliN, ~ : lliN .... lli and F O : lliN+m+ l .... lli are gi vcn analytic functions,

However their theory is only valid under the assumption that the states and the controls remain in a neighborhood of a fixed point (which is in general an equilibrum point). In spite of other attempts, there is of yet no systematic method for computing the optimal feedback law.

We are looking for an optimal feedback control u(x,t) which minimizes the cost functional J for all initial times t, to 5 t 5 T and all initial states x(t) = x. In addition, T is fixed, the endpoint is free and the variables x and u are assumed to be unconstrained.

We are proposing here a new approach based on the elimination of the adjoint vector. We have shown in previous papers (Fliess, 1984 ; Fliess and Bourdache-Siguerdidjane, 1984) that simple Lie bracket computations lead to a system of quasilinear partial differential equations satisfied by the optimal feedback control law . When the dimensions of the state space and the control vectors are equal, this system turns out to be one of first order which can be reduced to the integration of a system of ordinary differential equations by the method of characteristics. This result can also be derived via more classical methods, such as the Hamilton - Jacobi - Bellman equation, and this is in fact the approach that we will adopt here.

a) A summary of the Lie bracket approach (cf. Fliess ; Fliess and Bourdache Siguerdidjane)

One important special case arises when the cost function does not depend on the control variable, for then the system of partial differential equations degenerates into a set of algebraic equations, i.e., they are not differential equations. We illustrate our theory with two bilinear examples. One comes from the literature (Bell and YE, 1981) and the other is a realistic case-study of heat transfer in buildings (Bourdache-Siguerdidjane 1985).

Let us consider the following problem in Bolza form F(x,u,t) (1)

~(x(T»

+ fT F O (x,u,t) dt to

As usual, the Hamiltonian is defined by

~

'"

N

k

H = = I PkF (x,u,t) k=o where ~ = (pO' PI", .,PN) is the adjoint vector O and} is the vector of components (F ,F) . To the vector fields associate the new vector field

t,

A =

~t

+

t. .

N

N

Recall that a vector fields (cf. LEBORGNE) X:lli .... JR is defined in a local coordinate chart x=(xl" .,x~) as a first order linear differential operator N

I

X =

xk

k=1

3"k'

Us ing the Maximum Principle and the Hamiltonian formalism (cf BRYSON) one obtains

"

d·

~ t

H

'\.

ui

\..'

'V

=
ui

>~O , ·i =0,1,2

.. (2)

ad~ ~ui = [A, . .. [A' ~ui]] means the Lie bracket [A, t . ] iterated ~ times. Recall that a Lie bracket ofuEwo vector fields X, Y is the commutator [X,Y] = XY - YX.

where the llotation

28

H.

Bourdache-Si~lIerdidjane

Remark. Equation (2) is c lassical in analytic mechanics though there it is usually expressed in terms of Poisson brackets (Goldstein, 1956). It also often appears in singular optimal control theory (Krener, 1977 ; Lamnabhi - Laguarrigue, 1985). Corrolary : The optimal feedback control law satisfies the infinite hierarchy of necessary conditions


ad~ ~ui> = 0 , v = 0,1, ..•

o and

i = I, . .. ,m. + F~t + F~2 Ut = 0

"

Pk

The elimination of J*

Fk u.

(4)

1

p

FO ........ .. FN uI ... ....... uI FO ....... . . . FN uN 1lN ui

o i

[A, ~

] ° ...... .. .

U·1

1, . .. ,N .

]N

We obtain a first order system of N quasi-linear partial differential equations. In the case where the dimensions of the state and control vectors are different, such as N > m, the elimination of the adjoint vector leads to a higher order partial ' differential equation (Rouff, to submit) : dru e[u,x, t, dt r ' dr-Iu

is a matrix and

~

is a vector.

_F o

ut

+

(5)

T T + FT (IN S (F- FO)) F - FT FT F- FO u

ux

where fined P S Q IN is

u

u

x

u

u

S is the Kronecker product (Brewer,1978) deas foJ.lows : for P E lRPxr and Q E lRmxq ,

PijQ. the N dimensional identity matrix . We denote

=

by FT2 and F T the partionned matrices T u TUX T T )and(F I ... IF ) respecti ve l y . (F I ... IF uUI uUN uXI uXN T Setting CNxN = [F~2 - F~2 (IN S (F: F~))] and letting D equal the rieht hand side of (5), one can rewrite (5) in the condensed form

C ( dU dt

+

~ F) = D. dX

(5' )

Two special cases arise from this equation

N

dui + L Fk dui = dt k=1 dXk

The previous partial differential equations can also be derived quite simply from the Hamilton Jacobi - Bellman equation. This thereby shows the relationship between the modern differential geometric tools used in non linear system theory and the c l assical methods. Consider once again the system (I). Denote by J* the optimal value of J. The associated Hamilton Jacobi - Bellman equation is J* + min [ FT J* + FO] = 0 u

from the last

CN~N

invertible. Let S

equal C ID, then writing (5') in terms of its components

b) Hamilton - Jacobi - Bellman approach

t

xt

FT (IN S (F: T FO))] [dU +~ F] dt dX u u FT F- T FO _ FO F + FT F O ut u u x u ux

I) Nondegenerate case :

~[u,x,t, dt r - I '

J*

xx'

three equa ti ons gives the following equation in matrix notation [FO 2 u

which is a homogeneous system of 2N equations in N + I unknowns (PO,PI, ... ,PN). The adjoint vector is not identically zero, hence (4) has a solution if all the character istics determinants vanish

[A, ~

J*

x'

0

{kt Pk [ AJ ui ]k = 0 k=O

e

Differentiating the second equation with respect to x and t respectively, one obtains

(3)

For m = N, t he first order conditions, in other words those for v = 0 and I, lead to :

L

and. I. Fliess

x

or (see Willemstein for example)

s.

i = I, ... ,N

l'

we clearly see the characteristics differential equations (cf. Courant and Hilbert) , k

I, ... ,N.

(6)

The solution of Eq . (5) is the inteera l hypersurface which is generated by the characteristics curves of (6) and passing through a given curve f. The equation for r comes from the boundary conditions of (5). This is a well-known Ca uchy problem (cf. Courant and Hilbert).

o

2) Degenerate case (cf. Bourdache- Siguerdidjane)

o

When the cost function does not depend on the control variable, the optimal feedback control necessarily satisfies the algebraic equations

Differentiating the first equation with respect to x yields J* + (FT + uT FT) J* + FTJ* + FO + u T FO xx xt x x u x x x u which can be reduced to J* + FT J* + FT J* + FO = 0 x xt x x xx (notice, however that this is equivalent to the adjoint equation)

0

N

I:

k=1

Fk FO = ui xk

o.

Example I Consider the system (Bell and YE, 1981) which seems to describe some realistic case- studies (ecology, metallurgy, ... ) EXU

29

Computation of Optimal Nonlinear Feedback Controls The corresponding PDE is au

at

+

~xu

au ax

~qx

The equivalent electric circuit of the whole system (room + external climate) is :

2

and the Cauchy condition is u(x,T)

o

From the auxiliary system dx

du

~xu = ~qx2 =

dt

1

we get the following implicit analytic solution u + x

Iq

sin [~/qx2-u2 (T-t)] = 0

( 7)

Fig. 2.

the solution given in the literature, u(x, t) = - ~qx2 [(T-t) -

%q~2

(T-t) 3 x 2 ] + ... ,

is a Taylor expansion of (7) in the neighborhood of the point (u = O,x,t = T). So, it is only valid on a portion of the surface u = u(x,t). The graphical representation of u(x,t) is shown in Fig. 1.

xl characterizes the temperature of the glass wall, x2 the temperature of the concrete walls, U1 the ratio i(t)/i max where i(t) is the supplied power (in the form of heating), i max is the maximum available power and U2 is the external insulation (in the form of shutters on the glass wall).

U),

Writing Kirchoff' s current laws at the nodes (i) and t» of the circuit above we get the state equations (8) and the output T2 (for more details see (Bourdache-Siguerdidjane». The output T2 is the ambient temperature of the room. The criterion J to be minimized is the integral over a finite horizon of energy costs plus a quadratic penalty function which takes into account the user's comfort and a second quadratic penalty function which tends to keep the external insulation within physically admissable limits. w1(t) and w2(t) are functions of I(t) and Tex(t). The coefficient a . . , bi, a , 8, y are constants, 1J . ' K1 , K2 are the we1ght1ng parameters. Th e electr1city tarrifs are included in K1' TZ is the desired temperature of the room. Fig. 1.

u20 is the desired nominal value around wich the value of u2 must oscillate. In view of equation (5) we have

2:J(:~:)+(:~:1 :~:2) (:~) =(:~) 1

Example 2

2

with the Cauchy conditions

Let us consider the bilinear system

(8)

fT

Xl

x

[(T -T )2 + K u + K (U -U )2] dt 2 2 20 1 2 2

2

2 -2 y(a F 1+BF2 )- 2C1KZ(U2-U20) f(x,t) -2 (T 2-T:) D2 - D3

to

er ex (t)

2K2(u2-u20) f(x,t) This model is derived from a simplified problem in building heat transfer. We are interested in a single dwelling room. The wall facing south is composed entirely of double glass panes (supposed opaque) and all the other walls are made of concrete. The inputs are the solar flow I(t) and the external temperature Tex(t). Those used here correspond to typical values for an April day in the Trappes region of France.

+ bi(t} _ F1)

where D. are constants 1

I(t) = 8(159,9 - 232,65 cos wt + 68,57 cos 2 wt + 18,33 cos3wt) T (t) = 8 + 3,5 cos (wt + 2,62) ex -1 w = (2n/86400) s

30

H. Bourdache-Siguerdidjane and 1\1. Fliess

since it is difficult for us to plot the hypersurfaces ul(x I ' x 2 ' t) and u 2 (x I ' x 2 ' t), we show only the evolution of xI' x 2,u l' u 2 and the output T2 during a period of 24 hours. The authors would like to thank Jessy Grizzle from Urbana University, Illinois, for revising the english version of this paper.

12.121

e.e 12

3121.121

18.0

••••

Auxll.

e. , e .•

e.2 time 'In nouroS

e.e

12

35.13 28.0 21 .0

1-4.121

Output

TZ-f(t)

12.121

'.e tlme'ln nour .s

Fig. 3.

REFERENCES "L'BREKHT E.G (1962). On Optimal stabilizationofnonlinear system. J. Appl. Math. Mech., 25, 1254-1266 BELL D.J., YE Q. (1981). A perturbation method for sub-optimal feedback control of bilinear system. Internat. J. Systems Sci., 12, 1157-1168. BOURDACHE-SIGUERDIDJANE H. (1985) Contribution au calcul des lois de bouclage en commande optimale non lineaire. These de Docteur-Ingenieur, Universite Paris XI (ORSAY). BREWER J.W. (1978). Kronecker products and matrix calculus in system theory. IEEE Trans. Circuits Systems, 25, 772-781. BRUNOVSKY P. (1967) On optimal stabilization of nonlinear system. Mathematical Theory of Control, A.V. Balakrishman and Lucien W. Neustadt, eds., Academic Press, New York and London. BRYSON A.E., HO C.Y (1975) Applied Optimal Control. John Wiley, New York. COURANT R., HILBERT D. (1962) Methods of Mathematical Physics. Vol II, Interscience, New York. FLIESS M. (1984). Lie brackets and optimal nonlinear feedback regulation. Proc. IXth IFAC World Congress, Budapest. FLIESS M., BOURDACHE-SIGUERDIDJANE H. (1984). Quelques remarques elementaires sur le calcul des lois de bouclage en commande optimale nonlineaire. Proc. 6th Internat. Conf. Analysis Optimiz. System, Nice, June 1984, Notes Control Informat. Sci., Berlin: Springer, 499-512. GOLDSTEIN H. (1956) Classical mechanics. Addison Wesley Publ. Cy, Inc., Cambridge 42, Mass. KRENER A.J. (1977) The high order maximal principle and its application to singular extremals. SIAM J. Control Optimiz, IS, 256-293. LAMNABHI-LAGUARRIGUE F. (1985) Series de Volterra et commande optimale singuliere. '£hese d I Etat, Universite de Paris XI. LEBORGNE D. (1982) Calcul differentiel et geometrie Presses Universitaires de France, Paris. LUKES D.L. (1969) Optimal r egu lation of nonlinear dynamical systems. SIAM J. Control, 7, 75-100. ROUFF M. to submit WILLEMSTEIN A.P. (1977) Optimal regulation of nonlinear dynamical system on a finite interval SIAM J. Control Optimiz, IS, 1050-1069.