- Email: [email protected]
Control and Estimation in Stochastic Hereditary Systems
Co pyri ght © IFAC Stoc hastic Control Vilnius. Lithuan ian SSR . L'SSR . 1986
CONTROL AND ESTIMATION IN STOCHASTIC HEREDITARY SYSTEMS
v.
B. Kolmanovskii
Institl/te for Problellls ill .Heel/ rmirs , L'SSR Aeadnll\' of Seimefs , ,\1oseolt'
Abstract. A correct description ef a number phenomena in the control theory, mechanics, radio-physics, biology, immunology, economics, robototechnique and so on needs the utilization of hereditary systems. One of the problems, arising in this case, is connected with optimal control and esimation under stochastic perturbations. In this paper the optimal conditions are formulated, the control problems for linear and quasi-linear systems with square minimizing functional are studied, the duality relation between control and observation are established, and the formulas for optimal estimates of optimal hereditary system states are derived. Keit0rds. Optimal control; hereditary systems; filtering; stochastic dis urbances. HEREDITARY SYSTEMS CONTROL. OPTIMALITY CONDITIONS Consider the following system dx(t)=f(t,xt,u)dt+ S'(t,xt,U)d ~ (t); o~t~T,
LUy(t,x, 'V (t+9) ).L ov'I'(t ,x)+ fO (t,ijit,UW'f(t,x)/~x, Lo.~Tr(02V //ox 2 ) tS'(t,ii\,u) , fi'(t,ijit,u)+ 'dV,+,(t,x)/'Ot, (1.4) where by prime the transposition is denofed and by Tr the trace of matrix is denoted, 1 (t).x, .::v (t+El). 'V (h9). Theorem 1.1. (Kolmanovskii, Maizenberg, 197 3;19 77 ). Suppose that there exist the functional V (t,x, 'f (9» and the control u~(t, ~t) which satisfy to the conditions inf[LUv0 (t, IV (t), IV (t+8 »+F 1 (t, ~(t).:.u(t, 'i't» .LuOVo(t, 'V(t), 'V(t+El»+F 1 (t, '¥(t),uo(t, ~t»· cO (1.5) Vo(T, 'i' (T), It' (T+8»=F('V(T))(L6) for all t E[ O, T] and all admissi~ ble control u(t,x t ). Then the cont rol u is the optimal and u 0 0 ( 'f ) .. V0 ( 0 , If ( 0 ), If (t+9». Theorem 1.1 may be considered as Bellman'S prinCiple for time-lag systems (Bellman,195 7 ). Bellow the cases, when the theorem 1.1 allows effectively to construct the optimal control and to find the minimum value of index performance (1.) will be studied.
(1.1)
where vector x( t) E. ~,Rn is n-dimentional Euclidlan space, vector fERn and matrix ~ are given, l§. (t)ERl is the standard Wiener's process~instants to, Tare gi ven and xt.x( t+8), ~ ~ 0 is denoted. Solution of equation (1.1) is determined by initial condition x o.X(8). 'fee), 8~0 (1.2) where ~ is a given function. The control u E U must be choosed from the condition of the minimum of the functional (where U - is the given set of the space Rm, F, F1 are the given scalar functions, M is expectation) M [ F(X(T»+~ F 1 (t,x(t),u»dt • U( "j)(1}) Optimal control for problem (1.1),(1.3) we seek in the class of functionals of the form u.u(t,x t ). The control u(t,x t ) € U is the admissible control, if there exists the solution of problem (1.1) (1.Z) for u(t,x t ) and the functional (1.3) is ~ounded.Cons1der class D of scalar functionals V(t,x, ~ (9», 9 ",-0 such that function V ~ (t,x).V(t,x, 'f (t+8» is twice continuosly differentiable with respect to x and its derivative is bounded for all t E [t ,T] for arbitrary piecewise continuos 0 restricted function 'V (s )€~, s «00 and arbitrary t ~to' Ux€ R... We introduce in D the operator L sucH that
r
a
LINEAR- QUADRATI C PROBLEM Let the motion equation (1.1) be linear
3 11
V. B. Kolmano\'skii
312
B2 (t)_-B 2 (t+h)A(t+h),
0 ~ t ~ T,B 2 (T)-I,
B2 (s)-0, s >T. P1(t)-P1(t)B2(t)B1(t)B2(t)P1(t), (1.14) Then the solution P,Q,R problem (1.10), (1.11) is equal to P(t)-B 2 (t)P 1 (t)B 2 (t), Q(t,"t" )_-B 2 (t)P (t)B 2 (t+
,S )(1.15)
(1. 20) where M~ is mean value, computed under condition of that the trajectory of the process is gi ven for CC' ~ t and coincides with the given function ,process y(~) is determined by the stochastic equation without affere:fect dyeer;' )--B 1 ('I:' .jP(
in the unique instant of discontinuity t-T-h the derivative of function B is determined by the continuity from 2the left. Correctness of formulas (1.15) are verified by its substitution in (1.10), (1.11). If matrix No is nonsingular, the equation (1.14) is reduced to linear with the help of transfer to reciprocal matrix P 1-. p :,1. Approximate control for quasi-linear systems. The construction of optimal control based on the method of dynamic programming is adjoint with significant difficulties. Prasence of small parameter in the systems equations allows to develop the approximate method, based on expansion of Bellman's functional on degrees of small parameter (Kolmanovskii,1977;Kolmanovskii Schaichet, 1978). This method for hereditory systems based the construction of auxiliary control problems, the solutions of which coincide with introduced sequential approximations. Consider explicitly quasi-linear system (1.1) in the form
In order to obtain i-approximation uito optimal control it is sufficient to substitute in the right part of (1.18) i-partial sum from (1.19) instead of V. The difference between optimal and suboptimal values of minimizing functional is the value of t i+1 - order.
dx(t)-( E f(t,Xt)+B(t)u)dt+~(t)d 3 (t)(1.16)
O~t$T
with minimazing functional (1.3), (1.5). From (1.4)-(1.6) the Bellman equation in this case is in the form iijf LoV'f/(t,x)+( E..f(t, '" )+Bu) , ~ VIjI(t,x)1
-
i
1
-i
ESTIMATION OF THE STATES FOR THE HEREDITARY SYSTEMS Consider linear systems in a form (1.7) dx(t).A(t)x(t-h 1 )dt+ ~1(t)d ~ 1(t), (2.1)
(2. 2 ) x(s)-e, s <0, x(O)-x o Beyond systems continuous observations are conducted dy(t )cg( t )x( t-h)dt+ ~ 2( t)d ~ 2( t),
Id x+x'N 1x+u'N 2u .0, V (T,x)"X'N ox (1.17) 4I From (1.17) follows the formula for optimal control
uo(t'Xt)"'-;N~1(t)B'(t) oVx (t+9) (t~X(t»/dX(t)
-
Si(t,'f' ) .. f'(t, 'I')'dv ~ (t,x)/'Oxi-1 . 2: (d vJ(t,x)/ o X)'B 1 (t) j.1 '" 'd vj~i(t ,x)1 ~x
(1.18)
Present the solution V of equation (1.17) in the form V( t , x, IjJ ) _ vo ( t , x, \V ) + E. V1 ( t ,x, \jI)+ ••• ( 1 • 19 ) With a view to define the coefficient in that expansion we substitute (1.19) in (1.17) and equal to zero the coefficients on identiaal degrees of E. Function VO.x'P(t)x+g(t). The others approximations are the solutions of linear equations, for which the probability representation is correct
O~t~T,
yo(O)-O
(2.3)
In the equations (2.1)-(2.3) vectors x(t)€.R n , y(t)€.~, matrix A, (5' l' g, (S 2wi th piecewise continuous components are given, constant delays h1' h 3 0, by ~i are denoted the standard Wiener's processes, Xo is Gauss random variable such that Mxo.O, Do·MxoX~ 0, variables x o ' ~ 1,52 are mutual undependent. The problem is construction of the best in the mean-square estimation meT) for the vector x(T) under the observations yet) on the segment [O,T] • Assume that D(T).M [x(T)-m~~rx(T)-m(T)J". Since conditiona~ probability x(T.j Under YT is gaussian, then T
meT).
S u(t)dy(t) o
(2.4)
Control and Estima tion
From (2.1)-(2.4) it follows that matrix u(t) in (2.4) is the optimal control for determinate problem
. Uv (t
).-A' (t+h.1)
d..(~).O,'t>T,
Define z(t,s) by relations "d Z(t,S)/d t .. A(t) Z(t,s), Z(s,s)=I Then
d.. (t+h 1 )+g' (t+h)u( t+h). oUT~.I
3 13
(2.5)
T
~ 1+
D( t )-DA' -AD.. - DZ, (t )D+DZ,
J.TrD(T).Tr[c(.'(O)D d...(O)+S<<<.'(t)N,(t) o 0 d..(t)+U'(t)N 2 (t)U(t»dtJ-min u, (2.6)
+W 1 Z1D+[
where N • (S' ~, > 0, N • ~ el' , functions A a~d g2exterior io e~gme~t [O,T] are equal to zero. Solution of the problem (2.5),(2.6) is presented in § 1. Using it we deduce the expressions for estimation meT) and covariance matrix D(T). We have
Z1(t)c Z'(t-h,t)g'(t)( N2 (t»-1
U(t).(N 2 (t»-1 g (t) [P(t-h)ol(t-h)+
o
S
-h T~
t
Q (t-h,'1:)ol(t-'t-h)d~,
(2.7)
G'1(t)(f1(t)-t1Z'~1]' (2 . 11 )
g(t)Z(t-h,t), -
~1·
5t
Z(t,s)N,(s) Z'(t,S)dS t-h The equat ion f or estimat i on has the f orm t
1 ~h;u(t).o,O~
dm( t )-A(t)m(t)dt -(D( t )-
t< h;D(T).P(T)
5 a,(S)dS)
t-h
Mdtrices P, Q,R satisfy conditions
Z '(t, t -h)g'(t)( N2(t»~1
(B (t-h).g(t)(N (t»-1 g (t» o 2 P(t).Q(t,O)+Q'(t,O)+N (t)-P(t)B (t)P(t), 1 o R(t, O,
(dy(t)-g(t) Z(t-h,t)m(t)dt),
er: )=0, «(jdt + ~
Q(t,
(2.8)
+ ddg )R(t,'t, +Q,(t,'t' )Bo(t) Q(t, g ).0
f >+
Boundary conditions for equati ons (2.8) are form P(O)-Do, Q( O, 't ~
) .. R( O, 't" ,
f
)-O,-h 1 <
er, f
~
0
A(t+h1)P(t)- Q '(t1-h1). O , O ~ t~
T
2A(t+h 1 )Q(t,
m( O).mo ; c ) The case h .O and the equati on (2.1) correct 1only f or t~ o with initial conditi on (2.1 0 ). Then t he opt imal filter is des cribed by the equations (2.11),(2.12), where before the right terms is the common mu1 tipl1er ':t (t-h) . where 'X (t)- O for t~ 0 and 'X- (t). .1 f or t> O. Observe that in the same way one can obtain the solution for the problems of ext rapolati on and i nterpolati on f or equations (2.1), (2.3) with arbitrary affereffect.
(2.9) ,-h1~
-0
Thus, algorithm f or constructin~ of M(T) consists in the f ollowing (Ko1manovskii, 1974) 1) find P and Q, s o lvin ~ the problem ( 2. 8 ) , ( 2.9); 2 ) find ol. (t ), s olving the problem (2.5) f or control u(t), determined from formula (2.7), in which matrix P and Q are substituted; 3) find u(t), substituting determinated P, Q, in (2. 7 ); 4) at last D(T) is calculated from f ormula (2.7) and meT) is computed from (2.4). We consider s ome cases, when the filtrat ion al gorithm may be simp1ifed: a) 1f~ - 0 then problem (2. 8 ), ( 2.9) has explicit s oluti on in the form (1.15); b) The case hi- O and equati ons (2.1) are co rre c t f or a 1 t ~ -h. The initial condition is x(O)-x o , Mx 0 =m0 , M(xo-mo)(xo-mo ) '=Do
( 2.1 2)
( 2.1 0 )
CONTROL OF STOCHA~TI C HEREDITARY SYSTEMS AND UNKNOWN PARAMETERS (THE RESULTS OF THIS SECTION HAVE BEEN OBTAINED WITH L.E.SHAI CHET) Consider t he problem of control f or the systems having the f orm (1.1) with unknown parameter "t}€Rk dx(t).f(t'Xt, U )~dt
+
Ei' ( t ,x t ' u ) d ~ ( t ) , 0
t ~T
~
(3.1)
with initial condition (1.2) and index performance (1. 3 ).~ C on ditiona1 probabilit y distribution under condition '1 to is gaussian variable with mean m~(O) and covariance matrix D (0 ). Denote by ~ t
m.,., (t)-M('ry /Xto),D", (t)M [('l)-m'l') (t»(1 -m'l(t»' /xio
1.
314
V. B. Kolmano\'skii
dX(t).[A(t)x(t-h)+B(t)u] dt+
+6' (t)d 3 (t)
(1.7)
The functional (1.3) has a quadratic form (1.8) where matricies N are piecewise continuous, bounded an~ positive semidefinite, matrix N2 >0, i.e. it is positive definite. We shall seek the functional Vo ' satisfiing to the conditions (1.5), (1.6), in the quadratic form Vo(t,x, IfJ (t+9».x'P(t)x+x'
o
S
Q(t,'t' )'¥(t')
-b
)Q'(t,~ )xdcr + o 0 -h S S 'tI' ('t' )R( t, 't' , f) 't'(f) d't d ~
d
f't"(ct'
Consider the methods of construction of exact and approximate solutions problem (1.10),(1.11) for h ~ O. I f h.o then Vo (t,x,lt'(t+9».x'P(t)x+g(t)u and the boundary problem (1.10), (1.11) is reduced to Riccati's equation P(t)+A'P+PA-PB~P+N1.0, P(T).N for the matrix P and quadra£ure, which defines g( t).
SEQ1)ENTIAL APPROXIMATION OF SOLUTION FOR PROBLEM (1.10), (1.11)
(1.9)
-h -h
In order to define matricies P,Q,R and scalar function ~ in (1.8) we substitute (1.9) in (1.5), (1.6) and equate to zero the corresponding forms of If • Vii th respect to (1.7),(1.4) we get the equations;
Define arbitrary continuous matricies P (t),Q (t,cr).Construct the sequgnce o~ solutions for (1.10), (1.11), in which the nonlinear terms are linearized by the following rule PB 1 P -+Pi - 1B1 Pi +P i B1 Pi-1- Pi-1 B1~,
.(P(t).dP(t)/d+)
PB 1Q -Pi-sB1Qi+PiB1Qi-s-
B 1.. BN2 -1 B',
Q'B 1Q QiB1Qi-s+Q'i-1B1Qi-
o d R(t,O, er: )+(~,- ~,)Q(t, er )-P(t)B 1' Q(t,cr).O, 06t~T "0 d d ( W~,- W)R(t,'t',~ )-Q'(t,'t')B 1
- Q'i-s B1Qi-s } ~=i
(1.13)
P(t)+Q(t,0)+Q'(t,0)-P(t)B1(t~N1(t)=0,
Q(t, ~ ).0,
-h<:. CZ:-, ~ ~ 0
get ).-Tr &' (t) 6" (t)p(t) ,g-dg(t)/dt (1.10) Boundary conditions are P(T)=N o ' Q(T,'r).R(T, 't
'f ).. g(T)=O
A'(t)P(t)-Q'(t -h) .. O, 2A'(t)Q(t,cr)1 R(t,-h,'t )-R'(t, cr,-h).O, R(" f ,'t"). • R'(t,'t',
f)
(1.11)
The Optimal control u for problem (1.7), (1.3), (1.8) is equalOto U
o (t,x t )·-N;1(t)B'(t)[P(t)x(t) +
o
+ _~ Q(t,s)x(t+s)ds
J
We plot i- approximation of u. to the optimal control by the ~ formula (1.12) in which instead P and Q there are Pi - s and Qi-s' Under the control u the value of functionals (1.3~, (1.8) is given by the right part of (1.9) in which P,Q,R,g is replaced by Pi,Qi,Ri,gi' The solution (Pi,Qi,R i ) of linear problem (1.10), (1.11),(1.13) is derived in the explicit form aDd V. (t,x,1jI (t+~)) Vi 1(t,x, (t+9» fOr any i ~ 1, t c + [0, T ] , x eRn' The sequence (Pi,Qi,R i ) converge to exact solution of problem (1.10), (1.11) and the norm of difference between exact solution and i-approximation to it is a 1/i! - order value (Kolmanovskii, 1974 b).
(1.12)
Minimum value of index perfomance (1.3), (1.8) is V (0, ~(t ), '-g (t +8». Thus the soluti8n of liBear-quaRratic control problem (1.7), (1.3), (1.8) is completely defined by boundary problem (1.10), (1.11). Notice , that under certain assumptions on N. and piecewise continuity of matricies~A(t) and B(t), there exists unique solution of problem (1.10), (1.11) in the class of absolutely continuous functions with almost every where bounded derivatives.
EXACT SOLUTIONS OF LlNEAR-QUADRA'l'IC PROBLEM If in the index perfomance (1.3), (1.8) matrix N1.O, the problem (1.10), (1.11) allows the exact solution (Kolmanovskii, 1977). Define matricies B2 and B1 with the help of relations (I is unit matrix)
315
Control and Estimation
Equations, determining m and D ,have the form (Liptser, Shiryaev, 1978) (t)-D (t)f'( G' ~,)-1 [dx(t)"1 '1 -f(t,xt,u)m dtJ ' dm
(3.2)
•
'I')
D (t)--D
(t)B D'h (t),B f' ( () (, )-1 f
~
~
0
1
0
The process in the quadratic brackets from equation (3.2) is the Wiener's one with the same diffuSion matrix as in (3.1). _ Consider scalar functionals W( t ,x, ~ (e), y, z), where yE: Rand Z is nxn matrix. For fixed ~ we d~fine !he finction W .g(t,xty,z)-W(t,x, 'i (e),y,z). Similarly to (1.4) we determine the operator Lu
LuW.L~+~ Tr
(o:W'9/0y2)ZBoZ+
+2( ~ 2W\Jj /"0 x 0 y)zf'-2 -(/W'9 / ~zBoZ. where LUis given by the formula (1.4). Now the can establish the results similarly to the results of 61 fo r the problem (3.1),(1.3). Nam~ly, there is correct the analog of theorem 1.1, where V and LU are replaced on Wand L , respectively. Further, if the s~stem (3.1) is linear with respect to x,u, "7 and functional (1.3) is quadratic the control problem has the explicit solution determined by the system of determinate equations (1.10), (1.11). Finally for the quasi-linear case we have constructed the sequential approximations to the optimal control of quasilinear equations. Notice that for the systems without time delay the control problem with unknown parameters was studied in Kolosov (1984). The control problems for another hereditary systems are considered in Chernousko, Kolmanovskii (19 78), Kolmanovskii, Nosov (1981), Schaichet (1984). REFERENCES Bellman, R.E. (1957). ~amic programmiig, Princeton, N•• : Princeton Un versity Press. Chernousko, F.L., V.B.Kolmanovskii (1978) 0rtimal control under stochastic d sturbances. Nauka, Moscow (in Russian) • Kolmanovskii, V.B. (1974a). Filtration of stochastic processes with delay. Automation and remote control, 1, 42-49. Kolmanovskii, V.B. (1974b). Approximation of linear control system with delay. Problems of control and information the0Et' 1, 63-76. Kolmanovskii. V.B. 19 Optimal control of stochastic hereditary systems. (A survey). Trudy of the fourth All-Union conference on theory and atrlications of differential enua ons With deViating arfSment.aUka DUIDka. kiev, pp.177-~5. Kolma!iOVskii, V.B., T.L.Maizenberg. (1973). Optimal control of stochastic retarded systems. Automation and remote control, 1,
17).
Kolmanovskii, V.B., T.L.Maizenberg. (1977). Optimal estimation of systems states and problems of control of systems with delay. Prikl. Mat em. i Mech., 41~3:44b=456. Kolmanoyskii, V.B., V•• Nosov (1981). Stabilit and periodic modes of con rol systems With aftereffect. NaUka, Moscow (in Russian). Kolmanoyskii, V.B., L.E.Sch&ichet (1978). Approximate feedback control of quasilinear stochastic retarded systems. Prikl. Matem. i Mech., 42,6, 978-988. Kolosov, ·G.E. (1984). DeSign of optimal automatic systems under stochastic disturbances. Nauka, Moscow (in Russian). Liptser, R.S., A.N.Shiryaev. (1978). Statistics of random irocesses. Springer, Ber! n. Schaichet, L.E. (1984). Optimal control of Volterra equations Problems of control and information theory. 13.3.
t
141-152.
CONTROL AND ESTIMATION IN STOCHASTIC HEREDITARY SYSTEMS
v.
B. Kolmanovskii
Institl/te for Problellls ill .Heel/ rmirs , L'SSR Aeadnll\' of Seimefs , ,\1oseolt'
Abstract. A correct description ef a number phenomena in the control theory, mechanics, radio-physics, biology, immunology, economics, robototechnique and so on needs the utilization of hereditary systems. One of the problems, arising in this case, is connected with optimal control and esimation under stochastic perturbations. In this paper the optimal conditions are formulated, the control problems for linear and quasi-linear systems with square minimizing functional are studied, the duality relation between control and observation are established, and the formulas for optimal estimates of optimal hereditary system states are derived. Keit0rds. Optimal control; hereditary systems; filtering; stochastic dis urbances. HEREDITARY SYSTEMS CONTROL. OPTIMALITY CONDITIONS Consider the following system dx(t)=f(t,xt,u)dt+ S'(t,xt,U)d ~ (t); o~t~T,
LUy(t,x, 'V (t+9) ).L ov'I'(t ,x)+ fO (t,ijit,UW'f(t,x)/~x, Lo.~Tr(02V //ox 2 ) tS'(t,ii\,u) , fi'(t,ijit,u)+ 'dV,+,(t,x)/'Ot, (1.4) where by prime the transposition is denofed and by Tr the trace of matrix is denoted, 1 (t).x, .::v (t+El). 'V (h9). Theorem 1.1. (Kolmanovskii, Maizenberg, 197 3;19 77 ). Suppose that there exist the functional V (t,x, 'f (9» and the control u~(t, ~t) which satisfy to the conditions inf[LUv0 (t, IV (t), IV (t+8 »+F 1 (t, ~(t).:.u(t, 'i't» .LuOVo(t, 'V(t), 'V(t+El»+F 1 (t, '¥(t),uo(t, ~t»· cO (1.5) Vo(T, 'i' (T), It' (T+8»=F('V(T))(L6) for all t E[ O, T] and all admissi~ ble control u(t,x t ). Then the cont rol u is the optimal and u 0 0 ( 'f ) .. V0 ( 0 , If ( 0 ), If (t+9». Theorem 1.1 may be considered as Bellman'S prinCiple for time-lag systems (Bellman,195 7 ). Bellow the cases, when the theorem 1.1 allows effectively to construct the optimal control and to find the minimum value of index performance (1.) will be studied.
(1.1)
where vector x( t) E. ~,Rn is n-dimentional Euclidlan space, vector fERn and matrix ~ are given, l§. (t)ERl is the standard Wiener's process~instants to, Tare gi ven and xt.x( t+8), ~ ~ 0 is denoted. Solution of equation (1.1) is determined by initial condition x o.X(8). 'fee), 8~0 (1.2) where ~ is a given function. The control u E U must be choosed from the condition of the minimum of the functional (where U - is the given set of the space Rm, F, F1 are the given scalar functions, M is expectation) M [ F(X(T»+~ F 1 (t,x(t),u»dt • U( "j)(1}) Optimal control for problem (1.1),(1.3) we seek in the class of functionals of the form u.u(t,x t ). The control u(t,x t ) € U is the admissible control, if there exists the solution of problem (1.1) (1.Z) for u(t,x t ) and the functional (1.3) is ~ounded.Cons1der class D of scalar functionals V(t,x, ~ (9», 9 ",-0 such that function V ~ (t,x).V(t,x, 'f (t+8» is twice continuosly differentiable with respect to x and its derivative is bounded for all t E [t ,T] for arbitrary piecewise continuos 0 restricted function 'V (s )€~, s «00 and arbitrary t ~to' Ux€ R... We introduce in D the operator L sucH that
r
a
LINEAR- QUADRATI C PROBLEM Let the motion equation (1.1) be linear
3 11
V. B. Kolmano\'skii
312
B2 (t)_-B 2 (t+h)A(t+h),
0 ~ t ~ T,B 2 (T)-I,
B2 (s)-0, s >T. P1(t)-P1(t)B2(t)B1(t)B2(t)P1(t), (1.14) Then the solution P,Q,R problem (1.10), (1.11) is equal to P(t)-B 2 (t)P 1 (t)B 2 (t), Q(t,"t" )_-B 2 (t)P (t)B 2 (t+
,S )(1.15)
(1. 20) where M~ is mean value, computed under condition of that the trajectory of the process is gi ven for CC' ~ t and coincides with the given function ,process y(~) is determined by the stochastic equation without affere:fect dyeer;' )--B 1 ('I:' .jP(
in the unique instant of discontinuity t-T-h the derivative of function B is determined by the continuity from 2the left. Correctness of formulas (1.15) are verified by its substitution in (1.10), (1.11). If matrix No is nonsingular, the equation (1.14) is reduced to linear with the help of transfer to reciprocal matrix P 1-. p :,1. Approximate control for quasi-linear systems. The construction of optimal control based on the method of dynamic programming is adjoint with significant difficulties. Prasence of small parameter in the systems equations allows to develop the approximate method, based on expansion of Bellman's functional on degrees of small parameter (Kolmanovskii,1977;Kolmanovskii Schaichet, 1978). This method for hereditory systems based the construction of auxiliary control problems, the solutions of which coincide with introduced sequential approximations. Consider explicitly quasi-linear system (1.1) in the form
In order to obtain i-approximation uito optimal control it is sufficient to substitute in the right part of (1.18) i-partial sum from (1.19) instead of V. The difference between optimal and suboptimal values of minimizing functional is the value of t i+1 - order.
dx(t)-( E f(t,Xt)+B(t)u)dt+~(t)d 3 (t)(1.16)
O~t$T
with minimazing functional (1.3), (1.5). From (1.4)-(1.6) the Bellman equation in this case is in the form iijf LoV'f/(t,x)+( E..f(t, '" )+Bu) , ~ VIjI(t,x)1
-
i
1
-i
ESTIMATION OF THE STATES FOR THE HEREDITARY SYSTEMS Consider linear systems in a form (1.7) dx(t).A(t)x(t-h 1 )dt+ ~1(t)d ~ 1(t), (2.1)
(2. 2 ) x(s)-e, s <0, x(O)-x o Beyond systems continuous observations are conducted dy(t )cg( t )x( t-h)dt+ ~ 2( t)d ~ 2( t),
Id x+x'N 1x+u'N 2u .0, V (T,x)"X'N ox (1.17) 4I From (1.17) follows the formula for optimal control
uo(t'Xt)"'-;N~1(t)B'(t) oVx (t+9) (t~X(t»/dX(t)
-
Si(t,'f' ) .. f'(t, 'I')'dv ~ (t,x)/'Oxi-1 . 2: (d vJ(t,x)/ o X)'B 1 (t) j.1 '" 'd vj~i(t ,x)1 ~x
(1.18)
Present the solution V of equation (1.17) in the form V( t , x, IjJ ) _ vo ( t , x, \V ) + E. V1 ( t ,x, \jI)+ ••• ( 1 • 19 ) With a view to define the coefficient in that expansion we substitute (1.19) in (1.17) and equal to zero the coefficients on identiaal degrees of E. Function VO.x'P(t)x+g(t). The others approximations are the solutions of linear equations, for which the probability representation is correct
O~t~T,
yo(O)-O
(2.3)
In the equations (2.1)-(2.3) vectors x(t)€.R n , y(t)€.~, matrix A, (5' l' g, (S 2wi th piecewise continuous components are given, constant delays h1' h 3 0, by ~i are denoted the standard Wiener's processes, Xo is Gauss random variable such that Mxo.O, Do·MxoX~ 0, variables x o ' ~ 1,52 are mutual undependent. The problem is construction of the best in the mean-square estimation meT) for the vector x(T) under the observations yet) on the segment [O,T] • Assume that D(T).M [x(T)-m~~rx(T)-m(T)J". Since conditiona~ probability x(T.j Under YT is gaussian, then T
meT).
S u(t)dy(t) o
(2.4)
Control and Estima tion
From (2.1)-(2.4) it follows that matrix u(t) in (2.4) is the optimal control for determinate problem
. Uv (t
).-A' (t+h.1)
d..(~).O,'t>T,
Define z(t,s) by relations "d Z(t,S)/d t .. A(t) Z(t,s), Z(s,s)=I Then
d.. (t+h 1 )+g' (t+h)u( t+h). oUT~.I
3 13
(2.5)
T
~ 1+
D( t )-DA' -AD.. - DZ, (t )D+DZ,
J.TrD(T).Tr[c(.'(O)D d...(O)+S<<<.'(t)N,(t) o 0 d..(t)+U'(t)N 2 (t)U(t»dtJ-min u, (2.6)
+W 1 Z1D+[
where N • (S' ~, > 0, N • ~ el' , functions A a~d g2exterior io e~gme~t [O,T] are equal to zero. Solution of the problem (2.5),(2.6) is presented in § 1. Using it we deduce the expressions for estimation meT) and covariance matrix D(T). We have
Z1(t)c Z'(t-h,t)g'(t)( N2 (t»-1
U(t).(N 2 (t»-1 g (t) [P(t-h)ol(t-h)+
o
S
-h T~
t
Q (t-h,'1:)ol(t-'t-h)d~,
(2.7)
G'1(t)(f1(t)-t1Z'~1]' (2 . 11 )
g(t)Z(t-h,t), -
~1·
5t
Z(t,s)N,(s) Z'(t,S)dS t-h The equat ion f or estimat i on has the f orm t
1 ~h;u(t).o,O~
dm( t )-A(t)m(t)dt -(D( t )-
t< h;D(T).P(T)
5 a,(S)dS)
t-h
Mdtrices P, Q,R satisfy conditions
Z '(t, t -h)g'(t)( N2(t»~1
(B (t-h).g(t)(N (t»-1 g (t» o 2 P(t).Q(t,O)+Q'(t,O)+N (t)-P(t)B (t)P(t), 1 o R(t, O,
(dy(t)-g(t) Z(t-h,t)m(t)dt),
er: )=0, «(jdt + ~
Q(t,
(2.8)
+ ddg )R(t,'t, +Q,(t,'t' )Bo(t) Q(t, g ).0
f >+
Boundary conditions for equati ons (2.8) are form P(O)-Do, Q( O, 't ~
) .. R( O, 't" ,
f
)-O,-h 1 <
er, f
~
0
A(t+h1)P(t)- Q '(t1-h1). O , O ~ t~
T
2A(t+h 1 )Q(t,
m( O).mo ; c ) The case h .O and the equati on (2.1) correct 1only f or t~ o with initial conditi on (2.1 0 ). Then t he opt imal filter is des cribed by the equations (2.11),(2.12), where before the right terms is the common mu1 tipl1er ':t (t-h) . where 'X (t)- O for t~ 0 and 'X- (t). .1 f or t> O. Observe that in the same way one can obtain the solution for the problems of ext rapolati on and i nterpolati on f or equations (2.1), (2.3) with arbitrary affereffect.
(2.9) ,-h1~
-0
Thus, algorithm f or constructin~ of M(T) consists in the f ollowing (Ko1manovskii, 1974) 1) find P and Q, s o lvin ~ the problem ( 2. 8 ) , ( 2.9); 2 ) find ol. (t ), s olving the problem (2.5) f or control u(t), determined from formula (2.7), in which matrix P and Q are substituted; 3) find u(t), substituting determinated P, Q, in (2. 7 ); 4) at last D(T) is calculated from f ormula (2.7) and meT) is computed from (2.4). We consider s ome cases, when the filtrat ion al gorithm may be simp1ifed: a) 1f~ - 0 then problem (2. 8 ), ( 2.9) has explicit s oluti on in the form (1.15); b) The case hi- O and equati ons (2.1) are co rre c t f or a 1 t ~ -h. The initial condition is x(O)-x o , Mx 0 =m0 , M(xo-mo)(xo-mo ) '=Do
( 2.1 2)
( 2.1 0 )
CONTROL OF STOCHA~TI C HEREDITARY SYSTEMS AND UNKNOWN PARAMETERS (THE RESULTS OF THIS SECTION HAVE BEEN OBTAINED WITH L.E.SHAI CHET) Consider t he problem of control f or the systems having the f orm (1.1) with unknown parameter "t}€Rk dx(t).f(t'Xt, U )~dt
+
Ei' ( t ,x t ' u ) d ~ ( t ) , 0
t ~T
~
(3.1)
with initial condition (1.2) and index performance (1. 3 ).~ C on ditiona1 probabilit y distribution under condition '1 to is gaussian variable with mean m~(O) and covariance matrix D (0 ). Denote by ~ t
m.,., (t)-M('ry /Xto),D", (t)M [('l)-m'l') (t»(1 -m'l(t»' /xio
1.
314
V. B. Kolmano\'skii
dX(t).[A(t)x(t-h)+B(t)u] dt+
+6' (t)d 3 (t)
(1.7)
The functional (1.3) has a quadratic form (1.8) where matricies N are piecewise continuous, bounded an~ positive semidefinite, matrix N2 >0, i.e. it is positive definite. We shall seek the functional Vo ' satisfiing to the conditions (1.5), (1.6), in the quadratic form Vo(t,x, IfJ (t+9».x'P(t)x+x'
o
S
Q(t,'t' )'¥(t')
-b
)Q'(t,~ )xdcr + o 0 -h S S 'tI' ('t' )R( t, 't' , f) 't'(f) d't d ~
d
f't"(ct'
Consider the methods of construction of exact and approximate solutions problem (1.10),(1.11) for h ~ O. I f h.o then Vo (t,x,lt'(t+9».x'P(t)x+g(t)u and the boundary problem (1.10), (1.11) is reduced to Riccati's equation P(t)+A'P+PA-PB~P+N1.0, P(T).N for the matrix P and quadra£ure, which defines g( t).
SEQ1)ENTIAL APPROXIMATION OF SOLUTION FOR PROBLEM (1.10), (1.11)
(1.9)
-h -h
In order to define matricies P,Q,R and scalar function ~ in (1.8) we substitute (1.9) in (1.5), (1.6) and equate to zero the corresponding forms of If • Vii th respect to (1.7),(1.4) we get the equations;
Define arbitrary continuous matricies P (t),Q (t,cr).Construct the sequgnce o~ solutions for (1.10), (1.11), in which the nonlinear terms are linearized by the following rule PB 1 P -+Pi - 1B1 Pi +P i B1 Pi-1- Pi-1 B1~,
.(P(t).dP(t)/d+)
PB 1Q -Pi-sB1Qi+PiB1Qi-s-
B 1.. BN2 -1 B',
Q'B 1Q QiB1Qi-s+Q'i-1B1Qi-
o d R(t,O, er: )+(~,- ~,)Q(t, er )-P(t)B 1' Q(t,cr).O, 06t~T "0 d d ( W~,- W)R(t,'t',~ )-Q'(t,'t')B 1
- Q'i-s B1Qi-s } ~=i
(1.13)
P(t)+Q(t,0)+Q'(t,0)-P(t)B1(t~N1(t)=0,
Q(t, ~ ).0,
-h<:. CZ:-, ~ ~ 0
get ).-Tr &' (t) 6" (t)p(t) ,g-dg(t)/dt (1.10) Boundary conditions are P(T)=N o ' Q(T,'r).R(T, 't
'f ).. g(T)=O
A'(t)P(t)-Q'(t -h) .. O, 2A'(t)Q(t,cr)1 R(t,-h,'t )-R'(t, cr,-h).O, R(" f ,'t"). • R'(t,'t',
f)
(1.11)
The Optimal control u for problem (1.7), (1.3), (1.8) is equalOto U
o (t,x t )·-N;1(t)B'(t)[P(t)x(t) +
o
+ _~ Q(t,s)x(t+s)ds
J
We plot i- approximation of u. to the optimal control by the ~ formula (1.12) in which instead P and Q there are Pi - s and Qi-s' Under the control u the value of functionals (1.3~, (1.8) is given by the right part of (1.9) in which P,Q,R,g is replaced by Pi,Qi,Ri,gi' The solution (Pi,Qi,R i ) of linear problem (1.10), (1.11),(1.13) is derived in the explicit form aDd V. (t,x,1jI (t+~)) Vi 1(t,x, (t+9» fOr any i ~ 1, t c + [0, T ] , x eRn' The sequence (Pi,Qi,R i ) converge to exact solution of problem (1.10), (1.11) and the norm of difference between exact solution and i-approximation to it is a 1/i! - order value (Kolmanovskii, 1974 b).
(1.12)
Minimum value of index perfomance (1.3), (1.8) is V (0, ~(t ), '-g (t +8». Thus the soluti8n of liBear-quaRratic control problem (1.7), (1.3), (1.8) is completely defined by boundary problem (1.10), (1.11). Notice , that under certain assumptions on N. and piecewise continuity of matricies~A(t) and B(t), there exists unique solution of problem (1.10), (1.11) in the class of absolutely continuous functions with almost every where bounded derivatives.
EXACT SOLUTIONS OF LlNEAR-QUADRA'l'IC PROBLEM If in the index perfomance (1.3), (1.8) matrix N1.O, the problem (1.10), (1.11) allows the exact solution (Kolmanovskii, 1977). Define matricies B2 and B1 with the help of relations (I is unit matrix)
315
Control and Estimation
Equations, determining m and D ,have the form (Liptser, Shiryaev, 1978) (t)-D (t)f'( G' ~,)-1 [dx(t)"1 '1 -f(t,xt,u)m dtJ ' dm
(3.2)
•
'I')
D (t)--D
(t)B D'h (t),B f' ( () (, )-1 f
~
~
0
1
0
The process in the quadratic brackets from equation (3.2) is the Wiener's one with the same diffuSion matrix as in (3.1). _ Consider scalar functionals W( t ,x, ~ (e), y, z), where yE: Rand Z is nxn matrix. For fixed ~ we d~fine !he finction W .g(t,xty,z)-W(t,x, 'i (e),y,z). Similarly to (1.4) we determine the operator Lu
LuW.L~+~ Tr
(o:W'9/0y2)ZBoZ+
+2( ~ 2W\Jj /"0 x 0 y)zf'-2 -(/W'9 / ~zBoZ. where LUis given by the formula (1.4). Now the can establish the results similarly to the results of 61 fo r the problem (3.1),(1.3). Nam~ly, there is correct the analog of theorem 1.1, where V and LU are replaced on Wand L , respectively. Further, if the s~stem (3.1) is linear with respect to x,u, "7 and functional (1.3) is quadratic the control problem has the explicit solution determined by the system of determinate equations (1.10), (1.11). Finally for the quasi-linear case we have constructed the sequential approximations to the optimal control of quasilinear equations. Notice that for the systems without time delay the control problem with unknown parameters was studied in Kolosov (1984). The control problems for another hereditary systems are considered in Chernousko, Kolmanovskii (19 78), Kolmanovskii, Nosov (1981), Schaichet (1984). REFERENCES Bellman, R.E. (1957). ~amic programmiig, Princeton, N•• : Princeton Un versity Press. Chernousko, F.L., V.B.Kolmanovskii (1978) 0rtimal control under stochastic d sturbances. Nauka, Moscow (in Russian) • Kolmanovskii, V.B. (1974a). Filtration of stochastic processes with delay. Automation and remote control, 1, 42-49. Kolmanovskii, V.B. (1974b). Approximation of linear control system with delay. Problems of control and information the0Et' 1, 63-76. Kolmanovskii. V.B. 19 Optimal control of stochastic hereditary systems. (A survey). Trudy of the fourth All-Union conference on theory and atrlications of differential enua ons With deViating arfSment.aUka DUIDka. kiev, pp.177-~5. Kolma!iOVskii, V.B., T.L.Maizenberg. (1973). Optimal control of stochastic retarded systems. Automation and remote control, 1,
17).
Kolmanovskii, V.B., T.L.Maizenberg. (1977). Optimal estimation of systems states and problems of control of systems with delay. Prikl. Mat em. i Mech., 41~3:44b=456. Kolmanoyskii, V.B., V•• Nosov (1981). Stabilit and periodic modes of con rol systems With aftereffect. NaUka, Moscow (in Russian). Kolmanoyskii, V.B., L.E.Sch&ichet (1978). Approximate feedback control of quasilinear stochastic retarded systems. Prikl. Matem. i Mech., 42,6, 978-988. Kolosov, ·G.E. (1984). DeSign of optimal automatic systems under stochastic disturbances. Nauka, Moscow (in Russian). Liptser, R.S., A.N.Shiryaev. (1978). Statistics of random irocesses. Springer, Ber! n. Schaichet, L.E. (1984). Optimal control of Volterra equations Problems of control and information theory. 13.3.
t
141-152.