- Email: [email protected]
On Hamilton-Jacobi-Bellman-Isaacs Equation for Time-Delay Systems⁎
15th IFAC Workshop on Time Delay Systems 15th IFAC Workshop on Time Delay Systems Sinaia, Romania, September 2019 15th IFAC Workshop on Delay Systems 15th IFAC Workshop on Time Time9-11, Delay Systems Sinaia, Romania, September 9-11, 2019 Available online at www.sciencedirect.com Sinaia, Romania, September 9-11, 2019 15th IFAC Workshop on Time9-11, Delay Systems Sinaia, Romania, September 2019 Sinaia, Romania, September 9-11, 2019
ScienceDirect
IFAC PapersOnLine 52-18 (2019) 138–143
On On Hamilton-Jacobi-Bellman-Isaacs Hamilton-Jacobi-Bellman-Isaacs On Hamilton-Jacobi-Bellman-Isaacs Equation for Time-Delay Systems On Hamilton-Jacobi-Bellman-Isaacs Equation for Time-Delay Systems Equation for Time-Delay Systems Equation forAnton Time-Delay Systems Plaksin ∗∗ Anton Plaksin
∗ Anton Anton Plaksin Plaksin ∗∗ ∗ Anton Plaksin ∗ N.N. N.N. Krasovskii Krasovskii Institute Institute of of Mathematics Mathematics and and Mechanics Mechanics of of ∗ ∗Branch N.N. Krasovskii Institute of Mathematics and Mechanics of of the Russian Academy of Sciences, S.Kovalevskaya N.N. Krasovskii Institute of Mathematics and Mechanics of Branch of the Russian Academy of Sciences, S.Kovalevskaya ∗
the the Ural Ural the Str. 16, the Ural Str.Ural 16, N.N. Krasovskii Institute of Mathematics and Mechanics of the Ural Branch of the the Russian Russian Academy of Sciences, S.Kovalevskaya Str. 16, Yekaterinburg, 620990, Russia; Branch of Academy of Sciences, S.Kovalevskaya Str. 16, Yekaterinburg, 620990, Russia; Branch of theUniversity, Russian Academy of620990, Sciences, S.Kovalevskaya Str. 16, Yekaterinburg, 620990, Russia; Yekaterinburg, Russia; Ural Federal Mira str. 19, Yekaterinburg, 620002, Russia, Ural Federal University, Mira str. 620990, 19, Yekaterinburg, 620002, Russia, Yekaterinburg, Russia; Ural Mira str. Ural Federal Federal University, University, Mira str. 19, 19, Yekaterinburg, Yekaterinburg, 620002, 620002, Russia, Russia, (e-mail: [email protected]) (e-mail: [email protected]) Ural Federal University, Mira str. 19, Yekaterinburg, (e-mail: [email protected]) [email protected]) 620002, Russia, (e-mail: (e-mail: [email protected]) Abstract: The The paper paper deals deals with with aa two-person two-person zero-sum zero-sum differential differential game game for for aa dynamical dynamical system system Abstract: Abstract: The paper deals with a two-person zero-sum differential game for aa dynamical system which motion is described by a delay differential equation under an initial condition defined Abstract: The paper deals with a two-person zero-sum differential game for dynamical system defined by by which motion is paper described by a delay differential equation under angame initial Abstract: The deals with aFor two-person zero-sum differential forcondition a dynamical system which motion is by a differential equation an condition by a continuous function. value of this we the Hamiltonwhich motion is described described by a delay delay differential equation under an initial initial condition defined by a piecewise piecewise continuous function. For the the value functional functional ofunder this game, game, we derive derive the defined Hamiltonwhich motion is described bycoinvariant a delay differential equation anthat, initial condition by a piecewise continuous function. For the the value functional functional ofunder this game, game, weif derive derive the defined HamiltonJacobi type equation with derivatives. It is proved the solution of this a piecewise continuous function. For value of this we the HamiltonJacobi type equation with coinvariant derivatives. It is proved that, if the solution of this a piecewise continuous function. For the value functional ofcoincides this game, weif derive the functional. HamiltonJacobi type equation with coinvariant derivatives. It is is proved that, if the the value solution of this this equation satisfies certain smoothness conditions, then it with Jacobi type equation with coinvariant derivatives. It proved that, the solution of equation satisfies certain smoothness conditions, then it coincides withif the functional. Jacobi equation coinvariant Itofis proved that, the value solution this equation satisfies certain smoothness then it coincides with the value functional. On the the type other hand, it with is proved proved that,conditions, atderivatives. the points points coinvariant differentiability, theofvalue value equation satisfies certain smoothness conditions, then itcoinvariant coincides with the value functional. On other hand, it is that, at the of differentiability, the equation satisfies certain smoothness conditions, then it coincides with the value functional. On the other hand, it is proved that, at the points of coinvariant differentiability, the value functional satisfies the derived Hamilton-Jacobi equation. Therefore, this equation can be called On the other hand,theit derived is proved that, at the points of coinvariant differentiability, value Hamilton-Jacobi equation. Therefore, this equation canthe be called functional satisfies On the other hand,the is proved that, at the for points of coinvariant differentiability, value functional satisfies theit derived derived Hamilton-Jacobi equation. Therefore, this equation can canthe be called called the Hamilton-Jacobi-Bellman-Isaacs Hamilton-Jacobi-Bellman-Isaacs equation time-delay systems.this functional satisfies Hamilton-Jacobi equation. Therefore, equation be the equation for time-delay systems. functional satisfies the derived Hamilton-Jacobi equation. Therefore, the Hamilton-Jacobi-Bellman-Isaacs Hamilton-Jacobi-Bellman-Isaacs equation for for time-delay systems.this equation can be called the equation time-delay systems. Copyright © 2019. The Authors. Published by Elsevier All rightssystems. reserved. the Hamilton-Jacobi-Bellman-Isaacs equation for Ltd. time-delay Keywords: delay system, system, differential differential game, game, Hamilton-Jacobi equation, coinvariant coinvariant derivatives, derivatives, Keywords: delay Hamilton-Jacobi equation, Keywords: delay system, system, differential game, Hamilton-Jacobi equation, coinvariant derivatives, value functional, optimal strategies. Keywords: delay differential game, Hamilton-Jacobi equation, coinvariant derivatives, optimal strategies. value functional, Keywords: delay system, value functional, functional, optimal strategies. value optimal differential strategies. game, Hamilton-Jacobi equation, coinvariant derivatives, value functional, optimal strategies. 1. INTRODUCTION INTRODUCTION further further investigations investigations of of differential differential games games for for time-delay time-delay 1. 1. INTRODUCTION INTRODUCTION further investigations investigations of differential differential games for time-delay time-delay systems and the corresponding HJBI equations. 1. further of games for systems and the corresponding HJBI equations. 1. INTRODUCTION further of differential for time-delay systems investigations and the the corresponding corresponding HJBIgames equations. From the differential games theory for ordinary differsystems and HJBI equations. From the differential games theory for ordinary differ- systems and the corresponding HJBI equations. From the differential games theory for differential the equations (see, e.g., e.g., Isaacs (1965); Krasovskii and 2. From differential games theory for ordinary ordinary differential equations (see, Isaacs (1965); Krasovskii and 2. DIFFERENTIAL DIFFERENTIAL GAME GAME From the differential games theory for Hamilton-Jacobi ordinary differential equations (see, e.g., Isaacs (1965); Krasovskii and Subbotin (1988); Osipov (1971)) and 2. DIFFERENTIAL GAME GAME ential equations (see, e.g., Isaacs (1965); Krasovskii and 2. DIFFERENTIAL Subbotin (1988); Osipov (1971)) and Hamilton-Jacobi ential equations (see, e.g., (1971)) Isaacs (1965); Krasovskii and n Subbotin (1988); Osipov and theory Hamilton-Jacobi DIFFERENTIAL GAME (HJ) with partial derivatives (see, Subbotin (1988); Osipov and Hamilton-Jacobi Euclidian R n be the2. n-dimensional (HJ) equations equations with partial(1971)) derivatives theory (see, e.g., e.g., Let space with with the the Let Rnn be the n-dimensional Euclidian space Subbotin (1988); Osipov and theory Hamilton-Jacobi (HJ) equations equations with partial(1971)) derivatives theory (see, e.g., inner (1995); Crandall and Lions (1983); Clarke, (HJ) with partial derivatives (see, e.g., be the n-dimensional Euclidian space with the Let R product ·, · and the norm · . A function be the n-dimensional Euclidian space with the Let R Subbotin (1995); Crandall and Lions (1983); Clarke, inner product ·, · and the norm · . A function n (HJ) equations with partial derivatives theory (see, e.g., n Subbotin (1995);andCrandall Crandall and Lions it(1983); (1983); Clarke, Ledyaev, Stern Stern Wolenskiand (1998)), is well well Clarke, known x(·) be → theR ·, n-dimensional Euclidian the Let :R[a, Subbotin (1995); Lions inner product · and norm ·· space . function b] is piecewise if there n inner product ·, · called and the the norm continuous . A A with function Ledyaev, and Wolenski (1998)), it is known x(·) : [a, b] → R is called piecewise continuous if there n Subbotin (1995); Crandall and Lions it Clarke, Ledyaev, Stern and Wolenski (1998)), it(1983); isa well well known inner that, on the one hand, a value function of differential n=is product ·, · and the norm · . A function Ledyaev, Stern and Wolenski (1998)), is known x(·) : [a, b] → R called piecewise continuous if there exist numbers a ξ < ξ < . . . < ξ = b such that, for x(·) b] → R piecewise if there that, on the oneand hand, a value(1998)), functionitofisa well differential exist: [a, numbers a n=isξ11called < ξ22 < . . . < ξkkcontinuous = b such that, for Ledyaev, Stern Wolenski known x(·) that, on the one hand, value function function differential game on at points points of hand, differentiability satisfies of theaacorrespondcorrespond: i[a, b]1,→ R piecewise ifonthere that, the one aa value of differential exist numbers a1,= =is ξ11called < ξ22 < < . ..x(·) .< < ξξiskkcontinuous = bb such such that, that, for each ∈ k − the function continuous the exist numbers a ξ < ξ . . = for game at of differentiability satisfies the each i ∈ 1, k − 1, the function x(·) is continuous on the that, on the one a value function of acorresponddifferential game at points points of hand, differentiability satisfies the corresponding Hamilton-Jacobi-Bellman-Isaacs Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation with exist a)1, =and ξ1 there < ξ2 < . .x(·) .< ξis = blimit suchofthat, for game at of differentiability satisfies the each iinumbers ∈[ξ1, 1, k − 1, the function x(·) is continuous on the interval , ξ exists a finite x(ξ) as k continuous i i+1 each ∈ k − the function on the ing (HJBI) equation with as [ξ1, i , ξi+1 ) and there exists a finite limit of x(ξ) game atderivatives points of differentiability satisfies the correspondn ing Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation with interval partial and, on the other hand, the continueach i ∈ k − 1, the function x(·) is continuous on the ing Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation with interval [ξ , ξ ) and there exists a finite limit of x(ξ) as ξ approaches ξ from left. Denote by PC([a, b], R ) and n i+1 i+1 interval [ξii , ξi+1 ) and there exists a finite limit of x(ξ) as partial derivatives and, on the other hand, the continuξ approaches ξ from left. Denote by PC([a, b], R ) and i+1 n ing Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation with n partial derivatives and, on the other hand, the continuously differentiable solution of the HJBI equation coinn ) and interval [ξ , ξ ) and there exists a finite limit of x(ξ) as partial derivatives and, on the other hand, the continuξ approaches ξ from left. Denote by PC([a, b], R Lip([a, b], R ) the linear spaces of piecewise continuous i i+1 n i+1 ξLip([a, approaches ξthe from spaces left. Denote by PC([a, b], Rn ) and ously differentiable solution of the HJBI equation coini+1 linear b], R ) of piecewise continuous n ξ partial derivatives and, on the other hand,equation the continunb], R ) and ously differentiable solution of the HJBI coincides with with the value value function. In Lukoyanov (2003) the Lipschitz n ξLip([a, approaches from left. Denote by b] PC([a, ously differentiable solution of In the HJBI equation coinLip([a, b], R ) the linear spaces of piecewise continuous continuous functions x(·) : [a, → R i+1 linear n. b], R ) the spaces of piecewise continuous and cides the function. Lukoyanov (2003) the continuous functions x(·) : [a, b] → Rnn . n ously differentiable solution of In theLukoyanov HJBI equation coincides with theforvalue value function. In Lukoyanov (2003) the Lipschitz similarwith result differential games for time-delay systems Lip([a, b], R ) the linear spaces of piecewise continuous and cides the function. (2003) the Lipschitz continuous functions x(·) :: [a, R Lipschitz continuous functions x(·) [a, b] b] → → Rn .. similar result forvalue differential games for time-delay systems Let t < ϑ and h > 0. Let us denote cides with the function. In Lukoyanov (2003) the 0 similar result for differential differential games for time-delay time-delay systems Lipschitz and the corresponding HJBI equations with coinvariant continuous functions x(·) : [a, b] → R . similar result for games for systems Let t < ϑ and h > 0. Let us denote and theresult corresponding HJBIgames equations with coinvariant Let tt000 < <ϑ ϑ and and h h> > 0. 0. Let Let us denote denote similar for differential for time-delay systems n us n and the corresponding corresponding HJBI equations with coinvariant derivatives was Note that equations were Let and the coinvariant PC([−h, 0], R G = [t × R n ),us denote n × PC. derivatives was obtained. obtained.HJBI Noteequations that HJBI HJBIwith equations were 0 ,, ϑ] Let t0PC <= ϑ and h > 0. Let PC = PC([−h, 0], R ), G = [t ϑ] × R × PC. 0 and the corresponding HJBI equations with coinvariant n n derivatives was obtained. Note that HJBI equations were considered on the space of continuous functions. In the n ), n × PC. derivatives was obtained. Note that HJBI equationsInwere PC = PC([−h, 0], R G = [t , ϑ] × R 0 considered on the space of continuous functions. the PC = PC([−h, 0], R ), G = [t , ϑ] × R × PC. 0 n n derivatives was obtained. Note that general HJBI equations were norms PC: considered on the the spaceinof of continuous functions. InHJBI the Define paper the the similar similar result the more case for forIn considered on space continuous functions. the PCthe = following PC([−h, 0], R ), on = space [t0 , ϑ] × R × PC. Define the following norms onGthe the space PC: paper result inofthe more general case HJBI Define the following norms on the space PC: considered on the space continuous functions. In the 0 paper the similar similar result in the more general general case functions for HJBI HJBI Define the following equations on the the result space in of the piecewise continuous norms on the space PC: paper the more case for equations on space of piecewise continuous 00 norms on the space PC: paper the similar in more general case functions for HJBI Define the following equations on the the result space of the piecewise continuous functions is it to positional optimal 0 w(ξ) dξ, equations on space of piecewise continuous w(·) = w(·) = sup is proved. proved. Firstly, Firstly, it allows allows to construct construct positionalfunctions optimal 1 = w(·) w(·) ∞ sup w(ξ) . w(ξ) . 0 w(ξ) dξ, equations on the space of piecewise continuous functions 1 ∞ = ξ∈[−h,0] is proved. Firstly, it allows to construct positional optimal strategies of players in differential games for time-delay is proved. Firstly, it allows to construct positional optimal w(·) = w(ξ) dξ, w(·) = sup strategies of players in differential games for time-delay 1 ∞ w(·) 1 = −h w(ξ) dξ, w(·) ∞ = ξ∈[−h,0] sup w(ξ) . w(ξ) . is proved.inFirstly, it allows to construct positional optimal strategies ofthe players in differential games forHJBI time-delay systems, cases when the corresponding equa−h ξ∈[−h,0] strategies of players in differential games for time-delay w(·) 1 = w(ξ) dξ, w(·) ∞ = ξ∈[−h,0] sup w(ξ) . systems, inofthe cases in when the corresponding HJBI equa−h strategies players differential games for time-delay −h systems, insmooth the cases cases wheneven the corresponding corresponding HJBI equation has has aain solution if an an initial initial motion motion history ξ∈[−h,0] systems, the when the HJBIhistory equa- We consider a two-person differential game tion smooth solution even if systems, the cases wheneven theSecondly, corresponding HJBIhistory equa- We consider a−htwo-person differential game for for the the dynamdynamtion has aainsmooth smooth solution even if an an initial initial motion has of the choice of tion has solution if has points points of discontinuities. discontinuities. Secondly, themotion choicehistory of the the We We consider a two-person differential game for the dynamdynamical system described by the time-delay equation consider a two-person differential game for the tion has a smooth solution even if an initial motion history ical system described by the time-delay equation has points of discontinuities. Secondly, the choice of the space of piecewise continuous functions may in the future has of discontinuities. Secondly, may the choice of the We consider a two-person differential game for the dynamspacepoints of piecewise continuous functions in the future ical system described by the time-delay equation ical system described by the time-delay equation has points of discontinuities. Secondly, the choice of the space of piecewise piecewise continuous functions may mayofin in the the theory future ical system significantly facilitate many constructions constructions x(τ ˙˙ described )) = x(τ − v(τ space of continuous functions future by), x(τ = ff (τ, (τ, x(τ x(τ ),the x(τ time-delay − h), h), u(τ u(τ ), ), equation v(τ )), )), significantly facilitate many ofinthe the theory (1) space of piecewise continuous functions may the theory future significantly facilitate many constructions of coinvariant the theory x(τ ˙ )) = = ff (τ, (τ, x(τ x(τ ), ),nx(τ x(τ − − h), h), u(τ u(τ ), ), v(τ v(τ )), )), of generalized solutions of HJ equations with significantly facilitate many constructions of the x(τ ˙ (1) of generalized solutions of HJ equations with coinvariant τ ∈ [t , ϑ], x(τ ) ∈ R u(τ ) ∈ U, v(τ )) ∈ V, n, 0 (1) significantly facilitate many constructions of the theory x(τ ˙ ) = f (τ, x(τ ), x(τ − h), u(τ ), v(τ )), τ ∈ [t , ϑ], x(τ ) ∈ R , u(τ ) ∈ U, v(τ ∈ V, of generalized solutions of HJ equations with coinvariant (1) derivatives. Earlier, similar investigations of HJ equations 0 n of generalized solutions of HJ equations with n derivatives. Earlier, similar investigations of HJcoinvariant equations τ∈ ∈ [t [t00 ,, ϑ], ϑ], x(τ x(τ )) ∈ ∈R R ,, u(τ u(τ )) ∈ ∈ U, U, v(τ v(τ )) ∈ ∈ V, V, (1) of generalized solutions ofcontrol HJ equations with derivatives. Earlier, similar investigations offor HJcoinvariant equations andττthe corresponding to optimal problems time-delay n quality index derivatives. Earlier, similar investigations of HJ equations ∈ [tquality x(τ ) ∈ R , u(τ ) ∈ U, v(τ ) ∈ V, corresponding to optimal control problemsoffor time-delay and the 0 , ϑ], index derivatives. Earlier, similar investigations HJ equations corresponding to optimal optimal control problems for paper time-delay systems were were presented presented in control Plaksinproblems (2019). The The con- and index corresponding to for time-delay and the theγquality quality index systems in Plaksin (2019). paper con= σ(x(ϑ), x corresponding to optimal control problems for paper time-delay systems were presented presented ingives Plaksin (2019). The The paper continues Plaksin (2019) and a theoretical foundation for and the quality index γ = σ(x(ϑ), xϑϑ (·)) (·)) systems were in Plaksin (2019). con tinues Plaksin (2019) and gives a theoretical foundation for γ = σ(x(ϑ), x ϑ γ = σ(x(ϑ), xϑϑ (·)) (·)) systems were presented in Plaksin (2019). The paper con(2) tinues Plaksin (2019) and gives a theoretical foundation for ϑ tinues Plaksin (2019) and gives a theoretical foundation for 0 (2) γ = σ(x(ϑ), xϑx(ξ), (·)) x(ξ − h), u(ξ), v(ξ)) dξ. ϑϑ f 0 (ξ, This Plaksin + is supported bygives the Grant of the President of for the tinues (2019) and a theoretical foundation (2) This work + f (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ. (2) work is supported by the Grant of the President of the 0 0 t ϑ This work + t ff (ξ, (ξ, x(ξ), x(ξ), x(ξ x(ξ − − h), h), u(ξ), u(ξ), v(ξ)) v(ξ)) dξ. dξ. Russian Federation (project by no. the MK-3566.2019.1). is Grant (2) + This work is supported supported by the Grant of of the the President President of of the the Russian Federation (project no. MK-3566.2019.1). 0 + tt f (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ. Russian Federation (project by no. the MK-3566.2019.1). This work is supported Grant of the President of the Russian Federation (project no. MK-3566.2019.1).
t Russian Federation (project 2405-8963 Copyright © 2019. no. TheMK-3566.2019.1). Authors. Published by Elsevier Ltd. 260 All rights reserved. Copyright © 2019 IFAC Copyright © 2019 IFAC 260 Peer review© under responsibility of International Federation of Automatic Copyright 2019 IFAC IFAC 260 Control. Copyright © 2019 260 10.1016/j.ifacol.2019.12.220 Copyright © 2019 IFAC 260
2019 IFAC TDS Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
Here τ is the time variable; x(τ ) is the value of the state vector at the time τ ; t ∈ [t0 , ϑ] is the time of the control process beginning; hereinafter, for each τ ∈ [t0 , ϑ], the symbol xτ (·) denotes the function on the interval [−h, 0] defined by xτ (ξ) = x(τ + ξ), ξ ∈ [−h, 0]; u(τ ) and v(τ ) are control actions of the first and second players, respectively; U and V are known compacts of finite-dimensional spaces. The first player aims to minimize the value γ of the quality index, while the second player aims to maximize it. We assume that the following conditions hold: (f1 ) The functions f (t, x, y, u, v) ∈ Rn , f 0 (t, x, y, u, v) ∈ R, t ∈ [t0 , ϑ], x, y ∈ Rn , u ∈ U, v ∈ V, are continuous. (f2 ) For every α > 0, there exists λf > 0 such that f (t, x, y, u, v) − f (t, x , y , u, v) +|f 0 (t, x, y, u, v) − f 0 (t, x , y , u, v)| ≤ λf x − x + y − y
for any t ∈ [t0 , ϑ], x, y, x , y ∈ O(α) = {x ∈ R : x ≤ α}, u ∈ U and v ∈ V. (f3 ) There exists a constant cf > 0 such that f (t, x, y, u, v) + |f 0 (t, x, y, u, v)| ≤ cf 1 + x + y
n
for any t ∈ [t0 , ϑ], x, y ∈ Rn , u ∈ U and v ∈ V. (f4 ) For every t ∈ [t0 , ϑ] and x, y, s ∈ Rn , we have min max f (t, x, y, u, v), s + f 0 (t, x, y, u, v) u∈U v∈V = max min f (t, x, y, u, v), s + f 0 (t, x, y, u, v)
139
u(τ ) = U (τj , x(τj ), xτj (·)), τ ∈ [τj , τj+1 ), j = 1, l. (4)
This control law together with v(·) ∈ V(t) uniquely determine the control process realization {x(·), u(·), v(·)} and the value γ = γ(t, z, w(·); U, ∆δ ; v(·)) of quality index (2). The guaranteed result of the strategy U is defined by ρu (t, z, w(·); U ) = lim sup sup γ(t, z, w(·); U, ∆δ ; v(·)). (5) δ↓0 ∆δ v(·)
The optimal guaranteed result of the first player is ρ◦u (t, z, w(·)) = inf ρu (t, z, w(·); U ).
(6)
U
◦
A strategy U of the first player is called optimal if ρu (t, z, w(·); U ◦ ) = ρ◦u (t, z, w(·)). Similarly, with the corresponding changes, for the second player, we define a control strategy V : G → V, control law {V, ∆δ } that forms a function v(·) ∈ V(t) by v(τ ) = V (τj , x(τj ), xτj (·)), t ∈ [τj , τj+1 ), j = 1, l,
the guaranteed result of the strategy V
ρv (t, z, w(·); V ) = lim inf inf γ(t, z, w(·); u(·); V, ∆δ ), (7) δ↓0 ∆δ u(·)
and the optimal guaranteed result ρ◦v (t, z, w(·)) = sup ρv (t, z, w(·); V ).
(8)
V
A strategy V ◦ of the second player is called optimal if ρv (t, z, w(·); V ◦ ) = ρ◦v (t, z, w(·)). Due to definitions (5)–(8), we have
v∈V u∈U
(σ) For every α > 0, there exists λσ > 0 such that |σ(z, w(·)) − σ(z , w (·))| ≤ λσ z − z + w(·) − w (·) 1
for any (z, w(·)), (z , w (·)) ∈ P (α), where
P (α) = {(z, w(·)) ∈ Rn × PC : z ≤ α, w(·) ∞ ≤ α}.
Let (t, z, w(·)) ∈ G. Denote by Λ(t, z, w(·)) the set of functions x(·) ∈ PC([t − h, ϑ], Rn ) such that x(τ ) = w(τ − t), τ ∈ [t − h, t), x(τ ) = y(τ ), τ ∈ [t, ϑ], where y(·) ∈ Lip([t, ϑ], Rn ) and y(t) = z.
Denote by U(t) and V(t) sets of measurable functions u(·) : [t, ϑ] → U and v(·) : [t, ϑ] → V, respectively. It is known that, under the conditions above, for each u(·) ∈ U(t) and v(·) ∈ V(t), there exists a unique motion x(·) of system (1) that is a function x(·) ∈ Λ(t, z, w(·)) that satisfies equation (1) for almost every τ ∈ [t, ϑ]. The triple {x(·), u(·), v(·)} is called a control process realization. Note that this control process realization uniquely determines the value of quality index (2).
ρ◦v (t, z, w(·)) ≤ ρ◦u (t, z, w(·)), ◦
(t, z, w(·)) ∈ G.
ρ◦u (t, z, w(·)) ◦
(9)
ρ◦v (t, z, w(·))
= If the equality ρ (t, z, w(·)) := holds for any (t, z, w(·)) ∈ G, then ρ : G → R is called the value functional of differential game (1), (2). One can show (see, e.g., Krasovskii and Subbotin (1988)) that the value functional ρ◦ has the following properties: (ρu ) For every (t, z, w(·)) ∈ G, τ ∈ [t, ϑ], ε > 0 and v(·) ∈ V(t), there exists u(·) ∈ U such that, for the control process realization {x(·), u(·), v(·)}, we have τ ◦ ρ (τ, x(τ ), xτ (·)) + f 0 (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ t
≤ ρ◦ (t, z, w(·)) + ε.
(ρv ) For every (t, z, w(·)) ∈ G, τ ∈ [t, ϑ], ε > 0 and u(·) ∈ U (t), there exists v(·) ∈ V such that, for the control process realization {x(·), u(·), v(·)}, we have τ ◦ ρ (τ, x(τ ), xτ (·)) + f 0 (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ t
≥ ρ◦ (t, z, w(·)) − ε.
According to Krasovskii and Subbotin (1988) (see also Lukoyanov (2003) for time-delay systems), differential game (1), (2) is posed as follows.
3. PROPERTIES OF THE TIME-DELAY SYSTEM
By a control strategy of the first player, we mean an arbitrary function U : G → U. Let us fix (t, z, w(·)) ∈ G and a partition of the interval [t, ϑ]: ∆δ = {τj : τ1 = t, 0 < τj+1 −τj ≤ δ, j = 1, l, τl+1 = ϑ}. (3)
Taking the constant cf > 0 form (f3 ), we denote F η (x, y) = {f ∈ Rn : f ≤ cf (1 + x + y ) + η}, x, y ∈ Rn , η ≥ 0. (10) Let (t, z, w(·)) ∈ G and η ≥ 0. Denote by X η (t, z, w(·)) the set of functions x(·) ∈ Λ(t, z, w(·)) that satisfy the following time-delay differential inclusion:
The pair {U, ∆δ } defines a control law that forms a piecewise constant function u(·) ∈ U(t) according to the following step-by-step rule:
261
x(τ ˙ ) ∈ F η (x(τ ), x(τ − h)) for a.e. τ ∈ [t, ϑ].
(11)
2019 IFAC TDS 140 Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
Note that the set X η (t, z, w(·)) is not empty. In particular, for each u(·) ∈ U(t) and v(·) ∈ V(t), the motion x(·) of system (1) satisfies the inclusion (12) x(·) ∈ X 0 (t, z, w(·)) ⊂ X η (t, z, w(·)), η ≥ 0. Proposition 1. Let (t, z, w(·)) ∈ G and η ≥ 0. Then there exist αX , λX > 0 such that (x(τ ), xτ (·)) ∈ P (αX ), x(τ ) − x(τ ) ≤ λX |τ − τ |,
for any τ, τ ∈ [t, ϑ] and x(·) ∈ X η (t, z, w(·)). Proposition 2. Let (t, z, w(·)) ∈ G, t < ϑ and η ≥ 0. Let a sequence xk (·) ∈ X η (t, z, w(·)), k ∈ N be chosen. Then there exist a subsequence xki (·) and a function x∗ (·) ∈ X η (t, z, w(·)) such that max
τ ∈[t−h,ϑ]
xki (τ ) − x∗ (τ ) → 0 as i → ∞.
Proposition 3. Let (t, z, w(·)) ∈ G and η ≥ 0. For every ε > 0, there exists δ > 0 such that, for every x(·) ∈ X η (t, z, w(·)) and τ, τ ∈ [t, ϑ]: |τ − τ | ≤ δ, we have: x(τ ) − x(τ ) + xτ (·) − xτ (·)1 ≤ ε.
Proposition 1 and 2 can be proved similar to Proposition 3.1 and Lemma 4.8 in Plaksin (2019), respectively. Proposition 3 one can proved, using approximation of w(·) by a Lipschitz function (see, e.g., (Natanson, 1960, p. 214)) and Proposition 1. 4. HJBI EQUATION Following Kim (1999); Lukoyanov (2000), a functional ϕ : G → R is called coinvariantly differentiable (ci-differentiable) at a point (t, z, w(·)) ∈ G, t < ϑ if there exist ci ϕ(t, z, w(·)) ∈ R and ∇z ϕ(t, z, w(·)) ∈ Rn such that, ∂t,w for every v ∈ Rn , x(·) ∈ Λ(t, z, w(·)) and τ ∈ [t, ϑ], the following relation holds: ci ϕ(τ, v, xτ (·)) − ϕ(t, z, w(·)) = ∂t,w ϕ(t, z, w(·))(τ − t) (13) +v − z, ∇z ϕ(t, z, w(·)) + o(|τ − t| + v − z), where the function xτ (·) ∈ PC is defined by xτ (ξ) = x(τ + ξ), ξ ∈ [−h, 0], the value o(·) depends on the triplet {t, z, x(·)}, and o(δ)/δ → 0 as δ → +0. Then ci ϕ(t, z, w(·)) is called the ci-derivative of ϕ with respect ∂t,w to {t, w(·)} and ∇z ϕ(t, z, w(·)) is the gradient of ϕ with respect to z. Let us note that if ϕ does not depend on the functional variable w(·), then the definition of cidifferentiability coincides with the definition of differentiability of functions. For differential game (1), (2), we define the Hamiltonian H(t, x, y, s) = min max f (t, x, y, u, v), s (14) u∈U v∈V +f 0 (t, x, y, u, v) , t ∈ [t0 , ϑ], x, y, s ∈ Rn .
and consider the following HJ equation
ci ϕ(t, z, w(·)) + H(t, z, w(−h), ∇z ϕ(t, z, w(·))) = 0, ∂t,w
(t, z, w(·)) ∈ G,
t < ϑ,
(15)
with the terminal condition
ϕ(ϑ, z, w(·)) = σ(z, w(·)),
(ϑ, z, w(·)) ∈ G.
(16)
Following Plaksin (2019), define the class of functionals in which we will search a solution of the problem (15), (16). Denote by Φ the set of functionals ϕ = ϕ(t, z, w(·)) ∈ R, 262
(t, z, w(·)) ∈ G which are continuous with respect to t and satisfy the following Lipschitz condition: for every α > 0, there exists λϕ > 0 such that |ϕ(t, z, w(·))−ϕ(t, z , w (·))| ≤ λϕ z−z +w(·)−w (·)1 .
for any t ∈ [t0 , ϑ], (z, w(·)), (z , w (·)) ∈ P (α). The choice of this class is motivated, in particular, by the inclusion ρ◦ ∈ Φ, (17) which can be shown by scheme of Lemma 4.1 in Plaksin (2019). 5. OPTIMAL STRATEGIES Lemma 1. Let ϕ ∈ Φ, (t, z, w(·)) ∈ G and η ≥ 0. For every ε > 0, there exists δ = δ(ε) > 0 such that, for every x(·) ∈ X η (t, z, w(·)) and τ, τ ∈ [t, ϑ]: |τ − τ | ≤ δ, the following inequality holds: |ϕ(τ, x(τ ), xτ (·)) − ϕ(τ , x(τ ), xτ (·))| ≤ ε.
Proof. For the sake of a contradiction, suppose that there exist ε > 0 and xk (·) ∈ X η (t, z, w(·)), τk , τk ∈ [t, ϑ]: |τk − τk | ≤ 1/k, k ∈ N such that |ϕ(τk , xk (τk ), xkτk (·)) − ϕ(τk , xk (τk ), xkτ (·))| > ε, k
k ∈ N.
Without loss of generality, taking into account Proposition 2, we can suppose that there exist τ∗ ∈ [t, ϑ] and x∗ (·) ∈ X η (t, z, w(·)) such that τk → τ∗ ,
max
τ ∈[t−h,ϑ]
τk → τ∗ ,
xk (τ ) − x∗ (τ ) → 0,
as
k → ∞.
(18)
Due to Proposition 1 and the inclusion ϕ ∈ Φ, there exist k1 > 0 and λϕ > 0 such that, for every k > k1 , we have |ϕ(τk , xk (τk ), xkτk (·)) − ϕ(τ∗ , x∗ (τ∗ ), x∗τ∗ (·))|
≤ |ϕ(τk , xk (τk ), xkτk (·)) − ϕ(τk , x∗ (τ∗ ), x∗τ∗ (·))|
+|ϕ(τk , x∗ (τ∗ ), x∗τ∗ (·)) − ϕ(τ∗ , x∗ (τ∗ ), x∗τ∗ (·))| ≤ λϕ xk (τk ) − x∗ (τ∗ ) + xkτk (·) − x∗τ∗ (·)1 + ε/4.
According to Proposition 3 and (18), there exists k2 > 0 such that, for every k > k2 , we derive xk (τk ) − xk (τ∗ ) + xkτk (·) − xkτ∗ (·)1 ≤ ε/(8λϕ ), xk (τ∗ ) − x∗ (τ∗ ) + xkτ∗ (·) − x∗τ∗ (·)1 ≤ ε/(8λϕ ).
Thus, for k > max{k1 , k2 }, we obtain
|ϕ(τk , xk (τk ), xkτk (·)) − ϕ(τ∗ , x∗ (τ∗ ), x∗τ∗ (·))| ≤ ε/2.
In a similar way, we can show that there exists k3 > 0 such that, for every k > k3 , we have |ϕ(τk , xk (τk ), xkτ (·)) − ϕ(τ∗ , x∗ (τ∗ ), x∗τ∗ (·))| ≤ ε/2. k
Thus, for k > max{k1 , k2 , k3 }, we conclude ε < ε. Lemma 2. Let ϕ ∈ Φ, (t, z, w(·)) ∈ G and η ≥ 0. There exists β > 0 such that, for every x(·) ∈ X η (t, z, w(·)), the following estimate holds: |ϕ(τ, x(τ ), xτ (·))| ≤ β,
τ ∈ [t, ϑ].
(19)
Proof. Since ϕ ∈ Φ, there exists β0 > 0 satisfying |ϕ(τ, 0, θ(·) ≡ 0)| ≤ β0 , τ ∈ [t, ϑ]. Moreover, taking αX > 0 from Proposition 1, one can choose λϕ such that
2019 IFAC TDS Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
|ϕ(τ, x(τ ), xτ (·)) − ϕ(τ, 0, θ(·))| ≤ λϕ (x(τ ) + xτ (·)1 ), η
τ ∈ [t, ϑ],
x(·) ∈ X (t, z, w(·)).
Thus, defining β = β0 + λϕ (1 + h)αX , we obtain (19). Lemma 3. Let ϕ ∈ Φ be ci-differentiable at every point ci (t, z, w(·)) ∈ G, t < ϑ and the inclusions ∂t,w ϕ, ∇z ϕ ∈ Φ ∗ hold. Let (t, z, w(·)) ∈ G and x (·) ∈ X 0 (t, z, w(·)). Then the function ω ∗ (τ ) = ϕ(τ, x∗ (τ ), x∗τ (·)), τ ∈ [t, ϑ] is Lipschitz continuous and ci ω˙ ∗ (τ ) = ∂t,w ϕ(τ, x∗ (τ ), x∗τ (·)) ∗
+x˙ (τ ), ∇z ϕ(τ, x
∗
(τ ), x∗τ (·))
for a.e. τ ∈ [t, ϑ].
(20)
Proof. Taking cf > 0 from condition (f3 ) and αX , λX > 0 from Proposition 1, let us define η = cf (1 + 2αX ). Then, ci due to inclusions ∂t,w ϕ, ∇z ϕ ∈ Φ and Lemma 2, there exists β∂ϕ , β∇ϕ > 0 such that ci ϕ(τ, x(τ ), xτ (·))| |∂t,w
≤ β∂ϕ , |∇z ϕ(τ, x(τ ), xτ (·))| ≤ β∇ϕ , x(·) ∈ X η (t, z, w(·)).
τ ∈ [t, ϑ],
(21)
Let us define the functions xk (·) ∈ Λ(t, z, w(·)): τ k x (τ ) = k x∗ (max{ξ, t}) dξ, τ ∈ [t, ϑ], k ∈ N. τ −1/k
Then these functions are continuously differentiable and, for every τ ∈ [t, ϑ] and k ∈ N, satisfy relations (xk (τ ), xkτ (·)) ∈ P (αX ),
Moreover, one can show that
x˙ k (τ ) ≤ λX .
max x∗ (τ ) − xk (τ ) → 0 as k → ∞.
τ ∈[t,ϑ]
(22) (23)
0
and, using definition (11) of the set X (t, z, w(·)) and in accordance with the choice of the number η, for all τ ∈ [t, ϑ], we derive τ k x˙ ∗ (ξ) dξ x˙ (τ ) ≤ k max{τ −1/k,t}
τ
≤k
max{τ −1/k,t}
It means that
cf 1 + x∗ (ξ) + x∗ (ξ − h) dξ ≤ η.
xk (·) ∈ X η (t, z, w(·)), k
k
k ∈ N.
(24)
(τ ), xkτ (·)),
Define the functions ω (τ ) = ϕ(τ, x τ ∈ [t, ϑ], k ∈ N. Then, according to the inclusion ϕ ∈ Φ and (22), there exists λϕ > 0 such that, for every τ ∈ [t, ϑ], we have |ω ∗ (τ ) − ω k (τ )| ≤ λϕ x∗ (τ ) − xk (τ ) + x∗τ (·) − xkτ (·)1 . Then from (23) we obtain
max ω ∗ (τ ) − ω k (τ ) → 0
τ ∈[t,ϑ]
as k → ∞.
(25)
Taking into account ci-differentiability of ϕ, let us calculate right derivatives of the functions ω k (·) as follows ω k (ξ) − ω k (τ ) d+ ω k (τ )/ dτ = lim ξ→τ +0 ξ−τ (26) ci = ∂t,w ϕ(τ, xk (τ ), xkτ (·)) k
+x˙ (τ ), ∇z ϕ(τ, x
k
(τ ), xkτ (·)),
τ ∈ (t, ϑ).
141
ci Due to the inclusions ∂t,w ϕ, ∇z ϕ ∈ Φ, Lemma 1 and continuous differentiability of xk (·), we obtain that the right-hand side of this equation is continuous on (t, ϑ). Consequently, the function d+ ω k (τ )/ dτ is also continuous on (t, ϑ). Then one can show that the function ω(·) is differentiable on (t, ϑ) and ω˙ k (τ ) = d+ ω k (τ )/ dτ , τ ∈ (t, ϑ). Thus, from (21), (22), (24), (26), we derive
|ω˙ k (τ )| ≤ β∂ϕ + λX β∇ϕ := λω ,
It means that |ω k (τ ) − ω k (τ )| ≤ λω |τ − τ |,
τ ∈ (t, ϑ). τ, τ ∈ [t, ϑ].
Passing to the limit in this estimate as k → ∞, considering (25), we obtain that the function ω ∗ (·) is Lipschitz continuous. Equality (20) is obtained similar to (26) at the points of differentiability of functions ω ∗ (·) and x∗ (·). Let us consider the following players control strategies: U ◦ (t, z, w(·)) ∈ argmin max χ(t, z, w(·), u, v), u∈U
v∈V
v∈V
u∈U
V ◦ (t, z, w(·)) ∈ argmax min χ(t, z, w(·), u, v),
(27)
where (t, z, w(·)) ∈ G and χ(t, z, w(·), u, v) = f (t, z, w(−h), u, v), ∇ϕz (t, z, w(·)) +f 0 (t, z, w(−h), u, v). (28) Theorem 1. Let a functional ϕ ∈ Φ be ci-differentiable at every point (t, z, w(·)) ∈ G, t < ϑ, satisfies HJ equation (15) with terminal condition (16) and the inclusions ci ∂t,w ϕ, ∇z ϕ ∈ Φ hold. Then the control strategies U ◦ and V ◦ defined by (27) are optimal, and ϕ is the value functional of differential game (1), (2). Proof. The proof is carried out by the scheme from Lukoyanov (2003) (see also Gomoyunov and Plaksin (2018)). Let (t, z, w(·)) ∈ G. If t = ϑ, then the validity of the theorem follows from the definition of ρ◦ and (16). Let t < ϑ. In accordance with (6), (8) and (9), for proving the theorem, it is sufficient to show that ρu (t, z, w(·); U ◦ ) ≤ ϕ(t, z, w(·)) ≤ ρv (t, z, w(·); V ◦ ). (29)
Let us prove the first inequality. Due to (5), we should show that, for any ε > 0, there exists δ > 0 such that the following statement is valid. Let ∆δ be a partition (3). Then, for the control process realization {x(·), u(·), v(·)} generated by the control law of the first player {U ◦ , ∆δ } and v(·) ∈ V(t), the value γ = γ(t, z, w(·); U ◦ , ∆δ ; v(·)) of quality index (2) satisfies the inequality ϑ γ = σ(x(ϑ), xϑ (·)) + f 0 (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ t
≤ ϕ(t, z, w(·)) + ε.
(30)
Due to the inclusion ∇z ϕ ∈ Φ and Lemma 2, there exists β∇ϕ > 0 such that, for every x(·) ∈ X 0 (t, z, w(·)), we have |∇z ϕ(τ, x(τ ), xτ (·))| ≤ β∇ϕ , τ ∈ [t, ϑ]. (31) Denote
ε∗ = ε/(2(ϑ − t)).
(32)
Let −h < ξ1 < ξ2 < . . . < ξk < 0 are discontinuity points of the function w(·). Define the sets Ii = [ξi + t + h, ξi+1 + t + h) ∩ [t, ϑ],
i ∈ 1, k − 1
Ik = [ξl + t + h, t + h) ∩ [t, ϑ], Ik+1 = [min{t + h, ϑ}, ϑ]. 263
2019 IFAC TDS 142 Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
Then, according to piecewise continuity of w(·), Proposition 1 and condition (f1 ), there exists δf > 0 such that, for every x(·) ∈ X 0 (t, z, w(·)), i ∈ 1, k + 1, τ, τ ∈ Ii , u ∈ U and v ∈ V, the following estimates hold: f (τ, x(τ ), x(τ − h), u, v)
−f (τ , x(τ ), x(τ − h), u, v) ≤ ε∗ /β∇ϕ ,
|f 0 (τ, x(τ ), x(τ − h), u, v)
(33)
Moreover, there exists βf > 0 such that
x(·) ∈ X 0 (t, z, w(·)),
u ∈ U,
(34) v ∈ V.
ci ϕ, ∂t,w ϕ, ∇z ϕ 0
Due to Lemma 1 and the inclusions ∈ Φ, there exists δϕ > 0 such that, for every x(·) ∈ X (t, z, w(·)) and τ, τ ∈ [t, ϑ]: |τ − τ | ≤ δϕ , we obtain |ϕ(τ, x(τ ), xτ (·)) − ϕ(τ , x(τ ), xτ (·))| ≤ ε/(4(k + 1)),
ci ci ϕ(τ, x(τ ), xτ (·)) − ∂t,w ϕ(τ , x(τ ), xτ (·))| ≤ ε∗ , (35) |∂t,w
v∈V
= H(τj , x(τj ), x(τj − h), ∇ϕ(τj , x(τj ), xτj (·))).
Thus, taking into account (15), we obtain ω(τ ˙ ) ≤ ε∗ for a.e. τ ∈ [τj , τj+1 ], j ∈ J.
6. CI-DIFFERENTIABILITY PROPERTIES OF THE VALUE FUNCTIONAL
f (τ, x(τ ), x(τ − h), u, v) ≤ βf ,
τ ∈ [t, ϑ],
≤ max χ(τj , x(τj ), xτj (·), U ◦ (τj , x(τj ), xτj (·)), v)
Then, from (32), (39), we conclude (38). The first inequality in (29) is proved. Due to (f4 ), the second inequality in (29) can be proved in a similar way.
−f 0 (τ , x(τ ), x(τ − h), u, v)| ≤ ε∗ .
|f 0 (τ, x(τ ), x(τ − h), u, v)| ≤ βf ,
Due to (4), (14), (27), we have χ(τj , x(τj ), xτj (·), u(τ ), v(τ ))
|∇z ϕ(τ, x(τ ), xτ (·)) − ∇z ϕ(τ , x(τ ), xτ (·))| ≤ ε∗ /βf . Define δ = min{δf , δϕ , ε/(4βf (k + 1))}.
(36)
Let us show that this δ satisfies the statement above. Let ∆δ be a partition (3) and the control process realization {x(·), u(·), v(·)} be generated by the control law {U ◦ , ∆δ } and v(·) ∈ V(t). Define the function
Lemma 4. For every (t, z, w(·)) ∈ G, t < ϑ and s ∈ Rn , there exist τ∗ ∈ (t, ϑ] and x∗ (·) ∈ X 0 (t, z, w(·)) such that, for every τ ∈ [t, τ∗ ], the following inequality holds: ρ◦ (τ, x∗ (τ ), x∗τ (·)) τ + H(ξ, x∗ (ξ), x∗ (ξ − h), s) − x˙ ∗ (ξ), s dξ (40) t
Proof. Since w(·) ∈ PC, then there exists τ∗ ∈ (t, ϑ] such that w(·) is continuous on [−h, τ∗ −t−h]. Let k ∈ N. Denote τj = t+(τ∗ −t)j/k, j ∈ 0, k. According to (14) and (ρu ), let us define control process realizations {xk (·), uk (·), v k (·)} such that, for every j ∈ 0, k − 1, we have (41) v k (τ ) = vjk , τ ∈ [τj , τj+1 ), where vjk ∈ V is defined by
H(τj , xk (τj ), xk (τj − h), s) = min f (τj , xk (τj ), xk (τj − h), u, vjk ), s u∈U +f 0 (τj , xk (τj ), xk (τj − h), u, vjk ) ,
ω(τ ) = ϕ(τ, x(τ ), xτ (·)) +
τ t
f 0 (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ, τ ∈ [t, ϑ].
(37)
Then, taking into account terminal condition (16), for proving (30), it is sufficient to show that ω(ϑ) ≤ ω(t) + ε.
(43)
≤ ρ◦ (τj , xk (τj ), xkτj (·)) + (τj+1 − τj )/k.
(38)
j∈J
According to Lemma 3, the function ω(·) is Lipschitz continuous and, taking into account system (1) and (28), for almost every τ ∈ [t, ϑ], satisfies the equation ci ω(τ ˙ ) = ∂t,w ϕ(τ, x(τ ), xτ (·)) + χ(τ, x(τ ), xτ (·), u(τ ), v(τ )).
For j ∈ J and τ ∈ [τj , τj+1 ), in accordance with (31), (33)–(36), we derive ≤
and ρ◦ (τj+1 , xk (τj+1 ), xkτj+1 (·)) τ j+1 + f 0 (ξ, xk (ξ), xk (ξ − h), uk (ξ), v k (ξ)) dξ
(42)
τj
Denote by J the set of j ∈ 1, l such that there exists i ∈ 1, k + 1 such that [τj − h, τj+1 − h) ⊂ Ii . Note that, |J| ≥ l − k − 1. Then, from (34)–(37), we derive (ω(τj+1 ) − ω(τj )) + ε/2. (39) ω(ϑ) − ω(t) ≤
ci ϕ(τ, x(τ ), xτ (·)) ∂t,w
≤ ρ◦ (t, z, w(·)).
According to (12) and Proposition 2, without loss of generality, we can suppose that there exists x∗ (·) ∈ X 0 (t, z, w(·)) such that max x∗ (τ ) − xk (τ ) → 0 as k → ∞. (44) τ ∈[t,ϑ]
Let us fix ε > 0. Due to (f1 ) and Lemma 1, taking into account the continuity of w(·) on [−h, τ∗ − t − h], Proposition 1 and (17), there exists k1 > 0 such that, for every k ≥ k1 , x(·) ∈ X 0 (t, z, w(·)), u ∈ U, v ∈ V, j ∈ 0, k − 1 and τ ∈ [τj , τj+1 ], we have |f (τj , x(τj ), x(τj − h), u, v), s
−f (τ, x(τ ), x(τ − h), u, v), s| ≤ ε/(16(τ∗ − t)),
0
|f (τj , x(τj ), x(τj − h), u, v)
+ χ(τ, x(τ ), xτ (·), u(τ ), v(τ ))
ci ∂t,w ϕ(τj , x(τj ), xτj (·))
0
−f (τ, x(τ ), x(τ − h), u, v)| ≤ ε/(16(τ∗ − t)),
◦
|ρ (τj , x(τj ), xτj (·)) − ρ◦ (τ, x(τ ), xτ (·))| ≤ ε/6.
+χ(τj , x(τj ), xτj (·), u(τ ), v(τ )) + 4ε∗ . 264
(45)
2019 IFAC TDS Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
Then, in accordance with (14), we derive |H(τj , x(τj ), x(τj − h), s)
−H(τ, x(τ ), x(τ − h), s)| ≤ ε/(8(τ∗ − t)).
(46)
Moreover, according to (f1 ) and Proposition 1, there exists βf > 0 such that 0
|f (τ, x(τ ), x(τ − h), u, v)| ≤ βf ,
τ ∈ [t, τ∗ ], x(·) ∈ X 0 (t, z, w(·)), u ∈ U, v ∈ V.
(47)
Due to (17), (f1 ), (14) and (44), there exists k2 > 0 such that, for every k ≥ k2 , we have +
τ t
τ t
ρ◦ (τ, x∗ (τ ), x∗τ (·))
H(ξ, x∗ (ξ), x∗ (ξ − h), s) − x˙ ∗ (ξ), s dξ ≤ ρ◦ (τ, xk (τ ), xkτ (·))
(48)
H(ξ, xk (ξ), xk (ξ − h), s) − x˙ k (ξ), s dξ + ε/6.
k
k
k
H(ξ, x (ξ), x (ξ − h), s) − x˙ (ξ), s ≤ H(τj , xk (τj ), xk (τj − h), s)
k
k
k
−f (τj , x (τj ), x (τj − h), u 0
k
k
(ξ), vjk ), s+ k
≤ f (τj , x (τj ), x (τj − h), u
(ξ), vjk )
3ε/(16(τ∗ − t))
+ 3ε/(16(τ∗ − t))
Then, taking into account (47) and defining k3 = 4βf (τ∗ − t)/ε, for k ≥ k3 , j ∈ 0, k − 1 and τ ∈ [τj , τj+1 ), we have τ H(ξ, xk (ξ), xk (ξ − h), s) − x˙ k (ξ), s dξ τ
≤ ≤
t τj t
f 0 (ξ, xk (ξ), xk (ξ − h), uk (ξ), vjk ) dξ + ε/4
(49)
f 0 (ξ, xk (ξ), xk (ξ − h), uk (ξ), vjk ) dξ + ε/2.
Thus, from (41), (43), (45), (48), (49), for k ≥ max{6(τ∗ − t)/ε, k1 , k2 , k3 }, we obtain ρ◦ (τ, x∗ (τ ), x∗τ (·)) τ H(ξ, x∗ (ξ), x∗ (ξ − h), s) − x˙ ∗ (ξ), s dξ + t
≤ ρ◦ (t, z, w(·)) + ε.
It holds for any ε > 0, therefore we can put ε = 0. Lemma 5. For every (t, z, w(·)) ∈ G, t < ϑ and s ∈ Rn , there exist τ∗ ∈ (t, ϑ] and x∗ (·) ∈ X 0 (t, z, w(·)) such that, for every τ ∈ [t, τ∗ ], the following inequality holds: ρ◦ (τ, x∗ (τ ), x∗τ (·)) τ + H(ξ, x∗ (ξ), x∗ (ξ − h), s) − x˙ ∗ (ξ), s dξ t
Proof. Due to Lemma 4, defining s = ∇z ρ◦ (t, z, w(·)), let us take τ∗ ∈ (t, ϑ] and x∗ (·) ∈ X 0 (t, z, w(·)). By the definition of ci-differentiability of ρ◦ , we have ci ◦ ρ (t, z, w(·))(τ − t) ρ◦ (τ, x∗ (τ ), x∗τ (·))−ρ◦ (t, z, w(·)) = ∂t,w
+x∗ (τ ) − z, ∇z ρ◦ (t, z, w(·)) + o(τ − t),
Then, using (40), for τ ∈ [t, τ∗ ], we derive
+
τ
τ ∈ [t, ϑ].
ci ◦ ∂t,w ρ (t, z, w(·))(τ − t) + o(τ − t)
H(ξ, x∗ (ξ), x∗ (ξ − h), ∇z ρ◦ (t, z, w(·))) dξ ≤ 0.
Dividing this inequality by τ − t, passing to the limit as τ → t + 0, taking into account (f1 ) and (14), we get ci ◦ ∂t,w ρ (t, z, w(·)) + H(t, z, w(−h), ∇z ρ◦ (t, z, w(·))) ≤ 0.
In a similar way, using Lemma 5, we can obtain the opposite inequality. REFERENCES
≤ f 0 (ξ, xk (ξ), xk (ξ − h), uk (ξ), vjk ) + ε/(4(τ∗ − t)).
t
Proof. The Lemma can be proved similar to Lemma 4, using (ρv ) and (f4 ). Theorem 2. Let the value functional ρ◦ of differential game (1), (2) be ci-differentiable at a point (t, z, w(·)) ∈ G, t < ϑ. Then it satisfies HJ equation (15) at this point.
t
From (1), (42), (45), for every j ∈ 0, k − 1 and almost every ξ ∈ [τj , τj+1 ), we derive
143
≥ ρ◦ (t, z, w(·)). 265
Clarke, F.H., Ledyaev, Yu.S., Stern, R.J., and Wolenski, P.R. (1998). Nonsmooth analysis and control theory. Springer, New York. Crandall, M.G. and Lions, P.-L. (1983). Viscosity solutions of Hamilton-Jacobi equations. Trans. Amer. Math. Soc., 277 (1), 1–42. Gomoyunov, M. and Plaksin, A. (2018). On HamiltonJacobi equations for neutral-type differential games. IFAC PapersOnLine, 50 (1), 5109–5114. Isaacs., R. (1965). Differential games. John Wiley, New York. Kim, A.V. (1999). Functional differential equations. Application of i-smooth calculus. Kluwer, The Netherlands Dordrecht. Krasovskii, N.N. and Subbotin, A.I. (1988). Gametheoretical control problems. Springer, New York. Lukoyanov, N.Yu. (2000). Functional equations of the Hamilton-Jacobi type and differential games with hereditary information. Doklady Mathematics, 61 (2), 301– 304. Lukoyanov, N.Yu. (2003). Functional Hamilton-Jacobi type equations with ci-derivatives in control problems with hereditary information. Nonlinear Funct. Anal. and Appl., 8 (4), 535–556. Natanson, I.P. (1960). Theory of functions of a real variable. Volume 2. Frederick Ungar Publishing Co., New-York. Osipov, Yu.S. (1971). Differential games of systems with aftereffect. Dokl. Akad. Nauk SSSR, 196 (4), 779–782. Plaksin, A. (2019). Minimax and viscosity solutions of Hamilton-Jacobi-Bellman equations for time-delay systems. ArXiv:1901.04677 (submitted to Journal of Optimization Theory and Applications). Subbotin, A.I. (1995). Generalized solutions of first order PDEs: the dynamical optimization perspective. Birkh¨auser, Berlin.
ScienceDirect
IFAC PapersOnLine 52-18 (2019) 138–143
On On Hamilton-Jacobi-Bellman-Isaacs Hamilton-Jacobi-Bellman-Isaacs On Hamilton-Jacobi-Bellman-Isaacs Equation for Time-Delay Systems On Hamilton-Jacobi-Bellman-Isaacs Equation for Time-Delay Systems Equation for Time-Delay Systems Equation forAnton Time-Delay Systems Plaksin ∗∗ Anton Plaksin
∗ Anton Anton Plaksin Plaksin ∗∗ ∗ Anton Plaksin ∗ N.N. N.N. Krasovskii Krasovskii Institute Institute of of Mathematics Mathematics and and Mechanics Mechanics of of ∗ ∗Branch N.N. Krasovskii Institute of Mathematics and Mechanics of of the Russian Academy of Sciences, S.Kovalevskaya N.N. Krasovskii Institute of Mathematics and Mechanics of Branch of the Russian Academy of Sciences, S.Kovalevskaya ∗
the the Ural Ural the Str. 16, the Ural Str.Ural 16, N.N. Krasovskii Institute of Mathematics and Mechanics of the Ural Branch of the the Russian Russian Academy of Sciences, S.Kovalevskaya Str. 16, Yekaterinburg, 620990, Russia; Branch of Academy of Sciences, S.Kovalevskaya Str. 16, Yekaterinburg, 620990, Russia; Branch of theUniversity, Russian Academy of620990, Sciences, S.Kovalevskaya Str. 16, Yekaterinburg, 620990, Russia; Yekaterinburg, Russia; Ural Federal Mira str. 19, Yekaterinburg, 620002, Russia, Ural Federal University, Mira str. 620990, 19, Yekaterinburg, 620002, Russia, Yekaterinburg, Russia; Ural Mira str. Ural Federal Federal University, University, Mira str. 19, 19, Yekaterinburg, Yekaterinburg, 620002, 620002, Russia, Russia, (e-mail: [email protected]) (e-mail: [email protected]) Ural Federal University, Mira str. 19, Yekaterinburg, (e-mail: [email protected]) [email protected]) 620002, Russia, (e-mail: (e-mail: [email protected]) Abstract: The The paper paper deals deals with with aa two-person two-person zero-sum zero-sum differential differential game game for for aa dynamical dynamical system system Abstract: Abstract: The paper deals with a two-person zero-sum differential game for aa dynamical system which motion is described by a delay differential equation under an initial condition defined Abstract: The paper deals with a two-person zero-sum differential game for dynamical system defined by by which motion is paper described by a delay differential equation under angame initial Abstract: The deals with aFor two-person zero-sum differential forcondition a dynamical system which motion is by a differential equation an condition by a continuous function. value of this we the Hamiltonwhich motion is described described by a delay delay differential equation under an initial initial condition defined by a piecewise piecewise continuous function. For the the value functional functional ofunder this game, game, we derive derive the defined Hamiltonwhich motion is described bycoinvariant a delay differential equation anthat, initial condition by a piecewise continuous function. For the the value functional functional ofunder this game, game, weif derive derive the defined HamiltonJacobi type equation with derivatives. It is proved the solution of this a piecewise continuous function. For value of this we the HamiltonJacobi type equation with coinvariant derivatives. It is proved that, if the solution of this a piecewise continuous function. For the value functional ofcoincides this game, weif derive the functional. HamiltonJacobi type equation with coinvariant derivatives. It is is proved that, if the the value solution of this this equation satisfies certain smoothness conditions, then it with Jacobi type equation with coinvariant derivatives. It proved that, the solution of equation satisfies certain smoothness conditions, then it coincides withif the functional. Jacobi equation coinvariant Itofis proved that, the value solution this equation satisfies certain smoothness then it coincides with the value functional. On the the type other hand, it with is proved proved that,conditions, atderivatives. the points points coinvariant differentiability, theofvalue value equation satisfies certain smoothness conditions, then itcoinvariant coincides with the value functional. On other hand, it is that, at the of differentiability, the equation satisfies certain smoothness conditions, then it coincides with the value functional. On the other hand, it is proved that, at the points of coinvariant differentiability, the value functional satisfies the derived Hamilton-Jacobi equation. Therefore, this equation can be called On the other hand,theit derived is proved that, at the points of coinvariant differentiability, value Hamilton-Jacobi equation. Therefore, this equation canthe be called functional satisfies On the other hand,the is proved that, at the for points of coinvariant differentiability, value functional satisfies theit derived derived Hamilton-Jacobi equation. Therefore, this equation can canthe be called called the Hamilton-Jacobi-Bellman-Isaacs Hamilton-Jacobi-Bellman-Isaacs equation time-delay systems.this functional satisfies Hamilton-Jacobi equation. Therefore, equation be the equation for time-delay systems. functional satisfies the derived Hamilton-Jacobi equation. Therefore, the Hamilton-Jacobi-Bellman-Isaacs Hamilton-Jacobi-Bellman-Isaacs equation for for time-delay systems.this equation can be called the equation time-delay systems. Copyright © 2019. The Authors. Published by Elsevier All rightssystems. reserved. the Hamilton-Jacobi-Bellman-Isaacs equation for Ltd. time-delay Keywords: delay system, system, differential differential game, game, Hamilton-Jacobi equation, coinvariant coinvariant derivatives, derivatives, Keywords: delay Hamilton-Jacobi equation, Keywords: delay system, system, differential game, Hamilton-Jacobi equation, coinvariant derivatives, value functional, optimal strategies. Keywords: delay differential game, Hamilton-Jacobi equation, coinvariant derivatives, optimal strategies. value functional, Keywords: delay system, value functional, functional, optimal strategies. value optimal differential strategies. game, Hamilton-Jacobi equation, coinvariant derivatives, value functional, optimal strategies. 1. INTRODUCTION INTRODUCTION further further investigations investigations of of differential differential games games for for time-delay time-delay 1. 1. INTRODUCTION INTRODUCTION further investigations investigations of differential differential games for time-delay time-delay systems and the corresponding HJBI equations. 1. further of games for systems and the corresponding HJBI equations. 1. INTRODUCTION further of differential for time-delay systems investigations and the the corresponding corresponding HJBIgames equations. From the differential games theory for ordinary differsystems and HJBI equations. From the differential games theory for ordinary differ- systems and the corresponding HJBI equations. From the differential games theory for differential the equations (see, e.g., e.g., Isaacs (1965); Krasovskii and 2. From differential games theory for ordinary ordinary differential equations (see, Isaacs (1965); Krasovskii and 2. DIFFERENTIAL DIFFERENTIAL GAME GAME From the differential games theory for Hamilton-Jacobi ordinary differential equations (see, e.g., Isaacs (1965); Krasovskii and Subbotin (1988); Osipov (1971)) and 2. DIFFERENTIAL GAME GAME ential equations (see, e.g., Isaacs (1965); Krasovskii and 2. DIFFERENTIAL Subbotin (1988); Osipov (1971)) and Hamilton-Jacobi ential equations (see, e.g., (1971)) Isaacs (1965); Krasovskii and n Subbotin (1988); Osipov and theory Hamilton-Jacobi DIFFERENTIAL GAME (HJ) with partial derivatives (see, Subbotin (1988); Osipov and Hamilton-Jacobi Euclidian R n be the2. n-dimensional (HJ) equations equations with partial(1971)) derivatives theory (see, e.g., e.g., Let space with with the the Let Rnn be the n-dimensional Euclidian space Subbotin (1988); Osipov and theory Hamilton-Jacobi (HJ) equations equations with partial(1971)) derivatives theory (see, e.g., inner (1995); Crandall and Lions (1983); Clarke, (HJ) with partial derivatives (see, e.g., be the n-dimensional Euclidian space with the Let R product ·, · and the norm · . A function be the n-dimensional Euclidian space with the Let R Subbotin (1995); Crandall and Lions (1983); Clarke, inner product ·, · and the norm · . A function n (HJ) equations with partial derivatives theory (see, e.g., n Subbotin (1995);andCrandall Crandall and Lions it(1983); (1983); Clarke, Ledyaev, Stern Stern Wolenskiand (1998)), is well well Clarke, known x(·) be → theR ·, n-dimensional Euclidian the Let :R[a, Subbotin (1995); Lions inner product · and norm ·· space . function b] is piecewise if there n inner product ·, · called and the the norm continuous . A A with function Ledyaev, and Wolenski (1998)), it is known x(·) : [a, b] → R is called piecewise continuous if there n Subbotin (1995); Crandall and Lions it Clarke, Ledyaev, Stern and Wolenski (1998)), it(1983); isa well well known inner that, on the one hand, a value function of differential n=is product ·, · and the norm · . A function Ledyaev, Stern and Wolenski (1998)), is known x(·) : [a, b] → R called piecewise continuous if there exist numbers a ξ < ξ < . . . < ξ = b such that, for x(·) b] → R piecewise if there that, on the oneand hand, a value(1998)), functionitofisa well differential exist: [a, numbers a n=isξ11called < ξ22 < . . . < ξkkcontinuous = b such that, for Ledyaev, Stern Wolenski known x(·) that, on the one hand, value function function differential game on at points points of hand, differentiability satisfies of theaacorrespondcorrespond: i[a, b]1,→ R piecewise ifonthere that, the one aa value of differential exist numbers a1,= =is ξ11called < ξ22 < < . ..x(·) .< < ξξiskkcontinuous = bb such such that, that, for each ∈ k − the function continuous the exist numbers a ξ < ξ . . = for game at of differentiability satisfies the each i ∈ 1, k − 1, the function x(·) is continuous on the that, on the one a value function of acorresponddifferential game at points points of hand, differentiability satisfies the corresponding Hamilton-Jacobi-Bellman-Isaacs Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation with exist a)1, =and ξ1 there < ξ2 < . .x(·) .< ξis = blimit suchofthat, for game at of differentiability satisfies the each iinumbers ∈[ξ1, 1, k − 1, the function x(·) is continuous on the interval , ξ exists a finite x(ξ) as k continuous i i+1 each ∈ k − the function on the ing (HJBI) equation with as [ξ1, i , ξi+1 ) and there exists a finite limit of x(ξ) game atderivatives points of differentiability satisfies the correspondn ing Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation with interval partial and, on the other hand, the continueach i ∈ k − 1, the function x(·) is continuous on the ing Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation with interval [ξ , ξ ) and there exists a finite limit of x(ξ) as ξ approaches ξ from left. Denote by PC([a, b], R ) and n i+1 i+1 interval [ξii , ξi+1 ) and there exists a finite limit of x(ξ) as partial derivatives and, on the other hand, the continuξ approaches ξ from left. Denote by PC([a, b], R ) and i+1 n ing Hamilton-Jacobi-Bellman-Isaacs (HJBI) equation with n partial derivatives and, on the other hand, the continuously differentiable solution of the HJBI equation coinn ) and interval [ξ , ξ ) and there exists a finite limit of x(ξ) as partial derivatives and, on the other hand, the continuξ approaches ξ from left. Denote by PC([a, b], R Lip([a, b], R ) the linear spaces of piecewise continuous i i+1 n i+1 ξLip([a, approaches ξthe from spaces left. Denote by PC([a, b], Rn ) and ously differentiable solution of the HJBI equation coini+1 linear b], R ) of piecewise continuous n ξ partial derivatives and, on the other hand,equation the continunb], R ) and ously differentiable solution of the HJBI coincides with with the value value function. In Lukoyanov (2003) the Lipschitz n ξLip([a, approaches from left. Denote by b] PC([a, ously differentiable solution of In the HJBI equation coinLip([a, b], R ) the linear spaces of piecewise continuous continuous functions x(·) : [a, → R i+1 linear n. b], R ) the spaces of piecewise continuous and cides the function. Lukoyanov (2003) the continuous functions x(·) : [a, b] → Rnn . n ously differentiable solution of In theLukoyanov HJBI equation coincides with theforvalue value function. In Lukoyanov (2003) the Lipschitz similarwith result differential games for time-delay systems Lip([a, b], R ) the linear spaces of piecewise continuous and cides the function. (2003) the Lipschitz continuous functions x(·) :: [a, R Lipschitz continuous functions x(·) [a, b] b] → → Rn .. similar result forvalue differential games for time-delay systems Let t < ϑ and h > 0. Let us denote cides with the function. In Lukoyanov (2003) the 0 similar result for differential differential games for time-delay time-delay systems Lipschitz and the corresponding HJBI equations with coinvariant continuous functions x(·) : [a, b] → R . similar result for games for systems Let t < ϑ and h > 0. Let us denote and theresult corresponding HJBIgames equations with coinvariant Let tt000 < <ϑ ϑ and and h h> > 0. 0. Let Let us denote denote similar for differential for time-delay systems n us n and the corresponding corresponding HJBI equations with coinvariant derivatives was Note that equations were Let and the coinvariant PC([−h, 0], R G = [t × R n ),us denote n × PC. derivatives was obtained. obtained.HJBI Noteequations that HJBI HJBIwith equations were 0 ,, ϑ] Let t0PC <= ϑ and h > 0. Let PC = PC([−h, 0], R ), G = [t ϑ] × R × PC. 0 and the corresponding HJBI equations with coinvariant n n derivatives was obtained. Note that HJBI equations were considered on the space of continuous functions. In the n ), n × PC. derivatives was obtained. Note that HJBI equationsInwere PC = PC([−h, 0], R G = [t , ϑ] × R 0 considered on the space of continuous functions. the PC = PC([−h, 0], R ), G = [t , ϑ] × R × PC. 0 n n derivatives was obtained. Note that general HJBI equations were norms PC: considered on the the spaceinof of continuous functions. InHJBI the Define paper the the similar similar result the more case for forIn considered on space continuous functions. the PCthe = following PC([−h, 0], R ), on = space [t0 , ϑ] × R × PC. Define the following norms onGthe the space PC: paper result inofthe more general case HJBI Define the following norms on the space PC: considered on the space continuous functions. In the 0 paper the similar similar result in the more general general case functions for HJBI HJBI Define the following equations on the the result space in of the piecewise continuous norms on the space PC: paper the more case for equations on space of piecewise continuous 00 norms on the space PC: paper the similar in more general case functions for HJBI Define the following equations on the the result space of the piecewise continuous functions is it to positional optimal 0 w(ξ) dξ, equations on space of piecewise continuous w(·) = w(·) = sup is proved. proved. Firstly, Firstly, it allows allows to construct construct positionalfunctions optimal 1 = w(·) w(·) ∞ sup w(ξ) . w(ξ) . 0 w(ξ) dξ, equations on the space of piecewise continuous functions 1 ∞ = ξ∈[−h,0] is proved. Firstly, it allows to construct positional optimal strategies of players in differential games for time-delay is proved. Firstly, it allows to construct positional optimal w(·) = w(ξ) dξ, w(·) = sup strategies of players in differential games for time-delay 1 ∞ w(·) 1 = −h w(ξ) dξ, w(·) ∞ = ξ∈[−h,0] sup w(ξ) . w(ξ) . is proved.inFirstly, it allows to construct positional optimal strategies ofthe players in differential games forHJBI time-delay systems, cases when the corresponding equa−h ξ∈[−h,0] strategies of players in differential games for time-delay w(·) 1 = w(ξ) dξ, w(·) ∞ = ξ∈[−h,0] sup w(ξ) . systems, inofthe cases in when the corresponding HJBI equa−h strategies players differential games for time-delay −h systems, insmooth the cases cases wheneven the corresponding corresponding HJBI equation has has aain solution if an an initial initial motion motion history ξ∈[−h,0] systems, the when the HJBIhistory equa- We consider a two-person differential game tion smooth solution even if systems, the cases wheneven theSecondly, corresponding HJBIhistory equa- We consider a−htwo-person differential game for for the the dynamdynamtion has aainsmooth smooth solution even if an an initial initial motion has of the choice of tion has solution if has points points of discontinuities. discontinuities. Secondly, themotion choicehistory of the the We We consider a two-person differential game for the dynamdynamical system described by the time-delay equation consider a two-person differential game for the tion has a smooth solution even if an initial motion history ical system described by the time-delay equation has points of discontinuities. Secondly, the choice of the space of piecewise continuous functions may in the future has of discontinuities. Secondly, may the choice of the We consider a two-person differential game for the dynamspacepoints of piecewise continuous functions in the future ical system described by the time-delay equation ical system described by the time-delay equation has points of discontinuities. Secondly, the choice of the space of piecewise piecewise continuous functions may mayofin in the the theory future ical system significantly facilitate many constructions constructions x(τ ˙˙ described )) = x(τ − v(τ space of continuous functions future by), x(τ = ff (τ, (τ, x(τ x(τ ),the x(τ time-delay − h), h), u(τ u(τ ), ), equation v(τ )), )), significantly facilitate many ofinthe the theory (1) space of piecewise continuous functions may the theory future significantly facilitate many constructions of coinvariant the theory x(τ ˙ )) = = ff (τ, (τ, x(τ x(τ ), ),nx(τ x(τ − − h), h), u(τ u(τ ), ), v(τ v(τ )), )), of generalized solutions of HJ equations with significantly facilitate many constructions of the x(τ ˙ (1) of generalized solutions of HJ equations with coinvariant τ ∈ [t , ϑ], x(τ ) ∈ R u(τ ) ∈ U, v(τ )) ∈ V, n, 0 (1) significantly facilitate many constructions of the theory x(τ ˙ ) = f (τ, x(τ ), x(τ − h), u(τ ), v(τ )), τ ∈ [t , ϑ], x(τ ) ∈ R , u(τ ) ∈ U, v(τ ∈ V, of generalized solutions of HJ equations with coinvariant (1) derivatives. Earlier, similar investigations of HJ equations 0 n of generalized solutions of HJ equations with n derivatives. Earlier, similar investigations of HJcoinvariant equations τ∈ ∈ [t [t00 ,, ϑ], ϑ], x(τ x(τ )) ∈ ∈R R ,, u(τ u(τ )) ∈ ∈ U, U, v(τ v(τ )) ∈ ∈ V, V, (1) of generalized solutions ofcontrol HJ equations with derivatives. Earlier, similar investigations offor HJcoinvariant equations andττthe corresponding to optimal problems time-delay n quality index derivatives. Earlier, similar investigations of HJ equations ∈ [tquality x(τ ) ∈ R , u(τ ) ∈ U, v(τ ) ∈ V, corresponding to optimal control problemsoffor time-delay and the 0 , ϑ], index derivatives. Earlier, similar investigations HJ equations corresponding to optimal optimal control problems for paper time-delay systems were were presented presented in control Plaksinproblems (2019). The The con- and index corresponding to for time-delay and the theγquality quality index systems in Plaksin (2019). paper con= σ(x(ϑ), x corresponding to optimal control problems for paper time-delay systems were presented presented ingives Plaksin (2019). The The paper continues Plaksin (2019) and a theoretical foundation for and the quality index γ = σ(x(ϑ), xϑϑ (·)) (·)) systems were in Plaksin (2019). con tinues Plaksin (2019) and gives a theoretical foundation for γ = σ(x(ϑ), x ϑ γ = σ(x(ϑ), xϑϑ (·)) (·)) systems were presented in Plaksin (2019). The paper con(2) tinues Plaksin (2019) and gives a theoretical foundation for ϑ tinues Plaksin (2019) and gives a theoretical foundation for 0 (2) γ = σ(x(ϑ), xϑx(ξ), (·)) x(ξ − h), u(ξ), v(ξ)) dξ. ϑϑ f 0 (ξ, This Plaksin + is supported bygives the Grant of the President of for the tinues (2019) and a theoretical foundation (2) This work + f (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ. (2) work is supported by the Grant of the President of the 0 0 t ϑ This work + t ff (ξ, (ξ, x(ξ), x(ξ), x(ξ x(ξ − − h), h), u(ξ), u(ξ), v(ξ)) v(ξ)) dξ. dξ. Russian Federation (project by no. the MK-3566.2019.1). is Grant (2) + This work is supported supported by the Grant of of the the President President of of the the Russian Federation (project no. MK-3566.2019.1). 0 + tt f (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ. Russian Federation (project by no. the MK-3566.2019.1). This work is supported Grant of the President of the Russian Federation (project no. MK-3566.2019.1).
t Russian Federation (project 2405-8963 Copyright © 2019. no. TheMK-3566.2019.1). Authors. Published by Elsevier Ltd. 260 All rights reserved. Copyright © 2019 IFAC Copyright © 2019 IFAC 260 Peer review© under responsibility of International Federation of Automatic Copyright 2019 IFAC IFAC 260 Control. Copyright © 2019 260 10.1016/j.ifacol.2019.12.220 Copyright © 2019 IFAC 260
2019 IFAC TDS Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
Here τ is the time variable; x(τ ) is the value of the state vector at the time τ ; t ∈ [t0 , ϑ] is the time of the control process beginning; hereinafter, for each τ ∈ [t0 , ϑ], the symbol xτ (·) denotes the function on the interval [−h, 0] defined by xτ (ξ) = x(τ + ξ), ξ ∈ [−h, 0]; u(τ ) and v(τ ) are control actions of the first and second players, respectively; U and V are known compacts of finite-dimensional spaces. The first player aims to minimize the value γ of the quality index, while the second player aims to maximize it. We assume that the following conditions hold: (f1 ) The functions f (t, x, y, u, v) ∈ Rn , f 0 (t, x, y, u, v) ∈ R, t ∈ [t0 , ϑ], x, y ∈ Rn , u ∈ U, v ∈ V, are continuous. (f2 ) For every α > 0, there exists λf > 0 such that f (t, x, y, u, v) − f (t, x , y , u, v) +|f 0 (t, x, y, u, v) − f 0 (t, x , y , u, v)| ≤ λf x − x + y − y
for any t ∈ [t0 , ϑ], x, y, x , y ∈ O(α) = {x ∈ R : x ≤ α}, u ∈ U and v ∈ V. (f3 ) There exists a constant cf > 0 such that f (t, x, y, u, v) + |f 0 (t, x, y, u, v)| ≤ cf 1 + x + y
n
for any t ∈ [t0 , ϑ], x, y ∈ Rn , u ∈ U and v ∈ V. (f4 ) For every t ∈ [t0 , ϑ] and x, y, s ∈ Rn , we have min max f (t, x, y, u, v), s + f 0 (t, x, y, u, v) u∈U v∈V = max min f (t, x, y, u, v), s + f 0 (t, x, y, u, v)
139
u(τ ) = U (τj , x(τj ), xτj (·)), τ ∈ [τj , τj+1 ), j = 1, l. (4)
This control law together with v(·) ∈ V(t) uniquely determine the control process realization {x(·), u(·), v(·)} and the value γ = γ(t, z, w(·); U, ∆δ ; v(·)) of quality index (2). The guaranteed result of the strategy U is defined by ρu (t, z, w(·); U ) = lim sup sup γ(t, z, w(·); U, ∆δ ; v(·)). (5) δ↓0 ∆δ v(·)
The optimal guaranteed result of the first player is ρ◦u (t, z, w(·)) = inf ρu (t, z, w(·); U ).
(6)
U
◦
A strategy U of the first player is called optimal if ρu (t, z, w(·); U ◦ ) = ρ◦u (t, z, w(·)). Similarly, with the corresponding changes, for the second player, we define a control strategy V : G → V, control law {V, ∆δ } that forms a function v(·) ∈ V(t) by v(τ ) = V (τj , x(τj ), xτj (·)), t ∈ [τj , τj+1 ), j = 1, l,
the guaranteed result of the strategy V
ρv (t, z, w(·); V ) = lim inf inf γ(t, z, w(·); u(·); V, ∆δ ), (7) δ↓0 ∆δ u(·)
and the optimal guaranteed result ρ◦v (t, z, w(·)) = sup ρv (t, z, w(·); V ).
(8)
V
A strategy V ◦ of the second player is called optimal if ρv (t, z, w(·); V ◦ ) = ρ◦v (t, z, w(·)). Due to definitions (5)–(8), we have
v∈V u∈U
(σ) For every α > 0, there exists λσ > 0 such that |σ(z, w(·)) − σ(z , w (·))| ≤ λσ z − z + w(·) − w (·) 1
for any (z, w(·)), (z , w (·)) ∈ P (α), where
P (α) = {(z, w(·)) ∈ Rn × PC : z ≤ α, w(·) ∞ ≤ α}.
Let (t, z, w(·)) ∈ G. Denote by Λ(t, z, w(·)) the set of functions x(·) ∈ PC([t − h, ϑ], Rn ) such that x(τ ) = w(τ − t), τ ∈ [t − h, t), x(τ ) = y(τ ), τ ∈ [t, ϑ], where y(·) ∈ Lip([t, ϑ], Rn ) and y(t) = z.
Denote by U(t) and V(t) sets of measurable functions u(·) : [t, ϑ] → U and v(·) : [t, ϑ] → V, respectively. It is known that, under the conditions above, for each u(·) ∈ U(t) and v(·) ∈ V(t), there exists a unique motion x(·) of system (1) that is a function x(·) ∈ Λ(t, z, w(·)) that satisfies equation (1) for almost every τ ∈ [t, ϑ]. The triple {x(·), u(·), v(·)} is called a control process realization. Note that this control process realization uniquely determines the value of quality index (2).
ρ◦v (t, z, w(·)) ≤ ρ◦u (t, z, w(·)), ◦
(t, z, w(·)) ∈ G.
ρ◦u (t, z, w(·)) ◦
(9)
ρ◦v (t, z, w(·))
= If the equality ρ (t, z, w(·)) := holds for any (t, z, w(·)) ∈ G, then ρ : G → R is called the value functional of differential game (1), (2). One can show (see, e.g., Krasovskii and Subbotin (1988)) that the value functional ρ◦ has the following properties: (ρu ) For every (t, z, w(·)) ∈ G, τ ∈ [t, ϑ], ε > 0 and v(·) ∈ V(t), there exists u(·) ∈ U such that, for the control process realization {x(·), u(·), v(·)}, we have τ ◦ ρ (τ, x(τ ), xτ (·)) + f 0 (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ t
≤ ρ◦ (t, z, w(·)) + ε.
(ρv ) For every (t, z, w(·)) ∈ G, τ ∈ [t, ϑ], ε > 0 and u(·) ∈ U (t), there exists v(·) ∈ V such that, for the control process realization {x(·), u(·), v(·)}, we have τ ◦ ρ (τ, x(τ ), xτ (·)) + f 0 (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ t
≥ ρ◦ (t, z, w(·)) − ε.
According to Krasovskii and Subbotin (1988) (see also Lukoyanov (2003) for time-delay systems), differential game (1), (2) is posed as follows.
3. PROPERTIES OF THE TIME-DELAY SYSTEM
By a control strategy of the first player, we mean an arbitrary function U : G → U. Let us fix (t, z, w(·)) ∈ G and a partition of the interval [t, ϑ]: ∆δ = {τj : τ1 = t, 0 < τj+1 −τj ≤ δ, j = 1, l, τl+1 = ϑ}. (3)
Taking the constant cf > 0 form (f3 ), we denote F η (x, y) = {f ∈ Rn : f ≤ cf (1 + x + y ) + η}, x, y ∈ Rn , η ≥ 0. (10) Let (t, z, w(·)) ∈ G and η ≥ 0. Denote by X η (t, z, w(·)) the set of functions x(·) ∈ Λ(t, z, w(·)) that satisfy the following time-delay differential inclusion:
The pair {U, ∆δ } defines a control law that forms a piecewise constant function u(·) ∈ U(t) according to the following step-by-step rule:
261
x(τ ˙ ) ∈ F η (x(τ ), x(τ − h)) for a.e. τ ∈ [t, ϑ].
(11)
2019 IFAC TDS 140 Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
Note that the set X η (t, z, w(·)) is not empty. In particular, for each u(·) ∈ U(t) and v(·) ∈ V(t), the motion x(·) of system (1) satisfies the inclusion (12) x(·) ∈ X 0 (t, z, w(·)) ⊂ X η (t, z, w(·)), η ≥ 0. Proposition 1. Let (t, z, w(·)) ∈ G and η ≥ 0. Then there exist αX , λX > 0 such that (x(τ ), xτ (·)) ∈ P (αX ), x(τ ) − x(τ ) ≤ λX |τ − τ |,
for any τ, τ ∈ [t, ϑ] and x(·) ∈ X η (t, z, w(·)). Proposition 2. Let (t, z, w(·)) ∈ G, t < ϑ and η ≥ 0. Let a sequence xk (·) ∈ X η (t, z, w(·)), k ∈ N be chosen. Then there exist a subsequence xki (·) and a function x∗ (·) ∈ X η (t, z, w(·)) such that max
τ ∈[t−h,ϑ]
xki (τ ) − x∗ (τ ) → 0 as i → ∞.
Proposition 3. Let (t, z, w(·)) ∈ G and η ≥ 0. For every ε > 0, there exists δ > 0 such that, for every x(·) ∈ X η (t, z, w(·)) and τ, τ ∈ [t, ϑ]: |τ − τ | ≤ δ, we have: x(τ ) − x(τ ) + xτ (·) − xτ (·)1 ≤ ε.
Proposition 1 and 2 can be proved similar to Proposition 3.1 and Lemma 4.8 in Plaksin (2019), respectively. Proposition 3 one can proved, using approximation of w(·) by a Lipschitz function (see, e.g., (Natanson, 1960, p. 214)) and Proposition 1. 4. HJBI EQUATION Following Kim (1999); Lukoyanov (2000), a functional ϕ : G → R is called coinvariantly differentiable (ci-differentiable) at a point (t, z, w(·)) ∈ G, t < ϑ if there exist ci ϕ(t, z, w(·)) ∈ R and ∇z ϕ(t, z, w(·)) ∈ Rn such that, ∂t,w for every v ∈ Rn , x(·) ∈ Λ(t, z, w(·)) and τ ∈ [t, ϑ], the following relation holds: ci ϕ(τ, v, xτ (·)) − ϕ(t, z, w(·)) = ∂t,w ϕ(t, z, w(·))(τ − t) (13) +v − z, ∇z ϕ(t, z, w(·)) + o(|τ − t| + v − z), where the function xτ (·) ∈ PC is defined by xτ (ξ) = x(τ + ξ), ξ ∈ [−h, 0], the value o(·) depends on the triplet {t, z, x(·)}, and o(δ)/δ → 0 as δ → +0. Then ci ϕ(t, z, w(·)) is called the ci-derivative of ϕ with respect ∂t,w to {t, w(·)} and ∇z ϕ(t, z, w(·)) is the gradient of ϕ with respect to z. Let us note that if ϕ does not depend on the functional variable w(·), then the definition of cidifferentiability coincides with the definition of differentiability of functions. For differential game (1), (2), we define the Hamiltonian H(t, x, y, s) = min max f (t, x, y, u, v), s (14) u∈U v∈V +f 0 (t, x, y, u, v) , t ∈ [t0 , ϑ], x, y, s ∈ Rn .
and consider the following HJ equation
ci ϕ(t, z, w(·)) + H(t, z, w(−h), ∇z ϕ(t, z, w(·))) = 0, ∂t,w
(t, z, w(·)) ∈ G,
t < ϑ,
(15)
with the terminal condition
ϕ(ϑ, z, w(·)) = σ(z, w(·)),
(ϑ, z, w(·)) ∈ G.
(16)
Following Plaksin (2019), define the class of functionals in which we will search a solution of the problem (15), (16). Denote by Φ the set of functionals ϕ = ϕ(t, z, w(·)) ∈ R, 262
(t, z, w(·)) ∈ G which are continuous with respect to t and satisfy the following Lipschitz condition: for every α > 0, there exists λϕ > 0 such that |ϕ(t, z, w(·))−ϕ(t, z , w (·))| ≤ λϕ z−z +w(·)−w (·)1 .
for any t ∈ [t0 , ϑ], (z, w(·)), (z , w (·)) ∈ P (α). The choice of this class is motivated, in particular, by the inclusion ρ◦ ∈ Φ, (17) which can be shown by scheme of Lemma 4.1 in Plaksin (2019). 5. OPTIMAL STRATEGIES Lemma 1. Let ϕ ∈ Φ, (t, z, w(·)) ∈ G and η ≥ 0. For every ε > 0, there exists δ = δ(ε) > 0 such that, for every x(·) ∈ X η (t, z, w(·)) and τ, τ ∈ [t, ϑ]: |τ − τ | ≤ δ, the following inequality holds: |ϕ(τ, x(τ ), xτ (·)) − ϕ(τ , x(τ ), xτ (·))| ≤ ε.
Proof. For the sake of a contradiction, suppose that there exist ε > 0 and xk (·) ∈ X η (t, z, w(·)), τk , τk ∈ [t, ϑ]: |τk − τk | ≤ 1/k, k ∈ N such that |ϕ(τk , xk (τk ), xkτk (·)) − ϕ(τk , xk (τk ), xkτ (·))| > ε, k
k ∈ N.
Without loss of generality, taking into account Proposition 2, we can suppose that there exist τ∗ ∈ [t, ϑ] and x∗ (·) ∈ X η (t, z, w(·)) such that τk → τ∗ ,
max
τ ∈[t−h,ϑ]
τk → τ∗ ,
xk (τ ) − x∗ (τ ) → 0,
as
k → ∞.
(18)
Due to Proposition 1 and the inclusion ϕ ∈ Φ, there exist k1 > 0 and λϕ > 0 such that, for every k > k1 , we have |ϕ(τk , xk (τk ), xkτk (·)) − ϕ(τ∗ , x∗ (τ∗ ), x∗τ∗ (·))|
≤ |ϕ(τk , xk (τk ), xkτk (·)) − ϕ(τk , x∗ (τ∗ ), x∗τ∗ (·))|
+|ϕ(τk , x∗ (τ∗ ), x∗τ∗ (·)) − ϕ(τ∗ , x∗ (τ∗ ), x∗τ∗ (·))| ≤ λϕ xk (τk ) − x∗ (τ∗ ) + xkτk (·) − x∗τ∗ (·)1 + ε/4.
According to Proposition 3 and (18), there exists k2 > 0 such that, for every k > k2 , we derive xk (τk ) − xk (τ∗ ) + xkτk (·) − xkτ∗ (·)1 ≤ ε/(8λϕ ), xk (τ∗ ) − x∗ (τ∗ ) + xkτ∗ (·) − x∗τ∗ (·)1 ≤ ε/(8λϕ ).
Thus, for k > max{k1 , k2 }, we obtain
|ϕ(τk , xk (τk ), xkτk (·)) − ϕ(τ∗ , x∗ (τ∗ ), x∗τ∗ (·))| ≤ ε/2.
In a similar way, we can show that there exists k3 > 0 such that, for every k > k3 , we have |ϕ(τk , xk (τk ), xkτ (·)) − ϕ(τ∗ , x∗ (τ∗ ), x∗τ∗ (·))| ≤ ε/2. k
Thus, for k > max{k1 , k2 , k3 }, we conclude ε < ε. Lemma 2. Let ϕ ∈ Φ, (t, z, w(·)) ∈ G and η ≥ 0. There exists β > 0 such that, for every x(·) ∈ X η (t, z, w(·)), the following estimate holds: |ϕ(τ, x(τ ), xτ (·))| ≤ β,
τ ∈ [t, ϑ].
(19)
Proof. Since ϕ ∈ Φ, there exists β0 > 0 satisfying |ϕ(τ, 0, θ(·) ≡ 0)| ≤ β0 , τ ∈ [t, ϑ]. Moreover, taking αX > 0 from Proposition 1, one can choose λϕ such that
2019 IFAC TDS Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
|ϕ(τ, x(τ ), xτ (·)) − ϕ(τ, 0, θ(·))| ≤ λϕ (x(τ ) + xτ (·)1 ), η
τ ∈ [t, ϑ],
x(·) ∈ X (t, z, w(·)).
Thus, defining β = β0 + λϕ (1 + h)αX , we obtain (19). Lemma 3. Let ϕ ∈ Φ be ci-differentiable at every point ci (t, z, w(·)) ∈ G, t < ϑ and the inclusions ∂t,w ϕ, ∇z ϕ ∈ Φ ∗ hold. Let (t, z, w(·)) ∈ G and x (·) ∈ X 0 (t, z, w(·)). Then the function ω ∗ (τ ) = ϕ(τ, x∗ (τ ), x∗τ (·)), τ ∈ [t, ϑ] is Lipschitz continuous and ci ω˙ ∗ (τ ) = ∂t,w ϕ(τ, x∗ (τ ), x∗τ (·)) ∗
+x˙ (τ ), ∇z ϕ(τ, x
∗
(τ ), x∗τ (·))
for a.e. τ ∈ [t, ϑ].
(20)
Proof. Taking cf > 0 from condition (f3 ) and αX , λX > 0 from Proposition 1, let us define η = cf (1 + 2αX ). Then, ci due to inclusions ∂t,w ϕ, ∇z ϕ ∈ Φ and Lemma 2, there exists β∂ϕ , β∇ϕ > 0 such that ci ϕ(τ, x(τ ), xτ (·))| |∂t,w
≤ β∂ϕ , |∇z ϕ(τ, x(τ ), xτ (·))| ≤ β∇ϕ , x(·) ∈ X η (t, z, w(·)).
τ ∈ [t, ϑ],
(21)
Let us define the functions xk (·) ∈ Λ(t, z, w(·)): τ k x (τ ) = k x∗ (max{ξ, t}) dξ, τ ∈ [t, ϑ], k ∈ N. τ −1/k
Then these functions are continuously differentiable and, for every τ ∈ [t, ϑ] and k ∈ N, satisfy relations (xk (τ ), xkτ (·)) ∈ P (αX ),
Moreover, one can show that
x˙ k (τ ) ≤ λX .
max x∗ (τ ) − xk (τ ) → 0 as k → ∞.
τ ∈[t,ϑ]
(22) (23)
0
and, using definition (11) of the set X (t, z, w(·)) and in accordance with the choice of the number η, for all τ ∈ [t, ϑ], we derive τ k x˙ ∗ (ξ) dξ x˙ (τ ) ≤ k max{τ −1/k,t}
τ
≤k
max{τ −1/k,t}
It means that
cf 1 + x∗ (ξ) + x∗ (ξ − h) dξ ≤ η.
xk (·) ∈ X η (t, z, w(·)), k
k
k ∈ N.
(24)
(τ ), xkτ (·)),
Define the functions ω (τ ) = ϕ(τ, x τ ∈ [t, ϑ], k ∈ N. Then, according to the inclusion ϕ ∈ Φ and (22), there exists λϕ > 0 such that, for every τ ∈ [t, ϑ], we have |ω ∗ (τ ) − ω k (τ )| ≤ λϕ x∗ (τ ) − xk (τ ) + x∗τ (·) − xkτ (·)1 . Then from (23) we obtain
max ω ∗ (τ ) − ω k (τ ) → 0
τ ∈[t,ϑ]
as k → ∞.
(25)
Taking into account ci-differentiability of ϕ, let us calculate right derivatives of the functions ω k (·) as follows ω k (ξ) − ω k (τ ) d+ ω k (τ )/ dτ = lim ξ→τ +0 ξ−τ (26) ci = ∂t,w ϕ(τ, xk (τ ), xkτ (·)) k
+x˙ (τ ), ∇z ϕ(τ, x
k
(τ ), xkτ (·)),
τ ∈ (t, ϑ).
141
ci Due to the inclusions ∂t,w ϕ, ∇z ϕ ∈ Φ, Lemma 1 and continuous differentiability of xk (·), we obtain that the right-hand side of this equation is continuous on (t, ϑ). Consequently, the function d+ ω k (τ )/ dτ is also continuous on (t, ϑ). Then one can show that the function ω(·) is differentiable on (t, ϑ) and ω˙ k (τ ) = d+ ω k (τ )/ dτ , τ ∈ (t, ϑ). Thus, from (21), (22), (24), (26), we derive
|ω˙ k (τ )| ≤ β∂ϕ + λX β∇ϕ := λω ,
It means that |ω k (τ ) − ω k (τ )| ≤ λω |τ − τ |,
τ ∈ (t, ϑ). τ, τ ∈ [t, ϑ].
Passing to the limit in this estimate as k → ∞, considering (25), we obtain that the function ω ∗ (·) is Lipschitz continuous. Equality (20) is obtained similar to (26) at the points of differentiability of functions ω ∗ (·) and x∗ (·). Let us consider the following players control strategies: U ◦ (t, z, w(·)) ∈ argmin max χ(t, z, w(·), u, v), u∈U
v∈V
v∈V
u∈U
V ◦ (t, z, w(·)) ∈ argmax min χ(t, z, w(·), u, v),
(27)
where (t, z, w(·)) ∈ G and χ(t, z, w(·), u, v) = f (t, z, w(−h), u, v), ∇ϕz (t, z, w(·)) +f 0 (t, z, w(−h), u, v). (28) Theorem 1. Let a functional ϕ ∈ Φ be ci-differentiable at every point (t, z, w(·)) ∈ G, t < ϑ, satisfies HJ equation (15) with terminal condition (16) and the inclusions ci ∂t,w ϕ, ∇z ϕ ∈ Φ hold. Then the control strategies U ◦ and V ◦ defined by (27) are optimal, and ϕ is the value functional of differential game (1), (2). Proof. The proof is carried out by the scheme from Lukoyanov (2003) (see also Gomoyunov and Plaksin (2018)). Let (t, z, w(·)) ∈ G. If t = ϑ, then the validity of the theorem follows from the definition of ρ◦ and (16). Let t < ϑ. In accordance with (6), (8) and (9), for proving the theorem, it is sufficient to show that ρu (t, z, w(·); U ◦ ) ≤ ϕ(t, z, w(·)) ≤ ρv (t, z, w(·); V ◦ ). (29)
Let us prove the first inequality. Due to (5), we should show that, for any ε > 0, there exists δ > 0 such that the following statement is valid. Let ∆δ be a partition (3). Then, for the control process realization {x(·), u(·), v(·)} generated by the control law of the first player {U ◦ , ∆δ } and v(·) ∈ V(t), the value γ = γ(t, z, w(·); U ◦ , ∆δ ; v(·)) of quality index (2) satisfies the inequality ϑ γ = σ(x(ϑ), xϑ (·)) + f 0 (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ t
≤ ϕ(t, z, w(·)) + ε.
(30)
Due to the inclusion ∇z ϕ ∈ Φ and Lemma 2, there exists β∇ϕ > 0 such that, for every x(·) ∈ X 0 (t, z, w(·)), we have |∇z ϕ(τ, x(τ ), xτ (·))| ≤ β∇ϕ , τ ∈ [t, ϑ]. (31) Denote
ε∗ = ε/(2(ϑ − t)).
(32)
Let −h < ξ1 < ξ2 < . . . < ξk < 0 are discontinuity points of the function w(·). Define the sets Ii = [ξi + t + h, ξi+1 + t + h) ∩ [t, ϑ],
i ∈ 1, k − 1
Ik = [ξl + t + h, t + h) ∩ [t, ϑ], Ik+1 = [min{t + h, ϑ}, ϑ]. 263
2019 IFAC TDS 142 Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
Then, according to piecewise continuity of w(·), Proposition 1 and condition (f1 ), there exists δf > 0 such that, for every x(·) ∈ X 0 (t, z, w(·)), i ∈ 1, k + 1, τ, τ ∈ Ii , u ∈ U and v ∈ V, the following estimates hold: f (τ, x(τ ), x(τ − h), u, v)
−f (τ , x(τ ), x(τ − h), u, v) ≤ ε∗ /β∇ϕ ,
|f 0 (τ, x(τ ), x(τ − h), u, v)
(33)
Moreover, there exists βf > 0 such that
x(·) ∈ X 0 (t, z, w(·)),
u ∈ U,
(34) v ∈ V.
ci ϕ, ∂t,w ϕ, ∇z ϕ 0
Due to Lemma 1 and the inclusions ∈ Φ, there exists δϕ > 0 such that, for every x(·) ∈ X (t, z, w(·)) and τ, τ ∈ [t, ϑ]: |τ − τ | ≤ δϕ , we obtain |ϕ(τ, x(τ ), xτ (·)) − ϕ(τ , x(τ ), xτ (·))| ≤ ε/(4(k + 1)),
ci ci ϕ(τ, x(τ ), xτ (·)) − ∂t,w ϕ(τ , x(τ ), xτ (·))| ≤ ε∗ , (35) |∂t,w
v∈V
= H(τj , x(τj ), x(τj − h), ∇ϕ(τj , x(τj ), xτj (·))).
Thus, taking into account (15), we obtain ω(τ ˙ ) ≤ ε∗ for a.e. τ ∈ [τj , τj+1 ], j ∈ J.
6. CI-DIFFERENTIABILITY PROPERTIES OF THE VALUE FUNCTIONAL
f (τ, x(τ ), x(τ − h), u, v) ≤ βf ,
τ ∈ [t, ϑ],
≤ max χ(τj , x(τj ), xτj (·), U ◦ (τj , x(τj ), xτj (·)), v)
Then, from (32), (39), we conclude (38). The first inequality in (29) is proved. Due to (f4 ), the second inequality in (29) can be proved in a similar way.
−f 0 (τ , x(τ ), x(τ − h), u, v)| ≤ ε∗ .
|f 0 (τ, x(τ ), x(τ − h), u, v)| ≤ βf ,
Due to (4), (14), (27), we have χ(τj , x(τj ), xτj (·), u(τ ), v(τ ))
|∇z ϕ(τ, x(τ ), xτ (·)) − ∇z ϕ(τ , x(τ ), xτ (·))| ≤ ε∗ /βf . Define δ = min{δf , δϕ , ε/(4βf (k + 1))}.
(36)
Let us show that this δ satisfies the statement above. Let ∆δ be a partition (3) and the control process realization {x(·), u(·), v(·)} be generated by the control law {U ◦ , ∆δ } and v(·) ∈ V(t). Define the function
Lemma 4. For every (t, z, w(·)) ∈ G, t < ϑ and s ∈ Rn , there exist τ∗ ∈ (t, ϑ] and x∗ (·) ∈ X 0 (t, z, w(·)) such that, for every τ ∈ [t, τ∗ ], the following inequality holds: ρ◦ (τ, x∗ (τ ), x∗τ (·)) τ + H(ξ, x∗ (ξ), x∗ (ξ − h), s) − x˙ ∗ (ξ), s dξ (40) t
Proof. Since w(·) ∈ PC, then there exists τ∗ ∈ (t, ϑ] such that w(·) is continuous on [−h, τ∗ −t−h]. Let k ∈ N. Denote τj = t+(τ∗ −t)j/k, j ∈ 0, k. According to (14) and (ρu ), let us define control process realizations {xk (·), uk (·), v k (·)} such that, for every j ∈ 0, k − 1, we have (41) v k (τ ) = vjk , τ ∈ [τj , τj+1 ), where vjk ∈ V is defined by
H(τj , xk (τj ), xk (τj − h), s) = min f (τj , xk (τj ), xk (τj − h), u, vjk ), s u∈U +f 0 (τj , xk (τj ), xk (τj − h), u, vjk ) ,
ω(τ ) = ϕ(τ, x(τ ), xτ (·)) +
τ t
f 0 (ξ, x(ξ), x(ξ − h), u(ξ), v(ξ)) dξ, τ ∈ [t, ϑ].
(37)
Then, taking into account terminal condition (16), for proving (30), it is sufficient to show that ω(ϑ) ≤ ω(t) + ε.
(43)
≤ ρ◦ (τj , xk (τj ), xkτj (·)) + (τj+1 − τj )/k.
(38)
j∈J
According to Lemma 3, the function ω(·) is Lipschitz continuous and, taking into account system (1) and (28), for almost every τ ∈ [t, ϑ], satisfies the equation ci ω(τ ˙ ) = ∂t,w ϕ(τ, x(τ ), xτ (·)) + χ(τ, x(τ ), xτ (·), u(τ ), v(τ )).
For j ∈ J and τ ∈ [τj , τj+1 ), in accordance with (31), (33)–(36), we derive ≤
and ρ◦ (τj+1 , xk (τj+1 ), xkτj+1 (·)) τ j+1 + f 0 (ξ, xk (ξ), xk (ξ − h), uk (ξ), v k (ξ)) dξ
(42)
τj
Denote by J the set of j ∈ 1, l such that there exists i ∈ 1, k + 1 such that [τj − h, τj+1 − h) ⊂ Ii . Note that, |J| ≥ l − k − 1. Then, from (34)–(37), we derive (ω(τj+1 ) − ω(τj )) + ε/2. (39) ω(ϑ) − ω(t) ≤
ci ϕ(τ, x(τ ), xτ (·)) ∂t,w
≤ ρ◦ (t, z, w(·)).
According to (12) and Proposition 2, without loss of generality, we can suppose that there exists x∗ (·) ∈ X 0 (t, z, w(·)) such that max x∗ (τ ) − xk (τ ) → 0 as k → ∞. (44) τ ∈[t,ϑ]
Let us fix ε > 0. Due to (f1 ) and Lemma 1, taking into account the continuity of w(·) on [−h, τ∗ − t − h], Proposition 1 and (17), there exists k1 > 0 such that, for every k ≥ k1 , x(·) ∈ X 0 (t, z, w(·)), u ∈ U, v ∈ V, j ∈ 0, k − 1 and τ ∈ [τj , τj+1 ], we have |f (τj , x(τj ), x(τj − h), u, v), s
−f (τ, x(τ ), x(τ − h), u, v), s| ≤ ε/(16(τ∗ − t)),
0
|f (τj , x(τj ), x(τj − h), u, v)
+ χ(τ, x(τ ), xτ (·), u(τ ), v(τ ))
ci ∂t,w ϕ(τj , x(τj ), xτj (·))
0
−f (τ, x(τ ), x(τ − h), u, v)| ≤ ε/(16(τ∗ − t)),
◦
|ρ (τj , x(τj ), xτj (·)) − ρ◦ (τ, x(τ ), xτ (·))| ≤ ε/6.
+χ(τj , x(τj ), xτj (·), u(τ ), v(τ )) + 4ε∗ . 264
(45)
2019 IFAC TDS Sinaia, Romania, September 9-11, 2019
Anton Plaksin et al. / IFAC PapersOnLine 52-18 (2019) 138–143
Then, in accordance with (14), we derive |H(τj , x(τj ), x(τj − h), s)
−H(τ, x(τ ), x(τ − h), s)| ≤ ε/(8(τ∗ − t)).
(46)
Moreover, according to (f1 ) and Proposition 1, there exists βf > 0 such that 0
|f (τ, x(τ ), x(τ − h), u, v)| ≤ βf ,
τ ∈ [t, τ∗ ], x(·) ∈ X 0 (t, z, w(·)), u ∈ U, v ∈ V.
(47)
Due to (17), (f1 ), (14) and (44), there exists k2 > 0 such that, for every k ≥ k2 , we have +
τ t
τ t
ρ◦ (τ, x∗ (τ ), x∗τ (·))
H(ξ, x∗ (ξ), x∗ (ξ − h), s) − x˙ ∗ (ξ), s dξ ≤ ρ◦ (τ, xk (τ ), xkτ (·))
(48)
H(ξ, xk (ξ), xk (ξ − h), s) − x˙ k (ξ), s dξ + ε/6.
k
k
k
H(ξ, x (ξ), x (ξ − h), s) − x˙ (ξ), s ≤ H(τj , xk (τj ), xk (τj − h), s)
k
k
k
−f (τj , x (τj ), x (τj − h), u 0
k
k
(ξ), vjk ), s+ k
≤ f (τj , x (τj ), x (τj − h), u
(ξ), vjk )
3ε/(16(τ∗ − t))
+ 3ε/(16(τ∗ − t))
Then, taking into account (47) and defining k3 = 4βf (τ∗ − t)/ε, for k ≥ k3 , j ∈ 0, k − 1 and τ ∈ [τj , τj+1 ), we have τ H(ξ, xk (ξ), xk (ξ − h), s) − x˙ k (ξ), s dξ τ
≤ ≤
t τj t
f 0 (ξ, xk (ξ), xk (ξ − h), uk (ξ), vjk ) dξ + ε/4
(49)
f 0 (ξ, xk (ξ), xk (ξ − h), uk (ξ), vjk ) dξ + ε/2.
Thus, from (41), (43), (45), (48), (49), for k ≥ max{6(τ∗ − t)/ε, k1 , k2 , k3 }, we obtain ρ◦ (τ, x∗ (τ ), x∗τ (·)) τ H(ξ, x∗ (ξ), x∗ (ξ − h), s) − x˙ ∗ (ξ), s dξ + t
≤ ρ◦ (t, z, w(·)) + ε.
It holds for any ε > 0, therefore we can put ε = 0. Lemma 5. For every (t, z, w(·)) ∈ G, t < ϑ and s ∈ Rn , there exist τ∗ ∈ (t, ϑ] and x∗ (·) ∈ X 0 (t, z, w(·)) such that, for every τ ∈ [t, τ∗ ], the following inequality holds: ρ◦ (τ, x∗ (τ ), x∗τ (·)) τ + H(ξ, x∗ (ξ), x∗ (ξ − h), s) − x˙ ∗ (ξ), s dξ t
Proof. Due to Lemma 4, defining s = ∇z ρ◦ (t, z, w(·)), let us take τ∗ ∈ (t, ϑ] and x∗ (·) ∈ X 0 (t, z, w(·)). By the definition of ci-differentiability of ρ◦ , we have ci ◦ ρ (t, z, w(·))(τ − t) ρ◦ (τ, x∗ (τ ), x∗τ (·))−ρ◦ (t, z, w(·)) = ∂t,w
+x∗ (τ ) − z, ∇z ρ◦ (t, z, w(·)) + o(τ − t),
Then, using (40), for τ ∈ [t, τ∗ ], we derive
+
τ
τ ∈ [t, ϑ].
ci ◦ ∂t,w ρ (t, z, w(·))(τ − t) + o(τ − t)
H(ξ, x∗ (ξ), x∗ (ξ − h), ∇z ρ◦ (t, z, w(·))) dξ ≤ 0.
Dividing this inequality by τ − t, passing to the limit as τ → t + 0, taking into account (f1 ) and (14), we get ci ◦ ∂t,w ρ (t, z, w(·)) + H(t, z, w(−h), ∇z ρ◦ (t, z, w(·))) ≤ 0.
In a similar way, using Lemma 5, we can obtain the opposite inequality. REFERENCES
≤ f 0 (ξ, xk (ξ), xk (ξ − h), uk (ξ), vjk ) + ε/(4(τ∗ − t)).
t
Proof. The Lemma can be proved similar to Lemma 4, using (ρv ) and (f4 ). Theorem 2. Let the value functional ρ◦ of differential game (1), (2) be ci-differentiable at a point (t, z, w(·)) ∈ G, t < ϑ. Then it satisfies HJ equation (15) at this point.
t
From (1), (42), (45), for every j ∈ 0, k − 1 and almost every ξ ∈ [τj , τj+1 ), we derive
143
≥ ρ◦ (t, z, w(·)). 265
Clarke, F.H., Ledyaev, Yu.S., Stern, R.J., and Wolenski, P.R. (1998). Nonsmooth analysis and control theory. Springer, New York. Crandall, M.G. and Lions, P.-L. (1983). Viscosity solutions of Hamilton-Jacobi equations. Trans. Amer. Math. Soc., 277 (1), 1–42. Gomoyunov, M. and Plaksin, A. (2018). On HamiltonJacobi equations for neutral-type differential games. IFAC PapersOnLine, 50 (1), 5109–5114. Isaacs., R. (1965). Differential games. John Wiley, New York. Kim, A.V. (1999). Functional differential equations. Application of i-smooth calculus. Kluwer, The Netherlands Dordrecht. Krasovskii, N.N. and Subbotin, A.I. (1988). Gametheoretical control problems. Springer, New York. Lukoyanov, N.Yu. (2000). Functional equations of the Hamilton-Jacobi type and differential games with hereditary information. Doklady Mathematics, 61 (2), 301– 304. Lukoyanov, N.Yu. (2003). Functional Hamilton-Jacobi type equations with ci-derivatives in control problems with hereditary information. Nonlinear Funct. Anal. and Appl., 8 (4), 535–556. Natanson, I.P. (1960). Theory of functions of a real variable. Volume 2. Frederick Ungar Publishing Co., New-York. Osipov, Yu.S. (1971). Differential games of systems with aftereffect. Dokl. Akad. Nauk SSSR, 196 (4), 779–782. Plaksin, A. (2019). Minimax and viscosity solutions of Hamilton-Jacobi-Bellman equations for time-delay systems. ArXiv:1901.04677 (submitted to Journal of Optimization Theory and Applications). Subbotin, A.I. (1995). Generalized solutions of first order PDEs: the dynamical optimization perspective. Birkh¨auser, Berlin.