- Email: [email protected]
Dynamic programming for systems with hysteresis
Physica B 306 (2001) 200–205
Dynamic programming for systems with hysteresis S.A. Belbasa,*, I.D. Mayergoyzb b
a Mathematics Department, University of Alabama, Tuscaloosa, AL 35487-0350, USA Electrical and Computer Engineering Department, University of Maryland, College Park, MD 20742, USA
Abstract We introduce a novel type of dynamic programming equations for controlled differential equations with hysteresis on the control. The novelty of our approach is that we use Hamilton–Jacobi functions with one of the variables being setvalued, corresponding to the active set on the Preisach plane. The derivation of these new dynamic programming equations relies on a suitable notion of derivatives of set-valued functions, which we introduce in this paper. The results have applications to systems with sensors and actuators (e.g. piezoceramic materials) that exhibit hysteresis. r 2001 Elsevier Science B.V. All rights reserved. Keywords: Preisach hysteresis; Optimal control; Derivatives of set-valued functions; Dynamic programming; Hamilton–Jacobi function
1. Introduction In this paper, we introduce a novel type of dynamic programming equations in which part of the state is a set-valued function and the Hamilton–Jacobi function is a (generally non-additive) real-valued set-function; the relevant set-valued state is the active set, on the Preisach plane, for a hysteresis operator. Many modern control and estimation applications, including magnetostrictive actuators, involve hysteresis phenomena. It is, therefore, of practical interest to obtain optimality conditions, based on the principle of dynamic programming, for controlled systems with hysteresis nonlinearities. The mathematical model of Preisach hysteresis has been delineated in Refs. [1–3]. Among other
results, it is known that the Preisach model of hysteresis can be defined by a set of rules for the evolution of the active sets on the Preisach plane; if the relevant part of the Preisach plane is denoted by Pða0 ; b0 Þ :¼ fða; bÞ : b0 oboaoa0 g; and if, at some time t; the active set is WðtÞ; i.e. the hysterons with indices (a; b) are in the ‘‘on’’ position, then the hysteresis output at time t is given by ZZ HuðtÞ ¼ wða; bÞ da db Wþ
ZZ wða; bÞ da db W
¼2
ZZ
wða; bÞ da db Wþ
*Corresponding author. Fax: +1-301-530-6582. E-mail address: [email protected] (S.A. Belbas). 0921-4526/01/$ - see front matter r 2001 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 1 - 4 5 2 6 ( 0 1 ) 0 1 0 0 4 - 3
ZZ wða; bÞ da db;
Pða0 ;b0 Þ
ð1:1Þ
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
where Wþ ¼ WðtÞ; W ¼ Pða0 ; b0 ÞWWðtÞ: In this paper, we shall assume that the function wða; bÞ is continuous in the closure of Pða0 ; b0 Þ: Our goal is to define and calculate the time-derivatives of the set-valued function WðtÞ: The evolution in time of a hysteretic system will then be written as a system of generalized differential equations, with one differential equation (which includes a hysteresis operator) for the ordinary state of the system, and a second generalized differential equation for the evolution of the active sets. With such a description of the dynamics of a controlled system with hysteresis, we are then in a position to apply the methods of dynamic programming, where the Hamilton–Jacobi function is parametrized by a generalized state variable which includes the active sets, i.e. the Hamilton–Jacobi function is a realvalued set-function (for each value of the other, ordinary, independent variables). It is well known that standard Hamilton–Jacobi equations do not, in general, possess continuously differentiable solutions, and that an appropriate form of generalized solutions is necessary. Thus, we are faced with the following 4 questions: (1) What is the appropriate concept of derivatives of set-valued functions of a real variable, and what generalized differential equations describe the evolution of the active sets on the Preisach plane? (2) What is the appropriate concept of derivatives of real-valued set-functions (functions with an algebra of sets as their domain of definition), for the evolution of the cost-to-go in a controlled hysteretic system? (3) What are the dynamic programming equations, assuming sufficient smoothness of the Hamilton–Jacobi function? (4) What is the appropriate concept of generalized solutions of the dynamic programming equation when the Hamilton–Jacobi function is a real-valued set-function? In this paper, we provide answers to these questions. We note that other approaches to the dynamic programming for systems with Preisach hysteresis have been obtained in Ref. [4] and, for a finite
201
number of hysterons, in Ref. [5]. The general study of optimal control problems with hysteresis originated in the work of Brokate [6].
2. Statement of the optimal control problem We consider a controlled system with hysteresis dyðtÞ ¼ f ðt; yðtÞ; uðtÞ; HuðtÞÞ; tAð0; T; dt yð0Þ ¼ y0 ;
ð2:1Þ
where H is a Preisach hysteresis operator acting on the control function u: The function f is continuous and bounded in all its arguments, and Lipschitz in y; uniformly with respect to the other arguments. The class of admissible controls consists of all real-valued piecewise monotone functions on [0; T] which are continuously differentiable with derivatives bounded between two fixed real numbers v1 ; v2 : In this paper we are concerned with the dynamic programming equations for the Hamilton–Jacobi function, i.e. the infimum of the cost-to-go, not with the question of synthesis of an optimal control; there is no guarantee that the infimum will be achieved within the considered class of admissible controls. Other classes of admissible controls, e.g. LN ð½0; T/IRÞ can be treated by methods similar to the methods of this paper; also, relaxed control problems (in the sense of Gamkrelidze) can be treated by similar methods. The optimal control problem is the minimization of a functional Z T J :¼ Fðt; yðtÞ; uðtÞ; HuðtÞÞ dt þ F0 ðyðTÞÞ: ð2:2Þ 0
For continuous controls uðtÞ; the value of the control can be calculated if the active set at time t is known; indeed, if gðtÞ is the relative boundary of WðtÞ; i.e. gðtÞ :¼ qWðtÞWqPða0 ; b0 Þ; then ðuðtÞ; uðtÞÞ is the point of intersection of gðtÞ with the line a ¼ b on the Preisach plane. Consequently, we may write the dynamics Eq. (2.1) as dyðtÞ ¼ gðt; yðtÞ; WðtÞÞ; dt ð2:3Þ yð0Þ ¼ y0 ; Wð0Þ ¼ W0 :
202
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
0
The dependence of the control uðtÞ on the active set will be denoted by uðtÞ ¼ UðWðtÞÞ:
ð2:5Þ
We note that the dependence of uðtÞ on WðtÞ is of a very specific nature, and it will be further examined in Section 5 below.
3. A concept of derivatives of set-valued functions Various concepts of derivatives of set-valued functions have been investigated by several authors; representative examples are Refs. [7,8]. We note that the definitions given up to now have been motivated mostly by the study of multi-valued differential equations. By contrast, in this paper we give a novel definition, which is suitable for set-functions that will be used as domains of integration (for the definition of Preisach hysteresis operators). We formulate our definitions for sets in a Euclidean space. First, we need some terminology and notation. For any two sets A; B; we denote by A#B the ordered difference of the two sets, i.e. the ordered pair (AWB; BWA), where the simple difference is A=B :¼ fx : xAA and xeBg: We remark that the ordered difference is a particular case of the concept of a chain (as-used in differential geometry, e.g. Ref. [9]): the ordered difference is isomorphic to the chain ðþ1Þ½AWB þ ð1Þ½BWA; in particular, the ordered difference is anti-commutative. Our concept of differentiation is based on the so-called Peano definition of derivatives for real-valued functions. Definition. If WðtÞ is a set-valued function of the real variable t; with values in the algebra of Borel sets of IRn ; then we say that W is m-times differentiable at t ¼ t0 if there exist points aci;þ ; c aci; in IRn ; connected Borel sets Ui;7 and n n
matrix-valued functions Aci;7 ðeÞ; 1pipm; such that ! [ Wðt0 þ eÞWWðt0 ÞðWÞ 1pipm
[
!
! i ðaci;þ
þ
ci i Aci;þ ðeÞUi;þ Þ
¼ Rm;þ ðeÞ;
ci
Wðt0 ÞWWðt0 þ eÞðWÞ
!
[ 1pipm
[
!
! i ðaci;
þ
ci i Aci; ðeÞUi; Þ
¼ Rm; ðeÞ;
ð3:1Þ
ci
where the union and set-difference symbols in parentheses denote almost disjoint union (two sets are said to be almost disjoint if their intersection has n-dimensional Lebesgue measure zero) and set-difference modulo sets of n-dimensional Lebesgue measure zero, the matrices Aci;7 ðeÞ satisfy detðAci;7 ðeÞÞ ¼
ei i!
and the remainders satisfy jRm;7 ðeÞj ¼ oðem Þ (where |.| stands for n-dimensional Lebesgue measure). c Simple examples show that the sets Ui;7 are not necessarily unique; consequently, we define the ith derivative as an equivalence class of triples ða i;7 ; A i;7 ð Þ; Ui;7 Þ: We shall denote the ith derivative of WðtÞ at t ¼ t0 by qi Wðt0 Þ; and the collection of all derivatives up to order m by Dm Wðt0 Þ:
4. The evolution of active sets on the Preisach plane We consider sets WðtÞ of the following type: WðtÞ consists of all points below a non-increasing curve g that connects two points on the boundary of Pða0 ; b0 Þ; i.e. WðtÞ ¼ fða; bÞ : (ða1 ; b1 ÞAg a A
Similarly, the cost functional of Eq. (2.2) can be written as Z T Gðt; yðtÞ; WðtÞÞ dt þ F0 ðyðTÞÞ: ð2:4Þ J¼
pa1 4bpb1 4ða; bÞaða1 ; b1 Þg:
ð4:1Þ
203
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
We consider the following forms of g: (1) A non-increasing continuous curve g from one point ðu1 ; u1 Þof qPðao ; b0 Þ to some other point of qPðao ; b0 Þ and such that, in a sufficiently small neighborhood of ðu1 ; u1 Þ; g can be represented both as b ¼ jðaÞ; b0 pa1 oaoa2 oa0 and a ¼ cðbÞ; b0 pb1 obpb0 ; where j and c are strictly decreasing continuously differentiable functions. (2) A continuous curve from a point ðu1 ; u1 Þ consisting of a straight-line segment parallel to the a-axis, for u1 oapuþ 1 ; and continuous non-increasing part from ðuþ 1 ; u1 Þ to some point of qPðao ; b0 Þ þ such that, for uþ 1 oaou1 þ e; g can be represented as a ¼ cðbÞ with c non-increasing and continuously differentiable. (3) A continuous g from some point ðu1 ; u1 Þ; consisting of a straight-line segment parallel to the b-axis, for u 1 obou1 ; and a continuous non-increasing part from ðu1 ; u 1 Þ such that, for u 1 eobou1 ; g can be represented as b ¼ jðaÞ with j non-increasing and continuously differentiable. We note that the standard ‘‘staircase’’ curves that appear in the classical treatment of hysteresis [1–3] are particular cases of the 3 cases above; from the point of view of the differential calculus of setvalued functions, the staircase curves play the role of quadratic functions in ordinary differential calculus, in the sense that their derivatives of order higher than 2 vanish. In each of the 3 cases above, we can explicitly calculate the derivatives of the set-valued function WðtÞ at t ¼ t0 ; provided the input uðtÞ is differentiable at t ¼ t0 : We set v ¼ uðt ’ 0 Þ: In case (1), when v > 0; we have Wðt0 þ dtÞWWðt0 ÞðWÞ " # " # ! u1 1 dt 0
U2;þ ¼ R2;þ ðdtÞ; þ 2 0 dt u1 where U2;þ is the triangular domain with vertices ð0; 0Þ; ð2v; 2vÞ; ð2v; 2j0 ðu1 ÞvÞ; if we set A :¼ ðu1 ; u1 Þ; B :¼ ðu1 þ vdt; u1 þ vdtÞ; C :¼ ðu1 ; jðu1 þ vdtÞÞ; D :¼ ðu1 ; u1 þ j0 ðu1 ÞvdtÞ;
then the domain enclosed by the curvilinear triangle ABD is Wðt0 þ dtÞWWðt0 Þ; whereas the domain ABC is " # " # u1 1 dt 0 U2;þ : þ 2 0 dt u1 The remainder R2;þ ðdtÞ is the domain ACD, whose area is Z u1 þvdt jR2;þ ðdtÞj ¼ ½jðu1 Þ þ j0 ðu1 Þða u1 Þ u1
jðaÞda ¼ oðdt2 Þ: Still in case (1), if vo0; we have Wðt0 ÞWWðt0 þ dtÞðWÞ " # " # ! u1 1 dt 0
U2; ¼ R2; ðdtÞ; þ 2 0 dt u1 where U2; is the triangular domain with vertices ð0; 0Þ; ð2v; 2vÞ; ð2c0 ðu1 Þv; 2vÞ; and the remainder R2; ðdtÞ satisfies jR2; ðdtÞj ¼ oðdt2 Þ: In case (2), if v > 0; we have Wðt0 þ dtÞWWðt0 ÞðWÞ " # " # ! u1 1 dt 0 U2;þ ¼ R2;þ ðdtÞ; þ
2 0 dt u1 where U2;þ is the triangular domain with vertices ð0; 0Þ; ð2v; 2vÞ; ð2v; 0Þ: When vo0; we have ( " # " # ! 0 1 0 Wðt0 ÞWWðt0 þ dtÞðWÞ þ U1; u1 0 dt "
ð,Þð,Þ "
ð,Þ
uþ 1 u1
u1 u1
"
# þ
#
" þ
1 2
1 2
dt 0
dt 0
# ! 0 1 U2; dt
!) # 0 2 ¼ R2; ðdtÞ; U2; dt
where U1; is the rectangular domain with vertices þ 1 ðu1 ; 0Þ; ðu1 ; vÞ; ðuþ 1 ; 0Þ; ðu1 ; vÞ; U2; is the triangular domain with vertices ð0; 0Þ; ð0; 2vÞ; ð2v; 2vÞ; and 2 U2; is the triangular domain with vertices ð0; 0Þ; ð0; 2vÞ; ð2c0 ðuþ 1 Þv; 2vÞ:
204
"
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
In case (3) with v > 0; the first derivative is # " # u1 dt 0 ; ; U1;þ ; 0 0 1
As it is standard in utilizing the principle of dynamic programming, we consider a parametrized problem
where U1;þ is the rectangular domain with vertices ð0; u1 Þ; ðv; u1 Þ; ð0; u 1 Þ; ðv; u1 Þ; and the second derivative has connected components " # " # u1 dt 0 1 ; ; U2;þ 0 dt u1
dyðt; s; x; AÞ ¼ gðt; yðt; s; x; AÞ; Wðt; s; x; AÞÞ; t > s; dt D2 Wðt; s; x; AÞ ¼ FðWðt; s; x; AÞ; vðtÞÞ; t > s; yðs; s; x; AÞ ¼ x; Wðt; s; x; AÞ ¼ A; Z Jðs; x; AÞ :¼
and " # " # u1 dt 0 2 ; ; ; U2;þ 0 dt u 1
Gðt; yðt; s; x; AÞ; s
Wðt; s; x; AÞÞdt þ F0 ðyðT; s; x; AÞÞ: ð5:2Þ
1 where U2;þ is the triangular domain with vertices 2 ð0; 0Þ; ð2v; 0Þ; ð2v; 2vÞ and U2;þ is the triangular domain with vertices ð0; 0Þ; ð2v; 0Þ; ð2v; 2j0 ðu1 ÞvÞ: Still in case (3), if vo0; then U2; is the triangular domain with vertices ð0; 0Þ; ð0; 2vÞ; ð2v; 2vÞ:
5. The dynamic programming equations We re-formulate the problem of Section 2 as follows: the state consists of " # yðtÞ ; WðtÞ where WðtÞ takes values in the class of sets defined in Section 4. The accessory control function vðtÞ is the derivative of the original control function, and the class of admissible accessory control functions is Cð½0; T/½v1 ; v2 : The controlled dynamical system is written as dyðtÞ ¼ gðt; yðtÞ; WðtÞÞ; D2 WðtÞ ¼ FðWðtÞ; vðtÞÞ; dt yð0Þ ¼ y0 ; Wð0Þ ¼ W0
T
ð5:1Þ
and the cost functional is given by Eq. (2.4). We note that, in g and G; the set WðtÞ enters it two ways: as the domain of integration for calculating HuðtÞ and through the control function uðtÞ which, in turn, depends implicitly on WðtÞ and explicitly on some of the components of D2 WðtÞ:
Of course, y; W; and J also depend on the function vð Þ: We define the Hamilton–Jacobi function Vðs; x; AÞ by Vðs; x; AÞ :¼
inf
vACð½0;T/ðv1 ;v2 Þ
Jðs; x; AÞ:
ð5:3Þ
We shall say that a real-valued set-function C is differentiable at A if, for any family of sets SðeÞ; e0 pepe0 ; e0 > 0 with Sð0Þ ¼ A; and S twice differentiable at e ¼ 0; there exists a function DCðA; D2 Sð0ÞÞ such that CðSðeÞÞ CðAÞ ¼ DCðA; D2 Sð0ÞÞe þ oðeÞ:
ð5:4Þ
With this definition of differentiability, if the value function V is differentiable with respect to all its arguments, we obtain the dynamic programming equation ( qVðs; x; AÞ qVðs; x; AÞ þ inf gðs; x; AÞ aA½v1 ;v2 @s @x ) þ DA Vðs; x; A; D2a AÞ þ Gðs; x; AÞ ¼ 0; VðT; x; AÞ ¼ F0 ðxÞ:
ð5:5Þ
Here, the symbol D2a A denotes the derivative D2 WðsÞjWðsÞ¼A evaluated with the value vðsÞ ¼ a: The concept of generalized solutions is this: a function Vðs; x; AÞ is said to be a generalized solution of (5.4) if it satisfies the final conditions and, for every continuously differentiable function Cðs; x; AÞ; if V C has a minimum at ðs0 ; x0 ; A0 Þ;
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
then C is a subsolution of Eq. (5.4) at ðs0 ; x0 ; A0 Þ; i.e. ( qCðs; x; AÞ qCðs; x; AÞ þ inf gðs; x; AÞ aA½v ;v qs qx 1 2 þ DA Cðs; x; A; D2a AÞ ) þ Gðs; x; AÞ s¼s ;x¼x 0
0 ;A¼A0
p0
ð5:6Þ
and if V C has a maximum at ðs0 ; x0 ; A0 Þ then C is a supersolution of (5.4) at ðs0 ; x0 ; A0 Þ; i.e. ( qCðs; x; AÞ qCðs; x; AÞ þ inf gðs; x; AÞ: aA½v1 ;v2 qs qx þ DA Cðs; x; A; D2a AÞ ) þ Gðs; x; AÞ s¼s ;x¼x 0
0 ;A¼A0
X0:
ð5:7Þ
Acknowledgements The reported research has been supported in part by the US Department of Energy, Engineering Research Program.
205
References [1] I.D. Mayergoyz, Mathematical Models of Hysteresis, Springer, New York, 1991. [2] A. Visintin, Differential Models of Hysteresis, Springer, New York, 1994. [3] M. Brokate, J. Sprekels, Hysteresis and Phase Transitions, Springer, New York, 1996. [4] S.A. Belbas, I.D. Mayergoyz, Dynamic programming for systems with Preisach hysteresis, preprint, 2000. [5] F. Bagagiolo, Dynamic programming for some optimal control problems with hysteresis, Preprint No. 38, Max Planck Institut f. Math. in der Naturwiss., Leipzig, 2000. . [6] M. Brokate, Optimale Steuerung von gewohnlichen Differentialgleichungen mit Nichtlinearit.aten vom HysteresisTyp, Verlag Peter Lang, Frankfurt a. M., 1987. [7] H.T. Banks, M.Q. Jacobs, J. Math. Anal. Appl. 29 (1970) 246. . [8] G. Schroder, Differentiation of interval functions, Proc. Amer. Math. Soc. 36 (1972) 485. [9] M. Spivak, Differential geometry, Vol. 1, Publish or Perish, Berkeley, 1979.
Dynamic programming for systems with hysteresis S.A. Belbasa,*, I.D. Mayergoyzb b
a Mathematics Department, University of Alabama, Tuscaloosa, AL 35487-0350, USA Electrical and Computer Engineering Department, University of Maryland, College Park, MD 20742, USA
Abstract We introduce a novel type of dynamic programming equations for controlled differential equations with hysteresis on the control. The novelty of our approach is that we use Hamilton–Jacobi functions with one of the variables being setvalued, corresponding to the active set on the Preisach plane. The derivation of these new dynamic programming equations relies on a suitable notion of derivatives of set-valued functions, which we introduce in this paper. The results have applications to systems with sensors and actuators (e.g. piezoceramic materials) that exhibit hysteresis. r 2001 Elsevier Science B.V. All rights reserved. Keywords: Preisach hysteresis; Optimal control; Derivatives of set-valued functions; Dynamic programming; Hamilton–Jacobi function
1. Introduction In this paper, we introduce a novel type of dynamic programming equations in which part of the state is a set-valued function and the Hamilton–Jacobi function is a (generally non-additive) real-valued set-function; the relevant set-valued state is the active set, on the Preisach plane, for a hysteresis operator. Many modern control and estimation applications, including magnetostrictive actuators, involve hysteresis phenomena. It is, therefore, of practical interest to obtain optimality conditions, based on the principle of dynamic programming, for controlled systems with hysteresis nonlinearities. The mathematical model of Preisach hysteresis has been delineated in Refs. [1–3]. Among other
results, it is known that the Preisach model of hysteresis can be defined by a set of rules for the evolution of the active sets on the Preisach plane; if the relevant part of the Preisach plane is denoted by Pða0 ; b0 Þ :¼ fða; bÞ : b0 oboaoa0 g; and if, at some time t; the active set is WðtÞ; i.e. the hysterons with indices (a; b) are in the ‘‘on’’ position, then the hysteresis output at time t is given by ZZ HuðtÞ ¼ wða; bÞ da db Wþ
ZZ wða; bÞ da db W
¼2
ZZ
wða; bÞ da db Wþ
*Corresponding author. Fax: +1-301-530-6582. E-mail address: [email protected] (S.A. Belbas). 0921-4526/01/$ - see front matter r 2001 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 1 - 4 5 2 6 ( 0 1 ) 0 1 0 0 4 - 3
ZZ wða; bÞ da db;
Pða0 ;b0 Þ
ð1:1Þ
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
where Wþ ¼ WðtÞ; W ¼ Pða0 ; b0 ÞWWðtÞ: In this paper, we shall assume that the function wða; bÞ is continuous in the closure of Pða0 ; b0 Þ: Our goal is to define and calculate the time-derivatives of the set-valued function WðtÞ: The evolution in time of a hysteretic system will then be written as a system of generalized differential equations, with one differential equation (which includes a hysteresis operator) for the ordinary state of the system, and a second generalized differential equation for the evolution of the active sets. With such a description of the dynamics of a controlled system with hysteresis, we are then in a position to apply the methods of dynamic programming, where the Hamilton–Jacobi function is parametrized by a generalized state variable which includes the active sets, i.e. the Hamilton–Jacobi function is a realvalued set-function (for each value of the other, ordinary, independent variables). It is well known that standard Hamilton–Jacobi equations do not, in general, possess continuously differentiable solutions, and that an appropriate form of generalized solutions is necessary. Thus, we are faced with the following 4 questions: (1) What is the appropriate concept of derivatives of set-valued functions of a real variable, and what generalized differential equations describe the evolution of the active sets on the Preisach plane? (2) What is the appropriate concept of derivatives of real-valued set-functions (functions with an algebra of sets as their domain of definition), for the evolution of the cost-to-go in a controlled hysteretic system? (3) What are the dynamic programming equations, assuming sufficient smoothness of the Hamilton–Jacobi function? (4) What is the appropriate concept of generalized solutions of the dynamic programming equation when the Hamilton–Jacobi function is a real-valued set-function? In this paper, we provide answers to these questions. We note that other approaches to the dynamic programming for systems with Preisach hysteresis have been obtained in Ref. [4] and, for a finite
201
number of hysterons, in Ref. [5]. The general study of optimal control problems with hysteresis originated in the work of Brokate [6].
2. Statement of the optimal control problem We consider a controlled system with hysteresis dyðtÞ ¼ f ðt; yðtÞ; uðtÞ; HuðtÞÞ; tAð0; T; dt yð0Þ ¼ y0 ;
ð2:1Þ
where H is a Preisach hysteresis operator acting on the control function u: The function f is continuous and bounded in all its arguments, and Lipschitz in y; uniformly with respect to the other arguments. The class of admissible controls consists of all real-valued piecewise monotone functions on [0; T] which are continuously differentiable with derivatives bounded between two fixed real numbers v1 ; v2 : In this paper we are concerned with the dynamic programming equations for the Hamilton–Jacobi function, i.e. the infimum of the cost-to-go, not with the question of synthesis of an optimal control; there is no guarantee that the infimum will be achieved within the considered class of admissible controls. Other classes of admissible controls, e.g. LN ð½0; T/IRÞ can be treated by methods similar to the methods of this paper; also, relaxed control problems (in the sense of Gamkrelidze) can be treated by similar methods. The optimal control problem is the minimization of a functional Z T J :¼ Fðt; yðtÞ; uðtÞ; HuðtÞÞ dt þ F0 ðyðTÞÞ: ð2:2Þ 0
For continuous controls uðtÞ; the value of the control can be calculated if the active set at time t is known; indeed, if gðtÞ is the relative boundary of WðtÞ; i.e. gðtÞ :¼ qWðtÞWqPða0 ; b0 Þ; then ðuðtÞ; uðtÞÞ is the point of intersection of gðtÞ with the line a ¼ b on the Preisach plane. Consequently, we may write the dynamics Eq. (2.1) as dyðtÞ ¼ gðt; yðtÞ; WðtÞÞ; dt ð2:3Þ yð0Þ ¼ y0 ; Wð0Þ ¼ W0 :
202
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
0
The dependence of the control uðtÞ on the active set will be denoted by uðtÞ ¼ UðWðtÞÞ:
ð2:5Þ
We note that the dependence of uðtÞ on WðtÞ is of a very specific nature, and it will be further examined in Section 5 below.
3. A concept of derivatives of set-valued functions Various concepts of derivatives of set-valued functions have been investigated by several authors; representative examples are Refs. [7,8]. We note that the definitions given up to now have been motivated mostly by the study of multi-valued differential equations. By contrast, in this paper we give a novel definition, which is suitable for set-functions that will be used as domains of integration (for the definition of Preisach hysteresis operators). We formulate our definitions for sets in a Euclidean space. First, we need some terminology and notation. For any two sets A; B; we denote by A#B the ordered difference of the two sets, i.e. the ordered pair (AWB; BWA), where the simple difference is A=B :¼ fx : xAA and xeBg: We remark that the ordered difference is a particular case of the concept of a chain (as-used in differential geometry, e.g. Ref. [9]): the ordered difference is isomorphic to the chain ðþ1Þ½AWB þ ð1Þ½BWA; in particular, the ordered difference is anti-commutative. Our concept of differentiation is based on the so-called Peano definition of derivatives for real-valued functions. Definition. If WðtÞ is a set-valued function of the real variable t; with values in the algebra of Borel sets of IRn ; then we say that W is m-times differentiable at t ¼ t0 if there exist points aci;þ ; c aci; in IRn ; connected Borel sets Ui;7 and n n
matrix-valued functions Aci;7 ðeÞ; 1pipm; such that ! [ Wðt0 þ eÞWWðt0 ÞðWÞ 1pipm
[
!
! i ðaci;þ
þ
ci i Aci;þ ðeÞUi;þ Þ
¼ Rm;þ ðeÞ;
ci
Wðt0 ÞWWðt0 þ eÞðWÞ
!
[ 1pipm
[
!
! i ðaci;
þ
ci i Aci; ðeÞUi; Þ
¼ Rm; ðeÞ;
ð3:1Þ
ci
where the union and set-difference symbols in parentheses denote almost disjoint union (two sets are said to be almost disjoint if their intersection has n-dimensional Lebesgue measure zero) and set-difference modulo sets of n-dimensional Lebesgue measure zero, the matrices Aci;7 ðeÞ satisfy detðAci;7 ðeÞÞ ¼
ei i!
and the remainders satisfy jRm;7 ðeÞj ¼ oðem Þ (where |.| stands for n-dimensional Lebesgue measure). c Simple examples show that the sets Ui;7 are not necessarily unique; consequently, we define the ith derivative as an equivalence class of triples ða i;7 ; A i;7 ð Þ; Ui;7 Þ: We shall denote the ith derivative of WðtÞ at t ¼ t0 by qi Wðt0 Þ; and the collection of all derivatives up to order m by Dm Wðt0 Þ:
4. The evolution of active sets on the Preisach plane We consider sets WðtÞ of the following type: WðtÞ consists of all points below a non-increasing curve g that connects two points on the boundary of Pða0 ; b0 Þ; i.e. WðtÞ ¼ fða; bÞ : (ða1 ; b1 ÞAg a A
Similarly, the cost functional of Eq. (2.2) can be written as Z T Gðt; yðtÞ; WðtÞÞ dt þ F0 ðyðTÞÞ: ð2:4Þ J¼
pa1 4bpb1 4ða; bÞaða1 ; b1 Þg:
ð4:1Þ
203
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
We consider the following forms of g: (1) A non-increasing continuous curve g from one point ðu1 ; u1 Þof qPðao ; b0 Þ to some other point of qPðao ; b0 Þ and such that, in a sufficiently small neighborhood of ðu1 ; u1 Þ; g can be represented both as b ¼ jðaÞ; b0 pa1 oaoa2 oa0 and a ¼ cðbÞ; b0 pb1 obpb0 ; where j and c are strictly decreasing continuously differentiable functions. (2) A continuous curve from a point ðu1 ; u1 Þ consisting of a straight-line segment parallel to the a-axis, for u1 oapuþ 1 ; and continuous non-increasing part from ðuþ 1 ; u1 Þ to some point of qPðao ; b0 Þ þ such that, for uþ 1 oaou1 þ e; g can be represented as a ¼ cðbÞ with c non-increasing and continuously differentiable. (3) A continuous g from some point ðu1 ; u1 Þ; consisting of a straight-line segment parallel to the b-axis, for u 1 obou1 ; and a continuous non-increasing part from ðu1 ; u 1 Þ such that, for u 1 eobou1 ; g can be represented as b ¼ jðaÞ with j non-increasing and continuously differentiable. We note that the standard ‘‘staircase’’ curves that appear in the classical treatment of hysteresis [1–3] are particular cases of the 3 cases above; from the point of view of the differential calculus of setvalued functions, the staircase curves play the role of quadratic functions in ordinary differential calculus, in the sense that their derivatives of order higher than 2 vanish. In each of the 3 cases above, we can explicitly calculate the derivatives of the set-valued function WðtÞ at t ¼ t0 ; provided the input uðtÞ is differentiable at t ¼ t0 : We set v ¼ uðt ’ 0 Þ: In case (1), when v > 0; we have Wðt0 þ dtÞWWðt0 ÞðWÞ " # " # ! u1 1 dt 0
U2;þ ¼ R2;þ ðdtÞ; þ 2 0 dt u1 where U2;þ is the triangular domain with vertices ð0; 0Þ; ð2v; 2vÞ; ð2v; 2j0 ðu1 ÞvÞ; if we set A :¼ ðu1 ; u1 Þ; B :¼ ðu1 þ vdt; u1 þ vdtÞ; C :¼ ðu1 ; jðu1 þ vdtÞÞ; D :¼ ðu1 ; u1 þ j0 ðu1 ÞvdtÞ;
then the domain enclosed by the curvilinear triangle ABD is Wðt0 þ dtÞWWðt0 Þ; whereas the domain ABC is " # " # u1 1 dt 0 U2;þ : þ 2 0 dt u1 The remainder R2;þ ðdtÞ is the domain ACD, whose area is Z u1 þvdt jR2;þ ðdtÞj ¼ ½jðu1 Þ þ j0 ðu1 Þða u1 Þ u1
jðaÞda ¼ oðdt2 Þ: Still in case (1), if vo0; we have Wðt0 ÞWWðt0 þ dtÞðWÞ " # " # ! u1 1 dt 0
U2; ¼ R2; ðdtÞ; þ 2 0 dt u1 where U2; is the triangular domain with vertices ð0; 0Þ; ð2v; 2vÞ; ð2c0 ðu1 Þv; 2vÞ; and the remainder R2; ðdtÞ satisfies jR2; ðdtÞj ¼ oðdt2 Þ: In case (2), if v > 0; we have Wðt0 þ dtÞWWðt0 ÞðWÞ " # " # ! u1 1 dt 0 U2;þ ¼ R2;þ ðdtÞ; þ
2 0 dt u1 where U2;þ is the triangular domain with vertices ð0; 0Þ; ð2v; 2vÞ; ð2v; 0Þ: When vo0; we have ( " # " # ! 0 1 0 Wðt0 ÞWWðt0 þ dtÞðWÞ þ U1; u1 0 dt "
ð,Þð,Þ "
ð,Þ
uþ 1 u1
u1 u1
"
# þ
#
" þ
1 2
1 2
dt 0
dt 0
# ! 0 1 U2; dt
!) # 0 2 ¼ R2; ðdtÞ; U2; dt
where U1; is the rectangular domain with vertices þ 1 ðu1 ; 0Þ; ðu1 ; vÞ; ðuþ 1 ; 0Þ; ðu1 ; vÞ; U2; is the triangular domain with vertices ð0; 0Þ; ð0; 2vÞ; ð2v; 2vÞ; and 2 U2; is the triangular domain with vertices ð0; 0Þ; ð0; 2vÞ; ð2c0 ðuþ 1 Þv; 2vÞ:
204
"
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
In case (3) with v > 0; the first derivative is # " # u1 dt 0 ; ; U1;þ ; 0 0 1
As it is standard in utilizing the principle of dynamic programming, we consider a parametrized problem
where U1;þ is the rectangular domain with vertices ð0; u1 Þ; ðv; u1 Þ; ð0; u 1 Þ; ðv; u1 Þ; and the second derivative has connected components " # " # u1 dt 0 1 ; ; U2;þ 0 dt u1
dyðt; s; x; AÞ ¼ gðt; yðt; s; x; AÞ; Wðt; s; x; AÞÞ; t > s; dt D2 Wðt; s; x; AÞ ¼ FðWðt; s; x; AÞ; vðtÞÞ; t > s; yðs; s; x; AÞ ¼ x; Wðt; s; x; AÞ ¼ A; Z Jðs; x; AÞ :¼
and " # " # u1 dt 0 2 ; ; ; U2;þ 0 dt u 1
Gðt; yðt; s; x; AÞ; s
Wðt; s; x; AÞÞdt þ F0 ðyðT; s; x; AÞÞ: ð5:2Þ
1 where U2;þ is the triangular domain with vertices 2 ð0; 0Þ; ð2v; 0Þ; ð2v; 2vÞ and U2;þ is the triangular domain with vertices ð0; 0Þ; ð2v; 0Þ; ð2v; 2j0 ðu1 ÞvÞ: Still in case (3), if vo0; then U2; is the triangular domain with vertices ð0; 0Þ; ð0; 2vÞ; ð2v; 2vÞ:
5. The dynamic programming equations We re-formulate the problem of Section 2 as follows: the state consists of " # yðtÞ ; WðtÞ where WðtÞ takes values in the class of sets defined in Section 4. The accessory control function vðtÞ is the derivative of the original control function, and the class of admissible accessory control functions is Cð½0; T/½v1 ; v2 : The controlled dynamical system is written as dyðtÞ ¼ gðt; yðtÞ; WðtÞÞ; D2 WðtÞ ¼ FðWðtÞ; vðtÞÞ; dt yð0Þ ¼ y0 ; Wð0Þ ¼ W0
T
ð5:1Þ
and the cost functional is given by Eq. (2.4). We note that, in g and G; the set WðtÞ enters it two ways: as the domain of integration for calculating HuðtÞ and through the control function uðtÞ which, in turn, depends implicitly on WðtÞ and explicitly on some of the components of D2 WðtÞ:
Of course, y; W; and J also depend on the function vð Þ: We define the Hamilton–Jacobi function Vðs; x; AÞ by Vðs; x; AÞ :¼
inf
vACð½0;T/ðv1 ;v2 Þ
Jðs; x; AÞ:
ð5:3Þ
We shall say that a real-valued set-function C is differentiable at A if, for any family of sets SðeÞ; e0 pepe0 ; e0 > 0 with Sð0Þ ¼ A; and S twice differentiable at e ¼ 0; there exists a function DCðA; D2 Sð0ÞÞ such that CðSðeÞÞ CðAÞ ¼ DCðA; D2 Sð0ÞÞe þ oðeÞ:
ð5:4Þ
With this definition of differentiability, if the value function V is differentiable with respect to all its arguments, we obtain the dynamic programming equation ( qVðs; x; AÞ qVðs; x; AÞ þ inf gðs; x; AÞ aA½v1 ;v2 @s @x ) þ DA Vðs; x; A; D2a AÞ þ Gðs; x; AÞ ¼ 0; VðT; x; AÞ ¼ F0 ðxÞ:
ð5:5Þ
Here, the symbol D2a A denotes the derivative D2 WðsÞjWðsÞ¼A evaluated with the value vðsÞ ¼ a: The concept of generalized solutions is this: a function Vðs; x; AÞ is said to be a generalized solution of (5.4) if it satisfies the final conditions and, for every continuously differentiable function Cðs; x; AÞ; if V C has a minimum at ðs0 ; x0 ; A0 Þ;
S.A. Belbas, I.D. Mayergoyz / Physica B 306 (2001) 200–205
then C is a subsolution of Eq. (5.4) at ðs0 ; x0 ; A0 Þ; i.e. ( qCðs; x; AÞ qCðs; x; AÞ þ inf gðs; x; AÞ aA½v ;v qs qx 1 2 þ DA Cðs; x; A; D2a AÞ ) þ Gðs; x; AÞ s¼s ;x¼x 0
0 ;A¼A0
p0
ð5:6Þ
and if V C has a maximum at ðs0 ; x0 ; A0 Þ then C is a supersolution of (5.4) at ðs0 ; x0 ; A0 Þ; i.e. ( qCðs; x; AÞ qCðs; x; AÞ þ inf gðs; x; AÞ: aA½v1 ;v2 qs qx þ DA Cðs; x; A; D2a AÞ ) þ Gðs; x; AÞ s¼s ;x¼x 0
0 ;A¼A0
X0:
ð5:7Þ
Acknowledgements The reported research has been supported in part by the US Department of Energy, Engineering Research Program.
205
References [1] I.D. Mayergoyz, Mathematical Models of Hysteresis, Springer, New York, 1991. [2] A. Visintin, Differential Models of Hysteresis, Springer, New York, 1994. [3] M. Brokate, J. Sprekels, Hysteresis and Phase Transitions, Springer, New York, 1996. [4] S.A. Belbas, I.D. Mayergoyz, Dynamic programming for systems with Preisach hysteresis, preprint, 2000. [5] F. Bagagiolo, Dynamic programming for some optimal control problems with hysteresis, Preprint No. 38, Max Planck Institut f. Math. in der Naturwiss., Leipzig, 2000. . [6] M. Brokate, Optimale Steuerung von gewohnlichen Differentialgleichungen mit Nichtlinearit.aten vom HysteresisTyp, Verlag Peter Lang, Frankfurt a. M., 1987. [7] H.T. Banks, M.Q. Jacobs, J. Math. Anal. Appl. 29 (1970) 246. . [8] G. Schroder, Differentiation of interval functions, Proc. Amer. Math. Soc. 36 (1972) 485. [9] M. Spivak, Differential geometry, Vol. 1, Publish or Perish, Berkeley, 1979.