Age-structured linear-state differential games

European Journal of Operational Research 254 (2016) 269–278

Contents lists available at ScienceDirect

European Journal of Operational Research journal homepage: www.elsevier.com/locate/ejor

Innovative Applications of O.R.

Age-structured linear-state differential games Luca Grosset, Bruno Viscolani∗ Department of Mathematics, University of Padova, Via Trieste 63, I-35121 Padova, Italy

a r t i c l e

i n f o

Article history: Received 11 December 2014 Accepted 14 March 2016 Available online 22 March 2016 Keywords: Age-structured models Differential games Advertising

a b s t r a c t In this paper we search for conditions on age-structured differential games to make their analysis more tractable. We focus on a class of age-structured differential games which show the features of ordinary linear-state differential games, and we prove that their open-loop Nash equilibria are sub-game perfect. By means of a simple age-structured advertising problem, we provide an application of the theoretical results presented in the paper, and we show how to determine an open-loop Nash equilibrium.

1. Introduction In this paper we want to extend the concept of linear-state differential game to a family of models with age-structured dynamics. In the last few years there has been an increasing number of papers on applications of age-structured optimal control. For an introduction to this topic the reader can consult the book by Grass, Caulkins, Feichtinger, Tragler, and Behrens (2008, Chapter 8, pp. 417–421) or the book by Anit¸ a, Arna˘ utu, and Capasso (2011, Chapter 4, pp. 145–184). Moreover, three papers are important for our analysis. Feichtinger, Tragler, and Veliov (2003) introduce a very general set of Pontryagin-type necessary conditions, while Feichtinger, Hartl, Kort, and Veliov (2006) describe the anticipation effect: in age-structured models it is convenient to anticipate an investment flow to take advantage of the age evolution. Finally, Krastev (2013) presents a set of Arrow-type sufficient conditions. This is definitely a non-exhaustive list, but browsing the references in the just mentioned papers the reader can find a variety of different applications in this active research field. On the other hand, only a few applications of these mathematical techniques to differential games have been proposed so far. To the best of our knowledge, one of the first papers on applications of partial differential games is by Roxin (1977), who presents two examples: one on pollution control, the other on competitive fishing. A more technical paper on partial differential games is by Ichikawa (1976), who shows how to study the linear-quadratic differential games when the motion equation is described using a strongly continuous semigroup. Even if this approach is rather technical, the reference may be useful for further research on age-structured differential

© 2016 Elsevier B.V. All rights reserved.

games. Finally, as far as we know, the most recent application is by Reluga and Li (2013) in the field of mathematical biology. The fact that there are only few applications of age-structured optimal control to differential games is surely due to the complexity that arises in the analysis of the necessary conditions for this kind of models. However, even ordinary differential games are difficult to analyze, so that, in the applications, it is customary to focus on some special families of differential games. Two of them are the linear-quadratic differential games Engwerda (2005) and the linear-state differential games (Dockner, Jørgensen, Van Long, and Sorger, 20 0 0, Chapter 7, p. 187). The analysis of these kinds of differential games is mathematically tractable and allows the characterization of strong equilibria: for example it has been proved that an open-loop Nash equilibrium in a linear-state differential game is sub-game perfect (Dockner, Jørgensen, Van Long, and Sorger, 20 0 0, Chapter 7, p. 189). In this paper we define a class of age-structured differential games which has the features of ordinary linear-state differential games. In Section 2 we present a quick review of agestructured optimal control and we introduce a formulation of an age-structured differential game. In Section 3 we introduce the linear-state formulation of an age-structured differential game and we show that it is the right formulation, because we can prove the sub-game perfectness of the open-loop Nash equilibria. In Section 4 we describe a very simple advertising model with agestructured dynamics which is useful to show how to characterize an open-loop Nash equilibrium. 2. Age-structured models 2.1. Age-structured optimal control problems

∗

Corresponding author. Tel.: +390498271397; fax: +390498271479. E-mail address: [email protected] (B. Viscolani).

http://dx.doi.org/10.1016/j.ejor.2016.03.025 0377-2217/© 2016 Elsevier B.V. All rights reserved.

First of all we present a formal definition of an age-structured optimal control problem. We inform the readers that the material

270

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278

introduced in this subsection is presented to make the paper selfcontained and is taken from the seminal work by Feichtinger et al. (2003). Definition 1. An age-structured optimal control problem is defined by stating an objective functional to maximize



J (u(t, a ), v(t ) )

max

u(t,a )∈U,v(t )∈V



=

0

+

T

 ω 0

 ω 0

L(t, a, y(t, a ), p(t, a ), u(t, a ) )dadt

(a, y(T , a ) )da +



T 0



L(t, q(t ), v(t ) )dt ,

(1)

subject to a motion PDE

(∂t + ∂a )y(t, a ) = f (t, a, y(t, a ), p(t, a ), q(t ), u(t, a )), y(0, a ) = y0 (a ), y(t, 0 ) = k(t, q(t ), v(t ) ),

(2)

where the non-local variables p(t, a) and q(t) are defined as follows:

p(t, a ) = q(t ) =

 ω 0

 ω 0

g(t, a, σ , y(t, σ ), u(t, σ ) )dσ , h(t, σ , y(t, σ ), u(t, σ ) )dσ .

(3)

A feasible control u(t, a) is an element of the set of the measurable and essentially bounded functions L∞ ([0, T] × [0, ω]; U) where U is a compact subset of R; a feasible control v(t ) belongs to L∞ ([0, T]; V) where V is a compact and convex subset of R. Given a couple of feasible controls (u(t, a ), v(t ) ) there exists a unique state function y(t, a ) ∈ L∞ ([0, T ] × [0, ω]; R ) which satisfies (2) and (3). The couple of feasible controls (u∗ (t, a ), v∗ (t ) ) is optimal if and only if

J (u (t, a ), v (t, a ) ) ≥ J (u(t, a ), v(t, a ) ) ∗

∗

for all couples of feasible controls (u(t, a ), v(t, a ) ). In this definition we are assuming that the functions L, , L, f, g, h, k are measurable with respect to the variables t, a, σ , continuous with respect to the remaining variables, locally essentially bounded, and differentiable w.r.t. y, p, q, u, v. Moreover, the partial derivatives are measurable w.r.t. t, a, σ , continuous w.r.t. the remaining variables, and locally essentially bounded. For further details on the analytical setting we refer to the seminal paper by Feichtinger et al. (2003) where the Pontryagin-like necessary conditions are introduced. The idea behind this formulation is the following: the decision-maker wants to control the evolution of the PDE in the set [0, T] × [0, ω]; the first variable t ∈ [0, T] represents the time (the programming horizon is finite, T > 0), while the second variable a ∈ [0, ω] represents the age (ω > 0 is the maximum age we take into account). The quantity y(t, a) is the value of the state function at the time t for the “class” of age a. Time and age evolve together; the motion equation describing the state variable evolution in time and age is a linear PDE. In the two segments {0} × [0, ω] and [0, T] × {0} we define the boundary conditions as follows: at the time t = 0 the value of the state function is known for all a ∈ [0, ω] and it is given by the function y0 (a); while at each time t the value of the state function for the age a = 0 is given in an implicit way. This value is defined using the equation y(t, 0 ) = k(t, q(t ), v(t ) ) where v(t ) is a control, while q(t) is a “non-local” variable that depends on the shapes of the state and control functions along the segment {t} × (0, ω]. A clarifying example is provided in the field of population dynamics, where y(t, a) represents the number of people of age a at the time t. In this kind of model the number of newborns at a given time (i.e. y(t,

0)) depends on the age distribution of the population at that time (to explore this topic we suggest the interesting paper written by Simon, Skritek, and Veliov, 2013 and the references therein). The non-local variable q(t) is present also in the motion equation. In order to describe a different phenomenon another non-local variable p(t, a) is defined. This variable represents the influence of the “age-class” σ on the “age-class” a at a given fixed time t. In our opinion, an application that clearly explains the meaning of this non-local variable in the context of drug initiation is by Almeder, Caulkins, Feichtinger, and Tragler (2004). In this model y(t, a) represents the number of non-drug-users. At a fixed time t, for all σ ∈ [0, ω], the number of non-drug-users in the age-class σ (i.e. y(t, σ )) can influence the evolution of y(t, a) because of reputation interactions among different age groups. The decision maker chooses the control functions u(t, a) and v(t ) in order to maximize the objective functional (1). This is the sum of three terms: the first one depends on the values of the state and control functions on their whole domain [0, T] × [0, ω]; the second term depends on the values of the state function in the final segment {T} × [0, ω]; finally, the third term depends on the values of the state and control functions through the synthesis along the segment {t} × (0, ω] provided by the function q(t). The formulation of the problem in Definition 1 is less general than other formulations considered in literature, as we assume that the initial value for the state function y(t, a) is given a priori (it does not depend on further control), and that the quantity q(t) is defined explicitly ( the function h does not depend on p or q). This clarifies the presentation and makes the analytical setting simpler. Given the optimal control problem introduced in Definition 1, we define the distributed Hamiltonian

H (t, a, y, p, q, u, λ, η (t, σ ), ζ ) = L(t, a, y, p, q, u ) + λ f (t, a, y, p, q, u )  ω + η (t, σ )g(t, a, σ , y, u )dσ + ζ h(t, a, y, u )

(4)

0

and the boundary Hamiltonian

H b (t, q, v, λ(t, 0 ) ) = L(t, q, v ) + λ(t, 0 )k(t, q, v ) .

(5)

Moreover, we define the adjoint variables as follows: λ(t, a) satisfies

(∂t + ∂a )λ(t, a ) = −∂y H (t, a, y(t, a ), p(t, a ), q(t ), u(t, a ), λ(t, a ), η (t, σ ), ζ (t ) ), λ(t, ω ) = 0, λ(T , a ) = ∂y (a, y(T , a )),

(6)

while η(t, a) and ζ (t) are defined explicitly by the equations:

η (t, a ) = ∂ p H (t, a, y(t, a ), p(t, a ), q(t ), u(t, a ), λ(t, a ), η (t, a ), ζ (t ) ), ζ (t ) = ∂q H b (t , q(t ), v(t ), λ(t , 0 )) +

 ω 0

∂q H (t, a, y(t, a ), p(t, a ), q(t ), u(t, a ),

λ(t, a ), η (t, a ), ζ (t ) )da.

(7)

We observe that, under our assumptions, Eq. (7) defines the quantities η(t, a) and ζ (t) explicitly, as the functions ∂ q H and ∂ p H do not depend on the two variables η and ζ . The assumptions introduced after Definition 1 allow us to apply the necessary conditions for optimality described by Feichtinger et al. (2003) and recalled by Krastev (2013). Theorem 1 (Necessary conditions). Let (u∗ (t, a ), v∗ (t ) ) be an optimal couple of controls for the age-structured problem (1)–(3) and let y∗ (t, a) be the state function associated with that couple of controls.

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278

We denote by p∗ (t, a) and q∗ (t) the quantities associated with this optimal solution by (3). Then there exist λ(t, a ) ∈ L∞ ([0, T ] × [0, ω]; R ), η (t, a ) ∈ L∞ ([0, T ] × [0, ω]; R ), and ζ (t ) ∈ L∞ ([0, T ]; R ) that solve (6) and (7), and moreover the following relations hold

compact of R. Given a pair of feasible con and convex subset   trols u∗1 (t, a ), v∗1 (t ) , u∗2 (t, a ), v∗2 (t ) there exists a unique1 state function y(t, a ) ∈ L∞ ([0, T ] × [0, ω]; R ) which satisfies (10) and (11).

u∗ (t, a ) ∈ arg max H (t, a, y∗ (t, a ), p∗ (t, a ), q∗ (t ), u,

We define now the open-loop Nash equilibrium.

u∈U

λ(t, a ), η (t, a ), ζ (t )), v (t ) ∈ arg max H b (t , q∗ (t ), v, λ(t , 0 )) . ∗

v∈V

Proof. See Feichtinger et al. (2003) for the original version of the necessary conditions and see Krastev (2013) for the equivalent version used in this paper.  Theorem 2 (Sufficient conditions). Let (u∗ (t, a ), v∗ (t ) ) be a couple of feasible controls for the age-structured optimal control problem (1)–(3). Let us assume that these controls satisfy the necessary conditions described in Theorem 1 and let us define the maximized distributed Hamiltonian as follows:

H ∗ (t, a, y, p, q, λ, η (t, σ ), ζ ) = max H (t, a, y, p, q, u, λ, η (t, σ ), ζ ).

(8)

u∈U

Furthermore, let us assume that • •

•

•

U and V are convex sets, H∗ (t, a, y, p, q, λ, η(t, σ ), ζ ) is jointly concave with respect to y, p, q for all values of the other variables, H b (t, q, v, λ(t, 0 ) ) is jointly concave with respect to q, v for all values of the other variables, (a, y) is concave in y for all a, then (u∗ (t, a ), v∗ (t ) ) is optimal for (1)–(3).

Proof. See Krastev (2013).



2.2. Age-structured differential games Here we present a formal definition of a two-player agestructured differential game. Definition 2. A two-player, non-cooperative, age-structured differential game is defined by stating, for i = 1, 2, an objective functional to maximize

 

max

ui (t,a )∈Ui ,vi (t )∈Vi



=

T

 ω

0

+

0

 ω 0



Ji ui (t, a ), vi (t ), u j (t, a ), v j (t )

Li (t, a, y(t, a ), p(t, a ), u1 (t, a ), u2 (t, a ) )da dt

i (a, y(T , a ) )da +



T 0



Li (t , q(t ), v1 (t ), v2 (t ) )dt ,

(9)

where j = i, subject to a motion PDE

(∂t + ∂a )y(t, a ) = y ( 0, a ) = y(t, 0 ) =

f (t, a, y(t, a ), p(t, a ), q(t ), u1 (t, a ), u2 (t, a ) ), y0 (a ), k(t, q(t ), v1 (t ), v2 (t ) ),

(10)

where the non-local variables p(t, a) and q(t) are defined as follows:

p(t, a ) = q(t ) =

 ω 0

 ω 0

g(t, a, σ , y(t, σ ), u1 (t, σ ), u2 (t, σ ) )dσ , h(t, σ , y(t, σ ), u1 (t, σ ), u2 (t, σ ) )dσ .

271

(11)

A player i’s feasible control ui (t, a) is an element of the set of the measurable and essentially bounded functions L∞ ([0, T] × [0, ω]; Ui ) where Ui is a compact subset of R; a player i’s feasible control vi (t ) belongs to L∞ ([0, T]; Vi ) where Vi is a

Definition 3. A pair of feasible controls ((u∗1 (t, a ), v∗1 (t )), (u∗2 (t, a ), v∗2 (t ))) for the two-player, non-cooperative, age-structured differential game (9), (10), (11) is an Open-Loop Nash Equilibrium (OLNE for short) if and only if for all i, j ∈ {1, 2}, i = j, and for all feasible controls (ui (t, a ), vi (t ) ) ∈ L∞ ([0, T ] × [0, ω]; Ui ) × L∞ ([0, T ]; Vi ), the following inequalities hold





Ji u∗i (t, a ), v∗i (t ), u∗j (t, a ), v∗j (t )





≥ Ji ui (t, a ), vi (t ), u∗j (t, a ), v∗j (t ) . This definition is consistent with the similar ones presented in the books by Dockner et al. (20 0 0, Chapter 4, p. 86) and by Haurie, Krawczyk, and Zaccour (2012, Chapter 7, p. 247). The necessary conditions for an open-loop Nash equilibrium lead us to study a system of nonlinear-integro-partial differential equations. To the best of our knowledge, this problem can only be tackled using numerical methods. However, in theoretical models, it is often important to obtain a closed form solution. In ordinary differential games, where the necessary conditions have a lower complexity, two common families of models are introduced in order to obtain an analytically tractable system of conditions. The family of linear-quadratic differential games is illustrated for example in the book by Dockner et al. (20 0 0, Chapter 7, p. 170) and in the book by Engwerda (2005). The family of linear-state differential games is presented in the book by Dockner et al. (20 0 0, Chapter 7, p. 187) in the book by Haurie et al. (2012, Chapter 7, p. 262), and in the clear paper by Jørgensen, Martín-Herrán, and Zaccour (2010). The main features of the linear-state differential games are the following: • •

the optimal strategies do not depend on the state; the adjoint equations are decoupled from the motion equations.

The importance of these features is described in several papers and books, for example in Dockner et al. (20 0 0, Section 7.2, p. 187) and Haurie et al. (2012, p. 262). In our personal opinion, the clearest reference for this topic is the paper by Jørgensen et al. (2010). If the two features just outlined occur, we can solve the problem using the following procedure: solve the adjoint equations, characterize the optimal strategies, and find the optimal state function. In Sections 3 and 4 we will study the features of such linearstate differential games in age-structured problems. In the remaining part of this section we propose an example of a differentiable game with age-structure as of Definition 2. In such example we implement an elementary approach to extend the linear-state structure to model (9)–(11), by assuming that all the data are affine functions in the state y and additively separable in y and (u1 , u2 ), so that there is no multiplicative interaction between the state and the control variables as required in Dockner et al. (20 0 0, Chapter 7, p. 188). Such features are a natural requirement, albeit insufficient, towards the definition of a tractable class of differentiable games with age-structure. Unfortunately, they cannot guarantee that the procedure “solve adjoint equations, characterize optimal strategies, find optimal state” really works. The obstacles are due to the presence of a non-local variable that introduces an indirect link between the state and its adjoint function. This observation paves the way to the assumptions of Section 3. The reader may well skip

1 We are assuming the same hypotheses on the functions L, , L, f, g, h, k already introduced after Definition 1.

272

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278

the example and go on to the next section, if he simply wants to see our definition of linear-state age-structured differential game. Example 1. Let us consider the following symmetric agestructured differential game

 T 

max

ui (t,a )∈U

(∂t + ∂a )y(t, a ) q(t ) y ( 0, a ) y(t, 0 )

  ω 1 − u2i (t, a )dadt + y(T , a )da , 2 0 0 0 = q(t )(u1 (t, a ) + u2 (t, a ) ),  ω (12) = y(t, σ )dσ , ω

0

= 0, = q(t ) .

We notice that all the functions are linear in the state. The distributed Hamiltonian for the player i

Hi (t, a, y, p, q, u1 , u2 , λ1 , λ2 , ζ1 , ζ2 ) = −u2i /2 + λi q(u1 + u2 ) + ζi y (13) and the boundary Hamiltonian

Hib (t, q, v1 , v2 , λ1 (t, 0 ), λ2 (t, 0 ) ) = λi (t, 0 )q

(14)

are both linear functions of the state. We notice that maximizing the Hamiltonian of player i we obtain

u∗i (t, a ) = λi (t, a )q(t ),

 ω 0

+ λ2 (t, a ) )da



 

0

(16)

(17)

Hi (t, a, y, p, q, u1 , u2 , λi , ηi (t, σ ), ζi )

 



= L¯ i (t, a )y + L˜i (t, a ) p + Lˆi (t, a, u1 , u2 ) + λi ( f¯(t, a )y + f˜(t, a ) p + fˇ(t, a )q + fˆ(t, a, u1 , u2 ))  ω   + ηi (t, σ ) g¯ (t, a, σ )y + gˆ(t, a, σ , u1 , u2 ) dσ

λi (t, a )(λ1 (t, a ) (18)

and this shows that the adjoint equation is coupled with the motion equations by the presence of the non-local variable q(t). This definitely destroys the fundamental idea behind the linear-state structure of a differential game. 3. Linear-state age-structured differential games We need a definition of a special class of age-structured differential games to obtain the analytical features that characterize the ordinary linear-state differential games. The basic idea comes from observing the form of the necessary conditions: if we want to decouple the motion from the adjoint equations, we have to assume the linearity of the data functions both in the state variable and in the non-local variables. According to this intuition, we present the following definition: Definition 4. Let us consider the two-player non-cooperative agestructured differential game (9)–(11); we call it linear-state if the data functions are affine (linear) functions of the state y and of the non-local variables p, q, and additively separable in (y, p, q) and (u1 , u2 , v1 , v2 ), i.e. if for all i, j ∈ {1, 2},

Li (t, a, y, p, u1 , u2 ) = L¯ i (t, a )y + L˜i (t, a ) p + Lˆi (t, a, u1 , u2 ), i (a, y ) = ¯i (a )y + ˜i (a ), Li (t, q, v1 , v2 ) = Lˇi (t )q + Lˆi (t, v1 , v2 ),



Proof. If u∗1 (t, a ), v∗1 (t ) , u∗2 (t, a ), v∗2 (t ) is a Nash equilibrium, then it satisfies the necessary conditions for the control problems of both players. Player i’s distributed Hamiltonian is

Putting together all this information, we obtain that

 ω

(19)

In order to show that this definition is the right one, we observe some of its consequences. In the following result we prove that an Open-Loop Nash Equilibrium (OLNE) for a linear-state agestructured differential game is sub-game perfect, as defined in Dockner et al. (20 0 0, Chapter 4, p. 102). Roughly speaking, if we restrict the game to a subinterval [τ , T] for all τ ∈ [0, T), then the previous OLNE still remains a valid OLNE. This result has a twofold outcome. First of all, we show that the necessary conditions for an OLNE lead to an adjoint equation which is decoupled from the motion equation. Secondly, we prove that the linear-state assumption implies sub-game perfectness even when the motion equation is age-structured. The latter is a standard result for ordinary linearstate differential games as shown in Dockner et al. (20 0 0, Chapter 7, p. 188), and it is important because it gives “stability” to the equilibrium.



λi (t, a )(u1 (t, a ) + u2 (t, a ))da .

(∂t + ∂a )λi (t, a ) = −λi (t, 0 ) − q(t )

h(t, σ , y, u1 , u2 ) = h¯ (t, σ )y + hˆ (t, σ , u1 , u2 ), k(t, q, v1 , v2 ) = kˇ (t )q + kˆ 1 (t )v1 + kˆ 2 (t )v2 .

(15)

The variables ζ i (t) are explicitly defined by the equations:

ζi (t ) = λi (t, 0 ) +

g(t, a, σ , y, u1 , u2 ) = g¯ (t, a, σ )y + gˆ(t, a, σ , u1 , u2 ),

Theorem 3. Let u∗1 (t, a ), v∗1 (t ) , u∗2 (t, a ), v∗2 (t ) be an open-loop Nash equilibrium for the linear-state age-structured differential game (9)–(11) and (19); if Ui and Vi are convex sets and Lˆi (t, v1 , v2 ) is jointly concave in (v1 , v2 ) for all t ∈ [0, T], then the equilibrium is sub-game perfect.

where the adjoint variables are defined as follows:

(∂t + ∂a )λi (t, a ) = −ζi (t ), λi (t, ω ) = 0, λi (T , a ) = 1 .

f (t, a, y, p, q, u1 , u2 ) = f¯(t, a )y + f˜(t, a ) p + fˇ(t, a )q + fˆ(t, a, u1 , u2 ),

0

+ ζi (h¯ (t, a )y + hˆ (t, a, u1 , u2 )), while the boundary Hamiltonian is

Hib (t, q, v1 , v2 , λi (t, 0 ) ) = Lˇi (t )q + Lˆi (t, v1 , v2 ) + λi (t, 0 ) × (kˇ (t )q + kˆ 1 (t )v1 + kˆ 2 (t )v2 ). By the necessary conditions (see Theorem 1), there exist λi (t, a), ηi (t, a), ζ i (t) that solve the following integro-partial-differential equation

(∂t + ∂a )λi (t, a ) = −L¯ i (t, a ) − λi (t, a ) f¯(t, a ) −

 ω 0

ηi (t, σ )g¯ (t, a, σ )dσ − ζi (t )h¯ (t, a ),

λi (t, ω ) = 0, λi (T , a ) = ¯i (a ), where ηi (t, a) and ζ i (t) are

ηi (t, a ) = L˜i (t, a ) + λi (t, a ) f˜(t, a ), ζi (t ) = Lˇi (t ) + λi (t, 0 )kˇ (t ) + Moreover

 ω 0

λi (t, a ) fˇ(t, a )da .

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278



u∗1 (t, a ) = arg max Lˆ1 (t, a, u1 , u∗2 (t, a ) ) u1 ∈U1

+ λ1 (t, a ) fˆ(t, a, u1 , u∗2 (t, a ) )  ω + η1 (t, σ )gˆ(t, a, σ , u1 , u∗2 (t, a ))dσ 0



+ ζ1 (t )hˆ (t, σ , u1 , u∗2 (t, a ) ) ,

  v∗1 (t ) = arg max Lˆ1 (t, v1 , v∗2 (t ) ) + λ1 (t, 0 )kˆ 1 (t )v1 , v1 ∈V1

and symmetric equations hold for u∗2 (t, a ) and v∗2 (t ). We can notice that u∗i (t, a ) and v∗i (t ) depend on the adjoint variables λi (t, a), ηi (t, σ ), ζ i (t) only, and, in addition to that, the adjoint integro-PDEs are decoupled from the motion equation. These two observations imply that the optimal controls are independent of the state function, hence the functions u∗1 (t, a ), v∗1 (t ), u∗2 (t, a ), v∗2 (t ), restricted to the time interval [τ , T], satisfy the necessary conditions for an openloop Nash equilibrium concerning the same game starting at the time τ from any state yτ (a). These controls satisfy not only the necessary conditions, but also the sufficient ones. By the assumption of Theorem 3, Ui and Vi are convex for i = 1, 2. Moreover, the concavity assumptions required by the sufficient conditions are satisfied too, and to prove it we explicitly write player 1’s maximized distributed Hamiltonian:

(t, a, y, p, q, u2 , λ1 , η1 (t, σ ), ζ1 )   = L¯ 1 (t, a )y + L˜1 (t, a ) p + λ1 f¯(t, a )y + f˜(t, a ) p + fˇ(t, a )q

H1∗

 ω

η1 (t, σ )g¯ (t, a, σ )ydσ + ζ1 h¯ (t, a )y  + max Lˆ1 (t, a, u1 , u2 ) + λ1 fˆ(t, a, u1 , u2 ) +

0

u1 ∈U1

+

 ω 0

 η1 (t, σ )gˆ(t, a, σ , u1 , u2 )dσ + ζ1 hˆ (t, a, u1 , u2 ) .

We notice that this function is linear in y, p, q, by the assumption (19), hence it is jointly concave in y, p, q. Player 1’s boundary Hamiltonian

H1b (t, q, v1 , v2 , λi (t, 0 ) ) = Lˇ1 (t )q + Lˆ1 (t, v1 , v2 ) + λ1 (t, 0 ) × (kˇ (t )q + kˆ 1 (t )v1 + kˆ 2 (t )v2 ) is linear in q, by (19), and concave in v1 , by the assumption of the theorem, hence it is jointly concave in q, vi . Finally, 1 (a, y ) = ¯1 (a )y + ˜1 (a ) is linear in y, by (19), hence it is concave in y. The same  results hold for player 2. This implies that the restriction of u∗1 (t, a ), v∗1 (t ), u∗2 (t, a ), v∗2 (t ) to the time interval [τ , T] is an open-loop Nash equilibrium for the game starting at the time τ from any state value yτ (a). This proves that the open-loop Nash equilibrium is sub-game perfect.  Theorem 3 proves that, in a linear-state age-structured differential game, with an additional restriction concerning the control ranges Ui , Vi and the function Lˆi (t, v1 , v2 ), any OLNE is sub-game perfect. What is still missing in our analysis is a general result about the existence and the uniqueness of an OLNE for this kind of differential games. Unfortunately, we cannot present such a theorem and we think that the analytical form of the problem makes it very difficult to prove a general result on this issue. However, in the following, we suggest a solution search scheme that may be useful to characterize the OLNE using the necessary and sufficient conditions. It is inspired by an analogous scheme proposed by Bressan (2011) for a standard differential game. Let us consider a linear-state age-structured differential game. For the sake of simplicity we assume that the non-local variables p and q cannot be controlled directly by the players, i.e.

273

gˆ(t, a, σ , u1 , u2 ) = hˆ (t, a, u1 , u2 ) = 0. Moreover, we assume that the players cannot control the value of the state function on the boundary, i.e. kˆ 1 (t ) = kˆ 2 (t ) = 0. While searching for an OLNE of a linear-state age-structured differential game, we try to perform the following steps. Step 1 Find the best response of player i, by maximizing the distributed Hamiltonian with respect to ui





u#i t, a, u j , λi = arg max{Lˆi (t, a, u1 , u2 ) + λi fˆ(t, a, u1 , u2 )} ui ∈Ui

and by maximizing the boundary Hamiltonian with respect to vi

  v#i t, v j = arg max Lˆi (t, v1 , v2 ). vi ∈Vi









We assume that the functions u# t, a, u j , λi and v# t, v j i i are well-defined, although this is a restrictive assumption. Step 2 Find a solution (u∗1 , u∗2 , v∗1 , v∗2 ) of the nonlinear equations system



ui = u#i t, a, u j , λi   vi = v#i t, v j



∀i, j ∈ {1, 2}, i = j , ∀i, j ∈ {1, 2}, i = j .

We assume that such a system has  a unique solution which leads us to define the functions u∗i t, a, λi , λ j , v∗i (t ), i = 1, 2. In fact, the uniqueness of the solution is not always guaranteed, nor its existence. Step 3 Solve the following system of two coupled integro-PDEs (with i ∈ {1, 2})

(∂t + ∂a )y(t, a ) = f¯(t, a )y(t, a ) + f˜(t, a ) p(t, a ) + fˇ(t, a )q(t ) + λi (t, a ) fˆ(t, a, u∗1 , u∗2 ),

(∂t + ∂a )λi (t, a ) = −L¯ i (t, a ) − λi (t, a ) f¯(t, a )

 ω − ηi (t, σ )g¯ (t, a, σ )dσ −ζi (t )h¯ (t, σ ), 0  ω p(t, a ) = g¯ (t, a, σ )y(t, a )dσ , 0  ω h¯ (t, σ )y(t, σ )dσ , q(t ) = 0

ηi (t, a ) = L˜i (t, a ) + λi (t, a ) f˜(t, a ), ζi (t ) = Lˇi (t ) + λi (t, 0 )kˇ (t ) +

 ω 0

λi (t, a ) fˇ(t, a )da,

y(0, a ) = y0 (a ), y(t, 0 ) = k(t )q(t ),

λi (t, ω ) = 0, λi (T , a ) = ¯i (t ). In the following, we assume that we are able to find a unique solution to such system, although this search is a challenging task and it represents the main obstacle in the analysis of a linear-state age-structured differential game. If we can complete the three steps, then we have a unique solution that satisfies all the necessary conditions. The analytical forms of the distributed Hamiltonian and the boundary Hamiltonian in a linear-state age-structured differential game allow us to apply the sufficient conditions, provided that the control ranges Ui , Vi are convex and the function Lˆi (t, v1 , v2 ) is concave in (v1 , v2 ). Therefore, through this scheme we characterize the unique OLNE. In the following section we apply the scheme to an example and show a case in which we can explicitly perform all the required computations.

274

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278

4. A two-retailer game In this section we illustrate the use of the results we have just obtained, introducing a very simple model based on the papers by Grosset and Viscolani (2005) and by Faggian and Grosset (2013). We assume that two retailers sell the same product to different target markets. The market segmentation is made using the age as a fundamental parameter, hence the goodwill value is described by the state function y(t, a) (see Huang, Leng, and Liang, 2012 for a recent survey about dynamic advertising research, where our approach could be classified as an extention of the classical Nerlove– Arrow model). We denote by ui (t, a) ≥ 0 the advertising flow controlled by retailer i ∈ {1, 2} and directed at the time t towards the segment a. In this kind of models the investment process is irreversible and a negative advertising flow makes no sense. We assume that the goodwill evolution can be described by the following PDE:

(∂t + ∂a )y(t, a ) = −δ y(t, a ) + u1 (t, a ) + u2 (t, a ) + ρ u1 (t, a )u2 (t, a ),

for all a ∈ [0, ω] and we assume the further boundary condition y(t, 0 ) = 0, i.e. the goodwill for the youngest class of the population is equal to 0. These assumptions lead us to study the following agestructured differential game, where each retailer aims at maximizing his total discounted profit with discount rate r > 0,



T

max

ui (t,a )≥0 0



×

e−rt

ω

πi (a )y(t, a ) −

0

where δ > 0 is the constant goodwill decay factor. The first term in the r.h.s. of the PDE represents the goodwill decay, the second and third terms are the direct effects of the advertising flows of the two retailers, while the last term represents the interaction effect of the two advertising flows. With ρ = 0 we have the linear advertising productivity generalization of the Nerlove–Arrow’s model, as often used in cooperative advertising models (see: Jørgensen and Zaccour (2014)) and the age-distributed goodwill, as in Grosset and Viscolani (2005). With ρ > 0, we have a synergy situation with a positive correction to the linear part, as in Prasad and Sethi (2009) and Leitmann and Schmitendorf (1978). With ρ < 0 we have an interference situation with a negative correction to the linear part, which occurs, for example, when the two retailers have dissonant advertising messages; this agrees with the model by Grosset, Roberti, and Viscolani (2011). The revenue flow of retailer i at age a and time t is assumed to be proportional to the goodwill

q(t ) =

(πi (a ) + ξi )y(t, a ),

u#i t, a, u j , λi = ert

 ω 0

πi (a )y(t, a )da + ξi q(t ),

(22)

where ω > 0 is the maximum age considered in the market segmentation and

q(t ) =

 ω 0

y(t, σ )dσ

Ci (a, ui ) = κi (a )

(25)

 ω 0

y(t, σ )dσ .

(26)

(27)

4.1. Equilibrium conditions We apply the solution search scheme presented in Section 3 to the linear-state age-structured differential game (25)–(27). 4.1.1. Step 1 Maximizing the distributed Hamiltonian of retailer i,

Hi (t, a, y, u1 , u2 , λi , ζi ) = e−rt πi (a )y − e−rt

κi (a ) 2

u2i + λi (−δ y + u1 + u2 + ρ u1 u2 ) + ζi y, (28)

we obtain that the equilibrium advertising policies of the retailers are given by





 λi  1 + ρu j , κi (a )

i, j ∈ {1, 2}, i = j,

(29)

provided that λi ≥ 0. We will discuss this condition at the end of Step 3. 4.1.2. Step 2 We have to solve the system





ui = u#i t, a, u j , λi ,

i, j ∈ {1, 2}, i = j,

(30)

in the unknowns u1 , u2 , and we obtain





u∗i t, a, λi , λ j = ert λi

κ j (a ) + ρ ert λ j , i = j , κi (a )κ j (a ) − ρ 2 e2rt λ j λi

(31)

provided that κi (a )κ j (a ) − ρ 2 e2rt λ j λi > 0. We will check this condition at the end of Step 3 when we obtain an explicit form of the adjoint functions.

(23)

represents the global goodwill at the time t. Here, ξ i and π i (a) are the uniform and the age-specific parts of the marginal revenue of goodwill y(t, σ ), respectively. As frequently done, the advertising cost flow is quadratic for both retailers

u2i /2,

 (t, a ) da + ξi q(t ) dt,

(∂t + ∂a )y(t, a ) = −δ y(t, a ) + u1 (t, a ) + u2 (t, a ) + ρ u1 (t, a )u2 (t, a ), y(0, a ) = α (a ), y(t, 0 ) = 0, where

with piecewise continuous π i (a) and π i (a) ≥ 0, ξ i ≥ 0. For example, we can think that retailer i sells his product using two different distribution channels. The first one is an on-line store, where detailed information on the customers is available and retailer i can estimate the marginal revenue of goodwill in each segment a. The second channel is a traditional shop, where only general market information is available and retailer i is not able to profile the customers by age. This assumption means that the total revenue flow for retailer i at time t is

2

u2i

subject to the motion PDE

(20)

(21)

κi (a )

(24)

where the piecewise continuous function κ i (a) > 0 describes the cost for retailer i to reach the age-segment a. Finally, we assume that the initial value of the state function is given: y(0, a ) = α (a )

4.1.3. Step 3 Here we have to characterize the adjoint functions. First of all we notice that

ζi (t ) = e−rt ξi ,

(32)

therefore the adjoint functions λi (t, a) must satisfy the linear PDEs

(∂t + ∂a )λi (t, a ) = δλi (t, a ) − e−rt πi (a ) − e−rt ξi , λi (t, ω ) = 0, λi (T , a ) = 0 .

(33)

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278

We notice that the adjoint equations are decoupled from the motion equation. This is exactly what we want to obtain from the linear-state assumption of the model. Now we solve the adjoint PDE and then we characterize the remaining quantities. In order to solve the linear PDE (33), let us introduce the characteristic line σ (τ ) = a + τ . If λi (t, a) is a solution of the adjoint PDE, we define i (τ ) = λi (τ , σ (τ ) ) = λi τ , a + τ , which satisfies the linear ODE

˙ i (τ ) = δ i (τ ) − e−rτ πi (σ (τ ) ) − e−rτ ξi , hence

i (τ )

= e δ ( τ −θ )



θ τ

(πi (σ (s ) ) + ξi )e

(34)

−rs −δ (s−θ )

e

 ds + i (θ ) . (35)

Now, let (t, a) ∈ [0, T] × [0, ω] be a point in the time-age space and let σ (τ ) = a − t + τ be the unique characteristic line which passes through the point (t, a). This characteristic line may either intersect the segment [0, T) × {ω}, or the segment {T} × [0, ω]. •

If the characteristic line intersects the segment [0, T) × {ω}, then the intersection occurs at the time θ = ω − a + t ∈ [0, T ) such that σ (θ ) = ω, therefore a − t > ω − T . In this case we have that

i (θ ) = λi (θ , ω ) = 0, and

i (τ ) =

 ω−a+t τ

(πi (a − t + s ) + ξi )e−rs e−δ (s−τ ) ds ;

hence the value of the adjoint function at the point (t, a) is

λi (t, a ) = i (t ) = e−rt •

 ω a

(πi (u ) + ξi )e−(δ+r )(u−a) du ≥ 0 .

If the characteristic line intersects the segment {T} × [0, ω], then the final age is σ (T ) = a − t + T ≤ ω and therefore a − t ≤ ω − T . In this case we have that

i (T ) = λi (T , σ (T ) ) = 0, and

i (τ ) =



T

τ

(πi (a − t + s ) + ξi )e

λi (t, a ) = i (t ) = e−rt

−rs −δ (s−τ )

e

ds ;



a+T −t

a

(πi (u ) + ξi )e−(δ+r )(u−a) du ≥ 0 .

Finally, recollecting the two parts of the analysis, we obtain that

=

⎪ ⎩e−rt

ω

(πi (u ) + ξi )e

−(δ +r )(u−a )

du,

a − t > ω − T,

(37)

If this inequality holds, then we have characterized a unique solution that satisfies the necessary conditions for an OLNE. Finally, we observe that the control ranges are convex sets and the functions Lˆi (t, v1 , v2 ) are identically equal to zero in this problem; hence the sufficient conditions hold and the previous solution is really the unique OLNE for the linear-state age-structured differential game (25)–(27). We collect all this information in a proposition. Proposition 1. Let us assume that (37) holds, then the strategies (31), where λi (t, a) is substituted by (36), are the unique OLNE for (25)– (27).

4.2. Equilibrium advertising efforts In Eqs. (30) and (31) we can observe some general features of the equilibrium advertising efforts of the two retailers. We notice that inequality (37) is a sufficient condition for the existence and the uniqueness of an OLNE. If the interaction effect between the two advertising flows is small (i.e. if |ρ | is small), then the differential game admits an equilibrium. On the other hand, if such interaction effect is large (i.e. if |ρ | is large) then the quadratic term in the motion equation ρ u1 u2 is the predominant one. In particular, when ρ is large and positive, the return on investment in goodwill is greater than the adversing expenditure; hence both optimal advertising strategies diverge and the game does not admit an equilibrium (the quadratic advertising costs cannot balance the advertising effort increment required). In the following we comment on some features of the model due to the different signs of the parameter ρ . If ρ = 0, then each retailer behaves as if they were alone and their advertising effort were unaffected by the other retailer’s decision,

λi (t, a ) , κi (a )

(38)

as in the monopolistic problem treated in Grosset and Viscolani (2005). If ρ = 0, the relationship between the two retailers is more interesting and we have to consider the sign of ρ : •

•

a a+T −t a

κ¯ i · κ¯ j · (δ + r )2  . (π¯ i + ξi ) π¯ j + ξ j

u∗i (t, a ) = ert

hence the value of the adjoint function at the point (t, a) is

λi (t, a ) ⎧  ⎪ ⎨e−rt

ρ2 <

275

in the synergy situation, when ρ > 0, each retailer advertises more than in the case ρ = 0 as they exploit the synergy with the other retailer’s advertising effort; in the interference situation, when ρ < 0, each retailer advertises less, in order to limit the harmful interference of the other retailer’s advertising effort.

(πi (u ) + ξi )e−(δ+r )(u−a) du, a − t ≤ ω − T . (36)

Now we want to check whether our general assumptions are satisfied. The first assumption, λi (t, a) ≥ 0, is satisfied because the integrand function is positive. If π¯ i = supa∈[0,ω] πi (a ), then

(δ + r )ert λ (t, a ) ≤ π¯ i + ξi i



1 − e−(δ +r )(ω−a ) , a − t > ω − T , 1 − e−(δ +r )(T −t ) , a − t ≤ ω − T ,

hence λi (t, a ) ≤ e−rt (π¯ i + ξi )/(δ + r ). Moreover, if κ¯ i = infa∈[0,ω] κi (a ), then, using the previous inequality, we obtain a sufficient conditon for the second assumption:

4.3. A special instance If we assume that π1 (a ) = η1 I[ϑ1 ,ω] (a ), with η1 > 0, 0 < ϑ1 < ω, and ξ1 = 0, then retailer 1 observes a demand with constant marginal value of goodwill equal to η1 only from the customers

who are older than ϑ1 . On the other hand, if we assume that π 2 (a) ≡ 0 and ξ 2 > 0, then there is a segment-constant demand addressed to retailer 2. Substituting the values of the parameters π i (u) and ξ i into Eq. (36), setting ν = δ + r, and after some computations, we obtain:

276

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278

Fig. 2. Retailer 2’s ad effort, ρ = 0. Fig. 1. Retailer 1’s ad effort, ρ = 0.

λ1 (t, a ) =

⎧  ⎪ ⎨e−rt ⎪ ⎩e−rt

a



ω

η1 I[ϑ1 ,ω] (u )e−ν (u−a) du,

a+T −t

a

a > ω − T + t,

η1 I[ϑ1 ,ω] (u )e−ν (u−a) du, a ≤ ω − T + t,

⎧ −rt ν a −1  −ν max {a,ϑ } −νω  1 − e e η1 e ν e , a > ω − T + t, ⎪ ⎪ ⎪ ⎨ −rt ν a −1  −ν max {a,ϑ1 } −ν (a+T −t )  e η1 e ν e −e , a ≤ ω − T + t, = ⎪ a > ϑ1 − T + t, ⎪ ⎪ ⎩ 0, a ≤ ϑ1 − T + t, (39) and

λ2 (t, a ) =



e−rt ξ2 (1 − e−ν (ω−a ) )/ν, a > ω − T + t, e−rt ξ2 (1 − e−ν (T −t ) )/ν,

a ≤ ω−T +t.

(40)

In the following, we consider some numerical examples of the two-retailer game and we illustrate the results through some plots of the relevant functions.2 With reference to the notation in this subsection, let us set

δ = 0.2, ω = 80, T = 70, ξ2 = 10, ϑ1 = 40, η1 = 20, r = 0.02,

and let the advertising costs be age-independent

κ1 (a ) = κ2 (a ) = 30,

a ∈ [0, ω].

In particular, we want to examine the advertising strategies of the two retailers associated with the different signs of the interaction parameter ρ . 4.3.1. No advertising interaction case: ρ = 0 The Nash equilibrium advertising strategies of retailers 1 and 2 are plotted in Figs. 1 and 2, respectively. The equilibrium advertising flow of the first retailer is decreasing in time for all age-segments. This is due to the presence of the discount factor in the objective function and to the setting of the model in a finite horizon (the advertising flow must vanish at the end of the programming interval). As far as the age-segmentation is concerned, the advertising flow is maximum in the target market [ϑ1 , ω]. We can see an anticipation effect: the optimal 2 All the plots are obtained programming the explicit formula (31), (39) and (40) in Wolfram Mathematica 10.1. The file gv2015_a.nb, used to prepare these plots, is attached as supplementary content to this paper.

Fig. 3. Retailer 1’s ad effort, ρ < 0.

advertising flow is positive before ϑ1 , as already observed in Grosset and Viscolani (2005) and in Faggian and Grosset (2013). Moreover, the equilibrium advertising flow decreases quickly near the maximum age ω: it is not convenient to invest in these segments because they will soon exit the target market. The equilibrium advertising flow of the second retailer is also decreasing in time for all age-segments. Nevertheless, the target market of the second retailer is [0, ω], hence his advertising flow does not exhibit any anticipation effect. As shown in Fig. 1, the advertising flow decreases quickly near the maximum age ω. 4.3.2. Advertising interference case: ρ < 0 If we assume that ρ = −0.2, then an interference effect is observed between the advertising efforts. The Nash equilibrium advertising strategies of retailers 1 and 2 are plotted in Figs. 3 and 4, respectively. When a negative interaction exists between the advertising flows, we obtain that the equilibrium advertising effort of the first retailer has the same qualitative shape as before, but a smaller magnitude. On the other hand, the equilibrium advertising flow of the second retailer changes both in shape and magnitude with respect to the case ρ = 0. The second retailer focuses his advertising flow in the age-segments which do not belong to the target market of the

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278

277

Fig. 4. Retailer 2’s ad effort, ρ < 0. Fig. 6. Retailer 2’s ad effort, ρ > 0.

Acknowledgments We want to thank the anonymous reviewers for their stimulating suggestions and comments. This research was supported by the Italian Ministry of University and Research and by the University of Padova. Supplementary material Supplementary material associated with this article can be found, in the online version, at 10.1016/j.ejor.2016.03.025. References

Fig. 5. Retailer 1’s ad effort, ρ > 0.

first retailer. In other words, to avoid the negative interference of retailer 1’s advertising actions, the second retailer focuses his advertising flow in the market niche which is not covered by the first retailer. 4.3.3. Advertising synergy case: ρ > 0 If we assume ρ = 0.2, then a synergy effect is observed between advertising efforts. The Nash equilibrium advertising strategies of retailers 1 and 2 are plotted in Figs. 5 and 6, respectively. In the case of advertising synergy, both retailers focus their equilibrium advertising flows in the target market of the first retailer, in order to obtain the maximum profit from their investment by exploiting the synergy effect. The same results apply to the case where both retailers have an age-specific marginal revenue.3 If a negative interference effect is observed between the two advertising efforts, then each retailer concentrates his advertising flow mainly in the market niche which is not covered by the competitor. On the other hand, if an advertising synergy is observed, then both retailers address their advertising efforts mainly to the intersection of the spectra of the two marginal revenue functions.

3 All computations are described in the Mathematica notebook gv2015_b.nb attached as supplementary content to this paper.

Almeder, C., Caulkins, J. P., Feichtinger, G., & Tragler, G. (2004). An age-structured single-state drug initiation model-cycles of drug epidemics and optimal prevention programs. Socio-Economic Planning Sciences, 38(1), 91–109. doi:10.1016/ S0 038-0121(03)0 0 030-2. Anit¸ a, S., Arna˘ utu, V., & Capasso, V. (2011). An introduction to optimal control problems in life sciences and economics. Modeling and Simulation in science, engineering and technology. New York: Birkhäuser/Springer. doi:10.1007/ 978- 0- 8176- 8098- 5. Bressan, A. (2011). Noncooperative differential games. Milan Journal of Mathematics, 79(2), 357–427. doi:10.10 07/s0 0 032- 011- 0163- 6. Dockner, E., Jørgensen, S., Van Long, N., & Sorger, G. (20 0 0). Differential games in economics and management science. Cambridge: Cambridge University Press. doi:10.1017/CBO9780511805127. Engwerda, J. C. (2005). LQ dynamic optimization and differential games. Chichester: John Wiley & Sons. Faggian, S., & Grosset, L. (2013). Optimal advertising strategies with age-structured goodwill. Mathematical Methods of Operations Research, 78(2), 259–284. doi:10. 10 07/s0 0186-013-0443-8. Feichtinger, G., Hartl, R., Kort, P. M., & Veliov, V. M. (2006). Anticipation effect of technological progress on capital accumulation: A vintage-capital approach. Journal of Economic Theory, 126(1), 143–164. doi:10.1016/j.jet.2004.10.001. Feichtinger, G., Tragler, G., & Veliov, V. (2003). Optimality conditions for agestructured control systems. Journal of Mathematical Analysis and Applications, 288(1), 47–68. doi:10.1016/j.jmaa.20 03.07.0 01. Grass, D., Caulkins, J. P., Feichtinger, G., Tragler, G., & Behrens, D. A. (2008). Optimal control of nonlinear processes. Springer-Verlag, Berlin. doi:10.1007/ 978- 3- 540- 77647-5. Grosset, L., Roberti, P., & Viscolani, B. (2011). A goodwill model with predatory advertising. Operations Research Letters, 39(6), 419–422. doi:10.1016/j.orl.2011.10. 001. Grosset, L., & Viscolani, B. (2005). Advertising for the introduction of an agesensitive product. Optimal Control Applications and Methods, 26(3), 157–167. doi:10.1002/oca.758. Haurie, A., Krawczyk, J. B., & Zaccour, G. (2012). Games and dynamic games. Singapore: World Scientific Publishing. doi:10.1142/8442. Huang, J., Leng, M., & Liang, L. (2012). Recent developments in dynamic advertising research. European Journal of Operational Research, 220(3), 591–609. doi:10.1016/ j.ejor.2012.02.031. Ichikawa, A. (1976). Linear quadratic differential games in a Hilbert space. SIAM Journal on Control and Optimization, 14(1), 120–136. http://epubs.siam.org/doi/abs/ 10.1137/0314010.

278

L. Grosset, B. Viscolani / European Journal of Operational Research 254 (2016) 269–278

Jørgensen, S., Martín-Herrán, G., & Zaccour, G. (2010). The Leitmann–Schmitendorf advertising differential game. Applied Mathematics and Computation, 217(3), 1110–1116. doi:10.1016/j.amc.2010.01.047. Jørgensen, S., & Zaccour, G. (2014). A survey of game-theoretic models of cooperative advertising. European Journal of Operational Research, 237(1), 1–14. doi:10.1016/j.ejor.2013.12.017. Krastev, V. Y. (2013). Arrow-type sufficient conditions for optimality of agestructured control problems. Central European Journal of Mathematics, 11(6), 1094–1111. doi:10.2478/s11533- 013- 0219- 7. Leitmann, G., & Schmitendorf, W. (1978). Profit maximization through advertising: a nonzero sum differential game approach. IEEE Transactions on Automatic Control, 23(4), 645–650. doi:10.1109/TAC.1978.1101794.

Prasad, A., & Sethi, S. P. (2009). Integrated marketing communications in markets with uncertainty and competition. Automatica, 45(3), 601–610. doi:10.1016/j. automatica.2008.09.018. Reluga, T. C., & Li, J. (2013). Games of age-dependent prevention of chronic infections by social distancing. Journal of Mathematical Biology, 66(7), 1527–1553. doi:10.10 07/s0 0285- 012- 0543- 8. Roxin, E. O. (1977). Differential games with partial differential equations. In Proceedings of workshop on differential games and applications, Enschede, 1977 (pp. 186–204, Vol. 3). Berlin: Springer.Lecture Notes in Control and Information Science Simon, C., Skritek, B., & Veliov, V. M. (2013). Optimal immigration age-patterns in populations of fixed size. Journal of Mathematical Analysis and Applications, 405(1), 71–89. doi:10.1016/j.jmaa.2013.03.061.