Some aspects of decision making under uncertainty

Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632 www.elsevier.com/locate/jspi

Some aspects of decision making under uncertainty Ekaterina V. Bulinskaya Faculty of Mathematics and Mechanics, Moscow State University, GSP-2, 119992 Moscow, Russian Federation Received 17 February 2006; accepted 15 May 2006 Available online 14 January 2007

Abstract Two discrete-time insurance models are studied in the framework of cost approach. The models being non-deterministic one deals with decision making under uncertainty. Three different situations are investigated: (1) underlying processes are stochastic however their probability distributions are given; (2) information concerning the distribution laws is incomplete; (3) nothing is known about the processes under consideration. Mathematical methods useful for establishing the (asymptotically) optimal control are demonstrated in each case. Algorithms for calculation of critical levels are proposed. Numerical results are presented as well. © 2007 Elsevier B.V. All rights reserved. MSC: Primary, 93E99;; secondary 91B30; 93C41; 62G30 Keywords: Input–output models; Asymptotically optimal control; Insurance; Dynamic programming; Stability; Empirical measures

1. Introduction It is well known that in order to study any real-life system one has to construct its mathematical model. Since there exist a lot of models more or less precisely describing the system, an important question is how to choose an appropriate model, see, e.g., Bellman (1957). One necessary condition is certainly the model’s stability and we tackle it in the paper. On the other hand, the same mathematical model can be used for investigation of processes arising in various applications of probability theory such as inventory, storage, queueing, reliability theory, actuarial sciences or finance, as well as in many others. To this end, one only needs different interpretations of input, output, control and state processes, see, e.g., Prabhu (1980) and Bulinskaya (2003). Thus, methods productive in one research domain turn out to be of interest in others. To formulate optimization problems it is necessary to specify an objective function L measuring the performance quality of the system. The choice of objective function determines the approach used by a researcher. The most frequently used approaches are cost and reliability ones. In the former case one aims at maximizing profits or minimizing losses incurred by system performance. That means, the purpose of investigation is to obtain an optimal control U ∗ providing the desired extremum of L(U ). In the latter case the system quality is usually measured by the probability of its uninterrupted functioning. However it is almost always impossible to attain the maximal value of probability, equal to one, by any control. Hence, one has to be satisfied by establishing the so-called -optimal control U which gives, E-mail address: [email protected]. 0378-3758/$ - see front matter © 2007 Elsevier B.V. All rights reserved. doi:10.1016/j.jspi.2006.05.015

2614

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

for a small  > 0, L(U ) sup L(U ) − .

(1)

U

Obviously, an -optimal control may also be of interest in the framework of cost approach. It is determined by (1) if L(U ) represents profit under control U, whereas for the case of loss L(U ) inf L(U ) + . U

Another possibility, in the framework of reliability approach, is maximization of the expected time until the system’s failure or ruin, see, e.g., Bulinskaya (1998). The reliability approach still dominates in actuarial sciences, see e.g., Asmussen (1994) and De Vylder (1996), whereas the cost approach prevailed in inventory theory from the beginning, see, e.g., Arrow et al. (1958) and Bulinskaya (2005a). We consider below only discrete-time processes in the framework of cost approach. The paper is organized as follows. In Section 2 we investigate the optimal control for the case of known input and output distributions, establishing stability domains and providing calculation algorithms, as well as numerical results. Section 3 is devoted to the case of incomplete information. Stochastic orders turned out to be useful in this situation along with statistical tools. Conclusions and further research directions are presented in Section 4. Almost all the proofs are given in Appendix. 2. The case of known distributions The most frequently used method of obtaining an optimal control in this case is dynamic programming. Its fruitfulness in inventory theory was firmly established by the end of last century, see e.g., Chikán (1986). Usefulness of its application in finance and insurance was recently demonstrated, e.g., in Afanasieva and Bulinskaya (2004) and Bulinskaya (2003, 2004a, 2005c), see also references therein. We concentrate here on systems’ asymptotic behaviour, some algorithms and calculation problems. The insurance model introduced in Bulinskaya (2003) is used for illustration. (Its description is given in the next subsection.) At the same time we generalize the results stated in Bulinskaya (2005b). 2.1. Model description Performance of insurance company is investigated under the following assumptions. We consider a discrete-time process taking into account only the company state by the end of each year. By the beginning of a year the company can borrow or invest money, the corresponding interest rates being r1 and r2 . Let r3 be the interest rate in case of emergency loans for payment of unsatisfied claims by the end of the year, while  (0 <  1) denotes the discount factor and  stands for inflation rate (or lost investment opportunities). For simplicity, all the parameters r1 , r2 , r3 ,  and  are supposed to be known constants. Furthermore, let n be the planning horizon, whereas l denotes aggregate claims amount (output) and l is a premium amount (input) acquired during the lth year, l = 1, n. In contrast to classical models l may be random. The difference l = l − l represents the company losses in the year l if l > 0, respectively, |l | is the company profit if l < 0. The random variables l , l 1, called sometimes “net loss”, are supposed to be independent identically distributed, possessing a finite expectation and  t a density  which is positive in some finite or infinite interval (a, b) and equal to zero otherwise. Denote F (t) = −∞ (s) ds and F¯ (t) = 1 − F (t). Let Ul and Rl+1 be the cash amounts at the beginning and at the end of the lth year, respectively, R1 = x being the initial capital. It is natural to call Ul the company decision at the beginning of the lth year. In fact, making the decision to borrow the money amount y > 0 one obtains Ul = Rl + y. On the other hand, the cash amount Ul available (before premium payment) for meeting the claims is equal to Rl − z if the decision is to invest z > 0. Thus the decision to have the cash amount Ul by the beginning of the lth year entails the loss r1 (Ul − Rl ) if Ul > Rl or the gain r2 (Rl − Ul ) if Ul < Rl . It is clear that Rl+1 = Ul − l . If Rl+1 < 0, that is, the cash amount Ul plus the premium acquired is not enough to meet all the claims, the company has to get an emergency loan equal to |Rl+1 |, at interest rate r3 , or to pay a penalty r3 (l − Ul ). (Such a penalty may be also due to deposit withdrawal or assets selling at inappropriate time.) Another possibility Rl+1 > 0 entails a loss (Ul − l ) due to inflation. It can be also considered as a lost investment opportunity since the money amount left, that is, Rl+1 could have been invested by the beginning of the lth year.

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2615

The total expected discounted costs Ln (U (n) ) associated with company’s n years decision U (n) = (U1 , . . . , Un ) can be chosen as an objective function. Assuming that all the costs relating to the lth year are incurred by the end of the year, Ln (U (n) ) can be written as follows: E

n 

l [r1 (Ul − Rl )+ − r2 (Rl − Ul )+ + r3 (l − Ul )+ + (Ul − l )+ ].

l=1

The aim is to minimize these costs. More precisely, we are going to obtain fn (x) = −1 minU (n) ∈Rn Ln (U (n) ) and the decision U (n)∗ (x) letting to attain this minimum, for initial capital x and planning horizon n. The function fn (x) will be called the minimal n-step expected discounted costs. According to Bellman’s optimality principle fn (x), for n1, satisfy the following recurrent relations    +∞ fn (x) = min r1 (u − x)+ − r2 (x − u)+ + L(u) +  fn−1 (u − s)(s) ds (2) u

with

 L(u) = 

−∞

u −∞



+∞

(u − s)(s) ds + r3

(s − u)(s) ds,

f0 (x) ≡ 0.

(3)

u

Denote by u∗n (x) the value of u providing the minimum of the right-hand side in (2).As usual in dynamic programming, ∗ un (x) characterizes the optimal company’s behaviour (decision) at the first step of the n-step process, see e.g., Bellman (1957) and Bulinskaya (2004a). In other words, for given x and n, U1∗ (x) = u∗n (x), the next decision, at the beginning of the second year, is determined by the system state, that is, U2∗ (x) = u∗n−1 (U1∗ (x) − s) if 1 = s and so on. 2.2. Dependence of optimal policy on system’s parameters We begin by generalizing two results proved in Bulinskaya (2004a).  m l l ∗ Theorem 1. Let r3 > r2 and r3 m−1 l=0  r1 < r3 l=0  . Then, for n m, the optimal behaviour un (x) is determined by one parameter xn2 in a following way  x, x xn2 , ∗ (4) un (x) = xn2 , x > xn2 , whereas, for nm + 1 the optimal behaviour is determined by two constants xnk , k = 1, 2 (xn1 < xn2 ) and has the form ⎧ x , x < xn1 , ⎪ ⎨ n1 ∗ (5) un (x) = x, x ∈ [xn1 , xn2 ], ⎪ ⎩ xn2 , x > xn2 . If r1 > r3 /(1 − ) the optimal behaviour has the form (4) for all n. Corollary 1. The critical levels xn2 introduced in Theorem 1 are given by formulas x12 = H1inv (b2 (0)),

xn2 = Hninv (b2 ()),

(6)

n2.

Here H inv (·) denotes the function inverse to H (·), H1 (u) = F¯ (u). For 1 n m, functions Hn (u) satisfy recurrent relations  +∞ ¯ ¯ Hn+1 (u) = F (u) − b2 ()F (u − xn2 ) +  Hn (u − s)(s) ds,

(7)

u−xn2

whereas, for n > m, Hn+1 (u) has the form  ¯ ¯ ¯ F (u) +  Hn (xn1 )F (u − xn1 ) − Hn (xn2 )F (u − xn2 ) +

u−xn1 u−xn2

and bk () = (rk +  − r2 )/(r3 + ), 0  1, k = 1, 2.

Hn (u − s)(s) ds

(8)

2616

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

The sequence xn1 , n > m, given by xn1 = Hninv (b1 ()), is non-decreasing and has a finite limit x1 under assumptions of Theorem 1, while the non-decreasing sequence xn2 , n 1, has a finite limit x2 under an additional assumption r2 > r1 − . The proof of Theorem 1 and Corollary 1 is carried out by induction. Denoting  +∞ fn−1 (u − s)(s) ds Gnk (u) = rk u + L(u) +  −∞

it is possible to rewrite (2) in the form fn (x) = min(fn1 (x), fn2 (x)) where fn1 (x) = −r1 x + min Gn1 (u), ux

fn2 (x) = −r2 x + min Gn2 (u). ux

It can be established that, for n m + 1, there exist unique solutions xnk of equations Gnk (u) = 0, k = 1, 2, whereas, for n m, this is true only for k = 2. Thus one gets the base for proving (4) and (5). Inequalities Ak1 (u)Gnk (u) Ak2 (u) valid for n m + 2, with Akl (u) = rk − rl − r3 + (r3 + )F (u), k, l = 1, 2, are useful for studying the properties of sequences xnk , k = 1, 2. Since Gnk (u) = (r3 + )[bk () − Hn (u)], it is not difficult to get (6)–(8). For details see Appendix. Remark 1. To understand clearly the further reasoning it is advisable to read Appendix. Then it will be obvious that results established in Bulinskaya (2004a) are particular cases of Theorem 1 and Corollary 1 corresponding to m = 0, r1 > r2 . In other words, the optimal behaviour for the most interesting (from the practical point of view) case r3 > r1 > r2 has the form (5). Moreover, an assumption r2 > r3 leading to the conclusion x12 = −∞, is unrealistic since it means that there are no penalties for a deposit withdrawal or assets selling at an inappropriate time. Next we formulate a generalization of the result proved in Bulinskaya (2005c). Theorem 2. Assume r3 > r2 > r1 > r2 − . Then the optimal behaviour at the first step of n-step process is characterized by three critical levels xn2 < xn∗ < xn1 in a following way  xn1 , x < xn∗ , u∗n (x) = (9) xn2 , x > xn∗ . There exists x ∗ = limn→∞ xn∗ . The proof is in Appendix. We mention here only that Gnk (xnk ) = 0 and xn∗ = (r2 − r1 )−1 [Gn2 (xn2 ) − Gn1 (xn1 )]. At last we consider the case which turned out to be the simplest. Theorem 3. Suppose r3 > r2 = r1 , then the optimal behaviour at the first step of the n-step process is characterized by a single critical level x˜n in a following way: u∗n (x) = x˜n for any x. Moreover, x˜1 = F¯ inv (b1 (0))

(10)

and, for n2, x˜n = x˜ where x˜ = F¯ inv (b1 ())

(11)

with b1 () defined in Corollary 1. The proof is also in Appendix. Remark 2. Thus, we have established the stability domains of our model. Firstly, we have to consider only the case r2 < r3 . Secondly, the optimal behaviour at the first step of n-step process is determined by three parameters if r2 > r1

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2617

and by two parameters if r2 < r1 < r3 , whereas for r1 = r2 it depends only on one parameter. Thirdly, the region r1 > r3 is subdivided in parts where the optimal behaviour may be determined by one or two parameters according to relations between n and m. 2.3. Asymptotic behaviour Although we can provide an optimal control for any set of feasible parameters r1 , r2 , r3 , ,  and each n 1, there can arise some difficulties in its practical implementation. In fact, the main drawback of the dynamic programming is the necessity to know in advance the planning horizon n and impossibility to change it in the future. Moreover, in order to obtain the critical levels at the first step of n-step process one has to calculate all those pertaining to the future steps. Fortunately, after the study of systems limit behaviour, as n → ∞, we can construct the so-called stationary asymptotically optimal policies.  = (U (n) , n1) is asymptotically optimal if Definition 1. A policy U lim n−1 fn (x) = lim n−1 fn (x). n→∞

n→∞

(n) . Here fn (x) stands for the costs incurred under the control U Stationarity will mean that the decision made at any step depends neither on the length of the planning horizon nor the step of the process under consideration. Theorem 4. Under assumptions of Theorems 1–3, for 0 <  < 1, there exist f (x) = limn→∞ fn (x) and H (x) = limn→∞ Hn (x). The function f (x) is the solution of the following equation    +∞ f (x) = min r1 (u − x)+ − r2 (x − u)+ + L(u) +  f (u − s)(s) ds . (12) u

−∞

The proof is in Appendix. Turning to the case =1 (undiscounted costs) we establish the asymptotic optimality of a stationary policy depending (l) on two parameters x1 and x2 . Denote by fn (x) the n-step expected costs if the critical levels x1 and x2 are used at the first l steps, the optimal levels xqk , k = 1, 2, q n − l, defining the company behaviour during the last n − l steps. (0) Hence, the minimal expected costs fn (x) (under the optimal policy) are equal to fn (x), whereas the expected costs (n) fn (x) incurred under the stationary (x1 , x2 )-policy are, by definition, fn (x). Theorem 5. Under assumptions of Theorem 1 with 0 <  1 and m < ∞ lim n−1 fn (x) = lim n−1 fn (x).

n→∞

n→∞

(13)

The proof can be divided in three lemmas. Denote Fn (t) = P (Rn t) and (F, G) = supt∈(−∞,+∞) |F (t) − G(t)|.  for the sequence of random variables Rn , n1. Lemma 1. Under (x1 , x2 )-policy, there exists the limit distribution F ) Chn where h = x2 −x1 (s) ds and C is a constant. Moreover, (Fn , F x1 −x2 Proof. In our model the surplus process R undergoes changes only in discrete moments, therefore (applying (x1 , x2 )policy) we have the recurrent relations ⎧ x − n , Rn < x1 , ⎪ ⎨ 1 Rn+1 = Rn − n , Rn ∈ [x1 , x2 ], (14) ⎪ ⎩ x2 − n , Rn > x2 .

2618

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

Since Rn+1 = Un − n and {Un }n  1 form a Markov chain (random walk with two impenetrable boundaries) existence  is almost obvious. Moreover, after some transformations one obtains of the limit distribution F  x2 ¯ Fn+1 (t) = F (x2 − t) + Fn (u)(u − t) du x1

therefore (Fn+1 , Fn )  n = maxt∈[x1 ,x2 ] |Fn (t) − Fn−1 (t)|. In its turn n h n−1 thus leading to the desired result.  Lemma 2. There exists limn→∞ n−1 fn (x). Proof. The n-step expected costs under (x1 , x2 )-policy have the form fn (x) = E

n 

[V1 (Rm ) + V2 (Rm+1 )]

m=1

with V1 (Rm ) = r1 (x1 − Rm )+ − r2 (Rm − x2 )+ ,

+ V2 (Rm+1 ) = Rm+1 + r3 (−Rm+1 )+ .

Due to Lemma 1 and uniform integrability of V1 (Rm ) and V2 (Rm+1 ), m 1, the right-hand side of (13) exists and .  is equal to E[V1 (R) + V2 (R)]. Here R stands for a random variable having the distribution F Lemma 3. For all x, limn→∞ n−1 (fn (x) − fn (x)) = 0. (l)

Proof. According to definition of fn (x), l = 0, n, it is possible to write n 

fn (x) − fn (x) =

(fn(l) (x) − fn(l−1) (x)).

(15)

l=1

Taking into account convergence of xnk to xk , as n → ∞, k =1, 2, we find for any  > 0, such n0 () that, for l n−n0 (), each item in the right-hand side of (15) has the absolute value less than /2. The other ones being uniformly bounded it is not difficult to obtain the desired result.  2.4. Calculation algorithms Thus, the stationary policy using at each step xk instead of xnk , n 1, k = 1, 2, was proved to be asymptotically optimal. That is, the long-run one-step average costs are the same under this policy and optimal one. Therefore, it is of interest to obtain an explicit form of xk , k = 1, 2. To this end we introduce some notations. Let bk () be as in Corollary 1, ck () = (r3 + r1 − rk )/(r3 + ), g(t) = F (t) +  n (t) =

∞ 

n n (t),

n=1



x2

∞ 

n ¯ n (t),

(16)

n=1 x2

... x1

g(t) ¯ = F¯ (t) +

F (t − v1 )(v1 − v2 ) . . . (vn−1 − vn )(vn ) dv1 . . . dvn

(17)

x1

and ¯ n (t), n 1, have the form (17) with F¯ (t − v1 ) instead of F (t − v1 ). Theorem 6. Under assumptions of Theorem 1 with m < ∞ the critical levels xk , k = 1, 2, are given either by g(xk ) = ck () or by g(x ¯ k ) = bk (). Proof. It is easily seen that Hn (u), n 1, are non-increasing in u, there exist hn (u) = Hn (u) and Hn (−∞) = 1 + (r1 − r2 )(r3 + )−1 , Hn (+∞) = 0. The uniform, in u ∈ (−∞, +∞), convergence of Hn (u) to H (u), as n → ∞, follows

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2619

from Theorem 4. Hence, due to existence of xk = limn→∞ xnk , k = 1, 2, one obtains from (8) the following integral equation

 u−x1 ¯ ¯ ¯ H (u) = F (u) +  b1 ()F (u − x1 ) − b2 ()F (u − x2 ) + H (u − s)(s) ds (18) u−x2

and H (xk ) = bk (). Using (18) it is not difficult to prove that H (u) is differentiable and its derivative satisfies the integral equation  x2 h(u) = −(u) +  h(s)(u − s) ds x1

which can be rewritten as 

∞ 

h(u) = − (u) +

 n n (u)

(19)

n=1

with





x2

n (u) =

x2

... x1

(u − v1 ) . . . (vn−1 − vn )(vn ) dv1 . . . dvn .

x1

So one gets the desired result by integrating both parts of (19) and taking into account the form of H (−∞) or H (+∞).  It will be shown in the next subsection that for some distributions it is easy to get xk , k = 1, 2, explicitly as functions of systems parameters. For other distributions the critical levels can be obtained only numerically, using Theorem 6. If the distribution F of m is such that it is impossible to obtain g(t) or g(t) ¯ in a closed form one can use the following two basic algorithms for calculation of xk , k = 1, 2. (0)

(0)

(k)

(0)

Algorithm 1. Let the initial approximation xk be the solution of F (xk ) = c . That means xk = zk1 xk (see (n) Appendix, proof of Theorem 1). Then further approximations xk , n 1, k = 1, 2, are obtained recursively as solutions  (n) (n−1) (t), with of equations gn (xk ) = ck (). Here we put gn (t) = F (t) + nq=1 q q  (l) q (t) =



(l)

x2 (l)

...

x1

(l)

x2 (l)

F (t − v1 )(v1 − v2 ) . . . (vq−1 − vq )(vq ) dv1 . . . dvq .

x1

(n)

(0)

Functions gn (t) are non-decreasing in t and non-negative. Thus, for any n 1, one has xk < xk . (l) Moreover, one can simplify calculations introducing auxiliary functions q (t, u) of two variables t and u in a following way (l) 0 (t, u) = F (t

 − u),

(l) q (t, u) =

(l)

x2 (l) x1

(l)

q−1 (t, v)(v − u) dv,

q 1,

(20)

then gn (t) =

n 

q (l) q (t, 0).

(21)

q=0 (0) (0) Algorithm 2. Another possibility is to choose the initial approximation x¯k as solution of F¯ (x¯k ) = bk (), in other (n) (n) (0) words xk = zk2 . Further approximations x¯k are solutions of equations g¯ n (x¯k ) = bk (). The functions g¯ n (t) have the (n) (0) same structure as gn (t) with F¯ (·) instead of F (·). Hence, g¯ n (t) are non-increasing and xk > xk . Expressions similar to (20) and (21) are valid (with obvious changes) for g¯ n (t).

2620

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

 (u) as a sum of To emphasize the simplicity of both algorithms we give below an expression for hn+1 (u) = Hn+1 multiple integrals   n  l (n)  l (u) , hn+1 (u) = − (u) + l=1

where, for l < n − m + 1,  xn2  (n) l (u) = ... xn1

xn−l+1,2

(u − v1 ) . . . (vl−1 − vl )(vl ) dv1 . . . dvl ,

xn−l+1,1

and, for l n − m + 1, their form is more complicated  xn2  xm+1,2  xm2  xn−l+1,2 (u − v1 ) . . . (vl−1 − vl )(vl ) dv1 . . . dvl . ... ... xn1

xm+1,1

−∞

−∞

Hence, such a simple representation as in (20)–(21) is clearly impossible for Hn (u). 2.5. Some examples and numerical results We consider below three different distributions of net loss per year, namely, two-side exponential, shifted exponential and Pareto. Example 1. Suppose that for some 0 <  < 1, > 0,  > 0   t  e s , s < 0, t < 0, e , (s) = F (t) = −t , t > 0. s −  1 − (1 − )e (1 − )e , s > 0, Then

 L(u) =

−r3 (u +  −1 − (1 − )−1 ) + (r3 + ) −1 e u ,

u < 0,

(u +  −1 − (1 − )−1 ) + (1 − )(r3 + )−1 e−u , u > 0.

It is clear that, according to (36), x1k < 0 if  > k = (r3 − rk )(r3 + )−1 . Thus, we have three different situations, namely, both parameters x11 , x12 are either negative or positive and one of them is negative whereas the other one is positive. For example, if  > max(1 , 2 ) then x1k = −1 (ln(r3 − rk ) − ln[(r3 + )]), and if r2 > r1 x1∗

=

−1



k = 1, 2

r3 − r 1 r 3 − r2 ln(r3 − r1 ) − ln(r3 − r2 ) − ln[(r3 + )] − 1 . r2 − r 1 r2 − r 1

(22)

Although one can obtain xnk , k = 1, 2, xn∗ and fn (x), n1, in explicit form for all the combinations of system parameters, their expressions are too cumbersome and therefore omitted. For illustration we produce two graphics. Thus, Fig. 1 represents G2k (u), k = 1, 2, and their upper and lower bounds Akl (u), l = 1, 2, for  = 0.5,  = 0.3, r1 = 0.5, r2 = 0.2, r3 = 0.7,  = 0.6, = 0.1,  = 0.2. It is important to recall that Gnk (u), for all n > 2, are lying between G2k (u) and Ak1 (u). The next Fig. 2 represents the same curves as in Fig. 1 under assumption r2 = 0.5 and r1 = 0.2, that is, r2 > r1 , (other parameters being the same). It is interesting to compare the pictures and see the difference in their configuration. Example 2. Suppose now that  0, s < − c, (s) = −  (s+c) e , s > − c,

 F (t) =

0,

t < − c,

1 − e−(t+c) ,

t  − c.

(23)

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2621

0.6 0.4 0.2

-40

-20

20

40

20

40

-0.2 -0.4 -0.6

Fig. 1. Case r1 > r2 .

0.6 0.4 0.2

-40

-20 -0.2 -0.4 -0.6

Fig. 2. Case r1 < r2 .

Such a choice of the density means that the premium amount during a year is equal to c, whereas the claims amount is exponentially distributed with parameter . Obviously the most interesting case is c > 1, that is, the expected payment of insurance company is less than the corresponding premium amount. Denoting d = ln(r1 +  − r2 ) − ln(r2 +  − r2 ), it is easy to verify that this function is non-decreasing in  for  0, therefore d d0 = ln(r1 + ) − ln(r2 + ). Suppose r3 > r1 > r2 > r1 − , then using Corollary 1, we calculate x1k = −−1 ln bk (0) − c. It is clear that −c < x11 < x12 and (x12 − x11 ) = d0 . Next, assuming c > d we obtain, for all n 1 and any uxn1 , Hn (xn1 )F¯ (u − xn1 ) − Hn (xn2 )F¯ (u − xn2 ) = 0

(24)

thus the relation (8) is significantly simplified  u−xn1 Hn+1 (u) = F¯ (u) +  Hn (u − s)(s) ds,

uxn1 ,

n 1.

u−xn2

Finally, for n 3 and u xn−1,1 , Hn (u) = F¯ (u)(1 + an ),

an = g(1 + an−1 ),

Therefore, xnk = −1 [ln(1 + an ) − ln bk ()] − c.

a2 = d0 e−c ,

g = d e−c .

2622

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

It is not difficult to find the limits xk = limn→∞ xnk , k = 1, 2, and the convergence rate, since (1 + an ) → (1 − g), as n → ∞. Namely, the following relations take place xk = − −1 [ln b + ln(1 − g)] − c, (k)

xk − xnk = − −1 ln(1 − g n−2 (a3 − a2 )),

(25)

in other words, xk − xnk = O(g n ) as n → ∞. Note also, that xk < zk1 = −1 [ln(r3 + ) − ln(rk +  − r1 )] − c. For this distribution, g(t) ¯ can be obtained explicitly. Thus  ¯ n (t) =



x2

x2

... x1

n e−nc e−(t+c) dv1 . . . dvn = v n e−(t+c)

x1

with v = e−c (x2 − x1 ). Whence it follows that g(t) ¯ = (1 − v)−1 e−(t+c) and v = d e−c . Finally, xk = −−1 [ln(1 − v) + ln bk ()] − c, this expression coinciding with (25). Obviously, the second method (based on Theorem 6) of getting the desired result (25) is shorter and easier. (n)

(n)

Remark 3. For a shifted exponential distribution (23), it is also easy, using Algorithms 1 and 2, to get xk and x¯k in a closed form and their convergence to xk , as n → ∞, is very fast. Numerical calculations show that in the most cases there is no need to perform many iterations in order to obtain the limit values. Moreover, the same is true not only for the exponential distribution, but for Pareto one with a density (s) = aa (s +  + c)−(a+1) , s  − c, as well. Below one can see some numerical results using Algorithm 1. The first column of each table provides the planning exp exp horizon n, the second and the third ones contain the critical levels xn1 and xn2 , respectively, for the exponential par par distribution, whereas the critical values xn1 , xn2 for Pareto distribution with the same mean as in exponential case are given in columns four and five, respectively. 1. Assume  = 1,  = 0.3, c = 10,  = 0.5, r1 = 0.04, r2 = 0.02, r3 = 0.05, a = 1.5,  = 1, then exp

exp

par

par

n

xn1

xn2

xn1

xn2

1 2 3

−9.942025 −9.819959 −9.819906

−9.820776 −9.690882 −9.690828

−9.980484 −9.937258 −9.937257

−9.938437 −9.890460 −9.890459

2. Next, put  = 1,  = 0.3, c = 5,  = 0.3, r1 = 0.2, r2 = 0.1, r3 = 0.3, a = 1.3,  = 1, then exp

exp

par

par

n

xn1

xn2

xn1

xn2

1 2 3 4 5 6 7

−4.392262 −4.486482 −4.431072 −4.427547 −4.427321 −4.427306 −4.427305

−3.648450 −2.527542 −2.472132 −2.468606 −2.468380 −2.468366 −2.468365

−4.849442 −4.608647 −4.608596 −4.608596 −4.608596 −4.608596 −4.608596

−4.633987 −4.256231 −4.256148 −4.256148 −4.256148 −4.256148 −4.256148

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2623

0.6 0.4 0.2

-40

-20

40

20 -0.2 -0.4 -0.6

Fig. 3. Case r1 < r2 .

3. At last we take  = 0.5,  = 0.1, c = 2,  = 1, r1 = 0.2, r2 = 0.1, r3 = 0.3, a = 3,  = 2, then exp

exp

par

par

n

xn1

xn2

xn1

xn2

1 2 3 4 5 6

−1.712318 −1.502929 −1.495098 −1.494829 −1.494819 −1.494818

−1.306853 −0.992103 −0.984272 −0.984002 −0.983993 −0.983993

−1.798715 −1.649730 −1.648267 −1.648240 −1.648240 −1.648240

−1.480158 −1.209000 −1.207186 −1.207106 −1.207106 −1.207106

For completeness sake, we treat here also the case r2 > r1 , see Fig. 3, showing that Gnk (u) may be non-monotone in this case. The explicit form of control parameters xnk , k = 1, 2, and xn∗ , n 1, are given below along with their limits xk , k = 1, 2, and x ∗ . Introduce  = −1 − c and, for 0  1, q = −1 (r2 − r1 )−1 [(r1 − r2 + ) ln b1 () − (r2 − r2 + ) ln b2 ()]. It is obvious that x1k , k = 1, 2, have the same form as in the case r1 > r2 , although now x12 < x11 . Under assumption x11 < 0, x1∗ =  + q0 . Further, one obtains the following recurrent relations: xn∗ = −1 ln n−1 +  + q ,

∗

n = 1 − ϑexn

and

xnk = −1 (ln n−1 − ln bk ()) − c

with ϑ = (r2 − r1 )(r3 + )−1 . Hence there exist the limit values  = (1 + )−1 ,

x ∗ =  + q − −1 ln(1 + ),

xk = −1 (ln  − ln bk ()) − c

with  = ϑe+q . 3. The case of incomplete information Procedure to follow in this case depends on the type and amount of information in our possession. One possibility is the distribution known up to parameters, in other words, F (·) belongs to some parametric family. Such a situation was considered in Scarf (1959) for a single-product inventory model with periodic ordering and gamma distributed demand. A combination of dynamic programming and Bayes approach was used for obtaining an optimal control. The second case is only several first moments of the distribution are known. Some static inventory models were considered in Girlich (1973). Usefulness of stochastic orders technique for investigation of dynamic inventory models was demonstrated in Bulinskaya (2004b).

2624

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

The most frequently arising situation is no a priori knowledge about the distribution. A useful approach in such a case is to apply the empirical measures, see, e.g., Bulinskaya (1990). 3.1. Parametric families We begin by considering the distribution of Example 2, which depends on two parameters c and . The first parameter is usually known, whereas the second one can be estimated on the base of claims amounts 1 , . . . , n observed during n n (n) −1  l ) in (25) instead of  we obtain corresponding approximations  xk converging, previous years. Putting n =n( l=1  according to the strong law of large numbers, almost surely to xk , as n → ∞. It is reasonable to operate in a similar way, whenever xk , k = 1, 2, are explicit functions of parameters, namely, use their estimates. In order to establish the convergence rates it is advantageous to apply the technique of limit theorems, see, e.g., Zolotarev (1997). Next, let the distribution F, depending on unknown parameters, be such that we are unable to obtain xk , k = 1, 2, in explicit form. Nevertheless, combining the basic algorithms with statistical estimation of parameters one can construct an asymptotically optimal control in this case as well. Details are omitted however we give below some numerical results for Pareto distribution. Namely, a sample of size 100 was produced by simulation for a Pareto distributed random variable (with known  = 2), then parameter a was estimated using the moments method. The estimate a was put instead of a in Algorithm 1 (the other system parameters being  = 0.5,  = 0.3, r1 = 0.5, r2 = 0.2, r3 = 0.7). In the first column of the table below one finds the number of iterations, in the second and third ones values (n) (n) (n) (n) of approximations  x1 and  x2 , respectively, if c = 10, whereas in columns four and five values of  x1 and  x2 , respectively, if c = 2. Less than 6 iterations were needed to finish the procedure. n

(n)  x1

(n)  x2

(n)  x1

(n)  x2

1 2 3 4 5 6

−9.845564 −9.746820 −9.746828 −9.746828 −9.746828 −9.746828

−9.480158 −9.284290 −9.284305 −9.284305 −9.284305 −9.284305

−1.763932 −1.556021 −1.561463 −1.561562 −1.561565 −1.561565

−1.171573 −0.747663 −0.753304 −0.753466 −0.753469 −0.753469

3.2. Use of stochastic orders for models comparison Another possibility is unknown distribution with fixed m first moments. Technique of stochastic orders turns out to be useful in this situation for comparing the models, obtaining upper and lower bounds and making conservative decisions. We consider two models (of Section 2) assuming (i) with distribution function Fi to stand for the net loss per year (i) in the ith model and denote by fn (x) the corresponding minimal n years expected costs, i = 1, 2, x being the initial capital. Below we establish two results using the stop-loss order (
u

with L(i) (u) = El(u, (i) ), l(u, y) = r3 (y − u)+ + (u − y)+ and f0 (x) ≡ 0. (i)

n 1,

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2625

It is well known, see, e.g., Stoyan (1983) that F1
(26)

for any convex non-decreasing function f such that both expectations exist. Furthermore, if E(1) =E(2) then inequality (26) is valid for any convex f. (1) (2) Therefore L(1) (u)L(2) (u), for all u, and it is easily established that f1 (x) f1 (x), for all x, and both functions are convex. The next induction steps are omitted being obvious.  Suppose now that the distribution law F is not known however its expectation  is given. Theorem 7 provides a lower bound for minimal expected costs fn (x) corresponding to distribution F. (2)

(1)

Corollary 2. Under assumption r3 > r1 > r2 > r1 −  one has fn (x) hn (x) if  < 0 and fn (x) hn (x) if  > 0, with h(k) n (x) = rk 

n−1 

  + l

l=1

r1 ( − x), x < , r2 ( − x), x > ,

k = 1, 2.

Proof. The proof is based on the following property of the stop-loss order: E 0, for s > 0, and a finite mean. For simplicity, we assume the discount factor  = 1. Denote fn (x) the minimal expected costs during n periods, x being the initial capital. These functions satisfy the following recurrent relations fn (x) = −c1 x + min

uv x

(27)

Gn (u, v),

where 

∞

Gn (u, v) = c2 u + (c1 − c2 )v + L(v) +

fn−1 (u − s)(s) ds,

(28)

0

L(v) = E[(v − 1 )+ + r3 (1 − v)+ ]

and f0 (x) ≡ 0.

The parameters u and v define the company decision as follows: if the initial capital is equal to x, sell assets to obtain the amount v − x, the loan being u − v. It is easily seen that we get here an illustration of the fact mentioned in Introduction. That is, the same mathematical model describes an inventory system with two suppliers discussed in Bulinskaya (1990). So we can use some of the results obtained there to demonstrate the reasoning in the case of completely unknown distribution. Note also that if the assets amount possessed by the company is less than the optimal amount to sell, one has to modify the decision. We do not raise this issue here deferring it to later publications. Due to lack of space

2626

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

numerical results are not provided as well. We want only outline the procedure to follow in the case of completely unknown distribution. It was established in Bulinskaya (1990) that there exist three stability domains with different types of optimal behaviour. Thus, if c1 c2 or c1 > c2 + r3 , we obtain a model with only one source of money (with immediate delivery or one-period delay, respectively), whereas the case c2 < c1 c2 + r3 corresponds to involvement of both. We consider here only the last case. Denote by Sn and Kn the partial derivatives of Gn (u, v) in u and v, respectively. It is easily seen from (28) that Sn and Kn are functions of a single variable, moreover, Kn does not depend on n. ˜ To formulate the results we introduce a function L(u) given by  u−v¯ ˜ F (u − s) dF (s) (29) ¯ + (r3 + ) L(u) = c2 − c1 + (c1 − c2 − r3 )F (u − v) 0

with v¯ satisfying the equation Kn (v) = 0, in other words, F (v) ¯ = (r3 + c2 − c1 )/(r3 + ).

(30)

Theorem 8. For a given F (·) the optimal behaviour at the first step of the n-step process is the following: if c2 m/(m − 1) c1 c2 + r3 , m2, (l − 1)r3 < c1 lr 3 , l 1, (and thereby m l and un  v¯ for n m) take u = v = x, for n < l, and v = max(v, ¯ x), u = max(un , x), for nm. The parameters un uniquely defined by the relations Sn (un ) = 0, n m, form a bounded increasing sequence. Its limit u¯ is the unique solution of the equation ˜ L(u) = 0.

(31)

Now denote by v k and uk solutions of (30) and (31), respectively, if F (s) is replaced by Fk (s) and L˜ k (u) is defined by (29) with F and v¯ replaced by Fk and v k , respectively. We use the uniform probability metric (F, Fk ) to investigate the dependence of the optimal policy on the distribution. Corollary 3. Let Fk (u), k 1, be continuous and such that equation Fk (u) = has a unique solution for 0  < 1 and u0. Suppose also that (F, Fk ) → 0, as k → ∞. Then v k → v¯ and uk → u¯ as k → ∞. The stationary (u, ¯ v)-policy ¯  is defined as follows: if xk−1 is the money amount at the beginning of the kth period, k 1, sell assets to get (v¯ − xk−1 )+ , the loan amount being min[(u¯ − v), ¯ (u¯ − xk−1 )+ ]. Denote by fn (x) expected costs incurred during n periods under (u, ¯ v)-policy, ¯ if x0 = x. Theorem 9. Stationary (u, ¯ v)-policy ¯  is asymptotically optimal, that is, lim n−1 fn (x) = lim n−1 fn (x),

n→∞

(32)

n→∞

for all x. In contrast to Section 2 we establish here the explicit expression for the limit in (32). Thus the proof of Theorem 9 is divided in two parts. Lemma 4. For any x there exists limn→∞ n−1 fn (x) equal to  +∞  u− ¯ v¯ [c2 s + L(u¯ − s)](s) ds + [c1 s + (c2 − c1 )(u¯ − v) ¯ + L(v)](s) ¯ ds. d= 0

u− ¯ v¯

(33)

Lemma 5. For any x there exists limn→∞ n−1 fn (x) equal to d defined by (33). Next we study the same model as above with unknown distribution function F (·). Now let i be the observed value of excess within the ith period, i = 1, k, that is, we have k observations. Arranging them in increasing order,

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2627

viz. 1 = min1  i  k i 2  · · · k = max1  i  k i we construct the empirical distribution function F˜k and its k in a following way continuous analog F F˜k (s) = k (s)/k,

k (s) = k −1 F

k 

Ukl (s),

l=1

where k (s) = max{i : i s} and Ukl (s) is the distribution function of the uniform law on (l−1 , l ), l = 1, k, with 0 = 0. (We drop the argument  writing a random variable, as well as argument x if it entails no confusion.) k (s). Using Corollary 3 we obtain immediately. Denote by u¯ k and v¯k the values of uk and v k corresponding to Fk (s)= F Lemma 6. The following relations are valid k ) → 0, as k → ∞) = 1, P ( (F, F

P (u¯ k → u, ¯ v¯k → v, ¯ as k → ∞) = 1.

Let us define an “empirical” policy   as follows: at the first step of the process make no decision at all and if xk−1 is the money amount at the beginning of the kth step, k 2, sell assets to get (v¯k−1 − xk−1 )+ the loan amount being . min[(u¯ k−1 − v¯k−1 ), (u¯ k−1 − xk−1 )+ ]. Now we investigate the behaviour of xn under the policy  Lemma 7. For each  > 0 there exist a subset  and a positive number n() such that P ( ) > 1 −  and |u¯ − u¯ n | < , |v¯ − v¯n | < , xn < u¯ +  for all n n() and  ∈  . n (x) under the policy  At last we deal with the mean costs G . Denoting the costs incurred during the nth period by  n one has  1 = (x − 1 )+ + r3 (1 − x)+ , and, for n > 1,  n is given by (yn − n )+ + r3 (n − yn )+ + c1 (v¯n−1 − xn−1 )+ + c2 min[(u¯ n−1 − v¯n−1 ), (u¯ n−1 − xn−1 )+ ], n (x)= where yn =max(xn−1 , v¯n−1 ) is the money amount ready to meet the nth excess. Then G

n

k=1 dk

where dk =E k .

Lemma 8. The sequence dn converges, as n → ∞, to d defined by (33). The proof can be found in Bulinskaya (1990). Now it is possible to formulate the main result. Theorem 10. The policy   is asymptotically optimal, that is, for any x, n (x) = d. lim n−1 G

n→∞

The proof follows from Lemmas 5 and 8. Since an empirical distribution function is only piecewise continuous, Eqs. (30) and (31) with F (s) replaced by F˜k (s) may have many solutions or no solution at all. However we can define v˜ k and u˜ k as follows: v˜ k = max{v : F˜k (v) (r3 + c2 − c1 )/(r3 + )},

u˜ k = max{u : L˜ k (u)0}.

The maximal jump of F˜k (v) (and L˜ k (u)) tends to zero as k → ∞, therefore we easily reach the conclusion similar to Lemma 6 ¯ v˜ k → v, ¯ as k → ∞) = 1. P (u˜ k → u, The proofs of Lemmas 7–8 remain unchanged for u˜ k and v˜ k , so we obtain Corollary 4. The policy ˜ using (u˜ k , v˜ k ) instead of (u¯ k , v¯ k ) is also asymptotically optimal.

2628

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

4. Conclusions We treated in this paper two discrete-time insurance models. It is important to underline that a balance is struck by the end of a financial year, so it is natural to choose a year as an interval between successive decisions. Effectiveness of dynamic programming in providing the optimal decisions for finite-step processes (in the case of completely known distribution) was demonstrated along with its drawbacks. The advantage of stationary asymptotically optimal policies was established. Some calculation algorithms were proposed. Moreover, it was shown that using the statistical tools to obtain the policies parameters one conserves the asymptotic optimality. Having demonstrated the usefulness of empirical measures for an insurance model with two sources in case of unknown distribution we briefly sketch application of a similar method to the model of Section 2. At first we have to establish the model’s stability. Namely, using (16) we prove that small perturbations of function  entail only small deviations of xk . To formulate precisely these results one employs the probabilistic metrics, see e.g., Zolotarev (1997). The next step is a modification of the basic algorithm providing an asymptotically optimal control. As in Bulinskaya (1990) instead of usual empirical distribution functions we use their continuous analogs. Results of Csörgö et al. (1993) are used for proofs along with properties of probabilistic metrics. Due to the lack of space these results will be published elsewhere along with detailed sensitivity analysis. Acknowledgments The research was partially supported by RFBR Grant 05-01-00256 and the Leading Scientific Schools Grant 4129.2006.1. The author would like to thank her colleagues A. Cherny and A. Manita for the help in drawing graphics. The useful comments of anonymous referees on earlier version of the paper are gratefully acknowledged. Appendix Proof of Theorem 1 and Corollary 1. Integrating by parts in (3) one obtains  u  +∞ L(u) =  F (s) ds + r3 F¯ (s) ds. −∞

(34)

u

Hence it is easily seen that L (u) = F (u) − r3 F¯ (u) =  − (r3 + )F¯ (u) = −r3 + (r3 + )F (u)

(35)

and functions G1k (u) = rk u + L(u), k = 1, 2, are convex and have a unique minimum G1k (x1k ) with x1k given by G1k (x1k ) = 0. According to (35) the last equality can be written as F¯ (x1k ) = ( + rk )/( + r3 )

or F (x1k ) = (r3 − rk )/(r3 + ).

(36)

Thus, for m = 0, that is, r3 > r1 , there exist finite x1k , k = 1, 2, and x11 < x12 under assumption r1 > r2 . Therefore, as in Bulinskaya (2004a)   −r1 x + G11 (x11 ), x < x11 , L(x), x x12 , f11 (x) = f12 (x) = L(x), x x11 , −r2 x + G12 (x12 ), x > x12 . It is clear that in this case  −r1 x + G11 (x11 ), x < x11 , f1 (x) = L(x), x ∈ [x11 , x12 ], −r2 x + G12 (x12 ), x > x12 ,

 f1 (x) =

−r1 , L (x), −r2 ,

x < x11 , x ∈ [x11 , x12 ], x > x12 ,

and one completes the proof by induction, for details see the above mentioned paper. The typical form of f1 (x) is given by Fig. 4, fn (x), n > 1, are similar.

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2629

15 12.5 10 7.5 5 2.5 -20

-10

10

20

Fig. 4. Case r1 > r2 .

We do not consider here the case r2 > r1 deferred to Theorem 2, since for m > 0 it follows automatically r1 > r2 . On the other hand, if m > 0 then only x12 is finite whereas G11 (u) is increasing in u. Whence it follows f11 (x) = L(x) and  L(x), x x12 , f12 (x) = (37) −r2 x + G12 (x12 ), x > x12 , in other words, f1 (x) = f12 (x). Next, for n = 2, we have G2k (u) = G1k (u) − r2 F (u − x12 ) + 



+∞

L (u − s)(s) ds.

(38)

u−x12

It follows immediately from (38) that G2k (u) → rk − r3 − r3 , as u → −∞, and    +∞ G2k (u) = (r3 + ) (u) +  (u − s)(s) ds 0. u−x12

Thus, it is clear that functions G2k (u), k = 1, 2, are convex and there exist finite x2k satisfying G2k (x2k ) = 0 if m = 1, moreover, x21 < x22 and x1k < x2k , k = 1, 2, (with x11 = −∞). In this case  x < x21 , −r1 x + G21 (x21 ), +∞ f2 (x) = L(x) +  −∞ f1 (x − s)(s) ds, x ∈ [x21 , x22 ], (39) −r2 x + G22 (x22 ), x > x22 . Assuming that the optimal behaviour u∗n (x) has the form (5), that is, fn (x) is similar to (39) with obvious change of indices, one easily gets that the same is true for u∗n+1 (x). In fact, since −r1 fn (x)  − r2 the following system of inequalities is valid, for n2, Ak1 (u)Gn+1,k (u) Ak2 (u),

(40)

with Akl (u) = rk − rl − r3 + (r3 + )F (u), k, l = 1, 2, see Fig. 1. It is also interesting to note that Gn+1,k (−∞) = Ak1 (−∞) and Gn+1,k (+∞) = Ak2 (+∞). Obviously A11 (u) − A22 (u) = (r1 − r2 )(1 − ) 0 and the curves can coincide if either r1 = r2 or  = 1, otherwise A11 (u) > A22 (u). That means, for n > m, we have zk2 xnk zk1 with Akl (zkl ) = 0 and z11 z22 . Using the explicit form of Gn+1,k (u) it is possible to establish that Gn+1,k (u)Gnk (u), for all u, entailing xnk xn+1,k . Since z11 is always finite, there exists a finite limit x1 = limn→∞ xn1 . On the other hand, the sequence xn2 is non-decreasing and it is clear that x2 = limn→∞ xn2 < ∞ if z21 < ∞, that is, under additional assumption r2 > r1 − . If m > 1 one gets that only x22 is finite whereas G21 (u) is increasing. Hence,   +∞ L(x) +  −∞ f1 (x − s)s ds, x x22 , f2 (x) = (41) x > x22 , −r2 x + G22 (x22 ),

2630

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

 +∞ and G3k (u) = G1k (u) − F (u − x22 ) +  u−x22 f2 (u − s)(s) ds which gives along with (37), (41) G3k (u) →  rk − r3 2l=0 l , as u → −∞. Thus, G32 (−∞) < 0, for all m, and G31 (−∞) < 0, for m = 2, whereas G31 (−∞) 0 for m > 2. As previously, further reasoning is carried out by induction in n for a fixed m. Note that we have already proved the statement of Corollary 1 concerning the limits of critical levels xnk . To prove the other statements we need to write equations Gnk (u) = 0 in a different form. As follows from (35), if one puts H1 (u) = F¯ (u) then H1 (x12 ) = b2 (0). Rewriting (38) in the form G2k (u) = (r3 + )[bk () − H2 (u)] with  +∞ H2 (u) = F¯ (u) − H1 (x12 )F¯ (u − x12 ) +  H1 (u − s)(s) ds u−x12

one provides the first step for establishing (6)–(8). Further proof is going along the same lines therefore it is omitted.  Proof of Theorem 2. The first step is similar to that in the proof of Theorem 1. Since L (u) = −r3 + (r3 + )F (u) and L (u) = (r3 + )(u), it is clear that G1k (u) has a unique minimum attained at x1k = F inv ((r3 − rk )/(r3 + )), k = 1, 2. According to assumptions about parameters one has x12 < x11 . Hence   x x12 , G11 (x11 ), x < x11 , G12 (x), f11 (x) = −r1 x + f12 (x) = −r2 x + G11 (x), G12 (x12 ), x > x12 . x x11 , Taking into account that G1k (x) − rk x = L(x) we get  x < x12 , G11 (x) − G11 (x11 ), f12 (x) − f11 (x) = G12 (x12 ) − G11 (x11 ) − (r2 − r1 )x, x ∈ [x12 , x11 ], G12 (x12 ) − G12 (x), x > x11 . It is obvious that there exists x1∗ = (G12 (x12 ) − G11 (x11 ))/(r2 − r1 ) such that f12 (x) − f11 (x) > 0, for x < x1∗ , and f12 (x) − f11 (x) < 0, for x > x1∗ . That means,   −r1 x + G11 (x11 ), x x1∗ , −r1 , x < x1∗ ,  f1 (x) = f1 (x) = −r2 x + G12 (x12 ), x x1∗ , −r2 , x > x1∗ . A typical behaviour of f1 (x) is given by Fig. 5, the form of fn (x), n > 1, is similar. In contrast to the case r1 > r2 treated in Theorem 1, here the function f1 (x) is concave, moreover, its derivative is not continuous. It follows immediately that G2k (u) = G1k (u) − r2 F (u − x1∗ ) − r1 F¯ (u − x1∗ )

(42) 6 4 2

-30

-20

10

-10 -2 -4 -6 -8 Fig. 5. Case r1 < r2 .

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

2631

and G2k (−∞) = rk − r3 − r1 < 0, G2k (+∞) =  + rk − r2 > 0, k = 1, 2. That means, equation G2k (u) = 0 has at least one solution. According to (42), under assumption r1 < r2 , one has instead of (40) the inequalities Ak1 (u)G2k (u)Ak2 (u)

(43)

with A22 (u) − A11 (u) = (r2 − r1 )(1 − )0. Therefore, all the solutions of equation G22 (u) = 0 are less than any solution of G21 (u) = 0, for illustration see Figs. 2 and 3. Hence, if G2k (u) attains its global minimum at x2k , k = 1, 2, then x22 < x21 . Moreover, since G2k (x1k ) < 0, we have x1k < x2k , k = 1, 2. It is easily verified that  G22 (y22 (x)) − G21 (x21 ), x < x22 , f22 (x) − f21 (x) = (r1 − r2 )x + G22 (x22 ) − G21 (x21 ), x ∈ [x22 , x21 ], G22 (x22 ) − G21 (y21 (x)), x > x21 , where y2k (x), k = 1, 2, are either equal to x or the points of corresponding local minimums. Since f22 (x) − f21 (x)  G21 (x22 )−G21 (x21 ) > 0 for x < x22 and f22 (x)−f21 (x)G22 (x22 )−G22 (x21 ) < 0 for x > x21 , the optimal behaviour has the form (9). Thus   −r1 x + G21 (x21 ), x x2∗ , −r1 , x < x2∗ ,  f2 (x) = (x) = f ∗ 2 −r2 x + G22 (x22 ), x x2 , −r2 , x > x2∗ with x2∗ = (G22 (x22 ) − G21 (x21 ))/(r2 − r1 ). Now suppose that u∗n (x) is given by (9). It follows that Gn+1,k (u) is similar to (42), namely, we have to put there xn∗ instead of x1∗ . Using the fact that (43) is valid for Gn+1,k (u) as well, one easily establishes that u∗n+1 (x) has also the form (9), thus completing the proof. (Existence of limn→∞ xn∗ will be obtained as a part of proof of Theorem 4, see below.)  Proof of Theorem 3. Under assumption r1 = r2 one has Gn1 (u) = Gn2 (u) for all n and u. Since G11 (u) is convex it attains the minimum at the point x˜1 given by (10). So f1 (x) = −r1 x + G11 (x˜1 ), f1 (x) = −r1 , f1 (x) = 0 for all x. Moreover, G21 (u) = G11 (u) − r1 = r1 (1 − ) − r3 + (r3 + )F (u). It means, there exists a unique point x˜2 given by (11) and such that G21 (x˜2 ) = minu G21 (u). It is obvious that f2 (x) = −r1 x + G21 (x˜2 ) and f2 (x) = −r1 for all x. Hence, the same is true for all n > 2. This completes the proof.  Proof of Theorem 4. 1. At first, we tackle the case r1 > r2 . Consider n > m, that is, the optimal behaviour is given by (5). Using the fact xn1 xn+1,1 < xn2 xn+1,2 one can write ⎧ Gn+1,1 (xn+1,1 ) − Gn1 (xn1 ), x < xn1 , ⎪ ⎪ ⎪ x ∈ [xn1 , xn+1,1 ), ⎨ Gn+1,1 (xn+1,1 ) − Gn1 (x), fn+1 (x) − fn (x) = Gn+1,k (x) − Gnk (x) (44) x ∈ [xn+1,1 , xn2 ], ⎪ ⎪ (x) − G (x ), x ∈ (x , x ], G ⎪ n+1,2 n2 n2 n2 n+1,2 ⎩ Gn+1,2 (xn+1,2 ) − Gn2 (xn2 ), x > xn+1,2 . Furthermore, Gn+1,1 (xn+1,1 ) − Gn1 (xn+1,1 ) Gn+1,1 (xn+1,1 ) − Gn1 (xn1 ) Gn+1,1 (xn1 ) − Gn1 (xn1 ), and similar inequalities are valid for lines 2, 4 and 5 of (44), whereas the expression of line 3 is equal to  +∞  (fn (x − s) − fn−1 (x − s))(s) ds. −∞

Thus, it is not difficult to obtain the following inequalities (fn+1 , fn ) =

sup x∈(−∞,+∞)

|fn+1 (x) − fn (x)| 

max

x∈[xn1 ,xn+1,2 ]

|Gn+1,k (x) − Gnk (x)|,

whence it follows (fn+1 , fn )  (fn , fn−1 ). In other words, we provided the base for uniform convergence of fn (x) to f (x). This enables us to pass to the limit, as n → ∞, in (2) and obtain (12).

2632

E.V. Bulinskaya / Journal of Statistical Planning and Inference 137 (2007) 2613 – 2632

Since Hn (u) = bk () − (r3 + )−1 Gnk (u), their uniform convergence is easily verified. ∗ , 2. Now turn to the case r2 > r1 . Instead of (44) we have now, see the proof of Theorem 2, for xn∗ < xn+1  x xn∗ , Gn+1,1 (xn+1,1 ) − Gn1 (xn1 ), ∗ , fn+1 (x) − fn (x) = (r2 − r1 )x + Gn+1,1 (xn+1,1 ) − Gn2 (xn1 ), xn∗ x xn+1 ∗ x xn+1 , Gn+1,2 (xn+1,2 ) − Gn2 (xn2 ),

(45)

Proceeding as in the first part of the proof and using the linearity in x of the line 2 in (45) we get (fn+1 , fn )

max

x=xnk ,xn+1,k ,k=1,2

|Gn+1,k (x) − Gnk (x)|.

∗ In its turn, the right-hand side is less than  (fn , fn−1 ). After that the reasoning goes as in part 1. The case xn+1 < xn∗ is treated similarly. It is necessary to underline that ∗ |xn+1 − xn∗ | 2 (fn , fn−1 )(r2 − r1 )−1 ,

therefore xn∗ → x ∗ , as n → ∞. 3. Validity of Theorem 4 under assumption r1 = r2 is obvious.



References Afanasieva, L.G., Bulinskaya, E.V., 2004. Dependence of option price on probability of its fulfillment. In: Proceedings of Voronezh Winter Mathematical School-2004.pp. 1–12 (in Russian). Arrow, K., Karlin, S., Scarf, H., 1958. Studies in the Mathematical Theory of Inventory and Production. Stanford University Press, Stanford, California. Asmussen, S., 1994. Ruin Probabilities. World Scientific, Singapore. Bellman, R., 1957. Dynamic Programming. Princeton University Press, Princeton. Bulinskaya, E.V., 1990. Inventory control in case of unknown demand distribution. Engineering Costs and Production Economics 19, 301–306. Bulinskaya, E.V., 1998. Systems stability and optimal control. J. Math. Sci. 92, 3857–3872. Bulinskaya, E.V., 2003. On a cost approach in insurance. Rev. Appl. Indus. Math. 10, 276–286 (in Russian). Bulinskaya, E.V., 2004a. Multistep investment policy of insurance company. In: Transactions of XXIV International Seminar on Stability Problems for Stochastic Models. September 10–17, 2004, Jurmala, Latvia.pp. 193–200. Bulinskaya, E.V., 2004b. Stochastic orders and inventory problems. Internat. J. Production Econom. 88, 125–135. Bulinskaya, E.V., 2005a. On cost and reliability approaches in inventory theory. Cybernet. Systems Anal. 41, 69–81 (in Russian). Bulinskaya, E.V., 2005b. On decision making under uncertainty. In: Proceedings of the Fifth St. Petersburg Workshop on Simulation. June 26–July 2, 2005, St. Petersburg, Russia.pp. 181–186. Bulinskaya, E.V., 2005c. Investment policy of insurance company under incomplete information. In: Transactions of the XXV International Seminar on Stability Problems for Stochastic Models. Maiori/Salerno, Italy, September 20–24/2005.pp. 71–78. Chikán, A. (Ed.), 1986. Bibliography of Inventory Literature. Budapest, ISIR. Csörgö, M., Horváth, L., Shao, Q.-M., 1993. Convergence of integrals of uniform empirical and quantile processes. Stochastic Process. Appl. 45, 283–294. De Vylder, F.E., 1996. Advanced Risk Theory. Editions de l’université de Bruxelles. Swiss Association of Actuaries. Girlich, H.-J., 1973. Diskrete stochastische Entscheidung Prozesse und ihre Anwendung in der Lagerhaltung. Teubner Verlag, Prabhu, N.U., 1980. Stochastic Storage Processes. Queues, Insurance Risk and Dams. Springer, New York. Scarf, H., 1959. Bayes solutions of the statistical inventory problem. Ann. Math. Statist. 30, 490–580. Stoyan, D., 1983. Comparison Methods for Queues and Other Stochastic Models. Wiley, New York. Zolotarev, V.M., 1997. Modern Theory of Summation of Independent Random Variables. VSP, Utrecht.