Stochastically minimizing the number of customers in exponential queuing systems

European Journal of Operational Research 27 (1986) 111-116 North-Holland

111

Stochastically minimizing the number of customers in exponential queuing systems Pravin K. JOHRI

*

A T & T Bell Laboratories, Crawfords Corner Road, Holmdel, NJ 07733, U.S.A.

Abstract: This paper deals with the problem of controlling an exponential queuing system (that is, a system with exponential service times and Poisson arrivals) so as to stochastically minimize the number of customers in the system at any time t > 0. Sufficient (simple) conditions are developed for a policy to be optimal. Similar conditions are sufficient for a policy to stochastically minimize(maximize) any function of the state of the system. Two models are considered to illustrate the results. In both cases, optimal policies are shown to satisfy these conditions by a simple inductive procedure. Keywords: Exponential queuing systems, stochastic order

Introduction

This paper deals with the problem of controlling an exponential queuing system (that is, a system with Poisson arrivals and exponential service times) so as to stochastically minimize the number of customers in the system at any time t > 0. By allowing transitions that do not result in a change of state, as in Lippman (1975), the times between transitions become exponential with constant parameter. Consequently, they are independent of both the control policy employed and the state of the system. By exploiting this fact, sufficient (simple) conditions are developed for a policy to be optimal. As an immediate generalization, similar conditions are sufficient for a policy to stochastically minimize (maximize) any function of the state of the system. Two models are considered to illustrate the results. In both cases, optimal policies are shown to satisfy these conditions by a simple inductive procedure. The first model is a single server queue with m types of arrivals. Type i customers arrive according to a Poisson process at rate h i and require service which is exponentially distributed with rate #/. After completion of service they exit the system with probability P~ and rejoin it with probability (1 - P,.). Service pre-emption is allowed and the server can switch instantaneously from serving one type of customer to serving another type. It is shown that the policy, which always serves a customer with the largest value of p# among the customers present, is optimal. The second model is an m-server queue where the service times of server i are exponentially distributed with rate #~. Customers arrive according to a Poisson process at rate ~. N o server is allowed to be idle if a * This research was done while the author was at the State University of New York at Stony Brook. Received January 1985

0377-2217/86/$3.50 © 1986, Elsevier Science Publishers B.V. (North-Holland)

P.K. Johri / Minim&ing the number of customers in queuing systems

112

customer is waiting. Derman et al. (1980) have shown that the optimal policy (even under the more general conditions or arbitrary arrivals) is to always assign a customer to that free server whose service rate is largest. This policy is shown to Satisfy the conditions developed in this paper. The restriction that a server is not allowed to be idle if a customer is waiting is necessary. Larsen and Agrawala (1983) obtain a different policy which minimizes the average number of customers for a two-server queuing system when servers are allowed to be idle. The methodology developed in this paper is not entirely new. Crabill (1974), Nash and Weber (1982) and Winston (1979) have developed sufficient conditions for policies to be optimal. However, the conditions obtained in this paper are relatively simple. Minimization over the policy space is translated to minimization over the action space, and thus the conditions derived are only in terms of the optimal policy and the alternative actions. A simple induction is all that is necessary to show that these conditions hold. It should be noted that stochastic minimization is a strong criterion. An optimal policy may not exist and if it does it generally conforms rather well with intuition. Quite often, as in the examples given, it turns out to be an index type policy (see Varaiya et aL (1983)). For sake of brevity, only two examples are used to illustrate the procedure. Many other problems have been solved. See, for example, Winston (1977), Weber (1978), Nash and Weber (1982) and Schrage (1968). This method works equally well for these problems as for the two examples given in this paper. In Johri and Katehakis (submitted), the optimal policy is obtained for a tandem queue model using the same technique as in this paper.

Sufficient conditions for optimality An exponential queuing system is specified by four objects: a state space S, an action space A, a law of motion q and an (exponential) transition time t. Whenever the system is in state s ~ S, and an action a ~ A s _ A is chosen, two things happen: (i) The next state of the system is selected according to the probability distribution q(-Is, a). (ii) The time till the next transition is an exponential random variable with parameter ~s,,,- Assume that the set of ~'s has a finite upper bound and let A >/sups,aXs, ~. Redefine the laws of motion (as in Lippman (1975)) as follows:

q,(s,ls, a ) = [ [ A - { 1 - q ( s l s , a)}Xs,,]/A ~ Xs.~q(s'ls, a)/A

if s' = s ,

if s" ~ s.

(1)

For the new formulation, if a policy 7r is employed, let

s,(t)

denote s , ( n ) denote N~(t) denote N=(n) denote

the the the the

state at time t, state after the n-th transition, number of customers in the system at time t, and number of customers in the system after the n-th transition.

Let s(0) be the initial state of the system. A policy ~r° stochastically minimizes the number of customers at any time t > 0 if

P{ N,o(t) > k Is(0) = s } ~ k Is(0) = s ), k = 0, 1, ... ; s ~ S; for every ~.

(2)

Since the time between transitions is independent of both the state of the system and the control policy, and since the state does not change between transitions, inequalities (2) are equivalent to P{

> k Is(0) -- s }

k=0, 1.... ; n=l,

2.... ;

> k Is(0) = s

s ~ S ; for every ~.

}, (3)

Assume that ~r° is a deterministic policy and that the action prescribed by ~r° in state s is denoted by

~-°(s).

P.K. Johri / Minimizing the number of customers in queuing systems

113

Theorem 1.7r o stochastically minimizes the mtmber of customers in the system if it satisfies (3) for n = 1 and if

~_, q'(s'ls, ~ r ° ( s ) ) [ P { N . o ( n ) > k l s ( O ) = s ' } - P {

N.o(n) > kls(O)=s}]

$'ES

<~ E q'(s'ls, a ) [ P { N , o ( n ) > k l s ( O ) = s ' } - P { N , , o ( n ) > k l s ( O ) = s } ]

,

s'~S

aEA,; k=0,1 .... ; n=0,1 .....

(4)

Remark. Inequalities (4), since they are only in terms of ~r°, are simpler to establish than inequalities (3). Proof. It will be shown, by induction on n, that ~r° satisfies (3) for every n. Assume that it does for some n = n' >~ 1, which implies that if only the first n' transitions are of interest then rr ° should be employed. For n = n' + 1, if an action is taken initially according to policy rr °, then the result follows from the induction hypothesis. So consider a policy rr' which initially prescribes an action (or randomized actions) different than the initial action prescribed by ~r° and then is identical to 7r° for the next n' transitions. By conditioning on the state after the first transition and using the Markov property, P{N,~o(n'+ 1) > k l s ( 0 ) = s } = P { N = o ( n ' + 1) > k l s ( 1 ) = s } + [left hand side of (4) with n = n ' ] / A , whereas P ( N ~ . ( n ' + 1) > k Is(0) = s } = P ( N = , ( n ' +

1) > k I s ( l ) = s }

+ [right hand side of (4) with n = n ' ] / . 4 . It follows that ~r° satisfies inequalities (3) for n = n' + 1 and the induction is complete.

Optimal customer selection in an M / M / 1

queue ssJth feedback

For the first model described previously, the state of the system can be denoted by a vector s = (s~, s 2 . . . . . s,,,) where s~ = j if there are j customers of type i in the system, i = 1, 2 . . . . . m; j = 0, 1. . . . . Given a state s, let

C(s)

=(i:si>O},

( + 1 i , s)=(sx ..... si_l, s~ + l, si+ 1..... s,,), Isl

=s~ + s 2 +

--- +s,..

Let ~ = ~ t l "]- /'L2 -[" " ' " q - ~ m ,

~k = ~ k l "{- ~k2"}- " ' " -l-~krn,

and assume, without loss of generality, that PlPl >/P2#2 >/ "" " >/Pm/X.,, and that A=k+~=I. Define ~r- to be the policy which serves any customer with the least index among the customers present, that is, the action prescribed by ~r in state s can be denoted as serving a type ~r-(s) customer where =

i

C(s).

114

P.K. Johri / Minimizing the number of customers in queuing systenu

It is obvious that ~r satisfies ineqs. (3) for n = 1. Ineqs. (4) simplify to

0+,.+,[ P{ g~-(n) > kls(O)= ( - 1 + .

s)} - P{ g~-(,,)> k i s ( o ) = s

}]

< p;.j[ P { No-(,,) > ~ Is(O) = ( - ~, s) } - p (N~-(,,) > ~. s(O) = s }], (5)

s ~ S ; for everyj ~ C ( s ) ; n=O, 1 . . . . ; k=O, 1 . . . . . Define

h.(s; i; k) = pibti[P(N=-(n) >

kls(O)=

(-1.

s)} -

P{ N,~-( . ) > kls(O)

Lemma 1. For every s ~ S, for every i, j ~ C(s) such that i < j , n = 0, 1. . . .

=s}].

and k = O, 1. . . .

h.(s; i; k)<~h.(s; j ; k)~.>pjl~jand ] ( - l i , s)] = [ ( - l j , s)] < Is[, the result is true for n = 0 . Assume that it is true for some n' - 1 >/0. Let v=~r'(-li,

s)
and

u=~r-(-lj,

s)=rr-(s)<~i.

Case (i): u = i. By conditioning on the state after the first transition, and using the Markov property, h.,(s; i; k) = pd~,lP{N.-(n' ) > k Is(l) = ( - l v , - 1 , , s)}po~o +P{N,,-(n') > kls(1 ) = ( - 1 / , s ) ) ( # - pj%)

+ ~P(N.-(n')>kls(1)=(lr,-li,

s)}X r

r=l

- P { N,; (n') > k ls(1 ) = ( - 1i, s)} P,#i

-P{

N~-(n') > k I s ( l ) = s } (/~ - p,/t,)

-- r~= l

P{N~-(n')>kls(1)=(l~'s)}}~r]

=p,tt,h.._l((-l,, s); v; k)+(tt-pd~i)h.._,(s; i; k) + ~ )~rh.,_l((l,. s ) ; i; k ) . r~l

Similarly,

h,,(s; j; k ) = p d x i h w _ l ( ( - l i , s); j ; k)+(tt-Pitti)h,,_l(s; j; k) m

+ E ?,,h,,_,((a,, s); j; k). r=l

It follows from the inductive hypothesis that the result is true for n = n'.

(6)

P.K. Johri / Minimizing the number of customers in queuing systems

115

Case (ii): u < i. For this case, since v = u, the induction is simple and is omitted. The proof of lemma 1 is complete. It has just been shown that rr satisfies the conditions of Theorem 1 and hence, ~r stochastically minimizes the number of customers in the system at any time t > O. Optimal assignment of sen'ers in an

M/M/m

queue

For the second model described earlier, the state of the system is denoted by (N, s) where N equals the number of customers in the system and s = (s 1, s 2..... sin) is a binary vector such that s i = 1 (0) if server i is busy (idle). Given a state (N, s), let C(s) be the set of indices of busy servers, C'(s) be the set of indices of idle servers, ~(s) = Ei~ c(s)t~;, s i = (sl ..... s~-l, 1, s,+l . . . . . s,.), i, = ( s a. . . . . si_l, O, si+ 1..... s,,). Without loss of generality assume that /q >~/x2>~ - - - >~t,~, and that A =~,+/.tl +/.t2+ - - - + / I r a = 1 . Derman et al. (1980) have shown that the optimal poficy is to always assign a customer to that free server whose service rate is largest. So let ~r^ be the policy which always sends a customer to the ,r^(s)-th idle server, where ~r^(s) = m i n i ~ C'(s), whenever servers are free. It is obvious that ~r^ satisfies ineqs. (3) for n = 1. Since servers are not permitted to be idle when a customer is waiting, options exist only when a new customer arrives and two or more servers are free. To distinguish such states denote them as (1, N, s). If this new customer is assigned to server j then the new state is ( N + 1, sJ). For the purpose of establishing ineqs. (4), which are in terms of ~" only, the new state can be written as ( N + 1, s ) where s ~= s *i'). Ineqs. (4) can be written as

X[P{N;(n) > k I,(0)= (1, N + 1, s^)} - P { N ; (,,)> k Is(0)= (N+ 1, ,^)}] + Z rEC(s') s J ) } - P ( N . - ( n ) > k l s ( O ) = ( N + t . s^)}] ~.[P(N~r'(n)>kls(O)=(N,r(sJ))}-P{g~r'(tl)>kls(O)=(g+l,s')}],



+

E reC(sJ) j~C'(s);(1,

kls(0)=(1.

N,s)~S;

N + 1.

n = O , 1. . . . ; k = O , 1.....

which simplify to #.i,) [ P { N , , - ( n ) > k ls(0 ) = ( N , s)} - P ( N , , - ( n ) > k Is(0) = ( N + 1, s ' ) } ] Z I~.P(N~'(n) > kls(O) = ( N , ~C(s) k l s ( 0 ) = ( N , s)} - P { N , ; ( n ) > k Is(0) = ( N + 1, s^)}] + ~kP{ g . - ( n ) > k Is(0) = ( g + 2, (si) ^) } + ~_. ~ r P { N , , ' ( n ) > k l s ( O ) = ( N , ' ( s J ) ) }

+XP{N,r(n ) > k l s ( 0 ) = ( N + 2, ( s ' ) " ) } +

rEC(s)

j~C'(s);(1,

N,s)~S;

n=0,1 .... ;k=O,

1.....

r(s^))}

116

P.K. Johri / Mininzizing the number of customers in queuing systems

Lemma 2. For everyj ~ C'(s), (1, N, s) ~ S, n = 0, 1 . . . .

and k = 0, 1 . . . . .

P ( N ~ - ( n ) > k Is(0) = ( N + 1, s ' ) } < P ( N ~ - ( n ) > k l s ( 0 ) = ( N + 1, s J)} and

P{ N~-(n) > k Is(0) = (N, s)} -P{

N . , - ( n ) > k I s ( 0 ) = ( N + 1, s ' ) } < 0.

The proof, by induction on n, is similar to the proof o f Lemma 1 and is omitted. Since #~(s)>~#j, from Lemma 2 it follows that ~r satisfies ineqs. (4) and hence it stochastically minimizes the number of customers in the system at any time t > 0.

References Crabill, T.B. (1974), "Optimal control of a maintenance system with variable service rates", Operations Research 22, 736-745. Derman, C., Lieberman, G., and Ross, S.M. (1980), "On the optimal assignment of servers and a repairman", Journal of Applied Probability 17, 577-581. Johri, P.K., and Katehakis, M.N., "Scheduling service in tandem queues attended by a single server", OR Spektrum (submitted). Larsen, R.L., and Agrawala, A.K. (1983), "Control of a heterogeneous two-server exponential queuing system", IEEE Transaction on Software Engineering 9 (4), 522-526. Lippman, S.A. (1975), "Applying a new device in the optimization of expor~ential queuing systems", Operations Research 23, 687-710. Nash, P. and Weber, R.R. (1982), "Dominant strategies in stochastic allocation and scheduling problems", in: M.A.H. Dempster, J.K. Lenstra and A.G.H. Rinnooy-Kan (eds.), Deterministic and Stochastic Scheduling~ Reidel, Dordrec:ht, 385-398. Schrage, L. (1968), "A proof of the optimality of the shortest remaining service time discipline", Operations Research 16, 687-690. Varaiya, P. Walrand, J., and Buyukkoc, C. (1983), "Extension to the multi-armed bandit problem", in: Proceedings of the 22rid IEEE Conference on Decision and Control, San Antonio, TX; 1179-1180. Weber, R.R. (1978), "On the optimal assignment of customers to parallel servers", Journal of Applied Probability 15, 406-413. Winston, W.L. (1977), "Assignment of customers to ser~'ers in a heterogeneous queuing system with switching", Operations Research 25, 469-483. Winston, W.L. (1979), "Optimality of the shortest line discipline", Journal of Applied Probability 15, 181-189.