replacement rules for decaying service facilities

PURCHASE/REPLACEMENT RULES FOR DECAYING SERVICE FACILITIES CHARLES ERNESTLOVE* Collegeof Commerce. University of Saskatchewan, Saskatoon. Canada

Scopeand purpose-This paper concentrates on developing a computational algorithm for establishing a schedule of purchases and replacements for large fleets of equipment when the fleet is operating in a stochastic environment. Typically such systems exist in delivery systems and equipment rental systems. Emphasis is placed on obtaining a computationally feasible algorithm. Limiting conditions are established whereby the problem can be treated as an ordinary linear program. This allows for the investigation of a much wider range of scenarios and the imposition of many more corporate restrictions on the pur~hase!replacement schedule. Appropriate background material for this paper is contained in references[ 1.?,5 and 91 which discuss the application of dynamic programming to stochastic processes. Abstract-The economic structure of a pool of equipment in use as a service facility is examined. Typically in such circumstances, the economic decay of the equipment is quite rapid. A model for the acquisition and replacement of such equipment is developed which takes the form of a multi-period dynamic program. The result is a specification of a purchase/replacement age vector for each period under consideration. The size of the resulting problem makes ordinary dynamic programming infeasible. A method of “successive approximations’~is introduced which takes advantage of the special structure of the problem. Limiting conditions are established which allow the structure to asymtotically converge to a global optimum. A numerical example is introduced which illustrates the form of the solution. Additional assumptions are imposed which allow the problem to be approximated by an ordinary linear program.

INTRODUCTION

A pool of equipment is available for use by customers. Customer demands on the pool are normaI~y stochastic in nature. If a unit is available (that is, they are not all currently in use), then the customer takes it out for a period of time and returns it. The length of time it is out is also normally described by a stochastic process. It may or may not be that a fee or rental charge is levied for use of the equipment. Such rental charges are usually based on the length of time the equipment is in service. We seek to maintain a server system which, in supplying units for the customers, returns maximum benefits to the firm. THE OPERATING

SYSTEM

In many operating systems, the equipment undergoes significant deterioration over time. Thus, to continue using the equipment in the system, results in both increasing operating costs and declining resale value. At the same time, it is often the case that the demand pattern (of arriving customers) is itself significantly affected by such deterioration. As the equipment ages, customers will turn elsewhere for satisfaction. Therefore, the decision to replace equipment in the fleet depends not only on the increasing costs to the firm but, also such replacements as are necessary to maintain the demand for service. A further problem which must be explicitly accounted for is that the average demand for service changes significantly over time. Typically this takes the form of a seasonal demand structure. This has the effect of staging the acquisition/replacement process such that equipment is obtained just prior to periods of high demand and excess equipment is removed just prior to periods of low demand.

*Assistant Professor of Management Science at the University of Saskatchewan, Saskatoon, Canada. He holds the B. Engineering and M.B.A. from McMaster University, and the Ph.D. from the University of London. He has worked as a process and design engineer for Union Carbide of Canada Ltd. His current research interests are in the development of generalized decomposition procedures for large scale dynamic programming problems.

CHARLESERNESTLOVE

112 THE

QUEUING

PROCESS

The embedded queuing process of our operating system is the multiple server model with a finite number of servers. Customers arrive and continue to take out equipment as long as servers are available. When all servers are busy, customers either leave unsatisfied or queue up and wait for an available unit. Under normal circumstances the system has a finite queue length. Two stages in such processes can be identified. Initially the process is in a transient phase in which the probability of the system being in a particular state is changing over time. Ultimately, if the process is allowed to continue long enough, it may arrive at steady-state. In order to treat as general a problem as possible, we will assume that the arrival rate pattern follows a general Erlang process with mean arrival rate Ak customers per stage (where k is the number of stages in the process). Utilizing an Erlang process affords great flexibility in fitting empirical distributions. (When k = 1, a simple Poisson process results.)* In addition, it will be assumed that the service rate is a negative exponential process with mean time between servers of l/p time units (I* being the average number of services per unit time that a unit of equipment can completeS). Allow a maximum of U* customers in the system (where U* is in general greater than the number of units of equipment (U) available) before balking takes place. It is possible then for a customer to be in any one of k. U* + k possible states. Define the transient probability (see Whisler [9] p. 644) P,,(T, U’), as the probability of the system being in state I after an elapsed time T, given that the system started in state z with U’ units of equipment available in the period f. Thus r 5 k*U*’ + k. If no queue is permitted, then rsk-U’+k.

The c~cuIation of these transient probabilities can be handled quite simply utilizing numerical procedures. Neutsf61 and Grassmann present straightforward algorithms for such calculations.

THE

AGING

PROCESS

Central to decisions of purchase/replacement is the concept of equipment aging. At any time t, the firm has a pool of equipment available for use, this pool of equipment has a particular age structure associated with it. The folIowing aging process can be identified with such a pool. Define: U;, as the number of units of age class j in the pool at the beginning of period t. If there are J age classes in total, we can define: U’ = (Ui} as the equipment age vector for al1 these classes at start of period t. Equipment that remains in the system for period t automatically becomes one age class older, U,‘L’,at the end of the time period. This eliminates the possibility of equipment being randomly removed from service. Breakdowns are assumed to be repaired instantaneously. Also, equipment added or deleted in period t is valued as of the end of the period. Thus: N’ = {N,!+,},is the decision vector for period t. Such that: U,‘Z:= U; + N; for all t and all j u;ro

(1)

Equation (1) defines the aging process. Finafly define: Vi+, * Ni+, as the valuation terms. That is, it is the cost in period t of adding or deleting equipment from the system. (where V:+, is the market value of a unit equipment of age j + 1 in period t. V;,, can consist of two costs-one for purchasing, if I$+, is positive; and one for selling, if N;+, is negative).

tin some circumstances, a hyper-exponential distribution may be more appropriate. The queuing equations utilized in the paper would have to be altered accordingly. $We do not introduce a general Erlang (or hyper-exponential) service rate. The computational requirements of monitoring ail combinations of service states would be prohibitive except in the case of very small systems. However, in environments of interest to us, the assumption of a negative exponential service rate is not unreasonable. We have a large number of candidates competi~ for completion of service, each with a low probability of being selected. (See Wagner181 page 135).

Purchase/replacement THE

rules for decaying service facilities

PURCHASE/REPLACEMENT

113

PROBLEM

We can identify the following costs associated with operating such a system. H,‘, the holding cost (in period t) per period for each unit of equipment in class j that is held by the firm, whether in service or not. Thus: H*’ = i H,’ . U,’ are the total holding costs for maintaining the pool of equipment in j=, period t. (2) O,‘,the operating cost per period for each unit of equipment in class j that is in service in period t. We now make the further assumption that, as long as there are customers available, equipment is released for service on the basis of lowest operating cost first, (where all equipment in the same age class has the same operating cost). This presents no conceptual difficulty as it would be disadvantageous for the firm to adopt any other policy. Define z’ = I: zi, as the total amount of equipment in service at a point in time. Because we have placed an ordering on the release of equipment into service, we need only be concerned with the cumulative number in service. Then, the total expected operating cost for the period when the period begins with z’ units in service is given by:

s.t. r 5 k * U*’ + k

(3)

where Li is the lower limit of the amount of equipment in age class j and Mj is the upper limit of the amount of equipment in age class j (thus M: - Lj’ = U,‘). Furthermore define: S’ as the shortage cost incurred in period t if a customer is forced to leave the system, thus total expected shortage cost for the period becomes: E:,(S) = {S’ . A’ . P+ku*‘+k(T, U’,}

(4)

where A’ is the average arrival rate of customers during period t. Finally define: C’( U’, z’), as the optimum total cost in period t when entering the period with equipment vector U’ of which z’ units are in service. Thus for the single period f we have: C’( U’, z’) = minitim IV:+, +IV:+, + H*’ + E:(O) + ES(S)} IZSjZZ-l s.t. U;:: = U,’+ N:+, lzzj5I&I U,’ ro I ,=I

(5)

Equation (5) can be seen to be a dynamic programming problem of dimension J + 1 state variables (1 dimensions associated with the age class vector plus an additional state, z’, to specify the amount of equipment in service); and J decision variables. Dividing the time horizon of interest into L equal time periods, we can formulate the following multi-period model: C’( U’, 2’) = mini~m

V:,, ’ N;,, + H*’ + E:(O) t E:(S)

IZGj5,--l k.U*“‘+K

+a 5 s.t. U::: = U; t N:+, vi’ 2 0

c’+‘(u I+‘, r) . P,,,r(T,U’+‘)) ,5,c,+, ,S,l,.-l

Where x is an appropriate discount factor for future time periods.

(6)

CHARLESERNESTLOVE

114

Equation (6) is a multi-stage dynamic program, with J + 1 state variables and J decision variables in each time period. Each single period is simply the single period model of equation (5). The high speed memory requirements for such a system are:

where W’ is the maximum allowable units in any age class. Thus, even for relatively small problems, a standard dynamic programming solution is not possible. The problem becomes one of reducing a J + 1 dimensional space model to one of smaller dimensions such that total storage requirements become manageable. Larson [4] has described several techniques for taking advantage of the special structure of system equations in order to reduce the dimensionality of the problem. Such techniques center around successively solving smaller sub-problems. METHOD

OF SUCCESSIVE

APPROXIMATIONS

Referring again to the system described by equation (6), we note that the basic aging process is described by equation (1). It really describes state variable transitions. Such a transition process occurs for each age class (resulting in J such equations). However, in equation (6), we can see that each decision variable (N,‘), controls only a single transition between two state variables (U,‘_, to U,‘+‘).The remainder of the system is not affected by this control variable. In period t we select one state variable arbitrarily, say U;. Now equation (6) can be optimized only with respect to the single control variable Nj’,,. We now have a dynamic program in only two state variables (Vi’ and z’) and one control variable, N,!,,. Entering period t, with Vi’ units in class j, and with a total of z’ units currently in the system, we can optimize equation (6) to specify an optimal trajectory for leaving this stage. This solution is only an approximation because all other state variables have been held fixed at previously set values. Having found the optimum for all possible values of this entering state variable, we then move back to period t - 1. By virtue of equation (l), the entering state variable at this stage must be U::,‘. It can be seen that moving back from the horizon is in reality a backwards shift along a diagonal. We continue to shift backwards on a diagonal (calculating the optimal values), until reaching either t = 1, or j = 1. At this point our two-state dynamic program has resulted in the optimum trajectory for a given diagonal. This diagonal is then “fixed” at this optimal trajectory. Then a different diagonal is chosen and the procedure repeated. In this fashion, optimal trajectories for all diagonals can be found. When all diagonals have been evaluated, this completes the first cycle, and we have calculated the first cycle optimum total cost. We then repeat this same cycle, each time calculating optimum total costs. When optimum total costs between two successive cycles converge within acceptable limits the procedure is terminated. In order to apply this technique to our purchase/replacement problem, it was programmed for computer operation. A description of the algorithm follows in Fig. 1. Using this procedure of “successive approximations” then, the solution of a (J + 1) dimensional dynamic programming problem is reduced to a sequence of two dimensional problems. In this way, solution of such large non-linear problems with state variables inequalities is possible. CONDITIONS

FOR MONOTONIC

CONVERGENCE

Referring back to equations (5) and (6) the method of solution requires cycling through the set of two variable problems a sufficient number of times such that convergence to the global optimum is reached. It is straightforward to show that, for any given starting states, equation (5) will be discrete convex with respect to each decision variable iv,!+,.*For such a discrete convex control problem with bounded states and unbounded controls, Korsak and Larsen[S] have shown that successive approximation techniques will result in a monotonically convergent solution. *Equation (3) plus equation (4) define the sum of operating plus shortage costs for the system. Whisler[9], p. 645, has shown that such a function is discrete convex with respect to {V”‘} (the total number in the system during period t). Since each decision variable, N’,+, can only increase or decrease U”‘, it remains discrete convex. Valuation term and holding cost term are linear with respect to changing N;+,. Thus C’(U’, z’) remains discrete convex with respect to N;, >.

Purchase/replacement rules for decaying Establish

a nominal

service facilities

115

trajectory

fl,

Choose : +

This

any

starting

element,

automatically

selects

I$ the

diagonal.

Holding all other state entries constant, element between its max. and min. values. For value vary 2’ from 0 to its maximum. combination calculate Ct(lJt,z’) and store.

+

i_

vary For each

Shift along the diagonal to element U>I$,and Continue state dynamic programming problem. shifting until the diagonal is exhausted.

repeat the two this backwards

Select a new diagonal and starting with element ti!, all optimum values for this diagonal. Repeat until diagonals have been completed.

Calculate the optimum first costs between two successive terminate the able limits,

1

this state each such feasible

calculate all

period costs. If total optimum cycles converge within acceptprocedure; otherwise, not.

Fig. 1. A method of successive approximations.

However, it is not possible to maintain strict convexity in the case of equation (6). C’( U’, z’) is convex only with respect to each value of r but not with respect to all values of r. In this case, monotonic convergence to the global optimum cannot be guaranteed (see Whisler[9], p. 645). In such circumstances, it is very likely that a local minimum will result. However, for test problems run, convergence was obtained in several cycles. Allowing the program to continue cycling did not appreciably improve these near-optimum results. MODEL OPERATION

This model was tested on an automobile rental agency. Typically, such firms are subjected to a wide variety of changing conditions. For example, sudden changes in operating and holding costs, as well as in the market values for equipment, can dramatically affect purchaselreplacement strategies. Conditions can result in wide variation in the lead time required for acquisition of new equipment with a commensurate effect on the firm’s budgetary planning. Finally, changes in demand, specifically seasonality, can result in a cyclical pattern of purchase/replacement. Below in Table 1 is the output of a sample run when demand is highly seasonal. The following data were used. Both t and j represent a period of 3 months. The model was run for 8 periods or two years. Table 1. Purchase replacement problem

1

Age class J

1 12 2 8 3 7 4 5 5 3 60

State matrix (CJ,‘) time period, t + 2 3 4 5 10 12 8 7 5 0

2 10 12 8 0 0

8 2 10 0 0 0

12 8 2 10 0 0

6

7

8

10 12 8

2 10 12 8 2 0

8 2 10 0 0 0

1; 0

Buy and sell matrix (NJ time period, f + I2 345

Age class J

I

10

2 3 4 5 6

0 0 0 0 -3

2 0 0 0 -7 -5

8 000 0 -12 -8 0

12

10

0 0 0 0

0 0 0 0

(b) Total

costs for 2-year horizon = $183,250

67 2 00 0 _,a

8 0 -12 1; i

CHARLES ERNEST LOVE

116

(10,15,10,5,10,15,10,5)customers/day (Poisson) CL= l/3 services/day (Neg-exponential) S’ = 100$/lost customer 1lfSL lStSL,lzSjsJ H,‘=5O+lOj V,’ = 4000 1~t~L lzSjSJ--l,lstlL V,!,, = V,’ - l5Oj lSrSL,lljlJ O,‘= lO+lOj 1ljSJ zj = 0 (for all classes, the starting amount in use is zero). A’ =

At time period 1 the beginning state matrix has: 12units in class 1 8 units in class 2 7 units in class 3 5 units in class 4 3 units in class 5. Because of the magnitude of the problem, all possible solutions (for each combination of entering state variables), are not listed. Table 1 lists only the optimum solution. It can be seen that purchase and replacement rules are also highly seasonal but lag the peaks in the demand cycle. No units are kept beyond age 5, and on the average, units are removed after age 4 (1 year). DEMAND

ELASTICITY

In certain types of rental environments, the age structure of the pool can affect the demand for service. Firms are then obliged to replace older units or experience a decline in demand. We can account for the impact of such an effect with the following iterative procedure. (1) Assume that demand is affected, not by the average age of the fleet, but rather by the number of age classes in the fleet. (2) Using the expected demand structure, solve the resulting dynamic program using the method of successive approximation. (3) An output of this optimal policy will be the number of age classes present in each period. (4) Adjust the original estimate of demand structure for the number of age classes present. (5) Resolve for the new optimal policy. Continue to iterate in this manner until a consistant optimal policy is obtained. An important consideration must not be lost sight of when making such an adjustment. Compare the two firms-one that is constrained to carry relatively new equipment (by the above argument) and one that is not subject to such a constraint (by the nature of their clientel). It is apparent that the first firm must move towards establishing higher prices because of this restraint. Otherwise it would not be enjoying the equivalent return on assets as the latter firm. LIMITING MULTI-PERIOD MODEL

If we allow the length of time of each period to increase, the probability of the system being in state r at the end of the period becomes independent of the starting state of the system. i.e. lim P,,JT,U')+r(r,U') T.= s.t. r I k. U”’ + k That is, the distribution of ending states approaches steady state. Under this assumption equation (6) can be rewritten as: C’(U’) = minirim {V,‘+, aN;,, + H*’ + E’(S) + E’(S) + a C’+‘(U’+‘)) ISijZSmI

(7)

Purchase/replacement rules for decaying service facilities

117

Equation (7) can of course be solved directly be applying the method of “successive approximations”. Under steady-state, the system is independent of the number of units in service at the start of each period. As such, we have a discrete convex objective function in only J state variables and J control variables. Conditions for monotonic convergence established by Korsak and Larson[S] are satisfied and the method of “successive approximation” guarantees global optimality. We interpret this limiting case in the following manner. It is possible, in solving equation (7) to be called on to sell units in a period where they are still in service. This requires that we wait until the equipment is returned to the pool before any such sale can be made. The extent of this lag time (by assuming steady-state) then introduces a slight bias into our results. However, given that the period length is of sufficient duration, such a bias approaches zero. LINEAR PROGRAMMING

FORMULATION

We have assumed throughout that equipment from the pool is placed into service on the basis of lowest operating cost first. Let us now assume that for all equipment of age class j, held in period t, we can pay a maintainance service charge in order to avoid any additional operating costs that would occur as a result of operating older equipment. What we are in fact doing is paying an additional charge for each age class such that we will be indifferent (in terms of cost) as to the age of the unit of equipment sent into service. Now we can rewrite (3) and (4) respectively as:

where Hi’ is the revised holding cost per period which includes the above service fee mentioned.

{k”$:k k - a(r, U’))

E"(O) = 0' .

(9)

Total expected operating costs are now solely a function of the amount of equipment in the system. E’(S) = {S’ +A’ . r(k . U*’ + k, U’,}

(10)

Define a new set of variables, xit, such that: 0 I x,*5 1 for all i and t and $ xi’ = U’ for all t

(11)

That is, xi’ is the fraction of the ith unit of equipment in the system in period ti, where P is merely a large number to insure that enough xit will enter the solution in each period. Consider the fraction of the first unit brought in in period 1, x,‘. Equation (9) and (10) are linear in x,‘. The piecewise convexity of the problem insures that either all or none of x,’ will be used. Assuming x,’ reaches a value of 1, then some fraction of x2’ will be brought in. The expected marginal contribution of variable x*’ can be determined from equations (9) and (10); being the difference between expected operating plus shortage costs for 2 units less expected plus shortage costs for only 1 unit. Any fraction of xz’ will contribute at the same expected marginal rate. Again the piecewise convexity insure xII will enter at either 0 or 1. This same argument holds for all xi’. Define: MO,’ as the expected marginal reduction in operating costs by introducing the ith unit in period r; and MS,’ as the expected marginal reduction in shortage costs by introducing the ith unit in period t. [MO: and MS can be calculated directly using (9) and (IO) respectively].

CHARLES ERNESTLOVE

118

With these definitions, equation (7) can be reformulated into the following linear program. minimize

OSXit’;l

all t and all i

(121

Notice here that N,!,, is now merely a dummy decision variable and can take on either positive or negative values. From a computational point of view of course, the L.P. formulation has significant advantages over the dynamic programming formulation. Because of its flexibility, it allows for the investigation of a much wider class of problems without posing undue computational difficulties. In particular, imposition of added budgetary or operational constraints on the system can be handled quite simply in a linear model whereas such is not the case with a dynamic programming formulation. DISCUSSION

In this paper, a general stochastic dynamic programming formulation to the purchaselreplacement problem is presented. Problems of dimensionality preclude solution by ordinary programming techniques. However, by taking advantage of the special structure of the problem, a “successive approximations” algorithm is developed. conditions for the monotonic convergence of this procedure to the global optimum are presented. This technique allows a wide variety of operating scenarios to be investigated. By taking the limiting behavior of the system and by proposing a maintenance service charge to eliminate differentials in operating costs based on age structure, the problem can be converted to a standard linear program. As such, many more constraints can be imposed on the system without restricting computational feasibility. REFERENCES I. 2. 3, 4. 5. 6. 7. 8. 9.

R. Bellman, Adaptive Control Processes. Princeton University Press, Princeton, NJ (1961). R. Bellman and S. Dreyfus, Applied Dynamic Programming. Princeton University Press, Princeton, NJ (1962). W. K. Grassmann, Transient solutions in Markovian queuing systems, Comput. Ops Res. (to appear) (1976). R. E. Larson, State Increment ~numic Programming, ,&fodem A~a~yr~ca~and Computaf~off methods in Scietlce and ~athema#ics Series. Elsevier, New York (1968). A. J. Korsak and R. E, Larson, Convergence proofs for a dynamic programming successive approximations technique, Submitted to Fourth IFAC Congress, Warsaw, Poland, June (1969). M. F. Neuts, The single server queue in discrete time-numerical analysis I, Naoal Res. Logistics Quar. 20, (1973). M. Tainter, Some stochastic inventory models for rental situations, Mgmt Sci. 11, (1964). H. M. Wagner, Principles of Operations Research. Prentice-Hall, London (1%9). W. D. Whisler, A stochastic inventory model for rented equipment. Mgmt Sci. 13, (1967). (Received 28 October 1976)