Optimal Service Control of a Serial Production Line with Unreliable Workstations and Random Demand

PII: S0005–1098(98)00050–8

Automatica, Vol. 34, No. 9, pp. 1047—1060, 1998 ( 1998 Elsevier Science Ltd. All rights reserved Printed in Great Britain 0005-1098/98 $ — see front matter

Optimal Service Control of a Serial Production Line with Unreliable Workstations and Random Demand* DONG-PING SONG- and YOU-XIAN SUN-

¹he optimal control policy of an unreliable serial production line is characterized by a set of monotone switching manifolds. Asymptotic properties of the manifolds are established, which lead to easy-to-implement suboptimal policies. Key Words—Dynamic programming; optimal control; manufacturing systems; discrete event dynamic system.

backlog cost. A mathematically complete study has been developed by Akella and Kumar (1986), where it was first shown that the hedging point policy is truly optimal for a one part-type one machine manufacturing system. Bielecki and Kumar (1988) developed a probability approach to solve the same problem with long-run average cost, and the exact solution to the hedging point was obtained. Sharifnia (1988) and Algoet (1989) extended to prove that the HP policy is also optimal for multiple part-type multiple machine-state systems. Hu and Xiang (1994) investigated the structural properties of optimal production controllers for a one part-type multiple machine-system and obtained the result as follows: the closer to the zero capacity state the system is, the larger the hedging point should be. Hu et al. (1994) proved that the HP policy was also optimal for the one-part-type one-machine system in which the failure rate of machines depends on the rate of production. The problem of optimal control for failure-prone system with general up and down times were considered in Hu and Xiang (1993, 1995) and Tu et al. (1993). On the other hand, preventive maintenance was introduced to reduce the machine failure rates and improve the productivity of the system by Boukas and Haurie (1990), Boukas et al. (1995) and Boukas and Yang (1996). Sethi et al. (1992a) and Sethi and Zhang (1994) used viscosity-solution approach to study the optimal feedback control problem for one machine system with multiple part types and multiple machine states. Boukas et al. (1991), Haurie and Delft (1991) and Haurie et al. (1994) formulated the problem as a piecewise deterministic optimal control problem and proposed a policy improvement algorithm for solving the problem computationally. Hierarchical framework was established to decompose and approximate the system in Gershwin (1989, 1994).

Abstract—The problem considered here is to find the optimal service control policy for a serial production line with n failureprone workstations and random demand. The processing times of the part in workstations are exponentially distributed, and the service rates are controllable if the workstations are up. The objective function is the expected discounted cost caused by inventories of work-in-process and inventory or backlog of finished products. It is shown that the optimal policy is of bang-bang type and can be determined by a set of switching manifolds. For a given state of the workstations, one manifold determines the optimal decision of one workstation while it is up. The monotonicity and asymptotic behaviors of the manifolds are investigated. The relationship of the manifolds under different workstation states is studied, i.e. the more the workstations are down, the lower the switching manifolds locate in the state space. Based on the structural properties of the switching manifolds, some simple suboptimal policies are proposed, which are quite easy to implement in practical systems. Numeric examples are given to illustrate the results. ( 1998 Elsevier Science Ltd. All rights reserved.

1. INTRODUCTION

An interesting issue arising recently is how to operate a failure-prone manufacturing system so that it can meet the product demand and at the same time keep up lower in-process and finished-part inventories. Kimemia and Gershwin (1983) studied the optimal control of a manufacturing system with multiple machines and no internal buffers, and derived a quite simple policy, called hedging point (HP) policy, which minimizes both inventory and

* Received 30 September 1996; revised 6 May 1997; revised 18 November 1997; received in final form 17 February 1998. An earlier version of this paper was presented at the 1996 IFAC Congress, held in San Francisco, CA, USA, 1—5 July 1996. This paper was recommended for publication in revised form by Associate Editor Christos G. Cassandras under the direction of Editor Tamer Bas,ar. Corresponding author Mr. Dongping Song. Tel. #86 571 7951069; Fax. 86 571 7951 107; E-mail [email protected]. - National Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, People’s Republic of China. 1047

1048

D.-P. Song and Y.-X. Sun

When the internal buffers between machines are taken into account, the problem becomes quite complicated. Van Ryzin et al. (1993) considered a two-unreliable-machine system with an internal buffer and used dynamic programming to obtain a numerical solution of the optimal control. Lou et al. (1994) proved the optimal control policy of the two-machine system is a feedback control. Presman et al. (1995) extended the above results to a stochastic N-machine flowshop problem. However, analytic solution of the optimal control policy is too difficult to derive. Lou and Van Ryzin (1989) derived a dual-threshold policy, termed twoboundary control, based on the numerical results. Numerical simulation techniques, heuristic tools and approximation techniques have been applied (Sethi and Zhang, 1994; Sethi et al., 1992b). Perturbation analysis technique was introduced to approximate the optimal threshold parameters in Caramanis and Liberopoulos (1992), Song et al. (1992) Yan et al. (1994) and Liberopoulos and Caramanis (1995). Most of the above works are based on the continuous material flow model, i.e., the machines or workstations are assumed to process the part continuously. The controllable variables are the machine production rates. The randomness of machine processing was not under consideration. However, it is a good approximation for production of sufficient volume. Other authors focused on the optimal service rate control for manufacturing systems. The machine service times are random and the service rates are controllable. Hajek (1984) proved that the optimal control for a system with two interacting service stations is of switching structure. Xu and Chen (1993) studied the asymptote characters of the above optimal policy. Miyoshi et al. (1993) showed that the optimal service control of a station connected with two parallel substations is also of switching structure. Weber and Stidham (1987) considered a cycle of m queues and showed that the optimal control of service rates had a monotone structure. Glasserman and Yao (1994) investigated the monotone structure of optimal policies for discrete event systems described by controlled Markovian generalized semi-Markov process. However, these pioneering authors did not consider the system demand and station failure in their papers. Due to the discrete nature of manufacturing system, it is reasonable to deal with it as a discrete event dynamic system (Glasserman and Yao, 1994; Baccelli et al., 1992). Our aim is to formulate the manufacturing system as a discrete event system and meanwhile allow the random demand and random workstation failures and repairs. We have considered an unreliable one-workstation manu-

facturing system which can produce many part types and its total service rate is constrained by a fixed constant. The structural properties of the optimal service rate allocation policy were investigated (Song and Sun, 1996). A manufacturing system which consists of n workstations in parallel and followed by an assembly workstation was studied in Song et al. (1998). The n different part types are produced first in the parallel workstations respectively, and then are assembled in the last workstation. By minimizing the expected discounted cost caused by work-in-process and inventory or backlog of the finished product, an optimal control policy with switching structure is derived. However, it mainly treated the case that all workstations are reliable. In this paper, we deal with a serial production line with n unreliable workstations and random demand. The workstation service rate is controllable only if it is up. The results in Song et al. (1998) are extended to this system. The optimal service rate control for operational workstations is also of bang-bang type and can be described by a set of monotone switching manifolds. Actually, if the states of all workstations are given, the optimal decision of one workstation can be determined by one manifold. Detailed structural properties of the optimal value function and the switching manifolds are investigated. We also studied the relationship of the manifolds under different workstation states. The interesting result is that the more the workstations are down, the lower the manifolds locate in the state space. Moreover, since the workstations are allowed to be unreliable, the state space increases rapidly with the addition of the workstation number in line. This results in the computational difficulty to find the actual optimal policy. We propose several simple suboptimal policies based on the structural properties of the optimal policy. These suboptimal policies are featured by several threshold values and can be easily implemented in practice. To conclude, we make the following contributions in this paper: 1. Consider a serial production line with n unreliable workstations and random demand. An optimal policy is derived, which is of bang-bang type, state feedback and switching structure. 2. The monotonicity and asymptotic behaviors of the optimal value function and switching manifolds with respect to the buffer levels are derived. 3. The relationship of the manifolds under different workstation states is established. That is, the more the workstations are down, the lower the switching manifolds locate in the state space. It follows directly the relationship of the optimal policies under different states. For instance if the optimal decision for an up workstation is to stop producing at a given system state, then the

Optimal control of unreliable production lines optimal decision of this up workstation is also to stop producing at the states in which more workstations are down than that in the former state. 4. Based on the structural properties of the switching manifolds, simple suboptimal threshold control policies are proposed, which are featured by several threshold parameters. This paper is organized as follows: In the next section, the problem and model are formulated and some definitions are given. In Section 3, the optimal control policy is derived, and it is shown that the optimal policy is of bang—bang type and switching structure. In Section 4, the definitions of switching manifolds are given. Detailed structural properties, such as monotonicity and asymptotic behaviors of the optimal value function and the switching manifolds, are investigated. The relationship of the manifolds under different workstation states is derived. In Section 5, we provide the results for the two-WS system. In Section 6, several simple suboptimal control policies are presented based on the monotonicity and asymptotic properties of the switching manifolds. Numeric examples are given in Section 7 and conclusion is made in Section 8.

the time between two demands is exponentially distributed with average time 1/k. Let x (t) be the i number of parts in the ith buffer for i(n, apparently, x (t) is a non-negative integer. Let x (t) be the i n difference between the number of the cumulative product of ¼S and the number of the cumulative n product demand of the system; x (t) is an integer. It n is positive when the system has finished-part inventories and negative when the system is backlog. Denote: a(t)"(a (t), a (t), 2 , a (t)); x(t)"(x (t), 1 2 n 1 x (t), 2 , x (t)); u(t)"(j (t), j (t), 2 , j (t)). 2 n 1 2 n Some facts should be considered for the operation of such a system. Selecting the too fast service rate results in large finished-part inventories which are expensive to maintain. Selecting too slow service rate results in order cancellations or loss of goodwill. Finally, the stock carried in the buffer must be controlled, because it adds to in-process inventory costs and requires factory space. Our goal is to operate the system so that it meets the demand without overproducing, uderproducing or carrying large quantities of material in the buffer. To find a policy to producing such behavior can be reflected in the cost function. That is, to find the optimal control policy u*(t) to minimize the following infinite expected discounted cost J (x, a)"min E

2. MODEL AND PROBLEM

Consider a production line producing one part type pictured in Fig. 1. The system consists of n exponential service workstations (WS) and n unlimited-size buffers in series. The WSs are separated by the buffers. Parts enter the system to the first ¼S from outside, then to the first buffer, then to 1 ¼S , and so forth until they reach the last buffer. 2 The finished product in the last buffer leaves the system if the demand is not satisfied. It is assumed that an inexhaustible supply of parts is available upstream of the first WS. The workstations are subject to Markovian failures and repairs with failure rates m , m , 2 , m and repair rates 1 2 n g , g , 2 , g . The failure process is time dependent 1 2 n rather than operation dependent, that is, the probability of a failure occurring during a time interval does not depend on how much the WS has been used, but only on the length of the interval. Denote the state of ¼S by a (t), here a (t)"1 if ¼S is up i i i i and a (t)"0 if ¼S is down. i i The exponential service rate at ¼S is j , which i i can be selected to be any value s.t. j 3[0, jM ], i i i", 2, 2 , n. The product demand is random and

=

P0

e~bt g (x(t)) dt,

(1)

where 0(b(1 is a discount factor and g( ) ) is a function to penalize both in-process inventory and the difference between demand and actual production, such as g(x(t))"c x (t)#c x (t)#2#c x (t) 1 1 2 2 n~1 n~1 #c` x` (t)#c~ x~ (t), n n n n and x~ (t)" where x` (t)"max (x (t), 0) n n n max (!x (t), 0), where c , c` and c~ are fixed coeffin n i n cients. Let Z "M0, 1N and Zn be the product space 2 2 of n Z . Define mapping ¹ : ZnPZn and define 2 2 i 2 mappings R , A and D from Zn~1]Z to Zn~1]Z ` ` i i i as follows: f

f f

f f

f

Fig. 1. A production line with random demand.

1049

f

¹ a"(a , 2 , a , p , p , 2 , a ), where i 1 i~1 i i`1 n p ,a #1 mod 2, i"1, 2, 2 , n; i i R x"(x #1, x , 2 , x ); 1 1 2 n R x"(x , 2 , x , x !1, x #1, x , 2 , x ) i 1 i~2 i~1 i i`1 n if x '0, and R x"x if x "0, i~1 i i~1 i"2, 3, 2 , n; R x"(x , 2 , x , x !1); n`1 1 n~1 n A x"(x , 2 , x , x #1, x , 2 , x ), i"2, i 1 i~1 i i`1 n 3, 2 , n; D x"(x , 2 , x , x !1, x , 2 , x ), i"1, i 1 i~1 i i`1 n 2, 2 , n; ! (x)"MR : R xOx, i"1, 2, 2 , n#1N. i i

1050

D.-P. Song and Y.-X. Sun

Here R x (i"1, 2, 2 , n#1) and ¹ a (i"1, 2, 2 , i i n) are called events of the system, and !(x) is the active event set at state x. R x (i"1, 2, 2 , n) rei flects a service completion of a part at ¼S , i and R x reflects a departure of a product at ¼S n`1 n in order to meet the demand. ¹ a reflects a state i transition of the ith workstation. Clearly, A x"R x, D x"R x and R x"A D x 1 1 n n`1 i i i~1 if R 3! (x) for i"2, 3, 2 , n. In addition, i the following proposition concerning active set is clear. ¸emma 1. (a) R 3!(x), R 3!(x) for any x; and i n`1 R 3!(x) if and only if x '0. i i~1 (b) If R 3!(x), R 3!(x) and iOj, then R 3 i j i !(R x), R 3! (R x) and R R x"R R x. j j i i j j i 3. THE OPTIMAL CONTROL POLICY

Let S"M(x, a) D x3Z n`1]Z, a3Zn N be the 2 ` state space of the system. For our system, the state transition happens only if the event R x (i"1, i 2, 2 , n#1) or ¹ a (i"1, 2, 2 , n) happens, and i the ¼S is controllable only if ¼S is up (i.e. i i a (t)"1). Let )"Mu"(j , j , 2 , j ) D j 3[0, jM ]N i 1 2 n i i be the set of admissible controls at each state. It is assumed that control decision is made if and only if the system state transition happens and the workstation is up. In the remainder of this section, the continuous-time Markov chain problem is transformed into the equivalent discrete-time one, and the optimal dynamic equations are derived by the uniformization technique (Bertsekas, 1987; also cf. Song and Sun, 1996). Let l"k#+ i|n6 ( j1 #m #g ), where n"M1, 2, 2 , nN. For any i i i 6 given u3), the one-step transition probability function P ( ) D ), u) is given by P (yD(x, a), u)"

G

k/l if y"(R x, a); n`1 j /l if y"(R x, a), a "1, x '0; i i i i~1 m /l if y"(x,¹ a), a "1; i i i g /l if y"(x,¹ a), a "0; i i i j m g k i! + i! + i + 1! ! l l l l i|n6 ,ai/1,xi~1;0 i|n6 ,ai/1 i|n6 ,ai/0 if y"(x, a); 0 otherwise.

Let 0"t (t (2(t (2 be the potential 0 1 n state transition epochs. x " : x(t ) is the destination k k state of the kth transition. u " : u(t ) is the control k k decision of the kth transition. Thus, x(t)"x k and u(t)"u , if t3[t , t ). For the given conk k k`1 trol policy u(t), compute the objective function

(Bertsekas, 1987) E

=

=

P0 e~bt g(x(t)) dt"E k/0+ Pt

tk`1

k

e~bt g(x ) dt k

1 = " + hk Eg (x ), k b#l k/0 where h"l/(b#l). Now the equivalent discretetime Markov chain problem with positive but unbounded cost per step and infinite countable state space is established. By the dynamic programming approach, it yields the Bellman equation J (x, a)

C

1 " min g(x)#k J(D x, a) n b#l u |) j J (R x, a) # + i i i|n6 ,ai/1,xi~1;0 # + m J (x, ¹ a)# + g J (x, ¹ a) i i i i i|n6 ,ai/0 i|n6 ,ai/1

A

j # + ( j1 #m #g )! + i i i i i|n6 i|n6 ,ai/1,xi~1;0

B

! + m ! + g J (x, a) i i i|n6 ,ai/1 i|n6 ,ai/0 1 " g(x)#k J(D x, a) n b#l

C

D

# + (m J (x, ¹ a)#g J (x, a)) i i i i|n6 ,ai/1 # + ((m #j1 ) J (x, a)#g J (x, ¹ a)) i i i i i|n6 ,ai/0 # + j1 min M J (R x, a), J (x, a)N . (2) i i i|n6 ,ai/1 Clearly, the optimal control policy is of bang—bang type, that means j*"jM or 0. Without any loss of i i the generality, one can limit the admissible control set at each state to be )"Mu D u"(j , j , 2 , j ), 1 2 n j 3M0, jM NN. For this finite control set problem, i i next theorem is the direct result from Bertsekas (1987).

D

¹heorem 1. Let J (x, a)"0 and 0 J (x, a) k`1 1 " g(x)#k J (D x, a) k n b#l

C

# + (m J (x, ¹ a)#g J (x, a)) i k i i k i|n6 ,ai/1 # + ((m #j1 ) J (x, a)#g J (x, ¹ a)) i i k i k i i|n6 ,ai/0

D

# + j1 min MJ (R x, a), J (x, a)N . i k i k i|n6 ,ai/1

(3)

Optimal control of unreliable production lines Then

where

lim J (x, a)"J (x, a), (x, a)3S, k k?= where J (x, a) is defined in equation (1).

*g (l, x)"

¹heorem 2. For our system pictured in Fig. 1, the optimal control policy is of bang—bang type and switching structure, that is, u*"(j* , j* , 2 , j* ) in n 1 2 state (x, a), where j*" i jM J (R , x, a)4J (x, a) and x '0, a "1 i i i~1 i 0 otherwise.

G

Proof. It is the direct result of the Bellman equation (2). K 4. STRUCTURAL PROPERTIES

4.1. Switching manifolds of the optimal control policy Now we first investigate the monotone properties of the optimal value function from the Bellman Equation. Without any loss of the generality, let b#l"1. It will simplify our notation. Definition 1. J (x, a)!J (x@, a@) is called nonincreasing for R , if j J (x, a)!J (x@, a@)5J (R x, a)!J (R x@, a@) j j where R 3! (x) and R 3! (x@). If the above inj j equality reverses, we call J (x, a)!J (x@, a@) nondecreasing for R . j ¸emma 2. For every distinct pair (l, j ), we have: J (R x, a)!J (x, a) is nonincreasing for R , where l, l j j3M1, 2, 2 , n#1N and lOj. Proof. From Theorem 1, it yields lim J (x, a)" k?= k J (x, a), ∀ (x, a)3S, so the proof can be shown by induction on k using iteration equation (3). Since J (x, a),0, the assertion is true for k"0. Suppose 0 it holds for k, then we show that it also holds for k#1. From the iteration equation (3), we have J (R x, a)!J (x, a)"*g(l, x) k`1 l k`1 #[ k J (R R x, a)!kJ (R x, a)] k l n`1 k n`1 # + (m J (R x, ¹ a)#g J (R x, a) i k l i i k l i|n6 ,ai/1 !m J (x, ¹ a)!g J (x, a)) i k i i k # + ((m #jM ) (J (R x, a)!J (x, a)) i i k l k i|n6 ,ai/0 #g ( J (R x, ¹ a)!J (x, ¹ a))) i k l i k i # + j1 [min M J (R R x, a), J (R x, a)N i k i l k l i|n6 ,ai/1 !min MJ (R x, a), J (x, a)N], k i k

1051

G

!c #c if 14l4n!1, l~1 l !c #c` ) 1 Mx 50N!c~ ) 1 Mx (0N n n n n n~1 if l"n, !c` ) 1 Mx '0N#c~ ) 1 Mx 40N if l"n#1, n n n n where c " : and 1M ) N is the indicator function. It is 0 not difficult to verify that *g(l, x)5*g (l, R x), that j is, *g (l, x) is nonincreasing for R . By the induction j hypothesis, the second term, the third term and the forth term of the right-hand side of equation (4) are also nonincreasing for R . Now it suffices to show j that: For ∀ i3M1, 2, 2 , nN and a "1, i min M J (R R x, a), J (R x, a)N k i l k l !min M J (R x, a), J (x, a)N k i k 5min M J (R R R x, a), J (R R x, a)N k i j l k j l !min MJ (R R x, a), J (R x, a)N. k i j k j (A): If i"l, then min M J (R R x, a), J (R x, a)N k i l k l !min M J (R x, a), J (x, a)N k i k "min M J (R R x, a)!J (R x, a), 0N k l l k l #max M0, J (R x, a)!J (x, a)N. k l k Note that min and max preserve the monotonicity, by the condition R 3!(R x) and the induction j l hypothesis, it is nonincreasing for R if R 3! (R x). j l l If R N! (R x), it is clear. l l (B): If iOl, then it suffices to show that min M J ( R R x, a), J (R x, a)N k i l k l #min M J (R R x, a), J (R x, a)N k i j k j 5min M J (R R R x, a), J (R R x, a)N k i j l k j l #min M J (R x, a), J (x, a)N. (5) k i k (a) If i"j, then the above inequality is equivalent to min M0, J (R x, a)!J (R R x, a)N k l k j l #min MJ (R R x, a)!J (R x, a), 0N k j j k j 5min M J (R R R x, a)!J (R R x, a), 0N, k j j l k j l #min M0, J (x, a)!J (R x, a)N. (6) k k j If R 3! (R x), by the induction hypothesis and j j Lemma 1, we have

(4)

J (R x, a)!J (R R x, a)5J (x, a)!J (R x, a) k l k j l k k j J (R R x, a)!J (R x, a)5J (R R R x, a) k j j k j k l j j !J (R R x, a)"J (R R R x, a)!J (R R x, a). k l j k j j l k j l

1052

D.-P. Song and Y.-X. Sun

The above two inequalities yield directly the inequality (6). On the other hand, if R N! (R x), the j j inequality (6) is clear from R R x"R x and the j j j induction hypothesis. (b) If iOj. And if R 3! (x), then from Lemma 1 i and the induction hypothesis, we have J (R R x, a)#J (R R x, a)5J (R R R x, a) k i l k i j k i j l #J (R x, a)5RHS of (5), (7) k i J (R R x, a)#J (R x, a)"J (R R x, a) k i l k j k i l !J (R x, a)#J (R x, a)#J (R x, a) k l k l k j 5J (R R R x, a)!J (R R x, a) k i j l k j l #J (R x, a)#J (R x, a) k l k j 5J (R R R x, a)#J (x, a)5RHS of (5), (8) k i j l k J (R x, a)#J (R x, a)5J (R R x, a) k l k j k j l #J (x, a)5RHS of 5, (9) k J (R x, a)#J (R R x, a)"J (R R x, a) k l k i j k i j !J (R x, a)#J (R x, a)#J (R x, a) k j k j k l 5J (R R R x, a)!J (R R x, a) k i j l k j l #J (R x, a)#J (R x, a) k j k l 5J (R R R x, a)#J (x, a)5RHS of (5). (10) k i j l k The inequalities (7)—(10) lead to the inequality (5). If R N!(x), consider the case j"i!1. Note that i lOj, then we have R N! (R x) and equation (5) i l becomes J (R x, a)#min M J (R R x, a), J (R x, a)N k l k i j k j 5min M J (R R R x, a), J (R R x, a)N#J (x, a) k i j l k j l k that is J (R x, a)!J (x, a) k l k #min MJ (R R x, a)!J (R x, a), 0N k i j k j 5J (R R x, a)!J (R x, a) k j l k j #min MJ (R R R x, a)!J (R R x, a), 0N. (11) k i j l k j l Since R 3! (R x), R 3! (R R x) and lOi, lOj, l j l i j from Lemma 1 and the induction hypothesis, we have J (R R x, a)!J (R x, a)5J (R R R x, a) k i j k j k i j l !J (R R x, a). k j l That yields the inequality (11) and (5). For the case l"i!1 and the case jOi!1, lOi!1, the proof is very similar. Thus, the assertion holds for k#1. This completes the induction proof. K ¸emma 3. (a) J (R x, a)!J (x, a) is nondecreasing i or A ( j"i, i#1, 2 , n) and is nonincreasing for j A ( j"1, 2, 2 , i!1), where i"1, 2, 2 , n. In j

other words, J (R x, a)!J (x, a) is nondecreasing i in x ( j"i, i#1, 2 , n) and is nonincreasing in x j j ( j"1, 2, 2 , i!1); where i"1, 2, 2 , n. (b) J (R x, a)!J (x, a) is a nondecreasing for R , i i where i"1, 2, 2 , n. Proof. (a). From Lemma 2, for every distinct pair (i, j ), we have J (R x, a)!J (x, a)5J(R R x, a)!J (R x, a). i i j j Recall the definition of R , that yields i R 2 R R x"x; R 2R R x"D x; 2 1 n`1 j`2 j`1 j n`1 R 2 R R x"A x. j 2 1 j Then for any x s.t. x '0 and j5i, we have j J (R x, a)!J (x, a) i 5J (R R x, a)!J (R x, a) j`1 i j`1 5J (R R R x, a)!J (R R x, a) i`2 j`1 i j`2 j`1 525J (R 2 R R R x, a) n`1 j`2 j`1 i !J (R 2 R R x, a) n`1 j`2 j`1 "J (D R x, a)!J (D x, a) j i j "J (R D x, a)!J (D x, a). i j j That means J (R x, a)!J (x, a) is nondecreasing in i x ( j"i, i#1, 2 , n) for any feasible x. With the j same argument, we can show that it is nonincreasing in x ( j"1, 2, 2 , i!1). j (b) Note that R "A D , the assertion (b) can i i i~1 be derived simply from (a). This completes the proof. K Remark 1. Theorem 2 and Lemma 3 (a) state that for a given WS state a, if the optimal decision of ¼S is to produce with jM at state i i x"(x , x , 2 , x ), then so it is at the state 1 2 n (x , 2 , x , x #1, x , 2 , x , 2 , x ), where 1 j~1 j j`1 i n 14j(i; on the other hand, if the optimal decision of ¼S is to stop producing at the state i (x , x , 2 , x ), then so it is at the state 1 2 n (x , 2 , x , 2 , x , x #1, x , 2 , x ), where 1 i j~1 j j`1 n i4j4n. Intuitively, upstream WSs tend to produced nothing when inventories of downstream WSs increase. Define the switching manifolds as follows: z S (x , 2 , x , a)"max Mx : J (x, a) 1 2 n 1 x1 5J (R x, a)N; 1 z S (x , 2 , x , x , 2 , x , a) i 1 i~1 i`1 n "max Mx : J (x, a)5J (R x, a), x '0N, i i i~1 xi 1(i4n. If (x , 2 , x , x , 2 , x ) are given and 1 i~1 i`1 n J (x, a)(J (R x, a) for any x , then we define i i S (x , 2 , x , x , 2 , x , a) " : !1 for i"1, i 1 i~1 i`1 n 2, 2 , n!1, which means, the workstation ¼S i

Optimal control of unreliable production lines produces nothing. To simplify the notation, let (xC x , a) " : (x , 2 , x , x , 2 , x , a). From l 1 i~1 i`1 n Theorem 2 and Lemma 3(a), we have the following result. ¹heorem 3. The optimal control policy u*" (j* , j* , 2 , j* ) has the following switching struc1 2 n ture: jM , x3B (a) i , j*" i i 0, xNB (a) i where

G

B (a)"Mx : x 4S (xCx , a)N, i"1, 2, 2 , n. i i i i Remark 2. Theorem 3 states that the optimal policy can be described by the switching manifolds. For example, if the system state locates below the switching manifold S (xCx , a), then the optimal i i decision of ¼S is to produce at full speed; otheri wise, to stop producing. B (a) is called the optimal i control regions for ¼S . i It is worthwhile to investigate the structural properties of the switching manifolds, since they completely determine the optimal control policy. We study the further properties of the optimal value function in the Section 4.2 and then give the structural properties of the switching manifolds in Section 4.3. 4.2. Further properties of the optimal value function To simplify our narrative, with a slight abuse of the notation, for given l3M1, 2, 2 , nN denote and

(x, 1) " : (x, a), s.t. a "1; l

(x, 0) " : (x, a), s.t. a "0. l Then we have the following result ¸emma 4. J (x, 1)!J (x, 0) is nonincreasing for R , j where j3n and jOl. 1 Proof. Just like the proof of Lemma 2, we prove this proposition by induction on k using iteration equation (3). From Theorem 1, we have J (x, 1)!J (x, 0) k`1 k`1 "k [ J (R x, 1)!J (R x, 0)] k n`1 k n`1 #j1 [min M J (R x, 1), J (x, 1)N!J (x, 0)] l k l k k # + j1 [J (x, 1)!J (x, 0)] i k k i|n6 ,iOl,ai/0 # + m [J (x, 1, 0)!J (x, 0, 0)] i k k i|n6 ,iOl #g [J (x, 1, 1)!J (x, 0, 1)] i k k # + j1 [min MJ (R x, 1)!J (x, 1)N i k i k i|n6 ,iOl,ai/1 !min MJ (R x, 0), J (x, 0)N], (12) k i k

1053

where (x, 1, 0) " : (x, a) s.t. a "1, a "0 and (x, 0, 0), l i (x, 1, 1), (x, 0, 1) are similarly defined. Clearly, the 1st term, the 3rd term and the 4th term of the righthand side of equation (12) are nonincreasing for R by the induction hypothesis. On the other hand, j the second term of the right-hand side of equation (12) can be rewritten as follows: min MJ (R x, 1), J (x, 1)N!J (x, 0) k l k k "min MJ (R x, 1))!J (x, 1), 0N k l k #J (x, 1)!J (x, 0). k k From Lemma 2 and the induction hypothesis, the above expression is nonincreasing for R . Thus, it j suffices to show that the last term of the right-hand side of equation (12) is nonicreasing for R , i.e. j min M J (R x, 1), J (x, 1)N k i k !min MJ (R x, 0), J (x, 0)N k i k 5min M J (R R x, 1), J (R x, 1)N k i j k j !min MJ (R R x, 0), J (R x, 0)N, (13) k i j k j where a "1, i3n and iOl. i 1 equation (13) becomes If R N! (x), then i J (x, 1)!J (x, 0)5min M J (R R x, 1), k k k i j J (R x, 1)N!min MJ (R R x, 0), J (R x, 0)N. k j k i j k j The above inequality is true due to the following inequalities: J (x, 1)!J (x, 0)5J (R x, 1)!J (R x, 0) k k k j k j 5min MJ (R R x, 1), J (R x, 1)N!J (R x, 0), k i j k j k j J (x, 1)!J (x, 0)5J (R R x, 1)!J (R R x, 0) k k k i j k i j 5min MJ (R R x, 1), J (R x, 1)N!J (R R x, 0). k i j k j k i j In the following, we consider the case R 3! (x). i (A) If i"j, then equation (13) becomes min M J (R x, 1), J (x, 1)N!min MJ (R x, 0), J (x, 0)N k j k k j k 5min MJ (R R x, 1), J (R x, 1)N k j j k j !min M J (R R x, 0), J (R x, 0)N k j j k j i.e. minMJ (R R x, 0)!J (R x, 0), 0N k j j k j #max M0, J (R x, 0)!J (x, 0)N k j k 5minMJ (R R x, 1)!J (R x, 1), 0N k j j k j #max M0, J (R x, 1)!J (x, 1)N. (14) k j k If R N! (R x), the inequality (14) is clear. If j j R 3! (R x), then from the induction hypothesis, j j we have J (R R x, 0)!J (R x, 0)5J (R R x, 1) k j j k j k j j !J (R x, 1), k j J (R x, 0)!J (x, 0)5J (R x, 1)!J (x, 1). k j k k j k These two inequalities yield the inequality (14).

1054

D.-P. Song and Y.-X. Sun

(B) If iOj, then from Lemma 2 and the induction hypothesis, we have J (R R x, 1)!J (R x, 1)4J (R x, 1) k i j k j k i !J (x, 1)4J (R x, 0)!J (x, 0), k k i k J (R R x, 1)!J (R x, 1)4J (R R x, 0) k i j k j k i j !J (R x, 0)4J (R x, 0)!J (x, 0). k j k i k If J (R R x, 1)!J (R x, 1)50 or J (R x, 0)! k i j k j k i J (x, 0)40, then inequality (13) is quite easy to k show by the induction hypothesis. Otherwise, J (R R x, 1)!J (R x, 1)(0(J (R x, 0)!J (x, 0), k i j k j k i k then (13) can be simplified as min MJ (R x, 1), J (x, 1)N!J (x, 0)5J (R R x, 1) k i k k k i j !min MJ (R R x, 0), J (R x, 0)N. (15) k i j k j Note that J (R x, 1)!J (x, 0)"J (R x, 1)!J (R x, 0) k i k k i k i #( J (R x, 0)!J (x, 0)) k i k 5J (R R x, 1)!J (R R x, 0)#( J (R x, 0) k i j k i j k i !J (x, 0))'J (R R x, 1)!J (R R x, 0). k k i j k i j

(16)

The above first inequality follows from the induction hypothesis and the second inequality follows from the assumption J (R x, 0)!J (x, 0)'0. Morek i k over, it is easy to show that J (R x, 1)!J (x, 0)5J (R R x, 1)!J (R x, 0). k i k k i j k j (17) Then, (16) and (17) yield J (R x, 1)!J (x, 0)5RHS of equation (15) k i k (18) On the other hand, we have J (x, 1)!J (x, 0)5J (R x, 1)!J (R x, 0) k k k j k j 5J (R R x, 1)!J (R R x, 0) k i j k i j "J (R R x, 1)!J (R x, 0)#( J (R x, 0) k i j k j k j !J (R R x, 0))5J (R R x, 1)!J (R x, 0). k i j k i j k j (19) The above first and second inequalities follow from the induction hypothesis and the third inequality follows from the assumption J (R R x, 1)! k i j J (R x, 1)(0. Note that k j J (x, 1)!J (x, 0)5J (R x, 1)!J (R x, 0) k k k j k j 5J (R R x, 1)!J (R R x, 0). (20) k i j k i j Then, equations (19) and (20) yield J (x, 1)!J (x, 0)5RHS of equation (15). k k

(21)

From equations (18) and (21), the inequality (15) holds. This completes the proof of Lemma 4. K ¸emma 5. J (R x, a)!J (x, a)40 for any (x, a) s.t. n x (0. n Proof. It can be derived with the similar argument as Lemma 2. K ¸emma 6. (a) If c 4c , then J (R x, a)! l l~1 l J (x, a)40 for any feasible (x, a), where l"2, 3, 2 , n!1; , then J (R x, a)!J (x, a)40 for (b) If c`4c n n~1 n any feasible (x, a). Proof. (a) We show this statement by induction k using Theorem 2. The statement is true for k"0 due to J (x, a),0. Suppose it holds for k. Consider 0 the case of k#1. The iteration equation (3) yields J (R x, a)!J (x, a)"(c !c ) k`1 l k`1 l l~1 #[ k J (R R x, a)!k J (R x, a)] k l n`1 k n`1 # + (m J (R x, ¹ a)#g J (R x, a) i k l i i k l i|n6 ,ai/1 !m J (x, ¹ a)!g J (x, a)) i k i i k # + ((m #jM ) ( J (R x, a)!J (x, a)) i i k l k i|n6 ,ai/0 #g ( J (R x, ¹ a)!J (x, ¹ a))) i k l i k l # + j1 [min M J (R R x, a), J (R x, a)N i k i l k l i|n6 ,ai/1 !min M J (R x, a), J (x, a)N]. (22) k i k Clearly, the first term on the right-hand side of equation (22) is nonpositive by the condition of the assertion (a). By induction hypothesis, the second term and the third term both are nonpositive. Now it suffices to show that for any i3n , a "1 1 i min M J (R R x, a), J (R x, a)N k i l k l !min M J (R x, a), J (x, a)N40. (23) k i k If i"l, the inequality (23) is clear by the induction hypthesis. If iOl, and R 3! (x), then from Lemma 1 and i the induction hypothesis, we have: J (R R x, a)4 k i l J (R x, a) and J (R x, a)4J (x, a). That yields k i k l k equation (23) If i"l, and R N!(x), the inequality (23) is true i since R x"x and J (R x, a)4J (x, a). i k l k So the last term on the right-hand side of equation (22) is also nonpositive. Thus, we have J (R x, a)!J (x, a)40. k`1 l k`1 This completes the induction proof. The assertion (a) holds.

Optimal control of unreliable production lines (b) With the same argument, we can show the assertion (b) holds. K ¸emma 7. If c (c (2(c (c` , then for n 1 2 n~1 l4j, we have (a) lim J (R x, a)!J (x, a)'0 for any feasxj?`= l ible (x, a); (b) lim xn?~= J (R x, a)!J (x, a)'0 for any feasl xl?`= ible (x, a). Proof. (a) We first prove lim J (R x, a)! xj?`= k l J (x, a)50. From Theorem 1 and Lemma 3, lim J (R x, a)!J (x, a)" lim lim J (R x, a) l k l xj?`= xj?`= k?`= !J (x, a)" lim lim J (R x, a)!J (x, a). k l k k k?`= xj?`= So it suffices to show that for any k, lim J (R x, a)!J (x, a)50. We show it by xj?`= k l k induction on k. For k"0, it is clear. Suppose the assertion holds for k. Consider the case k#1. J (R x, a)!J (x, a)"(c !c ) k`1 l k`1 l l~1 #[ k J (R R x, a)!k J (R x, a)] k l n`1 k n`1 # + (m J (R x, ¹ a)#g J (R x, a) i k l i i k l i|n6 ,ai/1 !m J (x, ¹ a)!g J (x, a)) i k i i k # + ((m #jM ) ( J (R x, a)!J (x, a)) i i k l k i|n6 ,ai/0 #g ( J (R x, ¹ a)!J (x, ¹ a))) i k l i k i # + j1 [min M J (R R x, a), J (R x, a)N i k i l k l i|n6 ,ai/1 !min M J (R x, a), J (x, a)N]. (24) k i k Lemma 3 guarantees the existence of the limit of above expression as x P#R. From the condij tion and the induction hypothesis, the first four terms are not less than zero as x P#R. Now it j suffices to prove for any i3n , a "1 i 1 lim [min M J (R R x, a), J (R x, a)N —— k i l k l xj?`= !min M J (R x, a), J (x, a)N]50. (25) k i k Case 1: If i"l, it is clear from the induction hypothesis and Lemma 3. Case 2: If i"l#1, j"l, then lim J (R R x, a)!J (R x, a) k i l k i xj?`= " lim J (R R x, a)!J (R x, a)50, k l i k i xj?`= lim J (R x, a)!J (x, a)50. k l k xj?`= That yields the inequality (25). Case 3: If i"l#1, j'l, and R 3! (x), then the i argument is the same as Case 2.

1055

Case 4: If i"l#1, j'l, and R N! (x), then i R x"x and equation (25) becomes i lim —— min M Jk (Rl`1 Rl x, a), Jk (Rl x, a)N xj?`= !J (x, a)50. (26) k On the other hand, by the condition and the induction hypothesis, we have lim J (R R x, a)!J (R x, a)50, k l`1 l k l xj?`= lim J (R x, a)!J (x, a)50. k l k xj?`= It follows that equation (26) is true. Case 5: If iOl, l#1, and R 3! (x), then the i argument is the same as Case 2. Case 6: If iOl, l#1, and R N! (x), then R x"x i i and R R x"R x and (25) is clear. i l l Hence, we complete the induction proof. J (R x, a)!J (x, a)50. Now let That is, lim l xj?`= kP#R on both sides of equation (24) and then let x P#R, then the right hand side is positive j due to the condition c !c '0. That means l l~1 the left hand side is also positive, i.e. lim J (R x, a)!J (x, a)'0. xj?`= l (b) With the similar argument in (a) we can prove this assertion. K

4.3. Structural properties of the switching manifolds Now we can establish some useful structural properties of the switching manifolds, such as monotonicity, asymptotic behavior and the relationship of the different switching manifolds. ¹heorem 4. S (xCx , a) is nondecreasing in x ( j"1, i i j 2, 2 ,i!1) and nonincreasing in x ( j"i#1, 2 , n). j i"1, 2, 2 , n. Proof. Consider S (xCx , a). From the definition, 1 1 we have S (xCx , a)"S (x , x , 2 , x , a) 1 1 1 2 3 n "max Mx : J (R x, a)!J (x, a)40N, 1 1 x1 S (x , 2 , x , x !1, x , 2 , x , a) 1 2 j~1 j j`1 n "max Mx : J (R D x, a)!J (D x, a)40N. 1 1 j j x1 By Lemma 3(a), it follows: J (R x, a)!J (x, a)5J (R D x, a)!J (D x, a) 1 1 j j that yields S (xCx , a) 1 1 4S (x , 2 , x , x !1, x , 2 , x , a). 1 2 j~1 j j`1 n Thus, S (xCx , a) is nonincreasing in x for 1 1 j j"2, 3, 2 , n. The same arguments can prove the

1056

D.-P. Song and Y.-X. Sun

assertions of the other switching manifolds S (xCx , a) for i"2, 3, 2 , n. K i i Denote (xCx , 1) " : (xCx , a), s.t. a "1 and i i l (xCx , 0) " : (xCx , a), s.t. a "0. We have: i i l ¹heorem 5. S (xCx , 1)5S (xCx , 0) for i3n and i i i i 1 iOl. Proof. This is the direct result from Lemma 4. K Remark 3. Theorem 5 states the relationship of the switching maifolds under different system states, especially under the states with some WS up and down. In general, it says that the more the workstations are down, the lower the switching manifolds locate in the state space. For a given WS state a, let b denote the WS state in which some WSs become down but they are up in a. Then Theorems 5 and 3 yield: S (xCx , a)5S (xCx , b) and B (b)-B (a). i i i i i i In other words, if the optimal decision for ¼S is to i produce at its greatest rate at state (x, b), then, so it is at the state (x, a). On the other hand, if the optimal decision for ¼S is to stop producing at the i state (x, a), then so it is at the state (x, b).

That implies, S (xCx , a)4xN (a) when x 4x (a). l l l n 1n Since S (xCx , a) is nondecreasing upper bounded l l as x P!R, the assertion (b) is true. n (c) If x "#R, it means that the last WS has n~1 infinite part resource. In this case, the control decision of ¼S is equivalent to one-WS system. From n the result of Song and Sun (1996), S (xCx , a) conn n verges to a finite number as x P#R. That n~1 yields the assertion (c). K Remark 4. Intuitively, x "#R means there are j infinite inventories in the buffer of ¼S , thus, the j upstream workstations should remain zero-inventory. That is, S (xCx , a)"!1 for x "#R and l l j l(j. 5. TWO-WORKSTATION SYSTEM

In this section, we turn to the particular two-WS system to illustrate the structural properties of the switching curves, which can give intuitive ideas. Actually, the optimal policy can be described by two monotone curves for the two-WS system (n"2). We have, x"(x , x ) and u"(j , j ). 1 2 1 2 Their switching curves can be rewritten as follows:

¹heorem 6. (a) S (xCx , a)5!1 for any (x, a) s.t. n n a "1. n (b) If c 5c , then S (xCx , a)"#R for any l~1 l l l feasible (x, a), where l"2, 3, 2 , n!1; (c) If c 5c` , then S (xCx , a)"#R for n n n n~1 any feasible (x, a).

S (x , a)"max Mx D J (x, a)5J (R x, a)N; 1 2 1 1 x1 S (x , a)"max Mx D J (x, a)5J (R x, a)N. 2 1 2 2 x2 If x is given and J (x, a)(J (R x, a) for any x , 2 1 1 then, let S (x , a)"!1. 1 2

Proof. It can be derived from Lemma 5 and Lemma 6. K

Corollary 1. The switching curves S (x , a) is 1 2 nonincreasing and S (x , a) is nondecreasing. 2 1

¹heorem 7. If c (c (2(c (c` , then for n 1 2 n~1 l(j and j"2, 3, 2 , n, we have

Corollary 2. S (x , 1, 1)5S (x , 1, 0) and S (x , 1, 1) 1 2 1 2 2 1 5S (x , 0, 1). 2 1

(a) S (xCx , a) converges to !1 as x P#R. l l j (b) S (xCx , a) converges to a finite asymptote as l l x P!R. n (c) S (xCx , a) converges to a finite asymptote as n n x P#R. n~1 Proof. (a) For a given (x , 2 , x , x , 2 , x ) 1 j~1 j`1 n s.t. x "0, Lemma 7(a) implies that there exists an l xN (a), j J (R x, a)!J (x, a)'0 when x 5xN (a). l j j Recall Lemma 3(a), for any x we have l J (R x, a)!J (x, a)'0, when x 5xN (a). It follows l j j that S (xCx , a)"!1 when x 5xN (a) and the asl l j j sertion (a) holds. (b) For a given (x , 2 , x , x , 2 , x ), 1 l~1 l`1 n~1 Lemma 7(b) yields that there exist x (a) and xN (a) l 1n J (R x, a)!J (x, a)'0 when x 4x (a) l n 1n and x 5xN (a). l l

Corollary 3. (a) S (x , a)5!1 for any x and 2 1 1 a "1. 2 (b) If c 5c` , then S (x , a),#R. 2 2 1 1 (c) If c (c` , then S (x , a) converges to a finite 2 2 1 1 asymptote as x P#R. 1 (d) If c (c` , then S (x , a) converges to !1 2 1 2 1 as x P#R. 2 (e) If c (c` , then S (x , a) converges to a finite 2 1 2 1 asymptote as x P!R. 2 Remark 5. For (c)—(e), due to the monotone property of the switching curves as well as the fact that the switching curves only take integer, there actually exist finite integers (xN (a), x* (a)), xN (a) and 2 2 1 ( x (a), x* (a)), such that 1 2 1 S (x , a)"x* (a) if x 5xN (a), 2 1 1 2 1 !1 if x 5xN (a), 2 2 S (x , a)" 1 2 x* (a) if x 4x (a). 1 2 12

G

Optimal control of unreliable production lines

1057

Based on these asymptotic properties of the switching curves, the control regions can be simplified by

and

B (a)"Mx: x 4xN (a), x 4x* (a)N 1 2 2 1 1 B (a)"Mx: x 4x* (a)N. 2 2 2

Fig 2. Optimal switching curves and optimal control regions.

This information gives the useful insights of the optimal control regions. In fact, Theorem 3 for two-WS system can be described as follows. Corollary 4. The optimal control policy u*" (j* , j* ) for two-WS system is 1 2 jM , x3B (a), jM , x3B (a), 1 2 j*" 1 j*" 2 1 2 0, xNB (a), 0, xNB (a), 1 2 where B (a)"Mx: x 4S (x , a)N and B (a)" 1 1 1 2 2 Mx: x 4S (x , a)N. See Fig. 2. 2 2 1

G

G

6. SUBOPTIMAL CONTROL POLICIES

In reality, we always come across the systems with large number of workstations, which are difficult to derive the exact optimal control policy. On the other hand, even if we could obtain the exact optimal control policy, it may be too complex to be implemented in practice. A trade-off method is to find a simple suboptimal control policy, which is easy to implement and operate. In this Section, we aim to estabish some suboptimal control policies based on the structural properties of the optimal switching manifolds. In fact, these suboptimal policies are featured only by several threshold values. 6.1. ¹wo-¼S system In Corollary 4 and Fig. 2, the optimal control regions has been detailed outlined. Especially, the asymptotic behavior of the switching curves plays an important role in determining the optimal control regions. Actually, from Remark 5 there exist finite integers (xN (a), x* (a)), xN (a) and ( x (a), 1 2 2 12 x* (a)), such that 1 S (x , a)"x* (a) if x 5xN (a), 2 1 2 1 1 !1 if x 5xN (a), 2 2 S (x , a)" 1 2 x* (a) if x 4x (a). 1 2 12

G

That means, the control decision of the second WS is featured only by one threshold value x* (a). It 2 produces at its greatest rate if the buffer level of ¼S is not greater than x* (a), otherwise, it stops 2 2 producing. The control decision of ¼S is as fol1 lows: It produces at its greatest speed if the first buffer level is not greater than x* (a) and the second 1 buffer level is less than xN (a); otherwise, it stops 2 producing. Furthermore, note that ¼S stops producing if 2 x 5x* (a), it yields that the buffer level of ¼S will 2 2 1 increase if ¼S still produces. And at last it results 1 in the violation of inequalities x 4x* (a), so ¼S 1 1 1 is forced to produce nothing. Hence, we can further simplify the control region of B (a) as follows: 1 B (a)"Mx: x 4x* (a)N. 1 1 1 In summary, we get two simple threshold control policies: 1. Buffer Threshold Control (BTC)

G G

j1 x 4b (a), 1 1 j* (x, a)" 1 1 0 otherwise, j1 x 4b (a), 2 2 j* (x, a)" 2 2 0 otherwise. 2. Multiple Threshold Control (MTC)

G

j1 , j* (x, a)" 1 1 0,

x (b (a), x 4b (a), 1 11 2 12 otherwise,

G

j1 , x (b (a), 2 2 j* (x, a)" 2 2 0, otherwise. We call the first policy Buffer ¹hreshold Control, because the decision of any workstation only depends on its own buffer levels. However, the second control policy depends on all the buffer levels of downstream workstations. For instance, see the Example 1(b) in Section 7. For the workstation state (1, 0), if we let b (1, 0)"1 and b (1, 0)"!1 , then MTC policy 11 12 is very close to the optimal control. For the second workstation, if let b (a)"1, then the correspond2 ing threshold control is exactly the optimal control. 6.2. Multiple workstation system Similar to the two workstation system, we can summarize the following simple threshold control

1058

D.-P. Song and Y.-X. Sun

policies for n-workstation system using the structural properties of the switching manifolds in Section 4.3. 1. Buffer Threshold Control (BTC)

G

j1 , x 4b (a) i i , j* (x, a)" i i 0, otherwise

for i"1, 2, 2 , n.

2. Multiple Threshold Control (MTC)

G

j1 , x 4b (a), j"i, i#1, 2 , n i ij j* (x, a)" i , i 0, otherwise for i"1, 2, 2 , n. The first control policy is quite similar to the Kanban control system in Yan et al. (1994), which is developed for the flow rate control model. But the second policy is new. Clearly, MTC policy is better than BTC policy since BTC is only one of the simplest forms derived from MTC. On the other hand, it is possible to further simplify the threshold structure of the MTC to yield other simple suboptimal policies. Remark 6. (a) If the WS state a is given, then the control decision for each WS is only determined by one threshold value under BTC; the decision for ¼S is determined by n!i#1 threshold values i under MTC. They are quite simple to realize in pracitcal systems. (b) It should be pointed that, b (a) and b (a) i ij depend on the WS state a. However, according to Theorem 5, they should decrease if more WSs become down. (c) Due to the computation difficulty, it is not realistic to compute the exact optimal policy for the system with large n. However, by introducing above threshold control policies, the difficulty is reduced. One only needs to find several appropriate threshold parameters rather than the detailed switching manifolds. In addition, it is quite possible to develop effective algorithms to find these parameters of BTC and MTC. For instance, Perturbation Analysis technique, Likelihood Ratio technique and other methods developed recently may be applied to find the threshold values. Actually, some authors have applied perturbation analysis technique to optimize the threshold values for suboptimal policies in continuous flow manufacturing systems (Caramanis and Liberopoulos’ 1992; Song et al., 1992; Yan et al., 1994; Liberopoulos and Caramanis, 1995). 7. NUMERIC EXAMPLES

In this section, we use value iteration to compute the optimal control policies and cost functions for

some test systems. In order to perform the iteration, the state space should be limited in a finite region. Example 1. Two-WS systems with j1 "0.8, 1 b"0.5, k"0.5, c "1 and m "m "0.2, 1 1 2 g "g "0.8. The state space is limited in 60]60, 1 2 that is, 04x (60 and !304x (30. The 1 2 switching curves defined in Section 5 under different j , k, c` and c~ are computed as follows: 2 2 2 (a) If j1 "0.8, c`"1, c~"4, then 2 2 2 S (x , 1, 1)" 1 2 !1 if x 51, 2 0 if x "0, !1, S (x , 1, 1),#R; 2 2 1 1 if x 4!2, 2 S (x , 1, 0)" 1 2 !1 if x 50, 2 S (x , 0, 1),#R. 2 1 0 if x 4!1, 2 (b) If j1 "0.8, c`"2, c~"8, then 2 2 2 S (x , 1, 1)" 1 2 !1 if x 51, 2 0 if x "0, 2 S (x , 1, 1),1. 2 1 1 if !34x 4!1, 2 2 if x 4!4, 2 S (x , 1, 0)" 1 2 !1 if x 51, 2 0 if x "0, !1, S (x , 0, 1),1. 2 2 1 1 if x 4!2, 2 (c) If j1 "0.6, c`"2, c~"4, then 2 2 2 S (x , 1, 1) 1 2 !1 if x 51, 2 " S (x , 1, 1),0; 2 1 0 if x 40, 2 S (x , 1, 0) 1 2 !1 if x 50, 2 " S (x , 0, 1),0. 2 1 0 if x 4!1, 2 As an example, the switching curves of (b) are pictured in Fig. 3, where (A) shows the switching curves while both workstations are up, (B) shows the switching curve while the first WS is up and the second is down, and (C) shows the switching curve while the first WS is down and the second is up. The shadow area in Fig. 3 shows the control region for the corresponding workstation under certain WS state.

G

G

G

G

G

G

Example 2. Consider a three-WS systems with j1 "j1 "j1 "0.8, k"0.6, b"0.5, m "0.2, 1 2 3 3 g "0.8. The first two workstations are assumed 3 reliable. That means, only the last WS is failureprone. The state space is limited in 60]60]60, i.e.

Optimal control of unreliable production lines x 3[0, 60), x 3[0, 60) and x 3[!30, 30). Let 1 2 3 c "1, c "2, c`"4, c~"12. Then, the switch3 3 1 2 ing surfaces S (x , x ), S (x , x ) and S (x , x ) 1 2 3 2 1 3 3 1 2 are given in Tables 1—5 (here to simplify our

1059

notation, we omit the WS state in the switching surfaces). Example 1 and Fig. 3 illustrate the results presented in Section 5. Example 2 demonstrates the monotonicity and asymptotic behaviors of switching curves given in Section 4.3. It is clear that S (x , x ) is noninecreasing in x and x , S (x , x ) 1 2 3 2 3 2 1 3 is nondecreasing in x and nonincreasing in x and 1 3 S (x , x ) is nondecreasing in x and x whether or 3 1 2 1 2 not the WS is up or down. And the asymptotic behaviors of S (x , x ), S (x , x ) and S (x , x ) 1 2 3 2 1 3 3 1 2 shown in Tables 1—5 are consistent with Theorem 7. In addition, the switching surfaces when the third WS is up locate above those when the third WS is down. Table 5. The switching surface S (x , x ) while the third WS is up3 1 2 x 2

x "0 1 ,x ) S (x 3 1 2

x "1 1 ,x ) S (x 3 1 2

x "2 1 ,x ) S (x 3 1 2

x 53 1 ,x ) S (x 3 1 2

0 1 1

0 1 1

0 1 1

0 1 1

1 2 53 Fig 3. The switcing curves of case (b).

Table 1. The switching surface S (x , x ) while the third WS is up 1 2 3 x 2 0 1 2 53

x 4!3 3 S (x , x ) 1 2 3

x "!2 3 S (x , x ) 1 2 3

x "!1 3 S (x , x ) 1 2 3

x "0 3 S (x , x ) 1 2 3

x "1 3 S (x , x ) 1 2 3

x "2 3 S (x , x ) 1 2 3

x 53 3 S (x , x ) 1 2 3

0 0 !1 !1

0 0 !1 !1

0 0 !1 !1

0 !1 !1 !1

!1 !1 !1 !1

!1 !1 !1 !1

!1 !1 !1 !1

Table 2. The switching surface S (x , x ) while the third WS is down 1 2 3 x 2 0 1 52

x 4!3 3 S (x , x ) 1 2 3

x "!2 3 S (x , x ) 1 2 3

x "!1 3 S (x , x ) 1 2 3

x "0 3 S (x , x ) 1 2 3

x "1 3 S (x , x ) 1 2 3

x "2 3 S (x , x ) 1 2 3

x 53 3 S (x , x ) 1 2 3

0 !1 !1

0 !1 !1

0 !1 !1

0 !1 !1

!1 !1 !1

!1 !1 !1

!1 !1 !1

Table 3. The switching surface S (x , x ) while the third WS is up 2 1 3 x 1 1 2 53

x 4!4 3 S (x , x ) 2 1 3

x "!3 3 S (x , x ) 2 1 3

x "!2 3 S (x , x ) 2 1 3

x "!1 3 S (x , x ) 2 1 3

x "0 3 S (x , x ) 2 1 3

x "1 3 S (x , x ) 2 1 3

x 52 3 S (x , x ) 2 1 3

2 2 2

1 2 2

1 1 1

1 1 1

0 0 0

0 0 0

!1 !1 !1

Table 4. The switching surface S (x , x ) while the third WS is down 2 1 3 x 1 1 2 53

x 4!4 3 S (x , x ) 2 1 3

x "!3 3 S (x , x ) 2 1 3

x "!2 3 S (x , x ) 2 1 3

x "!1 3 S (x , x ) 2 1 3

x "0 3 S (x , x ) 2 1 3

x "1 3 S (x , x ) 2 1 3

x 52 3 S (x , x ) 2 1 3

1 1 1

1 1 1

1 1 1

0 0 0

0 0 0

!1 !1 !1

!1 !1 !1

1060

D.-P. Song and Y.-X. Sun 8. CONCLUSION

It has been shown that the optimal policy for the n-WS system pictured in Fig. 1 is of bang—bang type, state feedback and can be described by n switching manifolds. The structural properties, such as monotonicity and asymptotic behaviors of the switching manifolds are studied. The relationship of the switching manifolds at different WS up and down states is established. Further interesting questions arise in this field such as: (a) how to generalize the results to more complex systems such as multiple-part-type system; (b) is it possible to generalize the distributions of the service time, the time between demands, the WS up time and the WS down time? (c) how to deal with the similar problem if the workstations are operation dependent? (d) what is the optimal control policy for other objective functions? for instance, the expected average cost. etc. Acknowledgements—The authors would like to express their grateful acknowledgement to the Associate Editor and the anonymous referees for thir careful reading and constructive suggestions, which were very helpful in improving this paper. The work was supported in part by the National Science Foundation under Grant No 69635010 and in part by the National Lab. of Ind. Control Tech. at Zhejiang University under Grant No K97X01. REFERENCES Akella, R. and P. R. Kumar (1986). Optimal control of production rate in failure prone manufacturing system. IEEE ¹rans. Automat. Control, AC-31(2), 116—126. Algoet, P. H. (1989). Flow balance equations for the steady-state distribution of a flexible manufacturing system. IEEE ¹rans. Automat. Control, AC-34(8), 917—921. Baccelli, F., G. Cohen, G. J. Olsder and J. P. Quadrat (1992). Synchronization and ¸inearity—An Algebra for Discrete Event Systems. Wiley, Chichester. Bertsekas, D. P. (1987). Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Englewood Cliffs, NJ. Bielecki, T. and P. R. Kumar (1988). Optimality of zero-inventory policies for unreliable manufacturing systems. Oper. Res. 36(4), 532—541. Boukas, E. and A. Haurie (1990). Manufacturing flow control and preventive maintenance: a stochastic control approach. IEEE ¹rans. Automat. Control, AC-35(9), 1024—1031. Boukas, E., A. Haurie and C. van Delft (1991). A turnpike improvement algorithm for piecewise deterministic control. Optim. Control. Appl. Meth., 12, 1—18. Boukas, E. and H. Yang (1996). Optimal control of manufacturing flow and preventive maintenance. IEEE ¹rans. Automat. Control, AC-41(6), 881—885. Boukas, E., Q. Zhang and G. Yin (1995). Robust production and maintenance planning in stochastic manufacturing systems. IEEE ¹rans. Automat. Control, AC-40(6), 1098—1102. Caramanis, M. and G. Liberopolous (1992). Perturbation analysis for the design of flexible manufacturing system flow controllers. Oper. Res., 40(6), 1107—1125. Gershwin, S. B. (1989). Hierarchical flow control: a framework for scheduling and planning discrete events in manufacturing systems. Proc. IEEE, 77(1), 195—209. Gershwin, S. B. (1994). Manufacturing Systems Engineering. Prentice-Hall, Englewood Cliffs, NJ. Glasserman, P. and D. D. Yao (1994). Monotone Structure in Discrete-Event Systems. Wiley, New York.

Hajek, B. (1984). Optimal control of two interacting service stations. IEEE ¹rans. Automat. Control, AC-29, 491—498. Haurie, A. and C. Van Delft (1991). Turnpike properties for a class of piecewise deterministic control systems, Ann. Oper. Res., 29, 351—374. Haurie, A., A. Leizarowitz, and C. Van Delft (1994). Boundedly optimal control of piecewise deterministic processes. Eur. J. Oper. Res., 73, 237—251. Hu, J. Q., P. Vakili, and G. Yu (1994). Optimality of hedging point policies in the production control of failure prone manufacturing systems. IEEE ¹rans. Automat. Control, AC39(9), 1875—1880. Hu, J. Q. and D. Xiang (1993). A queuing equivalence to optimal control of manufacturing systems with failures. IEEE ¹rans. Automat. Control, AC-38, 499—502. Hu, J. Q. and D. Xiang (1994). Structural properties of optimal production controllers in failure-prone manufacturing systems. IEEE ¹rans. Automat. Control, AC-39(3), 640—643. Hu, J. Q. and D. Xiang (1995). Optimal control for systems with deterministic production cycles. IEEE ¹rans. Automat. Control, AC-40(2), 782—786. Kimemia, J. G. and S. B. Gershwin (1983). An algorithm for the computer control of production in flexible manufacturing system. IIE ¹rans. 15(4), 353—362. Liberopolous, G. and M. Caramanis (1995). Dynamics and design of a class of parameterized manufacturing flow controllers. IEEE ¹rans. Automat. Control, AC-40(6), 1018—1028. Lou, S. X. C. and G. V. Ryzin (1989). Optimal control rules for scheduling job shops. Ann. Oper. Res. 17, 233—248. Lou, S. X. C., S. Sethi and Q. Zhang (1994). Optimal feedback production planning in a stochastic two-machine flowshop. Eur. J. Oper. Res., 73, 331—345. Miyoshi, N., Ohnisi and N. Okino (1993). Optimal service control of a station connected with two parallel substations. IEEE ¹rans. Automat. Control, 38, 1567—1572. Presman, E., S. Sethi and Q. Zhang (1995). Optimal feedback production planning in a stochastic N-machine flowshop. Automatica, 31(9), 1325—1332. Ryzin, V., S. X. C. Lou and S. B. Gershwin (1993). Production control for a tandem two-machine system. IIE ¹rans. 25, 5—20. Sethi, S. P., H. M. Soner, Q. Zhang and J. Jiang (1992a). Turnpike sets and their analysis in stochastic production planning problems. Math. Oper. Res., 17, 932—950. Sethi, S and Q. Zhang (1994). Hierarchical Decision Making in Stochastic Manufacturing Systems. Birkhauser, Boston. Sethi, S., Q. Zhang and X. Y. Zhou (1992b). Hierarchical controls in a stochastic two-machine flowshop with a finite internal buffer. In Proc. 31st IEEE CDC, Tuscson, AZ, U.S.A., pp. 2074—2079. Sharifnia A. (1988). Production control of a manufacturing system with multiple machine states IEEE ¹rans. Automat. Control, AC-33(7), 620—625. Song, D. P., F. S. Tu and S. X. C. Lou (1992). Parameter optimization of a control policy for unreliable manufacturing systems. In Proc. 31st IEEE CDC, Tuscson, AZ, U.S.A., pp. 1655—1656. Song, D. P. and Y. X. Sun (1996). Optimal control structure of an unreliable manufacturing system and random demands. In Proc. 35th IEEE CDC, Kobe, Japan, pp. 4553—4557. Song, D. P., Y. X. Sun and W. Xing (1998). Optimal control of a stochastic assembly production line J. Optim. ¹heory Appl., 98(3). Tu, F. S., D. P. Song and S. X. C. Lou (1993). Preventive hedging point control and its realization. In the 12th IFAC ¼orld Congress, Vol. 5, Sydney, Autralia, pp. 13—16. Weber, R. R. and S. Stidham (1987). Optimal control of service rates in networks in queues. Adv. Appl. Probab., 19, 202—218. Xu, S. H. and H. Chen (1993). On the asymptote of the optimal routing policy for two service stations. IEEE ¹rans. Automat. Control, 38, 187—189. Yan, H., G. Yin and S. X. C. Lou (1994). Using stochastic optimization to determine threshold values for the control of unreliable manufacturing systems. J. Optim. ¹heory Appl., 83(3), 511—539.