- Email: [email protected]
Dynamic vehicle allocation control for automated material handling system in semiconductor manufacturing
Computers & Operations Research 40 (2013) 2329–2339
Contents lists available at SciVerse ScienceDirect
Computers & Operations Research journal homepage: www.elsevier.com/locate/caor
Dynamic vehicle allocation control for automated material handling system in semiconductor manufacturing James T. Lin a,n, Cheng-Hung Wu b, Chih-Wei Huang a a Department of Industrial Engineering and Engineering Management, National Tsing-Hua University, 101, Section 2, Kuang-Fu Road, Hsinchu 300, Taiwan, ROC b Institute of Industrial Engineering, National Taiwan University, 1, Section 4, Roosevelt Road, Taipei 10617, Taiwan, ROC
art ic l e i nf o
a b s t r a c t
Available online 24 April 2013
The current study examines the dynamic vehicle allocation problems of the automated material handling system (AMHS) in semiconductor manufacturing. With the uncertainty involved in wafer lot movement, dynamically allocating vehicles to each intrabay is very difficult. The cycle time and overall tool productivity of the wafer lots are affected when a vehicle takes too long to arrive. In the current study, a Markov decision model is developed to study the vehicle allocation control problem in the AMHS. The objective is to minimize the sum of the expected long-run average transport job waiting cost. An interesting exhaustive structure in the optimal vehicle allocation control is found in accordance with the Markov decision model. Based on this exhaustive structure, an efficient algorithm is then developed to solve the vehicle allocation control problem numerically. The performance of the proposed method is verified by a simulation study. Compared with other methods, the proposed method can significantly reduce the waiting cost of wafer lots for AMHS vehicle transportation. & 2013 Elsevier Ltd. All rights reserved.
Keywords: AMHS Vehicle allocation control Markov decision process Simulation
1. Introduction For the past 20 years, the automated material handling system (AMHS) has been successfully applied in semiconductor manufacturing factories. Related technological developments have generated large-scale, complicated transport systems with expanded capabilities that can be used for storage and wafer transport. The AMHS is a solution to fabrication (fab) automation in 300-mm semiconductor manufacturing, which is capable of optimizing productivity, improving tool utilization and ergonomics, as well as reducing particle contamination and vibration shocks on wafers. A typical AMHS consists of pods, vehicles, track system, stocker system, and material control system (MCS). Front opening unified pods (FOUPs) with a carrying capacity of 25 wafers are used to transport wafers in a 300-mm wafer fab. These 25 wafers form a manufacturing batch size, referred to as a wafer lot, for processing in the tool and are stored in the stocker. The MCS is a software system used to control vehicle routing, dispatching, and allocating. It is also used for transport job control of the material. The primary function of the AMHS is to execute the physical material handling jobs that transport wafers to their destinations, including tools and their local stockers. In addition, the AMHS stores wafers into the stockers.
n
Corresponding author. Tel.: +886 35742650. E-mail address: [email protected] (J.T. Lin).
0305-0548/$ - see front matter & 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.cor.2013.04.007
In semiconductor manufacturing, wafer movement occurs between and within different processing areas, such as thin-film, etching, diffusion, and photo. Wafers typically go through over 250 processing tools to undergo several hundreds of processing steps [1]. The general transport layout in the 300-mm semiconductor factory is developed in the form of a connecting transport AMHS to provide a higher transport volume and quick delivery time because of the highly reentrant nature and long processing flow of wafer fab [11,12]. The connecting transport AMHS layout (Fig. 1) shows the tools in different processing areas installed along several intrabay tracks. Each intrabay track is connected by a central interbay track. This configuration has stockers located at the junction of the interbay and each intrabay. The vehicles directly move wafer lots to various locations without passing through stockers, thus reducing the wafer transport time more efficiently compared with the AMHS without connecting interbay and intrabay tracks. With the complicated processing flow of wafer lots processed in different areas, wafer lot transport continuously occurs at different intrabays. The MCS controls hundreds of vehicles to perform different transport jobs between intrabays. However, wafer lots have to wait for a considerable amount of time for the vehicle to arrive, especially when vehicles rarely occur at the intrabay, affecting the cycle time and overall tool productivity of the wafer lots. These unpredictable transport demands of the wafer lots are highly related to the product type, work-in-process pool, availability of tools, tool dispatch strategies, and other
2330
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
Fig. 1. Representation of the connecting transport AMHS.
factors. The dynamic allocation of the vehicles to each intrabay is very difficult considering the uncertainty of wafer movement despite the direct movement of all vehicles within the connecting transport AMHS. Currently, fab managers apply an empirical method to establish the MCS system parameters in allocating all vehicles to each intrabay. Despite its periodic modification the MCS parameters, this method has remarkably influenced AMHS transport performances in which assumptions or conditions do not exist. Thus, dynamic allocation of vehicles has become a significant management topic for wafer fab. Previous studies have investigated improvements in AMHS design and performance. The AMHS vehicle management problem in these studies can be categorized into three: vehicle fleet size, vehicle dispatching rule, and vehicle routing decision. The vehicle fleet size problem involves the determination of the minimum number of vehicles required to transport the wafer lots in the factory. Maxwell and Muckstadt [14] formulated the automated guided vehicle (AGV) transport problem into a mathematical model to minimize the total travel times of empty vehicles. They determined the minimum number of vehicles that would satisfy a specific demand requirement. The use of dispatch rules to control vehicles minimizes vehicle waiting time and maximizes system throughput. Egbelu and Tanchoco [5] proposed two aspects of the AGV dispatching problem: work center-initiated versus vehicleinitiated task-assignment problems. Improved performances have been observed from the simulation results of the combined modified first-come, first-served (FCFS) rule and the nearest vehicle (NV) or longest idle vehicle (LIV) rules. Liao and Fu [10] proposed a two-phase approach to selecting the best vehicle dispatch policy and determining the number of vehicles. Their simulation results showed satisfactory performances of both transport and delivery time. The vehicle routing decision problem chooses the vehicle route and the job sequence to visit, thus minimizing the transport time of wafer lots. Baker and Ayechew [2] used a generic algorithm to solve basic vehicle routing problems and obtained solutions that were 0.5% above the bestknown results. Renaud et al. [16] applied meta-heuristics and tabu search, which were successfully implemented in other vehicle routing studies. Both meta-heuristics and tabu search were reported to perform efficiently. The occurrence of vehicle control problems over the past few years is discussed as the connecting transport AMHS is applied to the 300-mm fab. Lin et al. [11,12] first proposed four types of vehicles (A, B, C, and D) to perform the connecting transport tasks by using three combinations, namely, vehicles A–B, A–C, and A–D. They calculated the minimum number of vehicle types with a simple procedure. To improve the vehicle waiting time, Wang et al. [17] proposed the control of the upper or lower limit on the number of vehicles in the intrabay. Their results indicated that the control significantly affected travel time, waiting time, and empty vehicle utilization.
The above-mentioned studies only examined the optimization of vehicle number, dispatching, routing, and types. Thus, the relationship between the transport demand and AMHS states is not further discussed. Meanwhile, the explicit consideration of the real-time production and the AMHS states for vehicle control management is very important. Markov decision processes (MDPs) are useful in the real-time control of dynamic systems. Baras and Dorsey [3] analyzed the problem of selecting from two parallel queues the one that will be served with a single server. The control is selected at each time slot to serve one of the two queues. They formulated a finite horizon stochastic optimal control to generate an optimal policy and discussed the storage of these offline computation results to facilitate the on-line implementation of the strategy. Kobza et al. [7] proposed a discrete-time Markov chain model to find the moments and cumulative probabilities of the empty-vehicle travel time as the service time of a load request in the AGV system. Heung et al. [6] employed fuzzy logic and genetic algorithms to handle local control and learning processes, respectively. The local controllers allocated green time to the four phases of a traffic-light cycle by dynamic programming. Their simulation results showed that the delay per vehicle could be substantially reduced, particularly when the traffic demand reaches the junction capacity. Down and Lewis [4] examined a method for balancing the load dynamically in parallel process networks with dedicated arrival streams. With two servers, the optimal control policy preferred storing customers in the lowest cost queue. These results lead to a heuristic development for the model with more than two servers. Kyriakidis and Pavitsos [8] developed an efficient MDP solution algorithm for the optimal maintenance of a deteriorating production unit, that pulls the raw material from a buffer. This model generated a sequence of improving control-limit policies, and extensive numerical results proved that control-limit policies converge to the optimal policy. Li and Tao [9] have formulated a two-stage dynamic programming model to assist a car rental company serving two cities in determining the optimal fleet size and vehicle transfer policy. This numerical study revealed that their heuristic solution for the fleet size performs efficiently. The current paper defines the AMHS transport demand and vehicle number states. The present study also proposes a MDP model to solve the vehicle allocation control problem dynamically. To improve the computational efficiency, a dynamic vehicle allocation control (DVAC) is proposed to allocate the vehicles of each intrabay with the optimal vehicle control policy, which is generated by a mathematical model that considers real-time transport demand and vehicle number states. The objective is to provide a guideline for minimizing the long-term average waiting vehicle time of wafer lots. A software tool that implements the relative value iteration algorithm is developed to solve the DVAC problem. This decision support tool allows managers to control the vehicle allocation dynamically and is easily integrated into the MCS control program in the production fab. Compared with current industry practices, the proposed approach significantly reduces the waiting vehicle time of wafer lots, as proven in a simulation study. The rest of the present paper is organized as follows. Section 2 discusses the problem and mathematical models. The numerical implementation and simulation study are presented in Section 3. The conclusion is provided in Section 4.
2. Problem description and model formulation In semiconductor manufacturing, each wafer lot follows its specific processing steps to be processed in the different tools. Each wafer lot is sequentially moved from the preceding to the
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
succeeding tool. To complete a wafer lot movement, the MCS generates a transport job in the AMHS system. Hence, a transport job is an MCS software command that moves the wafer lot from a source to a destination node. Fig. 2 shows an event sequence diagram of a transport job execution. First, the wafer lot initiates a transport request to the MCS. The MCS then dispatches an empty vehicle to move this lot, after which the assigned vehicle loads the FOUP from the source node. After completely loading the FOUP, the vehicle transports it to the destination. Finally, the FOUP is unloaded at the destination and completes the transport job. The total time of a transport job, denoted by Ttotal, is the sum of T1, T2, T3, T4, and T5. T1 is the duration for which MCS dispatches an empty vehicle. T2 is the duration for which the FOUP waits for an arriving vehicle. T3 represents the duration for which the vehicle loads the FOUP from the source node. T4 denotes the duration for which the vehicle carries the FOUP from the source to the destination node. T5 corresponds to the duration for which the vehicle unloads the FOUP to the load port. To analyze the historical value of Ttotal, the key factors affecting performance are identified as T2 and T4, whereas T1, T3, and T5 are the system-controllable values. T2 performance is determined by the vehicle fleet size, empty vehicle location, and dispatching rules. T4 performance is determined by the transport distance between the source and destination nodes. The objective of the current paper is to improve T2 and T4 performances. To describe clearly the FOUP and vehicle states, two types of FOUP states and three types of vehicle states are used to specify an AMHS transport job scenario. The FOUP state is categorized into waiting and moving states, whereas the vehicle state is categorized into empty, assigned, and transport states. Fig. 3 shows all FOUP and vehicle transfer states when completing a transport job. When a wafer lot stored in the FOUP starts or completes a process step, the system sends a transport request to the MCS. The FOUP changes to a waiting state. The FOUP state then changes to a moving state when a vehicle arrives and then completes FOUP
Fig. 2. The event sequence diagram of a transport job execution.
Fig. 3. States transfer of FOUP and vehicle for a transport job.
2331
loading. The FOUP state finally becomes a non-waiting or nonmoving state when a transport job is completed. A vehicle changes from an empty to an assigned state when the MCS successfully dispatches the vehicle. The vehicle shifts to a transport state, and the FOUP loading is completed. The vehicle returns to an empty state after completing a transport job and waits for the next job. The FOUP changes from a waiting to moving state after a vehicle loads the FOUP. From the system point-of-view, each intrabay has a transport job queue. The length of the transport job queue is an index of the number of wafer lots waiting for an empty vehicle service. Similarly, the vehicle number is an index of the number of vehicles that provide transport job services. Therefore, these two state variables, the number of jobs in a transport job queue, and the vehicle number describe the transport job demand and AMHS service capability of each intrabay in the present study. The current research examines the vehicle allocation control problem in the connecting transport AMHS with a limited vehicle number on the tracks. The proposed system has two intrabays, intrabay_1 and intrabay_2, both of which are connected to another interbay. The vehicles running in the system can directly move the FOUP to various locations. This decision problem is modeled by MDP. The objective is to find an optimal vehicle allocation control policy that minimizes long-term average costs. The MDP model is solved by relative value iteration algorithm [15]. The transport job inter-arrival time and the vehicle transport time follows an exponential distribution. This model assumes that the inter-arrival times between jobs are exponentially distributed with rates of λ1 and λ2 (jobs/min) in intrabay_1 and intrabay_2, respectively. The total vehicle number is defined by V, and the vehicle numbers of intrabay_1 and intrabay_2 are represented by v1 and v2, respectively. The total vehicle number in the system is V ¼ v1 þ v2 . In AMHS system control, the assumption of exponentially distributed service and inter-arrival times seems to be reasonable. Without the memoryless property of exponential distribution, a decision maker requires additional information on the elapsed and remaining transportation or inter-arrival times. For large-scale problems, the consideration of elapsed or remaining times is not realistic or feasible. This restriction makes the Markov system assumption reasonable. Decision makers need to decide on a reallocation based on the current transport job and vehicle allocation in real semiconductor manufacturing systems. In the AMHS, the vehicle service rate is an index of the number of transport jobs completed within a specific time. This index can measure the overall transport capability of the system. The vehicle service rate is affected by the number of available vehicles in each intrabay. Therefore, in the current paper, when intrabay_1 and intrabay_2 have α and β vehicles, their vehicle service rates are defined as μ1 ðαÞ and μ2 ðβÞ (jobs/min), respectively. Obtaining the vehicle service rate by a mathematical model is not feasible because of the dynamic behavior of the systems. Thus, the vehicle service rates μ1 ðαÞ and μ2 ðβÞ are obtained by a simulation model. This simulation model, which is constructed of the actual transport distance for each source destination pair, cross-interbay ratio, transport job arrival rate and vehicle number, to generate the vehicle service rate for the MDP model. Fig. 4 shows the simulation outcome of the vehicle service rates of intrabay_1 and intrabay_2. Therefore, the vehicle service rate of intrabay_1 (intrabay_2) in the MDP model considers the wafer lot movements within intrabay and between intrabays. The cross-interbay ratios of intrabay_1 and intrabay_2 are f12 and f21, respectively. f12 is the ratio of the transport counts of intrabay_1 to intrabay_2, divided by the total transport counts generated in intrabay_1. Meanwhile, f21 is the ratio of the transport counts of intrabay_2 to intrabay_1, divided by the total transport
2332
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
Fig. 4. Vehicle service rate of intrabay_1 and intrabay_2.
counts generated in intrabay_2. f12 and f21 are expressed as follows: f 12 ¼
transport counts to bay 2 f rom bay 1 ; total transport count sin bay 1
f 21 ¼
transport counts to bay 1 f rom bay2 total transport count sin bay 2
defined by the following: φ ¼ λ1 þ λ2 þ Max½μ1 ðv1 Þ þ Max ½μ2 ðv2 Þ þ ω x1 ;v1
ð1Þ
for each transport queue job in intrabay_1 (intrabay_2), a unit waiting cost rate of c1 (c2) per minute is accrued. Let S ¼ fðx1 ; x2 ; v1 ; kÞjv1 ∈f0; 1; 2; :::; Vg; k∈f−1; 0; 1gg be the state space, where x1 and x2 represent the number of transport queue jobs in intrabay_1 and that in intrabay_2, respectively, and v1 and v2 denote the number of vehicles in intrabay_1 and that in intrabay_2, respectively. To define only one empty vehicle assigned to move from one intrabay to another intrabay at a time, the variable k∈f−1; 0; 1g represents one vehicle moving from intrabay_1 to intrabay_2, no vehicle moving, and one vehicle moving from intrabay_2 to intrabay_1, respectively. The action set a∈f−1; 0; 1g is to decrease, remain constant, and to increase a vehicle for intrabay_1. Considering that our model assumes only one vehicle that can be moved at a time, we assume the following: a ¼ 0; if k≠0 ði:e:; there is one vehicle intentionally moved between intrabaysÞ
ð3Þ
x2 ;V −v1
The one-step transition probability of the discrete-time equiva0 lent model is defined as Pðst−1 ¼ ðx10 ; x20 ; v10 ; k Þjst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ. In the discrete-time equivalent DVAC model, the one-step state transition probability includes the intrabay_1 (intrabay_2) transport job arrival, intrabay_1 (intrabay_2) transport job completion, an increase (decrease) in the number of vehicles in intrabay_1, and the vehicle move (do not move) between intrabays. We define the following one-step transition probability pi, where i¼1–9 and Σ 9i ¼ 1 pi ¼ 1 corresponds to each one-step transition type.
P1: transport job arrival in intrabay_1 p1 ¼ Pðst−1 ¼ ðx1 þ 1; x2 ; v1 ; k þ aÞjst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ ¼
λ1 φ
ð4Þ
P2: transport job completion in intrabay_1
p2 ¼ Pðst−1 ¼ ðx1 −1; x2 ; v1 ; k þ aÞst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ μ ðv1 −½k þ a− Þ⋅ð1−f 12 Þ ¼ 1 φ
ð5Þ
a∈f−1; 0; 1g if k ¼ 0 ði:e:; there is no vehicle moved between intrabaysÞ
Given the above assumption, when the action state a is −1 (or 1) and the state variable k ¼ 0, the state variable k will be immediately changed to −1 (or 1) from 0. a will then be changed to 0 from −1 (or 1) at the same time. When the action state a is 0 and the state variable k ¼ −1 (or 1), the state variable k remains −1 (or 1). The k will be changed to 0 until one vehicle is completely moved between intrabays. The moving time of decreasing or increasing a vehicle is exponentially distributed with a rate of ω (vehicle/min). The one-time moving cost cv is calculated when a vehicle is assigned to move. To describe clearly the vehicle moving status and action, we define the following operators: ½yþ ¼ maxf0; yg ½y− ¼ maxf0; −yg
for any real number y
ð2Þ
Uniformization by Lippman [13] is then applied to transfer the infinite horizon continuous-time problem into an infinite horizon discrete-time equivalent model. Uniformization substitutes the original continuous-time process with a Poisson process that triggers state transition events and a discrete-time process that models the state transition behavior. Under the infinite horizon setting, solving the discrete-time equivalent model will generate a solution that is also optimal for the continuous-time problem. In our model, state transition events include the transport job arrival, transport job completion, and the decrease/increase a vehicle event. In the current study, the uniformization rate φ is
P3: transport job arrival in intrabay_2 p3 ¼ Pðst−1 ¼ ðx1 ; x2 þ 1; v1 ; k þ aÞjst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ ¼
λ2 φ
ð6Þ
P4: transport job completion in intrabay_2
p4 ¼ Pðst−1 ¼ ðx1 ; x2 −1; v1 ; k þ aÞst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ μ ðv2 −½k þ aþ Þ⋅ð1−f 21 Þ ¼ 2 φ
ð7Þ
P5: transport job completion from intrabay_1 to intrabay_2 p5 ¼ Pðst−1 ¼ ðx1 −1; x2 ; v1 −1; k þ aÞst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ − μ ðv1 −½k þ a Þ⋅f 12 ¼ 1 φ
ð8Þ
P6: transport job completion from intrabay_2 to intrabay_1 p6 ¼ Pðst−1 ¼ ðx1 ; x2 −1; v1 þ 1; k þ aÞst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ μ ðv2 −½k þ aþ Þ⋅f 21 ¼ 2 φ
ð9Þ
P7: vehicle movement completion from intrabay_2 to intrabay_1 p7 ¼ Pðst−1 ¼ ðx1 ; x2 ; v1 þ 1; 0Þst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
( ¼
ω φ
if k ¼ 1 or a ¼ 1
0
otherwise
ð10Þ
P8: vehicle movement completion from intrabay_1 to intrabay_2
p8 ¼ Pðst−1 ¼ ðx1 ; x2 ; v1 −1; 0Þst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ ( ω if k ¼ −1 or a ¼ −1 ¼ φ 0 otherwise
ð11Þ
P9: dummy transition (system state does not change), except the number of vehicles on move p9 ¼ Pðst−1 ¼ ðx1 ; x2 ; v1 ; k 8
þ aÞjst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ ¼ 1− ∑ pi
ð12Þ
i¼1
let cðx1 ; x2 ; v1 ; k; aÞ be the expected instantaneous (one-period) cost that considers job waiting and vehicle moving costs. For each job at intrabay_1 (intrabay_2), a unit waiting cost rate c1 ðc2 Þ per unit of time is accrued. To move a vehicle, a unit cost rate cv is calculated. cðx1 ; x2 ; v1 ; aÞ is defined by Eq. (12) as follows: c1 x1 þ c2 x2 þ cv jaj φ
cðx1 ; x2 ; v1 ; k; aÞ ¼
ð13Þ
let U nt ðx1 ; x2 ; v1 ; kÞ be the relative value function in the relative value iteration algorithm, according to Puterman [15]. In the ^ relative value iteration algorithm, a specific state ðx^1 ; x^2 ; v^1 ; k Þ is selected from the state space S, and the optimality equation searches for an optimal action in each iteration that minimizes the relative value function. Given the initial condition U 0 ðx1 ; x2 ; v1 ; kÞ ¼ 0; ∀x1 ; x2 ; v1 ; k, the iterative definition of U nt ðx1 ; x2 ; v1 ; kÞ is provided in Eq. (14). Optimality equation:
simulation model is finally constructed in Phase 3 to evaluate the DVAC performance. Subsequently, the proposed model is compared with other vehicle allocation control methods. Fig. 6 shows the connecting transport AMHS layout containing a single loop of an interbay system, two intrabay systems, forty tools, and four stockers in the simulation study. The vehicle used is the overhead hoist transporter (OHT) that holds the FOUP by its top flange. Table 1 summarizes the parameters of all tools in the proposed system. Table 2 presents examples of the wafer movement information and Table 3 shows the from–to distances and arrival rates between the tools. The arrival rate of the wafer lots is defined as the quantity per hour. This information can be retrieved from the manufacturing execution system (MES). For instance, the lot coded by LT-01-A enters the tool T101 at 23:08:15 on 03/01/10, finishes processing, and leaves at 23:52:40. The lot arrives at 00:12:20 on 03/02/10 in the next tool, T118 on the same bay, and leaves at 00:32:52 on 03/02/10. Thus, the arrival rate of the wafer lots can be obtained for analysis with the MES data. A simulation software eM-Plant is used to construct a virtual AMHS to compare the transport performances of different vehicle allocation control methods. For each simulation run, several performance measures were collected after 10 days of warm-up time, and stored in a data file for validation. Each simulation was run for 100 days. The simulation results were confirmed by real data in a wafer fab in Taiwan to validate the model. A complete set of simulation outputs for one sample system is provided in Example 3.1 to demonstrate the DVAC method and facilitate an understanding of the simulation models and results. Example 3.1. Implementation of DVAC and comparison with other methods In this example, the unit waiting cost of a transport job in both intrabays is 5 per min, and the moving cost for each vehicle is 0 per min. The vehicle cannot serve the other request when it
U nt ðx1 ; x2 ; v1 ; kÞ ¼ minfcðx1 ; x2 ; v1 ; k; aÞ þ E½W nt−1 ðst−1 Þjst ¼ ðx1 ; x2 ; v1 ; kÞ; ag a
8 9 < cðx1 ; x2 ; v1 ; k; aÞ = ¼ min þ ∑ ðPðst−1 jst ¼ ðx1 ; x2 ; v1 ; kÞ; at Þ W nt−1 ðst−1 ÞÞ a : ;
ð14Þ
St−1
in which, ( a∈f−1; 0; 1g a¼0
if k ¼ 0 if k≠0 0
Fig. 5. DVAC methodology. 0
^
W nt−1 ðx10 ; x20 ; v10 ; k Þ≡U nt−1 ðx10 ; x20 ; v10 ; k Þ−U nt−1 ðx^1 ; x^2 ; v^1 ; k Þ; ∀x1 ∈f0; 1; 2; :::; L1 g; x2 ∈f0; 1; 2; :::; L2 g; v1 ∈f0; 1; 2; :::; Vg; k∈f−1; 0; 1g
3. Implementation of optimal control and the simulation study A software tool developed by Microsoft Visual Basic for solving an optimality equation is used to conduct the numerical study and verify the transport performance of the DVAC in the present study. Discrete-event simulation models are applied in analyzing a complex AMHS system to evaluate the effectiveness of the DVAC model. Fig. 5 shows the schematic diagram of the proposed DVAC and simulation approaches. In Phase 1, a simulation model that considers the transport from–to table, different levels of transport job arrival rates, and vehicle number is constructed to generate vehicle service rates. With the phase 1 simulation output and other input parameters, the dynamic programming model is applied to generate optimal policies in Phase 2. The second
2333
Fig. 6. A virtual AMHS model.
2334
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
moves from one bay to the other bay. The other system parameters are listed as follows:
The total vehicle number is 20. The vehicle dispatching rule uses the combination of SD–NV
(shortest distance and nearest vehicle) and FEFS (the foremostencounter-first-served) rule. The job arrival of intrabay_1 and intrabay_2 is Poisson with rates of 8.0 and 7.5 jobs/min, respectively. The vehicle service rate is exponentially distributed with a rate obtained by a simulation model.
The implementation procedure of the DVAC method is listed below:
Table 1 Tool parameters summary. Type
Model
Quantity
Load ports /per tool
Process time (s/lot)
WPH (pcs/h)
Stocker Tool Tool Tool Tool Tool Tool Tool Tool Tool Tool
– SxxN NxxC OxxY CxxD AxxE ExxT WxxB LxxM TxxL DxxS
4 7 3 4 4 2 5 5 3 3 4
2 2 2 2 2 2 2 2 2 2 2
– 2400 3000 240 300 180 2700 1800 1500 2100 2100
– 37.5 30.0 375.0 300.0 500.0 33.3 50.0 60.0 42.9 42.9
WPH: Wafer per hour.
Table 2 Some examples of the lot movement information.
Lot
Part
Pieces
Start
Finish
Area
Tool
LT-01-A
TKN-A-1
25
T101
TKN-A-1
25
LIT
T118
LT-01-A
TKN-A-1
25
03/01/10 23:52:40 03/02/10 00:32:52 03/02/10 01:24:02
LIT
LT-01-A
03/01/10 23:08:15 03/02/10 00:12:20 03/02/10 00:45:11
ETCH
T206
The vehicle allocation rate to decrease or increase a vehicle is exponentially distributed at a rate of 1 vehicle/min. The cross-interbay ratios of intrabay_1 and intrabay_2 are 0.16 and 0.18, respectively.
Step 1: Offline calculation of the DVAC policy A software tool is implemented to solve the optimality equation and output optimal actions of all states. The optimal actions are different for each state, as shown in Fig. 7. For example, in Fig. 7(a), the optimal action for state (10, 0, 10), dot M, is to move a vehicle from intrabay_2 to intrabay_1. However, in Fig. 7(c), dot N, the optimal action for state (10, 16, 10) is to move a vehicle from intrabay_1 to intrabay_2. Although dot N has the same x1 and v1 values as dot M, the intrabay_2 job queue x2 of dot N is 16, which results in a different optimal action to decrease a vehicle of intrabay_1. The control limit lines in Fig. 7 are those states that remain constant (i.e., no vehicle change action for any intrabay). Step 2: Implementation of the optimal policy in the simulation system The offline pre-calculated DVAC policies in Step 1 are stored in the simulation look-up tables. In the real-time operation of a manufacturing system, the MCS can search the look-up table and choose an optimal action, accordingly. The vehicle waiting time performance of transport jobs in the DVAC model is compared with other vehicle allocation control methods used in the practical production fab, including fixed vehicle allocation control (FVAC), empty vehicle balancing allocation control (EVBAC), and zone control (ZC). FVAC FVAC is widely used in the traditional AMHS system. In FVAC, each intrabay maintains a fixed vehicle number, and the MCS allocates vehicles to each intrabay based on the default setting. In the current study, the fixed vehicle numbers of both intrabay_1 and intrabay_2 are set to 10. To illustrate, when the vehicle number in intrabay_1 is higher than 10, the MCS allocates the vehicles to intrabay_2 until the vehicle number in intrabay_1 and intrabay_2 are both 10. When the vehicle
Table 3 From–to distance (Upper corner, meters) and arrival rate (Lower corner, lot moves/hour) between tools. From–to
S100
S100 T101 T102 T103 T104 T105 T106 T107 T108 T109 T110 ⫶
52.5 2.1 50.0 1.7 47.5 1.8 45.0 1.9 42.5 1.6 40.0 2.0 37.5 1.5 35.0 4.2 32.5 3.8 30.0 5.1 ⫶
T101
T102
T103
T104
T105
T106
T107
T108
T109
T110
2.5 6.6
5.0 4.4 2.5 0.0
7.5 5.5 5.0 0.0 2.5 0.0
10.0 6.6 7.5 0.0 5.0 0.0 2.5 0.0
12.5 4.4 10.0 0.0 7.5 0.0 5.0 0.0 2.5 0.0
15.0 5.5 12.5 0.0 10.0 0.0 7.5 0.0 5.0 0.0 2.5 0.0
17.5 2.2 15.0 0.0 12.5 0.0 10.0 0.0 7.5 0.0 5.0 0.0 2.5 0.0
20.0 5.5 17.5 1.0 15.0 0.6 12.5 1.0 10.0 1.0 7.5 0.8 5.0 0.7 2.5 0.6
22.5 3.3 20.0 0.4 17.5 0.7 15.0 0.7 12.5 0.8 10.0 0.5 7.5 0.5 5.0 0.7 2.5 0.0
25.0 4.4 22.5 0.9 20.0 0.8 17.5 1.3 15.0 0.8 12.5 0.9 10.0 0.7 7.5 0.8 5.0 0.0 2.5 0.0
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 40.0 0.0 37.5 0.0 35.0 0.0 32.5 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 40.0 0.0 37.5 0.0 35.0 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 40.0 0.0 37.5 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 40.0 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 ⫶
52.5 0.0 50.0 0.0 ⫶
52.5 0.0 ⫶
… … … … … … … … … … … …
⫶
⫶
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
2335
Fig. 7. An example of the optimal vehicle allocation control policy.
number in intrabay_1 is lower than 10, the MCS allocates the vehicles to intrabay_1 from intrabay_2 to balance the intrabay vehicles. EVBAC EVBAC is commonly used in the connecting transport AMHS system. In EVBAC, each intrabay keeps its expected empty vehicles, calculated as the total number of empty vehicles multiplied by a target ratio. The MCS automatically balances all empty vehicles to each intrabay according to its target ratio. In the current study, the balancing ratios of both intrabay_1 and intrabay_2 are equal to 0.5 (i.e., the total empty vehicles in this system are equally allocated to intrabay_1 and intrabay_2 using the EVBAC method). ZC Wang et al. [17] used a ZC strategy to control the feasible vehicle number at each intrabay. This control method restricts the actual number of vehicles at each intrabay at an interval value. The lower or upper limit of the vehicle number is set as the feasible solution. The MCS controls the vehicle number of each intrabay in this feasible solution. In the current paper, the lower and upper limits of each intrabay are set at 6 and 14, respectively. For example, the MCS allocates a vehicle from intrabay_2 to intrabay_1 when the vehicle number in intrabay_1 is 5. Meanwhile, the MCS allocates a vehicle from intrabay_1 to intrabay_2 when the vehicle number in intrabay_1 is 15. Besides ZC with (6, 14), two other ZC parameters at (7, 13) and (8, 12) are added for comparison.
We implemented the DVAC and all other methods in the simulation model. All vehicle allocation control methods were coded into the method object of the simulation model. We measured the vehicle waiting time, total transport time, and throughput of these three important indices in the model to evaluate transport performance. The vehicle waiting time T2 and the total transport time Ttotal are clearly defined in Section 2. The
Fig. 8. Vehicle waiting time under different vehicle allocation controls.
throughput measures the number of transport jobs completed within this simulation period. Wang et al. [17] concluded that the flow rate affects the transport performance of the connecting transport AMHS. In the current paper, the five flow rate levels were 930, 976.5, 1023, 1069.5, and 1116 lots/h. The standard flow rate was 930 lots/h, and the other four flow rates were 105%, 110%, 115%, and 120% of the standard loading. The comparison of the vehicle waiting time, total transport time, and throughput of these four methods with the statistical hypothesis test indicated a significant difference among these three performances under different flow rates. The three performance results (as shown in Figs. 8–10) show that the proposed DVAC outperforms all other methods in terms of vehicle waiting time, total transport time, and throughput by at least 7.2%, 2.1% and 1.3%, respectively. Table 4 presents the average values and the 95% confidence intervals of the three performance results. Table 5 shows the p-values of the paired t-tests of these measurements. These two tables reveal that the DVAC dominates over the other methods on vehicle waiting time and total transport times at all flow rates, which suggests that AMHS vehicles using the DVAC method provide a better service rate compared with others. In terms of throughput performance, only DVAC performs more transport counts at high flow rate compared with the other three
2336
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
methods. As an example, in a practical production fab, many transport jobs in the load ports of tools or stockers wait for the AMHS vehicle service at a high flow rate. When vehicles quickly move the wafer lots to tools and stockers, the manufacturing system can produce more wafer lots and minimize the tool idle time. Hence, the DVAC shows a more satisfactory throughput performance at a high flow rate with each vehicle having a better
Fig. 9. Total transport time under different vehicle allocation controls.
Fig. 10. Throughput performance under different vehicle allocation controls.
service rate and the AMHS system completing more transport jobs within the simulation period. Example 3.2. Comparisons of DVAC rules with different transport job waiting costs Minimizing the lost time of high utilization tools is a relevant management topic because of the high investment cost in semiconductor manufacturing. One of the reasons for the time loss of impact tools is the delay due to waiting for the AMHS vehicle to transport a wafer lot to the tool from the stockers or preceding tools. To reduce delays, managers have to allocate more vehicles to minimize vehicle waiting time in the intrabay. By applying the DVAC method, managers can easily set up different transport job waiting costs for each intrabay to allocate optimal empty vehicles to that specific intrabay. In the present example, we studied the effect of different transport job waiting cost parameters. In our numerical example, the system parameters were identical to those in Example 3.1, except for the job unit waiting cost. We assumed the transport job unit waiting costs of intrabay_1 and intrabay_2 (c1,c2) be (2,8), (3,7), (5,5), (7,3), and (8,2). Five different vehicle allocation control policies were then sequentially generated using the DVAC model: DVAC_2_8, DVAC_3_7, DVAC_5_5, DVAC_7_3, and DVAC_8_2. Fig. 11 shows an example of the vehicle control limit with all intrabay_2 jobs equal to 11 under different job unit waiting costs. Most states allocate a vehicle from intrabay_2 to intrabay_1 when the job unit waiting cost of intrabay_1 is higher than that of intrabay_2, such as the control limit DVAC_8_2. These observations suggest that if intrabay_1 has high-utilization tools, such as photo-scanners, the DVAC method can be applied to move more empty vehicles to this photo intrabay in many situations. This useful decision policy can be implemented to the MCS vehicle
Table 4 Comparison of vehicle allocation methods under different flow rates. Flow rate (lots/h)
Method
Vehicle waiting time (95% conf. intv.)
930.0
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
25.71 28.28 28.86 28.07 28.03 28.10
(25.64, 25.79) (28.21, 28.34) (28.78, 28.93) (27.95, 28.20) (27.93, 28.12) (28.00, 28.19)
84.61 86.65 87.97 87.04 86.65 86.51
(84.47, 84.75) (86.45, 86.86) (87.79, 88.15) (86.77, 87.31) (86.43, 86.86) (86.29, 86.74)
1839557 1842130 1833828 1831746 1836851 1840858
(1838063, 1841051) (1840283, 1843977) (1831869, 1835787) (1829686, 1833806) (1834727, 1838975) (1838525, 1843191)
976.5
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
34.94 37.26 37.63 37.62 37.46 37.32
(34.88, 34.99) (37.20, 37.31) (37.56, 37.70) (37.52, 37.72) (37.38, 37.54) (37.25, 37.38)
96.28 98.77 99.24 99.08 98.84 98.62
(96.19, 96.38) (98.59, 98.94) (99.10, 99.39) (98.86, 99.29) (98.66, 99.03) (98.45, 98.80)
1890447 1875665 1866911 1861176 1869648 1875767
(1888534, 1892360) (1873060, 1878270) (1864146, 1869676) (1858515, 1863837) (1867620, 1871676) (1873139, 1878395)
1023.0
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
39.53 41.83 42.21 42.26 42.12 41.96
(39.49, 39.57) (41.76, 41.89) (42.14, 42.28) (42.19, 42.33) (42.04, 42.20) (41.90, 42.03)
102.86 104.74 105.08 105.02 104.85 104.69
(102.76, 102.95) (104.61, 104.88) (104.92, 105.24) (104.87, 105.16) (104.69, 105.00) (104.56, 104.81)
1918965 1884191 1874267 1871376 1877447 1883763
(1916834, 1921096) (1880953, 1887429) (1870786, 1877748) (1868207, 1874545) (1874056, 1880838) (1880372, 1887154)
1069.5
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
42.98 45.45 45.86 45.95 45.71 45.56
(42.93, 43.03) (45.36, 45.54) (45.77, 45.94) (45.88, 46.01) (45.64, 45.78) (45.50, 45.63)
107.72 109.60 109.91 109.83 109.70 109.58
(107.59, 107.85) (109.44, 109.76) (109.78, 110.04) (109.69, 109.97) (109.56, 109.84) (109.42, 109.74)
1918282 1884166 1878214 1874160 1880534 1884568
(1915335, 1921229) (1880295, 1888037) (1875166, 1881262) (1870990, 1877330) (1877327, 1883741) (1880841, 1888295)
1116.0
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
45.93 48.12 48.59 48.54 48.40 48.24
(45.87, 45.98) (48.03, 48.20) (48.49, 48.68) (48.45, 48.63) (48.32, 48.48) (48.16, 48.32)
111.73 113.28 113.61 113.46 113.44 113.25
(111.57, 111.89) (113.11, 113.44) (113.44, 113.78) (113.32, 113.60) (113.30, 113.57) (113.07, 113.43)
1920159 1887498 1878474 1881195 1883250 1888349
(1916783, 1923535) (1883540, 1891456) (1874299, 1882649) (1878144, 1884246) (1880096, 1886404) (1880414, 1892684)
Total transport time (95% conf. intv.)
Throughput (95% conf. intv.)
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
2337
Table 5 p-values of the paired t-tests for the comparison of vehicle allocation methods. Flow rate (lots/h) DVAC vs. FVAC
930.0 976.5 1023.0 1069.5 1116.0
Vehicle waiting time 1.68E−29 3.39E−30 9.37E−33 1.93E−34 4.90E−32
DVAC vs. EVBAC Total transport time 3.96E−17 7.35E−22 1.93E−21 7.38E−23 4.14E−19
Throughput 0.0092 9.63E−10 4.83E−18 4.85E−18 7.70E−16
DVAC vs. ZC(6,14)
930.0 976.5 1023.0 1069.5 1116.0
Vehicle waiting time 4.58E−24 6.97E−29 1.83E−33 6.74E−39 5.01E−34
Vehicle waiting time 3.25E−31 1.85E−30 7.18E−32 2.99E−37 1.71E−33
Total transport time 3.75E−23 1.36E−25 1.09E−21 1.25E−24 5.80E−21
Throughput 5.87E−06 1.18E−15 4.08E−20 1.76E−21 3.50E−18
Total transport time 1.57E−17 7.71E−22 1.60E−20 4.33E−23 4.30E−23
Throughput 0.0149 9.85E−15 3.87E−20 1.66E−20 8.49E−21
DVAC vs. ZC(7,13) Total transport time 4.50E−17 1.59E−21 9.97E−21 6.86E−24 1.39E−21
Throughput 1.35E−07 1.56E−16 1.88E−21 1.19E−22 1.28E−18
Total transport time 4.37E−17 3.96E−21 3.73E−21 5.91E−23 1.80E−19
Throughput 0.1431 1.69E−09 1.52E−18 7.44E−18 1.00E−15
Vehicle waiting time 4.58E−25 3.90E−30 5.79E−33 2.39E−36 3.95E−35
DVAC vs. ZC(8,12)
930.0 976.5 1023.0 1069.5 1116.0
Vehicle waiting time 1.51E−25 8.26E−30 1.72E−32 6.57E−37 1.57E−35
control algorithm to satisfy intrabays with high empty-vehicle demands. This result reflects the actual behavior observed in the wafer fab. Example 3.3. Comparisons of DVAC rules with different vehicle moving costs In the present example, we study the effects of different cost parameters. In the numerical example, the system parameters are identical to those in Example 3.1, except for the vehicle unit moving cost. We set the vehicle moving cost cv of both intrabay_1 and intrabay_2 to 20. Two DVAC policies can be generated: DVAC with a vehicle moving cost of 0 and DVAC with a vehicle moving cost of 20. Fig. 12 shows the vehicle control limit under different vehicle unit moving costs. The DVAC with a vehicle moving cost of 0 is only a control limit characterized by a line with no vehicle change action in the diagram. An upside area indicates a vehicle allocation from intrabay_1 to intrabay_2, whereas a downside area indicates a vehicle allocation from intrabay_2 to intrabay_1. The DVAC with a vehicle moving cost of 20 has upper and lower control limits. The area between the upper and lower control limit lines indicates that keeping no vehicle changes the action. The upside area above the upper control limit line indicates a vehicle allocation from intrabay_1 to intrabay_2, whereas the downside area below the lower control limit line indicates a vehicle allocation from intrabay_2 to intrabay_1. The comparison of the DVAC method with different vehicle moving costs indicate no significant difference among the vehicle waiting time, total transport time, and throughput performance in the simulation experiment results. Figs. 13–15 show the transport permanence comparison of these two policies. The vehicle allocation control policy with a vehicle moving cost of 20 showed good transport performances similar to those without vehicle moving cost. This result indicates that the same transport performance can be obtained without adding a vehicle to one intrabay or subtracting one from another in many situations. The MCS only allocates an empty vehicle to the target intrabay whenever necessary. Aside from a satisfactory transport performance, this policy of using a vehicle moving cost of 20 also involves fewer vehicles cross-interbay
Fig. 11. Vehicle control limit under different job unit waiting costs.
movements and prevents vehicle traffic jam in the interbay connections by observing a simulation animation. In the practical production fab, managers can improve the AMHS transport performance as well as save vehicle energy resources and usage of consumer parts by applying this method.
4. Conclusion This paper examines the DVAC problems of the AMHS in semiconductor manufacturing. Two AMHS state variables, namely, the number of jobs in the transport job queue and the vehicle number, describe the transport job demand and AMHS service capability of each intrabay and are applied in the dynamic programming method. A DVAC method is proposed to adjust the vehicle number of each intrabay and to reduce the waiting time of wafer lots for AMHS vehicle service. Stochastic dynamic programming is used to solve the vehicle allocation control problem. The objective is to find optimal vehicle control policies to minimize the long-term average waiting cost of all transport jobs. Although AMHS systems are not rigorous “Markov” systems, MDP remains as an effective tool in modeling AMHS system. We do not assume the exponential transportation and inter-arrival time distribution in
2338
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
Fig. 12. Vehicle control limit under different vehicle moving costs.
Fig. 13. Vehicle waiting time under different DVAC rules.
Fig. 15. Throughput performance under different DVAC rules.
Fig. 14. Total transport time under different DVAC rules.
our simulation study. However, the simulation results strongly suggest that the DVAC method can significantly reduce the vehicle waiting time of the wafer lots as well as the total transport time compared with the other methods. For control, the optimal policy generated by the DVAC model can be stored in the MCS look-up tables and is applied to daily operations in a practical real-time production environment. By examining the model with only two intrabays, we gain valuable insights into the optimal policy structure in the N-intrabay system. However, in a more complicated system, overcoming the computational complexity issue in larger systems with expanded state space remains unclear. This consideration complicates our heuristics, and simultaneously provides an interesting challenge for further research.
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
Acknowledgment The authors wish to thank the anonymous referees of this manuscript whose thorough review greatly improved the clarity and readability of our paper. We thank the National Science Council, Taiwan ROC, for support under contract NSC96-2628-E007-032-MY3. References [1] Agrawal GK, Heragu SS. A survey of automated material handling systems in 300-mm semiconductor fabs. IEEE Transactions on Semiconductor Manufacturing 2006;19(1):112–20. [2] Baker BM, Ayechew MA. A generic algorithm for the vehicle routing problem. Computers and Operations Research 2003;30:787–800. [3] Baras JS, Dorsey AJ. Stochastic control of two partially observed competing queues. IEEE Transactions on Automatic Control 1981;26(5):1106–17. [4] Down DG, Lewis ME. Dynamic load balancing in parallel queueing systems. Stability and Optimal Control 2006;168(2):509–19. [5] Egbelu PJ, Tanchoco JMA. Characterization of automatic guided vehicle dispatching rules. International Journal of Production Research 1984;22 (3):359–74. [6] Heung TH, Ho TK, Fung YF. Coordinated road-junction traffic control by dynamic programming. IEEE Transactions on Intelligent Transportation Systems 2005;6(3):341–50.
2339
[7] Kobza JE, Shen YC, Reasor RJ. A stochastic model of empty-vehicle travel time and load request service time in light-traffic material handling system. IIE Transaction 1998;30:133–42. [8] Kyriakidis EG, Pavitsos A. Markov decision models for the optimal maintenance of a production unit with an upstream buffer. Computers and Operations Research 2009;36:1993–2006. [9] Li Z, Tao F. On determining optimal fleet size and vehicle transfer policy for a car rental company. Computers and Operations Research 2010;37:341–50. [10] Liao DY, Fu HS. Speed delivery: dynamic OHT allocation and dispatching in large-scale 300-mm AMHS management. IEEE Robotics and Automation Magazine 2004:22–32. [11] Lin JT, Wang FK, Wu CK. Simulation analysis of the connecting transport AMHS in a wafer fab. IEEE Transactions on Semiconductor Manufacturing 2003;16 (3):555–64. [12] Lin JT, Wang FK, Wu CK. Connecting transport AMHS in a wafer fab. International Journal of Production Research 2003;41(3):529–44. [13] Lippman SA. Applying a new device in the optimization of exponential queuing systems. Operations Research 1975;23(4):687–710. [14] Maxwell WL, Muckstadt JA. Design of automatic guided vehicle systems. IIE Transaction 1982;14(2):114–24. [15] Puterman M. Markov decision process: discrete stochastic dynamic programming. John Wiley & Sons Inc; 1994. [16] Renaud J, Laporte G, Boctor F. A tabu search heuristic for the multi-depot vehicle routing problem. Computers and Operations Research 1996;23: 229–35. [17] Wang FK, Lin JT. Performance evaluation of an automated material handling system for a wafer fab. Robotics and Computer Integrated Manufacturing 2004;20(2):91–100.
Contents lists available at SciVerse ScienceDirect
Computers & Operations Research journal homepage: www.elsevier.com/locate/caor
Dynamic vehicle allocation control for automated material handling system in semiconductor manufacturing James T. Lin a,n, Cheng-Hung Wu b, Chih-Wei Huang a a Department of Industrial Engineering and Engineering Management, National Tsing-Hua University, 101, Section 2, Kuang-Fu Road, Hsinchu 300, Taiwan, ROC b Institute of Industrial Engineering, National Taiwan University, 1, Section 4, Roosevelt Road, Taipei 10617, Taiwan, ROC
art ic l e i nf o
a b s t r a c t
Available online 24 April 2013
The current study examines the dynamic vehicle allocation problems of the automated material handling system (AMHS) in semiconductor manufacturing. With the uncertainty involved in wafer lot movement, dynamically allocating vehicles to each intrabay is very difficult. The cycle time and overall tool productivity of the wafer lots are affected when a vehicle takes too long to arrive. In the current study, a Markov decision model is developed to study the vehicle allocation control problem in the AMHS. The objective is to minimize the sum of the expected long-run average transport job waiting cost. An interesting exhaustive structure in the optimal vehicle allocation control is found in accordance with the Markov decision model. Based on this exhaustive structure, an efficient algorithm is then developed to solve the vehicle allocation control problem numerically. The performance of the proposed method is verified by a simulation study. Compared with other methods, the proposed method can significantly reduce the waiting cost of wafer lots for AMHS vehicle transportation. & 2013 Elsevier Ltd. All rights reserved.
Keywords: AMHS Vehicle allocation control Markov decision process Simulation
1. Introduction For the past 20 years, the automated material handling system (AMHS) has been successfully applied in semiconductor manufacturing factories. Related technological developments have generated large-scale, complicated transport systems with expanded capabilities that can be used for storage and wafer transport. The AMHS is a solution to fabrication (fab) automation in 300-mm semiconductor manufacturing, which is capable of optimizing productivity, improving tool utilization and ergonomics, as well as reducing particle contamination and vibration shocks on wafers. A typical AMHS consists of pods, vehicles, track system, stocker system, and material control system (MCS). Front opening unified pods (FOUPs) with a carrying capacity of 25 wafers are used to transport wafers in a 300-mm wafer fab. These 25 wafers form a manufacturing batch size, referred to as a wafer lot, for processing in the tool and are stored in the stocker. The MCS is a software system used to control vehicle routing, dispatching, and allocating. It is also used for transport job control of the material. The primary function of the AMHS is to execute the physical material handling jobs that transport wafers to their destinations, including tools and their local stockers. In addition, the AMHS stores wafers into the stockers.
n
Corresponding author. Tel.: +886 35742650. E-mail address: [email protected] (J.T. Lin).
0305-0548/$ - see front matter & 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.cor.2013.04.007
In semiconductor manufacturing, wafer movement occurs between and within different processing areas, such as thin-film, etching, diffusion, and photo. Wafers typically go through over 250 processing tools to undergo several hundreds of processing steps [1]. The general transport layout in the 300-mm semiconductor factory is developed in the form of a connecting transport AMHS to provide a higher transport volume and quick delivery time because of the highly reentrant nature and long processing flow of wafer fab [11,12]. The connecting transport AMHS layout (Fig. 1) shows the tools in different processing areas installed along several intrabay tracks. Each intrabay track is connected by a central interbay track. This configuration has stockers located at the junction of the interbay and each intrabay. The vehicles directly move wafer lots to various locations without passing through stockers, thus reducing the wafer transport time more efficiently compared with the AMHS without connecting interbay and intrabay tracks. With the complicated processing flow of wafer lots processed in different areas, wafer lot transport continuously occurs at different intrabays. The MCS controls hundreds of vehicles to perform different transport jobs between intrabays. However, wafer lots have to wait for a considerable amount of time for the vehicle to arrive, especially when vehicles rarely occur at the intrabay, affecting the cycle time and overall tool productivity of the wafer lots. These unpredictable transport demands of the wafer lots are highly related to the product type, work-in-process pool, availability of tools, tool dispatch strategies, and other
2330
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
Fig. 1. Representation of the connecting transport AMHS.
factors. The dynamic allocation of the vehicles to each intrabay is very difficult considering the uncertainty of wafer movement despite the direct movement of all vehicles within the connecting transport AMHS. Currently, fab managers apply an empirical method to establish the MCS system parameters in allocating all vehicles to each intrabay. Despite its periodic modification the MCS parameters, this method has remarkably influenced AMHS transport performances in which assumptions or conditions do not exist. Thus, dynamic allocation of vehicles has become a significant management topic for wafer fab. Previous studies have investigated improvements in AMHS design and performance. The AMHS vehicle management problem in these studies can be categorized into three: vehicle fleet size, vehicle dispatching rule, and vehicle routing decision. The vehicle fleet size problem involves the determination of the minimum number of vehicles required to transport the wafer lots in the factory. Maxwell and Muckstadt [14] formulated the automated guided vehicle (AGV) transport problem into a mathematical model to minimize the total travel times of empty vehicles. They determined the minimum number of vehicles that would satisfy a specific demand requirement. The use of dispatch rules to control vehicles minimizes vehicle waiting time and maximizes system throughput. Egbelu and Tanchoco [5] proposed two aspects of the AGV dispatching problem: work center-initiated versus vehicleinitiated task-assignment problems. Improved performances have been observed from the simulation results of the combined modified first-come, first-served (FCFS) rule and the nearest vehicle (NV) or longest idle vehicle (LIV) rules. Liao and Fu [10] proposed a two-phase approach to selecting the best vehicle dispatch policy and determining the number of vehicles. Their simulation results showed satisfactory performances of both transport and delivery time. The vehicle routing decision problem chooses the vehicle route and the job sequence to visit, thus minimizing the transport time of wafer lots. Baker and Ayechew [2] used a generic algorithm to solve basic vehicle routing problems and obtained solutions that were 0.5% above the bestknown results. Renaud et al. [16] applied meta-heuristics and tabu search, which were successfully implemented in other vehicle routing studies. Both meta-heuristics and tabu search were reported to perform efficiently. The occurrence of vehicle control problems over the past few years is discussed as the connecting transport AMHS is applied to the 300-mm fab. Lin et al. [11,12] first proposed four types of vehicles (A, B, C, and D) to perform the connecting transport tasks by using three combinations, namely, vehicles A–B, A–C, and A–D. They calculated the minimum number of vehicle types with a simple procedure. To improve the vehicle waiting time, Wang et al. [17] proposed the control of the upper or lower limit on the number of vehicles in the intrabay. Their results indicated that the control significantly affected travel time, waiting time, and empty vehicle utilization.
The above-mentioned studies only examined the optimization of vehicle number, dispatching, routing, and types. Thus, the relationship between the transport demand and AMHS states is not further discussed. Meanwhile, the explicit consideration of the real-time production and the AMHS states for vehicle control management is very important. Markov decision processes (MDPs) are useful in the real-time control of dynamic systems. Baras and Dorsey [3] analyzed the problem of selecting from two parallel queues the one that will be served with a single server. The control is selected at each time slot to serve one of the two queues. They formulated a finite horizon stochastic optimal control to generate an optimal policy and discussed the storage of these offline computation results to facilitate the on-line implementation of the strategy. Kobza et al. [7] proposed a discrete-time Markov chain model to find the moments and cumulative probabilities of the empty-vehicle travel time as the service time of a load request in the AGV system. Heung et al. [6] employed fuzzy logic and genetic algorithms to handle local control and learning processes, respectively. The local controllers allocated green time to the four phases of a traffic-light cycle by dynamic programming. Their simulation results showed that the delay per vehicle could be substantially reduced, particularly when the traffic demand reaches the junction capacity. Down and Lewis [4] examined a method for balancing the load dynamically in parallel process networks with dedicated arrival streams. With two servers, the optimal control policy preferred storing customers in the lowest cost queue. These results lead to a heuristic development for the model with more than two servers. Kyriakidis and Pavitsos [8] developed an efficient MDP solution algorithm for the optimal maintenance of a deteriorating production unit, that pulls the raw material from a buffer. This model generated a sequence of improving control-limit policies, and extensive numerical results proved that control-limit policies converge to the optimal policy. Li and Tao [9] have formulated a two-stage dynamic programming model to assist a car rental company serving two cities in determining the optimal fleet size and vehicle transfer policy. This numerical study revealed that their heuristic solution for the fleet size performs efficiently. The current paper defines the AMHS transport demand and vehicle number states. The present study also proposes a MDP model to solve the vehicle allocation control problem dynamically. To improve the computational efficiency, a dynamic vehicle allocation control (DVAC) is proposed to allocate the vehicles of each intrabay with the optimal vehicle control policy, which is generated by a mathematical model that considers real-time transport demand and vehicle number states. The objective is to provide a guideline for minimizing the long-term average waiting vehicle time of wafer lots. A software tool that implements the relative value iteration algorithm is developed to solve the DVAC problem. This decision support tool allows managers to control the vehicle allocation dynamically and is easily integrated into the MCS control program in the production fab. Compared with current industry practices, the proposed approach significantly reduces the waiting vehicle time of wafer lots, as proven in a simulation study. The rest of the present paper is organized as follows. Section 2 discusses the problem and mathematical models. The numerical implementation and simulation study are presented in Section 3. The conclusion is provided in Section 4.
2. Problem description and model formulation In semiconductor manufacturing, each wafer lot follows its specific processing steps to be processed in the different tools. Each wafer lot is sequentially moved from the preceding to the
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
succeeding tool. To complete a wafer lot movement, the MCS generates a transport job in the AMHS system. Hence, a transport job is an MCS software command that moves the wafer lot from a source to a destination node. Fig. 2 shows an event sequence diagram of a transport job execution. First, the wafer lot initiates a transport request to the MCS. The MCS then dispatches an empty vehicle to move this lot, after which the assigned vehicle loads the FOUP from the source node. After completely loading the FOUP, the vehicle transports it to the destination. Finally, the FOUP is unloaded at the destination and completes the transport job. The total time of a transport job, denoted by Ttotal, is the sum of T1, T2, T3, T4, and T5. T1 is the duration for which MCS dispatches an empty vehicle. T2 is the duration for which the FOUP waits for an arriving vehicle. T3 represents the duration for which the vehicle loads the FOUP from the source node. T4 denotes the duration for which the vehicle carries the FOUP from the source to the destination node. T5 corresponds to the duration for which the vehicle unloads the FOUP to the load port. To analyze the historical value of Ttotal, the key factors affecting performance are identified as T2 and T4, whereas T1, T3, and T5 are the system-controllable values. T2 performance is determined by the vehicle fleet size, empty vehicle location, and dispatching rules. T4 performance is determined by the transport distance between the source and destination nodes. The objective of the current paper is to improve T2 and T4 performances. To describe clearly the FOUP and vehicle states, two types of FOUP states and three types of vehicle states are used to specify an AMHS transport job scenario. The FOUP state is categorized into waiting and moving states, whereas the vehicle state is categorized into empty, assigned, and transport states. Fig. 3 shows all FOUP and vehicle transfer states when completing a transport job. When a wafer lot stored in the FOUP starts or completes a process step, the system sends a transport request to the MCS. The FOUP changes to a waiting state. The FOUP state then changes to a moving state when a vehicle arrives and then completes FOUP
Fig. 2. The event sequence diagram of a transport job execution.
Fig. 3. States transfer of FOUP and vehicle for a transport job.
2331
loading. The FOUP state finally becomes a non-waiting or nonmoving state when a transport job is completed. A vehicle changes from an empty to an assigned state when the MCS successfully dispatches the vehicle. The vehicle shifts to a transport state, and the FOUP loading is completed. The vehicle returns to an empty state after completing a transport job and waits for the next job. The FOUP changes from a waiting to moving state after a vehicle loads the FOUP. From the system point-of-view, each intrabay has a transport job queue. The length of the transport job queue is an index of the number of wafer lots waiting for an empty vehicle service. Similarly, the vehicle number is an index of the number of vehicles that provide transport job services. Therefore, these two state variables, the number of jobs in a transport job queue, and the vehicle number describe the transport job demand and AMHS service capability of each intrabay in the present study. The current research examines the vehicle allocation control problem in the connecting transport AMHS with a limited vehicle number on the tracks. The proposed system has two intrabays, intrabay_1 and intrabay_2, both of which are connected to another interbay. The vehicles running in the system can directly move the FOUP to various locations. This decision problem is modeled by MDP. The objective is to find an optimal vehicle allocation control policy that minimizes long-term average costs. The MDP model is solved by relative value iteration algorithm [15]. The transport job inter-arrival time and the vehicle transport time follows an exponential distribution. This model assumes that the inter-arrival times between jobs are exponentially distributed with rates of λ1 and λ2 (jobs/min) in intrabay_1 and intrabay_2, respectively. The total vehicle number is defined by V, and the vehicle numbers of intrabay_1 and intrabay_2 are represented by v1 and v2, respectively. The total vehicle number in the system is V ¼ v1 þ v2 . In AMHS system control, the assumption of exponentially distributed service and inter-arrival times seems to be reasonable. Without the memoryless property of exponential distribution, a decision maker requires additional information on the elapsed and remaining transportation or inter-arrival times. For large-scale problems, the consideration of elapsed or remaining times is not realistic or feasible. This restriction makes the Markov system assumption reasonable. Decision makers need to decide on a reallocation based on the current transport job and vehicle allocation in real semiconductor manufacturing systems. In the AMHS, the vehicle service rate is an index of the number of transport jobs completed within a specific time. This index can measure the overall transport capability of the system. The vehicle service rate is affected by the number of available vehicles in each intrabay. Therefore, in the current paper, when intrabay_1 and intrabay_2 have α and β vehicles, their vehicle service rates are defined as μ1 ðαÞ and μ2 ðβÞ (jobs/min), respectively. Obtaining the vehicle service rate by a mathematical model is not feasible because of the dynamic behavior of the systems. Thus, the vehicle service rates μ1 ðαÞ and μ2 ðβÞ are obtained by a simulation model. This simulation model, which is constructed of the actual transport distance for each source destination pair, cross-interbay ratio, transport job arrival rate and vehicle number, to generate the vehicle service rate for the MDP model. Fig. 4 shows the simulation outcome of the vehicle service rates of intrabay_1 and intrabay_2. Therefore, the vehicle service rate of intrabay_1 (intrabay_2) in the MDP model considers the wafer lot movements within intrabay and between intrabays. The cross-interbay ratios of intrabay_1 and intrabay_2 are f12 and f21, respectively. f12 is the ratio of the transport counts of intrabay_1 to intrabay_2, divided by the total transport counts generated in intrabay_1. Meanwhile, f21 is the ratio of the transport counts of intrabay_2 to intrabay_1, divided by the total transport
2332
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
Fig. 4. Vehicle service rate of intrabay_1 and intrabay_2.
counts generated in intrabay_2. f12 and f21 are expressed as follows: f 12 ¼
transport counts to bay 2 f rom bay 1 ; total transport count sin bay 1
f 21 ¼
transport counts to bay 1 f rom bay2 total transport count sin bay 2
defined by the following: φ ¼ λ1 þ λ2 þ Max½μ1 ðv1 Þ þ Max ½μ2 ðv2 Þ þ ω x1 ;v1
ð1Þ
for each transport queue job in intrabay_1 (intrabay_2), a unit waiting cost rate of c1 (c2) per minute is accrued. Let S ¼ fðx1 ; x2 ; v1 ; kÞjv1 ∈f0; 1; 2; :::; Vg; k∈f−1; 0; 1gg be the state space, where x1 and x2 represent the number of transport queue jobs in intrabay_1 and that in intrabay_2, respectively, and v1 and v2 denote the number of vehicles in intrabay_1 and that in intrabay_2, respectively. To define only one empty vehicle assigned to move from one intrabay to another intrabay at a time, the variable k∈f−1; 0; 1g represents one vehicle moving from intrabay_1 to intrabay_2, no vehicle moving, and one vehicle moving from intrabay_2 to intrabay_1, respectively. The action set a∈f−1; 0; 1g is to decrease, remain constant, and to increase a vehicle for intrabay_1. Considering that our model assumes only one vehicle that can be moved at a time, we assume the following: a ¼ 0; if k≠0 ði:e:; there is one vehicle intentionally moved between intrabaysÞ
ð3Þ
x2 ;V −v1
The one-step transition probability of the discrete-time equiva0 lent model is defined as Pðst−1 ¼ ðx10 ; x20 ; v10 ; k Þjst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ. In the discrete-time equivalent DVAC model, the one-step state transition probability includes the intrabay_1 (intrabay_2) transport job arrival, intrabay_1 (intrabay_2) transport job completion, an increase (decrease) in the number of vehicles in intrabay_1, and the vehicle move (do not move) between intrabays. We define the following one-step transition probability pi, where i¼1–9 and Σ 9i ¼ 1 pi ¼ 1 corresponds to each one-step transition type.
P1: transport job arrival in intrabay_1 p1 ¼ Pðst−1 ¼ ðx1 þ 1; x2 ; v1 ; k þ aÞjst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ ¼
λ1 φ
ð4Þ
P2: transport job completion in intrabay_1
p2 ¼ Pðst−1 ¼ ðx1 −1; x2 ; v1 ; k þ aÞst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ μ ðv1 −½k þ a− Þ⋅ð1−f 12 Þ ¼ 1 φ
ð5Þ
a∈f−1; 0; 1g if k ¼ 0 ði:e:; there is no vehicle moved between intrabaysÞ
Given the above assumption, when the action state a is −1 (or 1) and the state variable k ¼ 0, the state variable k will be immediately changed to −1 (or 1) from 0. a will then be changed to 0 from −1 (or 1) at the same time. When the action state a is 0 and the state variable k ¼ −1 (or 1), the state variable k remains −1 (or 1). The k will be changed to 0 until one vehicle is completely moved between intrabays. The moving time of decreasing or increasing a vehicle is exponentially distributed with a rate of ω (vehicle/min). The one-time moving cost cv is calculated when a vehicle is assigned to move. To describe clearly the vehicle moving status and action, we define the following operators: ½yþ ¼ maxf0; yg ½y− ¼ maxf0; −yg
for any real number y
ð2Þ
Uniformization by Lippman [13] is then applied to transfer the infinite horizon continuous-time problem into an infinite horizon discrete-time equivalent model. Uniformization substitutes the original continuous-time process with a Poisson process that triggers state transition events and a discrete-time process that models the state transition behavior. Under the infinite horizon setting, solving the discrete-time equivalent model will generate a solution that is also optimal for the continuous-time problem. In our model, state transition events include the transport job arrival, transport job completion, and the decrease/increase a vehicle event. In the current study, the uniformization rate φ is
P3: transport job arrival in intrabay_2 p3 ¼ Pðst−1 ¼ ðx1 ; x2 þ 1; v1 ; k þ aÞjst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ ¼
λ2 φ
ð6Þ
P4: transport job completion in intrabay_2
p4 ¼ Pðst−1 ¼ ðx1 ; x2 −1; v1 ; k þ aÞst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ μ ðv2 −½k þ aþ Þ⋅ð1−f 21 Þ ¼ 2 φ
ð7Þ
P5: transport job completion from intrabay_1 to intrabay_2 p5 ¼ Pðst−1 ¼ ðx1 −1; x2 ; v1 −1; k þ aÞst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ − μ ðv1 −½k þ a Þ⋅f 12 ¼ 1 φ
ð8Þ
P6: transport job completion from intrabay_2 to intrabay_1 p6 ¼ Pðst−1 ¼ ðx1 ; x2 −1; v1 þ 1; k þ aÞst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ μ ðv2 −½k þ aþ Þ⋅f 21 ¼ 2 φ
ð9Þ
P7: vehicle movement completion from intrabay_2 to intrabay_1 p7 ¼ Pðst−1 ¼ ðx1 ; x2 ; v1 þ 1; 0Þst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
( ¼
ω φ
if k ¼ 1 or a ¼ 1
0
otherwise
ð10Þ
P8: vehicle movement completion from intrabay_1 to intrabay_2
p8 ¼ Pðst−1 ¼ ðx1 ; x2 ; v1 −1; 0Þst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ ( ω if k ¼ −1 or a ¼ −1 ¼ φ 0 otherwise
ð11Þ
P9: dummy transition (system state does not change), except the number of vehicles on move p9 ¼ Pðst−1 ¼ ðx1 ; x2 ; v1 ; k 8
þ aÞjst ¼ ðx1 ; x2 ; v1 ; kÞ; aÞ ¼ 1− ∑ pi
ð12Þ
i¼1
let cðx1 ; x2 ; v1 ; k; aÞ be the expected instantaneous (one-period) cost that considers job waiting and vehicle moving costs. For each job at intrabay_1 (intrabay_2), a unit waiting cost rate c1 ðc2 Þ per unit of time is accrued. To move a vehicle, a unit cost rate cv is calculated. cðx1 ; x2 ; v1 ; aÞ is defined by Eq. (12) as follows: c1 x1 þ c2 x2 þ cv jaj φ
cðx1 ; x2 ; v1 ; k; aÞ ¼
ð13Þ
let U nt ðx1 ; x2 ; v1 ; kÞ be the relative value function in the relative value iteration algorithm, according to Puterman [15]. In the ^ relative value iteration algorithm, a specific state ðx^1 ; x^2 ; v^1 ; k Þ is selected from the state space S, and the optimality equation searches for an optimal action in each iteration that minimizes the relative value function. Given the initial condition U 0 ðx1 ; x2 ; v1 ; kÞ ¼ 0; ∀x1 ; x2 ; v1 ; k, the iterative definition of U nt ðx1 ; x2 ; v1 ; kÞ is provided in Eq. (14). Optimality equation:
simulation model is finally constructed in Phase 3 to evaluate the DVAC performance. Subsequently, the proposed model is compared with other vehicle allocation control methods. Fig. 6 shows the connecting transport AMHS layout containing a single loop of an interbay system, two intrabay systems, forty tools, and four stockers in the simulation study. The vehicle used is the overhead hoist transporter (OHT) that holds the FOUP by its top flange. Table 1 summarizes the parameters of all tools in the proposed system. Table 2 presents examples of the wafer movement information and Table 3 shows the from–to distances and arrival rates between the tools. The arrival rate of the wafer lots is defined as the quantity per hour. This information can be retrieved from the manufacturing execution system (MES). For instance, the lot coded by LT-01-A enters the tool T101 at 23:08:15 on 03/01/10, finishes processing, and leaves at 23:52:40. The lot arrives at 00:12:20 on 03/02/10 in the next tool, T118 on the same bay, and leaves at 00:32:52 on 03/02/10. Thus, the arrival rate of the wafer lots can be obtained for analysis with the MES data. A simulation software eM-Plant is used to construct a virtual AMHS to compare the transport performances of different vehicle allocation control methods. For each simulation run, several performance measures were collected after 10 days of warm-up time, and stored in a data file for validation. Each simulation was run for 100 days. The simulation results were confirmed by real data in a wafer fab in Taiwan to validate the model. A complete set of simulation outputs for one sample system is provided in Example 3.1 to demonstrate the DVAC method and facilitate an understanding of the simulation models and results. Example 3.1. Implementation of DVAC and comparison with other methods In this example, the unit waiting cost of a transport job in both intrabays is 5 per min, and the moving cost for each vehicle is 0 per min. The vehicle cannot serve the other request when it
U nt ðx1 ; x2 ; v1 ; kÞ ¼ minfcðx1 ; x2 ; v1 ; k; aÞ þ E½W nt−1 ðst−1 Þjst ¼ ðx1 ; x2 ; v1 ; kÞ; ag a
8 9 < cðx1 ; x2 ; v1 ; k; aÞ = ¼ min þ ∑ ðPðst−1 jst ¼ ðx1 ; x2 ; v1 ; kÞ; at Þ W nt−1 ðst−1 ÞÞ a : ;
ð14Þ
St−1
in which, ( a∈f−1; 0; 1g a¼0
if k ¼ 0 if k≠0 0
Fig. 5. DVAC methodology. 0
^
W nt−1 ðx10 ; x20 ; v10 ; k Þ≡U nt−1 ðx10 ; x20 ; v10 ; k Þ−U nt−1 ðx^1 ; x^2 ; v^1 ; k Þ; ∀x1 ∈f0; 1; 2; :::; L1 g; x2 ∈f0; 1; 2; :::; L2 g; v1 ∈f0; 1; 2; :::; Vg; k∈f−1; 0; 1g
3. Implementation of optimal control and the simulation study A software tool developed by Microsoft Visual Basic for solving an optimality equation is used to conduct the numerical study and verify the transport performance of the DVAC in the present study. Discrete-event simulation models are applied in analyzing a complex AMHS system to evaluate the effectiveness of the DVAC model. Fig. 5 shows the schematic diagram of the proposed DVAC and simulation approaches. In Phase 1, a simulation model that considers the transport from–to table, different levels of transport job arrival rates, and vehicle number is constructed to generate vehicle service rates. With the phase 1 simulation output and other input parameters, the dynamic programming model is applied to generate optimal policies in Phase 2. The second
2333
Fig. 6. A virtual AMHS model.
2334
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
moves from one bay to the other bay. The other system parameters are listed as follows:
The total vehicle number is 20. The vehicle dispatching rule uses the combination of SD–NV
(shortest distance and nearest vehicle) and FEFS (the foremostencounter-first-served) rule. The job arrival of intrabay_1 and intrabay_2 is Poisson with rates of 8.0 and 7.5 jobs/min, respectively. The vehicle service rate is exponentially distributed with a rate obtained by a simulation model.
The implementation procedure of the DVAC method is listed below:
Table 1 Tool parameters summary. Type
Model
Quantity
Load ports /per tool
Process time (s/lot)
WPH (pcs/h)
Stocker Tool Tool Tool Tool Tool Tool Tool Tool Tool Tool
– SxxN NxxC OxxY CxxD AxxE ExxT WxxB LxxM TxxL DxxS
4 7 3 4 4 2 5 5 3 3 4
2 2 2 2 2 2 2 2 2 2 2
– 2400 3000 240 300 180 2700 1800 1500 2100 2100
– 37.5 30.0 375.0 300.0 500.0 33.3 50.0 60.0 42.9 42.9
WPH: Wafer per hour.
Table 2 Some examples of the lot movement information.
Lot
Part
Pieces
Start
Finish
Area
Tool
LT-01-A
TKN-A-1
25
T101
TKN-A-1
25
LIT
T118
LT-01-A
TKN-A-1
25
03/01/10 23:52:40 03/02/10 00:32:52 03/02/10 01:24:02
LIT
LT-01-A
03/01/10 23:08:15 03/02/10 00:12:20 03/02/10 00:45:11
ETCH
T206
The vehicle allocation rate to decrease or increase a vehicle is exponentially distributed at a rate of 1 vehicle/min. The cross-interbay ratios of intrabay_1 and intrabay_2 are 0.16 and 0.18, respectively.
Step 1: Offline calculation of the DVAC policy A software tool is implemented to solve the optimality equation and output optimal actions of all states. The optimal actions are different for each state, as shown in Fig. 7. For example, in Fig. 7(a), the optimal action for state (10, 0, 10), dot M, is to move a vehicle from intrabay_2 to intrabay_1. However, in Fig. 7(c), dot N, the optimal action for state (10, 16, 10) is to move a vehicle from intrabay_1 to intrabay_2. Although dot N has the same x1 and v1 values as dot M, the intrabay_2 job queue x2 of dot N is 16, which results in a different optimal action to decrease a vehicle of intrabay_1. The control limit lines in Fig. 7 are those states that remain constant (i.e., no vehicle change action for any intrabay). Step 2: Implementation of the optimal policy in the simulation system The offline pre-calculated DVAC policies in Step 1 are stored in the simulation look-up tables. In the real-time operation of a manufacturing system, the MCS can search the look-up table and choose an optimal action, accordingly. The vehicle waiting time performance of transport jobs in the DVAC model is compared with other vehicle allocation control methods used in the practical production fab, including fixed vehicle allocation control (FVAC), empty vehicle balancing allocation control (EVBAC), and zone control (ZC). FVAC FVAC is widely used in the traditional AMHS system. In FVAC, each intrabay maintains a fixed vehicle number, and the MCS allocates vehicles to each intrabay based on the default setting. In the current study, the fixed vehicle numbers of both intrabay_1 and intrabay_2 are set to 10. To illustrate, when the vehicle number in intrabay_1 is higher than 10, the MCS allocates the vehicles to intrabay_2 until the vehicle number in intrabay_1 and intrabay_2 are both 10. When the vehicle
Table 3 From–to distance (Upper corner, meters) and arrival rate (Lower corner, lot moves/hour) between tools. From–to
S100
S100 T101 T102 T103 T104 T105 T106 T107 T108 T109 T110 ⫶
52.5 2.1 50.0 1.7 47.5 1.8 45.0 1.9 42.5 1.6 40.0 2.0 37.5 1.5 35.0 4.2 32.5 3.8 30.0 5.1 ⫶
T101
T102
T103
T104
T105
T106
T107
T108
T109
T110
2.5 6.6
5.0 4.4 2.5 0.0
7.5 5.5 5.0 0.0 2.5 0.0
10.0 6.6 7.5 0.0 5.0 0.0 2.5 0.0
12.5 4.4 10.0 0.0 7.5 0.0 5.0 0.0 2.5 0.0
15.0 5.5 12.5 0.0 10.0 0.0 7.5 0.0 5.0 0.0 2.5 0.0
17.5 2.2 15.0 0.0 12.5 0.0 10.0 0.0 7.5 0.0 5.0 0.0 2.5 0.0
20.0 5.5 17.5 1.0 15.0 0.6 12.5 1.0 10.0 1.0 7.5 0.8 5.0 0.7 2.5 0.6
22.5 3.3 20.0 0.4 17.5 0.7 15.0 0.7 12.5 0.8 10.0 0.5 7.5 0.5 5.0 0.7 2.5 0.0
25.0 4.4 22.5 0.9 20.0 0.8 17.5 1.3 15.0 0.8 12.5 0.9 10.0 0.7 7.5 0.8 5.0 0.0 2.5 0.0
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 40.0 0.0 37.5 0.0 35.0 0.0 32.5 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 40.0 0.0 37.5 0.0 35.0 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 40.0 0.0 37.5 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 40.0 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 42.5 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 45.0 0.0 ⫶
52.5 0.0 50.0 0.0 47.5 0.0 ⫶
52.5 0.0 50.0 0.0 ⫶
52.5 0.0 ⫶
… … … … … … … … … … … …
⫶
⫶
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
2335
Fig. 7. An example of the optimal vehicle allocation control policy.
number in intrabay_1 is lower than 10, the MCS allocates the vehicles to intrabay_1 from intrabay_2 to balance the intrabay vehicles. EVBAC EVBAC is commonly used in the connecting transport AMHS system. In EVBAC, each intrabay keeps its expected empty vehicles, calculated as the total number of empty vehicles multiplied by a target ratio. The MCS automatically balances all empty vehicles to each intrabay according to its target ratio. In the current study, the balancing ratios of both intrabay_1 and intrabay_2 are equal to 0.5 (i.e., the total empty vehicles in this system are equally allocated to intrabay_1 and intrabay_2 using the EVBAC method). ZC Wang et al. [17] used a ZC strategy to control the feasible vehicle number at each intrabay. This control method restricts the actual number of vehicles at each intrabay at an interval value. The lower or upper limit of the vehicle number is set as the feasible solution. The MCS controls the vehicle number of each intrabay in this feasible solution. In the current paper, the lower and upper limits of each intrabay are set at 6 and 14, respectively. For example, the MCS allocates a vehicle from intrabay_2 to intrabay_1 when the vehicle number in intrabay_1 is 5. Meanwhile, the MCS allocates a vehicle from intrabay_1 to intrabay_2 when the vehicle number in intrabay_1 is 15. Besides ZC with (6, 14), two other ZC parameters at (7, 13) and (8, 12) are added for comparison.
We implemented the DVAC and all other methods in the simulation model. All vehicle allocation control methods were coded into the method object of the simulation model. We measured the vehicle waiting time, total transport time, and throughput of these three important indices in the model to evaluate transport performance. The vehicle waiting time T2 and the total transport time Ttotal are clearly defined in Section 2. The
Fig. 8. Vehicle waiting time under different vehicle allocation controls.
throughput measures the number of transport jobs completed within this simulation period. Wang et al. [17] concluded that the flow rate affects the transport performance of the connecting transport AMHS. In the current paper, the five flow rate levels were 930, 976.5, 1023, 1069.5, and 1116 lots/h. The standard flow rate was 930 lots/h, and the other four flow rates were 105%, 110%, 115%, and 120% of the standard loading. The comparison of the vehicle waiting time, total transport time, and throughput of these four methods with the statistical hypothesis test indicated a significant difference among these three performances under different flow rates. The three performance results (as shown in Figs. 8–10) show that the proposed DVAC outperforms all other methods in terms of vehicle waiting time, total transport time, and throughput by at least 7.2%, 2.1% and 1.3%, respectively. Table 4 presents the average values and the 95% confidence intervals of the three performance results. Table 5 shows the p-values of the paired t-tests of these measurements. These two tables reveal that the DVAC dominates over the other methods on vehicle waiting time and total transport times at all flow rates, which suggests that AMHS vehicles using the DVAC method provide a better service rate compared with others. In terms of throughput performance, only DVAC performs more transport counts at high flow rate compared with the other three
2336
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
methods. As an example, in a practical production fab, many transport jobs in the load ports of tools or stockers wait for the AMHS vehicle service at a high flow rate. When vehicles quickly move the wafer lots to tools and stockers, the manufacturing system can produce more wafer lots and minimize the tool idle time. Hence, the DVAC shows a more satisfactory throughput performance at a high flow rate with each vehicle having a better
Fig. 9. Total transport time under different vehicle allocation controls.
Fig. 10. Throughput performance under different vehicle allocation controls.
service rate and the AMHS system completing more transport jobs within the simulation period. Example 3.2. Comparisons of DVAC rules with different transport job waiting costs Minimizing the lost time of high utilization tools is a relevant management topic because of the high investment cost in semiconductor manufacturing. One of the reasons for the time loss of impact tools is the delay due to waiting for the AMHS vehicle to transport a wafer lot to the tool from the stockers or preceding tools. To reduce delays, managers have to allocate more vehicles to minimize vehicle waiting time in the intrabay. By applying the DVAC method, managers can easily set up different transport job waiting costs for each intrabay to allocate optimal empty vehicles to that specific intrabay. In the present example, we studied the effect of different transport job waiting cost parameters. In our numerical example, the system parameters were identical to those in Example 3.1, except for the job unit waiting cost. We assumed the transport job unit waiting costs of intrabay_1 and intrabay_2 (c1,c2) be (2,8), (3,7), (5,5), (7,3), and (8,2). Five different vehicle allocation control policies were then sequentially generated using the DVAC model: DVAC_2_8, DVAC_3_7, DVAC_5_5, DVAC_7_3, and DVAC_8_2. Fig. 11 shows an example of the vehicle control limit with all intrabay_2 jobs equal to 11 under different job unit waiting costs. Most states allocate a vehicle from intrabay_2 to intrabay_1 when the job unit waiting cost of intrabay_1 is higher than that of intrabay_2, such as the control limit DVAC_8_2. These observations suggest that if intrabay_1 has high-utilization tools, such as photo-scanners, the DVAC method can be applied to move more empty vehicles to this photo intrabay in many situations. This useful decision policy can be implemented to the MCS vehicle
Table 4 Comparison of vehicle allocation methods under different flow rates. Flow rate (lots/h)
Method
Vehicle waiting time (95% conf. intv.)
930.0
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
25.71 28.28 28.86 28.07 28.03 28.10
(25.64, 25.79) (28.21, 28.34) (28.78, 28.93) (27.95, 28.20) (27.93, 28.12) (28.00, 28.19)
84.61 86.65 87.97 87.04 86.65 86.51
(84.47, 84.75) (86.45, 86.86) (87.79, 88.15) (86.77, 87.31) (86.43, 86.86) (86.29, 86.74)
1839557 1842130 1833828 1831746 1836851 1840858
(1838063, 1841051) (1840283, 1843977) (1831869, 1835787) (1829686, 1833806) (1834727, 1838975) (1838525, 1843191)
976.5
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
34.94 37.26 37.63 37.62 37.46 37.32
(34.88, 34.99) (37.20, 37.31) (37.56, 37.70) (37.52, 37.72) (37.38, 37.54) (37.25, 37.38)
96.28 98.77 99.24 99.08 98.84 98.62
(96.19, 96.38) (98.59, 98.94) (99.10, 99.39) (98.86, 99.29) (98.66, 99.03) (98.45, 98.80)
1890447 1875665 1866911 1861176 1869648 1875767
(1888534, 1892360) (1873060, 1878270) (1864146, 1869676) (1858515, 1863837) (1867620, 1871676) (1873139, 1878395)
1023.0
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
39.53 41.83 42.21 42.26 42.12 41.96
(39.49, 39.57) (41.76, 41.89) (42.14, 42.28) (42.19, 42.33) (42.04, 42.20) (41.90, 42.03)
102.86 104.74 105.08 105.02 104.85 104.69
(102.76, 102.95) (104.61, 104.88) (104.92, 105.24) (104.87, 105.16) (104.69, 105.00) (104.56, 104.81)
1918965 1884191 1874267 1871376 1877447 1883763
(1916834, 1921096) (1880953, 1887429) (1870786, 1877748) (1868207, 1874545) (1874056, 1880838) (1880372, 1887154)
1069.5
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
42.98 45.45 45.86 45.95 45.71 45.56
(42.93, 43.03) (45.36, 45.54) (45.77, 45.94) (45.88, 46.01) (45.64, 45.78) (45.50, 45.63)
107.72 109.60 109.91 109.83 109.70 109.58
(107.59, 107.85) (109.44, 109.76) (109.78, 110.04) (109.69, 109.97) (109.56, 109.84) (109.42, 109.74)
1918282 1884166 1878214 1874160 1880534 1884568
(1915335, 1921229) (1880295, 1888037) (1875166, 1881262) (1870990, 1877330) (1877327, 1883741) (1880841, 1888295)
1116.0
DVAC FVAC EVBAC ZC(6,14) ZC(7,13) ZC(8,12)
45.93 48.12 48.59 48.54 48.40 48.24
(45.87, 45.98) (48.03, 48.20) (48.49, 48.68) (48.45, 48.63) (48.32, 48.48) (48.16, 48.32)
111.73 113.28 113.61 113.46 113.44 113.25
(111.57, 111.89) (113.11, 113.44) (113.44, 113.78) (113.32, 113.60) (113.30, 113.57) (113.07, 113.43)
1920159 1887498 1878474 1881195 1883250 1888349
(1916783, 1923535) (1883540, 1891456) (1874299, 1882649) (1878144, 1884246) (1880096, 1886404) (1880414, 1892684)
Total transport time (95% conf. intv.)
Throughput (95% conf. intv.)
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
2337
Table 5 p-values of the paired t-tests for the comparison of vehicle allocation methods. Flow rate (lots/h) DVAC vs. FVAC
930.0 976.5 1023.0 1069.5 1116.0
Vehicle waiting time 1.68E−29 3.39E−30 9.37E−33 1.93E−34 4.90E−32
DVAC vs. EVBAC Total transport time 3.96E−17 7.35E−22 1.93E−21 7.38E−23 4.14E−19
Throughput 0.0092 9.63E−10 4.83E−18 4.85E−18 7.70E−16
DVAC vs. ZC(6,14)
930.0 976.5 1023.0 1069.5 1116.0
Vehicle waiting time 4.58E−24 6.97E−29 1.83E−33 6.74E−39 5.01E−34
Vehicle waiting time 3.25E−31 1.85E−30 7.18E−32 2.99E−37 1.71E−33
Total transport time 3.75E−23 1.36E−25 1.09E−21 1.25E−24 5.80E−21
Throughput 5.87E−06 1.18E−15 4.08E−20 1.76E−21 3.50E−18
Total transport time 1.57E−17 7.71E−22 1.60E−20 4.33E−23 4.30E−23
Throughput 0.0149 9.85E−15 3.87E−20 1.66E−20 8.49E−21
DVAC vs. ZC(7,13) Total transport time 4.50E−17 1.59E−21 9.97E−21 6.86E−24 1.39E−21
Throughput 1.35E−07 1.56E−16 1.88E−21 1.19E−22 1.28E−18
Total transport time 4.37E−17 3.96E−21 3.73E−21 5.91E−23 1.80E−19
Throughput 0.1431 1.69E−09 1.52E−18 7.44E−18 1.00E−15
Vehicle waiting time 4.58E−25 3.90E−30 5.79E−33 2.39E−36 3.95E−35
DVAC vs. ZC(8,12)
930.0 976.5 1023.0 1069.5 1116.0
Vehicle waiting time 1.51E−25 8.26E−30 1.72E−32 6.57E−37 1.57E−35
control algorithm to satisfy intrabays with high empty-vehicle demands. This result reflects the actual behavior observed in the wafer fab. Example 3.3. Comparisons of DVAC rules with different vehicle moving costs In the present example, we study the effects of different cost parameters. In the numerical example, the system parameters are identical to those in Example 3.1, except for the vehicle unit moving cost. We set the vehicle moving cost cv of both intrabay_1 and intrabay_2 to 20. Two DVAC policies can be generated: DVAC with a vehicle moving cost of 0 and DVAC with a vehicle moving cost of 20. Fig. 12 shows the vehicle control limit under different vehicle unit moving costs. The DVAC with a vehicle moving cost of 0 is only a control limit characterized by a line with no vehicle change action in the diagram. An upside area indicates a vehicle allocation from intrabay_1 to intrabay_2, whereas a downside area indicates a vehicle allocation from intrabay_2 to intrabay_1. The DVAC with a vehicle moving cost of 20 has upper and lower control limits. The area between the upper and lower control limit lines indicates that keeping no vehicle changes the action. The upside area above the upper control limit line indicates a vehicle allocation from intrabay_1 to intrabay_2, whereas the downside area below the lower control limit line indicates a vehicle allocation from intrabay_2 to intrabay_1. The comparison of the DVAC method with different vehicle moving costs indicate no significant difference among the vehicle waiting time, total transport time, and throughput performance in the simulation experiment results. Figs. 13–15 show the transport permanence comparison of these two policies. The vehicle allocation control policy with a vehicle moving cost of 20 showed good transport performances similar to those without vehicle moving cost. This result indicates that the same transport performance can be obtained without adding a vehicle to one intrabay or subtracting one from another in many situations. The MCS only allocates an empty vehicle to the target intrabay whenever necessary. Aside from a satisfactory transport performance, this policy of using a vehicle moving cost of 20 also involves fewer vehicles cross-interbay
Fig. 11. Vehicle control limit under different job unit waiting costs.
movements and prevents vehicle traffic jam in the interbay connections by observing a simulation animation. In the practical production fab, managers can improve the AMHS transport performance as well as save vehicle energy resources and usage of consumer parts by applying this method.
4. Conclusion This paper examines the DVAC problems of the AMHS in semiconductor manufacturing. Two AMHS state variables, namely, the number of jobs in the transport job queue and the vehicle number, describe the transport job demand and AMHS service capability of each intrabay and are applied in the dynamic programming method. A DVAC method is proposed to adjust the vehicle number of each intrabay and to reduce the waiting time of wafer lots for AMHS vehicle service. Stochastic dynamic programming is used to solve the vehicle allocation control problem. The objective is to find optimal vehicle control policies to minimize the long-term average waiting cost of all transport jobs. Although AMHS systems are not rigorous “Markov” systems, MDP remains as an effective tool in modeling AMHS system. We do not assume the exponential transportation and inter-arrival time distribution in
2338
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
Fig. 12. Vehicle control limit under different vehicle moving costs.
Fig. 13. Vehicle waiting time under different DVAC rules.
Fig. 15. Throughput performance under different DVAC rules.
Fig. 14. Total transport time under different DVAC rules.
our simulation study. However, the simulation results strongly suggest that the DVAC method can significantly reduce the vehicle waiting time of the wafer lots as well as the total transport time compared with the other methods. For control, the optimal policy generated by the DVAC model can be stored in the MCS look-up tables and is applied to daily operations in a practical real-time production environment. By examining the model with only two intrabays, we gain valuable insights into the optimal policy structure in the N-intrabay system. However, in a more complicated system, overcoming the computational complexity issue in larger systems with expanded state space remains unclear. This consideration complicates our heuristics, and simultaneously provides an interesting challenge for further research.
J.T. Lin et al. / Computers & Operations Research 40 (2013) 2329–2339
Acknowledgment The authors wish to thank the anonymous referees of this manuscript whose thorough review greatly improved the clarity and readability of our paper. We thank the National Science Council, Taiwan ROC, for support under contract NSC96-2628-E007-032-MY3. References [1] Agrawal GK, Heragu SS. A survey of automated material handling systems in 300-mm semiconductor fabs. IEEE Transactions on Semiconductor Manufacturing 2006;19(1):112–20. [2] Baker BM, Ayechew MA. A generic algorithm for the vehicle routing problem. Computers and Operations Research 2003;30:787–800. [3] Baras JS, Dorsey AJ. Stochastic control of two partially observed competing queues. IEEE Transactions on Automatic Control 1981;26(5):1106–17. [4] Down DG, Lewis ME. Dynamic load balancing in parallel queueing systems. Stability and Optimal Control 2006;168(2):509–19. [5] Egbelu PJ, Tanchoco JMA. Characterization of automatic guided vehicle dispatching rules. International Journal of Production Research 1984;22 (3):359–74. [6] Heung TH, Ho TK, Fung YF. Coordinated road-junction traffic control by dynamic programming. IEEE Transactions on Intelligent Transportation Systems 2005;6(3):341–50.
2339
[7] Kobza JE, Shen YC, Reasor RJ. A stochastic model of empty-vehicle travel time and load request service time in light-traffic material handling system. IIE Transaction 1998;30:133–42. [8] Kyriakidis EG, Pavitsos A. Markov decision models for the optimal maintenance of a production unit with an upstream buffer. Computers and Operations Research 2009;36:1993–2006. [9] Li Z, Tao F. On determining optimal fleet size and vehicle transfer policy for a car rental company. Computers and Operations Research 2010;37:341–50. [10] Liao DY, Fu HS. Speed delivery: dynamic OHT allocation and dispatching in large-scale 300-mm AMHS management. IEEE Robotics and Automation Magazine 2004:22–32. [11] Lin JT, Wang FK, Wu CK. Simulation analysis of the connecting transport AMHS in a wafer fab. IEEE Transactions on Semiconductor Manufacturing 2003;16 (3):555–64. [12] Lin JT, Wang FK, Wu CK. Connecting transport AMHS in a wafer fab. International Journal of Production Research 2003;41(3):529–44. [13] Lippman SA. Applying a new device in the optimization of exponential queuing systems. Operations Research 1975;23(4):687–710. [14] Maxwell WL, Muckstadt JA. Design of automatic guided vehicle systems. IIE Transaction 1982;14(2):114–24. [15] Puterman M. Markov decision process: discrete stochastic dynamic programming. John Wiley & Sons Inc; 1994. [16] Renaud J, Laporte G, Boctor F. A tabu search heuristic for the multi-depot vehicle routing problem. Computers and Operations Research 1996;23: 229–35. [17] Wang FK, Lin JT. Performance evaluation of an automated material handling system for a wafer fab. Robotics and Computer Integrated Manufacturing 2004;20(2):91–100.