- Email: [email protected]
Optimal Linear Quadratic Gaussian Problem Applied to Reverse Logistics System
Proceedings of the 13th IFAC Symposium on Information Control Problems in Manufacturing Moscow, Russia, June 3-5, 2009
Optimal Linear Quadratic Gaussian Problem Applied to Reverse Logistics System Oscar Salviano Silva Filho Centro de Tecnologia de Informação Renato Archer – CTI Campinas – SP – Brazil (e-mail: [email protected]) Abstract: Based on a reverse logistics system, a discrete-time linear quadratic stochastic problem is formulated. From this problem, the idea is to develop an optimal production policy, which combines the production of new and remanufactured products to meet the demand. Assuming the fluctuation of demand and return rate of used-products as stationary and normally distributed processes, an open-loop feedback strategy can be derived in order to provide such a production policy to the stochastic problem. This strategy allows drawing sub-optimal inventory-production scenarios by varying some parameters of the model such as: return rate of used-products, delay of collecting, or even both. At last, a simple example will be presented and it will allow comparing an open-loop feedback policy with a classical open-loop policy. Keywords: reverse logistics, supply chain, remanufacturing, optimization, simulation. 1. INTRODUCTION There are a lot of different definitions for reverse logistics, but essentially they refer to the logistics processes of dealing with recycling, waste disposal, and management of hazardous materials. These processes require typical activities of planning, implementing and controlling the flow of material throughout the forward and reverse channels of the supply chain. Generically, the objective of reverse logistics is to move used-products from the market to their final destination with the aim of capturing value, or proper disposal. Operationally, the reverse logistics must be understood as the process of recycling or remanufacturing used products in order to reduce waste. In literature, there are innumerous papers related to logistic reverses issues. Most of them are based on quantitative models that are used to represent remanufacturing and recycling activities in the reverse channel. Fleischmann et al (1997) provide a typology of quantitative models for reverse logistics, which is based on three kinds of problems, namely: (i) reverse distribution problems; (ii) inventory control problems in systems with return flows; and (iii) production planning problem with reuse of parts and materials. In short, the first is concerned with the collection and transportation of used-products and packages. According the authors, “the reverse distribution can take place through the original forward channel, through a separate reverse channel, or through combinations of the forward and the reverse channel”; the second is related to appropriate control mechanism that allows returning the used products into the market; and, the third is associated with the planning of the reuse of items, parts and products without any additional process of remanufacturing. At last, it is worth mentioning that there are many different approaches to deal with each one of these problems.
978-3-902661-43-2/09/$20.00 © 2009 IFAC
In this paper, the focus is on the second type of problems cited above. Such problems can be also separated into two categories (Fleischmann et al, 1997), namely: repair problems, which mean the replacement of failed items by spares; and recovery problems which mean to remanufacture a product entirely and then replaced it into the marketplace. Particularly, the interest here is to look at a problem that belongs to this second category. Such a problem is subject to production-inventory system with a special structure for recovering used-products from market and overhauling or disposing of them. Besides, the random nature of the return rate of used-products makes the remanufacturable inventory system a stochastic process. As a consequence, recovery problems should be formulated as stochastic optimization problems. It is worth realizing that stochastic remanufacturable inventory systems are very commons in reverse logistics problems where return rates are directly dependent on the demand, whose fluctuation over time-periods is described by a stochastic process. Fleischmann et al. (1997) have shown that the traditional classification of stochastic inventory systems based on periodic (discrete) or continuous-time review models can be replicated for application in recovery problems. In this context, the continuous-time deterministic inventory model introduced by Dobos (2003) is reformulated here in order to follow a discrete-time stochastic format, where demand is random variable. Assuming that the demand fluctuation is described by a normal stationary process, the stochastic problem can be transformed into an equivalent one. This equivalent problem is result of the application of a suboptimal approach named OLFC (i.e. Open-loop Feedback Controller); see Bertesekas (2007). The OLFC approach provides optimal production policies that are updated over time-periods. Such policies can be used to create inventoryproduction scenarios, which are built from the variation of
1008
10.3182/20090603-3-RU-2001.0040
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
Serviceable inventory
Manufacturing Remanufacturable inventory
new products
Remanufacturing
used products
Marketplace
disposal
Fig. 1. Diagram of a reverse logistics system some parameters of the stochastic problem, as for instance: return rate of used products, delays in the collection of usedproducts, customer-satisfaction level, among others. To illustrate the applicability of this model, a simple example is considered that compares the OLFC policy with classical open-loop policy.
which is related to the quality of returned product, which may be inappropriate for remanufacturing; and the second has a financial justification, that is: the idea of remanufacturing all products, which return in good condition, increase unnecessarily holding costs of serviceable products. So, the last strategy increases the overall cost for running the system. 2.1 Inventory-production system
2. THE STOCHASTIC MODEL Figure 1 illustrates the schematic diagram for reverse logistics system that is considered in this study. Note that there are two types of stockkeeping units in this scheme: the first unity denotes the serviceable inventory system. It stocks manufactured and remanufactured products that are used to meet demand in the marketplace. The second represents the remanufacturable inventory system. It stocks used-products that are collected from the market. After sorting, they can be remanufactured or sent to discard. It is valuable to note that Figure 1 represents situations often found in practice. In fact, it shows the forward and reverse channels of a supply chain, to which the processes of collection, storage and remanufacturing are economically viable. This means that costs with transport and storage in reverse channel should consume only a small fraction of the budget of the organization. Furthermore, it is considered that the cost of remanufacturing is less than the cost of manufacturing. Examples of organizations that can be identified by the diagram (shown in Figure 1) are those companies that manufacture and recycle, for instance, bottles, cans, containers, among others items. For the purposes of this paper, some features and properties, that the system of Figure 1 should exhibit, are listed in sequel: a) demand for new products should be met by the combination of both manufactured and remanufactured products; b) the fluctuation of demand is a stationary stochastic process that follows a normal distribution function whose first and second statistics moments are perfectly known; c) the return rate of use-products depends directly on the demand’s level and, as a consequence, it is a random variable that follows the same probability distribution of demand; d) there are upper boundaries for both serviceable and remanufacturable inventory storages; e) there are also upper limits of both manufacturing and remanufacturing capacities; and f) used-products may be discarded after being collected from the market. There are two main reasons to discard used-products: the first has a technical justification,
The inventory-production system, described in the figure 1, can be mathematically modelled by discrete-time stochastic equations, with two state variables and three control variables. In this context, state variables describe the inventory levels of the serviceable and remanufacturable storages; while control variables describe the rates of manufacturing, remanufacturing, and discard processes. Thus, the system can be defined as follows: The discrete-time stochastic system is described by the following two difference-equations, which represent inventory balance systems related, respectively, to forward and reverse channels of the supply chain: x1(k+1) = x1(k)+u1(k)+u2(k)-d(k)
(1)
x2(k+1) = x2(k)-u2(k)-u3(k)+ r(k)
(2)
where, for each period k of the planning horizon T, the following notation should be considered: x1(k) = inventory level in store 1; x2(k) = inventory level in store 2; u1(k) = production rate of manufacturing; u2(k) = production rate of remanufacturing; u3(k) = discard rate; d(k) = demand level (a random variable); r(k) = η⋅d(k). It denotes the return rate of used-products, which is dependent on the demand d(k). The parameter η denotes the percentage of expected returns (0≤η≤1). Such a percentage should be provided a priori by the decision maker. It is assumed that the demand variable is stationary and normally distributed, with first and second statistics moments given, respectively, by dˆ (k ) ≥ 0 and σ 2 finite for each d
period k. Note that the normality hypothesis adopted here to describe demand fluctuation follows a justification found in Graves (1999). The inventory levels x1(k) and x2(k) depend
1009
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
directly on the demand behavior then, as a consequence, they are random variables. Furthermore, due to the linearity of equations (1) and (2), such variables follow also a normal probability distribution. Therefore, these stochastic variables can be completely described by the evolution of first and second statistic moments as follows: ⎧⎪xˆ1 (k + 1) = xˆ1 (k ) + u1 (k ) + u 2 (k ) − dˆ (k ) Mean ⎨ ⎪⎩xˆ 2 (k + 1) = xˆ 2 (k ) − u 2 (k ) + u 3 (k ) − η ⋅ dˆ (k )
Variance
⎧⎪Var( x1 (k )) = k ⋅ σ d2 ⎨ ⎪⎩Var( x 2 (k )) = k ⋅ η2 ⋅ σ d2
(3)
(4)
⎧ σ2 ⎪Pr ob ( xˆ1 (k ) − Δx1 2 ≤ x1 (k ) ≤ xˆ1 (k ) + Δx1 2 ) ≥ 4 ⋅ k ⋅ d2 Δx1 ⎪ (6) ⎨ 2 2 η ⋅ σ ⎪ d x x Δ Δ 2 ˆ ≤ x 2 (k ) ≤ xˆ 2 (k ) + 2 ) ≥ 4 ⋅ k ⋅ ⎪Pr ob ( x 2 (k ) − 2 2 Δx 22 ⎩
Note that inequalities describe in (6) represent the situation where occurs the largest chances (i.e. probability) of the inventory variables x1(k) and x2(k) belong to the spaces defined by their respective constraints. In other words, the chances of violation of the constraints given in (5) are minimal. The dark area in the figure 2 tries to illustrate these theoretical aspects.’
Var( x (k )) Pr ob. ( 1 ⋅ Δx ≤ x (k ) ≤ 3 ⋅ Δx ) ≥ 1 − 4 ⋅ k ⋅ 2 2 Δx 2
2.2 Physical constraints
The serviceable and remanufacturable inventory systems (1) and (2) are, respectively, constrained by lower and upper limits of storage. This means that these inventory systems can receive minimum and maximum volumes of products for each period k. In fact, in the serviceable store (1) is required a minimum volume of products (i.e. a safety stock) given by x1 , and is only allowed a maximum volume of products (i.e.,
limit of storage), given by x1 . Similarly, in the remanufacturable store (2), there are a safety stock, denoted by x 2 = 0 , and an upper bound of storage capacity, which is denoted by x 2 . It is important to point out that the violation of these limits of capacity lead to unnecessary increases on inventory costs.
Fig. 2. Maximum concentration of probability Based on the arguments above, the probabilistic indexes αk and βk can be defined by: ⎧ σ2 ⎪α k = 1 − 4 ⋅ k ⋅ d2 Δx 1 ⎪ ⎨ 2 2 ⎪ β = 1 − 4 ⋅ k ⋅ η ⋅ σd 2 ⎪ k Δx 2 ⎩
Due to the randomness of inventory variables, constraints should be considered in probability for the purpose of keeping them explicitly into the problem’s formulation, as will see ahead. Thus,
with k=0,1,…,T-1
Note that (7) is only valid if Δx1 ≥ 2 ⋅ σ d ⋅ T ⎧⎪Pr ob ( x1 ≤ x1 (k ) ≤ x1 ) ≥ α k ⎨ ⎪⎩Pr ob ( x 2 ≤ x 2 (k ) ≤ x 2 ) ≥ β k
(5)
where αk and βk are probabilistic indexes. Now, the objective is to choose proper indexes in such a way to guarantee noviolation of constraints (5). For such, let’s assume first that variables x1(k) and x2(k) have probability distributions that can be approximated by equivalent normal distribution with first and second statistical moments known a priori; note that such statistics are given, respectively, by (3) and (4). Based on these statistics and taking into account the Chebycheff's inequality (see Papoulis and Pillai, 2002), it follows, then, that the maximum value of probability density function for these variables will be occur, exactly, when the mean values of these inventory variables (i.e., xˆ1 (k ) and xˆ 2 (k ) ) reach the centre of their constraints, that is, xˆ1 (k ) ≈ ( x1 + x1 ) 2 and xˆ 2 (k ) ≈ ( x 2 + x 2 ) 2 ,
As a consequence, it is possible to show that (see Jazwinski (2007)):
(7)
and
Δx 2 ≥ 2 ⋅ η ⋅ σ d ⋅ T are verified. Fortunately, for the most practical applications, it is found that Δx1 >> σ d and Δx 2 >> η ⋅ σ d , which validates (7).
Since the terms 4 ⋅ σ d2 Δx12 and 4 ⋅ η2 ⋅ σ d2 Δx 22 can be computed a priori, it is possible to show that αk and βk tend to decrease proportionally with the evolution of the period k. This previous feature can be interpreted as follows: “the risk of infeasibility of constraints (5) decreases proportionally with the growing of the time-periods, and it reaches the minimal at the end of the planning horizon (k=T).” 2.3 The Criterion
For each period k, the function that describes the production costs of running systems (1)-(2) is given as follows: Ζk(x1,x2,u1,u2,u3) = h1⋅E{x1(k)2}+h2⋅E{x2(k)2}+c1⋅u1(k)2+ (8) c2⋅u2(k)2+c3⋅u3(k)2
1010
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
where h1 and h2 denote the inventory holding coefficients (prices) in the stores (1) and (2), respectively. Coefficients c1, c2 and c3 denote prices related respectively to manufacture, remanufacture and discard operations. The symbol E{.} denotes the expectation operator and it related to randomness of inventory variables x1 and x2. In addition, note that the total production cost is given by Z T =
∑
T k =1
zk .
Still with regard to (8), it is worth commenting that: (a) the use of quadratics functions to represent production costs has some advantages when compared with other functions. For instance, quadratic cost induces high penalties for large deviations of the decision variables from the origin but relatively small penalties for small deviation; see Bertesekas (2007); (b) the holding costs are not deterministic. In fact, because of the stochastic nature of the systems (1)-(2), the variable x1(k) and x2(k) are random variables, and so their real values can only be estimated in probability. This explains the reason of applying expectation operator to these variables, that is, E{x1(k)2} and E{x2(k)2}; and (c) the cost of collection of used-products is not deterministic. The reason of this is that the amount of product retuned r(k) is directly proportional to demand d(k) and so, it is a random variable. 2.4 Discrete-time stochastic optimal control problem
Based on the exposed above, a linear quadratic Gaussian (LQG) model with constraints can be formulated as follows:
{
} ∑ {E [h
Min Z T = E h1 ⋅ x1 (T) 2 + h 2 ⋅ x 2 (T ) 2 +
u1 ,u 2 ,u 3
]
T −1
k =0
1
⋅ x1 (k ) 2 +
}
+ h 2 ⋅ x 2 (k ) 2 + c1 ⋅ u1 (k ) 2 + c 2 ⋅ u 2 (k ) 2 + c 3 ⋅ u 3 (k ) 2 subject to x1 (k + 1) = x1 (k ) + u1 (k ) + u 2 (k ) − d (k ) (9) x 2 (k + 1) = x 2 (k ) − u 2 (k ) − u 3 (k ) + r (k ) Pr ob. ( x1 ≤ x1 (k ) ≤ x1 ) ≥ α k Pr ob. ( x 2 ≤ x 2 (k ) ≤ x 2 ) ≥ β k 0 ≤ u1 ( k ) ≥ u1 ; 0 ≤ u 2 ( k ) ≥ u 2 ; u 3 ( k ) ≥ 0
The probability index α is often denoted as the level of customer satisfaction; and here, it is used to measure the chances of satisfying customers with the combination of manufactured and remanufactured products. At last, note that the problem (9) provides a sequential optimal production policy {u1(k), u2(k), and u3(k); with k = 0,1,2, …, T-1} that optimizes the performance of (1)-(2). 3. SOLVING THE PROBLEM (9) BY OLFC APPROACH Due to constraints and dimensionality associated to stochastic problem (9), computing a true optimal production policy for (9) (i.e. a closed-loop solution) is not a trivial task. However, from the literature, it is possible to find several sub-optimal approaches to solve such a problem. Some of these numerical techniques can provide revised optimal production policies that are very close to the true optimal solution (Bertesekas, 1995); such as the case of the Open-Loop Feedback Controller (OLFC), which will be discussed in sequel.
The OLFC is an approach that allows updating the optimal production policy continually, taking into account measures of current states of the inventory systems (1)-(2). Note that such a feedback characteristic of the OLFC makes it an adaptive procedure. This means that it provides a better solution than that provided by an Open-Loop Controller (OLC), which does not include any new updates in the solution, except the one taken in the first period of planning horizon. It is worth mentioning that an open-loop policy is a static solution, which is based entirely on the mean values of the decision variables of the problem (9). It is worth bearing in mind that the problem (9) is a stochastic optimal control problem under perfect state information (Bertesekas, 2007). So, then, this means that inventory levels x1(k) and x2(k) can be measured exactly in the beginning of each period k. As a consequence, the OLFC approach can be applied directly to solve problem (9), without requiring any mechanism for estimating the states. Basically, the steps to apply such a procedure are: (Step 1) In the beginning of each new period k=t (with t=0, 1, …, T-1), the exact position of the serviceable and remanufacturable inventory levels (see figure 1) should be measured, that is, x1(t) and x2(t) are perfectly taken from systems (1)-(2); and (Step 2) From these measures, an optimal production policy, given by {u1(k), u2(k), and u3(k); with k=t, t+1, …, N-1} is computed by solving the following equivalent problem: Min Z t = h1 ⋅ xˆ1 (T ) 2 + h 2 ⋅ xˆ 2 (T ) 2 +
∑ {h T −1 k =t
1
⋅ xˆ1 (k ) 2 +
}
h 2 ⋅ xˆ 2 (k ) 2 + c1 ⋅ u1 (k ) 2 + c 2 ⋅ u 2 (k ) 2 + c 3 ⋅ u 3 (k ) 2 + Κ t subject to (10) xˆ1 (k + 1) = xˆ1 (k ) + u1 (k ) + u 2 (k ) − dˆ (k ); xˆ1 (k ) = x1 ( t ) xˆ 2 (k + 1) = xˆ 2 (k ) − u 2 (k ) − u 3 (k ) + rˆ (k ); xˆ 2 (k ) = x 2 ( t ) x1,α (k ) ≤ xˆ1 (k ) ≤ x1,α (k ) x 2,β (k ) ≤ xˆ 2 (k ) ≤ x 2,β (k ) 0 ≤ u1 ( k ) ≥ u 1 ; 0 ≤ u 2 ( k ) ≥ u 2 ; u 3 ( k ) ≥ 0 k = t , t + 1,L, N − 1
where the integration constant of the criterion Zt, denoted by Kt, depends on the variance of the demand and the initial period t. It is given by: K t = (h1 + η ⋅ h 2 ) ⋅ σ d2 ⋅ (T − t )
(11)
Note also that lower and upper boundaries of serviceable and remanufacturable inventory constraints (5) are now given by x1, α (k ) = x1 − k ⋅ σ d ⋅ Φ −1 (α k ) x1, α (k ) = x1 + k ⋅ σ d ⋅ Φ −1 (α k )
x 2,β (k ) = x 2 − k ⋅ η ⋅ σ d ⋅ Φ −1 (β k )
(12)
x 2,β (k ) = x 2 + k ⋅ η ⋅ σ d ⋅ Φ −1 (β k )
Besides, given the initial inventory levels of inventory variable (i.e., x1(t) and x2(t)), the problem (10) is solved from
1011
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
the period t to T. This solution does not take into account any information about the current level of the inventory systems (1)-(2). Thus, it is an open-loop solution. (Step 3) Since new measures of inventory levels x1(k) and x2(k) are not allowed from systems (1)-(2) for k > t, the OLFC approach selects only the first elements of the optimal production policy (i.e., {u1* (t), u *2 (t), and u *3 (t)} ) to apply as input into systems (1)-(2). The others components of the optimal sequence (i.e., {u1* (k), u *2 (k), and u *3 (k); k=t+1, t+2, …, T}) are completely ignored. Thus, as soon as new measures of inventories variables are taken, in the beginning of new period t+1, the OLFC procedure starts again from the step 2 and problem (10) is reformulated in order to be solved for periods k=t+1, ..., T. 4. EXAMPLE Let's consider a simple example of a company that manufactures a certain kind of product to meet demand in the marketplace. After use, the product is collected from the market. Then, quality tests are carried out in order to decide whether the product should be re-used or discarded. The company uses two units of stores in forward and reverse channels of its supply chain, as illustrated in the figure 1. Besides, it is assumed that: (i) the demand for this product is stationary, which means that the fluctuation of sales can be estimated with a good accuracy over time periods of the planning horizon and (ii) the unit cost of remanufacturing a returned product is lower than the cost of manufacturing a new product, which means that used-products can be easily overhauled and replaced to market.
and β1=0,04; and when k=6 implies that α6=0,4 and β6=0,25. It is worth noting that indexes αk and βk decrease proportionally with the evolution of period k. This means that for time-periods close to the end of planning horizon, the risk of violation of the constraints (5) is lower. 4.1 Optimal policies via OLFC and OLC
In this section, two production scenarios for systems (1)-(2) are discussed: the first scenario shows a suboptimal solution generated from the application of the OLC approach in the problem (10); whereas the second shows the suboptimal policy provided from the application of the OLFC approach to the same problem. The basic difference between the two scenarios is in the use of available information. In fact, in the first scenario, only initial serviceable and remanufacturable inventory levels are available to solve problem (10) and so, the solution is said open-loop. On the other hands, in the second scenario, the solution is continually updated. 4.1(a) Scenario: Open-loop Controller (OLC) Figures 3 and 4 illustrate inventory and production trajectories for forward and reverse channels. The striking feature of this scenario is the high production of remanufacturable products. Practically, all production of the company is being met by this process. In this way, it is interesting to note that: as initial inventory levels of serviceable and remanufacturable stores are high, the demand for products, in early initial periods of planning horizon, tends to be met completely by products available into the serviceable inventory store and, simultaneously, by those that are being currently manufactured; see solid lines in figures 3 and 4, during periods k = 0.
Aiming to develop a production policy for the next six months (T=6), the company decides to implement an optimal strategy based on the production planning problem given in (10). The demand fluctuation is considered stationary and normally distributed over the periods of planning horizon. Its first and second statistics moments are given in the Table 1. Table 1: Statistics on the demand k Jan Feb Mar Apr May Jun d(k) 604 597 610 614 609 580 Standard-deviation σd ≈ 50
Inventory levels
350 300 250 200 150 100 50 0
Serviceable Remanufacturable
0
500
boundaries of manufacturing and remanufacturing facilities are u1 = u 2 = 0 , u1 = 450 and u 2 = 400 ; (iv) inventory,
300
1012
2
3
4
5
6 months
Fig.3. Inventory level of manufactured
Other information: (i) initial inventories for serviceable and remanufacturable products are, respectively: x01 = 300 and x02 = 200; (ii) lower and upper boundaries of serviceable and remanufacturable inventories are given respectively by x1 = x 2 = 25 and x1 = x 2 = 300 ; (iii) lower and upper
production and disposal costs are respectively: h1=2, h2=1, c1=2, c2=1 and c3=1; (v) the rate of return is 80% (i.e., η=0.8). Note that the use of this high rate of return means a greater possibility of reuse of returnable products; and (vi) probabilistic indexes defined in (7) have their values taken from the following ranges αk ∈[0,06 0,4] and βk ∈[0,04 0,25]. Thus, for instance, when k = 1 implies that α1=0,06
1
Production and disposal rates
400
Manufacture Remanufature Disposal
200 100 0 0
1
2
3
4
5 months
Fig.4. Optimal production and disposal rates
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
The following periods show that the process of remanufacture becomes intense and its maximum capacity of production is already reached during period k=1 (see Figure 4, dashed line). Thus, it can be said that the mean demand is being met almost completely by remanufactured products (note – from figure 4, solid line – that the production rate of manufactured products are quite negligible). In addition, many used products are being discarded in order to maintain low inventory levels, which can be seen as competitive strategy of the company to reduce cost with inventories. 4.1(b) Scenario: Open-loop Feedback Controller (OLFC) The optimal inventory and production trajectories for forward and reverse channels are respectively exhibited in figures 5 and 6. The trajectories depicted in this scenario are similar to that exhibited to the OLC procedure (see Figures 3 and 4). The reason of this is that the fluctuation of demand is based entirely on the average levels given in Table 1. As a result, the optimal solution with updating (OLFC) and no-updating (OLC) become very close. However, it is possible to observe small differences. One of them is the levels of serviceable and remanufacturable inventories of the OLC policy that tend to increase slightly during end-periods of the planning horizon. This means that such a policy is more conservative (i.e. averse to risk) than OLFC policy, thus it increases safety-stocks close to the end-periods of planning horizon in order to guarantee customer satisfaction (see periods 5 and 6 in the figure 4). The difference between the two approaches can be easily seen from a cost analysis, as shown in Table 2. Note that the operation of the systems (1) and (2), under application of OLFC policy, reduces the total cost in 21% in relation to OLC policy. Inventory levels
350 300 250 200 150 100 50 0
Serviceable Remanufacturable
Table 2. Costs ($) of the two scenarios Costs Serviceable inventory cost Remanufacturable inventory cost Manufactured cost Remanufactured cost Disposal cost Total cost
OLFC 7.597 6.288 328.110 851.660 202.190 1.395.845
OLC 196.520 43.644 460.760 851.668 215.308 1.767.900
5. CONCLUSION This paper analyzed a particular type of reverse logistics problem. The problem is defined by two productioninventory systems, which represent the forward and reverse channels of a supply chain. A linear quadratic Gaussian (LQG) model with constraints was formulated to provide optimal policies for manufacturing, remanufacturing and disposal variables. The model proposed considers chanceconstraints on inventory variables that are computed in such a way to get the best benefits of the physical intervals (see section 2.2). The paper discussed also about a sub-optimal approach (denoted as OLFC) in order to apply to LQG problem. It was shown that OLFC procedure can be used in a simulation scheme to provide production scenarios. Through these scenarios, it is possible to compare different production plans, helping managers to make decision about the use of available resources. The OLFC approach was applied to a simple example and it presented better solution than the classical OLC approach. The reason is that OLFC policy can be updated over the periods, whenever new information about the inventories is available. This means that a higher rate of return will lead to lower production costs and, as a consequence, greater profitability. Finally, it is worth mentioning that such profitability will occur only if the costs for remanufacturing are lower than those for manufacturing new products. REFERENCES
0
1
2
3
4
5
6 months
Fig. 5. Serviceable and remanufacturable inventory levels Production and disposal rates
500 400
Manufacture Remanufature Disposal
300 200 100 0 0
1
2
3
4
5 months
Fig. 6. Optimal production and disposal rates
Bertesekas, D. P. (2007). Dynamic Programming and Optimal Control, Athena Scientific, Vol. 1. USA. Dobos, Imre (2003) Optimal Production-inventory strategies for HMMS-type reverse logistics system, Int. J. Production Economics, 81-82, 351-360. Fleischman, M., Bloemhof-Ruwaard, J. M., Dekker, R., Van der Laan, E., Van Nunen, Jo A. E. E., and Van Wassenhove, L. N. (1997) Quantitative models for reverse logistics: A review, European Journal of Operational Research, 103, 1-17. Graves, S. C. (1999). A Single-Item Inventory Model for a Non-stationary Demand Process, Manufacturing & Service Operations Management, Vol. 1, No 1. Jazwinski, A. H. (2007) Stochastic Process and Filtering Theory, Dover Publications. Papoulis A. and Pillai S. U. (2002) Probability, Random Variables, and Stochastic Process, McGraw-Hill Publishing Co.
1013
Optimal Linear Quadratic Gaussian Problem Applied to Reverse Logistics System Oscar Salviano Silva Filho Centro de Tecnologia de Informação Renato Archer – CTI Campinas – SP – Brazil (e-mail: [email protected]) Abstract: Based on a reverse logistics system, a discrete-time linear quadratic stochastic problem is formulated. From this problem, the idea is to develop an optimal production policy, which combines the production of new and remanufactured products to meet the demand. Assuming the fluctuation of demand and return rate of used-products as stationary and normally distributed processes, an open-loop feedback strategy can be derived in order to provide such a production policy to the stochastic problem. This strategy allows drawing sub-optimal inventory-production scenarios by varying some parameters of the model such as: return rate of used-products, delay of collecting, or even both. At last, a simple example will be presented and it will allow comparing an open-loop feedback policy with a classical open-loop policy. Keywords: reverse logistics, supply chain, remanufacturing, optimization, simulation. 1. INTRODUCTION There are a lot of different definitions for reverse logistics, but essentially they refer to the logistics processes of dealing with recycling, waste disposal, and management of hazardous materials. These processes require typical activities of planning, implementing and controlling the flow of material throughout the forward and reverse channels of the supply chain. Generically, the objective of reverse logistics is to move used-products from the market to their final destination with the aim of capturing value, or proper disposal. Operationally, the reverse logistics must be understood as the process of recycling or remanufacturing used products in order to reduce waste. In literature, there are innumerous papers related to logistic reverses issues. Most of them are based on quantitative models that are used to represent remanufacturing and recycling activities in the reverse channel. Fleischmann et al (1997) provide a typology of quantitative models for reverse logistics, which is based on three kinds of problems, namely: (i) reverse distribution problems; (ii) inventory control problems in systems with return flows; and (iii) production planning problem with reuse of parts and materials. In short, the first is concerned with the collection and transportation of used-products and packages. According the authors, “the reverse distribution can take place through the original forward channel, through a separate reverse channel, or through combinations of the forward and the reverse channel”; the second is related to appropriate control mechanism that allows returning the used products into the market; and, the third is associated with the planning of the reuse of items, parts and products without any additional process of remanufacturing. At last, it is worth mentioning that there are many different approaches to deal with each one of these problems.
978-3-902661-43-2/09/$20.00 © 2009 IFAC
In this paper, the focus is on the second type of problems cited above. Such problems can be also separated into two categories (Fleischmann et al, 1997), namely: repair problems, which mean the replacement of failed items by spares; and recovery problems which mean to remanufacture a product entirely and then replaced it into the marketplace. Particularly, the interest here is to look at a problem that belongs to this second category. Such a problem is subject to production-inventory system with a special structure for recovering used-products from market and overhauling or disposing of them. Besides, the random nature of the return rate of used-products makes the remanufacturable inventory system a stochastic process. As a consequence, recovery problems should be formulated as stochastic optimization problems. It is worth realizing that stochastic remanufacturable inventory systems are very commons in reverse logistics problems where return rates are directly dependent on the demand, whose fluctuation over time-periods is described by a stochastic process. Fleischmann et al. (1997) have shown that the traditional classification of stochastic inventory systems based on periodic (discrete) or continuous-time review models can be replicated for application in recovery problems. In this context, the continuous-time deterministic inventory model introduced by Dobos (2003) is reformulated here in order to follow a discrete-time stochastic format, where demand is random variable. Assuming that the demand fluctuation is described by a normal stationary process, the stochastic problem can be transformed into an equivalent one. This equivalent problem is result of the application of a suboptimal approach named OLFC (i.e. Open-loop Feedback Controller); see Bertesekas (2007). The OLFC approach provides optimal production policies that are updated over time-periods. Such policies can be used to create inventoryproduction scenarios, which are built from the variation of
1008
10.3182/20090603-3-RU-2001.0040
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
Serviceable inventory
Manufacturing Remanufacturable inventory
new products
Remanufacturing
used products
Marketplace
disposal
Fig. 1. Diagram of a reverse logistics system some parameters of the stochastic problem, as for instance: return rate of used products, delays in the collection of usedproducts, customer-satisfaction level, among others. To illustrate the applicability of this model, a simple example is considered that compares the OLFC policy with classical open-loop policy.
which is related to the quality of returned product, which may be inappropriate for remanufacturing; and the second has a financial justification, that is: the idea of remanufacturing all products, which return in good condition, increase unnecessarily holding costs of serviceable products. So, the last strategy increases the overall cost for running the system. 2.1 Inventory-production system
2. THE STOCHASTIC MODEL Figure 1 illustrates the schematic diagram for reverse logistics system that is considered in this study. Note that there are two types of stockkeeping units in this scheme: the first unity denotes the serviceable inventory system. It stocks manufactured and remanufactured products that are used to meet demand in the marketplace. The second represents the remanufacturable inventory system. It stocks used-products that are collected from the market. After sorting, they can be remanufactured or sent to discard. It is valuable to note that Figure 1 represents situations often found in practice. In fact, it shows the forward and reverse channels of a supply chain, to which the processes of collection, storage and remanufacturing are economically viable. This means that costs with transport and storage in reverse channel should consume only a small fraction of the budget of the organization. Furthermore, it is considered that the cost of remanufacturing is less than the cost of manufacturing. Examples of organizations that can be identified by the diagram (shown in Figure 1) are those companies that manufacture and recycle, for instance, bottles, cans, containers, among others items. For the purposes of this paper, some features and properties, that the system of Figure 1 should exhibit, are listed in sequel: a) demand for new products should be met by the combination of both manufactured and remanufactured products; b) the fluctuation of demand is a stationary stochastic process that follows a normal distribution function whose first and second statistics moments are perfectly known; c) the return rate of use-products depends directly on the demand’s level and, as a consequence, it is a random variable that follows the same probability distribution of demand; d) there are upper boundaries for both serviceable and remanufacturable inventory storages; e) there are also upper limits of both manufacturing and remanufacturing capacities; and f) used-products may be discarded after being collected from the market. There are two main reasons to discard used-products: the first has a technical justification,
The inventory-production system, described in the figure 1, can be mathematically modelled by discrete-time stochastic equations, with two state variables and three control variables. In this context, state variables describe the inventory levels of the serviceable and remanufacturable storages; while control variables describe the rates of manufacturing, remanufacturing, and discard processes. Thus, the system can be defined as follows: The discrete-time stochastic system is described by the following two difference-equations, which represent inventory balance systems related, respectively, to forward and reverse channels of the supply chain: x1(k+1) = x1(k)+u1(k)+u2(k)-d(k)
(1)
x2(k+1) = x2(k)-u2(k)-u3(k)+ r(k)
(2)
where, for each period k of the planning horizon T, the following notation should be considered: x1(k) = inventory level in store 1; x2(k) = inventory level in store 2; u1(k) = production rate of manufacturing; u2(k) = production rate of remanufacturing; u3(k) = discard rate; d(k) = demand level (a random variable); r(k) = η⋅d(k). It denotes the return rate of used-products, which is dependent on the demand d(k). The parameter η denotes the percentage of expected returns (0≤η≤1). Such a percentage should be provided a priori by the decision maker. It is assumed that the demand variable is stationary and normally distributed, with first and second statistics moments given, respectively, by dˆ (k ) ≥ 0 and σ 2 finite for each d
period k. Note that the normality hypothesis adopted here to describe demand fluctuation follows a justification found in Graves (1999). The inventory levels x1(k) and x2(k) depend
1009
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
directly on the demand behavior then, as a consequence, they are random variables. Furthermore, due to the linearity of equations (1) and (2), such variables follow also a normal probability distribution. Therefore, these stochastic variables can be completely described by the evolution of first and second statistic moments as follows: ⎧⎪xˆ1 (k + 1) = xˆ1 (k ) + u1 (k ) + u 2 (k ) − dˆ (k ) Mean ⎨ ⎪⎩xˆ 2 (k + 1) = xˆ 2 (k ) − u 2 (k ) + u 3 (k ) − η ⋅ dˆ (k )
Variance
⎧⎪Var( x1 (k )) = k ⋅ σ d2 ⎨ ⎪⎩Var( x 2 (k )) = k ⋅ η2 ⋅ σ d2
(3)
(4)
⎧ σ2 ⎪Pr ob ( xˆ1 (k ) − Δx1 2 ≤ x1 (k ) ≤ xˆ1 (k ) + Δx1 2 ) ≥ 4 ⋅ k ⋅ d2 Δx1 ⎪ (6) ⎨ 2 2 η ⋅ σ ⎪ d x x Δ Δ 2 ˆ ≤ x 2 (k ) ≤ xˆ 2 (k ) + 2 ) ≥ 4 ⋅ k ⋅ ⎪Pr ob ( x 2 (k ) − 2 2 Δx 22 ⎩
Note that inequalities describe in (6) represent the situation where occurs the largest chances (i.e. probability) of the inventory variables x1(k) and x2(k) belong to the spaces defined by their respective constraints. In other words, the chances of violation of the constraints given in (5) are minimal. The dark area in the figure 2 tries to illustrate these theoretical aspects.’
Var( x (k )) Pr ob. ( 1 ⋅ Δx ≤ x (k ) ≤ 3 ⋅ Δx ) ≥ 1 − 4 ⋅ k ⋅ 2 2 Δx 2
2.2 Physical constraints
The serviceable and remanufacturable inventory systems (1) and (2) are, respectively, constrained by lower and upper limits of storage. This means that these inventory systems can receive minimum and maximum volumes of products for each period k. In fact, in the serviceable store (1) is required a minimum volume of products (i.e. a safety stock) given by x1 , and is only allowed a maximum volume of products (i.e.,
limit of storage), given by x1 . Similarly, in the remanufacturable store (2), there are a safety stock, denoted by x 2 = 0 , and an upper bound of storage capacity, which is denoted by x 2 . It is important to point out that the violation of these limits of capacity lead to unnecessary increases on inventory costs.
Fig. 2. Maximum concentration of probability Based on the arguments above, the probabilistic indexes αk and βk can be defined by: ⎧ σ2 ⎪α k = 1 − 4 ⋅ k ⋅ d2 Δx 1 ⎪ ⎨ 2 2 ⎪ β = 1 − 4 ⋅ k ⋅ η ⋅ σd 2 ⎪ k Δx 2 ⎩
Due to the randomness of inventory variables, constraints should be considered in probability for the purpose of keeping them explicitly into the problem’s formulation, as will see ahead. Thus,
with k=0,1,…,T-1
Note that (7) is only valid if Δx1 ≥ 2 ⋅ σ d ⋅ T ⎧⎪Pr ob ( x1 ≤ x1 (k ) ≤ x1 ) ≥ α k ⎨ ⎪⎩Pr ob ( x 2 ≤ x 2 (k ) ≤ x 2 ) ≥ β k
(5)
where αk and βk are probabilistic indexes. Now, the objective is to choose proper indexes in such a way to guarantee noviolation of constraints (5). For such, let’s assume first that variables x1(k) and x2(k) have probability distributions that can be approximated by equivalent normal distribution with first and second statistical moments known a priori; note that such statistics are given, respectively, by (3) and (4). Based on these statistics and taking into account the Chebycheff's inequality (see Papoulis and Pillai, 2002), it follows, then, that the maximum value of probability density function for these variables will be occur, exactly, when the mean values of these inventory variables (i.e., xˆ1 (k ) and xˆ 2 (k ) ) reach the centre of their constraints, that is, xˆ1 (k ) ≈ ( x1 + x1 ) 2 and xˆ 2 (k ) ≈ ( x 2 + x 2 ) 2 ,
As a consequence, it is possible to show that (see Jazwinski (2007)):
(7)
and
Δx 2 ≥ 2 ⋅ η ⋅ σ d ⋅ T are verified. Fortunately, for the most practical applications, it is found that Δx1 >> σ d and Δx 2 >> η ⋅ σ d , which validates (7).
Since the terms 4 ⋅ σ d2 Δx12 and 4 ⋅ η2 ⋅ σ d2 Δx 22 can be computed a priori, it is possible to show that αk and βk tend to decrease proportionally with the evolution of the period k. This previous feature can be interpreted as follows: “the risk of infeasibility of constraints (5) decreases proportionally with the growing of the time-periods, and it reaches the minimal at the end of the planning horizon (k=T).” 2.3 The Criterion
For each period k, the function that describes the production costs of running systems (1)-(2) is given as follows: Ζk(x1,x2,u1,u2,u3) = h1⋅E{x1(k)2}+h2⋅E{x2(k)2}+c1⋅u1(k)2+ (8) c2⋅u2(k)2+c3⋅u3(k)2
1010
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
where h1 and h2 denote the inventory holding coefficients (prices) in the stores (1) and (2), respectively. Coefficients c1, c2 and c3 denote prices related respectively to manufacture, remanufacture and discard operations. The symbol E{.} denotes the expectation operator and it related to randomness of inventory variables x1 and x2. In addition, note that the total production cost is given by Z T =
∑
T k =1
zk .
Still with regard to (8), it is worth commenting that: (a) the use of quadratics functions to represent production costs has some advantages when compared with other functions. For instance, quadratic cost induces high penalties for large deviations of the decision variables from the origin but relatively small penalties for small deviation; see Bertesekas (2007); (b) the holding costs are not deterministic. In fact, because of the stochastic nature of the systems (1)-(2), the variable x1(k) and x2(k) are random variables, and so their real values can only be estimated in probability. This explains the reason of applying expectation operator to these variables, that is, E{x1(k)2} and E{x2(k)2}; and (c) the cost of collection of used-products is not deterministic. The reason of this is that the amount of product retuned r(k) is directly proportional to demand d(k) and so, it is a random variable. 2.4 Discrete-time stochastic optimal control problem
Based on the exposed above, a linear quadratic Gaussian (LQG) model with constraints can be formulated as follows:
{
} ∑ {E [h
Min Z T = E h1 ⋅ x1 (T) 2 + h 2 ⋅ x 2 (T ) 2 +
u1 ,u 2 ,u 3
]
T −1
k =0
1
⋅ x1 (k ) 2 +
}
+ h 2 ⋅ x 2 (k ) 2 + c1 ⋅ u1 (k ) 2 + c 2 ⋅ u 2 (k ) 2 + c 3 ⋅ u 3 (k ) 2 subject to x1 (k + 1) = x1 (k ) + u1 (k ) + u 2 (k ) − d (k ) (9) x 2 (k + 1) = x 2 (k ) − u 2 (k ) − u 3 (k ) + r (k ) Pr ob. ( x1 ≤ x1 (k ) ≤ x1 ) ≥ α k Pr ob. ( x 2 ≤ x 2 (k ) ≤ x 2 ) ≥ β k 0 ≤ u1 ( k ) ≥ u1 ; 0 ≤ u 2 ( k ) ≥ u 2 ; u 3 ( k ) ≥ 0
The probability index α is often denoted as the level of customer satisfaction; and here, it is used to measure the chances of satisfying customers with the combination of manufactured and remanufactured products. At last, note that the problem (9) provides a sequential optimal production policy {u1(k), u2(k), and u3(k); with k = 0,1,2, …, T-1} that optimizes the performance of (1)-(2). 3. SOLVING THE PROBLEM (9) BY OLFC APPROACH Due to constraints and dimensionality associated to stochastic problem (9), computing a true optimal production policy for (9) (i.e. a closed-loop solution) is not a trivial task. However, from the literature, it is possible to find several sub-optimal approaches to solve such a problem. Some of these numerical techniques can provide revised optimal production policies that are very close to the true optimal solution (Bertesekas, 1995); such as the case of the Open-Loop Feedback Controller (OLFC), which will be discussed in sequel.
The OLFC is an approach that allows updating the optimal production policy continually, taking into account measures of current states of the inventory systems (1)-(2). Note that such a feedback characteristic of the OLFC makes it an adaptive procedure. This means that it provides a better solution than that provided by an Open-Loop Controller (OLC), which does not include any new updates in the solution, except the one taken in the first period of planning horizon. It is worth mentioning that an open-loop policy is a static solution, which is based entirely on the mean values of the decision variables of the problem (9). It is worth bearing in mind that the problem (9) is a stochastic optimal control problem under perfect state information (Bertesekas, 2007). So, then, this means that inventory levels x1(k) and x2(k) can be measured exactly in the beginning of each period k. As a consequence, the OLFC approach can be applied directly to solve problem (9), without requiring any mechanism for estimating the states. Basically, the steps to apply such a procedure are: (Step 1) In the beginning of each new period k=t (with t=0, 1, …, T-1), the exact position of the serviceable and remanufacturable inventory levels (see figure 1) should be measured, that is, x1(t) and x2(t) are perfectly taken from systems (1)-(2); and (Step 2) From these measures, an optimal production policy, given by {u1(k), u2(k), and u3(k); with k=t, t+1, …, N-1} is computed by solving the following equivalent problem: Min Z t = h1 ⋅ xˆ1 (T ) 2 + h 2 ⋅ xˆ 2 (T ) 2 +
∑ {h T −1 k =t
1
⋅ xˆ1 (k ) 2 +
}
h 2 ⋅ xˆ 2 (k ) 2 + c1 ⋅ u1 (k ) 2 + c 2 ⋅ u 2 (k ) 2 + c 3 ⋅ u 3 (k ) 2 + Κ t subject to (10) xˆ1 (k + 1) = xˆ1 (k ) + u1 (k ) + u 2 (k ) − dˆ (k ); xˆ1 (k ) = x1 ( t ) xˆ 2 (k + 1) = xˆ 2 (k ) − u 2 (k ) − u 3 (k ) + rˆ (k ); xˆ 2 (k ) = x 2 ( t ) x1,α (k ) ≤ xˆ1 (k ) ≤ x1,α (k ) x 2,β (k ) ≤ xˆ 2 (k ) ≤ x 2,β (k ) 0 ≤ u1 ( k ) ≥ u 1 ; 0 ≤ u 2 ( k ) ≥ u 2 ; u 3 ( k ) ≥ 0 k = t , t + 1,L, N − 1
where the integration constant of the criterion Zt, denoted by Kt, depends on the variance of the demand and the initial period t. It is given by: K t = (h1 + η ⋅ h 2 ) ⋅ σ d2 ⋅ (T − t )
(11)
Note also that lower and upper boundaries of serviceable and remanufacturable inventory constraints (5) are now given by x1, α (k ) = x1 − k ⋅ σ d ⋅ Φ −1 (α k ) x1, α (k ) = x1 + k ⋅ σ d ⋅ Φ −1 (α k )
x 2,β (k ) = x 2 − k ⋅ η ⋅ σ d ⋅ Φ −1 (β k )
(12)
x 2,β (k ) = x 2 + k ⋅ η ⋅ σ d ⋅ Φ −1 (β k )
Besides, given the initial inventory levels of inventory variable (i.e., x1(t) and x2(t)), the problem (10) is solved from
1011
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
the period t to T. This solution does not take into account any information about the current level of the inventory systems (1)-(2). Thus, it is an open-loop solution. (Step 3) Since new measures of inventory levels x1(k) and x2(k) are not allowed from systems (1)-(2) for k > t, the OLFC approach selects only the first elements of the optimal production policy (i.e., {u1* (t), u *2 (t), and u *3 (t)} ) to apply as input into systems (1)-(2). The others components of the optimal sequence (i.e., {u1* (k), u *2 (k), and u *3 (k); k=t+1, t+2, …, T}) are completely ignored. Thus, as soon as new measures of inventories variables are taken, in the beginning of new period t+1, the OLFC procedure starts again from the step 2 and problem (10) is reformulated in order to be solved for periods k=t+1, ..., T. 4. EXAMPLE Let's consider a simple example of a company that manufactures a certain kind of product to meet demand in the marketplace. After use, the product is collected from the market. Then, quality tests are carried out in order to decide whether the product should be re-used or discarded. The company uses two units of stores in forward and reverse channels of its supply chain, as illustrated in the figure 1. Besides, it is assumed that: (i) the demand for this product is stationary, which means that the fluctuation of sales can be estimated with a good accuracy over time periods of the planning horizon and (ii) the unit cost of remanufacturing a returned product is lower than the cost of manufacturing a new product, which means that used-products can be easily overhauled and replaced to market.
and β1=0,04; and when k=6 implies that α6=0,4 and β6=0,25. It is worth noting that indexes αk and βk decrease proportionally with the evolution of period k. This means that for time-periods close to the end of planning horizon, the risk of violation of the constraints (5) is lower. 4.1 Optimal policies via OLFC and OLC
In this section, two production scenarios for systems (1)-(2) are discussed: the first scenario shows a suboptimal solution generated from the application of the OLC approach in the problem (10); whereas the second shows the suboptimal policy provided from the application of the OLFC approach to the same problem. The basic difference between the two scenarios is in the use of available information. In fact, in the first scenario, only initial serviceable and remanufacturable inventory levels are available to solve problem (10) and so, the solution is said open-loop. On the other hands, in the second scenario, the solution is continually updated. 4.1(a) Scenario: Open-loop Controller (OLC) Figures 3 and 4 illustrate inventory and production trajectories for forward and reverse channels. The striking feature of this scenario is the high production of remanufacturable products. Practically, all production of the company is being met by this process. In this way, it is interesting to note that: as initial inventory levels of serviceable and remanufacturable stores are high, the demand for products, in early initial periods of planning horizon, tends to be met completely by products available into the serviceable inventory store and, simultaneously, by those that are being currently manufactured; see solid lines in figures 3 and 4, during periods k = 0.
Aiming to develop a production policy for the next six months (T=6), the company decides to implement an optimal strategy based on the production planning problem given in (10). The demand fluctuation is considered stationary and normally distributed over the periods of planning horizon. Its first and second statistics moments are given in the Table 1. Table 1: Statistics on the demand k Jan Feb Mar Apr May Jun d(k) 604 597 610 614 609 580 Standard-deviation σd ≈ 50
Inventory levels
350 300 250 200 150 100 50 0
Serviceable Remanufacturable
0
500
boundaries of manufacturing and remanufacturing facilities are u1 = u 2 = 0 , u1 = 450 and u 2 = 400 ; (iv) inventory,
300
1012
2
3
4
5
6 months
Fig.3. Inventory level of manufactured
Other information: (i) initial inventories for serviceable and remanufacturable products are, respectively: x01 = 300 and x02 = 200; (ii) lower and upper boundaries of serviceable and remanufacturable inventories are given respectively by x1 = x 2 = 25 and x1 = x 2 = 300 ; (iii) lower and upper
production and disposal costs are respectively: h1=2, h2=1, c1=2, c2=1 and c3=1; (v) the rate of return is 80% (i.e., η=0.8). Note that the use of this high rate of return means a greater possibility of reuse of returnable products; and (vi) probabilistic indexes defined in (7) have their values taken from the following ranges αk ∈[0,06 0,4] and βk ∈[0,04 0,25]. Thus, for instance, when k = 1 implies that α1=0,06
1
Production and disposal rates
400
Manufacture Remanufature Disposal
200 100 0 0
1
2
3
4
5 months
Fig.4. Optimal production and disposal rates
13th IFAC INCOM (INCOM'09) Moscow, Russia, June 3-5, 2009
The following periods show that the process of remanufacture becomes intense and its maximum capacity of production is already reached during period k=1 (see Figure 4, dashed line). Thus, it can be said that the mean demand is being met almost completely by remanufactured products (note – from figure 4, solid line – that the production rate of manufactured products are quite negligible). In addition, many used products are being discarded in order to maintain low inventory levels, which can be seen as competitive strategy of the company to reduce cost with inventories. 4.1(b) Scenario: Open-loop Feedback Controller (OLFC) The optimal inventory and production trajectories for forward and reverse channels are respectively exhibited in figures 5 and 6. The trajectories depicted in this scenario are similar to that exhibited to the OLC procedure (see Figures 3 and 4). The reason of this is that the fluctuation of demand is based entirely on the average levels given in Table 1. As a result, the optimal solution with updating (OLFC) and no-updating (OLC) become very close. However, it is possible to observe small differences. One of them is the levels of serviceable and remanufacturable inventories of the OLC policy that tend to increase slightly during end-periods of the planning horizon. This means that such a policy is more conservative (i.e. averse to risk) than OLFC policy, thus it increases safety-stocks close to the end-periods of planning horizon in order to guarantee customer satisfaction (see periods 5 and 6 in the figure 4). The difference between the two approaches can be easily seen from a cost analysis, as shown in Table 2. Note that the operation of the systems (1) and (2), under application of OLFC policy, reduces the total cost in 21% in relation to OLC policy. Inventory levels
350 300 250 200 150 100 50 0
Serviceable Remanufacturable
Table 2. Costs ($) of the two scenarios Costs Serviceable inventory cost Remanufacturable inventory cost Manufactured cost Remanufactured cost Disposal cost Total cost
OLFC 7.597 6.288 328.110 851.660 202.190 1.395.845
OLC 196.520 43.644 460.760 851.668 215.308 1.767.900
5. CONCLUSION This paper analyzed a particular type of reverse logistics problem. The problem is defined by two productioninventory systems, which represent the forward and reverse channels of a supply chain. A linear quadratic Gaussian (LQG) model with constraints was formulated to provide optimal policies for manufacturing, remanufacturing and disposal variables. The model proposed considers chanceconstraints on inventory variables that are computed in such a way to get the best benefits of the physical intervals (see section 2.2). The paper discussed also about a sub-optimal approach (denoted as OLFC) in order to apply to LQG problem. It was shown that OLFC procedure can be used in a simulation scheme to provide production scenarios. Through these scenarios, it is possible to compare different production plans, helping managers to make decision about the use of available resources. The OLFC approach was applied to a simple example and it presented better solution than the classical OLC approach. The reason is that OLFC policy can be updated over the periods, whenever new information about the inventories is available. This means that a higher rate of return will lead to lower production costs and, as a consequence, greater profitability. Finally, it is worth mentioning that such profitability will occur only if the costs for remanufacturing are lower than those for manufacturing new products. REFERENCES
0
1
2
3
4
5
6 months
Fig. 5. Serviceable and remanufacturable inventory levels Production and disposal rates
500 400
Manufacture Remanufature Disposal
300 200 100 0 0
1
2
3
4
5 months
Fig. 6. Optimal production and disposal rates
Bertesekas, D. P. (2007). Dynamic Programming and Optimal Control, Athena Scientific, Vol. 1. USA. Dobos, Imre (2003) Optimal Production-inventory strategies for HMMS-type reverse logistics system, Int. J. Production Economics, 81-82, 351-360. Fleischman, M., Bloemhof-Ruwaard, J. M., Dekker, R., Van der Laan, E., Van Nunen, Jo A. E. E., and Van Wassenhove, L. N. (1997) Quantitative models for reverse logistics: A review, European Journal of Operational Research, 103, 1-17. Graves, S. C. (1999). A Single-Item Inventory Model for a Non-stationary Demand Process, Manufacturing & Service Operations Management, Vol. 1, No 1. Jazwinski, A. H. (2007) Stochastic Process and Filtering Theory, Dover Publications. Papoulis A. and Pillai S. U. (2002) Probability, Random Variables, and Stochastic Process, McGraw-Hill Publishing Co.
1013