On Exploitation of Supply Chain Properties by Sequential Distributed MPC*

Proceedings of the 20th World The International Federation of Congress Automatic Control Proceedings of the 20th World Congress The International Federation of Congress Automatic Control Proceedings of the 20th9-14, World Toulouse, France, July 2017 The International Federation of Automatic Control Available online at www.sciencedirect.com Toulouse, France,Federation July 9-14, 2017 The International of Automatic Control Toulouse, France, July 9-14, 2017 Toulouse, France, July 9-14, 2017

ScienceDirect

On On On On

IFAC PapersOnLine 50-1 (2017) 7947–7952 Exploitation of Supply Chain Properties Exploitation of Supply Chain Properties Exploitation of Supply Chain Properties  by Sequential Distributed MPC Exploitation of Supply Chain Properties by Sequential Distributed MPC by Sequential Distributed MPC  by Sequential Distributed MPC Philipp N. Köhler ∗ Matthias A. Müller ∗ Jürgen Pannek ∗∗,∗∗∗

Philipp Philipp Philipp

∗ ∗ ∗∗,∗∗∗ N. A. ∗ ∗ ∗ Jürgen ∗∗,∗∗∗ N. Köhler Köhler ∗∗ Matthias Matthias A. Müller Müller Jürgen Pannek Pannek ∗∗,∗∗∗ Frank Allgöwer ∗ Frank Allgöwer ∗ ∗ Jürgen Pannek ∗∗,∗∗∗ ∗ N. Köhler ∗ Matthias A. Müller Frank Allgöwer Frank Allgöwer ∗ ∗ ∗ Institute for Systems Theory and Automatic Control, Systems Theory and Automatic Control, ∗ ∗ Institute for University Stuttgart, Institute for SystemsofTheory and Germany Automatic Control, ∗ University Stuttgart, Institute for Systemsof and Germany Automatic Control, University ofTheory Stuttgart, Germany (e-mail: {koehler,mueller,allgower}@ist.uni-stuttgart.de). ∗∗ (e-mail: {koehler,mueller,allgower}@ist.uni-stuttgart.de). University of Stuttgart, Germany Department of Production Engineering, University of Bremen, (e-mail: {koehler,mueller,allgower}@ist.uni-stuttgart.de). ∗∗ of Production Engineering, University of Bremen, ∗∗ Department ∗∗ (e-mail: {koehler,mueller,allgower}@ist.uni-stuttgart.de). Germany (e-mail: [email protected]). Department of Production Engineering, University of Bremen, ∗∗∗∗∗ Germany (e-mail: [email protected]). Department of Production Engineering, of GmbH, Bremen, Bremer Institut Produktion University und Logistik (e-mail:für [email protected]). ∗∗∗ BIBAGermany Bremer für Produktion ∗∗∗ ∗∗∗ BIBA (e-mail:Germany [email protected]). BIBAGermany Bremer Institut Institut für Produktion und und Logistik Logistik GmbH, GmbH, ∗∗∗ BIBA Bremer Institut Germany für Produktion und Logistik GmbH, Germany Germany Abstract: In this work, we propose a sequential distributed MPC algorithm for control of a Abstract: In this work, we propose a sequential distributed MPC algorithm for control of a linear supply The we algorithm closely taking account Abstract: In chain. this work, proposeisa developed sequential by distributed MPC into algorithm forsupply controlchain of a linear supply chain. The we algorithm isa developed by closely taking into account chain Abstract: this work, propose sequential distributed MPCare algorithm forsupply control of a specifics andInrequirements from practice, e.g., orders and leavings both treated as decision linear supply chain. The algorithm is developed by closely taking into account supply chain specifics and requirements from practice, e.g., orders and leavings are both treated as decision linear supply chain. The algorithm is developed by closely taking into account supply chain specifics and requirements from practice, e.g., orders and leavings are both treated as decision variables at each stage and communication between stages is kept low. We present the rather variables at each stage andfrom communication between stages is keptare low.both We treated present as the rather specifics requirements practice, e.g., orders and leavings surprising that terminal equality constraints employed the local MPC formulations are variables and atresult each stage and communication between stages isinkept low. We present thedecision rather surprising result that terminal equality constraints employed inkept the local MPC formulations are variables at each stage and communication between stages is low. We present the rather inherently satisfied forterminal the overall system, despite employed the presence of bilateral dynamic couplings surprising result that equality constraints in the local MPC formulations are inherently satisfied forterminal the overall system, despite employed the presence of bilateral dynamic couplings surprising result that equality constraints in the local MPC formulations are and solvingsatisfied the localfor MPC sequentially. due to the stock anddynamic flow nature of the inherently the problems overall system, despiteThis the ispresence of bilateral couplings and the local MPC problems sequentially. This due to stock and anddynamic flow nature nature of the the inherently satisfied the overallissystem, despite the is of bilateral couplings and solving solving theproposed localfor MPC problems sequentially. This ispresence duefeasible, to the the stock flow of problem. The algorithm shown to be recursively to asymptotically satisfy a problem. The proposed algorithm is shown to be recursively feasible, to asymptotically satisfy and solving theproposed local MPC problems sequentially. Thisconvergence is duefeasible, to theofstock and stock flow nature of theaa constant customer demand and to is achieve asymptotic the local andsatisfy backlog problem. The algorithm shown to be recursively to asymptotically constant customer demand and to is achieve asymptotic convergence of the local stock andsatisfy backlog problem. The proposed algorithm shown to be recursively feasible, to asymptotically to the desired levels. This isand illustrated byasymptotic numerical simulations. constant customer demand to achieve convergence of the local stock and backloga to the desired levels. This isand illustrated byasymptotic numerical convergence simulations. of the local stock and backlog constant customer demand to achieve to the desired levels. This is illustrated by numerical simulations. © 2017, IFAC (International of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. to the desired levels. This Federation is illustrated by numerical simulations. Keywords: Decentralized and distributed control, Complex logistic systems, Modeling and Keywords: Decentralized and distributed control, Complex Keywords: Decentralized andsystems, distributed control, systems. Complex logistic logistic systems, systems, Modeling Modeling and and decision making in complex Multiagent decision making in Multiagent Keywords: Decentralized andsystems, distributed control, systems. Complex logistic systems, Modeling and decision making in complex complex systems, Multiagent systems. decision making in complex systems, Multiagent systems. 1. INTRODUCTION and Maestre (2014) for an overview. However, only few 1. INTRODUCTION INTRODUCTION and Maestre (2014) for an overview. However, only few 1. of them suit the system setup of supply chains,only namely and Maestre (2014) for an overview. However, few of them suit the system setup of supply chains,only namely 1. INTRODUCTION Maestre (2014) for an overview. However, few systems dynamically coupled inofthe inputs. Dunbar and of them suit the system setup supply chains, namely Supply chains are networks of companies interconnected and systems dynamically coupled in the inputs. Dunbar and Supply chains are networks of companies interconnected of them suit the system setup of supply chains, namely systems dynamically coupled in the inputs. Dunbar and Supply are networks of companies interconnected (2007) proposed a DMPC scheme for supply chain by orderchains and delivery flows that are involved in satisfying Desa (2007) proposedcoupled a DMPC for supply chain by orderchains and delivery delivery flows that that are involved involved in satisfying satisfying Desa dynamically in scheme theasinputs. Dunbar and are networks of companies interconnected by order and flows are in systems considering only the orders a decision variable Desa (2007) proposed a DMPC scheme for supply chain aSupply customer demand (Chopra and Meindl, 2012). Transport systems systems considering only the orders as a decision variable a customer demand (Chopra and Meindl, 2012). Transport Desa (2007) proposed a DMPC scheme for supply chain bycustomer order delivery flowsand that are involved in Transport satisfying adelays demand (Chopra anddecision Meindl, 2012). and thereby replacing some dynamic fixed considering only the orders as couplings a decision by variable in and the material flow policies subject to systems and thereby replacing some dynamic couplings by fixed delays in the the material flow and and decision policies subject to to systems considering only the as systems a decision variable a customer demand (Chopra anddecision Meindl, 2012). Transport delays in material flow policies subject relations. DMPC schemes fororders general subject to and thereby replacing some dynamic couplings by fixed limited information usually cause oscillation, amplification relations. DMPC schemes for general systems subject to limited information usually cause oscillation, amplification and thereby replacing some dynamic couplings by fixed delays in the material flow and decision policies subject to relations. DMPC schemes for general systems subject to limited information usually amplification couplings were proposed, e.g., by Stewart et al. and phase lag in the flowscause of a oscillation, supply chain (Sterman, dynamic dynamic couplings were proposed, e.g., by Stewart et al. and phase lag in the flows of a supply chain (Sterman, relations. DMPC schemes for general systems subject to limited information usually cause oscillation, amplification and phase lag in the flows of a supply chain (Sterman, (2010); Maestre et al. (2011a,b), where the latter two also dynamic couplings were proposed, e.g., by Stewart et al. 2000). For example, a variation in customer demand is am- (2010); Maestre et al. (2011a,b), where the latter two also 2000). For example, example, variation in customer demand is amam- dynamic couplings were proposed, by latter Stewart al. and phase the flows a customer supply chain (Sterman, 2000). For aa variation in demand is investigated a supply chain as where an e.g., application example, Maestre et al. (2011a,b), the twoetalso plified alonglag theinsupply chainofwhat is commonly known as (2010); a supply chain as where an application plified along the supply supply chain chain what what is commonly commonly known as investigated (2010); et exploiting al. (2011a,b), the specific latterexample, two also 2000). For example, in customer demand is amplified along the is known as however,Maestre without supply investigated a supply chain or as analyzing an application example, the bullwhip effect. aTovariation cope with these difficulties, model however, without exploiting or analyzing specific supply the bullwhip effect. To cope with these difficulties, model investigated a supply chain as an application example, plified along the supply chain what is commonly known as however, without exploiting or analyzing specific supply the bullwhip effect. To cope with these difficulties, model characteristics and using a simplified model. predictive control (MPC) was applied to decision making chain characteristics and using simplifiedspecific model. predictive control (MPC) waswith applied todifficulties, decision making making however, without exploiting or aaanalyzing thesupply bullwhip effect.(MPC) To for cope these model chain predictive control was applied to decision chain characteristics and using simplified model. supply in chains, see example Braun et al. (2003); The work at hand serves as a first step towards our in supply chains, see for example Braun et al. (2003); chain characteristics and using a simplified model. predictive (MPC) was applied to in supply chains, (2008); see forSubramanian example Braun al. making (2003); work at serves first step towards our Wang and control Rivera et decision al.et(2013). Since The The work goal at hand hand serves as as aahow first step towards our long term of investigating the use of predictive Wang and Rivera (2008); Subramanian et al. (2013). Since in supply chains, see for example Braun et al. (2003); Wang Rivera (2008); Subramanian et al. (2013). Since term goal of investigating how the use of predictive MPC and ensures satisfaction of certain constraints and in- long The work goal atcanhand serves as ahow first step towards our information improve (overall and individual) system long term of investigating the use of predictive MPC ensures satisfaction of certain constraints and inWang and Rivera (2008); Subramanian et (2013). MPC ensures satisfaction of certain in- long information canofimprove (overall andtheindividual) system corporates some performance criterionconstraints in al. terms ofand aSince cost term goal investigating how use of predictive performance in supply chains by means of DMPC. To this information can improve (overall and individual) system corporates some performance criterion in terms of a cost MPC ensures satisfaction of certain in- performance in supply chains by means of DMPC. To this corporates some performance criterion ofand a cost function (Grüne and Pannek, 2011), constraints itin isterms an appealing information can improve (overall and individual) system in supply chains by means ofalgorithm DMPC. To this end, we propose and analyze a DMPC closely function (Grüne and Pannek, Pannek,criterion 2011), it itin is isterms an appealing appealing corporates some performance of a cost performance function (Grüne and 2011), an end, we propose and analyze aa DMPC algorithm closely control technique for such complex problems. However, performance in supply chains by means of DMPC. To this end, we propose and analyze DMPC algorithm closely tailored to supply chain system specifics in order to gain control technique for Pannek, such complex complex problems. However, function (Grüne and 2011), it to is an appealing control technique for such problems. However, tailored to supplyand chain system specificsalgorithm in order to gain this requires all information being sent a centralized end, weinto propose analyze a DMPC closely insight the effects and properties of a DMPC approach tailored to supply chain system specifics in order to gain this requires all information being sent to a centralized control technique for such problems. However, this requires all information being sent to centralized insight into the effects andsystem properties of a DMPC approach decision instance, which is complex not suitable in apractice due tailored tochain supply chain in order to work gain to supply management. Inspecifics particular, in this into the effects and properties of a DMPC approach decision instance, which is is not not suitable in apractice practice due insight thissecrecy requires information being to centralized decision instance, which in due to supply chain management. In particular, in this work to andallindividualism of suitable the sent separate companies. insight into the effects and properties of a DMPC approach to supply chain management. In particular, in this work consider a realistic supply chain model, in which both to secrecyinstance, and individualism individualism of suitable the separate separate companies. companies. decision is not due we to secrecy and of the we consider aa realistic supply chain model, in both Additionally, such which a centralized approachinispractice error-prone to chain management. In particular, this work we supply consider realistic supply and chain model, are ininwhich which both flow variables, namely orders leavings, treated as Additionally, such a centralized approach is error-prone to secrecy and individualism of the separate companies. Additionally, such a of centralized approach is error-prone flowconsider variables, namely orders and leavings, are treated as and limits scalability the system. To overcome these is- we a realistic supply chain model, in which both decision variables at each stage of the supply chain. In flow variables, namely orders and leavings, are treated as and limits scalability of the system. To overcome these isAdditionally, such a of centralized isschemes error-prone and limits scalability the system. To overcome these is- flow decision variables at each stage ofleavings, the supply chain. In sues, a number of distributed MPCapproach (DMPC) have variables, namely orders and are treated as order to reduce communication effort and to represent the decision variables at each stage of the supply chain. In sues, a number of distributed MPC (DMPC) schemes have and limits scalability of the system. To (2013); overcome these is- order to reduce communication effort and to represent the sues, aproposed, number ofsee distributed MPC schemes have been Christofides et (DMPC) al. Negenborn decision variables at each stage of the supply chain. In order to reduce communication effort and to represent the operating regime in supply chains, we propose a sebeen proposed, see Christofides et al. Negenborn sues, aproposed, number ofsee distributed MPC schemes have typical been Christofides et (DMPC) al. (2013); (2013); Negenborn typical regime in supply chains, we propose a the seorder tooperating reduce effort and to(customer) represent quential DMPCcommunication algorithm thewe typical operating regime in propagating supply chains, propose a dese been proposed, see Christofides et al. (2013); Negenborn P. Köhler, M. Müller and F. Allgöwer thank the German Research quentialoperating DMPC algorithm propagating thewe (customer) de P. Köhler, M. Müller and F. Allgöwer thank the German Research typical regime in supply chains, propose a semand along the supply chain. The separate MPC problems quential DMPC algorithm propagating the (customer) de Foundation for support of this work within AL Research 316/11-1 P. Köhler,(DFG) M. Müller and F. Allgöwer thank thegrant German mand along the supply chain. The separate MPC problems Foundation (DFG) for support of this work within grant AL 316/11-1 quential DMPC algorithm propagating the (customer) de are formulated the framework terminal equality conmand along the in supply chain. Theofseparate MPC problems P.within Köhler,the M.Cluster Müller and F. Allgöwer thank thegrant German and of Excellence Simulation Technology (EXC Foundation (DFG) for support of thisinwork within AL Research 316/11-1 are formulated in the framework of terminal equality conand within the Cluster of Excellence Simulation Technology (EXC mand along the We supply chain. MPC problems are formulated in the framework ofseparate terminal equality conFoundation (DFG) for support of thisin within and grant 316/11-1 strained MPC. present theThe surprising result that these 310/2) at the University Stuttgart. M. Müller J.AL Pannek are and within the Cluster of of Excellence inwork Simulation Technology (EXC strained MPC. We present the surprising result that these 310/2) at the University of Stuttgart. M. Müller and J. Pannek are are formulated in the framework of terminal equality conand theUniversity Cluster of of Excellence in M. Simulation Technology (EXC also within supported by the DFG, grant WO 2056/1. strained MPC. We present the surprising result that these 310/2) at the Stuttgart. Müller and J. Pannek are also supported by the DFG, grant WOM. 2056/1. strained MPC. We present the surprising result that these 310/2) at the University of Stuttgart. Müller and J. Pannek are also supported by the DFG, grant WO 2056/1.

also supported by the DFG, grant WO 2056/1. Copyright © 2017 IFAC 8223 Copyright 2017 IFAC 8223Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © 2017, IFAC (International Federation of Automatic Control) Copyright © 2017 IFAC 8223Control. Peer review©under of International Federation of Automatic Copyright 2017 responsibility IFAC 8223 10.1016/j.ifacol.2017.08.706

Proceedings of the 20th IFAC World Congress 7948 Philipp N. Köhler et al. / IFAC PapersOnLine 50-1 (2017) 7947–7952 Toulouse, France, July 9-14, 2017

ai−1 oi−1

stage i−1

li−1 di−1

τ =

ai

li stage i

oi

di

τ =

ai+1 oi+1

stage i+1

li+1 di+1

Fig. 1. Schematic of a linear supply chain system. terminal constraints are not restrictive, but are rather inherently recursively satisfied for the overall supply chain without being explicitly respected for neighboring stages due to the stock and flow nature of the system. Besides, we propose a two-step MPC update procedure to cope with the bidirectional dynamic coupling of neighboring stages in the particular supply chain structure. This paper is organized as follows. In Section 2, a detailed system description is given. Section 3 introduces and investigates the considered MPC problems. Subsequently, in Section 4, we state and analyze the distributed MPC algorithm. In Section 5, numerical simulations are shown before some concluding remarks are given in Section 6. Notation. Let I≥a denote the set of all integers larger or equal than a ∈ R and I[a,b] the set of all integers in the interval [a, b] ⊂ R. A continuous function α : [0, ∞) → [0, ∞) belongs to class K∞ if it is strictly increasing, α(0) = 0 and a(r) → ∞ as r → ∞. 2. PROBLEM DESCRIPTION In this work, we consider the discrete-time dynamics of a single echelon supply chain consisting of P ∈ I≥1 stages. Each stage of the supply chain has two state variables accumulating material as inventory stock si (t) ∈ R and unfulfilled received orders as backlog bi (t) ∈ R. At every time instant t ∈ I≥t0 , each stage submits an order oi (t) to its upstream neighbor i−1 and delivers a certain amount of material li (t) (leavings) to its downstream neighbor i + 1. In turn, it acquires a certain amount of material ai (t) (acquisition) from its upstream neighbor i−1 equalling this neighbor’s delayed leavings, i.e., ai (t) = li−1 (t−τ ) with the constant transport delay τ ∈ I≥0 . The demand di (t) from stage i is equal to the orders placed by its downstream neighbor i + 1 with no delay, i.e., di (t) = oi+1 (t). A schematic of the interconnection of stages in a supply chain is shown in Figure 1. This model is standard in the literature and was considered, e.g., in Dunbar and Desa (2007). In this work, we consider the discrete-time dynamics of each stage i ∈ I[1,P ] , i.e., si (t + 1) = si (t) + ai (t) − li (t) (1a) = si (t) + li−1 (t − τ ) − li (t) bi (t + 1) = bi (t) + di (t) − li (t) (1b) = bi (t) + oi+1 (t) − li (t) subject to the obvious positivity constraints si (t) ≥ 0, bi (t) ≥ 0, oi (t) ≥ 0, li (t) ≥ 0 for all t ∈ I≥t0 and i ∈ I[1,P ] . For every stage i ∈ I[1,P ] , leavings li (t) and orders oi (t) are decision variables, whereas the arrival ai (t) and demand di (t) are input variables obtained from the neighboring stages and thus can be considered as measurable disturbances. Order satisfaction at the supplier stage i = 1 at the upstream end of the supply chain is modeled as a pure time delay, i.e., a1 (t) = o1 (t − τ ). The demand at the retailer stage i = P at the downstream end of the supply chain is given as customer orders dP (t).

Later, we will analyze the proposed algorithm for the case of constant customer orders, i.e., dP (t) = d ∈ R+ 0 for all t ∈ I≥t0 . Note that in this case arbitrary stock and backlog levels sdi and bdi are steady states for the overall system with all order and material flows equalling the customer demand as according steady-state input, i.e., li (t) = oi (t) = d¯ for all i ∈ I[1,P ] and all t ∈ I≥t0 . In the literature, leavings li (t) are usually not considered as a decision variable, but are determined by a fixed relation, cf., Dunbar and Desa (2007). The latter allows to substitute some of the coupling variables in the dynamics of the separate stages. However, this substitution does not take the positivity constraints into account, which is why such a substituted model is not valid for analytical investigations. For this reason, and also with respect to practical requirements, in this work we treat the leavings li (t) as a decision variable and, hence, allow each stage not only to decide upon the orders oi (t) to be placed upstream, but also upon the material li (t) to be shipped downstream. The goal considered in the following is to asymptotically satisfy a constant customer demand dP (t) and to have all stock and backlog levels converge to the desired values, i.e., lP (t) → dP (t), si (t) → sdi and bi (t) → 0 for t → ∞ and for all i ∈ I[1,P ] . The decision variables should be determined by each stage locally based on neighboring information only. Due to the chain structure of the problem, we propose a sequential DMPC algorithm in this work. 3. DISTRIBUTED MPC FORMULATION In this work, the decision variables, i.e., the leavings li (t) and orders oi (t), are calculated in a distributed MPC fashion at each stage of the supply chain, i.e., at each time instant t, an optimization problem is solved at each stage to obtain values for the leavings and orders to be implemented. We employ a terminal equality constraint MPC formulation to simplify our analysis and to gain a thorough understanding for the essential effects resulting from the interaction between the stages in a supply chain. Besides, this also has a reasonable interpretation from a supply chain management point of view: A company might always plan their future trade such that at the end of the planning horizon all demand is satisfied and the stock level is as desired. Such terminal (equality) constraints can be interpreted as an assurance which companies in a supply chain give each other. These guarantees in terms of terminal equality constraints result in some interesting properties of the overall supply chain and can be exploited in a distributed MPC approach. We will study these properties in detail after stating the optimization problems solved in the MPC update procedure. The MPC update procedure at each stage is performed in two steps, i.e., two optimization problems are solved by each stage at each time instant, which will be discussed in Section 3.2. 3.1 MPC optimization problems In this section, we present the two optimization problems that are employed in the two-step MPC update procedure at each stage. The following optimization problem determines an optimal leaving trajectory li (t), whereas the auxiliary optimization problem given below determines an optimal order trajectory oi (t).

8224

Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 Philipp N. Köhler et al. / IFAC PapersOnLine 50-1 (2017) 7947–7952

Problem 1. min Ji (si (t), bi (t), li (t), ai (t), di (t)) li (t)

subject to si (0|t) = si (t) bi (0|t) = bi (t) si (k + 1|t) = si (k|t) + ai (k|t) − li (k|t) bi (k + 1|t) = bi (k|t) + di (k|t) − li (k|t) k = 0, . . . , N − 1 si (N |t) = sdi bi (N |t) = 0 si (k|t) ≥ 0, li (k|t) ≥ 0, ¯ li (k|t) = d,

K 

k=0

li (k|t) ≥

K 

k=0

bi (k|t) ≥ 0, k = 0, . . . , N k = 0, . . . , N − 1 k = N − τ, . . . , N − 1 ˆli (k|t), K = 0, . . . , N − 1

(2) (3a) (3b) (3c) (3d) (3e) (3f) (3g) (3h) (3i) (3j)

N −1 with Ji (si (t), bi (t), li (t), ai (t), di (t)) = k=0 Li (si (k|t), bi (k|t)) where N ∈ I≥τ denotes the prediction horizon. In this problem, si (t) := [si (0|t), . . . , si (N |t)] , bi (t) := [bi (0|t), . . . , bi (N |t)] and li (t) := [li (0|t), . . . , li (N −1|t)] are the predicted state and control trajectories and Li : R2 → R denotes the stage cost. Assumption 2. The stage cost function Li is continuous, it satisfies Li (sdi , 0) = 0 and there exists a function α1 ∈ K∞ such that Li (si , bi ) ≥ α1 (||[si − sdi , bi ] ||2 ) holds for all si ≥ 0 and bi ≥ 0. According to the system setup, the predicted arrival and demand trajectories, ai (t) and di (t), are set based on the predicted leaving and order trajectories of the neighboring stages and, due to the transport delay τ , also on the past leavings li−1 (t − τ ), . . . , li−1 (t − 1) to di (t) := oi+1 (t) (4a) ai (t) := [li−1 (t − τ ), . . . , li−1 (t − 1),

li−1 (0|t), . . . , li−1 (N − 1 − τ |t)] (4b)  ¯ ¯ with oP +1 (t) = [d, . . . , d] and l0 (t) := o1 (t). We will specify later, how oi+1 (t) and li−1 (t) are exactly defined, i.e., which input sequences stage i assumes for its neighbors. Likewise, for notational simplicity, we write d0i (t), ˆ i (t) for given o0 (t), ˆ oi+1 (t), and a0i (t), ˆ ai (t) for given d i+1 0 ˆ li−1 (t), li−1 (t), respectively. By ˆli (t) in (3j) we denote a candidate input trajectory. It results from shifting the previously applied input sequence l0i (t − 1) and extending ¯ i.e., it by the constant input d, ¯ . ˆli (t) := [l0 (1|t − 1), . . . , l0 (N − 1|t − 1), d] (5) i

i

Constraints (3i)–(3j) are discussed below. We call the second optimization problem given in the following “auxiliary optimization problem”, since it employs a heuristic to estimate how placed orders will affect the future acquisition and thus the future stock level. The auxiliary optimization problem determines an optimal order trajectory oi (t) and is similar to Problem 1. Problem 3. (Auxiliary optimization problem) min Jiaux (si (t), li (t), oi (t), ˆ oi (t), ˆli−1 (t)) (6) oi (t)

subject to s˜i (0|t) = si (t) ˜i (k|t) − li (k|t) s˜i (k + 1|t) = s˜i (k|t) + a k = 0, . . . , N − 1 s˜i (N |t) = sdi s˜i (k|t) ≥ 0, oi (k|t) ≥ 0, ¯ oi (k|t) = d,

K 

k=0

oi (k|t) ≥

K 

k=0

k = 0, . . . , N k = 0, . . . , N − 1 k = N − τ, . . . , N − 1 oˆi (k|t), K = 0, . . . , N − 1

7949

(7a) (7b) (7c) (7d) (7e) (7f) (7g)

oi (t), ˆli−1 (t)) = where J aux (s (t), li (t), oi (t), ˆ N −1 iaux i si (k|t)) and Laux : R → R denotes the i k=0 Li (˜ auxiliary stage cost. The trajectory ˜ ai (t) is the auxiliary acquisition trajectory, which implements a heuristic to predict how orders oi (t) affect future acquisition and is chosen as  li−1 (t − τ + k) k ∈ I[0,τ −1] . a ˜i (k|t) = ˆli−1 (k − τ |t) +  k ∈ I[τ,N −1] oi (k − τ |t) − oˆi (k − τ |t) (8) As before, the candidate trajectory ˆ oi (t) results from shifting the previously applied input sequence o0i (t − 1) ¯ i.e., and extending it by the constant input d, 0 0 ¯ . (9) oi (t) = [oi (1|t − 1), . . . , oi (N − 1|t − 1), d] ˆ The heuristic (8) is based on the following supposition: At the previous time instant t − 1, the order trajectory o0i (t−1) was answered by the leavings trajectory l0i−1 (t−1). At the current time instant t, we consider the shifted trajectory ˆ oi (t) and assume that an adaptation of this trajectory oi (t)−ˆ oi (t) is directly reflected in an adaptation of upstream leavings li−1 (t) − ˆli−1 (t) and thus defines the acquisition ai (t). Remark 4. The need for such a heuristic stems from the fact that the decision variable oi (t) does not directly affect any of the state variables at stage i. Hence, in order to decide for a suitable order trajectory at stage i, we employ a heuristic to estimate the effect of orders on the future stock. Thus, the employed heuristic models how the neighboring stage i − 1 answers orders oi (t) with leavings li−1 (t). More elaborate heuristics could include a more detailed model or incorporate uncertainties, which is part of our ongoing work. Note that the heuristic is only used as an estimate for future acquisition at stage i and does not actually have not to be satisfied by stage i − 1. Remark 5. Note that global knowledge of the value d¯ is required, since this is the steady-state input for each stage in the supply chain, which is required for the construction of feasible candidate trajectories. 3.2 Two-step MPC update procedure Unlike in standard MPC schemes, in each time step we solve two optimization problems at each stage i, first determining an optimal leaving trajectory and secondly determining an optimal order trajectory. The motivation for this approach is twofold. On the one hand, demand satisfaction usually is the primary objective in a supply chain. Placing orders to fill up the stock is done as a resulting step

8225

Proceedings of the 20th IFAC World Congress 7950 Philipp N. Köhler et al. / IFAC PapersOnLine 50-1 (2017) 7947–7952 Toulouse, France, July 9-14, 2017

to compensate for the predicted material outflow. On the other hand, such an approach establishes appealing systems theoretic properties in view of the couplings between the separate stages of the supply chain. By optimizing the leaving and order trajectories independently, each of the optimization problems is subject to couplings with only one neighboring stage. In this manner, the dynamic coupling throughout the supply chain is split at each stage, which facilitates a distributed approach. Particularly, this “splitting” of the supply chain is instrumental for establishing convergence of the overall system for the proposed sequential algorithm and is discussed below. 3.3 Terminal equality constraints in supply chain systems Next, we closely study the interplay between the stock and flow nature of the considered supply chain system and the terminal constraints employed in each of the local MPC optimization problems. It will turn out that this interplay results in inherent recursive feasibility of the overall system with respect to the terminal equality constraints of the local MPC optimization problems. By the term “inherent” we mean the fact that stages do not explicitly consider the terminal constraints of their neighbors. For this investigation, in this section we only consider the system dynamics constraints (3c),(3d),(7b) and the terminal equality constraints (3e),(3f),(7c) of Problems 1 and 3. Additionally we require the τ -step end pieces of the leaving and order trajectories to be fixed to ¯ i.e., constraints (3i) and (7f). In other the constant value d, words, the decision variables are only optimized over a horizon of N − 1 − τ , which is instrumental to establish recursive feasibility with respect to the terminal equality constraints in presence of the transport delay. In the following we assume that at time instant t ∈ I≥t0 feasible trajectories o0i (t) and l0i (t) and accordingly a0i (t) and d0i (t) are known for all i ∈ I[1,P ] , which fulfill the terminal constraints of Problems 1 and 3. Due to the simple system dynamics, we can explicitly calculate the resulting stock and backlog trajectories for each stage i ∈ I[1,P ] and obtain according to the terminal constraint (3e) N −1 N −1   (10) a0i (k|t) − li0 (k|t) sdi = si (t) +

constraint), as well as by the respective difference between the cumulative sum of the order trajectories o0i (t) and oi (t + 1) (through the virtual stock terminal constraint), i.e., ∆di (t + 1) = ∆li (t + 1) = ∆oi (t + 1) = ∆di+1 (t + 1). This is formalized by the following lemma. Lemma 6. Given the demand trajectories d0i (t) = o0i+1 (t) and di (t + 1) = oi+1 (t + 1) at stage i ∈ I[1,P ] , feasible leaving and order trajectories li (t + 1) and oi (t + 1) with respect to the terminal constraints on the backlog (3f) and the auxiliary stock (7c) of Problems 1 and 3, respectively, N −1 N −1 satisfy k=1 li0 (k|t) − k=0 li (k|t + 1) = ∆di (t + 1) and N −1−τ 0 N −1−τ oi (k|t) − k=0 oi (k|t + 1) = ∆di (t + 1). k=1 Remark 7. The heuristic (8) for estimation of the acquisition ˜ ai (t) employed to select an order trajectory oi (t+1) is crucial for the result of Lemma 6 to hold. Its key property is that it keeps track of previously placed but yet unfulfilled orders by considering the previously applied order trajectory o0i (t). Thereby, the neighboring backlog level is implicitly considered when selecting an order trajectory. Another approach to this is to introduce an additional state of unfulfilled orders, c.f. Dunbar and Desa (2007). Having obtained feasible leaving and order trajectories li (t+1) and oi (t+1) at stage i and time t+1, we next show that the terminal constraint on the stock (3e) of Problem 1 at stage i is indeed satisfied by a subsequently updated feasible leaving trajectory li−1 (t+1) obtained at stage i−1 based on the updated demand di−1 (t + 1) = oi (t + 1). Lemma 8. Given feasible leaving and order trajectories oi (t + 1) and li (t + 1) for di (t + 1) with respect to the terminal backlog constraint (3f) and the terminal auxiliary stock constraint (7c) of Problems 1 and 3 at stage i, the terminal stock constraint (3e) of Problem 2 at stage i is satisfied by an updated leaving trajectory li−1 (t + 1) subsequently obtained at stage i − 1. The proofs of Lemma 6 and 8 are omitted in this conference paper due to the space restrictions. In both proofs, the basic idea is to investigate the updated trajectories with respect to their compatibility to the terminal constraints, i.e., using (10) and (11).

Combining Lemma 6 and Lemma 8 shows that Problem 1 and Problem 3 are recursively feasible with respect to N −1−τ N −1  the terminal equality constraints when solved sequentially 0 = si (t) + li−1 (t − τ + k) + li−1 (k|t) − li0 (k|t) along the supply chain from i = P to i = 1. k=0 k=0 k=0 Remark 9. If the value ∆dP (t + 1) is communicated to and according to the terminal constraint (3f) all stages, then even solving Problem 1 and Problem 3 N −1 N −1 in parallel at all stages i ∈ I[1,P ] still yields recursive   bi (t) + o0i+1 (k|t) − li0 (k|t) = 0. (11) feasibility with respect to the terminal constraints. k=0 τ −1 

k=0

k=0

k=0

Next, we characterize feasible leaving and order trajectories of stage i at the subsequent time instant t + 1, i.e., li (t + 1) and oi (t + 1), given an updated demand trajectory di (t + 1). For this, we consider the difference between the cumulative sum of the remainder of the previous demand trajectory d0i (t) and the cumulative sum of the updated demand trajectory di (t + 1), i.e., ∆di (t + N −1 N −1 1) := k=1 o0i+1 (k|t) − k=0 oi+1 (k|t + 1). We first show that this difference is reflected both by the respective difference between the cumulative sum of the leaving trajectories l0i (t) and li (t + 1) (through the backlog terminal

4. SEQUENTIAL DISTRIBUTED MPC ALGORITHM In this section, we present a sequential update scheme for the MPC problems associated with each stage of the supply chain, starting at the downstream end of the supply chain at stage P and placing orders upstream along the supply chain. Thus, at each stage i, first the demand di (t) obtained from the downstream neighbor i + 1 is answered by computing an appropriate leaving trajectory li (t), which is then transmitted to stage i + 1. After that, stage i itself computes an order trajectory oi (t) which is

8226

Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 Philipp N. Köhler et al. / IFAC PapersOnLine 50-1 (2017) 7947–7952

sent to the upstream neighbor i − 1. The basic idea of this sequential scheme is based on Richards and How (2007). In the following sequential scheme, in every update step we consider each pair of stages i and i + 1 and calculate their respective pairwise cost. We introduce the following abbreviations for the respective cost with two superscripts (e.g. Jiˆ,0 (t)), where the left one denotes the type of the acquisition trajectory (on the left side) of a stage, and the right one denotes the type of the leaving and demand trajectories (on the right side) of a stage. We compare the cost resulting from candidate trajectories with the cost resulting from optimized trajectories. Thus, we consider ˆ i (t), ˆli (t), ˆ ai+1 (t) the candidate trajectories d ^,^ ˆ i (t)) J (t) := Ji (si (t), bi (t), ˆli (t), ˆ ai (t), d i

^,0 (t) := Ji+1 (si+1 (t), bi+1 (t), l0i+1 (t), ˆ ai+1 (t), d0i+1 (t)), Ji+1

(t), ltest (t), atest versus the optimized trajectories dtest i i i+1 (t) Ji^,test (t) := Ji (si (t), bi (t), ltest (t), ˆ ai (t), dtest (t)) i i

test,0 0 (t) := Ji+1 (si+1 (t), bi+1 (t), l0i+1 (t), atest Ji+1 i+1 (t), di+1 (t)). Algorithm 1. (DMPC of a supply chain)

(0) Initialization: Set t = t0 , communicate the customer demand d¯ to all stages i ∈ I[1,P ] and find feasible candidate input sequences ˆli (t0 ) and ˆ oi (t0 ) and according state trajectories. Communicate these to the ˆ i (t0 ), ˆ ai (t0 ) according neighboring stages and set d test ˆ P (t0 ). to (4a)–(4b) and dP (t0 ) := d (1) At time t ∈ I≥t0 , sequentially do for all stages i = P, . . . , 1: (a) Set ai (t) := ˆ ai (t) and di (t) := dtest (t) and solve i Problem 1. Denote its minimizer by ltest (t) and i obtain the resulting cost Ji^,test (t). (b) Transmit 1 ltest (t) to stage i + 1, set atest i i+1 (t) test,0 accordingly and compute Ji+1 (t). If test,0 ^,0 (t) ≤ Ji^,^ (t) + Ji+1 (t) (13) Ji^,test (t) + Ji+1

(t) and keep o0i+1 (t) := holds true, set l0i (t) := ltest i test oi+1 (t). Otherwise fall back to candidate trajectories and oi+1 (t). set l0i (t) := ˆli (t) and o0i+1 (t) := ˆ (c) Solve Problem 3 with li (t) = l0i (t) to obtain otest (t). i (d) Transmit 2 otest (t) to the upstream neighboring i stage i − 1, which sets dtest i−1 (t) accordingly. (2) At all stages i ∈ I[1,P ] apply oi (t) = o0i (0|t) and li (t) = li0 (0|t) to the plant. Calculate the candidate ˆ i (t+1) trajectories ˆ oi+1 (t+1) and ˆli−1 (t+1), and set d and ˆ ai (t + 1) accordingly. (3) Set t := t + 1 and go to Step 1.

for the combined resulting cost of stages i and i + 1. If this is not the case, the candidate leaving trajectory is applied instead. We note that a similar technique was also used, e.g., by Maestre et al. (2011a) and Müller et al. (2012). Note that the calculation of the different cost terms in step 1b of the algorithm is a pure evaluation of the respective cost functions and no optimization problem has to be solved. We next analyze the proposed sequential algorithm and show that it has the desired properties. Proposition 10. Suppose that Assumption 2 is satisfied and that there exists an initially feasible solution to Problems 1 and 3 for all stages i ∈ I[1,P ] at t = t0 . Then, Problems 1 and 3 in step 1a and 1c of Algorithm 1 are feasible for all t ∈ I≥t0 . Furthermore, the stock and backlog of all stages i ∈ I[1,P ] asymptotically converge to the desired values and the constant customer demand dP (t) = d¯ is satisfied. Proof. First observe that the candidate trajectories (5) and (9) indeed satisfy the constraints of Problems 1 and 3 for all i ∈ I[1,P ] thanks to the terminal constraints in each of the optimization problems and by construction ¯ In the case of a seof the candidate trajectory using d. quential update scheme, satisfaction of the terminal equality constraints of the Problems 1 and 3 directly follows from Lemma 6 and 8. Constraint (3j) and (7g) of the optimization problems represent compatibility constraints tailored to ensure satisfaction of the positivity constraints of stage i + 1’s stock and stage i − 1’s backlog, respectively. Since these constraints require the cumulative updated leaving and order trajectories of stage i to lie above its cumulative candidate leaving and order trajectory, the predicted neighboring stock and backlog levels are at each time instant larger or equal than the respective level obtained by the candidate leaving and order trajectories. This concludes the proof of recursive feasibility. We next show convergence of si (t), bi (t) and li (t) to the respective desired steady states at each stage i ∈ I[1,P ] . The negotiation Step 1b in Algorithm 1 ensures that for Ji0,0 (t) := Ji (si (t), bi (t), l0i (t), a0i (t), d0i (t)) and any i ∈ I[1,P −1] it holds 0,0 ^,0 (t) ≤ Ji^,^ (t) + Ji+1 (t). (14) Ji^,0 (t) + Ji+1 At the upstream end of the supply chain, for stage i = 1, we immediately get J10,0 ≤ J1^,0 due to the acquisition being modeled as a pure time delay. Accordingly, at the downstream end of the supply chain, for stage i = P , we always have JP^,0 = JP^,^ . Applying the sequential Algorithm 1, we thus obtain for the overall cost of the entire supply chain system

J(t) :=

The negotiation step 1b of the algorithm requires cooperation between the stages and ensures a cost decrease for the considered pair of stages by the updated leaving trajectory l0i (t). Condition (13) checks whether the optimized leaving trajectory ltest (t) obtained at stage i yields a cost decrease i 1 For stage i = P , skip Step 1b and set l0 (t) := l0,test (t) and P P ¯ . . . , d]. ¯ d0P (t + 1) := [d, 2 For stage i = 1, skip Step 1d and instead set o0 (t) := otest (t) 1 1 if J1test,0 (t) ≤ J1^,0 (t), and o01 (t) := ˆ o1 (t) otherwise. Set a01 (t) accordingly as in (4b).

7951

P 

Ji (si (t), bi (t), l0i (t), a0i (t), d0i (t)) =

i=1

≤ J1^,0 (t) + J20,0 (t) + (14)

P 

Ji0,0 (t)

i=1

P 

Ji0,0 (t)

i=3 P 

≤ J1^,^ (t) + J2^,0 (t) +

Ji0,0 (t).

i=3

Using the same argument recursively from i = 1 up to P P ^,^ (t). Thus, along the closed-loop yields J(t) ≤ i=1 Ji

8227

Proceedings of the 20th IFAC World Congress 7952 Philipp N. Köhler et al. / IFAC PapersOnLine 50-1 (2017) 7947–7952 Toulouse, France, July 9-14, 2017

80

40 i=3

60

i=1

20

bd i

30 20

0 0 5 10 15 20 Fig. 2. Closed-loop trajectories si (t) and bi (t) (solid lines) and desired values sdi and bdi (dashed lines). evolution of the overall supply chain dynamics we get P    Ji^,^ (t + 1) − Ji0,0 (t) J(t + 1) − J(t) ≤ =−

d(t)

80

i=3

60

i=1

i=2 d(t)

40

sd i

i=1 P 

i=2 i=1

i=2

40

i=3

20

0 5 10 15 20 0 5 10 15 20 Fig. 3. Closed-loop trajectories li (t) (left) and oi (t) (right) and customer demand d(t) (dashed line). employed in the MPC problems are inherently satisfied for the overall system when updated in sequence. The insights gained by the study of this supply chain specific DMPC approach serve as a first step towards ultimately relating the use of predictive information with (overall and individual) system performance in supply chains by means of DMPC.

Li (s0i (0|t), b0i (0|t))

i=1

which shows, by Assumption 2, asymptotic convergence si (t) → sdi and bi (t) → 0 for all i ∈ I[1,P ] and t → ∞ and, according to the system dynamics (1), also li (t) → d¯ and oi (t) → d¯ for all i ∈ I[1,P ] and t → ∞. 

Remark 11. The compatibility constraints (3j) and (7g) are quite restrictive. Other techniques to guarantee satisfaction of the positivity constraints are, e.g., to transmit additional lower bound trajectories or to introduce iterations between neighboring stages. 5. NUMERICAL SIMULATIONS

We demonstrate the effectiveness of the proposed algorithm by numerical simulations of an exemplary supply chain consisting of P = 3 stages and a step-change in customer demand at t0 = 4 from dP (t) = 20 for t ∈ I[0,t0 −1] to dP (t) = d¯ = 25 for t ∈ I≥t0 . We initialize the overall system at steady state, i.e., si (t) = sdi = 100, bi (t) = 0, oi (t) = li (t) = dP (t) for all i ∈ I[1,P ] and t ∈ I[0,t0 −1] . The transport delay is chosen as τ = 4 and the prediction horizon of all MPC problems is set to N = 8. The cost functions are chosen quadratic as Li (si , bi ) = (si − sdi )2 + 5b2i and Laux si ) = (˜ si −sdi )2 for all i ∈ I[1,P ] . This choice of i (˜ the cost functions stresses the primary objective of demand satisfaction by keeping the backlog low. Figures 2 and 3 show the resulting closed-loop leaving and order trajectories and according stock and backlog trajectories. Asymptotic convergence and constraint satisfaction as asserted by Proposition 10 is confirmed. The observed amplification in stock and backlog deviation along the supply chain is inevitable due to the transport delay, but it can be seen how predictive information yields a fast settling behavior after time instant t0 + τ . This is initiated by exploiting guaranteed order satisfaction at stage i = 1 in combination with availability of predictive information. 6. CONCLUSIONS In this paper, we proposed a sequential distributed MPC algorithm closely tailored to supply chain systems, and we investigated the specifics resulting from this combination. We showed that the terminal equality constraints

REFERENCES Braun, M., Rivera, D., Flores, M., Carlyle, W., and Kempf, K. (2003). A model predictive control framework for robust management of multi-product, multi-echelon demand networks. Annu Rev Control, 27(2), 229 – 245. Chopra, S. and Meindl, P. (2012). Supply chain management. Strategy, planning & operation. Prentice Hall, New Jersey, 5th edition. Christofides, P.D., Scattolini, R., de la Pena, D.M., and Liu, J. (2013). Distributed model predictive control: A tutorial review and future research directions. Comput Chem Eng, 51, 21–41. Dunbar, W.B. and Desa, S. (2007). Distributed MPC for dynamic supply chain management. In Assessment and future directions of nonlinear model predictive control, 607–615. Springer, BerlinHeidelberg. Grüne, L. and Pannek, J. (2011). Nonlinear Model Predictive Control: Theory and Algorithms. Communications and Control Engineering. Springer, London. Maestre, J., De La Pena, D.M., Camacho, E., and Alamo, T. (2011a). Distributed model predictive control based on agent negotiation. J Process Control, 21(5), 685–697. Maestre, J., Muñoz De La Peña, D., and Camacho, E. (2011b). Distributed model predictive control based on a cooperative game. Optimal Control Applications and Methods, 32(2), 153–176. Müller, M.A., Reble, M., and Allgöwer, F. (2012). Cooperative control of dynamically decoupled systems via distributed model predictive control. Int J Robust Nonlinear Control, 22(12), 1376– 1397. Negenborn, R.R. and Maestre, J. (2014). Distributed model predictive control: An overview and roadmap of future research opportunities. IEEE Control Syst. Mag., 34(4), 87–97. Richards, A. and How, J. (2007). Robust distributed model predictive control. Int J Control, 80(9), 1517–1531. Sterman, J.D. (2000). Business Dynamics: Systems Thinking and Modeling for a Complex World. McGraw-Hill, New York. Stewart, B.T., Venkat, A.N., Rawlings, J.B., Wright, S.J., and Pannocchia, G. (2010). Cooperative distributed model predictive control. Syst Control Lett, 59(8), 460–469. Subramanian, K., Rawlings, J.B., Maravelias, C.T., Flores-Cerrillo, J., and Megan, L. (2013). Integration of control theory and scheduling methods for supply chain management. Comput Chem Eng, 51, 4–20. Wang, W. and Rivera, D.E. (2008). Model predictive control for tactical decision-making in semiconductor manufacturing supply chain management. IEEE Trans. Control Syst. Technol., 16(5), 841–855.

8228