Distributed Norm Optimal Iterative Learning Control for Point-to-Point Consensus Tracking

Available online at www.sciencedirect.com

ScienceDirect IFAC PapersOnLine 52-29 (2019) 292–297

Distributed Distributed Norm Norm Optimal Optimal Iterative Iterative Distributed Norm Optimal Iterative Learning Control Point-to-Point Distributed Norm for Optimal Iterative Learning Control for Point-to-Point Learning Control for Point-to-Point Consensus Tracking Learning Control for Point-to-Point Consensus Tracking Consensus Tracking Consensus Tracking Bin Chen ∗∗ Bing Chu ∗∗

Bin Chen ∗∗ Bing Chu ∗∗ Bin Chen Chen Bing Bing Chu Chu Bin Bin Chen ∗ Bing Chu ∗ ∗ ∗ Department of Electronics and Computer Sciences, ∗ Department of Electronics and Computer Sciences, ∗ Department of Electronics Electronics and Computer Computer Sciences, University of Southampton, United Kingdom, of and Sciences, University of Southampton, Southampton, Southampton, United Kingdom, ∗ Department Department of Electronics and Computer Sciences, University of Southampton, Southampton, United Kingdom, (e-mail: {bc1n16, B.Chu }@ soton.ac.uk) University of Southampton, Southampton, United Kingdom, (e-mail: }@ soton.ac.uk) {bc1n16, B.Chu University(e-mail: of Southampton, Southampton, United {bc1n16, B.Chu }@ soton.ac.uk) (e-mail: {bc1n16, B.Chu }@ soton.ac.uk)Kingdom, (e-mail: {bc1n16, B.Chu }@ soton.ac.uk) Abstract: High performance Abstract: High performance consensus consensus tracking tracking of of networked networked dynamical dynamical systems systems working working Abstract: High performance consensus tracking of dynamical systems working repetitively an important class of coordination problems and it has found many applications Abstract: is High performance consensus tracking of networked networked dynamical systems working repetitively is an important class of coordination problems and it has found many applications Abstract: High performance consensus tracking of networked dynamical systems working repetitively is an important important class of coordination coordination problems andwhich it has hasdoes found many applications in different areas. Recently, iterative learning control (ILC), not require a highly repetitively is an class of problems and it found many applications in different areas. Recently, iterative learning control (ILC), which does not require a highly repetitively is an important class of coordination problems and it has found many applications in different areas. Recently, iterative learning control (ILC), which does not require a highly accurate model to achieve the high performance requirement, has been developed for the in different areas. Recently, iterative learning control (ILC), which does not require a highly accurate model to achieve the high performance requirement, has been developed for the in different areas. Recently, iterative learning control (ILC), which does not require a highly accurate model to achieve the high performance requirement, has been developed for the consensus tracking problem. Most of existing ILC algorithms consider about the tracking of accurate model to achieve the high performance requirement, has been developed for the consensus tracking problem. Most of existing ILC algorithms consider about the tracking of accurate model to achieve the high performance requirement, has been developed for the consensus tracking problem. Most of existing ILC algorithms consider about the tracking of a reference defined over the whole trial length, while the Point-to-Point (P2P) task where the problem. Most trial of existing consider about of aconsensus referencetracking defined over the whole length, ILC whilealgorithms the Point-to-Point (P2P) the tasktracking where the consensus tracking problem. Most of existing ILC algorithms consider about the tracking of a reference reference defined over the wholeoftrial trial length, while while the Point-to-Point (P2P) task where the the emphasis is placed on the tracking intermediate time instant points, has not been explored. To a defined over the whole length, the Point-to-Point (P2P) task where emphasis is placed on the tracking of intermediate time instant points, has not been explored. To a reference defined over the whole trial length, while the Point-to-Point (P2P) task where the emphasis is placed on the tracking of intermediate time instant points, has not been explored. To bridge this gap, we develop a optimal ILC algorithm P2P tracking emphasis placed the tracking intermediate time instant points,for has notconsensus been explored. To bridge thisis weon develop a norm normof ILC (NOILC) (NOILC) algorithm for P2P tracking emphasis isgap, placed on thenot tracking ofoptimal intermediate time instant points, has notconsensus beenerror explored. To bridge this gap, we develop a norm optimal ILC (NOILC) algorithm for P2P consensus tracking problem that guarantees only the monotonic convergence of consensus tracking norm bridge this gap, we develop a norm optimal ILC (NOILC) algorithm for P2P consensus tracking problem that guarantees notaonly the monotonic convergence of consensus tracking error tracking norm to to bridge this gap, we develop norm optimal ILC (NOILC) algorithm for P2P consensus problem that guarantees not only only the monotonic convergence of consensus tracking error norm to to zero, but also the convergence of input to the minimum input energy solution, which is desired in problem that guarantees not the monotonic convergence of consensus tracking error norm zero, butthat also guarantees the convergence of input to the minimum input of energy solution, which is desired in problem not only the monotonic convergence consensus tracking error norm to zero, but but Moreover, also the the convergence convergence of input input toalternating the minimum minimum input method energy solution, solution, whichwe is desired desired ina practice. using the idea of the direction of multipliers, develop zero, also of to the input energy which is in practice. using the idea of thetoalternating direction of multipliers, developina zero, but Moreover, also the convergence of input theproposed minimum input method energy solution, whichwe is algorithm desired practice. Moreover, using idea of the alternating direction method of we develop distributed implementation algorithm, allowing the resulting resulting practice. Moreover, using the themethod idea of for the the alternating direction method of multipliers, multipliers, we develop aa distributed implementation method for the proposed algorithm, allowing the algorithm practice. Moreover, using the idea of the alternating direction method of multipliers, we develop a distributed implementation method for the proposed algorithm, allowing the resulting algorithm to be applied to large scale networked dynamical systems. Rigorous analysis of the algorithm’s distributed implementation for dynamical the proposed algorithm, allowing the resulting algorithm to be applied to large scale method networked systems. Rigorous analysis of the algorithm’s distributed implementation method forsimulations the proposed algorithm, allowing the resulting algorithm to be be applied applied to large large scale scale networked dynamical systems. Rigorous analysis of the the algorithm’s algorithm’s properties is provided and numerical are given to verify its effectiveness. to to networked dynamical systems. Rigorous analysis of properties is provided and numerical given Rigorous to verify analysis its effectiveness. to be applied to large scale networkedsimulations dynamical are systems. of the algorithm’s properties is provided provided and numerical numerical simulations are given to to verify verify its its effectiveness. effectiveness. properties is and simulations are given © 2019, IFAC (International Automatic Control) Hosting by Elsevier Ltd. All rights reserved. properties is provided andFederation numericalofsimulations are given to verify its effectiveness. Keywords: Keywords: Iterative Iterative learning learning control, control, networked networked dynamical dynamical systems, systems, point point to to point point tasks, tasks, the the Keywords: Iterative learning control, networked dynamical systems, point to point tasks, alternating direction method of multipliers Keywords: Iterative learning control, networked dynamical systems, point to point tasks, the the alternating direction method of multipliers Keywords: networked dynamical systems, point to point tasks, the alternating direction method of alternating Iterative directionlearning method control, of multipliers multipliers alternating direction method of multipliers 1. worked systems; systems; Devasia Devasia (2017) (2017) proposes proposes aa model-inverse model-inverse 1. INTRODUCTION INTRODUCTION worked 1. worked systems; Devasia (2017) aa heterogeneous model-inverse based on frequency for linear 1. INTRODUCTION INTRODUCTION workedILC systems; Devasia domain (2017) proposes proposes model-inverse based ILC on frequency domain for linear heterogeneous 1. INTRODUCTION worked systems; Devasia (2017) model-inverse based on domain for linear networked systems. For more more ILCproposes algorithms for consensus consensus based ILC ILC on frequency frequency domain for lineara heterogeneous heterogeneous High performance networked dynamical systems working networked systems. For ILC algorithms for High performance networked dynamical systems working based ILC on frequency domain for linear heterogeneous networked systems. For more ILC algorithms for consensus tracking problem, please refer to Meng et al. (2012), Li and and networked systems. For more ILC algorithms for consensus High performance networked dynamical systems working in a repetitive manner and requiring high performance High performance networked dynamicalhigh systems working tracking problem, please refer to Meng et al. (2012), Li in a repetitive manner and requiring performance networked systems. For more ILC algorithms forYang consensus tracking problem, please refer to Meng et al. (2012), Li and High performance networked dynamical systems working Li (2014a,b), Jin (2016), Yang and Xu (2016), et al. tracking problem, please refer to Meng et al. (2012), Li and in aa repetitive manner and requiring high performance have attracted increasing attentions during the last two in repetitive manner and requiring high performance Li (2014a,b), Jin (2016), Yang and Xu (2016), Yang et al. have attracted increasing attentions during the last two tracking problem, please refer to Meng et al. (2012), Li and Li (2014a,b), Jin (2016), Yang and Xu (2016), Yang et al. in a repetitive manner and requiring high performance (2016, 2017), Shen and Xu (2018), Shen et al. (2019). Li (2014a,b), Jin (2016), Yang and Xu (2016), Yang et al. have during last two decades (Knorn increasing et al., al., 2016; 2016;attentions Olfati-Saber et al., al.,the 2007), e.g., have attracted attracted increasing attentions during the last e.g., two (2016, 2017), Shen and Xu (2018), et al. (2019). decades (Knorn et Olfati-Saber et 2007), Li (2014a,b), Jin (2016), Yang and Shen Xu (2016), Yang et al. (2016, 2017), Shen and Xu (2018), Shen et al. (2019). have attracted increasing attentions during the last two (2016, 2017), Shen and Xu (2018), Shen et al. (2019). decades (Knorn et al., 2016; Olfati-Saber et al., 2007), e.g., in logistics, traffic networks and manufacturing. manufacturing. Among decades (Knorn et al., 2016; Olfati-Saber et al., 2007), e.g., Most of2017), the existing existing ILC methods for consensus consensus tracking in logistics, traffic networks and Among (2016,of Shen and Xumethods (2018), Shen et al. (2019). Most the ILC for tracking decades (Knorn et al., 2016; Olfati-Saber et al., 2007), e.g., in traffic networks and manufacturing. Among all the coordination problems of high performance netin logistics, logistics, traffic networks and manufacturing. Among Most of the existing ILC methods for consensus tracking problem focus on the tracking performance over the whole all the coordination problems of high performance netMost of the existing ILC methods for consensus tracking focus on the tracking performance over the whole in trafficsystems, networks and manufacturing. Among all the coordination problems of high performance networked dynamical there exists an important class Most of the existing ILC methods for consensus tracking all logistics, the dynamical coordination problems ofexists high an performance net- problem problem focus on the tracking performance over the whole time horizon. However, there is a class of applications problem focus on the tracking performance over the whole worked systems, there important class horizon. However, there is aa class of applications all theconsensus coordination problems high an performance net- time worked dynamical systems, there important class called tracking, which requires all the the subsyssubsysproblem focus on thetracking tracking performance over thespecific whole worked dynamical systems, thereofexists exists an important class time horizon. However, there is class of applications that focus on the of the reference at time horizon. However, there is a class of applications called consensus tracking, which requires all that focus on the tracking of the reference at specific worked dynamical systems, there exists an important class called consensus tracking, which requires all the subsystems perfectly track a prescribed reference (even not all time horizon. However, there is a class of applications called consensus tracking, which requires all the subsysthat focus on the tracking of the reference at specific time instants, which is called as Point-to-Point (P2P) that focus on the tracking of the reference at specific tems perfectly a prescribed reference not all time instants, which is called as Point-to-Point (P2P) called consensustrack tracking, which requires all(even the subsystems perfectly track aaccess prescribed reference (even not all the have to reference signal). that on the tracking of the reference multi at specific temssubsystems perfectly track prescribed notFor all time instants, which is as (P2P) consensus tracking problem. As an Point-to-Point example, robot the subsystems have a access to the thereference reference(even signal). For time focus instants, which is called called as Point-to-Point (P2P) consensus tracking problem. As an example, multi robot tems perfectly track a prescribed reference (even not all the access reference For example, modernhave air transportation transportation contains great numnumtime instants, which is called as Point-to-Point (P2P) the subsystems subsystems have access to to the the contains referenceaasignal). signal). For consensus tracking problem. As an example, multi robot collaborative task requires all the robotic arms travel consensus tracking problem. As an example, multi robot example, modern air great task requires all the robotic arms travel the subsystems access to the reference For collaborative example, modern air transportation contains aasignal). great number of unmanned aerial vehicles (UAVs) working together consensus tracking problem. As an example, multi robot example, modernhave air transportation contains great numcollaborative task requires all the robotic arms travel from the initial position to the ‘pick’ position at time collaborative task requires all the robotic arms travel ber of unmanned aerial vehicles (UAVs) working together from the initial position to the ‘pick’ position at time example, modern air transportation contains a great number of unmanned aerial vehicles (UAVs) working together to transport the goods repetitively from the origin to collaborative task requires all the robotic arms travel ber of unmanned aerial vehicles (UAVs) working together from the initial position to the ‘pick’ position at time instant t ; then collaboratively move the objectives from from the initial position to the ‘pick’ position at time 1 to transport the goods repetitively from the origin to instant t ; then collaboratively move the objectives from ber of unmanned aerial vehicles (UAVs) working together to the goods repetitively from the to the destination with a precision In from the position to themove ‘pick’ at time to transport transport goods repetitively from the origin origin to instant tt111 ;;initial then collaboratively collaboratively move theposition objectives from the ‘pick’ position to the ‘place’ position at time instant the destinationthe with a high high precision requirement. requirement. In this this instant then the objectives from the ‘pick’ position to the ‘place’ position at time instant to transport the goods repetitively from the origin to the destination with aa of high precision requirement. In this this application, only part the UAVs know the reference for instant t ; then collaboratively move the objectives from the destination with high precision requirement. In ‘pick’1 position position tothe theinitial ‘place’position positionand at time time instant tthe return wait for the ‘pick’ the ‘place’ position at 2 ;; finally application, only part of the UAVs know the reference for tthe finally return to toto initial position wait instant for the the the destination high requirement. In this application, only part the UAVs the reference for security reason. During the control process, each UAV is next ‘pick’ position tothe the ‘place’ positionand atthe time instant application, onlywith parta of of theprecision UAVs know know the each reference for tt222 ;; finally return to the initial position and wait for experiment. This task lies emphasis on consensus finally return to the initial position and wait for the the security reason. During the control process, UAV is next experiment. This task lies emphasis on the consensus application, only part of the UAVs know the reference for security reason. During the control process, each UAV is required to negotiate negotiate the control strategy with its surt2 ; finally return to the initial position and wait fortime the security reason. Duringthe thecontrol controlstrategy process, with each its UAV is tracking next experiment. This task lies emphasis on the consensus at time instants t , t , and tracking at other next experiment. This task lies emphasis on the consensus 1 2 required to surtracking at time instants t , t , and tracking at other time security During thecontrol control process, each UAV is next required to negotiate the strategy with its surrounding UAVs and ‘agree’ the best control strategy to experiment. This task on the consensus required reason. to negotiate the control strategy with its surtracking at time time instants t111 ,,lies t222 ,, emphasis and tracking at other other time instants are of less interest. This allows infinite number of rounding UAVs and ‘agree’ the best control strategy to tracking at instants t t and tracking at time instants are of less interest. allows infinite number of required to negotiate the control strategy its surrounding UAVs and ‘agree’ the control strategy to achieve the desired transportation task. at time instants t1 , This t2greater , and tracking at other time roundingthe UAVs andtransportation ‘agree’ the best best controlwith strategy to tracking instants are ofwhich less interest. This allows infinite number of input choices, creating design flexibility, but instants are of less interest. This allows infinite number of achieve desired task. input choices, which creating greater design flexibility, but rounding UAVs and ‘agree’ the best control strategy to achieve the desired transportation task. instants are of less interest. This allows infinite numberbut of achieve theiterative desired transportation task. input choices, which creating greater design flexibility, on the other hand, it leads to significant design difficulties input choices, which creating greater design flexibility, but Recently, learning control (ILC) has been apon thechoices, other hand, itcreating leads to greater significant design difficulties achieve theiterative desired transportation task. Recently, learning control (ILC) has been apinput which design flexibility, but on the other hand, it leads to significant design difficulties too. Despite its importance in practical applications, the on the other hand, it leads toinsignificant difficulties Recently, iterative learning control (ILC) has applied to the the consensus tracking problem. ILC canbeen achieve Recently, iterative learning control (ILC) has been ap- too. Despite its importance practical design applications, the plied to consensus tracking problem. ILC can achieve on other hand, it leads toin design difficulties too.the Despite its importance insignificant practical applications, the Recently, iterative learning control (ILC) has been apgeneral P2P consensus tracking probelm has not been too. Despite its importance practical applications, the plied to the consensus tracking problem. ILC can achieve the high performance requirement even without using an plied to the consensus tracking problem. ILC can achieve general P2P consensus tracking probelm has not been the high performance requirement even without using an too. Despite its importance in out practical applications, the general P2P consensus tracking probelm has not been plied to the consensus tracking problem. ILC can achieve explored. It is worth pointing that a special class of general P2P consensus tracking probelm has not been the high performance requirement even without using an accurate model, in contrast to most of the conventional the high performance requirement even without using an explored. It isconsensus worth pointing outprobelm that a special class of accurate model, in contrast to most of the conventional general P2P tracking has not been explored. It is worth pointing out that a special class of the highmethods. performance requirement even without using an explored. ILC focusing on the consensus tracking at the final point, It is worth pointing out that a special class of accurate model, in contrast to most of the conventional control For instance, Meng et al. (2015a) propose accurate model, in contrast to most of the conventional ILC focusing on the consensus tracking at the final point, control methods. For instance, Meng et al. (2015a) propose explored. It is worth pointing out that a special class of ILC focusing on the consensus tracking at the final point, accurate model, in contrast to most of the conventional terminal ILC (e.g.,Meng and Jia (2011, 2012); Meng et al. ILC focusing on the consensus tracking at the Meng final point, control methods. For instance, Meng et al. (2015a) propose a Proportional-Integral-Derivative type ILC for linear netcontrol methods. For instance, Meng et al. (2015a) propose terminal ILC (e.g.,Meng and Jia (2011, 2012); et al. a Proportional-Integral-Derivative type ILC for linear netILC focusing on the consensus tracking at the Meng final point, terminal ILC (e.g.,Meng and Jia (2011, 2012); Meng et al. control methods. For instance, Meng et al. (2015a) propose (2013, 2014, 2015b); Lv et al. (2018)), cannot be applied terminal ILC (e.g.,Meng and Jia (2011, 2012); et al. aworked Proportional-Integral-Derivative type ILC for linear netsystems with with swithcing swithcing topology; topology; Li for andlinear Li (2016) (2016) a Proportional-Integral-Derivative type ILC net- (2013, 2014, 2015b); Lv et al.Jia (2018)), cannot be applied worked systems Li and Li terminal ILCgeneral (e.g.,Meng and (2011, 2012);problem. Meng et al. (2013, 2014, 2015b); Lv al. cannot be a Proportional-Integral-Derivative type ILC for netto solve the P2P consensus tracking (2013, 2014, 2015b); Lv et et al. (2018)), (2018)), cannot be applied applied worked systems with topology; Li and Li introduce an adaptive ILC for high-order networked systems with swithcing swithcing topology; Linonlinear andlinear Li (2016) (2016) to solve the general P2P consensus tracking problem. introduce an adaptive ILC for high-order nonlinear net(2013, 2014, 2015b); Lv et al. (2018)), cannot be applied to solve the general P2P consensus tracking problem. worked systems with swithcing topology; Li and Li (2016) introduce introduce an an adaptive adaptive ILC ILC for for high-order high-order nonlinear nonlinear netnet- to solve the general P2P consensus tracking problem. introduce an adaptive ILC for high-order nonlinear net- to solve the general P2P consensus tracking problem. 2405-8963 © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Peer review under responsibility of International Federation of Automatic Control. 10.1016/j.ifacol.2019.12.665



Bin Chen et al. / IFAC PapersOnLine 52-29 (2019) 292–297

To bridge this gap, this paper develops a norm optimal ILC (NOILC) algorithm for P2P consensus tracking problem, by extending our recent works on ILC for high performance networked dynamical systems in Chen and Chu (2019a,b). By incorporating a novel performance-based cost function into the NOILC framework, the resulting algorithm not only guarantees the monotonic convergence of the P2P consensus tracking error normto zero, but also produces a minimum energy consumption for a particular choice of initial input. Furthermore, the algorithm can be applied to non-minimum phase systems (since the algorithm does not involve the system inverse) and both homogeneous and heterogeneous networked dynamical systems. We further introduce a distributed implementation for the proposed NOILC algorithm using the alternating direction method of multipliers (ADMM) (Boyd et al., 2011), allowing the proposed framework to be applied to large scale networked dynamical systems. The rest of paper is organised as follows: Section 2 formulates the system dynamics, networked topology of networked dynamical systems and defines the P2P consensus tracking problem; Section 3 proposes a NOILC framework for the P2P consensus tracking problem and provides rigorous analysis for the algorithm’s convergence properties; Section 4 gives a distributed implementation for the proposed framework; Section 5 uses numerical simulations to verify the effectiveness of the proposed algorithm; and finally Section 6 concludes this paper and points out the future directions. 2. PROBLEM FORMULATION In this session, we model the system dynamics, networked topology of the networked dynamical system, and define the P2P ILC design problem. 2.1 System Dynamics Consider a (homogeneous or heterogeneous) networked dynamical system with p agents (subsystems) and the dynamics of ith (1 ≤ i ≤ p) subsystem is a discrete-time, linear time-invariant (LTI), single-input-single-output (SISO) system, which can be represented using the following system state-space model xi,k (t + 1) = Ai xi,k (t) + Bi ui,k (t), xi,k (0) = xi,0 (1) yi,k (t) = Ci xi,k (t) where the time index t ∈ [0, N ], N is the trial length, k represents the trial index; ui,k (·) ∈ R, xi,k (·) ∈ Rni (ni represents the order of ith subsystem), yi,k (·) ∈ R are the input, state and output of ith subsystem at k th trial; Ai , Bi , Ci are system matrices of ith subsystem with proper dimensions. The discussed networked system is working repetitively: in each trial, the system first operates from t = 0 to t = N ; then resets the time t to zero, the state to the initial state; and performs the same task again. For the P2P consensus tracking problem, it requires consensus tracking at M time instants tm (m = 1, 2, · · · , M ) , with the time instant vector defined as (2) Λ = [t1 t2 · · · tM ]T and the corresponding P2P reference vector is defined as rP = [r(t1 ) r(t2 ) · · · r(tM )]T (3)

293

Note that only limited agents have direct access to the P2P reference rP , making the design nontrivial. To facilitate later ILC design, we introduce a ‘lifted form’ to represent the system dynamics (Hatonen et al., 2004). Assuming each subsystem has unity relative degree, i.e., Ci Bi = 0. We can rewrite the system model (1) as yi,k = Gi ui,k + di (4) where the system matrix Gi is  Ci Bi 0 ··· 0  Ci Bi ··· 0   C i Ai B i Gi =  (5) .. .. ..  ..  . . . .  −1 Bi C i AN i

−2 C i AN Bi i

···

Ci Bi

di ∈ R is the response of initial conditions T di = [Ci Ai xi,0 Ci A2i xi,0 Ci A3i xi,0 · · · Ci AN i xi,0 ] N

(6)

th

and ui,k , yi,k are the ‘lifted form’ of i subsystem’s input ui,k (t), output yi,k (t), which defined as ui,k = [ui,k (0)

ui,k (1)

yi,k = [yi,k (1)

yi,k (2)

···

···

ui,k (N − 1)]T

yi,k (N )]T

(7)

P By defining ith subsystem’s P2P output yi,k as P = [yi,k (t1 ) yi,k

yi,k (t2 )

yi,k (tM )]T

···

(8)

the system model (4) can be rewritten as P P yi,k = GP i ui,k + di

GP i ,

(9)

dP i

are obtained by extracting the correspondwhere ing rows from Gi and di . Without loss of generality, rP is replaced by rP − dP 0 and we have dP i = 0. The system model (9) now becomes P = GP yi,k i ui,k

(10)

We define u ˆk , yˆkP , RP as the combined vectors of all the P agents’ input ui,k , output yi,k , and reference rP u ˆk = [u1,k T

u2,k T

P T [y1,k PT

P T y2,k PT

yˆkP =

··· ···

up,k T ]T T

P ]T yp,k

(11)

PT T

RP = [r r ··· r ] and the ‘lifted form’ system model is represented as yˆkP = GP u ˆk (12) P P P where G = diag (G1 , GP , · · · , G ). p 2

The P2P consensus tracking problem is defined as designing an input u ˆk such that the resulting P2P output yˆkP tracks the desired P2P reference RP at time instants Λ. 2.2 Networked Topology In this paper, we use an undirected graph G = (V ,E ) to represent the networked topology, where V = {1, 2, · · · , p} is the vertex set and E ⊂ V × V denotes the edges set. The neighbouring set of ith subsystem is represented as Ni := {j : (i, j) ∈ E }.

To represent the relationship between different subsystems, we introduce the Laplacian matrix L = {lij }, which is a real, positive semi-definite, symmetric matrix with element lij defined as below

Bin Chen et al. / IFAC PapersOnLine 52-29 (2019) 292–297

294

 if j ∈ Ni   −Wij lij = (13) Wij if j = i j∈Ni   0 otherwise where weight Wij denotes the edge’s connection strength (Bullo, 2018). When vertexes i and j are neighbours, Wij > 0; otherwise, Wij = 0. As mentioned above, only limited agents have direct access to rP . Hence, we introduce a diagonal matrix D = diag{dii } to indicate the relationship between ith subsystem and the P2P reference rP , which called as referenceaccessibility matrix. Its element dii is represented as  1 if agent i has access dii = (14) 0 if agent i does not have access To achieve the P2P consensus tracking, the following two assumptions on the networked topology are required: Assumption 1. The graph G is connected, i.e., each vertex can be reached by any other vertex through a path. Assumption 2. At least one agent has direct access to rP , i.e., dii = 0. 2.3 Iterative Learning Control Design For P2P consensus tracking of networked dynamical systems, the ILC design problem is stated as designing an input updating law defined as u ˆk+1 = f (ˆ uk , eˆP (15) k) P such that each subsystem’s P2P output yi,k converges to the desired P2P reference rP , i.e. P lim yi,k = rP , i = 1, 2, · · · , p (16) k→∞

P ˆkP denotes the ‘virtual’ P2P consensus where eˆP k = R −y tracking error. Note that only part of the error information is available to the design since not all the agents have access to the P2P reference.

The design of (15) is not a trivial problem, as the input uk needs to be updated at each time instant t, while the error information is only available at time instants t1 , t2 , · · · , tM . It should also be pointed out that there are infinite number of input solutions for the above problem, and among these solutions, there exists one with the minimum input energy consumption. By using the proposed NOILC framework, it can not only guarantee the P2P consensus tracking error norm converge monotonically to zero, but also converge to the minimum input energy solution, as will be shown later. 3. NORM OPTIMAL ILC FRAMEWORK FOR P2P CONSENSUS TRACKING PROBLEM In this session, a NOILC framework for the P2P consensus tracking problem is derived, and its properties are analysed rigorously. 3.1 Algorithm Description Algorithm 1. For any initial input choice u ˆ0 , the input series {ˆ uk+1 }k≥0 , defined as follows u ˆk+1 = arg min{Jk+1 (ˆ uk+1 )} (17)

provides a solution for the P2P consensus tracking problem, i.e. P = rP , i = 1, 2, · · · , p (18) lim yi,k k→∞

uk+1 ) is the cost function, defined as where Jk+1 (ˆ P Jk+1 (ˆ uk+1 ) = ||ˆ uk+1 − u ˆk ||2 + ||Ek+1 ||2

and the P2P consensus tracking error norm P ||2 ||Ek+1

2 ||Lˆ eP k+1 ||

P 2 Ek+1 2

(19) is

||Dˆ eP k+1 ||

= + (20) where L = L ⊗ IM , D = D ⊗ IM , ⊗ is the Kronecker product, || · || denotes the Euclidean norm and IM is an M by M identity matrix. Note that weightings on norm of the input difference and the P2P consensus tracking error can also be included in (20) without any difficulties, but not considered here for notational simplicity. Remark 1. Solving the optimisation problem (17) directly (centrally), yields a centralised input updating law T

u ˆk+1 = u ˆk +[GP (LT L + D T D)GP + IpM ]−1 T

× GP (LT L + D T D)ˆ eP k

(21)

However, the computational complexity is huge when applying the centralised implementation to large scale networked dynamical systems, which is unrealistic in practice. A distributed implementation of the proposed NOILC framework will be introduced in later section. 3.2 Convergence Properties of the Proposed Algorithm Algorithm 1 has desired convergence performance, as shown in following theorem. Theorem 1. For any initial input choice u ˆ0 , Algorithm 1 guarantees the P2P consensus tracking error norm converges monotonically to 0, i.e. P ||Ek+1 (22) ||2 ≤ ||EkP ||2 , lim ||EkP ||2 = 0 k→∞

Consequently, the P2P consensus tracking is achieved as k → ∞, i.e. P lim yi,k = rP , i = 1, 2, · · · , p (23) k→∞

Proof. The proof is omitted here for space reasons. In addition, the generated input has following property. Theorem 2. Algorithm 1 guarantees the generated input converges as follows lim u ˆk = u ˆ∗ (24) k→∞

where u ˆ∗ is the optimal solution for the following optimisation problem minimize ||ˆ u−u ˆ0 ||2 (25) subject to GP u ˆ − RP = 0 and when u ˆ0 is chosen to be 0, Algorithm 1 converges to the minimum input energy solution, i.e. minimize ||ˆ u||2 (26) subject to GP u ˆ − RP = 0 Proof. The proof is omitted here for space reasons. The above two theorems demonstrate that Algorithm 1 guarantees the P2P consensus tracking error norm converges monotonically to zero and the input converges to



Bin Chen et al. / IFAC PapersOnLine 52-29 (2019) 292–297

the minimum input energy solution, which is desired in practice. Later simulations will verify these two theorems. 4. DISTRIBUTED IMPLEMENTATION OF THE PROPOSED ALGORITHM This section introduces the idea of the alternating direction method of multipliers, and shows the distributed implementation of Algorithm 1, allowing the proposed design to be applied to large scale networked systems. 4.1 The Alternating Direction Method of Multipliers ADMM is a powerful and efficient distributed optimisation method, thanks to its superior robustness and convergence (Boyd et al., 2011). To illustrate the idea of ADMM, consider the following optimisation problem p  fi (xi ) minimize (27) i=1 i z = 0, i = 1, · · · , p subject to xi − E where xi ∈ Rpi denotes the local variable; z ∈ Rp denotes i is a matrix that maps the input the global variable; E plan to the corresponding global components. The augmented Lagrangian of (27) is p  Lρi (xi , z, γi ) Lρ (x, z, γ) = i=1

=

p 

i z) + ρ ||xi − E i z||2 ] [fi (xi ) + γiT (xi − E 2 i=1

(28)

For problem (27), ADMM iteratively uses three steps to solve it, with q th iteration defined as xiq+1 = arg min Lρi (xi , z q , γiq ) z q+1 = arg min Lρ (x q+1 , z, γ q ) i z q+1 ) γ q+1 = γ q + ρ(x q+1 − E i

i

i

(29) (30) (31)

The most attractive feature of ADMM is the superior convergence property: while solving p local optimisation problems distributively, the algorithm’s convergence performance is still guaranteed for any positive step size ρ. For more detailed proof and analysis, please refer to Boyd et al. (2011). 4.2 Distributed Implementation of Algorithm 1 To apply ADMM for the P2P consensus tracking problem, we separate (19) into p local cost functions p  uk+1 ) = Ji,k+1 (ˆ ui,k+1 ) (32) Jk+1 (ˆ i=1

ui,k+1 ) is the local cost function for ith where Ji,k+1 (ˆ subsystem, defined as  P 2 J i,k+1 (ˆ ui,k+1 ) = || (GP i ui,k+1 − Gj uj,k+1 )|| j∈Ni (33) P P 2 2 + ||dii (r − Gi ui,k+1 )|| + ||ui,k+1 − ui,k || and dii = dii ⊗ IM , u ˆi,k+1 is the local input plan for agent itself and its neighbours, e.g., if Ni = {l, m}, then u ˆi,k+1 = [ui,k+1 T

ul,k+1 T

um,k+1 T ]T

(34)

295

Algorithm 3 Distributed NOILC Algorithm for P2P Consensus Tracking Problem Input: System matrices Ai , Bi , Ci , reference -accessibility matrix D, Laplacian matrix L, P2P reference signal rP , maximum ILC trial kmax , maximum ADMM iteration qmax , step size ρ Output: Optimal input ui,kmax for each subsystem 1: Initialization: Set the trial number k = 0 2: Repeat: 3: Set k = k + 1, q = 0 4: Repeat: 5: Set q = q + 1 , i = 0 6: Repeat: 7: Set i = i + 1 8: Receive information from neighbouring agents q+1 minimization using updating law (35) 9: u ˆi,k+1 q+1 10: zi,k+1 minimization using updating law (36) q+1 11: γi,k+1 minimization using updating law (37) 12: Send information to neighbouring agents 13: Until i = p 14: Until q = qmax qmax 15: Convert local input plan u ˆi,k+1 into ui,k+1 16: Until k = kmax 17: Return: Optimal input ui,kmax for each subsystem By applying ADMM to implement Algorithm 1 distributively, the following distributed algorithm is obtained: Algorithm 2. At ILC trial k + 1, the local input sequence q+1 {ˆ ui,k+1 }q≥0 defined as follows T q+1 q q+1 i z q ) = arg min[γi,k+1 (ˆ ui,k+1 −E u ˆi,k+1 k+1 (35) ρ q+1 q+1 i z q ||2 + Ji,k+1 (ˆ ui,k+1 − E ui,k+1 )] + ||ˆ k+1 2  1 q+1 q+1 zi,k+1 = (ˆ uo,k+1 )i (36) 1 + |Ni |  o∈(Ni

q+1 γi,k+1

=

q γi,k+1

+

i)

q+1 ρ(ˆ ui,k+1

i z q+1 ) −E k+1

(37)

provides a distributed implementation for problem (17), i.e. q = arg min{Jk+1 (ˆ uk+1 )} (38) lim zk+1 q→∞

q+1 where (ˆ uo,k+1 )i denotes all the local input plans that related to zi,k+1 ; zi,k+1 denotes ith global variable; |Ni | is the total neighbourhood’s amount of subsystem i.

4.3 Distributed NOILC Algorithm Using Algorithm 2 to solve the optimisation problem (17) gives an efficient way to implement the proposed NOILC in a distributed manner, as shown in Algorithm 3. For Algorithm 3, it contains kmax ILC trials and each ILC trial has qmax ADMM iterations. By repetitively performing max step 5–13 for qmax times, the local input plan u ˆqi,k+1 is considered to be the optimal solution at ILC trial k + 1 (since ADMM guarantees the resulting input converges to the optimal solution). Although infinite iterations are required for ADMM to approach the centralised result in theory, a small qmax is often sufficient in most of the cases. Later simulations will verify this.

Bin Chen et al. / IFAC PapersOnLine 52-29 (2019) 292–297

296

Reference

1

2

4

3

5 6

7

Zoom In

Fig. 1. The graph structure of numerical example 5. NUMERICAL EXAMPLE In this session, we use a seven-agent system (p = 7) to verify the effectiveness of Algorithm 3. Each subsystem’s dynamics is chosen to be a non-minimum phase system with transfer function s−3 (39) Gi (s) = 0.1s2 + 0.6s + τi where τi = i, i = 1, 2, · · · , 7. The graph structure of the numerical example is shown in Figure 1 (only agent 1 and 2 know the reference signal) and the weight of each edge is chosen to be 1. All the agents’ initial input and initial conditions are set to zero. Assuming the trial length is 5s, the sampling time Ts = 0.05s (with a zero-order hold). All the subsystems are required to track five reference points, with the intermediate time instant set defined as follows (40) Λ = [20 40 60 80 100]T and the P2P reference signal rP is defined as rP = [−0.9511 0.5878 0.5878 − 0.9511 0]T (41)

Fig. 2. Convergence of tracking error norm over 100 trials

Fig. 3. Output of different subsystems on 200th trial

For Algorithm 3, its convergence speed will be affected by the maximum ADMM iteration qmax , and therefore we will investigate the effect of qmax . The maximum ILC trial kmax is set to 200; step size ρ is set to 1 and the maximum ADMM iteration qmax is chosen to be 2,5,10,15,40, respectively. Figure 2 compares the distributed algorithm with the centralised solution over the first 100 ILC trials. From the figure, we can find that both of them achieve monotonic convergence of the P2P consensus tracking error norm to 0, which verifies the result in Theorem 1. Furthermore, the convergence speed of the P2P consensus tracking error norm becomes faster as the increases of the maximum ADMM iteration qmax . However, this improvement is marginal: after qmax ≥ 15, the distributed result is indistinguishable from the centralised result, which demonstrates that ADMM is efficient in most of the cases. Figure 3 shows all the agents’ output on 200th ILC trial, demonstrating that all the agents can achieve the perfect tracking of prescribed reference points (even the reference information is only known by agent 1 and 2). Figure 4 shows the evolution of the input energy consumption over 200 ILC trials. For this numerical example, the optimal input solution ||ˆ u∗ ||2 = 33.5424 and the result obtained from Algorithm 3 does converge to the optimal solution as ILC trial increases. This phenomenon verifies Theorem 2.

Fig. 4. Convergence of input energy cost over 200 trials 6. CONCLUSION AND FUTURE WORK In this paper, we propose a norm optimal ILC algorithm to solve the P2P consensus tracking problem of networked dynamical systems, which has not been explored in the literature. The resulting algorithm guarantees the P2P consensus tracking error norm converges monotonically



Bin Chen et al. / IFAC PapersOnLine 52-29 (2019) 292–297

to zero and the control input converges to the minimum energy solution. Moreover, this elaborate algorithm can be applied to non-minimum phase systems and both homogeneous and heterogeneous networked dynamical systems. To avoid the huge computational complexity when dealing with large scale networked dynamical systems, we further propose a distributed implementation for the proposed NOILC algorithm using ADMM. Rigorous analysis of the algorithm’s properties is given and numerical simulations are provided to verify the algorithm’s effectiveness.

297

Li, J. and Li, J. (2014b). Adaptive fuzzy iterative learning control with initial-state learning for coordination control of leader-following multi-agent systems. Fuzzy sets and Systems, 248, 122–137. Lv, Y., Chi, R., and Feng, Y. (2018). Adaptive estimationbased TILC for the finite-time consensus control of nonlinear discrete-time MASs under directed graph. IET Control Theory & Applications, 12(18), 2516–2525. Meng, D. and Jia, Y. (2011). Finite-time consensus for multi-agent systems via terminal feedback iterative learning. IET control theory & applications, 5(18), 2098– For the sake of simplicity, this paper considers linear time2110. invariant, discrete time networked dynamical systems. Meng, D. and Jia, Y. (2012). Iterative learning approaches However, all the results in this paper can be easily applied to design finite-time consensus protocols for multi-agent to more general cases, e.g. linear time-varying, continuous systems. Systems & Control Letters, 61(1), 187–194. time networked dynamical systems. Future works include Meng, D., Jia, Y., and Du, J. (2013). Multi-agent iterative the handling of system constraints and rigorous robustness learning control with communication topologies dynamanalysis, which will be reported separately. ically changing in two directions. IET Control Theory & Applications, 7(2), 261–270. Meng, D., Jia, Y., and Du, J. (2014). Finite-time consensus ACKNOWLEDGEMENTS protocols for networks of dynamic agents by terminal iterative learning. International Journal of Systems This work was partially supported by the ZZU-Southampton Science, 45(11), 2435–2446. Collaborative Research Project 16306/01, and Royal SociMeng, D., Jia, Y., and Du, J. (2015a). Robust consensus ety International Exchanges Scheme IE161369. tracking control for multiagent systems with initial state shifts, disturbances, and switching topologies. IEEE Transactions on Neural Networks and Learning SysREFERENCES tems, 26(4), 809–824. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., Meng, D., Jia, Y., and Du, J. (2015b). Robust iterative et al. (2011). Distributed optimization and statistical learning protocols for finite-time consensus of multilearning via the alternating direction method of mulagent systems with interval uncertain topologies. Intipliers. Foundations and Trends in Machine learning, ternational Journal of Systems Science, 46(5), 857–871. 3(1), 1–122. Meng, D., Jia, Y., Du, J., and Yu, F. (2012). Tracking Bullo, F. (2018). Lectures on Network Systems. CreateScontrol over a finite interval for multi-agent systems with pace, 1 edition. a time-varying reference trajectory. Systems & Control Chen, B. and Chu, B. (2019a). Distributed norm optimal Letters, 61(7), 807–818. iterative learning control for formation of networked Olfati-Saber, R., Fax, J.A., and Murray, R.M. (2007). dynamical systems. In Conference on Decision and Consensus and cooperation in networked multi-agent Control (CDC), to appear 2019. IEEE. systems. Proceedings of the IEEE, 95(1), 215–233. Chen, B. and Chu, B. (2019b). Distributed norm opti- Shen, D. and Xu, J.X. (2018). Distributed learning mal iterative learning control for networked dynamical consensus for heterogenous high-order nonlinear multisystems. In 2019 American Control Conference (ACC), agent systems with output constraints. Automatica, 97, 2879–2884. IEEE. 64–72. Devasia, S. (2017). Iterative control for networked hetero- Shen, D., Zhang, C., and Xu, J.X. (2019). Distributed geneous multi-agent systems with uncertainties. IEEE learning consensus control based on neural networks for Transactions on Automatic Control, 62(1), 431–437. heterogeneous nonlinear multiagent systems. InternaHatonen, J.J., Owens, D.H., and Moore, K.L. (2004). tional Journal of Robust and Nonlinear Control, 29(13), An algebraic approach to iterative learning control. 4328–4347. International Journal of Control, 77(1), 45–54. Yang, S. and Xu, J.X. (2016). Leader–follower synchroJin, X. (2016). Adaptive iterative learning control for highnisation for networked lagrangian systems with uncerorder nonlinear multi-agent systems consensus tracking. tainties: a learning approach. International Journal of Systems & Control Letters, 89, 16–23. Systems Science, 47(4), 956–965. Knorn, S., Chen, Z., and Middleton, R.H. (2016). Yang, S., Xu, J.X., and Li, X. (2016). Iterative learning Overview: Collective control of multiagent systems. control with input sharing for multi-agent consensus IEEE Transactions on Control of Network Systems, tracking. Systems & Control Letters, 94, 97–106. 3(4), 334–347. Yang, S., Xu, J.X., Li, X., and Shen, D. (2017). Iterative Li, J. and Li, J. (2014a). Adaptive iterative learning Learning Control for Multi-agent Systems Coordination. control for coordination of second-order multi-agent John Wiley & Sons. systems. International Journal of Robust and Nonlinear Control, 24(18), 3282–3299. Li, J. and Li, J. (2016). Distributed adaptive fuzzy iterative learning control of coordination problems for higher order multi-agent systems. International Journal of Systems Science, 47(10), 2318–2329.