- Email: [email protected]
A Fault Tolerant Protocol for Clock Synchronization in Sensor Networks
6th IFAC Workshop on Distributed Estimation and Control in 6th 6th IFAC IFAC Workshop Workshop on Distributed Distributed Estimation Estimation and and Control Control in in Networked Systemson 6th IFAC Workshop on Distributed Estimation and Control in Networked Systems Available online at www.sciencedirect.com Networked Systems September 8-9, 2016. Tokyo, Japan Networked Systems September 8-9, 2016. Tokyo, Japan September September 8-9, 8-9, 2016. 2016. Tokyo, Tokyo, Japan Japan
ScienceDirect
IFAC-PapersOnLine 49-22 (2016) 181–186
A Fault Tolerant Protocol for Clock A Fault Tolerant Protocol for A Fault TolerantinProtocol for Clock Clock Synchronization Sensor Networks Synchronization in Sensor Networks Synchronization in Sensor Networks
Yuhei Kikuya ∗∗ Hideaki Ishii ∗∗ Yuhei Kikuya ∗∗ Hideaki Ishii ∗∗ Yuhei Yuhei Kikuya Kikuya Hideaki Hideaki Ishii Ishii ∗ Department of Computer Science, Tokyo Institute of Technology, ∗ ∗ Department of Computer Science, Tokyo Institute of Technology, ∗ Department of Computer Science, Tokyo Institute Yokohama 226-8502, Japan Department of Computer Science, Tokyo Institute of of Technology, Technology, Yokohama 226-8502, Japan Yokohama 226-8502, Japan Yokohama 226-8502, Japan Abstract: In wireless sensor networks, accurate clock synchronization among the sensor nodes Abstract: In wireless sensor networks, accurate clock synchronization among the sensor nodes Abstract: In sensor networks, clock among the nodes is critical, e.g., to maintain consistency in measurement data. We study a fully distributed Abstract: In wireless wireless sensor consistency networks, accurate accurate clock synchronization synchronization among the sensor sensor nodes is critical, e.g., to maintain in measurement data. We study a fully distributed is critical, e.g., to maintain consistency in measurement data. We study a fully distributed approach for realizing clock synchronization based on consensus-type algorithms. In distributed particular, is critical, e.g., to maintain consistency in measurement data. We study a fully approach for realizing clock synchronization based on consensus-type algorithms. In particular, approach for synchronization based on In we construct a robust clock scheme in a setting where some nodes may bealgorithms. faulty or even malicious approach for realizing realizing clock synchronization basedsome on consensus-type consensus-type algorithms. In particular, particular, we construct a robust scheme in a setting where nodes may be faulty or even malicious we construct a robust scheme in a setting where some nodes may be faulty or even malicious due to cyber attacks and behave differently from the predefined update protocols. For this we construct a robust scheme in a setting where some nodes may be faulty or even malicious due to attacks behave differently from the update protocols. For due to cyber cyber attacksa and and behave differently from consensus the predefined predefined update protocols. For this this purpose, we extend recently developed resilient algorithm where the non-faulty due to cyber attacks and behave differently from the predefined update protocols. For this purpose, we extend a recently resilient consensus where non-faulty purpose, we extend a recently developed resilient consensus algorithm where the non-faulty nodes ignore part of the collecteddeveloped data taking extreme values. Italgorithm is shown that forthe resilient clock purpose, we extend a recently developed resilient consensus algorithm where the non-faulty nodes ignore partthe of the collected data taking extreme values. It isproperty shown that for resilient clock nodes ignore of collected data extreme It shown for resilient clock synchronization, network of agents must possess graphical known robustness. nodes ignore part partthe of the the collected data taking taking extremeaa values. values. It is isproperty shown that that for as resilient clock synchronization, network of agents must possess graphical known as robustness. synchronization, the network of agents must possess a graphical property known as robustness. synchronization, the network of agents must possess a graphical property known as robustness. © 2016, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Clock synchronization, Sensor networks, Cyber security, Resilient consensus Keywords: Keywords: Clock Clock synchronization, synchronization, Sensor Sensor networks, networks, Cyber Cyber security, security, Resilient Resilient consensus consensus Keywords: Clock synchronization, Sensor networks, Cyber security, Resilient consensus 1. INTRODUCTION clock synchronization. While such protocols can accom1. clock synchronization. such can accom1. INTRODUCTION INTRODUCTION clock synchronization. While such protocols can accommodate changes in theWhile number of protocols nodes, one 1. INTRODUCTION clock synchronization. While such protocols canweakness accomchanges in the number of nodes, one weakness Many applications employing wireless sensor networks modate modate changes in the number of nodes, one weakness is that if any of the nodes become faulty and start to modate changes in the number of nodes, one weakness Many applications employing wireless sensor networks Many applications applications employing wireless sensor networks networks that if any of the nodes become faulty and start to (WSNs) require highly precise clock synchronization among is Many employing wireless sensor is that if any of the nodes become faulty and start to send wrong data of their clocks, all other nodes will be that if any of of thetheir nodes become faulty nodes and start to (WSNs) require precise clock among (WSNs) require highly preciseconsistency clock synchronization synchronization among is send wrong data clocks, all other will be the sensor nodeshighly to maintain in their collected (WSNs) require highly precise clock synchronization among send wrong data of their clocks, all other nodes will be easily influenced. This may lead to difficulties in arriving send wrong data of their clocks, all other nodes will be the sensor nodes to maintain consistency in their collected the sensor nodes to maintain consistency in their collected influenced. This may lead to difficulties in arriving datasensor (see, nodes e.g., (Wu et al. (2011))). Suchinissues are par- easily the to maintain consistency their collected easily influenced. This at proper synchronization. easily influenced. This may may lead lead to to difficulties difficulties in in arriving arriving data e.g., et (2011))). Such are data (see, (see,important, e.g., (Wu (Wu for et al. al. (2011))). Such issues issues are parparproper synchronization. ticularly example, in health monitoring of at data (see, e.g., (Wu et al. (2011))). Such issues are parat proper synchronization. at proper synchronization. ticularly important, for example, in health monitoring of ticularly important, for example, inbuildings; health monitoring monitoring of In this paper, we propose a robust protocol for clock large structures such for as bridges andin there, based ticularly important, example, health of In this paper, we propose a robust protocol for clock large structures bridges buildings; there, based this propose protocol for large structures such such as bridges and andfrom buildings; there, based synchronization by incorporating a resilient consensus alon measurements of as acceleration multiple points in In In this paper, paper, we we propose aa robust robust protocol for clock clock large structures such as bridges and buildings; there, based synchronization by incorporating aa resilient consensus alon measurements of acceleration from multiple points in synchronization by incorporating resilient consensus alon measurements of acceleration from multiple points in gorithm which can withstand faults and malicious atthe measurements structures, mode analysis can be carried out (see, e.g., synchronization by incorporating a resilient consensus alon of acceleration from multiple points in gorithm which can withstand faults and malicious atthe structures, mode analysis can be carried out (see, e.g., gorithm which can withstand faults and malicious atthe structures, mode analysis can be carried out (see, e.g., tacks. As the consensus algorithm, we follow the approach, Kim et al. (2007)). Moreover, for the purpose of reducing gorithm which can withstand faults and malicious atthe structures, mode analysis can be carried out (see, e.g., tacks. As the the consensus algorithm, we follow follow the approach, approach, Kim (2007)). Moreover, for As consensus algorithm, we the Kim et et al. al. (2007)). Moreover, accurate for the the purpose purpose of reducing sometimes called the mean subsequence reduced (MSR) battery power consumption, clocks of arereducing useful, tacks. tacks. As the consensus algorithm, we follow the approach, Kim et al. (2007)). Moreover, for the purpose of reducing sometimes called the mean subsequence reduced (MSR) battery power consumption, accurate clocks are useful, sometimes called the mean subsequence reduced (MSR) battery power consumption, accurate clocks are useful, algorithms, called studied inmean the area of computer science for enabling power the nodes to schedule their transmission times sometimes the subsequence reduced (MSR) battery consumption, accurate clocks are useful, algorithms, studied in the area of computer science for enabling the nodes to schedule their transmission times algorithms, studied in the area of computer science for enabling the nodes to schedule their transmission times several decades (see, e.g., Lynch (1997); Azadmanesh and and to remain idle otherwise. algorithms, studied in the area of computer science for enabling the nodes to schedule their transmission times several decades (see, e.g., Lynch (1997); Azadmanesh and and to remain idle otherwise. several decades (see, e.g., Lynch (1997); Azadmanesh and and to remain idle otherwise. Kiechafer (2002)). In this approach, the non-faulty nodes several decades (see, e.g., Lynch (1997); Azadmanesh and and to remain idle otherwise. (2002)). this approach, the non-faulty nodes In WSNs, clock synchronization among the sensor nodes Kiechafer Kiechafer (2002)). In this the nodes ignore some of theIn received from their neighbors (2002)). In data this approach, approach, the non-faulty non-faulty nodes In WSNs, synchronization among sensor nodes In necessary WSNs, clock clock synchronization among the the sensor nodes Kiechafer ignore some of the data received from their neighbors is because the clocks exhibit small differences In WSNs, clock synchronization among the sensor nodes ignore some of the data received from their neighbors which take the mostdata extreme values. Totheir thisneighbors end, we ignore some of the received from is necessary because the clocks exhibit small differences is necessary necessary because the clocks clocks exhibit small differences take the most extreme values. To this end, in their speeds. Usually, equipping them withdifferences GPSs is which is because the exhibit small which take the extreme values. To this we provide a variant of the algorithms (LeBlanc et we al. which take the most most extreme values.in To this end, end, we in their speeds. Usually, equipping them with GPSs is in their speeds. Usually, equipping them with GPSs is provide a variant of the algorithms in (LeBlanc et al. not an option in order to keep the sensor nodes small in in their speeds. Usually, equipping them with GPSs is provide a variant of the algorithms in (LeBlanc et (2013); Dibaji and Ishii (2015a,b)) that take account of a variant ofIshii the(2015a,b)) algorithmsthat in (LeBlanc et al. al. not order to the nodes not an an option incost. order to keep keep the sensor sensor nodes small small in in provide (2013); Dibaji and take account of size andoption low inin Hence, various clock synchronization not an option in order to keep the sensor nodes small in (2013); Dibaji and Ishii (2015a,b)) that take account of the characteristics in the(2015a,b)) distributedthat protocols for clock (2013); Dibaji and Ishii take account of size and low in cost. Hence, various clock synchronization size and and low lowhave in cost. cost. Hence, various clock for synchronization characteristics in the distributed protocols for clock algorithms been Hence, designed specifically WSNs in the the size in various clock synchronization the characteristics in the distributed protocols for clock synchronization. In in particular, this requires us tofor consider the characteristics the distributed protocols clock algorithms have been designed specifically for WSNs in the algorithms have beenFlooding designedprotocols specifically forvery WSNs in the the synchronization. particular, requires us to consider past decadehave or so. arefor common algorithms been designed specifically WSNs in synchronization. In particular, this requires to consensus in theIn of this attenuating noise, whose synchronization. Inpresence particular, this requires us us to consider consider past decade or so. Flooding protocols are very common past decade or so. Flooding protocols are very common consensus in the presence of attenuating noise, whose where a tree-structured network of nodes is built with its past decade or so. Flooding protocols are very common consensus in the presence of attenuating noise, whose analysis becomes rather complicated. consensus in the presence of attenuating noise, whose where aa tree-structured network of built with where tree-structured network of nodes nodes is built with its its analysis becomes rather complicated. root being the chosen reference node (Mar` ois ti et al. (2004)). where a tree-structured network of nodes is built with its analysis becomes rather complicated. analysis becomes rather complicated. root being the chosen reference node (Mar` o ti et al. (2004)). root time beingofthe the chosen reference node (Mar` ti et etdown al. (2004)). (2004)). The this reference nodenode is then passed along The paper is organized as follows. In Section 2, we present root being chosen reference (Mar` ooti al. The paper organized as In 2, present The time of this reference node is then passed down along paper is is organizedalgorithm as follows. follows.under In Section Section 2, we wemeasurepresent The time of this reference node is then passed down along aThe resilient consensus decaying the tree, and other nodes adjust their clocks to that time. The paper is organized as follows. In Section 2, we present The timeand of this reference node istheir thenclocks passedtodown along a resilient consensus algorithm under decaying measurethe tree, other nodes adjust that time. a resilient consensus algorithm under decaying measurethe tree, tree, and other other nodes adjust their clocks against to that that losses time. ament noises. This forms the basis for the proposed clock This approach however may lack their robustness resilient consensus algorithm under decaying measurethe and nodes adjust clocks to time. ment noises. This forms the basis the proposed clock This approach however lack against ment noises. This forms the basis for the proposed clock This approach however may may lack robustness robustness against losses approach studied infor Section 3. Finally, we of nodes and especially the reference node since thelosses tree synchronization ment noises. This forms the basis for the proposed clock This approach however may lack robustness against losses synchronization approach studied in Section 3. Finally, we of nodes and especially the reference node since the tree synchronization approach studied in Section 3. Finally, of nodes and especially the reference node since the tree provide concluding remarks in Section 4. needs to be reconstructed. approach studied in Section 3. Finally, we we of nodes andreconstructed. especially the reference node since the tree synchronization provide concluding remarks in Section 4. needs to be provide concluding concluding remarks remarks in in Section Section 4. 4. needs to to be be reconstructed. reconstructed. provide needs On the other hand, in recent years, fully distributed, 2. RESILIENT CONSENSUS UNDER NOISE On in On the the other other hand, hand, in recent recent years, fully distributed, consensus-based protocols have years, gainedfully muchdistributed, attention 2. RESILIENT RESILIENT CONSENSUS UNDER UNDER NOISE On the other hand, in recent years, fully distributed, 2. consensus-based protocols have gained much attention 2. RESILIENT CONSENSUS CONSENSUS UNDER NOISE NOISE consensus-based protocols have gained much attention (Solis et al. (2006); Carli have et al.gained (2011);much Schenato and 2.1 Preliminaries consensus-based protocols attention (Solis et al. (2006); Carli et al. (2011); Schenato and 2.1 Preliminaries (Solis et et al. al. (2006); Carli et(2014); al. (2011); (2011); Schenato and 2.1 Fiorentin (2011); He Carli et al. et Kadowaki and Ishii (Solis (2006); al. Schenato and 2.1 Preliminaries Preliminaries Fiorentin (2011); He al. and the network consisting of n nodes described as FiorentinThere, (2011);nodes He et et al. (2014); (2014); Kadowaki and Ishii Ishii (2015)). broadcast theirKadowaki current times and Consider Fiorentin (2011); He et al. (2014); Kadowaki and Ishii Consider the network consisting of n nodes described (2015)). There, nodes broadcast their current times and Consider the network consisting of described as the directed G = (V, E). Here, V = {1, . . . , n} as is (2015)). There, nodes broadcast their current times and Consider the graph network consisting of n n nodes nodes described as data to their neighbors to achieve robust and accurate (2015)). There,neighbors nodes broadcast their current times and the directed graph G = (V, E). Here, V = {1, . . . , n} is data to their to achieve robust and accurate the directed graph G = (V, E). Here, V = {1, . . . , n} is node set whereas E ⊆ V × V is the edge set. The data to their neighbors to achieve robust and accurate the directed graph G = (V, E). Here, V = {1, . . . , n} is data to their neighbors to achieve robust and accurate the node set whereas E ⊆ V × V is the edge set. The the node set whereas E ⊆ V × V is the edge set. The edge (j, i) ∈ E indicates that node j can send information 1 This work was supported in part by the JST-CREST Program the node set whereas E ⊆ V × V is the edge set. The edge (j, E that can send 1 edge (j, i) i)i;∈ ∈such E indicates indicates that node canincoming send information information supported in part for by the JST-CREST Program 1 This to node an edge is node calledjjj an edge of work was supported in by JST-CREST Program edge (j, i) ∈ E indicates that node can send information and by work JSPSwas under Grant-in-Aid Scientific Research Grant 1 This This work was supported in part part for by the the JST-CREST Program to node i; such an edge is called an incoming edge of and by JSPS under Grant-in-Aid Scientific Research Grant to node i; such an edge is called an incoming edge of node i. The set of neighbors for node i is denoted and by JSPS under Grant-in-Aid for Scientific Research Grant No. 15H04020. to node i; such an edge is called an incoming edge by of and by JSPS under Grant-in-Aid for Scientific Research Grant node i. The set of neighbors for node iiwritten is denoted by No. 15H04020. node i. The set of neighbors for node is denoted by No. 15H04020. N = {j : (j, i) ∈ E}; its cardinality is as |N E-mails: [email protected], [email protected] i i |. node i. The set of neighbors for node i is denoted by No. 15H04020. N = {j : (j, i) ∈ E}; its cardinality is written as |N |. E-mails: [email protected], [email protected] N E-mails: Niii = = {j {j :: (j, (j, i) i) ∈ ∈ E}; E}; its its cardinality cardinality is is written written as as |N |Niii |. |. E-mails: [email protected], [email protected], [email protected] [email protected]
Copyright © 2016, 2016 IFAC 181Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright 2016 IFAC 181 Copyright © 2016 IFAC 181 Peer review© of International Federation of Automatic Copyright ©under 2016 responsibility IFAC 181Control. 10.1016/j.ifacol.2016.10.393
2016 IFAC NECSYS 182 September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
For the graph G, the adjacency matrix A = [aij ] is given by aij ∈ [μ, 1)�if (j, i) ∈ E and aij = 0 otherwise. It is n assumed that j=1 aij < 1, and μ > 0 is a fixed lower bound for the non-zero entries. We introduce the notion of robust graphs (LeBlanc et al. (2013)). Definition 1. For the graph G = (V, E), if for each pair of nonempty disjoint subsets V1 , V2 ⊂ V satisfies one of the following conditions, it is called (r, s)-robust (r, s < n): � r � � � � � � � �XV � = |V1 |, �XVr � = |V2 |, �XVr � + �XVr � ≥ s. 1 2 1 2 Here, XVr� denotes the set of nodes in V� having at least r incoming edges from outside of V� . In particular, graphs which are (r, 1)-robust are called r-robust. It is noted that if the graph G is (r� + s − 1)-robust, then it is also (r� , s)-robust. 2.2 Multi-Agent Consensus under Decaying Noises Consider the multi-agent system with n agents whose network structure is given by the graph G (Mesbahi and Egerstedt (2010)). The interactions of the agents over this graph are assumed to be time varying. Each node i has the update scheme given by xi (k + 1) = xi (k) + ui (k), (1) where xi (k) and ui (k) are, respectively, the state and the input at time k ∈ Z+ . Agent i generates its input ui based on the data it receives from its neighbors. In this problem, however, we consider the case where the data contains noise which decays to zero over time. Specifically, agent i uses information received from each neighbor node j as xj (k) + wij (k), where wij (k) is the noise. It is assumed that the noise attenuates exponentially fast. That is, for some b ≥ 0 and ξ ∈ (0, 1), it holds for each (j, i) ∈ E |wij (k)| ≤ bξ k , k ∈ Z+ .
(2)
Regarding the interactions among the agents, we employ an asynchronous model. At each time k, only a subset of nodes makes updates and then those nodes will send their new states to the neighbors. Let U(k) be the set of such updating nodes at time k. The network used at time k is denoted by G(k). When node j does not send data to node i at time k, the corresponding entry aij (k) in the adjacency matrix A[k] is set to zero. Equivalently, we write Ni (k) to express the neighbors of agent i at time k. We assume that each agent makes an update at least once ¯ k0 +k−1 in k¯ time steps. So it holds that ∪k=k U(k) = V for 0 k0 ∈ Z+ . For node i, the input ui (k) is given by ⎧ � aij (k) {[xj (k) + wij (k)] − xi (k)} ⎪ ⎪ ⎨ j∈Ni (k) (3) ui (k) = if i ∈ U (k), ⎪ ⎪ ⎩ 0 otherwise.
�n trix whose ith entry di (k) is given by di (k) = �=1 ai� (k). n×n Then, let P (k) ∈ R be P (k) = I − D(k) + A(k). This matrix P (k) is row stochastic: Each entry is nonnegative and the sum of each row is equal to 1. �n Letting the noise w(k) be [w(k)]i = j=1 aij (k)wij (k), we can express the system (1) and (3) in vector form as x(k + 1) = P (k)x(k) + w(k), (4) Note that by (2), the noise vector w(k) is also exponentially decaying and its upper bound can be obtained as �w(k)� ≤ cξ k for some c > 0. 2.3 Resilient Consensus We describe the malicious nodes studied in this paper (e.g., LeBlanc et al. (2013); Dibaji and Ishii (2015b)). Malicious nodes may take arbitrary states at each time due to faults and/or external cyber attacks, potentially with intentions to severely affect the system behavior. Definition 2. If node i applies the input according to (3), then it is said to be normal. Otherwise, the node is called malicious. Denote the set of malicious nodes by M ⊂ V. The malicious node i ∈ M follows the update scheme in (1) with arbitrary input ui (k). As the class of malicious agents, we employ the so-called f -local model. Definition 3. The graph G is called f -local if within the neighbor set Ni of node i, the number of malicious nodes is no greater than f , i.e., |Ni ∩ M| ≤ f for i ∈ V\M. We now provide the notion of resilient consensus. Definition 4. Under the f -local model, if for any initial state values, any malicious inputs, any possible set of malicious agents, and any scenario of updating sets, the following two conditions hold, then the normal agents are said to achieve resilient consensus: (1) Safety condition: There exists a bounded interval S determined by the initial states of the normal agents and the bound on the noise such that x(k) ∈ S for i ∈ V \ M, k ∈ Z+ . (2) Consensus condition: There exists x∗ ∈ S such that xi (k) → x∗ as k → ∞ for i ∈ V \ M. To simplify the notation, we reorder the node indices so that the normal nodes are 1, 2, . . . , nN and the malicious nodes nN + 1, . . . , n. Then, the overall update scheme including the malicious nodes becomes � N � � N � � � x (k + 1) x (k) 0 = P (k) +w(k)+ uM (k). (5) InM xM (k + 1) xM (k)
We use the superscript N for the normal parts and M for the malicious parts. The normal nodes follow (4) while the malicious nodes follow (1) but may use arbitrary input M uM (k) ∈ Rn . For ease of notation, in (5), we keep all terms in P (k) and w(k), even those corresponding to the Notice that though the nodes make updates asynchronously, malicious nodes; clearly, these can be canceled by properly when node i does so at time k, it has the current noisy modifying the input uM (k). state values xj (k) + wij (k) of all of its neighbors. This is because when each node execute an update, its new state 2.4 Resilient Consensus Algorithm is immediately broadcasted as described earlier. We outline the algorithm for achieving resilient consensus, Now, we introduce the vector form expression of the multi- consisting of four steps. It will be referred to as the mean agent system. First, let D(k) ∈ Rn×n be the diagonal ma- subsequence reduced (MSR) algorithm. 182
2016 IFAC NECSYS September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
1. At time k, if normal node i ∈ V \M makes an update, i.e., i ∈ U (k), then it sorts the state values of its neighbors received most recently and the current state of itself in a decreasing order. 2. If the number of nodes taking values greater than xi (k) is less than f , then they are all ignored. Otherwise, the f values counting from the largest are ignored. 3. Similarly, if the number of nodes taking values less than xi (k) is less than f , then they are all ignored. Otherwise, the f values counting from the smallest are ignored. 4. Apply the input in (3) by setting aij (k) = 0 for the nodes j ignored in steps 2 and 3 above. The strategy of this algorithm is to ignore the neighbors possessing the extreme values regardless of its true identity of being normal or not. Clearly, each node requires to have at least 2f + 1 neighbors. As we see next, it turns out that simply having many neighbors is not enough to guarantee this algorithm to properly function, but certain structure in terms of graph robustness is necessary. 2.5 Consensus in the Presence of Malicious Nodes and Noise We are ready to state the main theorem of this section. For the safety interval, we use the following one: c c N N S = min xi (0) − , max xi (0) + . (6) i i 1−ξ 1−ξ Theorem 1. Under the f -local model, if the network of agents is (2f + 1)-robust, then with the MSR algorithm, the normal agents achieve resilient consensus. The safety interval is given by (6). Moreover, the rate of convergence to the consensus value x∗ is exponential. Conversely, if the normal agents achieve resilient consensus, then the network is (f + 1, f + 1)-robust. The proof is omitted due to space limitations. The theorem above is an extension of the result in (Dibaji and Ishii (2015b)), where the main difference lies in the presence of the decaying noise; by setting c = 0, the original result can be recovered. While the proof follows a relatively close line of arguments, it is much more technical. For example, due to the noise and the malicious behavior in the system, it is not straightforward to even establish the convergence of the upper bound maxi xN i (k) on the normal agents’ values. In the setting of (Dibaji and Ishii (2015b)), this can be easily shown since this bound becomes a decreasing function of time. To address this issue and moreover to establish that consensus among the normal agents has an exponential rate, we have changed the structure of the proof. We also note that exponential convergence is critical in the clock synchronization problem studied in the next section. 3. APPLICATION TO CLOCK SYNCHRONIZATION We propose a distributed algorithm for clock synchronization in WSNs when some of the nodes may have faulty or attacked clocks. It will be demonstrated that the MSR algorithm from the previous section can be applied to robustify the clocks. In particular, the decaying noise arises due to the interaction between two variables that each agent has and need to reach consensus. 183
183
3.1 Clock Models for WSNs We first present the models of clocks employed in this study (Schenato and Fiorentin (2011); Kadowaki and Ishii (2015)). Consider the network of sensor nodes connected over the graph G. Let t ∈ R+ denote the absolute time. Each node i is equipped with an internal clock whose time τi∗ (t) is an affine function of t as τi∗ (t) = αi∗ t + βi∗ , (7) ∗ where αi is the clock drift and βi∗ is the clock offset. These parameters are assumed to lie within some ranges, which will cause the differences among the clocks over the time. We assume that αi∗ is normalized around 1 and is bounded as |αi∗ − 1| ≤ σα We remark that the absolute time t is not accessible and hence the true values of the parameters αi∗ and βi∗ are unknown in general. For achieving clock synchronization, each node i computes and maintains the adjusted time τi (t) using its own internal time τi∗ (t) as τi (t) = αi (t)τi∗ (t) + βi (t), (8) where αi (t) and βi (t) are, respectively, the new parameters corresponding to the drift and offset, whose update schemes are to be introduced. We say that the sensor network achieves clock synchronization if the adjusted clocks of all nodes agree asymptotically as limt→∞ τi (t) − τj (t) = 0 for i, j ∈ V. The problem of this paper is to construct distributed update schemes for αi (t) and βi (t) for the sensor network to achieve clock synchronization. By (7) and (8), the adjusted time can be written as τi (t) = αi (t)αi∗ t + αi (t)βi∗ + βi (t) =: xi (t)t + yi (t), (9) where xi (t) := αi (t)αi∗ is called the modified drift and yi (t) := αi (t)βi∗ + βi (t) is the modified offset. It is now clear that clock synchronization is equivalent to having consensus among these variables, i.e., for i, j ∈ V, it holds lim xi (t) − xj (t) = 0, lim yi (t) − yj (t) = 0. t→∞
t→∞
3.2 Communication Protocols for Sensor Nodes
We outline the protocol for the communication among the sensor nodes. The nodes in the network exchange the current values of their parameters with their neighbors. Specifically, each node i makes updates in its parameters and then the new values will be transmitted immediately afterwards. This is done periodically with the fixed period T based on its own internal clock. This implies that for node i, the period in absolute time is Ti := T /αi∗ . The information to be sent includes its index i, the internal time τi∗ (t), the modified time τi (t), and the states αi (t) and βi (t). After receiving information from node j, node i stores the received data as well as the following: Its own internal time τi∗ (t) and the adjusted time τi (t) at the time t of reception. Old information of node j will be discarded. Note that node i makes a transmission each time it updated its states αi (t) and βi (t). So the neighbors always have the latest values of them. Since the nodes are not synchronized, we must keep track of the instants in absolute time when any of the nodes make transmissions/updates. These instants are denoted by tk with k ∈ Z+ in an increasing order tk < tk+1 . With
2016 IFAC NECSYS 184 September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
a slight abuse of notation, we indicate the time using the discrete time index k. So, for example, τi (k) means τi (tk ). As the nodes periodically make updates, each node transmits at least once in T /(mini αi∗ ) in absolute time, which can be translated to some discrete time step Δ ≥ 0. 3.3 Updates in the Drifts When a normal node i ∈ V\M updates its drift αi (k) at time k, it follows ρα {ηij αj (k) − αi (k)} , (10) αi (k + 1) = αi (k) + j∈Ni (k)
where ρα ∈ (0, 1/(maxi |Ni |)) is a tuning parameter; increasing its value can result in faster convergence. Note that the data αj (k) used above by node i is in fact received at some time k − Δij (k) in the past, where the delay time satisfies Δij (k) ∈ [0, Δ]. However, due to the asynchronous model employed here, the nodes make at least one update every Δ, so it actually holds that xj (k − Δij (k)) = xj (k). This relation will implicitly be used in the following as well. The parameter ηij is an estimate of the ratio αi ∗ /αj ∗ between the true drifts. This can in fact be obtained at node i via τj∗ (k) − τj∗ (k � ) ηij = ∗ , τi (k) − τi∗ (k � )
where k � < k denote the time instants when node i received data from node j. Hence, we assume that the true value is obtained as ηij = αi∗ /αj∗ by the time when the synchronization algorithm starts. It is noted that in (Schenato and Fiorentin (2011)), an iterative algorithm is proposed to reduce the effects of potential noises in τj∗ (k). On the other hand, the malicious node i ∈ M can update its value any time in an arbitrary manner. Hence, we have αi (k + 1) = αi (k) + uα,i (k),
(11)
where uα,i (k) is the input chosen arbitrarily. In this setting of clock synchronization, the MSR algorithm can be applied in the following manner: At each updating time k, normal node i ∈ U (k) computes ηij αj (k) for all of its neighbors j ∈ Ni . It then follows the MSR algorithm outlined in Section 2.4 by first sorting these values and comparing them with its own state αi (k). We are in the position to state the resilient consensus result for the clock drifts based on the MSR algorithm. Proposition 1. Suppose that every normal node i in the sensor network updates its drift αi (k) by the scheme in (10) using the MSR algorithm and the malicious nodes by (11). If the underlying network is (2f + 1)-robust, then the sensor network achieves resilient consensus in their modified drifts xi (k) = αi (k)αi∗ . Proof : To show that the MSR algorithm result is applicable, we must transform the update schemes (10) and (11) into those comparable to (5). For the normal nodes, multiply αi∗ on both sides of the equality of (10), and then add/subtract j∈Ni (k) αj (k)αj∗ on the right-hand side to obtain 184
αi (k + 1)αi∗ = αi (k)αi∗ +
j∈Ni (k)
ρα ηij αj (k)αi∗ − αi (k)αi∗
+ αj (k)αj∗ − αj (k)αj∗ = αi (k)αi∗ + ρα αj (k)αj∗ − αi (k)αi∗ j∈Ni (k)
α∗ + ηij i − 1 αj (k)αj∗ αj ∗ ρα αj (k)αj∗ − αi (k)αi∗ . = αi (k)αi +
Substituting xi (k) = yields
j∈Ni (k) αi (k)αi∗ from
xi (k + 1) = xi (k) +
j∈Ni (k)
(9) into the above
ρα xj (k) − xi (k) .
We see that this update scheme is in the form of (1) and (3). It is also clear that the malicious nodes can change their values αi (k + 1)αi∗ freely through (11). Hence, the system is in the form (5), and Theorem 1 is applicable (without the decaying noise, i.e., c = 0). Consequently, xi (k) or equivalently αi (k)αi∗ for i ∈ V\M arrive at resilient consensus. Observe that in the proof, we transformed the update scheme of the drifts in the standard form for consensus. We remark that this form with the state xi (k) = αi (k)αi∗ is exploited only in the analysis and is not needed in its implementation. This is an important aspect since the drift αi∗ of the internal clock is unknown in general. We also note that the update scheme (10) for the normal nodes is slightly different from the one in (Schenato and Fiorentin (2011)), where updates in αi (k) is performed pairwise with αj (k) each time the value of node j arrives at node i. In contrast, in our scheme (10), the values of all neighbors are used at once when an update takes place. This is necessary to run the MSR algorithm, which must find the largest and smallest values from the neighbors’. 3.4 Updates in the Offsets At time k, if a normal node i ∈ V\M initiates an update in its values, its offset βi (k) is changed as βi (k+1) = βi (k)+ ρβ τj (k−Δij (k))−τi (k−Δij (k)) , j∈Ni (k)
(12) where ρβ ∈ 0, 1/(maxi |Ni |) is a tuning parameter. The delay times Δij (k) ∈ [0, Δ] represent the time difference between the transmission time of node j and the updating time of node i as discussed earlier. Malicious node i ∈ M can change its offset to any value via its input uβ,i (k) as βi (k + 1) = βi (k) + uβ,i (k). (13) To apply the MSR algorithm in Section 2.4 for the offset values, normal node i uses the difference in adjusted times with its neighbor nodes j ∈ Ni at the time of data reception, i.e., τj (k −Δij (k))−τi (k −Δij (k)), and compare these values in step 2 of the MSR algorithm; here, the value corresponding to node i itself becomes 0 at all times. The following result is the counterpart of Proposition 1 for the offset values and is the main result of this paper. Theorem 2. Suppose that every normal node i in the sensor network updates its offset βi (k) by the scheme in
2016 IFAC NECSYS September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
185
500
2
3
4
5
6
7
Comm. range of node 1 8
9
10
11
12
13
14
15
16
400
tauhat
1
300 200 100 0
50
100
150
200
250 t
300
350
400
450
500
0
50
100
150
200
250 t
300
350
400
450
500
102
max-min
Fig. 1. Network structure of sensor nodes 800 600
tau
0
400
101
100
10-1
200 0
0
50
100
150
200
250 t
300
350
400
450
500
Fig. 2. Internal clocks τi∗ (t)
Fig. 3. Adjusted clocks τi (t) under the conventional scheme (i.e., with f = 0) yi (k + 1) = yi (k) + ρα {yj (k) − yi (k) + wβ,ij (k)} . j∈Ni (k)
(12) based on the MSR algorithm and the malicious nodes by (13). If the underlying network is (2f + 1)-robust, then the sensor network achieves resilient consensus in their modified offsets yi (k) = αi (k)βi∗ + βi (k). Proof : Similarly to the proof of Proposition 1, we show that the update schemes (12) for the normal nodes and (13) for the malicious nodes can be reduced to (5). For updates, normal node i uses information received from its neighbor j after they make updates at time τi (k − Δij (k)) in its own clock. According to the asynchronous communication protocol, this time can be represented as τi (k − Δij (k)) = αi (k)αi∗ tk−Δij (k) + αi (k)βi∗ + βi (k).
The same holds for τj (k − Δij (k)). By noting these relations, substitute (9) into (12), and then add αi (k + 1)βi∗ on both sides. Further, add and subtract αi (k)βi∗ on the right-hand side to obtain
+
ρβ
j∈Ni (k)
αj (k)αj∗ − αi (k)αi∗ tk−Δij (k) .
Substitute (9) into this to obtain ρβ {yj (k) − yi (k)} yi (k + 1) = yi (k) + j∈Ni (k)
β∗ + i∗ {xi (k + 1) − xi (k)} αi + ρβ (xj (k) − xi (k))tk−Δij (k) . j∈Ni (k)
(14)
Moreover, letting wβ,ij (k) :=
We note that having shown exponential convergence in the modified drifts xi is essential in the proof above. This is because the differences among these variables are treated as noise in the system for the modified offsets. In fact, observe that in (14), the noise wβ,i (k) contains the term (xj (k) − xi (k))tk−Δij (k) where the time tk−Δij (k) clearly grows linearly. 3.5 Numerical Example We present simulation results to illustrate the effectiveness of the proposed MSR-based algorithm for resilient clock synchronization. We use the sensor network of 16 nodes depicted in Fig. 1. The nodes are located in the 4-by-4 grid topology with unit distance in both x- and y-axes. The wireless communication range of each node is set to be with radius 3. It can be shown that the network is a 7-robust graph. For the MSR algorithm, we take f = 3. By Proposition 1 and Theorem 2, resilient time synchronization can be realized. The clock parameters αi∗ and βi∗ are chosen randomly with σα = 0.3. Parameters in the update schemes are chosen as T = 1, ρα = 0.01, and ρβ = 0.01. The initial states are set as αi (0) = 1 and βi (0) = 0 for all i, that is, τi (0) = τi∗ (0). We compare the performance of the proposed resilient scheme with that of the non-resilient conventional approach (by setting f = 0 in the proposed scheme).
βi (k + 1) + αi (k + 1)βi∗ = βi (k) + αi (k + 1)βi∗ + αi (k)βi∗ − αi (k)βi∗ + ρβ αj (k)βj∗ + βj (k) − αi (k)βi∗ + βi (k) j∈Ni (k)
By Proposition 1, the rate of reaching consensus in xi (k) is exponential. Hence, there exist by ≥ 0 and ξy ∈ (0, 1) such that |wβ,ij |(k) ≤ by ξyk for all i, j. Thus, we can apply Theorem 1 to conclude that yi (k) for i ∈ V \ M achieve resilient consensus.
βi∗ {xi (k + 1) − xi (k)} αi∗ · |Ni (k)| + ρβ (xj (k) − xi (k))tk−Δij (k) ,
we can rewrite (14) as
185
Nodes 2, 7, and 12 are chosen as the malicious nodes. They exhibit the following two types of irregular behaviors: (i) The states αi (k) and βi (k) of nodes 2 and 7 remain constant and do not change from their initial values at all times. (ii) Node 12 resets its states at t = 300 to their initial values. In Fig. 2, we show the internal clocks τi∗ (t) of the 16 nodes. We observe that the differences in the clocks linearly grow over time and that the clock of node 12 switches to zero at time t = 300.
2016 IFAC NECSYS 186 September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
600
500
500
400
tauhat
tauhat
400 300 200
50
100
150
200
250 t
300
350
400
450
0
500
102
102
100
101
max-min
max-min
0
200 100
100 0
300
10-2
10-4
0
50
100
150
200
250 t
300
350
400
450
50
100
150
200
250 t
300
350
400
450
500
0
50
100
150
200
250 t
300
350
400
450
500
100
10-1
500
0
Fig. 4. Adjusted clocks τi (t) under the proposed scheme with f = 3
Fig. 5. Adjusted clocks τi (t) under the proposed scheme with f = 2 over a non-robust graph
When the conventional scheme is used, time responses of the adjusted clocks are shown in Fig. 3. The top plot gives τi (t) and the bottom plot gives the maximum difference |τi (t) − τj (t)| in log scale over all normal agents. Observe that the normal nodes are affected by the faulty ones and achieve only a limited level of synchronization.
Dibaji, S. and Ishii, H. (2015a). Consensus of secondorder multi-agent system in presence of locally faults. Systems & Control Letters, 79, 23–29. Dibaji, S. and Ishii, H. (2015b). Resilient multi-agent consensus with asynchrony and delayed information. In Proc. 5th IFAC Workshop on Distributed Estimation and Control in Networked Systems (NecSys’15). He, J., Cheng, P., Shi, L., Chen, J., and Sun, Y. (2014). Time synchronization in WSNs: A maximum-valuebased consensus approach. IEEE Trans. Autom. Control, 59, 660–675. Kadowaki, Y. and Ishii, H. (2015). Event-based distributed clock synchronization for wireless sensor networks. IEEE Trans. Autom. Control, 60, 2266–2271. Kim, S., Pakzad, S., Culler, D., Demmel, J., Fenves, G., Glaser, S., and Toron, M. (2007). Health monitoring of civil infrastructures using wireless sensor networks. In Proc. 6th Int. Symp. Information Processing in Sensor Networks, 254–263. LeBlanc, H., Zhang, H., Koutsoukos, X., and Sundaram, S. (2013). Resilient asymptotic consensus in robust networks. IEEE J. Selected Areas Comm., 31, 766– 781. Lynch, N. (1997). Distributed Algorithms. Morgan Kaufmann, San Francisco, CA. Mar` oti, M., Kusy, B., Simon, G., and L`edeczi, A. (2004). The flooding time synchronization protocol. In Proc. 2nd ACM Conf. Embedded Networked Sensor Systems, 39–49. Mesbahi, M. and Egerstedt, M. (2010). Graph Theoretic Methods in Multiagent Networks. Princeton University Press. Schenato, L. and Fiorentin, F. (2011). Average TimeSynch: A consensus-based protocol for clock synchronization in wireless sensor networks. Automatica, 47, 1878–1886. Solis, R., Borkar, V., and Kumar, P.R. (2006). A new distributed time synchronization protocol for multihop wireless networks. In Proc. 45th IEEE Conf. on Decision and Control, 2734–2739. Wu, Y.C., Chaudhari, Q., and Serpedin, E. (2011). Clock synchronization of wireless sensor networks. IEEE Signal Processing Mag., 47(1), 124–138.
On the other hand, when the proposed scheme is applied, the responses of the modified clocks are presented in Fig. 4. It is clear that normal agents arrive at synchronization, without being influenced by the fixed clocks of nodes 2 and 7 and the clock of node 12 reset at time 300. In fact, in the bottom plot, we observe exponential convergence over time. Finally, we ran the proposed scheme over a graph with less connectivity. By reducing the communication range from 3 to 2.3, the graph becomes 6-robust. This is not enough to secure the system when three malicious nodes are present. The simulation result shown in Fig. 5 exhibits that the clocks are very much affected by the reset of node 12 that occurred at time 300. 4. CONCLUSION We have studied an MSR-based algorithm for resilient consensus for a network of agents where malicious agents may influence the overall system behavior. In particular, we have analyzed the case when decaying noise is present in the update scheme using the notion of robust graphs. The algorithm has then been applied to the distributed clock synchronization for sensor networks. In the future, we will study the case when delays are present in the information used in the updates. Acknowledgement: The authors would like to thank Seyed Mehran Dibaji for the helpful discussions. REFERENCES Azadmanesh, M.H. and Kiechafer, R.M. (2002). Asynchronous approximate agreement in partially connected networks. Int. J. Parallel and Distributed Networks, 5, 26–34. Carli, R., Chiuso, A., Schenato, L., and Zampieri, S. (2011). Optimal synchronization for networks of noisy double integrators. IEEE Trans. Autom. Control, 56, 1146–1152. 186
ScienceDirect
IFAC-PapersOnLine 49-22 (2016) 181–186
A Fault Tolerant Protocol for Clock A Fault Tolerant Protocol for A Fault TolerantinProtocol for Clock Clock Synchronization Sensor Networks Synchronization in Sensor Networks Synchronization in Sensor Networks
Yuhei Kikuya ∗∗ Hideaki Ishii ∗∗ Yuhei Kikuya ∗∗ Hideaki Ishii ∗∗ Yuhei Yuhei Kikuya Kikuya Hideaki Hideaki Ishii Ishii ∗ Department of Computer Science, Tokyo Institute of Technology, ∗ ∗ Department of Computer Science, Tokyo Institute of Technology, ∗ Department of Computer Science, Tokyo Institute Yokohama 226-8502, Japan Department of Computer Science, Tokyo Institute of of Technology, Technology, Yokohama 226-8502, Japan Yokohama 226-8502, Japan Yokohama 226-8502, Japan Abstract: In wireless sensor networks, accurate clock synchronization among the sensor nodes Abstract: In wireless sensor networks, accurate clock synchronization among the sensor nodes Abstract: In sensor networks, clock among the nodes is critical, e.g., to maintain consistency in measurement data. We study a fully distributed Abstract: In wireless wireless sensor consistency networks, accurate accurate clock synchronization synchronization among the sensor sensor nodes is critical, e.g., to maintain in measurement data. We study a fully distributed is critical, e.g., to maintain consistency in measurement data. We study a fully distributed approach for realizing clock synchronization based on consensus-type algorithms. In distributed particular, is critical, e.g., to maintain consistency in measurement data. We study a fully approach for realizing clock synchronization based on consensus-type algorithms. In particular, approach for synchronization based on In we construct a robust clock scheme in a setting where some nodes may bealgorithms. faulty or even malicious approach for realizing realizing clock synchronization basedsome on consensus-type consensus-type algorithms. In particular, particular, we construct a robust scheme in a setting where nodes may be faulty or even malicious we construct a robust scheme in a setting where some nodes may be faulty or even malicious due to cyber attacks and behave differently from the predefined update protocols. For this we construct a robust scheme in a setting where some nodes may be faulty or even malicious due to attacks behave differently from the update protocols. For due to cyber cyber attacksa and and behave differently from consensus the predefined predefined update protocols. For this this purpose, we extend recently developed resilient algorithm where the non-faulty due to cyber attacks and behave differently from the predefined update protocols. For this purpose, we extend a recently resilient consensus where non-faulty purpose, we extend a recently developed resilient consensus algorithm where the non-faulty nodes ignore part of the collecteddeveloped data taking extreme values. Italgorithm is shown that forthe resilient clock purpose, we extend a recently developed resilient consensus algorithm where the non-faulty nodes ignore partthe of the collected data taking extreme values. It isproperty shown that for resilient clock nodes ignore of collected data extreme It shown for resilient clock synchronization, network of agents must possess graphical known robustness. nodes ignore part partthe of the the collected data taking taking extremeaa values. values. It is isproperty shown that that for as resilient clock synchronization, network of agents must possess graphical known as robustness. synchronization, the network of agents must possess a graphical property known as robustness. synchronization, the network of agents must possess a graphical property known as robustness. © 2016, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Keywords: Clock synchronization, Sensor networks, Cyber security, Resilient consensus Keywords: Keywords: Clock Clock synchronization, synchronization, Sensor Sensor networks, networks, Cyber Cyber security, security, Resilient Resilient consensus consensus Keywords: Clock synchronization, Sensor networks, Cyber security, Resilient consensus 1. INTRODUCTION clock synchronization. While such protocols can accom1. clock synchronization. such can accom1. INTRODUCTION INTRODUCTION clock synchronization. While such protocols can accommodate changes in theWhile number of protocols nodes, one 1. INTRODUCTION clock synchronization. While such protocols canweakness accomchanges in the number of nodes, one weakness Many applications employing wireless sensor networks modate modate changes in the number of nodes, one weakness is that if any of the nodes become faulty and start to modate changes in the number of nodes, one weakness Many applications employing wireless sensor networks Many applications applications employing wireless sensor networks networks that if any of the nodes become faulty and start to (WSNs) require highly precise clock synchronization among is Many employing wireless sensor is that if any of the nodes become faulty and start to send wrong data of their clocks, all other nodes will be that if any of of thetheir nodes become faulty nodes and start to (WSNs) require precise clock among (WSNs) require highly preciseconsistency clock synchronization synchronization among is send wrong data clocks, all other will be the sensor nodeshighly to maintain in their collected (WSNs) require highly precise clock synchronization among send wrong data of their clocks, all other nodes will be easily influenced. This may lead to difficulties in arriving send wrong data of their clocks, all other nodes will be the sensor nodes to maintain consistency in their collected the sensor nodes to maintain consistency in their collected influenced. This may lead to difficulties in arriving datasensor (see, nodes e.g., (Wu et al. (2011))). Suchinissues are par- easily the to maintain consistency their collected easily influenced. This at proper synchronization. easily influenced. This may may lead lead to to difficulties difficulties in in arriving arriving data e.g., et (2011))). Such are data (see, (see,important, e.g., (Wu (Wu for et al. al. (2011))). Such issues issues are parparproper synchronization. ticularly example, in health monitoring of at data (see, e.g., (Wu et al. (2011))). Such issues are parat proper synchronization. at proper synchronization. ticularly important, for example, in health monitoring of ticularly important, for example, inbuildings; health monitoring monitoring of In this paper, we propose a robust protocol for clock large structures such for as bridges andin there, based ticularly important, example, health of In this paper, we propose a robust protocol for clock large structures bridges buildings; there, based this propose protocol for large structures such such as bridges and andfrom buildings; there, based synchronization by incorporating a resilient consensus alon measurements of as acceleration multiple points in In In this paper, paper, we we propose aa robust robust protocol for clock clock large structures such as bridges and buildings; there, based synchronization by incorporating aa resilient consensus alon measurements of acceleration from multiple points in synchronization by incorporating resilient consensus alon measurements of acceleration from multiple points in gorithm which can withstand faults and malicious atthe measurements structures, mode analysis can be carried out (see, e.g., synchronization by incorporating a resilient consensus alon of acceleration from multiple points in gorithm which can withstand faults and malicious atthe structures, mode analysis can be carried out (see, e.g., gorithm which can withstand faults and malicious atthe structures, mode analysis can be carried out (see, e.g., tacks. As the consensus algorithm, we follow the approach, Kim et al. (2007)). Moreover, for the purpose of reducing gorithm which can withstand faults and malicious atthe structures, mode analysis can be carried out (see, e.g., tacks. As the the consensus algorithm, we follow follow the approach, approach, Kim (2007)). Moreover, for As consensus algorithm, we the Kim et et al. al. (2007)). Moreover, accurate for the the purpose purpose of reducing sometimes called the mean subsequence reduced (MSR) battery power consumption, clocks of arereducing useful, tacks. tacks. As the consensus algorithm, we follow the approach, Kim et al. (2007)). Moreover, for the purpose of reducing sometimes called the mean subsequence reduced (MSR) battery power consumption, accurate clocks are useful, sometimes called the mean subsequence reduced (MSR) battery power consumption, accurate clocks are useful, algorithms, called studied inmean the area of computer science for enabling power the nodes to schedule their transmission times sometimes the subsequence reduced (MSR) battery consumption, accurate clocks are useful, algorithms, studied in the area of computer science for enabling the nodes to schedule their transmission times algorithms, studied in the area of computer science for enabling the nodes to schedule their transmission times several decades (see, e.g., Lynch (1997); Azadmanesh and and to remain idle otherwise. algorithms, studied in the area of computer science for enabling the nodes to schedule their transmission times several decades (see, e.g., Lynch (1997); Azadmanesh and and to remain idle otherwise. several decades (see, e.g., Lynch (1997); Azadmanesh and and to remain idle otherwise. Kiechafer (2002)). In this approach, the non-faulty nodes several decades (see, e.g., Lynch (1997); Azadmanesh and and to remain idle otherwise. (2002)). this approach, the non-faulty nodes In WSNs, clock synchronization among the sensor nodes Kiechafer Kiechafer (2002)). In this the nodes ignore some of theIn received from their neighbors (2002)). In data this approach, approach, the non-faulty non-faulty nodes In WSNs, synchronization among sensor nodes In necessary WSNs, clock clock synchronization among the the sensor nodes Kiechafer ignore some of the data received from their neighbors is because the clocks exhibit small differences In WSNs, clock synchronization among the sensor nodes ignore some of the data received from their neighbors which take the mostdata extreme values. Totheir thisneighbors end, we ignore some of the received from is necessary because the clocks exhibit small differences is necessary necessary because the clocks clocks exhibit small differences take the most extreme values. To this end, in their speeds. Usually, equipping them withdifferences GPSs is which is because the exhibit small which take the extreme values. To this we provide a variant of the algorithms (LeBlanc et we al. which take the most most extreme values.in To this end, end, we in their speeds. Usually, equipping them with GPSs is in their speeds. Usually, equipping them with GPSs is provide a variant of the algorithms in (LeBlanc et al. not an option in order to keep the sensor nodes small in in their speeds. Usually, equipping them with GPSs is provide a variant of the algorithms in (LeBlanc et (2013); Dibaji and Ishii (2015a,b)) that take account of a variant ofIshii the(2015a,b)) algorithmsthat in (LeBlanc et al. al. not order to the nodes not an an option incost. order to keep keep the sensor sensor nodes small small in in provide (2013); Dibaji and take account of size andoption low inin Hence, various clock synchronization not an option in order to keep the sensor nodes small in (2013); Dibaji and Ishii (2015a,b)) that take account of the characteristics in the(2015a,b)) distributedthat protocols for clock (2013); Dibaji and Ishii take account of size and low in cost. Hence, various clock synchronization size and and low lowhave in cost. cost. Hence, various clock for synchronization characteristics in the distributed protocols for clock algorithms been Hence, designed specifically WSNs in the the size in various clock synchronization the characteristics in the distributed protocols for clock synchronization. In in particular, this requires us tofor consider the characteristics the distributed protocols clock algorithms have been designed specifically for WSNs in the algorithms have beenFlooding designedprotocols specifically forvery WSNs in the the synchronization. particular, requires us to consider past decadehave or so. arefor common algorithms been designed specifically WSNs in synchronization. In particular, this requires to consensus in theIn of this attenuating noise, whose synchronization. Inpresence particular, this requires us us to consider consider past decade or so. Flooding protocols are very common past decade or so. Flooding protocols are very common consensus in the presence of attenuating noise, whose where a tree-structured network of nodes is built with its past decade or so. Flooding protocols are very common consensus in the presence of attenuating noise, whose analysis becomes rather complicated. consensus in the presence of attenuating noise, whose where aa tree-structured network of built with where tree-structured network of nodes nodes is built with its its analysis becomes rather complicated. root being the chosen reference node (Mar` ois ti et al. (2004)). where a tree-structured network of nodes is built with its analysis becomes rather complicated. analysis becomes rather complicated. root being the chosen reference node (Mar` o ti et al. (2004)). root time beingofthe the chosen reference node (Mar` ti et etdown al. (2004)). (2004)). The this reference nodenode is then passed along The paper is organized as follows. In Section 2, we present root being chosen reference (Mar` ooti al. The paper organized as In 2, present The time of this reference node is then passed down along paper is is organizedalgorithm as follows. follows.under In Section Section 2, we wemeasurepresent The time of this reference node is then passed down along aThe resilient consensus decaying the tree, and other nodes adjust their clocks to that time. The paper is organized as follows. In Section 2, we present The timeand of this reference node istheir thenclocks passedtodown along a resilient consensus algorithm under decaying measurethe tree, other nodes adjust that time. a resilient consensus algorithm under decaying measurethe tree, tree, and other other nodes adjust their clocks against to that that losses time. ament noises. This forms the basis for the proposed clock This approach however may lack their robustness resilient consensus algorithm under decaying measurethe and nodes adjust clocks to time. ment noises. This forms the basis the proposed clock This approach however lack against ment noises. This forms the basis for the proposed clock This approach however may may lack robustness robustness against losses approach studied infor Section 3. Finally, we of nodes and especially the reference node since thelosses tree synchronization ment noises. This forms the basis for the proposed clock This approach however may lack robustness against losses synchronization approach studied in Section 3. Finally, we of nodes and especially the reference node since the tree synchronization approach studied in Section 3. Finally, of nodes and especially the reference node since the tree provide concluding remarks in Section 4. needs to be reconstructed. approach studied in Section 3. Finally, we we of nodes andreconstructed. especially the reference node since the tree synchronization provide concluding remarks in Section 4. needs to be provide concluding concluding remarks remarks in in Section Section 4. 4. needs to to be be reconstructed. reconstructed. provide needs On the other hand, in recent years, fully distributed, 2. RESILIENT CONSENSUS UNDER NOISE On in On the the other other hand, hand, in recent recent years, fully distributed, consensus-based protocols have years, gainedfully muchdistributed, attention 2. RESILIENT RESILIENT CONSENSUS UNDER UNDER NOISE On the other hand, in recent years, fully distributed, 2. consensus-based protocols have gained much attention 2. RESILIENT CONSENSUS CONSENSUS UNDER NOISE NOISE consensus-based protocols have gained much attention (Solis et al. (2006); Carli have et al.gained (2011);much Schenato and 2.1 Preliminaries consensus-based protocols attention (Solis et al. (2006); Carli et al. (2011); Schenato and 2.1 Preliminaries (Solis et et al. al. (2006); Carli et(2014); al. (2011); (2011); Schenato and 2.1 Fiorentin (2011); He Carli et al. et Kadowaki and Ishii (Solis (2006); al. Schenato and 2.1 Preliminaries Preliminaries Fiorentin (2011); He al. and the network consisting of n nodes described as FiorentinThere, (2011);nodes He et et al. (2014); (2014); Kadowaki and Ishii Ishii (2015)). broadcast theirKadowaki current times and Consider Fiorentin (2011); He et al. (2014); Kadowaki and Ishii Consider the network consisting of n nodes described (2015)). There, nodes broadcast their current times and Consider the network consisting of described as the directed G = (V, E). Here, V = {1, . . . , n} as is (2015)). There, nodes broadcast their current times and Consider the graph network consisting of n n nodes nodes described as data to their neighbors to achieve robust and accurate (2015)). There,neighbors nodes broadcast their current times and the directed graph G = (V, E). Here, V = {1, . . . , n} is data to their to achieve robust and accurate the directed graph G = (V, E). Here, V = {1, . . . , n} is node set whereas E ⊆ V × V is the edge set. The data to their neighbors to achieve robust and accurate the directed graph G = (V, E). Here, V = {1, . . . , n} is data to their neighbors to achieve robust and accurate the node set whereas E ⊆ V × V is the edge set. The the node set whereas E ⊆ V × V is the edge set. The edge (j, i) ∈ E indicates that node j can send information 1 This work was supported in part by the JST-CREST Program the node set whereas E ⊆ V × V is the edge set. The edge (j, E that can send 1 edge (j, i) i)i;∈ ∈such E indicates indicates that node canincoming send information information supported in part for by the JST-CREST Program 1 This to node an edge is node calledjjj an edge of work was supported in by JST-CREST Program edge (j, i) ∈ E indicates that node can send information and by work JSPSwas under Grant-in-Aid Scientific Research Grant 1 This This work was supported in part part for by the the JST-CREST Program to node i; such an edge is called an incoming edge of and by JSPS under Grant-in-Aid Scientific Research Grant to node i; such an edge is called an incoming edge of node i. The set of neighbors for node i is denoted and by JSPS under Grant-in-Aid for Scientific Research Grant No. 15H04020. to node i; such an edge is called an incoming edge by of and by JSPS under Grant-in-Aid for Scientific Research Grant node i. The set of neighbors for node iiwritten is denoted by No. 15H04020. node i. The set of neighbors for node is denoted by No. 15H04020. N = {j : (j, i) ∈ E}; its cardinality is as |N E-mails: [email protected], [email protected] i i |. node i. The set of neighbors for node i is denoted by No. 15H04020. N = {j : (j, i) ∈ E}; its cardinality is written as |N |. E-mails: [email protected], [email protected] N E-mails: Niii = = {j {j :: (j, (j, i) i) ∈ ∈ E}; E}; its its cardinality cardinality is is written written as as |N |Niii |. |. E-mails: [email protected], [email protected], [email protected] [email protected]
Copyright © 2016, 2016 IFAC 181Hosting by Elsevier Ltd. All rights reserved. 2405-8963 © IFAC (International Federation of Automatic Control) Copyright 2016 IFAC 181 Copyright © 2016 IFAC 181 Peer review© of International Federation of Automatic Copyright ©under 2016 responsibility IFAC 181Control. 10.1016/j.ifacol.2016.10.393
2016 IFAC NECSYS 182 September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
For the graph G, the adjacency matrix A = [aij ] is given by aij ∈ [μ, 1)�if (j, i) ∈ E and aij = 0 otherwise. It is n assumed that j=1 aij < 1, and μ > 0 is a fixed lower bound for the non-zero entries. We introduce the notion of robust graphs (LeBlanc et al. (2013)). Definition 1. For the graph G = (V, E), if for each pair of nonempty disjoint subsets V1 , V2 ⊂ V satisfies one of the following conditions, it is called (r, s)-robust (r, s < n): � r � � � � � � � �XV � = |V1 |, �XVr � = |V2 |, �XVr � + �XVr � ≥ s. 1 2 1 2 Here, XVr� denotes the set of nodes in V� having at least r incoming edges from outside of V� . In particular, graphs which are (r, 1)-robust are called r-robust. It is noted that if the graph G is (r� + s − 1)-robust, then it is also (r� , s)-robust. 2.2 Multi-Agent Consensus under Decaying Noises Consider the multi-agent system with n agents whose network structure is given by the graph G (Mesbahi and Egerstedt (2010)). The interactions of the agents over this graph are assumed to be time varying. Each node i has the update scheme given by xi (k + 1) = xi (k) + ui (k), (1) where xi (k) and ui (k) are, respectively, the state and the input at time k ∈ Z+ . Agent i generates its input ui based on the data it receives from its neighbors. In this problem, however, we consider the case where the data contains noise which decays to zero over time. Specifically, agent i uses information received from each neighbor node j as xj (k) + wij (k), where wij (k) is the noise. It is assumed that the noise attenuates exponentially fast. That is, for some b ≥ 0 and ξ ∈ (0, 1), it holds for each (j, i) ∈ E |wij (k)| ≤ bξ k , k ∈ Z+ .
(2)
Regarding the interactions among the agents, we employ an asynchronous model. At each time k, only a subset of nodes makes updates and then those nodes will send their new states to the neighbors. Let U(k) be the set of such updating nodes at time k. The network used at time k is denoted by G(k). When node j does not send data to node i at time k, the corresponding entry aij (k) in the adjacency matrix A[k] is set to zero. Equivalently, we write Ni (k) to express the neighbors of agent i at time k. We assume that each agent makes an update at least once ¯ k0 +k−1 in k¯ time steps. So it holds that ∪k=k U(k) = V for 0 k0 ∈ Z+ . For node i, the input ui (k) is given by ⎧ � aij (k) {[xj (k) + wij (k)] − xi (k)} ⎪ ⎪ ⎨ j∈Ni (k) (3) ui (k) = if i ∈ U (k), ⎪ ⎪ ⎩ 0 otherwise.
�n trix whose ith entry di (k) is given by di (k) = �=1 ai� (k). n×n Then, let P (k) ∈ R be P (k) = I − D(k) + A(k). This matrix P (k) is row stochastic: Each entry is nonnegative and the sum of each row is equal to 1. �n Letting the noise w(k) be [w(k)]i = j=1 aij (k)wij (k), we can express the system (1) and (3) in vector form as x(k + 1) = P (k)x(k) + w(k), (4) Note that by (2), the noise vector w(k) is also exponentially decaying and its upper bound can be obtained as �w(k)� ≤ cξ k for some c > 0. 2.3 Resilient Consensus We describe the malicious nodes studied in this paper (e.g., LeBlanc et al. (2013); Dibaji and Ishii (2015b)). Malicious nodes may take arbitrary states at each time due to faults and/or external cyber attacks, potentially with intentions to severely affect the system behavior. Definition 2. If node i applies the input according to (3), then it is said to be normal. Otherwise, the node is called malicious. Denote the set of malicious nodes by M ⊂ V. The malicious node i ∈ M follows the update scheme in (1) with arbitrary input ui (k). As the class of malicious agents, we employ the so-called f -local model. Definition 3. The graph G is called f -local if within the neighbor set Ni of node i, the number of malicious nodes is no greater than f , i.e., |Ni ∩ M| ≤ f for i ∈ V\M. We now provide the notion of resilient consensus. Definition 4. Under the f -local model, if for any initial state values, any malicious inputs, any possible set of malicious agents, and any scenario of updating sets, the following two conditions hold, then the normal agents are said to achieve resilient consensus: (1) Safety condition: There exists a bounded interval S determined by the initial states of the normal agents and the bound on the noise such that x(k) ∈ S for i ∈ V \ M, k ∈ Z+ . (2) Consensus condition: There exists x∗ ∈ S such that xi (k) → x∗ as k → ∞ for i ∈ V \ M. To simplify the notation, we reorder the node indices so that the normal nodes are 1, 2, . . . , nN and the malicious nodes nN + 1, . . . , n. Then, the overall update scheme including the malicious nodes becomes � N � � N � � � x (k + 1) x (k) 0 = P (k) +w(k)+ uM (k). (5) InM xM (k + 1) xM (k)
We use the superscript N for the normal parts and M for the malicious parts. The normal nodes follow (4) while the malicious nodes follow (1) but may use arbitrary input M uM (k) ∈ Rn . For ease of notation, in (5), we keep all terms in P (k) and w(k), even those corresponding to the Notice that though the nodes make updates asynchronously, malicious nodes; clearly, these can be canceled by properly when node i does so at time k, it has the current noisy modifying the input uM (k). state values xj (k) + wij (k) of all of its neighbors. This is because when each node execute an update, its new state 2.4 Resilient Consensus Algorithm is immediately broadcasted as described earlier. We outline the algorithm for achieving resilient consensus, Now, we introduce the vector form expression of the multi- consisting of four steps. It will be referred to as the mean agent system. First, let D(k) ∈ Rn×n be the diagonal ma- subsequence reduced (MSR) algorithm. 182
2016 IFAC NECSYS September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
1. At time k, if normal node i ∈ V \M makes an update, i.e., i ∈ U (k), then it sorts the state values of its neighbors received most recently and the current state of itself in a decreasing order. 2. If the number of nodes taking values greater than xi (k) is less than f , then they are all ignored. Otherwise, the f values counting from the largest are ignored. 3. Similarly, if the number of nodes taking values less than xi (k) is less than f , then they are all ignored. Otherwise, the f values counting from the smallest are ignored. 4. Apply the input in (3) by setting aij (k) = 0 for the nodes j ignored in steps 2 and 3 above. The strategy of this algorithm is to ignore the neighbors possessing the extreme values regardless of its true identity of being normal or not. Clearly, each node requires to have at least 2f + 1 neighbors. As we see next, it turns out that simply having many neighbors is not enough to guarantee this algorithm to properly function, but certain structure in terms of graph robustness is necessary. 2.5 Consensus in the Presence of Malicious Nodes and Noise We are ready to state the main theorem of this section. For the safety interval, we use the following one: c c N N S = min xi (0) − , max xi (0) + . (6) i i 1−ξ 1−ξ Theorem 1. Under the f -local model, if the network of agents is (2f + 1)-robust, then with the MSR algorithm, the normal agents achieve resilient consensus. The safety interval is given by (6). Moreover, the rate of convergence to the consensus value x∗ is exponential. Conversely, if the normal agents achieve resilient consensus, then the network is (f + 1, f + 1)-robust. The proof is omitted due to space limitations. The theorem above is an extension of the result in (Dibaji and Ishii (2015b)), where the main difference lies in the presence of the decaying noise; by setting c = 0, the original result can be recovered. While the proof follows a relatively close line of arguments, it is much more technical. For example, due to the noise and the malicious behavior in the system, it is not straightforward to even establish the convergence of the upper bound maxi xN i (k) on the normal agents’ values. In the setting of (Dibaji and Ishii (2015b)), this can be easily shown since this bound becomes a decreasing function of time. To address this issue and moreover to establish that consensus among the normal agents has an exponential rate, we have changed the structure of the proof. We also note that exponential convergence is critical in the clock synchronization problem studied in the next section. 3. APPLICATION TO CLOCK SYNCHRONIZATION We propose a distributed algorithm for clock synchronization in WSNs when some of the nodes may have faulty or attacked clocks. It will be demonstrated that the MSR algorithm from the previous section can be applied to robustify the clocks. In particular, the decaying noise arises due to the interaction between two variables that each agent has and need to reach consensus. 183
183
3.1 Clock Models for WSNs We first present the models of clocks employed in this study (Schenato and Fiorentin (2011); Kadowaki and Ishii (2015)). Consider the network of sensor nodes connected over the graph G. Let t ∈ R+ denote the absolute time. Each node i is equipped with an internal clock whose time τi∗ (t) is an affine function of t as τi∗ (t) = αi∗ t + βi∗ , (7) ∗ where αi is the clock drift and βi∗ is the clock offset. These parameters are assumed to lie within some ranges, which will cause the differences among the clocks over the time. We assume that αi∗ is normalized around 1 and is bounded as |αi∗ − 1| ≤ σα We remark that the absolute time t is not accessible and hence the true values of the parameters αi∗ and βi∗ are unknown in general. For achieving clock synchronization, each node i computes and maintains the adjusted time τi (t) using its own internal time τi∗ (t) as τi (t) = αi (t)τi∗ (t) + βi (t), (8) where αi (t) and βi (t) are, respectively, the new parameters corresponding to the drift and offset, whose update schemes are to be introduced. We say that the sensor network achieves clock synchronization if the adjusted clocks of all nodes agree asymptotically as limt→∞ τi (t) − τj (t) = 0 for i, j ∈ V. The problem of this paper is to construct distributed update schemes for αi (t) and βi (t) for the sensor network to achieve clock synchronization. By (7) and (8), the adjusted time can be written as τi (t) = αi (t)αi∗ t + αi (t)βi∗ + βi (t) =: xi (t)t + yi (t), (9) where xi (t) := αi (t)αi∗ is called the modified drift and yi (t) := αi (t)βi∗ + βi (t) is the modified offset. It is now clear that clock synchronization is equivalent to having consensus among these variables, i.e., for i, j ∈ V, it holds lim xi (t) − xj (t) = 0, lim yi (t) − yj (t) = 0. t→∞
t→∞
3.2 Communication Protocols for Sensor Nodes
We outline the protocol for the communication among the sensor nodes. The nodes in the network exchange the current values of their parameters with their neighbors. Specifically, each node i makes updates in its parameters and then the new values will be transmitted immediately afterwards. This is done periodically with the fixed period T based on its own internal clock. This implies that for node i, the period in absolute time is Ti := T /αi∗ . The information to be sent includes its index i, the internal time τi∗ (t), the modified time τi (t), and the states αi (t) and βi (t). After receiving information from node j, node i stores the received data as well as the following: Its own internal time τi∗ (t) and the adjusted time τi (t) at the time t of reception. Old information of node j will be discarded. Note that node i makes a transmission each time it updated its states αi (t) and βi (t). So the neighbors always have the latest values of them. Since the nodes are not synchronized, we must keep track of the instants in absolute time when any of the nodes make transmissions/updates. These instants are denoted by tk with k ∈ Z+ in an increasing order tk < tk+1 . With
2016 IFAC NECSYS 184 September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
a slight abuse of notation, we indicate the time using the discrete time index k. So, for example, τi (k) means τi (tk ). As the nodes periodically make updates, each node transmits at least once in T /(mini αi∗ ) in absolute time, which can be translated to some discrete time step Δ ≥ 0. 3.3 Updates in the Drifts When a normal node i ∈ V\M updates its drift αi (k) at time k, it follows ρα {ηij αj (k) − αi (k)} , (10) αi (k + 1) = αi (k) + j∈Ni (k)
where ρα ∈ (0, 1/(maxi |Ni |)) is a tuning parameter; increasing its value can result in faster convergence. Note that the data αj (k) used above by node i is in fact received at some time k − Δij (k) in the past, where the delay time satisfies Δij (k) ∈ [0, Δ]. However, due to the asynchronous model employed here, the nodes make at least one update every Δ, so it actually holds that xj (k − Δij (k)) = xj (k). This relation will implicitly be used in the following as well. The parameter ηij is an estimate of the ratio αi ∗ /αj ∗ between the true drifts. This can in fact be obtained at node i via τj∗ (k) − τj∗ (k � ) ηij = ∗ , τi (k) − τi∗ (k � )
where k � < k denote the time instants when node i received data from node j. Hence, we assume that the true value is obtained as ηij = αi∗ /αj∗ by the time when the synchronization algorithm starts. It is noted that in (Schenato and Fiorentin (2011)), an iterative algorithm is proposed to reduce the effects of potential noises in τj∗ (k). On the other hand, the malicious node i ∈ M can update its value any time in an arbitrary manner. Hence, we have αi (k + 1) = αi (k) + uα,i (k),
(11)
where uα,i (k) is the input chosen arbitrarily. In this setting of clock synchronization, the MSR algorithm can be applied in the following manner: At each updating time k, normal node i ∈ U (k) computes ηij αj (k) for all of its neighbors j ∈ Ni . It then follows the MSR algorithm outlined in Section 2.4 by first sorting these values and comparing them with its own state αi (k). We are in the position to state the resilient consensus result for the clock drifts based on the MSR algorithm. Proposition 1. Suppose that every normal node i in the sensor network updates its drift αi (k) by the scheme in (10) using the MSR algorithm and the malicious nodes by (11). If the underlying network is (2f + 1)-robust, then the sensor network achieves resilient consensus in their modified drifts xi (k) = αi (k)αi∗ . Proof : To show that the MSR algorithm result is applicable, we must transform the update schemes (10) and (11) into those comparable to (5). For the normal nodes, multiply αi∗ on both sides of the equality of (10), and then add/subtract j∈Ni (k) αj (k)αj∗ on the right-hand side to obtain 184
αi (k + 1)αi∗ = αi (k)αi∗ +
j∈Ni (k)
ρα ηij αj (k)αi∗ − αi (k)αi∗
+ αj (k)αj∗ − αj (k)αj∗ = αi (k)αi∗ + ρα αj (k)αj∗ − αi (k)αi∗ j∈Ni (k)
α∗ + ηij i − 1 αj (k)αj∗ αj ∗ ρα αj (k)αj∗ − αi (k)αi∗ . = αi (k)αi +
Substituting xi (k) = yields
j∈Ni (k) αi (k)αi∗ from
xi (k + 1) = xi (k) +
j∈Ni (k)
(9) into the above
ρα xj (k) − xi (k) .
We see that this update scheme is in the form of (1) and (3). It is also clear that the malicious nodes can change their values αi (k + 1)αi∗ freely through (11). Hence, the system is in the form (5), and Theorem 1 is applicable (without the decaying noise, i.e., c = 0). Consequently, xi (k) or equivalently αi (k)αi∗ for i ∈ V\M arrive at resilient consensus. Observe that in the proof, we transformed the update scheme of the drifts in the standard form for consensus. We remark that this form with the state xi (k) = αi (k)αi∗ is exploited only in the analysis and is not needed in its implementation. This is an important aspect since the drift αi∗ of the internal clock is unknown in general. We also note that the update scheme (10) for the normal nodes is slightly different from the one in (Schenato and Fiorentin (2011)), where updates in αi (k) is performed pairwise with αj (k) each time the value of node j arrives at node i. In contrast, in our scheme (10), the values of all neighbors are used at once when an update takes place. This is necessary to run the MSR algorithm, which must find the largest and smallest values from the neighbors’. 3.4 Updates in the Offsets At time k, if a normal node i ∈ V\M initiates an update in its values, its offset βi (k) is changed as βi (k+1) = βi (k)+ ρβ τj (k−Δij (k))−τi (k−Δij (k)) , j∈Ni (k)
(12) where ρβ ∈ 0, 1/(maxi |Ni |) is a tuning parameter. The delay times Δij (k) ∈ [0, Δ] represent the time difference between the transmission time of node j and the updating time of node i as discussed earlier. Malicious node i ∈ M can change its offset to any value via its input uβ,i (k) as βi (k + 1) = βi (k) + uβ,i (k). (13) To apply the MSR algorithm in Section 2.4 for the offset values, normal node i uses the difference in adjusted times with its neighbor nodes j ∈ Ni at the time of data reception, i.e., τj (k −Δij (k))−τi (k −Δij (k)), and compare these values in step 2 of the MSR algorithm; here, the value corresponding to node i itself becomes 0 at all times. The following result is the counterpart of Proposition 1 for the offset values and is the main result of this paper. Theorem 2. Suppose that every normal node i in the sensor network updates its offset βi (k) by the scheme in
2016 IFAC NECSYS September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
185
500
2
3
4
5
6
7
Comm. range of node 1 8
9
10
11
12
13
14
15
16
400
tauhat
1
300 200 100 0
50
100
150
200
250 t
300
350
400
450
500
0
50
100
150
200
250 t
300
350
400
450
500
102
max-min
Fig. 1. Network structure of sensor nodes 800 600
tau
0
400
101
100
10-1
200 0
0
50
100
150
200
250 t
300
350
400
450
500
Fig. 2. Internal clocks τi∗ (t)
Fig. 3. Adjusted clocks τi (t) under the conventional scheme (i.e., with f = 0) yi (k + 1) = yi (k) + ρα {yj (k) − yi (k) + wβ,ij (k)} . j∈Ni (k)
(12) based on the MSR algorithm and the malicious nodes by (13). If the underlying network is (2f + 1)-robust, then the sensor network achieves resilient consensus in their modified offsets yi (k) = αi (k)βi∗ + βi (k). Proof : Similarly to the proof of Proposition 1, we show that the update schemes (12) for the normal nodes and (13) for the malicious nodes can be reduced to (5). For updates, normal node i uses information received from its neighbor j after they make updates at time τi (k − Δij (k)) in its own clock. According to the asynchronous communication protocol, this time can be represented as τi (k − Δij (k)) = αi (k)αi∗ tk−Δij (k) + αi (k)βi∗ + βi (k).
The same holds for τj (k − Δij (k)). By noting these relations, substitute (9) into (12), and then add αi (k + 1)βi∗ on both sides. Further, add and subtract αi (k)βi∗ on the right-hand side to obtain
+
ρβ
j∈Ni (k)
αj (k)αj∗ − αi (k)αi∗ tk−Δij (k) .
Substitute (9) into this to obtain ρβ {yj (k) − yi (k)} yi (k + 1) = yi (k) + j∈Ni (k)
β∗ + i∗ {xi (k + 1) − xi (k)} αi + ρβ (xj (k) − xi (k))tk−Δij (k) . j∈Ni (k)
(14)
Moreover, letting wβ,ij (k) :=
We note that having shown exponential convergence in the modified drifts xi is essential in the proof above. This is because the differences among these variables are treated as noise in the system for the modified offsets. In fact, observe that in (14), the noise wβ,i (k) contains the term (xj (k) − xi (k))tk−Δij (k) where the time tk−Δij (k) clearly grows linearly. 3.5 Numerical Example We present simulation results to illustrate the effectiveness of the proposed MSR-based algorithm for resilient clock synchronization. We use the sensor network of 16 nodes depicted in Fig. 1. The nodes are located in the 4-by-4 grid topology with unit distance in both x- and y-axes. The wireless communication range of each node is set to be with radius 3. It can be shown that the network is a 7-robust graph. For the MSR algorithm, we take f = 3. By Proposition 1 and Theorem 2, resilient time synchronization can be realized. The clock parameters αi∗ and βi∗ are chosen randomly with σα = 0.3. Parameters in the update schemes are chosen as T = 1, ρα = 0.01, and ρβ = 0.01. The initial states are set as αi (0) = 1 and βi (0) = 0 for all i, that is, τi (0) = τi∗ (0). We compare the performance of the proposed resilient scheme with that of the non-resilient conventional approach (by setting f = 0 in the proposed scheme).
βi (k + 1) + αi (k + 1)βi∗ = βi (k) + αi (k + 1)βi∗ + αi (k)βi∗ − αi (k)βi∗ + ρβ αj (k)βj∗ + βj (k) − αi (k)βi∗ + βi (k) j∈Ni (k)
By Proposition 1, the rate of reaching consensus in xi (k) is exponential. Hence, there exist by ≥ 0 and ξy ∈ (0, 1) such that |wβ,ij |(k) ≤ by ξyk for all i, j. Thus, we can apply Theorem 1 to conclude that yi (k) for i ∈ V \ M achieve resilient consensus.
βi∗ {xi (k + 1) − xi (k)} αi∗ · |Ni (k)| + ρβ (xj (k) − xi (k))tk−Δij (k) ,
we can rewrite (14) as
185
Nodes 2, 7, and 12 are chosen as the malicious nodes. They exhibit the following two types of irregular behaviors: (i) The states αi (k) and βi (k) of nodes 2 and 7 remain constant and do not change from their initial values at all times. (ii) Node 12 resets its states at t = 300 to their initial values. In Fig. 2, we show the internal clocks τi∗ (t) of the 16 nodes. We observe that the differences in the clocks linearly grow over time and that the clock of node 12 switches to zero at time t = 300.
2016 IFAC NECSYS 186 September 8-9, 2016. Tokyo, Japan
Yuhei Kikuya et al. / IFAC-PapersOnLine 49-22 (2016) 181–186
600
500
500
400
tauhat
tauhat
400 300 200
50
100
150
200
250 t
300
350
400
450
0
500
102
102
100
101
max-min
max-min
0
200 100
100 0
300
10-2
10-4
0
50
100
150
200
250 t
300
350
400
450
50
100
150
200
250 t
300
350
400
450
500
0
50
100
150
200
250 t
300
350
400
450
500
100
10-1
500
0
Fig. 4. Adjusted clocks τi (t) under the proposed scheme with f = 3
Fig. 5. Adjusted clocks τi (t) under the proposed scheme with f = 2 over a non-robust graph
When the conventional scheme is used, time responses of the adjusted clocks are shown in Fig. 3. The top plot gives τi (t) and the bottom plot gives the maximum difference |τi (t) − τj (t)| in log scale over all normal agents. Observe that the normal nodes are affected by the faulty ones and achieve only a limited level of synchronization.
Dibaji, S. and Ishii, H. (2015a). Consensus of secondorder multi-agent system in presence of locally faults. Systems & Control Letters, 79, 23–29. Dibaji, S. and Ishii, H. (2015b). Resilient multi-agent consensus with asynchrony and delayed information. In Proc. 5th IFAC Workshop on Distributed Estimation and Control in Networked Systems (NecSys’15). He, J., Cheng, P., Shi, L., Chen, J., and Sun, Y. (2014). Time synchronization in WSNs: A maximum-valuebased consensus approach. IEEE Trans. Autom. Control, 59, 660–675. Kadowaki, Y. and Ishii, H. (2015). Event-based distributed clock synchronization for wireless sensor networks. IEEE Trans. Autom. Control, 60, 2266–2271. Kim, S., Pakzad, S., Culler, D., Demmel, J., Fenves, G., Glaser, S., and Toron, M. (2007). Health monitoring of civil infrastructures using wireless sensor networks. In Proc. 6th Int. Symp. Information Processing in Sensor Networks, 254–263. LeBlanc, H., Zhang, H., Koutsoukos, X., and Sundaram, S. (2013). Resilient asymptotic consensus in robust networks. IEEE J. Selected Areas Comm., 31, 766– 781. Lynch, N. (1997). Distributed Algorithms. Morgan Kaufmann, San Francisco, CA. Mar` oti, M., Kusy, B., Simon, G., and L`edeczi, A. (2004). The flooding time synchronization protocol. In Proc. 2nd ACM Conf. Embedded Networked Sensor Systems, 39–49. Mesbahi, M. and Egerstedt, M. (2010). Graph Theoretic Methods in Multiagent Networks. Princeton University Press. Schenato, L. and Fiorentin, F. (2011). Average TimeSynch: A consensus-based protocol for clock synchronization in wireless sensor networks. Automatica, 47, 1878–1886. Solis, R., Borkar, V., and Kumar, P.R. (2006). A new distributed time synchronization protocol for multihop wireless networks. In Proc. 45th IEEE Conf. on Decision and Control, 2734–2739. Wu, Y.C., Chaudhari, Q., and Serpedin, E. (2011). Clock synchronization of wireless sensor networks. IEEE Signal Processing Mag., 47(1), 124–138.
On the other hand, when the proposed scheme is applied, the responses of the modified clocks are presented in Fig. 4. It is clear that normal agents arrive at synchronization, without being influenced by the fixed clocks of nodes 2 and 7 and the clock of node 12 reset at time 300. In fact, in the bottom plot, we observe exponential convergence over time. Finally, we ran the proposed scheme over a graph with less connectivity. By reducing the communication range from 3 to 2.3, the graph becomes 6-robust. This is not enough to secure the system when three malicious nodes are present. The simulation result shown in Fig. 5 exhibits that the clocks are very much affected by the reset of node 12 that occurred at time 300. 4. CONCLUSION We have studied an MSR-based algorithm for resilient consensus for a network of agents where malicious agents may influence the overall system behavior. In particular, we have analyzed the case when decaying noise is present in the update scheme using the notion of robust graphs. The algorithm has then been applied to the distributed clock synchronization for sensor networks. In the future, we will study the case when delays are present in the information used in the updates. Acknowledgement: The authors would like to thank Seyed Mehran Dibaji for the helpful discussions. REFERENCES Azadmanesh, M.H. and Kiechafer, R.M. (2002). Asynchronous approximate agreement in partially connected networks. Int. J. Parallel and Distributed Networks, 5, 26–34. Carli, R., Chiuso, A., Schenato, L., and Zampieri, S. (2011). Optimal synchronization for networks of noisy double integrators. IEEE Trans. Autom. Control, 56, 1146–1152. 186