Hybrid consensus for averager–copier–voter networks with non-rational agents

Chaos, Solitons and Fractals 110 (2018) 244–251

Contents lists available at ScienceDirect

Chaos, Solitons and Fractals Nonlinear Science, and Nonequilibrium and Complex Phenomena journal homepage: www.elsevier.com/locate/chaos

Hybrid consensus for averager–copier–voter networks with non-rational agents Yilun Shang School of Mathematical Sciences, Tongji University, Shanghai 200092, China

a r t i c l e

i n f o

Article history: Received 27 January 2018 Revised 17 March 2018 Accepted 26 March 2018 Available online 4 April 2018 Keywords: Social networks Opinion dynamics Non-rational agents Voter Copier Averager

a b s t r a c t For many social dynamical systems with heterogenous communicating components there exist nonrational agents, whose full profile (such as location and number) is not accessible to the normal agents a priori, posing threats to the group goal of the community. Here we demonstrate how to provide resilience against such non-cooperative behaviors in opinion dynamics. We focus in particular on the consensus of a hybrid network consisting of continuous-valued averager, copier agents and discrete-valued voter agents, where the averagers average the opinions of their neighbors and their own deterministically, while copiers and voters update their opinions following some stochastic strategies. Based upon a filtering strategy which removes some fixed number of opinion values, we establish varied necessary and sufficient conditions for the hybrid opinion network to reach consensus in mean in the presence of globally and locally bounded non-rational agents. The communication topologies are modeled as directed fixed as well as time-dependent robust networks. Although our results are shown to be irrespective of the proportion of the averager, copier, and voters, we find that the existence of voters has distinct influence on the evolution and consensus value of the negotiation process. © 2018 Elsevier Ltd. All rights reserved.

1. Introduction Decision-making in today’s networked complex systems requires coordination of a group of heterogenous agents, which may comprise cyber, physical, and human elements. These agents naturally engage in consensus-building on certain quantities of interest, such as heading of autonomous vehicles, temperature of the environment, and opinion of the cohort, through local interaction with their neighbors. For example, in the setting of social networks, interacting agents can influence each other and gradually form common opinions by ignoring minority opinions and allowing opinion differences [1]. A number of physical models have been developed to explore human opinion propagation despite the challenge of describing and evaluating the macroscopic collective behaviors involving physiological and psychological factors. In opinion dynamics, each agent maintains an opinion, i.e., state, which can be a continuous or a discrete quantity. In the discrete case, binary opinion models have dominated research, mostly in physics literature, due to their marked analogy with spin systems. The voter model [2], the Sznajd model [3] and the Galam majority-rule model [4] are wellknown examples of discrete opinion dynamics models. There are

E-mail address: [email protected] https://doi.org/10.1016/j.chaos.2018.03.037 0960-0779/© 2018 Elsevier Ltd. All rights reserved.

also situations where the opinion of an agent should be expressable in real and may vary smoothly between the extremes. Examples include attitudes, prices, or predictions about macroeconomic variables. Among continuous opinion dynamics models, the Deffuant–Weisbuch model [5] and the Hegselmann–Krause model [6] have attracted significant attention in sociophysics. The dependence of the final opinion distribution on the confidence bound in Deffuant–Weisbuch model, for example, has been studied in directed small-world networks [7]. The effect of self-affirmation on the opinion formation relating to the phase transition of the opinion configuration has also been examined for directed small-world networks [8]. It is noteworthy that most current models of opinion dynamics and more generally, distributed multi-agent coordination [9], are concerned with either continuous or discrete opinion (state) spaces, but not a mix of them. In practice, discrete-valued and continuous-valued players or opinions often exist concurrently: examples include discrete action selection based on fusion of sensed quantities [10], heterogenous cyber-physical systems [11], and political elections, where some participants have binary opinions while others are able to hold graded ones [12]. In the recent work [13], the author interprets a class of network dynamics with hybrid discrete-valued and continuous-valued opinions as averager–copier–voter models [14]. In this endeavor, the network is composed of three types of normal agents, which update their states in different fashions, namely,

Y. Shang / Chaos, Solitons and Fractals 110 (2018) 244–251

averaging, stochastic copying, and stochastic binary voting. Based upon moment methods and stochastic stability theory, it is proved that consensus can be achieved in mean-square sense if the agents perform independent random walks over a spatial network that is connected and non-bipartite [13]. This result is remarkable in that rigorous mathematical analyses for opinion dynamics have been so far essentially overlooked, with the notable exceptions of discretevalued voter model (e.g. [15]) and continuous-valued Deffuant– Weisbuch model (e.g. [16–18]) due to the complexity of the involved dynamical processes. An important challenge of decision-making in social networks is that the community contains non-rational behaviors that prevent the group of normal agents from achieving their goal [19]. For example, there can be stubborn agents, who influence others but do not adjust their opinions, or malicious agents, who deliberately change their opinions with the goal of manipulating the performance of entire network. In this paper, we aim to continue the line of research [13] by considering the averager–copier–voter networks with non-rational agents over fixed and time-varying networks. We incorporate a filtering strategy into the rules of averager, copier, and voter agents such that consensus can be achieved in the sense of convergence in mean. Previous works regarding opinion dynamics with non-rational agents (mostly stubborn ones; see e.g. [20– 23]) generally characterize how equilibrium depends on the location and number of stubborn agents and show that the presence of stubborn agents precludes convergence to consensus. On the other hand, opinion consensus algorithms based on leadership concept have been studied in [24], where adding minimum edges may lead the network to some established target, which is desirable in networks of firms and administrations. Here, we deal with essentially an inverse problem and present necessary and sufficient conditions for consensus when the identities and actual number of non-rational agents remain unknown to the normal agents. Our method is of practical significance in that it not only contributes to understanding the influence of general adversarial behaviors (from stubborn to highly malicious ones) in opinion formation within a community but offers effective ways to cope with them when each normal agent in the network only communicates with its neighbors, i.e., in a purely distributed manner. It is worth mentioning that there are some relevant works similar in concept to our current work in robotics and control communities. The work [25,26] introduced a linear consensus protocol, termed Weighted-Mean Subsequence Reduced algorithm, which leads the states of cooperative nodes to an agreement in a multiagent system asymptotically in the presence of attackers. However, these studies do not capture hybrid dynamics and the models considered therein are deterministic. Also in analogy with the protocols considered here, hybrid consensus formation has been studied in [27,28] focusing on a hybrid of discrete-time and continuoustime agent dynamics, where only continuous-valued states are considered.

2. System description for the averager–copier–voter model We consider a time-dependent directed graph of order n, denoted by G (t ) = (V, E (t )), where V = {v1 , · · · , vn } is the node set representing the agents in the network, and E (t ) ⊆ V × V is the directed edge set at time t ∈ N. Here, N means the set of nonnegative integers. Assume a partition of the node set, V = N ∪ M, where N is the set of normal agents and M is the set of nonrational agents which is unknown a priori to the normal ones. An edge (vi , vj ) captures the information flow from agent vi to agent vj . The neighborhood of agent vi at time t is defined by Ji (t ) = {v j : (v j , vi ) ∈ E (t )}. When considering fixed networks, we will omit the dependence on t accordingly in the above notations.

245

When the meaning is clear from the context, we also suppress t in time-dependent networks for simplicity. Let xi (t) represent the value, i.e., opinion, of agent vi at time step t ∈ N. Denote by xij (t ) the value sent from agent vj to agent

vi at time t, and assume xij (t ) = x j (t ) for v j ∈ N . We further assume that N = Na ∪ Nc ∪ Nv , where Na , Nc , and Nv represents the averager, copier, and voter agents, respectively. Fix R ∈ N (which is related to the number of opinion values to be removed or ignored when performing our strategy; see below) and assume xi (0) ∈ [0, 1] for all vi ∈ N . We do not impose any assumption on the initial values of non-rational agents. The normal agents are proposed to update their opinions according to the following 3-step strategy, executed at each time step t ∈ N: (1) Each normal agent vi ∈ N obtains the values xij (t ) of its neighbors, and creates a sorted list for {xij (t )}v j ∈Ji (t ) from largest to

smallest. (2) The largest R values that are strictly larger than xi (t) in this list are removed (if there are fewer than R larger values than xi (t), all of those values are removed). The similar removal process is applied to the smaller values. The set of nodes that are removed by agent vi at time t is denoted by Ri (t ). (3) For averager vi ∈ Na :



xi (t + 1 ) =

wi j (t )xij (t ),

(1)

v j ∈(Ji (t )∪{vi } )\Ri (t )

where {wij (t)} are weights satisfying (i) wi j (t ) = 0 if v j ∈ Ji (t ) ∪ {vi }, (ii) there exists a constant α ∈ (0, 1) independent of t, such that wij (t) ≥ α > 0 for any v j ∈ (Ji (t ) ∪ {vi } )\Ri (t ), and  (iii) v ∈(J (t )∪{v } )\R (t ) wi j (t ) = 1. Note that the weights {wij (t)} j

i

i

i

can be arbitrarily chosen as long as the conditions (i)–(iii) hold. This distributed averaging protocol maintains continuousvalued opinions and calculates a continuous-valued function to determined the next opinions. It has been widely adopted in cooperative coordinations [13,20,27], which models the agents desiring to match their opinions with neighbors’ and having good computation capability. For copier vi ∈ Nc : First, choose an agent vj with probability wij (t), and then, set xi (t + 1 ) = xij (t ), where {wij (t)} satisfy the above three conditions (i)–(iii). Here, each copier agent randomly selects a determining agent (namely, a neighbor or itself), and copies its opinion at the next time step. By doing so, the copier agents maintain a continuous-valued opinion but adjust this opinion via a discrete selection process. A copier can be thought of as one who does not have its own idea and is randomly influenced by others. For voter vi ∈ Nv : First, similarly as for a copier, choose an agent vj with probability wij (t), and then take xi (t + 1 ) = 1 with probability xij (t ) and 0 with probability 1 − xij (t ), where {wij (t)} satisfy the above three conditions (i)–(iii). We remark that xij (t ) ∈ [0, 1] for the models we are interested here (see below). Clearly, each voter is stochastically influenced by its neighbors’ opinions but maintains a binary opinion (0 or 1) itself via a discrete update procedure. The voters have some pre-determined ideas and use their neighbors’ opinions to make their own choice. Recent empirical studies in social learning have also revealed that the transmission of behaviors can happen in different ways of copying, and that the specific copying mechanisms involved in decision making can impact the resulting diffusion dynamics at population level [29]. The three individual behaviors introduced above, namely, averager, copier, and voter, can be roughly viewed as using “copying from the majority”, “copying from the random one”, and “copying from the best one”, respectively. On an intuitive level, these actions tend to produce consensus behaviors.

246

Y. Shang / Chaos, Solitons and Fractals 110 (2018) 244–251

We remark that Na , Nc , Nv are allowed to be empty sets. All normal agents follow the above strategy whereas the non-rational ones may apply different, arbitrary update rules, and their identities and number remain unknown to the normal agents. Specifically, we here consider two types of non-rational behaviors. An agent vi ∈ M is said to be stubborn if it sends xi (t) to all of its neighbors at every time step t, but behaves differently from averager, copier, or voter agents at some time step. Obviously, our concept of stubborn agents is much more general than the usual definition (see e.g. [20]), where stubborn agents simply keep their opinions constant. An agent vi ∈ M is said to be adversarial if it behaves differently from averager, copier, or voter agents or it does not send the same value to all of its neighbors at some time step. By definition, all stubborn agents are also adversarial, but not vice versa. An adversarial agent may collude with other adversarial agents and hence are more harmful to the network compared to the stubborn agents. Based on the number and location of the non-rational agents, the following two types of models are addressed in this paper: (i) R-globally bounded model, where the number of agents in M is upper bounded by a constant R ∈ N; and (ii) R-locally bounded model, where |Ji ∩ M| ≤ R for each vi ∈ N . In the Rlocally bounded model, each normal agent has at most R nonrational neighbors but there is no global bound on it. In both models, it is straightforward to check that all xij (t ) in Step (3) above lie in the unit interval [0,1], justifying our choice. In both globally and locally bounded models, non-rational agents threaten the system’s goal by preventing other agents from achieving valid status or driving their values into an unsafe region. Therefore, it is desirable to design resilient coordination strategies withstanding the compromise of non-rational agents. Since the opinions of agents are influenced stochastically by their neighbors, the resulting system is stochastic. The normal agents are said to achieve consensus in mean if there exists a real L such that limt→∞ Exi (t ) = L for all vi ∈ N for any choice of initial conditions {xi (0 )}vi ∈V . We remark here that although convergence in mean is slightly weaker than mean square convergence, it has been widely explored in consensus of stochastic multi-agent systems; see e.g. [13,30,31]. The above algorithm has low complexity and is purely distributed using only local information is assumed for each normal agent. No prior knowledge of the identities of non-rational agents or the network topology is required for normal agents. In what follows, we will refer to the above algorithm as the averager–copier– voter filtering strategy with parameter R. 3. Consensus analysis for the averager–copier–voter model In this section, we present the consensus analysis for the averager–copier–voter model in the presence of both stubborn and adversarial agents. In each case, we address resilience to both globally bounded and locally bounded threats. To start with, define xmax (t ) := maxvi ∈N xi (t ) and xmin (t ) := minvi ∈N xi (t ). It is easy to see that xmax (t) and xmin (t) are stochastic processes satisfying 0 ≤ xmin (t) ≤ xmax (t) ≤ 1. Let x(t ) = {xi (t )}vi ∈V be the opinions of all agents at time t ∈ N. Lemma 1. Suppose each normal agent adjusts its opinion according to the averager–copier–voter filtering strategy with parameter R. Then in the R-globally or locally bounded model with stubborn or adversarial agents, for each vi ∈ N , Exi (t + 1 ) ∈ [Exmin (t ), Exmax (t )] regardless of the network topology. Proof. We analyze the conditional expectation for the opinion of a normal agent vi at time t + 1 given x(t). For vi ∈ Na , we have E(xi (t + 1 )|x(t )) ∈ [xmin (t ), xmax (t )] since the opinions in (Ji (t ) ∪ {vi } )\Ri (t ) used in the averager–copier–voter filtering

strategy with parameter R lie in [xmin (t), xmax (t)] and the update rule (1) is a convex combination of these opinions. For vi ∈ Nc , the expectation can be found by further conditioning on which agent is chosen to be copied. In view of the law of total prob ability, E(xi (t + 1 )|x(t )) = v ∈V E(xi (t + 1 )|x(t ), βi j (t ))P(βi j (t )), j

where β ij (t) is the event that vi copies from vj . Noting that E(xi (t + 1 )|x(t ), βi j (t )) = xij (t ) and P(βi j (t )) = wi j (t ), we derive E(xi (t + 1 )|x(t )) ∈ [xmin (t ), xmax (t )] similarly. For vi ∈ Nv , we again  have E(xi (t + 1 )|x(t )) = v ∈V E(xi (t + 1 )|x(t ), βi j (t ))P(βi j (t )). j

Now E(xi (t + 1 )|x(t ), βi j (t )) = 1 · xij (t ) + 0 · (1 − xij (t )) = xij (t ) by the definition of Bernoulli random variable. Therefore, we reproduce E(xi (t + 1 )|x(t )) ∈ [xmin (t ), xmax (t )]. Finally, taking expectation with respect to x(t) yields the desired result.  Lemma 1 indicates that Exmin (t ) and Exmax (t ) are monotone and bounded functions of t. It is worth noting that voter agents are special in the sense that xmin (t) ≡ 0 and/or xmax (t) ≡ 1 if Nv = ∅. We will see in Section 4 later that voter agents have a prominent impact on the opinion configuration of the network. To present our main results, the following notions of reachable sets and network robustness in graph theory [26] are needed. Let r, s ∈ N. A set S ⊆ V is an r-reachable set if there exists a node vi ∈ S such that |Ji \S| ≥ r. Moreover, S is an (r, s)-reachable set if |{vi ∈ S : |Ji \S| ≥ r}| ≥ s. Obviously, r-reachability is equivalent to (r, 1)-reachability. A directed graph G is said to be r-robust if for any pair of nonempty, disjoint subsets of V, at least one of them is r-reachable. Moreover, G is (r, s)-robust if for any pair of nonempty, disjoint subsets S1 , S2 ⊆ V, at least one of the following three conditions is true: (a) |{vi ∈ S1 : |Ji \S1 | ≥ r}| = |S1 |, (b) |{vi ∈ S2 : |Ji \S2 | ≥ r}| = |S2 |, and (c) |{vi ∈ S1 : |Ji \S1 | ≥ r}| + |{vi ∈ S2 : |Ji \S2 | ≥ r}| ≥ s. 3.1. Consensus in the presence of stubborn agents: globally bounded model In this section, we derive necessary and sufficient conditions for opinion formation in the globally bounded model in the presence of stubborn agents. We first consider fixed network topology and then extend the results to time-varying networks. Theorem 1. Consider a fixed network modeled as a directed graph G = (V, E ), where each normal agent updates its opinion according to the averager–copier–voter filtering strategy with parameter R. Then in the R-globally bounded model with stubborn agents, consensus in mean is achieved if G is (2R + 1, R + 1 )-robust. Furthermore, G is (R + 1, R + 1 )-robust if consensus in mean is achieved in the Rglobally bounded model with stubborn agents. Proof. (Necessity) Suppose that G is not (R + 1, R + 1 )-robust, we will prove that consensus in mean cannot be achieved in the Rglobally bound model with stubborn agents. By assumption, there exist nonempty, disjoint sets S1 , S2 ⊆ V such that none of the conditions (a)–(c) above hold. Let xi (0 ) = 0 for all vi ∈ S1 , and xi (0 ) = 1 for all vi ∈ S2 . For vi ∈ V \{S1 ∪ S2 }, we set xi (0 ) = a for some fixed a ∈ (0, 1). Since |{vi ∈ S1 : |Ji \S1 | ≥ R + 1}| + |{vi ∈ S2 : |Ji \S2 | ≥ R + 1}| ≤ R, we may suppose that all agents in {vi ∈ S1 : |Ji \S1 | ≥ R + 1} and {vi ∈ S2 : |Ji \S2 | ≥ R + 1} are stubborn agents and that they keep their opinions constant. Note that there is at least one normal agent in both S1 and S2 , say, vi1 ∈ S1 and vi2 ∈ S2 , because |{vi ∈ S1 : |Ji \S1 | ≥ R + 1}| < |S1 | and |{vi ∈ S2 : |Ji \S2 | ≥ R + 1}| < |S2 |. If vi1 is an averager or copier, xi1 (t ) ≡ 0 since it will remove the R or less opinions of neighbors outside of S1 . If vi1 is a voter, it will take opinion 1 with probability zero. Therefore, we recover xi1 (t ) ≡ 0. A similar reasoning yields xi2 (t ) ≡ 1. It is clear that no consensus between them can be reached.

Y. Shang / Chaos, Solitons and Fractals 110 (2018) 244–251

(Sufficiency) In view of Lemma 1, we may assume that ρmax := limt→∞ Exmax (t ) ≥ ρmin := limt→∞ Exmin (t ). If ρmax = ρmin , consensus in mean is then reached. In what follows, we suppose that ρ max > ρ min and will prove that it can not be true by contradiction. Set ε 0 > 0 satisfying ρmax − ε0 > ρmin + ε0 . For t ∈ N and εk > 0, we define two sets Amax (t, εk ) := {vi ∈ V : Exi (t ) > ρM − εk } and Amin (t, εk ) := {vi ∈ V : Exi (t ) < ρm + εk }. By our choice of ε 0 , αN ε

Amax (t, ε 0 ) and Amin (t, ε 0 ) are disjoint. Fix ε < 1−α 0N and ε 0 > ε > 0, where N := |N | is the number of normal agents in G. Let tε be the time step such that Exmax (t ) < ρmax + ε and Exmin (t ) > ρmin − ε for all t ≥ tε . Consider the nonempty, disjoint sets Amax (tε , ε 0 ) and Amin (tε , ε 0 ). Since G is (2R + 1, R + 1 )-robust with no more than R stubborn agents, there must be a normal agent in their union that has at least 2R + 1 neighbors outside of its set. Without loss of generality, we assume that vi ∈ Amax (tε , ε0 ) ∩ N has at least 2R + 1 neighbors outside of Amax (tε , ε 0 ). By definition, these neighbors have expected opinions at most equal to ρmax − ε0 and at leat one of these neighbors will be in the set (Ji (tε ) ∪ {vi } )\Ri (tε ). If vi is an averager, we have

Exi (tε + 1 ) ≤ (1 − α )Exmax (tε ) + α (ρmax − ε0 ) ≤ ρmax − αε0 + (1 − α )ε ,

(2)

where we have employed the inequality Exmax (tε ) ≤ ρmax + ε , and the fact that the averager agent’s opinion is a convex combination (see (1)) of its own opinion and the opinions of its neighbors with coefficients bounded below by α and that the largest value vi will use at time tε is upper bounded by xmax (tε ) according to the averager–copier–voter filtering strategy with parameter R. The inequality (2) also applies to the updated opinion of any averager outside Amax (tε , ε 0 ) since such an agent will use its own opinion, whose expectation is also upper bounded by ρmax − ε0 , in the update procedure. If vi is a copier, we can reproduce (2) by  noting that E(xi (tε + 1 )|x(tε )) = v ∈(J (tε )∪{v } )\R (tε ) wi j (tε )xij (tε ) j

i

i

according to the averager–copier–voter filtering strategy with parameter R. Denote {tk } the time steps in which G (t ) is (2R + 1, R + 1 )robust. Then in the R-globally bounded model with stubborn agents, consensus in mean is achieved if |{tk }| = ∞ and there is a constant θ such that |tk+1 − tk | ≤ θ for all k. Proof. Similarly

ε<

α N θ ε0 1−α Nθ

as

in

the

proof

of

Theorem

1,

we

fix

and ε 0 > ε > 0. Let tε be the time step such that Exmax (t ) < ρmax + ε and Exmin (t ) > ρmin − ε for all t ≥ tε . By assumption, there exists τ1 ∈ {tε , tε + 1, · · · , tε + θ − 1} such that G (τ1 ) is (2R + 1, R + 1 )-robust. Let ε1 = αε0 − (1 − α )ε and 0 < ε < ε 1 < ε 0 . Arguing as in the proof of Theorem 1, we obtain |Amax (τ1 + 1, ε1 ) ∩ N | < |Amax (τ1 , ε0 ) ∩ N | or |Amin (τ1 + 1, ε1 ) ∩ N | < |Amin (τ1 , ε0 ) ∩ N | holds. We recursively define εk = αεk−1 − (1 − α )ε for 1 ≤ k ≤ Nθ . Similarly as in Theorem 1, we can show that any normal agent vi satisfying Exi (τ1 + 1 ) ≤ ρmax − ε1 will satisfy Exi (τ1 + k ) ≤ ρmax − εk for each 1 ≤ k ≤ Nθ . In the same way, any normal agent vi satisfying Exi (τ1 + 1 ) ≥ ρmin + ε1 will satisfy Exi (τ1 + k ) ≥ ρmin + εk for each 1 ≤ k ≤ Nθ . As a result, we derive |Amax (τ1 + k, εk ) ∩ N | ≤ |Amax (τ1 + k − 1, εk−1 ) ∩ N | or |Amin (τ1 + k, εk ) ∩ N | ≤ |Amin (τ1 + k − 1, εk−1 ) ∩ N | for each 1 ≤ k ≤ Nθ , regardless of the network topology. By assumption, there is an infinite sequence τ 1 , τ 2 , , at which G (τk ) is (2R + 1, R + 1 )-robust, both Amax (τk , ετk −τ1 ) ∩ N and Amin (τk , ετk −τ1 ) ∩ N are non-empty, and either |Amax (τk + 1, ετk +1−τ1 ) ∩ N | < |Amax (τk , ετk −τ1 ) ∩ N | or |Amin (τk + 1, ετk +1−τ1 ) ∩ N | < |Amin (τk , ετk −τ1 ) ∩ N |, or both, holds for k ≥ 1. Since there are N normal agents in the network G and |τN − τ1 | ≤ Nθ , there exists some time T ≤ Nθ such that either Amax (τ1 + T , εT ) ∩ N or Amin (τ1 + T , εT ) ∩ N is empty. By our choice of ε , we have ε T > 0, which yields a contradiction as in the proof of Theorem 1. The proof is complete.  3.2. Consensus in the presence of stubborn agents: locally bounded model

i

via the law of total probability. If vi is a voter, we can also recover (2) using the expectation of a Bernoulli random variable; c.f. Lemma 1. Similarly, the inequality (2) applies to the updated opinions of all normal agents (namely, averagers, copiers, and voters) outside Amax (tε , ε 0 ). If vi ∈ Amin (tε , ε0 ) ∩ N which has at least 2R + 1 neighbors outside of Amin (tε , ε 0 ), we can obtain a similar bound

Exi (tε + 1 ) ≥ ρmin + αε0 − (1 − α )ε ,

247

(3)

which also applies to the normal agents outside Amin (tε , ε 0 ). Next, we define ε1 = αε0 − (1 − α )ε , which satisfies 0 < ε < ε 1 < ε 0 . It follows that the sets Amax (tε + 1, ε1 ) and Amin (tε + 1, ε1 ) are disjoint. The above comment implies that |Amax (tε + 1, ε1 ) ∩ N | < |Amax (tε , ε0 ) ∩ N | or |Amin (tε + 1, ε1 ) ∩ N | < |Amin (tε , ε0 ) ∩ N | holds. We can then recursively define εk = αεk−1 − (1 − α )ε for each k ≥ 1 and εk < εk−1 holds. The above discussion can be applied to each time step tε + k as long as there are still normal agents in Amax (tε + k, εk ) and Amin (tε + k, εk ). Since there are N normal agents in the network G, there exists sometime T ≤ N such that either Amax (tε + T , εT ) ∩ N or Amin (tε + T , εT ) ∩ N is empty. However, εT = αεT −1 − (1 − α )ε = α T ε0 − (1 − α T )ε ≥ α N ε0 − (1 − α N )ε > 0 by our definition of ε . This suggests that all normal agents at time tε + T have expected opinions at most ρmax − εT < ρmax or have expected opinions at least ρmin + εT > ρmin . This contradicts the definition of ρ max or ρ min , thereby completing the proof.  For time-varying topologies, we have the following corollary. Corollary 1. Consider a time-varying network modeled as a directed graph G (t ) = (V, E (t )), where each normal agent updates its opinion

In locally bounded models, where non-rational agents are much more prevalent but bounded in each normal agent’s neighborhood, we have the following characterization for time-invariant network topology that is able to handle stubborn agents. Theorem 2. Consider a fixed network modeled as a directed graph G = (V, E ), where each normal agent updates its opinion according to the averager–copier–voter filtering strategy with parameter R. Then in the R-locally bounded model with stubborn agents, consensus in mean is achieved if G is 2R + 1-robust. Furthermore, G is R + 1-robust if consensus in mean is achieved in the R-locally bounded model with stubborn agents. Proof. (Necessity) Suppose that G is not R + 1-robust, then there are nonempty, disjoint sets S1 , S2 ⊆ V such that each agent in these two sets would have no more than R neighbors outside the set. Assume that there are normal agents in both S1 and S2 . Let xi (0 ) = 0 for any agent vi ∈ S1 , and xi (0 ) = 1 for any agent vi ∈ S2 . For vi ∈ V \{S1 ∪ S2 }, set xi (0 ) = c for some fixed c ∈ (0, 1). Apparently, the opinions xi (t) of normal agents in S1 and S2 will not reach agreement under the averager–copier–voter filtering strategy with parameter R since they would never use any opinions from outside their own sets. Hence, consensus in mean cannot be achieved. (Sufficiency) As in the proof of Theorem 1, we assume that ρmax := limt→∞ Exmax (t ) and ρmin := limt→∞ Exmin (t ). We will prove that ρmax = ρmin by contradiction. To this end, suppose that ρ max > ρ min . Choose ε 0 > 0 satisfying ρmax − ε0 > ρmin + ε0 . For t ∈ N and ε k > 0, we define two sets Amax (t, εk ) := {vi ∈ V : Exi (t ) > ρmax − εk } and Amin (t, εk ) := {vi ∈ V : Exi (t ) < ρmin + εk }. By the choice of ε 0 , Amax (t, ε 0 ) and Amin (t, ε 0 ) are disjoint. Fix ε <

α N ε0

1−α N

and ε 0 > ε > 0. Let tε be the time step such that

248

Y. Shang / Chaos, Solitons and Fractals 110 (2018) 244–251

Exmax (t ) < ρmax + ε and Exmin (t ) > ρmin − ε for all t ≥ tε . Consider the nonempty, disjoint sets Amax (tε , ε0 ) ∩ N and Amin (tε , ε0 ) ∩ N . Since the network G is 2R + 1-robust, at least one of these two sets is 2R + 1-reachable. Without loss of generality, we assume that Amax (tε , ε0 ) ∩ N is 2R + 1-reachable, and therefore there is an agent vi ∈ Amax (tε , ε0 ) ∩ N which has at least 2R + 1 neighbors outside its set. Since there are at most R stubborn agents in Ji , vi will use at least one of its normal neighbors’ opinions outside Amax (tε , ε0 ) ∩ N under the averager–copier–voter filtering strategy with parameter R, be it an averager, copier, or voter. Consequently, reasoning as in the proof of Theorem 1, we obtain Exi (tε + 1 ) ≤ ρmax − αε0 + (1 − α )ε, which also applies to the updated value of any normal agent outside Amax (tε , ε0 ) ∩ N since such an agent will use its own opinion in the update procedure. Analogously, if vi ∈ Amin (tε , ε0 ) ∩ N has at least 2R + 1 neighbors outside its set, we obtain a similar bound Exi (tε + 1 ) ≥ ρmin + αε0 − (1 − α )ε , which also applies to all normal agents outside Amin (tε , ε0 ) ∩ N . By setting ε1 = αε0 − (1 − α )ε which satisfies 0 < ε < ε 1 < ε 0 , we can resort to the same proof in Theorem 1 (through defining recursively ε k , k ≥ 1) to induce the contradiction. The sufficiency is then concluded.  For time-varying networks, we have the following corollary. It can be proved in the same manner as Corollary 1. Corollary 2. Consider a time-varying network modeled as a directed graph G (t ) = (V, E (t )), where each normal agent updates its value according to the averager–copier–voter filtering strategy with parameter R. Denote {tk } the time steps in which G (t ) is 2R + 1-robust. Then in the R-locally bounded model with stubborn agents, consensus in mean is achieved if |{tk }| = ∞ and there exists a constant θ such that |tk+1 − tk | ≤ θ for all k. 3.3. Consensus in the presence of adversarial agents: globally bounded model In this section, we derive necessary and sufficient conditions for stochastic consensus in the globally bounded model in the presence of adversarial agents. Recall that adversarial agents may send different information to different neighbors at any time step, and hence they are more difficult to cope with. Let GN (t ) = (N , EN (t )) be the subgraph of G (t ) = (V, E (t )) induced by N , where EN (t ) consists of all directed edges among the normal agents at time t. We first consider fixed network topology. Theorem 3. Consider a fixed network modeled as a directed graph G = (V, E ), where each normal agent updates its opinion according to the averager–copier–voter filtering strategy with parameter R. Then in the R-globally bounded model with adversarial agents, consensus in mean is achieved if GN is 2R + 1-robust. Furthermore, GN is R + 1-robust if consensus in mean is achieved in the R-globally bounded model with adversarial agents. Proof. (Necessity) Suppose that GN is not R + 1-robust, then there exist nonempty, disjoint sets S1 , S2 ⊆ N that are not R + 1 reachable. Hence, each agent in these two sets has at most R normal neighbors outside the set. Set xi (0 ) = 0 for any agent vi ∈ S1 , and xi (0 ) = 1 for any agent vi ∈ S2 . For vi ∈ V \{S1 ∪ S2 }, set xi (0 ) = c for some fixed c ∈ (0, 1). Suppose that all adversarial agents always send the opinion 0 to each agent vi in S1 , and the opinion 1 to each agent vi in S2 at each time step t. Consequently, under the averager–copier–voter filtering strategy with parameter R, agents in S1 and S2 will not use opinions from outside their own sets. Therefore, consensus cannot be achieved. (Sufficiency) Similarly, we set ρmax := limt→∞ Exmax (t ) and ρmin := limt→∞ Exmin (t ). Assume that ρ max > ρ min . Choose ε0 > 0 satisfying ρmax − ε0 > ρmin + ε0 . For t ∈ N and ε k > 0, we define two sets Bmax (t, εk ) := {vi ∈ N : Exi (t ) > ρmax − εk }

and

Bmin (t, εk ) := {vi ∈ N : Exi (t ) < ρmin + εk }.

By

our

choice αN ε

of ε 0 , Bmax (t, ε 0 ) and Bmin (t, ε 0 ) are disjoint. Fix ε < 1−α 0N which satisfies ε 0 > ε > 0. Let tε be the time step such that Exmax (t ) < ρmax + ε and Exmin (t ) > ρmin − ε for all time steps t ≥ tε . Consider the nonempty, disjoint sets Bmax (tε , ε 0 ) and Bmin (tε , ε 0 ). Since the network GN is R + 1-robust with at most R adversarial agents, there must be an agent in Bmax (tε , ε 0 ) or Bmin (tε , ε 0 ) that has no less than R + 1 normal neighbors outside of its set. Without loss of generality, we assume that vi ∈ Bmax (tɛ , ɛ0 ) has no less than R + 1 normal neighbors outside of Bmax (tε , ε 0 ). With the same reasoning as in Theorem 1, we obtain the estimation Exi (tε + 1 ) ≤ ρmax − αε0 + (1 − α )ε , which also applies to the updated opinions of all normal agents outside Bmax (tε , ε 0 ). Analogously, if vi ∈ Bmin (tɛ , ɛ0 ) which has no less than R + 1 normal neighbors outside of Bmin (tε , ε 0 ), we have again Exi (tε + 1 ) ≥ ρmin + αε0 − (1 − α )ε , which also applies to all normal agents outside Bmin (tε , ε 0 ). Define ε1 = αε0 − (1 − α )ε , which satisfies 0 < ε < ε 1 < ε 0 . Note that the sets Bmax (tε + 1, ε1 ) and Bmin (tε + 1, ε1 ) are disjoint. The above comment implies that |Bmax (tε + 1, ε1 )| < |Bmax (tε , ε0 )| or |Bmin (tε + 1, ε1 )| < |Bmin (tε , ε0 )| holds true. We can recursively define εk = αεk−1 − (1 − α )ε for each k ≥ 1 and note that εk < εk−1 . The above discussion can be applied to each time step tε + k as long as Bmax (tε + k, εk ) and Bmin (tε + k, εk ) are non-empty. Since the network GN contains N normal agents, there is some time T ≤ N such that either Bmax (tε + T , εT ) or Bmin (tε + T , εT ) becomes empty. On the other hand, εT = αεT −1 − (1 − α )ε = α T ε0 − (1 − α T )ε ≥ α N ε0 − (1 − α N )ε > 0 by the definition of ε. This suggests that all normal agents at time tε + T have opinions at most ρmax − εT < ρmax or have opinions at least ρmin + εT > ρmin . This contradicts the definition of ρ max or ρ min . The proof of sufficiency is complete.  For time-varying networks, we have the following corollary. Corollary 3. Consider a time-varying network modeled as a directed graph G (t ) = (V, E (t )), where each normal agent updates its opinion according to the averager–copier–voter filtering strategy with parameter R. Denote {tk } the time steps in which G (t ) is 2R + 1-robust. Then in the R-globally bounded model with adversarial agents, consensus in mean is achieved if |{tk }| = ∞ and there exists a constant θ such that |tk+1 − tk | ≤ θ for all k. Proof. When G (t ) is 2R + 1-robust, GN (t ) is R + 1 robust. This is because there are at most R adversarial agents in the entire network. In light of Theorem 3, a similar argument as in the proof of Corollary 1 yields the result.  3.4. Consensus in the presence of adversarial agents: locally bounded model Finally, we consider the locally bounded models with adversarial agents distributed over a fixed network. Theorem 4. Consider a fixed network modeled as a directed graph G = (V, E ), where each normal agent updates its opinion according to the averager–copier–voter filtering strategy with parameter R. Then in the R-locally bounded model with adversarial agents, consensus in mean is achieved if GN is 2R + 1-robust. Furthermore, GN is R + 1robust if consensus in mean is achieved in the R-locally bounded model with adversarial agents. Proof. (Necessity) The necessity follows from the same proof in Theorem 3. (Sufficiency) Suppose that ρmax := limt→∞ Exmax (t ) and ρmin := limt→∞ Exmin (t ). Assume that ρ max > ρ min . Choose ε 0 > 0 satisfying ρmax − ε0 > ρmin + ε0 . For t ∈ N and ε k > 0, we define two sets Bmax (t, εk ) := {vi ∈ N : Exi (t ) > ρmax − εk } and Bmin (t, εk ) := {vi ∈

Y. Shang / Chaos, Solitons and Fractals 110 (2018) 244–251

249

respectively) to make sure that there always exists a node in one of the two sets having at least three neighbors outside. Example 1. In this example, we consider the averager–copier– voter model with no voters. The network topology is shown as in Fig. 1, where Na = {v1 , v2 }, Nc = {v3 , v4 , v5 , v6 }, and M = {v7 }. The initial opinions of the seven agents are taken as x1 (0 ) = 0.8, x2 ( 0 ) = 0.3, x3 ( 0 ) = 0.1, x4 ( 0 ) = 0.6, x5 ( 0 ) = 0.9, x6 ( 0 ) = 0.2, and x7 (0 ) = 0.7. We assume that each normal agent vi ∈ N takes −1 the weight or probability wi j (t ) = (|Ji | + 1 − |Ri (t )| ) for v j ∈ (Ji ∪ {vi } )\Ri (t ) in the averager–copier–voter filtering strategy. Since the networks G and GN are both 3-robust, Theorem 1 (or Theorem 3, respectively) implies that consensus in mean can be achieved for our system with stubborn (or adversarial, respectively) agents when we implement the averager–copier–voter filtering strategy with parameter 1. Fig. 1. Network topologies G and GN are both 3-robust, where N = {v1 , · · · , v6 }.

N : Exi (t ) < ρmin + εk }. By the choice of ε 0 , Bmax (t, ε 0 ) and Bmin (t,

ε0 ) are disjoint. Fix ε <

α N ε0

which satisfies ε 0 > ε > 0. Let tε be the time step such that Exmax (t ) < ρmax + ε and Exmin (t ) > ρmin − ε for all time steps t ≥ tε . Consider the nonempty, disjoint sets Bmax (tε , ε 0 ) and Bmin (tε , ε 0 ). Since GN is R + 1-robust, there must be an agent in Bmax (tε , ε 0 ) or Bmin (tε , ε 0 ) that has no less than R + 1 normal neighbors outside of its set. Without loss of generality, we assume that vi ∈ Bmax (tɛ , ɛ0 ) has no less than R + 1 normal neighbors outside of Bmax (tε , ε 0 ). Since there are at most R adversarial agents in Ji , vi will use at least one of its normal neighbors’ opinions outside Bmax (tε , ε 0 ) under the averager–copier–voter filtering strategy with parameter R. With the same reasoning as in Theorem 1, we obtain Exi (tε + 1 ) ≤ ρmax − αε0 + (1 − α )ε , which also applies to the updated opinions of all normal agents outside Bmax (tε , ε 0 ). Analogously, if vi ∈ Bmin (tɛ , ɛ0 ) which has at least R + 1 normal neighbors outside of Bmin (tε , ε 0 ), we have Exi (tε + 1 ) ≥ ρmin + αε0 − (1 − α )ε, which also applies to the normal agents outside Bmin (tε , ε 0 ). Define ε1 = αε0 − (1 − α )ε , which satisfies 0 < ε < ε 1 < ε 0 . The same argument in the proof of Theorem 3 (by recursively defining εk = αεk−1 − (1 − α )ε for k ≥ 1) yields the desired contradiction. The proof of sufficiency is complete.  1−α N

For time-varying networks, we have the following result. It can be shown similarly as Corollary 3.

Corollary 4. Consider a time-varying network modeled as a directed graph G (t ) = (V, E (t )), where each normal agent updates its opinion according to the averager–copier–voter filtering strategy with parameter R. Denote {tk } the time steps in which G (t ) is 2R + 1-robust. Then in the R-locally bounded model with adversarial agents, consensus in mean is achieved if |{tk }| = ∞ and there exists a constant θ such that |tk+1 − tk | ≤ θ for all k.

4. Numerical studies In this section, we present simulations to illustrate our theoretical results. We first consider a 3-robust network G = (V, E ) with the node set V = {v1 , · · · , v7 } (see Fig. 1), in which node v7 is a non-rational agent. Moreover, the subnetwork GN induced by N = {v1 , · · · , v6 } is also 3-robust. To verify these facts, we have to exhaustively check every pair of non-disjoint subsets in V (and N ,

In Fig. 2(a) we show the opinion trajectories of the agents, where the agent v7 is stubborn and keeps its opinion unchanged. In Fig. 2(b) we assume that v7 is adversarial, which takes the strategy x17 (t ) = x27 (t ) = 0.7 and x7 (t ) = x47 (t ) = x57 (t ) = sin(t/4 ). Namely, the agent v7 in Fig. 2(b) sends opinion value 0.7 to both agents v1 and v2 , but a totally different value to v4 and v5 . We observe from Fig. 2 that the normal agents are able to reach consensus in both cases as predicted by our theoretical results. Example 2. Here, we consider the averager–copier–voter model with the existence of voters. The network topology is again shown as in Fig. 1, where Na = {v1 , v2 }, Nc = {v3 , v4 }, Nv = {v5 , v6 }, and M = {v7 }. The wij (t) functions and the initial opinions are taken as in Example 1. Similarly, Theorems 1 and 3 indicate that consensus in mean can be achieved for our system with stubborn or adversarial agents when we implement the averager–copier–voter filtering strategy with parameter 1. We show in Fig. 3(a) the opinion evolution of the agents, where the agent v7 is stubborn and keeps its opinion unchanged. In Fig. 3(b) we assume that v7 is adversarial, which takes the same strategy as specified in Example 1. From Fig. 3, we observe that the normal agents are able to reach consensus in both cases as one would expect. However, note that the final opinion value can only be 0 or 1 if consensus is reached when there exist voter agents (see Fig. 3). This is in line with the previous results on averager– copier–voter model without non-rational behaviors [13,14]. Moreover, the existence of voters appears to hinder the consensus process. For example, the system in Fig. 2 reaches consensus before t = 10 but consensus is reached until t is around 30 in the sample paths shown in Fig. 3. Similar phenomenon has been observed in [13] in systems free of non-rational agents. It is also worth mentioning that although consensus is achieved as shown in Figs. 2 and 3, the non-rational agent v7 may affect the opinion evolution and the ultimate consensus value. This is clear from our averager–copier–voter filtering algorithm that only the largest and the smallest opinions (R = 1 in our examples) at each time step may be removed, which is not necessarily the opinion of v7 . Example 3. Finally, we consider a social network of a karate club that was studied by Zachary [32]. The network has 34 nodes representing the members of a karate club, and links describing the social interaction between the members (see Fig. 4(a)). We assume that all nodes except the node v0 are normal agents, and v0 is a stubborn agent which keeps its opinion unchanged. Each normal agent is assumed to be a copier or an averager with probability 0.5. Note that v0 is directly linked to the most connected node in the network, posing a substantial threat to the group goal of reaching consensus.

250

Y. Shang / Chaos, Solitons and Fractals 110 (2018) 244–251

Fig. 2. Opinion evolution of 2 averagers {v1 , v2 } and 4 copiers {v3 , , v6 } for Example 1. In (a) v7 is a stubborn agent keeping its opinion value constant and in (b) v7 is an adversarial agent communicating different values to its neighbors.

Fig. 3. Opinion evolution of 2 averagers {v1 , v2 }, 2 copiers {v3 , v4 }, and 2 voters {v5 , v6 } for Example 2. In (a) v7 is a stubborn agent keeping its opinion value constant and in (b) v7 is an adversarial agent communicating different values to its neighbors.

Fig. 4. (a) Network topology of the Zachary karate club, where v0 is non-rational. (b) Consensus error (t) with respect to t for all normal agents with v0 being stubborn.

The initial opinions of the normal agents (here, averagers and copiers) are taken uniformly at random from the interval [0,1] and we keep x0 (t ) = 0.5 for all time t. Define the consensus error of the normal agents as (t ) = maxvi ,v j ∈N {|xi (t ) − x j (t )|}. We observe from Fig. 4(b) that (t) tends to zero with our averager– copier–voter filtering strategy with parameter 1 in spite of the existence of a non-rational agent v0 . It is worth noting that the Zachary karate club network is not 2-robust since there are nodes having only degree one. However, this does not contradict, for example, the necessity of Theorem 3 because Example 3 here only says that opinion consensus is possible when there is one specific node that is compromised by an adversary.

5. Conclusion We present a general model for the opinion formation in the averager–copier–voter network with non-rational agents. Our method distinguishes itself from prior work in that the location and number of non-rational agents are unknown a priori. We propose the averager–copier–voter filtering strategy that is capable of steering the robust opinion network toward a stochastic hybrid consensus in the sense of convergence in expectation. Necessary and sufficient conditions guaranteeing consensus in expectation are provided in the presence of stubborn and adversarial agents under both globally and locally bounded models. There are many interesting problems that remain open. For example, there are gaps

Y. Shang / Chaos, Solitons and Fractals 110 (2018) 244–251

between the necessary and sufficient conditions derived here. From the perspective of stochastic dynamical system, almost sure convergence is more desirable as compared to the weak convergence studied here in some cases. Moreover, communication delay issue has not been addressed in our paper, which is essential in many large-scale realistic systems. It is hoped that this work could stimulate further investigation on the resilience of opinion dynamics against non-rational and other non-cooperative behaviors in social networks. Acknowledgments The author would like to thank the reviewers for valuable comments and constructive suggestions. This work is funded by the National Natural Science Foundation of China (11505127). References [1] Castellano C, Fortunato S, Loreto V. Statistical physics of social dynamics. Rev Mod Phys 2009;81:591–646. [2] Clifford P, Sudbury A. A model for spatial conflict. Biometrika 1973;60:581–8. [3] Sznajd-Weron K, Sznajd J. Opinion evolution in closed community. Int J Mod Phys C 20 0 0;11:1157–65. [4] Galam S. Minority opinion spreading in random geometry. Eur Phys J B 2002;25:403–6. [5] Deffuant G, Neau D, Amblard F, Weisbuch G. Mixing beliefs among interacting agents. Advs Complex Syst 20 0 0;3:87–98. [6] Hegselmann R, Krause U. Opinion dynamics and bounded confidence: models, analysis and simulation. J Art Soc Soc Simul 2002;5:1–33. [7] Gandica Y, del Castillo-Mussot M, Vázquez GJ, Rojas S. Continuous opinion model in small-world directed networks. Physica A 2010;389:5864–70. [8] Jiang LL, Hua DY, Zhou JF, Wang BH, Zhou T. Opinion dynamics on directed small-world networks. Eur Phys J B 2008;65:251–5. [9] Ge X, Yang F, Han QL. Distributed networked control systems: a brief overview. Inf Sci 2017;380:117–31. [10] Dong Y, Zhan M, Kou G, Ding Z, Liang H. A survey on the fusion process in opinion dynamics. Inform Fus 2018;43:57–65. [11] Lee EA, Seshia SA. Introduction to embedded systems: a cyber-physical systems approach. 2nd ed.. Cambridge, MA: MIT Press; 2017. [12] Braha D, de Aguiar MAM. Voting contagion: modeling and analysis of a century of U.S. presidential elections. PLoS ONE 2017;12:e0177970.

251

[13] Shang Y. Consensus in averager–copier–voter networks of moving dynamical agents. Chaos 2017;27:023116. [14] Xue M, Roy S. Averager–copier–voter models for hybrid consensus. IFAC-PapersOnLine 2016;49-4:001–6. [15] Liggett TM. Interacting particle systems. Classics in mathematics series. Springer; 2005. [16] Shang Y. Deffuant model with general opinion distributions: first impression and critical confidence bound. Complexity 2013;19:38–49. [17] Shang Y. Deffuant model of opinion formation in one-dimensional multiplex networks. J Phys A 2015;48:395101. [18] Antonopoulos C, Shang Y. Opinion formation in multiplex networks with general initial distributions. Sci Rep 2018;8:2852. [19] Alatas H, Nurhimawan S, Asmat F, Hardhienata H. Dynamics of an agent-based opinion model with complete social connectivity network. Chaos Solitons Fract 2017;101:24–32. [20] Yildiz E, Ozdaglar A, Acemoglu D, Saberi A, Scaglione A. Binary opinion dynamics with stubborn agents. ACM Trans Econ Comp 2013;1(4):19. [21] Jin C, Li Y, Jin X. Political opinion formation: initial opinion distribution and individual heterogeneity of tolerance. Physica A 2017;467:257–66. [22] Steinbacher M, Steinbacher M. Opinion formation with imperfect agents as an evolutionary process. Comput Econ 2017. doi:10.1007/s10614- 017- 9751- z. [23] Topirceanu A, Udrescu M, Vladutiu M, Marculescu R. Tolerance-based interaction: a new model targeting opinion formation and diffusion in social networks. Peer J Comput Sci 2016;2:e42. [24] Dong Y, Ding Z, Martínez L, Herrera F. Managing consensus based on leadership in opinion dynamics. Inf Sci 2017;397–398:187–205. [25] Zhang H, Sundaram S. Robustness of information diffusion algorithms to locally bounded adversaries. In: Proc 2012 American control conference, Montreal, Canada; 2012. p. 5855–61. [26] LeBlanc HJ, Zhang H, Koutsoukos X, Sundaram S. Resilient asymptotic consensus in robust networks. IEEE J Select Areas Commun 2013;31(4):766–81. [27] Borra D, Lorenzi T. A hybrid model for opinion formation. Z Angrew Math Phys 2013;64:419–37. [28] Shang Y. Scaled consensus of switched multi-agent systems. IMA J Math Control Inform 2018. doi:10.1093/imamci/dnx062. [29] Kenda R, Hopper LM, Whiten A, Brosnan SF, Lambeth SP, Schapiro SJ, Hoppitt W. Chimpanzees copy dominant and knowledgeable individuals: implications for cultural diversity. Evol Hum Behav 2015;36:65–72. [30] Aysal TC, Barner KE. Convergence of consensus models with stochastic disturbances. IEEE Trans Inform Theory 2010;56:4101–13. [31] Shang Y. L1 group consensus of multi-agent systems with switching topologies and stochastic inputs. Phys Lett A 2013;377:1582–6. [32] Zachary WW. An information flow model for conflict and fission in small groups. J Anthropol Res 1977;33:452–73.