A distributed method to avoid higher-order deadlocks in multi-robot systems

Automatica 112 (2020) 108706

Contents lists available at ScienceDirect

Automatica journal homepage: www.elsevier.com/locate/automatica

A distributed method to avoid higher-order deadlocks in multi-robot systems✩ ∗

Yuan Zhou a , Hesuan Hu a,b , , Yang Liu a , Shang-Wei Lin a , Zuohua Ding c a

School of Computer Science and Engineering, College of Engineering, Nanyang Technological University, Singapore 639798, Singapore School of Electro-Mechanical Engineering, Xidian University, Xi’an, Shaanxi 710071, PR China c School of Information Sciences, Zhejiang Sci-Tech University, Hangzhou, Zhejiang 310018, PR China b

article

info

Article history: Received 25 September 2018 Received in revised form 13 July 2019 Accepted 20 October 2019 Available online xxxx Keywords: Autonomous mobile robots Deadlock Robot control Discrete-event systems Distributed control

a b s t r a c t Deadlock avoidance is a crucial problem in motion control of multi-robot systems since deadlocks can crash the systems and /or degrade their performance. However, deadlocks sometimes are difficult to predict in advance because of the existence of higher-order deadlocks, from which a system can lead to a deadlock inevitably. In this paper, we investigate the properties of higher-order deadlocks and propose a distributed approach to their avoidance in multi-robot systems where each robot has a predetermined and closed path to execute persistent motion. After modeling the motion of robots by labeled transition systems (LTSs), we first conclude that there exist at most the (N − 3)-th order deadlocks with N robots. This means that deadlocks, if any, will occur unavoidably within N − 3 steps of corresponding transitions. A distributed algorithm is then proposed to avoid deadlocks in such systems. In the algorithm, each robot only needs to look ahead at most N − 1 states, i.e., N − 3 intermediate states and two endpoint states, to decide whether its move can cause higher-order deadlocks. To execute the algorithm, each robot needs to communicate with its neighbors. The theoretical analysis and experimental study show that the proposed algorithm is practically operative. © 2019 Elsevier Ltd. All rights reserved.

1. Introduction A multi-robot system is a system where multiple mobile robots work together to finish tasks by moving around in a given environment. With cooperation among robots, such a system can execute complicated tasks efficiently and has been applied in many areas (Khamis, Hussein, & Elmogy, 2015). However, the cooperation also makes the system more sophisticated. Some problems, such as deadlocks, may arise. This paper focuses on the multi-robot systems where each robot moves along a predetermined and closed path persistently. Some potential applications are in transportation systems, warehouses, tourist areas, and public parks. For example, autonomous cars are required to move along particular circular roads to monitor real-time traffic conditions in a city; robots in warehouses ✩ The material in this paper was not presented at any conference. This paper was recommended for publication in revised form by Associate Editor Rong Su under the direction of Editor Christos G. Cassandras. ∗ Correspondence to: Cyber Security Lab, School of Computer Science and Engineering, Nanyang Technological University, 50 Nanyang Ave, 639798, Singapore. E-mail addresses: [email protected] (Y. Zhou), [email protected] (H. Hu), [email protected] (Y. Liu), [email protected] (S.-W. Lin), [email protected] (Z. Ding). https://doi.org/10.1016/j.automatica.2019.108706 0005-1098/© 2019 Elsevier Ltd. All rights reserved.

are required to continually load and unload materials and products in given circular lines. Such predetermined paths may also be generated from other path planning algorithms, which avoid environmental obstacles in advance. During their motion, a robot needs to avoid collisions with others, which may cause deadlocks. In our previous work (Zhou, Hu, Liu and Ding, 2017; Zhou, Hu, Liu, Lin and Ding, 2019), we have studied collision and deadlock avoidance for simple path networks, without consideration of higher-order deadlocks. A higher-order deadlock is a deadlockfree configuration, from which the system will lead to a deadlock inevitably. Higher-order deadlocks make it hard to predict and avoid deadlocks in advance. Although some technologies have been proposed to avoid deadlocks completely, there is little work about deep exploration on how long a deadlock will occur from a higher-order deadlock. So the current methods are either too aggressive, i.e., checking the whole state space, or too conservative, i.e., forbidding many acceptable states. Hence, in this paper, we investigate the properties and avoidance of higher-order deadlocks in the aforementioned multi-robot systems. Using labeled transition systems (LTSs) to describe robots’ motion, we first study the orders of higher-order deadlocks and conclude that N robots can form a higher-order deadlock with at most (N − 3)th order. This means from such a configuration, a deadlock will occur inevitably within (N − 3)-step moves of the robots that are always included in a circuit. We then propose a

2

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

distributed algorithm to avoid higher-order deadlocks. Each robot executes the algorithm to predict deadlocks by looking ahead at most N − 1 states and communicating with its neighbors, where N is the number of robots in the multi-robot system. The orders of higher-order deadlocks give the upper bound of states that a robot needs to check at each state. Compared with the state-of-the-art technologies for ordinary deadlock avoidance and prevention, a robot in our method does not need to check all reachable configurations of the system or the whole states to its nearest collision-free state. In this way, the method based on deadlock orders can achieve not only acceptable computation complexity and good flexibility but also more admissible motion. The main contributions of this work are twofold. First, we propose the concepts of higher-order deadlocks and their orders. The main result is that N robots can form a higher-order deadlock with at most (N − 3)th order. Second, we propose a distributed algorithm to avoid higher-order deadlocks, where each robot needs to look ahead at most N − 1 states, i.e., one starting state, one ending state, and N − 3 intermediate states. The rest of this paper is organized as follows. Section 2 is a brief review of related work. Section 3 gives the problem statement of deadlock avoidance in terms of LTSs. Section 4 gives a detailed control algorithm for higher-order deadlock avoidance and its distributed nature. Section 5 shows simulation results. Sections 6 and 7 give some discussions and the conclusion of this paper, respectively. 2. Related work Collision avoidance is one of the basic requirements during robot motion. There are usually two ways to avoid collisions. The first one is to plan/replan collision-free paths or trajectories during robot motion, while the second one is to control robots to pass through the same intersections at different time instants. For the first one, some technologies are developed to plan collision-free trajectories, such as optimization programming (Zhou, Hu, Liu, Lin et al., 2019), vector fields (Dimarogonas, Loizou, Kyriakopoulos, & Zavlanos, 2006; Ge & Cui, 2002), cell decomposition (Kloetzer, Mahulea, & Gonzalez, 2015), state lattice (Pivtoraiko & Kelly, 2011), reciprocal collision avoidance (Van Den Berg, Snape, Guy, & Manocha, 2011), and graph theory (Ma, Kumar, & Koenig, 2017; Yu & LaValle, 2016). They can be applied either online or offline. However, to apply these methods online, robots are allowed to change their paths during their running. For the second one, collisions are avoided by controlling robots to traverse a collision location at different times, such as (Akella & Hutchinson, 2002; Kim & Kumar, 2014; Soltero, Smith, & Rus, 2011; Wang, Kloetzer, Mahulea, & Silva, 2015). The key point of this kind of methods is to compute time delay or motion priorities of robots. However, only collision avoidance sometimes is not enough for safe motion since deadlocks may occur in order to avoid collisions. Deadlocks will stop robots from moving forward and even stagnate the whole system. Indeed, deadlocks are a great challenge to systems containing multiple subsystems with shared resources, such as automated manufacturing systems and multirobot systems, and have been widely studied (Alonso-Mora, DeCastro, Raman, Rus, & Kress-Gazit, 2018; Chen, Moarref, & Kress-Gazit, 2018; Chen, Wu, & Zhou, 2016; Fanti, Mangini, Pedroncelli, & Ukovich, 2015; Hu, Su, Zhou, & Liu, 2016; Hu & Zhou, 2015; Jager & Nebel, 2001; Kalinovcic, Petrovic, Bogdan, & Bobanac, 2011; Lee & Lin, 1995; Li, Wu, & Zhou, 2012; Moorthy, Hock-Guan, Wing-Cheong, & Chung-Piaw, 2003; Reveliotis & Roszkowska, 2011; Uzam & Zhou, 2007; Xing, Wang, Zhou, Lei, & Luo, 2018; Zhou, Hu, Liu and Ding, 2017; Zhou, Hu, Liu, Lin, & Ding, 2018). Among the existing work, there are mainly three

strategies to solve deadlocks: deadlock detection and resolution, deadlock prevention, and deadlock prediction and avoidance. For the deadlock detection and resolution strategy, such as Alonso-Mora et al. (2018), Chen et al. (2018), Jager and Nebel (2001) and Moorthy et al. (2003), resolution methods are applied to resolve deadlocks when they are detected. This means deadlocks are allowed to occur during system evolution. Due to the existence of deadlocks, this strategy is suitable for systems where deadlocks are rare and cannot result in severe catastrophes, and the recovery is affordable. Deadlock prevention, such as the work in Chen et al. (2016), Uzam and Zhou (2007) and Xing et al. (2018), is an off-line mechanism to avoid deadlocks. The key task is to compute liveness conditions or design a proper controller before a system is released, so deadlocks can never occur. However, such methods are with exponential computation complexity with regard to the net sizes (Chen et al., 2016). Deadlock prediction and avoidance is an online strategy to avoid deadlocks. Via looking ahead into the future system evolution, it can predict online whether the current evolution would cause deadlocks (Fanti et al., 2015; Kalinovcic et al., 2011; Lee & Lin, 1995; Moorthy et al., 2003; Reveliotis & Roszkowska, 2011; Zhou, Hu, Liu and Ding, 2017). The main cost for this strategy is the prediction of deadlocks. Since the optimal deadlock avoidance is NP-complete (Reveliotis, 2006), some conservative methods are proposed in practice, such as the Banker’s algorithm (Dijkstra, 1968) and its variants (Kalinovcic et al., 2011; Reveliotis & Roszkowska, 2011). The Banker’s algorithm requires that at any time, the move of a robot should guarantee that each robot can move to its destination sequentially in some order. While its variants focus on more relaxed movable conditions. For example, in Kalinovcic et al. (2011) and Reveliotis and Roszkowska (2011), a robot can move forward if it can move to an intermediate location that cannot be occupied by others. Indeed, during deadlock detection and avoidance, there is a tradeoff between computation cost and motion permissiveness. Too conservative policies usually have low computation cost but forbid many permissive motions, while too aggressive methods allow more permissive motions but the computation cost is high. In our method, each robot only checks the status of some of its collision states and communicates with its neighbors to retrieve local information of the system; it does not need to search for the whole state space or exhaustively analyze the global structure of the whole system. Hence, the proposed method allows more permissive motions with affordable computation cost. 3. Multi-robot systems and problem statement In this section, we first give a brief review of the building of the discrete model for robot motion described in Zhou et al. (2018). As well, we present our problem statement. 3.1. Multi-robot systems Let N be the number of robots in a system, and NN = {1, 2, . . ., N }. N+ is the set of all positive integers. ∀i ∈ NN , ri denotes a robot in the system. Rn0 (n0 = 2 or 3) is the motion space of robots in the 2-D or 3-D Euclidean space. Each robot has a predefined closed path. The closed path of ri , denoted as P i , is a closed curve defined by a parameter equation: P i = P i (θ ), θ ∈ [0, 1], mapping from [0, 1] to Rn0 , and P i (1) = P i (0). The motion direction is given by increasing θ . A segment of a path P i , denoted as Pki , k ∈ N+ , can be described as Pki = {P(θ ) : θ ∈ [θ1 (k), θ2 (k)]}, where θ1 (k) ∈ [0, 1] and θ2 (k) ∈ [0, 1]. Let j d(Pki , Pk′ ) = infx∈P i ,y∈P j d(x, y), where d(x, y) = ∥x − y∥2 , denote k

k′

j

the distance between Pki and Pk′ .

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

3

For the sake of safety, each robot has a safe radius, say ρ , during its motion. Thus, two robots are in a collision if d(x, y) < 2ρ , where x and y are their current positions, respectively. Let j A2ρ = {x ∈ Rn0 |∥x − P j (θ )∥2 ≤ 2ρ, θ ∈ [0, 1]}, then ri ’s collision j

locations with rj is P i ∩ A2ρ . It contains a set of continuous path segments of P i . j

Definition 1. Suppose Pki and Pk′ are maximal continuous segj j ments in P i ∩ A2ρ and P j ∩ Ai2ρ , respectively. (Pki , Pk′ ) is a collision j

pair between P i and P j if d(Pki , Pk′ ) ≤ 2ρ . Pki is called a collision segment of ri related to rj ; and rj related to ri .

j Pk′

is called a collision segment of

Note that in Definition 1, each maximal continuous segment of a robot is determined with respect to one and only one of other robots, rather than all of them. This means that two maximal continuous segments may be adjacent since they are with respect to two different robots. For example, in Fig. 1(a), segment ab is a maximal continuous collision segment with r4 , while bc is a maximal continuous collision segment with r3 ; ab and bc are adjacent. In Fig. 1(b), r1 and r2 may collide with each other only when they are respectively at ab and de or at bc and ef simultaneously, but the maximal continuous collision segments are abc and def . The evolution of a multi-robot system relies on a lot of things, such as the motion control algorithms, the sensors to monitor the environment, and the communication via wireless network. However, we cannot deal with all of them in one paper. As usual, the clarity of one perspective’s discussion can be attained by the negligence of others, i.e., their correctness is assured by default. In this paper, we focus on the design of motion control supervisors. Thus, to simplify the problem, we need some assumptions. Assumption 1. Each robot can always move along its path with a tolerant derivation, which can be addressed by considering it into the safe radius. Assumption 2. Each path is a one-way traffic, and robots are not allowed to move backward. Assumption 3. Different robots have different paths, and one path has only one robot. Assumption 4. Each collision segment can be retrieved via a sequence of communications with other robots. Note that with the limitation of communication range, our method, as well as other counterparts, cannot deal with the situation that a collision segment is larger than the communication range of robots. However, we can address it by deploying some simple equipments manually when we construct the path network. For example, once we evaluate that a path segment may be hard to be completely detected by a robot, we can add a monitor at either of the endpoints in advance to show the number of robots in the segment. Only when there are no robots in the segment can a robot move into this segment. Hence, such deployment does not require that each robot know the path network in advance. 3.2. Robot motion In this paper, by assuming local continuous controllers are available, we study high-level discrete control policies for robot motion. We use LTSs to model robots’ motion. An LTS is a triple ⟨S, Σ , →⟩, where S is the finite set of states, Σ is the finite set of events, and →⊆ S × Σ × S is the set of transitions. Usually, the

Fig. 1. Examples showing maximal continuous collision segments. (a) ab, bc, and cd are three maximal collision segments of r1 ; ef and fg are two maximal collision segments of r2 ; hi, ij, and jk are three maximal collision segments of r3 ; lm and mo are maximal collision segments of r4 . (b) abc is a maximal collision segment of r1 , and def is a maximal collision segment of r2 .

Fig. 2. Illustrative examples. (a) A deadlock: r1 − r3 form a deadlock. (b) A higher-order deadlock: r1 and r3 can move currently; however, the move of r1 (resp., r3 ) causes a deadlock with r2 and r3 (resp., r1 and r4 ).

transition from state si to sj triggered by the event δ , i.e., (si , δ , δ

sj ) ∈ →, is written as si → sj . Next, we present the procedure to construct the detailed LTS models for the motion of robots. The procedure contains the following steps. Step 1: Abstract discrete states. By searching for its path and communicating with its neighbors, each robot can determine its collision segments with different robots independently. Thus, the robot can obtain a set of non-overlapped segments by merging the overlapped ones. Then, it can determine the collision pairs with others via a sequence of communication. For a robot ri , each segment in a collision pair is abstracted as a collision state. Note that the segments in a collision pair are abstracted as the same collision state in system level. The set of collision states is denoted as Sαi . The rest part of P i is collision-free. It can be partitioned into a set of small segments, each of which is abstracted as a private state. The set of private states is denoted as Sβi . So the state space of ri is S i = Sαi ∪ Sβi . Step 2: Construct LTS model for each robot. The motion of a robot is abstracted as two events: mov e and stop. With the partition of the path P i , when a robot moves from one segment to another, the mov e event is triggered and the robot transits to another state. Hence, the set of transitions with mov e event can mov e be denoted as →i,mov e = {sik −→i sik′ : ∀sik ∈ S i }. Note that all the mov e transitions and the discrete states form a circle such that the robot at any state can move back to this state. To avoid collisions or deadlocks, a robot should stop at the current state, stop

resulting in the set of stop transitions, i.e., →i,stop = {sik −→i sik : ∀sik ∈ S i }. So the LTS model of ri ’s motion is Ti = ⟨S i , Σi = {mov e, stop}, →i ⟩, where →i =→i,move ∪ →i,stop . For example, the LTS models for the paths shown in Fig. 1(a) are given in Fig. 2(b). ab and mo are abstracted to a collision state s1 , bc and ij are abstracted to s2 , cd and ef are abstracted to s3 , fg and hi are abstracted to s4 , and jk and lm are abstracted to s5 .

4

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

Note that physically a discrete transition can be enabled and fired at the current state only when the robot arrives at the end of the related path segment. We assume that a suitable local continuous controller relying on the robot’s kinematics is available for each robot and can stop the robot in a short time so as to implement the discrete transitions. Let •i s and s•i be the unique preceding and succeeding states mov e mov e of s in S i , respectively, i.e., •i s −→i s and s −→i s•i . Sα = ∪i∈NN i Sα . 3.3. Problem statement Before giving our problem statement, we give some notations with respect to the system level. Without ambiguity, the state of a system is called configuration. Hence, a configuration of a system, denoted as c, is a vector composing the discrete states of all robots in the system, i.e., c = (s1 , s2 , . . ., sN ), where si ∈ S i . The state of ri at c can be denoted as c(i), i.e., si = c(i). Let ccur = (s1cur , . . . , sNcur ) be the current configuration of the system, where sicur is the current state of ri . Definition 2. A configuration c is a collision one if ∃i, j ∈ NN , i ̸ = j, such that c(i) = c(j). The set of collision configurations is denoted as Cc , so the set of collision-free configurations is Ccfree = C \ Cc . Definition 3. A configuration c is a deadlock configuration if there exist a set of robots, ri1 , ri2 , . . . , rik , such that ∀im ∈ {i1 , . . . , ik }, c(im )•im = c(im+1 ), where ik+1 = i1 . The set of deadlock configurations is denoted as Cdead , and the set of deadlockfree configurations is Cdfree = Ccfree \ Cdead . Using the logical operators ‘‘implication’’ (→) and ‘‘conjunction’’ (∧), and the temporal operator ‘‘eventually’’ (♦), we give the definition of higher-order deadlocks. Definition 4. A configuration c is a higher-order deadlock if (c ∈ Cdfree ) ∧ (c → ♦cd ), where cd ∈ Cdead . The set of higher-order deadlock configurations is denoted as Chdead . For example, Fig. 2 shows examples of deadlocks and higherorder deadlocks. To ensure safety, each admissible configuration should not be a higher-order deadlock. Let Cfree = Cdfree \ Chdead . In the sequel, we can give the problem statement studied in this paper. Problem: Given a multi-robot system with N robots, whose LTS models are {Ti }Ni=1 , find an online control policy for each robot such that at any state its move always leads the system to a configuration in Cfree . 4. Collision and higher-order deadlock avoidance In this section, we develop an algorithm to avoid collisions and deadlocks. 4.1. Collision avoidance strategy First, we briefly describe a collision avoidance strategy. Based on Definition 2, to avoid collisions, all the reachable configurations should be collision-free. Note that only when it is approaching to a state in Sα can a robot cause collisions with others. Hence, a collision-avoidance strategy allows at most one robot to be at s, ∀s ∈ Sα , at any moment. Let Sign be the set of the local signals: ∀s ∈ Sα , Sign(s) = 0 if s is not occupied by any robot; otherwise, Sign(s) = 1. Thus, a robot ri at the current state, say s, is movable if s•i is a private state or Sign(s•i ) = 0.

Since each robot checks its succeeding state independently, there may be several robots that can move into a state at the same time. Thus, to guarantee mutual exclusion and concurrent motion, they need to negotiate with each other to determine which one can actually move. There are many negotiation strategies, such as centralized algorithms, token based algorithms, and non-token based algorithms (Kshemkalyani & Singhal, 2011; Yadav, Yadav, & Mandiratta, 2015). Here we exemplify a fairly random selection method for mutual exclusion, which relies on real-time communication connection and is simple to implement. First, each robot communicates with its neighbors to identify the robots that are moving through the same collision region X . Let eX be the set of such movable robots. Each robot in eX generates a random time delay based on the same distribution. Then, they communicate with others to retrieve all delays. Finally, the robot with the minimum one obtains the right to move. This process is denoted as Neg(eX ). Thus, the collision avoidance strategy for ri is that: When it is about to move to s ∈ Sαi , ri first checks the value of Sign(s). If Sign(s) = 0 and it gets the right to move from the process Neg(es ), ri enters s, and the signal changes to 1. Otherwise, it stops at the current state. 4.2. Higher-order deadlocks and their detection In this subsection, we study the characteristics of higher-order deadlocks from the system level, based on which we develop a distributed method to detect higher-order deadlocks by each robot. Definition 5. An edge-colored digraph is a quadruple ⟨V , E , Ic , C ⟩, where V is a finite set of vertices, E is a finite set of edges, Ic is a finite set of colors, and C : E → Ic is a function assigning each edge with a color in Ic . Definition 6. The edge-colored digraph produced from a multirobot system, {Ti = ⟨S i , Σi , →i,mov e ⟩, i ∈ NN }, is a quadruple GT = ⟨V , E , Ic , C ⟩, where V = ∪Ni=1 S i , E = ∪Ni=1 →i,mov e , Ic = NN contains N different colors, and C satisfies ∀e ∈→i,mov e , C (e) = i. This definition means that in the graphic representation of a multi-robot system, all edges from the same robot are colored by the same color. Proposition 1. There are no two states that are connected by two or more different colored edges in GT . Proof. Suppose there are two states connected with two different j colored edges, and their corresponding collision pairs are (P1i , P1 ) j and (P2i , P2 ). Clearly, P1i and P2i are adjacent on P i , and may collide with rj . However, based on Definition 1, they should be one j j collision segment. Similarly, P1 and P2 should be one collision segment. This means that the two states should be abstracted to the same one. Hence, there are no two successive states that are connected by different colored edges. ■ Definition 7. An individual walk of robot ri in GT is a sequence of vertices and edges with the form ⟨s1 , (s1 , s2 ), s2 , . . ., sn−1 , (sn−1 , sn ), sn ⟩, where sk ∈ S i for all k ∈ Nn . Without ambiguity, w can also be simplified as w = ⟨s1 , s2 , . . ., sn ⟩. We denote s1 and sn as t(w ) and h(w ), respectively, i.e., the tail and the head of w . If a robot is at a private state, then it cannot block any robot. So in the rest, we only need to study the situation that a robot is at a collision state. We have the following definition.

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

5

Fig. 4. Sub-circuit W3′ of W3 given in Fig. 3(a).

p

Fig. 3. (a): A given configuration c. (b) − (d): Three circuits at c.

Definition 8. Let w (c) = ⟨s1 , s2 , . . ., sn ⟩ be an individual walk of ri at configuration c.

• w(c) is risky if (1) ∀k ∈ Nn , sk ∈ Sαi ; and (2) ri is at s1 , while sn is occupied by another robot rj at c.

• w(c) is safe if (1) ∀k ∈ Nn−1 , sk ∈ Sαi , and sn ∈ Sβi ; and (2) ri is at s1 whereas other states are empty at c.

The concepts of risky and safe walks are dependent on system configurations. Given a configuration c where a robot is at a collision state, it either has risky walks or contains a safe walk. Each robot can check risky or safe walks independently since it only needs to detect its own path. If a robot has a safe walk currently, its one-step move cannot cause collisions or deadlocks since as the last resort it can move to its private state while others stop at their current states. Hence, we only need to consider the robots that have risky walks. The following descriptions are related to the current configuration ccur even if it is not shown explicitly. For a risky walk w , the states between the tail and head are called intermediate states of w , denoted as SE (w ); the length of w is defined as L(w ) = |SE (w )|. Note that a risky walk with length l has l + 2 states, i.e., l intermediate states, its tail, and its head. For convenience, we sometimes use wij to denote ri ’s risky walk whose head is occupied by rj . For example, at the configuration shown in Fig. 3(a), r1 has two risky walks: w16 = ⟨s1 , s2 , s3 , s4 ⟩ and w12 = ⟨s1 , s2 , s3 , s4 , s5 ⟩; SE (w16 ) = {s2 , s3 }, L(w16 ) = 2; and SE (w12 ) = {s2 , s3 , s4 }, L(w12 ) = 3. Definition 9. W = ⟨w1 , w2 , . . ., wm ⟩ is a circuit if (1) ∀i ∈ Nm , wi is a risky walk, and h(wi ) = t(wi+1 ) where wm+1 = w1 , and (2) ∀i1 , i2 ∈ Nm , i1 ̸= i2 , wi1 and wi2 belong to two different robots. For example, the configuration shown in Fig. 3(a) contains three circuits, shown in Fig. 3(b)−(d). W1 = ⟨w16 , w61 ⟩ is a circuit since w16 and w61 are risky walks of r1 and r6 , respectively; h(w16 ) = s4 = t(w61 ) and h(w61 ) = s1 = t(w16 ). Similarly, W2 = ⟨w12 , w23 , w36 , w61 ⟩, and W3 = ⟨w12 , w23 , w34 , w45 , w56 , w61 ⟩ are circuits too. For a circuit W , the following notations are used in the sequel. I (W ) = {i1 , i2 , . . ., im } is the set of indices of robots in W , where rij is at t(wj ). SE (W ) = ∪m j=1 SE (wj ) is the set of intermediate states ij

i

of W . Sα (W ) = ∪ij ,ik ∈I (W )∧ij ̸=ik Sα ∩ Sαk denotes all collision states between any two different robots in I (W ), and Ii (W ) is the set of robots at the states of SE (W ). Next, we introduce the sub-circuit of a circuit. Suppose W = ⟨w1 , w2 , . . ., wm ⟩ is a circuit, where wj = ⟨sj1 , sj2 , . . ., sjkj ⟩ is a risky walk of rij , ∀j ∈ Nm . Select a movable robot rip , p ∈ Nm , and let it move one step forward. The risky walk wp changes to wp′ = ⟨sp2 , . . ., spkp ⟩. Then verification is done to check whether there still exists a circuit with some of the risky walks in W . The process starts from rip with wp′ . First, rip sends the information

of s2 to rip+1 . After it receives this message, rip+1 checks whether wp+1 passes over sp2 . If not, the message is again sent to rip+2 by rip+1 . Then rip+2 begins to check wp+2 . Repeat sending the message to robots until a robot, say rip1 , checks that its risky walk, wp1 , p passes over s2 . Note that during the iteration, rm , if needed, sends p the message to r1 , and r1 begins to check w1 . Thus, wp1 = ⟨s11 , p1 p1 p1 p1 p p ′ s2 , . . ., s2 , . . ., skp ⟩ changes to wp1 = ⟨s1 , s2 , . . ., s2 ⟩, and 1 W ′ = ⟨wp′ , wp+1 , . . ., wp1 −1 , wp′ 1 ⟩ is a circuit. We call W ′ subcircuit of W . Note the risky walks between wp′ and wp′ 1 in W ′ are the same as those in W , while wp′ and wp′ 1 are parts of wp and wp1 , respectively. For example, consider W3 = ⟨w12 , w23 , w34 , w45 , w56 , w61 ⟩ in ′ Fig. 4. Let r1 move to s2 . Then w12 = ⟨s2 , s3 , s4 , s5 ⟩, the risky walk of r5 becomes w51 = ⟨s8 , s2 ⟩, and r6 is excluded from W3 . Thus, ′ , w23 , w34 , w45 , w51 ⟩ is a sub-circuit of W3 . W3′ = ⟨w12 A circuit W may contain other smaller circuits. The difference between the smaller circuits and sub-circuits is that the smaller circuits must coexist with W , while sub-circuits are generated by the moves of some robots in W and cannot coexist with W . For example, as shown in Fig. 3, W1 and W2 are two smaller circuits of W3 at current configuration, but they are not sub-circuits of W3 . Definition 10. A deadlock cycle is a circuit where the length of each risky walk is 0. Proposition 2.

A deadlock cycle contains at least 3 robots.

Proof. First, consider there exists a deadlock cycle with two robots. Without loss of generality, suppose W = ⟨w1 , w2 ⟩ is a deadlock cycle. We have h(w1 ) = t(w2 ) and h(w2 ) = t(w1 ), implying that there are two vertices connected by two colors. This violates Proposition 1. Second, as shown in Fig. 2(a), there exists a deadlock cycle with three robots. ■ Proposition 3. A configuration c is a deadlock configuration if and only if there exists a deadlock cycle in c. Proof. Suppose c is a deadlock configuration satisfying c(im )•im = c(im+1 ) for m = 1, 2, . . . , k and ik+1 = i1 . Thus, rim has a risky walk wim = ⟨c(im ), c(im+1 )⟩. Hence, W = ⟨wi1 , wi2 , . . . , wim ⟩ is a deadlock cycle. Suppose W = ⟨wi1 , wi2 , . . . , wik ⟩ is a deadlock cycle at c. ∀m ∈ {1, 2, . . . , k}, since the length of wim is 0, wim = ⟨sim , sim+1 ⟩, where sim and sim+1 are the current states of rim and rim+1 , respectively; and ik+1 = i1 . So sim •im = sim+1 . This means ri1 , ri2 , . . . , rik satisfy Definition 3. Hence, c is a deadlock configuration. ■ As described before, it is difficult to predict deadlocks in advance owing to the existence of higher-order deadlocks. So we first study the characteristics of higher-order deadlocks. Definition 11. A circuit W is called a kth order deadlock if (1) for any movable robot, its one-step move causes a lower-order deadlock with these robots, and (2) there exists a robot such that

6

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

its one-step move causes a (k − 1)th order deadlock with these robots. Remark 1. There is no need to study a circuit containing a smaller one that is a higher-order deadlock. Indeed, when a robot r finds that a circuit W0 contains a small circuit W1 after it reaches its next state, W1 either contains r or does not contain r. If W1 contains r, then r should first check whether W1 is a higher-order deadlock; if ‘‘yes’’, r cannot move forward, and there is no need to further check W0 . If W1 does not contain r, note that in this case other robots forming W1 are at their current states; based on their higher-order deadlock avoidance algorithms, W1 should not be a higher-order deadlock. Hence, we focus on only simple higher-order deadlocks, meaning that the smaller circuits in a higher-order deadlock are always deadlock-free. Indeed, a kth order deadlock is a circuit W such that a deadlock occurs inevitably within k times of transitions. Here the number of transitions is counted by the moves of robots that are involved in the sub-circuits of W . Once a robot is excluded from a subcircuit of W , its future transitions are not counted to the order. For example, as shown in Fig. 4, when r1 moves to s3 , r5 is excluded from the sub-circuit, so its future transitions are not counted any more; while r1 −r4 are still in the new circuit, so their next transitions, if any, will be counted to the order. A deadlock cycle is also called the 0th order deadlock. Intuitively, a kth order deadlock occurs because the robots in the circuit W , i.e., rij , ij ∈ I (W ), are in a ‘‘circular wait’’ in order to avoid lower-order deadlocks or collisions. In other words, ri1 cannot move forward because its move can cause a lower-order deadlock. Thus, it has to wait for the robot in its path, say ri2 , to move away. However, ri2 cannot move forward since its move can also cause a lower-order deadlock. So ri2 also needs to wait for the move of the robot in its path, say ri3 . This process iterates until rim needs to wait for the move of ri1 . Thus, a circular wait occurs, and none of them can move. Note, collision avoidance leads to deadlock cycles while the (k−1)th order deadlock avoidance leads to the kth order deadlock. For example, the circuit in Fig. 4 is a second-order deadlock. At current moment, only r1 , r3 , and r4 are movable. (1) If r1 moves to s3 , r1 , . . . , r4 form a sub-circuit. For this sub-circuit, r1 and r3 are movable. However, no matter which one moves forward, a deadlock occurs. (2) If r3 moves to s4 , r1 , r3 , r4 , and r5 form a subcircuit. For this sub-circuit, r1 and r4 are movable. Similarly, the move of either of them forms a deadlock. (3) If r4 moves to s3 , r1 , r4 , and r5 form a deadlock. Thus, W3′ generates a deadlock within two steps, and it is a second-order deadlock. Lemma 1.

If W is a higher-order deadlock, SE (W ) ⊆ Sα (W ).

Proof. Suppose W = ⟨w1 , w2 , . . ., wm ⟩ is a higher-order deadlock, where wj is a risky walk of rij , ∀j ∈ Nm . If there exists s, s ∈ SE (wk ), k ∈ Nm , such that s ∈ / Sα (W ). Then, we can conclude that W is live. First, there exists an execution such that the robots between rik ’s current state and s can move away from wk without causing any deadlock except those with rik and the robot at h(wk ). Otherwise, there exists a lower-order deadlock in W , which is in conflict with the simple higher-order deadlock assumption. Then, let rik move to s. However, after it moves to s, rik cannot block the motion of other robots in W because of s ∈ / Sα (W ). This means there is no ‘‘circular wait’’ among the robots in W anymore. Thus, W is live. This is a contradiction. Hence, ∀s ∈ SE (W ), s ∈ Sα (W ), i.e., SE (W ) ⊆ Sα (W ). ■ Note that Lemma 1 describes a necessary but not necessarily sufficient condition. For example, robots r1 − r10 in Fig. 5 form a circuit W = ⟨w12 , w23 , w34 , w45 , w56 , w67 , w78 , w89 , w9,10 , w10,1 ⟩,

Fig. 5. A counter-example of the converse of Lemma 1.

Fig. 6. Illustrative circuit structure to proof of Lemma 2. The bold edges are the mov e transitions of rij and rij−1 , and the dotted ones represent the mov e transitions of the rest robots in W .

where w12 = ⟨s1 , s2 , s3 , s4 , s5 ⟩, w23 = ⟨s5 , s6 , s7 , s8 ⟩, w34 = ⟨s8 , s9 ⟩, w45 = ⟨s9 , s10 , s7 , s11 ⟩, w56 = ⟨s11 , s6 , s4 , s12 ⟩, w67 = ⟨s12 , s13 ⟩, w78 = ⟨s13 , s4 , s14 ⟩, w89 = ⟨s14 , s10 , s3 , s15 ⟩, w9,10 = ⟨s15 , s2 , s16 ⟩, and w10,1 = ⟨s16 , s1 ⟩. Clearly, W satisfies SE (W ) ⊆ Sα (W ). But it is live by leading r2 to s7 first, then r1 can move away from s5 . Hence, r10 , r9 , r8 , r7 , r6 , and r5 can move to their own private states in turns. Third, r4 moves to s10 . So r3 and r2 can move to their private states in sequence. At last, r4 can move to its private state. Lemma 2. k ≤ m − 3.

If W = ⟨w1 , w2 , . . ., wm ⟩ is a kth order deadlock,

Proof. For any movable robot rij , its one-step move will release at least the robot rij−1 from W [ ≜ T1 ]. We prove T1 using proof by contradiction. Suppose rij−1 is still in the sub-circuit of W after rij moves onej

j

j

step forward [≜ ¬T1 ]. Then, let wj = ⟨s1 , s2 , . . ., sk ⟩ be the risky j

walk of rij . wj−1 in W should satisfy: (1) j

exists at least one state between s2 and rij−1 can traverse

j s2 .

j s2 , j s1 ;

j s1

∈ wj−1 ; (2) there

and (3) only rij and

Without loss of generality, suppose wj−1 =

⟨s1j−1 , . . ., sjk−j−11 , sj2 , s, sj1 ⟩. Thus, W is with the structure shown in Fig. 6. Since W is a higher-order deadlock, there exists an execution such that rij and rij−1 still form a higher-order deadlock with some of the robots in this circuit, and the states between rij and rij−1 are empty. To release the circuit, there are two cases after such an execution. The first one is that rij does not need to move one step forward anymore, and the second one is that rij needs to move one step forward. j For the first one, let rij−1 move to s, then rij to s2 . So ri does not block any other robot. This means there is no circular wait anymore. Thus, there is no deadlock. However, W is a higherorder deadlock. So in this case, ¬T1 is not satisfied and thus T1 is satisfied. For the second case, it means that rij blocks some robots’ motion even though rij is not the robot that they need to wait

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

7

Algorithm 1: Process of higher-order deadlock detection for ri with state s. Input : Robot ri , the state to be checked, i.e., s. Output: Boolean value. 1 2

Fig. 7. m robots form an (m − 3)th order deadlock.

3 j

for leaving in the circular wait. Thus, when rij moves to s2 , other robots can move sequentially and finally all robots at the states of rij−1 can move away from their current states. Then, the robots that need to wait for the leaving of these robots can now leave the circuit. Thus, rij can move forward and does not block other 1 robots. This means rij−1 cannot form a deadlock with rij . However, if ¬T1 is satisfied, rij−1 should form a deadlock with rij . Thus, ¬T1 cannot be satisfied and T1 is satisfied. Thus, we prove that the first statement, i.e., T1 , is satisfied. Repeatedly applying T1 , we can conclude that a k-step move can release at least k robots. After k-step moves, there are still some robots that form a deadlock cycle. The number is not less than 3 based on Proposition 2. Thus, the total number of robots is not less than k + 3, i.e., m ≥ k + 3. Hence, k ≤ m − 3. On the other hand, we need to prove that there exists an (m − 3)th order deadlock with m robots. Consider the circuit W = ⟨w1 , w2 , . . ., wm ⟩ shown in Fig. 7, where (1) ∀i ∈ Nm , SE (wi ) ⊆ SE (w1 ); and (2) |SE (w1 )| = k, |SE (w2 )| = |SE (wm )| = 0, and |SE (wj )| = 1 for other risky walks. Clearly, k = m − 3. Moreover, a deadlock cycle can occur inevitably within m − 3 steps. Hence, W is an (m − 3)th order deadlock. ■

6

Break; else if ri detects another robot, say rj , at q then Ai (j) = k − 1; wij = w ; Wi = Wi ∪ {wij };

7

p = q; q = p•i ; w = ⟨w , q⟩;

4 5

8 9 10 11 12 13 14

15 16 17 18 19 20 21

If W is a kth order deadlock, |SE (W )| = k.

22

Proof. On one hand, during the motion forming a deadlock cycle, each robot moves along the intermediate states of its risky walk in W . once it has been traversed, an intermediate state is excluded from the sub-circuit. Thus, each intermediate state can be traversed at most one time during the process to form a deadlock cycle. Hence, |SE (W )| ≥ k. On the other hand, when some robots in W form a deadlock cycle after some steps of move, they are at the intermediate states of the original risky walks in W . Thus, for any higher-order deadlock, a deadlock cycle must occur when all the intermediate states are traversed. Hence, |SE (W )| ≤ k. From the above analysis, |SE (W )| = k. ■

24

Lemma 3.

According to Lemmas 2 and 3, we can get the boundary of the number of intermediate states in a higher-order deadlock. Lemma 4. For a higher-order deadlock W = ⟨w1 , w2 , . . ., wm ⟩, |SE (W )| ≤ m − 3. Moreover, for all i ∈ Nm , |SE (wi )| ≤ m − 3. The following statement gives criteria to check whether a circuit is a higher-order deadlock. Theorem 1. Suppose W = ⟨w1 , w2 , . . ., wm ⟩ is a circuit and |SE (W )| = k. W is live if (1) k > m − 3 or SE (W ) \ Sα (W ) ̸= ∅, or (2) there exists a movable robot such that its one-step move either causes no circuits or forms a sub-circuit W ′ satisfying |SE (W ′ )| > |I (W ′ )| − 3 or SE (W ′ ) \ Sα (W ′ ) ̸= ∅. Proof. If k > m − 3, W is deadlock-free based on the conversenegative proposition of Lemma 4; while if SE (W ) \ Sα (W ) ̸ = ∅, based on the converse-negative proposition of Lemma 1, W is deadlock-free. Thus, if condition (1) is satisfied, W is live. Suppose condition (2) is satisfied. If the one-step move of a robot causes no circuit among these robots, these robots cannot

/* Step 1: Search for the risky walks. p = s; q = p•i ; w = ⟨p, q⟩; Wi = ∅; Ai = ∞; for k = 1 to N − 2 do if q ∈ Sβi then /* Reach a private state.

23 25 26 27

/* Step 2: Search for the circuits that ri can form. Circuit = ∅; /* Collect circuits. Ni = {j : Ai (j) ̸ = ∞ and j ̸ = i}; /* Detect the neighbors on ri ’s path Index = NN ; for each j ∈ Ni do RW = ∅; /* RW stores the risky walks forming a circuit. RW = ⟨wij ⟩ ; Send message (rj , ⟨ri , s⟩, RW , Index) to rj and wait for response from rj ’s v isit function; Circuit = All returned circuits RWf from other robots; /* Step 3: Higher-order Deadlock Check. if Circuit == ∅ then

*/

*/

*/ */ */

*/

*/

return true; while Circuit ̸ = ∅ do Select the first circuit, say W , in Circuit; Circuit = Circuit \ {W }; Broadcast W to the system; Count SE (W ), I (W ), and Sα (W ); if (|SE (W )|> |I (W )|−3) ∨ (SE (W ) \ Sα (W ) ̸ = ∅) then return true; else Call each robot execute its check(W , rj ); return ∨j∈I (W ) check(W , rj );

be in a deadlock. Thus, W is deadlock-free. Otherwise, if the resulting sub-circuit W ′ satisfies one of the two described conditions, we can conclude W ′ is live based on the proof of condition (1). Thus, W is live. ■ Remark 2. If the computation time is available, the second condition can be extended to the following one: There exists a movable robot such that its finite-step move either causes no circuits or forms a sub-circuit W ′ satisfying |SE (W ′ )| > |I (W ′ )|− 3 or SE (W ′ ) \ Sα (W ′ ) ̸ = ∅. In the sequel, we describe the procedure to check higher-order deadlocks. The details are shown in Algorithm 1. In the algorithm, Ai is an N-dimensional vector, where Ai (j) = l < ∞ means that there exists a risky walk of ri such that the head is occupied by rj and the length is l; Ai (j) = ∞ means the current state of rj is not in S i or there exists at least one private state of ri between ri and rj ; Wi collects the risky walks that ri can construct if ri is at s. Note that each robot computes Ai and Wi independently. The algorithm contains three steps. The first one is that ri searches for its next N − 2 states and updates Ai and Wi , i.e., Lines 1−7. The second step is that ri searches for all circuits it can form by communicating with others, i.e., Lines 8−15. In this step, for each robot rj at its next N − 2 states, ri sends message (rj , ⟨ri , s⟩, RW , Index) to rj . The first parameter in the message identifies the receiver of the message; the second one denotes the robot

8

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

Function v isit(rj , ⟨ri , s⟩, RW , Index) Input: Received message (rj , ⟨ri , s⟩, RW , Index) from rj′ . 1

RW1 = RW ; j

p = scur ; q = p•j ; w = ⟨p, q⟩; Wj = ∅; Aj = ∞; 3 for k = 1 to N − 2 do /* rj checks its next N − 2 states. 2

7 8

p = q; q = p•j ; w = ⟨w , q⟩;

5 6

*/

j

if q ∈ Sβ then Break; else if rj detects a robot, say rl , at q then Aj (l) = k − 1; wjl = w ; Wj = Wj ∪ {wjl };

4

Index1 = Index \ {j}; Nj = {k : k ∈ Index1 and Aj (k) ̸ = ∞}; 11 if Nj == ∅ then 9

10

12 13 14 15 16 17 18 19 20

/* No circuits with respect to ri . RWf = ∅ and send it back to rj′ ;

*/

else for each k ∈ Nj do RWf = ⟨RW1 , wjk ⟩;/* Add a risky walk to RW1 . */ if rk == ri then Send RWf to rj′ ; else Send message (rk , ⟨ri , s⟩, RWf , Index1 ) to rk and wait for response; Once receive RWf from rk , transmit it to rj′ ;

Function check(W , rij ) 1

2 3 4 5 6

wj′ = ⟨sj2 , . . ., sjmj ⟩; W ′ = ⟨wj′ ⟩; /* Search for a sub-circuit of W with wj′ . for k = j + 1 to j + m − 2 do if k > m then wk = wk−m ; j

/* A sub-circuit is formed. wk′ = ⟨sk1 , . . ., sj2 ⟩; W ′ = ⟨W ′ , wk′ ⟩; Break; else

9

W′

10 11 12 13 14 15 16 17

j

j

Theorem 1, where W = ⟨w1 , . . ., wm ⟩ is a circuit and wj = ⟨s1 , s2 , . . ., sjmj ⟩ is a risky walk of rij . Each robot can execute this function independently once it receives the corresponding circuit W . Next, we give the complexity analysis of the deadlock detection process. Based on Algorithm 1, there are three steps in this process. The first step is to check the status of the next N − 2 states. It can be done in O(N) time. The second one is to search for the circuits by communicating with others. Note that each robot can send messages to different robots at the same time and the robots receiving the messages can check their own states simultaneously. Thus, the main computation for each robot is to check its next N − 2 states. Hence, the computation complexity for this step is O(N 2 ). The third step is to verify whether there are deadlocks in the received circuits. Suppose the number of detected circuits is M, then the computation complexity is O(M). Note that in theory, M = O(2N ) for general cases. However, in our method, during the circuit searching, each robot only needs to look ahead at most N − 1 states, and the latter robots will search for less than N − 1 states; thus, with our discretization, the number of circuits should not be large. Moreover, in practice, the number of robots in a multi-robot system cannot be too large or change greatly. Thus, our method cannot cause high complexity and can work well in practice. However, for swarm systems with complex path networks, our approach may not outperform some decentralized methods, such as the variants of Banker algorithm. 4.3. Collision and deadlock avoidance algorithm and its correctness

if s2 ∈ wk then

8

7

*/

(rk , ⟨ri , s⟩, RWf , Index) to rk , and waits for rk ’s response (Line 19); when it receives response with RWf from rk , rj further transmits this response to rj′ (Line 20). The third step of Algorithm 1 is checking of the circuits, i.e., Lines 16−27. Function check(W , rij ) in this step is used to check whether rij ’s move satisfies the second condition in

*/

= ⟨W ′ , wk ⟩;

if k == j + m − 1 then return true; else Count SE (W ′ ), I (W ′ ), and Sα (W ′ ); if (|SE (W ′ )|> |I (W ′ )|−3) ∨ (SE (W ′ ) \ Sα (W ′ ) ̸ = ∅) then return true; else return false;

activating the procedure of deadlock detection and its checked state; RW collects the risky walks delivered by the previous robots; and Index is the set of remaining robots, and it guarantees that different risky walks in the generated circuit are of different robots. Once it receives (rj , ⟨ri , s⟩, RW , Index), rj executes its v isit function, taking the received message as an input. The detailed procedure of a robot’s v isit function is given in Function v isit. Suppose rj receives a message (rj , ⟨ri , s⟩, RW , Index) from rj′ . rj first detects robots at its next N − 2 states, stored in Nj (Lines 3 − 10). If Nj = ∅, meaning that there are no circuits with rj , so rj sends RWf = ∅ back to rj′ (Lines 11 and 12). Otherwise, for each robot rk in Nj , rj adds its risky walk wik to RWf (Line 15). If rk is ri , then this branch is finished and rj sends the generated RWf back to rj′ (Lines 16 and 17); otherwise, rj sends a message

In this subsection, we give our distributed collision and deadlock avoidance algorithm and its correctness. Algorithm 2 gives the collision and deadlock avoidance algorithm for robot ri . The procedure can be described as follows. Before it moves to the next state, ri needs to check whether its transition is allowed. Lines 2−8 are the procedure for collision avoidance. If its next state is a private state, ri moves forward and updates its current state (Lines 2−6). If the next state is occupied by a robot, ri cannot move forward in order to avoid collision, and it tries a new attempt after a given delay (Lines 7−8). If collision avoidance is guaranteed, ri performs higher-order deadlock detection and determines the event to be triggered (Lines 9−19). If its move cannot cause higher-order deadlocks and it wins the negotiation, ri moves one step forward; otherwise, it re-executes the algorithm after a given delay. According to Algorithm 2, deadlock detection is to check whether there exists a higher-order deadlock when the robot is at its succeeding state. Thus, each robot looks ahead N −1 states, two endpoints and N − 3 intermediate states, to determine whether it can move forward. Theorem 2 states the correctness of Algorithm 2. Theorem 2. Under the control of Algorithm 2, the system is always live if the initial configuration c0 is live. Proof. On one hand, given a deadlock-free configuration, the new generated configuration by any robot executing its Algorithm 2 is deadlock-free. This is because the search process in Algorithm 1 makes sure that all circuits are searched. Thus, Lines 9−19 in Algorithm 2 guarantee that the move of a movable robot cannot cause any higher-order deadlock. Hence, the generated configuration is live. Since c0 is live, all reachable configurations

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

9

Algorithm 2: Collision and deadlock avoidance of ri . Input : Robot ri ’s LTS model Ti . Output: Motion without collisions and deadlocks of ri . •

i scur = ccur (i), snext = scur , and determine the negotiation region X ; i 2 if snext ∈ Sβ then

1

3 4 5 6 7

/* Next state is a private one. mov e Execute the transition scur −→i snext ; i if scur ∈ Sα then Sign(scur ) = 0; ccur (i) = snext ; else if Sign(snext ) == 1 then

*/

Fig. 8. Four robots are traversing a collision region.

Stop for a given delay and re-perform the algorithm; else 10 if Algorithm 1(ri , snext ) then 8

9

/* Without higher-order deadlocks. 11 12 13 14 15 16 17 18 19

*/

Determine the negotiation robots eX ; if Neg(eX ) == ri then mov e

Execute the transition scur −→i snext ; eX = ∅; if scur ∈ Sαi then Sign(scur ) = 0;

Fig. 9. The communication of r1 with other robots for the deadlock checking process. s2 is the state that r1 needs to check.

cnext (i) = snext ; Sign(snext ) = 1; else Stop for a given delay and re-perform the algorithm;

generated by Algorithm 2 are deadlock-free. On the other hand, we assert if its one-step move can cause a higher-order deadlock currently with a set of robots, ri first stops at its current state but can eventually move forward in the future. This is because the involved robots can move forward at least one by one due to the stop of ri , eventually allowing ri to move forward. Therefore, the system is live and each robot can move persistently. ■ 4.4. Distributed nature analysis At last, we give a brief description on the distributed nature of the proposed algorithm. According to Algorithms 1 and 2, to execute its algorithm, each robot may need to (1) check the status of its collision states via on-board sensors, and (2) communicate with its neighbors to check higher-order deadlocks. On one hand, each robot needs to retrieve the status of its own collision states, i.e., Sign(s), using its on-board sensors. Indeed, during the implementation, Sign is a set of independent local resources {Sign(s), ∀s ∈ Sα }, and each robot stores the local signals {Sign(s) : s ∈ Sαi }. Before making decision, the robot uses its sensors to retrieve the values of its collision states within its sensing range and update the related local signals stored in it. In this way, a robot manages its own local signals independently, and there is no need to synchronize the same local signal stored in different robots. For example, as shown in Fig. 8, r1 stores {Sign(s2 ), Sign(s3 ), Sign(s4 )}; r2 stores {Sign(s4 ), Sign(s6 )}; r3 stores {Sign(s3 ), Sign(s6 ), Sign(s7 )}; and r4 stores {Sign(s2 ), Sign(s7 )}. When the system is at the current configuration, r1 retrieves Sign(s2 ) = 0, Sign(s3 ) = 0, and Sign(s4 ) = 1. After r1 moves to s2 , the signal Sign(s2 ) stored in r1 is changed to 1. If it wants to move forward after r1 moves to s2 , r4 first updates its stored Sign(s2 ) to 1 and then performs its collision and deadlock avoidance algorithm. On the other hand, a robot may communicate with its neighbors for deadlock detection. For example, consider the system shown in Fig. 8. Suppose at the current configuration of the system, r1 activates the process of deadlock detection, i.e., Line

Fig. 10. A case study with 8 robots. The current state of a robot is marked with its index, and the dashed circles denote private states.

10 in Algorithm 2. To check whether its motion to s2 can cause a higher-order deadlock, r1 needs to obtain the circuits it can build if it is at s2 via communication among robots. The detailed communication process is shown in Fig. 9. First, r1 identifies r2 and thus sends a message (r2 , ⟨r1 , s2 ⟩, RW , Index) to r2 . When it receives this message, r2 invokes function v isit, adds a risky walk w23 into RW , and delivers the message to r3 . Similarly, r3 and r4 sequentially receive this message and invoke their v isit functions. Finally, r4 sends the circuit RW = ⟨w12 , w23 , w34 , w41 ⟩ back to r1 . Thus, the communication is finished, and r1 begins to check whether RW is a higher-order deadlock. Finally, as described in Section 4.1, a robot also needs to communicate with the robots in eX for the negotiation process. This process may be done by a local coordinator, which is selected from the robots in eX randomly. Once the current coordinator is failed, another one can be an alternative. Thus, there are no centralized components. 5. Simulation experiments and comparisons In this section, we give simulations of a system with 8 robots. Fig. 10 shows the LTS of this system. Suppose the initial configuration of the system is c0 = (sicur )8i=1 = (s1 , s6 , s7 , s10 , s11 , s12 , s13 , s14 ). We assume that once a robot moves away from the sequence of collision states in Fig. 10, it can move back to its initial state only through private states. Hence, our simulation goal is to ensure

10

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

Fig. 11. Estimation of time for higher-order deadlock prediction.

that the robots can move to the private states s10 , . . . , s80 (the dashed private ones); once a robot arrives at its dashed private state, we no longer consider its motion. Our simulation is implemented with MATLAB R2017a on a desktop running Windows 10, and equipped with an Intel(R) Xeon(R) CPU E5-1650 v3 3.5 GHz and 16 GB of RAM. Each robot can call its higher-order deadlock detection function independently. Since all the robots are moving to a collision region, they can communicate with each other.

r3 r7 r5 r6 r2 r2 r8 r6 r8 r6 r4 r7 r6 r4 r1 r5 r3 r7 r5 r3 r1 r3 r1 r1 r3 r1 r1 . Fig. 12 shows some snapshots of this evolution. First, r1 , r2 , and r4 cannot move forward since the move of r1 can cause a fifth-order deadlock, while the succeeding states of r2 and r4 are occupied by other robots. Thus, the set of movable robots is eX = {r3 , r5 , . . . , r8 }, and r3 obtains the right to move forward and moves to s5 . The generated configuration is c1 = (s1 , s6 , s5 , s10 , s11 , s12 , s13 , s14 ), which is shown in Fig. 12(a). Clearly, r4 is blocked by r5 at c1 , so the robots that can move forward are r2 , r5 , . . . , r8 . In the simulation, r7 is the winner of the negotiation process. Thus, r7 moves one step forward and the current configuration changes to c2 = (s1 , s6 , s5 , s10 , s11 , s12 , s3 , s14 ), shown in Fig. 12(b). From this configuration, robots r2 , r5 , r6 , and r8 are allowed to move, but finally only r5 moves forward, resulting in the configuration c3 shown in Fig. 12(c). Similarly, the next movable robot is r6 , and the generated configuration is shown in Fig. 12(d). The next two-step move of r2 leads r2 to its private state s20 , resulting in a configuration c5 = (s1 , s20 , s5 , s10 , s9 , s8 , s3 , s14 ), shown in Fig. 12(e). Next, we no longer consider the motion of r2 . With the next two successive moves of r8 r6 , the system arrives at configuration c6 = (s1 , s20 , s5 , s10 , s9 , s13 , s3 , s80 ) shown in Fig. 12(f). From this configuration, after r4 , r7 , r6 , and r4 move sequentially, r4 and r6 arrive at s40 and s60 , respectively. The configuration is shown in Fig. 12(g). The next move sequence is r1 r5 r3 r7 r5 , and the system arrives at configuration c8 , shown in Fig. 12(h). At this configuration, all robots, except r1 and r3 , arrive at their private states si0 , i = 2, 4, . . . , 8, respectively. From c8 , r1 and r3 can finally move to their own private states. The simulation video for state transitions can be found at https://youtu.be/VF15INmYM9g.

5.1. Simulation without higher-order deadlock avoidance 5.3. Comparison with other methods First, consider the evolution of the system without higherorder deadlock avoidance algorithm. Since all robots need to move into a crowded region with no private states, they have to negotiate. We apply Monte Carlo simulation method to do our simulation, i.e., each time the movable robots are randomly selected to move forward. Detailedly, we conduct 8 experiments with different simulation rounds: 10, 100, 500, 1000, 2000, 5000, 8000, and 10000. A round is an evolution of the system resulting in a deadlock or each robot to its dashed private state. A live round is an evolution of the system such that all robots can reach s10 − s80 . To count the number of live rounds, we further repeat each experiment 100 times and then compute the average number of live rounds. The average numbers of live rounds are 3.19, 30.22, 149.41, 296.9, 602.47, 1503.97, 2406.44, and 3017.65, respectively. From the results, we find that the probability is around 0.3 in this experiment. This means the system is with low reliability without any deadlock avoidance algorithm. 5.2. Simulation under higher-order deadlock avoidance Next, we show the evolution of the system under the control of Algorithm 2. We also run the system with the same experiments as before. The results show that none of them causes any deadlock. At each round, all robots can reach s10 − s80 after total 27 steps of transitions. To evaluate the efficiency of our approach, for each robot, we also compute the time for higher-order deadlock detection at each state. We run the system 100 rounds and compute the average prediction time. The results are shown in Fig. 11. From Fig. 11, we can find that the prediction can be done in milliseconds. At the initial configuration, r1 predicts a higherorder deadlock with all other robots; while r2 and r4 do not need to perform the process of higher-order deadlock prediction since their next states are occupied by r3 and r5 , respectively. We, in the sequel, show one evolution of the system from the simulation results. The sequence of the moves of robots is

Currently, one of the state-of-the-art decentralized methods is the variant of the Banker’s algorithm (VBanker) proposed in Kalinovcic et al. (2011) and Reveliotis and Roszkowska (2011). In this subsection, we compare our method with this method on the system shown in Fig. 8. The idea of VBanker is that a robot can move one step forward only when it can arrive at a private state that will not be occupied by other robots. By applying this idea to our case, a robot can move one step forward only when it contains a safe walk. Clearly, each configuration generated by VBanker is also admissible based on our method. We also give some numerical comparisons of these two methods. The metrics for comparison are the numbers of reachable configurations and routes of the system to arrive at a target configuration (s5 , s9 , s10 , s11 ). Here a route is a sequence of robots to move to the target configuration. By running with different initial configurations, we compute the numbers of reachable configurations and routes. The results are shown in Table 1. Clearly, our method allows more motion for robots. We also point out that our higher-order deadlock detection is a bit conservative compared with some centralized methods. This is because the proposed criteria for higher-order deadlocks are necessary but not sufficient, such as the counter-example given in Fig. 5. Usually, any centralized method on the discrete model can generate all admissible configurations for the system. However, it is with high computation complexity and there is lack of flexibility. 6. Discussion Compared with continuous path based methods, our method is a bit conservative. First, to avoid the situation that there are successive collision states between two robots, we consider maximal continuous collision segments between two robots. Sometimes,

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

11

Fig. 12. Some intermediate configurations during an evolution of the system. At each configuration, the current states of robots are marked by the numbers, which represent the corresponding robots. Table 1 Comparison on the numbers of reachable configurations and routes between our method and VBanker. Initialization

# configurations # routes

(s1 , s4 , s10 , s12 )

(s1 , s8 , s6 , s12 )

(s1 , s8 , s10 , s7 )

(s1 , s4 , s6 , s12 )

Ours

VBanker

Ours

VBanker

Ours

VBanker

Ours

VBanker

Ours

VBanker

18 903

14 859

34 165

28 204

19 830

14 913

7616

5960

2506

1187

5 155

4 023

9 860

8 191

5 407

4 021

2164

1669

661

316

such a maximal continuous collision segments may be refined into several successive collision states. For example, as shown in Fig. 1(b), the maximal continuous collision segments can be refined as two collision states, i.e., (ab, de) is abstracted to a collision state, and (bc , ef ) is to another one. However, such a situation is more complex. If they are moving in the opposite directions, it allows at most one robot at the two states. In this case, our method is not conservative since it is the same as one collision state. If the two robots are moving in the same direction, they can stay at the refined two states simultaneously. Our current study cannot deal with this situation. Second, to avoid unnecessarily high complexity, we do not take time into consideration in our model. Once time is considered, some deadlocks may not occur since when a robot arrives at the endpoint of its current segment, other robots have left their current states. In the future, we will consider these situations. In conclusion, there is a tradeoff between conservativeness and complexity. A more conservative method possesses lower complexity, while centralized methods can allow all possible motion but is with high complexity. Our method can keep an intelligent balance between conservativeness and complexity. Our current discussion focuses on paths satisfying that each discrete state is passed once in each round. For paths whose discrete states may be passed multiple times in a round, we can decompose it into an equivalent path containing only one circuit based on its unique motion. Suppose sj ∈ S i , and let di (sj ) be the number of occurrences of sj in the discrete path of ri . Then the decomposition can be done by decomposing sj into di (sj ) substates, say sj,1 , sj,2 , . . . , sj,di (s) , each of which replaces the original sj in the unique path of the sub-graph. Note that ri can determine the signal value of the corresponding sub-state if it detects a robot at sj . To broadcast its risky walks to others, ri still uses sj ; if it

(s1 , s4 , s6 , s7 )

receives a risky walk containing sj , ri replaces this state with the sub-state that is the nearest one from its current state. Thus, such a decomposition cannot affect others. Besides, even though our descriptions focus on closed paths, the proposed method can be applied to non-closed paths. For a non-closed path, each time a robot checks the states from its next state to at most the terminal state. Once it arrives at the terminal state, it stops forever. Moreover, on one hand, the LTS model can be changed in real time; on the other hand, our method also allows a robot to change its path in real-time as long as there is only one path for each robot at any time instant. Indeed, the modification is done based on the communication information and is detected by the sensors. Since the communication range is larger than the sensing range, a robot first detects the changes of its path and the paths of its neighbors, and modifies the related parts of its LTS model. Then, these modified states are detected by sensors when they are within the sensing range and are used for higherorder deadlock detection. Hence, the changed part of the LTS model cannot affect the current process of higher-order deadlock detection at each time instant. Besides, the two processes, i.e., LTS model modification and higher-order deadlock detection, can be done simultaneously. This also shows that our method is with flexibility and scalability. 7. Conclusion In this paper, we investigate deadlocks in multi-robot systems where each robot has a predetermined and closed path. Based on the discrete abstraction, we propose the concept of higher-order deadlocks and find that the highest order of a higher-order deadlock formed by N robots is N − 3. Then, a distributed algorithm is developed to avoid collisions and deadlocks. To perform the

12

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706

algorithm distributedly, each robot needs to check the status of its collision states ahead and communicate with its neighbors. In the future, there are several directions we can continue based on the current results. First, note that in this paper, we study higher-order deadlock avoidance without considering times spent at each state. However, having time associated to each state will be more interesting and practical but also more complicated. It is the first direction for our future work. Second, currently we do not consider the situation that there are multiple robots moving along the same path. In the future, we can study higherorder deadlocks for such a scenario, which is quite different from the current study. Third, we may also study more complex and dynamic environment, for example, there are humans and animals in the environment. We can further consider systems where a robot may have multiple paths. Performance optimization after collision and deadlock avoidance, especially when time is taken into account, is also an interesting topic. Acknowledgments This work was supported by the Natural Science Foundation of China under Grant Nos. 61573265, 61203037, 51305321, 61751210, 61572441, and 61973242, Fundamental Research Funds for the Central Universities under Grant Nos. K7215581201, K5051304004, and K5051304021, New Century Excellent Talents in University under Grant No. NCET-12-0921, and Academic Research Fund Tier 2 by Ministry of Education in Singapore under Grant No. MOE2015-T2-2-049. References Akella, S., & Hutchinson, S. (2002). Coordinating the motions of multiple robots with specified trajectories. IEEE international conference on robotics and automation (pp. 624-631), Washington, DC, USA. Alonso-Mora, J., DeCastro, J. A., Raman, V., Rus, D., & Kress-Gazit, H. (2018). Reactive mission and motion planning with deadlock resolution avoiding dynamic obstacles. Autonomous Robots, 42(4), 801–824. Chen, J., Moarref, S., & Kress-Gazit, H. (2018). Verifiable Control of Robotic Swarm from High-level Specifications. In Proceedings of the 17th international conference on autonomous agents and multiagent systems (pp. 568–576), Stockholm, Sweden. Chen, H., Wu, N. Q., & Zhou, M. C. (2016). A novel method for deadlock prevention of AMS by using resource-oriented Petri nets. Information Sciences, 363, 178–189. Dijkstra, E. W. (1968). Cooperating sequential processes. In F. Genuys (Ed.), The origin of concurrent programming (pp. 65–138). New York: Academic. Dimarogonas, D. V., Loizou, S. G., Kyriakopoulos, K. J., & Zavlanos, M. M. (2006). A feedback stabilization and collision avoidance scheme for multiple independent non-point agents. Automatica, 42(2), 229–243. Fanti, M. P., Mangini, A. M., Pedroncelli, G., & Ukovich, W. (2015). Decentralized deadlock-free control for AGV systems. In American control conference (pp. 2414–2419), Chicago, IL, USA. Ge, S. S., & Cui, Y. J. (2002). Dynamic motion planning for mobile robots using potential field method. Autonomous Robots, 13(3), 207–222. Hu, H., Su, R., Zhou, M., & Liu, Y. (2016). Polynomially complex synthesis of distributed supervisors for large-scale AMSs using Petri nets. IEEE Transactions on Control Systems Technology, 24(5), 1610–1622. Hu, H., & Zhou, M. C. (2015). A petri net-based discrete-event control of automated manufacturing systems with assembly operations. IEEE Transactions on Control Systems and Technology, 23(2), 513–524. Jager, M., & Nebel, B. (2001). Decentralized collision avoidance, deadlock detection, and deadlock resolution for multiple mobile robots. In IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 3 (pp. 1213–1219), Maui, HI, USA. Kalinovcic, L., Petrovic, T., Bogdan, S., & Bobanac, V. (2011). Modified Banker’s algorithm for scheduling in multi-AGV systems. In IEEE conference on automation science and engineering (pp. 351–356), Trieste, Italy. Khamis, A., Hussein, A., & Elmogy, A. (2015). Multi-robot task allocation: A review of the state-of-the-art. In A. Koubâa, & E. J. R. M. Dios (Eds.), Cooperative Robots and Sensor Networks 2015 (pp. 31–51). Switzerland: Springer.

Kim, K. D., & Kumar, P. R. (2014). An MPC-based approach to provable systemwide safety and liveness of autonomous ground traffic. IEEE Transactions on Automatic Control, 59(12), 3341–3356. Kloetzer, M., Mahulea, C., & Gonzalez, R. (2015). Optimizing cell decomposition path planning for mobile robots using different metrics. In International conference on system theory, control and computing (pp. 565–570), Cheile Gradistei, Romania. Kshemkalyani, A. D., & Singhal, M. (2011). Distributed computing: Principles, algorithms, and systems. Cambridge, UK: Cambridge University Press. Lee, C. C., & Lin, J. T. (1995). Deadlock prediction and avoidance based on Petri nets for zone-control automated guided vehicle systems. International Journal of Productions Research, 33(12), 3249–3265. Li, Z. W., Wu, N. Q., & Zhou, M. C. (2012). Deadlock control of automated manufacturing systems based on Petri nets – A literature review. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4), 437–462. Ma, H., Kumar, T. K. S., & Koenig, S. (2017). Multi-agent path finding with delay probabilities. In Thirty-First AAAI conference on artificial intelligence (pp. 3605–3612), San Francisco, CA, USA. Moorthy, R. L., Hock-Guan, W., Wing-Cheong, N., & Chung-Piaw, T. (2003). Cyclic deadlock prediction and avoidance for zone-controlled AGV system. International Journal of Production Economics, 83(3), 309–324. Pivtoraiko, M., & Kelly, A. (2011). Kinodynamic motion planning with state lattice motion primitives. In IEEE/RSJ international conference on intelligent robots and systems (pp. 2172–2179), San Francisco, CA, USA. Reveliotis, S. (2006). Real-time management of resource allocation systems: A discrete event systems approach. New York: Springer Science & Business Media. Reveliotis, S. A., & Roszkowska, E. (2011). Conflict resolution in free-ranging multivehicle systems: A resource allocation paradigm. IEEE Transactions on Robotics, 27(2), 283–296. Soltero, D. E., Smith, S. L., & Rus, D. (2011). Collision avoidance for persistent monitoring in multi-robot systems with intersecting trajectories. In IEEE/RSJ international conference on intelligent robots and systems (pp. 3645–3652), San Francisco, CA, USA. Uzam, M., & Zhou, M. C. (2007). An iterative synthesis approach to Petri netbased deadlock prevention policy for flexible manufacturing systems. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 37(3), 362–371. Van Den Berg, J., Snape, J., Guy, S. J., & Manocha, D. (2011). Reciprocal collision avoidance with acceleration-velocity obstacles. In IEEE international conference on robotics and automation (pp. 3475–3482), Shanghai, China. Wang, X., Kloetzer, M., Mahulea, C., & Silva, M. (2015). Collision avoidance of mobile robots by using initial time delays. In IEEE conference on decision and control (pp. 324–329), Osaka, Japan. Xing, K., Wang, F., Zhou, M. C., Lei, H., & Luo, J. (2018). Deadlock characterization and control of flexible assembly systems with Petri nets. Automatica, 87, 358–364. Yadav, N., Yadav, S., & Mandiratta, S. (2015). A review of various mutual exclusion algorithms in distributed environment. International Journal of Computer Applications, 129(14), 11–16. Yu, J., & LaValle, S. M. (2016). Optimal multirobot path planning on graphs: Complete algorithms and effective heuristics. IEEE Transactions on Robotics, 32(5), 1163–1177. Zhou, Y., Hu, H., Liu, Y., & Ding, Z. (2017). Collision and deadlock avoidance in multirobot systems: A distributed approach. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(7), 1712–1726. Zhou, Y., Hu, H., Liu, Y., Lin, S.-W., & Ding, Z. (2018). A distributed approach to robust control of multi-robot systems. Automatica, 98, 1–13. Zhou, Y., Hu, H., Liu, Y., Lin, S.-W., & Ding, Z. (2019). A real-time and fully distributed approach to motion planning for multirobot systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(12), 2636–2650.

Yuan Zhou received the M.S. degree in computational mathematics from Zhejiang Sci-Tech University, Hangzhou, China, in 2015, and the Ph.D. degree in computer science from Nanyang Technological University (NTU), Singapore, where he is currently a Research Fellow. His research interests include multi-robot systems, motion control, discrete event systems, and artificial intelligence in robotics.

Y. Zhou, H. Hu, Y. Liu et al. / Automatica 112 (2020) 108706 Hesuan Hu received the B.S. degree in computer engineering and the M.S. and Ph.D. degrees in electromechanical engineering from Xidian University, Xi’an, Shaanxi, China, in 2003, 2005, and 2010, respectively. He is currently a Full Professor at Xidian University. His research interests include discrete event systems and their supervisory control techniques, Petri nets, automated manufacturing systems, and artificial intelligence. Yang Liu graduated in 2005 with a Bachelor degree of Computer Science in the National University of Singapore (NUS). In 2010, he obtained his Ph.D. in Computer Science and continued with his post doctoral work in NUS. He is currently a Full Professor at School of Computer Science and Engineering, Nanyang Technological University, Singapore. His research focuses on security, cyber–physical systems, and formal methods.

13

Shang-Wei Lin received his B.S. degree in Information Management and the Ph.D. degree in Computer Science and Information Engineering, both from the National Chung Cheng University, Chiayi, Taiwan, in 2003 and 2010, respectively. He is currently an Assistant Professor at School of Computer Science and Engineering, Nanyang Technological University, Singapore. His research interests include formal methods, embedded systems, and cyber–physical systems. Zuohua Ding received the M.S. degree in computer science and the Ph.D. degree in mathematics, both from the University of South Florida, Tampa, FL, USA, in 1996 and 1998, respectively. He is currently a Professor and the Director with the Laboratory of Intelligent Computing and Software Engineering, Zhejiang Sci-Tech University, Hangzhou, China. His current research interests include system modeling and analysis, intelligent systems, and service robots.