- Email: [email protected]
LTL Planning in Dynamic Environments
LTL Planning in Dynamic Environments ! Marius Kloetzer ∗ Cristian Mahulea ∗∗ ∗
Dept. of Automatic Control and Applied Informatics, Technical University “Gh. Asachi” of Iasi, Romania (e-mail: [email protected]). ∗∗ Arag´ on Institute of Engineering Research (I3A), University of Zaragoza, Spain (e-mail: [email protected])
Abstract: This paper addresses the path planning problem of a mobile robot based on linear temporal logic formulae over a set of regions of interest in the environment. The regions have fixed and known locations but have some dynamic observations that can appear and disappear based on exponential probability density functions. The robot movement capabilities are modeled by a finite-state transition system. By drawing inspiration from temporal logic control for static environments, we find a run that has the greatest chance of satisfying the temporal logic specification. Then, we devise a supervising strategy for the robot motion along this run, which updates the execution based on the currently observed regions. The approach is supported by two simulations. Keywords: Temporal logic, mobile robots, transition systems, probabilistic models, formal languages. 1. INTRODUCTION Recent researches on mobile robots propose different methods for extending the motion tasks to rich and human-like specifications (Kress-Gazit, 2011; Fainekos et al., 2009; Kloetzer and Belta, 2008; Belta et al., 2007). Thus, it is possible for some classes of robots to automatically perform missions as “visit region A or B, and if A is visited go to C, otherwise visit infinitely often B and C while avoiding A” rather than classic problems as “go to A while avoiding obstacles” (Choset et al., 2005; LaValle, 2006). The first challenge in enabling rich tasks is the choice of a language allowing expressive specifications and intuitive translation from human language to syntactically correct formulas. Solutions were found in a variety of formal languages (Clarke et al., 1999; Hopcroft et al., 2006). After choosing a specification class, the second challenge is the development of algorithms that actually control the robot movement such that the desired mission is accomplished. Again, inspiration was drawn from formal analysis, and solutions were developed by appropriately adjusting and extending the available algorithms (Clarke et al., 1999; Hopcroft et al., 2006; Baier and Katoen, 2008). In this paper, we will express robotic missions by linear temporal logic formulas, which is a common choice for enabling human-like specifications (Loizou and Kyriakopoulos, 2004; Kloetzer and Belta, 2008; Fainekos et al., 2009; Wongpiromsarn et al., 2009). Tasks given in this logic usually imply the attainment of some regions from the environment in a specific order, the avoidance of other re! At the Technical University of Iasi, this work has been partially supported by the CNCS-UEFISCDI grant PN-II-RU PD 333/2010. At University of Zaragoza, this work has been partially supported by CICYT - FEDER project DPI2010-20413 and by Fundaci´ on Arag´ on I+D.
gions, and various logical and temporal connectives among regions. We extend the results of the mentioned works by assuming dynamic environments rather than static ones. Specifically, we consider that the regions of interest have fixed locations, but they appear and disappear based on some probabilistic density functions. As customary for this type of problems, the robot movement capabilities are abstracted to a finite-state description, and then the problem is solved by developing supervising methods for the obtained model. The focus of this work will be on providing such a supervising strategy, and therefore we assume a simple robot and we do not insist on the continuous control laws involved when constructing a finite-state model. However, this does not restrict the applicability of our approach to certain robot types or different dynamic systems for which abstraction techniques were already developed (Habets and van Schuppen, 2004; Habets et al., 2006; Belta and Habets, 2006; Kloetzer et al., 2010). In the discrete event driven model we embed probabilistic information on the appearance and disappearance of regions, and then we find a robotic path in form of a run that has the maximum probability of satisfying the given specification. For accomplishing this, we adapt model checking and temporal logic control strategies (Holzmann, 2004; Kloetzer and Belta, 2008). Then, we develop a supervising strategy that reacts to the regions of interest observed during the evolution by updating the system behavior. Related works that address the problem of planning mobile robots based on high level specifications and probabilistic information include (Lahijanian et al., 2010; Ding et al., 2011). In (Lahijanian et al., 2010) the probabilistic information comes from robot sensing and control uncertainties, and not from environment evolution, while the
specifications are restricted to another formalism, namely probabilistic computation tree logic. (Ding et al., 2011) assumes a scenario similar with the one we consider, with additional probabilities on the outcomes of robot control actions, but with specifications involving both robot locations and events that may happen in those locations. Also, the solution from (Ding et al., 2011) is different, involving robot models as Markov decision processes and more complex deterministic Rabin automata, while the control strategy is not adjusted based on current observations from the environment. The remainder of the paper is structured as follows. Sec. 2 introduces some necessary preliminaries and states the problem we solve. The solution is given in Sec. 3, two simulations are presented in Sec. 4, and Sec. 5 formulates some concluding remarks. 2. PRELIMINARIES AND PROBLEM FORMULATION Subsection 2.1 briefly introduces the formalism that will be used for specifying robotic motion tasks, and defines an automaton needed for a formal description of the solution. Subsection 2.2 gives the generic structure of a finite transition system, that will be adapted for obtaining a suitable model for a mobile robot, and subsection 2.3 formulates the problem we solve. 2.1 Linear Temporal Logic In this work, we consider motion tasks given as formulae of LT L−X , which is a fragment of Linear Temporal Logic (LTL) Clarke et al. (1999). Informally, an LT L−X formula is recursively defined over a set of atomic propositions Π, by using the standard Boolean operators (¬ - negation, ∨ - disjunction, ∧ - conjunction, ⇒ - implication, and ⇔ equivalence) and some temporal operators with straightforward meaning (! - always, ♦ - eventually, and U until). By interconnecting these operators one can obtain rich and human-like specifications, which we will use as motion specifications for a mobile robot. For exemplification, consider that Π contains the labels of some regions of interest from an environment. A standard navigation task Choset et al. (2005); LaValle (2006) as “ eventually reach π1 , while always avoiding π2 and π3 ” can be expressed by the LT L−X formula ♦π1 ∧ !¬(π2 ∨ π3 ). A surveillance mission - “visit π1 and then π2 infinitely often” - is described by formula !♦(π1 ∧ ♦π2 ), a choice reachability mission - “eventually reach either region π1 or π2 ” corresponds to formula ♦(π1 ∨ π2 ). A formula as ♦!(π1 ∧ π2 ) describes a convergence mission (“eventually reach π1 and π2 and stay there for all future times”) that can be satisfied by a single robot if regions π1 and π2 overlap (otherwise, at least two cooperating robots would be necessary). With respect to standard LTL, LT L−X lacks the “next” operator, which is meaningless for continuous trajectories (as are those generated by a moving mobile robot). However, the intended meaning of the “next” operator can be replaced by an adequate usage of the “until” operator, as in the following example. A task requiring that “region
π1 is visited, and then π2 is directly entered from π1 ” is expressed by the LT L−X formula ♦(π1 Uπ2 ). Therefore, we will simply denote by LTL the fragment that we use. A formal definition of the syntax and semantics of LTL formulas can be found in Clarke et al. (1999). LTL formulas are interpreted over infinite strings with elements from 2Π (the set of all subsets of Π, including the empty set ∅). Any LTL formula over set Π can be transformed into a B¨ uchi automaton (see Def. 1) that accepts all and only the input strings satisfying the formula Wolper et al. (1983). Available software tools allow such conversions Holzmann (2004); Gastin and Oddoux (2001). Definition 1. The B¨ uchi automaton corresponding to an LTL formula over the set Π has the structure B = (S, S0 , ΣB , →B , F ), where: • • • • •
S is a finite set of states; S0 ⊆ S is the set of initial states; ΣB = 2Π is the set of inputs; →B ⊆ S × ΣB × S is the transition relation; F ⊆ S is the set of final states.
The transitions in B can be non-deterministic, meaning that from a given state there may be multiple outgoing transitions enabled by the same input, i.e. we may have (s, σ, s# ) ∈→B and (s, σ, s## ) ∈→B , with s# += s## . Thus, an input sequence can produce more than one sequence of states (called run). An infinite input word (sequence with elements fromΣ B ) is accepted by B if the word produces at least one run of B that visits set F infinitely often. 2.2 Transition systems For a partitioned environment cluttered with possibly overlapping and static regions of interest from set Π, the movement capabilities of a mobile robot can be abstracted into a finite state transition system as in Def. 2. Environment partitions can be obtained with cell decomposition algorithms Berg et al. (2008), while details on abstractions suitable for different robot dynamics and cell shapes can be found in Habets et al. (2006); Belta and Habets (2006). The idea is that every cell from the partition corresponds to a single state in the finite state description, and transitions between states correspond to movement capabilities. A satisfaction map shows the regions of interest that are satisfied when the robot is inside a particular cell (by assuming that the regions and their intersections are equal with a union of partition cells). Definition 2. A finite" state transition system is a tuple ! T = Q, q0 , δT , ΠT , |= , where: • Q is the finite number of states (cells from partition); • q0 is the initial state (cell containing the initial deployment of robot); • δT ⊆ Q × Q is the transition relation, where (q, q # ) ∈ δT , with q += q # , if and only if the robot can be steered from any initial condition in q to the adjacent cell q # in finite time, and (q, q) ∈ δT if the robot can be kept inside cell q for an indefinite time; • ΠT = 2Π is the set of possible observations (outputs of states of T );
• |=: Q → ΠT is a satisfaction map, where |= (q) is the set of all regions from Π that contain cell labeled by q, and |= (q) = ∅ if cell q belongs to the left-over space (q is not included in any region of interest). Transition systems that we use are deterministic, fact which implies that any feasible transition in the current state can be chosen by imposing a specific robot control law. A run (or path) of T is an infinite sequence r = q0 q1 q2 ..., with the property that (qi , qi+1 ) ∈ δT , ∀i ≥ 0. A run corresponds to a robot movement through cells from partition, and it induces (through map |=) an output word, which is the observed sequence of elements fromΠ T . SinceΠ T = ΣB , an output word of T is an input word for B, and thus one can make the connection between robot trajectories and satisfaction of LTL specifications. Researches as Fainekos et al. (2005); Kloetzer and Belta (2008) proposed algorithms that abstract a robot motion into a transition system T and use model-checking inspired algorithms for controlling T such that it satisfies a given LTL formula. Unlike such works, we assume that regions of interest appear and disappear based on probability density functions. For accounting scenarios where regions of interest appear and disappear, in this work we will modify the finite state representation T and the path finding strategy from Kloetzer and Belta (2008), and then we will develop an automated supervising strategy for the mobile robot.
As in subsection 2.2, we assume that the environment is partitioned (e.g. with respect to regions from Π and their intersections) in convex and polyhedral cells, and the set of obtained cells is denoted by Q. This partition enables us to model the motion of the robot by a finite state representation (transition system from Def. 2). Moreover, since the robot is fully-actuated, the transition relation δT is reflexive and symmetric. For actually moving the robot between adjacent cells we will use the standard strategy from (Choset et al., 2005): the current robot position is linked by a line segment to the middle point of the common facet of the current and the next cell. Thus, a run of T corresponds to a robot trajectory formed by connected line segments, and it can be easily followed by a fully-actuated and point robot. For more complicated dynamics other control laws can be used, as in Habets and van Schuppen (2004); Belta and Habets (2006), but such details go beyond the scope of this paper. Problem: Given a task as an LTL formula over the set Π, find a robot control strategy that yields the maximum probability of satisfying the task. To solve the problem, in the next section (1) we extend the structure of the transition system from Def. 2 by adding probability information, (2) we compute a product automaton and find a run of T , and finally (3) we design a supervising method that adjusts the strategy for following the run based on the currently observed propositions from Π.
2.3 Problem formulation
3. SOLUTION
Assume an environment where a mobile robot evolves and some regions of interest are defined. Each region of interest has a fixed and known position, but it can appear or disappear during robotic motion. The robot is assumed to be of negligible size and fully-actuated, and the environment boundaries and the regions are polygonal. Let Π= {π1 , . . . , πn } be the set containing the labels of regions of interest, and let ∅ be a label corresponding to the free space (space not covered by a region). Initially, we assume that each region of interest πi is visible, but will disappear after a random delay with negative exponential probability density function. The disappearance of a region does not mean that the region is not accessible by the robot, but it means that its observation is not visible. The parameter of the probability density function associated with the disappearance of πi is the rate λdi . Therefore, the average time delay of passing from visible to invisible is 1/λdi . When invisible, πi will appear (become visible) after a random delay with negative exponential probability density function, in this case with rate λai . Since the exponential probability density function has the Markov property, the appearance / disappearance of a region is a Markov process, and therefore it is possible to compute the steady-state probability of each region πi to exist. The appearance/disappearance of different regions are independent (not correlated) and the probability of region πi to exist is computed immediately as p πi =
λai λai + λdi
(1)
As briefly mentioned, our solution follows three main steps. In subsection 3.1, the steady state probabilities from (1) are embedded in the transition system modeling the robot, by accounting possible overlapping of some regions. In subsection 3.2 we construct the product between the probabilistic transition system and the B¨ uchi automaton corresponding to the LTL formula, and we find a path with the maximum probability of producing observations that satisfy the formula. The supervising strategy from subsection 3.3 keeps track of formula satisfaction and updates if necessary the evolution of the transition system when following the imposed path. 3.1 Robot model ! " Let T = Q, q0 , δT , ΠT , |= be the transition system for an environment with static regions of interest. In our case, map |= - created when all regions exist (initial moment) - becomes inaccurate, because observations change when regions disappear. Therefore, we extend T by adding a probabilistic observation function ρ : Q × ΠT → [0, 1], where ρ(q, p) is the probability of observing (satisfying) all and only regions from p ∈ ΠT when the current state is q. We note that the empty observation ∅ in a state q of T means that (at a given time instant) there is no region appeared that includes cell q. For example, if the cell corresponding to state q is included in the intersection of two appeared regions π1 and π2 , than the observation at that instant would be p = {π1 , π2 }, and if both π1 and
π2 disappear at another instant, the observation would be p = ∅. The probabilistic observation function ρ can be computed based on the probabilities of regions from Π to be active (pπi computed in (1)). We find ρ(q, p), ∀p ∈ 2|=(q) as follows: # (1 − pπ ) (2) ρ(q, ∅) = π∈|=(q)
|=(q)
, p += ∅, % $ # pπ · ρ(q, p) =
∀p ∈ 2
π∈p
#
π∈|=(q)\p
(1 − pπ )
(3)
Recall that ρ(q, p) is the probability of observing all regions from set p, and not other additional regions from set |= (q). Result (2) comes from the fact that the appearance / disappearance of regions are not correlated, and 1 − pπ is the probability of not observing region π at a time instant. The first product from (3) means that all regions from set p are appeared, and the second product implies that no other additional region exists. 3.2 Path finding This subsection presents an automatic procedure for finding a path in T such that the produced word has the greatest chance of satisfying the LTL formula. This chance comes from the fact that observations of T are probabilistic. We begin by constructing a special type of product automaton between T and B. Let us denote by P(s→B s! ) ⊆ 2Π the set of all inputs of B that enable a transition from s to s# , i.e. P(s→B s! ) = {p ∈ 2Π | (s, p, s# ) ∈→B }. Definition 3. The product automaton A = T × B is constructed as follows: A = (SA , SA0 , δA , ωA , FA ), where: • SA = Q × S is the set of states; • SA0 = {q0 } × S0 is the set of initial states; • δA ⊆ SA × SA is the transition relation, defined by: ((q, s), (q # , s# )) ∈ δA if and only if (q, q # ) ∈ δT and ∃p ∈ ΠT such that ρ(q, p) > 0 and (s, p, s# ) ∈→B ; • ωA : δA → [', ∞), with ' > 0 being a small constant, is a weighting (cost) function for tran# # sitions , ωA ((q, s), (q , s )) = ' − *+ of A, defined by: # # log p∈P(s→B s! ) ρ(q, p) , ∀ ((q, s), (q , s )) ∈ δA ; • FA = Q × SA is the set of final states. Product automaton A is inspired by algorithms in (Kloetzer and Belta, 2008), but its construction is adapted such that it includes the available probabilistic information in relation δA and map ωA . A transition in A is a match between a transition of B caused by the current observation of T and a transition of T . Since observations of T are probabilistic, transitions of A have a probabilistic nature, in the sense that the same transition in A is feasible if T has a certain observation and it may be unfeasible for other observations. The probability of existence for a transition of A is captured by its cost assigned by map ωA . The sum of observation probabilities from ωA encapsulates all observations of T that can produce the same transition in B. We considered
the costs as logarithms of probabilities because in this way, when finding a minimum-cost path by standard search algorithms (e.g. Dijkstra (Corman et al., 2001)), we basically find the path whose product of probabilities is maximum. The ' constant guarantees that if more paths with the same optimum cost exist, the one with the smallest number of different states will be chosen (' = 0 would induce paths with unnecessary zero-weighted transitions). The acceptance condition of A is formulated similar to the one of B, in the sense that an infinite run is accepted by A if and only if it visits infinitely often the set of final states FA . If A has at least one accepted run, than it has an accepted run in a prefix-suffix form, consisting of a finite sequence of states (called prefix) followed by infinite repetitions of another finite sequence (called suffix). By using graph searches, we find in A an accepted path that has the sum of costs of prefix and suffix minimum - as explained above, this guarantees the maximum probability of having correct observations in T (observations that keep feasible the followed transitions of A). Of course, one can differently weight prefix / suffix costs, for example by considering that the suffix is infinitely repeated. The found path of A will be projected to a path of T , and the probability of the path of A is basically a measure that the generated observation string of T satisfies the LTL formula. However, for ensuring this we have to develop a supervising strategy, which adapts the robot motion plan (transitions to be taken in T ) each time the current observation of T is not consistent with the current transition of A. 3.3 Supervisory robot control Let us denote by runA = (q0 , s0 ) (q1 , s1 ) (q2 , s2 ) . . . (qs , ss ) . . . (qp , sp ) . . . the optimum path of A, where the suffix is marked by bold font. The corresponding run of T is runT = q0 q1 . . . qs . . . qp . . ., and the one of B is runB = s0 s1 . . . ss . . . sp . . .. From construction of automaton A, we have the guarantee that runB is an accepted run of B, and thus runT can produce a sequence of observations (inputs to B) that satisfies the LTL formula. As noted in subsection 3.2, optimization on runA guarantees the maximization of the probability that T produces an output word satisfying the LTL formula. However, when T follows its run (the mobile robot moves through partition cells corresponding to states in runT ), we have to check the current observation and decide if that produces the desired transition in runB . Let us denote by h ∈ ΠT the current observation of T (the set of appeared regions of interest that contain the mobile robot position at the current time instant). During robot movement, B plays the role of a test tool for the satisfaction of LTL formula. For accomplishing this, we store and update a set C B ⊆ S containing the possible current (or consistent) states from B that can be reached due to the inputs applied so far to B (these inputs are observations from T ). Initially, C B = {S0 }, and it is updated as in (4) whenever the robot enters a new region from runT or when the current observation h changes.
C B := {s ∈ S | ∃s# ∈ C B s.t. (s# , h, s) ∈→B }
(4)
During the motion, whenever the robot enters in a state (partition cell) qi from runT , and at any change in h, we update C B , check which of the following cases is true and perform the corresponding action. Of course, the following supervising strategy begins with i = 0 (the robot deployed in q0 ): (i) if si+1 ∈ C B , then T advances to the next state qi+1 ; (ii) if si+1 ∈ / C B , but si ∈ C B , then T stays (self-loops) in qi ; (iii) if si , si+1 ∈ / C B , but C B += ∅, then we have to modify the run of T to be followed from now. Consider q0 = qi in T , S0 = C B in B, and update the set of initial states of A. Search for a new optimum runA , find runT and runB , and restart the supervising strategy according to these paths; (iv) if C B = ∅, then stop the movement and report that the LTL formula was just violated by the current observation h. Informally, case (i) means that until now the desired run of B is produced by the actual observation of T . Case (ii) means that the desired run of B does not advance, but the current observation h enables a self-loop in its actual state. The robot waits for a change in observation of T - such a change can appear, because runB can be produced by some probabilistic observations along runT . In case (iii), the current observation h prevented runB of being followed, but it did not block the entire evolution of B. Therefore, we search for a different continuation of the evolution that satisfies the formula. Case (iv) can appear only when any feasible robot trajectory can produce (besides desired observations) some observations that are forbidden by the LTL formula - such an example will be included in Sec. 4. The probability-based optimization from subsection 3.2 implies that along the evolution case (i) is most likely to appear, case (iii) rarely appears, and case (iv) can appear only when there’s no way of satisfying the formula while avoiding its possible violation. The waiting times from case (ii) are also minimized by large probabilities of observing the desired output along runT . 4. SIMULATIONS Let us consider the planar environment from Fig. 1(a), composed by 22 triangular cells q1 , . . . , q22 and three regions of interest: Π= {π1 , π2 , π3 }. The satisfaction map |= for the initial moment is defined as follows: • • • • •
|= (q10 ) = |= (q12 ) = π1 ; |= (q9 ) = {π1 , π2 }; |= (q1 ) = |= (q4 ) = |= (q5 ) = |= (q8 ) = π2 ; |= (q16 ) = {π2 , π3 }; |= (q18 ) = |= (q20 ) = |= (q22 ) = π2 .
Let us assume that π1 has a disappearance rate of λd1 = 0.3, i.e., the average time delay after which the region disappears is 1/0.3, and an appearance rate of λa1 = 0.7. According to (1), the steady-state probability of observing 0.7 π1 is pπ1 = 0.7+0.3 = 0.7. For π2 we assume λa2 = λd2 = 0.5,
0.5 implying pπ2 = 0.5+0.5 = 0.5, while for π3 we assume 0.2 a d λ3 = 0.2 and λ3 = 0.8, obtaining pπ3 = 0.2+0.8 = 0.2.
Based on these probabilities, the probabilistic observation function ρ is immediately computed by applying (2) and (3). For example, for q9 , where two observations can appear (|= (q9 ) = {π1 , π2 }), we have: • • • •
ρ(q9 , ∅) = (1 − pπ1 ) · (1 − pπ2 ) = 0.3 · 0.5 = 0.15; ρ(q9 , {π1 }) = pπ1 · (1 − pπ2 ) = 0.7 · 0.5 = 0.35; ρ(q9 , {π2 }) = (1 − pπ1 ) · pπ2 = 0.3 · 0.5 = 0.15; ρ(q9 , {π1 , π2 }) = pπ1 · pπ2 = 0.7 · 0.5 = 0.35.
Let us assume that the robot is initially placed in q21 and its task is given by the following LTL formula: ♦ ((π1 ∧ π2 ) ∧ ♦π3 ) Consequently, the robot should first reach a region where both π1 and π2 are simultaneously satisfied (in our case the only possibility is to reach q9 ) and then it should reach a region with the observation π3 . Applying the results in subsection 3.2 the following path is obtained: runT = q21 q19 q17 q10 q12 q9 q8 q5 q16 q16 . . . (the suffix is q16 ). Fig. 1 shows an evolution to follow this trajectory by applying the supervisory control presented in subsection 3.3 by using a MATLAB implementation. The robot should first reach q9 (hoping that both π1 and π2 will be active). During the movement to q9 the robot passes through q21 , q19 , q17 , q10 , q12 and some regions disappear (π1 and π3 ), but π3 appears after some time. The positions of the robot when the regions changed are represented by bullets. However, since these changes of observations are not affecting the mission, the robot will move by following the computed path (case (i) in the supervising strategy). Once the robot arrives in q9 , π2 is not appeared and the robot waits (case (ii) in the supervising strategy). After some time, both π1 and π2 are observed, and the robot leaves q9 and moves towards q16 . Notice that when the robot arrives to q16 (Fig.1.(c)) the region π3 is not appeared and the robot waits. When π3 becomes visible the robot moves to the centroid of q16 and remains there. Let us assume the same environment and observation probabilities, but an initial robot position in q14 and a mission given by the following LTL formula: !¬ (π1 ∨ π2 ) ∧ ♦π3 Hence, the robot should reach a region with observation π3 , while always avoiding observations π1 and π2 . In order to accomplish the mission the robot should pass through at least one region with |= (q) = π1 or |= (q) = π2 , hoping that during the motion in a such region the corresponding observation will not be active. The path computed by method from subsection 3.2 is: runT = q14 q6 q1 q3 q2 q7 q20 q20 . . .. Notice that the robot should pass through q1 , where π2 may appear and violate the formula. A possible trajectory is presented in Fig. 2. For this situation, when the robot enters in q1 the region π2 is not observed (Fig. 2.(a)), but after a small amount of time π2 becomes active and the formula is violated. In this case, the robot stops (case (iv) in the supervising strategy). Of course, for other simulations it is possible that π2 does not appear while the robot crosses q1 , and the formula is
(a)
(b)
(c)
(d)
Fig. 1. Environment and continuous trajectories of the robot for the first example from Sec. 4. The bullets mark the positions of the robot when a region appears or disappears. (a) Planar environment with all regions appeared. (b) Robot waits in q9 the appearance of π1 . (c) Robot waits in q16 the appearance of π3 . (d) Mission is accomplished and the robot stops in the middle of q16 . satisfied. Observe that the strategy from subsection 3.2 did not produce a path that traverses cell q10 (instead of q1 ), because such a trajectory would have smaller chances of satisfying the formula (pπ1 > pπ2 ). We mention that future work will focus on performing examples with real robots on the experimental platform reported in (Kloetzer et al., 2012). 5. CONCLUSIONS This paper proposes an approach for supervising the motion of a mobile robot based on an LTL formula over a set of regions in a partitioned environment. The regions of interest appear and disappear based on exponential probability density function’s, and for accounting this we model the robot by a transition system with probabilistic observations. We adapt model checking algorithms for finding a path of the transition system that is most likely to satisfy the formula. The robot is moved according to this path, and based on the regions that are observed
by the robot, a supervising strategy alters the motion by pausing it or by recomputing another path from the current position. The method is supported by two case studies. Future work can be conducted for extending the approach to computationally attractive methods for planning a whole team of robots, and for considering the region related probability density function’s in the robot model, rather than just the steady-state appearance probabilities. REFERENCES Baier, C. and Katoen, J.P. (2008). Principles of Model Checking. MIT Press, Boston. Belta, C., Bicchi, A., Egerstedt, M., Frazzoli, E., Klavins, E., and Pappas, G.J. (2007). Symbolic planning and control of robot motion. IEEE Robotics and Automation Magazine, special issue on grand challenges for robotics, 14(1), 61–71. Belta, C. and Habets, L. (2006). Controlling a class of nonlinear systems on rectangles. IEEE Transactions on
(a)
(b)
Fig. 2. Second example, where the robot is initially deployed in q14 . (a) Trajectory until the robot reaches q1 . (b) While traversing q1 , region π2 appears and the formula is violated. Automatic Control, 51(11), 1749–1759. Berg, M.D., Cheong, O., and van Kreveld, M. (2008). Computational Geometry: Algorithms and Applications. Springer, 3rd edition. Choset, H., Lynch, K.M., Hutchinson, S., Kantor, G., Burgard, W., Kavraki, L.E., and Thrun, S. (2005). Principles of Robot Motion: Theory, Algorithms, and Implementations. MIT Press, Boston. Clarke, E.M.M., Peled, D., and Grumberg, O. (1999). Model checking. MIT Press. Corman, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. (2001). Introduction to Algorithms. The MIT Press and McGraw-Hill Book Company, Cambridge, Massachusetts and New York, 2nd edition. Ding, X.C., Smith, S.L., Belta, C., and Rus, D. (2011). LTL control in uncertain environments with probabilistic satisfaction guarantees. In 18th IFAC World Congress. Milan, Italy. Fainekos, G.E., Girard, A., Kress-Gazit, H., and Pappas, G.J. (2009). Temporal logic motion planning for dynamic robots. Automatica, 45(2), 343–352. Fainekos, G.E., Kress-Gazit, H., and Pappas, G.J. (2005). Hybrid controllers for path planning: A temporal logic approach. In Proceedings of the 44th IEEE Conference on Decision and Control, 4885 – 4890. Gastin, P. and Oddoux, D. (2001). Fast ltl to b¨ uchi automata translation. In H.C. G. Berry and A. Finkel (eds.), Proceedings of the 13th Conference on Computer Aided Verification (CAV’01), number 2102 in LNCS, 53–65. SPRINGER. Habets, L.C.G.J.M., Collins, P.J., and van Schuppen, J.H. (2006). Reachability and control synthesis for piecewiseaffine hybrid systems on simplices. IEEE Transactions on Automatic Control, 51, 938–948. Habets, L.C.G.J.M. and van Schuppen, J.H. (2004). A control problem for affine dynamical systems on a fulldimensional polytope. Automatica, 40, 21–35. Holzmann, G. (2004). The Spin Model Checker, Primer and Reference Manual. Addison-Wesley, Reading, Massachusetts.
Hopcroft, J., Motwani, R., and Ullman, J. (2006). Introduction to Automata Theory, Languages, and Computation (3rd Edition). Addison-Wesley Longman Publishing Co., Boston, MA. Kloetzer, M. and Belta, C. (2008). A fully automated framework for control of linear systems from temporal logic specifications. IEEE Transactions on Automatic Control, 53(1), 287–297. Kloetzer, M., Magdici, S., and Burlacu, A. (2012). Experimental Platform and Matlab Toolbox for Planning Mobile Robots. In 16th International Conference on System Theory, Control and Computing. Sinaia, Romania. Accepted. Kloetzer, M., Mahulea, C., Belta, C., and Silva, M. (2010). An automated framework for formal verification of timed continuous petri nets. IEEE Transactions on Industrial Informatics, 6(3), 460–471. Kress-Gazit, H. (ed.) (2011). IEEE Robotics and Automation Magazine. Special issue on Formal Methods for Robotics and Automation, volume 18(3). Lahijanian, M., Wasniewski, J., Andersson, S.B., and Belta, C. (2010). Motion planning and control from temporal logic specifications with probabilistic satisfaction guarantees. In IEEE International Conference on Robotics and Automation, 3227–3232. LaValle, S.M. (2006). Planning Algorithms. Cambridge. Available at http://planning.cs.uiuc.edu. Loizou, S.G. and Kyriakopoulos, K.J. (2004). Automatic synthesis of multiagent motion tasks based on LTL specifications. In IEEE Conference on Decision and Control, volume 1, 153– 158. Wolper, P., Vardi, M., and Sistla, A. (1983). Reasoning about infinite computation paths. In E.N. et al. (ed.), Proceedings of the 24th IEEE Symposium on Foundations of Computer Science, 185–194. Tucson, AZ. Wongpiromsarn, T., Topcu, U., and Murray, R. (2009). Receding horizon temporal logic planning for dynamical systems. In IEEE Conference on Decision and Control, 5997–6004.
Dept. of Automatic Control and Applied Informatics, Technical University “Gh. Asachi” of Iasi, Romania (e-mail: [email protected]). ∗∗ Arag´ on Institute of Engineering Research (I3A), University of Zaragoza, Spain (e-mail: [email protected])
Abstract: This paper addresses the path planning problem of a mobile robot based on linear temporal logic formulae over a set of regions of interest in the environment. The regions have fixed and known locations but have some dynamic observations that can appear and disappear based on exponential probability density functions. The robot movement capabilities are modeled by a finite-state transition system. By drawing inspiration from temporal logic control for static environments, we find a run that has the greatest chance of satisfying the temporal logic specification. Then, we devise a supervising strategy for the robot motion along this run, which updates the execution based on the currently observed regions. The approach is supported by two simulations. Keywords: Temporal logic, mobile robots, transition systems, probabilistic models, formal languages. 1. INTRODUCTION Recent researches on mobile robots propose different methods for extending the motion tasks to rich and human-like specifications (Kress-Gazit, 2011; Fainekos et al., 2009; Kloetzer and Belta, 2008; Belta et al., 2007). Thus, it is possible for some classes of robots to automatically perform missions as “visit region A or B, and if A is visited go to C, otherwise visit infinitely often B and C while avoiding A” rather than classic problems as “go to A while avoiding obstacles” (Choset et al., 2005; LaValle, 2006). The first challenge in enabling rich tasks is the choice of a language allowing expressive specifications and intuitive translation from human language to syntactically correct formulas. Solutions were found in a variety of formal languages (Clarke et al., 1999; Hopcroft et al., 2006). After choosing a specification class, the second challenge is the development of algorithms that actually control the robot movement such that the desired mission is accomplished. Again, inspiration was drawn from formal analysis, and solutions were developed by appropriately adjusting and extending the available algorithms (Clarke et al., 1999; Hopcroft et al., 2006; Baier and Katoen, 2008). In this paper, we will express robotic missions by linear temporal logic formulas, which is a common choice for enabling human-like specifications (Loizou and Kyriakopoulos, 2004; Kloetzer and Belta, 2008; Fainekos et al., 2009; Wongpiromsarn et al., 2009). Tasks given in this logic usually imply the attainment of some regions from the environment in a specific order, the avoidance of other re! At the Technical University of Iasi, this work has been partially supported by the CNCS-UEFISCDI grant PN-II-RU PD 333/2010. At University of Zaragoza, this work has been partially supported by CICYT - FEDER project DPI2010-20413 and by Fundaci´ on Arag´ on I+D.
gions, and various logical and temporal connectives among regions. We extend the results of the mentioned works by assuming dynamic environments rather than static ones. Specifically, we consider that the regions of interest have fixed locations, but they appear and disappear based on some probabilistic density functions. As customary for this type of problems, the robot movement capabilities are abstracted to a finite-state description, and then the problem is solved by developing supervising methods for the obtained model. The focus of this work will be on providing such a supervising strategy, and therefore we assume a simple robot and we do not insist on the continuous control laws involved when constructing a finite-state model. However, this does not restrict the applicability of our approach to certain robot types or different dynamic systems for which abstraction techniques were already developed (Habets and van Schuppen, 2004; Habets et al., 2006; Belta and Habets, 2006; Kloetzer et al., 2010). In the discrete event driven model we embed probabilistic information on the appearance and disappearance of regions, and then we find a robotic path in form of a run that has the maximum probability of satisfying the given specification. For accomplishing this, we adapt model checking and temporal logic control strategies (Holzmann, 2004; Kloetzer and Belta, 2008). Then, we develop a supervising strategy that reacts to the regions of interest observed during the evolution by updating the system behavior. Related works that address the problem of planning mobile robots based on high level specifications and probabilistic information include (Lahijanian et al., 2010; Ding et al., 2011). In (Lahijanian et al., 2010) the probabilistic information comes from robot sensing and control uncertainties, and not from environment evolution, while the
specifications are restricted to another formalism, namely probabilistic computation tree logic. (Ding et al., 2011) assumes a scenario similar with the one we consider, with additional probabilities on the outcomes of robot control actions, but with specifications involving both robot locations and events that may happen in those locations. Also, the solution from (Ding et al., 2011) is different, involving robot models as Markov decision processes and more complex deterministic Rabin automata, while the control strategy is not adjusted based on current observations from the environment. The remainder of the paper is structured as follows. Sec. 2 introduces some necessary preliminaries and states the problem we solve. The solution is given in Sec. 3, two simulations are presented in Sec. 4, and Sec. 5 formulates some concluding remarks. 2. PRELIMINARIES AND PROBLEM FORMULATION Subsection 2.1 briefly introduces the formalism that will be used for specifying robotic motion tasks, and defines an automaton needed for a formal description of the solution. Subsection 2.2 gives the generic structure of a finite transition system, that will be adapted for obtaining a suitable model for a mobile robot, and subsection 2.3 formulates the problem we solve. 2.1 Linear Temporal Logic In this work, we consider motion tasks given as formulae of LT L−X , which is a fragment of Linear Temporal Logic (LTL) Clarke et al. (1999). Informally, an LT L−X formula is recursively defined over a set of atomic propositions Π, by using the standard Boolean operators (¬ - negation, ∨ - disjunction, ∧ - conjunction, ⇒ - implication, and ⇔ equivalence) and some temporal operators with straightforward meaning (! - always, ♦ - eventually, and U until). By interconnecting these operators one can obtain rich and human-like specifications, which we will use as motion specifications for a mobile robot. For exemplification, consider that Π contains the labels of some regions of interest from an environment. A standard navigation task Choset et al. (2005); LaValle (2006) as “ eventually reach π1 , while always avoiding π2 and π3 ” can be expressed by the LT L−X formula ♦π1 ∧ !¬(π2 ∨ π3 ). A surveillance mission - “visit π1 and then π2 infinitely often” - is described by formula !♦(π1 ∧ ♦π2 ), a choice reachability mission - “eventually reach either region π1 or π2 ” corresponds to formula ♦(π1 ∨ π2 ). A formula as ♦!(π1 ∧ π2 ) describes a convergence mission (“eventually reach π1 and π2 and stay there for all future times”) that can be satisfied by a single robot if regions π1 and π2 overlap (otherwise, at least two cooperating robots would be necessary). With respect to standard LTL, LT L−X lacks the “next” operator, which is meaningless for continuous trajectories (as are those generated by a moving mobile robot). However, the intended meaning of the “next” operator can be replaced by an adequate usage of the “until” operator, as in the following example. A task requiring that “region
π1 is visited, and then π2 is directly entered from π1 ” is expressed by the LT L−X formula ♦(π1 Uπ2 ). Therefore, we will simply denote by LTL the fragment that we use. A formal definition of the syntax and semantics of LTL formulas can be found in Clarke et al. (1999). LTL formulas are interpreted over infinite strings with elements from 2Π (the set of all subsets of Π, including the empty set ∅). Any LTL formula over set Π can be transformed into a B¨ uchi automaton (see Def. 1) that accepts all and only the input strings satisfying the formula Wolper et al. (1983). Available software tools allow such conversions Holzmann (2004); Gastin and Oddoux (2001). Definition 1. The B¨ uchi automaton corresponding to an LTL formula over the set Π has the structure B = (S, S0 , ΣB , →B , F ), where: • • • • •
S is a finite set of states; S0 ⊆ S is the set of initial states; ΣB = 2Π is the set of inputs; →B ⊆ S × ΣB × S is the transition relation; F ⊆ S is the set of final states.
The transitions in B can be non-deterministic, meaning that from a given state there may be multiple outgoing transitions enabled by the same input, i.e. we may have (s, σ, s# ) ∈→B and (s, σ, s## ) ∈→B , with s# += s## . Thus, an input sequence can produce more than one sequence of states (called run). An infinite input word (sequence with elements fromΣ B ) is accepted by B if the word produces at least one run of B that visits set F infinitely often. 2.2 Transition systems For a partitioned environment cluttered with possibly overlapping and static regions of interest from set Π, the movement capabilities of a mobile robot can be abstracted into a finite state transition system as in Def. 2. Environment partitions can be obtained with cell decomposition algorithms Berg et al. (2008), while details on abstractions suitable for different robot dynamics and cell shapes can be found in Habets et al. (2006); Belta and Habets (2006). The idea is that every cell from the partition corresponds to a single state in the finite state description, and transitions between states correspond to movement capabilities. A satisfaction map shows the regions of interest that are satisfied when the robot is inside a particular cell (by assuming that the regions and their intersections are equal with a union of partition cells). Definition 2. A finite" state transition system is a tuple ! T = Q, q0 , δT , ΠT , |= , where: • Q is the finite number of states (cells from partition); • q0 is the initial state (cell containing the initial deployment of robot); • δT ⊆ Q × Q is the transition relation, where (q, q # ) ∈ δT , with q += q # , if and only if the robot can be steered from any initial condition in q to the adjacent cell q # in finite time, and (q, q) ∈ δT if the robot can be kept inside cell q for an indefinite time; • ΠT = 2Π is the set of possible observations (outputs of states of T );
• |=: Q → ΠT is a satisfaction map, where |= (q) is the set of all regions from Π that contain cell labeled by q, and |= (q) = ∅ if cell q belongs to the left-over space (q is not included in any region of interest). Transition systems that we use are deterministic, fact which implies that any feasible transition in the current state can be chosen by imposing a specific robot control law. A run (or path) of T is an infinite sequence r = q0 q1 q2 ..., with the property that (qi , qi+1 ) ∈ δT , ∀i ≥ 0. A run corresponds to a robot movement through cells from partition, and it induces (through map |=) an output word, which is the observed sequence of elements fromΠ T . SinceΠ T = ΣB , an output word of T is an input word for B, and thus one can make the connection between robot trajectories and satisfaction of LTL specifications. Researches as Fainekos et al. (2005); Kloetzer and Belta (2008) proposed algorithms that abstract a robot motion into a transition system T and use model-checking inspired algorithms for controlling T such that it satisfies a given LTL formula. Unlike such works, we assume that regions of interest appear and disappear based on probability density functions. For accounting scenarios where regions of interest appear and disappear, in this work we will modify the finite state representation T and the path finding strategy from Kloetzer and Belta (2008), and then we will develop an automated supervising strategy for the mobile robot.
As in subsection 2.2, we assume that the environment is partitioned (e.g. with respect to regions from Π and their intersections) in convex and polyhedral cells, and the set of obtained cells is denoted by Q. This partition enables us to model the motion of the robot by a finite state representation (transition system from Def. 2). Moreover, since the robot is fully-actuated, the transition relation δT is reflexive and symmetric. For actually moving the robot between adjacent cells we will use the standard strategy from (Choset et al., 2005): the current robot position is linked by a line segment to the middle point of the common facet of the current and the next cell. Thus, a run of T corresponds to a robot trajectory formed by connected line segments, and it can be easily followed by a fully-actuated and point robot. For more complicated dynamics other control laws can be used, as in Habets and van Schuppen (2004); Belta and Habets (2006), but such details go beyond the scope of this paper. Problem: Given a task as an LTL formula over the set Π, find a robot control strategy that yields the maximum probability of satisfying the task. To solve the problem, in the next section (1) we extend the structure of the transition system from Def. 2 by adding probability information, (2) we compute a product automaton and find a run of T , and finally (3) we design a supervising method that adjusts the strategy for following the run based on the currently observed propositions from Π.
2.3 Problem formulation
3. SOLUTION
Assume an environment where a mobile robot evolves and some regions of interest are defined. Each region of interest has a fixed and known position, but it can appear or disappear during robotic motion. The robot is assumed to be of negligible size and fully-actuated, and the environment boundaries and the regions are polygonal. Let Π= {π1 , . . . , πn } be the set containing the labels of regions of interest, and let ∅ be a label corresponding to the free space (space not covered by a region). Initially, we assume that each region of interest πi is visible, but will disappear after a random delay with negative exponential probability density function. The disappearance of a region does not mean that the region is not accessible by the robot, but it means that its observation is not visible. The parameter of the probability density function associated with the disappearance of πi is the rate λdi . Therefore, the average time delay of passing from visible to invisible is 1/λdi . When invisible, πi will appear (become visible) after a random delay with negative exponential probability density function, in this case with rate λai . Since the exponential probability density function has the Markov property, the appearance / disappearance of a region is a Markov process, and therefore it is possible to compute the steady-state probability of each region πi to exist. The appearance/disappearance of different regions are independent (not correlated) and the probability of region πi to exist is computed immediately as p πi =
λai λai + λdi
(1)
As briefly mentioned, our solution follows three main steps. In subsection 3.1, the steady state probabilities from (1) are embedded in the transition system modeling the robot, by accounting possible overlapping of some regions. In subsection 3.2 we construct the product between the probabilistic transition system and the B¨ uchi automaton corresponding to the LTL formula, and we find a path with the maximum probability of producing observations that satisfy the formula. The supervising strategy from subsection 3.3 keeps track of formula satisfaction and updates if necessary the evolution of the transition system when following the imposed path. 3.1 Robot model ! " Let T = Q, q0 , δT , ΠT , |= be the transition system for an environment with static regions of interest. In our case, map |= - created when all regions exist (initial moment) - becomes inaccurate, because observations change when regions disappear. Therefore, we extend T by adding a probabilistic observation function ρ : Q × ΠT → [0, 1], where ρ(q, p) is the probability of observing (satisfying) all and only regions from p ∈ ΠT when the current state is q. We note that the empty observation ∅ in a state q of T means that (at a given time instant) there is no region appeared that includes cell q. For example, if the cell corresponding to state q is included in the intersection of two appeared regions π1 and π2 , than the observation at that instant would be p = {π1 , π2 }, and if both π1 and
π2 disappear at another instant, the observation would be p = ∅. The probabilistic observation function ρ can be computed based on the probabilities of regions from Π to be active (pπi computed in (1)). We find ρ(q, p), ∀p ∈ 2|=(q) as follows: # (1 − pπ ) (2) ρ(q, ∅) = π∈|=(q)
|=(q)
, p += ∅, % $ # pπ · ρ(q, p) =
∀p ∈ 2
π∈p
#
π∈|=(q)\p
(1 − pπ )
(3)
Recall that ρ(q, p) is the probability of observing all regions from set p, and not other additional regions from set |= (q). Result (2) comes from the fact that the appearance / disappearance of regions are not correlated, and 1 − pπ is the probability of not observing region π at a time instant. The first product from (3) means that all regions from set p are appeared, and the second product implies that no other additional region exists. 3.2 Path finding This subsection presents an automatic procedure for finding a path in T such that the produced word has the greatest chance of satisfying the LTL formula. This chance comes from the fact that observations of T are probabilistic. We begin by constructing a special type of product automaton between T and B. Let us denote by P(s→B s! ) ⊆ 2Π the set of all inputs of B that enable a transition from s to s# , i.e. P(s→B s! ) = {p ∈ 2Π | (s, p, s# ) ∈→B }. Definition 3. The product automaton A = T × B is constructed as follows: A = (SA , SA0 , δA , ωA , FA ), where: • SA = Q × S is the set of states; • SA0 = {q0 } × S0 is the set of initial states; • δA ⊆ SA × SA is the transition relation, defined by: ((q, s), (q # , s# )) ∈ δA if and only if (q, q # ) ∈ δT and ∃p ∈ ΠT such that ρ(q, p) > 0 and (s, p, s# ) ∈→B ; • ωA : δA → [', ∞), with ' > 0 being a small constant, is a weighting (cost) function for tran# # sitions , ωA ((q, s), (q , s )) = ' − *+ of A, defined by: # # log p∈P(s→B s! ) ρ(q, p) , ∀ ((q, s), (q , s )) ∈ δA ; • FA = Q × SA is the set of final states. Product automaton A is inspired by algorithms in (Kloetzer and Belta, 2008), but its construction is adapted such that it includes the available probabilistic information in relation δA and map ωA . A transition in A is a match between a transition of B caused by the current observation of T and a transition of T . Since observations of T are probabilistic, transitions of A have a probabilistic nature, in the sense that the same transition in A is feasible if T has a certain observation and it may be unfeasible for other observations. The probability of existence for a transition of A is captured by its cost assigned by map ωA . The sum of observation probabilities from ωA encapsulates all observations of T that can produce the same transition in B. We considered
the costs as logarithms of probabilities because in this way, when finding a minimum-cost path by standard search algorithms (e.g. Dijkstra (Corman et al., 2001)), we basically find the path whose product of probabilities is maximum. The ' constant guarantees that if more paths with the same optimum cost exist, the one with the smallest number of different states will be chosen (' = 0 would induce paths with unnecessary zero-weighted transitions). The acceptance condition of A is formulated similar to the one of B, in the sense that an infinite run is accepted by A if and only if it visits infinitely often the set of final states FA . If A has at least one accepted run, than it has an accepted run in a prefix-suffix form, consisting of a finite sequence of states (called prefix) followed by infinite repetitions of another finite sequence (called suffix). By using graph searches, we find in A an accepted path that has the sum of costs of prefix and suffix minimum - as explained above, this guarantees the maximum probability of having correct observations in T (observations that keep feasible the followed transitions of A). Of course, one can differently weight prefix / suffix costs, for example by considering that the suffix is infinitely repeated. The found path of A will be projected to a path of T , and the probability of the path of A is basically a measure that the generated observation string of T satisfies the LTL formula. However, for ensuring this we have to develop a supervising strategy, which adapts the robot motion plan (transitions to be taken in T ) each time the current observation of T is not consistent with the current transition of A. 3.3 Supervisory robot control Let us denote by runA = (q0 , s0 ) (q1 , s1 ) (q2 , s2 ) . . . (qs , ss ) . . . (qp , sp ) . . . the optimum path of A, where the suffix is marked by bold font. The corresponding run of T is runT = q0 q1 . . . qs . . . qp . . ., and the one of B is runB = s0 s1 . . . ss . . . sp . . .. From construction of automaton A, we have the guarantee that runB is an accepted run of B, and thus runT can produce a sequence of observations (inputs to B) that satisfies the LTL formula. As noted in subsection 3.2, optimization on runA guarantees the maximization of the probability that T produces an output word satisfying the LTL formula. However, when T follows its run (the mobile robot moves through partition cells corresponding to states in runT ), we have to check the current observation and decide if that produces the desired transition in runB . Let us denote by h ∈ ΠT the current observation of T (the set of appeared regions of interest that contain the mobile robot position at the current time instant). During robot movement, B plays the role of a test tool for the satisfaction of LTL formula. For accomplishing this, we store and update a set C B ⊆ S containing the possible current (or consistent) states from B that can be reached due to the inputs applied so far to B (these inputs are observations from T ). Initially, C B = {S0 }, and it is updated as in (4) whenever the robot enters a new region from runT or when the current observation h changes.
C B := {s ∈ S | ∃s# ∈ C B s.t. (s# , h, s) ∈→B }
(4)
During the motion, whenever the robot enters in a state (partition cell) qi from runT , and at any change in h, we update C B , check which of the following cases is true and perform the corresponding action. Of course, the following supervising strategy begins with i = 0 (the robot deployed in q0 ): (i) if si+1 ∈ C B , then T advances to the next state qi+1 ; (ii) if si+1 ∈ / C B , but si ∈ C B , then T stays (self-loops) in qi ; (iii) if si , si+1 ∈ / C B , but C B += ∅, then we have to modify the run of T to be followed from now. Consider q0 = qi in T , S0 = C B in B, and update the set of initial states of A. Search for a new optimum runA , find runT and runB , and restart the supervising strategy according to these paths; (iv) if C B = ∅, then stop the movement and report that the LTL formula was just violated by the current observation h. Informally, case (i) means that until now the desired run of B is produced by the actual observation of T . Case (ii) means that the desired run of B does not advance, but the current observation h enables a self-loop in its actual state. The robot waits for a change in observation of T - such a change can appear, because runB can be produced by some probabilistic observations along runT . In case (iii), the current observation h prevented runB of being followed, but it did not block the entire evolution of B. Therefore, we search for a different continuation of the evolution that satisfies the formula. Case (iv) can appear only when any feasible robot trajectory can produce (besides desired observations) some observations that are forbidden by the LTL formula - such an example will be included in Sec. 4. The probability-based optimization from subsection 3.2 implies that along the evolution case (i) is most likely to appear, case (iii) rarely appears, and case (iv) can appear only when there’s no way of satisfying the formula while avoiding its possible violation. The waiting times from case (ii) are also minimized by large probabilities of observing the desired output along runT . 4. SIMULATIONS Let us consider the planar environment from Fig. 1(a), composed by 22 triangular cells q1 , . . . , q22 and three regions of interest: Π= {π1 , π2 , π3 }. The satisfaction map |= for the initial moment is defined as follows: • • • • •
|= (q10 ) = |= (q12 ) = π1 ; |= (q9 ) = {π1 , π2 }; |= (q1 ) = |= (q4 ) = |= (q5 ) = |= (q8 ) = π2 ; |= (q16 ) = {π2 , π3 }; |= (q18 ) = |= (q20 ) = |= (q22 ) = π2 .
Let us assume that π1 has a disappearance rate of λd1 = 0.3, i.e., the average time delay after which the region disappears is 1/0.3, and an appearance rate of λa1 = 0.7. According to (1), the steady-state probability of observing 0.7 π1 is pπ1 = 0.7+0.3 = 0.7. For π2 we assume λa2 = λd2 = 0.5,
0.5 implying pπ2 = 0.5+0.5 = 0.5, while for π3 we assume 0.2 a d λ3 = 0.2 and λ3 = 0.8, obtaining pπ3 = 0.2+0.8 = 0.2.
Based on these probabilities, the probabilistic observation function ρ is immediately computed by applying (2) and (3). For example, for q9 , where two observations can appear (|= (q9 ) = {π1 , π2 }), we have: • • • •
ρ(q9 , ∅) = (1 − pπ1 ) · (1 − pπ2 ) = 0.3 · 0.5 = 0.15; ρ(q9 , {π1 }) = pπ1 · (1 − pπ2 ) = 0.7 · 0.5 = 0.35; ρ(q9 , {π2 }) = (1 − pπ1 ) · pπ2 = 0.3 · 0.5 = 0.15; ρ(q9 , {π1 , π2 }) = pπ1 · pπ2 = 0.7 · 0.5 = 0.35.
Let us assume that the robot is initially placed in q21 and its task is given by the following LTL formula: ♦ ((π1 ∧ π2 ) ∧ ♦π3 ) Consequently, the robot should first reach a region where both π1 and π2 are simultaneously satisfied (in our case the only possibility is to reach q9 ) and then it should reach a region with the observation π3 . Applying the results in subsection 3.2 the following path is obtained: runT = q21 q19 q17 q10 q12 q9 q8 q5 q16 q16 . . . (the suffix is q16 ). Fig. 1 shows an evolution to follow this trajectory by applying the supervisory control presented in subsection 3.3 by using a MATLAB implementation. The robot should first reach q9 (hoping that both π1 and π2 will be active). During the movement to q9 the robot passes through q21 , q19 , q17 , q10 , q12 and some regions disappear (π1 and π3 ), but π3 appears after some time. The positions of the robot when the regions changed are represented by bullets. However, since these changes of observations are not affecting the mission, the robot will move by following the computed path (case (i) in the supervising strategy). Once the robot arrives in q9 , π2 is not appeared and the robot waits (case (ii) in the supervising strategy). After some time, both π1 and π2 are observed, and the robot leaves q9 and moves towards q16 . Notice that when the robot arrives to q16 (Fig.1.(c)) the region π3 is not appeared and the robot waits. When π3 becomes visible the robot moves to the centroid of q16 and remains there. Let us assume the same environment and observation probabilities, but an initial robot position in q14 and a mission given by the following LTL formula: !¬ (π1 ∨ π2 ) ∧ ♦π3 Hence, the robot should reach a region with observation π3 , while always avoiding observations π1 and π2 . In order to accomplish the mission the robot should pass through at least one region with |= (q) = π1 or |= (q) = π2 , hoping that during the motion in a such region the corresponding observation will not be active. The path computed by method from subsection 3.2 is: runT = q14 q6 q1 q3 q2 q7 q20 q20 . . .. Notice that the robot should pass through q1 , where π2 may appear and violate the formula. A possible trajectory is presented in Fig. 2. For this situation, when the robot enters in q1 the region π2 is not observed (Fig. 2.(a)), but after a small amount of time π2 becomes active and the formula is violated. In this case, the robot stops (case (iv) in the supervising strategy). Of course, for other simulations it is possible that π2 does not appear while the robot crosses q1 , and the formula is
(a)
(b)
(c)
(d)
Fig. 1. Environment and continuous trajectories of the robot for the first example from Sec. 4. The bullets mark the positions of the robot when a region appears or disappears. (a) Planar environment with all regions appeared. (b) Robot waits in q9 the appearance of π1 . (c) Robot waits in q16 the appearance of π3 . (d) Mission is accomplished and the robot stops in the middle of q16 . satisfied. Observe that the strategy from subsection 3.2 did not produce a path that traverses cell q10 (instead of q1 ), because such a trajectory would have smaller chances of satisfying the formula (pπ1 > pπ2 ). We mention that future work will focus on performing examples with real robots on the experimental platform reported in (Kloetzer et al., 2012). 5. CONCLUSIONS This paper proposes an approach for supervising the motion of a mobile robot based on an LTL formula over a set of regions in a partitioned environment. The regions of interest appear and disappear based on exponential probability density function’s, and for accounting this we model the robot by a transition system with probabilistic observations. We adapt model checking algorithms for finding a path of the transition system that is most likely to satisfy the formula. The robot is moved according to this path, and based on the regions that are observed
by the robot, a supervising strategy alters the motion by pausing it or by recomputing another path from the current position. The method is supported by two case studies. Future work can be conducted for extending the approach to computationally attractive methods for planning a whole team of robots, and for considering the region related probability density function’s in the robot model, rather than just the steady-state appearance probabilities. REFERENCES Baier, C. and Katoen, J.P. (2008). Principles of Model Checking. MIT Press, Boston. Belta, C., Bicchi, A., Egerstedt, M., Frazzoli, E., Klavins, E., and Pappas, G.J. (2007). Symbolic planning and control of robot motion. IEEE Robotics and Automation Magazine, special issue on grand challenges for robotics, 14(1), 61–71. Belta, C. and Habets, L. (2006). Controlling a class of nonlinear systems on rectangles. IEEE Transactions on
(a)
(b)
Fig. 2. Second example, where the robot is initially deployed in q14 . (a) Trajectory until the robot reaches q1 . (b) While traversing q1 , region π2 appears and the formula is violated. Automatic Control, 51(11), 1749–1759. Berg, M.D., Cheong, O., and van Kreveld, M. (2008). Computational Geometry: Algorithms and Applications. Springer, 3rd edition. Choset, H., Lynch, K.M., Hutchinson, S., Kantor, G., Burgard, W., Kavraki, L.E., and Thrun, S. (2005). Principles of Robot Motion: Theory, Algorithms, and Implementations. MIT Press, Boston. Clarke, E.M.M., Peled, D., and Grumberg, O. (1999). Model checking. MIT Press. Corman, T.H., Leiserson, C.E., Rivest, R.L., and Stein, C. (2001). Introduction to Algorithms. The MIT Press and McGraw-Hill Book Company, Cambridge, Massachusetts and New York, 2nd edition. Ding, X.C., Smith, S.L., Belta, C., and Rus, D. (2011). LTL control in uncertain environments with probabilistic satisfaction guarantees. In 18th IFAC World Congress. Milan, Italy. Fainekos, G.E., Girard, A., Kress-Gazit, H., and Pappas, G.J. (2009). Temporal logic motion planning for dynamic robots. Automatica, 45(2), 343–352. Fainekos, G.E., Kress-Gazit, H., and Pappas, G.J. (2005). Hybrid controllers for path planning: A temporal logic approach. In Proceedings of the 44th IEEE Conference on Decision and Control, 4885 – 4890. Gastin, P. and Oddoux, D. (2001). Fast ltl to b¨ uchi automata translation. In H.C. G. Berry and A. Finkel (eds.), Proceedings of the 13th Conference on Computer Aided Verification (CAV’01), number 2102 in LNCS, 53–65. SPRINGER. Habets, L.C.G.J.M., Collins, P.J., and van Schuppen, J.H. (2006). Reachability and control synthesis for piecewiseaffine hybrid systems on simplices. IEEE Transactions on Automatic Control, 51, 938–948. Habets, L.C.G.J.M. and van Schuppen, J.H. (2004). A control problem for affine dynamical systems on a fulldimensional polytope. Automatica, 40, 21–35. Holzmann, G. (2004). The Spin Model Checker, Primer and Reference Manual. Addison-Wesley, Reading, Massachusetts.
Hopcroft, J., Motwani, R., and Ullman, J. (2006). Introduction to Automata Theory, Languages, and Computation (3rd Edition). Addison-Wesley Longman Publishing Co., Boston, MA. Kloetzer, M. and Belta, C. (2008). A fully automated framework for control of linear systems from temporal logic specifications. IEEE Transactions on Automatic Control, 53(1), 287–297. Kloetzer, M., Magdici, S., and Burlacu, A. (2012). Experimental Platform and Matlab Toolbox for Planning Mobile Robots. In 16th International Conference on System Theory, Control and Computing. Sinaia, Romania. Accepted. Kloetzer, M., Mahulea, C., Belta, C., and Silva, M. (2010). An automated framework for formal verification of timed continuous petri nets. IEEE Transactions on Industrial Informatics, 6(3), 460–471. Kress-Gazit, H. (ed.) (2011). IEEE Robotics and Automation Magazine. Special issue on Formal Methods for Robotics and Automation, volume 18(3). Lahijanian, M., Wasniewski, J., Andersson, S.B., and Belta, C. (2010). Motion planning and control from temporal logic specifications with probabilistic satisfaction guarantees. In IEEE International Conference on Robotics and Automation, 3227–3232. LaValle, S.M. (2006). Planning Algorithms. Cambridge. Available at http://planning.cs.uiuc.edu. Loizou, S.G. and Kyriakopoulos, K.J. (2004). Automatic synthesis of multiagent motion tasks based on LTL specifications. In IEEE Conference on Decision and Control, volume 1, 153– 158. Wolper, P., Vardi, M., and Sistla, A. (1983). Reasoning about infinite computation paths. In E.N. et al. (ed.), Proceedings of the 24th IEEE Symposium on Foundations of Computer Science, 185–194. Tucson, AZ. Wongpiromsarn, T., Topcu, U., and Murray, R. (2009). Receding horizon temporal logic planning for dynamical systems. In IEEE Conference on Decision and Control, 5997–6004.