- Email: [email protected]
Answer Set Programming for Reasoning with Semantic Knowledge in Collaborative Housekeeping Robotics
10th IFAC Symposium on Robot Control International Federation of Automatic Control September 5-7, 2012. Dubrovnik, Croatia
Answer Set Programming for Reasoning with Semantic Knowledge in Collaborative Housekeeping Robotics Erdi Aker ∗ Volkan Patoglu ∗∗ Esra Erdem ∗ ∗
Computer Science and Engineering, Faculty of Engineering and Natural ˙ Sciences, Sabancı University, Istanbul, TURKEY ∗∗ Mechatronics, Faculty of Engineering and Natural Sciences, ˙ Sabancı University, Istanbul, TURKEY
Abstract: Answer Set Programming (ASP) is a knowledge representation and reasoning paradigm with high-level expressive logic-based formalism, and efficient solvers; it is applied to solve hard problems in various domains, such as, systems biology, wire routing, space shuttle control. In this paper, we present an application of ASP to housekeeping robotics, by showing how the following problems are addressed using computational methods/tools of ASP: 1) embedding commonsense knowledge automatically extracted from the commonsense knowledge base ConceptNet, into high-level representation, 2) embedding (continuous) geometric reasoning and temporal reasoning about durations of actions, into (discrete) high-level reasoning. We illustrate the applicability of ASP on several housekeeping robotics problems, and report on the computational efficiency in terms of CPU time and memory. Keywords: Knowledge representation, reasoning, housekeeping robotics, answer set programming, commonsense knowledge 1. INTRODUCTION
Second, as in our earlier work (Erdem et al. (2011b)), we embed continuous geometric reasoning (e.g., to avoid robot-robot, robot-stationary object and robot-moveable object collisions) also in the high-level discrete representation of robots’ actions in ASP. In this way, not only the ASP solver guides the motion planner by providing a discrete task plan that satisfies some given complex temporal goals (e.g., some actions like carrying heavy objects should be postponed to later steps in a plan) or by conveying information as to how some actions should be executed (e.g., a fragile object should be carried slowly), but also the motion planner guides the ASP solver by providing information about the configurations that would lead to collisions so that the ASP solver can find feasible plans. Also, for multiple autonomous robots to complete cleaning a house by a given time, durations of actions should also be taken into account. However, this brings about further challenges. Since the robots can help each other, when a help request is received, the robot should be able to autonomously decide whether he has enough time to complete his tasks and help the other robot by the given deadline. Since changing locations to be able to help each other takes some time, transportation delay (depending on the length of the continuous trajectory) should also be taken into account. To address these challenges we also embed temporal reasoning about durations of actions into high-level representation and reasoning.
Answer Set Programming (ASP) (Lifschitz (2008); Brewka et al. (2011)) is a knowledge representation and reasoning paradigm with high-level expressive logic-based formalism, and efficient solvers (e.g., iClingo (Gebser et al. (2008))). Due to these advantages, it has been applied in many domains, such as systems biology (Tari et al. (2010); Gebser et al. (2010)), biomedical informatics (Erdem et al. (2011a)), wire routing (Erdem and Wong (2004)), space shuttle control (Nogueira et al. (2001)), etc. The idea is to formally represent the computational problems (e.g., actions of robots and planning problems) in ASP, and use the state-of-the-art ASP solvers to compute models of the formulation that characterize the solutions (e.g., discrete plans) of the given problem. In this paper, we illustrate how computational methods/tools of ASP can be used effectively for the housekeeping robotics problem (i.e., multiple autonomous robots cleaning a house collaboratively). Advantages of using ASP for representing and reasoning about housekeeping robotics domains can be summarized along with the main contributions of this paper as follows. First, one of the challenges of housekeeping robotics domain is the necessity of commonsense knowledge for the robots to behave intelligently while tidying in a house. For instance, in a tidy house, books are in the bookcase, dirty dishes are in the dishwasher, pillows are in the closet; some objects, like glasses, are fragile, whereas some others, like pillows are not. Using high-level formalism, the computational methods and tools of ASP, we can automatically extract such commonsense knowledge from the commonsense knowledge base ConceptNet (Liu and Singh (2004)), and embed it in the high-level representation of robots’ actions in ASP. 978-3-902823-11-3/12/$20.00 © 2012 IFAC
We have earlier studied (Aker et al. (2011b,a); Erdem and Patoglu (2012)) housekeeping robotics problem using a different knowledge representation and reasoning framework: we have represented the robots’ actions and planning problems in the high-level action description language C+ (Giunchiglia et al. (2004)) and used the causal reasoner CC ALC (McCain and Turner (1997)) to solve task planning problems. Apart from the difference in representation and reasoning frameworks, the 77
10.3182/20120905-3-HR-2030.00169
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
presented work is different from our earlier studies. First, the problem of automatically extracting and embedding commonsense knowledge in high-level representation of housekeeping domain is not addressed in our earlier studies (Aker et al. (2011b,a)). Also, we consider a more involved housekeeping domain where durations of actions are taken into account; in our earlier studies, actions are assumed to be instantaneous. It is important to emphasize here that, unlike action description languages, like C+, ASP allows us to represent non-Markovian domains where the effects of actions cannot simply be described in terms of the current state and the next state, but in terms of several consecutive states. Also, the ASP solver iClingo computes an answer to a planning/prediction problem incrementally, and thus uses less memory compared to the causal reasoner CC ALC. We observe these advantages of ASP over our earlier approach using causality-based formalisms and solvers, also in the experiments that we performed over some housekeeping problems instances. 2. ANSWER SET PROGRAMMING The idea of ASP is to represent knowledge (e.g., actions of multiple robots) as a “program” and to reason about the knowledge (e.g., find a plan of robots’ actions) by computing models, called “answer sets” (Gelfond and Lifschitz (1991)), of the program using “ASP solvers” like iClingo.
Fig. 1. A sample house keeping domain A group of rules that follow a pattern can be often described in a compact way using “schematic variables”. For instance, we can write the program pi ← not pi+1 (1 ≤ i ≤ 7) as follows index(1). index(2). ... index(7). p(i) ← not p(i + 1), index(i). The auxiliary predicate index(i) is introduced to describe the ranges of variables. ASP solvers compute an answer set for a given program that contains variables, after “grounding” the program. The “definitions” of such auxiliary predicates tell the ASP solver how to substitute specific values for variables in schematic expressions. Variables can be also used “locally” to describe the list of formulas. For instance, the rule 1{p1 , . . . , p7 }1 can be represented as follows: 1{p(i) : index(i)}1.
An (ASP) program is a finite set of rules of the form L0 ← L1 , . . . , Lk , not Lk+1 , . . . , not Lm where m ≥ k ≥ 0, L0 is a literal (a propositional atom p or its negation ¬p) or ⊥, and each Li (i > 0) is a literal. In such a rule, L0 is called the head and L1 , . . . , Lk , not Lk+1 , . . . , not Lm is called the body of the rule. If the head of a rule is ⊥, then the rule is called a constraint; and ⊥ is simply dropped from the head. Note the two sorts of negation: classical negation ¬ as in classical logic, and default negation not. Intuitively, ¬p means that “it is known that p is not true” whereas not p means that “it is not known that p is true”. Default negation is useful in expressing default values of atoms. For instance, closed ← not ¬closed expresses that a door is closed by default (i.e., unless it is not known that it is open). Intuitively, an “answer set” for an ASP program is a “minimal” (consistent) set of literals that satisfies the program. If an atom p is included in an answer set, then p is known to be “true”. If ¬p is included in an answer set, then p is known to be “false”. Otherwise, neither p nor ¬p is in an answer set, p is unknown. Due to its semantics, ASP is different from Prolog. For a precise definition of an answer set we refer the reader to (Gelfond and Lifschitz (1991)).
3. REPRESENTING HOUSEKEEPING DOMAIN IN ASP Consider a house consisting of three rooms: a bedroom, a living room and a kitchen as shown in Fig. 1. There are three cleaning robots in the house. The furniture is stationary and their locations are known to the robots a priori. Other objects are movable. There are three types of movable objects: books (green pentagon shaped objects), pillows (red triangular objects) and dishes (blue circular objects). The locations of the movable objects are not known to the robots a priori. The goal is for the cleaning robots to tidy the house collaboratively in a given amount of time.
When we represent a problem in ASP, we use special constructs of the form l{L1 , . . . , Lk }u (called cardinality expressions) where each Li is a literal and l and u are nonnegative integers denoting the “lower bound” and the “upper bound” (Simons et al. (2002)). Programs using these constructs can be viewed as abbreviations for normal programs (Ferrein et al. (2005)). Such an expression describes the subsets of the set {L1 , . . . , Lk } whose cardinalities are at least l and at most u. Such expressions when used in heads of rules generate many answer sets whose cardinality is at least l and at most u, and when used in constraints eliminate some answer sets.
We represent the housekeeping domain in ASP and then solve planning problems using the ASP solver iClingo. In the following, we describe how housekeeping domain is represented in ASP. Fluents and actions To represent the robots’ actions and the change in housekeeping domain, we think of the world as being 78
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
in one of many states. To be able to describe the states of the world, we need to introduce fluents (atoms whose truth values depend on time).
According to the formulation above, actions can be executed concurrently. To ensure that some actions cannot be executed concurrently, we make use of constraints as well. For instance, the following rules express that a robot cannot attach to an object while moving to a location: ← goto(r, x, y), attach(r).
We view the house as a grid; so the robots and the endpoints of objects (discrete set of points on the object through which the object can be grasped) are located at grid-points. Here are some of the fluents introduced for housekeeping domain: at(th, x, y, t) (“thing th is at (x, y) at time step t”), connected(r, ep, t) (“robot r is connected to endpoint ep of an object at time step t”).
4. EMBEDDING COMMONSENSE KNOWLEDGE IN PLANNING In the housekeeping domain, the robots need to know that books are expected to be in the bookcase, dirty dishes in the dishwasher, and pillows in the closet. Moreover, a bookcase is expected to be in the living-room, dishwasher in the kitchen, and the closet in the bedroom. In addition, the robots should have an understanding of a tidy house to be able to clean a house autonomously: tidying a house means that the objects are at their desired locations. Also, while cleaning a house, robots should pay more attention while carrying fragile objects; for that they should have an understanding of what a fragile object is. Such commonsense knowledge is formally represented already in commonsense knowledge bases, such as ConceptNet.
We introduce the following elementary actions to represent robots’ actions: goto(r, x, y, t) (“robot r goes to (x, y) at time step t”), detach(r, t) (“robot r detaches from the object it is connected to, at time step t”), and attach(r, t) with an attribute attach point(r, ep, t)(“robot r attaches to an object at its endpoint ep at time step t”). Direct effects of actions We describe the direct effects of the actions above by ASP rules. For instance, the following rule expresses that the direct effect of the action of a robot r going to a location (x, y): at(r, x, y, t + 1) ← goto(r, x, y, t) Similarly, we can describe the direct effects of the action of a robot r detaching from the endpoints of an object it is connected to: ¬connected(r, ep, t + 1) ← detach(r, t), connected(r, ep, t) and the direct effect of the action of a robot r attaching to an endpoint of an object: connected(r, ep, t + 1) ← attach(r, t), attach point(r, ep, t)
ASP allows us to extract and embed commonsense knowledge from these knowledge bases by means of “external predicates.” External predicates are not part of the signature of the domain description (i.e., they are not declared as fluents or actions). They are implemented as functions in some programming language of the users choice, such as C++ or Prolog. External predicates take as input not only some parameters from the domain description (e.g., the locations of robots) but also detailed information that is not a part of the action domain description (e.g., commonsense knowledge). They are used to externally check some conditions.
Preconditions of actions We describe preconditions of actions using constraints. For instance, at any time step t, we can describe that a robot r cannot go to a location (x, y) if the robot is already at that location, as the following: ← goto(r, x, y, t), at(r, x, y, t)
Expected locations of objects We represent the expected locations of objects by a new fluent at desired location(ep, t) describing that an object ep is at its expected position in the house at time step t. Unlike the fluents above, we can define at desired location(ep, t) in terms of the others (much like a “derived predicate”): at desired location(ep, t) ← at(ep, x, y, t), in place(ep, x, y)
(1)
To describe that a robot r cannot go to a location (x, y) if that location is already occupied by a stationary object, we need to know in advance the locations of stationary objects in the house. Such knowledge does not change in time, and is independent from the rest of the domain representation. Therefore, we represent it as a part of the “background knowledge” using atoms of the form occupied(x, y), and make use of it as follows: ← goto(r, x, y, t), occupied(x, y)
¬at desired location(ep, t) ← not at desired location(ep, t) The second rule expresses that normally the movable objects in an untidy house are not at their desired locations. The first rule formalizes that the object ep is at its desired location if it is at some “appropriate” position (x, y) in the right room. Here in place is not a fluent, but an external predicate; it is defined externally in Prolog as follows:
(2)
Ramifications If the robot is holding an object, then the location of the object changes as the location of the robot changes. In that sense, the change of the location of the object is a ramification of robot’s action. ASP allows us to represent this ramification as follows: at(ep, x, y, t) ← connected(r, ep, t), at(r, x, y, t)
in_place(EP,X,Y) :- belongs(EP,Obj), type_of(Obj,Type), el(Type,Room), area(Room,Xmin,Xmax,Ymin,Ymax), X>=Xmin, X<=Xmax, Y>=Ymin, Y>=Ymax. Here belongs(EP,OBJ), type of(OBJ,Type) describes the type Type of an object Obj that the endpoint EP belongs to, and el(Type,Room) describes the expected room of an object of type Type. The rest of the body of the rule above checks that the endpoint’s location (X,Y) is a desired part of the room Room.
Constraints ASP allows us to represent constraints on the states of the world. For instance, the following constraint expresses that, at any time step t, two objects ep and ep1 do not reside at the same location by the constraint ← at(ep, x, y, t), at(ep1, x, y, t) (ep 6= ep1). 79
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
area(Room,Xmin,Xmax,Ymin,Ymax) describes either an exchange area (if the object does not belong to the room where it is at) or a deposit area (if the object belongs to the room where it is at). Note also that knowledge about specific objects in a room as well as specific deposit and exchange areas are not common knowledge to all robots; each robot has a different knowledge of objects in their room.
Acquiring commonsense knowledge In order to acquire the knowledge of expected rooms of objects, which is represented by el(Type,Room), we make use of existing commonsense knowledge bases. Specifically, we use ConceptNet. Using Python API of ConceptNet 4.0, we can easily query which objects are likely to be located at a specific room. The network provides a well-suited relation for our purpose, called “At Location”. As the name itself suggests, it is a relation denoting the locations of objects.
Tidy house Once we define the expected locations of objects in a house as described above, we can introduce another statically determined fluent tidy to define a tidy house: ¬tidy(t) ← ¬at desired location(ep, t)
For instance, we query the objects which are likely to be located in the bedroom, and automatically generate a list of facts as follows:
tidy(t) ← not ¬tidy(t)
result = Assertion.objects.filter( \ relation=atLocation, concept2=bedroom, \ score__gte=threshold)
The second rule above expresses that the house is tidy normally. The first rule above describes the exceptions: when an object is not at its expected location, the house is untidy.
for assertion in result: print ’el(%s, bedroom).’ \ % assertion.concept1.text
5. EMBEDDING GEOMETRIC REASONING AND TEMPORAL REASONING IN PLANNING
Here atLocation represents the “At Location” relation, and bedroom is representing the “Bedroom” concept. The query outcome denoted by result is a set of assertions. An assertion in ConceptNet is simply an object which includes two related concepts, and a relation connecting these two concepts. Assertions also have some intrinsic properties like the language of the assertion, and the frequency in which the given two concepts are related two each other by the given relation. “Score” is one of these intrinsic values which denotes the reliability of an assertion. In order to eliminate possibly false assertions, we filter out the ones with a score less than the value of threshold. The value of threshold is determined empirically, and is equal to 5 for this case.
We can embed geometric reasoning in causal reasoning by making use of external predicates as well. For instance, suppose that the external predicate path exists(x, y, x1, y1) is implemented in C++ utilizing Rapidly exploring Random Trees (RRTs) (Lavalle (1998)); so it returns 1 (resp. 0) if there is a (resp. there is no) collision-free path between (x, y) and (x1, y1). Then we can express that the robot r cannot go from (x1, y1) to (x, y) where path exists(x, y, x1, y1) does not hold, by the following constraint: ← goto(r, x, y, t), at(r, x1, y1, t), @path exists(x1, y1, x, y) == 0.
After obtaining the query result, we simply represent it in Prolog using atoms of the form el(Type,Room). Here are some assertions about what is expected in the bedroom:
For multiple autonomous robots to complete cleaning a house by a given time, durations of actions should also be taken into account. Since the robots can help each other, when a help request is received, the robot should be able to autonomously decide whether he has enough time to complete his tasks and help the other robot by the given deadline. Since changing locations to be able to help each other takes some time, transportation delay (depending on the length of the continuous trajectory) should also be taken into account.
el(dresser, bedroom). el(mirror, bedroom). el(bed, bedroom). el(pillow, bedroom). el(closet, bedroom). el(person, bedroom). el(blanket, bedroom). el(pillowcase, bedroom). el(wardrobe, bedroom).
To represent duration of actions, we introduce the fluent robot time(r, d, t) (“duration of the action performed by robot r at time step t is d”). Then we define effects of actions on these fluents accordingly. For instance, the rule below expresses that if the action of a robot r attaching to an object takes 3 units of time: robot time(r, 3, t + 1) ← attach(r, t)
We could have obtained a much longer list of facts if the threshold had been set to a lower value; but then we would have jeopardized the integrity of the facts. ConceptNet is generated automatically using the data gathered collaboratively by Open Mind Common Sense Project, and as a result, contains some unreliable knowledge. Still, eliminating the unreliability is possible as described above, with the help of the easy-to-use API of ConceptNet.
Every attach performed by the robots is nearly identical, therefore it is reliable to assign a constant value for the duration of this action. But the idea may not be applicable to other actions, such as changing the location. Since the path traveled by the robot is the main factor contributing to the duration of a movement of the robot, we tried to obtain an estimate by taking the length of the path into account using the following rule:
Note that the expected location of an object depends on where it is: for instance, the expected location of a book on the floor of a kitchen is the exchange area between kitchen and living room (so that the robot whose goal is to tidy the living room can pick it up and put it in the bookcase); on the other hand, the expected location of a plate on the floor of kitchen is the dishwasher. Therefore, the predicate
robot time(r, @time estimate(x1, y1, x2, y2), t + 1) ← goto(r, x1, y1, t), at(r, x2, y2, t) 80
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
occupied by the stationary objects; so, in Scenario 1, the robot can go to where the red pillow is, and given the following planning problem
Here @time estimate(x1, y1, x2, y2) is an external predicate implemented in C++. It calls a motion planner to find a trajectory from an initial position (x1, y1) to a final position (x2, y2), and then returns a duration estimate between 1 and 4, based on the total length of this trajectory. Therefore, the rule expresses that if the robot r goes from one point to another, the execution time is proportional to the length of the path it follows.
%% Planning problem for Scenario 1 %% initial state % Robot 1 is at (4,6) in bedroom :- not at(r1,4,6,0). % Robot 1 is free :- not free(0). % red pillow is (1,2) in bedroom :- not at(redpillow1,1,2,0). % initially elapsed time is 0 :- not elapsed_time(0,0).
There is no obligation that every robot should perform an action at every state. Therefore, we define the default value for the duration of an action to be 0: robot time(r, 0, t) ← not ¬robot time(r, 0, t) While robot time(r, d, t) estimates the duration of a single action, we also need a fluent to keep track of the total amount of time starting from the initial state. For that we introduce a statically determined fluent elapsed time(d, t) (“it takes duration d to reach time step t”), and define it essentially by accumulating the durations of each action.
%% goal :- not tidy(t). % the bedroom is tidy :- not free(t). % Robot1 finished cleaning naive task planning using iClingo without geometric feasibility checks computes the following geometrically infeasible plan.
6. HYBRID PLANNING USING ASP Once the housekeeping domain is represented, and both the background/commonsense knowledge and geometric/temporal reasoning are embedded in the high-level representation, we can solve planning problems using ASP.
%% Plan (without external predicates): % go to where the red pillow is 0. goto(r1,1,2,0) % pick the red pillow 1. attach(r1,1) % go to where pillow is expected to be 2. goto(r1,5,1,2) % put the red pillow 3. detach(r1,3)
Since geometric reasoning and temporal reasoning (via a motion planner) are embedded in the computation, the calculated plans can be seen as hybrid plans integrating discrete ASP planning and continuous motion planning. Hybrid plans help computation of plans that are geometrically feasible as well as temporally feasible. Let us show these two advantages of hybrid planning with examples.
However, our hybrid planning approach identifies that such a planning problem is infeasible (i.e., iClingo returns that no solution exists) thanks to the geometric reasoning embedded into high-level representation. The second example shows the usefulness of embedding temporal reasoning in planning, by means of estimating the durations of actions based on the length of paths (computed by motion planners), to identify infeasible plans due to temporal constraints. Consider Scenario 2, described by the following planning problem %% Planning problem for Scenario 2 %% initial state % Robot 2 is at (2,2) in kitchen :- not at(r1,2,2,0). % Robot 2 is free :- not free(0). % location of plate1 is (5,5) in kitchen :- not at(plate1,5,5,0). % location of fork1 is (1,5) in kitchen :- not at(fork1,1,5,0). % location of spoon1 is (7,6) in kitchen :- not at(spoon1,7,6,0). % location of mug1 is (7,5) in kitchen :- not at(mug1,7,5,0). % initially elapsed time is 0 :- not elapsed_time(0,0).
Fig. 2. Housekeeping domain for Scenario 1 Consider Scenario 1, depicted in Figure 2, in which one of the nightstands together with the bed forms a narrow passage possibly blocking the robot to reach the north part of the bedroom where the red pillow is located. Note that this kind of geometric feasibility checks can only be performed using full geometric models of the robot and the room at a continuous level, and cannot be abstracted at the grid level without loosing completeness. Therefore, without geometric reasoning, naive task planning may lead to infeasible plans.
%% goal :- not tidy(t). % the kitchen is tidy :- not free(t). % Robot1 finished cleaning
Indeed, at the grid-level, preconditions (1) and (2) prohibit the robot to go to the locations occupied by the robot itself and 81
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
Table 1. Plans for Scenario 2 Time Step 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Plan with duration estimation goto(r2,1,5,0) attach(r2,1) goto(r2,0,3,2) detach(r2,3) goto(r2,5,5,4) attach(r2,5) goto(r2,1,3,6) detach(r2,7) goto(r2,7,5,8) attach(r2,9) goto(r2,0,0,10) detach(r2,11) goto(r2,7,6,12) attach(r2,13) goto(r2,2,3,14) detach(r2,15)
Elapsed Time 1 2 3 4 6 7 9 10 13 14 17 18 21 22 24 25
Table 3. Planning Time (seconds)
Plan without duration estimation
Elapsed Time
goto(r2,7,6,0) attach(r2,1) goto(r2,2,3,2) detach(r2,3) goto(r2,7,5,4) attach(r2,5) goto(r2,0,0,6) detach(r2,7) goto(r2,5,5,8) attach(r2,9) goto(r2,1,2,10) detach(r2,11) goto(r2,1,5,12) attach(r2,13) goto(r2,1,0,14) detach(r2,15)
Problem 1 Problem 2 Problem 3 Problem 4 Problem 5
2 3 5 6 9 10 13 14 17 18 20 21 22 23 25 26
Problem 1 Problem 2 Problem 3 Problem 4 Problem 5
CC ALC- ManySAT 2703.18 2800.62 2757.25 3235.79 3798.89
iClingo 79.34 138.48 44.53 78.39 249.15
We formalized housekeeping domain with multiple cleaning robots, in ASP, and solved some housekeeping problem instances using the ASP iClingo solver as part of a planning and monitoring framework. In particular, using ASP, we could embed three sorts of semantic knowledge into high-level representation of housekeeping domain for intelligent reasoning: 1) commonsense knowledge automatically extracted from the commonsense knowledge base ConceptNet, 2) feasibility check of plans via (continuous) geometric reasoning (e.g., RRTbased motion planning), 3) estimated durations of actions computed also by means of motion planning algorithms. We have also shown by some experiments that the use of ASP and iClingo over action language C+ and the reasoner CC ALC is advantageous in terms of memory consumption.
Note that the hybrid planning approach can also be extended to allow for collaborative actions of multiple housekeeping robots (Aker et al. (2011b,a)). In particular, consider Scenario 3, a variation of Scenario 2 where there exists a second robot that is available to help Robot 2 in the kitchen. A collaborative plan for these two robots calculated by iClingo is presented in Table 2. Note that the collaboration of robots decrease the total plan duration to 14 time units.
In our approach, we do not explicitly address environment uncertainty, that is, we assume we have complete knowledge of the environment. Although this assumption may seem restrictive, in fact it is quite reasonable to describe the basic results of this paper. Nevertheless, practical applications can easily integrate our approach with sensor-based updates, high level exploration behaviors and feedback control to effectively handle such uncertainty.
Table 2. Collaborative Plan for Scenario 3
goto(r2,7,5,0) goto(r1,7,6,0) attach(r2,1) attach(r1,1) goto(r2,1,0,2) goto(r1,0,3,2) detach(r2,3) detach(r1,3) goto(r2,5,5,4) goto(r1,1,5,4) attach(r2,5) attach(r1,5) goto(r2,1,3,6) detach(r2,7) detach(r1,7)
CC ALC- MiniSAT 2737.56 2706.93 2713.46 3399.64 3797.79
7. CONCLUSION AND DISCUSSION
Table 1 presents these two plans. Note that, even though both plans last 16 time steps at the discrete level, the plan computed with the duration estimation obeys the temporal constraint of 25 time units, while the plan computed without duration estimation lasts for 26 time units, violating the temporal constraint. Indeed, according to the continuous trajectories computed for each “go” action, the total length of the trajectory for the plan computed without temporal constraints is 55 units, whereas the total length of the trajectory for the plan computed with temporal constraints is 51 units; assuming a constant velocity of 3 units leads to duration estimates of 26 and 25 units respectively.
0 1 2 3 4 5 6 7
iClingo 129.60 197.07 65.53 129.65 276.35
Experiments are performed at a workstation with two 1.60GHz Intel Xeon E5310 Quad-Core Processor and 16GB RAM. Results are summarized in Tables 3 and 4. According to these results, the timings are comparable, whereas CC ALC subsumes much more memory than iClingo does. This is due to the incremental computation of a plan by iClingo.
% total elapsed time is <= 25 units :- elapsed_time(Dur,t), Dur>25.
Plan with two robots
CC ALC- ManySAT 110.58 146.13 87.47 197.26 264.44
Table 4. Memory Usage (MB)
We compute two shortest plans (of length 16) for this problem using iClingo: one without any temporal constraints, and one with the temporal constraint where the total duration of the plan is limited to 25 units of time. For the latter problem we add the following line to the planning problem description above:
Time Step
CC ALC- MiniSAT 113.58 152.05 93.51 190.49 252.88
Elapsed Time
Suppose that, once each robot obtains the current state of the world from the sensor information, each cleaning robot autonomously finds a hybrid plan to achieve her tasks in a given time, using the ASP-based approach. Then each robot starts executing her plan. However, a plan execution may fail, for instance, due to the unpredictable interference of human. Human may relocate an object which is to be carried by robot, or bring a new movable object to the room. Since the robots are unaware of these dynamic changes, they may collide with these obstacles. Furthermore, while most of the moveable objects are carried with only one robot, some of these objects are heavy and their manipulation requires two robots. The robots do not know in advance which objects are heavy, but discover a heavy object only when they attempt to move it. Also, a failure may cause a delay and put the time constraint at stake. In such a case, a robot may need assistance of another robot to meet
3 4 7 8 10 11 13 14
To compare the computational efficiency (in terms of CPU time and memory consumption) of the ASP-based approach to housekeeping, with our C+-based approach, we have experimented with 5 different problem instances in housekeeping domain. We have used iClingo Version 3.0.2, CCalc Version 2.0 with the SAT solver MiniSAT (E´en and S¨orensson (2003)) Version 2.2. and ManySAT (Hamadi et al. (2009)) Version 1.5. 82
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
Domain Description
Sensor Information
Geometric Models & Kinematic Relations
Commonsense Knowledge
Erdem, E., Haspalamutgil, K., Patoglu, V., and Uras, T. (2012). Causality-based planning and diagnostic reasoning for cognitive factories. In Proc. of ETFA. Erdem, E., Erdem, Y., Erdogan, H., and Oztok, U. (2011a). Finding answers and generating explanations for complex biomedical queries. In Proc. of AAAI. Erdem, E., Haspalamutgil, K., Palaz, C., Patoglu, V., and Uras, T. (2011b). Combining high-level causal reasoning with lowlevel geometric reasoning and motion planning for robotic manipulation. In Proc. of ICRA. Erdem, E. and Patoglu, V. (2012). Applications of action languages in cognitive robotics. In Correct Reasoning, 229– 246. Erdem, E. and Wong, M.D.F. (2004). Rectilinear steiner tree construction using answer set programming. In Proc. of ICLP, 386–399. Ferrein, A., Fritz, C., and Lakemeyer, G. (2005). Using GOLOG for deliberation and team coordination in robotic soccer. K¨unstliche Intelligenz, 19(1), 24–30. Gebser, M., Kaminski, R., Kaufmann, B., Ostrowski, M., Schaub, T., and Thiele, S. (2008). Engineering an incremental ASP solver. In Proc. of ICLP, 190–205. Gebser, M., Guziolowski, C., Ivanchev, M., Schaub, T., Siegel, A., Thiele, S., and Veber, P. (2010). Repair and prediction (under inconsistency) in large biological networks with answer set programming. In Proc. of KR. Gelfond, M. and Lifschitz, V. (1991). Classical negation in logic programs and disjunctive databases. New Generation Computing, 9, 365–385. Giunchiglia, E., Lee, J., Lifschitz, V., McCain, N., and Turner, H. (2004). Nonmonotonic causal theories. Artificial Intelligence, 153, 49–104. Hamadi, Y., Jabbour, S., and Sais, L. (2009). Control-based clause sharing in parallel sat solving. In Proc. of IJCAI, 499– 504. Lavalle, S.M. (1998). Rapidly-exploring random trees: A new tool for path planning. Technical report. Lifschitz, V. (2008). What is answer set programming? In Proc. of AAAI. Liu, H. and Singh, P. (2004). ConceptNet: A practical commonsense reasoning toolkit. BT Technology Journal, 22. McCain, N. and Turner, H. (1997). Causal theories of action and change. In Proc. of AAAI/IAAI, 460–465. Nogueira, M., Balduccini, M., Gelfond, M., Watson, R., and Barry, M. (2001). An a-prolog decision support system for the space shuttle. In Proc. of PADL, 169–183. Springer. Simons, P., Niemel¨a, I., and Soininen, T. (2002). Extending and implementing the stable model semantics. Artificial Intelligence, 138. Tari, L., Anwar, S., Liang, S., Hakenberg, J., and Baral, C. (2010). Synthesis of pharmacokinetic pathways through knowledge acquisition and automated reasoning. In Proc. of PSB, 465–476.
Planning Problem
TASK PLANNING & MOTION PLANNING
Execute the Plan
Check for a Discrepancy
Failure?
No
Yes
Diagnose the Cause of the Failure Unknown Object
Find a New Trajectory to the Next Configuration
Object not Found
Find a New Plan with the Updated Initial State
Heavy Object
Find a New Plan that Involves Two Robots
EXECUTION & MONITORING
Fig. 3. An execution and monitoring algorithm for housekeeping domain. its deadline. When such incidents occur, robots can identify the cause of the failure and act accordingly, e.g., according to a planning and monitoring algorithm similar to that of (Aker et al. (2011a)) shown in Fig. 3. In addition, some failures can be diagnosed automatically using high-level representation and reasoning and recovered by extending the execution and monitoring algorithm accordingly Erdem et al. (2012). We have shown the applicability of the ASP-based approach to hybrid planning, in such a planning and monitoring framework with a simulation of a housekeeping domain. The domain is modeled using VRML 2.0 and the execution of the actions is implemented in Matlab/Simulink. Video clips illustrating this simulation can be found at http://cogrobo.sabanciuniv. edu/demos/housekeeping/housekeeping.flv. 8. ACKNOWLEDGMENTS We thank anonymous reviewers for their useful comments. This work has been partially supported by Sabanci University IRP Grant, and TUBITAK Grant 111E116. REFERENCES Aker, E., Erdogan, A., Erdem, E., and Patoglu, V. (2011a). Causal reasoning for planning and coordination of multiple housekeeping robots. In Proc. of LPNMR. Aker, E., Erdogan, A., Erdem, E., and Patoglu, V. (2011b). Housekeeping with multiple autonomous robots: Representation, reasoning and execution. In Proc. of Commonsense. Brewka, G., Eiter, T., and Truszczynski, M. (2011). Answer set programming at a glance. Communications of the ACM, 54(12), 92–103. E´en, N. and S¨orensson, N. (2003). An extensible sat-solver. In Proc. of SAT, 502–518. 83
Answer Set Programming for Reasoning with Semantic Knowledge in Collaborative Housekeeping Robotics Erdi Aker ∗ Volkan Patoglu ∗∗ Esra Erdem ∗ ∗
Computer Science and Engineering, Faculty of Engineering and Natural ˙ Sciences, Sabancı University, Istanbul, TURKEY ∗∗ Mechatronics, Faculty of Engineering and Natural Sciences, ˙ Sabancı University, Istanbul, TURKEY
Abstract: Answer Set Programming (ASP) is a knowledge representation and reasoning paradigm with high-level expressive logic-based formalism, and efficient solvers; it is applied to solve hard problems in various domains, such as, systems biology, wire routing, space shuttle control. In this paper, we present an application of ASP to housekeeping robotics, by showing how the following problems are addressed using computational methods/tools of ASP: 1) embedding commonsense knowledge automatically extracted from the commonsense knowledge base ConceptNet, into high-level representation, 2) embedding (continuous) geometric reasoning and temporal reasoning about durations of actions, into (discrete) high-level reasoning. We illustrate the applicability of ASP on several housekeeping robotics problems, and report on the computational efficiency in terms of CPU time and memory. Keywords: Knowledge representation, reasoning, housekeeping robotics, answer set programming, commonsense knowledge 1. INTRODUCTION
Second, as in our earlier work (Erdem et al. (2011b)), we embed continuous geometric reasoning (e.g., to avoid robot-robot, robot-stationary object and robot-moveable object collisions) also in the high-level discrete representation of robots’ actions in ASP. In this way, not only the ASP solver guides the motion planner by providing a discrete task plan that satisfies some given complex temporal goals (e.g., some actions like carrying heavy objects should be postponed to later steps in a plan) or by conveying information as to how some actions should be executed (e.g., a fragile object should be carried slowly), but also the motion planner guides the ASP solver by providing information about the configurations that would lead to collisions so that the ASP solver can find feasible plans. Also, for multiple autonomous robots to complete cleaning a house by a given time, durations of actions should also be taken into account. However, this brings about further challenges. Since the robots can help each other, when a help request is received, the robot should be able to autonomously decide whether he has enough time to complete his tasks and help the other robot by the given deadline. Since changing locations to be able to help each other takes some time, transportation delay (depending on the length of the continuous trajectory) should also be taken into account. To address these challenges we also embed temporal reasoning about durations of actions into high-level representation and reasoning.
Answer Set Programming (ASP) (Lifschitz (2008); Brewka et al. (2011)) is a knowledge representation and reasoning paradigm with high-level expressive logic-based formalism, and efficient solvers (e.g., iClingo (Gebser et al. (2008))). Due to these advantages, it has been applied in many domains, such as systems biology (Tari et al. (2010); Gebser et al. (2010)), biomedical informatics (Erdem et al. (2011a)), wire routing (Erdem and Wong (2004)), space shuttle control (Nogueira et al. (2001)), etc. The idea is to formally represent the computational problems (e.g., actions of robots and planning problems) in ASP, and use the state-of-the-art ASP solvers to compute models of the formulation that characterize the solutions (e.g., discrete plans) of the given problem. In this paper, we illustrate how computational methods/tools of ASP can be used effectively for the housekeeping robotics problem (i.e., multiple autonomous robots cleaning a house collaboratively). Advantages of using ASP for representing and reasoning about housekeeping robotics domains can be summarized along with the main contributions of this paper as follows. First, one of the challenges of housekeeping robotics domain is the necessity of commonsense knowledge for the robots to behave intelligently while tidying in a house. For instance, in a tidy house, books are in the bookcase, dirty dishes are in the dishwasher, pillows are in the closet; some objects, like glasses, are fragile, whereas some others, like pillows are not. Using high-level formalism, the computational methods and tools of ASP, we can automatically extract such commonsense knowledge from the commonsense knowledge base ConceptNet (Liu and Singh (2004)), and embed it in the high-level representation of robots’ actions in ASP. 978-3-902823-11-3/12/$20.00 © 2012 IFAC
We have earlier studied (Aker et al. (2011b,a); Erdem and Patoglu (2012)) housekeeping robotics problem using a different knowledge representation and reasoning framework: we have represented the robots’ actions and planning problems in the high-level action description language C+ (Giunchiglia et al. (2004)) and used the causal reasoner CC ALC (McCain and Turner (1997)) to solve task planning problems. Apart from the difference in representation and reasoning frameworks, the 77
10.3182/20120905-3-HR-2030.00169
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
presented work is different from our earlier studies. First, the problem of automatically extracting and embedding commonsense knowledge in high-level representation of housekeeping domain is not addressed in our earlier studies (Aker et al. (2011b,a)). Also, we consider a more involved housekeeping domain where durations of actions are taken into account; in our earlier studies, actions are assumed to be instantaneous. It is important to emphasize here that, unlike action description languages, like C+, ASP allows us to represent non-Markovian domains where the effects of actions cannot simply be described in terms of the current state and the next state, but in terms of several consecutive states. Also, the ASP solver iClingo computes an answer to a planning/prediction problem incrementally, and thus uses less memory compared to the causal reasoner CC ALC. We observe these advantages of ASP over our earlier approach using causality-based formalisms and solvers, also in the experiments that we performed over some housekeeping problems instances. 2. ANSWER SET PROGRAMMING The idea of ASP is to represent knowledge (e.g., actions of multiple robots) as a “program” and to reason about the knowledge (e.g., find a plan of robots’ actions) by computing models, called “answer sets” (Gelfond and Lifschitz (1991)), of the program using “ASP solvers” like iClingo.
Fig. 1. A sample house keeping domain A group of rules that follow a pattern can be often described in a compact way using “schematic variables”. For instance, we can write the program pi ← not pi+1 (1 ≤ i ≤ 7) as follows index(1). index(2). ... index(7). p(i) ← not p(i + 1), index(i). The auxiliary predicate index(i) is introduced to describe the ranges of variables. ASP solvers compute an answer set for a given program that contains variables, after “grounding” the program. The “definitions” of such auxiliary predicates tell the ASP solver how to substitute specific values for variables in schematic expressions. Variables can be also used “locally” to describe the list of formulas. For instance, the rule 1{p1 , . . . , p7 }1 can be represented as follows: 1{p(i) : index(i)}1.
An (ASP) program is a finite set of rules of the form L0 ← L1 , . . . , Lk , not Lk+1 , . . . , not Lm where m ≥ k ≥ 0, L0 is a literal (a propositional atom p or its negation ¬p) or ⊥, and each Li (i > 0) is a literal. In such a rule, L0 is called the head and L1 , . . . , Lk , not Lk+1 , . . . , not Lm is called the body of the rule. If the head of a rule is ⊥, then the rule is called a constraint; and ⊥ is simply dropped from the head. Note the two sorts of negation: classical negation ¬ as in classical logic, and default negation not. Intuitively, ¬p means that “it is known that p is not true” whereas not p means that “it is not known that p is true”. Default negation is useful in expressing default values of atoms. For instance, closed ← not ¬closed expresses that a door is closed by default (i.e., unless it is not known that it is open). Intuitively, an “answer set” for an ASP program is a “minimal” (consistent) set of literals that satisfies the program. If an atom p is included in an answer set, then p is known to be “true”. If ¬p is included in an answer set, then p is known to be “false”. Otherwise, neither p nor ¬p is in an answer set, p is unknown. Due to its semantics, ASP is different from Prolog. For a precise definition of an answer set we refer the reader to (Gelfond and Lifschitz (1991)).
3. REPRESENTING HOUSEKEEPING DOMAIN IN ASP Consider a house consisting of three rooms: a bedroom, a living room and a kitchen as shown in Fig. 1. There are three cleaning robots in the house. The furniture is stationary and their locations are known to the robots a priori. Other objects are movable. There are three types of movable objects: books (green pentagon shaped objects), pillows (red triangular objects) and dishes (blue circular objects). The locations of the movable objects are not known to the robots a priori. The goal is for the cleaning robots to tidy the house collaboratively in a given amount of time.
When we represent a problem in ASP, we use special constructs of the form l{L1 , . . . , Lk }u (called cardinality expressions) where each Li is a literal and l and u are nonnegative integers denoting the “lower bound” and the “upper bound” (Simons et al. (2002)). Programs using these constructs can be viewed as abbreviations for normal programs (Ferrein et al. (2005)). Such an expression describes the subsets of the set {L1 , . . . , Lk } whose cardinalities are at least l and at most u. Such expressions when used in heads of rules generate many answer sets whose cardinality is at least l and at most u, and when used in constraints eliminate some answer sets.
We represent the housekeeping domain in ASP and then solve planning problems using the ASP solver iClingo. In the following, we describe how housekeeping domain is represented in ASP. Fluents and actions To represent the robots’ actions and the change in housekeeping domain, we think of the world as being 78
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
in one of many states. To be able to describe the states of the world, we need to introduce fluents (atoms whose truth values depend on time).
According to the formulation above, actions can be executed concurrently. To ensure that some actions cannot be executed concurrently, we make use of constraints as well. For instance, the following rules express that a robot cannot attach to an object while moving to a location: ← goto(r, x, y), attach(r).
We view the house as a grid; so the robots and the endpoints of objects (discrete set of points on the object through which the object can be grasped) are located at grid-points. Here are some of the fluents introduced for housekeeping domain: at(th, x, y, t) (“thing th is at (x, y) at time step t”), connected(r, ep, t) (“robot r is connected to endpoint ep of an object at time step t”).
4. EMBEDDING COMMONSENSE KNOWLEDGE IN PLANNING In the housekeeping domain, the robots need to know that books are expected to be in the bookcase, dirty dishes in the dishwasher, and pillows in the closet. Moreover, a bookcase is expected to be in the living-room, dishwasher in the kitchen, and the closet in the bedroom. In addition, the robots should have an understanding of a tidy house to be able to clean a house autonomously: tidying a house means that the objects are at their desired locations. Also, while cleaning a house, robots should pay more attention while carrying fragile objects; for that they should have an understanding of what a fragile object is. Such commonsense knowledge is formally represented already in commonsense knowledge bases, such as ConceptNet.
We introduce the following elementary actions to represent robots’ actions: goto(r, x, y, t) (“robot r goes to (x, y) at time step t”), detach(r, t) (“robot r detaches from the object it is connected to, at time step t”), and attach(r, t) with an attribute attach point(r, ep, t)(“robot r attaches to an object at its endpoint ep at time step t”). Direct effects of actions We describe the direct effects of the actions above by ASP rules. For instance, the following rule expresses that the direct effect of the action of a robot r going to a location (x, y): at(r, x, y, t + 1) ← goto(r, x, y, t) Similarly, we can describe the direct effects of the action of a robot r detaching from the endpoints of an object it is connected to: ¬connected(r, ep, t + 1) ← detach(r, t), connected(r, ep, t) and the direct effect of the action of a robot r attaching to an endpoint of an object: connected(r, ep, t + 1) ← attach(r, t), attach point(r, ep, t)
ASP allows us to extract and embed commonsense knowledge from these knowledge bases by means of “external predicates.” External predicates are not part of the signature of the domain description (i.e., they are not declared as fluents or actions). They are implemented as functions in some programming language of the users choice, such as C++ or Prolog. External predicates take as input not only some parameters from the domain description (e.g., the locations of robots) but also detailed information that is not a part of the action domain description (e.g., commonsense knowledge). They are used to externally check some conditions.
Preconditions of actions We describe preconditions of actions using constraints. For instance, at any time step t, we can describe that a robot r cannot go to a location (x, y) if the robot is already at that location, as the following: ← goto(r, x, y, t), at(r, x, y, t)
Expected locations of objects We represent the expected locations of objects by a new fluent at desired location(ep, t) describing that an object ep is at its expected position in the house at time step t. Unlike the fluents above, we can define at desired location(ep, t) in terms of the others (much like a “derived predicate”): at desired location(ep, t) ← at(ep, x, y, t), in place(ep, x, y)
(1)
To describe that a robot r cannot go to a location (x, y) if that location is already occupied by a stationary object, we need to know in advance the locations of stationary objects in the house. Such knowledge does not change in time, and is independent from the rest of the domain representation. Therefore, we represent it as a part of the “background knowledge” using atoms of the form occupied(x, y), and make use of it as follows: ← goto(r, x, y, t), occupied(x, y)
¬at desired location(ep, t) ← not at desired location(ep, t) The second rule expresses that normally the movable objects in an untidy house are not at their desired locations. The first rule formalizes that the object ep is at its desired location if it is at some “appropriate” position (x, y) in the right room. Here in place is not a fluent, but an external predicate; it is defined externally in Prolog as follows:
(2)
Ramifications If the robot is holding an object, then the location of the object changes as the location of the robot changes. In that sense, the change of the location of the object is a ramification of robot’s action. ASP allows us to represent this ramification as follows: at(ep, x, y, t) ← connected(r, ep, t), at(r, x, y, t)
in_place(EP,X,Y) :- belongs(EP,Obj), type_of(Obj,Type), el(Type,Room), area(Room,Xmin,Xmax,Ymin,Ymax), X>=Xmin, X<=Xmax, Y>=Ymin, Y>=Ymax. Here belongs(EP,OBJ), type of(OBJ,Type) describes the type Type of an object Obj that the endpoint EP belongs to, and el(Type,Room) describes the expected room of an object of type Type. The rest of the body of the rule above checks that the endpoint’s location (X,Y) is a desired part of the room Room.
Constraints ASP allows us to represent constraints on the states of the world. For instance, the following constraint expresses that, at any time step t, two objects ep and ep1 do not reside at the same location by the constraint ← at(ep, x, y, t), at(ep1, x, y, t) (ep 6= ep1). 79
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
area(Room,Xmin,Xmax,Ymin,Ymax) describes either an exchange area (if the object does not belong to the room where it is at) or a deposit area (if the object belongs to the room where it is at). Note also that knowledge about specific objects in a room as well as specific deposit and exchange areas are not common knowledge to all robots; each robot has a different knowledge of objects in their room.
Acquiring commonsense knowledge In order to acquire the knowledge of expected rooms of objects, which is represented by el(Type,Room), we make use of existing commonsense knowledge bases. Specifically, we use ConceptNet. Using Python API of ConceptNet 4.0, we can easily query which objects are likely to be located at a specific room. The network provides a well-suited relation for our purpose, called “At Location”. As the name itself suggests, it is a relation denoting the locations of objects.
Tidy house Once we define the expected locations of objects in a house as described above, we can introduce another statically determined fluent tidy to define a tidy house: ¬tidy(t) ← ¬at desired location(ep, t)
For instance, we query the objects which are likely to be located in the bedroom, and automatically generate a list of facts as follows:
tidy(t) ← not ¬tidy(t)
result = Assertion.objects.filter( \ relation=atLocation, concept2=bedroom, \ score__gte=threshold)
The second rule above expresses that the house is tidy normally. The first rule above describes the exceptions: when an object is not at its expected location, the house is untidy.
for assertion in result: print ’el(%s, bedroom).’ \ % assertion.concept1.text
5. EMBEDDING GEOMETRIC REASONING AND TEMPORAL REASONING IN PLANNING
Here atLocation represents the “At Location” relation, and bedroom is representing the “Bedroom” concept. The query outcome denoted by result is a set of assertions. An assertion in ConceptNet is simply an object which includes two related concepts, and a relation connecting these two concepts. Assertions also have some intrinsic properties like the language of the assertion, and the frequency in which the given two concepts are related two each other by the given relation. “Score” is one of these intrinsic values which denotes the reliability of an assertion. In order to eliminate possibly false assertions, we filter out the ones with a score less than the value of threshold. The value of threshold is determined empirically, and is equal to 5 for this case.
We can embed geometric reasoning in causal reasoning by making use of external predicates as well. For instance, suppose that the external predicate path exists(x, y, x1, y1) is implemented in C++ utilizing Rapidly exploring Random Trees (RRTs) (Lavalle (1998)); so it returns 1 (resp. 0) if there is a (resp. there is no) collision-free path between (x, y) and (x1, y1). Then we can express that the robot r cannot go from (x1, y1) to (x, y) where path exists(x, y, x1, y1) does not hold, by the following constraint: ← goto(r, x, y, t), at(r, x1, y1, t), @path exists(x1, y1, x, y) == 0.
After obtaining the query result, we simply represent it in Prolog using atoms of the form el(Type,Room). Here are some assertions about what is expected in the bedroom:
For multiple autonomous robots to complete cleaning a house by a given time, durations of actions should also be taken into account. Since the robots can help each other, when a help request is received, the robot should be able to autonomously decide whether he has enough time to complete his tasks and help the other robot by the given deadline. Since changing locations to be able to help each other takes some time, transportation delay (depending on the length of the continuous trajectory) should also be taken into account.
el(dresser, bedroom). el(mirror, bedroom). el(bed, bedroom). el(pillow, bedroom). el(closet, bedroom). el(person, bedroom). el(blanket, bedroom). el(pillowcase, bedroom). el(wardrobe, bedroom).
To represent duration of actions, we introduce the fluent robot time(r, d, t) (“duration of the action performed by robot r at time step t is d”). Then we define effects of actions on these fluents accordingly. For instance, the rule below expresses that if the action of a robot r attaching to an object takes 3 units of time: robot time(r, 3, t + 1) ← attach(r, t)
We could have obtained a much longer list of facts if the threshold had been set to a lower value; but then we would have jeopardized the integrity of the facts. ConceptNet is generated automatically using the data gathered collaboratively by Open Mind Common Sense Project, and as a result, contains some unreliable knowledge. Still, eliminating the unreliability is possible as described above, with the help of the easy-to-use API of ConceptNet.
Every attach performed by the robots is nearly identical, therefore it is reliable to assign a constant value for the duration of this action. But the idea may not be applicable to other actions, such as changing the location. Since the path traveled by the robot is the main factor contributing to the duration of a movement of the robot, we tried to obtain an estimate by taking the length of the path into account using the following rule:
Note that the expected location of an object depends on where it is: for instance, the expected location of a book on the floor of a kitchen is the exchange area between kitchen and living room (so that the robot whose goal is to tidy the living room can pick it up and put it in the bookcase); on the other hand, the expected location of a plate on the floor of kitchen is the dishwasher. Therefore, the predicate
robot time(r, @time estimate(x1, y1, x2, y2), t + 1) ← goto(r, x1, y1, t), at(r, x2, y2, t) 80
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
occupied by the stationary objects; so, in Scenario 1, the robot can go to where the red pillow is, and given the following planning problem
Here @time estimate(x1, y1, x2, y2) is an external predicate implemented in C++. It calls a motion planner to find a trajectory from an initial position (x1, y1) to a final position (x2, y2), and then returns a duration estimate between 1 and 4, based on the total length of this trajectory. Therefore, the rule expresses that if the robot r goes from one point to another, the execution time is proportional to the length of the path it follows.
%% Planning problem for Scenario 1 %% initial state % Robot 1 is at (4,6) in bedroom :- not at(r1,4,6,0). % Robot 1 is free :- not free(0). % red pillow is (1,2) in bedroom :- not at(redpillow1,1,2,0). % initially elapsed time is 0 :- not elapsed_time(0,0).
There is no obligation that every robot should perform an action at every state. Therefore, we define the default value for the duration of an action to be 0: robot time(r, 0, t) ← not ¬robot time(r, 0, t) While robot time(r, d, t) estimates the duration of a single action, we also need a fluent to keep track of the total amount of time starting from the initial state. For that we introduce a statically determined fluent elapsed time(d, t) (“it takes duration d to reach time step t”), and define it essentially by accumulating the durations of each action.
%% goal :- not tidy(t). % the bedroom is tidy :- not free(t). % Robot1 finished cleaning naive task planning using iClingo without geometric feasibility checks computes the following geometrically infeasible plan.
6. HYBRID PLANNING USING ASP Once the housekeeping domain is represented, and both the background/commonsense knowledge and geometric/temporal reasoning are embedded in the high-level representation, we can solve planning problems using ASP.
%% Plan (without external predicates): % go to where the red pillow is 0. goto(r1,1,2,0) % pick the red pillow 1. attach(r1,1) % go to where pillow is expected to be 2. goto(r1,5,1,2) % put the red pillow 3. detach(r1,3)
Since geometric reasoning and temporal reasoning (via a motion planner) are embedded in the computation, the calculated plans can be seen as hybrid plans integrating discrete ASP planning and continuous motion planning. Hybrid plans help computation of plans that are geometrically feasible as well as temporally feasible. Let us show these two advantages of hybrid planning with examples.
However, our hybrid planning approach identifies that such a planning problem is infeasible (i.e., iClingo returns that no solution exists) thanks to the geometric reasoning embedded into high-level representation. The second example shows the usefulness of embedding temporal reasoning in planning, by means of estimating the durations of actions based on the length of paths (computed by motion planners), to identify infeasible plans due to temporal constraints. Consider Scenario 2, described by the following planning problem %% Planning problem for Scenario 2 %% initial state % Robot 2 is at (2,2) in kitchen :- not at(r1,2,2,0). % Robot 2 is free :- not free(0). % location of plate1 is (5,5) in kitchen :- not at(plate1,5,5,0). % location of fork1 is (1,5) in kitchen :- not at(fork1,1,5,0). % location of spoon1 is (7,6) in kitchen :- not at(spoon1,7,6,0). % location of mug1 is (7,5) in kitchen :- not at(mug1,7,5,0). % initially elapsed time is 0 :- not elapsed_time(0,0).
Fig. 2. Housekeeping domain for Scenario 1 Consider Scenario 1, depicted in Figure 2, in which one of the nightstands together with the bed forms a narrow passage possibly blocking the robot to reach the north part of the bedroom where the red pillow is located. Note that this kind of geometric feasibility checks can only be performed using full geometric models of the robot and the room at a continuous level, and cannot be abstracted at the grid level without loosing completeness. Therefore, without geometric reasoning, naive task planning may lead to infeasible plans.
%% goal :- not tidy(t). % the kitchen is tidy :- not free(t). % Robot1 finished cleaning
Indeed, at the grid-level, preconditions (1) and (2) prohibit the robot to go to the locations occupied by the robot itself and 81
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
Table 1. Plans for Scenario 2 Time Step 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Plan with duration estimation goto(r2,1,5,0) attach(r2,1) goto(r2,0,3,2) detach(r2,3) goto(r2,5,5,4) attach(r2,5) goto(r2,1,3,6) detach(r2,7) goto(r2,7,5,8) attach(r2,9) goto(r2,0,0,10) detach(r2,11) goto(r2,7,6,12) attach(r2,13) goto(r2,2,3,14) detach(r2,15)
Elapsed Time 1 2 3 4 6 7 9 10 13 14 17 18 21 22 24 25
Table 3. Planning Time (seconds)
Plan without duration estimation
Elapsed Time
goto(r2,7,6,0) attach(r2,1) goto(r2,2,3,2) detach(r2,3) goto(r2,7,5,4) attach(r2,5) goto(r2,0,0,6) detach(r2,7) goto(r2,5,5,8) attach(r2,9) goto(r2,1,2,10) detach(r2,11) goto(r2,1,5,12) attach(r2,13) goto(r2,1,0,14) detach(r2,15)
Problem 1 Problem 2 Problem 3 Problem 4 Problem 5
2 3 5 6 9 10 13 14 17 18 20 21 22 23 25 26
Problem 1 Problem 2 Problem 3 Problem 4 Problem 5
CC ALC- ManySAT 2703.18 2800.62 2757.25 3235.79 3798.89
iClingo 79.34 138.48 44.53 78.39 249.15
We formalized housekeeping domain with multiple cleaning robots, in ASP, and solved some housekeeping problem instances using the ASP iClingo solver as part of a planning and monitoring framework. In particular, using ASP, we could embed three sorts of semantic knowledge into high-level representation of housekeeping domain for intelligent reasoning: 1) commonsense knowledge automatically extracted from the commonsense knowledge base ConceptNet, 2) feasibility check of plans via (continuous) geometric reasoning (e.g., RRTbased motion planning), 3) estimated durations of actions computed also by means of motion planning algorithms. We have also shown by some experiments that the use of ASP and iClingo over action language C+ and the reasoner CC ALC is advantageous in terms of memory consumption.
Note that the hybrid planning approach can also be extended to allow for collaborative actions of multiple housekeeping robots (Aker et al. (2011b,a)). In particular, consider Scenario 3, a variation of Scenario 2 where there exists a second robot that is available to help Robot 2 in the kitchen. A collaborative plan for these two robots calculated by iClingo is presented in Table 2. Note that the collaboration of robots decrease the total plan duration to 14 time units.
In our approach, we do not explicitly address environment uncertainty, that is, we assume we have complete knowledge of the environment. Although this assumption may seem restrictive, in fact it is quite reasonable to describe the basic results of this paper. Nevertheless, practical applications can easily integrate our approach with sensor-based updates, high level exploration behaviors and feedback control to effectively handle such uncertainty.
Table 2. Collaborative Plan for Scenario 3
goto(r2,7,5,0) goto(r1,7,6,0) attach(r2,1) attach(r1,1) goto(r2,1,0,2) goto(r1,0,3,2) detach(r2,3) detach(r1,3) goto(r2,5,5,4) goto(r1,1,5,4) attach(r2,5) attach(r1,5) goto(r2,1,3,6) detach(r2,7) detach(r1,7)
CC ALC- MiniSAT 2737.56 2706.93 2713.46 3399.64 3797.79
7. CONCLUSION AND DISCUSSION
Table 1 presents these two plans. Note that, even though both plans last 16 time steps at the discrete level, the plan computed with the duration estimation obeys the temporal constraint of 25 time units, while the plan computed without duration estimation lasts for 26 time units, violating the temporal constraint. Indeed, according to the continuous trajectories computed for each “go” action, the total length of the trajectory for the plan computed without temporal constraints is 55 units, whereas the total length of the trajectory for the plan computed with temporal constraints is 51 units; assuming a constant velocity of 3 units leads to duration estimates of 26 and 25 units respectively.
0 1 2 3 4 5 6 7
iClingo 129.60 197.07 65.53 129.65 276.35
Experiments are performed at a workstation with two 1.60GHz Intel Xeon E5310 Quad-Core Processor and 16GB RAM. Results are summarized in Tables 3 and 4. According to these results, the timings are comparable, whereas CC ALC subsumes much more memory than iClingo does. This is due to the incremental computation of a plan by iClingo.
% total elapsed time is <= 25 units :- elapsed_time(Dur,t), Dur>25.
Plan with two robots
CC ALC- ManySAT 110.58 146.13 87.47 197.26 264.44
Table 4. Memory Usage (MB)
We compute two shortest plans (of length 16) for this problem using iClingo: one without any temporal constraints, and one with the temporal constraint where the total duration of the plan is limited to 25 units of time. For the latter problem we add the following line to the planning problem description above:
Time Step
CC ALC- MiniSAT 113.58 152.05 93.51 190.49 252.88
Elapsed Time
Suppose that, once each robot obtains the current state of the world from the sensor information, each cleaning robot autonomously finds a hybrid plan to achieve her tasks in a given time, using the ASP-based approach. Then each robot starts executing her plan. However, a plan execution may fail, for instance, due to the unpredictable interference of human. Human may relocate an object which is to be carried by robot, or bring a new movable object to the room. Since the robots are unaware of these dynamic changes, they may collide with these obstacles. Furthermore, while most of the moveable objects are carried with only one robot, some of these objects are heavy and their manipulation requires two robots. The robots do not know in advance which objects are heavy, but discover a heavy object only when they attempt to move it. Also, a failure may cause a delay and put the time constraint at stake. In such a case, a robot may need assistance of another robot to meet
3 4 7 8 10 11 13 14
To compare the computational efficiency (in terms of CPU time and memory consumption) of the ASP-based approach to housekeeping, with our C+-based approach, we have experimented with 5 different problem instances in housekeeping domain. We have used iClingo Version 3.0.2, CCalc Version 2.0 with the SAT solver MiniSAT (E´en and S¨orensson (2003)) Version 2.2. and ManySAT (Hamadi et al. (2009)) Version 1.5. 82
IFAC SYROCO 2012 September 5-7, 2012. Dubrovnik, Croatia
Domain Description
Sensor Information
Geometric Models & Kinematic Relations
Commonsense Knowledge
Erdem, E., Haspalamutgil, K., Patoglu, V., and Uras, T. (2012). Causality-based planning and diagnostic reasoning for cognitive factories. In Proc. of ETFA. Erdem, E., Erdem, Y., Erdogan, H., and Oztok, U. (2011a). Finding answers and generating explanations for complex biomedical queries. In Proc. of AAAI. Erdem, E., Haspalamutgil, K., Palaz, C., Patoglu, V., and Uras, T. (2011b). Combining high-level causal reasoning with lowlevel geometric reasoning and motion planning for robotic manipulation. In Proc. of ICRA. Erdem, E. and Patoglu, V. (2012). Applications of action languages in cognitive robotics. In Correct Reasoning, 229– 246. Erdem, E. and Wong, M.D.F. (2004). Rectilinear steiner tree construction using answer set programming. In Proc. of ICLP, 386–399. Ferrein, A., Fritz, C., and Lakemeyer, G. (2005). Using GOLOG for deliberation and team coordination in robotic soccer. K¨unstliche Intelligenz, 19(1), 24–30. Gebser, M., Kaminski, R., Kaufmann, B., Ostrowski, M., Schaub, T., and Thiele, S. (2008). Engineering an incremental ASP solver. In Proc. of ICLP, 190–205. Gebser, M., Guziolowski, C., Ivanchev, M., Schaub, T., Siegel, A., Thiele, S., and Veber, P. (2010). Repair and prediction (under inconsistency) in large biological networks with answer set programming. In Proc. of KR. Gelfond, M. and Lifschitz, V. (1991). Classical negation in logic programs and disjunctive databases. New Generation Computing, 9, 365–385. Giunchiglia, E., Lee, J., Lifschitz, V., McCain, N., and Turner, H. (2004). Nonmonotonic causal theories. Artificial Intelligence, 153, 49–104. Hamadi, Y., Jabbour, S., and Sais, L. (2009). Control-based clause sharing in parallel sat solving. In Proc. of IJCAI, 499– 504. Lavalle, S.M. (1998). Rapidly-exploring random trees: A new tool for path planning. Technical report. Lifschitz, V. (2008). What is answer set programming? In Proc. of AAAI. Liu, H. and Singh, P. (2004). ConceptNet: A practical commonsense reasoning toolkit. BT Technology Journal, 22. McCain, N. and Turner, H. (1997). Causal theories of action and change. In Proc. of AAAI/IAAI, 460–465. Nogueira, M., Balduccini, M., Gelfond, M., Watson, R., and Barry, M. (2001). An a-prolog decision support system for the space shuttle. In Proc. of PADL, 169–183. Springer. Simons, P., Niemel¨a, I., and Soininen, T. (2002). Extending and implementing the stable model semantics. Artificial Intelligence, 138. Tari, L., Anwar, S., Liang, S., Hakenberg, J., and Baral, C. (2010). Synthesis of pharmacokinetic pathways through knowledge acquisition and automated reasoning. In Proc. of PSB, 465–476.
Planning Problem
TASK PLANNING & MOTION PLANNING
Execute the Plan
Check for a Discrepancy
Failure?
No
Yes
Diagnose the Cause of the Failure Unknown Object
Find a New Trajectory to the Next Configuration
Object not Found
Find a New Plan with the Updated Initial State
Heavy Object
Find a New Plan that Involves Two Robots
EXECUTION & MONITORING
Fig. 3. An execution and monitoring algorithm for housekeeping domain. its deadline. When such incidents occur, robots can identify the cause of the failure and act accordingly, e.g., according to a planning and monitoring algorithm similar to that of (Aker et al. (2011a)) shown in Fig. 3. In addition, some failures can be diagnosed automatically using high-level representation and reasoning and recovered by extending the execution and monitoring algorithm accordingly Erdem et al. (2012). We have shown the applicability of the ASP-based approach to hybrid planning, in such a planning and monitoring framework with a simulation of a housekeeping domain. The domain is modeled using VRML 2.0 and the execution of the actions is implemented in Matlab/Simulink. Video clips illustrating this simulation can be found at http://cogrobo.sabanciuniv. edu/demos/housekeeping/housekeeping.flv. 8. ACKNOWLEDGMENTS We thank anonymous reviewers for their useful comments. This work has been partially supported by Sabanci University IRP Grant, and TUBITAK Grant 111E116. REFERENCES Aker, E., Erdogan, A., Erdem, E., and Patoglu, V. (2011a). Causal reasoning for planning and coordination of multiple housekeeping robots. In Proc. of LPNMR. Aker, E., Erdogan, A., Erdem, E., and Patoglu, V. (2011b). Housekeeping with multiple autonomous robots: Representation, reasoning and execution. In Proc. of Commonsense. Brewka, G., Eiter, T., and Truszczynski, M. (2011). Answer set programming at a glance. Communications of the ACM, 54(12), 92–103. E´en, N. and S¨orensson, N. (2003). An extensible sat-solver. In Proc. of SAT, 502–518. 83