Planning and execution of tasks for a domestic robot

257

Robotics in Health and Human Services

Planning and Execution of Tasks for a Domestic Robot J a m e s L. C R O W L E Y

1. Introduction

7 Rue de Lilas, 38240 Meylan, France This paper proposes an architecture for an autonomous robot designed to perform tasks in a partially structured domestic environment. Particular attention is given to the roles of symbolic reasoning for planning and plan execution in such a robot. An architecture is proposed in which manipulation, mobility and perception are performed by a hierarchy of asynchronous processes. The hierarchies for manipulation, mobility and perception are controlled by a knowedge based "supervisor" which plans and executes domestic tasks. Planning tasks is presented as a problem of search. A review of heuristic search techniques is presented. In this review, the introduction of knowledge in the form of heuristics and subgoals is shown to greatly reduce" the computational complexity of planning. Problems related to plan execution are then discussed and a taxonomy of plan execution systems is proposed. The difficulty of plan execution is illustrated using the task "clear the dishes for the dining room table". The conclusion discusses the state of the art in each of the areas manipulation, mobility, percepetion, and symbolic reasoning as it applies to an autonomous domestic robot. The final conclusion asserts that progress requires experimentation with integrated systems.

Keywords: Domestic robots, Service robots, Autonomous robots, Planning, Artificial intelligence, Task execution, Real time knowledge-based systems.

This work was sponsored by Transitions Research Corporation North-Holland Robotics and Autonomous Systems 5 (1989) 257-272

It is sometimes claimed that we are in the midst of a third industrial revolution, this time with the systematic substitution of computing machines for mental labor and roots for repetitive labor. Evidence to support this claim can be found in the manufacturing industries. However, we note an important domain for which little progress has been made towards the substitution of machines for human labor: the domestic environment. This paper is concerned with the information processing technology necessary for the development of domestic robots. An architecture is proposed in which a knowledge based supervisor controls asynchronous hierarchies for perception, manipulation and locomotion. This architecture is James L. Crowley completed a Ph.D. at Carnegie Mellon University, in Pittsburgh, USA in 1981. From 1980 until 1985 he was a Research Scientist in the C M U Robotics Institute, where he directed the "Laboratory for Household Robotics". Since 1985 he has held the position of Professor at the Institute National Polytechnique de Grenoble (I.N.P.G.) in Grenoble, France. Professor Crowley performs his research within the Laboratoire d'Informatique Fondamental et Intelligence Artificielle (LIFIA), where he is responsible for the group: Mobile Robotics and Active Vision. In the area of machine vision, he has recently directed projects in the measurement of image motion and in the construction of a 3-D scene model, as part of project ESPRIT P 940. He is the technical coordinator of project ESPRIT BRA 3038, "Vision as Process", which has as its goal to develop an integrated goal driven system for active vision. He is also organizer and coordinator of the ESPRIT BRA "Working Group on Vision" (ESPRIT BRA 3352). In the area of mobile robotics, Professeur Crowley has developed techniques for dynamic world modeling and navigation using ultrasonic range sensors. Professor Crowley currently participates in the development of a navigation, perception and plan execution systems for project M I T H R A ( E U R E K A EU 110), a family of surveillance robots. Professor Crowley is the author of over 30 articles on vision and mobile robotics.

0921-8830/89/$3.50 © 1989, Elsevier Science Publishers B.V. (North-Holland)

258

J.L. Crowley/ Planningand Execution of Domestic Robot Tasks

based on recent experiences in developing an autonomous surveillance robot. The main goal of this paper is to expose the techniques currently available for the implementation of a knowledge based supervisor for a domestic robot. Knowledge, according to Newell [15], corresponds to competence. The knowledge, or competence, of a domestic robot should have many forms: Declarative statements which are easily communicated and modified, Relational structures (networks of "frames") which permit reasoning about routes, places, and objects, Procedural knowledge which direct reasoning (i.e. rules), Procedural "actions" which form the locomotion, perception and manipulation "skills" of the system. Of the types of knowledge representation listed above, the most controversial and least developed are the locomotion, perception and manipulation "skills". While the robotics and vision literature is rich with such techniques, the state of the art remains primitive, The few attempts to form integrated systems have been restricted to the simplest techniques, partly because of computational requirements. We assert that the integration of such techniques is fundamental to competence in a quasi-structured environment.

Table 1 Independent layers of technologyin underlying a computing engine Level Program level Architecture (RTL) level Logic-Gatelevel Circuit level Device level

Techniques Software engineering Sequential logic Boolean logic Electronics Solid state physics

layer. An entity with an identifiable architecture and a developing technology base. Newell has named this entity "The Intelligent Agent", and its layer " T h e Knowledge Level", The techniques for the knowledge level come from the field of Artificial Intelligence. An intelligent agent is composed of a physical body, a set of goals, a set of actions, and a knowledge base, The behavior of an agent is determined according to a "principle of rationality": Principle of Rationality: A n intelligent agent chooses its own actions to accomplish its goals. A computer architecture is an entity which exists independent of the logic family with which it is designed. In the same way, an intelligent agent is an entity which exists independent of the programming technology with which it is realized.

1.1. The Intelligent Agent

1.2. The Task Domain (or How I Hate Do the Dishes)

The technologies which make up a modern digital computer have evolved as a set of independent layers. These layers, and the analytic techniques which are used by engineers at each layer are summarized in Table 1. These layers are independent. That is, each layer can be implemented completely independent of the layer underneath. It is also interesting that this decomposition evolved naturally, and was observed afterwards. The conceptual entities at each layer in Table 1 involve certain components, independent of their realization. For example, a computer architecture is composed of a memory, a program address register, an instruction register, and an ALU. In his 1980 presidential address to the AAAI, Newell [15] proposed that the domain of artificial intelligence is concerned with a new layer in this hierarchy; a layer which sits above the program

This paper is concerned with the problem of how to construct the information processing architecture for an intelligent agent capable of manipulation and locomotion in a domestic environment. The need for manipulation and locomotion in a domestic environment is easily motivated by a range of applications. It is not hard to estimate that at least half of all human labor is performed on such tasks. Such a domain is also interesting on scientific terms. The domestic environment is a classic case of a "quasi-structured" environment. That is, while the over-all form of the environment is fixed and can be known in advance, the environment is likely to be cluttered in an absolutely unpredictable manner. Thus an agent which must perform tasks in such an environment can use knowledge of the environment to plan actions and to organize his

J.L. Crowley / Planning and Execution of Domestic Robot Tasks perceptions. However, the execution of actions must be based on immediate perceptions. A number of routine domestic tasks could serve as paradigms. For the problem of manipulation, these include: 1. Clearing the dinner table: After a meal, the dishes, silver etc. must be removed from the table and transported to the kitchen. While the set of objects on the table can be known in advance, their position is very difficult to predict. Plates are also likely to be covered with food debris which must be properly disposed of. 2. Picking up objects from the floor: Small children are not easily trained to put away their toys. Adults may also, from time to time, leave such objects as shoes, socks or books on the floor. While it may be difficult to predict the existence of such objects, they are readily detected. Anything which disturbs the "smoothness" of the floor, and which is not too large, is suspect. Such objects must be detected, picked up, identified and returned to their place. 3. Moving about in a cluttered environment: Humans are messy. They can leave a chair of footstool anywhere. A classic problem for a robot in a quasi-structured environment is to travel from point A to point B through a space occupied by both predicted and unpredicted obstacles. Of these chores, moving about in a cluttered environment is the most widely addressed. For variety, we conclude below with a discussion of the chore of clearing the table.

259

introduction of additional knowledge, in the form of subgoals, the complexity of the planning process can be greatly reduced. In particular, structuring problems as a hierarchy of subgoals can reduce the computational complexity of a planning process from exponential to linear complexity. Section four is concerned with the problem of plan execution. This section shows that the hierarchical decomposition of the planning process is limited by partial knowledge of the problem domain. This section also illustrates the importance of a goal directed architecture. Section five describes the planning and execution for a sample domestic chore "clear the table". The concept of a chore "script" is presented. A chore is defined as a set of tasks (or goals) which are decomposed into sub-tasks (or subgoals). The section ends with examples of OPS-5 rules which perform the decomposition from chore to action. Section six draws some conclusions, by comparing the situation of the domestic robot industry to the aviation industry at the time of the Wright brothers' first flights.

2. Asynchronous Hierarchical Control This section proposes a hierarchical asynchronous architecture for a domestic robot. It discusses the basis for the decomposition of the architecture into "levels". It then briefly describes each of the levels in such an architecture. 2. I. Layered A rchitecture

1.3. Contents" and Summary The following section proposes an information processing architecture for a domestic robot. The essential principle of this architecture is hierarchical asynchronous control. Such an architecture is increasingly recognized as naturally adapted to the function of autonomous robots in a quasi-structured or unstructured environment. This architecture contains control hierarchies for manipulation, locomotion and perception. Each of these three hierarchies contains a motor control level, an effecter level and an procedural action level. All three hierarchies are controlled by a knowledge based supervisor. Section three is a review of the state of the art in planning and problem solving. Planning is presented as a search process. We show that with the

Locomotion and manipulation in an unstructured or quasi-structured environment requires control based on perception. However, perceptual systems typically produce a hierarchy of descriptions of the environment in different coordinate systems, different levels of abstractions and different response times. No single level of this perception hierarchy is sufficient for control of locomotion or manipulation. Interestingly, the control of a manipulator or a vehicle can also be expressed as a hierarchy of control loops. As with perception, each control loop is defined by a coordinate space of the variables which are measured and controlled, and by the response times which are required. Indeed, it is possible to " m a t c h - u p " the levels in the perception, locomotion, and manipulation hierarchies in

J.L. Crowley / Planningand Execution of Domestic Robot Tasks

260

Supervisor (Production System)

. o.o"oo Actions

Actions

Actions

Control

Modeling

Centre

] Control Me,or I Motors,

sen,or Da= I Description Cameras and Sensors

Encoders

M=or

Control

Motors, Encoclers

Fig. 1. A layered architecture for hierarchical asynchronous control.

such a way that each level works in the same coordinates space and with the same level of abstraction. Adjusting the timing so that perception "keeps-up" with locomotion and manipulation is more difficult, but not impossible. It requires attention to implementation details in the perception hierarchy. The hierarchies may be tied together both by communication at each of the levels, and by a top-level "knowledge based" supervisor. The role of the supervisor is to decompose goals into subgoals, and to specify the actions necessary to accomplish each sub-goal. Such an activity depends on a large body of situation-specific knowledge. It is not easily organized as a collection of procedures. For this reason we propose that such knowledge be stored as rules, and that the supervisor be implemented as a production system. Both the control and perception hierarchies may be organized into a set of levels as shown in Fig. 1.

2. 2. The Motor Level At the lowest level, each hierarchy asynchronously processes raw signals. In the locomotion and manipulation hierarchies, this processing involves closed loop control of the motors to maintain a specified velocity, as well as capture of proprioceptive sensor signals for estimating position and velocity. In the perception hierarchy, processing involves acquiring sensor signals (in camera or sensor coordinates) and converting these

to an initial symbolic representation in vehicle coordinates. At the motor level, variables are in sensor coordinates (e.g. encoder counts, or image or camera coordinates). Response times for motor control should be on the order of 10 -3 to 10 -2 seconds. The processing cycle in the perception hierarchy, is likely to be closer to 10-1 seconds.

2.3. The Effecter Level Manipulators and vehicles are commonly controlled by processes which generate commands for each motor in order to achieve a certain coordinated behavior. In most commercial manipulators, the behavior is specified in terms of an external Cartesian coordinate space. We strongly argue that the vehicle behavior should also be expressed and controlled in an external coordinate space. Note. that for an arm mounted on a mobile robot. we propose that both the arm controller and the vehicle controller work in a common "external" coordinate system. The location of the arm in external coordinates is simply the composition of the arm configuration in Cartesian arm coordinates and the configuration (position and orientation) of the vehicle. A similar level of abstraction exists in the perception hierarchy. At this level of abstraction, the environment is described in terms of the geometric entities in external, or environmental, coordinates. An example of such geometric constructs are planar surface patches, contours and corners. The position of a sensor in the common external coordinate system is provided by the composition of the position of the sensor with regard to the vehicle and the configuration of the vehicle. At the effecter level, the perception system can use the sensor position to project abstract descriptions of sensor data into the common "external" coordinate system. The projected information may then be used to update a "unified" composition model of the environment. As a side effect of the update process, errors in the estimated position may be detected and relayed to the vehicle controller. In summary, the variables at the effecter level are expressed in a common "external" coordinate system. The entities at this level are the position and orientation of the effecter and vehicle, and the geometric structures in the environment. The cycle times for the effecter controllers should be on the

J.L. Crowley / Planning and Execution of Domestic Robot Tasks

order of 10 -2 to 10-1 seconds. For the perception hierarchy, a cycle time on the order 10-~ seconds is usually sufficient. 2.4. The Action Level

An action is a coordinated sequence of effecter commands. The concept of an action is central to both the planning and execution of tasks in the supervisor, and to the coordination and control or effecters and perception. From the perspective of the effecters, an action is a parametrized sequence of effecter commands. The sequence of steps in an action is defined in advance. However, the timing and position parameters are determined in real time based on perception. Because the sequence of measurement and control steps in an action are determined in advance, actions are best expressed as procedures. Indeed, such procedures are a form of calculation which is often difficult to express in a production system. In human terms, action level procedures correspond to learned "skills" such as driving a car, or understanding the environment in terms of rigid objects. Examples of actions in the locomotion hierarchy include following a wall, following a hall, turning a corner, or chasing a person. For manipulation, examples of actions include picking up an object, moving to a location, and placing an object. Such processes necessarily depend on asynchronous access to perceptual information. The information needed from perception may also be calculated by an action: a perceptual action. A significant amount of the coordination between effecter actions and perception can occur directly between the action level procedures. The supervisor need not be aware of such communication. Indeed, during an action, the supervisor may be occupied by other, longer term activity. 2.5. The Supervisor

A domestic robot is " p r o g r a m m e d " by specifying a sequence of tasks which it is to accomplish. Each task specifies a goal which is to be accomplished. Before its execution a task may be decomposed into a sequence of actions which comprise a plan. However, because the environment is not perfectly known in advance, the decomposition of the task into actions is not sure to succeed. For this reason, the domestic robot needs a supervisor

261

to monitor the execution of each action, and to dynamically correct the plan to account for unforeseen circumstances. Both planning and plan execution are knowledge intensive processes which are adapted to a forward chaining production system. For this reason we propose that the supervisor be realized by such a production system architecture. Our own experiments with a supervisor for a surveillance robot [7-9] employ a version of the OPS-5 language to which we have added an asynchronous message passing facility.

3. Planning as Search A human " m a s t e r " specifies a chore for a domestic robot as a set of tasks which are to be accomplished. The planning system of the domestic robot verifies that the tasks are not incompatible, and then decomposes each task into a set of subgoals to be accomplished. The lowest level subgoals in this decomposition trigger actions. This planning system should be an integral part of the supervisor; dynamic planning during task execution is essential to operation in a quasi-structured environment, The decomposition of tasks into a sequence of actions serves: 1. to verify in advance that the robot knows how to accomplish each task; 2. to verify, in advance, that the task as well as the chore, can be realized within the constraints of time and energy; and, 3. to determine any common requirements between tasks so that the tasks may be optimized. Before plunging into the problem of planning and executing tasks in a domestic environment, it is instructive to review the results of some of the recent literature concerning planning and problem solving. In the literature we find a paradigm which is increasingly recognized within the last ten years. This is the paradigm of "Planning as Search". 3.1. Planning as Search: States and Actions in a Problem Space

Most modern literature on planning and problem solving defines the problem in terms of a " p r o b l e m space" [13,15]. A problem space is defined by: 1. A set { U } of states of the universe, and

262

J.L. Crowley / Planningand Execution of Domestic Robot Tasks

2. A set (A } of actions which transform the universe between the states of ( U ) . This section is concerned with the representation of the states of a universe, and the actions which transform such states. 3.1.1. The States of a Universe A universe (or a domain) is represented by a finite set of states. Each state is equivalent to a conjunction of assertions. This conjunction of assertions may be expressed by a conjunction of expressions in predicate calculus, by a vector of state variables, or by any number of other representations. To illustrate the notion of state, consider the case of a mobile robot. The state of a mobile robot is often represented by the vector of position and orientation (x, y, O), sometimes called a posture. If these values are represented with a high precision, the set of possible position states can grow very large. Often, for planning the position states are quantified by associating intervals of the position vector (i.e. areas) to known "places". This gives a second, more abstract representation in which the state is represented by the single assertion: "The robot is at place A". Manipulation brings about a somewhat different problem. A manipulator is represented (typically) by a vector of six variables. While it may be helpful to quantize this vector into a finite set of "zones", the real problems in manipulation concern the objects which are to be manipulated. The universe is composed of assertions concerning the existence, location, and shape of these objects. As with our robot, we may quantize these variables into "zones" to simplify our planning. It is important to remark that any description of a real domain is partial. That is, in any real domain, the complete description of a state would require an infinite number of assertions. Since states are restricted to a finite number of assertions, the description is not complete, that is, partial. The incompletude has important ramifications for a plan execution system. It is also useful to remark that if a state, A, is defined by a conjunction of assertions, then a sub-set of those assertions defines a set of states (including the state A). Actions and goals are often identified using such subsets. As a result the same action may be executed in many circumstances, and a goal may be achieved in many ways.

3.1.2. The Actions in a Universe Actions are represented by l. A set of Preconditions, and 2. A set of Postconditions. The representation of actions has changed little since the days of Shakey and STRIPS [16,17]. The set of preconditions is a set of assertions which must be true before an action may be executed. The postcondition, represent a conjunction of assertions concerning the universe after the action has executed. Typically the preconditions and postconditions are a subset of the assertions which define the universe. In STRIPS the postconditions are further broken down into: 1. a delete list: assertions in the preconditions which are no longer true after the execution of the action, and 2. an add list: assertions which are made true by the action. 3.1.3. Problem Spaces In this problem space formulation, a problem is stated as: 1. A universe (or problem domain) { U }, 2. An initial state of the universe, S. 3. A set of goal states, { G). Solving a problem is equivalent to finding a sequence of actions from { A } which will transform the domain { U ) from the initial state, S, to any of the goal states, (G }. The fundamental principle for solving problems is often identified by the slogan: "Generate and Test". That is, solving a problem is a cyclic process of generating a set of adjacent states, and testing the set of adjacent states to select a state from which to continue. Such a process is known as searching i n a graph: 3.2. Searching the State Graph

In the absence of additional knowledge, planning is a process of exhaustively generating the tree of possible states using the available actions, until a state in the goal set has been detected. The path in the tree from the start state to a goal state then gives a plan, in terms of a sequence of actions, which may solve the problem. If the least-cost plan is desired then the graph must be exhaustively searched, a possibility that is only feasible in certain restricted domains, In real world problems, we are often more concerned with the cost of finding any solution at all.

J.L. Crow/ey/ Planningand Executionof DomesticRobot Tasks 3.2.1. The Cost of Searching a State Graph The cost of searching a state graph depends on two factors: b. The average branching factor for a state. d. The minimum depth to a solution. In a forward chaining system, b is the average number of actions which may be taken at any state. In a backward chaining system, b is the average number of actions which lead to each state. The depth, d, is the minimum number of actions which can be found to transform the start state into the goal state. In the absence of additional knowledge, the search tree may be generated breadth first or depth first.

3.2.2. Breadth First Search Breadth first search is the simplest to program. In breadth first search we enumerate all of the states accessible from the start state, then all of the states accessible within two actions of the start state, etc. The cost for computing a breadth first search is O(b J) in computing time and O(b J) in memory. Often the memory requirements dominate, because it is necessary to store a description of each state in the search tree.

3.2.3. Depth First Search It is known in advance that the solution will be found within N states, then we can perform the search "depth first". Depth first search requires on the average a computation time of O(ba). However, as each path is abandoned, the memory required for storing state descriptions may be freed, resulting in a memory requirement of O(d). Unfortunately, we are rarely sure to find the solution within a fixed depth. In such a case we can adopt an algorithm known as depth-first iterative deepening [13]. That is, perform a depth first search to level 1, if it is not successful repeat with depth 2 then 3 etc. until a solution is found. Such an algorithm also has a cost of O(b a) in time and O(d) in memory, without the restriction of knowing the number of levels, As we see, searching the state graphs can be expensive. The classic solution to reduce the cost is the introduction of additional knowledge.

263

3.3. Heuristic Search: The Algorithm GRAPHSEARCH and A* The word "heuristic" is from the Greek word which means " t o guide". A heuristic is a guide. One of the early results in artificial intelligence was the a general formalization of exhaustive and heuristically guided search: The algorithm GRAPHSEARCH. Although the algorithm G R A P H S E A R C H is well known, we will repeat a representation of the algorithm for completeness. The interested reader is referred to [17] for a formal definition and a thorough discussion. The GraphSearch algorithm organizes search as a cyclic process which starts with the start state and contains the following phases: 1. Given a current state, create a list of all directly accessible states (the "new" list). 2. Any state in this list which has already been explored is removed from the new list. 3. The remaining states in the new list are added to the search tree and to a list of states which are available for exploration (The "open" list). 4. A state from the open list is selected to continue the process. If this state is in the goal list the process halts. Otherwise, the cycle repeats with this selected state. If the open list is maintained as a queue, the search will be breadth first. If the open list is maintained as a stack, the search will be depth first. If the open list is sorted, based on an estimated cost " f " , the search will be a heuristic search, and the algorithm is known as algorithm A. The cost f ( n ) is the estimated cost of a solution which passes through the specified state, n. The cost f ( n ) is composed of two costs, g(n) and h(n):

f(n)=g(n)+h(n). The cost g(n) is the (estimated) cost of the path from the start state to the state n. The cost h(n) is the estimated cost of the path from the state n to the goal state. Let us define the true cost minimum path through two states n 1 and n= as k * ( n l , n2). We can then define g*(n) = k*(S, n): the true minimum cost of a path from S to a state n. h*(n) = k*(n, G): the true minimum cost from n to a state goal state, G.

J.L. Crowley / Planning and Execution of Domestic Robot Tasks

264

Thus

f*(n)=g*(n)+h*(n). If we can guarantee that the estimated cost f(n) is less than the true cost f*(n), then we can be sure that the first solution found is the least cost solution. In most cases, it is sufficient to prove that our heuristic cost, h, is such that

h(n)<~ *(n).

3.4. I. Sorts of Subgoals

Heuristic graph search is useful for two reasons; 1. It guarantees that the first solution found is the best, and 2. It reduces the computational cost of finding the solution from

O(b d)

to O(cd),

It would seem that the more subgoals a system can generate, the more efficient will be its planning system. Unfortunately, this is not the case. As the number of subgoals increases, they loose their benefit. We simply replace search through the state space with a matching problem of the same complexity. The advantage of subgoals is evident if they are few and well chosen.

where c~
Such a reduction in the branching factor can have an important role in the size of the domain in which problems can be solved. Nonetheless, such planning still has a cost which is exponential in depth. Even more importantly, in manipulation problems, it is often not all clear that a numerical heuristic exists to guide the research. In such a case, additional knowledge is needed to find a solution.

3.4. Planning with Subgoals The computational cost of planning may be reduced by the use of sub-goals. A subgoal is a (set of) intermediate states which lie between the start state and the nearest goal state. A subgoal divides the planning problem of depth d into two smaller problems of depth d 1 and d 2. The combined computational complexity of the two subproblem is:

O( b(d, +a2)) = O( bmax{a,.a2} ) as long as max( d 1, d 2 } < d, the complexity will be reduced. If the subgoal lies on the path from S to B, then d = dl + d2, and the solution will have a minimum cost. The best case is when subgoal lies half-way between S and B, in which case d = dl + d2 and d l = d2 = d/2. In this case the reduction of computational cost of planning is also maximized. Humans make extensive use of subgoals in everyday life. The reason is clear. Subgoals make everyday planning and problem solving tractable.

Typically, a problem is not decomposed into a single subgoal, but into a set of subgoals. In the planning literature, three forms of subgoals are generally identified: 1. independent subgoals 2. serializable subgoals 3. non-serializable subgoals. With independent subgoals, the order in which the subgoats are achieved is not important. For example, to set the table, a robot needs to place the dishes, the silver ware, and the glasses. In general these items may be placed on the table in any order. Many everyday tasks can be decomposed into a set of independent subgoats. Serializable subgoals must be accomplished in a certain order. For example, to serve the water, the robot must get a pitcher, fill it with water, bring it to the table, then set the pitcher on the table. (We"ll let the humans do the pouring for the moment.) The order of these operations is essential. Nonserializable subgoals are the hardest case for planning systems. A nonserializable subgoal obliges the system to undo previous subgoals to arrive at the next subgoal. If the pitcher is on the table and empty, and the goal is to have the pitcher on the table and full, it is necessary to remove the pitcher from the table in order to fill it. Non-serializable subgoals require the system to group subgoals and actions into chunks.

3.4.2. Using Subgoals in Planning Where do subgoals come from? Subgoals are little nuggets of domain specific (or situation specific) knowledge. Humans are constantly fine tuning their subgoals, both in refining the conditions in which to apply them and learning subgoals for new situations. Despite r e c e n t progress in this direction (see for example [14]) machine learning has not yet advanced to the point where we can count on automatic techniques for subgoal

J.L. Crowley / Planning and Execution of Domestic Robot Tasks

formation. Thus, for now, subgoals will have to be part of the program task for a domestic robot.

4. Plan Execution

Given a specification of a task as well as sufficient domain knowledge, the task may be decomposed into a sequence of actions. A common misconception is that because this decomposition has been performed in advance, the robot knows, in principle, what actions it must take to accomplish each task. Given that such care has been taken in forming plans, we must ask: Why do plans fail? What can be done when a plan fails? These are the questions which are addressed in this section. 4.1. Why Do Plans Fail?

When an application domain is perfectly known and perfectly predictable, The actions to accomplish a task may be completely determined in advance. Unfortunately, such a condition is only true in highly artificial toy situations in which the robot is the only agent. In any real world domain we have at least three problems: 1. The robot's descriptions of each state in the domain is partial, 2. The robot's knowledge of the effects of his actions is partial (and in particular, actions may unexpectedly fail), 3. Other agents may have unpredictably altered the state of the domain. The first problem was remarked in the previous section. A planning system represents the domain as a set of states. However, the description of the domain in any state is necessarily partial. As a result, any action may alter un-modeled aspects of the environment, causing unexpected side effects. The second problem is related to the first. The description of the environment is incomplete (some details are omitted) and imprecise (details are approximated). Such imprecision and incompletude are fundamental, simply because the universe is infinitely large and infinite precise. As a result, no action is sure to succeed with exactly the foreseen results. We should also note that it would be unrealistic to expect our procedural actions to be perfectly adapted to all circumstances. Nature

265

has been fine tuning animal systems for millions of years, and yet even for highly evolved animal species such as humans, simple actions can fail. The third problem is often ignored in problem solving research, but is a very real problem for a domestic robot. Children (and playful adults) will want to play tricks on a domestic robot. It can also happen that the robot is starting to do something stupid, and nearly humans find it desirable to interfere. (HEY! Don't take my plate! I haven't finished eating.) In the following sections we present a set of increasingly robust plan execution systems. As an organizational artifact, we organize this presentation according to a set of "levels". 4.2. Simple Plan Execution: Le~'~el 0

The failure of a plan is in fact the failure of the plan execution system to accomplish a specified task within the specified constraints. To construct our plan execution system, let us start by examining the simplest form of a plan execution system: a system with no re-planning ability which simply verifies that each action was successful. By listing the failures which can arrive with such a system, we can develop a taxonomy of systems with more sophisticated abilities to recover from failures. Our level O plan execution system is as follows. Each action is read from the plan and the specified action execution procedures are called. If the action execution procedure reports success, the next cycle is repeated for the next action. If a procedure reports failure, the system abandons the plan and executes an emergency recovery procedure (Go-Home-and-Cry). Similarly, the execution system keeps track of its expenses in time and energy. If the pre-established constraints are violated, the plan is also abandoned. Thus we see two sources of plan failure for the level O system: 1. Failure reported by an action procedure, 2. Failure to complete the chore within the established constraints. In the level O system, the most likely cause of failure for an action is that its preconditions were not satisfied. This could be because of outside influences, or because a previous action terminated successfully without yielding the required results. These two situations can both be avoided if the supervisor verifies that the domain is in the desired state before and after each action. Both verifica-

266

J.L. Crowley / Planningand Execution qf Domestic Robot Tasks

tions are necessary because the preconditions of an action may not co-coincide with the post conditions of the precedent action. The result is a system which operates in terms of desired domain states, and not in terms of actions. Indeed, the specific actions need not be determined until the moment of execution. A plan in such a system takes the form of a sequence of desired "states" of the domain. Let us call this a "level 1" plan execution system. 4.3. The Level 1 System: Choosing Actions to Accomplish Goals In the level 1 system, a plan is presented as a sequence of desired states of the universe. The difference between these states must be of the same grain as the actions with which the system may reason. In the simplest such system, this "grain" is at the level of individual procedural actions. In such a system the decomposition of task goals into action goals is performed during planning. For such a system, the supervisor operates in a cycle. In each cycle, the next goal state is retrieved from the task plan. The goal state is matched to the goal states of the available actions, and a set of possible actions is determined (usually unique). The precondition of each action is then matched to the current state of the universe, and any unsatisfied actions are rejected. If more than one action remains, the costs of the actions are estimated, and the least cost action is selected and executed. At the end of the execution, the system verifies that the goal state has been achieved. If it has not been achieved, the system repeats the cycle, searching again for an action which will bring the domain to the goal state. Because the goal states were determined by planning based on the same set of actions, for most state transitions there will be a unique action. Matching the preconditions serves to verify that each action can succeed, and to establish the values of variables, such as the location of objects, which are needed for the action. This level 1 system is better adapted to an environment which is not entirely predictable, than a level O system. Nonetheless, it can happen that no single action available to the system is capable of transforming the current domain state to the goal state. In this case the level 1 system would

also fail; a system with a more sophisticated action planning ability is required. 4.4. The Level 2 System: Planning Sequences of Actions A level 2 plan execution system is capable of composing a sequence of actions to arrive at a goal state. As with the level 1 system, if the task decomposition has been performed with the same vocabulary of actions, in normal circumstances, a single unique action will be found to transform the domain to the next goal state: However, when the domain deviates substantially from the prediction made during planning, or when the failure of an action radically transforms the state of the domain the system can construct a sequence of actions to bring the domain to the desired goal state. The construction of a sequence of actions to arrive at a goal state is exactly the task planning problem addressed in the previous section. The construction of such a plan is search problem. In most cases, it is difficult to determine a numeric heuristics to guide such search. The choice between backward chaining of actions from the goal state or forward chaining from the start state is determined by the branching factor. In manipulation problems, the branching factor often favors a backward chaining search. In locomotion, the branching factor is the average number of paths leading to or from each place. As long as the paths are two-way, the branching factor is equal for forward and backward chaining, and there is no basis for a preference. However, in domains where one seeks to impose order on a domain, the number of actions which establish a state is much less than the number of possible actions. In such a case, the branching factor tends to favor a backward chaining solution. In summary, a level 2 system works from a plan expressed as a sequence of domain states at the grain of individual actions. It operates in a cycle, always driven by the next goal state: In each cycle, it performs a backward chaining search through the space of actions from the goal state to the current state to determine an "action-level" plan. It then executes t h e sequence of actions given by this plan. If an action fails, a new "action-level" plan is constructed and executed. If no such "ac-

J.L. Crowley / Planning and Execution of Domestic Robot Tasks

tion-level" plan is found, the task is abandoned. When each action-level goal is accomplished, the system repeats the cycle for the next action-level goal. If a level 2 system is unable to find a sequence of actions to accomplish an action-level goal, then it must replan the task. That is, it must reinvoke the task planner to find a new sequence of actions and states between the current domain state and the task goal state. If such a sequence is found, the state sequence is adopted as the new task plan. If no such sequence is found, then the robot has no option but to give up the task.

4.5. Level 3: Satisfying Global Constraints In the plan execution systems discussed above, we have left aside the problem of executing a task within constraints. However, a real domestic robot is likely to be limited in autonomy due to limited energy. It is also possible that we would need to impose certain temporal constraints of the robot. In this section we consider the problem posed by the existence of global limitations on time and energy. During task specification, a budget should be established for constraints such as time and energy. Then as tasks are decomposed, each goal state should contain an estimate of the time and energy required to arrive at the state and the time and energy required to arrive at the task goal. Similarly, at the level of chores, each task state should contain an estimate for the time and energy required to arrive at the state, and the time and energy needed to arrive at the goal. During planning, the system will often have the possibility to optimize (find a simultaneous minimum) for its constraints. During execution, unforeseen circumstances may lead to a situation in which the current plan will exceed the global constraints. In such a case, it may be possible to construct a new plan which will remain within the global constraints. Such a system must have the ability to detect that the current plan will not succeed and to form a new plan. Let us define a level 3 system as a level 2 system to which we add the ability to detect when a plan will exceed its global constraints, and to replan the task in such a case. Detecting that a plan will exceed constraints requires an estimate during each action execution cycle, of the energy

267

and time needed for the entire task. Such an estimate has three parts: 1. The values of constraint already consumed, 2. The values estimated to be required to accomplish the current action, and 3. The values estimated to be required once the next action has been accomplished. The first quantity is simply the running total of the constraint. The second quantity must be estimated as each action is selected. The third quantity can be generated during task planning. If the sum of these three quantities is found to exceed the global constraint, then the task plan will fail. A new task plan should be prepared, as early as possible, to give a m a x i m u m of flexibility in avoiding the violation of the limiting constraint.

4.6. Conclusions Concerning Planning and Plan Execution We can summarize this section by listing four conclusions: 1. A task is a specification of a Goal to be accomplished, 2. A planning system decomposes a task into sub-goals, 3. An initial decomposition may be useful for determining the feasibility and cost constraints of a plan, 4. The final decomposition of a task into subgoals is best done during plan execution in order to account for unforeseen circumstances. These points are merely abstract principles. To prove their worth, thus must be applied to real world domains, such as the planning and execution of domestic chores.

5. Planning and Executing a Domestic Chore In this chapter we attempt to apply the lessons of the previous chapters to the supervisor for a domestic robot. As an example, we will take the problem of clearing the table after a meal. We will assume that the robot is equipped with a dish tray which permits it to simultaneously carry the required number of plates, glasses, silver, and serving dishes. We will sketch the behavior of the system as it locates and manipulates each of the objects on the table.

268

J.L. Crowley / Planning and Execution of Domestic Robot Tasks

5.1. Tasks and Chores

In common usage the work "task" may be used to refer to either a goal to be accomplished, or the actions which accomplish a goal. As we saw in the previous section, manipulation and locomotion in a quasi-structured environment require goal-directed behavior. Thus we will use the word " t a s k " to refer to a goal which is to be accomplished. With this definition, let us define a "chore" to be a sequence of tasks. A chore for a domestic robot is at the same level of abstraction as a mission for a surveillance robot. As with missions, a chore may be learned, and then repeated at specified intervals. Constraints concerning tasks and chores must be provided by the human master. To simplify our system, we will assume that a task is "defined" for the robot as an ordered sequence of tasks. The order in which the tasks which compose a chore must be accomplished is specified by human at the time at which the chore is defined. The chore also comes with certain global constraints, such as in time and energy. We can assume that the robot is to commence the chore after a certain time and to complete the chore before a certain time. 5.2. The Decompositions of a Chore

We distinguish three successively finer phases of decomposition of a chore: 1. The chore script, 2. Planning an instance of a chore, 3. Execution of a chore. A chore script is defined in advance as an ordered sequence of tasks. Planning techniques such as those described above can permit the chore script to be automatically planned from a chore goal. However, such techniques require an extensive knowledge base about the robot's domain and possible tasks. While such a knowledge base is desirable it is rather costly and time consuming to construct. We can sidestep the need for this knowledge base by having the user specify the definition of the chore in terms of the tasks which are to be accomplished. A chore is a general specification which can be commanded in a variety of circumstances. Once an instance of a chore has been commanded it is possible to plan the subgoals which compose certain tasks. Such a decomposition may be provi-

sionally performed at the time that the task instance is commanded. However, unforeseen circumstances may change the decomposition. Thus we find below that the final decomposition should be performed during plan execution, at the start of each task. The final decomposition of a task occurs during plan execution. We stress that the planning system specifies subgoals, not actions. To maintain an ability to adapt to unforeseen circumstances, actions should be selected as they are to be executed. 5. 3. Specifying the Tasks in a Chore

A chore is specified as a set of tasks. The degree of abstraction of the task specification is limited by the knowledge of the robot, and is actualized by a vocabulary of task verbs. Tasks may be specified by a menu driven system, or with sufficient investment, by a system which interprets pseudo-natural language using, for example, an ATN grammar. Let us assume that our robot knows that the word "Dishes" refers to the set of objects {plates, glasses, cups, saucers, forks, knives, spoons}. Let us further assume, that the r o b o t knows the geometric form of each object in the class "dishes" in the domain in which he works. That is, for each class of items in dishes, the robot is able to call the perceptual action "find-object" to determine the location of all items of this class on a table. Tasks verbs can be defined with a set of slots such as (VERB)(SUBJECT)(LOCATION)[(MODIFIER)]. Each verb can be labeled with a set of legal values for each of the slots. For example, we can define a task verb of the form Pick-up with slot values: Verb: Pick-up subject: one-of (clothes, dishes, shoes, garbage) location: one-of (table, dish-washer, bed, kitchen-floor} modifier: one-of {carefully, quickly, quietly}. Of course, every entry in the subject list must correspond to a class of known, identifiable, objects. Every member of the location list must correspond to a known location. The modifiers must correspond to constraints on the execution of actions. For example, 'carefully'can be translated to make all movements below a certain speed.

J.L. Crowley / Planning and Execution of Dornestic Robot Tasks

The representation of the sentence structure for "Pick-Up" is an example of an increasingly used technique for pseudo-natural language communication. The sentence structure is represented by a "schema" or frame. A schema may be created for an abstract action or for a specific verb. A set of legal values are listed for the subject of the verb, object phrases, and modifiers. These possible values may be explicit lists or pointers to other schema which represent classes of objects. A crude but simple c o m m a n d analyzer may operate by simple keyword recognition. While such analysis is easily accomplished, it may give unexpected results if the lists of objects and subjects have a non-null intersection. A more sophisticated system may be constructed in which schema are also used to represent possible sentence structures. Such schema may guide the lexical analysis of phrases. The representation of knowledge as schema (or units, or frames) is basic to most of the popular expert system tools, such as KEE, KnowledgeCraft, and ART. Such structures are also easily programmed using object-oriented programming languages, such as Flavors in C o m m o n Lisp, or Loops in Inter-Lisp. A primitive form of such facility is provided in C + + . Chore scripts are also quite naturally represented by such techniques.

5.4. An Example of a Chore Thus we are ready to specify an example chore: Chore: "Clear the table" The chore script would road something like: Constraints: Start: When commanded. Stop: unspecified. Chore Script: 1. Go to Table 2. Pickup all the Dishes from the Table 3. Go to the Dishwasher 4. Put all the Dishes in the Dishwasher 5. Start the Dishwasher. The first task in our script translates directly to a navigation task. This task cannot be planned until we know where the robot will be when the chore is to commence. The navigation task " G o to Place" is easily planned by graph search in a network of places, and may be planned and executed at the time the script begins. Task 2 of this chore is "Pickup all the Dishes

269

from the Table". This task must be represented by a set of rules which dynamically generate the necessary actions to accomplish this task. In the sections below we will use this task to illustrate the functioning of the supervisor.

5.5, The Trade-off between Actions and Rules Task 2 will start off with a rule such as (1) IF task 2 status is active. AND no dishes are known on table T H E N find-all dishes on table set status of task 2 to searching. The perceptual action, "find-all dishes on table" illustrates a problem: there are many kinds of objects which belong to the class dishes, and there may also be other objects, which are not in the class dishes, which are on the table. One approach to controlling a scene understanding is to organize the system to be "goal directed". In such an approach, a basic operation is the perceptual action "find-object". This procedure operates with a description of a class of objects to return a list of all instances of that class within a specified region of the scene. Thus "find all dishes on table" must be composed of calls to "find-object" for each class of dishes. The question is should this composition be accomplished by rules or as a composite action? First of all, one possible strategy is to simplify identify everything on the table. We will call this the "data-driven" strategy. 1. Data driven: The system attempts to identify every item on the table using an action level perception procedure. The list of items is entered into the working memory, when the items which belong to class dishes may be selected and pickedup.

We reject the data driven approach because it is inefficient. There may be numerous items on the table with which the robot is not concerned. The larger the base of known objects, the more time must be wasted trying to identify all of the things on the table. We propose instead that the recognition of objects on the table be "goal directed". However, even with a goal directed approach, there are alternatives. 2. Goal driven by class of object: A perception procedure "find-object" is called by the supervisor for each class of objects in dishes. The procedure

270

J.L. Crowley / Planningand Execution of Domestic Robot Tasks

returns its result by sending messages to the supervisor which translate to instances of working memory elements for each object. The presence of an object triggers the goal place-in-dishrack for the object. 3. Goal driven by superclass: A perception procedure Find-objects-from-superclass is written which accepts a class of object classes as its parameter. This procedure then calls "find-object" for each object class. For each detected object, a message is sent to the supervisor to add an element to working memory. From the standpoint of the supervisor, approach 3 is the easiest. In this approach, object location is completely autonomous. The supervisor simply poses the question, and the perception actions do the work. From the standpoint of the entire system, approach 2 is perhaps the more parsimonious, avoiding the requirement for a special procedure "find-objects-from-superclass". Additionally, the search for objects is more efficient, as it may be halted as soon as find-object returns a non null result for a particular class. Approach 1 is the most time consuming and least reliable. There is no guarantee that the system will not waste much time trying to identify a crumpled napkin. It is possible that the transition from approach 3 to 2 reflects a learning process in humans. When a task is new, a human must exert much conscious thought to accomplish it. He must explicitly remember and search for each kind of object that is to be picked up. As the task is repeated, this process becomes "compiled". In a sense, the human "learns" the specialized function "find-dishes", which returns all instances of objects from the class dishes. Since we can "program" our domestic robot, and since we know in advance that one of its tasks will be to find dishes on a table, we can directly program an optimized form of perceptual action for this task: "find-dishes". The perception action "find-dishes" takes as argument the geometric region in which it is to search. Using a database which tells it the list of object classes in dishes, and a description of the geometric form of each class, find-dishes calls the perception action "find-object" for each classes. On the first successful to find-object, the procedure returns the list of located objects to the supervisor and halts.

5.6. Representing a Task as a Set of Rules

Suppose it is four in the afternoon and the robot is in the den. We have just tasted a piece of aunt Martha's chocolate cake and a glass of milk on the kitchen table. We enter the den and command the robot to "clear the table". The "clear the table" script starts with the task: (l) Go to kitchen-table. " G o To Tasks" can be planned by graph search [17] through a network of places as soon as the robot knows the starting location. Assuming that the location of the kitchen table has not changed, task 1 can be planned and executed. (2) Pickup all the dishes from the kitchen-table The logic for task 2 can be expressed as rules of the form. As a first start, let us sketch some of these rules, leaving aside for the moment the problem of control, that is the problem of assuring that the desired rule has priority. In pseudo natural language, task 2 decomposes into rules such as the following: (1) IF task 2 status is active A N D no dishes are known on table T H E N find-all dishes on table set status of task 2 to searching Note the predicate " n o dishes are known". Literally, this corresponds to the existence of an working memory element of type dishes, as distinct from the presence of dishes on the table. The fact that no dishes are known, merely triggers a search for dishes. When dishes are found, the work begins. (2) IF task 2 status is searching A N D dish d is known to be on table T H E N set task 2 status to active Goals are created dynamically in response to the presence of each dish. (3) IF task 2 status is active A N D dish d is known to be on table T H E N make goal place-in-dishrack dish d status active A goal decomposes into subgoals which must be executed in a certain order. (4) IF goal place-in-dishrack dish d status is active T H E N make subgoal find space on dish-rack for dish d make subgoal grasp dish d make subgoal place dish d at space in dish-rack

J.L. Crowlev / Planning and Execution of Domestic Robot Tasks

Subgoals generally translate directly to actions. (5) IF subgoal grasp dish d T H E N grasp dish d However, if a subgoal fails, this generates a new intermediate subgoal. (6) IF subgoal grasp dish d A N D path-blocked by object B T H E N make subgoal displace object B The above is, of course, only a subset of the rules needed. This subset is just rich enough to be useful to illustrate the use of a rule based supervisor, yet simple enough not to loose our audience. A complete set of rules for clearing the table would likely have between 40 and 50 rules. The only reliable manner to construct such a program is by experimentation. For a first prototype, the physical problems of controlling a robot and a perception system can be sidestepped by using a simulation. However, to construct a system which will work in a real world situation will require experiments with a real robot and a real perception system (operating in real time!).

6. Conclusions

The technical state of the art for a domestic robot can be compared to the situation in aviation at about the time of the Wright brothers first manned flights. The Wright brothers had grasped that the key ingredients were power, lift and control. Of these, lift had been thoroughly investigated by the scientific community, power was a new technology developing in response to the demands of many applications, and control was the missing ingredient. To draw a parallel, a domestic robot needs perception, computing power, and control. A very important amount of research energy is currently dedicated to perception. As a scientific community, we are seeing solid progress. Computing power is doubling every 18 months, pushed by a number of forces. The key ingredient is control. Not necessarily in the sense of control theory models (although the play an important role) but in the sense of organizing the system in such a way that it is able to use goals to perceive and function in a quasi-structured environment. An aviation pioneer, wishing to create an aviation company in the year 1900, had to adapt and advance a number of technologies, and to in-

271

tegrate these technologies in a working aircraft. It was important to follow and encourage the evolution of lightweight power sources, but such advances were best left to the specialist in that industry. Lift technology was fundamental, for propellers and wings as well as for elevators and rudders. A number of other supporting technologies, related to materials and fabrication techniques and instrumentation were also important. However, the real advances were made in system architecture and in integration. Such advances were made by building prototypes. Let us finish by briefly summarizing the state of technology in some of the key areas for a domestic robot. 6.1. Locomotion and Manipulation

A number of technologies, necessary for a domestic robot, are sufficiently advanced as to be readily adapted. Asynchronous motor controllers are commercially available. By asynchronous control, we refer to the ability to specify a new desired position or velocity c o m m a n d which supersedes an existing command. A motor controller that "locks out" new commands before completion of the current c o m m a n d has disastrous consequences for the rest of the system architecture. Techniques for creating Cartesian arm controllers and vehicle controllers are well understood. Techniques for dynamically estimating the precision of movements (arm or vehicle) are understood. Techniques for determining the location of a mobile robot using beacons are understood, and improving. 6.2. Perception

Performing tasks in a quasi-structured environment is not feasible without perception. The minimum requirements for mobility is a dynamically maintained model of the limits of free-space, and a system for estimating position of the robot within the model. Satisfaction of both requirements has been demonstrated using a dynamically maintained local model based on ultrasonic ranging. This technology is ready for commercial exploitation. The minimum requirement for manipulation in a quasi-structured environment is a dynamically maintained model of the three dimensional objects

272

J.L. Crowley / Planning and Execution of Domestic Robot Tasks

in a work space. Such a model can be constructed from any number of sensing techniques including light stripe sensing, laser ranging, stereo and motion. Such a model is much easier to construct if composed of objects for which a geometric model exists. However, exploiting knowledge about object form does not obviate the need for perception. Three dimensional perception is required both to detect presence of known objects, and to describe the limits of free space due to unknown objects. Rapid progress is being made in techniques for building an abstract description of images as video rates and for matching such descriptions to obtain geometric information about a scene. As cost descends, these techniques will become ready for commercial application. The development of techniques for integrating 3-D perception to maintaining a geometric model of an environment is a very active area of research. Techniques are beginning to appear. Exploiting such a model to plan and execute manipulation actions will follow naturally.

6. 3. On-board Knowledge Based Systems The supervisor for a domestic robot will require an onboard knowledge based system to coordinate action and perception. The state of the art for such systems is to use "schema" or frames for representing knowledge and to use rules for reasoning. Schema based knowledge representation is easily provided by object oriented programming. This integration of schema and rules is fundamental to most of the "expert system" tools available on the market, such as KEE, Knowledge Craft, and Art. Unfortunately, these systems are written in LISP and designed to operate as dosed boxes on a lisp machine. A domestic robot will require this technology to be adopted to an open architecture operating in real time on an on-board computer.

6.4. System Integration The main effort of the Curtis-Wright aircraft company was dedicated to development of aircraft achitectures with improved performance and reliability. What we now need are experiments in prototype integrated systems. Such systems should, when possible, be composed of proven technolo-

gies. Such prototypes demonstrate lessons concerning the overall structure of such systems. They also make clear the roles of diverse technologies and make evident which technologies are crucial and which technologies are superfluous.

References [1] R.C. Bolles and P. Horaud, Configuration understanding in range data, Second 1SRR (August, 1984). [2] R.A. Brooks, Solving the find-path problem by good representations of free space, IEEE Trans. on Sys. Man and Cybernetics, SMC 13(3) (March/April 1983). [3] L. Brownstown, R. Farrell, E. Kant and N. Martin, Programming Expert Systems in OPS5 (Addison Wesley, Reading Mass., 1985). [4] J.L. Crowley, Navigation for an intelligent mobile robot. IEEE Journal on Robotics and Automation 1(1) (March 1985). [5] J.L. Crowley, Representation and maintenance of a composite surface model, IEEE Conference on Robotics and Automation, San Francisco, Cal. (April, 1986). [6] J.L. Crowley and J. Coutaz, Navigation et Modrlisation pour un robot mobile, Technique et Sciences en lnformatique (Octobre/Novembre 1986) (in French). [7] J.L Crowley, Coordination of action and perception in a surveillance robot, IEEE Expert 2(4) (Winter 1987) 32-43. [8] J.L. Crowtey, Using a composite surface model for perceptual tasks, 4rh IEEE Conference on Robotics and Automation, Raleigh, N.C. (April, 1987). [9] J.L. Crowley and F. Ramparany, Mathematical tools for representing uncertainty in perception, A A A I Workshop on Spatial Reasoning and Multi-Sensor Fusion (October, 1987). [10] O.D. Faugeras, and M. Hebert, A 3-D recognition and positioning algorithm using geometrical matching between primitive surfaces, Proc of Eighth IJCA1 (Aug. 1983). [11] W.E.L. Grimson and T. Lozano-P&ez, Model based recognition and localization from sparse range and tactile data, Int. Jour. of Robotics Research 3(3) (Fall 1984). [12] Y. Kanayama, Trajectory generation for mobile robots, Third Int. Syrup. on Robotics Research, ISRR-3, Paris (Oct. 1985). [13] R. Korf, Planning as search, Artificial Intelligence 33(1) (Sept. 1987). [14] J.E. Laird, P.S. Rosenbloom and A. Newel, Chunking in SOAR: The anatomy of a general learning mechanism, Machine Learning 1(1) (1986). [15] A. Newell, The knowledge level, Artificial Intelligence 2(2) (1981). [16] N.J. Nilsson, A mobile automoton: An application of artificial intelligence techniques, Proceedings of the First I.J.CA.I., Washington D.C. (May 1969). [17] N.J. Nilsson, Principles of Artificial Intelligence (Tioga Press, 1980). [18] R.W. Wallace et al., First results in road following, IJCAI 85, Los Angeles (August 1985).