Integrating Robot Task Planning into Off-Line Programming Systems

Copyright © IFAC Infonnation Control in Manufacturing, Nancy - Metz, France, 1998

INTEGRA TING ROBOT TASK PLANNING INTO OFF -LINE PROGRAMMING SYSTEMS

Hongyan Sun* and Uri I. Kroszynski**

*Department of Information Technology, Building 344 **Depanment of Control and Engineering Design, Building 358 Technical University of Denmark, DK-2800 Lyngby, Denmark

Abstract: The addition of robot task planning in off-line programming systems aims at improving the capability of current state-of-the-art commercially available off-line programming systems, by integrating modelling, task planning, programming and simulation together under one platform. This article proposes a system architecture for integrated robot task planning. It identifies and describes the components considered necessary for implementation. The focus is on the functionality of these elements as well as on the information flow. A pilot implementation of such an integrated system architecture for a robot assembly task is discussed . Copyright© 1998lFAC

Keywords: Robotics, Off-line programming, Robot programming, Simulation, Path planning, Architectures, Models.

1. INTRODUCTION

actual execution. In most such systems, planning and programming are separated, so users are required to explicitly code long sequences of simple instructions in order to achieve higher-level task goals. This proves tedious and time consuming when the task is complicated and particularly difficult when fine motions are introduced (Lozano-Perez, et aI. , 1984). In these cases, automated robot task planning becomes necessary. It can be loosely stated as follows: Deciding what actions and motions for a robot to perform in order to achieve goaL arrangements of physical objects (Latombe, 1993). Those actions and motions are plans which can be transferred into robot programs through robot programming languages.

Industrial robots are employed in many production and manufacturing processes in order to increase flexibility and productivity (McKerrow, 1991). However, the flexibility of a robot is mainly determined by its capability of being easily programmed (Trostrnann, et aI., 1991). Since each and every drive governing the movement of the robot manipulator articulations necessitates proper control signals to produce a co-ordinated movement, one of the ultimate goals in robot programming is to create an "automated" robot programming system. Such a programming system will also accept high-level descriptions of tasks and will produce robot machine code automatically with minimal human intervention. The input descriptions will specify what the user wants done rather than how to do it.

In this paper a system architecture for integrating modelling, robot task planning, programming and simulation together is presented. The motivation behind this integration is that often a sequence of actions from task level planning may turn out to be un feasible when developed at the motion level since task level planning is task-oriented. In this situation the knowledge that the motion cannot be realised should enrich a "knowledge & data base" so that the

Off-line programming systems have been developed to address some important issues of "automated" robot programming (Trostmann and Nielsen, 1992), e.g. CAD-based modelling, high-level robot language interpreters or compilers and simulation tools to validate robot movements before down-loading for 343

2.2 World modelling system

task level planner can produce another plan. The safest way to test a plan and obtain the resulting information is through off-line programming and simulation rather than through on-line tests (Sun, 1995). An implementation of such an integrated system for robot assembly tasks is also explained. The prototype is built on top of the commercial offline programming and simulation system ROBCAD, marketed world-wide by the company Technomatix.

The World modelling system provides tools for constructing the knowledge & data base which contains world models, e.g., task model, robot manipulator model, robot controller model (Raibert and Craig, 1981), etc., with knowledge obtained from experts and literature, as well as dedicated software application packages. The modelling system should also provide facilities for importing models from other systems (Kroszynski, et aI., 1993).

In the next section, the integrated system architecture is presented. Section 3 illustrates an implementation of the proposed approach for a robot assembly task. and describes an example with some results. Section 4 presents some concluding remarks.

2.3 Task planning system

The Task planning system is composed of two levels of planners. The task level planner decomposes user specified task descriptions into a sequence of robot primitive actions. The motion level planner decomposes the primitive actions into elementary robot motions. Each primitive action will be interpreted and transferred to the corresponding subplanner, i.e. the gross motion planner, the fine motion planner, or the grasp motion planner.

2. THE INTEGRATED ROBOT TASK PLANNING AND PROGRAMMING SYSTEM The proposed integrated system architecture is sketched in Figure 1. It consists of five functional components: User interface, World modelling system, Task planning system, Programming & simulation system, and Knowledge & Data base. The dotted lines in the figure represent user and system feedback information.

+-----+ L------r~----~

•

•

Knowledge

&

Data base

• Task planning system

.< IProgramming

The function of the gross motion planner is to find a collision free path (optimally the shortest), so that the robot or the end-effector moves along that path avoiding obstacles in the work scene. This planner should include the algorithms needed to provide such a path with little user intervention. The fine motion planner in this paper deals mainly with compliant motion synthesis (Mason, 1981; Lozano-Perez, et aI., 1984). Its function is to decompose each primitive action into a sequence of compliant motion commands (Sun, 1995). The grasp planner mainly deals with finding the optimal grasp with respect to some criteria, e.g. choose correct gripper, decide how to approach the object, and how to grasp it (Volz, et aI., 1985) .

~I Simulation

2.4 Programming and simulation

Programming & simulation system

The Programming and simulation system converts the robot motion commands into a robot program through the programming module, simulates the program execution through the simulation modules, and displays a realistic animation on the computer screen through the graphical visualisation modules.

Fig. 1. The proposed integrated system architecture.

2.1 User interface

The User interface is an environment where users can communicate with the World modelling system, the Task planning system and the Programming & simulation system. Modern computer programming techniques provide various flexible tools for user interaction design, e.g. menus, windows, icons, and graphics. Apart from these interaction tools, the important issue is what kind of information the user should be prompted to enter in order to communicate with the computer system, so that users can clearly express their intentions, and the computer system can correctly interpret these intentions.

The simulation results enrich the Knowledge & Data base, which will be used by the Task planning system to evaluate the plans. The programming module contains functions for translating robot commands into a robot program. The robot program is written in a programming language supported by the off-line programming system. The simulation modules include a manipulator simulation module, a controller simulation module and a task simulation module corresponding to the manipulator, controller and task models respectively.

344

2.5 Knowledge & Data base

After this interaction, the user has formally given the system a task description like:

The Knowledge & Data base stores (and provides) all information needed for the planning, programming and simulation. The information consists of the task model, the robot manipulator model, the controller model and the general rules. The planning system relies on this information to make decisions and calculations.

insert (objectl (name: objl_name, location:objl loc)} into (object2 (name: obj2_name, location: obj2joc)} until (tenninal condition).

Details on the formal task specification are found in (Sun, 1995). Similarly, menus for assisting planning and simulation are in the task planning and the task simulation groups.

The general rules provide the strategy to be used by the planners to make decisions, e.g. reasoning about spatial relationships between features . Information about the current world state, e.g. position of the manipulator, elapsed time, etc., is readily obtained from this database.

3.2 The Task Planner The task planner TaskPlanner consists of two levels: the task level planner and the motion level planner. The task level planner TaskLevelPlanner decomposes the task description specified by the user into primitive actions as illustrated in Figure 3.

3. THE IMPLEMENTATION OF A ROBOT ASSEMBLY TASK PLANNING SYSTEM A pilot system for robot assembly task planning was implemented using the functional components structure of Figure 1. The system is built on top of the off-line robot programming and simulation system ROB CAD, which is also used for world modelling purposes. The task selected for illustration is the insertion of a peg into a hole.

TaskLevelPlanner : : I I I

I

I

I

I

I I

I I

I

I

I I

I I

I

I

I

I

I

I

Fig. 3. Main modules in TaskLevelPlanner The interpreter queries the knowledge & data base in order to obtain the sequence of primitive actions that match task description keywords. ConstructActList constructs the primitive action lists. The parameters of the lists are calculated by CalculateParam.

-

x: y:

For the "insertion" task, for instance, the sequence of primitive actions is: { move (objl_app_loc_name) grasp (objCname) move (obj2_app_loc_name) insert (objl_name, obj2_name, tenninaCcondition) ungrasp (objCname) depart (home_loc) }

Force oondition

enninAle oonditions

Foroes:

-

Fx: Fy: Fz:

z:

Time:

tx:

Rx : Ry :

DiSWlCe

ty: tz:

Rz :

Defauh aa:ep<

Interpreter

Action Commands

The task description group provides user high-level task description buttons (i.e. commands) mainly for assembly tasks. For example, the peg-into-hole assembly task corresponds to the "insertion" button of the task description menu. Pressing the "insertion" button opens a series of pop-up input forms, as illustrated in Figure 2, awaiting for the user to input the parameters relevant to the task. obj UocatiOfl:

:

L ______________________________ 1

The Userinterface package consists of several modules providing hierarchical menu-driven user interfaces, which are categorised into three group: task description, task planning and task simulation.

Object Ioco
: I I I

3.i The User interface Package

objUwne: obj2_nam'"

Task Descriptions

i

1-------- ----------------------,

where, move, grasp, insert, ungrasp and depart are primitive action keywords stored in the knowledge & data base. Following each keyword are parameters. These parameters are obtained from the calculation or directly from the knowledge & data base.

I ..,."

Fig. 2. Task description menu Object names and locations can be input either by typing names and values or by graphically pointing at the objects in the workcell. The terminate conditions define the state of task completion. The default condition for the peg-into-hole task in the pilot implementation is a combination of force and distance travelled.

The motion level planner, MotionLevelPlanner, decomposes each primitive action in the sequence created by the task level planner into a series of robot motion commands. The main modules considered are illustrated in Figure 4.

345

gelose (gripper_name. parameter] , parameter2); move (peg_app_Ioc_name. [motion_type. speed]); move along (path_name); move (hole_app_locname. [motion_type. speed]); bmove (bias. bcm_dir. bterm_cond); cmove (cm_dir. term_cant!); gopen (gripper_name. parameter] , parameter2); move (home_lac); end;

Action Commands

r-------------- ----------------------.

:

~--~--~

I I

: I

I I

I I

: , - -_ _....L-_ _ _ _---,

I

: L -__--.-____- ' I I

I I

I I I

I

I I I I

I

I I I I I 1_____ _ - - - - - - - - -

3.3 Task Simulation

I I I

I

TaskSimulation is the package that corresponds to the Programming and simulation system of Figure 1. Its

----------------------~

structure is illustrated in Figure 5. Fig. 4. Modules in motion level planner

,---------------

The GrossPlannner should produces a collision free path from the current location to the target location of the primitive action without user intervention. At present the pilot system only provides a procedure to decompose the move action into linear move motion commands because the workcell considered is simple enough to assume that no obstacles block the robot motion. More complex scenes require the inclusion of proper strategy algorithms. Moreover, rather than using a separate grasp planner, calculation for the grasping location is also included here.

•

• •

•

----------------------,

~--1---~

1 1 1 1 1

I

1 I

1 1

:

1

TaskSimulation :

~---L..----_,

1 1 : L-.__--,-____...J 1 I

i'-_L_-_-_-_I __-_-_-_-_-_-_-_-'______________________ _ Fig. 5. TaskSimulation modules

The fine motion planner, FinePlanner, consists of several modules to accomplish the following goals. •

1 1 1

The Interpreter reads the task plan and interprets each command breaking it down into executable instructions for the simulation. Compliant motion commands are passed to the CompliantMotion submodule for further calculation.

Add to the task models the spatial relationships between mating features of the assembly parts. Determine the task frame (in the peg-into-hole assembly task example, the task frame is attached to the tip of the peg). Create a terminal condition data structure. Determine which directions are force controlled and which directions are position controlled. FinePlanner generates the compliant motion command: cmove (cm_dir. term_cond) ; where cm_dir and term_cond are parameter pointers to the control direction data structure and to the terminal condition data structure, respectively. Use a biased search strategy (Sun, 1995) to find the exact location of the target hole and issue the motion command: bmove (bias. bcm_dir. bterm_cond); Here, bcm_dir and bterm_cond are parameter pointers to the biased search direction and to the terminal condition respectively. The parameter bias is a measure of the bias.

The GrossMotion module in the pilot system is a built-in part of the ROBCAD simulation system. The CompliantMotion module accepts compliant motion commands and produces a series of executable motion instructions for the simulation. It has three main functional sub-modules: •

•

As a result of the motion level planner, the sequence of primitive actions for the peg-into-hole task is further decomposed into the following sequence of robot motion commands:

•

begin init ( ); move (peg_app_Ioc_name. [motion_type. speed]) ; gOpen (gripper_name. parameterl, parameter2) ; move (peg....grasp_Iocname [motion_type. speed]);

SimuSensor is a sub-module for the force/torque sensor simulation. It calculates contact forces developed between the peg and the hole during the assembly process based upon a simple "geometric penetration" model (Shimano and Roth, 1975; Sun, 1995). The calculation is carried out at the task frame. ForceTransform converts the values calculated in the task frame to corresponding values in the sensor frame (Sun, 1995). CompliantControl is an active compliant motion control simulation sub-module. The control strategy in this module is based on the spring nature of compliance (Whitney, 1982), as illustrated in Figure 6.

A virtual spring system with stiffness K is assumed to exist on the wrist of the manipulator arm as illustrated

346

S_R (PEG, HOLE) = { Aligned (cp, ch) 11 {trans (z), rot (z)},

in Figure 6(a). During insertion along the -z direction of the task frame , a contact force F causes a deformation L1X on the spring system as sketched in Figure 6(b). At the same time the sensor feeds back the force information to the robot control system. The control system produces a robot motion to neutralise this force and thus to release the deformation of the spring system, as seen in Figure 6(c).

S_R denotes the spatial relationships. trans denotes the translation along the z axis of reference frame, trans (x,y) the translation on the plane relative to the reference frame and rot (z) rotation around the z axis .

-,IlI'

Rules for the planning system are expressed as production rules (Frost, 1986), which are classified into four groups: rules for task level planning. for motion level planning. for simulation and for common use. For example, one rule for the motion level planning is stated as:

(a)

Against (jp,jh) 11 {trans (x , y), rot (z) I};

(z) the x-y the

(c )

(b )

If the primitive action command belongs to the action command set and it has the keyword "move" then pass it to GrossPlanner

Fig. 6. Compliant motion control strategy The stiffness K is a six by six diagonal matrix. L1X is a six element vector describing the translational and angular deformations along and around each of the three Cartesian co-ordinate axes respectively. F is also a six element vector indicating three force- and three torque-components along and around the Cartesian co-ordinate axes respectively.

3.5. An actual example The Workcell. The workcell (world models) for the peg-into-hole insertion task consists of a robot manipulator model (6 DOF irb2000 robot) with a motion control model embedded in it, a tool model (two-jaw gripper), a wrist force/torque sensor model , a work table model, and two assembly part models (a peg and an object containing a cylindrical hole) .

The animated simulation results can be displayed on the graphical screen by means of the ROB CAD animation module. Moreover, the results can be presented in the form of graphs showing contact forces developed during the task process, and paths of the robot tool centre. Graph plots were developed as add-ons for the prototype system.

Task description. The task objects, peg and hole, are sketched in Figure 8. Figure 8(a) illustrates the geometry of the peg (solid cylinder with radius rand height hi) and the hole object (ml *m2*h2 box with a through hole of radius R) in two different views. Figure 8(b) illustrates the initial state of the peg and the hole object after the biased search in the x-z plane and illustrates the error variables.

3.4 Knowledge & Data base The workcell model is stored database and administrated by Additional data which are used to coded on top of the system. Such illustrated in Figure 7.

in the ROB CAD ROB CAD itself. model the task are a task model is as

--

Feature

rnl

(p' r

Task descriptio n: insert (peg (nam: : peg. locati o n: P 1) into (hole (nam: : hole. locatio n: If)) until (fo= (Fz=-Icf). distanc< (I=h2))

I

I

. .. 1

It

g;;fJ1H%;1l1HiJXi H~B!

(a)

(b)

T. SR. MF . ..

T: tolerance 1 SR: spatial relationshir 1 MF: mating fealUrt:

I

Fig. 8 . Task descriptions I 1

~I¥h'J!:~_llt*L-.J

h I is taken large enough to allow for gripping and full insertion, and R-r is half the tolerance. Frames {P} and (H) locate the peg and the hole. The task frame {T} in this case is identical to {P}.

I Taskmodcl

Fig. 7. An illustration of a task model

Ci and Si are initial position and orientation errors of

MF specifies which features are mating upon assembly (Nnaji, 1993). SR specifies what kind of relations these features carry, e.g. for the peg-intohole insertion task, SR is expressed as follows :

{T} relati ve to {H). Here Si is the angle between the z axes of {T} and {H}, and ci is distance from the origin of {T} to the z axis of {H}, when viewed on a 347

x-z plane projection. The friction coefficient between the peg and the hole is assumed constant.

ACKNOWLEDGEMENTS The authors wish to thank the Department of Control and Engineering Design and the Department of Information Technology at the Technical University of Denmark, as well as the Daloon Foundation, for their support. The authors are grateful to Professor Erik Trostrnann who inspired this article and to Dr. A. P. Ravn who contributed valuable suggestions.

Fz is the force along the z direction of IT} and kf is its ~omi.nal value, I is the distance travelled along the z direction of IT} and h2 is its nominal value. This task description is entered to the task planning system through interaction via the user interface.

Results. The motion plan computed by the system for this particular example is formally similar to the one described in section 3. The plan contains 22 lines of motion commands, while the input to the planner contains only 3 lines of task descriptions created through the user interface. The plan is translated to ROBCAD's TDL robot programming language and tested through the ROB CAD simulation module for several groups of different tolerances and initial position errors. The results show the feasible robot motions in the sense of collision-free gross motions and compliant fine motions which are within given constraints during the insertion process. The coding was done using C and the ROSE facilities of ROBCAD version 3.1.

REFERENCES Frost, R.A (1986). Introduction to Knowledge Base Systems. William Collins Sons & Co., London. Kroszynski, U.I., T. S0rensen and E. Trostmann (1993). Issues in modelling techniques for the STEP based exchange of robotics data. In: Interfaces in Industrial Systems for Production and Engineering (Rix and Schlechtendahl (Ed», IFIP B·10, 101-113. North-Holland, Amsterdam. Latombe, le. (1993). Robot Motion Planning. Kluwer Academic Publishers, Norwell, USA Lozano-Perez, T., M.T. Mason and R:H. Taylor (1984). Automatic Synthesis of Fine-Motion Strategies for Robotics. International Journal of Robotics Research, 3, 65-95 . Mason, M.T. (1981). Compliance and Force Control for Computer Manipulators. IEEE Transactions on Systems. Man and Cybernetics, 11/6, 418-432. McKerrow, P.l . (1991). Introduction to Robotics Addison-Wesley Publishing Company, Sydney. ' Nnaji, B.O. (1993). Theory of Automatic Robot Assembly and Programming, Chapman & Hall, London. Raibert, M.H. and 1.1. Craig (1981). Hybrid Position! Force control Manipulators, ASME J. of Dynamic Systems. measurement and Control, 102,126-133. Shimano, B. and B. Roth (1975). On Force Sensing Information and its Use in Controlling Manipulators, Proceedings. ninth symposium on Industrial Robots, Washington, D.e., pp.119-126. Sun, H. (1995). An Architecture for Robot Task Planning. Ph.D. thesis, Control Engineering Institute, Technical University of Denmark. Trostmann, E., F. Conrad., T. Clausen, S. Trostmann and L.F. Nielsen (1991). Integration Using the Neutral Interfaces; IRDATAlICR in Robot Programming Does Work!. In: Computer Applications in Production and Engineering , Proc. IFIP Conf. CAPE'91 (G. Doumeingts, et ai., (Ed», 571-578. North-Holland, Amsterdam. Trostmann, S. and L.F. Nielsen (1992). Model Driven Simulation of Robot Systems. Models and Algorithms. Ph.D. thesis, Control Engineering Institute, Technical University of Denmark. Volz, R.A , 1.D. Wolten and Ae. Woo (1985). Automatic Generation of Gripping Positions. IEEE Transactions on Systems. Man and Cybernetics, 15, 204-213. Whitney, D.E. (1982). Quasi-Static Assembly of Compliantly Supported Rigid Parts. J. Dynamic Systems. Measurement and Control, . 104,65-77.

4. CONCLUSION An architecture which integrates modelling, robot task planning, off-line programming and simulation under one platform is described. The implementation of such an architecture for a robot assembly task planning in the ROBCAD system demonstrates the functionality and feasibility of the integrated approach. The integration improves the capabilities of a currently available commercial robot off-line programming system in the following aspects:

•

•

•

The system requires less user explicit definition, since only high-level task descriptions are needed as input, as shown in section 2.1, by means of the high-level task specification user interface. The architecture allows to integrate feature based task models, improved manipulator models, compliant motion simulation models, etc. It is also possible for the planner to re-plan according to the feedback information from the lower level components of the architecture. Fine motion (i.e. compliant motion) planning and simulation are considered of essence in order to produce more reliable programs for down-load to real robots. This is beyond the capabilities of most current off-line programming systems.

The rob~t task planner in the pilot implementation does not mclude many of the capabilities envisaged in the proposed architecture. It is based on an ideal workcell plus an initial error estimation between the geometric position of the peg and hole. More elaborate techniques are still required to estimate and calibrate the world models.

348