Modeling and design of integrated sensing, planning and control in robotic manufacturing work-cells

MODELTNG AND DESIGN OF INTEGRATED SENSING, PLANNIN ...

14th World Congress ofTFAC

B-ld-02-1

Copyright iD 1999 IFAC 14th Triennial World Congress, Beiiing, P.R. China

MODELING AND DESIGN OF INTEGRATED SENSING, PLANNING AND CONTROL IN ROBOTIC MANUFACTURING WORK-CELLS 1 Mumin Song' Tzyh-Jong Tarn •• Ning Xi ***

System.~ Department, MD3135, SRL, Ford Motor Company, Dearborn, MLf8121-2053, U.S.A. Department of Systems Science and Mathematics Washington University St. Louis, MO 63130, U.S.A. *** Department of Electrical Engineering, Michigan State University, East Lansing, MI48824-1226, U.S.A

* Manufacturing

n

Abstract: The analytical and robust integration of low-level system sensing and simple control with high-level system behavior and perception is a challenging problem in the study of intelligent control. This paper presents a novel and generic technique on modeling and design to solve the problem in an intelligent robotic manufacturing system. We propose Max-Plus Algebra model combined with event-based planning and control to form an advanced mechanism to efficiently integrate task scheduling, sensing, planning and real-time execution so that task scheduling, which usually deals with discrete events, as well as action plann.ing, which usually deals with continuous events, can be treated in a unified analytical model, and the design of task synchronization becomes much easier. More importantly, this unified analytic model naturally build up continuous interaction between discrete and continuous events. This characteristic of the model allows the designed automated system to intelligently cope with unexpected events and uncertainties during well-scheduled tasks and improves the robustness and reliability of the manufacturing system. A robotic manubcturing system working on a part-sorting task is used to illustrate the proposed approach. The experimental results successfully demonstrate the advantages of the proposed approach. Copyright © 1999lFAC Keywords: Task Scheduling, Intelligent Control, Manufacturing Work-cell, Hybrid Systelll

1. INTRODUCTION

trol scheme has been considered to be extremely critical in manufacturing technology.

Driven by the stressful market competition, manufacturers are desperate to demand automated manufacturing systems which have the ability to agilely react to new products, the flexibility in system configuration, and intelligence in dealing with unexpected system faults and uncertainties. Therefore, designing a generic and intelligent con-

Traditionally, a manufacturing system involves three major layers. First, in order to have produds through the manufacturing system, an offline task scheduling procedure is usually applied to generate a sequence of logical comlllands which use to execute the processes sequentially or concurrently. Various methods have been presented for this task scheduling procedure, including operation research type of ap-

1 Research partially supported under NSF Grant IRI9706160 and DMI-9729272.

523

Copyright 1999 IF AC

ISBN: 008 0432484

MODELING AND DESIGN OF INTEGRATED SENSING, PLANNIN ...

14th World Congress oflFAC

proaches (Luh, 1997), heuristic approaches (Kim and Lee, 1995)(Hsu and Fu, 1995) (Shin and Zheng, 1991)(Sriskandrajah et al., 1986), AND/OR graph approach (Homem and Sanderson, 1991), Petri Net approach (Kanehara et al., 1993), as well as the recently developed discrete event system approach (Brandin et al., 1992). Secondly, an off-line action planning procedure can translate an executed process into a desired motion , such as desired paths or trajectories, etc. These desired information is numerical commands and input to a lower-level closed-loop controller which drives the an object. Methods have also been proposed to solve the time-based robotic action planning (Latombe, 1991)(Tarn et al., 1992) (Khatib, 1986). Lastly, a sensing system can provide the lowerlevel control system output information and close feedback loop. In most of currently used manufacturing systems, task scheduling, action planning and control are treated separately and designed uniquely for specific purposes. They are hardly reconfigured when launching new products. At the mf'.an time few continuous both-way influences among the' task scheduling level, the action planning level and the lower-level controller and the sensing system exists. In other words, there is few system feedback information which can be utilized in real-time for decision making. This results in the very poor system adaptability, so that the current manufacturing systems do not deliver promise for finishing desired goals in the sense that any disturbances in the lower-level systems can destroy well-scheduled processes and logical sequence. The reason why these shortcomings in manufacturing systems can not be resolved for a long time is that, it is hard to describe the task scheduling and action planning/control in one unified analytical model or framework, and no generic and simple method could be applied to design a mechanisms to control such a complex systems, so called hybrid systems (Brockett, 1993)(Gollu and Varaiya, 1989), which involves both discrete and continuous events. For several years, considerable effort has been made to investigate hybrid systems. A three layer hierarchical model of controller and planner was introduced (Saridis, 1987) by adding a high level monitoring layer to a basic system in order to deal with discrete decisions. Recently, several new methods have been proposed for designing a hybrid system. (Nerode and Kohn, 1992) presented Computer-Aided Control Engineering environment which supports automatic generation of automata that simultaneously comply with discrete and continuous dynamics. (Bencze and Franklin, 1995) designed a Real-timefBoolean Translator to interface between decision-making logic and manipulator controller. (McCarragher

and Asada, 1993) applied hybrid system structure to formulate transitions between constrained motions for a peg-in-hole task in a robotic manufacturing system. However, these methods are either heuristics or one-of-a-kind desigIlB. This paper uses robotic manufacturing work-cells as examples to present a novel analytical design method for the general and complex hybrid systems. First, a Max-Plus Algebra model is introduced to describe discrete event dynamics. The logical and temporal relationships between discrete events will be determined by solving the Max-Plus Algebra dynamic model. The newly designed event-bWled planning and control scheme creates the continuous interaction between higherlevel system behavior and lower-level system planning/control, such that task scheduling, sensing, planning and control are integrated. Finally, the experim.ental results from a sorting task in a robotic manufacturing work-cell clearly demonstra.te the state-of-art characteristic of the designed system.

2. A MAX-PLUS ALGEBRA MODEL Coordinating the machines and the manufacturing tasks requires complex control mechanisms, which usually make it impossible to describe the dynamic behavior of such system in terms of differential equations, as used for physical phenomena. Instead the dynamics of Burn system can he described using the two paradigms of "sync.hronization" and "concurrency." In the task scheduling , all the dynamics of the discrete event(machine operation) system had better be described as a mathematical model so that the corresponding numerical. solutions can be obtained on-line to determine the sequence of logical commands. Max-Plus Algebra gives us a supportive tool to describe the time behavior of the discrete event dynamic system. In the manufacturing system, n discrete events are assumed to happen in order to accomplish a goal. xi(k) (i = 1,2, ... , n) denotes "the time which the i-th event happens at k-th times." 'ltp(k) (p = 1,2, ... ,111.) represents that "the time which i-th controlling event happens at k-th times." ai' indicates "the time cost from j-th event happen~ ing to i-th event happening." Hence, if there is no connection between i-th and j-th event, ai;" E, otherwise Uij equals to a nonnegative real value. Following these representations, if i-th event hap· pens at k-th times is dependent on events which include controlling event 'Up happening at Ck -1)th times, it is not diffi<.;ult to write an equation by using operators mB-X = EIJ and + = ® to describe the time behavior of the event a.s the following:

=

X(k)

=A

® X(k - 1)

Et}

B ® u(h - 1), (1)

524

Copyright 1999 IFAC

ISBN: 0 08 043248 4

14th World Congress oflFAC

MODELING AND DESIGN OF INTEGRATED SENSING, PLANNIN ...

it allows the high-level scheduling to continuously track and utilize the system instant information. From systematic point of view, the high-level task scheduling and lower-level planning/control interacts through the newly designed unified motion reference.

where xdk) xz(k)

X(k) = [

:

1

. X n

:

. u17l(k)

xn(k) , and A is a n B E ~nx'71l..

[ 1 ul(k) u2(k)

,u(k) =

matrix with element aij, 4. EXAMPLE: PARTS' SORTING TASK

The system (1) has the solution (Baccelli et al., 1992) 10-1

X(k) = Ak ® X(O) ffi

E

[Ak-i- 1 ® B ®

uU)] .

j=O

Comparing the magnitudes of the elements in vector X(k) 1 an ordered sequence of critical events can be determined. Therefore, a sequence of the logical commands can be generated on-line to execute the event sequentially or concurrently for accomplishing a goal successfully in the manufacturing system.

3. EVENT-BASED PLANNING AND CONTRDL An event-based planning and control method has been successfully applied to a single robotic operation (Tarn et al., 1992). An intuitive idea of this method is to introduce a new action reference variable different from time. It is related to the measurements of system. directly. Instead of using time as the action reference, the action plan or the desired system input is parameterized by using the new reference variable. This action reference can be designed to carry the sensory information, which is needed for a planner to adjust and modify the original plan to generate the desired system input. As a result, the desired system input becomes a function of the system output and the sensory measurements. This gives a real time planning process to adjust and modify the plan based on the system output and sensory information. Achieving a goal in the manufacturing system, usually contains a sequence of operations. The action references of these operations are mostly different from each other. However, as the result of the Max-Plus Algebra model, the dependence of all processes are clearly determined. Then, the designed system can either switch the motion references for sequential processes or combine the motion references for concurrent processes. Hence, a unified action reference is created by chaining the actual motion references and the combined motion reference. Obviously, this unified action reference carries the logical information. Meanwhile, since the unified motion reference is also designed by the each individual motion reference which is directly related to the system outputs,

Consider a robotic wanufacturing work-cell which consists of a single robotic m.anipulator and a disc conveyor working on a parts sorting task, as in Figure 1. There are three different types of parts, which can be denoted as a set M = {m(k)lk = 1,2, 3}. Since parts are located at different places on the disc conveyer, the robot picking positions are different. Without loss of generality, three different picking positions are assumed and denoted as a set L1 = {11(k)lk = 1, 2,3}. The robot needs to sort the parts into three different locations , denoted as a set L2 = {12(k)lk = 1,2,3}. Then, a sequence of the robotic operations and the disc conveyor in the work-cell is described as follows: (1) The disc conveyor moves part m(k) to location ll(k). The time of the part arriving is u(k) . (2) The robot reaches the part m(k ) at location ll(k) from another location in the set L2 and picks it up. The time of the operation is a12(k), where the index represents "from location in L2 to Id k). II Xl (k) denotes the beginning tim.e of this operation. (3) The robot moves the part m(k) from location llCk) to location 12(k) and drops it. The total time ofthe operation is a~l1 Ck) where the subscript indicates "from location II Ck) to location (z(k)." xz(k) represents the beginning time of this operation. (4) Procedures (1) to (3) are repeated. Following the modeling procedure in section 2, the time behavior of both robotic operations and disc operations in this task can be represented in a Max-Plus Algebra model as the following:

X(k

+ 1) =

A(k)X(k) ffi B(k)u(k - 1), (2)

where

A(k) _ -

BCk) =

[e

e aZ1(k

aI2(k)

+ 1) ® a12(k)

]

,

[a~H C: + 1) ] .

Solving this equation on-line, knowing the time of parts feeding is necessary. In other word, the configuration of parts in the robotic task space n eeds to be detected. A centralized. da.ta sensor fusion s('.heme (Tarn et al., 1996), which can fuse the information from a vision system and a disc encoder, is applied to provide the needed parts'

525

Copyright 1999 IFAC

ISBN: 0 08 043248 4

MODELTNG AND DESIGN OF INTEGRATED SENSING, PLANNIN ...

14th World Congress ofTFAC

6. CONCLUSION

information. Then a sequence of logical commands is generated in order to execute the corresponding operations.

A new modeling and design approach for integration of task scheduling, sensing, planning and control in an intelligent robotic Illanufacturin.g systems has been proposed. A Max-Plus Algebra model can be solved recursively in order to determine a task reference. Thus a unified action reference variable can be obtained, which carries all the sensory information that is necessary for task synchronization. As a result, all operations are synchronized by the unified action reference, and the integrated system is capable of intelligently coping with uncertainties and unexpected events occurring during a well-scheduled task. Obviously, the state-of-art characteristic of this approach can achieve intelligent, robust and efficient performance for robotic systeIIls.

In event~based planning level, the arc-length the robot travels is considered a9 the new reference va.riable for the robotic planning, and the arclength the disk rotates is the new action reference variable for the disc conveyor. Since both action reference are related to the traveling distance measured in real-time from the outputs of the robotic operation and the disc conveyor's operation, the operations of the robot and the disc conveyor can be synchronized easily in the sense that, relying on the different total distance which the robot and the disk need to move, the different velocity profile can be planned in order that the robot meets the part at the same location at the same time. More importantly, when any operations are stopped by a unexpected event, the action references will stop simultaneously so that all operations can stop. After recovering the system from the unexpected event, the system will seamlessly resume without any re-planning. This unique function makes the manufacturing system capable of dealing with the unexpected events so that a local disturbance will not become a global one. The resulting system is shown in Figure 2.

7. REFERENCES

5. EXPERIMENTAL RESULTS An experimental system has been setup in the Center for Robotics and Automation at Washington University in St. Louis. It consists of two PUMA560 robot manipulators and a controllable disc conveyor. There are three bolts with different height (O.l1m, O.085m, 0.045m). Tasks assigned in this work-cell are to let robot pick up the parts in order of their height, from highest to lowest, and move each of the parts to three different specific locations. The experimental results are shown in Figure 3 and Figure 4. In Figure 3, the trajectories of the robot and the bolts in XY Z coordinates are shown. This result demonstrates that, through a unified action reference, the sensing, planning and control are well integrated. Figure 4 shows that the robot was accidentally stopped by an obstacle in the time interval [14.923sec,20.255sec]. Since the action plans for the robot and the disc conveyor can be adapted by the low-level system outputs and logical comIIlands. As a result, the disc conveyor also stops rotating. Once the obstacle is removed, the assembly process seamlessly resumes without any replaning. This expedment demonstrates that the IIlethod of the event-based planning and control is capable of intelligently coping with unexpected events occurring during a well-scheduled task.

Baccelli, Ftancois Louis, Cohen, Guy, Olsder, Geert Jan and Quadrat, Jean-Pierre, Eds.) (1992). Synchronization and Linearity. John Wiley & Sons. Bencze, W. and G. Franklin (1995). A separation principle for hybrid control system design. IEEE Control System Magazi.ne pp. 8(J-..85. Brandin, B.A., W.M. Wonham and B. Benhabib (1992). Manufacturing cell supervisory control-a time discrete even system approach. In: IEEE/RSJ International Con. ference on Intelligent Robotics and Systems. Nice, France. Brockett, R. W. (1993). Hybrid models for motion control systems. Essays on Control: Perspectives in the Theory and its Applications pp. 29-53. Gollu, Aleks and Pravin Varaiya (1989). Hybrid dynamical system. In: The 28th Conference on Deci.sion and Control. Tampa, Florida. pp. 2708-2712. Homem, L. S. and A. C. Sanderson (1991). Representation of mechanical assembly sequence. IEEE Tra'n!lactions on Robotics and Automation 7, 211-227. Hsu, H. and L. H. Fu (1995). Fully automated robotic assembly cell: Scheduling and simulation. In: IEEE International Conference on Robotics and Automation. Nagoya, Japan. pp. 208-213. Kanehara, T., T. Suzuki, A. Inaba and S. Okuma (1993). On algebraic and graph structural prop=ties of assembly petri net: Search by linear programming. In: IEEE/RSJ International Conference on Intelligent Robotics and Systems. Atlanta, Georgia. pp. 2286-2293. Khatib, O. (1986). Real-time obstacle avoidance for manipulators and mobile robots. Interna· tional Journal of Robotics Research 5,90-98.

526

Copyright 1999 IF AC

ISBN: 008 0432484

MODELTNG AND DESIGN OF INTEGRATED SENSING, PLANNIN ...

14th World Congress ofTFAC

Kim, G. H. and C. S. G. Lee (1995). Genetic reinforcement learning approach to the machine scheduling problem. In: IEEE International Conference on Robotics and Automation. Nagoya, Japan. pp. 196-201. Latombe, J. C. (1991). Robot Motion Planning. Kluwer Aca.demic Publisher. Luh, P. B. (1997). Scheduling of flexible manufacturing systems. Lecture Notes in Control and Infol'maion Science: Control Problems in robotics and Automation pp. 227-243. McCarragher, B. and H. Asada (1993). A discrete event approach to the control of robotic assembly tasks. In: IEEE International Conference on Robotics and Automation. Atlanta, GA. pp. 331-336. Nerode, A. and W. Kahn (1992). An autonomous systems control theory: An overview. In: IEEE International Symposium on Computer-Aided Control System Design. Napa , CA. Saridis, G. N. (1987). Knowledge implementation: Structure of intelligent control system. In: IEEE International Symposium on Intelligent Control. Shin, K. S. and Q. Zheng (1991). Scheduling job operations in an automatic line. IEEE Transactions on Robotics and Automation. Sriskandrajah, C., Ladet, P. and Germain, R., Eds.) (1986). Scheduling Methods for Manufacturing System. Elsevier Science Publisher. Tarn, T. J., A. K. Bejczy and N. Xi (1992). Motion planning in phase space for intelligent robot arm control. In: IEEE/RSJ International Conference on Intelligent Robotics and Systems. Raleigh, NC. pp. 1507-1514. Tarn, T. J., M. Song, N. Xi and B. K. Ghosh (1996). Multi-sensor fusion scheme for calibration-free stereo vision in a. manufacturing workcell. In: IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent SysternB. Washington D.e., U.S.A .. pp. 41&-423.

527

Copyright 1999 IF AC

ISBN: 008 0432484

MODELTNG AND DESIGN OF INTEGRATED SENSING, PLANNIN",

14th World Congress ofTFAC

Fig. 1. Single Arm Robotic Work-cell with A Disc Conveyor

J\. n l":~j \J \~

On,-----~----~----~----~----~----~----,

1-

oA

/~

E

>=' -O.2~: -OA

' t \

...o.tl.

,-,-t'

"

.

i

' I ...

I

":~--~~

, I ',I

,

...... ----.

..

/

______ '

'

"

",'

..#

solid line "."; robo1 dct1ed line

",~:

1:)0lt1

dashed line "--": bol12

:.-,-'

dashdot linn "-" : boltS -0.80,--------,.':-0-----:2C::O-----,3:':D-----4.,.,D,----~50,----~"D::-----,J70

s*

I(seo,

.

----------~--------~-

... ---,

Fig. 4. Parts sorting operation with disturbance

Fig. 2. Unified Manufacturing Work-cell

Fig. 3. Parts sorting operation without djsturbance

528

Copyright 1999 IF AC

ISBN: 008 0432484