- Email: [email protected]
The development of a rapid-prototyping technique for mechatronic-augmented heavy plant
ELSEVIER
Automation in Construction 5 (1997) 365-378
The development of a rapid-prototyping technique for mechatronic-augmented heavy plant ’ G. Melling a, D.A. Bradley b, H. McKee ‘, M.B. Widden ’ a Consultant, b School
St. Annes, UK
ofElectronic Engineering and Computer Science, Uniuersity of Wales, Bangor, UK ’ Engineering Department,
Lancaster Uniuersity
Lancaster,
UK
Abstract Telechiric, semi-autonomous and autonomous heavy plant is finding an increasing role in applications such as construction, sub-sea work and decommissioning. There is a need for improved operator interfaces for such plant, and hence for rapid-prototyping tools which link the development of the operator interface with control and operational strategies and with machine geometries. The paper sets out a strategy by which different operator interfaces can be readily evaluated while at the same time generating the requisite information structure for the control of real items of plant. The proposed system is based on the use of interconnected PCs, one to simulate the operator interface and another to provide a kinematic representation of the machine using an appropriate “desk-top reality” environment. This system offers a safe, practical, rapid and cost-effective means of assessing proposed operator interfaces, as well as facilitating the development of machine kinematic structures and the associated operational and control strategies. Keywords:
Mechatronics; Robotics; Prototyping; Design; Human-machine interfaces
1. Introduction In recent years there has been increasing interest in the development and deployment of a range of telechiric, semi-autonomous and autonomous heavy plant in industries such as construction, mining and forestry, as well as for applications such as sub-sea work and both chemical and nuclear decommissioning [l-3]. The size and complexity of such plant make its development an expensive process, and create a range of new problems for the operator in
I
Discussion is open until August 1997 (please submit your discussion paper to the Editor on Constuction Technologies and Engineering, M. J. Skibniewski).
0926-5805/97/$17.00 Copyright PII SO926-5805(96)00161-6
achieving effective control. The difficulty experienced by an operator in attempting to control multiple degrees of freedom also acts as an inhibition on the development of machines with unusual and novel geometries. Where such machines have been produced, as for instance in the case of a recently introduced excavator which incorporates an additional degree of freedom at its “shoulder” (Fig. 1) to enable close operation alongside walls and other obstacles in a manner which is not possible using conventional slew control, they cannot be operated at full effectiveness using an approach based on the direct control of individual joints by the operator. In practice, with this system the operator moves the bucket into position using the additional degrees of
0 1997 Elsevier Science B.V. All rights reserved.
366
G. Melling et al. /Automation Dipper
Boom
(a) Conventional machine geometry with 3 degree-of-freedom an.
Added degrees-of-freedom
(b) Modified machine geometry with two added degrees of freedom in arm.
4;
degrees-of-freedom
(c) Plan view of machine with added degrees-of-freedom showing additional capability.
Fig. 1. Excavator
in Construction 5 (1997) 365-378
tion effectively precludes physical prototyping on a large scale, and so prevents machine designers from properly exploring the effects of changes in kinematic geometry on performance. This inability to prototype on a significant scale also makes it diffcult to evaluate operator interface strategies and to investigate the matching of the performance of the interface to the application. It is apparent therefore that there is a need for the development and provision of “rapid-prototyping” and visualisation tools which link the development of the operator interface with control and operational strategies as well as with machine geometries, and which enable different operational strategies to be evaluated rapidly within defined task environments [5-81. Such rapid-prototyping systems will also be able to provide support for software development, as well as for the investigation of new and novel control strategies. The paper describes an approach to the development of a rapid-prototyping system for mechatronicaugmented heavy plant which uses PC-based software, including a desk-top reality environment, to model machine behaviour and to explore and evaluate different operator interfaces and interface strategies.
2. System requirements
with added degrees-of-freedom.
freedom and then “locks off’ certain of the degrees of freedom before proceeding. The introduction of a computer-based controller able to encompass extra degrees of freedom would raise the productivity of such machines, as well as increasing the scope for further development. For instance, previous work has indicated that there may be significant operational benefits to be gained from the use of a multi-degree-of-freedom joystick to control excavator operation, particularly when combined with an improved operator interface providing additional information on bucket position and motion through the ground [4]. With the exception of certain special-purpose machines for the nuclear and other industries, the cost of developing machines of the type under considera-
Traditionally, the operation of items of heavy plant such as those used in the construction industry has relied on the ability of the operator both to position the plant at a given location and, once in position, to perform manipulative and other tasks. This is usually achieved by regulating the flow of hydraulic fluid to the machine actuators using either manual, directly connected joysticks or electronically connected joysticks as in Fig. 2. For a mechatronicaugmented item of plant such as the autonomous, robotic excavator (Lancaster University Computerised Intelligent Excavator - LUCIE) developed by the Intelligent Machines and Systems Group in the Engineering Department at Lancaster University [9,10] the servo control of the arm is taken over by the control processor as in Fig. 3. Such a system can operate either autonomously under program control or in response to operator inputs. In the latter case
G. Melling et al./Automarion in Construction 5 (1997) 365-378 Engine Contmls
(a) Direct contml of arm wing mechanically connected joysticks.
rn -,
$q-iq Pump
l
(b) Direct amtml of arm using dedmnic pmporlional joysticks.
Fig. 2. Excavator
with manual and electronic joysticks.
the operator is freed from the need to control the individual joints, and can concentrate on controlling the motion of the bucket or other end-effector. As a result of the ,work on the LUCIE system and other investigations, a number of functional requirements have been identified for full-scale systems which must be reproduced by any rapid-prototyping system. These are: . The ability to provide a range of controlled motions within an #operator-defined envelope: for instance, the provision of true straight-line motion of the bucket or other end-effector when mounted
361
on the end of an arm that includes revolute joints, in order to facilitate activities such as the digging of flat-bottomed trenches, both horizontal and inclined. The ability to define the operating envelope and limits on operation for the machine and the orientation of the end-effector within that envelope. Once movement outside preset limits has been prohibited, the operator is able to position the end-effector rapidly in a “carefree” manner, without needing to worry about the possibility of colliding with obstacles. The provision of operator feedback on the current task status. In the case of an excavator this could take the form of an indication of swept volume, and hence of bucket fill, when digging blind. The evaluation of machines with different kinematic geometries within a given task environment and the investigation of different operator strategies for such machines. Support for the development of operating software both for the operator interface and for the control of the machine. This should include both advanced, autonomous machines and developments such as software-defined “soft stops” for conventional machines to control the rates of deceleration when approaching the operational limits of the machine. In this context, it is worth noting that there were no perceived benefits to the operator in the control of acceleration. Indeed, the introduction of acceleration control tended to give the impression of a “sticky” actuator for which operators compensated by increasing the initial demand setting until the actuator apparently, and suddenly, became “unstuck”. In order to provide a high degree of flexibility, it was determined that the rapid-prototyping facility should be based around the following major sub-systems: A controlling sub-system dealing with the operator inputs, interface, displays, machine kinematics and control algorithms. - A controlled sub-system capable of emulating the plant and its operating environment at an appropriate degree of resolution, in order to provide feedback on the effectiveness of input devices, operator interfaces and control strategies and to evaluate the effect of changes in plant kinematic geometry on operation. l
368
G. Melling et al. /Automation
in Construction 5 (1997) 365-378
Electronic Joystick (Multi-degree-of-freedom)
;
CONTROLLER
Display
:m
I i
._._._._ ._.. ?_._._ ._._._._ ._..
I
:_
Bases
/
:
.j ‘1
I
i !
li, i
I I I
! I
i
I
MECHANICAL SYSTEM
i
j LEE 1.._....._........_..... ._._._.__....._. _._._._._ ._._._.__._._._._._._I _._..._._._._._._._._._._._ ..._.._..._._.__.__._._.___.____._._._.__._._., .;
Fig. 3. Mechatronic
approach
to system operation with the incorporation
For similar reasons, it was further decided that the message passing between the controlling sub-system and the controlled sub-system should be of the same form and format as that required by a real machine. This latter feature is determined by the requirement to ultimately replace the emulation with the real machine following the development phase. For the purposes of the initial work, the target machine has been the LUCIE system already in place in the laboratories. The target mode of application was operator-controlled excavation using either X-Y or
of the control processor.
conventional - direct control of joint actuators modes of operation, both with and without soft stops.
3. System design Following on from the definition of the system requirements it was apparent that the benefits of a rapid-prototyping approach will only be realisable, and the methodology will only be acceptable, if the
G. Melling et al. /Automation
computer-based protsotyping facility provides the following features: It is easy to use. It is easy and rapid to develop. It provides the required levels of functionality. It models the real world in a realistic way. It is of relatively low cost when compared with the cost of producing a physical prototype. The plant emulation and control environment is flexible, rapid and realistic. The system has tlhe ability both to emulate and to drive target systems in real time. The system is portable across a range of platforms and environments. These requirements forced a decision that the operator interface sllmulator and the plant emulator should be implemented on separate computers with appropriate interconnections and interfacing. The resulting systems are referred to as LOIS (Lancaster Operator Interface Simulator: Software Development and Operator Interface Design) and LEE (Lancaster Equipment Emulator: System Emulation and World Modelling). Fig. 3 shows the partitioning of functions between LOIS and LEE that would take place in a practical system (not all of the features shown in the figure have been realised at the time of writing), whilst Fig. 4 shows the distribution of software functionality between them. The LOIS/LEE configuration introduces a significant degree of flexibility into the prototyping process in that LOIS can be used to drive either LEE or
Knowledge Bases
+
Control Model
Y
Kinematic
Model
220”
+;
i
I
!
I
Pi Lo’sICF, LEE1 ,,,.I
yi3fNiming’
Interface
the real plant. Similarly, LEE can be driven either by LOIS or by the plant microcontroller system. Additionally, the use of two separate systems enables the optimum choice of hardware and software to be made and allows for the easy extension of the system to include other features, perhaps based on additional processors. Care must be taken to ensure the proper synchronisation of data between the two systems, as any failure in this area would have adverse effects on the operation of both systems. There is a commonality of certain of the data between the systems, for instance motion limits and rates. Other data such as the joystick input is required only by LOIS, whilst LEE is concerned with the sensory information. As the ultimate aim is to link together disparate systems, this requires that consideration be given throughout to the mechanisms of data management and data transfer. A typical design, build and test cycle using LOIS and LEE follows the stages shown in Fig. 5. Initially, the overall system, including the task environment, is evaluated using the LOIS/LEE rapid prototyping system configured as in Fig. 5(a) to assess factors such as the effects of different control strategies and machine kinematic geometries on operation, the configuration of the operator interfaces and the implications for software and associated hardware. Once the design and operational procedures have been developed using LOIS/LEE, these are then confirmed and refined by using LOIS to drive the
fi
Task Planning
Operator
369
in Construction 5 (1997) 365-378
7 J
Fig. 4. Distribution
L
Model
j senson /
of software functions between LOIS and LEE.
I
I j
G. Melling et d/Automation
370
in Construction 5 (1997) 365-378
BQBlQ@4
structured around and deploys a number of hardware and software sub-systems as follows.
(a) Development phase; LOIS drives LEE.
3. I. 1. Hardware Fast, available, non-exotic hardware with a wide range of software support was considered an essential prerequisite for LOIS, and a PC with a 486 processor was therefore selected for this purpose. Communication is by means of an interface providing a number of I/O channels which can be used either to drive LEE directly or, via some additional external conditioning circuitry, to provide the drive signals for the hydraulic valves on the target machine.
LOIS
LEE
0 LOIS
PLANT
(b) LOIS drives plant to confirm strategy.
Microcontroller (c)
LEE
Microcontroller drives LEE to confum transfer.
Microcontroller
PLANT
(d) Fully developed system. Fig. 5. System development using LOIS/LEE.
prototype plant directly as in Fig. 5(b). Following this confirmation, the software is transferred to the target processor and the transfer confirmed using LEE as in Fig. 5(c). After the software has been proven on its host, a full system test is carried out as in Fig. 5(d). As with any design process, this sequential progression is unlikely to occur in practice, and there will be a degree of iteration and parallel development, for instance to produce a hardware prototype while still developing software. This progression was seen in the early work on the LUCIE system, before the computer-based prototyping facility had been created, in the use of a one-fifth scale model for development purposes [lo].
3.1. LOIS The role of LOIS is to provide a rapid-prototyping and software-development facility in areas such as the development of control algorithms and operational strategies, as well as to support operator interface design and development. LOIS is therefore
3.1.2. Software In order to support the simulation of the operator’s control panel and associated displays, a Windowsbased system was considered to be essential, and it quickly became evident that to support the rapid prototyping of this interface and the associated control algorithms a powerful yet highly intuitive, simple and user-friendly programming environment was required. A survey of the market revealed that a range of powerful but complex packages were available. However, after consideration of the performance requirements and alternatives, it was decided to use Visual Basic for Windows to support the initial development of the system. 3.1.3. System design strategy In designing the operator interface it was necessary to consider the requirements for both the operator’s panel and the control algorithms, and the interactions between them. It would be possible to do this by means of a text file which could then be read by the main program. However, this approach - would require a sophisticated parser capable of identifing and interpreting the structure of the control algorithms; - would not utilise the inherently simple “drag and drop” window construction facilities offered by Visual Basic; - would not support the use of the interactive debugging facilities of Visual Basic. The initial strategy adopted was therefore to code each operator-interface structure or control-algorithm
371
G. Melling et al. /Automation in Construction 5 (1997) 365-378
tures of Visual Basic. The resulting structure and interrelationships of the LOIS software modules are as shown in Fig. 6.
change separately, and to assemble these into a new program which could then be developed rapidly and interactively, making use of the strengths and fea-
Run_LOIS ‘START
’
LOlSrs~~ / (alminatusLOlS thenresiarbL0l.s inilialiss data)
9 / Initialise Data LOIS clefaunr
! Set
/ pmject name en6
[ to
!
j / 4
j
j
/
I
--j
j
[Wk___---______,
n
I
I_
r^l ’ -
v JOYSW
END
’
LEE
lnput/Outpul
-
Convantiona~
proCeSS6S
______
Event&ban
Fig. 6. Relationships between LOIS software modules.
Display window
j 1
372
G. Melling et al. /Automation
Taking the LUCIE system already referred to as the model target environment, the following interface elements were created: A system-specific data-display panel, used primarily for system development and debugging purposes. An operator’s display panel. One format considered is shown in Fig. 7 and represents an operator’s display while digging. Information provided includes active systems, soft stops in the version shown, the current position of the excavator arm, the operating boundary of the machine and the limit set by the operator and the swept volume. An operator’s control panel. This enabled the operator to select the following operating modes: Direct mode. Operation corresponds to that of a conventional excavator, with each joint being controlled by means of the joysticks. X-Y mode. The joysticks are used to control the orthogonal motion of the bucket tip, with the system assuming responsibility for the control of individual joints. Support mode. This uses either of the direct or X-Y options, but the motion of the bucket is constrained to within the preset limits shown on the operator display. addition, the user is able to select from menus on the operator display, for instance to select or disengage the stop facility.
Boom
Options Selected
Soft stop I’
in Construction 5 (1997) 365-378
3.2. LEE The challenge with the equipment emulator was to provide an environment which complemented the LOIS component of the prototyping system whilst supporting the modelling of the machine kinematics in an easily assimilated form. For this purpose, an animated, real-time visualisation was considered to be essential as the means of providing effective feedback on the performance of the simulated machine. Initial considerations suggested an approach based on the adoption of a virtual reality (VR) environment. Though a full VR approach was ruled out in the initial stages of development on the grounds of system costs, an approach based on the concept of a desk-top reality system was adopted as providing a practical development while allowing a move to more sophisticated systems at a later date [ 11,121. 3.2.1. Hardware As is the case with LOIS, the requirement is for fast, available, non-exotic hardware with a wide range of software support, leading again to the choice of a PC with a 486 processor together with an appropriate I/O board as the platform for LEE. 3.2.2. So&are Update rates of 6 to 12 frames per second are required for smooth motion, but these are beyond the
Dipper
!,
\ ‘\
i / I
Machine Body
Swept Area (Volume)
User Defined Link
Machine Operating Limits
Fig. 7. Prototype of operator’s display panel for excavation
using LOIS.
G. Melling et al. /Automation
capability of current high-level packages; for instance, Visual Basic would require several minutes in a Windows environment to compute and render a LEE screen. A fast, dedicated package incorporating display and visualisation features and with direct access to the PC resources is therefore required. Several packages were examined and rejected because they could only be interfaced to external hardware for which the supplier had written an interface driver, which did r-rot include the interface boards to be used in this instance. Attention then focused on a simplified approach based on the use 01- a C library as this would allow complete control over the functions that LEE was required to implem~snt, albeit at the expense of having to write a dlzdicated C program as driver. REND386 is a software library that facilitates the construction of virtual reality PC applications on a “free and non-commercial use” basis; it has been modified and released as AVRIL. * To avoid any problems of incompatibility, the Borland Turbo C + + ~3.0 language used to develop AVRIL was then selected as the development environment. 3.2.3. System design strategy A basic C application skeleton is provided with the AVRIL library that allows the user to load “ worlds” and “figures”. The user can then navigate through and explore the world, interacting with figures and objects within that world in the process. LEE is constructed to be entirely generic, in that both the world and the figure are described in independent text files. In the initial phase of development, the world consists of two walls and a floor covered in 1-metre-grid squares, whilst the model of the excavator in that world is described by means of a simple articulated. representation of stretched cubes as in Fig. 8. After reading the world and figure files, LEE continuously scans LOIS for the valve driver signals and then moves the figure in the virtual world as appropriate by calling the appropriate AVRIL routines. The resulting orientation is then used to provide the equivalent sensor signal feedback to LOIS
313
in Construction 5 (1997) 365-378
Fig. 8. World and system model in AVRIL.
representing the output of the joint coders on board the excavator.
3.2.4. Enhanced system representation Though the simple representation of Fig. 8 demonstrates the capability to model system operation, in order to model a real environment and system it will be advantageous to link the creation of the virtual world and the machine model with material from other sources such as CAD packages. Fig. 9 shows a representation of an excavator produced by one such package. Once the drawing has been prepared, the three-dimensional (3D) information needs to be transferred to the virtual world. This could be achieved - directly from the drawing, or - indirectly using some transfer format. The first of these approaches has the advantage of directness. However, CAD data is commonly held in a proprietary, and non-disclosed, format which is
I babuck
I chassis-
tradk 2
A Virtual Reality Interface
Library.
and other en-
Fig. 9. CAD drawing drawing blocks.
of excavator
assembled
rotbase from named
sub
374
G. Melling et al./Automation
in Construction 5 (1997) 365-378
facto industry standard of the Drawing Exchange Format (DXF) for information transfer. It was therefore decided to use the DXF format in this instance as it was felt that the use of an industry-based standard was preferable to one as yet rarely ratified. Using this approach, the relevant CAD drawings are prepared, and a listing of components, scales and rotations is generated, together with associated movement data. The tile transfer is then initiated by the user to generate the representation in the virtual
subject to change with revisions of the software. This means that any converter is not only system-specific but also may rapidly become out of date. The first recognised standard for data transfer was IGES, but this suffers from the disadvantages of being lengthy and difficult to interface to. The current standard advocated for the transfer of 3D data is PDES/STEP, though few CAD systems are currently equipped for this. On a more limited front, the widespread use of AutoCAD software has made a de
Prototype subcomponent details: Tramheels Rotating base Amw Link rotations etc.
/ J
Base Plant
I
Data (Generic)
I
I
/ ! I I
[ Drawing of desired I prototype assembled I, from subcomponents
Transfer data extraction comprising:
f Attribute extraction 1 template
’ \
i I
Subcomponent names Locations in X, Y & 2 ScalesinX.Y&Z Rotations in X, Y 8 Z
1
Prototype Data (Specific) 1
_________________________________l___l__~~~~~~~~~~~~-~~~~~~~-~-~~~~~~~ r_______________________________________-__-______________________________~
/
1 Processing
!
I I
/
$-G-msp)
u
[ I
SrY
I
]
+
I
Site drawing: Surface geometry Excavation geometry Spoil removal
i..._.. _.......H ,
Transfer
F&
).-._.. ..l..l.l........._..l_....._ / ..._._....- .._...“.“.“.._.._ j
I I SiteData i I_____________________________________~~~~~~__________________________~ i
-b
Implemented
data flows
.......b
Fig. 10. Data structures for prototype
Additional generation.
data flows
375
G. Melling et al./Automation in Construction 5 (1997) 365-378
world. The implementation of the file transfer process and the resulting image in the virtual world is shown in Fig. 10, Fig. 11 and Fig. 12.
tion plant, and it will serve as the basis for further research and development in this area. A number of problems were identified with the system as initially configured, associated with the requirements of individual software packages in use and the need to provide a smooth, flicker-free display in both environments. These problems have largely been resolved and the system functions demonstrated. In addition, LEE has been operated in a stand-alone mode with direct joystick control of the model repre-
3.3. System evaluation The system as described has demonstrated a promising capability as an approach to the rapid prototyping and operational evaluation of construc-
Date fib tih
suba~mpmmt
narms. localims. scales and Paalwense4 s&p data fib defming mohma. lime. rate& dour% maltions and harrtvras dMals.
Lmcaeter
Operator hlttiace
SlrmAltor (LOIS)
Rs&s ioystick irpus. generates CMmol signals. receives feedtaadc dsp4ayr pasitim data, prodrcs apefatw display. I
L
4 i
i i in format reqhed world modelbr.
t$ the
I
_I
Datalink
+
I
Lancaster Equipment Emulator (LEE) b r Al8 tining mltims aId ’ fmtimpa-amsbrsfcr~ -b pmtyp in Uwfmnat req&ad by the wald modelbr. \
I
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . .. . .. . . . . ...” . . . .. . . . . . . . ..
---+
Reads comnmd inputs, prcdwm 30 visuabatim fw “SW sebaed view. rahmw pcsitim data.
lmplaented
software data now
.......... ............I ...................”................................................. ..........b
Fig. 11. Information
Potential soflwara datanows_._._+ flows in prototyping
toolset.
Ha&am
data W.vs
376
G. Melling et al. /Automation
in Construction 5 (1997) 365-378
Fig. 12. Modified world with CAD model of excavator
senting current machine tro-hydraulic valves.
control systems using elec-
4. System development As currently implemented, the LOIS/LEE configuration is itself a prototype of a larger system in which there will be a more refined representation of both the world system and the prototype system within the world modeller. There will also be features such as the ability to detect and report collisions and to simulate operations within that environment. This last feature will be essential where sophisticated operational investigations or training are major objectives of the prototyping process. In the case of the LUCIE system this could for instance introduce a requirement for the generation of a “ virtual trench” within the world model, such as that shown in Fig. 13. It is in these areas that current work is being focused, and certain of the features that are under consideration for inclusion in the next generation of both LOIS and LEE are listed in Table 1. As indicated in Table 1, in order to accommodate more sophisticated kinematic structures together with mass and inertia effects, the representation of the system model within the virtual reality environment will need to be further enhanced, perhaps through the use of a link to a kinematic modelling package or robot simulator.
inserted.
Where the development of automated and robotic plant and the interaction of such plant with its environment are the primary concern, this will place additional requirements on the software for the prototyping system. It is likely that such automated and robotic plant will utilise a modular architecture, and this in turn will imply a revision of the basic configuration of the rapid-prototyping environment to accommodate the processing requirements and development features that will be needed. In the development programme, particular stress is placed on the production of a high-level naturallanguage based ‘ ‘operator command and definition language” which will permit a user to rapidly define and configure a model user interface and will also
Excavated
,
Partially excavated /
Fig. 13.
G. Melling et al. /Automation
in Construction
311
5 (1997) 365-378
Table 1 Areas under review for future development Enhancement Link geometry
non-linearity
Improved kinematic
Control algorithm
representation
generation
Operator definition capability Operator menu definition capability Load monitoring Driver viewpoint Links to other systems
support
the programming
Description
Application
The relationships between the motion of the joint actuator and the joint motion is non-linear and this needs to be integrated within the kinematic model. The present kinematic representation takes no account of mass or inertia in determining system behaviour. For tasks involving the manipulation of large loads at long reach these effects will need to be incorporated in order to support features such as behavioural analysis and the development of control strategies. To allow for the investigation of advanced servo-control strategies and the introduction of model-based control. To allow an operator to define features such as bucket-full indication without the need to produce the code in full. To allow an operator to create a command menu structure directly. The provision of an estimate of power demands during the prototyping phase to support engine management. Provide a “driver’s eye-view” of the arm as it operates. To facilitate the incorporation of the other software systems such as those for kinematic modelling and representation.
LOIS + LEE
of the machine
at the task
level.
5. Conclusions The develpment of future generations of heavy plant for the construction and other industries presents significant challenges in the areas of software development, hardware configuration and kinematics and operator interface design and operability. In the work reported in the paper it has been shown that PC-based software can provide a rapid-prototyping facility for mechatronic-augmented heavy plant utilising window and de;sk-top reality techniques to support the development of such plant. This approach has been studied using as its target application the LUCIE autonomous and robotic excavator system developed in the Engineering Department at Lancaster University. In particular, the work has demonstrated that such a prototyping approach provides, at relatively low cost, the ability to investigate control strategies and operator interface development, and to identify errors and problems with these that would not be easily
LOIS + LEE
LOIS LOIS LOIS LOIS LEE LOIS + LEE
detected and identified by traditional design methodologies such as design reviews and code walkthrough. The approach also allows for the development and testing of control strategies in a laboratory environment, and contributes to a reduction in the overall development timescale due to the early detection of requirement errors. The basic principle of rapid prototyping as applied to mechatronic-augmented heavy plant having been demonstrasted, the research team is working on the development of the basic system described, to provide improved performance and to begin to evaluate the possibilities for the introduction of a high level “operator language” to facilitate system programming in the field.
References [l] Webster, D.A. and Challinor, S.F., Long-reach manipulators for decommissioning, Nuclear Energy, 32 (3) (1993) 173176. [2] Taylor, A.J., Design issues for underwater manipulator systems, Mechatronics, 3 (4) (1993) 419-432. [3] Boeke, M., Aust, E. and Schultheiss, G.F., Underwater operation of a 6-axes robot based on off-line programming and
318
[4]
[5]
[6]
[7]
G. Melling et al. /Automation graphical simulation, ‘91 ICAR, Fifth International Conference on Advanced Robotics - Robots in Unstructured Environments (19911, Vol 2, pp. 1342-1347. Taylor, S.R., Forceball control of a hydraulic backhoe excavator, MSc Disserration, Engineering Department, Lancaster University (1992). Thayer, S., Gourley, C., Butler, P., Costello, H., Trivedi, M., ChuXin Chen and Marapane, S., Three-dimensional sensing, graphics and interactive control in a human-machine system for decontamination and decommissioning applications, The International Society for Optical Engineering, SPIE, 1828 (1992) 74-85. Bejczy, A.K., Kim, W.-S. and Schenker, P.S., The role of computer graphics in advanced teleoperation, ASCE Conference on Robotics for Challenging Enuironments, Albuquerque (1994). pp. l-9. Hoffman, R. and Simmons, R., Simulation of autonomous
in Construction 5 (1997) 365-378 robot excavation, AXE Conference on Robotics for Challenging Environments, Albuquerque (1994). pp. 115-122. [S] Li, S. and Xu, Y., A software package for the graphic simulation of robots, Acta Automatica Sinica, 26 (4) (1990) 380-382. [9] Bradley, D.A., Seward, D.W., Mann, J.E. and Goodwin, M.R., Artificial intelligence in the control and operation of construction plant - the autonomous robot excavator, Automation in Construcrion, 2 (1993) 217-228. [lo] Bradley, D.A. and Seward, D.W., The Lancaster University computerised intelligent excavator programme, IMechE Conference on Mechatronics - Designing Intelligent Machines, Dundee (1992). pp. 163-168. [ll] Kalawsky, R.S., The Science of Virtual Real@ and Virtual Environments, Addison-Wesley, Reading, MA (1993). [12] Adams, L., Visualisarion and Virtual Reality, McGraw-Hill, New York (1994).
Automation in Construction 5 (1997) 365-378
The development of a rapid-prototyping technique for mechatronic-augmented heavy plant ’ G. Melling a, D.A. Bradley b, H. McKee ‘, M.B. Widden ’ a Consultant, b School
St. Annes, UK
ofElectronic Engineering and Computer Science, Uniuersity of Wales, Bangor, UK ’ Engineering Department,
Lancaster Uniuersity
Lancaster,
UK
Abstract Telechiric, semi-autonomous and autonomous heavy plant is finding an increasing role in applications such as construction, sub-sea work and decommissioning. There is a need for improved operator interfaces for such plant, and hence for rapid-prototyping tools which link the development of the operator interface with control and operational strategies and with machine geometries. The paper sets out a strategy by which different operator interfaces can be readily evaluated while at the same time generating the requisite information structure for the control of real items of plant. The proposed system is based on the use of interconnected PCs, one to simulate the operator interface and another to provide a kinematic representation of the machine using an appropriate “desk-top reality” environment. This system offers a safe, practical, rapid and cost-effective means of assessing proposed operator interfaces, as well as facilitating the development of machine kinematic structures and the associated operational and control strategies. Keywords:
Mechatronics; Robotics; Prototyping; Design; Human-machine interfaces
1. Introduction In recent years there has been increasing interest in the development and deployment of a range of telechiric, semi-autonomous and autonomous heavy plant in industries such as construction, mining and forestry, as well as for applications such as sub-sea work and both chemical and nuclear decommissioning [l-3]. The size and complexity of such plant make its development an expensive process, and create a range of new problems for the operator in
I
Discussion is open until August 1997 (please submit your discussion paper to the Editor on Constuction Technologies and Engineering, M. J. Skibniewski).
0926-5805/97/$17.00 Copyright PII SO926-5805(96)00161-6
achieving effective control. The difficulty experienced by an operator in attempting to control multiple degrees of freedom also acts as an inhibition on the development of machines with unusual and novel geometries. Where such machines have been produced, as for instance in the case of a recently introduced excavator which incorporates an additional degree of freedom at its “shoulder” (Fig. 1) to enable close operation alongside walls and other obstacles in a manner which is not possible using conventional slew control, they cannot be operated at full effectiveness using an approach based on the direct control of individual joints by the operator. In practice, with this system the operator moves the bucket into position using the additional degrees of
0 1997 Elsevier Science B.V. All rights reserved.
366
G. Melling et al. /Automation Dipper
Boom
(a) Conventional machine geometry with 3 degree-of-freedom an.
Added degrees-of-freedom
(b) Modified machine geometry with two added degrees of freedom in arm.
4;
degrees-of-freedom
(c) Plan view of machine with added degrees-of-freedom showing additional capability.
Fig. 1. Excavator
in Construction 5 (1997) 365-378
tion effectively precludes physical prototyping on a large scale, and so prevents machine designers from properly exploring the effects of changes in kinematic geometry on performance. This inability to prototype on a significant scale also makes it diffcult to evaluate operator interface strategies and to investigate the matching of the performance of the interface to the application. It is apparent therefore that there is a need for the development and provision of “rapid-prototyping” and visualisation tools which link the development of the operator interface with control and operational strategies as well as with machine geometries, and which enable different operational strategies to be evaluated rapidly within defined task environments [5-81. Such rapid-prototyping systems will also be able to provide support for software development, as well as for the investigation of new and novel control strategies. The paper describes an approach to the development of a rapid-prototyping system for mechatronicaugmented heavy plant which uses PC-based software, including a desk-top reality environment, to model machine behaviour and to explore and evaluate different operator interfaces and interface strategies.
2. System requirements
with added degrees-of-freedom.
freedom and then “locks off’ certain of the degrees of freedom before proceeding. The introduction of a computer-based controller able to encompass extra degrees of freedom would raise the productivity of such machines, as well as increasing the scope for further development. For instance, previous work has indicated that there may be significant operational benefits to be gained from the use of a multi-degree-of-freedom joystick to control excavator operation, particularly when combined with an improved operator interface providing additional information on bucket position and motion through the ground [4]. With the exception of certain special-purpose machines for the nuclear and other industries, the cost of developing machines of the type under considera-
Traditionally, the operation of items of heavy plant such as those used in the construction industry has relied on the ability of the operator both to position the plant at a given location and, once in position, to perform manipulative and other tasks. This is usually achieved by regulating the flow of hydraulic fluid to the machine actuators using either manual, directly connected joysticks or electronically connected joysticks as in Fig. 2. For a mechatronicaugmented item of plant such as the autonomous, robotic excavator (Lancaster University Computerised Intelligent Excavator - LUCIE) developed by the Intelligent Machines and Systems Group in the Engineering Department at Lancaster University [9,10] the servo control of the arm is taken over by the control processor as in Fig. 3. Such a system can operate either autonomously under program control or in response to operator inputs. In the latter case
G. Melling et al./Automarion in Construction 5 (1997) 365-378 Engine Contmls
(a) Direct contml of arm wing mechanically connected joysticks.
rn -,
$q-iq Pump
l
(b) Direct amtml of arm using dedmnic pmporlional joysticks.
Fig. 2. Excavator
with manual and electronic joysticks.
the operator is freed from the need to control the individual joints, and can concentrate on controlling the motion of the bucket or other end-effector. As a result of the ,work on the LUCIE system and other investigations, a number of functional requirements have been identified for full-scale systems which must be reproduced by any rapid-prototyping system. These are: . The ability to provide a range of controlled motions within an #operator-defined envelope: for instance, the provision of true straight-line motion of the bucket or other end-effector when mounted
361
on the end of an arm that includes revolute joints, in order to facilitate activities such as the digging of flat-bottomed trenches, both horizontal and inclined. The ability to define the operating envelope and limits on operation for the machine and the orientation of the end-effector within that envelope. Once movement outside preset limits has been prohibited, the operator is able to position the end-effector rapidly in a “carefree” manner, without needing to worry about the possibility of colliding with obstacles. The provision of operator feedback on the current task status. In the case of an excavator this could take the form of an indication of swept volume, and hence of bucket fill, when digging blind. The evaluation of machines with different kinematic geometries within a given task environment and the investigation of different operator strategies for such machines. Support for the development of operating software both for the operator interface and for the control of the machine. This should include both advanced, autonomous machines and developments such as software-defined “soft stops” for conventional machines to control the rates of deceleration when approaching the operational limits of the machine. In this context, it is worth noting that there were no perceived benefits to the operator in the control of acceleration. Indeed, the introduction of acceleration control tended to give the impression of a “sticky” actuator for which operators compensated by increasing the initial demand setting until the actuator apparently, and suddenly, became “unstuck”. In order to provide a high degree of flexibility, it was determined that the rapid-prototyping facility should be based around the following major sub-systems: A controlling sub-system dealing with the operator inputs, interface, displays, machine kinematics and control algorithms. - A controlled sub-system capable of emulating the plant and its operating environment at an appropriate degree of resolution, in order to provide feedback on the effectiveness of input devices, operator interfaces and control strategies and to evaluate the effect of changes in plant kinematic geometry on operation. l
368
G. Melling et al. /Automation
in Construction 5 (1997) 365-378
Electronic Joystick (Multi-degree-of-freedom)
;
CONTROLLER
Display
:m
I i
._._._._ ._.. ?_._._ ._._._._ ._..
I
:_
Bases
/
:
.j ‘1
I
i !
li, i
I I I
! I
i
I
MECHANICAL SYSTEM
i
j LEE 1.._....._........_..... ._._._.__....._. _._._._._ ._._._.__._._._._._._I _._..._._._._._._._._._._._ ..._.._..._._.__.__._._.___.____._._._.__._._., .;
Fig. 3. Mechatronic
approach
to system operation with the incorporation
For similar reasons, it was further decided that the message passing between the controlling sub-system and the controlled sub-system should be of the same form and format as that required by a real machine. This latter feature is determined by the requirement to ultimately replace the emulation with the real machine following the development phase. For the purposes of the initial work, the target machine has been the LUCIE system already in place in the laboratories. The target mode of application was operator-controlled excavation using either X-Y or
of the control processor.
conventional - direct control of joint actuators modes of operation, both with and without soft stops.
3. System design Following on from the definition of the system requirements it was apparent that the benefits of a rapid-prototyping approach will only be realisable, and the methodology will only be acceptable, if the
G. Melling et al. /Automation
computer-based protsotyping facility provides the following features: It is easy to use. It is easy and rapid to develop. It provides the required levels of functionality. It models the real world in a realistic way. It is of relatively low cost when compared with the cost of producing a physical prototype. The plant emulation and control environment is flexible, rapid and realistic. The system has tlhe ability both to emulate and to drive target systems in real time. The system is portable across a range of platforms and environments. These requirements forced a decision that the operator interface sllmulator and the plant emulator should be implemented on separate computers with appropriate interconnections and interfacing. The resulting systems are referred to as LOIS (Lancaster Operator Interface Simulator: Software Development and Operator Interface Design) and LEE (Lancaster Equipment Emulator: System Emulation and World Modelling). Fig. 3 shows the partitioning of functions between LOIS and LEE that would take place in a practical system (not all of the features shown in the figure have been realised at the time of writing), whilst Fig. 4 shows the distribution of software functionality between them. The LOIS/LEE configuration introduces a significant degree of flexibility into the prototyping process in that LOIS can be used to drive either LEE or
Knowledge Bases
+
Control Model
Y
Kinematic
Model
220”
+;
i
I
!
I
Pi Lo’sICF, LEE1 ,,,.I
yi3fNiming’
Interface
the real plant. Similarly, LEE can be driven either by LOIS or by the plant microcontroller system. Additionally, the use of two separate systems enables the optimum choice of hardware and software to be made and allows for the easy extension of the system to include other features, perhaps based on additional processors. Care must be taken to ensure the proper synchronisation of data between the two systems, as any failure in this area would have adverse effects on the operation of both systems. There is a commonality of certain of the data between the systems, for instance motion limits and rates. Other data such as the joystick input is required only by LOIS, whilst LEE is concerned with the sensory information. As the ultimate aim is to link together disparate systems, this requires that consideration be given throughout to the mechanisms of data management and data transfer. A typical design, build and test cycle using LOIS and LEE follows the stages shown in Fig. 5. Initially, the overall system, including the task environment, is evaluated using the LOIS/LEE rapid prototyping system configured as in Fig. 5(a) to assess factors such as the effects of different control strategies and machine kinematic geometries on operation, the configuration of the operator interfaces and the implications for software and associated hardware. Once the design and operational procedures have been developed using LOIS/LEE, these are then confirmed and refined by using LOIS to drive the
fi
Task Planning
Operator
369
in Construction 5 (1997) 365-378
7 J
Fig. 4. Distribution
L
Model
j senson /
of software functions between LOIS and LEE.
I
I j
G. Melling et d/Automation
370
in Construction 5 (1997) 365-378
BQBlQ@4
structured around and deploys a number of hardware and software sub-systems as follows.
(a) Development phase; LOIS drives LEE.
3. I. 1. Hardware Fast, available, non-exotic hardware with a wide range of software support was considered an essential prerequisite for LOIS, and a PC with a 486 processor was therefore selected for this purpose. Communication is by means of an interface providing a number of I/O channels which can be used either to drive LEE directly or, via some additional external conditioning circuitry, to provide the drive signals for the hydraulic valves on the target machine.
LOIS
LEE
0 LOIS
PLANT
(b) LOIS drives plant to confirm strategy.
Microcontroller (c)
LEE
Microcontroller drives LEE to confum transfer.
Microcontroller
PLANT
(d) Fully developed system. Fig. 5. System development using LOIS/LEE.
prototype plant directly as in Fig. 5(b). Following this confirmation, the software is transferred to the target processor and the transfer confirmed using LEE as in Fig. 5(c). After the software has been proven on its host, a full system test is carried out as in Fig. 5(d). As with any design process, this sequential progression is unlikely to occur in practice, and there will be a degree of iteration and parallel development, for instance to produce a hardware prototype while still developing software. This progression was seen in the early work on the LUCIE system, before the computer-based prototyping facility had been created, in the use of a one-fifth scale model for development purposes [lo].
3.1. LOIS The role of LOIS is to provide a rapid-prototyping and software-development facility in areas such as the development of control algorithms and operational strategies, as well as to support operator interface design and development. LOIS is therefore
3.1.2. Software In order to support the simulation of the operator’s control panel and associated displays, a Windowsbased system was considered to be essential, and it quickly became evident that to support the rapid prototyping of this interface and the associated control algorithms a powerful yet highly intuitive, simple and user-friendly programming environment was required. A survey of the market revealed that a range of powerful but complex packages were available. However, after consideration of the performance requirements and alternatives, it was decided to use Visual Basic for Windows to support the initial development of the system. 3.1.3. System design strategy In designing the operator interface it was necessary to consider the requirements for both the operator’s panel and the control algorithms, and the interactions between them. It would be possible to do this by means of a text file which could then be read by the main program. However, this approach - would require a sophisticated parser capable of identifing and interpreting the structure of the control algorithms; - would not utilise the inherently simple “drag and drop” window construction facilities offered by Visual Basic; - would not support the use of the interactive debugging facilities of Visual Basic. The initial strategy adopted was therefore to code each operator-interface structure or control-algorithm
371
G. Melling et al. /Automation in Construction 5 (1997) 365-378
tures of Visual Basic. The resulting structure and interrelationships of the LOIS software modules are as shown in Fig. 6.
change separately, and to assemble these into a new program which could then be developed rapidly and interactively, making use of the strengths and fea-
Run_LOIS ‘START
’
LOlSrs~~ / (alminatusLOlS thenresiarbL0l.s inilialiss data)
9 / Initialise Data LOIS clefaunr
! Set
/ pmject name en6
[ to
!
j / 4
j
j
/
I
--j
j
[Wk___---______,
n
I
I_
r^l ’ -
v JOYSW
END
’
LEE
lnput/Outpul
-
Convantiona~
proCeSS6S
______
Event&ban
Fig. 6. Relationships between LOIS software modules.
Display window
j 1
372
G. Melling et al. /Automation
Taking the LUCIE system already referred to as the model target environment, the following interface elements were created: A system-specific data-display panel, used primarily for system development and debugging purposes. An operator’s display panel. One format considered is shown in Fig. 7 and represents an operator’s display while digging. Information provided includes active systems, soft stops in the version shown, the current position of the excavator arm, the operating boundary of the machine and the limit set by the operator and the swept volume. An operator’s control panel. This enabled the operator to select the following operating modes: Direct mode. Operation corresponds to that of a conventional excavator, with each joint being controlled by means of the joysticks. X-Y mode. The joysticks are used to control the orthogonal motion of the bucket tip, with the system assuming responsibility for the control of individual joints. Support mode. This uses either of the direct or X-Y options, but the motion of the bucket is constrained to within the preset limits shown on the operator display. addition, the user is able to select from menus on the operator display, for instance to select or disengage the stop facility.
Boom
Options Selected
Soft stop I’
in Construction 5 (1997) 365-378
3.2. LEE The challenge with the equipment emulator was to provide an environment which complemented the LOIS component of the prototyping system whilst supporting the modelling of the machine kinematics in an easily assimilated form. For this purpose, an animated, real-time visualisation was considered to be essential as the means of providing effective feedback on the performance of the simulated machine. Initial considerations suggested an approach based on the adoption of a virtual reality (VR) environment. Though a full VR approach was ruled out in the initial stages of development on the grounds of system costs, an approach based on the concept of a desk-top reality system was adopted as providing a practical development while allowing a move to more sophisticated systems at a later date [ 11,121. 3.2.1. Hardware As is the case with LOIS, the requirement is for fast, available, non-exotic hardware with a wide range of software support, leading again to the choice of a PC with a 486 processor together with an appropriate I/O board as the platform for LEE. 3.2.2. So&are Update rates of 6 to 12 frames per second are required for smooth motion, but these are beyond the
Dipper
!,
\ ‘\
i / I
Machine Body
Swept Area (Volume)
User Defined Link
Machine Operating Limits
Fig. 7. Prototype of operator’s display panel for excavation
using LOIS.
G. Melling et al. /Automation
capability of current high-level packages; for instance, Visual Basic would require several minutes in a Windows environment to compute and render a LEE screen. A fast, dedicated package incorporating display and visualisation features and with direct access to the PC resources is therefore required. Several packages were examined and rejected because they could only be interfaced to external hardware for which the supplier had written an interface driver, which did r-rot include the interface boards to be used in this instance. Attention then focused on a simplified approach based on the use 01- a C library as this would allow complete control over the functions that LEE was required to implem~snt, albeit at the expense of having to write a dlzdicated C program as driver. REND386 is a software library that facilitates the construction of virtual reality PC applications on a “free and non-commercial use” basis; it has been modified and released as AVRIL. * To avoid any problems of incompatibility, the Borland Turbo C + + ~3.0 language used to develop AVRIL was then selected as the development environment. 3.2.3. System design strategy A basic C application skeleton is provided with the AVRIL library that allows the user to load “ worlds” and “figures”. The user can then navigate through and explore the world, interacting with figures and objects within that world in the process. LEE is constructed to be entirely generic, in that both the world and the figure are described in independent text files. In the initial phase of development, the world consists of two walls and a floor covered in 1-metre-grid squares, whilst the model of the excavator in that world is described by means of a simple articulated. representation of stretched cubes as in Fig. 8. After reading the world and figure files, LEE continuously scans LOIS for the valve driver signals and then moves the figure in the virtual world as appropriate by calling the appropriate AVRIL routines. The resulting orientation is then used to provide the equivalent sensor signal feedback to LOIS
313
in Construction 5 (1997) 365-378
Fig. 8. World and system model in AVRIL.
representing the output of the joint coders on board the excavator.
3.2.4. Enhanced system representation Though the simple representation of Fig. 8 demonstrates the capability to model system operation, in order to model a real environment and system it will be advantageous to link the creation of the virtual world and the machine model with material from other sources such as CAD packages. Fig. 9 shows a representation of an excavator produced by one such package. Once the drawing has been prepared, the three-dimensional (3D) information needs to be transferred to the virtual world. This could be achieved - directly from the drawing, or - indirectly using some transfer format. The first of these approaches has the advantage of directness. However, CAD data is commonly held in a proprietary, and non-disclosed, format which is
I babuck
I chassis-
tradk 2
A Virtual Reality Interface
Library.
and other en-
Fig. 9. CAD drawing drawing blocks.
of excavator
assembled
rotbase from named
sub
374
G. Melling et al./Automation
in Construction 5 (1997) 365-378
facto industry standard of the Drawing Exchange Format (DXF) for information transfer. It was therefore decided to use the DXF format in this instance as it was felt that the use of an industry-based standard was preferable to one as yet rarely ratified. Using this approach, the relevant CAD drawings are prepared, and a listing of components, scales and rotations is generated, together with associated movement data. The tile transfer is then initiated by the user to generate the representation in the virtual
subject to change with revisions of the software. This means that any converter is not only system-specific but also may rapidly become out of date. The first recognised standard for data transfer was IGES, but this suffers from the disadvantages of being lengthy and difficult to interface to. The current standard advocated for the transfer of 3D data is PDES/STEP, though few CAD systems are currently equipped for this. On a more limited front, the widespread use of AutoCAD software has made a de
Prototype subcomponent details: Tramheels Rotating base Amw Link rotations etc.
/ J
Base Plant
I
Data (Generic)
I
I
/ ! I I
[ Drawing of desired I prototype assembled I, from subcomponents
Transfer data extraction comprising:
f Attribute extraction 1 template
’ \
i I
Subcomponent names Locations in X, Y & 2 ScalesinX.Y&Z Rotations in X, Y 8 Z
1
Prototype Data (Specific) 1
_________________________________l___l__~~~~~~~~~~~~-~~~~~~~-~-~~~~~~~ r_______________________________________-__-______________________________~
/
1 Processing
!
I I
/
$-G-msp)
u
[ I
SrY
I
]
+
I
Site drawing: Surface geometry Excavation geometry Spoil removal
i..._.. _.......H ,
Transfer
F&
).-._.. ..l..l.l........._..l_....._ / ..._._....- .._...“.“.“.._.._ j
I I SiteData i I_____________________________________~~~~~~__________________________~ i
-b
Implemented
data flows
.......b
Fig. 10. Data structures for prototype
Additional generation.
data flows
375
G. Melling et al./Automation in Construction 5 (1997) 365-378
world. The implementation of the file transfer process and the resulting image in the virtual world is shown in Fig. 10, Fig. 11 and Fig. 12.
tion plant, and it will serve as the basis for further research and development in this area. A number of problems were identified with the system as initially configured, associated with the requirements of individual software packages in use and the need to provide a smooth, flicker-free display in both environments. These problems have largely been resolved and the system functions demonstrated. In addition, LEE has been operated in a stand-alone mode with direct joystick control of the model repre-
3.3. System evaluation The system as described has demonstrated a promising capability as an approach to the rapid prototyping and operational evaluation of construc-
Date fib tih
suba~mpmmt
narms. localims. scales and Paalwense4 s&p data fib defming mohma. lime. rate& dour% maltions and harrtvras dMals.
Lmcaeter
Operator hlttiace
SlrmAltor (LOIS)
Rs&s ioystick irpus. generates CMmol signals. receives feedtaadc dsp4ayr pasitim data, prodrcs apefatw display. I
L
4 i
i i in format reqhed world modelbr.
t$ the
I
_I
Datalink
+
I
Lancaster Equipment Emulator (LEE) b r Al8 tining mltims aId ’ fmtimpa-amsbrsfcr~ -b pmtyp in Uwfmnat req&ad by the wald modelbr. \
I
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . .. . .. . . . . ...” . . . .. . . . . . . . ..
---+
Reads comnmd inputs, prcdwm 30 visuabatim fw “SW sebaed view. rahmw pcsitim data.
lmplaented
software data now
.......... ............I ...................”................................................. ..........b
Fig. 11. Information
Potential soflwara datanows_._._+ flows in prototyping
toolset.
Ha&am
data W.vs
376
G. Melling et al. /Automation
in Construction 5 (1997) 365-378
Fig. 12. Modified world with CAD model of excavator
senting current machine tro-hydraulic valves.
control systems using elec-
4. System development As currently implemented, the LOIS/LEE configuration is itself a prototype of a larger system in which there will be a more refined representation of both the world system and the prototype system within the world modeller. There will also be features such as the ability to detect and report collisions and to simulate operations within that environment. This last feature will be essential where sophisticated operational investigations or training are major objectives of the prototyping process. In the case of the LUCIE system this could for instance introduce a requirement for the generation of a “ virtual trench” within the world model, such as that shown in Fig. 13. It is in these areas that current work is being focused, and certain of the features that are under consideration for inclusion in the next generation of both LOIS and LEE are listed in Table 1. As indicated in Table 1, in order to accommodate more sophisticated kinematic structures together with mass and inertia effects, the representation of the system model within the virtual reality environment will need to be further enhanced, perhaps through the use of a link to a kinematic modelling package or robot simulator.
inserted.
Where the development of automated and robotic plant and the interaction of such plant with its environment are the primary concern, this will place additional requirements on the software for the prototyping system. It is likely that such automated and robotic plant will utilise a modular architecture, and this in turn will imply a revision of the basic configuration of the rapid-prototyping environment to accommodate the processing requirements and development features that will be needed. In the development programme, particular stress is placed on the production of a high-level naturallanguage based ‘ ‘operator command and definition language” which will permit a user to rapidly define and configure a model user interface and will also
Excavated
,
Partially excavated /
Fig. 13.
G. Melling et al. /Automation
in Construction
311
5 (1997) 365-378
Table 1 Areas under review for future development Enhancement Link geometry
non-linearity
Improved kinematic
Control algorithm
representation
generation
Operator definition capability Operator menu definition capability Load monitoring Driver viewpoint Links to other systems
support
the programming
Description
Application
The relationships between the motion of the joint actuator and the joint motion is non-linear and this needs to be integrated within the kinematic model. The present kinematic representation takes no account of mass or inertia in determining system behaviour. For tasks involving the manipulation of large loads at long reach these effects will need to be incorporated in order to support features such as behavioural analysis and the development of control strategies. To allow for the investigation of advanced servo-control strategies and the introduction of model-based control. To allow an operator to define features such as bucket-full indication without the need to produce the code in full. To allow an operator to create a command menu structure directly. The provision of an estimate of power demands during the prototyping phase to support engine management. Provide a “driver’s eye-view” of the arm as it operates. To facilitate the incorporation of the other software systems such as those for kinematic modelling and representation.
LOIS + LEE
of the machine
at the task
level.
5. Conclusions The develpment of future generations of heavy plant for the construction and other industries presents significant challenges in the areas of software development, hardware configuration and kinematics and operator interface design and operability. In the work reported in the paper it has been shown that PC-based software can provide a rapid-prototyping facility for mechatronic-augmented heavy plant utilising window and de;sk-top reality techniques to support the development of such plant. This approach has been studied using as its target application the LUCIE autonomous and robotic excavator system developed in the Engineering Department at Lancaster University. In particular, the work has demonstrated that such a prototyping approach provides, at relatively low cost, the ability to investigate control strategies and operator interface development, and to identify errors and problems with these that would not be easily
LOIS + LEE
LOIS LOIS LOIS LOIS LEE LOIS + LEE
detected and identified by traditional design methodologies such as design reviews and code walkthrough. The approach also allows for the development and testing of control strategies in a laboratory environment, and contributes to a reduction in the overall development timescale due to the early detection of requirement errors. The basic principle of rapid prototyping as applied to mechatronic-augmented heavy plant having been demonstrasted, the research team is working on the development of the basic system described, to provide improved performance and to begin to evaluate the possibilities for the introduction of a high level “operator language” to facilitate system programming in the field.
References [l] Webster, D.A. and Challinor, S.F., Long-reach manipulators for decommissioning, Nuclear Energy, 32 (3) (1993) 173176. [2] Taylor, A.J., Design issues for underwater manipulator systems, Mechatronics, 3 (4) (1993) 419-432. [3] Boeke, M., Aust, E. and Schultheiss, G.F., Underwater operation of a 6-axes robot based on off-line programming and
318
[4]
[5]
[6]
[7]
G. Melling et al. /Automation graphical simulation, ‘91 ICAR, Fifth International Conference on Advanced Robotics - Robots in Unstructured Environments (19911, Vol 2, pp. 1342-1347. Taylor, S.R., Forceball control of a hydraulic backhoe excavator, MSc Disserration, Engineering Department, Lancaster University (1992). Thayer, S., Gourley, C., Butler, P., Costello, H., Trivedi, M., ChuXin Chen and Marapane, S., Three-dimensional sensing, graphics and interactive control in a human-machine system for decontamination and decommissioning applications, The International Society for Optical Engineering, SPIE, 1828 (1992) 74-85. Bejczy, A.K., Kim, W.-S. and Schenker, P.S., The role of computer graphics in advanced teleoperation, ASCE Conference on Robotics for Challenging Enuironments, Albuquerque (1994). pp. l-9. Hoffman, R. and Simmons, R., Simulation of autonomous
in Construction 5 (1997) 365-378 robot excavation, AXE Conference on Robotics for Challenging Environments, Albuquerque (1994). pp. 115-122. [S] Li, S. and Xu, Y., A software package for the graphic simulation of robots, Acta Automatica Sinica, 26 (4) (1990) 380-382. [9] Bradley, D.A., Seward, D.W., Mann, J.E. and Goodwin, M.R., Artificial intelligence in the control and operation of construction plant - the autonomous robot excavator, Automation in Construcrion, 2 (1993) 217-228. [lo] Bradley, D.A. and Seward, D.W., The Lancaster University computerised intelligent excavator programme, IMechE Conference on Mechatronics - Designing Intelligent Machines, Dundee (1992). pp. 163-168. [ll] Kalawsky, R.S., The Science of Virtual Real@ and Virtual Environments, Addison-Wesley, Reading, MA (1993). [12] Adams, L., Visualisarion and Virtual Reality, McGraw-Hill, New York (1994).