- Email: [email protected]
Autonomous robot technology for advanced wheelchair and robotic aids for people with disabilities
ELSEVIER
Robotics and Autonomous Systems Robotics and AutonomousSystems 14 (1995) 213-222
Autonomous robot technology for advanced wheelchair and robotic aids for people with disabilities Christian Biihler a,*, Ralf Hoelper b, Helmut Hoyer b, Wolfram Humann a a Forschungsinstitut Technologie-Behindertenhilfe (FTB) der Evangelischen Stiftung Volmarstein, Postfach 280, 5802 Wetter, Germany I, Prozeflsteuerungs- und Regelungstechnik FernUniversitiit Hagen, Hagen, Germany
Abstract
The use of robotic technology in assistive devices opens new opportunities for people with severe disabilities (tetraplegia, spinal cord injuries, etc.) at work and in their private homes. It can reduce social exclusion and assist social and vocational integration. Highly manoeuvrable wheelchairs for indoor use and wheelchair mounted arms are of particular importance. Due to their mobility, they are available on different locations, e.g. in different rooms of a dwelling for use in activities of daily living (ADL). Practical experiences and tests of a wheelchair mountable robot by users with disability are reported. This user analysis leads to further development of the human-machine interface and system integration towards an improved usability. A new type of highly manoeuvrable wheelchair is introduced. The modular software design provides comfortable means of integration. Parts of the work described are carried out within the ESPRIT project PMMA, the SPRINT project IMMeDIAte and the TIDE project OMNI. Keywords: Intelligent ,;ystems; Automated guided vehicles; Omnidirectional manoeuvrability; Service robots; People with disabilities; User interface
1. Introduction
The use of assistive technology is very common today, but leads often to problems in practice. Concerning independence many problems remain unsolved. Within the project "Integrated control systems for the handicapped and the elderly people" at FFB of tile Protestant Rehab Centre Volmarstein, the usability of a wheelchair mountable manipulator and a highly manoeuvrable wheelchair has been considered. It is targeted at
* Corresponding author. Tel.: +49 2335 9681-0; Fax: +49 2335 9681-19.
people using a powered wheelchair. The manipulator is to be used by disabled persons (e.g. muscular dystrophy) as an appliance for tasks, which these people cannot perform by themselves because of a lack of handpower or restricted motor performance. The wheelchair is designed for indoor use and highly movable also within restricted space. As the user wants to operate both technical aids via the very same but individual I / O procedure a uniform H M I design has been considered. Preliminary user tests with prototypes have shown that the qualities of this system have raised the radius of action of the user. Due to an increase of personal liberty, it supports an independent and self-assured life.
0921-8890/95/$09.50 © 1995 Elsevier Science B.V. All rights reserved SSD! 0921-8890(95)00030-1
214
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
2. System requirements In order to obtain an individually adaptable aid for the handicapped, the wheelchair has to be provided with a flexible, configurable functionality. That means, besides the 'normal' function of user-controlled movement several additional modes of operation should be available to the user. The functionality can be classified within a hierarchy, beginning with movements without sensorics, proceeding to sensor-supported control, and ending with path planning. The different layers of the hierarchy are shown in Fig. 1. The lowest layer is represented by the possibility to control the wheelchair by simply using a joystick. Proceeding within this hierarchy, the functionality of the wheelchair increases. In this next level, basic sensor systems (e.g. ultrasonic sensors) are integrated into the control system to support the user for example in collision avoidance and in generating safe movements. The next layer is built by an environment-directed movement of the wheelchair (driving along a wall, driving through a door) up to full sensor guidance e.g. t h e docking at a station. Special complex manoeuvres like play-back movement (redriving a recorded path, e.g. in ,the bathroom) or back-tracing (inversion of the recent manoeuvres) up to an automated guided movement (using devices of the vocational environment, e.g. magnetic ferrite marker lane, IR-landmarks, etc.) represent the top of the hierarchy. Fig. 1 reflects the distribution of effort between the user and the control system. As the wheelchair becomes more au-
tonomous by using the higher-level control, the burden on the user becomes less. While the functionality increases, safety aspects become more important. Similar aspects apply to the manipulator. Especially, there is a number of tasks which occur frequently in daily life, like drinking or picking things up. Control of the manipulator would be facilitated if these tasks were available and need not be composed of basic movements every time they occur. To give an adjusted and matched aid system to the individual, the functionality has to be easily adaptable to the specific handicaps of the user, that means, the system has to realise a high flexibility to allow transitions between the different layers. Furthermore, this operation on different levels of autonomy ensures that the user never feels patronised by the system. And although much autonomous robot technology is used in the system, it does not act totally autonomous in most cases because the user is always able to interrupt or modify ongoing action.
3. The robotic aid system 3.1. T h e w h e e l c h a i r m o u n t a b l e
Transferring industrial robot technology to wheelchair mountable robots is not a trivial task. The underlying reason is a number of specific requirements that need to be observed for this special application: -
-
-
-
wheelchair-control
~
user
Fig. 1. Hierarchy of wheelchair functionality.
robot
S e c u r i t y : In most cases the user is located within the workspace of the manipulator. The construction of the manipulator must therefore rule out the possibility of any harm to the user. Functionality: U s e r - o r i e n t e d functionality should enable every user to perform common tasks of h e r / h i s private and vocational life. F l e x i b i l i t y : Configuration and control need to be individualised to the needs as well as the physical and mental abilities of the user. M o b i l i t y : Low weight and low energy consumption are critical issues for mobile systems in general.
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222 Easy operation: Covers all aspects that facilitate the operation of the system like clear structure, reliability, and intuitive, user-oriented " h u m a n - m a n i p u l a t o r " interface.
3.1.1. Functionality and hardware The wheelchair mountable manipulator MANUS is designed especially for people with disabilities. Due to its special construction it guarantees, beside,~ functionality and flexibility, the safety of the user. The user is always able to modify the course of ongoing action or to stop the movement of the system. An external computer can be connected to the manipulator's control-box via serial interface, but is only necessary
215
to build new control-procedures or new configurations. The manipulator consists of a swivel arm with 6 rotational degrees of freedom, mounted on a basetube with a (redundant) translational degree of freedom and a gripper (see Fig. 2). Most joints are equipped with friction clutches to protect the user against excessive forces. The slipping moment is adjusted so that on the one hand the outstretched arm can lift up to 1.5 kg, but on the other hand the unpowered arm (as well as the gripper) can still be moved manually by an assistant. Maximum speed is limited to 0.5 m / s . Placing all motors and gears in the basetube minimises the mass and thereby the moment of inertia of the arm. These motors drive the peripheral joints using belt drives, cog wheels and hollow shafts. The electronics are divided into two parts: the basetube contains two PCBs with motor drives, optical encoders for the position of the joints, and interfaces. An external control-box contains the processor board, memory, converters, power supply and communication interfaces. 3.1.2. User tests The functionality of MANUS has been evaluated using two wheelchairs, which already exist on the market (electronic wheelchair type GENIUS from the company M E Y R A and an electric wheelchair type G A R A N T from the company INVACARE) [3]. These wheelchairs have different wheel combinations and hence different driving characteristics. It has also been considered that these characteristics could change due to the mechanical integration of MANUS. (A load compensation was necessary and implemented by adjustment of pneumatic springs.)
Fig. 2. The MANUS manipulator and its area of travel.
Evaluation o f the standard user interface. Separate control has been used for the devices: a standard joystick for the wheelchair, a 4 × 4 matrix minikeyboard for the manipulator. A configuration which assigns a command to each of these 16 keys is called a patchboard. The original patchboards group linear and rotational gripper movements in one patchboard each. Every function is offered in three speeds, giving a
216
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
total of six patchboards. Separate patchboards for translation and rotation result in the use of at least two patchboards for every task to be performed. Persons with different handicaps (muscular dystrophy, spast, tetraplegia, polyomyelitis, ICP, spina bifida) performed the first user tests. The operation of the coordination of linear movements in the three dimensions was subjectively judged as good by 3, medium by 4 and bad by 6 users. The standard keyboard itself was a poor solution for 8 users for different reasons. The users assessed the change of patchboards very negatively because it changes the function of the keys. This caused many mistakes and could all in all lead to refusal/rejection of the manipulator. Furthermore the separate controls (joystick for the wheelchair and keyboard for the manipulator) was recognised to be limiting. This made further development of the user interface inevitable.
Fig. 3. Daily task: shaving.
Improved patchboards. Newly developed configurations combine all important basic moves of the manipulator in one patchboard which is offered in two speeds (slow-quick). A single key-stroke switches from slow to fast and vice versa. Most movements are set off by the same keys on both patchboards, resulting in unambiguous, easy-tomemorise key-labels. The number of key-strokes required to perform certain tasks can be reduced by the use of special patchboards based on analyses of realhand movements for everyday activities (e.g. eating, drinking, opening) [3]. These complex movements can be composed of elementary movements of MANUS. All movements necessary for one task have been put together in one patchboard.
Test of daily tasks. The second set of tests was performed with a configuration of ergonomical and task-oriented patchboards in combination with an individualised keyboard. Some typical daily tasks tested are: to take care of oneself (e.g. shaving, Fig. 3); - to open doors and drawers; -
Fig. 4. Daily task: bottle handling.
Fig. 5. Vocational task in the library.
to - to to - to -
-
eat, drink and pour out (Fig. 4); get papers out of a file; grab and handle objects (Figs. 5, 6); lift up objects from the floor/ground.
217
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
Fig. 6. Daily task: handling goods from the bed.
The test users performed these tasks significantly different and one user requested to operate the robot only at a work station. Overall, it has to be noticed that all our users were keen to enhance their handling capabilities by the robot. However, we recognised that a certain mental power is needed to make real use of the current setup. Also the input device and the predefined movements for the respective users' needs are critical issues for success. In principle the mobility of the combination of wheelchair and manipulator was appreciated by all users, but the enlarged width (when driving through corridors or doors) and the devices separated controls were noticed to be significant disadvantages. While the disadvantage of the width can be solved by the use of a chair with small wheels in front, the control and operation problems have stimulated further development of HMI.
for the use in the rehabilitation was conducted. Wheeled mobile robots (WMR) with such driving features are well known in the field of robotics, e.g. Muir [9] has proved that WMRs using four omni-directional, actuated and sensed wheels provide three degree-of-freedom (DOF) locomotion. Within the scope of the study, experiments with such an omni-directional wheel-driven vehicle have shown that its performance is almost equal to those based on conventional wheels, with respect to various kinds of floor (stone, PVC, carpet) under several conditions (dry, wet, slippery) and on different gradients (flat to 25%) [4,5]. Moreover, simulations have illustrated the increase of mobility that can be yielded by the use of such omni-directional driving concepts, which allow to move even within packed indoor environments because of a non-restricted posi-
i¸¸%¸¸i¸
!iii!ii!iiiii:ii ~ii
3.2. The omnidirectional wheelchair platform 3.2.1. Mechanics To satisfy the demand of a higher mobility, new driving concepts have to be taken into consideration. At the Chair of Process Control, FernUniversit~it Hagen, mobile robot systems with different driving concepts have been developed. Based on this, a study [5] of the suitability of vehicles with omni-directional steering capability
Fig. 7. Wheelchair prototype.
218
C. Biihler et al. /Robotics and Autonomous Systems 14 (1995) 213-222
tioning capability. As a result, a prototype wheelchair (see Fig. 7) equipped with four socalled Mecanum wheels has been developed, able to move with three D O F in a plane, so that, in spite of its floor constraints, the wheelchair almost has the movability of a hovercraft. The prototype wheelchair has demonstrated the increase of mobility that can be yielded by the use of such omni-directional driving concepts, which allow to move even within packed indoor environments because of a non-restricted positioning capability. The omni-directional wheelchair not only overcomes the kinematic constraints of conventional ones and provides the user with a higher manoeuvrability, but also allows to simulate other wheelchairs with steering axes, which makes it useful for individual training.
3.2.2. Wheelchair control The control of the wheelchair platform is realised on a VMEbus based system using pSOS +, which is an operating system kernel providing multi-tasking and multi-processing features. The feedback-control structure which can be used for a three-wheeled as well as a four-wheeled omnidirectional driving concept is shown in Fig. 8. The illustrated control system, which is a kinematics based design, allows to track a reference trajectory and provides the operator with the possibility to control the vehicle in robot np coordinates or floor Fp coordinates, which is more
~
position L_~..:""coordinate"":~ control ~ transform. PID- filter [
/_
I
easy than a joint-space control. The control system encompasses an inner loop, which controls the wheel velocities and a position control loop. The actual platform position can be obtained by dead reckoning, i.e. by calculating it from the wheel sensor measurement (e.g. shaft encoder). First, the platform velocity is determined in dependence of the measured wheel velocities qi by applying the sensed forward velocity solution and a coordinate transformation in the case of a position control in floor coordinates. In a second step, the platform velocity is integrated over each sample period to get the actual position. After applying the control algorithm to the position error, which is obtained by comparing the desired with the actual position, the desired wheel velocities can be calculated with the actuated inverse solution. The input signal of the pulse-width modulators for driving the motors results from the velocity error and the velocity control filter. The actuated inverse as well as the sensed forward velocity solution are both derived from the com~osite 91atform equation of motion:
"ll~
0
...
0
where Ji denotes the Jacobian matrix for the wheel i and I represents the identity matrix.
actuated inverse velocity solution
Vce~tc~
Rp, Fp reference position
integration I
N
sensed forward I~ ' : : transform. ! _r_o_bot_pooy..! 1 velocity ]solution Fig. 8. Control structure.
I incrementalencodersignal for velocityfeedback
C. Biihleret al. / Robotics and Autonomous Systems 14 (1995) 213-222 Four-wheeled omni-directional wheelchair. Con-
~ 5
sider the four-wheeled vehicle design sketched in Fig. 9. The Mecanum-wheels are mounted at the corners of a rectangular frame. Supposing that the wheel slippage is negligible, the actuated inverse solution, i.e. the mathematical equation between the platform velocities in the stationary floor coordinate system and the four wheel-velocities is given by:
/1/11
+lb (l---ila+Ib)
O,)2111 0)3 =--~
1 1
0)4
1 1
Cl
Y]' C
whereby l a is the half width and l~ the half length of the vehicle (see Fig. 9). vx, Vy and F~ represent the velocities of the vehicle and its orientation related to the floor coordinate system. 0)1,-.., 0)4 are the wheel velocities and R is the wheel radius.
Three-wheeled omni-directional wheelchair.. Unlike the four-wheeled design, the wheels are placed at an angle of 120 degrees related to each other. By solving the composite platform equation of motion for the three-wheeled design, the actuated inverse solution can be calculated
(0)1)[-1
=R
Fig. 10. Three-wheeled vehicle design.
whereby l denotes the distance from the platform coordinate system to the wheel coordinate system
C i• Several functions of the low-level control, e.g. the servo-controller (motion unit), require realtime processing. Therefore modules, which use hardware resources, are realised on a VME bus based system using pSOS +, which is an operating system kernel providing multi-task and multiprocessor features.
3.3. The HMI
as"
0)2
°
C2
X
--(la+lb) (la + Ib)
219
1
l](FVx
1 1- vr3 -2---
1+V~---2--
l
l+v~
12V~-
l)t~
Fy
~°3
t ib
--la Fig. 9. Four-wheeled vehicle design.
'
Depending on the user's special needs it could be useful to have additional devices available within the system. Communication aids (e.g. BASCO [2]) for people with speech impairment and environmental control systems (ECS [6]) for the remote control of TV, door opener, light etc. are just two examples. This multiplies the already stated problem of separate controls for each device. A solution to this problem is a dedicated human machine interface (HMI). Fig. 11 shows the major functional components of such a HMI (e.g. Uni-Face [1]). A configurable set of objects is presented to the user on a screen. Objects are usually displayed as icons or text, organised in rows and columns. Each object relates to an action at a particular device (e.g. 'open gripper' relates to the robot, 'drive forward' relates to the wheelchair). The user can access all objects using the same input devices. These inputs can be one
220
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
' r
~.Usir~[
Input
]Human-Machine-Interlace P
devices ] ' ~ - - . ~ ~ , [Menu[lmonitorl.,: ~. [ oevices~ / devices ~ 1 generator erator I"~" l chairbus i
+ No or low additional hardware costs, + Various types of PCs available (desktop, laptop, industrial, notebook), - Some types of PCs (e.g. notebooks) have only a few ports.
Ontput i" ......... devices I
Fig. 11. Components of a HMI.
or more switches, a sip-and-puff device, mouse or joystick depending on the user's needs and abilities. Complex actions can be composed of a sequence of objects (e.g. a sentence for the communication aid is composed of several words or phrases). The advantages of such a H M I can be summarised as follows: - uniform access - configurable - large selection of input devices available, - interfacable to all devices in the system. Interfacing the H M I to the devices is addressed further in Section 4.
4. S y s t e m i n t e g r a t i o n
There are basically two ways to combine HMI, wheelchair, robot and, if necessary, additional components into a system: - All components connect directly to the H M I and the H M I forwards commands directly to the port which is connected to the respective device. - The H M I as well as all the other devices are connected to a chair bus system. Two examples are given to discuss these alternatives: If the H M I runs on a PC one could connect the other devices to the ports of that PC. The advantages and disadvantages of this solution are: + Each device can be connected to the type of port that fits best (RS232, parallel, PS/2),
A different solution is to use the M3S-bus as a chair bus for the system. The M3S-bus is an eight-wire bus based on the CAN-bus (properties: designed for automotive environments, highly reliable data transfer, high speed - here: 250 kbaud, multi-master) with additional lines for power supply and bus-independent safety functions. The M3S-bus has been developed for communication between devices of integrated wheelchair systems. Due to the open specification every manufacturer is free to develop his own M3S compatible devices. Such a system is specified by: + Easy system integration for a set of M3S compatible devices, + Direct inter-device communication possible, + Additional devices can be integrated later without problems, -Additional hardware (a configuration and control module CCM), - Every device needs to implement the M3S software protocol and hardware interface. Both options are currently being tested and assessed. The usage of both setups being almost identical, the decision will be based on aspects like costs and flexibility/configurability mainly.
5. C o n c l u s i o n s
Technology which has been developed in the area of autonomous and industrial robots can be applied successfully and profitably to the area of robotic aids for people with severe disabilities. The combination of a highly manoeuvrable wheelchair and a wheelchair mounted manipulator is especially useful, because the wheelchair makes even packed offices and similar narrow environments accessible and once the desired point is reached, the manipulator offers handling capabilities right there. A flexible and adaptable
C. Bfihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
h u m a n - m a c h i n e interface is another important issue. In combination with a tailored system integration it enables the user to control the complete system using the same input and output devices.
References [1] Ch. Biihler, Uniform user interface for communication and control, Proc. 2nd European Conf. on the Advancement of Rehabilitation Technology (ECART 2), Stockholm, Sweden (1993) No. 22.3. [2] Ch. Biihler and H. Heck, The versatile communication aid BASCO helps people with speech impairment, Journal of Microcomputer Applications (1993) 233-241. [3] Ch. Biihler, H. Heck and J. Nedza, Rollstuhlmontiertes Handhabungssystem f'tir Menschen mit Handicap, VDIFachtagung "Intel,!igente Steuerung und Regelung con Robotern", Langen, (1993); VDI Berichte 1094, 153-164. [4] Ch. Biihler and W. Humann, Smart wheelchair with high manoeuvrability, Proc. 2nd European Conf. on the Advancement of Rehabilitation Technology (ECART 2), Stockholm, Sweden (1993) No. P.I. [5] FTB, Elektro-Rollstuhl mit hoher Man6vrierf~ihigkeit, Interne Studie B92-2, Forschungsinstitut TechnologieBehindertenhilfe (FTB), Wetter 1992. [6] J.C. Gabus, JAMES - allgemeine Betrachtungen, Proc. Congress "Technohggie und Handicap", Neuchatel (1990) 303-324. [7] H. Hoyer and R. Hoelper, Open control architecture for an intelligent omn~idirectional wheelchair, Rehabilitation Technology, Proc. 1st TIDE Congress, (IOS Press, Brussels, 1993) 93-97. [8] H.H. Kwee, Rehabilitation robotics: Softening the hardware, Proc. 1990 h~tern. Conf. on Rehabilitation Robotics, A.I. duPont Institute, Wilmington, Delaware (1990). [9] P.F. Muir and P. Neumann, Kinematic modelling for feedback control of an omnidirectional wheeled mobile robot, IEEE Int. Conf. on Robotics and Automation (1987) 1772-1778. [10] F. Puppe, Einfiihrung in die Expertensysteme (Springer Verlag, Berlin, 1988). [11] G. Rzevski, ed., Applications of Artificial Intelligence in Engineering V (Computational Mechanics Publications, USA, 1990)
221
................................
Christian Biihler worked as a research associate from 1984-1985 at the Chair for Automation and Information Processing, Fern Universit~it of Hagen on industrial process control problems. At the University of Dortmund he worked from 1985-1990 as a research associate at the Institute of Robotics Research (IRF), where he became in 1988 Manager of the department of Control and MultiRobot Systems. Since 1991 Dr. Biihler is the Director of the Forschungsinstitut Technologie-Behindertenhilfe (FTB), a private Research centre on technologies for people with disabilities, of the Evangelische Stiftung Volmarstein, a large rehabilitation centre for people with motor and multiple impairments. His current interests relate especially to the field of assistive technology, e.g. control systems, prosthesis, computer applications and human-machine interaction for people with disabilities. In this field he carries out projects in research, development, test and evaluation with users, for the instruction and advice of rehab staff and the adaptation of assistive devices. Dr. Biihler is the national representative for Germany in the TIDE MMC and provides consultancy in the field of assistive technology to governmental, standardization and social institutions. Dr. Biihler is a member of VDI, the German Association for Rehabilitation, ISPO and RESNA. Ralf Hoelper was born in Bad Marienberg, Germany, in 1965. He received his Dipl.-Ing. degree in electrical engineering from the Technical University of Darmstadt, Germany, in 1991. From 1989 to 1990 he studied at the Ecole Centrale de Lyon, France. Since 1991 he has been with the Chair of Process Control (PRT), Department of Electrical Engineering, FernUniversit~it Hagen as research associate. His research interests are in robotics, especially the development of an intelligent, flexible configurable control systems for an omni-directional wheelchair, user-oriented path-planning and collision avoidance.
Helmut Hoyer received the Dipl.-Ing. degree in electrical engineering (especially control) in 1975 from the Universit~it (TH) Karlsruhe, Germany, and the Ph.D. degree from the Department of Electrical Engineering, FernUniversit[it Hagen, Germany, in 1984. After working as a group manager at the Institute of Robotics Research, Universit~it Dortmund, he is full professor and head of the Chair of Process Control (PRT) at FernUniversit~it Hagen since 1988. His research interests are in the field of robotics with special emphasis on control and programming, multi-robot systems, mobile robots and service robots. In the field of rehabilitation engineering the most recent topic concern flexible configurable control systems for the handling of mobile devices. Dr. Hoyer is involved in several projects in the field of intelligent control of wheelchairs.
222
C. Biihler et al. /Robotics and Autonomous Systems 14 (1995) 213-222
H u m a n n was born in Bochum, Germany, in 1965. He studied electrical engineering at the RuhrUniversit~it Bochum from 1986 to 1991. During this time he participated in an exchange program at Purdue University, West-Lafayette, Indiana, USA. In 1991 he received his Dipl.-Ing. degree in electrical engineering. Since 1992 he is working with the Forschungsinstitut TechnologieBehindertenhilfe. He is interested in circuit design in general, interfacing solutions for technical aids for disabled people, and advanced solutions for powered wheelchairs. Wolfram
Robotics and Autonomous Systems Robotics and AutonomousSystems 14 (1995) 213-222
Autonomous robot technology for advanced wheelchair and robotic aids for people with disabilities Christian Biihler a,*, Ralf Hoelper b, Helmut Hoyer b, Wolfram Humann a a Forschungsinstitut Technologie-Behindertenhilfe (FTB) der Evangelischen Stiftung Volmarstein, Postfach 280, 5802 Wetter, Germany I, Prozeflsteuerungs- und Regelungstechnik FernUniversitiit Hagen, Hagen, Germany
Abstract
The use of robotic technology in assistive devices opens new opportunities for people with severe disabilities (tetraplegia, spinal cord injuries, etc.) at work and in their private homes. It can reduce social exclusion and assist social and vocational integration. Highly manoeuvrable wheelchairs for indoor use and wheelchair mounted arms are of particular importance. Due to their mobility, they are available on different locations, e.g. in different rooms of a dwelling for use in activities of daily living (ADL). Practical experiences and tests of a wheelchair mountable robot by users with disability are reported. This user analysis leads to further development of the human-machine interface and system integration towards an improved usability. A new type of highly manoeuvrable wheelchair is introduced. The modular software design provides comfortable means of integration. Parts of the work described are carried out within the ESPRIT project PMMA, the SPRINT project IMMeDIAte and the TIDE project OMNI. Keywords: Intelligent ,;ystems; Automated guided vehicles; Omnidirectional manoeuvrability; Service robots; People with disabilities; User interface
1. Introduction
The use of assistive technology is very common today, but leads often to problems in practice. Concerning independence many problems remain unsolved. Within the project "Integrated control systems for the handicapped and the elderly people" at FFB of tile Protestant Rehab Centre Volmarstein, the usability of a wheelchair mountable manipulator and a highly manoeuvrable wheelchair has been considered. It is targeted at
* Corresponding author. Tel.: +49 2335 9681-0; Fax: +49 2335 9681-19.
people using a powered wheelchair. The manipulator is to be used by disabled persons (e.g. muscular dystrophy) as an appliance for tasks, which these people cannot perform by themselves because of a lack of handpower or restricted motor performance. The wheelchair is designed for indoor use and highly movable also within restricted space. As the user wants to operate both technical aids via the very same but individual I / O procedure a uniform H M I design has been considered. Preliminary user tests with prototypes have shown that the qualities of this system have raised the radius of action of the user. Due to an increase of personal liberty, it supports an independent and self-assured life.
0921-8890/95/$09.50 © 1995 Elsevier Science B.V. All rights reserved SSD! 0921-8890(95)00030-1
214
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
2. System requirements In order to obtain an individually adaptable aid for the handicapped, the wheelchair has to be provided with a flexible, configurable functionality. That means, besides the 'normal' function of user-controlled movement several additional modes of operation should be available to the user. The functionality can be classified within a hierarchy, beginning with movements without sensorics, proceeding to sensor-supported control, and ending with path planning. The different layers of the hierarchy are shown in Fig. 1. The lowest layer is represented by the possibility to control the wheelchair by simply using a joystick. Proceeding within this hierarchy, the functionality of the wheelchair increases. In this next level, basic sensor systems (e.g. ultrasonic sensors) are integrated into the control system to support the user for example in collision avoidance and in generating safe movements. The next layer is built by an environment-directed movement of the wheelchair (driving along a wall, driving through a door) up to full sensor guidance e.g. t h e docking at a station. Special complex manoeuvres like play-back movement (redriving a recorded path, e.g. in ,the bathroom) or back-tracing (inversion of the recent manoeuvres) up to an automated guided movement (using devices of the vocational environment, e.g. magnetic ferrite marker lane, IR-landmarks, etc.) represent the top of the hierarchy. Fig. 1 reflects the distribution of effort between the user and the control system. As the wheelchair becomes more au-
tonomous by using the higher-level control, the burden on the user becomes less. While the functionality increases, safety aspects become more important. Similar aspects apply to the manipulator. Especially, there is a number of tasks which occur frequently in daily life, like drinking or picking things up. Control of the manipulator would be facilitated if these tasks were available and need not be composed of basic movements every time they occur. To give an adjusted and matched aid system to the individual, the functionality has to be easily adaptable to the specific handicaps of the user, that means, the system has to realise a high flexibility to allow transitions between the different layers. Furthermore, this operation on different levels of autonomy ensures that the user never feels patronised by the system. And although much autonomous robot technology is used in the system, it does not act totally autonomous in most cases because the user is always able to interrupt or modify ongoing action.
3. The robotic aid system 3.1. T h e w h e e l c h a i r m o u n t a b l e
Transferring industrial robot technology to wheelchair mountable robots is not a trivial task. The underlying reason is a number of specific requirements that need to be observed for this special application: -
-
-
-
wheelchair-control
~
user
Fig. 1. Hierarchy of wheelchair functionality.
robot
S e c u r i t y : In most cases the user is located within the workspace of the manipulator. The construction of the manipulator must therefore rule out the possibility of any harm to the user. Functionality: U s e r - o r i e n t e d functionality should enable every user to perform common tasks of h e r / h i s private and vocational life. F l e x i b i l i t y : Configuration and control need to be individualised to the needs as well as the physical and mental abilities of the user. M o b i l i t y : Low weight and low energy consumption are critical issues for mobile systems in general.
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222 Easy operation: Covers all aspects that facilitate the operation of the system like clear structure, reliability, and intuitive, user-oriented " h u m a n - m a n i p u l a t o r " interface.
3.1.1. Functionality and hardware The wheelchair mountable manipulator MANUS is designed especially for people with disabilities. Due to its special construction it guarantees, beside,~ functionality and flexibility, the safety of the user. The user is always able to modify the course of ongoing action or to stop the movement of the system. An external computer can be connected to the manipulator's control-box via serial interface, but is only necessary
215
to build new control-procedures or new configurations. The manipulator consists of a swivel arm with 6 rotational degrees of freedom, mounted on a basetube with a (redundant) translational degree of freedom and a gripper (see Fig. 2). Most joints are equipped with friction clutches to protect the user against excessive forces. The slipping moment is adjusted so that on the one hand the outstretched arm can lift up to 1.5 kg, but on the other hand the unpowered arm (as well as the gripper) can still be moved manually by an assistant. Maximum speed is limited to 0.5 m / s . Placing all motors and gears in the basetube minimises the mass and thereby the moment of inertia of the arm. These motors drive the peripheral joints using belt drives, cog wheels and hollow shafts. The electronics are divided into two parts: the basetube contains two PCBs with motor drives, optical encoders for the position of the joints, and interfaces. An external control-box contains the processor board, memory, converters, power supply and communication interfaces. 3.1.2. User tests The functionality of MANUS has been evaluated using two wheelchairs, which already exist on the market (electronic wheelchair type GENIUS from the company M E Y R A and an electric wheelchair type G A R A N T from the company INVACARE) [3]. These wheelchairs have different wheel combinations and hence different driving characteristics. It has also been considered that these characteristics could change due to the mechanical integration of MANUS. (A load compensation was necessary and implemented by adjustment of pneumatic springs.)
Fig. 2. The MANUS manipulator and its area of travel.
Evaluation o f the standard user interface. Separate control has been used for the devices: a standard joystick for the wheelchair, a 4 × 4 matrix minikeyboard for the manipulator. A configuration which assigns a command to each of these 16 keys is called a patchboard. The original patchboards group linear and rotational gripper movements in one patchboard each. Every function is offered in three speeds, giving a
216
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
total of six patchboards. Separate patchboards for translation and rotation result in the use of at least two patchboards for every task to be performed. Persons with different handicaps (muscular dystrophy, spast, tetraplegia, polyomyelitis, ICP, spina bifida) performed the first user tests. The operation of the coordination of linear movements in the three dimensions was subjectively judged as good by 3, medium by 4 and bad by 6 users. The standard keyboard itself was a poor solution for 8 users for different reasons. The users assessed the change of patchboards very negatively because it changes the function of the keys. This caused many mistakes and could all in all lead to refusal/rejection of the manipulator. Furthermore the separate controls (joystick for the wheelchair and keyboard for the manipulator) was recognised to be limiting. This made further development of the user interface inevitable.
Fig. 3. Daily task: shaving.
Improved patchboards. Newly developed configurations combine all important basic moves of the manipulator in one patchboard which is offered in two speeds (slow-quick). A single key-stroke switches from slow to fast and vice versa. Most movements are set off by the same keys on both patchboards, resulting in unambiguous, easy-tomemorise key-labels. The number of key-strokes required to perform certain tasks can be reduced by the use of special patchboards based on analyses of realhand movements for everyday activities (e.g. eating, drinking, opening) [3]. These complex movements can be composed of elementary movements of MANUS. All movements necessary for one task have been put together in one patchboard.
Test of daily tasks. The second set of tests was performed with a configuration of ergonomical and task-oriented patchboards in combination with an individualised keyboard. Some typical daily tasks tested are: to take care of oneself (e.g. shaving, Fig. 3); - to open doors and drawers; -
Fig. 4. Daily task: bottle handling.
Fig. 5. Vocational task in the library.
to - to to - to -
-
eat, drink and pour out (Fig. 4); get papers out of a file; grab and handle objects (Figs. 5, 6); lift up objects from the floor/ground.
217
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
Fig. 6. Daily task: handling goods from the bed.
The test users performed these tasks significantly different and one user requested to operate the robot only at a work station. Overall, it has to be noticed that all our users were keen to enhance their handling capabilities by the robot. However, we recognised that a certain mental power is needed to make real use of the current setup. Also the input device and the predefined movements for the respective users' needs are critical issues for success. In principle the mobility of the combination of wheelchair and manipulator was appreciated by all users, but the enlarged width (when driving through corridors or doors) and the devices separated controls were noticed to be significant disadvantages. While the disadvantage of the width can be solved by the use of a chair with small wheels in front, the control and operation problems have stimulated further development of HMI.
for the use in the rehabilitation was conducted. Wheeled mobile robots (WMR) with such driving features are well known in the field of robotics, e.g. Muir [9] has proved that WMRs using four omni-directional, actuated and sensed wheels provide three degree-of-freedom (DOF) locomotion. Within the scope of the study, experiments with such an omni-directional wheel-driven vehicle have shown that its performance is almost equal to those based on conventional wheels, with respect to various kinds of floor (stone, PVC, carpet) under several conditions (dry, wet, slippery) and on different gradients (flat to 25%) [4,5]. Moreover, simulations have illustrated the increase of mobility that can be yielded by the use of such omni-directional driving concepts, which allow to move even within packed indoor environments because of a non-restricted posi-
i¸¸%¸¸i¸
!iii!ii!iiiii:ii ~ii
3.2. The omnidirectional wheelchair platform 3.2.1. Mechanics To satisfy the demand of a higher mobility, new driving concepts have to be taken into consideration. At the Chair of Process Control, FernUniversit~it Hagen, mobile robot systems with different driving concepts have been developed. Based on this, a study [5] of the suitability of vehicles with omni-directional steering capability
Fig. 7. Wheelchair prototype.
218
C. Biihler et al. /Robotics and Autonomous Systems 14 (1995) 213-222
tioning capability. As a result, a prototype wheelchair (see Fig. 7) equipped with four socalled Mecanum wheels has been developed, able to move with three D O F in a plane, so that, in spite of its floor constraints, the wheelchair almost has the movability of a hovercraft. The prototype wheelchair has demonstrated the increase of mobility that can be yielded by the use of such omni-directional driving concepts, which allow to move even within packed indoor environments because of a non-restricted positioning capability. The omni-directional wheelchair not only overcomes the kinematic constraints of conventional ones and provides the user with a higher manoeuvrability, but also allows to simulate other wheelchairs with steering axes, which makes it useful for individual training.
3.2.2. Wheelchair control The control of the wheelchair platform is realised on a VMEbus based system using pSOS +, which is an operating system kernel providing multi-tasking and multi-processing features. The feedback-control structure which can be used for a three-wheeled as well as a four-wheeled omnidirectional driving concept is shown in Fig. 8. The illustrated control system, which is a kinematics based design, allows to track a reference trajectory and provides the operator with the possibility to control the vehicle in robot np coordinates or floor Fp coordinates, which is more
~
position L_~..:""coordinate"":~ control ~ transform. PID- filter [
/_
I
easy than a joint-space control. The control system encompasses an inner loop, which controls the wheel velocities and a position control loop. The actual platform position can be obtained by dead reckoning, i.e. by calculating it from the wheel sensor measurement (e.g. shaft encoder). First, the platform velocity is determined in dependence of the measured wheel velocities qi by applying the sensed forward velocity solution and a coordinate transformation in the case of a position control in floor coordinates. In a second step, the platform velocity is integrated over each sample period to get the actual position. After applying the control algorithm to the position error, which is obtained by comparing the desired with the actual position, the desired wheel velocities can be calculated with the actuated inverse solution. The input signal of the pulse-width modulators for driving the motors results from the velocity error and the velocity control filter. The actuated inverse as well as the sensed forward velocity solution are both derived from the com~osite 91atform equation of motion:
"ll~
0
...
0
where Ji denotes the Jacobian matrix for the wheel i and I represents the identity matrix.
actuated inverse velocity solution
Vce~tc~
Rp, Fp reference position
integration I
N
sensed forward I~ ' : : transform. ! _r_o_bot_pooy..! 1 velocity ]solution Fig. 8. Control structure.
I incrementalencodersignal for velocityfeedback
C. Biihleret al. / Robotics and Autonomous Systems 14 (1995) 213-222 Four-wheeled omni-directional wheelchair. Con-
~ 5
sider the four-wheeled vehicle design sketched in Fig. 9. The Mecanum-wheels are mounted at the corners of a rectangular frame. Supposing that the wheel slippage is negligible, the actuated inverse solution, i.e. the mathematical equation between the platform velocities in the stationary floor coordinate system and the four wheel-velocities is given by:
/1/11
+lb (l---ila+Ib)
O,)2111 0)3 =--~
1 1
0)4
1 1
Cl
Y]' C
whereby l a is the half width and l~ the half length of the vehicle (see Fig. 9). vx, Vy and F~ represent the velocities of the vehicle and its orientation related to the floor coordinate system. 0)1,-.., 0)4 are the wheel velocities and R is the wheel radius.
Three-wheeled omni-directional wheelchair.. Unlike the four-wheeled design, the wheels are placed at an angle of 120 degrees related to each other. By solving the composite platform equation of motion for the three-wheeled design, the actuated inverse solution can be calculated
(0)1)[-1
=R
Fig. 10. Three-wheeled vehicle design.
whereby l denotes the distance from the platform coordinate system to the wheel coordinate system
C i• Several functions of the low-level control, e.g. the servo-controller (motion unit), require realtime processing. Therefore modules, which use hardware resources, are realised on a VME bus based system using pSOS +, which is an operating system kernel providing multi-task and multiprocessor features.
3.3. The HMI
as"
0)2
°
C2
X
--(la+lb) (la + Ib)
219
1
l](FVx
1 1- vr3 -2---
1+V~---2--
l
l+v~
12V~-
l)t~
Fy
~°3
t ib
--la Fig. 9. Four-wheeled vehicle design.
'
Depending on the user's special needs it could be useful to have additional devices available within the system. Communication aids (e.g. BASCO [2]) for people with speech impairment and environmental control systems (ECS [6]) for the remote control of TV, door opener, light etc. are just two examples. This multiplies the already stated problem of separate controls for each device. A solution to this problem is a dedicated human machine interface (HMI). Fig. 11 shows the major functional components of such a HMI (e.g. Uni-Face [1]). A configurable set of objects is presented to the user on a screen. Objects are usually displayed as icons or text, organised in rows and columns. Each object relates to an action at a particular device (e.g. 'open gripper' relates to the robot, 'drive forward' relates to the wheelchair). The user can access all objects using the same input devices. These inputs can be one
220
C. Biihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
' r
~.Usir~[
Input
]Human-Machine-Interlace P
devices ] ' ~ - - . ~ ~ , [Menu[lmonitorl.,: ~. [ oevices~ / devices ~ 1 generator erator I"~" l chairbus i
+ No or low additional hardware costs, + Various types of PCs available (desktop, laptop, industrial, notebook), - Some types of PCs (e.g. notebooks) have only a few ports.
Ontput i" ......... devices I
Fig. 11. Components of a HMI.
or more switches, a sip-and-puff device, mouse or joystick depending on the user's needs and abilities. Complex actions can be composed of a sequence of objects (e.g. a sentence for the communication aid is composed of several words or phrases). The advantages of such a H M I can be summarised as follows: - uniform access - configurable - large selection of input devices available, - interfacable to all devices in the system. Interfacing the H M I to the devices is addressed further in Section 4.
4. S y s t e m i n t e g r a t i o n
There are basically two ways to combine HMI, wheelchair, robot and, if necessary, additional components into a system: - All components connect directly to the H M I and the H M I forwards commands directly to the port which is connected to the respective device. - The H M I as well as all the other devices are connected to a chair bus system. Two examples are given to discuss these alternatives: If the H M I runs on a PC one could connect the other devices to the ports of that PC. The advantages and disadvantages of this solution are: + Each device can be connected to the type of port that fits best (RS232, parallel, PS/2),
A different solution is to use the M3S-bus as a chair bus for the system. The M3S-bus is an eight-wire bus based on the CAN-bus (properties: designed for automotive environments, highly reliable data transfer, high speed - here: 250 kbaud, multi-master) with additional lines for power supply and bus-independent safety functions. The M3S-bus has been developed for communication between devices of integrated wheelchair systems. Due to the open specification every manufacturer is free to develop his own M3S compatible devices. Such a system is specified by: + Easy system integration for a set of M3S compatible devices, + Direct inter-device communication possible, + Additional devices can be integrated later without problems, -Additional hardware (a configuration and control module CCM), - Every device needs to implement the M3S software protocol and hardware interface. Both options are currently being tested and assessed. The usage of both setups being almost identical, the decision will be based on aspects like costs and flexibility/configurability mainly.
5. C o n c l u s i o n s
Technology which has been developed in the area of autonomous and industrial robots can be applied successfully and profitably to the area of robotic aids for people with severe disabilities. The combination of a highly manoeuvrable wheelchair and a wheelchair mounted manipulator is especially useful, because the wheelchair makes even packed offices and similar narrow environments accessible and once the desired point is reached, the manipulator offers handling capabilities right there. A flexible and adaptable
C. Bfihler et al. / Robotics and Autonomous Systems 14 (1995) 213-222
h u m a n - m a c h i n e interface is another important issue. In combination with a tailored system integration it enables the user to control the complete system using the same input and output devices.
References [1] Ch. Biihler, Uniform user interface for communication and control, Proc. 2nd European Conf. on the Advancement of Rehabilitation Technology (ECART 2), Stockholm, Sweden (1993) No. 22.3. [2] Ch. Biihler and H. Heck, The versatile communication aid BASCO helps people with speech impairment, Journal of Microcomputer Applications (1993) 233-241. [3] Ch. Biihler, H. Heck and J. Nedza, Rollstuhlmontiertes Handhabungssystem f'tir Menschen mit Handicap, VDIFachtagung "Intel,!igente Steuerung und Regelung con Robotern", Langen, (1993); VDI Berichte 1094, 153-164. [4] Ch. Biihler and W. Humann, Smart wheelchair with high manoeuvrability, Proc. 2nd European Conf. on the Advancement of Rehabilitation Technology (ECART 2), Stockholm, Sweden (1993) No. P.I. [5] FTB, Elektro-Rollstuhl mit hoher Man6vrierf~ihigkeit, Interne Studie B92-2, Forschungsinstitut TechnologieBehindertenhilfe (FTB), Wetter 1992. [6] J.C. Gabus, JAMES - allgemeine Betrachtungen, Proc. Congress "Technohggie und Handicap", Neuchatel (1990) 303-324. [7] H. Hoyer and R. Hoelper, Open control architecture for an intelligent omn~idirectional wheelchair, Rehabilitation Technology, Proc. 1st TIDE Congress, (IOS Press, Brussels, 1993) 93-97. [8] H.H. Kwee, Rehabilitation robotics: Softening the hardware, Proc. 1990 h~tern. Conf. on Rehabilitation Robotics, A.I. duPont Institute, Wilmington, Delaware (1990). [9] P.F. Muir and P. Neumann, Kinematic modelling for feedback control of an omnidirectional wheeled mobile robot, IEEE Int. Conf. on Robotics and Automation (1987) 1772-1778. [10] F. Puppe, Einfiihrung in die Expertensysteme (Springer Verlag, Berlin, 1988). [11] G. Rzevski, ed., Applications of Artificial Intelligence in Engineering V (Computational Mechanics Publications, USA, 1990)
221
................................
Christian Biihler worked as a research associate from 1984-1985 at the Chair for Automation and Information Processing, Fern Universit~it of Hagen on industrial process control problems. At the University of Dortmund he worked from 1985-1990 as a research associate at the Institute of Robotics Research (IRF), where he became in 1988 Manager of the department of Control and MultiRobot Systems. Since 1991 Dr. Biihler is the Director of the Forschungsinstitut Technologie-Behindertenhilfe (FTB), a private Research centre on technologies for people with disabilities, of the Evangelische Stiftung Volmarstein, a large rehabilitation centre for people with motor and multiple impairments. His current interests relate especially to the field of assistive technology, e.g. control systems, prosthesis, computer applications and human-machine interaction for people with disabilities. In this field he carries out projects in research, development, test and evaluation with users, for the instruction and advice of rehab staff and the adaptation of assistive devices. Dr. Biihler is the national representative for Germany in the TIDE MMC and provides consultancy in the field of assistive technology to governmental, standardization and social institutions. Dr. Biihler is a member of VDI, the German Association for Rehabilitation, ISPO and RESNA. Ralf Hoelper was born in Bad Marienberg, Germany, in 1965. He received his Dipl.-Ing. degree in electrical engineering from the Technical University of Darmstadt, Germany, in 1991. From 1989 to 1990 he studied at the Ecole Centrale de Lyon, France. Since 1991 he has been with the Chair of Process Control (PRT), Department of Electrical Engineering, FernUniversit~it Hagen as research associate. His research interests are in robotics, especially the development of an intelligent, flexible configurable control systems for an omni-directional wheelchair, user-oriented path-planning and collision avoidance.
Helmut Hoyer received the Dipl.-Ing. degree in electrical engineering (especially control) in 1975 from the Universit~it (TH) Karlsruhe, Germany, and the Ph.D. degree from the Department of Electrical Engineering, FernUniversit[it Hagen, Germany, in 1984. After working as a group manager at the Institute of Robotics Research, Universit~it Dortmund, he is full professor and head of the Chair of Process Control (PRT) at FernUniversit~it Hagen since 1988. His research interests are in the field of robotics with special emphasis on control and programming, multi-robot systems, mobile robots and service robots. In the field of rehabilitation engineering the most recent topic concern flexible configurable control systems for the handling of mobile devices. Dr. Hoyer is involved in several projects in the field of intelligent control of wheelchairs.
222
C. Biihler et al. /Robotics and Autonomous Systems 14 (1995) 213-222
H u m a n n was born in Bochum, Germany, in 1965. He studied electrical engineering at the RuhrUniversit~it Bochum from 1986 to 1991. During this time he participated in an exchange program at Purdue University, West-Lafayette, Indiana, USA. In 1991 he received his Dipl.-Ing. degree in electrical engineering. Since 1992 he is working with the Forschungsinstitut TechnologieBehindertenhilfe. He is interested in circuit design in general, interfacing solutions for technical aids for disabled people, and advanced solutions for powered wheelchairs. Wolfram