Development of rescue robot system with human body grasping function

Copyright © IFAC Robot Control, Wroclaw, Poland, 2003

ELSEVIER

PUBLICATIONS www.elsevier.comllocate/ifac

DEVELOPMENT OF RESCUE ROBOT SYSTEM WITH HUMAN BODY GRASPING FUNCTION Daeheui Kim, Ryosuke Masuda

School ofInformation Technology and Electrics. Tolwi University 1117 Kitalwname Hiratsuka-shi Kanagawa. 259-1292. Japan

Abstract: In situations such as earthquakes and fires, rescue workers are called upon to do very dangerous and hazardous work. Consequently, there is a strong demand for rescue robots to do rescue operations instead of human rescue workers. This paper discusses our ongoing development of a rescue robot system that can provide direct help to disaster victims. To enable the system to perform rescue operations, the robot hands are equipped with multiple tactile sensors. A model of the rescue robot system and its multi-sensor control system was made, and it successfully performed fundamental experiments in rescue operations. Copyright ©2003 IFAC Keywords: Robot control, sensors

1. INTRODUCTION In recent years, cities have become larger and more complicated, with the result that the damage they incur in disasters is apt to be very serious and complicated. This is gradually making the rescue work required of human rescue workers very difficult and dangerous. Therefore, there is a growing demand for rescue robots that can come to the aid of disaster victims and prevent rescue workers from being exposed to secondary disasters (Takahashi, et aI., 1999; Kobayashi, et aI., 1993). Given this background, the research we have conducted has been focused on developing a rescue robot system that is able to grasp parts of the human body directly and safely (Ouhashi, et aI., 2000; Masuda, et aI., 1999; Chiba, et aI., 1998). Rescue robots need to be able to perform safe and delicate operations to rescue disaster victims, including searching for victims, removing obstacles, saving victims, and transporting them to safety zones. However, it is difficult for one comprehensive system to do all of these tasks for any given situations. Therefore, we concentrated on a system that is able to grasp a victim's wrist directly and lift it up. When an operator operates a robot to let it grasp a victim's wrist, the robot is likely to inflict pain on the

victim. To prevent this, we are working to develop a system incorporating a multi-sensor system that enables the robot to grasp the wrist as safely as possible. The multi-sensor system comprises a distributed tactile sensor, a force/torque sensor and a slip sensor mounted on the robot hand. Rescue task experiments were performed to show the method's effectiveness in ensuring victim safety.

2. RESCUE METHOD AND ROBOT SYSTEM When human operators rescue an earthquake or fire victim, they may employ one of many rescue techniques. One is to hold the victim's body in both arms, another is to grasp the victim's hand in me hand and support the victim's back with the other. The latter method was the one we selected for our rescue robot. Figure I shows an image of rescue work. R-.:ue Robot

Fig. 2 shows the composition of the rescue robot model we made. The rescue robot has two arms, with which it grasps and lifts up the victim by the wrist. The features of the parts of the rescue robot are explained in the following paragraphs.

r~-"

_s-~ I llponur

Fig. 2 Composition of rescue robot system model 2.1 Robot arms, hands, and mobile unit The rescue robot has two arms, each of which comprises a five-degree-Qf-freedom perpendicular joint type manipulator as the main arm and a four-degree-Qf-freedom perpendicular joint type manipulator as the sub-arm (Fig. 2). The robot hands fitted to the main arm and sub-arm have a two-finger adaptive mechanism. The robot hands have two-planet-gear mechanism joints, as shown in Fig. 3. These allow the robot hand to move flexibly in accordance with one of three conditions of an object. First, when no object exists, the robot finger moves to the dotted line shown in Fig. 3(a). Second, when the robot hand wraps itself around an object, the robot finger moves to the dotted line shown in Fig. 3(b). Third, when the robot hand pulls on an object, the robot finger moves to the dotted line shown in Fig. 3(c). Three types of tactile sensors are fitted on the robot hand to control it.

Fig. 4 Structure and view of multi-sensor hand 2.3 Sensors As shown in Fig. 2, several sensors are attached to the robot system. These comprise a vision sensor and three types of tactile sensors. The features of each sensor are explained in the following paragraphs. Vision sensor The vision sensor, mounted on the main arm as shown in Fig. 2, takes images of the prevailing conditions, displays them to the operator, and measures the distance between the robot and the object. Distributed tactile sensor The distributed tactile sensor, mounted on the inside of the robot hand as shown in Fig. 5, detects the shape of the object grasped by the robot hand, measures the force exerted by the robot hand, and controls the grasping force. . ,.Sensin~ line

51''ill

SliD "OSOI. •

JIrn 11:iFig. 3 Basic adaptive movements of robot hand A wheel-type mechanism with a DC motor adopted as the mobile unit.

IS

2.2 Multi-sensor robot hand The hand of the rescue robot is required to be able to grasp and lift up a part of the human body firmly and safely. Because the object of the rescue robot is a human victim, careful handling is required so as to prevent injury. As shown in Fig. 4, three tactile sensors, that is, a distributed tacti le sensor, a force/torque sensor, and a slip sensor, are attached to the rescue robot hand.

Fig. 5 View of distributed tactile sensor

[llilil

Particles of carbon

Slip sensor The slip sensor, attached to one side of the hand, uses a ball and two rotating encoders to detect the x-, y-directional relative slip motions between the robot hand and object (see Fig. 8). This sensor controls the grasping force to prevent an object from being dropped. The sensor resolution is measured as 0.13 mm.

Pressure-sensitive

.- . ~~ .~

Preasuresensitive rubber

-.l

T

1.5..

Film -shaped electrode

Fig. 6 Structure of distributed tactile sensor The distributed tactile sensor has a structure in which pressure-sensitive rubber is laid on a film-shaped electrode, as shown in Fig. 6. The pressure-sensitive rubber, made by kneading carbon particles into silicon rubber, detects changes in the electric resistance caused by changes in carbon density due to external pressure. The electrode lines are spaced at 1.5mm intervals, the resistance between the lines varies from 50 MO to 50 0 and the force that the electrode detects varies from 0.36 g/cm2 to 1.5 kg/cm2• The unit of the distributed tactile sensor has 32 sensing lines on each x- and y-axis. The distribution of grasping pressure acquired by the sensor is utilized to appropriately control the grasping force of the robot hand.

Fig. 8 Structure and view of slip sensor Control system Two computers are used to control the rescue robot system. One of them detects and processes the data of arm, mobile unit and the sensors attached to the robot hand. The other processes the information provided by the vision sensor. The processing interfaces between computers, sensors, and motor are shown in Table I. Table I Interfaces between computers, sensors, and !!lQ!Qr

Part Tactile sensor Force/torque sensor Slip sensor Vision sensor Hand motor Main-arm Sub-arm Mobile unit

Force/torque sensor The force/torque sensor, attached to the wrist portion of the main arm as shown in Fig. 7, detects the x-, y-, and z-axis -directional force/torque between the robot hand and the object. This sensor controls the pose and trajectory of the robot arm and robot hand so as not to harm the victim. The force and torque resolutions of the sensor are 5 gf and 30 gfcm, and its rated loads of force and torque are 5 kgf and 50 kgfcm.

3. FEEDBACK CONTROL OF MULTI-SENSOR ROBOT HAND SYSTEM

Xaxis

To ensure safe handling of victims, sensor feedback control is used to control the multi-sensor hand system. Safety criteria for handling the human body with a machine have been described in Ref. (Japan Robot Association, 2001)

~Forre

o

Processing Interface AID Converter RS-232C (Serial porter) Mouse porter Parallel I/O board Parallel I/O board Parallel porter Manual operation Parallel I/O board

Toroue

However, no exact criteria have been established for handling victims with injuries. Accordingly, in our research the criteria were determined by using the information provided by the distributed tactile sensor, the force/torque sensor, and the slip sensor. Appropriate safety criteria for each of the three tactile sensors were provisionally determined in basic experiments, and were used as the criteria for the control. Feedback control with distributed tactile sensor The appropriate safety criterion of the distributed tactile sensor is used to control the conditions for grasping the victim. As described above, the distributed tactile sensor unit contains 32 sensing lines, and so the

Fig. 7 Structure and view of force/torque sensor

·U5

average grasping pressure, Tavc , of the unit is calculated as follows: T",

=(L T,)f32[kg fern 'J. i = 1.2.3 •..., 32

0-=2: CormJI of wrist-pitch

(l)

where T; is the grasping force of each sensor line and i is the number of sensor lines. The maximum Tav• was measured as 1.5 kglcm2 , and the appropriate safety tactile criterion of the grasping pressure without giving injury, Tasc;, was measured as 1.2±0.25 kglcm2 . Tuc is used for a tactile correction case, Tcue, which is classified as the grasping pressure conditions (Table 2). The performance of tactile correction action taken for each tactile case is illustrated in Fig. 9. Table 2 Conditions oftactile correction cases Case I 2

3

Condition Tavc = T asc T avc < Tasc T avc > Tasc

Tactile correction action None Increase grasping force up to Tuc Decrease grasping force to T asc

Fig. ID Torque correction actions

Feedback control with slip sensor The slip sensor measures x-axis directional slip, S., and y-axis directional slip, &-' between the robot hand and the part of the victim's body grasped. The appropriate safety slip criterion, ~sc, was measured as 5 ± 0.0 mm. This criterion is used to perform the slip correction action according to the slip correction case, Scasc, as shown in Table 4. The slip correction actions taken by the slip sensor are illustrated in Fig. 11.

T case=2 : Increase grasping force /

Tcase=3 : Decrease grasping force

Table 4 Conditions of slip correction cases Fig. 9 Tactile correction actions

Case 1

Feedback control with force/torque sensor The force/torque sensor measures how much of a burden is imparted to the shoulder of the victim. Only the torque data of the sensor is used for the feedback control. Torque measurements were taken so as to avoid imparting an excess burden to the shoulder of victim, and from the results appropriate safety torque criteria were determined for three types of torque data. For each of these, i.e.• x-axis torque(Q.), y-axis torque(Qy), and z-axis torque(Qz), the criterion was measured to be 0.I4±0.02 kgfcm Using the measured safety torque criterion, Qasc, the torque correction action is performed according to the torque correction case, Qasc' as shown in Table 3. Figure 10 shows the torque correction action performed according to the twist between the robot hand and victim's arm as determined by force/torque sensor data.

2 3

S~ =2

Condition

1

Qx,y,z = Qase

2

Q. > Qasc Qy > Qasc Qz> Qasc

3 4

Slip correction action None Increase grasping force Increase grasping force : Increase 2rasoin2 force

S_=3 : Increase

Fig. 11 Slip correction actions

Table 3 Conditions of torque correction cases Case

Condition S.. Sy = Sasc S. > Sasc S. > Sasc

4. EXPERIMENTS WITH RESCUE ROBOT

Torque correction action None Control of wrist-pitch Stop Control of wrist-roll

This section describes the rescue operation process and the results of rescue operation experiments performed with the rescue robot.

-W6

carefully and lift it up, and support the back of victim with sub-arm. A view of the rescue operation is shown in Fig. 14.

Fig. 12 Flowchart of rescue operation performed by rescue robot

Fig. 14 View of rescue operation

Flowchart of rescue operation Figure 12 shows the rescue operation process performed by the rescue robot in accordance with several pre-established steps. First, the operator operates the robot to move it into the desired site, then observes the display images taken by the vision sensor to make a thorough search for the victim at the site. The operator selects the appropriate part of the victim's body to grasp and lift up, carefully grasp and lift it up. While the operator is controlling the arm in this step, multi-sensor control is carried out automatically. Finally, the operator operates the robot to place the victim on a transporter, and transports the victim to a safe place.

Results of rescue operation Experimental rescue operations were carried out through manual control of the two robot arms by the operator and the autonomous feedback control of the multi-sensor hand system. Figure 15-17 shows the time responses of the sensor data when the rescue operations were executed with and without sub-arm supporting. Taaile data (No suppon )

4.72

f-~--~r----;--:-----v-' "'::-:..r'f""=

~ 4.67 u

~ 4.62

f-

Human-typed model as vIctim The human-type model shown in Fig. 13 was adopted as the victim for the experiments. The model, made of wood and soft materials, is based on dimensional data for a 12-year-old male human being, but the weight of 3 kg is only 1/14 that ofa typical male human being of that age.

" "(

4.57 4.52 L-

_ Wcx'
4.72

...

Tactile data ( Supponing with sub-arm) -ij '"'

A

"l

~ 4.67 :WO

2S4 ;) (

u

;)

~"o

'l~' ..

lIO

.=

"

4.62

l.--

4.57 4.52

WorlUng stq>s

Unitfmm]

Fig. 15 Results of distributed tactile sensor control Torque data ( No support ) 600 ,------:.----'----'-''---'----

I-----------!::-I\+.---

--x

~ 200 ~ 0 "'=~~~~_frt!_~4~~

--v

400

Fig. 13 Human-type model for exp eriments

!-200 1----1----....::...------

View of rescue operation The rescue operation is performed by remote control executed by the operator. The principal steps in the operation are to determine the appropriate part of the victim's body to grasp, use the main arm to grasp the determined part

-400

1-------------

-600

L....

_

Working sreps

447

--z

appropriate safety slip criterion), which is achieved by feedback control.

Torque data (Supporting with sub-ann)

600 400

x

0 P~1-Jr-tJl..Lb~,i!2~!:\-,t.:=

~

E"

5. CONCLUSION

--:,

[] f-------.:L--------:..

~-200

f--------------

-400

-600 ' - - - - - - - - - - - WorkinS steps

Fig. 16 Results of force/torque sensor control Slip data ( No support ) 200 150

I

.9Ui

BJ

100 50 0 -50

200

.

--V axis

M

Working steps

Slip data ( Supporting with sub·arm )

150

B

E 100 oS 0-

Vi

v.is

--v

50

axi

0 -50

This paper has described the development of a rescue robot system that incorporates a multi-sensor robot hand system. Controlling a rescue robot is very important and very difficult, because its object is to save human victims. Because of this, the control system for a rescue robot must be adaptive to and safe for human beings. The rescue robot system proposed in this paper is being developed to enable rescue operations to be carried out as safely as possible. The means of achieving this objective is the use of a multi-sensor system, which comprises a distributed tactile sensor, a force/torque sensor, and a slip sensor attached to the robot hand. The multi-sensor hand system is autonomously controlled by feedback control, which also reduces the burden on operators who operates the rescue robot. The system is autonomously operated by feedback control, and experimental results confirmed that the use of feedback control is effective in enabling parts of the human body to be grasped safely, thus reducing the burden on the operator. A rescue robot for use at actual disaster sites must: I) Be capable of saving human victims 2) Have high heat-resistance tolerance against water and dust 3) Be absolutely safe

'-----..".w,-or1<:-on-g-steps-----

Fig. 17 Results of slip sensor control The results of distributed tactile sensor control are converted into a voltage unit, which is the average voltage of all the distributed tactile sensor lines mounted on the robot hand of the main arm. As shown in Fig. IS, the data of the distributed tactile sensor control is maintained at a fixed level, i.e., the appropriate safety tactile criterion, by feedback control of the multi-sensor robot hand. The results obtained with and without supporting the victim with the sub-arm differ, which strongly indicates that the sub-arm action affects the feedback control performed by the distributed tactile sensor. Figure 16 shows the results of torque sensor control obtained both with and without sub-arm support. As can be seen from the figure, in both cases the torque sensor data converges at a fixed level (the appropriate safety torque criterion), which is controlled by feedback control with the torque sensor and control devices. This demonstrates that torque feedback control is performed irrespective of the effect of sub-arm actions. Figure 17 shows the results of slip sensor control obtained both with and without sub-arm support. As can be seen from the figure, in both cases the slip sensor data is maintained at a fixed level (the

A model of the developed robot system was described and examined in this paper. The next research step will be the development of an actual rescue system that combines manual and autonomous control systems to achieve operator and sensor feedback control. REFERENCES Japan Robot Association (200 I), The research report ofintelligent robot standardization for plant. Ouhashi, et al. (2000), Control and sensing ofrescue robot system, The Society of Instrument and Control Engineers of Japan, pp. 117-118. Masuda, et al. (1999), Control of Rescue Robot Hand/Tool with Multi-sensor System, International Symposium on Robots. Takahashi, et al. (1999), The section meeting research report oflarge-scale disaster rescue robot system development, The Japan Society of Mechanical Engineers. Chiba, et al. (1998), The two-armed rescue robot for human handling, The Robotics and Mechatronics section of the Japan Society of Mechanical Engineers. Kobayashi, et al. (1993), The committee report of rescue robot development, Tokyo Disaster Education Association.