- Email: [email protected]
An experimental teletaction system for sensing and teleperception of human pulse
Available online at www.sciencedirect.com
Mechatronics 18 (2008) 195–207
An experimental teletaction system for sensing and teleperception of human pulse J. Dargahi, W.F. Xie *, Peng Ji Department of Mechanical and Industrial Engineering, Concordia University, 1455 de Maisonneuve Blvd. W. Montreal, Quebec, Canada H3G 1M8 Received 22 June 2006; accepted 4 December 2007
Abstract Teletaction is the sensing of a remote object by transmitting tactile information from a remote tactile sensor to an operator’s skin through tactile interface devices. A tactile interface is used to reproduce the information such as force (static and dynamic), texture, roughness, temperature, and shape. This paper describes the design, modeling, simulation, fabrication and testing of an experimental teletaction system to detect and reproduce the human pulse. It consists of three major components: pulse sensing system, pulse teleperception system and the data processing system. Two 25 lm thick polyvinylidene fluoride (PVDF) film sensors had been fabricated and calibrated for pulse detection and force measurement. A computer controlled force feedback system was developed for achieving pulse teleperception. The simulation and experimental results demonstrated the close matching between the sensed and perceived pulse. A psychophysics test showed that the fingertip feeling at the pulse teleperception end closely matched the feeling of touching a human pulse directly. Ó 2007 Elsevier Ltd. All rights reserved. Keywords: Teletaction; Tactile sensing; Tactile display; Pulse sensing; Pulse teleperception; Strain matching
1. Introduction The development of teletaction systems has become a hot issue in a number of research areas, including robotic assisted minimally invasive surgery (MIS) and tele-operated manipulation [1, 2]. Teletaction is the sensing of a remote object by transmitting cutaneous information from a remote tactile sensor to an operator’s skin (typically the fingertips) through tactile interface devices [3]. This information is important in applications such as tele-surgery, where the feel of the environment cannot be obtained by purely visual means. The tactile interface reproduces, as accurately as possible, the parameters such as force (static and dynamic), texture, roughness, temperature, and shape. The typical teletaction system comprises a tactile sensor, tactile display (the name typically given to a tactile feed*
Corresponding author. Tel.: +1 514 848 2424x4193; fax: +1 514 848 3175. E-mail address: [email protected] (W.F. Xie). 0957-4158/$ - see front matter Ó 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.mechatronics.2007.12.001
back device) and data processing unit. The tactile sensors interact with the environment and measure a contact force distribution. Tactile display systems are typically constructed with an array of actuators which deflect proportionally to strain on each sensing element of the tactile sensor. In a teletaction system, the sensed information is compared with the feedback from display devices and the comparison error is used as an input to the controller. The control system manipulates the tactile display in such a way that when an operator touches the tactile display with his/her finger, it can ideally provide the same tactile feeling as if the operator directly touches the object [4]. In this paper, we use pulse teleperception to represent tactile display in our pulse teletaction system. Most tactile sensors are based on various technologies such as piezoelectric, piezoresistive, strain gauging, optical, ultrasonic, electromagnetic and capacitive [5–7]. A typical tactile sensing system designed to detect force and pressure information has about 100 sensing elements in one square centimeter. This sensor array samples all sensing locations
196
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
in a short duration of time. Tactile sensors have numerous applications in robotics and medical areas. One of the medical applications is measuring and recording the human pulse. In recent years, a number of new methods of human pulse measurement and analysis have been developed such as the cardio-tachometer which measures and displays the pulse on a monitor. Piezoelectric technology is also used to measure the human pulse. An artificial finger, fitted with a PVDF tactile sensor array mounted on its tip, has been developed by Zheng et al. [18]. This device is able to record human pulse data with great accuracy. A typical pressure stimulation tactile display system has been described for the application of remote palpation tasks such as breast tumor localization or liver palpation [8]. A tactile display system, which recreates shape (pressure) profiles on the finger, has been designed using an array of mechanical pins actuated by commercially available radio controlled (RC) servomotors [8]. Another new type of tactile display, based on force feedback technique using an electrostatic linear actuator display system, was designed by Ishii et al. The tactile sensation is obtained from the shearing force generated on the fingertip by the electrostatic linear actuator [9]. Howe et al. [10,11] have developed a tactile shaped display for deployment on the handles of surgical instruments. They used shape memory alloy (SMA) as the actuation method because of its high power-to-weight and force-toweight ratios. However, SMA has nonlinear behavioral characteristics, contains high hysteresis properties and has a slow response time that poses a challenge to controller design. Cohn et al. [12] have developed a five by five tactile display of pneumatic actuators driven by solenoid valves. This tactile display has the advantage of economy as well as binary operation allowing for uniform open-loop response from actuator to actuator. Also, a continuously variable force output is possible using pulse-width modulation. Tactile displays using electro-rheological gel are also under investigation [13,14]. However, these types of tactile displays pose a possible safety hazard due to the relatively large voltages involved. There have also been tactile display designs utilizing actuators based on solenoids [15], and electrostatics [16]. A complete review of these tactile displays can be found in [16,17]. The combination of tactile sensors and tactile display systems can potentially be used in applications such as medical diagnostics, health monitoring and tele-operation manipulation. In this paper, an effective pulse sensing and teleperception system has been built to detect and reproduce a human pulse. The system can measure the human pulse and recreate the pulse sensation by using a linear actuator. A PVDF film is used for the pulse sensing and force feedback system due to its high reliability, low power consumption and low cost. It also has high sensitivity to a low level dynamic load such as a pulse. A linear actuator and an elastic layer are used for the pulse teleperception system. The advantages of stepping motors are
their low cost, high reliability and high torque at low speeds. Controlled by a force feedback control system, the linearly moving probe of the actuator produces a similar motion as the output of the pulse sensor. In addition, by integrating an elastomeric layer with the tip of an actuator, we are able to reproduce the pulse sensation. Therefore, when a human finger is in contact with the elastomer on the tip of the actuator, a similar sensation is perceived as when touching the pulse directly. Both simulation and experimental results demonstrate the success of the designed system in achieving human pulse teletaction. In this paper, Section 2 describes the system prototype and the working principles for the pulse sensing, pulse teleperception and tactile data processing. Section 3 builds the mathematical model for strain matching based on the elastic half-space principle. Section 4 details the system performance of strain matching in simulation. The experimental procedure and results are presented in Section 5 and the conclusion is drawn in Section 6. 2. System structure and working principle 2.1. System schematic As mentioned earlier, the objective of the proposed system is to replicate fingertip sensation of the human pulse by using a pulse teletaction system. Fig. 1 shows the complete system setup. The main components consist of a pulse sensing system, pulse teleperception system and tactile data processing unit. To produce identical fingertip feeling at the pulse teleperception side with the one at the sensing side, a strain matching method is proposed to achieve pulse perception. We use strain matching due to the fact that the outputs from both PVDF films in the pulse sensing and pulse teleperception sides change with the strain placed on the PVDF film [19]. The cutaneous mechanoreceptors in a human fingertip, however, may respond best to strain energy [4]. Fig. 2 illustrates the diagram of the system model. In this project, two PVDF film sensors have been fabricated for pulse detection and force measurement in the pulse sensing system. A linear actuator is controlled to apply load on a rubber layer block to produce the pulse sensation in the pulse teleperception system. The tactile data processing unit comprises a computer and an A/D converter which are used as a data processor and digital control system. The detailed design of each component in the system is illustrated in the following sections. 2.2. PVDF sensor design As shown in Fig. 3A, at the pulse sensing side, the essential component is the 2.5 micron thick metalized and poled PVDF film wrapped on the tip of the sensor probe consisting of plexi-glass and a rubber layer mounted between the probe tip and the PVDF layer. Consider the spatial impulse
J. Dargahi et al. / Mechatronics 18 (2008) 195–207 Charge Amplifier
197
Stepping Motor Drive
Pulse Sensing I/O Card
Computer Control System
Pulse Teleperception
Fig. 1. A photograph of the teletaction system setup.
Pulse Sensing System
Tactile Data Processing Computer & A/D Converter
Force Sensor
Feedback
+
Driving Electric Unit
Pulse Display System
Fig. 2. System diagram.
Probe
Rubber Layer PVDF Sensing Film Pulse Force Artery
Fig. 3A. PVDF sensor at pulse sensing.
response of the pulse teleperception system. It is very difficult to achieve stimulator density comparable to human mechanoreceptor density (on the order of 200 cm2), a rubber layer, which acts as a spatial low-pass filter, is essential [3]. In this way, the pulse teleperception system feels like touching the real pulse through a rubber layer. The rubber layer functions as low-pass filters to convert the surface stress to strain and the stress/strain model is discussed in detail in Section 3. At the pulse teleperception side, the same type of PVDF film is used in the force feedback system. As shown in Fig. 3B, PVDF film is mounted on the pulse teleperception block between the rubber layer and base block. The measured voltage produced by the PVDF film is used as a feedback signal in the force control system. The corresponding
198
J. Dargahi et al. / Mechatronics 18 (2008) 195–207 Rubber Layer
PVDF Force Sensor Plexi-glass Base Block
V pv ¼ ðk g31 g31 þ k g32 g32 Þrpv d pv
Fig. 3B. PVDF sensor at pulse teleperception.
Output Voltage from Sensor (mVolts)
by the probe, the stress is transmitted to the PVDF film laterally. The reason is that the PVDF sensors for both pulse sensing and teleperception are designed as membrane sensors. Thus the applied load in the thickness direction would be zero. Based on piezoelectric properties [19] for the voltage mode, the voltage output Vpv from PVDF film is expressed as
where g31 and g32 are the voltage mode piezo coefficients in the membrane lateral and transverse direction respectively (shown in Fig. 5), kg31 and kg32 are the coefficients for g31 and g32 respectively, rpv is the stress developed on the cross-sectional area of PVDF film and dpv is the thickness of the PVDF film. The parameters and specifications of the PVDF film are listed in Table 1. Since the PVDF film is biaxial, one has
PVDF Sensor Calibration 120 100 80 60 40 20
k g31 ¼ k g32 ¼ 1:
0 0
0.5
1
ð1Þ
1.5
2
2.5
Reference Loading (Newtons)
At the pulse sensing side, the stress rpv1 is obtained as rpv1 ¼
Fig. 4. PVDF sensor calibration.
calibration of the PVDF film at the pulse teleperception side is shown in Fig. 4. A similar linear calibration result was obtained for the PVDF film at the pulse sensing side. 2.3. PVDF sensor modeling At the pulse sensing side, the PVDF sensing film is mounted on the spherical tip of the sensing probe. At the pulse teleperception side, the area of contact between the spherical tip of the linear moving probe and the PVDF film is also circular in shape. Therefore, an assumption can be made that the effective load from pulse and actuator lies in the g31 and g32 direction and the effective loading area is the cross-sectional area contacting the PVDF membrane (shown in Fig. 5). When a load is applied to the PVDF film
ð2Þ
Fp Aload1
ð3Þ
where Fp is the pulse load on the cross-sectional area of the PVDF film and Aload1 is the effective loading area in PVDF film. Aload1 is written as Aload1 ¼ 2prpv1 d pv
ð4Þ
where rpv1 is the radius of the contacting area at the pulse sensing side. Therefore, from Eqs. (1)–(4), the pulse load can be expressed as Fp ¼
2V pv1 prpv1 : ðg31 þ g32 Þ
ð5Þ
At the pulse teleperception side, the stress rpv2 is obtained as rpv2 ¼
Fa Aload2
ð6Þ
g 33 (d 33 ) Normal to the PVDF Surface
g 32 (d 32 ) in Transverse Direction
g 31 (d 31 ) in Lateral Direction Contacting Area
Effective Loading Area
Fig. 5. Piezoelectric properties in PVDF sensing membrane.
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
where Fa is the actuator load on the cross-sectional area of the PVDF film and Aload2 is the cross-sectional area of PVDF. Aload2 is written as Aload2 ¼ 2prpv2 d pv
ð7Þ
where, rpv2 is the radius of the contacting area at the pulse teleperception side. Thus, from Eqs. (1), (2), (6) and (7), the pulse load can be expressed as Fa ¼
2V pv2 prpv2 : ðg31 þ g32 Þ
ð8Þ
Eqs. (5) and (8) are used in the strain matching validation in the later section. 2.4. Pulse teleperception and data processing As shown in Fig. 6, the pulse teleperception system comprises the linear actuator, pulse perception block (see Fig. 2 in Section 2.1) and the force feedback control system. In order to reproduce the human pulse, a closed-loop force feedback control system (Fig. 7) is designed to guarantee both amplitude and direction matching of strain so that identical fingertip feelings are obtained at both the pulse sensing and teleperception sides [20]. The tactile data processing detail is illustrated in Fig. 8. The linear actuator (Dyadic SCN-010-AS) for the pulse teleperception is driven by a two-phase hybrid stepping motor (detailed parameters for the linear actuator are shown in Tables 2A and 2B). As previously stated, the advantages of stepping motors are their low cost, high reliability and high torque at low speeds. A QUANSER MultiQ-PCI I/O card is applied to implement the data acquisition and A/D & D/A conversion. The human pulse force Fp is sampled by the PVDF sensor at the pulse sensing side. The output actuator force signal Fa is measured by another PVDF sensor at the pulse teleperception side. By using MATLAB SIMULINK, a real-time control system was built to control output force Fa to approach pulse force Fp. In response to pressure changes created by the applied human pulse, the PVDF produces high-impedance output charge signals [19]. Both signals at the pulse sensing and teleperception sides are converted to low-impedance voltage signals by the charge amplifier. A low-pass filter is used
Fig. 6. Pulse teleperception system setup.
199
to prevent unwanted 50/60 Hz AC line interference from entering the sensor. After being converted to a digital signal by an A/D converter in the I/O card, the two signals are fed to the comparator to generate the error signal e. The most commonly used PID controller is adopted to control step motor due to its simple structure [31]. The PID controller in the control system calculates this error e to produce a control signal u which is fed to the voltage-controlled-oscillator (VCO). By introducing the VCO in the control system [21, 22], the magnitude and speed of the stepping motor can be controlled based on the input control signal u. The stepping motor direction is determined by a logical control program based on the stepping motor operating principle. By choosing appropriate PID (proportional, integral and derivative) parameters, the output pulse load Fa tracks the reference input pulse load Fp in real-time. 3. Mathematical model for strain matching At the pulse sensing and teleperception sides, the rubber layers function as reconstruction filters to convert the surface stress to strain ez=d1 and ez=d2, respectively. It is critical to build a stress/strain model for the elastic layer that comes into contact with the fingertip. This specific modeling problem is related to contact mechanics where the theory of halfspace is widely employed when a probe or an indenter is pressed against a solid. According to the theory of elastic half-space model [26], when the elastic bodies make contact over an area whose dimensions are small compared with the radii of curvature of surfaces not deformed, the contact stresses are highly concentrated close to the contact region and decrease rapidly in intensity with distance from the point of contact. Provided the dimensions of the bodies are large compared to the dimensions of the contact area, the stresses in this region are neither critically dependent upon the shape of the bodies far from the contact area, nor upon the precise way in which they are supported. The stresses may be calculated with good approximation by considering each body as a semi-infinite elastic solid bounded by a plane surface (an elastic half-space). This ideal assumption, in which bodies having an arbitrary surface profile are regarded as semi-infinite in extent and having a plane surface, simplifies the boundary conditions. Hence the half-space theory is valid in the case where the strains remain small and the contact area is small compared with the dimension of the rubber layer. In this project, since the deflection of the rubber layer is small and there is no permanent deformation, the theory is appropriate for building the stress/strain models of rubber layers. The half-space theory can be extended to the linear and nonlinear viscoelastic models too, specifically when it is used to model rubber and biological issues [28]. In case of large deformation, the effects of large deformations and hyperelasticity should be considered [29]. However, these effects are negligible in this application and hence are ignored. Some dynamic contacting models for the strain matching have been developed to describe the accurate dynamic
200
J. Dargahi et al. / Mechatronics 18 (2008) 195–207 Pulse
Fp PVDF Sensor-- Pulse Sensing
e
u Controller
PVDF Sensor—Pulse Teleperceptron
Actuator
_
Fa
Force Feedback
Fig. 7. Force feedback control system.
Fp
Human Pulse signal
+
Charge AMP & Low Pass Filter A/D Converter
Force Sensor
Charge AMP & Low Pass Filter
Fa Step Motor Control Pulse
e _
Force Feedback
PID
VoltageControlledOscillator
Magnitude & Speed
u
D/A Converter
Stepping Motor Direction Control
Stepping Motor Direction Control
Step Motor Control Direction
Actuator
I/O Card
Computer Control Program
Fig. 8. Block diagram of data processing.
behavior of stress/strain of rubber. However, it is difficult to build such a dynamic contacting model for the strain matching in the system due to the complicated biomechanical characteristics of human fingertip of which two examples model its nonlinear viscoelasticity and relaxation [23– 25]. Therefore, without considering the fingertip factor, a simplified rubber layer model is built based on the elastic half-space model. 3.1. Principle of elastic half-space model In the reference frame shown in Fig. 9, the boundary surface is in the x–y plane and its z-axis is directed into the solid. The loaded strip lies parallel to the y-axis and has a width (r0 + r0) in the x-direction. The strip carries normal and tangential forces which are a function of x only. Let us assume that a state of plane strain (ey = 0) is produced in the half-space by the line loading. Therefore, in Fig. 9, d is the elastic layer thickness and P is the magnitude of the applied load acting in a direction normal to the surface of the rubber layer. Consider any point B (x, z) at the halfspace. The stresses along the x and z axes are denoted as
Fig. 9. Signal flow model for strain matching.
Consider A (0, d), which is the target point (pulse output point) for the project. The stresses acting along x and z can be represented as rx jA ¼ 0
ð11Þ
2P rz jA ¼ pd
ð12Þ
2
rx ¼ rr sin2 h ¼ rz ¼ rr cos2 h ¼
2Px z pðz2 þ x2 Þ
ð9Þ
2
2Pz3 pðz2 þ x2 Þ
2
:
ð10Þ
where h is the angle between line OB and the z-axis, rx is the tangential part of stress and rz is the normal part stress.
which relates the stresses to the strains. Based on Hooke’s law, the strains can be obtained as 1 ð1 t2 Þrx tð1 þ tÞrz E 1 ez ¼ ð1 t2 Þrz tð1 þ tÞrx E
ex ¼
ð13Þ ð14Þ
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
where E is the modulus of elasticity of the layer, and t is Poisson’s ratio. From Eqs. (11)–(14), the strains at the target point A can be derived as 1 2P tð1 þ tÞ ex ¼ ð15Þ E pd 2P 2 ðt 1Þ ð16Þ ez ¼ Epd where ex is the tangential strain at point A, and ez is the normal strain. In this pulse teleperception system, the normal strain plays a critical role in pulse feeling. Thus, only the normal strain ez is considered for the strain matching modeling. 3.2. Half-space model for strain matching The elastic models for both pulse sensing and pulse teleperception sides are built by applying the principle of the half-space model for the strain matching analysis. The pulse shall be considered as a narrow strip, or line loading, which produce stresses and deformations in an elastic halfspace. With the above results, the rubber layer model for the strain matching is built as shown in Fig. 10.
201
At the pulse sensing side, the surface concentrated load is the pulse load Fp. From Eq. (16) the strain e1 at pulse sensing side point A1 is derived as e1 ¼
2F p ðt2 1Þ Erub1 pd rub1 rub1
ð17Þ
where drub1 is the thickness of the rubber layer, Erub1 is the modulus of elasticity and trub1 is Poisson’s ratio. At the pulse teleperception (pulse display) side, the surface concentrated load is the actuator load Fa. From Eq. (16), the strain at pulse teleperception point A2 is derived as e2 ¼
2F a ðt2 1Þ Erub2 pd rub2 rub2
ð18Þ
where drub2 is the thickness of the rubber, Erub2 is the modulus of elasticity and trub2 is Poisson’s ratio. Comparing the elastic half-space model with the other linear and nonlinear viscoelastic models [30], one may find that the elastic half-space model does not consider the effects of creep and stress relaxation in rubber. Since we are dealing with pulse detection, where very small amount of load is applied with the finger, the effect of creep and stress relaxation is negligible. However, the viscoelastic models used in our previous work [28–30] will be included in our future studies in area of softness teletaction. 4. Computer simulation for strain matching A closed-loop strain matching system is designed to control the output strain ez=d2 to follow the input strain ez=d1. The signal flow model for the data processing is illustrated in Fig. 11. In order to validate the strain matching analysis, a system modeling and simulation was implemented by using the Matlab Simulink system. Fig. 12 shows the block diagram of the teletaction system including the mathematical models and transfer functions of the components. From Eqs. (17) and (18), the relationships between strains e1, e2 and pulse load Fp and Fa are denoted as 2F p ðt2 1Þ Erub1 pd rub1 rub1 2F a e2 ¼ h2 ðF a Þ ¼ ðt2 1Þ: Erub2 pd rub2 rub2 e1 ¼ h1 ðF p Þ ¼
Fig. 10. Cross-sectional view of elastic half-space model for concentrated normal force.
Strain ε z = d 1
Surface Stress Elastic Layer— Pulse Sensing
_
Surface Stress PID Controller
ð20Þ
From Eq. (5), the relationship between pulse load Fp and voltage Vpv1 are denoted as
δ
+
ð19Þ
Linear Actuator
Fig. 11. Rubber layer model for strain matching.
Strain ε z= d 2 Elastic Layer—Pulse Teleperception
202
J. Dargahi et al. / Mechatronics 18 (2008) 195–207 ε1
Fp Pulse
h ( Fp )
p1 (ε 1 )
δ
+
PID Controller
Table 2A Linear actuator (dyadic SCN-010-AS) parameters
u g (s )
_
Fa Strain Matching
ε2
p 2 (ε 2 )
Fig. 12. Block diagram of system modeling.
F p ¼ hðV pv1 Þ ¼
2V pv1 prpv1 ðg31 þ g32 Þ
Value
Unit
Stroke Max. thrust Load capacity Max. speed Rod diameter Weights Dimensions
0.1 100/10.2 5–15 0.4 0.016 1 21.5 7 4.5
m (N)/(kgf) N m/s m kg cm
ð21Þ
The actuator (stepping motor) model is identified as a second-order transfer function by using system identification with experimental data. The transfer function between the voltage input u and force output is expressed as gðsÞ ¼
Name
Fa 1 ¼ 2 ms þ cs þ k u
ð22Þ
where m is the mass of the actuator, c is damping coefficient and k is stiffness coefficient. By implementing the PID controller the output voltage Vo, which represents the output strain e2, can be controlled to match the input voltage Vin, which represents the input strain e1. In this simulation, the parameter values of the PID controller are obtained through trial-and-error. The detailed parameters for each block in the simulation are presented in Tables 1–5. The simulation result of the strain matching for the first 6 s is shown in Fig. 13. The root mean square error (RMSE) for the strain matching simulation during the first six seconds is vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N u1 X RMSEe1 ¼ t ð23Þ e2 ¼ 9:4249e 009 N i¼1 e1
Table 2B Stepping motor (Japan Servo Kh42KM2) parameters Name
Value
Unit
Drive method Number of phases Step angle Weights
Bi-polar 2 1.8 0.35
– – Deg./step kg
Table 3 Parameters of elastic (rubber) layer modeling Name
Description
Value
Unit
Erub drub1 drub2
Modulus of rubber layer Thickness of rubber layer at pulse sensing side Thickness of rubber layer at pulse teleperception side Poisson’s ratio
0.05 2 2
GPa mm mm
tpsn
0.5
Table 4 Parameters of actuator Name
Description
Value
Unit
m c k
Mass of actuator Damping coefficient Stiffness coefficient
0.65 0.61 4.23
kg N s/m N/m
Table 1 Parameters of PVDF modeling Name g31 g32 d31 d32 Epv rpv1 rpv2 dpv1 dpv2 kg31 kg32
Description Piezo coefficient in lateral direction (voltage mode) Piezo coefficient in transverse direction (voltage mode) Piezo coefficient in lateral direction (charge mode) Piezo coefficient in transverse direction (charge mode) Tensile and transverse modulus Contacting radius of PVDF film at pulse sensing side Contacting radius of PVDF film at pulse teleperception side Thickness of PVDF film at pulse sensing side Thickness of PVDF film at pulse teleperception side Coefficient for g31 Coefficient for g32
Value
Unit 1
Table 5 Parameters of PID controller in strain matching simulation (force feedback control experiment)
0.15
Vm N
2
pC N1
Name
Description
Value
18
pC N1
Proportional Integral Derivative
13 (15) 0.85 (0.15) 0.5 (0.03)
2
pC N1
Kp Ki Kd
2.2 8
Gpa mm
9
mm
0.0025
mm
0.0025
mm
1 1
where ee1 = e2 e1 is the error of strain between pulse teleperception and pulse sensing. The strain matching of the pulse signal during the first 3.8–4.7 s is shown in Fig. 14. The results demonstrate that the output strain from pulse teleperception matches the input strain from pulse sensing. Therefore, the mathematical elastic model and the strain matching control system are proved to be applicable for the teletaction system.
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
203
Fig. 13. System simulation result with strain matching (6 s).
Fig. 14. System simulation result with strain matching (1 s).
5. Experimental results In Section 2.1, Fig. 1 shows the system configuration for two experiments conducted in real-time in which two volunteers participated. At the pulse sensing side, one of the participants placed his wrist at the bottom base of the frame with the pulse sensor positioned at the tip of the probe measuring his pulse. At the pulse teleperception side, the other participant placed his fingertip on the top of the rubber of the pulse perception block to feel the pulse from the rubber layer placed at the tip of the actuator. 5.1. Human pulse sampling test The graph in Fig. 15 shows the experimental result of the pulse which was sampled by the PVDF probe sensor on one of the subjects from which it was determined that the heart rate was 75 beats per minute. The Welch power spectral density (PSD) estimation (shown in Fig. 16) for the pulse signal is obtained by MATLAB. From PSD analysis, the following conclusions are drawn:
(1) All the power spectra of the pulse signal are distributed in the range of 0–40 Hz. Most of the power spectra of the pulse signal are distributed within 20 Hz. (2) The spectral energy of the pulse within 20 Hz is about 99.7% of the total energy. Within 10 Hz it is about 89.4%, within 5 Hz is about 57.4%, within 1 Hz is about 12.4% and within 0.4 Hz is about 4.8%. This data illustrates that most of the spectral energy is distributed in the range of 1–10 Hz with only part of the spectral energy of the pulse signal being distributed below and above this range. Therefore, the obtained pulse did match the general condition of the actual human pulse [27]. In order to obtain a more detailed profile of the pulse, the sample taken from between 3.8 and 4.8 s is shown in Fig. 17. 5.2. Force feedback result The result of force control, together with the measured reference human pulse taken between 3.8 and 4.7 s, is shown in Fig. 17. The parameters of the PID controller
204
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
Fig. 15. Sampled human pulse (15 s).
Fig. 16. Power spectra of pulse signals for the sampled human pulse.
Fig. 17. Sampled human pulse (1 s).
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
205
Fig. 18. Sampled force control result (1 s).
are tuned by trial-and-error (refer to Table 5 for the PID parameters) and are close to the values which are used in the strain matching simulation. From the force control result illustrated in Fig. 18, the discrepancy and mismatching between the force from pulse sensing and teleperception can be observed clearly. The reason is that in the force feedback control system, the dynamic response of the linear actuator is not fast enough to track the input due to physical sluggishness of the actuator system. The RMSE for the force feedback control experiment during first 6 s is: vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N u1 X RMSEF ¼ t e2 ¼ 0:020863 N i¼1 F
ð24Þ
where eF = Fp Fa is the error between the force from pulse teleperception and pulse sensing.
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N u1 X e2 ¼ 5:7802e 008 RMSEe2 ¼ t N i¼1 e2
ð25Þ
where ee2 is the error between the strains from pulse teleperception and sensing (based on the force feedback control result). Remark: In the real-time experiment, the magnitude and speed of the stepping motor was controlled based on the input control signal u. The direction of the stepping motor is determined by a logical control program based on the stepping motor operating principle. In addition to the magnitude matching, the motor control system achieves the speed matching of strains as well. In this way, the pulse teletaction system produces the identical fingertip feeling at both pulse teleperception and pulse sensing sides. Fig. 20 shows the comparison between the computer simulation result and experimental result. The experimental output strain has relatively larger mismatch errors for the following reasons:
5.3. Experimental results of strain matching In the experiment, the measured voltages from both PVDF films in the pulse sensing and pulse teleperception sides represent the input pulse and output actuator load. Considering strain matching, the voltage signal is converted to a force signal by using Eq. (21). The strains at both sides can be obtained from Eqs. (19) and (20). As a result, the strain matching simulation based on the force control experiment is obtained by selecting the samples taken between 3.8 and 4.7 s, as shown in Fig. 19. The RMSE for the strain matching result during the first 6 s is
The slow response of system due to the inertia of each instrument. Environmental disturbances. Other random factors such as vibration, man-made errors etc. However, with the same input pulse and PID controller parameters, the results show close strain matching between the pulse sensing and teleperception. Therefore, the experimental system proved successful in sensing and reproducing the human pulse by force feedback control and strain matching.
206
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
Fig. 19. Sampled strain matching result (1 s).
Fig. 20. Strain matching result comparison (left – simulation result, right – experimental result).
5.4. Psychophysics experimental results
6. Conclusion
Based on the force control results and the strain matching simulation, a psychophysics experiment was conducted. When the system was running, six subjects placed their fingertips on the top surface of the rubber layer to feel the stimulation and compared the palpation feeling with the direct touching of human pulse on the wrist. The results indicate that 66% of subjects felt that it was close to directly touching the pulse itself and thereby demonstrated that the designed system was successful in achieving human pulse teletaction.
This paper details how two PVDF sensors were fabricated and calibrated for pulse detection and force measurement of pulse teleperception using a computer controlled force feedback system. A mathematical model was built for the rubber layer based on the half-space principle. Simulation and experimental tests verified the close matching between the sensed and perceived pulse as confirmed in the psychophysics results which indicated that the fingertip feeling at the pulse teleperception end is virtually identical to that of the actual pulse. In order to improve on these
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
results, however, an actuator with a faster response is required. One important application of the system is that it can be used for artery localization which is embedded within complex tissues in MIS. It might also eventually be applied in areas such as liver palpation. References [1] Howe RD. Tactile sensing and control of robotic manipulation. J Adv Robot 1994;8(3):245–61. [2] Howe RD, Peine WJ, Kontarinis DA, Son JS. Remote palpation technology. IEEE Eng Med Biol Mag 1995;14:318–23. [3] Fearing RS, Moy G, Tan E. Some basic issues in teletaction. In: IEEE international conference in robotics and automation, April 1997, vol. 4. p. 3093–9. [4] Moy G, Fearing R. Effect of shear stress in teletaction and human perception. In: 7th symposium on haptic interfaces for virtual environment and teleoperator systems. Proceedings ASME dynamic systems and control division 1998, DSC-vol 64. p. 265–72. [5] Kerpa O, Weiss K, Worn H. Development of a flexible tactile sensor system for a humanoid robot. In: Proceedings of IEEE international conference on intelligent robots systems October 2003, vol. 1. p. 1–6. [6] da Silva JG, De Carvalho AA, da Silva DD. A strain gauge tactile sensor for finger-mounted applications. IEEE Trans Instrum Meas 2002;51(1):18–22. [7] Dargahi J, Sedaghati R, Singh H, Najarian S. Modeling and testing of an endoscopic piezoelectric-based tactile sensor. Mechatronics 2007;17:462–7. [8] Feller RL, Lau CKL, Wagner CR, Perrin DP, Howe RD. The effect of force feedback on remote palpation. In: IEEE international conference on robotics and automation 2004, vol. 1. p. 782–8. [9] Ishii T, Yamamoto A, Higuchi T. Tactile display using thin type electrostatic linear actuator. Trans IEE Jpn 2002;122-E(10):474–9. [10] Howe RD, Peine WJ, Kontarinis DA, Son JS. Remote palpation technology or surgical applications. IEEE Eng Med Biol Mag 1995;14(3):318–23. [11] Feller RL, Lau CKL, Wagner CR, Perrin DP, Howe RD. The effect of force feedback on remote palpation. In: Robotics and automation 2004, vol. 1. p. 782–8. [12] Cohn MB, Lam M, Fearing R. Tactile feedback for teleoperation. In: ISPIE, telemanipulation technology 1993, vol. 1833. p. 240–54. [13] Voyles Jr RM, Fedder G, Khosla PK. Design of a modular tactile sensor and actuator based on an electrorheological gel. In: Proceedings of IEEE international conference on robotics and automation April 1996, vol. 1. p. 13–7. [14] Klein D, Freimuth H, Monkman GJ, Egersdo¨rfer S, Meier A, Bo¨se H, et al. Electrorheological tactel elements. Mechatron Sept 2005; 15(7):883–97.
207
[15] Pawluk DTV, van Buskirk CP, Killebrew JH, Hsiao SS, Johnson KO. Control and pattern specification for high density tactile array. In: Proceedings of the ASME dynamic systems and control division. New York, NY, USA; 1998. [16] Jungman M, Schlaak HF. Miniaturized electrostatic tactile display with high structural compliance. In: Proceeding of Eurohaptics. Edinburgh, UK; 2002. [17] Eltaib MEH, Hewit JR. Tactile sensing technology for minimal access surgery – a review. Mechatron Dec 2003;13(10):1163–77. [18] Zheng Xingyi, Lu Weixue, Yu Lie, Li Yuehua. A dexterous anthropomorphic finger for measuring human pulse. In: Singapore international conference on intelligent control and instrumentation Feb. 1992, vol. 1. p. 368–70. [19] Rosen CK. Piezoelectricity. American Institute of Physics 1992. [20] Phillips Charles L, Harbor Royce D. Feedback control systems. 4th ed. Upper Saddle River, NJ: Prentice Hall; 2000. [21] Paul Acarnely. Stepping motors: a guide to theory and practice. 2nd ed. 1984. [22] Kenjo Takashi. Stepping motors and their microprocessor controls. Oxford, New York: Clarendon Press, Oxford University Press; 1984. [23] Fung YC. Foundations of solid mechanics. Prentice Hall; 1965. [24] Pawluk DTV, Howe RD. Dynamic contact of the human fingerpad against a flat surface. ASME J Biomech Eng Dec 1999;121(6): 605–11. [25] Wu JZ, Dong RG, Smutz WP, Rakheja S, Schopp AW. Simulation of mechanical responses of fingertip to dynamic loading. Med Eng Phys 2004;24:253–64. [26] Johnson KL. Contact mechanics. Cambridge University Press; 1985. [27] Wang BH, Xiang Jinglin JL. Detecting system and power-spectral analysis of pulse signals of human body. In: Fourth international conference on signal processing, ICSP 1998, vol. 2. p. 1646–9. [28] Bonakdar A, Dargahi J, Bhat R. Investigations on the grasping contact analysis of biological tissues with applications in minimally invasive surgery. Am J Appl Sci 2007;4(12):1016–23. [29] Bonakdar A, Dargahi J, Bhat R. Grasping contact analysis of tissues with semi-cylindrical teeth grasper for minimally invasive surgery application. In Proceedings of ASME bioengineering conference, Keystone Colorado, 2007. [30] Bonakdar A, Dargahi J, Bhat R. Grasping contact analysis of viscoelastic materials with applications in minimally invasive surgery. In: Proceedings of ASME international mechanical engineering congress and exposition, Chicago, Illinois, 2006. [31] Tang KZ, Huang SN, Tan KK, Lee TH. Combined PID and adaptive nonlinear control for servo mechanical system. Mechatronics 2004;14:701–14.
Mechatronics 18 (2008) 195–207
An experimental teletaction system for sensing and teleperception of human pulse J. Dargahi, W.F. Xie *, Peng Ji Department of Mechanical and Industrial Engineering, Concordia University, 1455 de Maisonneuve Blvd. W. Montreal, Quebec, Canada H3G 1M8 Received 22 June 2006; accepted 4 December 2007
Abstract Teletaction is the sensing of a remote object by transmitting tactile information from a remote tactile sensor to an operator’s skin through tactile interface devices. A tactile interface is used to reproduce the information such as force (static and dynamic), texture, roughness, temperature, and shape. This paper describes the design, modeling, simulation, fabrication and testing of an experimental teletaction system to detect and reproduce the human pulse. It consists of three major components: pulse sensing system, pulse teleperception system and the data processing system. Two 25 lm thick polyvinylidene fluoride (PVDF) film sensors had been fabricated and calibrated for pulse detection and force measurement. A computer controlled force feedback system was developed for achieving pulse teleperception. The simulation and experimental results demonstrated the close matching between the sensed and perceived pulse. A psychophysics test showed that the fingertip feeling at the pulse teleperception end closely matched the feeling of touching a human pulse directly. Ó 2007 Elsevier Ltd. All rights reserved. Keywords: Teletaction; Tactile sensing; Tactile display; Pulse sensing; Pulse teleperception; Strain matching
1. Introduction The development of teletaction systems has become a hot issue in a number of research areas, including robotic assisted minimally invasive surgery (MIS) and tele-operated manipulation [1, 2]. Teletaction is the sensing of a remote object by transmitting cutaneous information from a remote tactile sensor to an operator’s skin (typically the fingertips) through tactile interface devices [3]. This information is important in applications such as tele-surgery, where the feel of the environment cannot be obtained by purely visual means. The tactile interface reproduces, as accurately as possible, the parameters such as force (static and dynamic), texture, roughness, temperature, and shape. The typical teletaction system comprises a tactile sensor, tactile display (the name typically given to a tactile feed*
Corresponding author. Tel.: +1 514 848 2424x4193; fax: +1 514 848 3175. E-mail address: [email protected] (W.F. Xie). 0957-4158/$ - see front matter Ó 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.mechatronics.2007.12.001
back device) and data processing unit. The tactile sensors interact with the environment and measure a contact force distribution. Tactile display systems are typically constructed with an array of actuators which deflect proportionally to strain on each sensing element of the tactile sensor. In a teletaction system, the sensed information is compared with the feedback from display devices and the comparison error is used as an input to the controller. The control system manipulates the tactile display in such a way that when an operator touches the tactile display with his/her finger, it can ideally provide the same tactile feeling as if the operator directly touches the object [4]. In this paper, we use pulse teleperception to represent tactile display in our pulse teletaction system. Most tactile sensors are based on various technologies such as piezoelectric, piezoresistive, strain gauging, optical, ultrasonic, electromagnetic and capacitive [5–7]. A typical tactile sensing system designed to detect force and pressure information has about 100 sensing elements in one square centimeter. This sensor array samples all sensing locations
196
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
in a short duration of time. Tactile sensors have numerous applications in robotics and medical areas. One of the medical applications is measuring and recording the human pulse. In recent years, a number of new methods of human pulse measurement and analysis have been developed such as the cardio-tachometer which measures and displays the pulse on a monitor. Piezoelectric technology is also used to measure the human pulse. An artificial finger, fitted with a PVDF tactile sensor array mounted on its tip, has been developed by Zheng et al. [18]. This device is able to record human pulse data with great accuracy. A typical pressure stimulation tactile display system has been described for the application of remote palpation tasks such as breast tumor localization or liver palpation [8]. A tactile display system, which recreates shape (pressure) profiles on the finger, has been designed using an array of mechanical pins actuated by commercially available radio controlled (RC) servomotors [8]. Another new type of tactile display, based on force feedback technique using an electrostatic linear actuator display system, was designed by Ishii et al. The tactile sensation is obtained from the shearing force generated on the fingertip by the electrostatic linear actuator [9]. Howe et al. [10,11] have developed a tactile shaped display for deployment on the handles of surgical instruments. They used shape memory alloy (SMA) as the actuation method because of its high power-to-weight and force-toweight ratios. However, SMA has nonlinear behavioral characteristics, contains high hysteresis properties and has a slow response time that poses a challenge to controller design. Cohn et al. [12] have developed a five by five tactile display of pneumatic actuators driven by solenoid valves. This tactile display has the advantage of economy as well as binary operation allowing for uniform open-loop response from actuator to actuator. Also, a continuously variable force output is possible using pulse-width modulation. Tactile displays using electro-rheological gel are also under investigation [13,14]. However, these types of tactile displays pose a possible safety hazard due to the relatively large voltages involved. There have also been tactile display designs utilizing actuators based on solenoids [15], and electrostatics [16]. A complete review of these tactile displays can be found in [16,17]. The combination of tactile sensors and tactile display systems can potentially be used in applications such as medical diagnostics, health monitoring and tele-operation manipulation. In this paper, an effective pulse sensing and teleperception system has been built to detect and reproduce a human pulse. The system can measure the human pulse and recreate the pulse sensation by using a linear actuator. A PVDF film is used for the pulse sensing and force feedback system due to its high reliability, low power consumption and low cost. It also has high sensitivity to a low level dynamic load such as a pulse. A linear actuator and an elastic layer are used for the pulse teleperception system. The advantages of stepping motors are
their low cost, high reliability and high torque at low speeds. Controlled by a force feedback control system, the linearly moving probe of the actuator produces a similar motion as the output of the pulse sensor. In addition, by integrating an elastomeric layer with the tip of an actuator, we are able to reproduce the pulse sensation. Therefore, when a human finger is in contact with the elastomer on the tip of the actuator, a similar sensation is perceived as when touching the pulse directly. Both simulation and experimental results demonstrate the success of the designed system in achieving human pulse teletaction. In this paper, Section 2 describes the system prototype and the working principles for the pulse sensing, pulse teleperception and tactile data processing. Section 3 builds the mathematical model for strain matching based on the elastic half-space principle. Section 4 details the system performance of strain matching in simulation. The experimental procedure and results are presented in Section 5 and the conclusion is drawn in Section 6. 2. System structure and working principle 2.1. System schematic As mentioned earlier, the objective of the proposed system is to replicate fingertip sensation of the human pulse by using a pulse teletaction system. Fig. 1 shows the complete system setup. The main components consist of a pulse sensing system, pulse teleperception system and tactile data processing unit. To produce identical fingertip feeling at the pulse teleperception side with the one at the sensing side, a strain matching method is proposed to achieve pulse perception. We use strain matching due to the fact that the outputs from both PVDF films in the pulse sensing and pulse teleperception sides change with the strain placed on the PVDF film [19]. The cutaneous mechanoreceptors in a human fingertip, however, may respond best to strain energy [4]. Fig. 2 illustrates the diagram of the system model. In this project, two PVDF film sensors have been fabricated for pulse detection and force measurement in the pulse sensing system. A linear actuator is controlled to apply load on a rubber layer block to produce the pulse sensation in the pulse teleperception system. The tactile data processing unit comprises a computer and an A/D converter which are used as a data processor and digital control system. The detailed design of each component in the system is illustrated in the following sections. 2.2. PVDF sensor design As shown in Fig. 3A, at the pulse sensing side, the essential component is the 2.5 micron thick metalized and poled PVDF film wrapped on the tip of the sensor probe consisting of plexi-glass and a rubber layer mounted between the probe tip and the PVDF layer. Consider the spatial impulse
J. Dargahi et al. / Mechatronics 18 (2008) 195–207 Charge Amplifier
197
Stepping Motor Drive
Pulse Sensing I/O Card
Computer Control System
Pulse Teleperception
Fig. 1. A photograph of the teletaction system setup.
Pulse Sensing System
Tactile Data Processing Computer & A/D Converter
Force Sensor
Feedback
+
Driving Electric Unit
Pulse Display System
Fig. 2. System diagram.
Probe
Rubber Layer PVDF Sensing Film Pulse Force Artery
Fig. 3A. PVDF sensor at pulse sensing.
response of the pulse teleperception system. It is very difficult to achieve stimulator density comparable to human mechanoreceptor density (on the order of 200 cm2), a rubber layer, which acts as a spatial low-pass filter, is essential [3]. In this way, the pulse teleperception system feels like touching the real pulse through a rubber layer. The rubber layer functions as low-pass filters to convert the surface stress to strain and the stress/strain model is discussed in detail in Section 3. At the pulse teleperception side, the same type of PVDF film is used in the force feedback system. As shown in Fig. 3B, PVDF film is mounted on the pulse teleperception block between the rubber layer and base block. The measured voltage produced by the PVDF film is used as a feedback signal in the force control system. The corresponding
198
J. Dargahi et al. / Mechatronics 18 (2008) 195–207 Rubber Layer
PVDF Force Sensor Plexi-glass Base Block
V pv ¼ ðk g31 g31 þ k g32 g32 Þrpv d pv
Fig. 3B. PVDF sensor at pulse teleperception.
Output Voltage from Sensor (mVolts)
by the probe, the stress is transmitted to the PVDF film laterally. The reason is that the PVDF sensors for both pulse sensing and teleperception are designed as membrane sensors. Thus the applied load in the thickness direction would be zero. Based on piezoelectric properties [19] for the voltage mode, the voltage output Vpv from PVDF film is expressed as
where g31 and g32 are the voltage mode piezo coefficients in the membrane lateral and transverse direction respectively (shown in Fig. 5), kg31 and kg32 are the coefficients for g31 and g32 respectively, rpv is the stress developed on the cross-sectional area of PVDF film and dpv is the thickness of the PVDF film. The parameters and specifications of the PVDF film are listed in Table 1. Since the PVDF film is biaxial, one has
PVDF Sensor Calibration 120 100 80 60 40 20
k g31 ¼ k g32 ¼ 1:
0 0
0.5
1
ð1Þ
1.5
2
2.5
Reference Loading (Newtons)
At the pulse sensing side, the stress rpv1 is obtained as rpv1 ¼
Fig. 4. PVDF sensor calibration.
calibration of the PVDF film at the pulse teleperception side is shown in Fig. 4. A similar linear calibration result was obtained for the PVDF film at the pulse sensing side. 2.3. PVDF sensor modeling At the pulse sensing side, the PVDF sensing film is mounted on the spherical tip of the sensing probe. At the pulse teleperception side, the area of contact between the spherical tip of the linear moving probe and the PVDF film is also circular in shape. Therefore, an assumption can be made that the effective load from pulse and actuator lies in the g31 and g32 direction and the effective loading area is the cross-sectional area contacting the PVDF membrane (shown in Fig. 5). When a load is applied to the PVDF film
ð2Þ
Fp Aload1
ð3Þ
where Fp is the pulse load on the cross-sectional area of the PVDF film and Aload1 is the effective loading area in PVDF film. Aload1 is written as Aload1 ¼ 2prpv1 d pv
ð4Þ
where rpv1 is the radius of the contacting area at the pulse sensing side. Therefore, from Eqs. (1)–(4), the pulse load can be expressed as Fp ¼
2V pv1 prpv1 : ðg31 þ g32 Þ
ð5Þ
At the pulse teleperception side, the stress rpv2 is obtained as rpv2 ¼
Fa Aload2
ð6Þ
g 33 (d 33 ) Normal to the PVDF Surface
g 32 (d 32 ) in Transverse Direction
g 31 (d 31 ) in Lateral Direction Contacting Area
Effective Loading Area
Fig. 5. Piezoelectric properties in PVDF sensing membrane.
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
where Fa is the actuator load on the cross-sectional area of the PVDF film and Aload2 is the cross-sectional area of PVDF. Aload2 is written as Aload2 ¼ 2prpv2 d pv
ð7Þ
where, rpv2 is the radius of the contacting area at the pulse teleperception side. Thus, from Eqs. (1), (2), (6) and (7), the pulse load can be expressed as Fa ¼
2V pv2 prpv2 : ðg31 þ g32 Þ
ð8Þ
Eqs. (5) and (8) are used in the strain matching validation in the later section. 2.4. Pulse teleperception and data processing As shown in Fig. 6, the pulse teleperception system comprises the linear actuator, pulse perception block (see Fig. 2 in Section 2.1) and the force feedback control system. In order to reproduce the human pulse, a closed-loop force feedback control system (Fig. 7) is designed to guarantee both amplitude and direction matching of strain so that identical fingertip feelings are obtained at both the pulse sensing and teleperception sides [20]. The tactile data processing detail is illustrated in Fig. 8. The linear actuator (Dyadic SCN-010-AS) for the pulse teleperception is driven by a two-phase hybrid stepping motor (detailed parameters for the linear actuator are shown in Tables 2A and 2B). As previously stated, the advantages of stepping motors are their low cost, high reliability and high torque at low speeds. A QUANSER MultiQ-PCI I/O card is applied to implement the data acquisition and A/D & D/A conversion. The human pulse force Fp is sampled by the PVDF sensor at the pulse sensing side. The output actuator force signal Fa is measured by another PVDF sensor at the pulse teleperception side. By using MATLAB SIMULINK, a real-time control system was built to control output force Fa to approach pulse force Fp. In response to pressure changes created by the applied human pulse, the PVDF produces high-impedance output charge signals [19]. Both signals at the pulse sensing and teleperception sides are converted to low-impedance voltage signals by the charge amplifier. A low-pass filter is used
Fig. 6. Pulse teleperception system setup.
199
to prevent unwanted 50/60 Hz AC line interference from entering the sensor. After being converted to a digital signal by an A/D converter in the I/O card, the two signals are fed to the comparator to generate the error signal e. The most commonly used PID controller is adopted to control step motor due to its simple structure [31]. The PID controller in the control system calculates this error e to produce a control signal u which is fed to the voltage-controlled-oscillator (VCO). By introducing the VCO in the control system [21, 22], the magnitude and speed of the stepping motor can be controlled based on the input control signal u. The stepping motor direction is determined by a logical control program based on the stepping motor operating principle. By choosing appropriate PID (proportional, integral and derivative) parameters, the output pulse load Fa tracks the reference input pulse load Fp in real-time. 3. Mathematical model for strain matching At the pulse sensing and teleperception sides, the rubber layers function as reconstruction filters to convert the surface stress to strain ez=d1 and ez=d2, respectively. It is critical to build a stress/strain model for the elastic layer that comes into contact with the fingertip. This specific modeling problem is related to contact mechanics where the theory of halfspace is widely employed when a probe or an indenter is pressed against a solid. According to the theory of elastic half-space model [26], when the elastic bodies make contact over an area whose dimensions are small compared with the radii of curvature of surfaces not deformed, the contact stresses are highly concentrated close to the contact region and decrease rapidly in intensity with distance from the point of contact. Provided the dimensions of the bodies are large compared to the dimensions of the contact area, the stresses in this region are neither critically dependent upon the shape of the bodies far from the contact area, nor upon the precise way in which they are supported. The stresses may be calculated with good approximation by considering each body as a semi-infinite elastic solid bounded by a plane surface (an elastic half-space). This ideal assumption, in which bodies having an arbitrary surface profile are regarded as semi-infinite in extent and having a plane surface, simplifies the boundary conditions. Hence the half-space theory is valid in the case where the strains remain small and the contact area is small compared with the dimension of the rubber layer. In this project, since the deflection of the rubber layer is small and there is no permanent deformation, the theory is appropriate for building the stress/strain models of rubber layers. The half-space theory can be extended to the linear and nonlinear viscoelastic models too, specifically when it is used to model rubber and biological issues [28]. In case of large deformation, the effects of large deformations and hyperelasticity should be considered [29]. However, these effects are negligible in this application and hence are ignored. Some dynamic contacting models for the strain matching have been developed to describe the accurate dynamic
200
J. Dargahi et al. / Mechatronics 18 (2008) 195–207 Pulse
Fp PVDF Sensor-- Pulse Sensing
e
u Controller
PVDF Sensor—Pulse Teleperceptron
Actuator
_
Fa
Force Feedback
Fig. 7. Force feedback control system.
Fp
Human Pulse signal
+
Charge AMP & Low Pass Filter A/D Converter
Force Sensor
Charge AMP & Low Pass Filter
Fa Step Motor Control Pulse
e _
Force Feedback
PID
VoltageControlledOscillator
Magnitude & Speed
u
D/A Converter
Stepping Motor Direction Control
Stepping Motor Direction Control
Step Motor Control Direction
Actuator
I/O Card
Computer Control Program
Fig. 8. Block diagram of data processing.
behavior of stress/strain of rubber. However, it is difficult to build such a dynamic contacting model for the strain matching in the system due to the complicated biomechanical characteristics of human fingertip of which two examples model its nonlinear viscoelasticity and relaxation [23– 25]. Therefore, without considering the fingertip factor, a simplified rubber layer model is built based on the elastic half-space model. 3.1. Principle of elastic half-space model In the reference frame shown in Fig. 9, the boundary surface is in the x–y plane and its z-axis is directed into the solid. The loaded strip lies parallel to the y-axis and has a width (r0 + r0) in the x-direction. The strip carries normal and tangential forces which are a function of x only. Let us assume that a state of plane strain (ey = 0) is produced in the half-space by the line loading. Therefore, in Fig. 9, d is the elastic layer thickness and P is the magnitude of the applied load acting in a direction normal to the surface of the rubber layer. Consider any point B (x, z) at the halfspace. The stresses along the x and z axes are denoted as
Fig. 9. Signal flow model for strain matching.
Consider A (0, d), which is the target point (pulse output point) for the project. The stresses acting along x and z can be represented as rx jA ¼ 0
ð11Þ
2P rz jA ¼ pd
ð12Þ
2
rx ¼ rr sin2 h ¼ rz ¼ rr cos2 h ¼
2Px z pðz2 þ x2 Þ
ð9Þ
2
2Pz3 pðz2 þ x2 Þ
2
:
ð10Þ
where h is the angle between line OB and the z-axis, rx is the tangential part of stress and rz is the normal part stress.
which relates the stresses to the strains. Based on Hooke’s law, the strains can be obtained as 1 ð1 t2 Þrx tð1 þ tÞrz E 1 ez ¼ ð1 t2 Þrz tð1 þ tÞrx E
ex ¼
ð13Þ ð14Þ
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
where E is the modulus of elasticity of the layer, and t is Poisson’s ratio. From Eqs. (11)–(14), the strains at the target point A can be derived as 1 2P tð1 þ tÞ ex ¼ ð15Þ E pd 2P 2 ðt 1Þ ð16Þ ez ¼ Epd where ex is the tangential strain at point A, and ez is the normal strain. In this pulse teleperception system, the normal strain plays a critical role in pulse feeling. Thus, only the normal strain ez is considered for the strain matching modeling. 3.2. Half-space model for strain matching The elastic models for both pulse sensing and pulse teleperception sides are built by applying the principle of the half-space model for the strain matching analysis. The pulse shall be considered as a narrow strip, or line loading, which produce stresses and deformations in an elastic halfspace. With the above results, the rubber layer model for the strain matching is built as shown in Fig. 10.
201
At the pulse sensing side, the surface concentrated load is the pulse load Fp. From Eq. (16) the strain e1 at pulse sensing side point A1 is derived as e1 ¼
2F p ðt2 1Þ Erub1 pd rub1 rub1
ð17Þ
where drub1 is the thickness of the rubber layer, Erub1 is the modulus of elasticity and trub1 is Poisson’s ratio. At the pulse teleperception (pulse display) side, the surface concentrated load is the actuator load Fa. From Eq. (16), the strain at pulse teleperception point A2 is derived as e2 ¼
2F a ðt2 1Þ Erub2 pd rub2 rub2
ð18Þ
where drub2 is the thickness of the rubber, Erub2 is the modulus of elasticity and trub2 is Poisson’s ratio. Comparing the elastic half-space model with the other linear and nonlinear viscoelastic models [30], one may find that the elastic half-space model does not consider the effects of creep and stress relaxation in rubber. Since we are dealing with pulse detection, where very small amount of load is applied with the finger, the effect of creep and stress relaxation is negligible. However, the viscoelastic models used in our previous work [28–30] will be included in our future studies in area of softness teletaction. 4. Computer simulation for strain matching A closed-loop strain matching system is designed to control the output strain ez=d2 to follow the input strain ez=d1. The signal flow model for the data processing is illustrated in Fig. 11. In order to validate the strain matching analysis, a system modeling and simulation was implemented by using the Matlab Simulink system. Fig. 12 shows the block diagram of the teletaction system including the mathematical models and transfer functions of the components. From Eqs. (17) and (18), the relationships between strains e1, e2 and pulse load Fp and Fa are denoted as 2F p ðt2 1Þ Erub1 pd rub1 rub1 2F a e2 ¼ h2 ðF a Þ ¼ ðt2 1Þ: Erub2 pd rub2 rub2 e1 ¼ h1 ðF p Þ ¼
Fig. 10. Cross-sectional view of elastic half-space model for concentrated normal force.
Strain ε z = d 1
Surface Stress Elastic Layer— Pulse Sensing
_
Surface Stress PID Controller
ð20Þ
From Eq. (5), the relationship between pulse load Fp and voltage Vpv1 are denoted as
δ
+
ð19Þ
Linear Actuator
Fig. 11. Rubber layer model for strain matching.
Strain ε z= d 2 Elastic Layer—Pulse Teleperception
202
J. Dargahi et al. / Mechatronics 18 (2008) 195–207 ε1
Fp Pulse
h ( Fp )
p1 (ε 1 )
δ
+
PID Controller
Table 2A Linear actuator (dyadic SCN-010-AS) parameters
u g (s )
_
Fa Strain Matching
ε2
p 2 (ε 2 )
Fig. 12. Block diagram of system modeling.
F p ¼ hðV pv1 Þ ¼
2V pv1 prpv1 ðg31 þ g32 Þ
Value
Unit
Stroke Max. thrust Load capacity Max. speed Rod diameter Weights Dimensions
0.1 100/10.2 5–15 0.4 0.016 1 21.5 7 4.5
m (N)/(kgf) N m/s m kg cm
ð21Þ
The actuator (stepping motor) model is identified as a second-order transfer function by using system identification with experimental data. The transfer function between the voltage input u and force output is expressed as gðsÞ ¼
Name
Fa 1 ¼ 2 ms þ cs þ k u
ð22Þ
where m is the mass of the actuator, c is damping coefficient and k is stiffness coefficient. By implementing the PID controller the output voltage Vo, which represents the output strain e2, can be controlled to match the input voltage Vin, which represents the input strain e1. In this simulation, the parameter values of the PID controller are obtained through trial-and-error. The detailed parameters for each block in the simulation are presented in Tables 1–5. The simulation result of the strain matching for the first 6 s is shown in Fig. 13. The root mean square error (RMSE) for the strain matching simulation during the first six seconds is vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N u1 X RMSEe1 ¼ t ð23Þ e2 ¼ 9:4249e 009 N i¼1 e1
Table 2B Stepping motor (Japan Servo Kh42KM2) parameters Name
Value
Unit
Drive method Number of phases Step angle Weights
Bi-polar 2 1.8 0.35
– – Deg./step kg
Table 3 Parameters of elastic (rubber) layer modeling Name
Description
Value
Unit
Erub drub1 drub2
Modulus of rubber layer Thickness of rubber layer at pulse sensing side Thickness of rubber layer at pulse teleperception side Poisson’s ratio
0.05 2 2
GPa mm mm
tpsn
0.5
Table 4 Parameters of actuator Name
Description
Value
Unit
m c k
Mass of actuator Damping coefficient Stiffness coefficient
0.65 0.61 4.23
kg N s/m N/m
Table 1 Parameters of PVDF modeling Name g31 g32 d31 d32 Epv rpv1 rpv2 dpv1 dpv2 kg31 kg32
Description Piezo coefficient in lateral direction (voltage mode) Piezo coefficient in transverse direction (voltage mode) Piezo coefficient in lateral direction (charge mode) Piezo coefficient in transverse direction (charge mode) Tensile and transverse modulus Contacting radius of PVDF film at pulse sensing side Contacting radius of PVDF film at pulse teleperception side Thickness of PVDF film at pulse sensing side Thickness of PVDF film at pulse teleperception side Coefficient for g31 Coefficient for g32
Value
Unit 1
Table 5 Parameters of PID controller in strain matching simulation (force feedback control experiment)
0.15
Vm N
2
pC N1
Name
Description
Value
18
pC N1
Proportional Integral Derivative
13 (15) 0.85 (0.15) 0.5 (0.03)
2
pC N1
Kp Ki Kd
2.2 8
Gpa mm
9
mm
0.0025
mm
0.0025
mm
1 1
where ee1 = e2 e1 is the error of strain between pulse teleperception and pulse sensing. The strain matching of the pulse signal during the first 3.8–4.7 s is shown in Fig. 14. The results demonstrate that the output strain from pulse teleperception matches the input strain from pulse sensing. Therefore, the mathematical elastic model and the strain matching control system are proved to be applicable for the teletaction system.
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
203
Fig. 13. System simulation result with strain matching (6 s).
Fig. 14. System simulation result with strain matching (1 s).
5. Experimental results In Section 2.1, Fig. 1 shows the system configuration for two experiments conducted in real-time in which two volunteers participated. At the pulse sensing side, one of the participants placed his wrist at the bottom base of the frame with the pulse sensor positioned at the tip of the probe measuring his pulse. At the pulse teleperception side, the other participant placed his fingertip on the top of the rubber of the pulse perception block to feel the pulse from the rubber layer placed at the tip of the actuator. 5.1. Human pulse sampling test The graph in Fig. 15 shows the experimental result of the pulse which was sampled by the PVDF probe sensor on one of the subjects from which it was determined that the heart rate was 75 beats per minute. The Welch power spectral density (PSD) estimation (shown in Fig. 16) for the pulse signal is obtained by MATLAB. From PSD analysis, the following conclusions are drawn:
(1) All the power spectra of the pulse signal are distributed in the range of 0–40 Hz. Most of the power spectra of the pulse signal are distributed within 20 Hz. (2) The spectral energy of the pulse within 20 Hz is about 99.7% of the total energy. Within 10 Hz it is about 89.4%, within 5 Hz is about 57.4%, within 1 Hz is about 12.4% and within 0.4 Hz is about 4.8%. This data illustrates that most of the spectral energy is distributed in the range of 1–10 Hz with only part of the spectral energy of the pulse signal being distributed below and above this range. Therefore, the obtained pulse did match the general condition of the actual human pulse [27]. In order to obtain a more detailed profile of the pulse, the sample taken from between 3.8 and 4.8 s is shown in Fig. 17. 5.2. Force feedback result The result of force control, together with the measured reference human pulse taken between 3.8 and 4.7 s, is shown in Fig. 17. The parameters of the PID controller
204
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
Fig. 15. Sampled human pulse (15 s).
Fig. 16. Power spectra of pulse signals for the sampled human pulse.
Fig. 17. Sampled human pulse (1 s).
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
205
Fig. 18. Sampled force control result (1 s).
are tuned by trial-and-error (refer to Table 5 for the PID parameters) and are close to the values which are used in the strain matching simulation. From the force control result illustrated in Fig. 18, the discrepancy and mismatching between the force from pulse sensing and teleperception can be observed clearly. The reason is that in the force feedback control system, the dynamic response of the linear actuator is not fast enough to track the input due to physical sluggishness of the actuator system. The RMSE for the force feedback control experiment during first 6 s is: vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N u1 X RMSEF ¼ t e2 ¼ 0:020863 N i¼1 F
ð24Þ
where eF = Fp Fa is the error between the force from pulse teleperception and pulse sensing.
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N u1 X e2 ¼ 5:7802e 008 RMSEe2 ¼ t N i¼1 e2
ð25Þ
where ee2 is the error between the strains from pulse teleperception and sensing (based on the force feedback control result). Remark: In the real-time experiment, the magnitude and speed of the stepping motor was controlled based on the input control signal u. The direction of the stepping motor is determined by a logical control program based on the stepping motor operating principle. In addition to the magnitude matching, the motor control system achieves the speed matching of strains as well. In this way, the pulse teletaction system produces the identical fingertip feeling at both pulse teleperception and pulse sensing sides. Fig. 20 shows the comparison between the computer simulation result and experimental result. The experimental output strain has relatively larger mismatch errors for the following reasons:
5.3. Experimental results of strain matching In the experiment, the measured voltages from both PVDF films in the pulse sensing and pulse teleperception sides represent the input pulse and output actuator load. Considering strain matching, the voltage signal is converted to a force signal by using Eq. (21). The strains at both sides can be obtained from Eqs. (19) and (20). As a result, the strain matching simulation based on the force control experiment is obtained by selecting the samples taken between 3.8 and 4.7 s, as shown in Fig. 19. The RMSE for the strain matching result during the first 6 s is
The slow response of system due to the inertia of each instrument. Environmental disturbances. Other random factors such as vibration, man-made errors etc. However, with the same input pulse and PID controller parameters, the results show close strain matching between the pulse sensing and teleperception. Therefore, the experimental system proved successful in sensing and reproducing the human pulse by force feedback control and strain matching.
206
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
Fig. 19. Sampled strain matching result (1 s).
Fig. 20. Strain matching result comparison (left – simulation result, right – experimental result).
5.4. Psychophysics experimental results
6. Conclusion
Based on the force control results and the strain matching simulation, a psychophysics experiment was conducted. When the system was running, six subjects placed their fingertips on the top surface of the rubber layer to feel the stimulation and compared the palpation feeling with the direct touching of human pulse on the wrist. The results indicate that 66% of subjects felt that it was close to directly touching the pulse itself and thereby demonstrated that the designed system was successful in achieving human pulse teletaction.
This paper details how two PVDF sensors were fabricated and calibrated for pulse detection and force measurement of pulse teleperception using a computer controlled force feedback system. A mathematical model was built for the rubber layer based on the half-space principle. Simulation and experimental tests verified the close matching between the sensed and perceived pulse as confirmed in the psychophysics results which indicated that the fingertip feeling at the pulse teleperception end is virtually identical to that of the actual pulse. In order to improve on these
J. Dargahi et al. / Mechatronics 18 (2008) 195–207
results, however, an actuator with a faster response is required. One important application of the system is that it can be used for artery localization which is embedded within complex tissues in MIS. It might also eventually be applied in areas such as liver palpation. References [1] Howe RD. Tactile sensing and control of robotic manipulation. J Adv Robot 1994;8(3):245–61. [2] Howe RD, Peine WJ, Kontarinis DA, Son JS. Remote palpation technology. IEEE Eng Med Biol Mag 1995;14:318–23. [3] Fearing RS, Moy G, Tan E. Some basic issues in teletaction. In: IEEE international conference in robotics and automation, April 1997, vol. 4. p. 3093–9. [4] Moy G, Fearing R. Effect of shear stress in teletaction and human perception. In: 7th symposium on haptic interfaces for virtual environment and teleoperator systems. Proceedings ASME dynamic systems and control division 1998, DSC-vol 64. p. 265–72. [5] Kerpa O, Weiss K, Worn H. Development of a flexible tactile sensor system for a humanoid robot. In: Proceedings of IEEE international conference on intelligent robots systems October 2003, vol. 1. p. 1–6. [6] da Silva JG, De Carvalho AA, da Silva DD. A strain gauge tactile sensor for finger-mounted applications. IEEE Trans Instrum Meas 2002;51(1):18–22. [7] Dargahi J, Sedaghati R, Singh H, Najarian S. Modeling and testing of an endoscopic piezoelectric-based tactile sensor. Mechatronics 2007;17:462–7. [8] Feller RL, Lau CKL, Wagner CR, Perrin DP, Howe RD. The effect of force feedback on remote palpation. In: IEEE international conference on robotics and automation 2004, vol. 1. p. 782–8. [9] Ishii T, Yamamoto A, Higuchi T. Tactile display using thin type electrostatic linear actuator. Trans IEE Jpn 2002;122-E(10):474–9. [10] Howe RD, Peine WJ, Kontarinis DA, Son JS. Remote palpation technology or surgical applications. IEEE Eng Med Biol Mag 1995;14(3):318–23. [11] Feller RL, Lau CKL, Wagner CR, Perrin DP, Howe RD. The effect of force feedback on remote palpation. In: Robotics and automation 2004, vol. 1. p. 782–8. [12] Cohn MB, Lam M, Fearing R. Tactile feedback for teleoperation. In: ISPIE, telemanipulation technology 1993, vol. 1833. p. 240–54. [13] Voyles Jr RM, Fedder G, Khosla PK. Design of a modular tactile sensor and actuator based on an electrorheological gel. In: Proceedings of IEEE international conference on robotics and automation April 1996, vol. 1. p. 13–7. [14] Klein D, Freimuth H, Monkman GJ, Egersdo¨rfer S, Meier A, Bo¨se H, et al. Electrorheological tactel elements. Mechatron Sept 2005; 15(7):883–97.
207
[15] Pawluk DTV, van Buskirk CP, Killebrew JH, Hsiao SS, Johnson KO. Control and pattern specification for high density tactile array. In: Proceedings of the ASME dynamic systems and control division. New York, NY, USA; 1998. [16] Jungman M, Schlaak HF. Miniaturized electrostatic tactile display with high structural compliance. In: Proceeding of Eurohaptics. Edinburgh, UK; 2002. [17] Eltaib MEH, Hewit JR. Tactile sensing technology for minimal access surgery – a review. Mechatron Dec 2003;13(10):1163–77. [18] Zheng Xingyi, Lu Weixue, Yu Lie, Li Yuehua. A dexterous anthropomorphic finger for measuring human pulse. In: Singapore international conference on intelligent control and instrumentation Feb. 1992, vol. 1. p. 368–70. [19] Rosen CK. Piezoelectricity. American Institute of Physics 1992. [20] Phillips Charles L, Harbor Royce D. Feedback control systems. 4th ed. Upper Saddle River, NJ: Prentice Hall; 2000. [21] Paul Acarnely. Stepping motors: a guide to theory and practice. 2nd ed. 1984. [22] Kenjo Takashi. Stepping motors and their microprocessor controls. Oxford, New York: Clarendon Press, Oxford University Press; 1984. [23] Fung YC. Foundations of solid mechanics. Prentice Hall; 1965. [24] Pawluk DTV, Howe RD. Dynamic contact of the human fingerpad against a flat surface. ASME J Biomech Eng Dec 1999;121(6): 605–11. [25] Wu JZ, Dong RG, Smutz WP, Rakheja S, Schopp AW. Simulation of mechanical responses of fingertip to dynamic loading. Med Eng Phys 2004;24:253–64. [26] Johnson KL. Contact mechanics. Cambridge University Press; 1985. [27] Wang BH, Xiang Jinglin JL. Detecting system and power-spectral analysis of pulse signals of human body. In: Fourth international conference on signal processing, ICSP 1998, vol. 2. p. 1646–9. [28] Bonakdar A, Dargahi J, Bhat R. Investigations on the grasping contact analysis of biological tissues with applications in minimally invasive surgery. Am J Appl Sci 2007;4(12):1016–23. [29] Bonakdar A, Dargahi J, Bhat R. Grasping contact analysis of tissues with semi-cylindrical teeth grasper for minimally invasive surgery application. In Proceedings of ASME bioengineering conference, Keystone Colorado, 2007. [30] Bonakdar A, Dargahi J, Bhat R. Grasping contact analysis of viscoelastic materials with applications in minimally invasive surgery. In: Proceedings of ASME international mechanical engineering congress and exposition, Chicago, Illinois, 2006. [31] Tang KZ, Huang SN, Tan KK, Lee TH. Combined PID and adaptive nonlinear control for servo mechanical system. Mechatronics 2004;14:701–14.