- Email: [email protected]
An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance
Computers and Electrical Engineering xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Computers and Electrical Engineering journal homepage: www.elsevier.com/locate/compeleceng
An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance q Yaser Maddahi, Kourosh Zareinia, Nariman Sepehri ⇑ Department of Mechanical Engineering, The University of Manitoba, Winnipeg, MB R3T5V6, Canada
a r t i c l e
i n f o
Article history: Available online xxxx
a b s t r a c t Virtual fixtures can be used in haptic-enabled hydraulic telemanipulators to facilitate certain tasks. Using this concept, however, the operator may tend to move the master fast due to relying on the virtual fixture. As a result, the slave manipulator could start to lag due to latency in the hydraulic actuation control system. This paper describes how to mitigate the position errors between master and slave robots by overlaying an augmentation force on the master that is collinear but opposite of the master instantaneous velocity. The magnitude of this force is proportional to the position error at the slave end-effector. Experiments, conducted on a teleoperated hydraulic manipulator to perform several live-line maintenance tasks, show that the augmented scheme exhibits less position error at the slave side, better task quality, but longer task completion time as compared to the virtual fixture alone. Ó 2014 Elsevier Ltd. All rights reserved.
1. Introduction Human-operated machines, equipped with hydraulic manipulators, are widely used in industry [1]. Examples include excavators and underwater manipulators [2–4]. These machines are remotely controlled by human, which extends their ability to perform tasks, especially when the environment is unsafe. A human-controlled manipulator system is generally composed of master side where an operator utilizes a hand-controller, slave side where a manipulator emulates behavior of the hand-controller, a communication channel connecting slave and master sides and, a feedback control system. The feedback system can be built upon the operator’s sensation about the slave site (telepresence) [5]. When the assigned task implies a contact with the environment (force tracking), or the slave needs to follow a particular trajectory (position tracking), the use of haptic sensation can be helpful to enhance operator’s performance [6–11]. For example, when teleoperated manipulators are used for repetitive tasks, a virtual fixture force can be applied to the operator’s hand to enhance task performances [12]. Within this context, Kang et al. [13] used the concept of virtual fixtures to provide passive constraint to the operator’s motion. They found that virtual fixtures could improve accuracy and task completion time for performing decontaminating and decommissioning. Abbott et al. [14] discussed design of two types of virtual fixtures: ‘guidance virtual fixtures’ (GVFs) that help the operator move the haptic implement along a pre-defined trajectory and ‘forbidden-region virtual fixtures’ (FRVFs), which prevent the haptic implement from penetrating into forbidden regions (defined at the master or the slave manipulator workspace). Marayong et al. [15] employed vision-based virtual fixtures to provide different levels of guidance to the operator. In this work, the complete guidance offered the best improvement
q
Reviews processed and proposed for publication to the Editor-in-Chief by Guest Editor Dr. Ankit Chaudhary.
⇑ Corresponding author.
E-mail address: [email protected] (N. Sepehri). http://dx.doi.org/10.1016/j.compeleceng.2014.07.006 0045-7906/Ó 2014 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
2
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
in both execution time and error reduction in tasks involving general path-following. Eduardo et al. [16] implemented the concept of impedance-type FRVFs to help operators keep the manipulator out of a defined forbidden region of the workspace. A FRVF was also implemented in a latest study on a teleoperated hydraulic manipulator, performing live-line maintenance tasks [17]. Results of qualitative evaluations by six hydro linemen showed that the concept of virtual fixtures was simple to use, reduced the physical load, and did not require long training. In the above studies, the haptic force was produced based on the information at the master site, and no feedback information was obtained from the slave manipulator. Although the virtual fixture force keeps the operator’s hand on the defined virtual path, an accurate motion at the slave side cannot be guaranteed. For instance, the position tracking of the slave manipulator can easily be violated due to fast motion of the operator’s hand at the master side reflecting the mismatch between the master dynamics and the dynamics of the slave. Particularly, in hydraulic telemanipulators, since hydraulic actuators exhibit significant nonlinear characteristics [18], the requirement of position tracking is generally more challenging than that of their electromechanical counterparts. Thus, in order to mitigate the position errors between the master implement and the slave end-effector, we propose the addition of another force, which is proportional to the magnitude of the position error at the slave end-effector, but in the direction opposite to the operator’s hand velocity vector at the master implement. When position error at the slave end-effector is evident, the augmentation force is activated alerting the operator to slow down the hand movement. Using this scheme, the combined virtual fixture and augmentation force reduces position errors at both the master device implement and the slave manipulator end-effector. Note that the concept of using position error, in slave manipulators, to provide a haptic sensation is not new and has been implemented in few research studies with different goals [19,20]. Abbott and Okamura [21] compared several FRVFs on four common telemanipulator control architectures. A single degree-of-freedom teleoperated system was used to simulate working near a known forbidden region. One of the evaluations relates to combining the position error at the slave manipulator with the slave interaction force with the environment, to produce a haptic force. Kontz et al. [22] proposed a strategy that commands the haptic device to generate a slave position-referenced force that couples the motion of a haptic device to the excavator bucket movement. Hayn and Schwarzmann [23] implemented a slave position-referenced force for the operation of a hydraulic excavator, whereby the positions of the haptic device (master) and the hydraulic manipulator (slave) were used to provide reference positions for each other. The utilization of the conventional slave position-referenced force concept in the above studies, results in producing a haptic force which is parallel to, and in proportion with, the position error vector at the slave end-effector. Indeed, this force indicates to the operator whether the slave is moving ahead or behind the intended desired trajectory. However, it does not serve to slow down the hand’s motion. In this paper, we propose to redirect the force generated based on the magnitude of the slave position error, to be parallel with, but in opposite direction of, the operator’s hand instantaneous velocity. The intention is to slow down the operator’s hand, thereby reducing the following (tracking) error caused by the mismatch between the dynamics of the master device and the slave manipulator. We will evaluate the performance of this new concept when combined with virtual fixtures in a live-line maintenance application. The rest of this paper is organized as follows. The experimental setup and coordinate mapping are described in Section 2. This section allows the readers to understand the system and maintenance tasks, investigated in this paper. The description of, and need for augmenting the currently used virtual fixtures concept is described by presenting preliminary experimental results in Section 3. Effectiveness of the application of the proposed concept is then evaluated in Section 4, by emulating several live transmission line maintenance tasks. The concept is further compared with the conventional slave positionreferenced force mode. Evaluation criteria used are position error at the master implement, and position error at the slave end-effector. Conclusions are provided in Section 5.
2. Experimental setup Fig. 1 shows the test rig developed to examine the performance of proposed augmented virtual fixture scheme. This system comprises an industrial hydraulic manipulator (slave side), and a widely used the PHANToM Desktop haptic device (master side) that allows the operator to control the manipulator end-effector trajectory [24]. The first four degrees of freedom (DOFs) of this manipulator are used to perform live-line maintenance tasks, and the last two DOFs are deactivated. The active motions are rotations about arm (hs1 ), shoulder (hs2 ), main elbow (hs3 ) and extended elbow (hs4 ) axes (Figs. 2a and 3a) P se ) emulates motion of the haptic device implement (~ Pm [24]. The slave manipulator end-effector (~ i ). A typical live-line maintenance task, designed to loosen or tighten a nut, is depicted in Fig. 1c. This test rig has been constructed for an ongoing research that aims to employ robots to work cooperatively with linemen at service interruption free maintenance and inspection of live transmission lines. Fig. 2a shows coordinate frames of the hydraulic manipulator. The PHANToM Desktop haptic device is shown in Fig. 2b. Superscripts ‘s’ and ‘m’ indicate the parameter belongs to manipulator (slave) or haptic device (master), respectively. Subm m s s s scripts ‘e’ and ‘i’ stand for the end-effector and implement, respectively. Frames fxm i yi zi g and fxe ye ze g are the coordinate systems attached to the master device implement and the slave manipulator end-effector, respectively. The fixed (global) coordinate system is denoted by fxo yo zo g. T Now let the position vector of manipulator end-effector be ~ P se ¼ ½ xse yse zse . Using the inverse kinematics solution, the s s s s angular displacements of the slave manipulator joints (h1 ;h2 ;h3 ;h4 ) are determined as follows: Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
3
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Fig. 1. (a) Kodiak 1000 hydraulic manipulator; (b) the PHANToM Desktop haptic device; and (c) typical live-line maintenance task.
Fig. 2. Coordinate frames of (a) the slave manipulator, and (b) the PHANToM Desktop haptic device.
hs1 ¼ tan1
s ye xse
ð1Þ
0qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1 2 2 2 k1 þ k2 k3 k 2 A þ tan1 @ hs2 ¼ tan1 k1 k3
hs3
1
¼ tan
2l2 k2 c2 2l2 k1 s2 2
2
2
ð2Þ
! ð3Þ
2
k1 þ k2 l2 l3
hs4 ¼ hs2 hs3
ð4Þ 2
2
2
2
where ci ¼ cosðhsi Þ and si ¼ sinðhsi Þ. Note that k1 ¼ xse c1 þ yse s1 l1 l4 , k2 ¼ zse , and k3 ¼ ðk1 þ k2 þ l2 l3 Þ=ð2l2 Þ. Note that in order to allow the manipulator to move along given trajectories, the tool located at the end-effector must remain horizontal, which results in Eq. (4). Furthermore, due to the square root function in Eq. (2), the inverse kinematics solution produces two solution sets. The acceptable one is chosen according to the joint workspaces. Table 1 shows the workspace of each joint of Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
4
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Fig. 3. Virtual fixture concept.
Table 1 Range of joint motions for Kodiak 1000. Joint
Lower limit
Upper limit
Arm, hs1 Shoulder, hs2 Main elbow, hs3 Extended elbow, hs4
12 0 48 90
43 90 180 30
the slave manipulator with respect to the home position. Home position is where the shoulder, main elbow and extended elbow links are horizontal. With reference to Fig. 2, to control manipulator’s end-effector, the coordinated-mode mapping system is chosen, in which m m s s s the haptic implement position, fxm i yi zi g, is mapped into the manipulator end-effector position, fxe ye ze g. As the haptic device m ~ moves, its position vector (P ) is continuously recorded, and scaled by a factor (ks). This gives the desired position of the a
P sd : manipulator end-effector, ~
~ Psd ¼ ks~ Pm a
ð5Þ
Psd ¼ xse;d where ~
T
P sd , the vector of joint angular displacements is . Given desired vector~ s ~ calculated using Eqs. (1)–(4). On the other hand, the actual vector P is obtained by substituting actual joint angular displaceyse;d
zse;d
xm Pm and ~ i;a a ¼
T
ym i;a
zm i;a
a
ments, measured by Hall Effect sensors, hsa , and using forward kinematic equations of the manipulator. In practice, the desired and actual values of joint angular displacements are not necessarily equal due to limitations of the control scheme Es ) appears at the slave end-effector, used, and difference between responsiveness of each link. Therefore, the position error (~ e
which is defined as:
2
ese;x
3
2
xse;a xse;d
3
2
xse;a ks xm i;a
3
6 7 6 7 6 7 ~ Ese ¼ 4 ese;y 5 ¼ 4 yse;a yse;d 5 ¼ 4 yse;a ks ym i;a 5 s s s s m ze;a ze;d ee;z ze;a ks zi;a
ð6Þ
~ Ese will later be used to define augmented virtual fixture force. 3. Augmented virtual fixture 3.1. Impedance-type virtual fixture and observation F VF , is proportional to the A typical impedance type virtual fixture is depicted in Fig. 3. As shown, the generated force, ~ Em error between the actual (point A) and desired (point D) positions of the haptic implement, ~ [14,17]. i
~ F VF ¼ GVF~ Em i
ð7Þ
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
5
m m m m Fig. 4. Desired (Om d ) and actual (Oa ) positions of the haptic implement. Position error at the implement is expressed by (ei;x ; ei;y , ei;z ) in coordinate frame {x0 m y0 z0 }. Om d and Oa represent point D and point A, in Fig. 3, respectively.
where GVF is called the virtual fixture impedance matrix, which represents the ability of virtual fixture in providing different levels of fixture compliance. The higher the values of diagonal terms in this matrix, the more force is generated to keep the operator’s hand on the intended trajectory. The haptic implement position error (~ Em i ) is the difference between the actual and desired positions (see Fig. 4):
2
3 m xm i;a xi;d 6 em 7 6 ym ym 7 ~ Em i;d 5 i ¼ 4 i;y 5 ¼ 4 i;a m m z z em i;a i;d i;z em i;x
3
2
ð8Þ
The virtual fixture is employed in unilateral mode, where there is no feedback from the slave side [17]. This scheme helps the operator keep the hand along the virtual fixture path, and corrects motion of the master device implement [24]. However, this is not adequate to ensure that position error at the slave manipulator end-effector maintains low. Since the virtual fixture scheme provides no feedback from these sources, even if the operator moves closely along the virtual fixture path, the accurate motion of the slave manipulator cannot be guaranteed. The slave end-effector errors can also originate from other sources such as speed of the operator’s hand motion along tangential trajectory at which no restriction in motion is applied by the virtual fixture [14,17]. Preliminary experiments were performed that show how the operator’s hand speed affects the position error at the slave end-effector as well as the haptic implement. The tests were conducted on the test rig shown in Fig. 1. In these tests, an operator was asked to move the haptic implement along a circular trajectory (see Fig. 1c) with four different speeds, which resulted in different task completion times. The measured times were: 30.58 s, 22.50 s, 13.52 s and 8.77 s. For each speed, the position errors at the master implement and the slave end-effector were recorded and compared. The impedance matrix of the virtual fixture, GVF, was set to GVF ¼ diag½1500 N=m 1500 N=m 1500 N=m. Fig. 5a and b shows position error at the haptic implement and slave end-effector, respectively, under different hand speeds and along the circular path shown in Fig. 1c. As observed, by increasing the hand speed, no obvious changes on the value of position error at the haptic device can be noticed, i.e. the ranges of position error are almost similar. The maximum position error was about 1.7 mm, which indicates the virtual fixture was able to keep the operator’s hand close to the desired track. However, the position error at the manipulator end-effector significantly increased beyond the error inherently originated from the controller response (see Fig. 5b). 3.2. Augmenting the virtual fixture The solution to reduce position error at the slave end-effector, proposed in this paper, is to add an augmentation force to the virtual fixture force. The augmentation force alerts the operator to slow down the haptic implement (hand) motion when position error becomes larger than the accuracy expected from the controller at the slave end-effector. The direction of this force should be apposite to the haptic implement (hand) instantaneous velocity. Indeed, this force allows the operator to realize position error at the slave side. The augmentation force is defined as:
~ F AU ¼
8 m > ~s ~s > < GAU Ee Ethreshold v^ ins > > :
0
~s ~s
Ee > Ethreshold
!s s
E 6 ~ Ethreshold
e
ð9Þ
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
6
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
(a) Haptic implement position error
1.5
1
0.5
25
1.5 1 0.5
40 20
5
15
10
20
30
25
Time (s)
5
10
15
Time (s)
80 60 40 20 0 0
20
5
10
15
20
Time (s) 100
2
1.5
1
0.5
2
4
6
Time (s)
8
10
80 60 40 20 0 0
12
2
4
6
8
10
12
Time (s) 100
Slave Position Error (mm)
2
1.5
1
0.5
0 0
60
0 0
30
Slave Position Error (mm)
Master Position Error (mm)
20
80
100
0 0
Master Position Error (mm)
15
Time (s)
2
0 0
Task time=13.52 s
10
5
Slave Position Error (mm)
Master Position Error (mm)
Task time=22.50 s
0 0
Task time=8.77 s
(b) Slave end-effector position error 100
2
Slave Position Error (mm)
Master Position Error (mm)
Task time=30.58 s
Task
1
2
3
4
5
Time (s)
6
7
8
80 60 40 20 0 0
1
2
3
4
5
6
7
8
Time (s)
Fig. 5. Position error at the haptic implement and manipulator end-effector, given the same task as in Fig. 1c and under different speeds.
Ese is the vector of position error at the manipulator end-effector, and is expressed by three components As shown in Fig. 6, ~ in frame fx0 y0 z0 g. Note that due to inherent error of the controller, the augmentation force is only generated when the position error is greater than a threshold. ~ Esthreshold is defined based on the steady-state positioning error at the slave end-effector
s ~s
Ee 6 Ethreshold , the haptic that originates from the manipulator’s controller as well as the resolution of the sensors. When ~
s ~s
Ee > Ethreshold , ~ F AU is proportional to ~ Ese ~ Esthreshold in terms of device does not produce any augmentation force. When ~ ^m magnitude, and parallel to v ins , which is the unit vector of haptic implement instantaneous velocity. The negative sign indicates the augmentation force is in the opposite direction of the hand instantaneous velocity. GAU is a diagonal matrix. The threshold is considered to prevent repeated activation-deactivation cycles. Fig. 7 shows how the augmented virtual fixture force is calculated using the virtual fixture and the augmentation forces. F VF ) pulls the operator’s hand toward the haptic desired path, the As illustrated in Fig. 7, while the virtual fixture force (~
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
7
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Fig. 6. Desired (Osd ) and actual (Osa ) positions of the slave manipulator end-effector. Position error at the end-effector is expressed by (ese;x , ese;y , ese;z ) in coordinate frame {xs0 ys0 zs0 }.
Fig. 7. Augmented virtual fixture scheme.
F AU ) slows down the operator’s hand motion. The augmented virtual fixture force (~ F AVF ) is then calcuaugmentation force (~ lated as:
~ F AVF ¼ qVF~ F VF þ qAU~ F AU
ð10Þ
where qVF and qAU are positive weighting factors to adjust the relative effect of virtual fixture and augmentation forces, respectively (qVF þ qAU ¼ 1). The block diagram of the augmented virtual fixture scheme is illustrated in Fig. 8. As observed, the actual position vector Pm ) is multiplied by scalar ks to indicate the desired position of the slave manipulator end-effector. of the haptic implement (~ a
Pm The actual position of the haptic implement (~ a ) is continuously recorded to determine position error at the haptic implem m ~ ~ F VF ). On the slave ment, E . Next, E is multiplied by the master impedance, GVF, in order to calculate the virtual fixture force (~ i
i
side, the actual position of the haptic device is multiplied by the scaling factor, ks, in order to calculate the desired position of P s . The actual position of the slave manipulator end-effector, ~ Ps , is calculated by substituting the manipulator end-effector, ~ d
a
hsa ) into forward kinematic equations of the manipulator. Having manipulator the actual angular displacements of joints (~ end-effector position error vector, the augmentation force, ~ F AU , is calculated using Eq. (9). The combined virtual fixture and augmentation forces form the augmented virtual fixture force. 4. Experimental results and discussion 4.1. Test procedure Two sets of experiments are presented in which eight operators were asked to perform maintenance tasks, using the PHANToM Desktop haptic device. Four tasks were performed using the experimental setup. They were: pulling out a cotter pin (Task A, Fig. 9a), loosening or tightening a nut (Task B, Fig. 9b), connecting or disconnecting a ball and socket joint (Task Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
8
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Fig. 8. Block diagram of augmented virtual fixture scheme.
(a) Task A: Pulling the cotter pin (line)
(b) Task B: Tightening the nut (horizontal circle)
end-effector (c) Task C: Connecting the joint (square)
(d) Task D: Twisting tie-wire (vertical circle)
Fig. 9. Typical maintenance tasks and corresponding expected paths to complete each task.
C, Fig. 9c) and, twisting tie wire (Task D, Fig. 9d). Fig. 9 shows virtual fixture paths along which the operators were asked to move the hydraulic manipulator end-effector. The experiments were first conducted using the virtual fixture force only (qVF = 1, qAU = 0). Next, the augmented virtual fixture force was applied (qVF = 0.5, qAU = 0.5). The impedance matrix of the force augmentation scheme was a diagonal matrix, GAU ¼ diag½150 N=m150 N=m150 N=m, the values of its elements were determined to give a feeling of an unsaturated Em has its maximum value during the test. Effectiveness of each mode was evaluated using two indices: (i) posiforce when ~ i
tion error at the haptic device implement, and (ii) position error at the manipulator end-effector. The operators were asked to hold stylus of the haptic device like a pen, and try to trace each path. The experiments were repeated 10 times by each operator, for each force mode. A total number of 2(force modes) 4(tasks) 10(trials) Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
9
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
8(participants) = 640 trials were collected. Although the operators were asked to repeat particular tasks, some deviations from trajectories were unavoidable. The nonlinear PI (NPI) controller, developed earlier [25], was employed to control each joint of the manipulator. This controller produced maximum steady-state error reflecting a threshold of 1.54 mm, 1.06 mm and 2.07 mm at the end-effector along x0 , y0 and z0 , respectively; thus, ~ Esthreshold ¼ ð1:54 mm;1:06 mm;2:07 mmÞT . 4.2. Results Fig. 10 illustrates typical trajectories of slave manipulator end-effector for all four tasks and under the two force modes. This figure shows how accurate the operator performs the tasks. As observed, there exist some deviations from the desired path in slave end-effector trajectory which result in position error. By comparing trajectories depicted in each row, it is seen that the position error at the slave end-effector, using the virtual fixture scheme only, is larger than the augmented virtual fixture scheme. For instance, comparing Fig. 10c and e, it is seen that the operator could not follow closely given circular and rectangular trajectories using the virtual fixture only. Likewise, the manipulator end-effector, in Fig. 10g, was not properly guided along the desired vertical curve. The values of task completion time indicate that, in augmented virtual fixture, the operator guided the slave end-effector slower than the virtual fixture mode to match its dynamics to dynamics of the manipulator. Note that these typical trajectories were randomly chosen from 640 trial runs. The quality of slave end-effector trajectory could change from an operator to another one; but, similar observation can be obtained for each test. As mentioned, the augmented force is zero if the magnitude of the position error at the slave end-effector is less than the controller inherent error (~ Esthreshold ). Otherwise, once a position error is greater than ~ Esthreshold appears, the augmentation force
Virtual fixture
Augmented virtual fixture 150 ze (mm)
150
145
s
145
s
ze (mm)
A: Pulling out a Cotter pin
Task
140 140
100 yse (mm)
140 140
260
120
80
60 220
260
120
100 yse (mm)
240 xse (mm)
80
(b) time=10.29 s 100
90 85
90
s
s
ze (mm)
95
ze (mm)
B: Loosening or tightening a nut
(a) time=4.97 s
80 200 150 yse (mm) 100
50 200
300 250 s xe (mm)
80 200
350
150 yse (mm)
100 50 200
150
100
s
ze (mm)
100 50 150
270 140
130 120 yse (mm)
265 110
100 260
270 50 150
xse (mm)
265 140
130 120 yse (mm)
(e) time=6.90 s
100 260
xse (mm)
200 ze (mm)
s
110
(f) time=27.77 s
200 150
100
s
ze (mm)
350
300 xse (mm)
150
s
ze (mm)
C: Connect a ball and socket joint
250
(d) time=27.04 s
(c) time=8.74 s
D: Twisting tie wire
240 xse (mm)
60 220
100
0 95
50 95 yse
90 (mm) 85
80 200
300 250 s xe (mm)
(g) time=7.84 s
350
90 yse
(mm)
85
80 200
250
300
350
xse (mm)
(h) time=26.41 s
Fig. 10. Typical trajectories of hydraulic manipulator end-effector for tasks shown in Fig. 9, and under (a) virtual fixture mode, and (b) augmented virtual fixture mode. Solid (black) and dashed (red) lines show desired and actual trajectories, respectively. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
10
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
starts functioning. Fig. 11 depicts magnitudes of virtual fixture force and augmented virtual fixture force that were applied to the operator’s hand during performing tasks shown in Fig. 10. As seen, regardless of the value of force or the task, that was completed, the generated forces (FVF and FAVF) are continuous, and no immediate change is seen in the force signals. Fig. 12 depicts the mean value of radial position errors at the haptic device implement, over 640 trials. Position errors were calculated by averaging absolute values of position errors measured from sampled points on the haptic implement actual trajectory. As observed, both the virtual fixture and the augmented virtual fixture modes showed small position error at the haptic implement which is less than 2.20 mm. Fig. 13 shows the mean values of radial position errors at the slave endeffector for all 640 tests. The mean errors in virtual fixture force mode are approximately three times larger than the ones from the augmented virtual fixture force mode. The mean values of radial errors at the slave end-effector are 20.80 mm in virtual fixture, and 6.02 mm in augmented virtual fixture. Table 2 shows the mean values of radial position errors at slave end-effector (over 640 trials) for each force mode. As seen, the augmented virtual fixture has less position error at the hydraulic manipulator end-effector as compared to the virtual fixture mode. Specifically, adding the augmentation force to the virtual fixture decreased the slave position error by 71%. In general, the augmented virtual fixture was found better in achieving small tracking position error in the tested hydraulic telemanipulator. Note that as a result of reducing position error at the slave end-effector, the task completion time is naturally increased using the augmented virtual fixture. Fig. 14 depicts the task completion times of each force mode. In all four tasks, the mean value of task completion times in the augmented virtual fixture mode is about three times more than the completion times in the virtual fixture mode. Finally, the purpose of implementing the augmented virtual fixture is to allow the slave end-effector to move with minimal position error. Thus, we need to further ensure that measured position errors are repeatable, i.e. the similar values of position error will be observed in future tests. The form of distribution diagram is used to predict the future behavior of
Task
Virtual fixture (VF)
Augmented virtual fixture (AVF)
Haptic force (N)
A: Pulling out a Cotter pin
FVF
4
FAVF AU
3 2 1 0
0
1
2
3
Haptic force (N)
5
5
1 2
VF
FAU
AVF
3 2 1
0
2
4
6
Haptic force (N)
Haptic force (N)
B: Loosening or tightening a nut
5
F
4
1 5
15
20
25
5
5
F
4
VF
F
AVF AU
3 2 1 0
2
4
F
4
F
AVF AU
VF
3 2 1 0 0
6
5
10
20
15
25
Time (s)
FVF
4
FAVF AU
3 2 1 0
1
2
3
Time (s)
4
5
Haptic force (N)
5
5
Haptic force (N)
10
Time (s) Haptic force (N)
Haptic force (N)
C: Connect a ball and socket joint
AVF AU
2
Time (s)
D: Twisting tie wire
F
VF
3
0 0
8
F
4
Time (s)
0
10
8
6
4
Time (s)
5
0
AVF AU
2
Time (s)
0
F
VF
3
0 0
4
F
4
F
4
F
AVF AU
VF
3 2 1 0 0
5
10
15
20
25
Time (s)
Fig. 11. Virtual fixture force (FVF) and augmentation (FAU) force, for typical tests shown in Fig. 10. In virtual fixture force mode, the value of FAU is zero.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
11
Mean value of position error (mm)
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
3
Virtual fixture Augmented virtual fixture
2.5 2 1.5 1 0.5 0
0
50
100
150
200
250
300
Trial run
Mean value of position error (mm)
Fig. 12. Mean value of radial position errors at haptic device implement.
30
Virtual fixture Augmented virtual fixture
25 20 15 10 5 0
0
50
100
150
200
250
300
Trial run Fig. 13. Mean value of radial position errors at manipulator end-effector.
Table 2 Statistical indices of slave position errors in tested force modes. Force mode
Mean value
Kurtosis
Skewness
Virtual fixture Augmented virtual fixture
20.80 6.02
0.935 0.070
1.800 0.593
40
40
Task A
20 10 0 0
20
40
60
Task C
20
20 10
20
40
60
80
60
80
Task D
30
Time (s)
Time (s)
10
40
30
0
20
0 0
80
40
0
Task B
30
Time (s)
Time (s)
30
40
60
80
20 10 0
0
20
Trial run Virtual fixture
40
Trial run Augmented virtual fixture
Fig. 14. Task completion times for all four tasks.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
12
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
(a) Virtual fixture augmented by proposed position-referenced scheme
(b) Virtual fixture augmented by conventional position-referenced scheme 85
X (cm)
X (cm)
85 80 75 70
0
20
40
60
80 75 70
80
0
20
Time (s) 20
Y (cm)
Y (cm)
20 10 0 -10
0
20
40
60
10 0 -10
80
0
20
Time (s) 0
Z (cm)
Z (cm)
40
Time (s)
1 0 -1 -2
40
Time (s)
0
20
40
60
-1 -2 -3
80
0
20
Time (s)
40
Time (s)
Fig. 15. Typical trajectories of hydraulic manipulator end-effector for Task B: (a) with proposed slave position-referenced force; and (b) with conventional slave position-referenced force. Solid (black) and dashed (red) lines show desired and actual trajectories, respectively. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
results. Measures, used to determine the type of distribution, are kurtosis and skewness. Kurtosis is defined as the measure of the ‘‘peakedness’’ of the probability distribution of a real-valued random variable [26], i.e., position error at the manipulator end-effector. Skewness is the measure of the asymmetry of the probability distribution of a real-valued random variable [26]. Table 1 also shows the values of kurtosis and skewness of position errors at the slave end-effector. The kurtosis or skewness absolute values of more than 2, indicate data groups differ or skew to a significant degree [27]. As seen from Table 1, the kurtosis and skewness values show that both virtual fixture and augmented virtual fixture modes exhibited acceptable ranges when the random variable is position error at the slave end-effector. However, when the virtual fixture mode is tested, the absolute measures are closer to 2 as compared to the augmented virtual fixture. This indicates, in addition to having larger values of position error at the slave end-effector, the virtual fixture mode is less reliable than the augmented virtual fixture in repeating similar results. 4.3. Comparison with conventional slave position-referenced force scheme
80 60 40 20 0
0
10
20
Trial run
30
40
Slave position error (mm)
Master position error (mm)
In order to further substantiate the approach presented here, we asked one of the operators to repeat Tasks B and C. However, this time the virtual fixture force was augmented by the conventional slave position-referenced force [19–23], in which the direction of the added force was along the vector of position error at the slave end-effector. We then compared the results with the ones obtained from our approach. The test was repeated 20 times for each task. In total, 2(force modes) 2(tasks) 20(trials) = 80 trials were collected. Typical trajectories are illustrated in Fig. 15. Fig. 16 shows mean of radial position error values at both master implement and slave end-effector, for each of the 80 trials. It is clearly seen that augmenting the virtual fixture by the conventional position-referenced force was substantially less effective in reducing the error as compared with the proposed method. More specifically, the mean values of errors at 80 60 40 20 0
0
10
20
30
40
Trial run
Combined virtual fixture and proposed slave position-referenced scheme Combined virtual fixture and conventional slave position-referenced scheme Fig. 16. Mean values of position error magnitudes at master implement and slave manipulator end-effector.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
13
the slave end-effector were reduced five times, when the proposed scheme was introduced. This is due to the fact that the conventional position-referenced force does not entirely serve to slow down the operator’s hand motion, and therefore does not effectively reduce the tracking error. 5. Conclusions In this paper, we introduced a simple, yet effective method, which aims at reducing the tracking (following) error in teleoperated hydraulic manipulators by overlaying an augmentation force onto the conventional virtual fixture force on the master. The augmentation force is proportional to the difference between the desired and actual positions of the slave end effector; its direction is collinear but opposite to the master instantaneous velocity vector. Indeed, while the virtual fixture force serves to pull the operator’s hand back on track when it deviates from the haptic predefined path, the augmentation force guides the operator to slow down her/his hand motion when the slave position error exceeds beyond a threshold. Effectiveness of the proposed scheme was experimentally evaluated on a hydraulic manipulator performing several live-line maintenance tasks. Evaluation indices were position error at the haptic device implement and position error at the hydraulic manipulator end-effector. Two sets of tests were performed: (i) tests using only conventional virtual fixture force generated by the haptic device, and (ii) tests when the virtual fixture force was augmented by an additional force reflecting the error observed at the slave side. The augmented virtual fixture showed a marked improvement over the virtual fixture mode alone, in that it resulted in lesser position error at the manipulator (slave) side. Specifically, for the tests conducted here, the augmented virtual fixture mode reduced position error at the manipulator end-effector by at least 71%, and this was done by guiding the operator to slow down the hand motion allowing the slave manipulator to catch up with the commands coming from the master haptic device. As expected, as a downside, the task completion time increased up to three times when the augmentation force was added to the virtual fixture force. References [1] Chuang CW, Shiu LC. CPLD based DIVSC of hydraulic position control systems. J Comput Electr Eng 2004;30(7):527–41. [2] Kontz ME, Beckwith J, Book WJ. Evaluation of a teleoperated haptic forklift. In: IEEE/ASME international conference on advanced intelligent mechatronics, USA; 2005. p. 295–300. [3] Tafazoli S, Salcudean SE, Hashtrudi-Zaad K, Lawrence PD. Impedance control of a teleoperated excavator. IEEE Trans Control Syst Technol 2002;10(3):355–67. [4] Papadopoulos E, Mu B, Frenette R. On modeling, identification, and control of a heavy-duty electrohydraulic harvester manipulator. IEEE/ASME Trans Mechatron 2003;8(2):178–87. [5] Ni T, Zhang H, Yu C, Zhao D, Liu S. Design of highly realistic virtual environment for excavator simulator. J Comput Electr Eng 2013;39(7):2112–23. [6] Hokayem PF, Spong MW. Bilateral teleoperation: an historical survey. J Automatica 2006;42(12):2035–57. [7] Kontz ME, Book WJ. Position/rate haptic control of a hydraulic forklift. ASME Int Mech Eng Congr, USA 2003:801–8. [8] Ni T, Zhanga H, Xua P, Yamada H. Vision-based virtual force guidance for tele-robotic system. J Comput Electr Eng 2013;37(7):2135–44. [9] Hainsworth DW. Teleoperation user interfaces for mining robotics. J Autonomous Robots 2001;11(1):19–28. [10] Yoerger D, Slotine JJ. Supervisory control architecture for underwater teleoperation. In: IEEE international conference on robotics and automation, vol. 4; 1987. p. 2068–73. [11] Salisbury JK, Srinivasan MA. Phantom-based haptic interaction with virtual objects. IEEE Comput Graphics Appl 1997;17(5):6–10. [12] Turro N, Khatib O, Coste-Maniere E. Haptically augmented teleoperation. In: IEEE international conference on robotics and automation; 2001. p. 386– 92. [13] Kang H, Park YS, Ewing TF, Faulring E, Colgate JE. Visually and haptically augmented teleoperation in DD tasks using virtual fixtures. In: International conference on robotics and remote systems for hazardous environments; 2004. p. 466–71. [14] Abbott JJ, Marayong P, Okamura AM. Haptic virtual fixtures for robot-assisted manipulation. Robotics Res, Springer Tracts Adv 2007;28:49–64. [15] Marayong P, Bettini A, Okamura A. Effect of virtual fixture compliance on human-machine cooperative manipulation. IEEE/RSJ Int Conf Intell Robots Syst 2002;2:1089–95. [16] Eduardo V, Khokar K, Alqasemi R, Dubey R. Laser assisted real time and scaled telerobotic control of a manipulator for defense and security applications. SPIE Unmanned Systems Technology XI; 2009. [17] Zareinia K, Maddahi Y, Olson T, Mueller W, Sepehri N. Unilateral control of teleoperated hydraulic manipulators applied to live line maintenance: comparative study. J Rob Autom 2014;29(2):162–74. [18] Heinrichs B, Sepehri N, Thornton-Trump A. Position-based impedance control of an industrial hydraulic manipulator. IEEE Control Syst Mag 1997;17(1):46–52. [19] Maddahi Y, Rahman RA, Fung WK, Sepehri N. Effect of network quality on performance of bilateral teleoperated hydraulic actuators: a comparative study. J Control Intell Syst 2013;41(1):11–25. [20] Zarei-nia K, Goharrizi AY, Sepehri N, Fung WK. Experimental evaluation of bilateral control schemes applied to hydraulic actuators: a comparative study. Trans Canadian Soc Mech Eng 2009;33(3):377–98. [21] Abbott JJ, Okamura AM. Virtual fixture architectures for telemanipulation. In: IEEE international conference on robotics and automation, Taiwan; 2003. p. 2798–805. [22] Kontz ME, Herrera MC, Huggins JD, Book WJ. Impedance shaping for improved feel in hydraulic systems. ASME Int Mech Eng Congr Expos, USA 2008:185–94. [23] Hayn H, Schwarzmann D. Control concept for a hydraulic mobile machine using a haptic operating device. In: International conferences advances in computer – human interactions, Mexico; 2009. p. 348–53. [24] Maddahi Y, Zareinia K, Sepehri N, Olson T, Mueller W. Live-line maintenance training using robotics technology. In: IEEE international conference on haptics, Korea; 2013. p. 587–92. [25] Sepehri N, Khayyat AA, Heinrichs B. Development of a nonlinear PI controller for accurate positioning of an industrial hydraulic manipulator. J Mechatronics 1997;7(8):683–700. [26] Wuensch KL. Encyclopedia of statistics in behavioral science. Chichester, UK: Wiley; 2005. [27] Brown JD. Statistics corner: questions and answers about language testing statistics: skewness and kurtosis, vol. 11. Shiken: JALT Testing and Evaluation SIG Newsletter; 1997. p. 20–3.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
14
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Yaser Maddahi received his PhD in Mechanical Engineering from the University of Manitoba, Canada in 2013. He is currently an Eyes High postdoctoral fellow in project neuroArm at University of Calgary and a visiting academic scholar at the University of Manitoba. Yaser’s research interests include robotics, haptics, fMRI-compatible devices, control of teleoperation systems, and hydraulic telemanipulators. Kourosh Zareinia received his BSc and MSc in Electrical Engineering from Isfahan University of Technology and University of Tehran, Iran, respectively. He earned his PhD in Mechanical Engineering from the University of Manitoba, Canada. His research interests include robotics, haptics, hydraulic manipulators and tactile feedback. Currently, Kourosh works on design, development and integration of advanced haptics and robotics on neuroArm. Nariman Sepehri is a professor with the Department of Mechanical Engineering, at the University of Manitoba, Canada. He received M.Sc. and Ph.D. degrees from the University of British Columbia, Canada. His research and development activities are primarily centered in all fluid power related aspects of systems, manipulation, diagnosis and control.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
Contents lists available at ScienceDirect
Computers and Electrical Engineering journal homepage: www.elsevier.com/locate/compeleceng
An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance q Yaser Maddahi, Kourosh Zareinia, Nariman Sepehri ⇑ Department of Mechanical Engineering, The University of Manitoba, Winnipeg, MB R3T5V6, Canada
a r t i c l e
i n f o
Article history: Available online xxxx
a b s t r a c t Virtual fixtures can be used in haptic-enabled hydraulic telemanipulators to facilitate certain tasks. Using this concept, however, the operator may tend to move the master fast due to relying on the virtual fixture. As a result, the slave manipulator could start to lag due to latency in the hydraulic actuation control system. This paper describes how to mitigate the position errors between master and slave robots by overlaying an augmentation force on the master that is collinear but opposite of the master instantaneous velocity. The magnitude of this force is proportional to the position error at the slave end-effector. Experiments, conducted on a teleoperated hydraulic manipulator to perform several live-line maintenance tasks, show that the augmented scheme exhibits less position error at the slave side, better task quality, but longer task completion time as compared to the virtual fixture alone. Ó 2014 Elsevier Ltd. All rights reserved.
1. Introduction Human-operated machines, equipped with hydraulic manipulators, are widely used in industry [1]. Examples include excavators and underwater manipulators [2–4]. These machines are remotely controlled by human, which extends their ability to perform tasks, especially when the environment is unsafe. A human-controlled manipulator system is generally composed of master side where an operator utilizes a hand-controller, slave side where a manipulator emulates behavior of the hand-controller, a communication channel connecting slave and master sides and, a feedback control system. The feedback system can be built upon the operator’s sensation about the slave site (telepresence) [5]. When the assigned task implies a contact with the environment (force tracking), or the slave needs to follow a particular trajectory (position tracking), the use of haptic sensation can be helpful to enhance operator’s performance [6–11]. For example, when teleoperated manipulators are used for repetitive tasks, a virtual fixture force can be applied to the operator’s hand to enhance task performances [12]. Within this context, Kang et al. [13] used the concept of virtual fixtures to provide passive constraint to the operator’s motion. They found that virtual fixtures could improve accuracy and task completion time for performing decontaminating and decommissioning. Abbott et al. [14] discussed design of two types of virtual fixtures: ‘guidance virtual fixtures’ (GVFs) that help the operator move the haptic implement along a pre-defined trajectory and ‘forbidden-region virtual fixtures’ (FRVFs), which prevent the haptic implement from penetrating into forbidden regions (defined at the master or the slave manipulator workspace). Marayong et al. [15] employed vision-based virtual fixtures to provide different levels of guidance to the operator. In this work, the complete guidance offered the best improvement
q
Reviews processed and proposed for publication to the Editor-in-Chief by Guest Editor Dr. Ankit Chaudhary.
⇑ Corresponding author.
E-mail address: [email protected] (N. Sepehri). http://dx.doi.org/10.1016/j.compeleceng.2014.07.006 0045-7906/Ó 2014 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
2
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
in both execution time and error reduction in tasks involving general path-following. Eduardo et al. [16] implemented the concept of impedance-type FRVFs to help operators keep the manipulator out of a defined forbidden region of the workspace. A FRVF was also implemented in a latest study on a teleoperated hydraulic manipulator, performing live-line maintenance tasks [17]. Results of qualitative evaluations by six hydro linemen showed that the concept of virtual fixtures was simple to use, reduced the physical load, and did not require long training. In the above studies, the haptic force was produced based on the information at the master site, and no feedback information was obtained from the slave manipulator. Although the virtual fixture force keeps the operator’s hand on the defined virtual path, an accurate motion at the slave side cannot be guaranteed. For instance, the position tracking of the slave manipulator can easily be violated due to fast motion of the operator’s hand at the master side reflecting the mismatch between the master dynamics and the dynamics of the slave. Particularly, in hydraulic telemanipulators, since hydraulic actuators exhibit significant nonlinear characteristics [18], the requirement of position tracking is generally more challenging than that of their electromechanical counterparts. Thus, in order to mitigate the position errors between the master implement and the slave end-effector, we propose the addition of another force, which is proportional to the magnitude of the position error at the slave end-effector, but in the direction opposite to the operator’s hand velocity vector at the master implement. When position error at the slave end-effector is evident, the augmentation force is activated alerting the operator to slow down the hand movement. Using this scheme, the combined virtual fixture and augmentation force reduces position errors at both the master device implement and the slave manipulator end-effector. Note that the concept of using position error, in slave manipulators, to provide a haptic sensation is not new and has been implemented in few research studies with different goals [19,20]. Abbott and Okamura [21] compared several FRVFs on four common telemanipulator control architectures. A single degree-of-freedom teleoperated system was used to simulate working near a known forbidden region. One of the evaluations relates to combining the position error at the slave manipulator with the slave interaction force with the environment, to produce a haptic force. Kontz et al. [22] proposed a strategy that commands the haptic device to generate a slave position-referenced force that couples the motion of a haptic device to the excavator bucket movement. Hayn and Schwarzmann [23] implemented a slave position-referenced force for the operation of a hydraulic excavator, whereby the positions of the haptic device (master) and the hydraulic manipulator (slave) were used to provide reference positions for each other. The utilization of the conventional slave position-referenced force concept in the above studies, results in producing a haptic force which is parallel to, and in proportion with, the position error vector at the slave end-effector. Indeed, this force indicates to the operator whether the slave is moving ahead or behind the intended desired trajectory. However, it does not serve to slow down the hand’s motion. In this paper, we propose to redirect the force generated based on the magnitude of the slave position error, to be parallel with, but in opposite direction of, the operator’s hand instantaneous velocity. The intention is to slow down the operator’s hand, thereby reducing the following (tracking) error caused by the mismatch between the dynamics of the master device and the slave manipulator. We will evaluate the performance of this new concept when combined with virtual fixtures in a live-line maintenance application. The rest of this paper is organized as follows. The experimental setup and coordinate mapping are described in Section 2. This section allows the readers to understand the system and maintenance tasks, investigated in this paper. The description of, and need for augmenting the currently used virtual fixtures concept is described by presenting preliminary experimental results in Section 3. Effectiveness of the application of the proposed concept is then evaluated in Section 4, by emulating several live transmission line maintenance tasks. The concept is further compared with the conventional slave positionreferenced force mode. Evaluation criteria used are position error at the master implement, and position error at the slave end-effector. Conclusions are provided in Section 5.
2. Experimental setup Fig. 1 shows the test rig developed to examine the performance of proposed augmented virtual fixture scheme. This system comprises an industrial hydraulic manipulator (slave side), and a widely used the PHANToM Desktop haptic device (master side) that allows the operator to control the manipulator end-effector trajectory [24]. The first four degrees of freedom (DOFs) of this manipulator are used to perform live-line maintenance tasks, and the last two DOFs are deactivated. The active motions are rotations about arm (hs1 ), shoulder (hs2 ), main elbow (hs3 ) and extended elbow (hs4 ) axes (Figs. 2a and 3a) P se ) emulates motion of the haptic device implement (~ Pm [24]. The slave manipulator end-effector (~ i ). A typical live-line maintenance task, designed to loosen or tighten a nut, is depicted in Fig. 1c. This test rig has been constructed for an ongoing research that aims to employ robots to work cooperatively with linemen at service interruption free maintenance and inspection of live transmission lines. Fig. 2a shows coordinate frames of the hydraulic manipulator. The PHANToM Desktop haptic device is shown in Fig. 2b. Superscripts ‘s’ and ‘m’ indicate the parameter belongs to manipulator (slave) or haptic device (master), respectively. Subm m s s s scripts ‘e’ and ‘i’ stand for the end-effector and implement, respectively. Frames fxm i yi zi g and fxe ye ze g are the coordinate systems attached to the master device implement and the slave manipulator end-effector, respectively. The fixed (global) coordinate system is denoted by fxo yo zo g. T Now let the position vector of manipulator end-effector be ~ P se ¼ ½ xse yse zse . Using the inverse kinematics solution, the s s s s angular displacements of the slave manipulator joints (h1 ;h2 ;h3 ;h4 ) are determined as follows: Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
3
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Fig. 1. (a) Kodiak 1000 hydraulic manipulator; (b) the PHANToM Desktop haptic device; and (c) typical live-line maintenance task.
Fig. 2. Coordinate frames of (a) the slave manipulator, and (b) the PHANToM Desktop haptic device.
hs1 ¼ tan1
s ye xse
ð1Þ
0qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi1 2 2 2 k1 þ k2 k3 k 2 A þ tan1 @ hs2 ¼ tan1 k1 k3
hs3
1
¼ tan
2l2 k2 c2 2l2 k1 s2 2
2
2
ð2Þ
! ð3Þ
2
k1 þ k2 l2 l3
hs4 ¼ hs2 hs3
ð4Þ 2
2
2
2
where ci ¼ cosðhsi Þ and si ¼ sinðhsi Þ. Note that k1 ¼ xse c1 þ yse s1 l1 l4 , k2 ¼ zse , and k3 ¼ ðk1 þ k2 þ l2 l3 Þ=ð2l2 Þ. Note that in order to allow the manipulator to move along given trajectories, the tool located at the end-effector must remain horizontal, which results in Eq. (4). Furthermore, due to the square root function in Eq. (2), the inverse kinematics solution produces two solution sets. The acceptable one is chosen according to the joint workspaces. Table 1 shows the workspace of each joint of Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
4
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Fig. 3. Virtual fixture concept.
Table 1 Range of joint motions for Kodiak 1000. Joint
Lower limit
Upper limit
Arm, hs1 Shoulder, hs2 Main elbow, hs3 Extended elbow, hs4
12 0 48 90
43 90 180 30
the slave manipulator with respect to the home position. Home position is where the shoulder, main elbow and extended elbow links are horizontal. With reference to Fig. 2, to control manipulator’s end-effector, the coordinated-mode mapping system is chosen, in which m m s s s the haptic implement position, fxm i yi zi g, is mapped into the manipulator end-effector position, fxe ye ze g. As the haptic device m ~ moves, its position vector (P ) is continuously recorded, and scaled by a factor (ks). This gives the desired position of the a
P sd : manipulator end-effector, ~
~ Psd ¼ ks~ Pm a
ð5Þ
Psd ¼ xse;d where ~
T
P sd , the vector of joint angular displacements is . Given desired vector~ s ~ calculated using Eqs. (1)–(4). On the other hand, the actual vector P is obtained by substituting actual joint angular displaceyse;d
zse;d
xm Pm and ~ i;a a ¼
T
ym i;a
zm i;a
a
ments, measured by Hall Effect sensors, hsa , and using forward kinematic equations of the manipulator. In practice, the desired and actual values of joint angular displacements are not necessarily equal due to limitations of the control scheme Es ) appears at the slave end-effector, used, and difference between responsiveness of each link. Therefore, the position error (~ e
which is defined as:
2
ese;x
3
2
xse;a xse;d
3
2
xse;a ks xm i;a
3
6 7 6 7 6 7 ~ Ese ¼ 4 ese;y 5 ¼ 4 yse;a yse;d 5 ¼ 4 yse;a ks ym i;a 5 s s s s m ze;a ze;d ee;z ze;a ks zi;a
ð6Þ
~ Ese will later be used to define augmented virtual fixture force. 3. Augmented virtual fixture 3.1. Impedance-type virtual fixture and observation F VF , is proportional to the A typical impedance type virtual fixture is depicted in Fig. 3. As shown, the generated force, ~ Em error between the actual (point A) and desired (point D) positions of the haptic implement, ~ [14,17]. i
~ F VF ¼ GVF~ Em i
ð7Þ
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
5
m m m m Fig. 4. Desired (Om d ) and actual (Oa ) positions of the haptic implement. Position error at the implement is expressed by (ei;x ; ei;y , ei;z ) in coordinate frame {x0 m y0 z0 }. Om d and Oa represent point D and point A, in Fig. 3, respectively.
where GVF is called the virtual fixture impedance matrix, which represents the ability of virtual fixture in providing different levels of fixture compliance. The higher the values of diagonal terms in this matrix, the more force is generated to keep the operator’s hand on the intended trajectory. The haptic implement position error (~ Em i ) is the difference between the actual and desired positions (see Fig. 4):
2
3 m xm i;a xi;d 6 em 7 6 ym ym 7 ~ Em i;d 5 i ¼ 4 i;y 5 ¼ 4 i;a m m z z em i;a i;d i;z em i;x
3
2
ð8Þ
The virtual fixture is employed in unilateral mode, where there is no feedback from the slave side [17]. This scheme helps the operator keep the hand along the virtual fixture path, and corrects motion of the master device implement [24]. However, this is not adequate to ensure that position error at the slave manipulator end-effector maintains low. Since the virtual fixture scheme provides no feedback from these sources, even if the operator moves closely along the virtual fixture path, the accurate motion of the slave manipulator cannot be guaranteed. The slave end-effector errors can also originate from other sources such as speed of the operator’s hand motion along tangential trajectory at which no restriction in motion is applied by the virtual fixture [14,17]. Preliminary experiments were performed that show how the operator’s hand speed affects the position error at the slave end-effector as well as the haptic implement. The tests were conducted on the test rig shown in Fig. 1. In these tests, an operator was asked to move the haptic implement along a circular trajectory (see Fig. 1c) with four different speeds, which resulted in different task completion times. The measured times were: 30.58 s, 22.50 s, 13.52 s and 8.77 s. For each speed, the position errors at the master implement and the slave end-effector were recorded and compared. The impedance matrix of the virtual fixture, GVF, was set to GVF ¼ diag½1500 N=m 1500 N=m 1500 N=m. Fig. 5a and b shows position error at the haptic implement and slave end-effector, respectively, under different hand speeds and along the circular path shown in Fig. 1c. As observed, by increasing the hand speed, no obvious changes on the value of position error at the haptic device can be noticed, i.e. the ranges of position error are almost similar. The maximum position error was about 1.7 mm, which indicates the virtual fixture was able to keep the operator’s hand close to the desired track. However, the position error at the manipulator end-effector significantly increased beyond the error inherently originated from the controller response (see Fig. 5b). 3.2. Augmenting the virtual fixture The solution to reduce position error at the slave end-effector, proposed in this paper, is to add an augmentation force to the virtual fixture force. The augmentation force alerts the operator to slow down the haptic implement (hand) motion when position error becomes larger than the accuracy expected from the controller at the slave end-effector. The direction of this force should be apposite to the haptic implement (hand) instantaneous velocity. Indeed, this force allows the operator to realize position error at the slave side. The augmentation force is defined as:
~ F AU ¼
8 m > ~s ~s > < GAU Ee Ethreshold v^ ins > > :
0
~s ~s
Ee > Ethreshold
!s s
E 6 ~ Ethreshold
e
ð9Þ
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
6
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
(a) Haptic implement position error
1.5
1
0.5
25
1.5 1 0.5
40 20
5
15
10
20
30
25
Time (s)
5
10
15
Time (s)
80 60 40 20 0 0
20
5
10
15
20
Time (s) 100
2
1.5
1
0.5
2
4
6
Time (s)
8
10
80 60 40 20 0 0
12
2
4
6
8
10
12
Time (s) 100
Slave Position Error (mm)
2
1.5
1
0.5
0 0
60
0 0
30
Slave Position Error (mm)
Master Position Error (mm)
20
80
100
0 0
Master Position Error (mm)
15
Time (s)
2
0 0
Task time=13.52 s
10
5
Slave Position Error (mm)
Master Position Error (mm)
Task time=22.50 s
0 0
Task time=8.77 s
(b) Slave end-effector position error 100
2
Slave Position Error (mm)
Master Position Error (mm)
Task time=30.58 s
Task
1
2
3
4
5
Time (s)
6
7
8
80 60 40 20 0 0
1
2
3
4
5
6
7
8
Time (s)
Fig. 5. Position error at the haptic implement and manipulator end-effector, given the same task as in Fig. 1c and under different speeds.
Ese is the vector of position error at the manipulator end-effector, and is expressed by three components As shown in Fig. 6, ~ in frame fx0 y0 z0 g. Note that due to inherent error of the controller, the augmentation force is only generated when the position error is greater than a threshold. ~ Esthreshold is defined based on the steady-state positioning error at the slave end-effector
s ~s
Ee 6 Ethreshold , the haptic that originates from the manipulator’s controller as well as the resolution of the sensors. When ~
s ~s
Ee > Ethreshold , ~ F AU is proportional to ~ Ese ~ Esthreshold in terms of device does not produce any augmentation force. When ~ ^m magnitude, and parallel to v ins , which is the unit vector of haptic implement instantaneous velocity. The negative sign indicates the augmentation force is in the opposite direction of the hand instantaneous velocity. GAU is a diagonal matrix. The threshold is considered to prevent repeated activation-deactivation cycles. Fig. 7 shows how the augmented virtual fixture force is calculated using the virtual fixture and the augmentation forces. F VF ) pulls the operator’s hand toward the haptic desired path, the As illustrated in Fig. 7, while the virtual fixture force (~
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
7
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Fig. 6. Desired (Osd ) and actual (Osa ) positions of the slave manipulator end-effector. Position error at the end-effector is expressed by (ese;x , ese;y , ese;z ) in coordinate frame {xs0 ys0 zs0 }.
Fig. 7. Augmented virtual fixture scheme.
F AU ) slows down the operator’s hand motion. The augmented virtual fixture force (~ F AVF ) is then calcuaugmentation force (~ lated as:
~ F AVF ¼ qVF~ F VF þ qAU~ F AU
ð10Þ
where qVF and qAU are positive weighting factors to adjust the relative effect of virtual fixture and augmentation forces, respectively (qVF þ qAU ¼ 1). The block diagram of the augmented virtual fixture scheme is illustrated in Fig. 8. As observed, the actual position vector Pm ) is multiplied by scalar ks to indicate the desired position of the slave manipulator end-effector. of the haptic implement (~ a
Pm The actual position of the haptic implement (~ a ) is continuously recorded to determine position error at the haptic implem m ~ ~ F VF ). On the slave ment, E . Next, E is multiplied by the master impedance, GVF, in order to calculate the virtual fixture force (~ i
i
side, the actual position of the haptic device is multiplied by the scaling factor, ks, in order to calculate the desired position of P s . The actual position of the slave manipulator end-effector, ~ Ps , is calculated by substituting the manipulator end-effector, ~ d
a
hsa ) into forward kinematic equations of the manipulator. Having manipulator the actual angular displacements of joints (~ end-effector position error vector, the augmentation force, ~ F AU , is calculated using Eq. (9). The combined virtual fixture and augmentation forces form the augmented virtual fixture force. 4. Experimental results and discussion 4.1. Test procedure Two sets of experiments are presented in which eight operators were asked to perform maintenance tasks, using the PHANToM Desktop haptic device. Four tasks were performed using the experimental setup. They were: pulling out a cotter pin (Task A, Fig. 9a), loosening or tightening a nut (Task B, Fig. 9b), connecting or disconnecting a ball and socket joint (Task Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
8
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Fig. 8. Block diagram of augmented virtual fixture scheme.
(a) Task A: Pulling the cotter pin (line)
(b) Task B: Tightening the nut (horizontal circle)
end-effector (c) Task C: Connecting the joint (square)
(d) Task D: Twisting tie-wire (vertical circle)
Fig. 9. Typical maintenance tasks and corresponding expected paths to complete each task.
C, Fig. 9c) and, twisting tie wire (Task D, Fig. 9d). Fig. 9 shows virtual fixture paths along which the operators were asked to move the hydraulic manipulator end-effector. The experiments were first conducted using the virtual fixture force only (qVF = 1, qAU = 0). Next, the augmented virtual fixture force was applied (qVF = 0.5, qAU = 0.5). The impedance matrix of the force augmentation scheme was a diagonal matrix, GAU ¼ diag½150 N=m150 N=m150 N=m, the values of its elements were determined to give a feeling of an unsaturated Em has its maximum value during the test. Effectiveness of each mode was evaluated using two indices: (i) posiforce when ~ i
tion error at the haptic device implement, and (ii) position error at the manipulator end-effector. The operators were asked to hold stylus of the haptic device like a pen, and try to trace each path. The experiments were repeated 10 times by each operator, for each force mode. A total number of 2(force modes) 4(tasks) 10(trials) Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
9
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
8(participants) = 640 trials were collected. Although the operators were asked to repeat particular tasks, some deviations from trajectories were unavoidable. The nonlinear PI (NPI) controller, developed earlier [25], was employed to control each joint of the manipulator. This controller produced maximum steady-state error reflecting a threshold of 1.54 mm, 1.06 mm and 2.07 mm at the end-effector along x0 , y0 and z0 , respectively; thus, ~ Esthreshold ¼ ð1:54 mm;1:06 mm;2:07 mmÞT . 4.2. Results Fig. 10 illustrates typical trajectories of slave manipulator end-effector for all four tasks and under the two force modes. This figure shows how accurate the operator performs the tasks. As observed, there exist some deviations from the desired path in slave end-effector trajectory which result in position error. By comparing trajectories depicted in each row, it is seen that the position error at the slave end-effector, using the virtual fixture scheme only, is larger than the augmented virtual fixture scheme. For instance, comparing Fig. 10c and e, it is seen that the operator could not follow closely given circular and rectangular trajectories using the virtual fixture only. Likewise, the manipulator end-effector, in Fig. 10g, was not properly guided along the desired vertical curve. The values of task completion time indicate that, in augmented virtual fixture, the operator guided the slave end-effector slower than the virtual fixture mode to match its dynamics to dynamics of the manipulator. Note that these typical trajectories were randomly chosen from 640 trial runs. The quality of slave end-effector trajectory could change from an operator to another one; but, similar observation can be obtained for each test. As mentioned, the augmented force is zero if the magnitude of the position error at the slave end-effector is less than the controller inherent error (~ Esthreshold ). Otherwise, once a position error is greater than ~ Esthreshold appears, the augmentation force
Virtual fixture
Augmented virtual fixture 150 ze (mm)
150
145
s
145
s
ze (mm)
A: Pulling out a Cotter pin
Task
140 140
100 yse (mm)
140 140
260
120
80
60 220
260
120
100 yse (mm)
240 xse (mm)
80
(b) time=10.29 s 100
90 85
90
s
s
ze (mm)
95
ze (mm)
B: Loosening or tightening a nut
(a) time=4.97 s
80 200 150 yse (mm) 100
50 200
300 250 s xe (mm)
80 200
350
150 yse (mm)
100 50 200
150
100
s
ze (mm)
100 50 150
270 140
130 120 yse (mm)
265 110
100 260
270 50 150
xse (mm)
265 140
130 120 yse (mm)
(e) time=6.90 s
100 260
xse (mm)
200 ze (mm)
s
110
(f) time=27.77 s
200 150
100
s
ze (mm)
350
300 xse (mm)
150
s
ze (mm)
C: Connect a ball and socket joint
250
(d) time=27.04 s
(c) time=8.74 s
D: Twisting tie wire
240 xse (mm)
60 220
100
0 95
50 95 yse
90 (mm) 85
80 200
300 250 s xe (mm)
(g) time=7.84 s
350
90 yse
(mm)
85
80 200
250
300
350
xse (mm)
(h) time=26.41 s
Fig. 10. Typical trajectories of hydraulic manipulator end-effector for tasks shown in Fig. 9, and under (a) virtual fixture mode, and (b) augmented virtual fixture mode. Solid (black) and dashed (red) lines show desired and actual trajectories, respectively. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
10
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
starts functioning. Fig. 11 depicts magnitudes of virtual fixture force and augmented virtual fixture force that were applied to the operator’s hand during performing tasks shown in Fig. 10. As seen, regardless of the value of force or the task, that was completed, the generated forces (FVF and FAVF) are continuous, and no immediate change is seen in the force signals. Fig. 12 depicts the mean value of radial position errors at the haptic device implement, over 640 trials. Position errors were calculated by averaging absolute values of position errors measured from sampled points on the haptic implement actual trajectory. As observed, both the virtual fixture and the augmented virtual fixture modes showed small position error at the haptic implement which is less than 2.20 mm. Fig. 13 shows the mean values of radial position errors at the slave endeffector for all 640 tests. The mean errors in virtual fixture force mode are approximately three times larger than the ones from the augmented virtual fixture force mode. The mean values of radial errors at the slave end-effector are 20.80 mm in virtual fixture, and 6.02 mm in augmented virtual fixture. Table 2 shows the mean values of radial position errors at slave end-effector (over 640 trials) for each force mode. As seen, the augmented virtual fixture has less position error at the hydraulic manipulator end-effector as compared to the virtual fixture mode. Specifically, adding the augmentation force to the virtual fixture decreased the slave position error by 71%. In general, the augmented virtual fixture was found better in achieving small tracking position error in the tested hydraulic telemanipulator. Note that as a result of reducing position error at the slave end-effector, the task completion time is naturally increased using the augmented virtual fixture. Fig. 14 depicts the task completion times of each force mode. In all four tasks, the mean value of task completion times in the augmented virtual fixture mode is about three times more than the completion times in the virtual fixture mode. Finally, the purpose of implementing the augmented virtual fixture is to allow the slave end-effector to move with minimal position error. Thus, we need to further ensure that measured position errors are repeatable, i.e. the similar values of position error will be observed in future tests. The form of distribution diagram is used to predict the future behavior of
Task
Virtual fixture (VF)
Augmented virtual fixture (AVF)
Haptic force (N)
A: Pulling out a Cotter pin
FVF
4
FAVF AU
3 2 1 0
0
1
2
3
Haptic force (N)
5
5
1 2
VF
FAU
AVF
3 2 1
0
2
4
6
Haptic force (N)
Haptic force (N)
B: Loosening or tightening a nut
5
F
4
1 5
15
20
25
5
5
F
4
VF
F
AVF AU
3 2 1 0
2
4
F
4
F
AVF AU
VF
3 2 1 0 0
6
5
10
20
15
25
Time (s)
FVF
4
FAVF AU
3 2 1 0
1
2
3
Time (s)
4
5
Haptic force (N)
5
5
Haptic force (N)
10
Time (s) Haptic force (N)
Haptic force (N)
C: Connect a ball and socket joint
AVF AU
2
Time (s)
D: Twisting tie wire
F
VF
3
0 0
8
F
4
Time (s)
0
10
8
6
4
Time (s)
5
0
AVF AU
2
Time (s)
0
F
VF
3
0 0
4
F
4
F
4
F
AVF AU
VF
3 2 1 0 0
5
10
15
20
25
Time (s)
Fig. 11. Virtual fixture force (FVF) and augmentation (FAU) force, for typical tests shown in Fig. 10. In virtual fixture force mode, the value of FAU is zero.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
11
Mean value of position error (mm)
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
3
Virtual fixture Augmented virtual fixture
2.5 2 1.5 1 0.5 0
0
50
100
150
200
250
300
Trial run
Mean value of position error (mm)
Fig. 12. Mean value of radial position errors at haptic device implement.
30
Virtual fixture Augmented virtual fixture
25 20 15 10 5 0
0
50
100
150
200
250
300
Trial run Fig. 13. Mean value of radial position errors at manipulator end-effector.
Table 2 Statistical indices of slave position errors in tested force modes. Force mode
Mean value
Kurtosis
Skewness
Virtual fixture Augmented virtual fixture
20.80 6.02
0.935 0.070
1.800 0.593
40
40
Task A
20 10 0 0
20
40
60
Task C
20
20 10
20
40
60
80
60
80
Task D
30
Time (s)
Time (s)
10
40
30
0
20
0 0
80
40
0
Task B
30
Time (s)
Time (s)
30
40
60
80
20 10 0
0
20
Trial run Virtual fixture
40
Trial run Augmented virtual fixture
Fig. 14. Task completion times for all four tasks.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
12
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
(a) Virtual fixture augmented by proposed position-referenced scheme
(b) Virtual fixture augmented by conventional position-referenced scheme 85
X (cm)
X (cm)
85 80 75 70
0
20
40
60
80 75 70
80
0
20
Time (s) 20
Y (cm)
Y (cm)
20 10 0 -10
0
20
40
60
10 0 -10
80
0
20
Time (s) 0
Z (cm)
Z (cm)
40
Time (s)
1 0 -1 -2
40
Time (s)
0
20
40
60
-1 -2 -3
80
0
20
Time (s)
40
Time (s)
Fig. 15. Typical trajectories of hydraulic manipulator end-effector for Task B: (a) with proposed slave position-referenced force; and (b) with conventional slave position-referenced force. Solid (black) and dashed (red) lines show desired and actual trajectories, respectively. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
results. Measures, used to determine the type of distribution, are kurtosis and skewness. Kurtosis is defined as the measure of the ‘‘peakedness’’ of the probability distribution of a real-valued random variable [26], i.e., position error at the manipulator end-effector. Skewness is the measure of the asymmetry of the probability distribution of a real-valued random variable [26]. Table 1 also shows the values of kurtosis and skewness of position errors at the slave end-effector. The kurtosis or skewness absolute values of more than 2, indicate data groups differ or skew to a significant degree [27]. As seen from Table 1, the kurtosis and skewness values show that both virtual fixture and augmented virtual fixture modes exhibited acceptable ranges when the random variable is position error at the slave end-effector. However, when the virtual fixture mode is tested, the absolute measures are closer to 2 as compared to the augmented virtual fixture. This indicates, in addition to having larger values of position error at the slave end-effector, the virtual fixture mode is less reliable than the augmented virtual fixture in repeating similar results. 4.3. Comparison with conventional slave position-referenced force scheme
80 60 40 20 0
0
10
20
Trial run
30
40
Slave position error (mm)
Master position error (mm)
In order to further substantiate the approach presented here, we asked one of the operators to repeat Tasks B and C. However, this time the virtual fixture force was augmented by the conventional slave position-referenced force [19–23], in which the direction of the added force was along the vector of position error at the slave end-effector. We then compared the results with the ones obtained from our approach. The test was repeated 20 times for each task. In total, 2(force modes) 2(tasks) 20(trials) = 80 trials were collected. Typical trajectories are illustrated in Fig. 15. Fig. 16 shows mean of radial position error values at both master implement and slave end-effector, for each of the 80 trials. It is clearly seen that augmenting the virtual fixture by the conventional position-referenced force was substantially less effective in reducing the error as compared with the proposed method. More specifically, the mean values of errors at 80 60 40 20 0
0
10
20
30
40
Trial run
Combined virtual fixture and proposed slave position-referenced scheme Combined virtual fixture and conventional slave position-referenced scheme Fig. 16. Mean values of position error magnitudes at master implement and slave manipulator end-effector.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
13
the slave end-effector were reduced five times, when the proposed scheme was introduced. This is due to the fact that the conventional position-referenced force does not entirely serve to slow down the operator’s hand motion, and therefore does not effectively reduce the tracking error. 5. Conclusions In this paper, we introduced a simple, yet effective method, which aims at reducing the tracking (following) error in teleoperated hydraulic manipulators by overlaying an augmentation force onto the conventional virtual fixture force on the master. The augmentation force is proportional to the difference between the desired and actual positions of the slave end effector; its direction is collinear but opposite to the master instantaneous velocity vector. Indeed, while the virtual fixture force serves to pull the operator’s hand back on track when it deviates from the haptic predefined path, the augmentation force guides the operator to slow down her/his hand motion when the slave position error exceeds beyond a threshold. Effectiveness of the proposed scheme was experimentally evaluated on a hydraulic manipulator performing several live-line maintenance tasks. Evaluation indices were position error at the haptic device implement and position error at the hydraulic manipulator end-effector. Two sets of tests were performed: (i) tests using only conventional virtual fixture force generated by the haptic device, and (ii) tests when the virtual fixture force was augmented by an additional force reflecting the error observed at the slave side. The augmented virtual fixture showed a marked improvement over the virtual fixture mode alone, in that it resulted in lesser position error at the manipulator (slave) side. Specifically, for the tests conducted here, the augmented virtual fixture mode reduced position error at the manipulator end-effector by at least 71%, and this was done by guiding the operator to slow down the hand motion allowing the slave manipulator to catch up with the commands coming from the master haptic device. As expected, as a downside, the task completion time increased up to three times when the augmentation force was added to the virtual fixture force. References [1] Chuang CW, Shiu LC. CPLD based DIVSC of hydraulic position control systems. J Comput Electr Eng 2004;30(7):527–41. [2] Kontz ME, Beckwith J, Book WJ. Evaluation of a teleoperated haptic forklift. In: IEEE/ASME international conference on advanced intelligent mechatronics, USA; 2005. p. 295–300. [3] Tafazoli S, Salcudean SE, Hashtrudi-Zaad K, Lawrence PD. Impedance control of a teleoperated excavator. IEEE Trans Control Syst Technol 2002;10(3):355–67. [4] Papadopoulos E, Mu B, Frenette R. On modeling, identification, and control of a heavy-duty electrohydraulic harvester manipulator. IEEE/ASME Trans Mechatron 2003;8(2):178–87. [5] Ni T, Zhang H, Yu C, Zhao D, Liu S. Design of highly realistic virtual environment for excavator simulator. J Comput Electr Eng 2013;39(7):2112–23. [6] Hokayem PF, Spong MW. Bilateral teleoperation: an historical survey. J Automatica 2006;42(12):2035–57. [7] Kontz ME, Book WJ. Position/rate haptic control of a hydraulic forklift. ASME Int Mech Eng Congr, USA 2003:801–8. [8] Ni T, Zhanga H, Xua P, Yamada H. Vision-based virtual force guidance for tele-robotic system. J Comput Electr Eng 2013;37(7):2135–44. [9] Hainsworth DW. Teleoperation user interfaces for mining robotics. J Autonomous Robots 2001;11(1):19–28. [10] Yoerger D, Slotine JJ. Supervisory control architecture for underwater teleoperation. In: IEEE international conference on robotics and automation, vol. 4; 1987. p. 2068–73. [11] Salisbury JK, Srinivasan MA. Phantom-based haptic interaction with virtual objects. IEEE Comput Graphics Appl 1997;17(5):6–10. [12] Turro N, Khatib O, Coste-Maniere E. Haptically augmented teleoperation. In: IEEE international conference on robotics and automation; 2001. p. 386– 92. [13] Kang H, Park YS, Ewing TF, Faulring E, Colgate JE. Visually and haptically augmented teleoperation in DD tasks using virtual fixtures. In: International conference on robotics and remote systems for hazardous environments; 2004. p. 466–71. [14] Abbott JJ, Marayong P, Okamura AM. Haptic virtual fixtures for robot-assisted manipulation. Robotics Res, Springer Tracts Adv 2007;28:49–64. [15] Marayong P, Bettini A, Okamura A. Effect of virtual fixture compliance on human-machine cooperative manipulation. IEEE/RSJ Int Conf Intell Robots Syst 2002;2:1089–95. [16] Eduardo V, Khokar K, Alqasemi R, Dubey R. Laser assisted real time and scaled telerobotic control of a manipulator for defense and security applications. SPIE Unmanned Systems Technology XI; 2009. [17] Zareinia K, Maddahi Y, Olson T, Mueller W, Sepehri N. Unilateral control of teleoperated hydraulic manipulators applied to live line maintenance: comparative study. J Rob Autom 2014;29(2):162–74. [18] Heinrichs B, Sepehri N, Thornton-Trump A. Position-based impedance control of an industrial hydraulic manipulator. IEEE Control Syst Mag 1997;17(1):46–52. [19] Maddahi Y, Rahman RA, Fung WK, Sepehri N. Effect of network quality on performance of bilateral teleoperated hydraulic actuators: a comparative study. J Control Intell Syst 2013;41(1):11–25. [20] Zarei-nia K, Goharrizi AY, Sepehri N, Fung WK. Experimental evaluation of bilateral control schemes applied to hydraulic actuators: a comparative study. Trans Canadian Soc Mech Eng 2009;33(3):377–98. [21] Abbott JJ, Okamura AM. Virtual fixture architectures for telemanipulation. In: IEEE international conference on robotics and automation, Taiwan; 2003. p. 2798–805. [22] Kontz ME, Herrera MC, Huggins JD, Book WJ. Impedance shaping for improved feel in hydraulic systems. ASME Int Mech Eng Congr Expos, USA 2008:185–94. [23] Hayn H, Schwarzmann D. Control concept for a hydraulic mobile machine using a haptic operating device. In: International conferences advances in computer – human interactions, Mexico; 2009. p. 348–53. [24] Maddahi Y, Zareinia K, Sepehri N, Olson T, Mueller W. Live-line maintenance training using robotics technology. In: IEEE international conference on haptics, Korea; 2013. p. 587–92. [25] Sepehri N, Khayyat AA, Heinrichs B. Development of a nonlinear PI controller for accurate positioning of an industrial hydraulic manipulator. J Mechatronics 1997;7(8):683–700. [26] Wuensch KL. Encyclopedia of statistics in behavioral science. Chichester, UK: Wiley; 2005. [27] Brown JD. Statistics corner: questions and answers about language testing statistics: skewness and kurtosis, vol. 11. Shiken: JALT Testing and Evaluation SIG Newsletter; 1997. p. 20–3.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006
14
Y. Maddahi et al. / Computers and Electrical Engineering xxx (2014) xxx–xxx
Yaser Maddahi received his PhD in Mechanical Engineering from the University of Manitoba, Canada in 2013. He is currently an Eyes High postdoctoral fellow in project neuroArm at University of Calgary and a visiting academic scholar at the University of Manitoba. Yaser’s research interests include robotics, haptics, fMRI-compatible devices, control of teleoperation systems, and hydraulic telemanipulators. Kourosh Zareinia received his BSc and MSc in Electrical Engineering from Isfahan University of Technology and University of Tehran, Iran, respectively. He earned his PhD in Mechanical Engineering from the University of Manitoba, Canada. His research interests include robotics, haptics, hydraulic manipulators and tactile feedback. Currently, Kourosh works on design, development and integration of advanced haptics and robotics on neuroArm. Nariman Sepehri is a professor with the Department of Mechanical Engineering, at the University of Manitoba, Canada. He received M.Sc. and Ph.D. degrees from the University of British Columbia, Canada. His research and development activities are primarily centered in all fluid power related aspects of systems, manipulation, diagnosis and control.
Please cite this article in press as: Maddahi Y et al. An augmented virtual fixture to improve task performance in robot-assisted live-line maintenance. Comput Electr Eng (2014), http://dx.doi.org/10.1016/j.compeleceng.2014.07.006