- Email: [email protected]
Training in Minimally Invasive Lobectomy: Thoracoscopic Versus Robotic Approaches
Training in Minimally Invasive Lobectomy: Thoracoscopic Versus Robotic Approaches Mark K. Ferguson, MD, Konstantin Umanskiy, MD, Cindy Warnes, RN, Amy D. Celauro, PA, MS, Wickii T. Vigneswaran, MD, and Vivek N. Prachand, MD Department of Surgery, University of Chicago, Chicago, Illinois
Background. Skills required for thoracoscopic and robotic operations likely differ. The needs and abilities of trainees learning these approaches require assessment. Methods. Trainees performed initial components of minimally invasive lobectomies using thoracoscopic or robotic approaches. Component difficulty was scored by trainees using the NASA task load index (NASATLX). Performance of each component was graded by trainees and attending surgeons on a 5-point ordinal scale (naïve, beginning learner, advanced learner, competent, master). Results. Eleven surgical trainees performed 87 replications among three lobectomy components (divide pulmonary ligament; dissect level 7/8/9 nodes; dissect level 4/5 nodes). Before performance NASATLX scores did not differ among components or between surgical approaches. Trainees’ after performance NASATLX scores appropriately calibrated task load for the components. After performance NASATLX scores were significantly lower for thoracoscopy than before performance estimates; robotic scores were similar before surgery and
after performance. Task load was higher for robotic than for thoracoscopic approaches. Trainees rated their performance higher than did attending surgeons in domains of knowledge and thinking, but ratings for other domains were similarly low. Ratings for performance improved significantly as component performance repetitions increased. Conclusions. Trainees did not differentiate task load among components or surgical approaches before attempting them. Task load scores differentiated difficulty among initial components of lobectomy, and were greater for robotic than for thoracoscopic approaches. Trainees overestimated their level of cognitive performance compared with attending physician evaluation of trainee performance. The study provides insights into how to customize training for thoracoscopic and robotic lobectomy and identifies tools to assess training effectiveness.
T
sometimes a different physical orientation for the surgeon (from anterior or inferior rather than from posterior), moderate magnification, and a two-dimensional view of the field; haptic feedback is largely retained. In contrast, robotic procedures, when compared with VATS procedures, often maintain the same surgical field orientation but involve substantially greater magnification, a threedimensional view, and loss of haptic feedback. Assessment of perceived difficulty and performance of specific steps of lobectomy may help identify components of this complex operation that are more challenging for VATS or robotic approaches. Such knowledge may help determine trainee educational needs to design curricular improvements in minimally invasive lobectomy training. We performed a pilot investigation of trainee ratings of component difficulty, the association of surgical approach to component difficulty, and trainee and surgical attending ratings of learner performance.
he explosion of minimally invasive surgery in the 1990s led to new approaches to caring for surgical patients, including thoracic surgical patients. In many centers, video-assisted thoracic surgery (VATS) lobectomy is now the most common technique for performing lung resections, and penetrance overall is approximately 40% [1]. A similar transition to robotic lobectomy has not yet occurred despite the more than 10-year interval since the first clinical reports of robotic lobectomy, with a recent publication citing a 3.4% penetrance in 2010 [1]. The reasons for this are manifold, including costs, availability, lack of standardized approaches to robotic lobectomy, lack of evidence for improved outcomes in the face of increased costs, and challenges in teaching new skills required for robotic lobectomy [2–4]. Robotic procedures require a different skill set than do thoracoscopic procedures, possibly to the same extent that thoracoscopic procedures require a different skill set than open thoracic procedures. Compared with open surgery, VATS operations often entail a different visual and
(Ann Thorac Surg 2014;-:-–-) Ó 2014 by The Society of Thoracic Surgeons
Material and Methods Accepted for publication Jan 14, 2014. Address correspondence to Dr Ferguson, Department of Surgery, University of Chicago, 5841 S Maryland Ave, MC 5040, Chicago, IL 60637; e-mail: [email protected].
Ó 2014 by The Society of Thoracic Surgeons Published by Elsevier Inc
This was a prospective single-institution pilot study evaluating task load and performance characteristics of senior level surgical trainees participating in minimally invasive pulmonary lobectomies. 0003-4975/$36.00 http://dx.doi.org/10.1016/j.athoracsur.2014.01.055
2
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
Subjects Study subjects (trainees) included senior general surgical residents and cardiothoracic surgery fellows who were in the process of learning minimally invasive lobectomy using the da Vinci robot (Intuitive Surgical, Sunnyvale, CA) or VATS. Inclusion criteria were (1) anticipated performance of 5 or more components using VATS or the robot during the study period; (2) performance of 2 or fewer components before study entry by any approach (VATS, robotic, open); (3) presence of an attending surgeon during the performance of study components; and (4) completion of a minimum of 1 hour of mentored practice with the robot before beginning the clinical portion of the study. Otherwise, robotic and VATS simulation exercises were not offered to the participants. Prior experience with laparoscopy was not assessed as part of this study, but was assumed to be extensive. Institutional Review Board approval was obtained before initiation of the study, and trainees were required to sign a study-specific consent form.
Component Definition Lobectomy was divided into individual components (Table 1). Components were characterized according to the perceived level of difficulty based on procedural and cognitive content as low, medium, or high. Components were selected for evaluation in this study that were feasible for all trainees to perform safely and effectively under proper supervision (components 1 through 4 in Table 1).
Subject Performance of Components Trainees were required to have reviewed the componentspecific steps using self-selected materials, to have discussed each with the attending surgeon, and to have completed an initial self-evaluation of difficulty before participating performance of the component. This initial evaluation of difficulty was intended to come from a naïve perspective. Trainees performed individual components using robotic or VATS techniques according to the clinical needs of the patient as determined by the attending surgeon and according to the attending surgeon’s perception of the trainee’s comfort/skill level. It was the attending surgeon’s responsibility to exercise appropriate clinical judgment in ensuring that performance of each component was proceeding safely and adequately if a Table 1. Components of Minimally Invasive Lobectomy Component 1a 2a 3a 4a 5 6 7 a
Description Divide pulmonary ligament Dissect level 8, 9, 7 lymph nodes Dissect level 4 or 5/6 lymph nodes Dissect, divide pulmonary vein Dissect, divide pulmonary artery branches Dissect, divide bronchus Complete fissure(s)
Components that were included as study components.
Difficulty Low Medium Medium Medium High High Medium
Ann Thorac Surg 2014;-:-–-
trainee was involved in its performance regardless of whether the performance of that component was being studied. It was not necessary that trainees performed most or all of a lobectomy to participate in an operation for purposes of this study. Completion of a component by the trainee was not required for the trainee to be able to participate in subsequent evaluation of the performance of that component.
Evaluation of Component Performance After performing a component, the trainee immediately rated that component using the NASA task load index (NASATLX; version 2.0) [5]. Task load is a subjective estimate of overall workload of a specific task. This instrument incorporates separate visual scales graded from 1 to 21 for mental demands, physical demands, temporal demands, own performance, effort, and frustration (Appendix 1). Trainees also immediately rated their performance on each component using a modified version of the Surgical Training and Assessment Tool (STAT) [6, 7]. Domains of performance included knowledge (understood rationale, anatomy, potential hazards); thinking (asked for retraction, positioned assistants, instrument selection, recognized problems if they arose); skill (technical aptitude including instrument and tissue handling, efficiency, did the trainee complete the component); independence (was guidance necessary in anatomy, instrument selection, or approach; did the trainee make an error in thought or action that required correction); and overall. Domains were rated using the choices “naïve,” “beginning learner,” “advanced learner,” “competent,” and “master” in an ordinal scale (Appendix 2). At the conclusion of each operation, the attending surgeon also rated trainee performance using STAT.
Statistical Analyses The NASATLX overall mean scores were calculated from the subscales; weighted NASATLX scores were not calculated. The STAT results were evaluated according to each of the STAT domains. Numerical data were compared using analysis of variance or unpaired t test, and categorical data were compared using c2 analysis. Kappa values (assessing exact matches among raters; a value of 1 indicates complete agreement, a value of 0 indicates that agreement occurred by chance) and Kendall’s tau (assessing rank correlation, in which proximity to another score was taken into account; a value of 1 indicates complete agreement, a value of 0 indicates that agreement occurred by chance) were calculated. All computations were performed using Minitab 16 (Minitab Inc, State College, PA).
Results Subjects and Components Eleven trainees completed 97 components of VATS or robotic lobectomy in 37 patients from February 2012 through January 2013. Nine were senior general surgery residents and two were cardiothoracic fellows. Two
Ann Thorac Surg 2014;-:-–-
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
attending thoracic surgeons participated in the study. There were three components that had adequate numbers to permit analysis, totaling 87 component performances: divide pulmonary ligament, dissect level 7/8/9 lymph nodes, and dissect level 4/5 lymph nodes (level 4 for right-sided resections, level 5 for left-sided resections).
3
or more performance, but borderline statistical significance was achieved only for the component with the lowest task load—divide pulmonary ligament—driven by significant task load decreases over time for temporal, effort, and frustration elements (Table 3). This finding indicates that a reduction in task load requires more than a few repetitions of a component.
Task Load Trainees did not differentiate among components or between approaches in their before performance task load scores. That was true for mean NASATLX scores as well as for each individual NASATLX element. Mean after performance task load scores differentiated among the three components, and this was also true for each NASATLX element except for task load associated with performance. These findings indicate appropriate calibration of difficulty by trainees relative to investigators’ estimated difficulty. This calibration was evident for VATS approaches, but there was no statistical difference among components performed robotically (Table 2). After performance mean task load scores were significantly lower for components performed using a VATS approach compared with before performance scores; these significant differences were evident for the first VATS performance of a component and thus were not entirely a result of learning with repeated VATS performances of a component. That indicates that trainees substantially overestimated task load associated with components of VATS lobectomy before actually performing these components. Task load scores for robotic components were not significantly different comparing before and after performance scores, indicating that the trainees accurately estimated robotic component task load before performing the components. The VATS scores after performance were significantly lower than robotic scores for each component (p < 0.05 for each), indicating that actual task load was lower for VATS than for robotic approaches.
Effect of Learning on Task Load Mean task loads changed somewhat with repeat performance of components, grouped as first, second, and third
Performance Scores The STAT performance scores assigned by trainees and attending surgeons ranged from naïve to competent. Performance scores were skewed toward the intermediate ratings, with only 73 (8.4%) rated as naïve and 24 (2.8%) rated as competent. The distribution among trainees was different than that among attending surgeons, with trainees demonstrating a higher likelihood of rating their performance as competent and a lower likelihood of rating their performance as naive (p ¼ 0.003). Because of the small numbers of performance scores in the naïve and competent categories, performance scores were combined into two categories for further analysis: “naïve or beginning learner” and “advanced learner or competent.” The distribution among domain scores was significantly different for both trainees and attending surgeons. This finding indicates that trainees and attending surgeons are able to discriminate among the domains in assigning scores (Table 4). Trainees rated their performance significantly higher than did attending surgeons in the domains of knowledge and thinking. That was related in part to significant differences in these domains for the more difficult components, whereas performance ratings in these domains for division of the pulmonary ligament were similar. Robotic approaches were rated significantly differently by trainees and attending surgeons for the knowledge domain, whereas VATS approaches were rated significantly differently for the thinking domain (data not shown). In contrast, scores for trainees and attending surgeons were similarly skewed toward naïve/beginning learner for skill, independence, and overall performance (Table 4).
Table 2. Mean Task Load Scores Before Performance and After Performance Component Timing Before performance After performance p value (t test) Before performance After performance p value (t test) Before performance After performance p value (t test) ANOVA ¼ analysis of variance;
Approach VATS
Robotic
Both
Pulmonary Ligament
Level 7
Level 4/5
p Value (ANOVA)
12.2 4.0 6.4 2.9 <0.001 12.7 5.0 9.8 4.8 0.233 12.5 4.4 7.1 3.6 <0.001
14.0 3.3 8.4 2.8 <0.001 14.1 4.3 12.4 3.1 0.484 14.1 3.8 8.9 3.2 <0.001
14.1 3.1 9.3 2.6 <0.001 13.8 4.1 13.7 1.8 0.967 13.9 3.5 10.1 2.9 <0.001
0.379 0.002 . 0.753 0.275 . 0.340 0.003 .
VATS ¼ video-assisted thoracic surgery.
4
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
Ann Thorac Surg 2014;-:-–-
Table 3. Changes in Mean Task Load With Repeat Component Performance Component Repetition Number 1
2
3
p Value (ANOVA)
8.2 3.3 9.0 3.3 10.9 2.3
8.1 4.1 9.5 2.1 10.8 4.0
5.1 2.8 8.1 3.4 8.4 2.4
0.050 0.744 0.154
Component Pulmonary ligament Level 7 Level 4/5
Note: robotic and video-assisted thoracic surgery procedures are combined within each component category. ANOVA ¼ analysis of variance.
These findings were amplified by kappa and tau scores for each domain (Table 5). Kappa scores (assessing exact matches) demonstrated slight agreement for knowledge and independence and fair agreement for the other domains. The best agreement was for skill, which ranked as moderate. In contrast, Kendall’s tau values (assessing exact and close matches) were overall high, indicating substantial agreement between trainees and attending surgeons for all domains. Together, the kappa and tau statistics indicate that agreement between rater groups was not very good when exact matches were considered, but that there was very good agreement between groups when individual ratings that were relatively close to each other in rank were considered. Performance ratings improved across STAT domains with repeated performances of components when surgical approaches were combined. This was evident for all three components and both sets of raters. Performance ratings did not importantly improve for robotic approaches (a maximum of only two repetitions were available for analysis), but significantly improved for VATS approaches (Figs 1 through 3).
Comment Challenges to the education of surgical trainees in performing complex minimally invasive procedures are increasing in an environment in which the time allotted for training is shrinking. One response has been to develop courses with standardized performance metrics
through use of simulation, such as Fundamentals of Laparoscopic Surgery (FLS) [8]. Although the basic skills assessed in the FLS program may be applicable to VATS, they are neither sufficiently specific or nor complex enough to enable trainees to perform VATS lobectomy or other advanced thoracoscopic or laparoscopic surgery without significant additional training. A similar set of training fundamentals is being developed for robotic surgery, but those, too, will not directly benefit cardiothoracic trainees in the performance of complex robotic procedures. In addition, it is likely that the skills required for VATS lobectomy are different than those required for robotic lobectomy. The current pilot study examined early learners performing minimally invasive lobectomy to identify taskand approach-specific needs with regard to education and evaluation. We used trainee-rated reports of task load as well as trainee and attending surgeon evaluation of performance to assess differing challenges of VATS and robotic approaches to components of minimally invasive lobectomy. We found that, based on NASATLX scores before performance, early learners had limited insight into the diverse challenges of different components and approaches before performing the components. They overestimated task load for VATS approaches and were unable to distinguish between levels of difficulty among different components. After trainees were engaged in performing minimally invasive lobectomy, they accurately calibrated differing task loads for individual components and reported
Table 4. Performance Scores for All Components According to Surgical Training and Assessment Tool Domains Trainee Performance Self-Assessment STAT Domain Knowledge Thinking Skill Independence Overall
Naïve or Beginning Learner (%) 21 35 47 60 53
Advanced Learner or Competent (%)
(24.1) (40.2) (54.0) (69.0) (60.9)
66 52 40 27 34 p < 0.001
(75.9) (59.8) (46.0) (31.0) (39.1)
Attending Surgeon Performance Assessment Naïve or Beginning Learner (%) 38 52 51 61 53
Advanced Learner or Competent (%)
(43.7) (59.8) (58.6) (70.1) (60.9)
49 35 36 26 34
(56.3) (40.2) (41.4) (29.9) (39.1)
p Value 0.006 0.010 0.541 0.869 1.000
p ¼ 0.011
Rows represent c2 tests comparing trainees to attending surgeons within STAT domains for all components, and paired columns represent c2 tests among STAT domains for all components. STAT ¼ Surgical Training and Assessment Tool.
Ann Thorac Surg 2014;-:-–-
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
5
Table 5. Interrater Exact Agreement (Kappa) and Rank Correlation (Kendall’s Tau) for All Components by Surgical Training and Assessment Tool Domain Kappa Domain
Naïve
Beginning Learner
Knowledge Thinking Skill Independence Overall
0.024 0.055 0.504 0.349 0.231
0.229 0.210 0.445 0.252 0.333
Advanced Learner
Competent
0.120 0.308 0.483 0.418 0.386
0.074 0.030 0.012 0.012 0.018
differences in task load between VATS and robotic approaches. A moderate number of repetitions of specific components did not alter the task load for most components. In self-assessment of their performance using STAT, trainees rated their knowledge and thinking substantially higher than did attending surgeons, whereas both trainees and attending surgeons similarly rated skill, independence, and overall performance as fair. A relatively small number of repetitions of a component was associated with a substantial increase in performance ratings by both trainees and attending surgeons, especially for VATS approaches. The tools used in this study have been previously validated in other settings. NASATLX has demonstrated utility in laparoscopic cholecystectomy [9] and has been used to validate and evaluate performance during simulation exercises [10–12]. An alternative multidimensional scale evaluating task load related to surgery is the surgery task load index (SURG-TLX). It has the potential advantage of discriminating different sources of stress, but has been evaluated thus far only in a simulation setting and was not available at the time this study was designed [13]. There are a variety of methods of evaluating procedurerelated performance. For this study, we selected a modified version of STAT, which enables self-ratings and
Overall
Kendall’s Tau
p Value for Kendall’s Tau
0.134 0.218 0.461 0.325 0.328
0.610 0.735 0.789 0.748 0.735
0.080 0.003 <0.001 0.002 0.003
expert ratings and which we have used for trainee performance evaluation for several years. The STAT was designed to be comprehensive and flexible, and as a result lacks elements specific to minimally invasive operations. Validated performance evaluation tools specific to minimally invasive surgery include the global assessment tool of intraoperative laparoscopic skills (GOALS) and global evaluative assessment of robotic skills (GEARS) [14, 15]. These tools have the advantage of rating task-specific variables such as depth perception and bimanual dexterity, but do not provide assessments aimed at development of competence or mastery of a specific portion of a particular procedure. We are not aware of a validated performance rating tool that is specific for VATS. Most studies comparing minimally invasive to roboticassisted minimally invasive surgery have involved simulation rather than actual operations, with conflicting results. The learning curve for simulated robotic tasks was longer than for laparoscopic tasks, whereas robotic assistance aided novices in performing tasks faster, an advantage that disappeared in experienced surgeons [16]. In another series of studies using simulation, intracorporeal suturing task load was reduced by the addition of robotic assistance [17–19]. There are limited studies that compare outcomes between VATS and robotic lobectomy. Most evaluate perioperative results
Fig 1. Changes in the percentage of performances rated as advanced learner/competent for the Surgical Training and Assessment Tool (STAT) domains according to the number of components performed (1, 2, 3) grouped by performance graders. (Trainee 1 ¼ brown bars; trainee 2 ¼ orange bars; trainee 3 ¼ lime green bars; attending surgeon 1 ¼ turquoise bars; attending surgeon 2 ¼ dark blue bars; attending surgeon 3 ¼ purple bars; Att Surg ¼ attending surgeon.)
6
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
Ann Thorac Surg 2014;-:-–-
Fig 2. Changes in the percentage of performances rated as advanced learner/competent for the Surgical Training and Assessment Tool (STAT) domains according to the number of components performed (1, 2, 3) grouped by surgical components. Values are for video-assisted thorascopic surgery and robotic approaches combined. (Pulmonary ligament 1 ¼ brown bars; pulmonary ligament 2 ¼ orange bars; pulmonary ligament 3 ¼ lime green bars; level 7, 1 ¼ turquoise bars; level 7, 2 ¼ dark blue bars; level 7, 3 ¼ purple bars; level 4/5, 1 ¼ orange bars; level 4/5, 2 ¼ lime green bars; level 4/5, 3 ¼ green bars.)
and focus on feasibility, safety, efficacy, and efficiency [2, 4, 20]. None of these studies compares work load or performance between laparoscopic/VATS and robotic approaches to operations. There are several potential limitations to this pilot study. The numbers of components, participating surgeons, and participating trainees were relatively small. That led to the inability to analyze paired data more directly comparing robotic to VATS approaches and prevented parsing of differences among trainees and among attending surgeons. The small numbers of robotically performed components, largely a result of limited access to the robot, also opened the possibility of a type II error in our analyses. We were only able to focus on the beginning steps of minimally invasive lobectomy, and we suspect that more challenging components may have revealed more striking differences between VATS and robotic approaches. Our study demonstrated the utility of NASATLX in evaluating task load and trainee performance of components of a complex surgical task. We identified fundamental differences in task load for individual components of and different approaches to minimally invasive
Fig 3. Changes in the percentage of performances rated as advanced learner/competent for the Surgical Training and Assessment Tool (STAT) domains according to the number of components performed (1, 2, 3) grouped by surgical approach. (Robotic 1 ¼ brown bars; robotic 2 ¼ orange bars; videoassisted thoracic surgery [VATS] 1 ¼ green bars; VATS 2 ¼ turquoise bars; VATS 3 ¼ dark blue bars.)
lobectomy. The ability to rank components according to task load will help identify which components should be the focus of specific simulation exercises. The clear difference in task load between VATS and robotic approaches indicates that more time spent in robotic simulation training may be necessary to create equivalent task loads for individual components to facilitate adequate progress toward competency. It has been shown that intentionally increasing task load as a means to improve proficiency does not produce the desired effect [21]. The utility of STAT in assessing performance was also demonstrated. Trainees rated their thinking and knowledge as more advanced than did attending surgeons, suggesting that more time should be spent preoperatively on understanding specific challenges of individual components. This investment of time may bring knowledge and thinking ratings into better alignment and mitigate the cognitive conflict between trainees and surgeons. Finally, the improvement in performances ratings, despite lack of important change in task loads, indicates that trainees progress toward competence at a relatively fast pace if they are able to perform multiple repetitions of a component in a short time.
Ann Thorac Surg 2014;-:-–-
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
This study was supported by an educational research grant from Intuitive Surgical, Inc, Sunnyvale, California.
11.
References
12.
1. Kent M, Wang T, Whyte R, Curran T, Flores R, Gangadharan S. Open, video-assisted thoracic surgery, and robotic lobectomy: review of a national database. Ann Thorac Surg 2014;97:236–44. 2. Cerfolio RJ, Bryant AS, Skylizard L, Minnich DJ. Initial consecutive experience of completely portal robotic pulmonary resection with 4 arms. J Thorac Cardiovasc Surg 2011;142:740–6. 3. Jang HJ, Lee HS, Park SY, Zo JI. Comparison of the early robot-assisted lobectomy experience to video-assisted thoracic surgery lobectomy for lung cancer: a singleinstitution case series matching study. Innovations (Phila) 2011;6:305–10. 4. Augustin F, Bodner J, Maier H, et al. Robotic-assisted minimally invasive vs. thoracoscopic lung lobectomy: comparison of perioperative results in a learning curve setting. Langenbecks Arch Surg 2013;398:895–901. 5. NASA. NASA Task Load Index (TLX): computerized version (version 2.0) [computer software]. Moffett Field, CA: NASAAmes Research Center, Aerospace Human Factors Research Division, 2003. 6. Roach PB, Roggin KK, Selkov G, Posner MC, Silverstein JC. Continuous, data-rich appraisal of surgical trainees’ operative abilities: a novel approach for measuring performance and providing feedback. J Surg Educ 2009;66:255–63. 7. Roach PB, Roggin KK, Selkov E, Posner MC, Silverstein JC. Use of a novel, web-based educational platform facilitates intraoperative training in a surgical oncology fellowship program. Ann Surg Oncol 2009;16:1100–7. 8. Peters JH, Fried GM, Swanstrom LL, et al. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 2004;135:21–7. 9. Youssef Y, Lee G, Godinez C, et al. Laparoscopic cholecystectomy poses physical injury risk to surgeons: analysis of hand technique and standing position. Surg Endosc 2011;25: 2168–74. 10. Montero PN, Acker CE, Heniford BT, Stefanidis D. Single incision laparoscopic surgery (SILS) is associated with poorer
13.
14. 15.
16.
17. 18.
19.
20. 21.
7
performance and increased surgeon workload compared with standard laparoscopy. Am Surg 2011;77:73–7. Dulan G, Rege RV, Hogg DC, et al. Proficiency-based training for robotic surgery: construct validity, workload, and expert levels for nine inanimate exercises. Surg Endosc 2012;26:1516–21. Yurko YY, Scerbo MW, Prabhu AS, Acker CE, Stefanidis D. Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool. Simul Healthc 2010;5:267–71. Wilson MR, Poolton JM, Malhotra N, Ngo K, Bright E, Masters RS. Development and validation of a surgical workload measure: the surgery task load index (SURG-TLX). World J Surg 2011;35:1961–9. Vassiliou MC, Feldman LS, Andrew CG, et al. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 2005;190:107–13. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ. Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 2012;187:247–52. Heemskerk J, van Gemert WG, de Vries J, Greve J, Bouvy ND. Learning curves of robot-assisted laparoscopic surgery compared with conventional laparoscopic surgery: an experimental study evaluating skill acquisition of robotassisted laparoscopic tasks compared with conventional laparoscopic tasks in inexperienced users. Surg Laparosc Endosc Percutan Tech 2007;17:171–4. Chandra V, Nehra D, Parent R, et al. A comparison of laparoscopic and robotic assisted suturing performance by experts and novices. Surgery 2010;147:830–9. Stefanidis D, Wang F, Korndorffer JR, Dunne JB, Scott DJ. Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surg Endosc 2010;24:377–82. Stefanidis D, Hope WW, Scott DJ. Robotic suturing on the FLS model possesses construct validity, is less physically demanding, and is favored by more surgeons compared with laparoscopy. Surg Endosc 2011;25:2141–6. Dylewski MR, Ohaeto AC, Pereira JF. Pulmonary resection using a total endoscopic robotic video-assisted approach. Semin Thorac Cardiovasc Surg 2011;23:36–42. Stefanidis D, Korndorffer JR, Markley S, Sierra R, Heniford BT, Scott DJ. Closing the gap in operative performance between novices and experts: does harder mean better for laparoscopic simulator training? J Am Coll Surg 2007;205:307–13.
8
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
Ann Thorac Surg 2014;-:-–-
Appendix 1. NASATLX Scoring Instrument Mental demand How mentally demanding was the task?
Physical demand How physically demanding was the task?
Temporal demand How hurried or rushed was the pace of the task?
Performance How successful were you in accomplishing what you were asked to do?
Effort How hard did you have to work to accomplish your level of performance?
Frustration How insecure, discouraged, irritated, stressed, and annoyed were you?
After Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, eds. Human mental workload. Amsterdam: North Holland Press, 1988.
Appendix 2. Surgical Training and Assessment Tool Scoring Instrument Assessment STAT Domain Knowledge Understood rational, anatomy; recognized potential hazards Thinking Asked for retraction; positioned assistants; instrument selection; recognized problems if they arose Skill Technical aptitude; efficiency; completion of component Independence How much guidance was necessary; did trainee make an error that required correction Overall Adapted from Roach et al [6]. STAT ¼ Surgical Training and Assessment Tool.
Naïve
Beginning Learner
Advanced Learner
Competent
Background. Skills required for thoracoscopic and robotic operations likely differ. The needs and abilities of trainees learning these approaches require assessment. Methods. Trainees performed initial components of minimally invasive lobectomies using thoracoscopic or robotic approaches. Component difficulty was scored by trainees using the NASA task load index (NASATLX). Performance of each component was graded by trainees and attending surgeons on a 5-point ordinal scale (naïve, beginning learner, advanced learner, competent, master). Results. Eleven surgical trainees performed 87 replications among three lobectomy components (divide pulmonary ligament; dissect level 7/8/9 nodes; dissect level 4/5 nodes). Before performance NASATLX scores did not differ among components or between surgical approaches. Trainees’ after performance NASATLX scores appropriately calibrated task load for the components. After performance NASATLX scores were significantly lower for thoracoscopy than before performance estimates; robotic scores were similar before surgery and
after performance. Task load was higher for robotic than for thoracoscopic approaches. Trainees rated their performance higher than did attending surgeons in domains of knowledge and thinking, but ratings for other domains were similarly low. Ratings for performance improved significantly as component performance repetitions increased. Conclusions. Trainees did not differentiate task load among components or surgical approaches before attempting them. Task load scores differentiated difficulty among initial components of lobectomy, and were greater for robotic than for thoracoscopic approaches. Trainees overestimated their level of cognitive performance compared with attending physician evaluation of trainee performance. The study provides insights into how to customize training for thoracoscopic and robotic lobectomy and identifies tools to assess training effectiveness.
T
sometimes a different physical orientation for the surgeon (from anterior or inferior rather than from posterior), moderate magnification, and a two-dimensional view of the field; haptic feedback is largely retained. In contrast, robotic procedures, when compared with VATS procedures, often maintain the same surgical field orientation but involve substantially greater magnification, a threedimensional view, and loss of haptic feedback. Assessment of perceived difficulty and performance of specific steps of lobectomy may help identify components of this complex operation that are more challenging for VATS or robotic approaches. Such knowledge may help determine trainee educational needs to design curricular improvements in minimally invasive lobectomy training. We performed a pilot investigation of trainee ratings of component difficulty, the association of surgical approach to component difficulty, and trainee and surgical attending ratings of learner performance.
he explosion of minimally invasive surgery in the 1990s led to new approaches to caring for surgical patients, including thoracic surgical patients. In many centers, video-assisted thoracic surgery (VATS) lobectomy is now the most common technique for performing lung resections, and penetrance overall is approximately 40% [1]. A similar transition to robotic lobectomy has not yet occurred despite the more than 10-year interval since the first clinical reports of robotic lobectomy, with a recent publication citing a 3.4% penetrance in 2010 [1]. The reasons for this are manifold, including costs, availability, lack of standardized approaches to robotic lobectomy, lack of evidence for improved outcomes in the face of increased costs, and challenges in teaching new skills required for robotic lobectomy [2–4]. Robotic procedures require a different skill set than do thoracoscopic procedures, possibly to the same extent that thoracoscopic procedures require a different skill set than open thoracic procedures. Compared with open surgery, VATS operations often entail a different visual and
(Ann Thorac Surg 2014;-:-–-) Ó 2014 by The Society of Thoracic Surgeons
Material and Methods Accepted for publication Jan 14, 2014. Address correspondence to Dr Ferguson, Department of Surgery, University of Chicago, 5841 S Maryland Ave, MC 5040, Chicago, IL 60637; e-mail: [email protected].
Ó 2014 by The Society of Thoracic Surgeons Published by Elsevier Inc
This was a prospective single-institution pilot study evaluating task load and performance characteristics of senior level surgical trainees participating in minimally invasive pulmonary lobectomies. 0003-4975/$36.00 http://dx.doi.org/10.1016/j.athoracsur.2014.01.055
2
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
Subjects Study subjects (trainees) included senior general surgical residents and cardiothoracic surgery fellows who were in the process of learning minimally invasive lobectomy using the da Vinci robot (Intuitive Surgical, Sunnyvale, CA) or VATS. Inclusion criteria were (1) anticipated performance of 5 or more components using VATS or the robot during the study period; (2) performance of 2 or fewer components before study entry by any approach (VATS, robotic, open); (3) presence of an attending surgeon during the performance of study components; and (4) completion of a minimum of 1 hour of mentored practice with the robot before beginning the clinical portion of the study. Otherwise, robotic and VATS simulation exercises were not offered to the participants. Prior experience with laparoscopy was not assessed as part of this study, but was assumed to be extensive. Institutional Review Board approval was obtained before initiation of the study, and trainees were required to sign a study-specific consent form.
Component Definition Lobectomy was divided into individual components (Table 1). Components were characterized according to the perceived level of difficulty based on procedural and cognitive content as low, medium, or high. Components were selected for evaluation in this study that were feasible for all trainees to perform safely and effectively under proper supervision (components 1 through 4 in Table 1).
Subject Performance of Components Trainees were required to have reviewed the componentspecific steps using self-selected materials, to have discussed each with the attending surgeon, and to have completed an initial self-evaluation of difficulty before participating performance of the component. This initial evaluation of difficulty was intended to come from a naïve perspective. Trainees performed individual components using robotic or VATS techniques according to the clinical needs of the patient as determined by the attending surgeon and according to the attending surgeon’s perception of the trainee’s comfort/skill level. It was the attending surgeon’s responsibility to exercise appropriate clinical judgment in ensuring that performance of each component was proceeding safely and adequately if a Table 1. Components of Minimally Invasive Lobectomy Component 1a 2a 3a 4a 5 6 7 a
Description Divide pulmonary ligament Dissect level 8, 9, 7 lymph nodes Dissect level 4 or 5/6 lymph nodes Dissect, divide pulmonary vein Dissect, divide pulmonary artery branches Dissect, divide bronchus Complete fissure(s)
Components that were included as study components.
Difficulty Low Medium Medium Medium High High Medium
Ann Thorac Surg 2014;-:-–-
trainee was involved in its performance regardless of whether the performance of that component was being studied. It was not necessary that trainees performed most or all of a lobectomy to participate in an operation for purposes of this study. Completion of a component by the trainee was not required for the trainee to be able to participate in subsequent evaluation of the performance of that component.
Evaluation of Component Performance After performing a component, the trainee immediately rated that component using the NASA task load index (NASATLX; version 2.0) [5]. Task load is a subjective estimate of overall workload of a specific task. This instrument incorporates separate visual scales graded from 1 to 21 for mental demands, physical demands, temporal demands, own performance, effort, and frustration (Appendix 1). Trainees also immediately rated their performance on each component using a modified version of the Surgical Training and Assessment Tool (STAT) [6, 7]. Domains of performance included knowledge (understood rationale, anatomy, potential hazards); thinking (asked for retraction, positioned assistants, instrument selection, recognized problems if they arose); skill (technical aptitude including instrument and tissue handling, efficiency, did the trainee complete the component); independence (was guidance necessary in anatomy, instrument selection, or approach; did the trainee make an error in thought or action that required correction); and overall. Domains were rated using the choices “naïve,” “beginning learner,” “advanced learner,” “competent,” and “master” in an ordinal scale (Appendix 2). At the conclusion of each operation, the attending surgeon also rated trainee performance using STAT.
Statistical Analyses The NASATLX overall mean scores were calculated from the subscales; weighted NASATLX scores were not calculated. The STAT results were evaluated according to each of the STAT domains. Numerical data were compared using analysis of variance or unpaired t test, and categorical data were compared using c2 analysis. Kappa values (assessing exact matches among raters; a value of 1 indicates complete agreement, a value of 0 indicates that agreement occurred by chance) and Kendall’s tau (assessing rank correlation, in which proximity to another score was taken into account; a value of 1 indicates complete agreement, a value of 0 indicates that agreement occurred by chance) were calculated. All computations were performed using Minitab 16 (Minitab Inc, State College, PA).
Results Subjects and Components Eleven trainees completed 97 components of VATS or robotic lobectomy in 37 patients from February 2012 through January 2013. Nine were senior general surgery residents and two were cardiothoracic fellows. Two
Ann Thorac Surg 2014;-:-–-
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
attending thoracic surgeons participated in the study. There were three components that had adequate numbers to permit analysis, totaling 87 component performances: divide pulmonary ligament, dissect level 7/8/9 lymph nodes, and dissect level 4/5 lymph nodes (level 4 for right-sided resections, level 5 for left-sided resections).
3
or more performance, but borderline statistical significance was achieved only for the component with the lowest task load—divide pulmonary ligament—driven by significant task load decreases over time for temporal, effort, and frustration elements (Table 3). This finding indicates that a reduction in task load requires more than a few repetitions of a component.
Task Load Trainees did not differentiate among components or between approaches in their before performance task load scores. That was true for mean NASATLX scores as well as for each individual NASATLX element. Mean after performance task load scores differentiated among the three components, and this was also true for each NASATLX element except for task load associated with performance. These findings indicate appropriate calibration of difficulty by trainees relative to investigators’ estimated difficulty. This calibration was evident for VATS approaches, but there was no statistical difference among components performed robotically (Table 2). After performance mean task load scores were significantly lower for components performed using a VATS approach compared with before performance scores; these significant differences were evident for the first VATS performance of a component and thus were not entirely a result of learning with repeated VATS performances of a component. That indicates that trainees substantially overestimated task load associated with components of VATS lobectomy before actually performing these components. Task load scores for robotic components were not significantly different comparing before and after performance scores, indicating that the trainees accurately estimated robotic component task load before performing the components. The VATS scores after performance were significantly lower than robotic scores for each component (p < 0.05 for each), indicating that actual task load was lower for VATS than for robotic approaches.
Effect of Learning on Task Load Mean task loads changed somewhat with repeat performance of components, grouped as first, second, and third
Performance Scores The STAT performance scores assigned by trainees and attending surgeons ranged from naïve to competent. Performance scores were skewed toward the intermediate ratings, with only 73 (8.4%) rated as naïve and 24 (2.8%) rated as competent. The distribution among trainees was different than that among attending surgeons, with trainees demonstrating a higher likelihood of rating their performance as competent and a lower likelihood of rating their performance as naive (p ¼ 0.003). Because of the small numbers of performance scores in the naïve and competent categories, performance scores were combined into two categories for further analysis: “naïve or beginning learner” and “advanced learner or competent.” The distribution among domain scores was significantly different for both trainees and attending surgeons. This finding indicates that trainees and attending surgeons are able to discriminate among the domains in assigning scores (Table 4). Trainees rated their performance significantly higher than did attending surgeons in the domains of knowledge and thinking. That was related in part to significant differences in these domains for the more difficult components, whereas performance ratings in these domains for division of the pulmonary ligament were similar. Robotic approaches were rated significantly differently by trainees and attending surgeons for the knowledge domain, whereas VATS approaches were rated significantly differently for the thinking domain (data not shown). In contrast, scores for trainees and attending surgeons were similarly skewed toward naïve/beginning learner for skill, independence, and overall performance (Table 4).
Table 2. Mean Task Load Scores Before Performance and After Performance Component Timing Before performance After performance p value (t test) Before performance After performance p value (t test) Before performance After performance p value (t test) ANOVA ¼ analysis of variance;
Approach VATS
Robotic
Both
Pulmonary Ligament
Level 7
Level 4/5
p Value (ANOVA)
12.2 4.0 6.4 2.9 <0.001 12.7 5.0 9.8 4.8 0.233 12.5 4.4 7.1 3.6 <0.001
14.0 3.3 8.4 2.8 <0.001 14.1 4.3 12.4 3.1 0.484 14.1 3.8 8.9 3.2 <0.001
14.1 3.1 9.3 2.6 <0.001 13.8 4.1 13.7 1.8 0.967 13.9 3.5 10.1 2.9 <0.001
0.379 0.002 . 0.753 0.275 . 0.340 0.003 .
VATS ¼ video-assisted thoracic surgery.
4
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
Ann Thorac Surg 2014;-:-–-
Table 3. Changes in Mean Task Load With Repeat Component Performance Component Repetition Number 1
2
3
p Value (ANOVA)
8.2 3.3 9.0 3.3 10.9 2.3
8.1 4.1 9.5 2.1 10.8 4.0
5.1 2.8 8.1 3.4 8.4 2.4
0.050 0.744 0.154
Component Pulmonary ligament Level 7 Level 4/5
Note: robotic and video-assisted thoracic surgery procedures are combined within each component category. ANOVA ¼ analysis of variance.
These findings were amplified by kappa and tau scores for each domain (Table 5). Kappa scores (assessing exact matches) demonstrated slight agreement for knowledge and independence and fair agreement for the other domains. The best agreement was for skill, which ranked as moderate. In contrast, Kendall’s tau values (assessing exact and close matches) were overall high, indicating substantial agreement between trainees and attending surgeons for all domains. Together, the kappa and tau statistics indicate that agreement between rater groups was not very good when exact matches were considered, but that there was very good agreement between groups when individual ratings that were relatively close to each other in rank were considered. Performance ratings improved across STAT domains with repeated performances of components when surgical approaches were combined. This was evident for all three components and both sets of raters. Performance ratings did not importantly improve for robotic approaches (a maximum of only two repetitions were available for analysis), but significantly improved for VATS approaches (Figs 1 through 3).
Comment Challenges to the education of surgical trainees in performing complex minimally invasive procedures are increasing in an environment in which the time allotted for training is shrinking. One response has been to develop courses with standardized performance metrics
through use of simulation, such as Fundamentals of Laparoscopic Surgery (FLS) [8]. Although the basic skills assessed in the FLS program may be applicable to VATS, they are neither sufficiently specific or nor complex enough to enable trainees to perform VATS lobectomy or other advanced thoracoscopic or laparoscopic surgery without significant additional training. A similar set of training fundamentals is being developed for robotic surgery, but those, too, will not directly benefit cardiothoracic trainees in the performance of complex robotic procedures. In addition, it is likely that the skills required for VATS lobectomy are different than those required for robotic lobectomy. The current pilot study examined early learners performing minimally invasive lobectomy to identify taskand approach-specific needs with regard to education and evaluation. We used trainee-rated reports of task load as well as trainee and attending surgeon evaluation of performance to assess differing challenges of VATS and robotic approaches to components of minimally invasive lobectomy. We found that, based on NASATLX scores before performance, early learners had limited insight into the diverse challenges of different components and approaches before performing the components. They overestimated task load for VATS approaches and were unable to distinguish between levels of difficulty among different components. After trainees were engaged in performing minimally invasive lobectomy, they accurately calibrated differing task loads for individual components and reported
Table 4. Performance Scores for All Components According to Surgical Training and Assessment Tool Domains Trainee Performance Self-Assessment STAT Domain Knowledge Thinking Skill Independence Overall
Naïve or Beginning Learner (%) 21 35 47 60 53
Advanced Learner or Competent (%)
(24.1) (40.2) (54.0) (69.0) (60.9)
66 52 40 27 34 p < 0.001
(75.9) (59.8) (46.0) (31.0) (39.1)
Attending Surgeon Performance Assessment Naïve or Beginning Learner (%) 38 52 51 61 53
Advanced Learner or Competent (%)
(43.7) (59.8) (58.6) (70.1) (60.9)
49 35 36 26 34
(56.3) (40.2) (41.4) (29.9) (39.1)
p Value 0.006 0.010 0.541 0.869 1.000
p ¼ 0.011
Rows represent c2 tests comparing trainees to attending surgeons within STAT domains for all components, and paired columns represent c2 tests among STAT domains for all components. STAT ¼ Surgical Training and Assessment Tool.
Ann Thorac Surg 2014;-:-–-
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
5
Table 5. Interrater Exact Agreement (Kappa) and Rank Correlation (Kendall’s Tau) for All Components by Surgical Training and Assessment Tool Domain Kappa Domain
Naïve
Beginning Learner
Knowledge Thinking Skill Independence Overall
0.024 0.055 0.504 0.349 0.231
0.229 0.210 0.445 0.252 0.333
Advanced Learner
Competent
0.120 0.308 0.483 0.418 0.386
0.074 0.030 0.012 0.012 0.018
differences in task load between VATS and robotic approaches. A moderate number of repetitions of specific components did not alter the task load for most components. In self-assessment of their performance using STAT, trainees rated their knowledge and thinking substantially higher than did attending surgeons, whereas both trainees and attending surgeons similarly rated skill, independence, and overall performance as fair. A relatively small number of repetitions of a component was associated with a substantial increase in performance ratings by both trainees and attending surgeons, especially for VATS approaches. The tools used in this study have been previously validated in other settings. NASATLX has demonstrated utility in laparoscopic cholecystectomy [9] and has been used to validate and evaluate performance during simulation exercises [10–12]. An alternative multidimensional scale evaluating task load related to surgery is the surgery task load index (SURG-TLX). It has the potential advantage of discriminating different sources of stress, but has been evaluated thus far only in a simulation setting and was not available at the time this study was designed [13]. There are a variety of methods of evaluating procedurerelated performance. For this study, we selected a modified version of STAT, which enables self-ratings and
Overall
Kendall’s Tau
p Value for Kendall’s Tau
0.134 0.218 0.461 0.325 0.328
0.610 0.735 0.789 0.748 0.735
0.080 0.003 <0.001 0.002 0.003
expert ratings and which we have used for trainee performance evaluation for several years. The STAT was designed to be comprehensive and flexible, and as a result lacks elements specific to minimally invasive operations. Validated performance evaluation tools specific to minimally invasive surgery include the global assessment tool of intraoperative laparoscopic skills (GOALS) and global evaluative assessment of robotic skills (GEARS) [14, 15]. These tools have the advantage of rating task-specific variables such as depth perception and bimanual dexterity, but do not provide assessments aimed at development of competence or mastery of a specific portion of a particular procedure. We are not aware of a validated performance rating tool that is specific for VATS. Most studies comparing minimally invasive to roboticassisted minimally invasive surgery have involved simulation rather than actual operations, with conflicting results. The learning curve for simulated robotic tasks was longer than for laparoscopic tasks, whereas robotic assistance aided novices in performing tasks faster, an advantage that disappeared in experienced surgeons [16]. In another series of studies using simulation, intracorporeal suturing task load was reduced by the addition of robotic assistance [17–19]. There are limited studies that compare outcomes between VATS and robotic lobectomy. Most evaluate perioperative results
Fig 1. Changes in the percentage of performances rated as advanced learner/competent for the Surgical Training and Assessment Tool (STAT) domains according to the number of components performed (1, 2, 3) grouped by performance graders. (Trainee 1 ¼ brown bars; trainee 2 ¼ orange bars; trainee 3 ¼ lime green bars; attending surgeon 1 ¼ turquoise bars; attending surgeon 2 ¼ dark blue bars; attending surgeon 3 ¼ purple bars; Att Surg ¼ attending surgeon.)
6
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
Ann Thorac Surg 2014;-:-–-
Fig 2. Changes in the percentage of performances rated as advanced learner/competent for the Surgical Training and Assessment Tool (STAT) domains according to the number of components performed (1, 2, 3) grouped by surgical components. Values are for video-assisted thorascopic surgery and robotic approaches combined. (Pulmonary ligament 1 ¼ brown bars; pulmonary ligament 2 ¼ orange bars; pulmonary ligament 3 ¼ lime green bars; level 7, 1 ¼ turquoise bars; level 7, 2 ¼ dark blue bars; level 7, 3 ¼ purple bars; level 4/5, 1 ¼ orange bars; level 4/5, 2 ¼ lime green bars; level 4/5, 3 ¼ green bars.)
and focus on feasibility, safety, efficacy, and efficiency [2, 4, 20]. None of these studies compares work load or performance between laparoscopic/VATS and robotic approaches to operations. There are several potential limitations to this pilot study. The numbers of components, participating surgeons, and participating trainees were relatively small. That led to the inability to analyze paired data more directly comparing robotic to VATS approaches and prevented parsing of differences among trainees and among attending surgeons. The small numbers of robotically performed components, largely a result of limited access to the robot, also opened the possibility of a type II error in our analyses. We were only able to focus on the beginning steps of minimally invasive lobectomy, and we suspect that more challenging components may have revealed more striking differences between VATS and robotic approaches. Our study demonstrated the utility of NASATLX in evaluating task load and trainee performance of components of a complex surgical task. We identified fundamental differences in task load for individual components of and different approaches to minimally invasive
Fig 3. Changes in the percentage of performances rated as advanced learner/competent for the Surgical Training and Assessment Tool (STAT) domains according to the number of components performed (1, 2, 3) grouped by surgical approach. (Robotic 1 ¼ brown bars; robotic 2 ¼ orange bars; videoassisted thoracic surgery [VATS] 1 ¼ green bars; VATS 2 ¼ turquoise bars; VATS 3 ¼ dark blue bars.)
lobectomy. The ability to rank components according to task load will help identify which components should be the focus of specific simulation exercises. The clear difference in task load between VATS and robotic approaches indicates that more time spent in robotic simulation training may be necessary to create equivalent task loads for individual components to facilitate adequate progress toward competency. It has been shown that intentionally increasing task load as a means to improve proficiency does not produce the desired effect [21]. The utility of STAT in assessing performance was also demonstrated. Trainees rated their thinking and knowledge as more advanced than did attending surgeons, suggesting that more time should be spent preoperatively on understanding specific challenges of individual components. This investment of time may bring knowledge and thinking ratings into better alignment and mitigate the cognitive conflict between trainees and surgeons. Finally, the improvement in performances ratings, despite lack of important change in task loads, indicates that trainees progress toward competence at a relatively fast pace if they are able to perform multiple repetitions of a component in a short time.
Ann Thorac Surg 2014;-:-–-
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
This study was supported by an educational research grant from Intuitive Surgical, Inc, Sunnyvale, California.
11.
References
12.
1. Kent M, Wang T, Whyte R, Curran T, Flores R, Gangadharan S. Open, video-assisted thoracic surgery, and robotic lobectomy: review of a national database. Ann Thorac Surg 2014;97:236–44. 2. Cerfolio RJ, Bryant AS, Skylizard L, Minnich DJ. Initial consecutive experience of completely portal robotic pulmonary resection with 4 arms. J Thorac Cardiovasc Surg 2011;142:740–6. 3. Jang HJ, Lee HS, Park SY, Zo JI. Comparison of the early robot-assisted lobectomy experience to video-assisted thoracic surgery lobectomy for lung cancer: a singleinstitution case series matching study. Innovations (Phila) 2011;6:305–10. 4. Augustin F, Bodner J, Maier H, et al. Robotic-assisted minimally invasive vs. thoracoscopic lung lobectomy: comparison of perioperative results in a learning curve setting. Langenbecks Arch Surg 2013;398:895–901. 5. NASA. NASA Task Load Index (TLX): computerized version (version 2.0) [computer software]. Moffett Field, CA: NASAAmes Research Center, Aerospace Human Factors Research Division, 2003. 6. Roach PB, Roggin KK, Selkov G, Posner MC, Silverstein JC. Continuous, data-rich appraisal of surgical trainees’ operative abilities: a novel approach for measuring performance and providing feedback. J Surg Educ 2009;66:255–63. 7. Roach PB, Roggin KK, Selkov E, Posner MC, Silverstein JC. Use of a novel, web-based educational platform facilitates intraoperative training in a surgical oncology fellowship program. Ann Surg Oncol 2009;16:1100–7. 8. Peters JH, Fried GM, Swanstrom LL, et al. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 2004;135:21–7. 9. Youssef Y, Lee G, Godinez C, et al. Laparoscopic cholecystectomy poses physical injury risk to surgeons: analysis of hand technique and standing position. Surg Endosc 2011;25: 2168–74. 10. Montero PN, Acker CE, Heniford BT, Stefanidis D. Single incision laparoscopic surgery (SILS) is associated with poorer
13.
14. 15.
16.
17. 18.
19.
20. 21.
7
performance and increased surgeon workload compared with standard laparoscopy. Am Surg 2011;77:73–7. Dulan G, Rege RV, Hogg DC, et al. Proficiency-based training for robotic surgery: construct validity, workload, and expert levels for nine inanimate exercises. Surg Endosc 2012;26:1516–21. Yurko YY, Scerbo MW, Prabhu AS, Acker CE, Stefanidis D. Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool. Simul Healthc 2010;5:267–71. Wilson MR, Poolton JM, Malhotra N, Ngo K, Bright E, Masters RS. Development and validation of a surgical workload measure: the surgery task load index (SURG-TLX). World J Surg 2011;35:1961–9. Vassiliou MC, Feldman LS, Andrew CG, et al. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 2005;190:107–13. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ. Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 2012;187:247–52. Heemskerk J, van Gemert WG, de Vries J, Greve J, Bouvy ND. Learning curves of robot-assisted laparoscopic surgery compared with conventional laparoscopic surgery: an experimental study evaluating skill acquisition of robotassisted laparoscopic tasks compared with conventional laparoscopic tasks in inexperienced users. Surg Laparosc Endosc Percutan Tech 2007;17:171–4. Chandra V, Nehra D, Parent R, et al. A comparison of laparoscopic and robotic assisted suturing performance by experts and novices. Surgery 2010;147:830–9. Stefanidis D, Wang F, Korndorffer JR, Dunne JB, Scott DJ. Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surg Endosc 2010;24:377–82. Stefanidis D, Hope WW, Scott DJ. Robotic suturing on the FLS model possesses construct validity, is less physically demanding, and is favored by more surgeons compared with laparoscopy. Surg Endosc 2011;25:2141–6. Dylewski MR, Ohaeto AC, Pereira JF. Pulmonary resection using a total endoscopic robotic video-assisted approach. Semin Thorac Cardiovasc Surg 2011;23:36–42. Stefanidis D, Korndorffer JR, Markley S, Sierra R, Heniford BT, Scott DJ. Closing the gap in operative performance between novices and experts: does harder mean better for laparoscopic simulator training? J Am Coll Surg 2007;205:307–13.
8
FERGUSON ET AL TRAINING IN MINIMALLY INVASIVE LOBECTOMY
Ann Thorac Surg 2014;-:-–-
Appendix 1. NASATLX Scoring Instrument Mental demand How mentally demanding was the task?
Physical demand How physically demanding was the task?
Temporal demand How hurried or rushed was the pace of the task?
Performance How successful were you in accomplishing what you were asked to do?
Effort How hard did you have to work to accomplish your level of performance?
Frustration How insecure, discouraged, irritated, stressed, and annoyed were you?
After Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, eds. Human mental workload. Amsterdam: North Holland Press, 1988.
Appendix 2. Surgical Training and Assessment Tool Scoring Instrument Assessment STAT Domain Knowledge Understood rational, anatomy; recognized potential hazards Thinking Asked for retraction; positioned assistants; instrument selection; recognized problems if they arose Skill Technical aptitude; efficiency; completion of component Independence How much guidance was necessary; did trainee make an error that required correction Overall Adapted from Roach et al [6]. STAT ¼ Surgical Training and Assessment Tool.
Naïve
Beginning Learner
Advanced Learner
Competent