- Email: [email protected]
Acquiring Surgical Skills: A Comparative Study of Open versus Laparoscopic Surgery
European Urology
European Urology 45 (2004) 346–351
Acquiring Surgical Skills: A Comparative Study of Open versus Laparoscopic Surgery Kesavapillai Subramoniana,*, Suren DeSylvaa, Peter Bishaib, Peter Thompsona, Gordon Muira a
Department of Urology, King’s College Hospital, London, UK Department of Surgery, King’s College Hospital, London, UK
b
Accepted 4 September 2003 Published online 24 October 2003
Abstract Objectives: A preliminary study to evaluate the feasibility of a protocol for comparing the learning curves for open and laparoscopic surgical procedures. Participants and Methods: Thirteen pre-clinical medical students with no previous surgical training were given intensive coaching in open and laparoscopic surgical techniques for 12 weeks. At the end of this period, their open and laparoscopic skills were assessed by three independent examiners. Individual and aggregate ability scores in various aspects of open and laparoscopic surgery and the time taken to perform the procedures were compared using Student’s t-test. Results: There was no statistically significant difference in the overall scores by the two different techniques ( p ¼ 0:057). However, differences between the two techniques were significant in certain criteria including tissue dissection ( p ¼ 0:024), tidiness of gall bladder ( p ¼ 0:034) and liver ( p ¼ 0:016) specimens and the time taken for the two techniques ( p 0:001). Conclusions: This study suggests that when inexperienced subjects are given equal training in laparoscopy and open surgery, the overall skills acquired were similar by both methods when assessed after 6 weeks. However, on detailed analysis of the different components of surgery, the laparoscopic skills were deficient in finer dissection, identification of correct planes and two-dimensional perception when compared to open surgery and required more operative time. Our study group perceived that laparoscopy was more difficult to learn than open surgery even after the training. The study group also felt that the training in basic surgical skills during their undergraduate careers would make them more interested in studying surgery and choosing it as a career. # 2003 Elsevier B.V. All rights reserved. Keywords: Learning curve; Laparoscopy; Open surgery; Surgical training
1. Introduction The history of surgical craft dates back to the history of mankind and until recently surgery has traditionally been taught by apprenticeship [1]. With modern surgical training, skills are acquired by a didactic process *
Corresponding author. Present address: 18 Place Farm Avenue, Orpington, BR6 8DQ, Kent, UK. Tel. þ44-7715172317; Fax: þ44-1322428779. E-mail address: [email protected] (K. Subramonian).
of observing, assisting and performing surgical procedures under supervision, often with practical workshops and simulators to assist evaluation. Training schemes vary in different parts of the world with regard to criteria for completion of training. Some schemes are time-based whereas others are competency-based. However, the competency achieved by a trainee to perform a particular procedure depends on the type and complexity of surgery, the aptitude and manual dexterity of the surgeon, and the quality of training received.
0302-2838/$ – see front matter # 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.eururo.2003.09.021
K. Subramonian et al. / European Urology 45 (2004) 346–351
While many papers have recently referred to the ‘‘learning curve’’, particularly with regard to minimally invasive surgery, it does not really have a clear definition. We believe that the ‘‘learning curve’’ could be best defined as the number of procedures an average surgeon needs to undertake in order to be able to confidently perform that procedure independently with a reasonable outcome. This does presuppose the ability to identify an ‘‘average’’ surgeon! While assessing the learning curve, several factors should be taken into account such as time taken for the procedure, frequency of performing the surgery, operative skill and the clinical outcome. Learning curves for each surgical procedure could be plotted with the number of procedures done on the X-axis and the above factors on the Y-axis. The shape of the curve may vary depending on the complexity of the procedure and the manual dexterity of the surgeon. It has been postulated that the learning curve for minimally invasive procedures is longer than that for open surgical procedures. This belief seems to have been proposed mainly by more senior open surgeons, but lacks any objective proof, since no work has addressed the place of a traditional surgical training in preparing surgeons to learn new procedures, nor dealt with the formidably steep curves that may be encountered during training for traditional procedures. While video gaming may equip lay persons to manage virtual reality surgical simulators to the same degree as experienced surgeons, there are major problems in extrapolating virtual reality results to the clinical environment. In a review of literature about learning curves in health technologies, Ramsay and colleagues describe various statistical methods used by the investigators such as simple graphs, splitting data chronologically and performing a t-test, curve fitting and other model fitting [2]. The ideal way of comparing the learning curve is to assess a cohort of surgeons performing open and laparoscopic surgery from the start and compare their curves. This is a significant undertaking needing a great deal of co-ordination. An alternative way is to assess the skills and operative time after a set number of procedures and compare the two at that point of the learning curve. This could be used as a proxy measure to compare the slope of the learning curves from the start to this point. One caveat of this method is the inability to predict the slope of the learning curve beyond the point of assessment. We wished to look at the hypothesis that the skills needed to perform laparoscopic surgery are so different to those needed for open surgery that the learning curve might be no different for given tasks in open and laparoscopic procedures. To assess the learning curve
347
of laparoscopic and open surgery independently, a group of medical students with no previous surgical experience were trained to perform a simulated procedure by the two different methods. We also wished to see if any preconceptions about technical difficulty were changed by a course of practical training. 2. Materials and methods Thirteen third-year medical students with no previous surgical training were recruited for the study. They were given intensive coaching for 4 hours per week for a period of 12 weeks. The teaching modules consisted of knot tying, tissue handling, tissue dissection and cholecystectomy in pig liver. These were performed by open and laparoscopic methods, with equal time apportioned to each approach. Initially the students were trained in using dry materials, but after the sixth week, inanimate ‘‘wet’’ materials (pig mesentery, pig vessels, pig liver and gall bladder) were introduced. In the wet lab training, each student performed five cholecystectomies before the end of training. On completion of the training period, their ability to perform cholecystectomy by the two methods was assessed by a practical examination in which each student performed an open and laparoscopic cholecystectomy in random order. The skills were scored by three independent examiners who were experienced surgeons in laparoscopic and/or open surgery. The assessment criteria included dissection of the cystic artery and cystic duct, instrument handling, knot tying/clipping, tissue dissection, tidiness of gall bladder specimen and liver bed, and the time taken for the procedures. A score of 1 to 4 was assigned for each criterion based on the performance (1 ¼ poor, 2 ¼ fair, 3 ¼ good and 4 ¼ excellent). Scoring was based on the direct observation by the examiners at the time of surgery. The mean score for each criterion was calculated to avoid inter-observer variation and aggregate scores for each candidate in the two methods calculated. The scores obtained by each student in the two techniques and the time taken was compared using Student’s t-test. The students were also asked to fill in questionnaires regarding their interest in surgery, and their perception about learning different techniques of surgery both before and after the training (see Appendix A). The perception scores before and after training were compared using paired sample t-test. Their self-assessed proficiency and time spent in playing video games in the previous year was also measured. These skills were correlated with the scores in the practical examination.
3. Results The average scores for each criterion in different techniques were plotted in a frequency distribution curve. These values followed a normal distribution and hence further statistical analysis was performed using paired sample t-test (Table 1). The mean scores for each criterion and the p-values are listed in Table 2. There was no statistically significant difference in the overall scores by the two different techniques ( p ¼ 0:057). However, differences between the two techniques were significant in certain criteria including
348
K. Subramonian et al. / European Urology 45 (2004) 346–351
Table 1 Paired sample t-test Comparison data
Paired differences Mean
Duct dissection Artery dissection Knot tying Instrument handling Dissection techniques Gall bladder specimen Liver bed Time taken
Df
Significance (2-tailed)
Std Deviation
Std error of mean
95% CI of the difference Lower
Upper
0.115 0.238 0.269 0.154 0.423 0.585 0.546 33.15
0.5398 0.5561 0.4820 0.5010 0.5890 0.8792 0.7067 19.304
0.1497 0.1542 0.1337 0.1390 0.1634 0.2439 0.1960 5.354
0.211 0.098 0.560 0.149 0.067 0.053 0.119 44.82
0.442 0.575 0.022 0.457 0.779 1.116 0.973 21.49
0.771 1.546 2.014 1.107 2.590 2.397 2.787 6.192
12 12 12 12 12 12 12 12
0.456 0.148 0.067 0.290 0.024 0.034 0.016 0.0001
1.715
2.9359
0.8143
0.059
3.490
2.107
12
0.057
Total score
Table 2 Showing mean scores by the open and laparoscopic methods Criterion
t
Mean scores Open method
Duct dissection 2.8 Artery dissection 2.9 Knot tying/clipping 2.5 Instrument handling 2.9 Dissection techniques 2.9 Gall bladder specimen 3.1 Liver bed 3.1 Total score 20.1 Total time 50.1
S.D.
p-value
0.540 0.556 0.482 0.501 0.589 0.879 0.707 2.936 19.304
0.456 0.148 0.067 0.290 0.024 0.034 0.016 0.057 0.000
Laparoscopy 2.7 2.7 2.7 2.7 2.4 2.5 2.6 18.4 83.2
tissue dissection ( p ¼ 0:024), tidiness of gall bladder ( p ¼ 0:034) and liver ( p ¼ 0:016) specimens and the time taken for the two techniques ( p 0:001). Analysis of the questionnaires showed that 10 out of 13 students wanted to be surgeons. All the students who wanted to be surgeons and two of the others were positively influenced by the training. By comparing those students who did or did not want to be surgeons using paired sample analysis, there was no significant
difference in surgical skills ( p 0:05). It was also found that those students with proficiency in playing video games did not differ in the open and laparoscopic skills from others. On analysis of the perception before the training, all the students perceived that learning laparoscopy was significantly more difficult than learning open surgery. ( p ¼ 0:011). Even after the training they perceived that laparoscopy was significantly more difficult than open surgery ( p ¼ 0:004). However, they found that both techniques were comparatively easier than they had initially perceived (Table 3).
4. Discussion Although this study has looked at a group of completely inexperienced subjects in a limited environment, it suggests that the overall learning curves for open and laparoscopic surgery might be similar when assessed at a particular point along the curve. However, when it comes to finer dissection, identification of correct planes and two-dimensional perception, laparoscopy requires
Table 3 Results of the questionnaire survey (A) Comparison of perception about learning the surgical skills (paired sample t-test) Mean scores
Perception before training Perception after training
Open surgery
Laparoscopy
5.53 7.23
3.15 4.92
Significance (2-tailed) 0.011 0.004
(B) Correlation between self-assessed scores in video games and surgical skill scores. (Mann–Whitney test) No. of subjects with above average skills Video skills
6/13
Significance (1-tailed) Open surgery
Laparoscopy
0.366 (NS)
0.295 (NS)
K. Subramonian et al. / European Urology 45 (2004) 346–351
more experience and more operative time. Overall, laparoscopic tasks took more time to finish than with open surgery by a proportion of 1.5. The validity of assessing the skills at a particular point along the learning curve and using it as a proxy measure to compare the learning curves is debatable. As mentioned earlier, the shape of the curve may differ before and after the point of assessment. This preliminary study was conducted to evaluate the feasibility of such a protocol rather than a definitive answer. A more comprehensive study involving the surgeons starting a new procedure is necessary to address this issue. Our observation about the difference in the finer dissection could be due to absence of tactile feedback and the fulcrum effect in performing laparoscopy. Fulcrum effect is due to the disparity in the direction of movement of the surgeon’s hand and the tip of the laparoscopic instrument [3]. These problems could be overcome by regular and intense training on inanimate trainers [4]. The bench laboratory setting has become a valid ground for assessing competency in surgical craft [5]. Direct comparison of learning curves in laparoscopic and open hysterectomy has been performed by Laminen [6] et al., showing that the learning curves for open and laparoscopic hysterectomy are similar for a single surgeon. One of the problems of this and many other similar studies is that the initial experience of the participants of the study may vary in open and laparoscopic techniques. Competence assessment in laparoscopic surgery is an integral part of any training programme. There are several validated tools available for this purpose such as Skills assessment device (SAD, Smith et al. [7]) and Imperial College Surgical assessment device (ICSAD, Darzi et al. [8]). It may in the future become mandatory for those embarking on laparoscopic surgery to assess their competence using such a device. The general perception that laparoscopy is more difficult to learn was shared by our study group, and persisted after the training. It is generally believed that proficiency in playing video games could translate into good laparoscopic skills, but we found no evidence to support this in an environment that did not rely on computer generated virtual reality. Another important observation from this study is that neither learning curve was linked to the career ambitions of the students. Currently, the selection process of surgical trainees is largely based on academic achievements and performance in an interview. In a large study conducted by Cuschieri et al. [9], the majority of the master trainers who took part in the study were of the
349
opinion that some form of skills and personality assessment should be included in the selection process. Overall, we were interested to note that while laparoscopic procedures appeared to be slightly more complex for novices to master, there was only a marginal difference between these and open skills in terms of a basic learning curve. When pronouncing on the ‘‘laparoscopic learning curve’’ we must be careful to remember that such a curve also exists for open surgery. Finally, although we had not set out to address the issue, we were encouraged to see that 12/13 (93%) of our students felt that this exposure to basic surgical techniques during their undergraduate careers would make them more interested in studying surgery and choosing it as a career. With the perceived drop in traditional surgical and urological teaching in many undergraduate curricula, perhaps we should think of extending our new training techniques to this audience to enthuse them for the future. Acknowledgements We would like to thank Mr. Neil Barber for helping with the assessment of surgical skills and the King’s College surgical study group for their participation in the study. King’s College surgical study group: Alexander Martin-Bates, Ameen Kamlana, Bethan Hale, Christopher Mitchell, James Jackson, Joel Newman, Johnson Osei-hwedieh, Junaid Bajwa, Laura Jeffreys, Matthew Campbell, Nadya Rahman, Sumita Mallick, Saraj Jayasundera. Appendix A. Questionnaire for the SSM students 1. Do you have any previous surgical training/ experience before this SSM? 2. Do you want to be a surgeon? 3. If yes, why? 4. Do you think that this SSM has changed your ambition to be a surgeon (þ or )? 5. What do you think of the role of surgical training in undergraduate training? 6. Your comments about the SSM: 7. Is there anything that we could have done to improve the SSM? 8. Have you played video games/PlayStation? 9. How many hours/minutes you might have played in the last year?
350
K. Subramonian et al. / European Urology 45 (2004) 346–351
10. Do you consider yourself good in playing video games/PlayStation? 11. Did you think that laparoscopy was more easy/ difficult to learn than open surgery before you started the SSM? (Score the difficulty from 1 to 10 for laparoscopy and open surgery, 1 ¼ not easy and 10 ¼ very easy.)
12. Has this idea changed after the SSM? (Score from 1 to 10, same as above.) 13. Do you think that the method of assessment used for the practical test was reliable (and show your true skills)? 14. Can you suggest a better way of assessing surgical training/ability to perform surgical skills?
References [1] Collins A, Brown J, Newman S. Cognitive apprenticeship: teaching the art of reading, writing and mathematics. In: Reznick L, editor. Cognition and instruction: issues and agenda. Hillsdale, NJ: Lawrence Earlbaum; 1990. p. 1–34. [2] Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Mon Russell IT. Assessment of the learning curve in health technologies. A systematic review. Int J Technol Assess Health Care 2000;16:1095–108. [3] Gallagher AG, McClure N, McGuigan J, et al. An ergonomic analysis of the fulcrum effect in the acquisition of endoscopy skills. Endoscopy 1998;30:617–20. [4] Traxer O, Gettman MT, Napper CA, Scott DJ, Jones DB, Roehrborn CG, et al. The impact of intense laparoscopic skills training on the operative performance of urology residents. J Urol 2001;166: 1658–61.
[5] Lentz GM, Mandel LS, Lee D, Gardella C, Melville J, Goff BA. Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: validity and reliability. Am J Obstet Gynecol 2001;184:1462–8. [6] Laminen A. Comparison between personal learning curves for abdominal and laparoscopic hysterectomy. Acta Obstet Gynecol Scand 2000;12:100–4. [7] Smith CD, Farrell TM, McNatt SS, Metreveli RE. Assessing laparoscopic manipulative skills. Am J Surg 2001;181:547–50. [8] Darzi A, Datta V, MacKay S. The challenge of objective assessment of surgical skill. Am J Surg 2001;181:484–6. [9] Cuschieri A, Francis N, Crosby J, Hanna GB. What do master surgeons think of surgical competence and revalidation? Am J Surg 2001;182:110–6.
Editorial Comment
Fit Faber fabricando? ‘‘The learning curve’’ is a concept that has risen recently with emergence of new techniques. This concept supports the idea that the quality of results improves with experience. This implies that knowledge cannot be completely taught or transferred from a teacher to a trainee without a lack of transfer, and that one needs to build his/her own experience while learning from mistakes. In our surgical field, the goal of a teacher becomes then to reduce the number of patients needed to get the required experience. The misnomer ‘‘steep learning curve’’ is often used to illustrate the difficulty of a procedure (a steep slope is difficult to climb), but in fact the goal is to make the curve (quality on Y, and number of cases on X) as steep as possible, and certainly not flat. This raises two questions, one on each axis.
the learning curve would be only applicable to difficult techniques, and a new question ‘‘What does the concept of difficulty in surgery mean?’’ pops up. This would bring us too far from the topic of this Editorial Comment, but is certainly part of the problem and deserves thoughts [1]. Nevertheless, if one speaks about quality, it becomes indispensable to imagine comparison and to evaluate the questions related to that: Better quality compared to what? What are the criteria of quality? What should be the base line below which the quality is not acceptable? Is this ‘‘base line’’ the same for each patient, surgeon or health care system? What is the influence of the time over the definition of quality? Furthermore, part of the difficulty is based upon the fact that the criteria of quality are different from one technique to another. For instance, time is certainly a factor of quality in partial nephrectomy but does not have the same value in nerve-sparing radical prostatectomy. Is a surgeon ‘‘better’’ because he is able to tie a knot in 1 minute rather than 2 minutes?
For the Y axis, what is quality? The quality is a concept difficult to assess. In fact, it seems that for some surgical techniques, the learning curve is not a real issue because the technique is supposed to be easy to master. ESWL is a good example of such a technique, but the easiness has never been confirmed or refuted. Then, it seems that
For the X axis, what is the relevant parameter? Intuitively, the number of cases seems to be the better parameter to build a learning curve. Certainly, the notion of time is also implied in this concept, since the value to have operated on X patients over one year doesn’t provide the same experience than the same number of patients over ten years.
Bertrand Guillonneau, New York, USA
K. Subramonian et al. / European Urology 45 (2004) 346–351
On the other hand, the number of patients operated on is certainly not the best parameter—or at least not the only one—to assess the surgeon’s quality [2]. Indeed in this study, it has been demonstrated that even the most experienced surgeons in retropubic radical prostatectomy have a wide discrepancy among their results, more unlikely than was expected. This could be explained by the fact that even a very experienced surgeon by surgical volume can repeatedly make the same mistake. And since time is a factor that is part of the concept, one could question whether the age of the surgeon influences the slope and then the long-term shape of the curve. To conclude, it appears that the ‘‘learning curve’’ is certainly a correct idea, but the concept must be further refined and framed within these two important guidelines: First, the number of patients and quality are not so tightly linked, and, to increase the quality, evaluation of results is the essential tool, whatever the number. Second, learning curve is, as long as the results are evaluated, a continual process. There is always room for improvement: I am still learning about a procedure that I perform daily.
References [1] Guillonneau B, Abbou CC, Doublet JD, Gaston R, Janetschek G, Mandressi A, et al. Proposal for a ‘‘European Scoring System for Laparoscopic Operations in Urology’’. Eur Urol 2001;40(1):2–6. [2] Begg CB, Riedel ER, Bach PB, Kattan MW, Schrag D, Warren JL, et al. Variations in morbidity after radical prostatectomy. N Engl J Med 2002;346(15):1138–44.
351
open surgical skills. One of the most striking observations represents the fact that the acquired skills were similar by both methods when assessed after 6 weeks. This underlines clearly that in this millenium all kind of surgical procedures should be applied clinically only after a successful training program. According to the specific operative technique, this could include different types of training options, such as simulators, invitro models, animal models or even training at cadavers [1–3]. Using their in-vitro model of open versus laparoscopic cholecystectomy, the authors found longer operating times in the laparoscopy group, which is not surprising for beginners which are not used to the reduced tactile feedback and different eye-hand coordination (‘‘Fulcrum effect’’). Based on studies at different centers (i.e. ParisCreteil, Heilbronn) the concept of a standardized step-by-step program to improve laparoscopic skills has been introduced, which enables trainees not experienced in laparoscopy to perfrom finally an urethrovesical anastomosis [4–6]. The recent evaluation of the program involving participants of the ESUT Expert Training and EAU-fellowship program revealed that after a time of 40 hours all trainees were able to perform all steps in time and completed an accurate anastomosis in 30 minutes [6]. These findings underline the importance of such training conepts for all types of surgical procedures in the future. This signifies the end of the ‘‘see one, do one, teach one’’ approach.
References Editorial Comment Jens Rassweiler, Heilbronn, Germany More than one decade after the first description of laparoscopic nephrectomy, an increasing number of laparoscopic procedure were performed worldwide. Nevertheless, there still exists a significant lack of standardized training programs and information about the learning curve of such technically demanding laparoscopic procedures (i.e. pyeloplasty, radical prostatectomy). Whereas, in former times for open surgery the rule of ‘‘see one, do one, teach one’’ has been applied frequently. For laparoscopy, such an approach is absolutely obsolete. This experimental study provides significant information about the training of both laparoscopic and
[1] Pilar-Laguna M, Hatzinger M, Rassweiler J. Simulators and endourological training. Curr Opin Urol 2002;12:209–15. [2] Michel MS, Knoll T, Ko¨ hrmann KU, Alken P. The URO Mentor: development and evaluation of a new computer-based interactive training system for virtual life-like simulation of diagnostic and therapeutic procedures. BJU Int 2002;89:174–7. [3] Rassweiler J, Frede T. Robotics, telesurgery and telementoring—their position in modern urological laparoscopy. Arch Esp Urol 2002;55: 610–28. [4] Frede T, Stock C, Renner C, Budair Z, Abdel-Salam Y, Rassweiler J. Geometry of laparoscopic suturing and knotting techniques. J Endourol 199;13:191–198. [5] Katz R, Nadu A, Olson LE, Hoznek A, de la Taille A, Salomon L, et al. A simplified 5-step model for traing laprosopic urethrovesical anastomosis. J Urol 2003;169:2041–4. [6] Teber D, Frede T, Dekel Y, Rassweiler. The ESUT laparoscopic training program: a stepwise approach to endoscopic suturing. Eur Urol Today 2003.
European Urology 45 (2004) 346–351
Acquiring Surgical Skills: A Comparative Study of Open versus Laparoscopic Surgery Kesavapillai Subramoniana,*, Suren DeSylvaa, Peter Bishaib, Peter Thompsona, Gordon Muira a
Department of Urology, King’s College Hospital, London, UK Department of Surgery, King’s College Hospital, London, UK
b
Accepted 4 September 2003 Published online 24 October 2003
Abstract Objectives: A preliminary study to evaluate the feasibility of a protocol for comparing the learning curves for open and laparoscopic surgical procedures. Participants and Methods: Thirteen pre-clinical medical students with no previous surgical training were given intensive coaching in open and laparoscopic surgical techniques for 12 weeks. At the end of this period, their open and laparoscopic skills were assessed by three independent examiners. Individual and aggregate ability scores in various aspects of open and laparoscopic surgery and the time taken to perform the procedures were compared using Student’s t-test. Results: There was no statistically significant difference in the overall scores by the two different techniques ( p ¼ 0:057). However, differences between the two techniques were significant in certain criteria including tissue dissection ( p ¼ 0:024), tidiness of gall bladder ( p ¼ 0:034) and liver ( p ¼ 0:016) specimens and the time taken for the two techniques ( p 0:001). Conclusions: This study suggests that when inexperienced subjects are given equal training in laparoscopy and open surgery, the overall skills acquired were similar by both methods when assessed after 6 weeks. However, on detailed analysis of the different components of surgery, the laparoscopic skills were deficient in finer dissection, identification of correct planes and two-dimensional perception when compared to open surgery and required more operative time. Our study group perceived that laparoscopy was more difficult to learn than open surgery even after the training. The study group also felt that the training in basic surgical skills during their undergraduate careers would make them more interested in studying surgery and choosing it as a career. # 2003 Elsevier B.V. All rights reserved. Keywords: Learning curve; Laparoscopy; Open surgery; Surgical training
1. Introduction The history of surgical craft dates back to the history of mankind and until recently surgery has traditionally been taught by apprenticeship [1]. With modern surgical training, skills are acquired by a didactic process *
Corresponding author. Present address: 18 Place Farm Avenue, Orpington, BR6 8DQ, Kent, UK. Tel. þ44-7715172317; Fax: þ44-1322428779. E-mail address: [email protected] (K. Subramonian).
of observing, assisting and performing surgical procedures under supervision, often with practical workshops and simulators to assist evaluation. Training schemes vary in different parts of the world with regard to criteria for completion of training. Some schemes are time-based whereas others are competency-based. However, the competency achieved by a trainee to perform a particular procedure depends on the type and complexity of surgery, the aptitude and manual dexterity of the surgeon, and the quality of training received.
0302-2838/$ – see front matter # 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.eururo.2003.09.021
K. Subramonian et al. / European Urology 45 (2004) 346–351
While many papers have recently referred to the ‘‘learning curve’’, particularly with regard to minimally invasive surgery, it does not really have a clear definition. We believe that the ‘‘learning curve’’ could be best defined as the number of procedures an average surgeon needs to undertake in order to be able to confidently perform that procedure independently with a reasonable outcome. This does presuppose the ability to identify an ‘‘average’’ surgeon! While assessing the learning curve, several factors should be taken into account such as time taken for the procedure, frequency of performing the surgery, operative skill and the clinical outcome. Learning curves for each surgical procedure could be plotted with the number of procedures done on the X-axis and the above factors on the Y-axis. The shape of the curve may vary depending on the complexity of the procedure and the manual dexterity of the surgeon. It has been postulated that the learning curve for minimally invasive procedures is longer than that for open surgical procedures. This belief seems to have been proposed mainly by more senior open surgeons, but lacks any objective proof, since no work has addressed the place of a traditional surgical training in preparing surgeons to learn new procedures, nor dealt with the formidably steep curves that may be encountered during training for traditional procedures. While video gaming may equip lay persons to manage virtual reality surgical simulators to the same degree as experienced surgeons, there are major problems in extrapolating virtual reality results to the clinical environment. In a review of literature about learning curves in health technologies, Ramsay and colleagues describe various statistical methods used by the investigators such as simple graphs, splitting data chronologically and performing a t-test, curve fitting and other model fitting [2]. The ideal way of comparing the learning curve is to assess a cohort of surgeons performing open and laparoscopic surgery from the start and compare their curves. This is a significant undertaking needing a great deal of co-ordination. An alternative way is to assess the skills and operative time after a set number of procedures and compare the two at that point of the learning curve. This could be used as a proxy measure to compare the slope of the learning curves from the start to this point. One caveat of this method is the inability to predict the slope of the learning curve beyond the point of assessment. We wished to look at the hypothesis that the skills needed to perform laparoscopic surgery are so different to those needed for open surgery that the learning curve might be no different for given tasks in open and laparoscopic procedures. To assess the learning curve
347
of laparoscopic and open surgery independently, a group of medical students with no previous surgical experience were trained to perform a simulated procedure by the two different methods. We also wished to see if any preconceptions about technical difficulty were changed by a course of practical training. 2. Materials and methods Thirteen third-year medical students with no previous surgical training were recruited for the study. They were given intensive coaching for 4 hours per week for a period of 12 weeks. The teaching modules consisted of knot tying, tissue handling, tissue dissection and cholecystectomy in pig liver. These were performed by open and laparoscopic methods, with equal time apportioned to each approach. Initially the students were trained in using dry materials, but after the sixth week, inanimate ‘‘wet’’ materials (pig mesentery, pig vessels, pig liver and gall bladder) were introduced. In the wet lab training, each student performed five cholecystectomies before the end of training. On completion of the training period, their ability to perform cholecystectomy by the two methods was assessed by a practical examination in which each student performed an open and laparoscopic cholecystectomy in random order. The skills were scored by three independent examiners who were experienced surgeons in laparoscopic and/or open surgery. The assessment criteria included dissection of the cystic artery and cystic duct, instrument handling, knot tying/clipping, tissue dissection, tidiness of gall bladder specimen and liver bed, and the time taken for the procedures. A score of 1 to 4 was assigned for each criterion based on the performance (1 ¼ poor, 2 ¼ fair, 3 ¼ good and 4 ¼ excellent). Scoring was based on the direct observation by the examiners at the time of surgery. The mean score for each criterion was calculated to avoid inter-observer variation and aggregate scores for each candidate in the two methods calculated. The scores obtained by each student in the two techniques and the time taken was compared using Student’s t-test. The students were also asked to fill in questionnaires regarding their interest in surgery, and their perception about learning different techniques of surgery both before and after the training (see Appendix A). The perception scores before and after training were compared using paired sample t-test. Their self-assessed proficiency and time spent in playing video games in the previous year was also measured. These skills were correlated with the scores in the practical examination.
3. Results The average scores for each criterion in different techniques were plotted in a frequency distribution curve. These values followed a normal distribution and hence further statistical analysis was performed using paired sample t-test (Table 1). The mean scores for each criterion and the p-values are listed in Table 2. There was no statistically significant difference in the overall scores by the two different techniques ( p ¼ 0:057). However, differences between the two techniques were significant in certain criteria including
348
K. Subramonian et al. / European Urology 45 (2004) 346–351
Table 1 Paired sample t-test Comparison data
Paired differences Mean
Duct dissection Artery dissection Knot tying Instrument handling Dissection techniques Gall bladder specimen Liver bed Time taken
Df
Significance (2-tailed)
Std Deviation
Std error of mean
95% CI of the difference Lower
Upper
0.115 0.238 0.269 0.154 0.423 0.585 0.546 33.15
0.5398 0.5561 0.4820 0.5010 0.5890 0.8792 0.7067 19.304
0.1497 0.1542 0.1337 0.1390 0.1634 0.2439 0.1960 5.354
0.211 0.098 0.560 0.149 0.067 0.053 0.119 44.82
0.442 0.575 0.022 0.457 0.779 1.116 0.973 21.49
0.771 1.546 2.014 1.107 2.590 2.397 2.787 6.192
12 12 12 12 12 12 12 12
0.456 0.148 0.067 0.290 0.024 0.034 0.016 0.0001
1.715
2.9359
0.8143
0.059
3.490
2.107
12
0.057
Total score
Table 2 Showing mean scores by the open and laparoscopic methods Criterion
t
Mean scores Open method
Duct dissection 2.8 Artery dissection 2.9 Knot tying/clipping 2.5 Instrument handling 2.9 Dissection techniques 2.9 Gall bladder specimen 3.1 Liver bed 3.1 Total score 20.1 Total time 50.1
S.D.
p-value
0.540 0.556 0.482 0.501 0.589 0.879 0.707 2.936 19.304
0.456 0.148 0.067 0.290 0.024 0.034 0.016 0.057 0.000
Laparoscopy 2.7 2.7 2.7 2.7 2.4 2.5 2.6 18.4 83.2
tissue dissection ( p ¼ 0:024), tidiness of gall bladder ( p ¼ 0:034) and liver ( p ¼ 0:016) specimens and the time taken for the two techniques ( p 0:001). Analysis of the questionnaires showed that 10 out of 13 students wanted to be surgeons. All the students who wanted to be surgeons and two of the others were positively influenced by the training. By comparing those students who did or did not want to be surgeons using paired sample analysis, there was no significant
difference in surgical skills ( p 0:05). It was also found that those students with proficiency in playing video games did not differ in the open and laparoscopic skills from others. On analysis of the perception before the training, all the students perceived that learning laparoscopy was significantly more difficult than learning open surgery. ( p ¼ 0:011). Even after the training they perceived that laparoscopy was significantly more difficult than open surgery ( p ¼ 0:004). However, they found that both techniques were comparatively easier than they had initially perceived (Table 3).
4. Discussion Although this study has looked at a group of completely inexperienced subjects in a limited environment, it suggests that the overall learning curves for open and laparoscopic surgery might be similar when assessed at a particular point along the curve. However, when it comes to finer dissection, identification of correct planes and two-dimensional perception, laparoscopy requires
Table 3 Results of the questionnaire survey (A) Comparison of perception about learning the surgical skills (paired sample t-test) Mean scores
Perception before training Perception after training
Open surgery
Laparoscopy
5.53 7.23
3.15 4.92
Significance (2-tailed) 0.011 0.004
(B) Correlation between self-assessed scores in video games and surgical skill scores. (Mann–Whitney test) No. of subjects with above average skills Video skills
6/13
Significance (1-tailed) Open surgery
Laparoscopy
0.366 (NS)
0.295 (NS)
K. Subramonian et al. / European Urology 45 (2004) 346–351
more experience and more operative time. Overall, laparoscopic tasks took more time to finish than with open surgery by a proportion of 1.5. The validity of assessing the skills at a particular point along the learning curve and using it as a proxy measure to compare the learning curves is debatable. As mentioned earlier, the shape of the curve may differ before and after the point of assessment. This preliminary study was conducted to evaluate the feasibility of such a protocol rather than a definitive answer. A more comprehensive study involving the surgeons starting a new procedure is necessary to address this issue. Our observation about the difference in the finer dissection could be due to absence of tactile feedback and the fulcrum effect in performing laparoscopy. Fulcrum effect is due to the disparity in the direction of movement of the surgeon’s hand and the tip of the laparoscopic instrument [3]. These problems could be overcome by regular and intense training on inanimate trainers [4]. The bench laboratory setting has become a valid ground for assessing competency in surgical craft [5]. Direct comparison of learning curves in laparoscopic and open hysterectomy has been performed by Laminen [6] et al., showing that the learning curves for open and laparoscopic hysterectomy are similar for a single surgeon. One of the problems of this and many other similar studies is that the initial experience of the participants of the study may vary in open and laparoscopic techniques. Competence assessment in laparoscopic surgery is an integral part of any training programme. There are several validated tools available for this purpose such as Skills assessment device (SAD, Smith et al. [7]) and Imperial College Surgical assessment device (ICSAD, Darzi et al. [8]). It may in the future become mandatory for those embarking on laparoscopic surgery to assess their competence using such a device. The general perception that laparoscopy is more difficult to learn was shared by our study group, and persisted after the training. It is generally believed that proficiency in playing video games could translate into good laparoscopic skills, but we found no evidence to support this in an environment that did not rely on computer generated virtual reality. Another important observation from this study is that neither learning curve was linked to the career ambitions of the students. Currently, the selection process of surgical trainees is largely based on academic achievements and performance in an interview. In a large study conducted by Cuschieri et al. [9], the majority of the master trainers who took part in the study were of the
349
opinion that some form of skills and personality assessment should be included in the selection process. Overall, we were interested to note that while laparoscopic procedures appeared to be slightly more complex for novices to master, there was only a marginal difference between these and open skills in terms of a basic learning curve. When pronouncing on the ‘‘laparoscopic learning curve’’ we must be careful to remember that such a curve also exists for open surgery. Finally, although we had not set out to address the issue, we were encouraged to see that 12/13 (93%) of our students felt that this exposure to basic surgical techniques during their undergraduate careers would make them more interested in studying surgery and choosing it as a career. With the perceived drop in traditional surgical and urological teaching in many undergraduate curricula, perhaps we should think of extending our new training techniques to this audience to enthuse them for the future. Acknowledgements We would like to thank Mr. Neil Barber for helping with the assessment of surgical skills and the King’s College surgical study group for their participation in the study. King’s College surgical study group: Alexander Martin-Bates, Ameen Kamlana, Bethan Hale, Christopher Mitchell, James Jackson, Joel Newman, Johnson Osei-hwedieh, Junaid Bajwa, Laura Jeffreys, Matthew Campbell, Nadya Rahman, Sumita Mallick, Saraj Jayasundera. Appendix A. Questionnaire for the SSM students 1. Do you have any previous surgical training/ experience before this SSM? 2. Do you want to be a surgeon? 3. If yes, why? 4. Do you think that this SSM has changed your ambition to be a surgeon (þ or )? 5. What do you think of the role of surgical training in undergraduate training? 6. Your comments about the SSM: 7. Is there anything that we could have done to improve the SSM? 8. Have you played video games/PlayStation? 9. How many hours/minutes you might have played in the last year?
350
K. Subramonian et al. / European Urology 45 (2004) 346–351
10. Do you consider yourself good in playing video games/PlayStation? 11. Did you think that laparoscopy was more easy/ difficult to learn than open surgery before you started the SSM? (Score the difficulty from 1 to 10 for laparoscopy and open surgery, 1 ¼ not easy and 10 ¼ very easy.)
12. Has this idea changed after the SSM? (Score from 1 to 10, same as above.) 13. Do you think that the method of assessment used for the practical test was reliable (and show your true skills)? 14. Can you suggest a better way of assessing surgical training/ability to perform surgical skills?
References [1] Collins A, Brown J, Newman S. Cognitive apprenticeship: teaching the art of reading, writing and mathematics. In: Reznick L, editor. Cognition and instruction: issues and agenda. Hillsdale, NJ: Lawrence Earlbaum; 1990. p. 1–34. [2] Ramsay CR, Grant AM, Wallace SA, Garthwaite PH, Mon Russell IT. Assessment of the learning curve in health technologies. A systematic review. Int J Technol Assess Health Care 2000;16:1095–108. [3] Gallagher AG, McClure N, McGuigan J, et al. An ergonomic analysis of the fulcrum effect in the acquisition of endoscopy skills. Endoscopy 1998;30:617–20. [4] Traxer O, Gettman MT, Napper CA, Scott DJ, Jones DB, Roehrborn CG, et al. The impact of intense laparoscopic skills training on the operative performance of urology residents. J Urol 2001;166: 1658–61.
[5] Lentz GM, Mandel LS, Lee D, Gardella C, Melville J, Goff BA. Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: validity and reliability. Am J Obstet Gynecol 2001;184:1462–8. [6] Laminen A. Comparison between personal learning curves for abdominal and laparoscopic hysterectomy. Acta Obstet Gynecol Scand 2000;12:100–4. [7] Smith CD, Farrell TM, McNatt SS, Metreveli RE. Assessing laparoscopic manipulative skills. Am J Surg 2001;181:547–50. [8] Darzi A, Datta V, MacKay S. The challenge of objective assessment of surgical skill. Am J Surg 2001;181:484–6. [9] Cuschieri A, Francis N, Crosby J, Hanna GB. What do master surgeons think of surgical competence and revalidation? Am J Surg 2001;182:110–6.
Editorial Comment
Fit Faber fabricando? ‘‘The learning curve’’ is a concept that has risen recently with emergence of new techniques. This concept supports the idea that the quality of results improves with experience. This implies that knowledge cannot be completely taught or transferred from a teacher to a trainee without a lack of transfer, and that one needs to build his/her own experience while learning from mistakes. In our surgical field, the goal of a teacher becomes then to reduce the number of patients needed to get the required experience. The misnomer ‘‘steep learning curve’’ is often used to illustrate the difficulty of a procedure (a steep slope is difficult to climb), but in fact the goal is to make the curve (quality on Y, and number of cases on X) as steep as possible, and certainly not flat. This raises two questions, one on each axis.
the learning curve would be only applicable to difficult techniques, and a new question ‘‘What does the concept of difficulty in surgery mean?’’ pops up. This would bring us too far from the topic of this Editorial Comment, but is certainly part of the problem and deserves thoughts [1]. Nevertheless, if one speaks about quality, it becomes indispensable to imagine comparison and to evaluate the questions related to that: Better quality compared to what? What are the criteria of quality? What should be the base line below which the quality is not acceptable? Is this ‘‘base line’’ the same for each patient, surgeon or health care system? What is the influence of the time over the definition of quality? Furthermore, part of the difficulty is based upon the fact that the criteria of quality are different from one technique to another. For instance, time is certainly a factor of quality in partial nephrectomy but does not have the same value in nerve-sparing radical prostatectomy. Is a surgeon ‘‘better’’ because he is able to tie a knot in 1 minute rather than 2 minutes?
For the Y axis, what is quality? The quality is a concept difficult to assess. In fact, it seems that for some surgical techniques, the learning curve is not a real issue because the technique is supposed to be easy to master. ESWL is a good example of such a technique, but the easiness has never been confirmed or refuted. Then, it seems that
For the X axis, what is the relevant parameter? Intuitively, the number of cases seems to be the better parameter to build a learning curve. Certainly, the notion of time is also implied in this concept, since the value to have operated on X patients over one year doesn’t provide the same experience than the same number of patients over ten years.
Bertrand Guillonneau, New York, USA
K. Subramonian et al. / European Urology 45 (2004) 346–351
On the other hand, the number of patients operated on is certainly not the best parameter—or at least not the only one—to assess the surgeon’s quality [2]. Indeed in this study, it has been demonstrated that even the most experienced surgeons in retropubic radical prostatectomy have a wide discrepancy among their results, more unlikely than was expected. This could be explained by the fact that even a very experienced surgeon by surgical volume can repeatedly make the same mistake. And since time is a factor that is part of the concept, one could question whether the age of the surgeon influences the slope and then the long-term shape of the curve. To conclude, it appears that the ‘‘learning curve’’ is certainly a correct idea, but the concept must be further refined and framed within these two important guidelines: First, the number of patients and quality are not so tightly linked, and, to increase the quality, evaluation of results is the essential tool, whatever the number. Second, learning curve is, as long as the results are evaluated, a continual process. There is always room for improvement: I am still learning about a procedure that I perform daily.
References [1] Guillonneau B, Abbou CC, Doublet JD, Gaston R, Janetschek G, Mandressi A, et al. Proposal for a ‘‘European Scoring System for Laparoscopic Operations in Urology’’. Eur Urol 2001;40(1):2–6. [2] Begg CB, Riedel ER, Bach PB, Kattan MW, Schrag D, Warren JL, et al. Variations in morbidity after radical prostatectomy. N Engl J Med 2002;346(15):1138–44.
351
open surgical skills. One of the most striking observations represents the fact that the acquired skills were similar by both methods when assessed after 6 weeks. This underlines clearly that in this millenium all kind of surgical procedures should be applied clinically only after a successful training program. According to the specific operative technique, this could include different types of training options, such as simulators, invitro models, animal models or even training at cadavers [1–3]. Using their in-vitro model of open versus laparoscopic cholecystectomy, the authors found longer operating times in the laparoscopy group, which is not surprising for beginners which are not used to the reduced tactile feedback and different eye-hand coordination (‘‘Fulcrum effect’’). Based on studies at different centers (i.e. ParisCreteil, Heilbronn) the concept of a standardized step-by-step program to improve laparoscopic skills has been introduced, which enables trainees not experienced in laparoscopy to perfrom finally an urethrovesical anastomosis [4–6]. The recent evaluation of the program involving participants of the ESUT Expert Training and EAU-fellowship program revealed that after a time of 40 hours all trainees were able to perform all steps in time and completed an accurate anastomosis in 30 minutes [6]. These findings underline the importance of such training conepts for all types of surgical procedures in the future. This signifies the end of the ‘‘see one, do one, teach one’’ approach.
References Editorial Comment Jens Rassweiler, Heilbronn, Germany More than one decade after the first description of laparoscopic nephrectomy, an increasing number of laparoscopic procedure were performed worldwide. Nevertheless, there still exists a significant lack of standardized training programs and information about the learning curve of such technically demanding laparoscopic procedures (i.e. pyeloplasty, radical prostatectomy). Whereas, in former times for open surgery the rule of ‘‘see one, do one, teach one’’ has been applied frequently. For laparoscopy, such an approach is absolutely obsolete. This experimental study provides significant information about the training of both laparoscopic and
[1] Pilar-Laguna M, Hatzinger M, Rassweiler J. Simulators and endourological training. Curr Opin Urol 2002;12:209–15. [2] Michel MS, Knoll T, Ko¨ hrmann KU, Alken P. The URO Mentor: development and evaluation of a new computer-based interactive training system for virtual life-like simulation of diagnostic and therapeutic procedures. BJU Int 2002;89:174–7. [3] Rassweiler J, Frede T. Robotics, telesurgery and telementoring—their position in modern urological laparoscopy. Arch Esp Urol 2002;55: 610–28. [4] Frede T, Stock C, Renner C, Budair Z, Abdel-Salam Y, Rassweiler J. Geometry of laparoscopic suturing and knotting techniques. J Endourol 199;13:191–198. [5] Katz R, Nadu A, Olson LE, Hoznek A, de la Taille A, Salomon L, et al. A simplified 5-step model for traing laprosopic urethrovesical anastomosis. J Urol 2003;169:2041–4. [6] Teber D, Frede T, Dekel Y, Rassweiler. The ESUT laparoscopic training program: a stepwise approach to endoscopic suturing. Eur Urol Today 2003.