- Email: [email protected]
Computer-assisted Learning versus a Lecture and Feedback Seminar for Teaching a Basic Surgical Technical Skill 1
Computer-assisted Learning versus a Lecture and Feedback Seminar for Teaching a Basic Surgical Technical Skill David A. Rogers, MD, Augusta, Georgia, Glenn Regehr, PhD, Toronto, Ontario, Canada, Karen A. Yeh, MD, Thomas R. Howdieshell, MD, Augusta, Georgia
BACKGROUND: Rapid improvements in computer technology allow us to consider the use of computer-assisted learning (CAL) for teaching technical skills in surgical training. The objective of this study was to compare in a prospective, randomized fashion, CAL with a lecture and feedback seminar (LFS) for the purpose of teaching a basic surgical skill. METHODS: Freshman medical students were randomly assigned to spend 1 hour in either a CAL or LFS session. Both sessions were designed to teach them to tie a two-handed square knot. Students in both groups were given knot tying boards and those in the CAL group were asked to interact with the CAL program. Students in the LFS group were given a slide presentation and were given individualized feedback as they practiced this skill. At the end of the session the students were videotaped tying two complete knots. The tapes were independently analyzed, in a blinded fashion, by three surgeons. The total time for the task was recorded, the knots were evaluated for squareness, and each subject was scored for the quality of performance. RESULTS: Data from 82 subjects were available for the final analysis. Comparison of the two groups demonstrated no significant difference between the proportion of subjects who were able to tie a square knot. There was no difference between the average time required to perform the task. The CAL group had significantly lower quality of performance (t 5 5.37, P <0.0001). CONCLUSIONS: CAL and LFS were equally effective in conveying the cognitive information associated with this skill. However, the significantly lower performance score demonstrates that the
students in the CAL group did not attain a proficiency in this skill equal to the students in the LFS group. Comments by the students suggest that the lack of feedback in this model of CAL was the significant difference between these two educational methods. Am J Surg. 1998;175:508 – 510. © 1998 by Excerpta Medica, Inc.
T
eaching technical skills is an important and unique aspect of surgical education.1,2 Traditionally, surgical skills have been taught in the operating room. However, pressures to increase clinical productivity in the academic environment have increased the temptation to abandon the role of teacher and simply finish the case ourselves. We suspect that this is particularly true when the surgery resident has not yet mastered the most basic surgical skills. These skills can be taught outside of the operating room, and this has usually been done in a skill laboratory.3– 8 While this eliminates the expense of the operating room, it continues to require the presence of the faculty to demonstrate the task. Improvements in computer technology have made CAL a potential solution to the problem of “surgeons in a rush.”9 To provide the most advantage to the faculty, the CAL tool would be used in an environment without the faculty present. In this setting, the major disadvantage of this methodology is the loss of external feedback from the expert. Instead, novices are required to identify their own errors and search for solutions. Our goal in this study was to compare in a blinded prospective, randomized, controlled manner CAL with a traditional form of education. We selected the LFS because we thought that this was the optimal traditional method currently available. The LFS format allows the student to see the expert demonstrate the task, and the student then receives feedback from the expert as he or she practices the skill.
METHOD From the Department of Surgery (DAR, KAY, TRH), Medical College of Georgia, Augusta, Georgia, and the Department of Surgery (GR), University of Toronto, Toronto, Ontario, Canada. Requests for reprints should be addressed to David A. Rogers, MD, Section of Pediatric Surgery, Department of Surgery, Medical College of Georgia, BAN 8216, 1120 15th Street, Augusta, Georgia 30912-4070. Presented at the 17th Annual Meeting of the Association for Surgical Education, Philadelphia, Pennsylvania, April 9 –12, 1997. Manuscript submitted August 13, 1997 and accepted in revised form March 16, 1998.
508
© 1998 by Excerpta Medica, Inc. All rights reserved.
Approval was obtained from the IRB at the Medical College of Georgia. Freshman students were enrolled in the study, and each was randomly assigned to either the CAL or LFS session. There were 10 to 12 students in each group. Each student was given a knot tying board and tying rope. The students were not paid for participating in the study but were allowed to keep the knot tying board and rope. In the LFS session, a series of 2 3 2 photographic slides was used to demonstrate the skill of tying a pair of twohanded square knots. The pictorial slides were alternated with text slides that described that portion of the skill. 0002-9610/98/$19.00 PII S0002-9610(98)00087-7
APPROACHES TO TEACHING BASIC SURGICAL SKILL/ROGERS ET AL
TABLE Performance Characteristics of Groups Proportion square Average time Performance Score
CAL (n 5 40)
LFS (n 5 42)
Test
Significance
85% 19.6 6 9.6 sec 12.8 6 4.14
90% 17.4 6 6.8 sec 17.4 6 3/53
Chi square 5 0.57 t 5 0.66 t 5 5.37
NS NS P ,.0001
Values for average time and performance score are the means 6 standard deviation. CAL 5 computer-assisted learning; LFS 5 lecture and feedback seminar.
Three perspectives were developed: a view from over the shoulder, a view at 90 degrees to the left of the surgeon, and a view of the skill being performed left-handed. These sessions took place in a standard conference room, and two faculty members were present to give feedback throughout the session. For the CAL session, a videotape was made of the surgical skill. The perspectives were identical to that used in the slide presentation. The video images were digitized, and a multimedia program was created using the software Director 4.0 (Macromedia, Inc.). In the multimedia program the student could choose to view the entire task in real time or in slow motion. We also divided the entire task into 12 separate steps that corresponded to those used in the slide demonstration. Text was shown in each of these steps that was identical to the text slides used in the LFS. The CAL sessions took place in a classroom equipped with a sufficient number computers so that there was one for each student. Each educational session lasted for 1 hour. At the end of each session, the students were instructed to tie their best and fastest pair of two-handed square knots. Each student was videotaped, and these tapes were independently reviewed, in a blinded fashion, by three surgical faculty members. The surgeons recorded whether or not they thought the knot was square. In the cases where all three surgeons did not agree on the squareness of the knot, the majority opinion was used. Further, the surgeons evaluated the quality of the knot tying using a rating scale that explicitly identified all of the actions necessary for an optimal performance. The number generated using the rating scale was termed the performance score, with a maximum score of 24. Finally, the time for the task was recorded for each of the students. After the experimental session and the videotape evaluation, the students participated in the alternate educational session and were then given a survey asking them which of the training sessions they preferred and reasons for this preference. The data were entered into a Quattro Pro 6.0 spreadsheet (Novell, Inc), and statistical analysis was done with this software program and SPSS for Windows, Release 7.0 (SPSS, Inc.) The interrater reliability of the checklist was evaluated using Cronbach’s alpha. The proportion of students who correctly tied a square knot and the survey responses were compared using chi square of proportions. The mean performance scores and the mean times were compared using Students’ t-test for independent samples.
RESULTS Ninety-one freshman medical students were enrolled in the study. Eight of the subjects were excluded because they had previous experience in the operating room or emergency room. The data from 1 of the subjects was excluded because it was not possible to evaluate the task on the videotape, leaving data from 82 of the students available for analysis. A total of 40 students participated in the CAL session and 42 students participated in the LFS session. Average age, gender distribution, handedness, and the proportion of students with computers at home were similar for each group. Cronbach’s alpha for the interrater reliability of the checklist was 0.79, indicating that the evaluators were able to reliably distinguish between inferior and superior performances. Measurements of outcome between the two groups were similar (Table) with the exception of the performance score. The mean performance score for the group that participated in the CAL session was significantly lower than that of the LFS group. After having experienced both educational sessions, the student’s were asked which methodology they preferred. Seventy-three of the 82 students (89%) preferred the LFS session. The lack of feedback was the item most often cited as the most significant negative feature of this model of CAL and was mentioned by 44% of the total group.
COMMENT A similar proportion of students in both groups were able to tie a square knot, suggesting that both groups grasped the cognitive portion of the task. By contrast, the significant difference in performance scores indicates that students in the LFS group attained a significantly higher level of proficiency with the skill than students in the CAL group. The finding of equivalence of cognition between the two methods is consistent with the existing literature on the use of CAL in medical education.10,11 Given the relatively high cost of developing CAL materials and the technology required to use it, CAL must offer an advantage over existing educational techniques.12 However, the available literature does not suggest that such an advantage exists when using CAL to teach purely cognitive information. CAL does seem to offer some features that could make it superior for teaching a motor task. It seems logical that viewing the entire motion of the task would offer a significant advantage compared with viewing still photographs or illustrations of individual steps in the tasks. This is possible by digitizing a traditional videotape and importing it into multimedia software. When viewing this digitized file, students can see the motion involved in the task
THE AMERICAN JOURNAL OF SURGERY® VOLUME 175 JUNE 1998
509
APPROACHES TO TEACHING BASIC SURGICAL SKILL/ROGERS ET AL
and can model their performance after the experts. The digitized image of the skill can be broken down into component parts that allows the student to customize the learning experience and concentrate on the particular portion of the skill that is causing difficulty. There is extremely limited experience with using CAL to teach medically relevant psychomotor skills. The experience most applicable to our interest is in teaching cardiopulmonary resuscitation (CPR). The American Heart Association developed a CAL tool to teach CPR, and two prospective randomized studies examined the use of this technology to teach this skill.13,14 These studies showed no difference between the two groups in the performance of the skill. In contrast, we found that the students who had participated in the CAL performed the motor task with much less proficiency than students in the LFS group. In the CPR experience, the motor skill testing is recorded as simply pass or fail. We suspect that had they used multiple outcome measurements, they may have detected a difference between the two groups. If we had measured only the time required to perform the task or the proportion of knots that were correctly squared, we would have found no difference between the methods. Survey responses indicate that the lack of feedback was the important difference between the two educational methods. While the student grasped the idea of the task, the lack of feedback led to an increased number of errors in the performance of this task. These errors lowered the performance score but did not affect the time required to perform the task. The students were aware that their performance did not match the experts but could not correct their own mistakes. This frustration is evident in their survey comments and in their overwhelming preference for the LFS format. The preference of students for a format where the instructor is present was also found in the experience with CAL to teach CPR.13 We can learn much about teaching surgical skills by examining the sports education literature where there is a long experience teaching motor skills.1 Feedback is one of the elements of instruction that is felt to be important in teaching motor skill associated with sports.15 We can expect that computer technology will play an increasingly important role in surgical education.16 Our interest at the beginning of this project was to use CAL to move the instruction of the most basic surgical skills out of the operating room and to relieve the faculty of the responsibility of being present to teach these skills. This feature has been cited as one of the major potential advan-
510
tages of the most advanced form of CAL, computer simulation.17 We have shown that, particularly when teaching a novice a new skill, that the involvement of the expert is still important. The results of our study should remind us that until CAL, including computer simulation, can faithfully reproduce all of the elements of motor skills teaching, it cannot replace traditional teaching methods.
REFERENCES 1. Barnes RW. Surgical handicraft: teaching and learning surgical skills. Am J Surg. 1987;153:422– 427. 2. Baker RJ. Teaching (and using) surgical technique. Curr Surg. 1985;42:105–106. 3. Allen PI. Anastomosis: a craft workshop for surgical trainees. Aust N Z J Surg. 1990;60:943–945. 4. Heppell J, Beauchamp F, Chollet A. Ten-year experience with a basic technical skills and perioperative management workshop for first-year residents. Can J Surg. 1995;38:27–32. 5. Bevan PG. The anastomosis workshop, March 1981. Ann R Coll Surg Engl. 1981;63:405– 408. 6. Lossing AG, Hatswell EM, Gilas T, et al. A technical-skills course for 1st-year residents in general surgery: a descriptive study. Can J Surg. 1992;35:536 –540. 7. Barnes RW, Lang NP, Whiteside MF. Halstedian technique revisited: innovations in teaching surgical skills. Ann Surg. 1989; 210:118 –121. 8. Ellis H. Teaching surgeons to stitch. Aust N Z J Surg. 1993;63: 919 –920. 9. Reznick RK. Teaching and testing technical skills. Am J Surg. 1993;165:358 –361. 10. Cohen PA, Dacanay L. A meta-analysis of computer-based instruction in nursing education. Comput Nurs. 1994;12:89 –97. 11. Jelovsek FR, Adebonojo L. Learning principles as applied to computer-assisted instruction. MD Comput. 1993;10:165–172. 12. Keane DR, Norman GR, Vickers J. The inadequacy of recent research on computer-assisted instruction. Acad Med. 1991;66: 444 – 448. 13. Fabius DB, Grissom EL, Fuentes A. Recertification in cardiopulmonary resuscitation: a comparison of two teaching methods. J Nurs Staff Dev. 1994;10:262–268. 14. Edwards MJ, Hannah KJ. An examination of the use of interactive videodisc cardiopulmonary resuscitation instruction for the lay community. Comput Nurs. 1985;3:250 –252. 15. Lee AM, Keh NC, Magill RA. Instructional effects of teacher feedback in physical education. J Teach Phys Ed. 1993;12:228 –243. 16. Dunnington GL, DaRosa DA. Changing surgical education strategies in an environment of changing health care delivery systems. World J Surg. 1994;18:734 –737. 17. Satava RM. Virtual reality surgical simulator: the first steps. Surg Endosc. 1993;7:203–205.
THE AMERICAN JOURNAL OF SURGERY® VOLUME 175 JUNE 1998
BACKGROUND: Rapid improvements in computer technology allow us to consider the use of computer-assisted learning (CAL) for teaching technical skills in surgical training. The objective of this study was to compare in a prospective, randomized fashion, CAL with a lecture and feedback seminar (LFS) for the purpose of teaching a basic surgical skill. METHODS: Freshman medical students were randomly assigned to spend 1 hour in either a CAL or LFS session. Both sessions were designed to teach them to tie a two-handed square knot. Students in both groups were given knot tying boards and those in the CAL group were asked to interact with the CAL program. Students in the LFS group were given a slide presentation and were given individualized feedback as they practiced this skill. At the end of the session the students were videotaped tying two complete knots. The tapes were independently analyzed, in a blinded fashion, by three surgeons. The total time for the task was recorded, the knots were evaluated for squareness, and each subject was scored for the quality of performance. RESULTS: Data from 82 subjects were available for the final analysis. Comparison of the two groups demonstrated no significant difference between the proportion of subjects who were able to tie a square knot. There was no difference between the average time required to perform the task. The CAL group had significantly lower quality of performance (t 5 5.37, P <0.0001). CONCLUSIONS: CAL and LFS were equally effective in conveying the cognitive information associated with this skill. However, the significantly lower performance score demonstrates that the
students in the CAL group did not attain a proficiency in this skill equal to the students in the LFS group. Comments by the students suggest that the lack of feedback in this model of CAL was the significant difference between these two educational methods. Am J Surg. 1998;175:508 – 510. © 1998 by Excerpta Medica, Inc.
T
eaching technical skills is an important and unique aspect of surgical education.1,2 Traditionally, surgical skills have been taught in the operating room. However, pressures to increase clinical productivity in the academic environment have increased the temptation to abandon the role of teacher and simply finish the case ourselves. We suspect that this is particularly true when the surgery resident has not yet mastered the most basic surgical skills. These skills can be taught outside of the operating room, and this has usually been done in a skill laboratory.3– 8 While this eliminates the expense of the operating room, it continues to require the presence of the faculty to demonstrate the task. Improvements in computer technology have made CAL a potential solution to the problem of “surgeons in a rush.”9 To provide the most advantage to the faculty, the CAL tool would be used in an environment without the faculty present. In this setting, the major disadvantage of this methodology is the loss of external feedback from the expert. Instead, novices are required to identify their own errors and search for solutions. Our goal in this study was to compare in a blinded prospective, randomized, controlled manner CAL with a traditional form of education. We selected the LFS because we thought that this was the optimal traditional method currently available. The LFS format allows the student to see the expert demonstrate the task, and the student then receives feedback from the expert as he or she practices the skill.
METHOD From the Department of Surgery (DAR, KAY, TRH), Medical College of Georgia, Augusta, Georgia, and the Department of Surgery (GR), University of Toronto, Toronto, Ontario, Canada. Requests for reprints should be addressed to David A. Rogers, MD, Section of Pediatric Surgery, Department of Surgery, Medical College of Georgia, BAN 8216, 1120 15th Street, Augusta, Georgia 30912-4070. Presented at the 17th Annual Meeting of the Association for Surgical Education, Philadelphia, Pennsylvania, April 9 –12, 1997. Manuscript submitted August 13, 1997 and accepted in revised form March 16, 1998.
508
© 1998 by Excerpta Medica, Inc. All rights reserved.
Approval was obtained from the IRB at the Medical College of Georgia. Freshman students were enrolled in the study, and each was randomly assigned to either the CAL or LFS session. There were 10 to 12 students in each group. Each student was given a knot tying board and tying rope. The students were not paid for participating in the study but were allowed to keep the knot tying board and rope. In the LFS session, a series of 2 3 2 photographic slides was used to demonstrate the skill of tying a pair of twohanded square knots. The pictorial slides were alternated with text slides that described that portion of the skill. 0002-9610/98/$19.00 PII S0002-9610(98)00087-7
APPROACHES TO TEACHING BASIC SURGICAL SKILL/ROGERS ET AL
TABLE Performance Characteristics of Groups Proportion square Average time Performance Score
CAL (n 5 40)
LFS (n 5 42)
Test
Significance
85% 19.6 6 9.6 sec 12.8 6 4.14
90% 17.4 6 6.8 sec 17.4 6 3/53
Chi square 5 0.57 t 5 0.66 t 5 5.37
NS NS P ,.0001
Values for average time and performance score are the means 6 standard deviation. CAL 5 computer-assisted learning; LFS 5 lecture and feedback seminar.
Three perspectives were developed: a view from over the shoulder, a view at 90 degrees to the left of the surgeon, and a view of the skill being performed left-handed. These sessions took place in a standard conference room, and two faculty members were present to give feedback throughout the session. For the CAL session, a videotape was made of the surgical skill. The perspectives were identical to that used in the slide presentation. The video images were digitized, and a multimedia program was created using the software Director 4.0 (Macromedia, Inc.). In the multimedia program the student could choose to view the entire task in real time or in slow motion. We also divided the entire task into 12 separate steps that corresponded to those used in the slide demonstration. Text was shown in each of these steps that was identical to the text slides used in the LFS. The CAL sessions took place in a classroom equipped with a sufficient number computers so that there was one for each student. Each educational session lasted for 1 hour. At the end of each session, the students were instructed to tie their best and fastest pair of two-handed square knots. Each student was videotaped, and these tapes were independently reviewed, in a blinded fashion, by three surgical faculty members. The surgeons recorded whether or not they thought the knot was square. In the cases where all three surgeons did not agree on the squareness of the knot, the majority opinion was used. Further, the surgeons evaluated the quality of the knot tying using a rating scale that explicitly identified all of the actions necessary for an optimal performance. The number generated using the rating scale was termed the performance score, with a maximum score of 24. Finally, the time for the task was recorded for each of the students. After the experimental session and the videotape evaluation, the students participated in the alternate educational session and were then given a survey asking them which of the training sessions they preferred and reasons for this preference. The data were entered into a Quattro Pro 6.0 spreadsheet (Novell, Inc), and statistical analysis was done with this software program and SPSS for Windows, Release 7.0 (SPSS, Inc.) The interrater reliability of the checklist was evaluated using Cronbach’s alpha. The proportion of students who correctly tied a square knot and the survey responses were compared using chi square of proportions. The mean performance scores and the mean times were compared using Students’ t-test for independent samples.
RESULTS Ninety-one freshman medical students were enrolled in the study. Eight of the subjects were excluded because they had previous experience in the operating room or emergency room. The data from 1 of the subjects was excluded because it was not possible to evaluate the task on the videotape, leaving data from 82 of the students available for analysis. A total of 40 students participated in the CAL session and 42 students participated in the LFS session. Average age, gender distribution, handedness, and the proportion of students with computers at home were similar for each group. Cronbach’s alpha for the interrater reliability of the checklist was 0.79, indicating that the evaluators were able to reliably distinguish between inferior and superior performances. Measurements of outcome between the two groups were similar (Table) with the exception of the performance score. The mean performance score for the group that participated in the CAL session was significantly lower than that of the LFS group. After having experienced both educational sessions, the student’s were asked which methodology they preferred. Seventy-three of the 82 students (89%) preferred the LFS session. The lack of feedback was the item most often cited as the most significant negative feature of this model of CAL and was mentioned by 44% of the total group.
COMMENT A similar proportion of students in both groups were able to tie a square knot, suggesting that both groups grasped the cognitive portion of the task. By contrast, the significant difference in performance scores indicates that students in the LFS group attained a significantly higher level of proficiency with the skill than students in the CAL group. The finding of equivalence of cognition between the two methods is consistent with the existing literature on the use of CAL in medical education.10,11 Given the relatively high cost of developing CAL materials and the technology required to use it, CAL must offer an advantage over existing educational techniques.12 However, the available literature does not suggest that such an advantage exists when using CAL to teach purely cognitive information. CAL does seem to offer some features that could make it superior for teaching a motor task. It seems logical that viewing the entire motion of the task would offer a significant advantage compared with viewing still photographs or illustrations of individual steps in the tasks. This is possible by digitizing a traditional videotape and importing it into multimedia software. When viewing this digitized file, students can see the motion involved in the task
THE AMERICAN JOURNAL OF SURGERY® VOLUME 175 JUNE 1998
509
APPROACHES TO TEACHING BASIC SURGICAL SKILL/ROGERS ET AL
and can model their performance after the experts. The digitized image of the skill can be broken down into component parts that allows the student to customize the learning experience and concentrate on the particular portion of the skill that is causing difficulty. There is extremely limited experience with using CAL to teach medically relevant psychomotor skills. The experience most applicable to our interest is in teaching cardiopulmonary resuscitation (CPR). The American Heart Association developed a CAL tool to teach CPR, and two prospective randomized studies examined the use of this technology to teach this skill.13,14 These studies showed no difference between the two groups in the performance of the skill. In contrast, we found that the students who had participated in the CAL performed the motor task with much less proficiency than students in the LFS group. In the CPR experience, the motor skill testing is recorded as simply pass or fail. We suspect that had they used multiple outcome measurements, they may have detected a difference between the two groups. If we had measured only the time required to perform the task or the proportion of knots that were correctly squared, we would have found no difference between the methods. Survey responses indicate that the lack of feedback was the important difference between the two educational methods. While the student grasped the idea of the task, the lack of feedback led to an increased number of errors in the performance of this task. These errors lowered the performance score but did not affect the time required to perform the task. The students were aware that their performance did not match the experts but could not correct their own mistakes. This frustration is evident in their survey comments and in their overwhelming preference for the LFS format. The preference of students for a format where the instructor is present was also found in the experience with CAL to teach CPR.13 We can learn much about teaching surgical skills by examining the sports education literature where there is a long experience teaching motor skills.1 Feedback is one of the elements of instruction that is felt to be important in teaching motor skill associated with sports.15 We can expect that computer technology will play an increasingly important role in surgical education.16 Our interest at the beginning of this project was to use CAL to move the instruction of the most basic surgical skills out of the operating room and to relieve the faculty of the responsibility of being present to teach these skills. This feature has been cited as one of the major potential advan-
510
tages of the most advanced form of CAL, computer simulation.17 We have shown that, particularly when teaching a novice a new skill, that the involvement of the expert is still important. The results of our study should remind us that until CAL, including computer simulation, can faithfully reproduce all of the elements of motor skills teaching, it cannot replace traditional teaching methods.
REFERENCES 1. Barnes RW. Surgical handicraft: teaching and learning surgical skills. Am J Surg. 1987;153:422– 427. 2. Baker RJ. Teaching (and using) surgical technique. Curr Surg. 1985;42:105–106. 3. Allen PI. Anastomosis: a craft workshop for surgical trainees. Aust N Z J Surg. 1990;60:943–945. 4. Heppell J, Beauchamp F, Chollet A. Ten-year experience with a basic technical skills and perioperative management workshop for first-year residents. Can J Surg. 1995;38:27–32. 5. Bevan PG. The anastomosis workshop, March 1981. Ann R Coll Surg Engl. 1981;63:405– 408. 6. Lossing AG, Hatswell EM, Gilas T, et al. A technical-skills course for 1st-year residents in general surgery: a descriptive study. Can J Surg. 1992;35:536 –540. 7. Barnes RW, Lang NP, Whiteside MF. Halstedian technique revisited: innovations in teaching surgical skills. Ann Surg. 1989; 210:118 –121. 8. Ellis H. Teaching surgeons to stitch. Aust N Z J Surg. 1993;63: 919 –920. 9. Reznick RK. Teaching and testing technical skills. Am J Surg. 1993;165:358 –361. 10. Cohen PA, Dacanay L. A meta-analysis of computer-based instruction in nursing education. Comput Nurs. 1994;12:89 –97. 11. Jelovsek FR, Adebonojo L. Learning principles as applied to computer-assisted instruction. MD Comput. 1993;10:165–172. 12. Keane DR, Norman GR, Vickers J. The inadequacy of recent research on computer-assisted instruction. Acad Med. 1991;66: 444 – 448. 13. Fabius DB, Grissom EL, Fuentes A. Recertification in cardiopulmonary resuscitation: a comparison of two teaching methods. J Nurs Staff Dev. 1994;10:262–268. 14. Edwards MJ, Hannah KJ. An examination of the use of interactive videodisc cardiopulmonary resuscitation instruction for the lay community. Comput Nurs. 1985;3:250 –252. 15. Lee AM, Keh NC, Magill RA. Instructional effects of teacher feedback in physical education. J Teach Phys Ed. 1993;12:228 –243. 16. Dunnington GL, DaRosa DA. Changing surgical education strategies in an environment of changing health care delivery systems. World J Surg. 1994;18:734 –737. 17. Satava RM. Virtual reality surgical simulator: the first steps. Surg Endosc. 1993;7:203–205.
THE AMERICAN JOURNAL OF SURGERY® VOLUME 175 JUNE 1998